Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!eweka.nl!lightspeed.eweka.nl!194.109.133.83.MISMATCH!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.001 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'python.': 0.02; 'value,': 0.04; 'attribute': 0.07; 'error:': 0.07; 'work!': 0.07; '__init__': 0.09; 'tests,': 0.09; 'unittest': 0.09; 'used.': 0.09; 'python': 0.11; 'def': 0.12; 'wrote': 0.14; 'thread': 0.14; 'changes': 0.15; 'posted': 0.15; '"i': 0.16; '37,': 0.16; '49,': 0.16; 'ah,': 0.16; 'errors:': 0.16; 'skip:" 70': 0.16; 'skip:" 80': 0.16; 'threads.': 0.16; 'subject:python': 0.16; 'prevent': 0.16; 'wrote:': 0.18; 'all,': 0.19; 'module': 0.19; 'trying': 0.19; 'file,': 0.19; 'server,': 0.19; 'thu,': 0.19; 'seems': 0.21; '8bit%:5': 0.22; '>>>': 0.22; 'example': 0.22; 'import': 0.22; 'email addr:gmail.com>': 0.22; 'otherwise,': 0.22; 'tests': 0.22; 'exists': 0.24; 'file.': 0.24; "i've": 0.25; '>': 0.26; 'class.': 0.26; 'logging': 0.26; 'skip:" 40': 0.26; 'task': 0.26; 'pass': 0.26; 'header:In-Reply-To:1': 0.27; 'to:2**1': 0.27; 'received:209.85.217': 0.29; 'skip:& 60': 0.30; 'message-id:@mail.gmail.com': 0.30; 'url:mailman': 0.30; 'code': 0.31; 'easier': 0.31; "skip:' 10": 0.31; 'that.': 0.31; '25,': 0.31; '>>>>': 0.31; 'explained': 0.31; 'fine,': 0.31; 'file': 0.32; 'class': 0.32; 'thanks!': 0.32; 'skip:- 30': 0.32; 'url:python': 0.33; '(most': 0.33; '-----': 0.33; 'fri,': 0.33; 'skip:_ 10': 0.34; 'problem': 0.35; 'received:209.85': 0.35; 'received:google.com': 0.35; 'there': 0.35; 'version': 0.36; '2.6': 0.36; 'module.': 0.36; 'skip:" 50': 0.36; 'done': 0.36; 'url:listinfo': 0.36; 'thanks': 0.36; 'responsible': 0.36; 'hi,': 0.36; 'url:org': 0.36; 'should': 0.36; 'received:209': 0.37; 'implement': 0.38; 'server': 0.38; 'skip:& 10': 0.38; 'thank': 0.38; '8bit%:4': 0.38; 'to:addr:python-list': 0.38; 'pm,': 0.38; 'recent': 0.39; 'skip:_ 30': 0.39; 'skip:& 20': 0.39; 'to:addr:python.org': 0.39; 'url:mail': 0.40; 'how': 0.40; 'applicable': 0.60; 'skip:y 20': 0.60; 'most': 0.60; 'skip:t 30': 0.61; 'url:3': 0.61; 'you.': 0.62; 'information': 0.63; 'url:blogspot': 0.65; 'here': 0.66; 'notice:': 0.67; '26,': 0.68; 'person,': 0.68; 'privileged.': 0.69; 'disclose': 0.74; '2.5.': 0.84; 'br,': 0.84; 'missing.': 0.84; '"how': 0.91; 'medium.': 0.91; 'skip:& 80': 0.91; '2013': 0.98 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=/onwpjgFUqgFErl7Kfb5bWuZo7VJHEsi7GG/MrLZCLM=; b=HEB0OIdohCUgs6Dk+QySqs3U8O/pRgPT0r/0IMJhovGoeER3B2VtIZ3WSfR2aesoiH pUncOZ3fNhL7zuj5nqGqKgc1J3Sf89Spta+X7qKSfCLsZB8Y6Ncrqj78av1bEhG7P9PJ jAFpf37S8TXgf5KHi9azpLI3EY7wjgApriR1Nh8aAnSjxho97QH9nc5LjWdgV7JPx3yi fAcu77u1zlGU+fgkEL8V9kSc/K6uuB5dfGADBbsrZYo9j0tWBNMazG4C4JBWXlJBNW7Y his50km/pzkb/MGW/AhZXY/bKvTuWX92sIxLmYMBMiFR356+UxVFlzjzyZe07DMtkwuA Vmug== MIME-Version: 1.0 X-Received: by 10.112.166.101 with SMTP id zf5mr2724552lbb.59.1367505236633; Thu, 02 May 2013 07:33:56 -0700 (PDT) In-Reply-To: References: <229947553.5215691.1366982379397.JavaMail.root@sequans.com> Date: Thu, 2 May 2013 16:33:56 +0200 Subject: Re: Efficient way of looging in python From: Maitrey Mishra To: Jean-Michel Pichavant , python-list@python.org Content-Type: multipart/alternative; boundary=001a11c38506dbadbb04dbbd1ffd X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 322 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1367505239 news.xs4all.nl 15923 [2001:888:2000:d::a6]:56614 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:44631 --001a11c38506dbadbb04dbbd1ffd Content-Type: text/plain; charset=ISO-8859-1 Sometimes, I notice the below errors: python call_Test.py Traceback (most recent call last): File "call_Test.py", line 1, in from Test_Task import TestTask File "/cygdrive/d/smoketest/targetflasher/flasher_server/Source/Test_Task.py", line 5, in import Logging.logQueueHandler as logq File "/cygdrive/d/smoketest/targetflasher/flasher_server/Source/Logging/logQueueHandler.py", line 72, in queue = multiprocessing.Queue(-1) File "/usr/lib/python2.6/multiprocessing/__init__.py", line 213, in Queue return Queue(maxsize) File "/usr/lib/python2.6/multiprocessing/queues.py", line 37, in __init__ self._rlock = Lock() File "/usr/lib/python2.6/multiprocessing/synchronize.py", line 117, in __init__ SemLock.__init__(self, SEMAPHORE, 1, 1) File "/usr/lib/python2.6/multiprocessing/synchronize.py", line 49, in __init__ sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue) OSError: [Errno 17] File exists On Thu, May 2, 2013 at 4:13 PM, Maitrey Mishra wrote: > I wrote some tests for logging module. I use python 2.6 and use the > example as in cookbook > file : Test_Task.py > from threading import Thread > import Logging.logQueueHandler as logq > > class TestTask(threading.Thread): > def __init__(self): > pass > > def run(self): > t = Thread(target=logq.log, args=("ERROR", "Hi")) > t.start() > q = Thread(target=logq.log, args=("ERROR", "Hello")) > q.start() > p = Thread(target=logq.log, args=("ERROR", "How are you?")) > p.start() > l = Thread(target=logq.log, args=("ERROR", "I am fine, Thank You!")) > l.start() > call_test.py > from Test_Task import TestTask > import unittest > > task = TestTask() > task.run() > However, In the log messages I see a number of messages missing. > With the tests, my aim is to see how logging works with multiple threads. > > > On Fri, Apr 26, 2013 at 3:19 PM, Jean-Michel Pichavant < > jeanmichel@sequans.com> wrote: > >> Yep that's the one I used. >> >> ------------------------------ >> >> Ah, >> http://plumberjack.blogspot.de/2010/09/using-logging-with-multiprocessing.html >> I use this now, it seems to work! >> Thanks! >> >> >> On Fri, Apr 26, 2013 at 2:16 PM, Maitrey Mishra < >> maitreymishra23@gmail.com> wrote: >> >>> Hi All, >>> Thanks for the inputs. >>> I use python2.6. It seems to be easier with python3.2 and QueueHandler >>> class. >>> I pasted the class in my code however I still get the >>> error: AttributeError: 'module' object has no attribute 'QueueHandler' >>> What changes did you do for adapting to python2.5? >>> >>> Thanks! >>> Br, >>> Maitrey >>> >>> >>> >>> On Thu, Apr 25, 2013 at 4:32 PM, Jean-Michel Pichavant < >>> jeanmichel@sequans.com> wrote: >>> >>>> ----- Original Message ----- >>>> > Hi, >>>> > >>>> > I need an efficient way of logging using python. >>>> > My problem statemnt: >>>> > 1. I have multiple processes using the same logging file. >>>> > I need solutions to the following: >>>> > a) If multiple processes are trying to write to the same file, I need >>>> > to prevent that. Otherwise, the logging messages will be puzzling to >>>> > the user as he would see logging messages from one module than may >>>> > be some other following it. >>>> > >>>> > Please let me know if there is an efficient way to do it. >>>> > >>>> > Thanks! >>>> > >>>> > Br, >>>> > Maitrey >>>> > -- >>>> > http://mail.python.org/mailman/listinfo/python-list >>>> >>>> Everything you need should be explained here >>>> >>>> http://docs.python.org/release/3.2/howto/logging-cookbook.html#logging-to-a-single-file-from-multiple-processes >>>> >>>> Most of the concepts are applicable to any version of python I've done >>>> it with python 2.5. I used some Vinay's code posted on the net to implement >>>> a logging server, all my processes are logging to that server which is >>>> responsible for writing the file. >>>> >>>> JM >>>> >>>> >>>> -- IMPORTANT NOTICE: >>>> >>>> The contents of this email and any attachments are confidential and may >>>> also be privileged. If you are not the intended recipient, please notify >>>> the sender immediately and do not disclose the contents to any other >>>> person, use it for any purpose, or store or copy the information in any >>>> medium. Thank you. >>>> >>> >>> >> >> >> >> -- IMPORTANT NOTICE: >> >> The contents of this email and any attachments are confidential and may >> also be privileged. If you are not the intended recipient, please notify >> the sender immediately and do not disclose the contents to any other >> person, use it for any purpose, or store or copy the information in any >> medium. Thank you. >> > > --001a11c38506dbadbb04dbbd1ffd Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Sometimes, I notice the below errors:
=A0python c= all_Test.py
Traceback (most recent call last):
=A0 File= "call_Test.py", line 1, in <module>
=A0 =A0 from= Test_Task import TestTask
=A0 File "/cygdrive/d/smoketest/targetflasher/flasher_server/Sour= ce/Test_Task.py", line 5, in <module>
=A0 =A0 import L= ogging.logQueueHandler as logq
=A0 File "/cygdrive/d/smokete= st/targetflasher/flasher_server/Source/Logging/logQueueHandler.py", li= ne 72, in <module>
=A0 =A0 queue =3D multiprocessing.Queue(-1)
=A0 File "/= usr/lib/python2.6/multiprocessing/__init__.py", line 213, in Queue
=A0 =A0 return Queue(maxsize)
=A0 File "/usr/lib/pyth= on2.6/multiprocessing/queues.py", line 37, in __init__
=A0 =A0 self._rlock =3D Lock()
=A0 File "/usr/lib/pytho= n2.6/multiprocessing/synchronize.py", line 117, in __init__
= =A0 =A0 SemLock.__init__(self, SEMAPHORE, 1, 1)
=A0 File "/u= sr/lib/python2.6/multiprocessing/synchronize.py", line 49, in __init__=
=A0 =A0 sl =3D self._semlock =3D _multiprocessing.SemLock(kind, value,= maxvalue)
OSError: [Errno 17] File exists



On = Thu, May 2, 2013 at 4:13 PM, Maitrey Mishra <maitreymishra23@gma= il.com> wrote:
I wrote some tests for= logging module. I use python 2.6 and use the example as in cookbook
<= div> file : Test_Task.py
from threading import Thread
import= Logging.logQueueHandler as logq

class TestTask(threading.Thread):
def __init_= _(self):
pass

def run(self):
t= =3D Thread(target=3Dlogq.log, args=3D("ERROR", "Hi"))<= /div>
t.start()
q =3D Thread(target=3Dlogq.log, args=3D("ERRO= R", "Hello"))
q.start()
p =3D Thread(tar= get=3Dlogq.log, args=3D("ERROR", "How are you?"))
=
p.start()
l =3D Thread(target=3Dlogq.log, args=3D("ERROR", "I am = fine, Thank You!"))
l.start()
call_test.py
from Test_Task import TestTask
import unittest

<= /div>
task =3D TestTask()
task.run()
However, In the log= messages I see a number of messages missing.
With the tests, my = aim is to see how logging works with multiple threads.


On Fri, Apr 26, 2013 at 3:19 PM, Jean-Mi= chel Pichavant <jeanmichel@sequans.com> wrote:
Yep that's the one I used.



On Fri, Apr 26, 2013 at 2:16 PM, Maitrey Mishra <maitre= ymishra23@gmail.com> wrote:
Hi All,
Thanks for the = inputs.
I use python2.6. It seems to be easier with python3.2 and= QueueHandler class.
I pasted the class in my code however I still get the error:=A0Attribu= teError: 'module' object has no attribute 'QueueHandler'
What changes did you do for adapting to python2.5?

Thanks!
Br,
Maitrey

=


On Thu, Apr 25, 2013 at 4:32 PM, Jean-Michel Pichavant &l= t;jeanmichel@se= quans.com> wrote:
----- Original Message -----
> Hi,
>
> I need an efficient way of logging using python.
> My problem statemnt:
> 1. I have multiple processes using the same logging file.
> I need solutions to the following:
> a) If multiple processes are trying to write to the same file, I = need
> to prevent that. Otherwise, the logging messages will be puzzlin= g to
> the user as he would see logging messages from one module than may
> be some other following it.
>
> Please let me know if there is an efficient way to do it.
>
> Thanks!
>
> Br,
> Maitrey
> --
> http://mail.python.org/mailman/listinfo/python-list

Everything you need should be explained here
http://d= ocs.python.org/release/3.2/howto/logging-cookbook.html#logging-to-a-single-= file-from-multiple-processes

Most of the concepts are applicable to any version of python I've done = it with python 2.5. I used some Vinay's code posted on the net to imple= ment a logging server, all my processes are logging to that server which is= responsible for writing the file.

JM


-- IMPORTANT NOTICE:

The contents of this email and any attachments are confidential and may als= o be privileged. If you are not the intended recipient, please notify the s= ender immediately and do not disclose the contents to any other person, use= it for any purpose, or store or copy the information in any medium. Thank = you.





-- IMPORTANT NOTICE:

The contents of this email and any attachments are confidential and may als= o be privileged. If you are not the intended recipient, please notify the s= ender immediately and do not disclose the contents to any other person, use= it for any purpose, or store or copy the information in any medium. Thank = you.


--001a11c38506dbadbb04dbbd1ffd--