Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #21167 > unrolled thread

RotatingFileHandler Fails

Back to article view | Back to comp.lang.python

Contents

  RotatingFileHandler Fails nac <cookfitz@gmail.com> - 2012-03-03 03:42 -0800
    Re: RotatingFileHandler Fails Jean-Michel Pichavant <jeanmichel@sequans.com> - 2012-03-05 11:27 +0100
      Re: RotatingFileHandler Fails nac <cookfitz@gmail.com> - 2012-03-05 18:17 -0800
        Re: RotatingFileHandler Fails arun1 <arun.sathyan@ust-global.com> - 2012-03-07 05:20 -0800
        Re: RotatingFileHandler Fails arun1 <arun.sathyan@ust-global.com> - 2012-03-07 07:33 -0800

#21167 — RotatingFileHandler Fails

The RotatingFileHandler running on win 7 64-bit; py 2.7 is failing
when the script launches a process using subprocess.Popen. Works fine
if the subprocess is not launched

The exception thrown
Traceback (most recent call last):
  File "C:\Python27\lib\logging\handlers.py", line 78, in emit
    self.doRollover()
  File "C:\Python27\lib\logging\handlers.py", line 141, in doRollover
    os.rename(self.baseFilename, dfn)
WindowsError: [Error 32] The process cannot access the file because it
is being used by another process

Anyone have an idea how to fix this?


import os, sys
import logging
import logging.handlers
import subprocess

def chomp(s):
      "remove trailing carriage return"
      if s[-1:]=='\n': return s[:-1]
      else: return s

logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %
(name)-2s %(levelname)-8s %(threadName)-12s %(message)s')
q5Logger = logging.getLogger('Q5')
logFileHandler = logging.handlers.RotatingFileHandler(filename='c:\
\logger\\q5.log', mode= 'a', maxBytes=100, backupCount=5)
logFileHandler.setFormatter(logging.Formatter('%(asctime)s %(name)-2s %
(levelname)-8s %(threadName)-12s %(message)s'))
logFileHandler.setLevel(logging.DEBUG)
q5Logger.addHandler(logFileHandler)

progOutput = subprocess.Popen(r'dir *.*', shell=True, bufsize=1000,
stdout=subprocess.PIPE).stdout
line = progOutput.readline()
while (line) != "":
      q5Logger.info( chomp(line))
      line = progOutput.readline()
rc = progOutput.close()

[toc] | [next] | [standalone]

#21226

nac wrote:
> The RotatingFileHandler running on win 7 64-bit; py 2.7 is failing
> when the script launches a process using subprocess.Popen. Works fine
> if the subprocess is not launched
>
> The exception thrown
> Traceback (most recent call last):
>   File "C:\Python27\lib\logging\handlers.py", line 78, in emit
>     self.doRollover()
>   File "C:\Python27\lib\logging\handlers.py", line 141, in doRollover
>     os.rename(self.baseFilename, dfn)
> WindowsError: [Error 32] The process cannot access the file because it
> is being used by another process
>
> Anyone have an idea how to fix this?
>
>
> import os, sys
> import logging
> import logging.handlers
> import subprocess
>
> def chomp(s):
>       "remove trailing carriage return"
>       if s[-1:]=='\n': return s[:-1]
>       else: return s
>
> logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %
> (name)-2s %(levelname)-8s %(threadName)-12s %(message)s')
> q5Logger = logging.getLogger('Q5')
> logFileHandler = logging.handlers.RotatingFileHandler(filename='c:\
> \logger\\q5.log', mode= 'a', maxBytes=100, backupCount=5)
> logFileHandler.setFormatter(logging.Formatter('%(asctime)s %(name)-2s %
> (levelname)-8s %(threadName)-12s %(message)s'))
> logFileHandler.setLevel(logging.DEBUG)
> q5Logger.addHandler(logFileHandler)
>
> progOutput = subprocess.Popen(r'dir *.*', shell=True, bufsize=1000,
> stdout=subprocess.PIPE).stdout
> line = progOutput.readline()
> while (line) != "":
>       q5Logger.info( chomp(line))
>       line = progOutput.readline()
> rc = progOutput.close()
>   
I don't think you can open a file from 2 different processes, hence the 
error message. The only exception would be that one of them opens it in 
read only mode which won't happen for log file.

You cannot fix it, it is the operand system that blocks the operation.
Using thread may allow you to log into the same file, but could bring a 
bunch of other problems.

I know some ppl use a log server, your processes using a socket (client) 
logger. You could do that on your local machine. There's a tutorial on 
that in the logging documentation.

JM

[toc] | [prev] | [next] | [standalone]

#21252

On Mar 5, 4:27 am, Jean-Michel Pichavant <jeanmic...@sequans.com>
wrote:
> nac wrote:
> > The RotatingFileHandler running on win 7 64-bit; py 2.7 is failing
> > when the script launches a process using subprocess.Popen. Works fine
> > if the subprocess is not launched
>
> > The exception thrown
> > Traceback (most recent call last):
> >   File "C:\Python27\lib\logging\handlers.py", line 78, in emit
> >     self.doRollover()
> >   File "C:\Python27\lib\logging\handlers.py", line 141, in doRollover
> >     os.rename(self.baseFilename, dfn)
> > WindowsError: [Error 32] The process cannot access the file because it
> > is being used by another process
>
> > Anyone have an idea how to fix this?
>
> > import os, sys
> > import logging
> > import logging.handlers
> > import subprocess
>
> > def chomp(s):
> >       "remove trailing carriage return"
> >       if s[-1:]=='\n': return s[:-1]
> >       else: return s
>
> > logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %
> > (name)-2s %(levelname)-8s %(threadName)-12s %(message)s')
> > q5Logger = logging.getLogger('Q5')
> > logFileHandler = logging.handlers.RotatingFileHandler(filename='c:\
> > \logger\\q5.log', mode= 'a', maxBytes=100, backupCount=5)
> > logFileHandler.setFormatter(logging.Formatter('%(asctime)s %(name)-2s %
> > (levelname)-8s %(threadName)-12s %(message)s'))
> > logFileHandler.setLevel(logging.DEBUG)
> > q5Logger.addHandler(logFileHandler)
>
> > progOutput = subprocess.Popen(r'dir *.*', shell=True, bufsize=1000,
> > stdout=subprocess.PIPE).stdout
> > line = progOutput.readline()
> > while (line) != "":
> >       q5Logger.info( chomp(line))
> >       line = progOutput.readline()
> > rc = progOutput.close()
>
> I don't think you can open a file from 2 different processes, hence the
> error message. The only exception would be that one of them opens it in
> read only mode which won't happen for log file.
>
> You cannot fix it, it is the operand system that blocks the operation.
> Using thread may allow you to log into the same file, but could bring a
> bunch of other problems.
>
> I know some ppl use a log server, your processes using a socket (client)
> logger. You could do that on your local machine. There's a tutorial on
> that in the logging documentation.
>
> JM

Thanks for taking the time to look at the problem. In case you are
interested I was able to get this to work with the code at URL below.
Seemed to work because the subprocess I am running  is not actually
using the log file but the OS make it inherit a handle to the log
file. The code below disables that inheritance. Do you see any
problems with the approach?

http://bugs.python.org/file14420/NTSafeLogging.py

[toc] | [prev] | [next] | [standalone]

#21317

Hi nac,

NTSafeLogging.py is working fine without any errors, but its not rotating
the log files as rotatingfilehandler does.
Do you have any working sample with  NTSafeLogging which rotates the log
file.

logging issue with subprocess.Popen  can be solved using  this code

import threading
lock = threading.RLock()
def acquire_lock():
	lock.acquire()

def release_lock():
	lock.release()

call the  acquire_lock()   at the begining  of method and release_lock() at
the end





--
View this message in context: http://python.6.n6.nabble.com/RotatingFileHandler-Fails-tp4542769p4554381.html
Sent from the Python - python-list mailing list archive at Nabble.com.

[toc] | [prev] | [next] | [standalone]

#21321

Hi,

Actually NonInheritedRotatingFileHandler is rotating the log files but some
times it falis and showing I/O errors while the log file limit reaches the
given size.

Thanks

Arun 


--
View this message in context: http://python.6.n6.nabble.com/RotatingFileHandler-Fails-tp4542769p4554781.html
Sent from the Python - python-list mailing list archive at Nabble.com.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web