Path: csiph.com!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder1.enfer-du-nord.net!newsfeed.eweka.nl!eweka.nl!feeder3.eweka.nl!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
Sender: mhrivnak@gmail.com
In-Reply-To: <jtbsb1$8fj$1@dough.gmane.org>
References: <CAOV1wRVtm27yWez1HZuN8=ia-TyM2aXp9QCUbSZ5aZExP_ZChA@mail.gmail.com> <jtbsb1$8fj$1@dough.gmane.org>
Date: Sun, 8 Jul 2012 12:47:25 -0400
Subject: Re: How to safely maintain a status file
From: Michael Hrivnak <mhrivnak@hrivnak.org>
To: Christian Heimes <lists@cheimes.de>
Content-Type: text/plain; charset=ISO-8859-1
Cc: python-list@python.org
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.1920.1341766046.4697.python-list@python.org>
Lines: 59
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:25044

What are you keeping in this status file that needs to be saved
several times per second?  Depending on what type of state you're
storing and how persistent it needs to be, there may be a better way
to store it.

Michael

On Sun, Jul 8, 2012 at 7:53 AM, Christian Heimes <lists@cheimes.de> wrote:
> Am 08.07.2012 13:29, schrieb Richard Baron Penman:
>> My initial solution was a thread that writes status to a tmp file
>> first and then renames:
>>
>> open(tmp_file, 'w').write(status)
>> os.rename(tmp_file, status_file)
>
> You algorithm may not write and flush all data to disk. You need to do
> additional work. You must also store the tmpfile on the same partition
> (better: same directory) as the status file
>
> with open(tmp_file, "w") as f:
>     f.write(status)
>     # flush buffer and write data/metadata to disk
>     f.flush()
>     os.fsync(f.fileno())
>
> # now rename the file
> os.rename(tmp_file, status_file)
>
> # finally flush metadata of directory to disk
> dirfd = os.open(os.path.dirname(status_file), os.O_RDONLY)
> try:
>     os.fsync(dirfd)
> finally:
>     os.close(dirfd)
>
>
>> This works well on Linux but Windows raises an error when status_file
>> already exists.
>> http://docs.python.org/library/os.html#os.rename
>
> Windows doesn't suppport atomic renames if the right side exists.  I
> suggest that you implement two code paths:
>
> if os.name == "posix":
>     rename = os.rename
> else:
>     def rename(a, b):
>         try:
>             os.rename(a, b)
>         except OSError, e:
>             if e.errno != 183:
>                 raise
>             os.unlink(b)
>             os.rename(a, b)
>
> Christian
>
> --
> http://mail.python.org/mailman/listinfo/python-list