Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!ecngs!feeder2.ecngs.de!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
To: python-list@python.org
From: Christian Heimes <lists@cheimes.de>
Subject: Re: How to safely maintain a status file
Date: Sun, 08 Jul 2012 13:53:05 +0200
References: <CAOV1wRVtm27yWez1HZuN8=ia-TyM2aXp9QCUbSZ5aZExP_ZChA@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:12.0) Gecko/20120430 Thunderbird/12.0.1
In-Reply-To: <CAOV1wRVtm27yWez1HZuN8=ia-TyM2aXp9QCUbSZ5aZExP_ZChA@mail.gmail.com>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.1917.1341748397.4697.python-list@python.org>
Lines: 49
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:25043

Am 08.07.2012 13:29, schrieb Richard Baron Penman:
> My initial solution was a thread that writes status to a tmp file
> first and then renames:
> 
> open(tmp_file, 'w').write(status)
> os.rename(tmp_file, status_file)

You algorithm may not write and flush all data to disk. You need to do
additional work. You must also store the tmpfile on the same partition
(better: same directory) as the status file

with open(tmp_file, "w") as f:
    f.write(status)
    # flush buffer and write data/metadata to disk
    f.flush()
    os.fsync(f.fileno())

# now rename the file
os.rename(tmp_file, status_file)

# finally flush metadata of directory to disk
dirfd = os.open(os.path.dirname(status_file), os.O_RDONLY)
try:
    os.fsync(dirfd)
finally:
    os.close(dirfd)


> This works well on Linux but Windows raises an error when status_file
> already exists.
> http://docs.python.org/library/os.html#os.rename

Windows doesn't suppport atomic renames if the right side exists.  I
suggest that you implement two code paths:

if os.name == "posix":
    rename = os.rename
else:
    def rename(a, b):
        try:
            os.rename(a, b)
        except OSError, e:
            if e.errno != 183:
                raise
            os.unlink(b)
            os.rename(a, b)

Christian