Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: Neil Cerutti <neilc@norwich.edu>
Newsgroups: comp.lang.python
Subject: Re: Using "with open(filename, 'ab'):" and calling code only if the file is new?
Date: 30 Oct 2013 13:23:52 GMT
Organization: Norwich University
Lines: 52
Message-ID: <bdcfj8Fk2pU1@mid.individual.net>
References: <68bd6cb6-44b2-446c-b0e2-043e3ac1c35b@googlegroups.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-Trace: individual.net OKuZs8Njua7VeubkGICO4gwYicFk8gdLahUNpvEhoIH8BN1nc6
Cancel-Lock: sha1:tvFdcnBHO4VU644dKUUKmw8r8wM=
User-Agent: slrn/0.9.9p1/mm/ao (Win32)
Xref: csiph.com comp.lang.python:58037

On 2013-10-30, Victor Hooi <victorhooi@gmail.com> wrote:
> Hi,
>
> I have a CSV file that I will repeatedly appending to.
>
> I'm using the following to open the file:
>
>     with open(self.full_path, 'r') as input, open(self.output_csv, 'ab') as output:
>         fieldnames = (...)
>         csv_writer = DictWriter(output, filednames)
>         # Call csv_writer.writeheader() if file is new.
>         csv_writer.writerows(my_dict)
>
> I'm wondering what's the best way of calling writeheader() only
> if the file is new?
>
> My understanding is that I don't want to use os.path.exist(),
> since that opens me up to race conditions.
>
> I'm guessing I can't use try-except with IOError, since the
> open(..., 'ab') will work whether the file exists or not.
>
> Is there another way I can execute code only if the file is new?

A heavy-duty approach involves prepending the old contents to a
temporary file.

fieldnames = (...)

with tempfile.TempDirectory() as temp:
    tempname = os.path.join(temp, 'output.csv')
    with open(tempname, 'wb') as output:
        writer = csv.DictWriter(output, fieldnames=fieldnames)
        writer.writeheader()
        try:
            with open(self.output_csv, 'b') old_data:
                reader = csv.DictReader(old_data)
                for rec in reader:
                    writer.writerow(rec)
        except IOError:
            pass
        with open(self.full_path, 'b') as infile:
            # etc...
    shutil.copy(tempname, self.output_csv)

This avoids clobbering output_csv unless new data is succesfully
written. I believe TempDirectory isn't available in Python 2, so
some other way of creating that path will be needed, and I'm too
lazy to look up how. ;)

-- 
Neil Cerutti