Re: realtime output and csv files

Path	csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From	Tim Chase <python.list@tim.thechases.com>
Newsgroups	comp.lang.python
Subject	Re: realtime output and csv files
Date	Sat, 6 Feb 2016 06:45:53 -0600
Lines	47
Message-ID	<mailman.40.1454766205.2317.python-list@python.org> (permalink)
References	<n92ucb$1h5s$1@gioia.aioe.org> <56B4FEAC.4080706@gmail.com> <20160205222715.06320d38@bigbox.christie.dr> <CAERt=-yX2Oy_9EpcL=5UjBYo6+ggax5_OTJwtc5X=JTiaMJEJQ@mail.gmail.com>
Mime-Version	1.0
Content-Type	text/plain; charset=US-ASCII
Content-Transfer-Encoding	7bit
X-Trace	news.uni-berlin.de fek0woZzAx5Y/fD79LwHngJpoKnzSKnoHKz0Q4htIx0w==
Return-Path	<python.list@tim.thechases.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.005
X-Spam-Evidence	'H': 0.99; 'S': 0.00; 'escape': 0.07; "'rb')": 0.09; 'csv': 0.09; 'delimiter': 0.09; 'subject:files': 0.09; 'tab': 0.09; 'assume': 0.11; '-tkc': 0.16; 'character:': 0.16; 'comma': 0.16; 'correctly,': 0.16; 'double-quote': 0.16; 'from:addr:python.list': 0.16; 'from:addr:tim.thechases.com': 0.16; 'from:name:tim chase': 0.16; "module's": 0.16; 'naive': 0.16; 'obviously,': 0.16; 'received:10.122': 0.16; 'received:io': 0.16; 'received:psf.io': 0.16; 'trivially': 0.16; 'tweak': 0.16; 'wrote:': 0.16; 'module,': 0.18; 'default,': 0.22; 'transfers': 0.23; 'header:In-Reply-To:1': 0.24; 'module': 0.25; "i've": 0.25; "doesn't": 0.26; 'finally,': 0.27; 'format,': 0.27; 'specify': 0.27; 'character': 0.29; 'starts': 0.29; 'guess': 0.31; 'supposed': 0.31; 'help,': 0.32; 'common': 0.33; 'schemes': 0.33; 'fail': 0.35; 'but': 0.36; 'instead': 0.36; 'there': 0.36; 'to:addr:python-list': 0.36; 'subject:: ': 0.37; 'received:10': 0.37; 'charset:us-ascii': 0.37; 'things': 0.38; 'several': 0.38; 'files': 0.38; 'data': 0.39; 'sure': 0.39; 'format': 0.39; 'to:addr:python.org': 0.40; 'default': 0.61; 'making': 0.62; 'alphanumeric': 0.66; 'manner': 0.69; 'intelligent': 0.76; 'bernardo': 0.84; 'column.': 0.84; 'dialect': 0.84; 'escapes,': 0.84; 'existence.': 0.84; 'received:23': 0.84; 'doubling': 0.91
X-Sender-Id	wwwh\|x-authuser\|tim@thechases.com
X-Sender-Id	wwwh\|x-authuser\|tim@thechases.com
X-MC-Relay	Neutral
X-MailChannels-SenderId	wwwh\|x-authuser\|tim@thechases.com
X-MailChannels-Auth-Id	wwwh
X-MC-Loop-Signature	1454762909882:3190701163
X-MC-Ingress-Time	1454762909882
In-Reply-To	<CAERt=-yX2Oy_9EpcL=5UjBYo6+ggax5_OTJwtc5X=JTiaMJEJQ@mail.gmail.com>
X-Mailer	Claws Mail 3.11.1 (GTK+ 2.24.25; x86_64-pc-linux-gnu)
X-AuthUser	tim@thechases.com
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.21rc2
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Xref	csiph.com comp.lang.python:102582

Show key headers only | View raw

On 2016-02-06 02:53, Bernardo Sulzbach wrote:
>> And even if you have things to escape or format correctly, the
>> stdlib has a "csv" module that makes this trivially easy:
>>  
> 
> I supposed it had one. Obviously, I've never used it myself,
> otherwise I would be sure about its existence. Nice to know about
> it, although I assume that for many data transfers it will not be
> painless if you need several escapes, when CSV starts to become
> complicated it starts to fail because of quirky and naive parsers.

If you read up on the csv module's help, it has all manner of knobs
to twiddle when it comes to the "dialect" of CSV:

- delimiters:  comma is the default, but you can specify others like
  tab or pipe)

- quote character:  double-quote is the default, but you
  can use single-quote or otherwise

- escaping: doubling up the quote-character is the default but I think
  you can specify other schemes

There are others, but that's what I have to tweak most often.

Finally, there's a sniffer in the module, so even if you don't know
the format, you can make an intelligent guess about it:

  with open('example.csv', 'rb') as csvfile:
    dialect = csv.Sniffer().sniff(csvfile.read(1024*4))
    csvfile.seek(0)
    reader = csv.reader(csvfile, dialect)

The only edge case I've hit regularly is that it doesn't (didn't? not
sure if it's been fixed) do the right thing on CSV files containing a
single column, as it would try to accept a common alphanumeric
character as the delimiter instead of making the better assumption
that it was a single column.

-tkc

Back to comp.lang.python | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

realtime output and csv files lucan <lucan@NNN.it> - 2016-02-05 20:49 +0100
  Re: realtime output and csv files Bernardo Sulzbach <mafagafogigante@gmail.com> - 2016-02-05 17:57 -0200
    Re: realtime output and csv files lucan <lucan@NNN.it> - 2016-02-05 22:09 +0100
      Re: realtime output and csv files Bernardo Sulzbach <mafagafogigante@gmail.com> - 2016-02-05 19:13 -0200
      Re: realtime output and csv files Joel Goldstick <joel.goldstick@gmail.com> - 2016-02-05 16:43 -0500
      Re: realtime output and csv files Bernardo Sulzbach <mafagafogigante@gmail.com> - 2016-02-05 19:51 -0200
        Re: realtime output and csv files lucan <lucan@NNN.it> - 2016-02-06 14:40 +0100
      Re: realtime output and csv files Joel Goldstick <joel.goldstick@gmail.com> - 2016-02-05 17:12 -0500
  Re: realtime output and csv files Tim Chase <python.list@tim.thechases.com> - 2016-02-05 22:27 -0600
  Re: realtime output and csv files Bernardo Sulzbach <mafagafogigante@gmail.com> - 2016-02-06 02:53 -0200
  Re: realtime output and csv files Tim Chase <python.list@tim.thechases.com> - 2016-02-06 06:45 -0600

csiph-web