Re: Pythonic infinite for loop?

Path	csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!news.albasani.net!newsfeed.straub-nv.de!news.glorb.com!news2.glorb.com!news.glorb.com!news-out.octanews.net!indigo.octanews.net!auth.brown.octanews.com.POSTED!not-for-mail
From	Paul Rubin <no.email@nospam.invalid>
Newsgroups	comp.lang.python
Subject	Re: Pythonic infinite for loop?
References	<mailman.377.1302833455.9059.python-list@python.org> <4da7ae8a$0$29986$c3e8da3$5496439d@news.astraweb.com> <BANLkTinV2NTUgGNDf+mM1R98vf2TmoQ5Rw@mail.gmail.com> <mailman.382.1302839905.9059.python-list@python.org> <4da83f8f$0$29986$c3e8da3$5496439d@news.astraweb.com> <mailman.396.1302881747.9059.python-list@python.org>
Date	Fri, 15 Apr 2011 09:10:18 -0700
Message-ID	<7xmxjrxzpx.fsf@ruckus.brouhaha.com> (permalink)
Organization	Nightsong/Fort GNOX
User-Agent	Gnus/5.13 (Gnus v5.13) Emacs/23.1 (gnu/linux)
Cancel-Lock	sha1:yaI1dZ2oV7IrwEkrvj/UDHqyWFk=
MIME-Version	1.0
Content-Type	text/plain; charset=us-ascii
Lines	9
NNTP-Posting-Date	15 Apr 2011 11:10:18 CDT
X-Complaints-To	abuse@octanews.net
Xref	x330-a1.tempe.blueboxinc.net comp.lang.python:3273

Show key headers only | View raw

Chris Angelico <rosuav@gmail.com> writes:
> This whole code is inside a loop that we took, in smoke testing, to a
> couple hundred million rows (I think), with the intention of having no
> limit at all. So this might only look at 60-100 headers, but it will
> be doing so in a tight loop.

If you're talking about data sets that large, you should rethink the
concept of using python dictionaries with a key for every row.  Try
importing your CSV into an SQL database and working from there instead.

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

Pythonic infinite for loop? Chris Angelico <rosuav@gmail.com> - 2011-04-15 12:10 +1000
  Re: Pythonic infinite for loop? Nobody <nobody@nowhere.com> - 2011-04-15 03:33 +0100
  Re: Pythonic infinite for loop? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-15 02:33 +0000
    Re: Pythonic infinite for loop? Chris Angelico <rosuav@gmail.com> - 2011-04-15 13:58 +1000
      Re: Pythonic infinite for loop? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-15 12:52 +0000
        Re: Pythonic infinite for loop? Roy Smith <roy@panix.com> - 2011-04-15 09:13 -0400
        Re: Pythonic infinite for loop? Chris Angelico <rosuav@gmail.com> - 2011-04-16 01:35 +1000
          Re: Pythonic infinite for loop? Paul Rubin <no.email@nospam.invalid> - 2011-04-15 09:10 -0700
  Re: Pythonic infinite for loop? Paul Rubin <no.email@nospam.invalid> - 2011-04-15 00:24 -0700
    Re: Pythonic infinite for loop? Chris Angelico <rosuav@gmail.com> - 2011-04-15 17:35 +1000
      Re: Pythonic infinite for loop? Paul Rubin <no.email@nospam.invalid> - 2011-04-15 00:47 -0700
        Re: Pythonic infinite for loop? Peter Otten <__peter__@web.de> - 2011-04-15 10:11 +0200
  Re: Pythonic infinite for loop? Peter Otten <__peter__@web.de> - 2011-04-15 10:25 +0200
    Re: Pythonic infinite for loop? Chris Angelico <rosuav@gmail.com> - 2011-04-15 18:32 +1000
      Re: Pythonic infinite for loop? Peter Otten <__peter__@web.de> - 2011-04-15 11:32 +0200
      Re: Pythonic infinite for loop? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-15 12:48 +0000

csiph-web