Re: Pythonic infinite for loop?

From	Paul Rubin <no.email@nospam.invalid>
Newsgroups	comp.lang.python
Subject	Re: Pythonic infinite for loop?
References	(1 earlier) <4da7ae8a$0$29986$c3e8da3$5496439d@news.astraweb.com> <BANLkTinV2NTUgGNDf+mM1R98vf2TmoQ5Rw@mail.gmail.com> <mailman.382.1302839905.9059.python-list@python.org> <4da83f8f$0$29986$c3e8da3$5496439d@news.astraweb.com> <mailman.396.1302881747.9059.python-list@python.org>
Date	2011-04-15 09:10 -0700
Message-ID	<7xmxjrxzpx.fsf@ruckus.brouhaha.com> (permalink)
Organization	Nightsong/Fort GNOX

Show all headers | View raw

Chris Angelico <rosuav@gmail.com> writes:
> This whole code is inside a loop that we took, in smoke testing, to a
> couple hundred million rows (I think), with the intention of having no
> limit at all. So this might only look at 60-100 headers, but it will
> be doing so in a tight loop.

If you're talking about data sets that large, you should rethink the
concept of using python dictionaries with a key for every row.  Try
importing your CSV into an SQL database and working from there instead.

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

Pythonic infinite for loop? Chris Angelico <rosuav@gmail.com> - 2011-04-15 12:10 +1000
  Re: Pythonic infinite for loop? Nobody <nobody@nowhere.com> - 2011-04-15 03:33 +0100
  Re: Pythonic infinite for loop? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-15 02:33 +0000
    Re: Pythonic infinite for loop? Chris Angelico <rosuav@gmail.com> - 2011-04-15 13:58 +1000
      Re: Pythonic infinite for loop? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-15 12:52 +0000
        Re: Pythonic infinite for loop? Roy Smith <roy@panix.com> - 2011-04-15 09:13 -0400
        Re: Pythonic infinite for loop? Chris Angelico <rosuav@gmail.com> - 2011-04-16 01:35 +1000
          Re: Pythonic infinite for loop? Paul Rubin <no.email@nospam.invalid> - 2011-04-15 09:10 -0700
  Re: Pythonic infinite for loop? Paul Rubin <no.email@nospam.invalid> - 2011-04-15 00:24 -0700
    Re: Pythonic infinite for loop? Chris Angelico <rosuav@gmail.com> - 2011-04-15 17:35 +1000
      Re: Pythonic infinite for loop? Paul Rubin <no.email@nospam.invalid> - 2011-04-15 00:47 -0700
        Re: Pythonic infinite for loop? Peter Otten <__peter__@web.de> - 2011-04-15 10:11 +0200
  Re: Pythonic infinite for loop? Peter Otten <__peter__@web.de> - 2011-04-15 10:25 +0200
    Re: Pythonic infinite for loop? Chris Angelico <rosuav@gmail.com> - 2011-04-15 18:32 +1000
      Re: Pythonic infinite for loop? Peter Otten <__peter__@web.de> - 2011-04-15 11:32 +0200
      Re: Pythonic infinite for loop? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-04-15 12:48 +0000

csiph-web