Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #44085

Re: itertools.groupby

Show all headers | View raw

On 2013-04-22, Oscar Benjamin <oscar.j.benjamin@gmail.com> wrote:
> On 22 April 2013 15:24, Neil Cerutti <neilc@norwich.edu> wrote:
>>
>> Hrmmm, hoomm. Nobody cares for slicing any more.
>>
>> def headered_groups(lst, header):
>>     b = lst.index(header) + 1
>>     while True:
>>         try:
>>             e = lst.index(header, b)
>>         except ValueError:
>>             yield lst[b:]
>>             break
>>         yield lst[b:e]
>>         b = e+1
>
> This requires the whole file to be read into memory. Iterators
> are typically preferred over list slicing for sequential text
> file access since you can avoid loading the whole file at once.
> This means that you can process a large file while only using a
> constant amount of memory.

I agree, but this application processes unknowns-sized slices,
you have to build lists anyhow. I find slicing much more
convenient than accumulating in this case, but it's possibly a
tradeoff.

> with open('data.txt') as inputfile:
>     for group in headered_groups(map(str.strip, inputfile)):
>         print(group)

Thanks, that's a nice improvement.

-- 
Neil Cerutti

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

itertools.groupby Jason Friedman <jsf80238@gmail.com> - 2013-04-20 11:09 -0600
  Re: itertools.groupby Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-04-21 00:13 +0000
    Re: itertools.groupby Joshua Landau <joshua.landau.ws@gmail.com> - 2013-04-22 04:09 +0100
  Re: itertools.groupby Neil Cerutti <neilc@norwich.edu> - 2013-04-22 14:24 +0000
    Re: itertools.groupby Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2013-04-22 15:49 +0100
      Re: itertools.groupby Neil Cerutti <neilc@norwich.edu> - 2013-04-22 15:04 +0000
    Re: itertools.groupby Chris Angelico <rosuav@gmail.com> - 2013-04-23 01:14 +1000

csiph-web