Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #33121

Re: A gnarly little python loop

Show key headers only | View raw

Steve Howell, 11.11.2012 04:03:
> On Nov 10, 2:58 pm, Roy Smith <r...@panix.com> wrote:
>> I'm trying to pull down tweets with one of the many twitter APIs.  The
>> particular one I'm using (python-twitter), has a call:
>>
>> data = api.GetSearch(term="foo", page=page)
>>
>> The way it works, you start with page=1.  It returns a list of tweets.
>> If the list is empty, there are no more tweets.  If the list is not
>> empty, you can try to get more tweets by asking for page=2, page=3, etc.
>> I've got:
>>
>>     page = 1
>>     while 1:
>>         r = api.GetSearch(term="foo", page=page)
>>         if not r:
>>             break
>>         for tweet in r:
>>             process(tweet)
>>         page += 1
>>
>> It works, but it seems excessively fidgety.  Is there some cleaner way
>> to refactor this?
> 
> I think your code is perfectly readable and clean, but you can flatten
> it like so:
> 
>     def get_tweets(term, get_page):
>         page_nums = itertools.count(1)
>         pages = itertools.imap(api.getSearch, page_nums)
>         valid_pages = itertools.takewhile(bool, pages)
>         tweets = itertools.chain.from_iterable(valid_pages)
>         return tweets

I'd prefer the original code ten times over this inaccessible beast.

Stefan

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

A gnarly little python loop Roy Smith <roy@panix.com> - 2012-11-10 17:58 -0500
  Re: A gnarly little python loop Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-10 16:17 -0700
  Re: A gnarly little python loop Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-11-11 00:23 +0000
  Re: A gnarly little python loop Steve Howell <showell@domaintools.com> - 2012-11-10 19:03 -0800
    Re: A gnarly little python loop Stefan Behnel <stefan_ml@behnel.de> - 2012-11-11 08:56 +0100
  Re: A gnarly little python loop rusi <rustompmody@gmail.com> - 2012-11-11 23:09 -0800
    Re: A gnarly little python loop rusi <rustompmody@gmail.com> - 2012-11-12 07:21 -0800
      Re: A gnarly little python loop Peter Otten <__peter__@web.de> - 2012-11-12 16:49 +0100
      Re: A gnarly little python loop Steve Howell <showell30@yahoo.com> - 2012-11-12 08:09 -0800
        Re: A gnarly little python loop rusi <rustompmody@gmail.com> - 2012-11-12 20:14 -0800

csiph-web