Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #33133

Re: A gnarly little python loop

Show all headers | View raw

On Sunday, November 11, 2012 1:54:46 AM UTC-8, Peter Otten wrote:
> Paul Rubin wrote:
> 
> 
> 
> > Cameron Simpson <cs@zip.com.au> writes:
> 
> >> | I'd prefer the original code ten times over this inaccessible beast.
> 
> >> Me too.
> 
> > 
> 
> > Me, I like the itertools version better.  There's one chunk of data
> 
> > that goes through a succession of transforms each of which
> 
> > is very straightforward.
> 
> 
> 
> [Steve Howell]
> 
> >     def get_tweets(term, get_page):
> 
> >         page_nums = itertools.count(1)
> 
> >         pages = itertools.imap(api.getSearch, page_nums)
> 
> >         valid_pages = itertools.takewhile(bool, pages)
> 
> >         tweets = itertools.chain.from_iterable(valid_pages)
> 
> >         return tweets
> 
>  
> 
> 
> 
> But did you spot the bug(s)?
> 

My first version was sketching out the technique, and I don't have handy access to the API.

Here is an improved version:

    def get_tweets(term):
        def get_page(page):
            return getSearch(term, page)
        page_nums = itertools.count(1)
        pages = itertools.imap(get_page, page_nums)
        valid_pages = itertools.takewhile(bool, pages)
        tweets = itertools.chain.from_iterable(valid_pages)
        return tweets

    for tweet in get_tweets("foo"):
            process(tweet)

This is what I used to test it:


    def getSearch(term = "foo", page = 1):
        # simulate api for testing
        if page < 5:
            return [
                'page %d, tweet A for term %s' % (page, term),
                'page %d, tweet B for term %s' % (page, term),
            ]
        else:
            return None

    def process(tweet):
        print tweet

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: A gnarly little python loop Cameron Simpson <cs@zip.com.au> - 2012-11-11 19:48 +1100
  Re: A gnarly little python loop Paul Rubin <no.email@nospam.invalid> - 2012-11-11 01:09 -0800
    Re: A gnarly little python loop Peter Otten <__peter__@web.de> - 2012-11-11 10:54 +0100
      Re: A gnarly little python loop Steve Howell <showell30@yahoo.com> - 2012-11-11 09:16 -0800
      Re: A gnarly little python loop Steve Howell <showell30@yahoo.com> - 2012-11-11 09:16 -0800
    Re: A gnarly little python loop Steve Howell <showell30@yahoo.com> - 2012-11-11 09:29 -0800
      Re: A gnarly little python loop Peter Otten <__peter__@web.de> - 2012-11-11 19:34 +0100
        Re: A gnarly little python loop Steve Howell <showell30@yahoo.com> - 2012-11-11 11:16 -0800
          Re: A gnarly little python loop Cameron Simpson <cs@zip.com.au> - 2012-11-12 11:43 +1100
            Re: A gnarly little python loop Steve Howell <showell30@yahoo.com> - 2012-11-11 17:38 -0800
        Re: A gnarly little python loop Roy Smith <roy@panix.com> - 2012-11-11 14:23 -0500

csiph-web