Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #33139

Re: A gnarly little python loop

Show all headers | View raw

Steve Howell wrote:

> On Nov 11, 1:09 am, Paul Rubin <no.em...@nospam.invalid> wrote:
>> Cameron Simpson <c...@zip.com.au> writes:
>> > | I'd prefer the original code ten times over this inaccessible beast.
>> > Me too.
>>
>> Me, I like the itertools version better.  There's one chunk of data
>> that goes through a succession of transforms each of which
>> is very straightforward.
> 
> Thanks, Paul.
> 
> Even though I supplied the "inaccessible" itertools version, I can
> understand why folks find it inaccessible.  As I said to the OP, there
> was nothing wrong with the original imperative approach; I was simply
> providing an alternative.
> 
> It took me a while to appreciate itertools, but the metaphor that
> resonates with me is a Unix pipeline.  It's just a metaphor, so folks
> shouldn't be too literal, but the idea here is this:
> 
>   page_nums -> pages -> valid_pages -> tweets
> 
> The transforms are this:
> 
>   page_nums -> pages: call API via imap
>   pages -> valid_pages: take while true
>   valid_pages -> tweets: use chain.from_iterable to flatten results
> 
> Here's the code again for context:
> 
>     def get_tweets(term):
>         def get_page(page):
>             return getSearch(term, page)
>         page_nums = itertools.count(1)
>         pages = itertools.imap(get_page, page_nums)
>         valid_pages = itertools.takewhile(bool, pages)
>         tweets = itertools.chain.from_iterable(valid_pages)
>         return tweets
> 

Actually you supplied the "accessible" itertools version. For reference, 
here's the inaccessible version:

class api:
    """Twitter search API mock-up"""
    pages = [
        ["a", "b", "c"],
        ["d", "e"],
        ]
    @staticmethod
    def GetSearch(term, page):
        assert term == "foo"
        assert page >= 1
        if page > len(api.pages):
            return []
        return api.pages[page-1]

from collections import deque
from functools import partial
from itertools import chain, count, imap, takewhile

def process(tweet):
    print tweet

term = "foo"

deque(
    imap(
        process,
        chain.from_iterable(
            takewhile(bool, imap(partial(api.GetSearch, term), count(1))))),
    maxlen=0)

;)

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: A gnarly little python loop Cameron Simpson <cs@zip.com.au> - 2012-11-11 19:48 +1100
  Re: A gnarly little python loop Paul Rubin <no.email@nospam.invalid> - 2012-11-11 01:09 -0800
    Re: A gnarly little python loop Peter Otten <__peter__@web.de> - 2012-11-11 10:54 +0100
      Re: A gnarly little python loop Steve Howell <showell30@yahoo.com> - 2012-11-11 09:16 -0800
      Re: A gnarly little python loop Steve Howell <showell30@yahoo.com> - 2012-11-11 09:16 -0800
    Re: A gnarly little python loop Steve Howell <showell30@yahoo.com> - 2012-11-11 09:29 -0800
      Re: A gnarly little python loop Peter Otten <__peter__@web.de> - 2012-11-11 19:34 +0100
        Re: A gnarly little python loop Steve Howell <showell30@yahoo.com> - 2012-11-11 11:16 -0800
          Re: A gnarly little python loop Cameron Simpson <cs@zip.com.au> - 2012-11-12 11:43 +1100
            Re: A gnarly little python loop Steve Howell <showell30@yahoo.com> - 2012-11-11 17:38 -0800
        Re: A gnarly little python loop Roy Smith <roy@panix.com> - 2012-11-11 14:23 -0500

csiph-web