Re: Filter versus comprehension (was Re: something about split()???)

Newsgroups	comp.lang.python
Date	2012-08-24 07:44 -0700
References	(4 earlier) <CAExgZOiHi-mE1mVvxYOST5RDbosDn3fP1X2nxkyDiKk=0ajjSw@mail.gmail.com> <CADYZVBCZZ3cAHGDe=wyi62Hg9+qU_r2U8F+85gHEOzs0Ekv1BA@mail.gmail.com> <k11ro8$cer$1@ger.gmane.org> <k121ob$lp9$1@ger.gmane.org> <mailman.3665.1345653816.4697.python-list@python.org>
Subject	Re: Filter versus comprehension (was Re: something about split()???)
From	Ramchandra Apte <maniandram01@gmail.com>
Message-ID	<mailman.3757.1345819477.4697.python-list@python.org> (permalink)

Show all headers | View raw

On Wednesday, 22 August 2012 22:13:04 UTC+5:30, Terry Reedy  wrote:
> On 8/22/2012 3:30 AM, Mark Lawrence wrote:
> 
> > On 22/08/2012 06:46, Terry Reedy wrote:
> 
> >> On 8/21/2012 11:43 PM, mingqiang hu wrote:
> 
> >>> why filter is bad when use lambda ?
> 
> >>
> 
> >> Inefficient, not 'bad'. Because the equivalent comprehension or
> 
> >> generator expression does not require a function call.
> 
> 
> 
> for each item in the iterable.
> 
> 
> 
> > A case of premature optimisation? :)
> 
> 
> 
> No, as regards my post. I simply made a factual statement without 
> 
> advocating a particular action.
> 
> 
> 
> filter(lambda x: <expr>, iterable)
> 
> (x for x in iterable if <expr>)
> 
> 
> 
> both create iterators that produce the items in iterable such that 
> 
> bool(<expr>) is true. The following, with output rounded, shows 
> 
> something of the effect of the extra function call.
> 
> 
> 
>  >>> timeit.timeit("list(i for i in ranger if False)", "ranger=range(0)")
> 
> 0.91
> 
>  >>> timeit.timeit("list(i for i in ranger if False)", "ranger=range(20)")
> 
> 1.28
> 
>  >>> timeit.timeit("list(filter(lambda i: False, ranger))", 
> 
> "ranger=range(0)")
> 
> 0.83
> 
>  >>> timeit.timeit("list(filter(lambda i: False, ranger))", 
> 
> "ranger=range(20)")
> 
> 2.60
> 
> 
> 
> Simply keeping true items is faster with filter -- at least on my 
> 
> particular machine with 3.3.0b2.
> 
> 
> 
>  >>> timeit.timeit("list(filter(None, ranger))", "ranger=range(20)")
> 
> 1.03
> 
> 
> 
> Filter is also faster if the expression is a function call.
> 
> 
> 
>  >>> timeit.timeit("list(filter(f, ranger))", "ranger=range(20); 
> 
> f=lambda i: False")
> 
> 2.5033614114454394
> 
>  >>> timeit.timeit("list(i for i in ranger if f(i))", "ranger=range(20); 
> 
> f=lambda i: False")
> 
> 3.2394095327040304
> 
> 
> 
> ---
> 
> Perhaps or even yes as regards the so-called rule 'always use 
> 
> comprehension'. If one prefers filter as more readable, if one only 
> 
> wants to keep true items, if the expression is a function call, if 
> 
> evaluating the expression takes much more time than the extra function 
> 
> call so the latter does not matter, if the number of items is few enough 
> 
> that the extra time does not matter, then the rule is not needed or even 
> 
> wrong.
> 
> 
> 
> So I think PyLint should be changed to stop its filter fud.
> 
> 
> 
> -- 
> 
> Terry Jan Reedy

When filtering for true values, filter(None,xxx) can be used
Your examples with lambda i:False are unrealistic - you are comparing `if False` vs <lambda function>(xx) - function call vs boolean check

Back to comp.lang.python | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Filter versus comprehension (was Re: something about split()???) Terry Reedy <tjreedy@udel.edu> - 2012-08-22 12:43 -0400
  Re: Filter versus comprehension (was Re: something about split()???) Ramchandra Apte <maniandram01@gmail.com> - 2012-08-24 07:44 -0700
    Re: Filter versus comprehension (was Re: something about split()???) Terry Reedy <tjreedy@udel.edu> - 2012-08-24 12:04 -0400
    Re: Filter versus comprehension (was Re: something about split()???) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2012-08-24 14:29 -0400
      Re: Filter versus comprehension (was Re: something about split()???) Walter Hurry <walterhurry@lavabit.com> - 2012-08-24 19:03 +0000
        Re: Filter versus comprehension (was Re: something about split()???) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2012-08-24 17:56 -0400
          Re: Filter versus comprehension (was Re: something about split()???) Walter Hurry <walterhurry@lavabit.com> - 2012-08-24 22:55 +0000
        Re: Filter versus comprehension (was Re: something about split()???) Terry Reedy <tjreedy@udel.edu> - 2012-08-24 18:03 -0400
        Re: Filter versus comprehension (was Re: something about split()???) Emile van Sebille <emile@fenx.com> - 2012-08-24 15:15 -0700
        Re: Filter versus comprehension (was Re: something about split()???) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2012-08-24 23:28 +0100
        Re: Filter versus comprehension (was Re: something about split()???) Ned Deily <nad@acm.org> - 2012-08-24 15:36 -0700
        Re: Filter versus comprehension (was Re: something about split()???) Ned Deily <nad@acm.org> - 2012-08-24 15:39 -0700
        Re: Filter versus comprehension (was Re: something about split()???) David Robinow <drobinow@gmail.com> - 2012-08-25 08:57 -0400
        Re: Filter versus comprehension (was Re: something about split()???) Tim Golden <mail@timgolden.me.uk> - 2012-08-25 16:31 +0100
  Re: Filter versus comprehension (was Re: something about split()???) Ramchandra Apte <maniandram01@gmail.com> - 2012-08-24 07:44 -0700

csiph-web