Filter versus comprehension (was Re: something about split()???)

From	Terry Reedy <tjreedy@udel.edu>
Subject	Filter versus comprehension (was Re: something about split()???)
Date	2012-08-22 12:43 -0400
References	(3 earlier) <502A865E.4030504@sequans.com> <CAExgZOiHi-mE1mVvxYOST5RDbosDn3fP1X2nxkyDiKk=0ajjSw@mail.gmail.com> <CADYZVBCZZ3cAHGDe=wyi62Hg9+qU_r2U8F+85gHEOzs0Ekv1BA@mail.gmail.com> <k11ro8$cer$1@ger.gmane.org> <k121ob$lp9$1@ger.gmane.org>
Newsgroups	comp.lang.python
Message-ID	<mailman.3665.1345653816.4697.python-list@python.org> (permalink)

Show all headers | View raw

On 8/22/2012 3:30 AM, Mark Lawrence wrote:
> On 22/08/2012 06:46, Terry Reedy wrote:
>> On 8/21/2012 11:43 PM, mingqiang hu wrote:
>>> why filter is bad when use lambda ?
>>
>> Inefficient, not 'bad'. Because the equivalent comprehension or
>> generator expression does not require a function call.

for each item in the iterable.

> A case of premature optimisation? :)

No, as regards my post. I simply made a factual statement without 
advocating a particular action.

filter(lambda x: <expr>, iterable)
(x for x in iterable if <expr>)

both create iterators that produce the items in iterable such that 
bool(<expr>) is true. The following, with output rounded, shows 
something of the effect of the extra function call.

 >>> timeit.timeit("list(i for i in ranger if False)", "ranger=range(0)")
0.91
 >>> timeit.timeit("list(i for i in ranger if False)", "ranger=range(20)")
1.28
 >>> timeit.timeit("list(filter(lambda i: False, ranger))", 
"ranger=range(0)")
0.83
 >>> timeit.timeit("list(filter(lambda i: False, ranger))", 
"ranger=range(20)")
2.60

Simply keeping true items is faster with filter -- at least on my 
particular machine with 3.3.0b2.

 >>> timeit.timeit("list(filter(None, ranger))", "ranger=range(20)")
1.03

Filter is also faster if the expression is a function call.

 >>> timeit.timeit("list(filter(f, ranger))", "ranger=range(20); 
f=lambda i: False")
2.5033614114454394
 >>> timeit.timeit("list(i for i in ranger if f(i))", "ranger=range(20); 
f=lambda i: False")
3.2394095327040304

---
Perhaps or even yes as regards the so-called rule 'always use 
comprehension'. If one prefers filter as more readable, if one only 
wants to keep true items, if the expression is a function call, if 
evaluating the expression takes much more time than the extra function 
call so the latter does not matter, if the number of items is few enough 
that the extra time does not matter, then the rule is not needed or even 
wrong.

So I think PyLint should be changed to stop its filter fud.

-- 
Terry Jan Reedy

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Filter versus comprehension (was Re: something about split()???) Terry Reedy <tjreedy@udel.edu> - 2012-08-22 12:43 -0400
  Re: Filter versus comprehension (was Re: something about split()???) Ramchandra Apte <maniandram01@gmail.com> - 2012-08-24 07:44 -0700
    Re: Filter versus comprehension (was Re: something about split()???) Terry Reedy <tjreedy@udel.edu> - 2012-08-24 12:04 -0400
    Re: Filter versus comprehension (was Re: something about split()???) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2012-08-24 14:29 -0400
      Re: Filter versus comprehension (was Re: something about split()???) Walter Hurry <walterhurry@lavabit.com> - 2012-08-24 19:03 +0000
        Re: Filter versus comprehension (was Re: something about split()???) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2012-08-24 17:56 -0400
          Re: Filter versus comprehension (was Re: something about split()???) Walter Hurry <walterhurry@lavabit.com> - 2012-08-24 22:55 +0000
        Re: Filter versus comprehension (was Re: something about split()???) Terry Reedy <tjreedy@udel.edu> - 2012-08-24 18:03 -0400
        Re: Filter versus comprehension (was Re: something about split()???) Emile van Sebille <emile@fenx.com> - 2012-08-24 15:15 -0700
        Re: Filter versus comprehension (was Re: something about split()???) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2012-08-24 23:28 +0100
        Re: Filter versus comprehension (was Re: something about split()???) Ned Deily <nad@acm.org> - 2012-08-24 15:36 -0700
        Re: Filter versus comprehension (was Re: something about split()???) Ned Deily <nad@acm.org> - 2012-08-24 15:39 -0700
        Re: Filter versus comprehension (was Re: something about split()???) David Robinow <drobinow@gmail.com> - 2012-08-25 08:57 -0400
        Re: Filter versus comprehension (was Re: something about split()???) Tim Golden <mail@timgolden.me.uk> - 2012-08-25 16:31 +0100
  Re: Filter versus comprehension (was Re: something about split()???) Ramchandra Apte <maniandram01@gmail.com> - 2012-08-24 07:44 -0700

csiph-web