Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #47603 > unrolled thread

Split a list into two parts based on a filter?

Back to article view | Back to comp.lang.python

Contents

  Split a list into two parts based on a filter? roy@panix.com (Roy Smith) - 2013-06-10 16:34 -0400
    Re: Split a list into two parts based on a filter? Chris Angelico <rosuav@gmail.com> - 2013-06-11 08:50 +1000
    Re: Split a list into two parts based on a filter? Roel Schroeven <roel@roelschroeven.net> - 2013-06-11 00:50 +0200
      Re: Split a list into two parts based on a filter? Roy Smith <roy@panix.com> - 2013-06-11 00:11 -0400
        Re: Split a list into two parts based on a filter? Serhiy Storchaka <storchaka@gmail.com> - 2013-06-11 18:27 +0300
          Re: Split a list into two parts based on a filter? Roy Smith <roy@panix.com> - 2013-06-11 20:33 -0400
            Re: Split a list into two parts based on a filter? Phil Connell <pconnell@gmail.com> - 2013-06-12 07:32 +0100
              Re: Split a list into two parts based on a filter? Roy Smith <roy@panix.com> - 2013-06-12 07:39 -0400
                Re: Split a list into two parts based on a filter? Fábio Santos <fabiosantosart@gmail.com> - 2013-06-12 12:51 +0100
                Re: Split a list into two parts based on a filter? Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2013-06-12 15:06 +0300
                Re: Split a list into two parts based on a filter? Terry Reedy <tjreedy@udel.edu> - 2013-06-12 14:07 -0400
            Re: Split a list into two parts based on a filter? Serhiy Storchaka <storchaka@gmail.com> - 2013-06-12 19:28 +0300
            Re: Split a list into two parts based on a filter? Fábio Santos <fabiosantosart@gmail.com> - 2013-06-12 17:57 +0100
            Re: Split a list into two parts based on a filter? Terry Reedy <tjreedy@udel.edu> - 2013-06-12 14:47 -0400
            Re: Split a list into two parts based on a filter? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2013-06-13 10:43 +0100
    Re: Split a list into two parts based on a filter? Chris Rebert <clp2@rebertia.com> - 2013-06-10 16:03 -0700
    Re: Split a list into two parts based on a filter? Tim Chase <python.list@tim.thechases.com> - 2013-06-10 18:10 -0500
    Re: Split a list into two parts based on a filter? Fábio Santos <fabiosantosart@gmail.com> - 2013-06-11 00:08 +0100
      Re: Split a list into two parts based on a filter? alex23 <wuwei23@gmail.com> - 2013-06-11 17:44 -0700
    Re: Split a list into two parts based on a filter? Chris Angelico <rosuav@gmail.com> - 2013-06-11 09:12 +1000
    Re: Split a list into two parts based on a filter? Peter Otten <__peter__@web.de> - 2013-06-11 02:11 +0200
    Re: Split a list into two parts based on a filter? Peter Otten <__peter__@web.de> - 2013-06-11 08:43 +0200
    Re: Split a list into two parts based on a filter? Jonas Geiregat <jonas@geiregat.org> - 2013-06-11 08:47 +0200
    Re: Split a list into two parts based on a filter? Fábio Santos <fabiosantosart@gmail.com> - 2013-06-11 14:48 +0100
      Re: Split a list into two parts based on a filter? rusi <rustompmody@gmail.com> - 2013-06-11 09:37 -0700
        Re: Split a list into two parts based on a filter? Fábio Santos <fabiosantosart@gmail.com> - 2013-06-11 18:05 +0100
          Re: Split a list into two parts based on a filter? rusi <rustompmody@gmail.com> - 2013-06-11 10:23 -0700
            Re: Split a list into two parts based on a filter? Chris Angelico <rosuav@gmail.com> - 2013-06-12 03:37 +1000
              Re: Split a list into two parts based on a filter? rusi <rustompmody@gmail.com> - 2013-06-11 11:13 -0700
            Re: Split a list into two parts based on a filter? Fábio Santos <fabiosantosart@gmail.com> - 2013-06-11 19:05 +0100
    Re: Split a list into two parts based on a filter? Joshua Landau <joshua.landau.ws@gmail.com> - 2013-06-11 15:22 +0100
    Re: Split a list into two parts based on a filter? Serhiy Storchaka <storchaka@gmail.com> - 2013-06-11 18:28 +0300
    Re: Split a list into two parts based on a filter? Chris Angelico <rosuav@gmail.com> - 2013-06-12 03:28 +1000
      Re: Split a list into two parts based on a filter? Roy Smith <roy@panix.com> - 2013-06-11 20:12 -0400
    Re: Split a list into two parts based on a filter? Peter Otten <__peter__@web.de> - 2013-06-11 20:13 +0200
    Re: Split a list into two parts based on a filter? Peter Otten <__peter__@web.de> - 2013-06-11 20:18 +0200
    Re: Split a list into two parts based on a filter? Chris Angelico <rosuav@gmail.com> - 2013-06-12 04:27 +1000
    Re: Split a list into two parts based on a filter? Roel Schroeven <roel@roelschroeven.net> - 2013-06-11 22:22 +0200

Page 2 of 2 — ← Prev page 1 [2]

#47621

Chris Angelico wrote:

> On Tue, Jun 11, 2013 at 6:34 AM, Roy Smith <roy@panix.com> wrote:
>> new_songs = [s for s in songs if s.is_new()]
>> old_songs = [s for s in songs if not s.is_new()]
> 
> Hmm. Would this serve?
> 
> old_songs = songs[:]
> new_songs = [songs.remove(s) or s for s in songs if s.is_new()]
> 
> Python doesn't, AFAIK, have a "destructive remove and return"
> operation, and del is a statement rather than an expression/operator,
> but maybe this basic idea could be refined into something more useful.
> It guarantees to call is_new only once per song.
> 
> The iterator version strikes my fancy. Maybe this isn't of use to you,
> but I'm going to try my hand at making one anyway.

> >>> def iterpartition(pred,it):
>         """Partition an iterable based on a predicate.
> 
>         Returns two iterables, for those with pred False and those 
True."""
>         falses,trues=[],[]
>         it=iter(it)
>         def get_false():
>                 while True:
>                         if falses: yield falses.pop(0)
>                         else:
>                                 while True:
>                                         val=next(it)
>                                         if pred(val): trues.append(val)
>                                         else: break
>                                 yield val
>         def get_true():
>                 while True:
>                         if trues: yield trues.pop(0)
>                         else:
>                                 while True:
>                                         val=next(it)
>                                         if not pred(val): 
falses.append(val)
>                                         else: break
>                                 yield val
>         return get_false(),get_true()

An alternative implementation, based on itertools.tee:

import itertools

def partition(items, predicate=bool):
    a, b = itertools.tee((predicate(item), item) for item in items)
    return ((item for pred, item in a if not pred),
            (item for pred, item in b if pred))

if __name__ == "__main__":
    false, true = partition(range(10), lambda item: item % 2)
    print(list(false))
    print(list(true))

    def echo_odd(item):
        print("checking", item)
        return item % 2

    false, true = partition(range(10), echo_odd)

    print("FALSE", [next(false) for _ in range(3)])
    print("TRUE", next(true))
    print("FALSE", list(false))
    print("TRUE", list(true))

[toc] | [prev] | [next] | [standalone]

#47640

Fábio Santos wrote:

> On 10 Jun 2013 23:54, "Roel Schroeven" <roel@roelschroeven.net> wrote:
>>
>> You could do something like:
>>
>> new_songs, old_songs = [], []
>> [(new_songs if s.is_new() else old_songs).append(s) for s in songs]
>>
>> But I'm not sure that that's any better than the long version.
> 
> This is so beautiful!

It makes me cringe. 

This code does spurious work to turn what is naturally written as a for loop 
into a list comprehension that is thrown away immediately. I think I'd even 
prefer this "gem"

>>> evens = []
>>> odds = [item for item in range(10) if item % 2 or evens.append(item)]
>>> evens, odds
([0, 2, 4, 6, 8], [1, 3, 5, 7, 9])

but if I have my way every side effect in a list comprehension should be 
punished with an extra hour of bug-hunting ;)

[toc] | [prev] | [next] | [standalone]

#47642

[Multipart message — attachments visible in raw view] — view raw

On 11 Jun 2013, at 01:08, Fábio Santos wrote:

> 
> On 10 Jun 2013 23:54, "Roel Schroeven" <roel@roelschroeven.net> wrote:
> >
> > You could do something like:
> >
> > new_songs, old_songs = [], []
> > [(new_songs if s.is_new() else old_songs).append(s) for s in songs]
> >
> > But I'm not sure that that's any better than the long version.
> 
> This is so beautiful!
> 
> 

I must disagree , this is unreadable and in my honor opinion not Pythonic at all.
I've learned it's always better to be explicit then implicit, and this snippet of code does a lot in an implicit way.

[toc] | [prev] | [next] | [standalone]

#47661

[Multipart message — attachments visible in raw view] — view raw

On 11 Jun 2013 07:47, "Peter Otten" <__peter__@web.de> wrote:
>
> Fábio Santos wrote:
>
> > On 10 Jun 2013 23:54, "Roel Schroeven" <roel@roelschroeven.net> wrote:
> >>
> >> You could do something like:
> >>
> >> new_songs, old_songs = [], []
> >> [(new_songs if s.is_new() else old_songs).append(s) for s in songs]
> >>
> >> But I'm not sure that that's any better than the long version.
> >
> > This is so beautiful!
>
> It makes me cringe.
>
> This code does spurious work to turn what is naturally written as a for
loop
> into a list comprehension that is thrown away immediately. I think I'd
even
> prefer this "gem"
>
> >>> evens = []
> >>> odds = [item for item in range(10) if item % 2 or evens.append(item)]
> >>> evens, odds
> ([0, 2, 4, 6, 8], [1, 3, 5, 7, 9])
>
> but if I have my way every side effect in a list comprehension should be
> punished with an extra hour of bug-hunting ;)
>

What I like so much about it is the .. if .. else .. Within the parenthesis
and the append() call outside these parenthesis.

I agree it would be best written as a for loop, to increase readability and
avoid confusion. I always expect list comprehensions to be used as
expressions, and its use as a (single-expression) statement is rather odd.

On 11 Jun 2013 07:52, "Jonas Geiregat" <jonas@geiregat.org> wrote:
> I must disagree , this is unreadable and in my honor opinion not Pythonic
at all.
> I've learned it's always better to be explicit then implicit, and this
snippet of code does a lot in an implicit way.
>

(Read above)

[toc] | [prev] | [next] | [standalone]

#47676

On Jun 11, 6:48 pm, Fábio Santos <fabiosantos...@gmail.com> wrote:
>
> What I like so much about it is the .. if .. else .. Within the parenthesis
> and the append() call outside these parenthesis.

You can do this -- which does not mix up functional and imperative
styles badly and is as much a 2-liner as Roy's original.

new_songs, old_songs = [], []
for s in songs: (new_songs if s.is_new() else old_songs).append(s)

[Of course I would prefer a 3-liner where the body of the for is
indented :-) ]

[toc] | [prev] | [next] | [standalone]

#47680

[Multipart message — attachments visible in raw view] — view raw

On 11 Jun 2013 17:47, "rusi" <rustompmody@gmail.com> wrote:
> [Of course I would prefer a 3-liner where the body of the for is
> indented :-) ]

Is this an aside comprehension?

[toc] | [prev] | [next] | [standalone]

#47684

On Jun 11, 10:05 pm, Fábio Santos <fabiosantos...@gmail.com> wrote:
> On 11 Jun 2013 17:47, "rusi" <rustompm...@gmail.com> wrote:
>
> > [Of course I would prefer a 3-liner where the body of the for is
> > indented :-) ]
>
> Is this an aside comprehension?

Eh?

Its a for-loop. Same as:
for s in songs:
   (new_songs if s.is_new() else old_songs).append(s)

Maybe you are unfamiliar with the line
... A suite can be one or more semicolon-separated simple statements
on the same line as the header,
??

from http://docs.python.org/2/reference/compound_stmts.html

Or you are joking??

What with the offside rule (indentation rule in Haskell) becoming the
'aside rule' causing (in)comprehension in reading comprehensions??

Ok... Jokes aside... Dunno what you are asking.

[toc] | [prev] | [next] | [standalone]

#47688

On Wed, Jun 12, 2013 at 3:23 AM, rusi <rustompmody@gmail.com> wrote:
> On Jun 11, 10:05 pm, Fábio Santos <fabiosantos...@gmail.com> wrote:
>> On 11 Jun 2013 17:47, "rusi" <rustompm...@gmail.com> wrote:
>>
>> > [Of course I would prefer a 3-liner where the body of the for is
>> > indented :-) ]
>>
>> Is this an aside comprehension?
>
> Eh?

I know they always say "don't explain the joke", but I'll have a shot
at it. It's somewhat like an autopsy though - you find out why it
ticks, but in the process, you prove that it's no longer ticking...

An aside comprehension is a means of generating an aside in-line, as
an expression. In this case, Fabio believes that you were iterating
over the smiley to generate comments about three-liners, and the
square brackets delimit the comprehension just as they do in a list
comprehension.

It's another case of code syntax cropping up in English, like this
example of ternary punctuation from a C++ program...

//TODO?: blah blah blah

That doesn't translate too well into Python though.

ChrisA

[toc] | [prev] | [next] | [standalone]

#47691

On Jun 11, 10:37 pm, Chris Angelico <ros...@gmail.com> wrote:
> On Wed, Jun 12, 2013 at 3:23 AM, rusi <rustompm...@gmail.com> wrote:
> > On Jun 11, 10:05 pm, Fábio Santos <fabiosantos...@gmail.com> wrote:
> >> On 11 Jun 2013 17:47, "rusi" <rustompm...@gmail.com> wrote:
>
> >> > [Of course I would prefer a 3-liner where the body of the for is
> >> > indented :-) ]
>
> >> Is this an aside comprehension?
>
> > Eh?
>
> I know they always say "don't explain the joke", but I'll have a shot
> at it. It's somewhat like an autopsy though - you find out why it
> ticks, but in the process, you prove that it's no longer ticking...

Ok :-)

Missed that my aside looked like a comprehension. Or rather did not
comprehend!

[toc] | [prev] | [next] | [standalone]

#47689

[Multipart message — attachments visible in raw view] — view raw

On 11 Jun 2013 18:48, "Chris Angelico" <rosuav@gmail.com> wrote:
>
> On Wed, Jun 12, 2013 at 3:23 AM, rusi <rustompmody@gmail.com> wrote:
> > On Jun 11, 10:05 pm, Fábio Santos <fabiosantos...@gmail.com> wrote:
> >> On 11 Jun 2013 17:47, "rusi" <rustompm...@gmail.com> wrote:
> >>
> >> > [Of course I would prefer a 3-liner where the body of the for is
> >> > indented :-) ]
> >>
> >> Is this an aside comprehension?
> >
> > Eh?
>
> I know they always say "don't explain the joke", but I'll have a shot
> at it. It's somewhat like an autopsy though - you find out why it
> ticks, but in the process, you prove that it's no longer ticking...
>
> An aside comprehension is a means of generating an aside in-line, as
> an expression. In this case, Fabio believes that you were iterating
> over the smiley to generate comments about three-liners, and the
> square brackets delimit the comprehension just as they do in a list
> comprehension.
>
> It's another case of code syntax cropping up in English, like this
> example of ternary punctuation from a C++ program...
>
> //TODO?: blah blah blah
>
> That doesn't translate too well into Python though.
>
> ChrisA
> --
> http://mail.python.org/mailman/listinfo/python-list

That was the joke. Unfortunately I didn't have a better synonym for "aside"
so it wasn't so obvious.

[toc] | [prev] | [next] | [standalone]

#47662

On 11 June 2013 01:11, Peter Otten <__peter__@web.de> wrote:
> def partition(items, predicate=bool):
>     a, b = itertools.tee((predicate(item), item) for item in items)
>     return ((item for pred, item in a if not pred),
>             (item for pred, item in b if pred))

I have to tell you this is the coolest thing I've read today. I'd
never have thought of that.

[toc] | [prev] | [next] | [standalone]

#47666

11.06.13 01:50, Chris Angelico написав(ла):
> On Tue, Jun 11, 2013 at 6:34 AM, Roy Smith <roy@panix.com> wrote:
>> new_songs = [s for s in songs if s.is_new()]
>> old_songs = [s for s in songs if not s.is_new()]
>
> Hmm. Would this serve?
>
> old_songs = songs[:]
> new_songs = [songs.remove(s) or s for s in songs if s.is_new()]

O(len(songs)**2) complexity.

[toc] | [prev] | [next] | [standalone]

#47686

On Wed, Jun 12, 2013 at 1:28 AM, Serhiy Storchaka <storchaka@gmail.com> wrote:
> 11.06.13 01:50, Chris Angelico написав(ла):
>
>> On Tue, Jun 11, 2013 at 6:34 AM, Roy Smith <roy@panix.com> wrote:
>>>
>>> new_songs = [s for s in songs if s.is_new()]
>>> old_songs = [s for s in songs if not s.is_new()]
>>
>>
>> Hmm. Would this serve?
>>
>> old_songs = songs[:]
>> new_songs = [songs.remove(s) or s for s in songs if s.is_new()]
>
>
> O(len(songs)**2) complexity.

Which isn't significant if len(songs) is low. We weren't told the
relative costs - is the is_new call ridiculously expensive? Everything
affects algorithmic choice.

ChrisA

[toc] | [prev] | [next] | [standalone]

#47715

In article <mailman.3032.1370971724.3114.python-list@python.org>,
 Chris Angelico <rosuav@gmail.com> wrote:

> On Wed, Jun 12, 2013 at 1:28 AM, Serhiy Storchaka <storchaka@gmail.com> wrote:
> > 11.06.13 01:50, Chris Angelico написав(ла):
> >
> >> On Tue, Jun 11, 2013 at 6:34 AM, Roy Smith <roy@panix.com> wrote:
> >>>
> >>> new_songs = [s for s in songs if s.is_new()]
> >>> old_songs = [s for s in songs if not s.is_new()]
> >>
> >>
> >> Hmm. Would this serve?
> >>
> >> old_songs = songs[:]
> >> new_songs = [songs.remove(s) or s for s in songs if s.is_new()]
> >
> >
> > O(len(songs)**2) complexity.

If I didn't want to make two passes over songs, I probably don't want 
something that's O(len(songs)^2) :-)

> Which isn't significant if len(songs) is low. We weren't told the
> relative costs - is the is_new call ridiculously expensive? Everything
> affects algorithmic choice.

Assume is_new() is cheap.  It's essentially:

return  (datetime.utcnow() - self.create_time) < [[a pre-defined timedelta]]

[toc] | [prev] | [next] | [standalone]

#47692

Chris Angelico wrote:

> On Wed, Jun 12, 2013 at 1:28 AM, Serhiy Storchaka <storchaka@gmail.com>
> wrote:
>> 11.06.13 01:50, Chris Angelico написав(ла):
>>
>>> On Tue, Jun 11, 2013 at 6:34 AM, Roy Smith <roy@panix.com> wrote:
>>>>
>>>> new_songs = [s for s in songs if s.is_new()]
>>>> old_songs = [s for s in songs if not s.is_new()]
>>>
>>>
>>> Hmm. Would this serve?
>>>
>>> old_songs = songs[:]
>>> new_songs = [songs.remove(s) or s for s in songs if s.is_new()]

I think you meant old_songs.remove(s).

>> O(len(songs)**2) complexity.
> 
> Which isn't significant if len(songs) is low. We weren't told the
> relative costs - is the is_new call ridiculously expensive? Everything
> affects algorithmic choice.

But is it correct? In the general case, no:

>>> numbers = [1, 1.0, 2.0, 2]
>>> ints = numbers[:]
>>> floats = [ints.remove(n) or n for n in numbers if isinstance(n, float)]
>>> floats
[1.0, 2.0]
>>> ints
[1.0, 2] # hmm

[toc] | [prev] | [next] | [standalone]

#47693

Joshua Landau wrote:

> On 11 June 2013 01:11, Peter Otten <__peter__@web.de> wrote:
>> def partition(items, predicate=bool):
>>     a, b = itertools.tee((predicate(item), item) for item in items)
>>     return ((item for pred, item in a if not pred),
>>             (item for pred, item in b if pred))
> 
> I have to tell you this is the coolest thing I've read today. I'd
> never have thought of that.

:)

[toc] | [prev] | [next] | [standalone]

#47694

On Wed, Jun 12, 2013 at 4:13 AM, Peter Otten <__peter__@web.de> wrote:
> Chris Angelico wrote:
>
>> On Wed, Jun 12, 2013 at 1:28 AM, Serhiy Storchaka <storchaka@gmail.com>
>> wrote:
>>> 11.06.13 01:50, Chris Angelico написав(ла):
>>>
>>>> On Tue, Jun 11, 2013 at 6:34 AM, Roy Smith <roy@panix.com> wrote:
>>>>>
>>>>> new_songs = [s for s in songs if s.is_new()]
>>>>> old_songs = [s for s in songs if not s.is_new()]
>>>>
>>>>
>>>> Hmm. Would this serve?
>>>>
>>>> old_songs = songs[:]
>>>> new_songs = [songs.remove(s) or s for s in songs if s.is_new()]
>
> I think you meant old_songs.remove(s).

Ah, yes, editing fail. I started by mutating the original list, then
thought "Oh, better to work with a copy"... and forgot to complete the
edit.

>>> O(len(songs)**2) complexity.
>>
>> Which isn't significant if len(songs) is low. We weren't told the
>> relative costs - is the is_new call ridiculously expensive? Everything
>> affects algorithmic choice.
>
> But is it correct? In the general case, no:
>
>>>> numbers = [1, 1.0, 2.0, 2]
>>>> ints = numbers[:]
>>>> floats = [ints.remove(n) or n for n in numbers if isinstance(n, float)]
>>>> floats
> [1.0, 2.0]
>>>> ints
> [1.0, 2] # hmm

Sure, but the implication of the original is that they're uniquely
identifiable. Anyway, it wasn't meant to be absolutely perfect, just
another notion getting thrown out there... and then thrown out :)

ChrisA

[toc] | [prev] | [next] | [standalone]

#47702

Peter Otten schreef:
> Fábio Santos wrote:
> 
>> On 10 Jun 2013 23:54, "Roel Schroeven" <roel@roelschroeven.net> wrote:
>>> You could do something like:
>>>
>>> new_songs, old_songs = [], []
>>> [(new_songs if s.is_new() else old_songs).append(s) for s in songs]
>>>
>>> But I'm not sure that that's any better than the long version.
>> This is so beautiful!
> 
> It makes me cringe. 
> 
> This code does spurious work to turn what is naturally written as a for loop 
> into a list comprehension that is thrown away immediately. 

I agree. I see two problems with it: it's not very readable or pythonic, 
and it creates a list that is not needed and is immediately thrown away.

I wrote that code just to see if I could make it work that way. In real 
code I would never write it that way, let's make that clear :)


-- 
"People almost invariably arrive at their beliefs not on the basis of
proof but on the basis of what they find attractive."
         -- Pascal Blaise

roel@roelschroeven.net

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.python

csiph-web