Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #64205 > unrolled thread

How to write this as a list comprehension?

Back to article view | Back to comp.lang.python

Contents

  How to write this as a list comprehension? Piet van Oostrum <piet@vanoostrum.org> - 2014-01-18 00:19 +0100
    Re: How to write this as a list comprehension? Dan Stromberg <drsalists@gmail.com> - 2014-01-17 15:49 -0800
    Re: How to write this as a list comprehension? Rustom Mody <rustompmody@gmail.com> - 2014-01-17 19:25 -0800
      Re: How to write this as a list comprehension? Piet van Oostrum <piet@vanoostrum.org> - 2014-01-18 11:00 +0100
    Re: How to write this as a list comprehension? Peter Otten <__peter__@web.de> - 2014-01-18 09:36 +0100
      Re: How to write this as a list comprehension? Rustom Mody <rustompmody@gmail.com> - 2014-01-18 07:20 -0800
        Re: How to write this as a list comprehension? Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2014-01-18 18:00 +0200
          Re: How to write this as a list comprehension? "Rhodri James" <rhodri@wildebst.org.uk> - 2014-01-19 23:41 +0000
            Re: How to write this as a list comprehension? Piet van Oostrum <piet@vanoostrum.org> - 2014-01-20 12:02 +0100
              Re: How to write this as a list comprehension? Rustom Mody <rustompmody@gmail.com> - 2014-01-20 03:47 -0800
    Re: How to write this as a list comprehension? matej@ceplovi.cz (Matěj Cepl) - 2014-01-18 11:57 +0100
    Re: How to write this as a list comprehension? Alain Ketterlin <alain@dpt-info.u-strasbg.fr> - 2014-01-18 12:53 +0100
      Re: How to write this as a list comprehension? Piet van Oostrum <piet@vanoostrum.org> - 2014-01-18 18:40 +0100
    Re: How to write this as a list comprehension? John Allsup <pydev@allsup.co> - 2014-01-19 01:24 +0000
      Re: How to write this as a list comprehension? Piet van Oostrum <piet@vanoostrum.org> - 2014-01-19 23:06 +0100

#64205 — How to write this as a list comprehension?

Hi,

I am looking for an elegant way to write the following code as a list
comprehension:

labels = []
for then, name in mylist:
    _, mn, dy, _, _, _, wd, _, _ = localtime(then)
    labels.append(somefunc(mn, day, wd, name))

So mylist is a list of tuples, the first member of the tuple is a time
(as epoch offset) and I neeed to apply a function on some fields of the
localtime of it.

I could define a auxiliary function like:

def auxfunc(then, name):
    _, mn, dy, _, _, _, wd, _, _ = localtime(then)
    return somefunc(mn, day, wd, name)

and then use 
[auxfunc(then, name) for then, name in mylist]

or even
[auxfunc(*tup) for tup in mylist]

But defining the auxfunc takes away the elegance of a list comprehension. I would like to integrate the unpacking of localtime() and calling somefunc within the list comprehension, but I don't see a simple way to do that.

somefunc(mn, day, wd, name) for _, mn, dy, _, _, _, wd, _, _ in [localtime(then)] 
(i.e. using a list comprehension on a one element list to do the variable shuffling) 
works but I don't find that very elegant.

labels = [somefunc(mn, day, wd, name) 
            for then, name in mylist
            for _, mn, dy, _, _, _, wd, _, _ in [localtime(then)]]

Python misses a 'where' or 'let'-like construction as in Haskell.

Anybody has a more elegant solution?
-- 
Piet van Oostrum <piet@vanoostrum.org>
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]

[toc] | [next] | [standalone]

#64206

On Fri, Jan 17, 2014 at 3:19 PM, Piet van Oostrum <piet@vanoostrum.org> wrote:
> Hi,
>
> I am looking for an elegant way to write the following code as a list
> comprehension:
>
> labels = []
> for then, name in mylist:
>     _, mn, dy, _, _, _, wd, _, _ = localtime(then)
>     labels.append(somefunc(mn, day, wd, name))

My recomendation: Don't use a list comprehension.  List comprehensions
and generator expressions are great for quick little things, but
become less readable when you have to string them over multiple
physical lines.

> labels = [somefunc(mn, day, wd, name)
>             for then, name in mylist
>             for _, mn, dy, _, _, _, wd, _, _ in [localtime(then)]]

[toc] | [prev] | [next] | [standalone]

#64214

On Saturday, January 18, 2014 4:49:55 AM UTC+5:30, Piet van Oostrum wrote:
> Hi,

> I am looking for an elegant way to write the following code as a list
> comprehension:

> labels = []
> for then, name in mylist:
>     _, mn, dy, _, _, _, wd, _, _ = localtime(then)
>     labels.append(somefunc(mn, day, wd, name))

> So mylist is a list of tuples, the first member of the tuple is a time
> (as epoch offset) and I neeed to apply a function on some fields of the
> localtime of it.

> I could define a auxiliary function like:

> def auxfunc(then, name):
>     _, mn, dy, _, _, _, wd, _, _ = localtime(then)
>     return somefunc(mn, day, wd, name)

> and then use 
> [auxfunc(then, name) for then, name in mylist]

> or even
> [auxfunc(*tup) for tup in mylist]

> But defining the auxfunc takes away the elegance of a list comprehension. I would like to integrate the unpacking of localtime() and calling somefunc within the list comprehension, but I don't see a simple way to do that.

> somefunc(mn, day, wd, name) for _, mn, dy, _, _, _, wd, _, _ in [localtime(then)] 
> (i.e. using a list comprehension on a one element list to do the variable shuffling) 
> works but I don't find that very elegant.

> labels = [somefunc(mn, day, wd, name) 
>             for then, name in mylist
>             for _, mn, dy, _, _, _, wd, _, _ in [localtime(then)]]

> Python misses a 'where' or 'let'-like construction as in Haskell.

+1
Yes Ive often been bitten by the lack of a 'comprehension-let'

Something like this is possible??


[somefunc(mn,day,wd,name) for (_, mn,dy,_,_,_,wd,_,_), name) in [localtime(then), name for then, name in mylist]]

Some debugging of the structure will be necessary (if at all possible)
I dont have your functions so cant do it

[toc] | [prev] | [next] | [standalone]

#64222

Rustom Mody <rustompmody@gmail.com> writes:

> On Saturday, January 18, 2014 4:49:55 AM UTC+5:30, Piet van Oostrum wrote:
[...]
>
>> Python misses a 'where' or 'let'-like construction as in Haskell.
>
> +1
> Yes Ive often been bitten by the lack of a 'comprehension-let'

If it used only in a comprehension as in my example you can write instead of 'where vars = expr':
   for vars in [expr], unnecessarily construction a one element list.
If there would be a syntax like:
   for vars = expr
this could be avoided.
>
> Something like this is possible??
>
>
> [somefunc(mn,day,wd,name) for (_, mn,dy,_,_,_,wd,_,_), name) in
> [localtime(then), name for then, name in mylist]]
It works modulo some corrections in the parentheses:

[somefunc(mn,day,wd,name) for (_, mn,dy,_,_,_,wd,_,_), name in
  [(localtime(then), name) for then, name in mylist]]
but I find that hardly more elegant.
-- 
Piet van Oostrum <piet@vanoostrum.org>
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]

[toc] | [prev] | [next] | [standalone]

#64218

Piet van Oostrum wrote:

> Hi,
> 
> I am looking for an elegant way to write the following code as a list
> comprehension:
> 
> labels = []
> for then, name in mylist:
>     _, mn, dy, _, _, _, wd, _, _ = localtime(then)
>     labels.append(somefunc(mn, day, wd, name))
> 
> So mylist is a list of tuples, the first member of the tuple is a time
> (as epoch offset) and I neeed to apply a function on some fields of the
> localtime of it.
> 
> I could define a auxiliary function like:
> 
> def auxfunc(then, name):
>     _, mn, dy, _, _, _, wd, _, _ = localtime(then)
>     return somefunc(mn, day, wd, name)
> 
> and then use
> [auxfunc(then, name) for then, name in mylist]
> 
> or even
> [auxfunc(*tup) for tup in mylist]
> 
> But defining the auxfunc takes away the elegance of a list comprehension.
> I would like to integrate the unpacking of localtime() and calling
> somefunc within the list comprehension, but I don't see a simple way to do
> that.
> 
> somefunc(mn, day, wd, name) for _, mn, dy, _, _, _, wd, _, _ in
> [localtime(then)] (i.e. using a list comprehension on a one element list
> to do the variable shuffling) works but I don't find that very elegant.
> 
> labels = [somefunc(mn, day, wd, name)
>             for then, name in mylist
>             for _, mn, dy, _, _, _, wd, _, _ in [localtime(then)]]
> 
> Python misses a 'where' or 'let'-like construction as in Haskell.
> 
> Anybody has a more elegant solution?

Options I can think of: 

You could do it in two steps...

time_name_pairs = ((localtime(then), name) for then, name in mylist)
labels = [somefunc(t.tm_mon, t.tm_mday, t.tm_wday, name) 
          for t, name in time_name_pairs]

...or you could inline the helper function...

mon_mday_wday = operator.attrgetter("tm_mon", "tm_day", "tm_wday")
labels = [somefunc(*mon_mday_wday(localtime(then)), name=name) 
          for then, name in mylist]

-- but both seem less readable than the classical for-loop.

What would a list-comp with `let` or `where` look like? Would it win the 
beauty contest against the loop?

[toc] | [prev] | [next] | [standalone]

#64234

On Saturday, January 18, 2014 2:06:29 PM UTC+5:30, Peter Otten wrote:

> Options I can think of: 

> You could do it in two steps...

> time_name_pairs = ((localtime(then), name) for then, name in mylist)
> labels = [somefunc(t.tm_mon, t.tm_mday, t.tm_wday, name) 
>           for t, name in time_name_pairs]

> ...or you could inline the helper function...

> mon_mday_wday = operator.attrgetter("tm_mon", "tm_day", "tm_wday")
> labels = [somefunc(*mon_mday_wday(localtime(then)), name=name) 
>           for then, name in mylist]

> -- but both seem less readable than the classical for-loop.

> What would a list-comp with `let` or `where` look like? Would it win the 
> beauty contest against the loop?

For me this is neat

[somefunc(mn,day,wd,name) for (then, name) in mylist let (_,mn,dy,_,_,_,wd,_,_) = localtime(then)]
   
Others may not find it so!

See it across > 1 line (as I guess it will come after being posted!) and its not so neat.

[toc] | [prev] | [next] | [standalone]

#64235

Rustom Mody writes:

> On Saturday, January 18, 2014 2:06:29 PM UTC+5:30, Peter Otten wrote:
>
> > What would a list-comp with `let` or `where` look like? Would it
> > win the beauty contest against the loop?
> 
> For me this is neat
> 
> [somefunc(mn,day,wd,name) for (then, name) in mylist let (_,mn,dy,_,_,_,wd,_,_) = localtime(then)]
>    
> Others may not find it so!
> 
> See it across > 1 line (as I guess it will come after being posted!)
> and its not so neat.

I would write that on three lines anyway, properly indented:

  [ somefunc(mn,day,wd,name)
    for (then, name) in mylist
    let (_,mn,dy,_,_,_,wd,_,_) = localtime(then) ]

It could be made to use existing keywords:

  [ somefunc(mn,day,wd,name)
    for (then, name) in mylist
    with localtime(then) as (_,mn,dy,_,_,_,wd,_,_) ]

[toc] | [prev] | [next] | [standalone]

#64333

On Sat, 18 Jan 2014 16:00:45 -0000, Jussi Piitulainen  
<jpiitula@ling.helsinki.fi> wrote:

> Rustom Mody writes:
>
>> On Saturday, January 18, 2014 2:06:29 PM UTC+5:30, Peter Otten wrote:
>>
>> > What would a list-comp with `let` or `where` look like? Would it
>> > win the beauty contest against the loop?
>>
>> For me this is neat
>>
>> [somefunc(mn,day,wd,name) for (then, name) in mylist let  
>> (_,mn,dy,_,_,_,wd,_,_) = localtime(then)]
>>
>> Others may not find it so!

Count me firmly in the "others" camp.  It looks ugly, it flows appallingly  
badly as English, and its only going to get worse as you pile in more  
variables and expressions.  -100 from me.

>> See it across > 1 line (as I guess it will come after being posted!)
>> and its not so neat.
>
> I would write that on three lines anyway, properly indented:
>
>   [ somefunc(mn,day,wd,name)
>     for (then, name) in mylist
>     let (_,mn,dy,_,_,_,wd,_,_) = localtime(then) ]
>
> It could be made to use existing keywords:
>
>   [ somefunc(mn,day,wd,name)
>     for (then, name) in mylist
>     with localtime(then) as (_,mn,dy,_,_,_,wd,_,_) ]

Better, in that it's readable.  It's still storing up trouble, though.

Seriously, what's inelegant about this?

def meaningful_name(then, name):
     _,mn,dy,_,_,_,wd,_,_ = localtime(then)
     return somefunc(mn, dy, wd, name)

...

     [meaningful_name(then, name) for (then, name) in mylist]

I assume there's some good reason you didn't want somefunc() to do the  
localtime() itself?

-- 
Rhodri James *-* Wildebeest Herder to the Masses

[toc] | [prev] | [next] | [standalone]

#64354

"Rhodri James" <rhodri@wildebst.org.uk> writes:

> On Sat, 18 Jan 2014 16:00:45 -0000, Jussi Piitulainen
> <jpiitula@ling.helsinki.fi> wrote:

[...]

>> I would write that on three lines anyway, properly indented:
>>
>>   [ somefunc(mn,day,wd,name)
>>     for (then, name) in mylist
>>     let (_,mn,dy,_,_,_,wd,_,_) = localtime(then) ]
>>
>> It could be made to use existing keywords:
>>
>>   [ somefunc(mn,day,wd,name)
>>     for (then, name) in mylist
>>     with localtime(then) as (_,mn,dy,_,_,_,wd,_,_) ]
>
> Better, in that it's readable.  It's still storing up trouble, though.
>
> Seriously, what's inelegant about this?
>
> def meaningful_name(then, name):
>     _,mn,dy,_,_,_,wd,_,_ = localtime(then)
>     return somefunc(mn, dy, wd, name)
>
> ...
>
>     [meaningful_name(then, name) for (then, name) in mylist]
>
> I assume there's some good reason you didn't want somefunc() to do the
> localtime() itself?

Actually in my use case somefunc(mn,day,wd,name) was an expression, not
really a function call. I just presented it like that to make it a more
generalised abstract case. So in my case that would introduce an
additional function outside of the comprehension.

Let me summarize what my problem really was: In a comprehension you
generally need local name(s), which can only be introduced with the
'for' construction. But then the name(s) must be bound to an element of
a sequence/iterator. There is no way to bind the name(s) to a single
object other than putting that object in a one element sequence. I was
just looking for a way to avoid that. Functional programming languages
have a way to do this with the 'let' or 'where' construction which is
missing in Python.

Thanks everybody for your thoughts on this.
-- 
Piet van Oostrum <piet@vanoostrum.org>
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]

[toc] | [prev] | [next] | [standalone]

#64359

On Monday, January 20, 2014 4:32:40 PM UTC+5:30, Piet van Oostrum wrote:
> "Rhodri James" writes:

> > On Sat, 18 Jan 2014 16:00:45 -0000, Jussi Piitulainen wrote:

> [...]

> >> I would write that on three lines anyway, properly indented:
> >>   [ somefunc(mn,day,wd,name)
> >>     for (then, name) in mylist
> >>     let (_,mn,dy,_,_,_,wd,_,_) = localtime(then) ]
> >> It could be made to use existing keywords:
> >>   [ somefunc(mn,day,wd,name)
> >>     for (then, name) in mylist
> >>     with localtime(then) as (_,mn,dy,_,_,_,wd,_,_) ]
> > Better, in that it's readable.  It's still storing up trouble, though.
> > Seriously, what's inelegant about this?
> > def meaningful_name(then, name):
> >     _,mn,dy,_,_,_,wd,_,_ = localtime(then)
> >     return somefunc(mn, dy, wd, name)
> > ...
> >     [meaningful_name(then, name) for (then, name) in mylist]
> > I assume there's some good reason you didn't want somefunc() to do the
> > localtime() itself?

> Actually in my use case somefunc(mn,day,wd,name) was an expression, not
> really a function call. I just presented it like that to make it a more
> generalised abstract case. So in my case that would introduce an
> additional function outside of the comprehension.

Nice! Thanks

One of things that people who decry comprehensions (and in general
hi-level constructs) dont understand is this:

One fundamental characteristic of increasing hi-level-ness of a programming
language is increased *freedom-in-naming* structures.

The 'structures' could be expressions, statements, or whatever is
the thing of the programming language.

A couple of examples will illustrate:

1. In most (imperative) programming languages when a computation is described
as a function it must willy-nilly be named. In lambda-calculus thats an option.

2. In any hi-level programming language, if we have an expression like
sqrt(b*b - 4*a*c)

we can choose to 'chunk it out' in a variety of ways:
1.
t = b*b - 4*a*c
sqrt(t)

2.
t1 = b*b
t2 = 4*a*c
t3 = t1 -t2
sqrt(t3)

Or the original: sqrt(b*b - 4*a*c)

The assembly language programmer has no such choice. He has to write
(the equivalent of)
t1 = b*b
t2 = 4*a
t3 = t2*c
t4 = t1 - t3
sqrt(t4)

Compared to a comprehension, a loop is in assembly-language category
since the list being 'comprehended' is compelled to be named.

And BTW thanks for the singleton list trick:
Sure 'v=e' is way neater than 'for v in [e]' but the for is better than nothing

[toc] | [prev] | [next] | [standalone]

#64224

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2014-01-17, 23:19 GMT, you wrote:
> But defining the auxfunc takes away the elegance of a list 
> comprehension.

Au contraire! Remember, that brevity is the sister of talent.  

I would definitively vote for 

    labels = [make_label(then, name) for then, name in mylist]

(always use descriptive names of functions and variables; 
auxfunc is a short way to the hell)

Beauty of the list comprehensions is that they show nicely what 
list is being processed, how it is filtered (if at all), and 
what we do with each element of the generated list. Anything you 
add to this simplicity is wrong. Whenever you start to feel you 
are missing some methods how to stuff more commands into 
a comprehension (or for example multiple embedded ones), you 
should start new function.

The same rule applies here as with any other lambda function 
(because these are in fact lambda functions): the best way how 
to write lambda is to write algorithm somewhere on the side, 
describe what this function does in one word, then add `def` in 
front of that name, and use so created named function instead.

Best,

Matěj

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iD8DBQFS2l4X4J/vJdlkhKwRAjEgAJ4n1OuANYlVFzlgBZ0f1uMhO/t36gCfdFjE
VmYDJ+F7aN0khzvlY50i0iA=
=Trcc
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#64227

Piet van Oostrum <piet@vanoostrum.org> writes:

[...]
> I could define a auxiliary function like:
>
> def auxfunc(then, name):
>     _, mn, dy, _, _, _, wd, _, _ = localtime(then)
>     return somefunc(mn, day, wd, name)
>
> and then use 
> [auxfunc(then, name) for then, name in mylist]

[...]

> labels = [somefunc(mn, day, wd, name) 
>             for then, name in mylist
>             for _, mn, dy, _, _, _, wd, _, _ in [localtime(then)]]
>
> Python misses a 'where' or 'let'-like construction as in Haskell.

"let x = v in e" really is (lambda x:e)(v)

In your case:

[ (lambda na,ti : somefunc(ti[1],ti[2],ti[6],na))(name,localtime(then))
  for then,name in mylist ]

Warning: absolutely untested (not even syntax-checked).

You may also use *localtime(...) and keep underscores.

-- Alain.

[toc] | [prev] | [next] | [standalone]

#64238

Alain Ketterlin <alain@dpt-info.u-strasbg.fr> writes:

> Piet van Oostrum <piet@vanoostrum.org> writes:
> [...]
>> Python misses a 'where' or 'let'-like construction as in Haskell.
>
> "let x = v in e" really is (lambda x:e)(v)
>
You are right, but it is a lot less readable IMHO.
-- 
Piet van Oostrum <piet@vanoostrum.org>
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]

[toc] | [prev] | [next] | [standalone]

#64272

Hi,

I'd agree with the advice that it's not the best idea: readability sucks 
here, but consider the following:


import time

def somefunc(a,b,c,d): # dummy function
     return "{} - {} - {} : {}".format(a,b,c,d)
l = [(time.time(),"name {}".format(n)) for n in range(100)] # dummy data

# the line in question
labels = [somefunc(*(lambda t,n: 
(t.tm_mon,t.tm_mday,t.tm_wday,n))(time.localtime(x[0]),x[1])) for x in l]


print(labels) # just to see the result


If you find that hard to decipher, the consider the maintainability of 
code you write that uses such comprehensions.  You need to include 
comments that explain what this does, and it is easier to write a 
longhand version using .append() and variable assignments.  I presume
performance won't be an issue determining the right approach, since then 
you'd be using C or C++.

John

On 17/01/2014 23:49, Dan Stromberg wrote:
> On Fri, Jan 17, 2014 at 3:19 PM, Piet van Oostrum <piet@vanoostrum.org> wrote:
>> Hi,
>>
>> I am looking for an elegant way to write the following code as a list
>> comprehension:
>>
>> labels = []
>> for then, name in mylist:
>>      _, mn, dy, _, _, _, wd, _, _ = localtime(then)
>>      labels.append(somefunc(mn, day, wd, name))
>
> My recomendation: Don't use a list comprehension.  List comprehensions
> and generator expressions are great for quick little things, but
> become less readable when you have to string them over multiple
> physical lines.
>
>> labels = [somefunc(mn, day, wd, name)
>>              for then, name in mylist
>>              for _, mn, dy, _, _, _, wd, _, _ in [localtime(then)]]

[toc] | [prev] | [next] | [standalone]

#64330

John Allsup <pydev@allsup.co> writes:

> Hi,
>
> I'd agree with the advice that it's not the best idea: readability sucks
> here, but consider the following:
>
>
> import time
>
> def somefunc(a,b,c,d): # dummy function
>     return "{} - {} - {} : {}".format(a,b,c,d)
> l = [(time.time(),"name {}".format(n)) for n in range(100)] # dummy data
>
> # the line in question
> labels = [somefunc(*(lambda t,n:
> (t.tm_mon,t.tm_mday,t.tm_wday,n))(time.localtime(x[0]),x[1])) for x in
> l]
>
>
> print(labels) # just to see the result
>
>
> If you find that hard to decipher, the consider the maintainability of
> code you write that uses such comprehensions.  You need to include
> comments that explain what this does, and it is easier to write a
> longhand version using .append() and variable assignments.  I presume
> performance won't be an issue determining the right approach, since then
> you'd be using C or C++.

I'll stay with

labels = [somefunc(mn, day, wd, name) 
            for then, name in mylist
            for _, mn, dy, _, _, _, wd, _, _ in [localtime(then)]]

where (sic!) the last part means as much as 
           where _, mn, dy, _, _, _, wd, _, _ = localtime(then)

I find the list comprehension preferable because it makes it more clear
that a new list is constructed from an existing list, something that is
not as immediately clear with the append construction.
-- 
Piet van Oostrum <piet@vanoostrum.org>
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web