Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #31497 > unrolled thread

Re: list comprehension question

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: list comprehension question Dave Angel <d@davea.name> - 2012-10-17 08:32 -0400
    Re: list comprehension question rusi <rustompmody@gmail.com> - 2012-10-17 07:06 -0700
      Re: list comprehension question rusi <rustompmody@gmail.com> - 2012-10-17 07:33 -0700
      Re: list comprehension question Dave Angel <d@davea.name> - 2012-10-17 10:36 -0400
        Re: list comprehension question 88888 Dihedral <dihedral88888@googlemail.com> - 2012-10-17 07:45 -0700
        Re: list comprehension question 88888 Dihedral <dihedral88888@googlemail.com> - 2012-10-17 07:45 -0700
        Re: list comprehension question rusi <rustompmody@gmail.com> - 2012-10-17 07:50 -0700
          Re: list comprehension question 88888 Dihedral <dihedral88888@googlemail.com> - 2012-10-17 08:06 -0700

#31497 — Re: list comprehension question

On 10/17/2012 12:43 AM, Kevin Anthony wrote:
> Is it not true that list comprehension is much faster the the for loops?
> 
> If it is not the correct way of doing this, i appoligize.
> Like i said, I'm learing list comprehension.
> 
(Please don't top-post;  it ruins the ordering.  In these forums, put
your response after the part you quote from earlier messages.  Or even
better, after each part you quote.  Then trim off the parts you didn't
reference.)

list comprehensions CAN be much faster, but not necessarily.  The most
complex a loop, the less likely it'll help much.

In any case, only the inner loop will be affected.  Nesting two list
comprehensions will make a trivial difference.

On the other hand, Hans Mulder shows some other factoring which seems
much more readable than yours.

Studying (and testing) those could teach you a lot about comprehensions,
as well as about the libraries that can help.  Note especially what
zip(*b) yields, and think about what it means.


-- 

DaveA

[toc] | [next] | [standalone]

#31503

On Oct 17, 5:33 pm, Dave Angel <d...@davea.name> wrote:
> On 10/17/2012 12:43 AM, Kevin Anthony wrote:> Is it not true that list comprehension is much faster the the for loops?
>
> > If it is not the correct way of doing this, i appoligize.
> > Like i said, I'm learing list comprehension.

> list comprehensions CAN be much faster, but not necessarily.  The most
> complex a loop, the less likely it'll help much.

One-lining the comprehension seems to make a difference of about 10%
out here. Maybe Ive missed something? Seems too large…

# My original suggestion
def dot(p,q): return sum (x*y for x,y in zip(p,q))
def transpose(m): return zip(*m)
def mm(a,b): return mmt(a, transpose(b))
def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a]

# One-liner (Thanks Hans for reminding me of sum)

def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
zip(*b)] for ra in a]

>>> t1=Timer("res=mm1(m,m)", setup="from __main__ import mm1, m")
>>> t1.timeit(1000)
12.276363849639893
>>> t0=Timer("res=mm(m,m)", setup="from __main__ import mm, m")
>>> t0.timeit(1000)
13.453603029251099

[toc] | [prev] | [next] | [standalone]

#31509

On Oct 17, 7:06 pm, rusi <rustompm...@gmail.com> wrote:
> On Oct 17, 5:33 pm, Dave Angel <d...@davea.name> wrote:
>
> > On 10/17/2012 12:43 AM, Kevin Anthony wrote:> Is it not true that list comprehension is much faster the the for loops?
>
> > > If it is not the correct way of doing this, i appoligize.
> > > Like i said, I'm learing list comprehension.
> > list comprehensions CAN be much faster, but not necessarily.  The most
> > complex a loop, the less likely it'll help much.
>
> One-lining the comprehension seems to make a difference of about 10%
> out here. Maybe Ive missed something? Seems too large…
>
> # My original suggestion
> def dot(p,q): return sum (x*y for x,y in zip(p,q))
> def transpose(m): return zip(*m)
> def mm(a,b): return mmt(a, transpose(b))
> def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a]
>
> # One-liner (Thanks Hans for reminding me of sum)
>
> def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
> zip(*b)] for ra in a]
>
> >>> t1=Timer("res=mm1(m,m)", setup="from __main__ import mm1, m")
> >>> t1.timeit(1000)
> 12.276363849639893
> >>> t0=Timer("res=mm(m,m)", setup="from __main__ import mm, m")
> >>> t0.timeit(1000)
>
> 13.453603029251099

In case anyone wants to try out with the same data, I used:

m = [range(i,i+30) for i in range(30)]

[toc] | [prev] | [next] | [standalone]

#31512

On 10/17/2012 10:06 AM, rusi wrote:
> On Oct 17, 5:33 pm, Dave Angel <d...@davea.name> wrote:
>> On 10/17/2012 12:43 AM, Kevin Anthony wrote:> Is it not true that list comprehension is much faster the the for loops?
>>
>>> If it is not the correct way of doing this, i appoligize.
>>> Like i said, I'm learing list comprehension.
>> list comprehensions CAN be much faster, but not necessarily.  The most
>> complex a loop, the less likely it'll help much.
> One-lining the comprehension seems to make a difference of about 10%
> out here. Maybe Ive missed something? Seems too large…
>
> # My original suggestion
> def dot(p,q): return sum (x*y for x,y in zip(p,q))
> def transpose(m): return zip(*m)
> def mm(a,b): return mmt(a, transpose(b))
> def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a]
>
> # One-liner (Thanks Hans for reminding me of sum)
>
> def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
> zip(*b)] for ra in a]
>
>>>> t1=Timer("res=mm1(m,m)", setup="from __main__ import mm1, m")
>>>> t1.timeit(1000)
> 12.276363849639893
>>>> t0=Timer("res=mm(m,m)", setup="from __main__ import mm, m")
>>>> t0.timeit(1000)
> 13.453603029251099

And I'd wager all the improvement is in the inner loop, the dot() function.


-- 

DaveA

[toc] | [prev] | [next] | [standalone]

#31513

Dave Angel於 2012年10月17日星期三UTC+8下午10時37分01秒寫道：
> On 10/17/2012 10:06 AM, rusi wrote:
> 
> > On Oct 17, 5:33 pm, Dave Angel <d...@davea.name> wrote:
> 
> >> On 10/17/2012 12:43 AM, Kevin Anthony wrote:> Is it not true that list comprehension is much faster the the for loops?
> 
> >>
> 
> >>> If it is not the correct way of doing this, i appoligize.
> 
> >>> Like i said, I'm learing list comprehension.
> 
> >> list comprehensions CAN be much faster, but not necessarily.  The most
> 
> >> complex a loop, the less likely it'll help much.
> 
> > One-lining the comprehension seems to make a difference of about 10%
> 
> > out here. Maybe Ive missed something? Seems too large�
> 
> >
> 
> > # My original suggestion
> 
> > def dot(p,q): return sum (x*y for x,y in zip(p,q))
> 
> > def transpose(m): return zip(*m)
> 
> > def mm(a,b): return mmt(a, transpose(b))
> 
> > def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a]
> 
> >
> 
> > # One-liner (Thanks Hans for reminding me of sum)
> 
> >
> 
> > def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
> 
> > zip(*b)] for ra in a]
> 
> >
> 
> >>>> t1=Timer("res=mm1(m,m)", setup="from __main__ import mm1, m")
> 
> >>>> t1.timeit(1000)
> 
> > 12.276363849639893
> 
> >>>> t0=Timer("res=mm(m,m)", setup="from __main__ import mm, m")
> 
> >>>> t0.timeit(1000)
> 
> > 13.453603029251099
> 
> 
> 
> And I'd wager all the improvement is in the inner loop, the dot() function.
> 
> 
> 
> 
> 
> -- 
> 
> 
> 
> DaveA

Thanks for the tips of matrix operations over some fields or rings 
other than the real field and the complex field.

[toc] | [prev] | [next] | [standalone]

#31514

Dave Angel於 2012年10月17日星期三UTC+8下午10時37分01秒寫道：
> On 10/17/2012 10:06 AM, rusi wrote:
> 
> > On Oct 17, 5:33 pm, Dave Angel <d...@davea.name> wrote:
> 
> >> On 10/17/2012 12:43 AM, Kevin Anthony wrote:> Is it not true that list comprehension is much faster the the for loops?
> 
> >>
> 
> >>> If it is not the correct way of doing this, i appoligize.
> 
> >>> Like i said, I'm learing list comprehension.
> 
> >> list comprehensions CAN be much faster, but not necessarily.  The most
> 
> >> complex a loop, the less likely it'll help much.
> 
> > One-lining the comprehension seems to make a difference of about 10%
> 
> > out here. Maybe Ive missed something? Seems too large�
> 
> >
> 
> > # My original suggestion
> 
> > def dot(p,q): return sum (x*y for x,y in zip(p,q))
> 
> > def transpose(m): return zip(*m)
> 
> > def mm(a,b): return mmt(a, transpose(b))
> 
> > def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a]
> 
> >
> 
> > # One-liner (Thanks Hans for reminding me of sum)
> 
> >
> 
> > def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
> 
> > zip(*b)] for ra in a]
> 
> >
> 
> >>>> t1=Timer("res=mm1(m,m)", setup="from __main__ import mm1, m")
> 
> >>>> t1.timeit(1000)
> 
> > 12.276363849639893
> 
> >>>> t0=Timer("res=mm(m,m)", setup="from __main__ import mm, m")
> 
> >>>> t0.timeit(1000)
> 
> > 13.453603029251099
> 
> 
> 
> And I'd wager all the improvement is in the inner loop, the dot() function.
> 
> 
> 
> 
> 
> -- 
> 
> 
> 
> DaveA

Thanks for the tips of matrix operations over some fields or rings 
other than the real field and the complex field.

[toc] | [prev] | [next] | [standalone]

#31515

On Oct 17, 7:37 pm, Dave Angel <d...@davea.name> wrote:

> And I'd wager all the improvement is in the inner loop, the dot() function.

Sorry -- red herring!

Changing

def mm1(a,b): return [[sum(x*y for x,y in zip(ra,rb)) for rb in
zip(*b)] for ra in a]

to

def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
zip(*b)] for ra in a]

makes the speed diff vanish

[toc] | [prev] | [next] | [standalone]

#31517

rusi於 2012年10月17日星期三UTC+8下午10時50分11秒寫道：
> On Oct 17, 7:37 pm, Dave Angel <d...@davea.name> wrote:
> 
> 
> 
> > And I'd wager all the improvement is in the inner loop, the dot() function.
> 
> 
> 
> Sorry -- red herring!
> 
> 
> 
> Changing
> 
> 
> 
> def mm1(a,b): return [[sum(x*y for x,y in zip(ra,rb)) for rb in
> 
> zip(*b)] for ra in a]
> 
> 
> 
> to
> 
> 
> 
> def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
> 
> zip(*b)] for ra in a]
> 
> 
> 
> makes the speed diff vanish

I think a lot people don't work on computations over 
fields other real and complex.

That is why a lot people keep complaining about the speeds 
of python programs executed.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web