Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #35768 > unrolled thread

dict comprehension question.

Back to article view | Back to comp.lang.python

Contents

  dict comprehension question. Quint Rankid <qbr567@gmail.com> - 2012-12-29 11:48 -0800
    Re: dict comprehension question. Roy Smith <roy@panix.com> - 2012-12-29 14:58 -0500
    Re: dict comprehension question. Mitya Sirenef <msirenef@lightbird.net> - 2012-12-29 15:01 -0500
    Re: dict comprehension question. Mitya Sirenef <msirenef@lightbird.net> - 2012-12-29 15:09 -0500
    Re: dict comprehension question. Joel Goldstick <joel.goldstick@gmail.com> - 2012-12-29 15:15 -0500
    Re: dict comprehension question. Peter Otten <__peter__@web.de> - 2012-12-29 21:32 +0100
    Re: dict comprehension question. MRAB <python@mrabarnett.plus.com> - 2012-12-29 20:39 +0000
    Re: dict comprehension question. Mitya Sirenef <msirenef@lightbird.net> - 2012-12-29 16:40 -0500
    Re: dict comprehension question. Terry Reedy <tjreedy@udel.edu> - 2012-12-29 18:22 -0500
    Re: dict comprehension question. Terry Reedy <tjreedy@udel.edu> - 2012-12-29 18:56 -0500
      Re: dict comprehension question. Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-01-01 03:10 +0000
        Re: dict comprehension question. 88888 Dihedral <dihedral88888@googlemail.com> - 2013-01-01 00:40 -0800
    Re: dict comprehension question. Tim Chase <python.list@tim.thechases.com> - 2012-12-29 18:26 -0600
    Re: dict comprehension question. Joel Goldstick <joel.goldstick@gmail.com> - 2012-12-29 23:10 -0500

#35768 — dict comprehension question.

Newbie question.  I've googled a little and haven't found the answer.

Given a list like:
w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
I would like to be able to do the following as a dict comprehension.
a = {}
for x in w:
    a[x] = a.get(x,0) + 1
results in a having the value:
{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

I've tried a few things
eg
a1 = {x:self.get(x,0)+1 for x in w}
results in error messages.

And
a2 = {x:a2.get(x,0)+1 for x in w}
also results in error messages.

Trying to set a variable to a dict before doing the comprehension
a3 = {}
a3 = {x:a3.get(x,0)+1 for x in w}
gets this result, which isn't what I wanted.
{1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1}

I'm not sure that it's possible to do this, and if not, perhaps the
most obvious question is what instance does the get method bind to?

TIA

[toc] | [next] | [standalone]

#35770

In article 
<724d4fea-606a-4503-b538-87442f6bcebc@ci3g2000vbb.googlegroups.com>,
 Quint Rankid <qbr567@gmail.com> wrote:

> Newbie question.  I've googled a little and haven't found the answer.
> 
> Given a list like:
> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
> I would like to be able to do the following as a dict comprehension.
> a = {}
> for x in w:
>     a[x] = a.get(x,0) + 1

Why are you trying to do this mind-blowing thing?  Other than as an 
entry in an obfuscated code contest, what is this for?

Anyway, I don't think this is possible with a dict comprehension.  
Entries in the dict depend on entries previously put into the dict.  I 
don't see any way a dict comprehension can deal with this.

[toc] | [prev] | [next] | [standalone]

#35771

On 12/29/2012 02:48 PM, Quint Rankid wrote:
> Newbie question.  I've googled a little and haven't found the answer.
>
> Given a list like:
> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
> I would like to be able to do the following as a dict comprehension.
> a = {}
> for x in w:
>      a[x] = a.get(x,0) + 1
> results in a having the value:
> {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}
>
> I've tried a few things
> eg
> a1 = {x:self.get(x,0)+1 for x in w}
> results in error messages.
>
> And
> a2 = {x:a2.get(x,0)+1 for x in w}
> also results in error messages.
>
> Trying to set a variable to a dict before doing the comprehension
> a3 = {}
> a3 = {x:a3.get(x,0)+1 for x in w}
> gets this result, which isn't what I wanted.
> {1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1}
>
> I'm not sure that it's possible to do this, and if not, perhaps the
> most obvious question is what instance does the get method bind to?
>
> TIA

Will this do?:

 >>> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
 >>> {x: w.count(x) for x in w}
{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}


  - mitya

-- 
Lark's Tongue Guide to Python: http://lightbird.net/larks/

[toc] | [prev] | [next] | [standalone]

#35772

On 12/29/2012 03:01 PM, Mitya Sirenef wrote:
> On 12/29/2012 02:48 PM, Quint  Rankid wrote:
 >> Newbie question. I've googled a little and haven't found the answer.
 >>
 >> Given a list like:
 >> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
 >> I would like to be able to do the following as a dict comprehension.
 >> a = {}
 >> for x in w:
 >> a[x] = a.get(x,0) + 1
 >> results in a having the value:
 >> {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}
 >>
 >> I've tried a few things
 >> eg
 >> a1 = {x:self.get(x,0)+1 for x in w}
 >> results in error messages.
 >>
 >> And
 >> a2 = {x:a2.get(x,0)+1 for x in w}
 >> also results in error messages.
 >>
 >> Trying to set a variable to a dict before doing the comprehension
 >> a3 = {}
 >> a3 = {x:a3.get(x,0)+1 for x in w}
 >> gets this result, which isn't what I wanted.
 >> {1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1}
 >>
 >> I'm not sure that it's possible to do this, and if not, perhaps the
 >> most obvious question is what instance does the get method bind to?
 >>
 >> TIA
 >
 > Will this do?:
 >
 > >>> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
 > >>> {x: w.count(x) for x in w}
 > {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}
 >
 >
 > - mitya
 >

I should probably add that this might be inefficient for large lists as
it repeats count for each item. If you need it for large lists, profile
against the 'for loop' version and decide if performance is good enough
for you, for small lists it's a nice and compact solution.

In a more general case, you can't refer to the list/dict/etc
comprehension as it's being constructed, that's just not a design goal
of comprehensions.

  -m

-- 
Lark's Tongue Guide to Python: http://lightbird.net/larks/

[toc] | [prev] | [next] | [standalone]

#35773

[Multipart message — attachments visible in raw view] — view raw

On Sat, Dec 29, 2012 at 3:09 PM, Mitya Sirenef <msirenef@lightbird.net>wrote:

> On 12/29/2012 03:01 PM, Mitya Sirenef wrote:
>
>> On 12/29/2012 02:48 PM, Quint  Rankid wrote:
>>
> >> Newbie question. I've googled a little and haven't found the answer.
> >>
> >> Given a list like:
> >> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
> >> I would like to be able to do the following as a dict comprehension.
> >> a = {}
> >> for x in w:
> >> a[x] = a.get(x,0) + 1
> >> results in a having the value:
> >> {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}
> >>
> >> I've tried a few things
> >> eg
> >> a1 = {x:self.get(x,0)+1 for x in w}
> >> results in error messages.
> >>
> >> And
> >> a2 = {x:a2.get(x,0)+1 for x in w}
> >> also results in error messages.
> >>
> >> Trying to set a variable to a dict before doing the comprehension
> >> a3 = {}
> >> a3 = {x:a3.get(x,0)+1 for x in w}
> >> gets this result, which isn't what I wanted.
> >> {1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1}
> >>
> >> I'm not sure that it's possible to do this, and if not, perhaps the
> >> most obvious question is what instance does the get method bind to?
> >>
> >> TIA
> >
> > Will this do?:
> >
> > >>> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
> > >>> {x: w.count(x) for x in w}
> > {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}
> >
> >
> > - mitya
> >
>
> I should probably add that this might be inefficient for large lists as
> it repeats count for each item. If you need it for large lists, profile
> against the 'for loop' version and decide if performance is good enough
> for you, for small lists it's a nice and compact solution.
>
> In a more general case, you can't refer to the list/dict/etc
> comprehension as it's being constructed, that's just not a design goal
> of comprehensions.
>

Would this help:

     >>> w = [1,2,3,1,2,4,4,5,6,1]
     >>> s = set(w)
     >>> s
     set([1, 2, 3, 4, 5, 6])
     >>> {x:w.count(x) for x in s}
     {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}
>>>


>  -m
>
>
> --
> Lark's Tongue Guide to Python: http://lightbird.net/larks/
>
> --
> http://mail.python.org/**mailman/listinfo/python-list<http://mail.python.org/mailman/listinfo/python-list>
>



-- 
Joel Goldstick

[toc] | [prev] | [next] | [standalone]

#35774

Quint Rankid wrote:

> Newbie question.  I've googled a little and haven't found the answer.
> 
> Given a list like:
> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
> I would like to be able to do the following as a dict comprehension.
> a = {}
> for x in w:
>     a[x] = a.get(x,0) + 1
> results in a having the value:
> {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}
> 
> I've tried a few things
> eg
> a1 = {x:self.get(x,0)+1 for x in w}
> results in error messages.
> 
> And
> a2 = {x:a2.get(x,0)+1 for x in w}
> also results in error messages.
> 
> Trying to set a variable to a dict before doing the comprehension
> a3 = {}
> a3 = {x:a3.get(x,0)+1 for x in w}
> gets this result, which isn't what I wanted.
> {1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1}
> 
> I'm not sure that it's possible to do this, and if not, perhaps the
> most obvious question is what instance does the get method bind to?

The name a3 will not be rebound until after the right side is evaluate. To 
spell it with a loop:

a3 = {}
_internal = {} # You have no access to this var. 
               # Even if you can beat a particular 
               # Python implementation -- you shouldn't

for x in w:
    _internal[x] = a3.get(x, 0) + 1

a3 = _internal

That should make it clear that x will be looked up in the "old" empty a3 
dict.

The closest you can get to a self-updating dict is probably

>>> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
>>> a1 = {}
>>> a1.update((x, a1.get(x, 0)+1) for x in w)
>>> a1
{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

but that it works doesn't mean it is a good idea. 
If your Python version supports it the obvious choice is

>>> from collections import Counter
>>> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
>>> Counter(w)
Counter({1: 3, 2: 2, 4: 2, 3: 1, 5: 1, 6: 1})

[toc] | [prev] | [next] | [standalone]

#35776

On 2012-12-29 19:48, Quint Rankid wrote:
> Newbie question.  I've googled a little and haven't found the answer.
>
> Given a list like:
> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
> I would like to be able to do the following as a dict comprehension.
> a = {}
> for x in w:
>      a[x] = a.get(x,0) + 1
> results in a having the value:
> {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}
>
> I've tried a few things
> eg
> a1 = {x:self.get(x,0)+1 for x in w}
> results in error messages.
>
> And
> a2 = {x:a2.get(x,0)+1 for x in w}
> also results in error messages.
>
> Trying to set a variable to a dict before doing the comprehension
> a3 = {}
> a3 = {x:a3.get(x,0)+1 for x in w}
> gets this result, which isn't what I wanted.
> {1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1}
>
> I'm not sure that it's possible to do this, and if not, perhaps the
> most obvious question is what instance does the get method bind to?
>
You can't do it with a comprehension.

The best way is probably with the 'Counter' class:

 >>> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
 >>> from collections import Counter
 >>> Counter(w)
Counter({1: 3, 2: 2, 4: 2, 3: 1, 5: 1, 6: 1})

If you want the result be a dict, then just the result to 'dict':

 >>> dict(Counter(w))
{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

[toc] | [prev] | [next] | [standalone]

#35783

On 12/29/2012 03:15 PM, Joel Goldstick wrote:
>
>
>
> On Sat, Dec 29, 2012 at 3:09 PM, Mitya Sirenef <msirenef@lightbird.net 
> <mailto:msirenef@lightbird.net>> wrote:
>
>     On 12/29/2012 03:01 PM, Mitya Sirenef wrote:
>
>         On 12/29/2012 02:48 PM, Quint  Rankid wrote:
>
>     >> Newbie question. I've googled a little and haven't found the
>     answer.
>     >>
>     >> Given a list like:
>     >> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
>     >> I would like to be able to do the following as a dict
>     comprehension.
>     >> a = {}
>     >> for x in w:
>     >> a[x] = a.get(x,0) + 1
>     >> results in a having the value:
>     >> {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}
>     >>
>     >> I've tried a few things
>     >> eg
>     >> a1 = {x:self.get(x,0)+1 for x in w}
>     >> results in error messages.
>     >>
>     >> And
>     >> a2 = {x:a2.get(x,0)+1 for x in w}
>     >> also results in error messages.
>     >>
>     >> Trying to set a variable to a dict before doing the comprehension
>     >> a3 = {}
>     >> a3 = {x:a3.get(x,0)+1 for x in w}
>     >> gets this result, which isn't what I wanted.
>     >> {1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1}
>     >>
>     >> I'm not sure that it's possible to do this, and if not, perhaps the
>     >> most obvious question is what instance does the get method bind to?
>     >>
>     >> TIA
>     >
>     > Will this do?:
>     >
>     > >>> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
>     > >>> {x: w.count(x) for x in w}
>     > {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}
>     >
>     >
>     > - mitya
>     >
>
>     I should probably add that this might be inefficient for large
>     lists as
>     it repeats count for each item. If you need it for large lists,
>     profile
>     against the 'for loop' version and decide if performance is good
>     enough
>     for you, for small lists it's a nice and compact solution.
>
>     In a more general case, you can't refer to the list/dict/etc
>     comprehension as it's being constructed, that's just not a design goal
>     of comprehensions.
>
>
> Would this help:
>
>      >>> w = [1,2,3,1,2,4,4,5,6,1]
>      >>> s = set(w)
>      >>> s
>      set([1, 2, 3, 4, 5, 6])
>      >>> {x:w.count(x) for x in s}
>      {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}
> >>>


Indeed, this is much better -- I didn't think of it..


-- 
Lark's Tongue Guide to Python: http://lightbird.net/larks/

[toc] | [prev] | [next] | [standalone]

#35790

On 12/29/2012 4:40 PM, Mitya Sirenef wrote:
> On 12/29/2012 03:15 PM, Joel Goldstick wrote:

>> Would this help:
>>
>>      >>> w = [1,2,3,1,2,4,4,5,6,1]
>>      >>> s = set(w)
>>      >>> s
>>      set([1, 2, 3, 4, 5, 6])
>>      >>> {x:w.count(x) for x in s}
>>      {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}
>> >>>
>
>
> Indeed, this is much better -- I didn't think of it..

It still turns an O(n) problem into an O(k*n) problem, where k is the 
number of distinct items.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#35791

On 12/29/2012 2:48 PM, Quint Rankid wrote:

> Given a list like:
> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
> I would like to be able to do the following as a dict comprehension.
> a = {}
> for x in w:
>      a[x] = a.get(x,0) + 1
> results in a having the value:
> {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

Let me paraphrase this: "I have nice, clear, straightforward, 
*comprehensible* code that I want to turn into an incomprehensible mess 
with a 'comprehension." That is the ironic allure of comprehensions.

Comprehensions do not allow for interactions between the source items. 
Mitya and Joel worked around this with solutions that do redundant 
calculation and multiply the time order.

Reductions do allow for interactions. Doing everything as a reduction 
was the fad before comprehensions came along ;-)

from functools import reduce
w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
def update(dic, n):
     "Mutate and return dic (contrary to usual Python policy)"
     dic[n] = dic.get(n, 0) + 1
     return dic
counts = reduce(update, w, {})
print(counts == {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1})

# prints True

The above is how to rewrite your code in a functional language that does 
not have statements and explicit iteration. In Python, I would only 
bother to wrap the body of the loop in a function if I needed the same 
body in multiple places.

Comprehensions are filtered mappings and that both filter and map can be 
written as reduction, so reduction included comprehension. It is more 
powerful because it can also do sequential interaction. Indeed, I would 
say that it should only be used when there is sequential interaction.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#35871

On Sat, 29 Dec 2012 18:56:57 -0500, Terry Reedy wrote:

> On 12/29/2012 2:48 PM, Quint Rankid wrote:
> 
>> Given a list like:
>> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
>> I would like to be able to do the following as a dict comprehension. 
>> a = {}
>> for x in w:
>>      a[x] = a.get(x,0) + 1
>> results in a having the value:
>> {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}
> 
> Let me paraphrase this: "I have nice, clear, straightforward,
> *comprehensible* code that I want to turn into an incomprehensible mess
> with a 'comprehension." That is the ironic allure of comprehensions.

But... but... one liner! ONE LINNNNNNEEEERRRRR!!!! Won't somebody think 
of the lines I'll save!!!!

*wink*


In case it's not obvious, I'm 100% agreeing with Terry here. List comps 
and dict comps are wonderful things, but they can't do everything, and 
very often even if they can do something they shouldn't because it makes 
the code inefficient or unreadable.

There's nothing wrong with a two or three liner.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#35880

On Tuesday, January 1, 2013 11:10:48 AM UTC+8, Steven D'Aprano wrote:
> On Sat, 29 Dec 2012 18:56:57 -0500, Terry Reedy wrote:
> 
> 
> 
> > On 12/29/2012 2:48 PM, Quint Rankid wrote:
> 
> > 
> 
> >> Given a list like:
> 
> >> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
> 
> >> I would like to be able to do the following as a dict comprehension. 
> 
> >> a = {}
> 
> >> for x in w:
> 
> >>      a[x] = a.get(x,0) + 1
> 
> >> results in a having the value:
> 
> >> {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}
> 
> > 
> 
> > Let me paraphrase this: "I have nice, clear, straightforward,
> 
> > *comprehensible* code that I want to turn into an incomprehensible mess
> 
> > with a 'comprehension." That is the ironic allure of comprehensions.
> 
> 
> 
> But... but... one liner! ONE LINNNNNNEEEERRRRR!!!! Won't somebody think 
> 
> of the lines I'll save!!!!
> 
> 
> 
> *wink*
> 
> 
> 
> 
> 
> In case it's not obvious, I'm 100% agreeing with Terry here. List comps 
> 
> and dict comps are wonderful things, but they can't do everything, and 
> 
> very often even if they can do something they shouldn't because it makes 
> 
> the code inefficient or unreadable.
> 
> 
> 
> There's nothing wrong with a two or three liner.
> 
> 
> 
> 
> 
> 
> 
> -- 
> 
> Steven

This is useful for not being choked in sorting a list
by the notorious quick-sort.

[toc] | [prev] | [next] | [standalone]

#35793

On 12/29/12 15:40, Mitya Sirenef wrote:
>>       >>> w = [1,2,3,1,2,4,4,5,6,1]
>>       >>> s = set(w)
>>       >>> s
>>       set([1, 2, 3, 4, 5, 6])
>>       >>> {x:w.count(x) for x in s}
>>       {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}
>
> Indeed, this is much better -- I didn't think of it..

Except that you're still overwhelmed by iterating over every element 
in "w" for every distinct element.  So you've gone from O(N**2) to 
O(k*N).

The cleanest way to write it (IMHO) is MRAB's

  >>> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
  >>> from collections import Counter
  >>> results = dict(Counter(w))

which should gather all the statistics in one single pass across "w" 
making it O(N), and it's Pythonically readable.

-tkc

[toc] | [prev] | [next] | [standalone]

#35797

[Multipart message — attachments visible in raw view] — view raw

On Sat, Dec 29, 2012 at 7:26 PM, Tim Chase <python.list@tim.thechases.com>wrote:

> On 12/29/12 15:40, Mitya Sirenef wrote:
>
>>       >>> w = [1,2,3,1,2,4,4,5,6,1]
>>>       >>> s = set(w)
>>>       >>> s
>>>       set([1, 2, 3, 4, 5, 6])
>>>       >>> {x:w.count(x) for x in s}
>>>       {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}
>>>
>>
>> Indeed, this is much better -- I didn't think of it..
>>
>
> Except that you're still overwhelmed by iterating over every element in
> "w" for every distinct element.  So you've gone from O(N**2) to O(k*N).
>
> The cleanest way to write it (IMHO) is MRAB's
>
>
>  >>> w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
>  >>> from collections import Counter
>  >>> results = dict(Counter(w))
>
> which should gather all the statistics in one single pass across "w"
> making it O(N), and it's Pythonically readable.
>
> -tkc
>
> I like this too.  I haven't learned about collections module yet.  Thanks
for the pointer

>
>
> --
> http://mail.python.org/**mailman/listinfo/python-list<http://mail.python.org/mailman/listinfo/python-list>
>



-- 
Joel Goldstick

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web