Re: Comprehension with two variables - explanation needed

Path	csiph.com!usenet.pasdenom.info!weretis.net!feeder1.news.weretis.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed2a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path	<roy@panix.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.000
X-Spam-Evidence	'H': 1.00; 'S': 0.00; 'third-party': 0.04; 'column': 0.07; 'nested': 0.07; 'subject:two': 0.07; 'expression:': 0.09; 'imported': 0.09; 'lines.': 0.09; 'matched': 0.09; 'cc:addr:python-list': 0.11; 'python': 0.11; 'wrote': 0.14; 'mostly': 0.14; '"for"': 0.16; '(there': 0.16; '1400': 0.16; '23,': 0.16; 'correlation': 0.16; 'crude': 0.16; 'doing,': 0.16; 'expressions)': 0.16; 'grep': 0.16; 'ivan': 0.16; 'merely': 0.16; 'naming': 0.16; 'nesting': 0.16; 'otoh,': 0.16; 'pity': 0.16; 'received:166.84': 0.16; 'received:166.84.1': 0.16; 'received:166.84.1.89': 0.16; 'received:mailbackend.panix.com': 0.16; 'received:panix.com': 0.16; 'roy': 0.16; 'sure.': 0.16; ':-)': 0.16; 'wrote:': 0.18; 'trying': 0.19; 'received:10.0.1': 0.19; 'received:166': 0.19; 'work,': 0.20; 'cc:addr:python.org': 0.22; 'rapidly': 0.24; 'skip': 0.24; 'stick': 0.24; 'cc:2**0': 0.24; 'source': 0.25; 'header:In-Reply-To:1': 0.27; 'leave': 0.29; 'am,': 0.29; 'generally': 0.29; "i'm": 0.30; 'code': 0.31; 'regular': 0.32; 'subject:with': 0.35; "can't": 0.35; "who's": 0.35; 'but': 0.35; 'next': 0.36; 'shows': 0.36; 'charset:us- ascii': 0.36; 'received:10.0': 0.36; 'experience,': 0.37; 'two': 0.37; 'list': 0.37; 'received:10': 0.37; 'nov': 0.38; 'subject:needed': 0.38; 'needed': 0.38; 'enough': 0.39; 'read': 0.60; 'full': 0.61; 'subject:skip:C 10': 0.61; 'simply': 0.61; 'header:Message-Id:1': 0.63; 'name': 0.63; 'such': 0.63; 'our': 0.64; 'more': 0.64; 'to:addr:gmail.com': 0.65; 'between': 0.67; 'smith': 0.68; 'glad': 0.83; '11:45': 0.84; '2014,': 0.84; 'email addr:panix.com': 0.84
Subject	Re: Comprehension with two variables - explanation needed
Mime-Version	1.0 (Apple Message framework v1283)
Content-Type	text/plain; charset=us-ascii
From	Roy Smith <roy@panix.com>
In-Reply-To	<CANc-5UxMAVzgM56LfRSyvBErqPedSZ5LreKBz6ksSeXXG4W9YQ@mail.gmail.com>
Date	Sun, 23 Nov 2014 12:08:19 -0500
Content-Transfer-Encoding	quoted-printable
References	<mailman.16184.1416754678.18130.python-list@python.org> <d1194d28-e9ed-4bd3-86e9-899e13a17097@googlegroups.com> <002501d00733$6995dce0$3cc196a0$@gmail.com> <mailman.16191.1416757555.18130.python-list@python.org> <roy-B11F64.10563923112014@news.panix.com> <CANc-5UxMAVzgM56LfRSyvBErqPedSZ5LreKBz6ksSeXXG4W9YQ@mail.gmail.com>
To	Skip Montanaro <skip.montanaro@gmail.com>
X-Mailer	Apple Mail (2.1283)
Cc	Python <python-list@python.org>
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.15
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.16200.1416762503.18130.python-list@python.org> (permalink)
Lines	58
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1416762503 news.xs4all.nl 2881 [2001:888:2000:d::a6]:50816
X-Complaints-To	abuse@xs4all.nl
Xref	csiph.com comp.lang.python:81392

Show key headers only | View raw

We seem to be somewhat more liberal about nested comprehensions here, but I can't say I'm proud of that :-)

906 Python source files, 109k lines. 

$ find . -name '*.py' | xargs grep '\[.* for .*\]' | wc -l
729

$ find . -name '*.py' | xargs grep '\[.* for .* for .*\]' | wc -l
46

Without naming names, a quick look into our nested comprehensions with "hg blame" shows a very strong correlation between "people who wrote many nested comprehensions" and "people who's code I find difficult to read in general".  I see one of these that has my name on the commit:

   report = [keys] + [[line[key] for key in keys] for line in results]

but I'm glad to report that this was merely some third-party code that I imported into our repo.  Whew, that was close!

On Nov 23, 2014, at 11:45 AM, Skip Montanaro wrote:

> On Sun, Nov 23, 2014 at 9:57 AM, Roy Smith <roy@panix.com> wrote:
>> 
>> If it was complicated enough that you needed to loopify it to
>> understand what it's doing, have pity on the next person who has to
>> maintain your code and leave it as a loop
> 
> Well, sure. I was mostly trying to give Ivan a path out of the weeds.
> Poking through the code I'm involved with at work, using a crude
> measuring stick (a couple regular expressions) I found over 1400 one
> line list comprehensions, those which matched this regular expression:
> 
> \[.* for .*\]
> 
> OTOH, I only found 14 with two "for" keywords:
> 
> \[.* for .* for .*\]
> 
> (There may well have been more nested loop list comprehensions, simply
> because it's hard to cram the full listcomp into 80 columns, my
> self-imposed column limit.)
> 
> In my experience, list comprehensions rapidly get unwieldy once you
> get to nesting them. I generally unravel such items into a nested set
> of for loops.
> 
> Skip
> 


--
Roy Smith
roy@panix.com

Thread

Comprehension with two variables - explanation needed "Ivan Evstegneev" <webmailgroups@gmail.com> - 2014-11-23 16:56 +0200
  Re: Comprehension with two variables - explanation needed Rustom Mody <rustompmody@gmail.com> - 2014-11-23 07:09 -0800
    RE: Comprehension with two variables - explanation needed "Ivan Evstegneev" <webmailgroups@gmail.com> - 2014-11-23 17:37 +0200
    Re: Comprehension with two variables - explanation needed Chris Angelico <rosuav@gmail.com> - 2014-11-24 02:42 +1100
    RE: Comprehension with two variables - explanation needed Skip Montanaro <skip.montanaro@gmail.com> - 2014-11-23 09:45 -0600
      Re: Comprehension with two variables - explanation needed Roy Smith <roy@panix.com> - 2014-11-23 10:57 -0500
        Re: Comprehension with two variables - explanation needed Rustom Mody <rustompmody@gmail.com> - 2014-11-23 08:45 -0800
          Re: Comprehension with two variables - explanation needed Rustom Mody <rustompmody@gmail.com> - 2014-11-23 09:17 -0800
          RE: Comprehension with two variables - explanation needed "Ivan Evstegneev" <webmailgroups@gmail.com> - 2014-11-23 19:27 +0200
          Re: Comprehension with two variables - explanation needed Steven D'Aprano <steve@pearwood.info> - 2014-11-24 04:42 +0000
            Re: Comprehension with two variables - explanation needed Chris Angelico <rosuav@gmail.com> - 2014-11-24 15:51 +1100
              Re: Comprehension with two variables - explanation needed Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-25 09:20 +1100
            Re: Comprehension with two variables - explanation needed Rustom Mody <rustompmody@gmail.com> - 2014-11-23 22:47 -0800
            Re: Comprehension with two variables - explanation needed Dan Stromberg <drsalists@gmail.com> - 2014-11-24 16:42 -0800
        Re: Comprehension with two variables - explanation needed Skip Montanaro <skip.montanaro@gmail.com> - 2014-11-23 10:45 -0600
        Re: Comprehension with two variables - explanation needed Roy Smith <roy@panix.com> - 2014-11-23 12:08 -0500
        Re: Comprehension with two variables - explanation needed Jon Ribbens <jon+usenet@unequivocal.co.uk> - 2014-11-23 17:46 +0000

csiph-web