Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #60658 > unrolled thread

Python and PEP8 - Recommendations on breaking up long lines?

Back to article view | Back to comp.lang.python

Contents

  Python and PEP8 - Recommendations on breaking up long lines? Victor Hooi <victorhooi@gmail.com> - 2013-11-27 17:57 -0800
    Re: Python and PEP8 - Recommendations on breaking up long lines? Victor Hooi <victorhooi@gmail.com> - 2013-11-27 18:03 -0800
      Re: Python and PEP8 - Recommendations on breaking up long lines? Ben Finney <ben+python@benfinney.id.au> - 2013-11-28 13:55 +1100
      Re: Python and PEP8 - Recommendations on breaking up long lines? Terry Reedy <tjreedy@udel.edu> - 2013-11-27 22:03 -0500
      Re: Python and PEP8 - Recommendations on breaking up long lines? Ned Batchelder <ned@nedbatchelder.com> - 2013-11-27 22:05 -0500
      Re: Python and PEP8 - Recommendations on breaking up long lines? Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2013-11-28 08:12 +0200
        Re: Python and PEP8 - Recommendations on breaking up long lines? Chris Angelico <rosuav@gmail.com> - 2013-11-28 17:22 +1100
      Re: Python and PEP8 - Recommendations on breaking up long lines? Roy Smith <roy@panix.com> - 2013-11-28 10:08 -0500
    Re: Python and PEP8 - Recommendations on breaking up long lines? Ben Finney <ben+python@benfinney.id.au> - 2013-11-28 13:47 +1100
      Re: Python and PEP8 - Recommendations on breaking up long lines? Steven D'Aprano <steve@pearwood.info> - 2013-11-28 03:02 +0000
        Re: Python and PEP8 - Recommendations on breaking up long lines? Ben Finney <ben+python@benfinney.id.au> - 2013-11-28 14:14 +1100
    Re: Python and PEP8 - Recommendations on breaking up long lines? Terry Reedy <tjreedy@udel.edu> - 2013-11-27 21:55 -0500
    Re: Python and PEP8 - Recommendations on breaking up long lines? Ned Batchelder <ned@nedbatchelder.com> - 2013-11-27 21:59 -0500
      Re: Python and PEP8 - Recommendations on breaking up long lines? Steven D'Aprano <steve@pearwood.info> - 2013-11-28 03:58 +0000
        Re: Python and PEP8 - Recommendations on breaking up long lines? Tim Chase <python.list@tim.thechases.com> - 2013-11-28 08:04 -0600
        Re: Python and PEP8 - Recommendations on breaking up long lines? Chris Angelico <rosuav@gmail.com> - 2013-11-29 01:21 +1100
        Re: Python and PEP8 - Recommendations on breaking up long lines? Ned Batchelder <ned@nedbatchelder.com> - 2013-11-28 12:26 -0500
    Re: Python and PEP8 - Recommendations on breaking up long lines? Ben Finney <ben+python@benfinney.id.au> - 2013-11-28 14:06 +1100
    Re: Python and PEP8 - Recommendations on breaking up long lines? Neil Cerutti <mr.cerutti@gmail.com> - 2013-11-27 22:09 -0500
    Re: Python and PEP8 - Recommendations on breaking up long lines? Ethan Furman <ethan@stoneleaf.us> - 2013-11-27 19:15 -0800
    Re: Python and PEP8 - Recommendations on breaking up long lines? Steven D'Aprano <steve@pearwood.info> - 2013-11-28 03:57 +0000
      Re: Python and PEP8 - Recommendations on breaking up long lines? Steven D'Aprano <steve@pearwood.info> - 2013-11-28 04:03 +0000
    Re: Python and PEP8 - Recommendations on breaking up long lines? MRAB <python@mrabarnett.plus.com> - 2013-11-28 12:43 +0000
    Re: Python and PEP8 - Recommendations on breaking up long lines? Walter Hurry <walterhurry@lavabit.com> - 2013-11-28 17:38 +0000
    Re: Python and PEP8 - Recommendations on breaking up long lines? Roel Schroeven <roel@roelschroeven.net> - 2013-11-28 19:37 +0100
    Re: Python and PEP8 - Recommendations on breaking up long lines? Nick Mellor <thebalancepro@gmail.com> - 2013-12-04 17:47 -0800

Page 2 of 2 — ← Prev page 1 [2]

#60674

On Wed, 27 Nov 2013 17:57:13 -0800, Victor Hooi wrote:

> Hi,
> 
> I'm running pep8 across my code, and getting warnings about my long
> lines (> 80 characters).
> 
> I'm wonder what's the recommended way to handle the below cases, and fit
> under 80 characters.
> 
> First example - multiple context handlers:
> 
>             with open(self.full_path, 'r') as input,
>             open(self.output_csv, 'ab') as output:


if True:  # add indentation, just for the example's sake.
    if True:
        if True:
            with (open(self.full_path, 'r') as input,
                      open(self.output_csv, 'ab') as output):
                do_this()
                do_that()


 
> Second example - long error messages:
> 
>             self.logger.error('Unable to open input or output file - %s.
>             Please check you have sufficient permissions and the file
>             and parent directory exist.' % e)

Long strings are always ugly :-(

But you can use implicit concatenation to make them a little less so.



if True:  # add indentation, just for the example's sake.
    if True:
        if True:
            self.logger.error(
                'Unable to open input or output file - %s.'
                ' Please check you have sufficient permissions and
                ' the file and parent directory exist.' 
                % e)

Notice that my convention is to use a leading space in the strings when 
doing implicit concatenation, if possible. E.g.:

("Hello"
 " World!")

rather than:

("Hello "
 "World!")


> Third example - long comments:
> 
>             """ NB - We can't use Psycopg2's parametised statements
>             here, as that automatically wraps everything in single
>             quotes. So s3://my_bucket/my_file.csv.gz would become
>             s3://'my_bucket'/'my_file.csv.gz'. Hence, we use Python's
>             normal string formating - this could potentially exposes us
>             to SQL injection attacks via the config.yaml file.
>             I'm not aware of any easy ways around this currently though
>             - I'm open to suggestions though.
>             See
>             http://stackoverflow.com/questions/9354392/psycopg2-cursor-
execute-with-sql-query-parameter-causes-syntax-error
>             for further information. """
> 
> In this case, I'm guessing a using triple quotes (""") is a better idea
> with multi-line comments, right?

*shrug* That's a matter of personal preference.


> However, I've noticed that I can't seem to put in line-breaks inside the
> comment without triggering a warning. For example, trying to put in
> another empty line in between lines 6 and 7 above causes a warning.
> 
> Also, how would I split up the long URLs? Breaking it up makes it
> annoying to use the URL. Thoughts?

I hate long URLs.

There's no good solution to long URLs. Just leave them as they come, and 
if your style checker allows you to flag an exception to the long-line 
rules, use it.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#60676

On Thu, 28 Nov 2013 03:57:03 +0000, Steven D'Aprano wrote:


> if True:  # add indentation, just for the example's sake.
>     if True:
>         if True:
>             with (open(self.full_path, 'r') as input,
>                       open(self.output_csv, 'ab') as output):
>                 do_this()
>                 do_that()


Bah! Apparently you can't do that.

py> with (open(self.full_path, 'r') as input,
  File "<stdin>", line 1
    with (open(self.full_path, 'r') as input,
                                     ^
SyntaxError: invalid syntax




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#60684

On 28/11/2013 03:06, Ben Finney wrote:
> Ned Batchelder <ned@nedbatchelder.com> writes:
>
>> The important thing in a with statement is that the assigned name will
>> be closed (or otherwise exited) automatically.  The open call is just
>> the expression used to assign the name.  The expression there isn't
>> really important.  This looks odd, but works the same as what you
>> have:
>>
>>     input = open(self.full_path)
>>     output = open(self.output_csv, 'ab')
>>     with input as input, output as output:
>>         ...
>
> Excellent! That solves for me a significant irritation of the syntax for
> multiple context managers. Thank you.
>
What if the second 'open' raises an exception?

[toc] | [prev] | [next] | [standalone]

#60718

On Thu, 28 Nov 2013 10:08:30 -0500, Roy Smith wrote:

> In article <34479463-b8a8-4417-9989-cd29369461c2@googlegroups.com>,
>  Victor Hooi <victorhooi@gmail.com> wrote:
> 
>>     cur.executemany("INSERT INTO foobar_foobar_files VALUES (?)",
>>                     [[os.path.relpath(filename, foobar_input_folder)]
>>                     for filename in filenames])
> 
> I don't often write raw SQL embedded in Python.  I'm much more likely to
> use some sort of ORM layer.  But, if I were doing this, I would break it
> up something like:
> 
> 
> 
> There's a few different strategies I employed there.  My first thought
> was a logical break of computing the list of pathnames vs. inserting
> them into the database.  That got me here:
> 
>     paths = [[os.path.relpath(filename, foobar_input_folder)] \
>       for filename in filenames]
>     cur.executemany("INSERT INTO foobar_foobar_files VALUES (?)",
>                     paths)
> 
> I wouldn't have actually broken the first line with the backslash, but
> I'm doing that to appease my news posting software.
> 
> My next step would be some simple textual changes; I'd get rid of the
> overly-line variable names.  In general, I don't like very short
> variable names, but I'm OK with them as long as the scope is very small,
> as it is here:
> 
>     paths = [[os.path.relpath(fn, folder)] for fn in filenames]
>     cur.executemany("INSERT INTO foobar_foobar_files VALUES (?)",
>                     paths)
> 
> I'd probably factor out the double lookup of os.path.relpath.  I think
> this is easier to read:
> 
>     relpath = os.path.relpath paths = [[relpath(fn, folder)] for fn in
>     filenames] cur.executemany("INSERT INTO foobar_foobar_files VALUES
>     (?)",
>                     paths)
> 
> If filenames was a very long list, it would also be a little bit faster
> to execute, but that's such a small factor as to probably be
> unmeasurable.
> 
> And, finally, I'd probably move one set of square brackets down into the
> SQL statement.  It really makes more sense there anyway; the bundling up
> of the arguments into a sequence is more a part of the database API than
> it is inherent to the data.
> 
>     relpath = os.path.relpath paths = [relpath(fn, foobar_input_folder)
>     for fn in filenames]
>     cur.executemany("INSERT INTO foobar_foobar_files VALUES (?)",
>                     [paths])

I had a problem similar to this some time ago. Someone (here) suggested 
textwrap.dedent, which for me was just the ticket.

[toc] | [prev] | [next] | [standalone]

#60723

Victor Hooi schreef:
>             with open(self.full_path, 'r') as input, open(self.output_csv, 'ab') as output:

I think I would write that as

     with open(self.full_path, 'r') as input:
         with open(self.output_csv, 'ab') as output:

That introduces an extra indentation level, but that doesn't really 
bother me. Does this seem like a good idea?


Best regards,
Roel

-- 
"Met een spitsvondig citaat bewijs je niets."
         -- Voltaire

[toc] | [prev] | [next] | [standalone]

#61068

Hi Victor,

I use PyCharm which is set up by default to warn when line length exceeds 120 chars, not 80. Perhaps times have changed?

I often break comprehensions at the for, in and else clauses. It's often not for line length reasons but because it's easier to follow the code that way. I have heard this is how Haskell programmers tend to use comprehensions (comprehensions are from Haskell originally):

location=random.choice([loc['pk']                                 
                        for loc                                   
                        in locations.whole_register()             
                        if loc['fields']['provider_id'] == provider_id])))

The other suggestion I have is to put the with clauses in a generator function. This saves you a level or more of indentation and modularises the code usefully. Here's an example:

def spreadsheet(csv_filename):
    with open(csv_filename) as csv_file:
        for csv_row in list(csv.DictReader(csv_file, delimiter='\t')):
            yield csv_row

then invoked using:

for row in spreadsheet("...")
    # your processing code here

Cheers,

Nick

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.python

csiph-web