Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #60658 > unrolled thread

Python and PEP8 - Recommendations on breaking up long lines?

Back to article view | Back to comp.lang.python

Contents

  Python and PEP8 - Recommendations on breaking up long lines? Victor Hooi <victorhooi@gmail.com> - 2013-11-27 17:57 -0800
    Re: Python and PEP8 - Recommendations on breaking up long lines? Victor Hooi <victorhooi@gmail.com> - 2013-11-27 18:03 -0800
      Re: Python and PEP8 - Recommendations on breaking up long lines? Ben Finney <ben+python@benfinney.id.au> - 2013-11-28 13:55 +1100
      Re: Python and PEP8 - Recommendations on breaking up long lines? Terry Reedy <tjreedy@udel.edu> - 2013-11-27 22:03 -0500
      Re: Python and PEP8 - Recommendations on breaking up long lines? Ned Batchelder <ned@nedbatchelder.com> - 2013-11-27 22:05 -0500
      Re: Python and PEP8 - Recommendations on breaking up long lines? Jussi Piitulainen <jpiitula@ling.helsinki.fi> - 2013-11-28 08:12 +0200
        Re: Python and PEP8 - Recommendations on breaking up long lines? Chris Angelico <rosuav@gmail.com> - 2013-11-28 17:22 +1100
      Re: Python and PEP8 - Recommendations on breaking up long lines? Roy Smith <roy@panix.com> - 2013-11-28 10:08 -0500
    Re: Python and PEP8 - Recommendations on breaking up long lines? Ben Finney <ben+python@benfinney.id.au> - 2013-11-28 13:47 +1100
      Re: Python and PEP8 - Recommendations on breaking up long lines? Steven D'Aprano <steve@pearwood.info> - 2013-11-28 03:02 +0000
        Re: Python and PEP8 - Recommendations on breaking up long lines? Ben Finney <ben+python@benfinney.id.au> - 2013-11-28 14:14 +1100
    Re: Python and PEP8 - Recommendations on breaking up long lines? Terry Reedy <tjreedy@udel.edu> - 2013-11-27 21:55 -0500
    Re: Python and PEP8 - Recommendations on breaking up long lines? Ned Batchelder <ned@nedbatchelder.com> - 2013-11-27 21:59 -0500
      Re: Python and PEP8 - Recommendations on breaking up long lines? Steven D'Aprano <steve@pearwood.info> - 2013-11-28 03:58 +0000
        Re: Python and PEP8 - Recommendations on breaking up long lines? Tim Chase <python.list@tim.thechases.com> - 2013-11-28 08:04 -0600
        Re: Python and PEP8 - Recommendations on breaking up long lines? Chris Angelico <rosuav@gmail.com> - 2013-11-29 01:21 +1100
        Re: Python and PEP8 - Recommendations on breaking up long lines? Ned Batchelder <ned@nedbatchelder.com> - 2013-11-28 12:26 -0500
    Re: Python and PEP8 - Recommendations on breaking up long lines? Ben Finney <ben+python@benfinney.id.au> - 2013-11-28 14:06 +1100
    Re: Python and PEP8 - Recommendations on breaking up long lines? Neil Cerutti <mr.cerutti@gmail.com> - 2013-11-27 22:09 -0500
    Re: Python and PEP8 - Recommendations on breaking up long lines? Ethan Furman <ethan@stoneleaf.us> - 2013-11-27 19:15 -0800
    Re: Python and PEP8 - Recommendations on breaking up long lines? Steven D'Aprano <steve@pearwood.info> - 2013-11-28 03:57 +0000
      Re: Python and PEP8 - Recommendations on breaking up long lines? Steven D'Aprano <steve@pearwood.info> - 2013-11-28 04:03 +0000
    Re: Python and PEP8 - Recommendations on breaking up long lines? MRAB <python@mrabarnett.plus.com> - 2013-11-28 12:43 +0000
    Re: Python and PEP8 - Recommendations on breaking up long lines? Walter Hurry <walterhurry@lavabit.com> - 2013-11-28 17:38 +0000
    Re: Python and PEP8 - Recommendations on breaking up long lines? Roel Schroeven <roel@roelschroeven.net> - 2013-11-28 19:37 +0100
    Re: Python and PEP8 - Recommendations on breaking up long lines? Nick Mellor <thebalancepro@gmail.com> - 2013-12-04 17:47 -0800

Page 1 of 2  [1] 2  Next page →

#60658 — Python and PEP8 - Recommendations on breaking up long lines?

Hi,

I'm running pep8 across my code, and getting warnings about my long lines (> 80 characters).

I'm wonder what's the recommended way to handle the below cases, and fit under 80 characters.

First example - multiple context handlers:

            with open(self.full_path, 'r') as input, open(self.output_csv, 'ab') as output:

and in my case, with indents, the 80-character marks is just before the ending "as output".

What's the standard recognised way to split this across multiple lines, so that I'm under 80 characters?

I can't just split after the "as input," as that isn't valid syntax, and there's no convenient parentheses for me to split over.

Is there a standard Pythonic way?

Second example - long error messages:

            self.logger.error('Unable to open input or output file - %s. Please check you have sufficient permissions and the file and parent directory exist.' % e)

I can use triple quotes:

            self.logger.error(
                """Unable to open input or output file - %s. Please check you
                have sufficient permissions and the file and parent directory
                exist.""" % e)

However, that will introduce newlines in the message, which I don't want.

I can use backslashes:

            self.logger.error(
                'Unable to open input or output file - %s. Please check you\
                have sufficient permissions and the file and parent directory\
                exist.' % e)

which won't introduce newlines.

Or I can put them all as separate strings, and trust Python to glue them together:

            self.logger.error(
                'Unable to open input or output file - %s. Please check you'
                'have sufficient permissions and the file and parent directory'
                'exist.' % e)

Which way is the recommended Pythonic way?

Third example - long comments:

            """ NB - We can't use Psycopg2's parametised statements here, as
            that automatically wraps everything in single quotes.
            So s3://my_bucket/my_file.csv.gz would become s3://'my_bucket'/'my_file.csv.gz'.
            Hence, we use Python's normal string formating - this could
            potentially exposes us to SQL injection attacks via the config.yaml
            file.
            I'm not aware of any easy ways around this currently though - I'm
            open to suggestions though.
            See
            http://stackoverflow.com/questions/9354392/psycopg2-cursor-execute-with-sql-query-parameter-causes-syntax-error
            for further information. """

In this case, I'm guessing a using triple quotes (""") is a better idea with multi-line comments, right?

However, I've noticed that I can't seem to put in line-breaks inside the comment without triggering a warning. For example, trying to put in another empty line in between lines 6 and 7 above causes a warning.

Also, how would I split up the long URLs? Breaking it up makes it annoying to use the URL. Thoughts?

Cheers,
Victor

[toc] | [next] | [standalone]

#60659

Hi,

Also, forgot two other examples that are causing me grief:

    cur.executemany("INSERT INTO foobar_foobar_files VALUES (?)",
                    [[os.path.relpath(filename, foobar_input_folder)] for filename in filenames])

I've already broken it up using the parentheses, not sure what's the tidy way to break it up again to fit under 80? In this case, the 80-character mark is hitting me around the "for filename" towards the end.

and:

                    if os.path.join(root, file) not in previously_processed_files and os.path.join(root, file)[:-3] not in previously_processed_files:

In this case, the 80-character mark is actually partway through "previously processed files" (the first occurrence)...

Cheers,
Victor

On Thursday, 28 November 2013 12:57:13 UTC+11, Victor Hooi  wrote:
> Hi,
> 
> 
> 
> I'm running pep8 across my code, and getting warnings about my long lines (> 80 characters).
> 
> 
> 
> I'm wonder what's the recommended way to handle the below cases, and fit under 80 characters.
> 
> 
> 
> First example - multiple context handlers:
> 
> 
> 
>             with open(self.full_path, 'r') as input, open(self.output_csv, 'ab') as output:
> 
> 
> 
> and in my case, with indents, the 80-character marks is just before the ending "as output".
> 
> 
> 
> What's the standard recognised way to split this across multiple lines, so that I'm under 80 characters?
> 
> 
> 
> I can't just split after the "as input," as that isn't valid syntax, and there's no convenient parentheses for me to split over.
> 
> 
> 
> Is there a standard Pythonic way?
> 
> 
> 
> Second example - long error messages:
> 
> 
> 
>             self.logger.error('Unable to open input or output file - %s. Please check you have sufficient permissions and the file and parent directory exist.' % e)
> 
> 
> 
> I can use triple quotes:
> 
> 
> 
>             self.logger.error(
> 
>                 """Unable to open input or output file - %s. Please check you
> 
>                 have sufficient permissions and the file and parent directory
> 
>                 exist.""" % e)
> 
> 
> 
> However, that will introduce newlines in the message, which I don't want.
> 
> 
> 
> I can use backslashes:
> 
> 
> 
>             self.logger.error(
> 
>                 'Unable to open input or output file - %s. Please check you\
> 
>                 have sufficient permissions and the file and parent directory\
> 
>                 exist.' % e)
> 
> 
> 
> which won't introduce newlines.
> 
> 
> 
> Or I can put them all as separate strings, and trust Python to glue them together:
> 
> 
> 
>             self.logger.error(
> 
>                 'Unable to open input or output file - %s. Please check you'
> 
>                 'have sufficient permissions and the file and parent directory'
> 
>                 'exist.' % e)
> 
> 
> 
> Which way is the recommended Pythonic way?
> 
> 
> 
> Third example - long comments:
> 
> 
> 
>             """ NB - We can't use Psycopg2's parametised statements here, as
> 
>             that automatically wraps everything in single quotes.
> 
>             So s3://my_bucket/my_file.csv.gz would become s3://'my_bucket'/'my_file.csv.gz'.
> 
>             Hence, we use Python's normal string formating - this could
> 
>             potentially exposes us to SQL injection attacks via the config.yaml
> 
>             file.
> 
>             I'm not aware of any easy ways around this currently though - I'm
> 
>             open to suggestions though.
> 
>             See
> 
>             http://stackoverflow.com/questions/9354392/psycopg2-cursor-execute-with-sql-query-parameter-causes-syntax-error
> 
>             for further information. """
> 
> 
> 
> In this case, I'm guessing a using triple quotes (""") is a better idea with multi-line comments, right?
> 
> 
> 
> However, I've noticed that I can't seem to put in line-breaks inside the comment without triggering a warning. For example, trying to put in another empty line in between lines 6 and 7 above causes a warning.
> 
> 
> 
> Also, how would I split up the long URLs? Breaking it up makes it annoying to use the URL. Thoughts?
> 
> 
> 
> Cheers,
> 
> Victor

[toc] | [prev] | [next] | [standalone]

#60663

Victor Hooi <victorhooi@gmail.com> writes:

>     cur.executemany("INSERT INTO foobar_foobar_files VALUES (?)",
>                     [[os.path.relpath(filename, foobar_input_folder)] for filename in filenames])
>
> I've already broken it up using the parentheses

But now the continuation line indentation is needlessly dependent on the
content of the previous line.

Better to just use a standard additional indentation (I prefer 8 columns
to easily distinguish from block indentation) for any continuation
line::

    cur.executemany(
            "INSERT INTO foobar_foobar_files VALUES (?)", [
                [os.path.relpath(filename, foobar_input_folder)]
                for filename in filenames])

Notice that further continuation within an existing continuation doesn't
get another 8, but only another 4 columns. The whole statement is
already distinguished, I don't need to add huge amounts of further
indentation for that purpose.

>     if os.path.join(root, file) not in previously_processed_files and os.path.join(root, file)[:-3] not in previously_processed_files:

In cases where the line is long because of a long expression, enclosing
that expression in parens for clarification will then give you an easy
way to break for continuation lines::

    if (
            os.path.join(root, file) not in previously_processed_files
            and os.path.join(root, file)[:-3] not in previously_processed_files):

But that's still some long lines, and your repeated use of a computed
value is an ideal opportunity to bind it to a convenience name for local
use::

    file_path = os.path.join(root, file)
    if (
            file_path not in previously_processed_files
            and file_path[:-3] not in previously_processed_files):

Not least because you're very likely going to use the value of
‘file_path’ yet again in the body of that ‘if’ block.

-- 
 \         “Two paradoxes are better than one; they may even suggest a |
  `\                                         solution.” —Edward Teller |
_o__)                                                                  |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#60667

On 11/27/2013 9:03 PM, Victor Hooi wrote:
> Hi,
>
> Also, forgot two other examples that are causing me grief:
>
>      cur.executemany("INSERT INTO foobar_foobar_files VALUES (?)",
>                      [[os.path.relpath(filename, foobar_input_folder)] for filename in filenames])
>
> I've already broken it up using the parentheses, not sure what's the tidy way to break it up again to fit under 80? In this case, the 80-character mark is hitting me around the "for filename" towards the end.

add another linebreak, you are still inside the parens.

> and:
>
>                      if os.path.join(root, file) not in previously_processed_files and os.path.join(root, file)[:-3] not in previously_processed_files:
>
> In this case, the 80-character mark is actually partway through "previously processed files" (the first occurrence)...

You can add parens. In this case, factoring out the common subexpression 
and replaced the repeated long name does the job.
                     p = os.path.join(root, file)
                     ppf = previously_processed_files
                     if p not in ppf and p[:-3] not in ppf:

[snip previous post]

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#60668

On 11/27/13 9:03 PM, Victor Hooi wrote:
> Hi,
>
> Also, forgot two other examples that are causing me grief:
>
>      cur.executemany("INSERT INTO foobar_foobar_files VALUES (?)",
>                      [[os.path.relpath(filename, foobar_input_folder)] for filename in filenames])
>
> I've already broken it up using the parentheses, not sure what's the tidy way to break it up again to fit under 80? In this case, the 80-character mark is hitting me around the "for filename" towards the end.

     file_values = [
         (os.path.relpath(filename, foobar_input_folder),)
         for filename in filenames
     ]
     cur.executemany(
         "INSERT INTO foobar_foobar_files VALUES (?)",
         file_values
     )

>
> and:
>
>                      if os.path.join(root, file) not in previously_processed_files and os.path.join(root, file)[:-3] not in previously_processed_files:
>
> In this case, the 80-character mark is actually partway through "previously processed files" (the first occurrence)...

     full_file = os.path.join(root, file)
     full_not_in = full_file not in previously_processed_files
     tail_not_in = full_file[:-3] not in previously_processed_files
     if full_not_in and tail_not_in:
         ...

This has the advantage of naming these complex intermediate conditions, 
and giving you natural places to write comments explaining them.  Why 
[:-3] for example?

More, shorter, statements.

--Ned.

>
> Cheers,
> Victor
>

[toc] | [prev] | [next] | [standalone]

#60678

Victor Hooi <victorhooi@gmail.com> writes:

> Hi,
> 
> Also, forgot two other examples that are causing me grief:
 
     cur.executemany("INSERT INTO foobar_foobar_files VALUES (?)",
                     [[os.path.relpath(filename, foobar_input_folder)]
                      for filename in filenames])
 
> I've already broken it up using the parentheses, not sure what's the
> tidy way to break it up again to fit under 80? In this case, the
> 80-character mark is hitting me around the "for filename" towards
> the end.

That's a natural break. I did it to your code above. Another is the
condition, if it's there:

     cur.executemany("INSERT INTO foobar_foobar_files VALUES (?)",
                     [[os.path.relpath(filename, foobar_input_folder)]
                      for filename in filenames
                      if not filename.startswith('tmp')])

There's much freedom of indentation inside the brackets, but these
points are natural.

> and:

Put the long expression in parentheses and you are again free to break
and indent; I tend to have extra spaces inside these parentheses, but
I have no idea about any standards:

                     if ( os.path.join(root, file)
                          not in previously_processed_files and
                          os.path.join(root, file)[:-3]
                          not in previously_processed_files ):

> In this case, the 80-character mark is actually partway through
> "previously processed files" (the first occurrence)...

Try to find natural breaks, between phrases, and maybe highlight
operators by putting them in the beginning of a line like.

Or even this:
                     pre = previously_processed_files
                     if ( os.path.join(root, file) not in pre and
                          os.path.join(root, file)[:-3] not in pre ):

[toc] | [prev] | [next] | [standalone]

#60680

On Thu, Nov 28, 2013 at 5:12 PM, Jussi Piitulainen
<jpiitula@ling.helsinki.fi> wrote:
>                      if ( os.path.join(root, file)
>                           not in previously_processed_files and
>                           os.path.join(root, file)[:-3]
>                           not in previously_processed_files ):

Eww, don't like this. The indentation puts the two halves of the
condition on equal footing, which they're not.

ChrisA

[toc] | [prev] | [next] | [standalone]

#60693

In article <34479463-b8a8-4417-9989-cd29369461c2@googlegroups.com>,
 Victor Hooi <victorhooi@gmail.com> wrote:

>     cur.executemany("INSERT INTO foobar_foobar_files VALUES (?)",
>                     [[os.path.relpath(filename, foobar_input_folder)] for 
>                     filename in filenames])

I don't often write raw SQL embedded in Python.  I'm much more likely to 
use some sort of ORM layer.  But, if I were doing this, I would break it 
up something like:



There's a few different strategies I employed there.  My first thought 
was a logical break of computing the list of pathnames vs. inserting 
them into the database.  That got me here:

    paths = [[os.path.relpath(filename, foobar_input_folder)] \
      for filename in filenames]
    cur.executemany("INSERT INTO foobar_foobar_files VALUES (?)",
                    paths)

I wouldn't have actually broken the first line with the backslash, but 
I'm doing that to appease my news posting software.

My next step would be some simple textual changes; I'd get rid of the 
overly-line variable names.  In general, I don't like very short 
variable names, but I'm OK with them as long as the scope is very small, 
as it is here:

    paths = [[os.path.relpath(fn, folder)] for fn in filenames]
    cur.executemany("INSERT INTO foobar_foobar_files VALUES (?)",
                    paths)

I'd probably factor out the double lookup of os.path.relpath.  I think 
this is easier to read:

    relpath = os.path.relpath
    paths = [[relpath(fn, folder)] for fn in filenames]
    cur.executemany("INSERT INTO foobar_foobar_files VALUES (?)",
                    paths)

If filenames was a very long list, it would also be a little bit faster 
to execute, but that's such a small factor as to probably be 
unmeasurable.

And, finally, I'd probably move one set of square brackets down into the 
SQL statement.  It really makes more sense there anyway; the bundling up 
of the arguments into a sequence is more a part of the database API than 
it is inherent to the data.

    relpath = os.path.relpath
    paths = [relpath(fn, foobar_input_folder) for fn in filenames]
    cur.executemany("INSERT INTO foobar_foobar_files VALUES (?)",
                    [paths])

[toc] | [prev] | [next] | [standalone]

#60662

Victor Hooi <victorhooi@gmail.com> writes:

> I'm running pep8 across my code, and getting warnings about my long
> lines (> 80 characters).

Great! Thank you for working to make your code readable by keeping lines
reasonably short.

> I'm wonder what's the recommended way to handle the below cases, and
> fit under 80 characters.

In general, I advise:

* Avoid line-end backslashes like the plague. Sometimes they're
  necessary, but try very hard to write code that doesn't need them.

* Break a long line on existing open-parenthesis syntax (whether parens,
  brackets, braces, triple-quotes); early in the line is better, so the
  reader doesn't need to scan a long way back to the next line.

* Ignore the code on the first line for aligning, and instead indent
  continuation lines to a consistent level (i.e. don't line up with
  some arbitrary character on the first line).

* Use eight-column indentation for continuation lines (to clearly
  distinguish from four-column block indentation).

> First example - multiple context handlers:

I haven't used multiples in the same statement yet, and I'm annoyed that
they simultaneously encourage long statements, and have no obvious way
to break on an open parenthesis syntax.

>     with open(self.full_path, 'r') as input, open(self.output_csv, 'ab') as output:

So in this case I don't see a way to avoid the ugly line-end backslash::

    with \
            open(self.full_path, 'r') as input, \
            open(self.output_csv, 'ab') as output:

Or::

    with open(self.full_path, 'r') as input, \
            open(self.output_csv, 'ab') as output:

Both look terrible to me. I'd love to know of a way to avoid
backslashes, while still breaking the line at the comma.

> Second example - long error messages:
>
>     self.logger.error('Unable to open input or output file - %s. Please check you have sufficient permissions and the file and parent directory exist.' % e)

Break at the first open-paren. Use implicit string literal continuation
to break that literal across multiple lines inside the parens. Like so::

    self.logger.error(
            'Unable to open input or output file - %s.'
            ' Please check you have sufficient permissions'
            ' and the file and parent directory exist.'
            % e)

Also, with such a large literal this is an ideal example of why named
parameters are preferable::

    self.logger.error(
            'Unable to open input or output file - {error}.'
            ' Please check you have sufficient permissions'
            ' and the file and parent directory exist.'.format(
                error=e))

> Third example - long comments:
>
>     """ NB - We can't use Psycopg2's parametised statements here, as
>     that automatically wraps everything in single quotes.
>     So s3://my_bucket/my_file.csv.gz would become s3://'my_bucket'/'my_file.csv.gz'.
>     Hence, we use Python's normal string formating - this could
>     potentially exposes us to SQL injection attacks via the config.yaml
>     file.
>     I'm not aware of any easy ways around this currently though - I'm
>     open to suggestions though.
>     See
>     http://stackoverflow.com/questions/9354392/psycopg2-cursor-execute-with-sql-query-parameter-causes-syntax-error
>     for further information. """

That's not syntactically a comment, and I don't think pretending
triple-quoted strings are comments is good practice. If nothing else,
you'll need a special case if you want to enclose something with
existing triple-quotes.

Better to use the standard Python comment style::

    # NB - We can't use Psycopg2's parametised statements here, as
    # that automatically wraps everything in single quotes.
    # So s3://my_bucket/my_file.csv.gz would become s3://'my_bucket'/'my_file.csv.gz'.
    # Hence, we use Python's normal string formating - this could
    # potentially exposes us to SQL injection attacks via the config.yaml
    # file.
    # I'm not aware of any easy ways around this currently though - I'm
    # open to suggestions though.
    # See
    # http://stackoverflow.com/questions/9354392/psycopg2-cursor-execute-with-sql-query-parameter-causes-syntax-error
    # for further information.

which most programmer's text editors can do for you easily by applying
or un-applying comments to a whole selected region of lines with a
single command.

The comments then are correctly identified as comments by all your
programming tools, and you don't have to think about using a different
comment style depending on the content — the same style is used for all
comments.

> Also, how would I split up the long URLs? Breaking it up makes it
> annoying to use the URL. Thoughts?

I'd try very hard to find an equivalent URL that isn't so long :-) but
URLs in comments are a good example of a PEP 8 exception: if the line is
over 80 characters because it contains a long URL in a comment, that's
fine, as I'm not expecting the human reader to be scanning it carefully
like other text in the code.

-- 
 \        “I fly Air Bizarre. You buy a combination one-way round-trip |
  `\    ticket. Leave any Monday, and they bring you back the previous |
_o__)     Friday. That way you still have the weekend.” —Steven Wright |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#60666

On Thu, 28 Nov 2013 13:47:22 +1100, Ben Finney wrote:


>> Third example - long comments:
>>
>>     """ NB - We can't use Psycopg2's parametised statements here, as
>>     that automatically wraps everything in single quotes. So
>>     s3://my_bucket/my_file.csv.gz would become
>>     s3://'my_bucket'/'my_file.csv.gz'. Hence, we use Python's normal
>>     string formating - this could potentially exposes us to SQL
>>     injection attacks via the config.yaml file. I'm not aware of any
>>     easy ways around this currently though - I'm open to suggestions
>>     though.
>>     See
>>     http://stackoverflow.com/questions/9354392/psycopg2-cursor-execute-
with-sql-query-parameter-causes-syntax-error
>>     for further information. """
> 
> That's not syntactically a comment, and I don't think pretending
> triple-quoted strings are comments is good practice. If nothing else,
> you'll need a special case if you want to enclose something with
> existing triple-quotes.

The CPython core devs disagree with you. Using bare strings for comments 
is supported by Python, and the compiler strips them out at compile-time:


steve@runes:~$ python3.3
Python 3.3.0rc3 (default, Sep 27 2012, 18:31:58) 
[GCC 4.4.5] on linux
Type "help", "copyright", "credits" or "license" for more information.
py> import dis
py> dis.dis('x = 1; """Comment"""; y = 1')
  1           0 LOAD_CONST               0 (1) 
              3 STORE_NAME               0 (x) 
              6 LOAD_CONST               0 (1) 
              9 STORE_NAME               1 (y) 
             12 LOAD_CONST               1 (None) 
             15 RETURN_VALUE         
py> 


I'm on the fence on this one. I can it's useful; but it's also 
inconsistent with docstrings. And I wonder why other "dead objects" 
aren't also dropped:

py> dis.dis('x = 1; {23: "a", "b": 42}; y = 1')
  1           0 LOAD_CONST               0 (1) 
              3 STORE_NAME               0 (x) 
              6 BUILD_MAP                2 
              9 LOAD_CONST               1 ('a') 
             12 LOAD_CONST               2 (23) 
             15 STORE_MAP            
             16 LOAD_CONST               3 (42) 
             19 LOAD_CONST               4 ('b') 
             22 STORE_MAP            
             23 POP_TOP              
             24 LOAD_CONST               0 (1) 
             27 STORE_NAME               1 (y) 
             30 LOAD_CONST               5 (None) 
             33 RETURN_VALUE         


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#60671

Steven D'Aprano <steve@pearwood.info> writes:

> On Thu, 28 Nov 2013 13:47:22 +1100, Ben Finney wrote:
> > [A triple-quoted string is] not syntactically a comment, and I don't
> > think pretending triple-quoted strings are comments is good
> > practice. If nothing else, you'll need a special case if you want to
> > enclose something with existing triple-quotes.
>
> The CPython core devs disagree with you. Using bare strings for comments 
> is supported by Python, and the compiler strips them out at
> compile-time:

That's not the same as disagreeing with what I said above. Triple-quoted
strings are not syntactically comments, they're still syntactically
treated as string literals. The implementation detail of them being
stripped out doesn't change that.

Also, “is supported by Python” doesn't mean the developers agree with
the practice.

-- 
 \      “Be careless in your dress if you must, but keep a tidy soul.” |
  `\                              —Mark Twain, _Following the Equator_ |
_o__)                                                                  |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#60664

On 11/27/2013 8:57 PM, Victor Hooi wrote:

[sorry if the re-wrapping mis-formats anything]

> I'm running pep8

Ah yes, the module that turns PEP8 into the straightjacket it explicitly 
says it is not meant to be.

> across my code,

We mostly to not change existing stdlib modules to conform unless they 
are being edited anyway for fixes and features.

> and getting warnings about my long lines (> 80 characters).

You are free to ignore such


> I'm wonder what's the recommended way to handle the below cases, and
> fit under 80 characters.
>
> First example - multiple context handlers:
>
> with open(self.full_path, 'r') as input, open(self.output_csv, 'ab')
> as output:
>
> and in my case, with indents, the 80-character marks is just before
> the ending "as output".
>
> What's the standard recognised way to split this across multiple
> lines, so that I'm under 80 characters?

I believe () does not work, so put \ after 'input,'.

> I can't just split after the "as input," as that isn't valid syntax,
> and there's no convenient parentheses for me to split over.
>
> Is there a standard Pythonic way?
>
> Second example - long error messages:
>
> self.logger.error('Unable to open input or output file - %s. Please
> check you have sufficient permissions and the file and parent
> directory exist.' % e)
>
> I can use triple quotes:
>
> self.logger.error( """Unable to open input or output file - %s.
> Please check you have sufficient permissions and the file and parent
> directory exist.""" % e)
>
> However, that will introduce newlines in the message, which I don't
> want.
>
> I can use backslashes:
>
> self.logger.error( 'Unable to open input or output file - %s. Please
> check you\ have sufficient permissions and the file and parent
> directory\ exist.' % e)
>
> which won't introduce newlines.
>
> Or I can put them all as separate strings, and trust Python to glue
> them together:

Such gluing in a documented feature and I think you can trust it to not 
disappear (for at least a decade ;-).

> self.logger.error( 'Unable to open input or output file - %s. Please
> check you' 'have sufficient permissions and the file and parent
> directory' 'exist.' % e)
>
> Which way is the recommended Pythonic way?
>
> Third example - long comments:
>
> """ NB - We can't use Psycopg2's parametised statements here, as that
> automatically wraps everything in single quotes. So
> s3://my_bucket/my_file.csv.gz would become
> s3://'my_bucket'/'my_file.csv.gz'. Hence, we use Python's normal
> string formating - this could potentially exposes us to SQL injection
> attacks via the config.yaml file. I'm not aware of any easy ways
> around this currently though - I'm open to suggestions though. See
> http://stackoverflow.com/questions/9354392/psycopg2-cursor-execute-with-sql-query-parameter-causes-syntax-error
>
>for further information. """
>
> In this case, I'm guessing a using triple quotes (""") is a better
> idea with multi-line comments, right?

Either are ok.

> However, I've noticed that I can't seem to put in line-breaks inside
> the comment without triggering a warning. For example, trying to put
> in another empty line in between lines 6 and 7 above causes a
> warning.

Blank lines are normal in triple-quoted strings and recommended for 
docstrings longer than a line. So I am puzzled as to your problem.

> Also, how would I split up the long URLs?

Don't if you can possibly avoid it.

> Breaking it up makes it annoying to use the URL.

Yep.

> Thoughts?

Some urls, especially long ones, have extra tracking/query junk that can 
be removed while still pointing to the page.


-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#60665

On 11/27/13 8:57 PM, Victor Hooi wrote:
> Hi,
>
> I'm running pep8 across my code, and getting warnings about my long lines (> 80 characters).
>
> I'm wonder what's the recommended way to handle the below cases, and fit under 80 characters.

My recommendations usually amount to: write more statements, each 
shorter than what you have.

> First example - multiple context handlers:
>
>              with open(self.full_path, 'r') as input, open(self.output_csv, 'ab') as output:
>
> and in my case, with indents, the 80-character marks is just before the ending "as output".
>
> What's the standard recognised way to split this across multiple lines, so that I'm under 80 characters?
>
> I can't just split after the "as input," as that isn't valid syntax, and there's no convenient parentheses for me to split over.
>
> Is there a standard Pythonic way?

The important thing in a with statement is that the assigned name will 
be closed (or otherwise exited) automatically.  The open call is just 
the expression used to assign the name.  The expression there isn't 
really important.  This looks odd, but works the same as what you have:

     input = open(self.full_path)
     output = open(self.output_csv, 'ab')
     with input as input, output as output:
         ...

(Use different names for the two parts of the "as" clauses if you like.)

>
> Second example - long error messages:
>
>              self.logger.error('Unable to open input or output file - %s. Please check you have sufficient permissions and the file and parent directory exist.' % e)
>
> I can use triple quotes:
>
>              self.logger.error(
>                  """Unable to open input or output file - %s. Please check you
>                  have sufficient permissions and the file and parent directory
>                  exist.""" % e)
>
> However, that will introduce newlines in the message, which I don't want.
>
> I can use backslashes:
>
>              self.logger.error(
>                  'Unable to open input or output file - %s. Please check you\
>                  have sufficient permissions and the file and parent directory\
>                  exist.' % e)
>
> which won't introduce newlines.
>
> Or I can put them all as separate strings, and trust Python to glue them together:
>
>              self.logger.error(
>                  'Unable to open input or output file - %s. Please check you'
>                  'have sufficient permissions and the file and parent directory'
>                  'exist.' % e)
>
> Which way is the recommended Pythonic way?

Use the separate strings, but don't forget the spaces:

     self.logger.error(
         "Unable to open input or output file - %s. Please check you "
         "have sufficient permissions and the file and parent directory "
         "exist." % e
     )

>
> Third example - long comments:
>
>              """ NB - We can't use Psycopg2's parametised statements here, as
>              that automatically wraps everything in single quotes.
>              So s3://my_bucket/my_file.csv.gz would become s3://'my_bucket'/'my_file.csv.gz'.
>              Hence, we use Python's normal string formating - this could
>              potentially exposes us to SQL injection attacks via the config.yaml
>              file.
>              I'm not aware of any easy ways around this currently though - I'm
>              open to suggestions though.
>              See
>              http://stackoverflow.com/questions/9354392/psycopg2-cursor-execute-with-sql-query-parameter-causes-syntax-error
>              for further information. """
>
> In this case, I'm guessing a using triple quotes (""") is a better idea with multi-line comments, right?

As Ben pointed out, using an actual comment is best.

> However, I've noticed that I can't seem to put in line-breaks inside the comment without triggering a warning. For example, trying to put in another empty line in between lines 6 and 7 above causes a warning.

I don't know what you mean about line-breaks causing warnings.

--Ned.

>
> Also, how would I split up the long URLs? Breaking it up makes it annoying to use the URL. Thoughts?
>
> Cheers,
> Victor
>

[toc] | [prev] | [next] | [standalone]

#60675

On Wed, 27 Nov 2013 21:59:56 -0500, Ned Batchelder wrote:

> The important thing in a with statement is that the assigned name will
> be closed (or otherwise exited) automatically.  The open call is just
> the expression used to assign the name.  The expression there isn't
> really important.  This looks odd, but works the same as what you have:
> 
>      input = open(self.full_path)
>      output = open(self.output_csv, 'ab')
>      with input as input, output as output:
>          ...
> 
> (Use different names for the two parts of the "as" clauses if you like.)


That's really clever! Why didn't I think of that?


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#60688

On 2013-11-28 03:58, Steven D'Aprano wrote:
>>      input = open(self.full_path)
>>      output = open(self.output_csv, 'ab')
>>      with input as input, output as output:
>>          ...
> 
> That's really clever! Why didn't I think of that?

Because if the 2nd output fails, the input doesn't get closed as
it {w,sh}ould in a with-statement? :-)

You could work around this with:

  from functools import partial
  in_ = partial(open, self.full_path)
  out_ = partial(open, self.output_csv, 'ab')
  with in_() as input, out_() as output:
    do_stuff()

There's still room for programmer error if you don't have an
"output_csv" property on self and you get an AttributeError, but
that's more of a "dumb programmer error" rather than an actual
runtime exception.

-tkc

[toc] | [prev] | [next] | [standalone]

#60690

On Fri, Nov 29, 2013 at 1:04 AM, Tim Chase
<python.list@tim.thechases.com> wrote:
> You could work around this with:
>
>   from functools import partial
>   in_ = partial(open, self.full_path)
>   out_ = partial(open, self.output_csv, 'ab')
>   with in_() as input, out_() as output:
>     do_stuff()

Yeah, I think I'd prefer a backslash continuation to that!

ChrisA

[toc] | [prev] | [next] | [standalone]

#60716

On 11/28/13 9:04 AM, Tim Chase wrote:
> On 2013-11-28 03:58, Steven D'Aprano wrote:
>>>       input = open(self.full_path)
>>>       output = open(self.output_csv, 'ab')
>>>       with input as input, output as output:
>>>           ...
>>
>> That's really clever! Why didn't I think of that?
>
> Because if the 2nd output fails, the input doesn't get closed as
> it {w,sh}ould in a with-statement? :-)

D'oh, sorry.  A doubly-nested with statement?

     input = open(self.full_path)
     with input:
         output = open(self.output_csv, 'ab')
         with output:
             ...

Of course, if you were willing to use a doubly-nested with statement, 
then the original open-in-the-with form would have fit in 80 chars... 
Also, I have a sneaking suspicion there's still an error case not 
handled well, someone will point it out if so!

--Ned.

>
> -tkc

[toc] | [prev] | [next] | [standalone]

#60669

Ned Batchelder <ned@nedbatchelder.com> writes:

> The important thing in a with statement is that the assigned name will
> be closed (or otherwise exited) automatically.  The open call is just
> the expression used to assign the name.  The expression there isn't
> really important.  This looks odd, but works the same as what you
> have:
>
>     input = open(self.full_path)
>     output = open(self.output_csv, 'ab')
>     with input as input, output as output:
>         ...

Excellent! That solves for me a significant irritation of the syntax for
multiple context managers. Thank you.

-- 
 \      “The most common way people give up their power is by thinking |
  `\                               they don't have any.” —Alice Walker |
_o__)                                                                  |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#60670

tOn Wed, Nov 27, 2013 at 8:57 PM, Victor Hooi <victorhooi@gmail.com> wrote:
>             with open(self.full_path, 'r') as input, open(self.output_csv, 'ab') as output:

There are two nice and clean solutions for this.

I usually nest them.

with open(self.full_path, 'r') as input:
  with open(self.ouptut_csv, 'ab') as output:

I use a non-standard (shortened) indentation for nested with
statements, to save space. I don't feel bad about this at all; the
with statement's syntax makes me do it!

Otherwise, contextlib.ExitStack is another way to separate them.

with contextlib.ExitStack() as stack:
    input = stack.enter_context(open(self.full_path, 'r'))
    writer = csv.writer(stack.enter_context(open(self.output_csv)))

When working with a csv file I like how it removes the output
temporary file object variable, though if you needed it for some
reason you could keep it.

-- 
Neil Cerutti

[toc] | [prev] | [next] | [standalone]

#60673

On 11/27/2013 06:59 PM, Ned Batchelder wrote:
>
> The important thing in a with statement is that the assigned name will be closed (or otherwise exited) automatically.
> The open call is just the expression used to assign the name.  The expression there isn't really important.  This looks
> odd, but works the same as what you have:
>
>      input = open(self.full_path)
>      output = open(self.output_csv, 'ab')
>      with input as input, output as output:
>          ...
>
> (Use different names for the two parts of the "as" clauses if you like.)

Or skip the `as` clauses all together:

   input = ...
   output = ...
   with input, output:
     ...

works just fine.  (At least on 2.7 where I tested it. ;)

--
~Ethan~

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.python

csiph-web