Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #88022 > unrolled thread

Re: A simple single line, triple-quoted comment is giving syntax error. Why?

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: A simple single line, triple-quoted comment is giving syntax error. Why? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2015-03-26 05:35 +0100
    Re: A simple single line, triple-quoted comment is giving syntax error. Why? Ian Kelly <ian.g.kelly@gmail.com> - 2015-03-25 23:09 -0600
      Re: A simple single line, triple-quoted comment is giving syntax error. Why? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2015-03-26 17:45 +0100
        Re: A simple single line, triple-quoted comment is giving syntax error. Why? Ian Kelly <ian.g.kelly@gmail.com> - 2015-03-26 12:29 -0600
          Re: A simple single line, triple-quoted comment is giving syntax error. Why? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2015-03-26 19:54 +0100
            Re: A simple single line, triple-quoted comment is giving syntax error. Why? Ian Kelly <ian.g.kelly@gmail.com> - 2015-03-26 17:27 -0600
              Re: A simple single line, triple-quoted comment is giving syntax error. Why? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2015-03-28 19:20 +0100
                Re: A simple single line, triple-quoted comment is giving syntax error. Why? Ian Kelly <ian.g.kelly@gmail.com> - 2015-03-29 01:56 -0600
                  Re: A simple single line, triple-quoted comment is giving syntax error. Why? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2015-03-29 12:41 +0200
                    Re: A simple single line, triple-quoted comment is giving syntax error. Why? Ian Kelly <ian.g.kelly@gmail.com> - 2015-03-31 01:23 -0600
                      Re: A simple single line, triple-quoted comment is giving syntax error. Why? sohcahtoa82@gmail.com - 2015-03-31 16:10 -0700
                        Re: A simple single line, triple-quoted comment is giving syntax error. Why? Chris Angelico <rosuav@gmail.com> - 2015-04-01 10:36 +1100
                        Re: A simple single line, triple-quoted comment is giving syntax error. Why? Ian Kelly <ian.g.kelly@gmail.com> - 2015-03-31 17:43 -0600
                      Re: A simple single line, triple-quoted comment is giving syntax error. Why? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2015-04-02 23:31 +0200
                        Re: A simple single line, triple-quoted comment is giving syntax error. Why? sohcahtoa82@gmail.com - 2015-04-02 15:26 -0700
                        Re: A simple single line, triple-quoted comment is giving syntax error. Why? Ian Kelly <ian.g.kelly@gmail.com> - 2015-04-02 18:21 -0600
                        Re: A simple single line, triple-quoted comment is giving syntax error. Why? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-04-03 16:40 +1100
                          Re: A simple single line, triple-quoted comment is giving syntax error. Why? Chris Angelico <rosuav@gmail.com> - 2015-04-03 16:57 +1100
                            Re: A simple single line, triple-quoted comment is giving syntax error. Why? Marko Rauhamaa <marko@pacujo.net> - 2015-04-03 10:13 +0300
                              r"\"" ??? (was A simple single line, triple-quoted comment) Rustom Mody <rustompmody@gmail.com> - 2015-04-03 05:52 -0700
                                Re: r"\"" ??? (was A simple single line, triple-quoted comment) Chris Angelico <rosuav@gmail.com> - 2015-04-04 01:40 +1100
                                  Re: r"\"" ??? (was A simple single line, triple-quoted comment) Rustom Mody <rustompmody@gmail.com> - 2015-04-03 08:14 -0700
                              Re: A simple single line, triple-quoted comment is giving syntax error. Why? Ian Kelly <ian.g.kelly@gmail.com> - 2015-04-03 09:25 -0600
                          Re: A simple single line, triple-quoted comment is giving syntax error. Why? Ian Kelly <ian.g.kelly@gmail.com> - 2015-04-03 00:40 -0600
          Re: A simple single line, triple-quoted comment is giving syntax   error. Why? Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-03-27 11:43 +1300
            Re: A simple single line, triple-quoted comment is giving syntax error. Why? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2015-03-28 19:06 +0100
        Re: A simple single line, triple-quoted comment is giving syntax error. Why? Ian Kelly <ian.g.kelly@gmail.com> - 2015-03-26 12:32 -0600
          Re: A simple single line, triple-quoted comment is giving syntax error. Why? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2015-03-26 19:56 +0100
            Re: A simple single line, triple-quoted comment is giving syntax error. Why? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-27 10:15 +1100
              Re: A simple single line, triple-quoted comment is giving syntax error. Why? Tim Chase <python.list@tim.thechases.com> - 2015-03-26 18:47 -0500
              Re: A simple single line, triple-quoted comment is giving syntax error. Why? Ian Kelly <ian.g.kelly@gmail.com> - 2015-03-26 19:38 -0600
              Re: A simple single line, triple-quoted comment is giving syntax error. Why? Marko Rauhamaa <marko@pacujo.net> - 2015-03-27 07:34 +0200
    Re: A simple single line, triple-quoted comment is giving syntax error. Why? Dave Angel <davea@davea.name> - 2015-03-26 01:23 -0400
      Re: A simple single line, triple-quoted comment is giving syntax error. Why? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2015-03-26 18:25 +0100
        Re: A simple single line, triple-quoted comment is giving syntax error. Why? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2015-03-26 18:30 +0100

Page 2 of 2 — ← Prev page 1 [2]

#88483 — Re: r"\"" ??? (was A simple single line, triple-quoted comment)

On Fri, Apr 3, 2015 at 11:52 PM, Rustom Mody <rustompmody@gmail.com> wrote:
> Speaking about silliness of definitions, I was knocked out in class by this today:
>
>>>> r"\""
> '\\"'
>
> Seeing the docs
> https://docs.python.org/3.4/reference/lexical_analysis.html#string-and-bytes-literals
> it talks of this explicitly
>
> But I still find it strange -- I would have expected this kind of error behavior:
>
>>>> "A string" "another incomplete
> SyntaxError: EOL while scanning string literal
>>>>

I'm sorry, I'm not understanding your confusion here. A raw string
literal allows escaped quote characters - is that the problem?

ChrisA

[toc] | [prev] | [next] | [standalone]

#88484 — Re: r"\"" ??? (was A simple single line, triple-quoted comment)

On Friday, April 3, 2015 at 8:10:54 PM UTC+5:30, Chris Angelico wrote:
> On Fri, Apr 3, 2015 at 11:52 PM, Rustom Mody  wrote:
> > Speaking about silliness of definitions, I was knocked out in class by this today:
> >
> >>>> r"\""
> > '\\"'
> >
> > Seeing the docs
> > https://docs.python.org/3.4/reference/lexical_analysis.html#string-and-bytes-literals
> > it talks of this explicitly
> >
> > But I still find it strange -- I would have expected this kind of error behavior:
> >
> >>>> "A string" "another incomplete
> > SyntaxError: EOL while scanning string literal
> >>>>
> 
> I'm sorry, I'm not understanding your confusion here. A raw string
> literal allows escaped quote characters - is that the problem?
> 
> ChrisA

Yeah...
I guess you could call it that. Raw? half-cooked? Or double-cooked?

# This looks properly raw
>>> r"\a"
'\\a'
>>> 
# This not so surprising
>>> r"\\"
'\\\\'
#  Thats 2 not 4 backslashes  - still ok
>>> len(r"\\")
2
# Until you try this
>>> r"\"
SyntaxError: EOL while scanning string literal
#... which seems to be 'correctable' by
>>> r"\""
'\\"'
>>> 

In short I see some behavior
What's the pattern or intention I dont get.

Does a backslash escape?? Seems to depend on the moonphase¹ <wink>

>>> print ("\n")


>>> print (r"\n")
\n
-------
¹ Worsened by eclipse tomorrow

[toc] | [prev] | [next] | [standalone]

#88486

On Fri, Apr 3, 2015 at 1:13 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
>  2. The counterexample "abc" "def" *does* demonstrate that expressions
>     can at times follow each other immediately. It is a nice point even
>     if not all that consequential.
>
>     Somewhat analogously:
>      * ord  is an expression
>      * ("a")  is an expression
>      * ord("a")  is an expression

I'm tired of this also, so I'll make this response short. By analogy to English:

* "far" is an English adverb.
* "lands" is an English noun.
* "far lands" is an English noun phrase.

* Therefore, a noun phrase in English can consist of an adverb
immediately followed by a noun that is modified by the adverb

The fallacy here is that the "far" in "far lands" is used as an
adjective, not an adverb. I think that the same fallacy applies to the
Python expressions above. If you disagree, that's fine; I'll let it
go.

[toc] | [prev] | [next] | [standalone]

#88473

On Thu, Apr 2, 2015 at 11:40 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> This sterile, pointless arguing about the minutia of pedantic definitions is
> not even close to useful. Honestly Thomas, Ian, nobody cares any more. If I
> were a betting man, I'd bet that neither of you can describe, in one
> sentence, what is the actual disagreement between the two of you, let alone
> why it matters for Python programming.

Oh, either of us *could* describe it in one sentence; it's just that
our descriptions of how we disagree would not agree. ;-)

[toc] | [prev] | [next] | [standalone]

#88105 — Re: A simple single line, triple-quoted comment is giving syntax error. Why?

Ian Kelly wrote:
> What I mean is that if you construct a parse tree of "foo" "bar" using
> that grammar, it looks like this:
> 
>      expr
>        |
>     STRING+
>      /   \
> STRING  STRING

Not quite -- STRING+ is not a symbol in the grammar, it's
a shorthand for a combination of symbols. The parse tree
is actually just

       expr
       /   \
  STRING  STRING

> Not like this:
> 
>     expr
>      |
>    STRING+
>     /  \
>  expr  expr
>   |      |
> STRING  STRING

That would be

 >     expr
 >     /  \
 >  expr  expr
 >   |      |
 > STRING  STRING

Thomas 'PointedEars' Lahn wrote:
 > Prove it.

To get a parse tree like the above, there would need to be
a production like this in the grammar:

    expr ::= expr+

There is no such production, so that parse tree is impossible.
QED.

What you seem to be doing in your mind is taking this
production:

    expr ::= STRING

and using it "backwards" to justify expanding any occurrence
of STRING into expr. But grammars don't work that way. You're
committing a logical fallacy: just because some exprs are
STRINGs doesn't mean that all STRINGs are exprs.

-- 
Greg

[toc] | [prev] | [next] | [standalone]

#88232

Gregory Ewing wrote:

> Ian Kelly wrote:
>> What I mean is that if you construct a parse tree of "foo" "bar" using
>> that grammar, it looks like this:
>> 
>>      expr
>>        |
>>     STRING+
>>      /   \
>> STRING  STRING
> 
> Not quite -- STRING+ is not a symbol in the grammar, it's
> a shorthand for a combination of symbols. The parse tree
> is actually just
> 
>        expr
>        /   \
>   STRING  STRING

Or if there is a single STRING,

  expr
   |
  STRING
 
AISB.

>> Not like this:
>> 
>>     expr
>>      |
>>    STRING+
>>     /  \
>>  expr  expr
>>   |      |
>> STRING  STRING
> […]
> 
> To get a parse tree like the above, there would need to be
> a production like this in the grammar:
> 
>     expr ::= expr+
> 
> There is no such production, so that parse tree is impossible.
> QED.

Correct.

> What you seem to be doing in your mind

Who is “you” here?  You appear to be confused, probably supported by your 
unconventional quotation style, who made which statement.

> is taking this production:
> 
>     expr ::= STRING
> 
> and using it "backwards" to justify expanding any occurrence
> of STRING into expr.

No.

> But grammars don't work that way. You're committing a logical fallacy:
> just because some exprs are STRINGs doesn't mean that all STRINGs are 
> exprs.

The fallacy – a straw man – is yours here.  Nobody – in particular not me – 
claimed that “STRING” can *only* be produced by “expr” in the first place.

Production rules in a formal grammar of the form

  expr           → … | any_goal | …
  any_goal       → … | any_other_goal | …
  any_other_goal → … | STRING+ | …

where the steps in-between are optional, mean *exactly* that (*all*) 
“STRING”s are “expr”s – but that not all “expr”s are “STRING”s.  Because it 
follows from the rules above that these production chains – *but not _only_ 
these* – are possible:

  a) expr ⇒ … ⇒ STRING
  b) expr ⇒ … ⇒ STRING STRING
  c) expr ⇒ … ⇒ STRING STRING STRING

and so on.  The truth of the statement “(All) STRING( literal)s are 
expr(ession)s” is what follows from the possibility of the first production 
chain.  (That does not preclude the possibility that “STRING”s are also an 
element of another set.)

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.

[toc] | [prev] | [next] | [standalone]

#88083

On Thu, Mar 26, 2015 at 12:29 PM, Ian Kelly <ian.g.kelly@gmail.com> wrote:
> On Thu, Mar 26, 2015 at 10:45 AM, Thomas 'PointedEars' Lahn
>> No, in the used flavour of EBNF the unquoted “+” following a goal symbol
>> clearly means the occurrence of *at least one* of the immediately preceding
>> symbol, meaning either one *or more than one*.
>
> It means one or more *tokens*, not one or more literals.

Although reading the documentation, it seems that it also conflates
string literals with tokens, so on that I'll have to concede the
point.

https://docs.python.org/3.4/reference/lexical_analysis.html#string-and-bytes-literals

[toc] | [prev] | [next] | [standalone]

#88085

Ian Kelly wrote:

> On Thu, Mar 26, 2015 at 12:29 PM, Ian Kelly <ian.g.kelly@gmail.com> wrote:
>> On Thu, Mar 26, 2015 at 10:45 AM, Thomas 'PointedEars' Lahn
>>> No, in the used flavour of EBNF the unquoted “+” following a goal symbol
>>> clearly means the occurrence of *at least one* of the immediately
>>> preceding symbol, meaning either one *or more than one*.
>>
>> It means one or more *tokens*, not one or more literals.
> 
> Although reading the documentation, it seems that it also conflates
> string literals with tokens,

There is nothing to conflate here.  String literals *are* tokens.

> so on that I'll have to concede the point.

Too late, the rebuttal is already underway :-p

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.

[toc] | [prev] | [next] | [standalone]

#88107

On Fri, 27 Mar 2015 05:56 am, Thomas 'PointedEars' Lahn wrote:
[snip argument]


Hey guys, I'm trying to follow the argument but I must admit you've
completely lost me with the angels-on-the-head-of-a-pin nitpicking about
EBNF. I love a good pedant-brawl as much as you, so let me see if I've got
this straight, correct me if I'm wrong.

You're arguing whether or not in the following line of code:

spam = "abcd" "efgh"
# implicitly concatenated to "abcdefgh" at compile time

the right hand side pair of strings counts as a single token or two? Am I
right, or am I missing something?

If that's all it is, why don't you just run the tokenizer over it and see
what it says?

py> from cStringIO import StringIO
py> code = StringIO('spam = "abcd" "efgh"\n')
py> import tokenize
py> for item in tokenize.generate_tokens(code.readline):
...     print item
...
(1, 'spam', (1, 0), (1, 4), 'spam = "abcd" "efgh"\n')
(51, '=', (1, 5), (1, 6), 'spam = "abcd" "efgh"\n')
(3, '"abcd"', (1, 7), (1, 13), 'spam = "abcd" "efgh"\n')
(3, '"efgh"', (1, 14), (1, 20), 'spam = "abcd" "efgh"\n')
(4, '\n', (1, 20), (1, 21), 'spam = "abcd" "efgh"\n')
(0, '', (2, 0), (2, 0), '')


Looks to me that the two string literals each get their own token, and are
concatenated at a later stage of compilation, not during parsing.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#88110

On 2015-03-27 10:15, Steven D'Aprano wrote:
> If that's all it is, why don't you just run the tokenizer over it
> and see what it says?
> 
> py> from cStringIO import StringIO
> py> code = StringIO('spam = "abcd" "efgh"\n')
> py> import tokenize
> py> for item in tokenize.generate_tokens(code.readline):
> ...     print item
> ...
> (1, 'spam', (1, 0), (1, 4), 'spam = "abcd" "efgh"\n')
> (51, '=', (1, 5), (1, 6), 'spam = "abcd" "efgh"\n')
> (3, '"abcd"', (1, 7), (1, 13), 'spam = "abcd" "efgh"\n')
> (3, '"efgh"', (1, 14), (1, 20), 'spam = "abcd" "efgh"\n')
> (4, '\n', (1, 20), (1, 21), 'spam = "abcd" "efgh"\n')
> (0, '', (2, 0), (2, 0), '')
> 
> 
> Looks to me that the two string literals each get their own token,

Nice.  I haven't played with the tokenize module before, but
resolving arguments on comp.lang.python is one of the best possible
uses.

It was interesting to try other feeders to generate_tokens(), my favorite being

>>> import tokenize
>>> i = iter(["spam = 'abc' 'def'"])
>>> for item in tokenize.generate_tokens(lambda: next(i)):
...     print(item)
... 
TokenInfo(type=1 (NAME), string='spam', start=(1, 0), end=(1, 4), line="spam = 'abc' 'def'")
TokenInfo(type=52 (OP), string='=', start=(1, 5), end=(1, 6), line="spam = 'abc' 'def'")
TokenInfo(type=3 (STRING), string="'abc'", start=(1, 7), end=(1, 12), line="spam = 'abc' 'def'")
TokenInfo(type=3 (STRING), string="'def'", start=(1, 13), end=(1, 18), line="spam = 'abc' 'def'")
TokenInfo(type=0 (ENDMARKER), string='', start=(2, 0), end=(2, 0), line='')


It's also nice to have the translation from token-type to token-type-name in Py3

-tkc

[toc] | [prev] | [next] | [standalone]

#88118

On Thu, Mar 26, 2015 at 5:15 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> Looks to me that the two string literals each get their own token, and are
> concatenated at a later stage of compilation, not during parsing.

Thanks. The dispute was about expressions, though. I think we're all
in agreement that there are two tokens.

[toc] | [prev] | [next] | [standalone]

#88134

Steven D'Aprano <steve+comp.lang.python@pearwood.info>:

> You're arguing whether or not in the following line of code:
>
> spam = "abcd" "efgh"
> # implicitly concatenated to "abcdefgh" at compile time
>
> the right hand side pair of strings counts as a single token or two? Am I
> right, or am I missing something?
>
> If that's all it is, why don't you just run the tokenizer over it and
> see what it says?

Now, someone *could* write a tokenizer that took care of string
concatenation on the spot--as long as it dealt with comments as well:

   ("abc"
   # hello
    "def")

It would be even possible to write a parser that didn't have a separate
lexical analyzer at all.

Arguing about terminology is pretty useless. Both sides in this fight
are correct, namely:

   * string literal concatenation is part of expression syntax

   * what goes on inside an atom stays inside an atom

For example, this expression is illegal:

   "abc" ("def")


Marko

[toc] | [prev] | [next] | [standalone]

#88027

On 03/26/2015 01:09 AM, Ian Kelly wrote:
 >
 >
 >>>>> 
https://docs.python.org/3/reference/lexical_analysis.html#string-literal-concatenation
>
> What the grammar that you quoted from shows is that STRING+ is an
> expression. The individual STRINGs of a STRING+ are not expressions,
> except to the extent that they can be parsed in isolation as a
> STRING+. By the same token, a STRING+ is a single string literal, not
> an aggregate of several.
>

That's the way I also read the BNF.  But something I cannot find in that 
chapter of the reference is the definition of STRING+

Naturally searching for 'string' finds way too many spurious refs.



-- 
DaveA

[toc] | [prev] | [next] | [standalone]

#88080

Dave Angel wrote:

[Fixed quotation]

> On 03/26/2015 01:09 AM, Ian Kelly wrote:
>>>>> Thomas 'PointedEars' Lahn wrote:
>>>>>> <https://docs.python.org/3/reference/lexical_analysis.html#string->>>>>> literal-concatenation>
>>
>> What the grammar that you quoted from shows is that STRING+ is an
>> expression. The individual STRINGs of a STRING+ are not expressions,
>> except to the extent that they can be parsed in isolation as a
>> STRING+. By the same token, a STRING+ is a single string literal, not
>> an aggregate of several.
> 
> That's the way I also read the BNF.

Then I am afraid you need to refresh your knowledge of formal grammars.

> But something I cannot find in that chapter of the reference is the
> definition of STRING+

You *definitely* need to refresh your knowledge of formal grammars.

“STRING+” in this flavor of _E_BNF is – rather obviously – equivalent to

  <multiple-string> ::= <STRING> <STRING>*
  <STRING>          ::= '"' <no-unescaped-doublequote>* '"'
                      | "'" <no-unescaped-singlequote>* "'"
                      | '"""' <no-triple-doublequote>* '"""'
                      | "'''" <no-triple-singlequote>* "'''"

in BNF and

  multiple-string = STRING *STRING
  STRING          = '"' *no-unescaped-doublequote '"'
                  / "'" *no-unescaped-singlequote '"'
                  / '"""' *no-unescaped-triple-doublequote '"""'
                  / "'''" *no-unescaped-triple-singlequote "'''"

in ABNF.  I suspect that in this flavor of EBNF the definition of STRING 
looks similar to the following:

  STRING: ('"' no_unescaped_doublequote* '"'
         | "'" no_unescaped_singlequote* "'"
         | '"""' no_unescaped_triple_doublequote* '"""'
         | "'''" no_unescaped_triple_singlequote* "'''")

Definition of the still undefined goal symbols is left as an exercise to the 
reader.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.

[toc] | [prev] | [next] | [standalone]

#88081

Thomas 'PointedEars' Lahn wrote:

>   multiple-string = STRING *STRING
> […]
> in ABNF.

JFTR: ABNF allows for

  multiple-string = 1*STRING

to be equivalent to the above.

<http://en.wikipedia.org/wiki/Augmented_Backus%E2%80%93Naur_Form> pp.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.python

csiph-web