Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From: Ian Kelly <ian.g.kelly@gmail.com>
Newsgroups: comp.lang.python
Subject: Re: What is a function parameter =[] for?
Date: Tue, 24 Nov 2015 11:04:05 -0700
Lines: 73
Message-ID: <mailman.22.1448388286.20593.python-list@python.org>
References: <d9045bdd-a5d4-4c5c-9fd3-3d54f6b50862@googlegroups.com> <564df258$0$1604$c3e8da3$5496439d@news.astraweb.com> <n2l0s3$5l6$1@dont-email.me> <mailman.473.1447955117.16136.python-list@python.org> <n2l3ne$hbp$1@dont-email.me> <mailman.480.1447960748.16136.python-list@python.org> <n2leck$urt$1@dont-email.me> <mailman.495.1447973754.16136.python-list@python.org> <n2lobe$7gs$1@dont-email.me> <8601c9af-a7d9-4642-ba1c-8edd1e4c3390@googlegroups.com> <56546985.8060704@rece.vub.ac.be> <CAPTjJmomomPO6ujJjZtkSXM6Eeqo8_c7i-bpad24C2zveVU0kg@mail.gmail.com> <56547337.4000709@rece.vub.ac.be> <CAPTjJmoGt3B_zTVd8dVLHNeanX=4g1yhN-PLpXzZoYiKb9HYuw@mail.gmail.com> <56547C53.8030407@rece.vub.ac.be> <CAPTjJmqAMwnxBZ97umsNg255v47dsQD6qEr0ysECtGiZ409k0Q@mail.gmail.com> <5654864E.2070906@rece.vub.ac.be> <CAPTjJmoUtwHzOEwcObh5hzUDbz-713kVE_Kq-rg_bmYFHE_tYw@mail.gmail.com> <56549324.8020607@rece.vub.ac.be> <CALwzidmAKgmxjR=nDU4bPn5iy9AqCCiT=nTQYSORADCtpdM+SA@mail.gmail.com> <56549F15.40206@rece.vub.ac.be> <CALwzidkuiSfyXxiQ8H6YSEtEL5cqj6sOUPXdUru6n8XtGnvzug@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
In-Reply-To: <CALwzidkuiSfyXxiQ8H6YSEtEL5cqj6sOUPXdUru6n8XtGnvzug@mail.gmail.com>
Precedence: list
Xref: csiph.com comp.lang.python:99379

On Tue, Nov 24, 2015 at 10:53 AM, Ian Kelly <ian.g.kelly@gmail.com> wrote:
> On Tue, Nov 24, 2015 at 10:32 AM, Antoon Pardon
> <antoon.pardon@rece.vub.ac.be> wrote:
>> Op 24-11-15 om 17:56 schreef Ian Kelly:
>>
>>>
>>>> So on what grounds would you argue that () is not a literal.
>>>
>>> This enumerates exactly what literals are in Python:
>>>
>>> https://docs.python.org/3/reference/lexical_analysis.html#literals
>>>
>>> I think it's a rather pedantic point, though. How are nuances of the
>>> grammar at all related to user expectations?
>>>
>>
>> I think that enumaration is too limited. The section starts with:
>>
>>    Literals are notations for constant values of some built-in types.
>>
>> () satisfies that definition, which is confirmed by the byte code
>> produced for it.
>
> Literals are a type of lexical token. All of the literals shown in
> that section are, indeed, tokens. Now I would point you to the grammar
> specification:
>
> https://docs.python.org/3/reference/grammar.html
>
> And specifically the "atom" rule, which defines both list displays and
> list comprehensions (as well as literals) as being atoms.
> Specifically, it parses () as the token '(', followed by an optional
> yield_expr or testlist_comp, followed by the token ')'. In no way is
> that a single token, nor therefore a literal.

In case reading the grammar doesn't convince, we can also get this
result from playing with the language:

>>> import tokenize
>>> from io import StringIO
>>> list(tokenize.generate_tokens(StringIO('()').readline))
[TokenInfo(type=52 (OP), string='(', start=(1, 0), end=(1, 1),
line='()'), TokenInfo(type=52 (OP), string=')', start=(1, 1), end=(1,
2), line='()'), TokenInfo(type=0 (ENDMARKER), string='', start=(2, 0),
end=(2, 0), line='')]

Two separate tokens of the OP type.

>>> list(tokenize.generate_tokens(StringIO('42').readline))
[TokenInfo(type=2 (NUMBER), string='42', start=(1, 0), end=(1, 2),
line='42'), TokenInfo(type=0 (ENDMARKER), string='', start=(2, 0),
end=(2, 0), line='')]

One token, of a literal type.

>>> list(tokenize.generate_tokens(StringIO('(1,2,3)').readline))
[TokenInfo(type=52 (OP), string='(', start=(1, 0), end=(1, 1),
line='(1,2,3)'), TokenInfo(type=2 (NUMBER), string='1', start=(1, 1),
end=(1, 2), line='(1,2,3)'), TokenInfo(type=52 (OP), string=',',
start=(1, 2), end=(1, 3), line='(1,2,3)'), TokenInfo(type=2 (NUMBER),
string='2', start=(1, 3), end=(1, 4), line='(1,2,3)'),
TokenInfo(type=52 (OP), string=',', start=(1, 4), end=(1, 5),
line='(1,2,3)'), TokenInfo(type=2 (NUMBER), string='3', start=(1, 5),
end=(1, 6), line='(1,2,3)'), TokenInfo(type=52 (OP), string=')',
start=(1, 6), end=(1, 7), line='(1,2,3)'), TokenInfo(type=0
(ENDMARKER), string='', start=(2, 0), end=(2, 0), line='')]

Two tokens for the parentheses, plus three literal tokens for the
ints, plus two more tokens for the separating commas. Definitely not a
literal.

Credit to Steven for using this approach to make a similar point about
literals in a previous thread.