Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #105799 > unrolled thread

Re: How to make Python interpreter a little more strict?

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: How to make Python interpreter a little more strict? John Pote <johnhpote@o2.co.uk> - 2016-03-26 23:30 +0000
    Re: How to make Python interpreter a little more strict? BartC <bc@freeuk.com> - 2016-03-27 00:36 +0000
    Re: How to make Python interpreter a little more strict? Steven D'Aprano <steve@pearwood.info> - 2016-03-27 14:28 +1100
      Re: How to make Python interpreter a little more strict? Tim Chase <python.list@tim.thechases.com> - 2016-03-27 15:32 -0500
        Re: How to make Python interpreter a little more strict? BartC <bc@freeuk.com> - 2016-03-27 21:49 +0100
          Re: How to make Python interpreter a little more strict? Chris Angelico <rosuav@gmail.com> - 2016-03-28 07:59 +1100
          Re: How to make Python interpreter a little more strict? Steven D'Aprano <steve@pearwood.info> - 2016-03-28 12:24 +1100
    Re: How to make Python interpreter a little more strict? Nobody <nobody@nowhere.invalid> - 2016-03-28 00:26 +0100

#105799 — Re: How to make Python interpreter a little more strict?

On 26/03/2016 12:05, Chris Angelico wrote:
> On Fri, Mar 25, 2016 at 11:06 PM, Aleksander Alekseev <afiskon@devzen.ru> wrote:
>> Recently I spend half an hour looking for a bug in code like this:
>>
>> eax@fujitsu:~/temp$ cat ./t.py
>> #!/usr/bin/env python3
>>
>> for x in range(0,5):
>>      if x % 2 == 0:
>>          next
>>      print(str(x))
>>
>> eax@fujitsu:~/temp$ ./t.py
>> 0
>> 1
>> 2
>> 3
>> 4
>>
>> Is it possible to make python complain in this case? Or maybe solve
>> such an issue somehow else?
> I think what you're looking for here is an acknowledgement that
> evaluating the name "next" accomplishes nothing. That's not really
> something the Python interpreter should be looking at (hey, you might
> have good reason for doing that), but there are linters that can
> detect this kind of dead code. Some of them tie into programmer's
> editors, so you could get a nice little warning message right in the
> window where you're typing your code. Look into some of the top-end
> editors (free or commercial) and see what you think of them - they can
> save you no end of time.
>
> ChrisA
So intrigued by this question I tried the following
def fnc( n ):
     print "fnc called with parameter '%d'" % n
     return n

for i in range(0,5):
     if i%2 == 0:
         fnc
         next
     print i

and got the same result as the OP

D:\projects\python
 >>python next.py
0
1
2
3
4
D:\projects\python
 >>

A couple of tests showed that the only important thing about the name in 
the if clause is that it is known at runtime and then it is silently 
ignored.
However, if the name is not known/accessible at run time a 'NameError' 
is raised,
NameError: name 'mynewfn123' is not defined

On the other hand the following if clause
     if i%2 == 0:
         fnc    next

results in a compiler error,
D:\projects\python
 >>python next.py
   File "next.py", line 9
     fnc next
            ^
SyntaxError: invalid syntax

This is all for Python 2.7.9. (Don't know about Python 3....).

So I have sympathy with the OP, I would expect the compiler to pick this 
up - indeed it does so for two (or more ?) unused names on a single 
line. That is unless someone can give a useful use of this behaviour or 
is there something going on under the Python hood I'm not aware of?

It would be all to easy to write a series of lines just calling 
functions and forget the () on one of them. Not fun programming.
It's also a good reminder that the meaning of a keyword in language A is 
not necessarily the same in language B (ie 'next', Python)
So on this last point is this behaviour of Python defined somewhere in 
the docs?

Regards all,
John

[toc] | [next] | [standalone]

#105800

On 26/03/2016 23:30, John Pote wrote:

> So intrigued by this question I tried the following
> def fnc( n ):
>      print "fnc called with parameter '%d'" % n
>      return n
>
> for i in range(0,5):
>      if i%2 == 0:
>          fnc
>          next
>      print i
>
> and got the same result as the OP

> A couple of tests showed that the only important thing about the name in
> the if clause is that it is known at runtime and then it is silently
> ignored.
> However, if the name is not known/accessible at run time a 'NameError'
> is raised,
> NameError: name 'mynewfn123' is not defined
>
> On the other hand the following if clause
>      if i%2 == 0:
>          fnc    next

The results aren't surprising once you know what's going on.

If you have:

    fnc

then this name is evaluated (it refers to a function object given your 
definition. Then that value is discarded. But this:

    kljgkjhgjk

is not defined anywhere, and it can't evaluate it, raising the name 
error. While in:

    fnc()

the 'fnc' is evaluated, and then it's called (when it should give a 
parameter error as your def expects one). But

    abc def

is just a syntax error as usually two identifiers can't be adjacent 
(AFAIK) except where the first is a reserved word.

> So I have sympathy with the OP, I would expect the compiler to pick this
> up - indeed it does so for two (or more ?) unused names on a single
> line. That is unless someone can give a useful use of this behaviour or
> is there something going on under the Python hood I'm not aware of?

This one of those features I think that do have the odd use but most of 
the time just result in perplexing results or hard-to-find bugs. It 
could have been eliminated from the language (especially as many people 
aren't even aware of the possibilities) with little loss.

Someone who desperately wants to evaluate a name or expression can 
always use something like:

  def test(x):pass

  test(fnc)

for the same purpose. But this time it's obvious. (Ideally it would be a 
built-in construct to avoid the call overhead.)


-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#105807

On Sun, 27 Mar 2016 10:30 am, John Pote wrote:

> So intrigued by this question I tried the following
> def fnc( n ):
>      print "fnc called with parameter '%d'" % n
>      return n
> 
> for i in range(0,5):
>      if i%2 == 0:
>          fnc
>          next
>      print i
> 
> and got the same result as the OP

In this case, the two lines "fnc" and "next" simply look up the function
names, but without actually calling them. They're not quite "no-ops", since
they can fail and raise NameError if the name doesn't exist, but otherwise
they might as well be no-ops.


> A couple of tests showed that the only important thing about the name in
> the if clause is that it is known at runtime and then it is silently
> ignored.

Right. What actually happens is that Python evaluates the expression,
generates the result, and then if that result isn't used, a microsecond
later the garbage collector deletes it. In this case, the expression
consists of only a single name: "fnc", or "next".

If you have a more complex expression, Python can do significant work before
throwing it away:

[x**3 for x in range(10000)]


generates a list of cubed numbers [0, 1, 8, 27, ...]. Then the garbage
collector sees that nothing refers to that list, and it is available to be
deleted, so it deletes it.

Why does Python bother generating the list only to throw it away a
microsecond later? Because the interpreter can't easily tell if the
calculations will have any side-effects. It might turn out that something
in the expression ends up setting a global variable, or printing output, or
writing to a file, or who knows what?

Now, you and I can read the line and see that (assuming range hasn't been
overridden) there are no side-effects from calculating cubes of numbers.
But the Python interpreter is very simple-minded and not as smart as you or
I, so it can't tell, and so it plays it safe and does the calculations just
in case. Future versions of the interpreter may be smarter.

In cases where Python *can* tell that there are no side-effects, it may
ignore stand-alone expressions that don't do anything useful.


> However, if the name is not known/accessible at run time a 'NameError'
> is raised,
> NameError: name 'mynewfn123' is not defined

Correct.



> On the other hand the following if clause
>      if i%2 == 0:
>          fnc    next
> 
> results in a compiler error,
> D:\projects\python
>  >>python next.py
>    File "next.py", line 9
>      fnc next
>             ^
> SyntaxError: invalid syntax
> 
> This is all for Python 2.7.9. (Don't know about Python 3....).

Python 3 will be more-or-less the same. Python 3 might be a bit smarter
about recognising expressions that have no side-effects and that can be
ignored, but not much more.


> So I have sympathy with the OP, I would expect the compiler to pick this
> up - indeed it does so for two (or more ?) unused names on a single
> line. That is unless someone can give a useful use of this behaviour or
> is there something going on under the Python hood I'm not aware of?

In Python, variables AND FUNCTIONS are created dynamically, and can be
deleted or created as needed. So Python doesn't know if your function "fnc"
actually exists or not until runtime. Just because you defined it using def
doesn't mean that it will still be around later: maybe you have called:

del fnc

or possibly reassigned the variable:

fnc = "Surprise! Not a function any more!"

So the compiler can't tell whether 

    fnc

is a legal line or not. Maybe fnc exists, maybe it doesn't, it will have to
actually evaluate the expression to find out, and that happens at runtime.

But the compiler can recognise syntax errors:

    fnc next

is illegal syntax, as are:

    1234fnc
    fnc?
    x = ) (

and for those, you will get an immediate SyntaxError before the code starts
running.


> It would be all to easy to write a series of lines just calling
> functions and forget the () on one of them. Not fun programming.

In theory, you are correct. But in practice, it's not really as big a
problem as you might think. Very, very few functions take no arguments.
There is this one:

import random
r = random.random  # oops, forgot to call the function

for example, but most functions take at least one argument:

r = random.randint(1, 6)

There are a few functions or methods that you call for their side-effects,
like Pascal "procedures" or C "void functions":

mylist.sort()

and yes, it is annoying when you forget the brackets (round brackets or
parentheses for any Americans reading) and the list doesn't sort, but
that's the exception rather than the rule. And if you're worried about
that, you can run a "linter" which check your source code for such
potential problems.

Google for PyLint, PyFlakes, PyChecker, Jedi, etc if you want more
information about linters, or just ask here. I can't tell you too much
about them, as I don't use them, but somebody will probably answer.





-- 
Steven

[toc] | [prev] | [next] | [standalone]

#105872

On 2016-03-27 14:28, Steven D'Aprano wrote:
> > So intrigued by this question I tried the following
> > def fnc( n ):
> >      print "fnc called with parameter '%d'" % n
> >      return n
> > 
> > for i in range(0,5):
> >      if i%2 == 0:
> >          fnc
> >          next
> >      print i
> > 
> > and got the same result as the OP  
> 
> In this case, the two lines "fnc" and "next" simply look up the
> function names, but without actually calling them. They're not
> quite "no-ops", since they can fail and raise NameError if the name
> doesn't exist, but otherwise they might as well be no-ops.

Which is actually useful.  I've got some 2.4 code that reads

  try:
    any
  except NameError:
    def any(...):
      ...

(with a similar block for all() )

I don't want to call any() or all(), I simply want to test whether
they exist.

-tkc

[toc] | [prev] | [next] | [standalone]

#105873

On 27/03/2016 21:32, Tim Chase wrote:
> On 2016-03-27 14:28, Steven D'Aprano wrote:

>> In this case, the two lines "fnc" and "next" simply look up the
>> function names, but without actually calling them. They're not
>> quite "no-ops", since they can fail and raise NameError if the name
>> doesn't exist, but otherwise they might as well be no-ops.
>
> Which is actually useful.  I've got some 2.4 code that reads
>
>    try:
>      any
>    except NameError:
>      def any(...):
>        ...
>
> (with a similar block for all() )
>
> I don't want to call any() or all(), I simply want to test whether
> they exist.

But would it have been much of an imposition to have typed:

     try:
       test = any
     except NameError:
       def any(...):
         ...

? (Or any of the half dozen ways there must be to test the existence of 
a name.)

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#105875

On Mon, Mar 28, 2016 at 7:49 AM, BartC <bc@freeuk.com> wrote:
> On 27/03/2016 21:32, Tim Chase wrote:
>>
>> On 2016-03-27 14:28, Steven D'Aprano wrote:
>
>
>>> In this case, the two lines "fnc" and "next" simply look up the
>>> function names, but without actually calling them. They're not
>>> quite "no-ops", since they can fail and raise NameError if the name
>>> doesn't exist, but otherwise they might as well be no-ops.
>>
>>
>> Which is actually useful.  I've got some 2.4 code that reads
>>
>>    try:
>>      any
>>    except NameError:
>>      def any(...):
>>        ...
>>
>> (with a similar block for all() )
>>
>> I don't want to call any() or all(), I simply want to test whether
>> they exist.
>
>
> But would it have been much of an imposition to have typed:
>
>     try:
>       test = any
>     except NameError:
>       def any(...):
>         ...
>
> ? (Or any of the half dozen ways there must be to test the existence of a
> name.)

It shifts the uselessness to "why did you assign to that name?". While
that's not too bad (and nothing like as bad as a dummy function call),
it's generally reserved for the places where Python syntax mandates an
assignment:

for _ in range(4): next(iter) # discard the headers

If you're building a linter, I would expect "name assigned to and
never used" to be its own warning; but also, the try/except block
hints that this isn't useless. Generally, "dummy expressions" like
this are going to explicitly catch at least one exception that can be
raised only in that expression (ie a really small try block). That
pretty much counts as your language-level marker for conscious dummy
expression usage.

ChrisA

[toc] | [prev] | [next] | [standalone]

#105887

On Mon, 28 Mar 2016 07:49 am, BartC wrote:

> On 27/03/2016 21:32, Tim Chase wrote:
>> On 2016-03-27 14:28, Steven D'Aprano wrote:
> 
>>> In this case, the two lines "fnc" and "next" simply look up the
>>> function names, but without actually calling them. They're not
>>> quite "no-ops", since they can fail and raise NameError if the name
>>> doesn't exist, but otherwise they might as well be no-ops.
>>
>> Which is actually useful.  I've got some 2.4 code that reads
>>
>>    try:
>>      any
>>    except NameError:
>>      def any(...):
>>        ...
>>
>> (with a similar block for all() )
>>
>> I don't want to call any() or all(), I simply want to test whether
>> they exist.
> 
> But would it have been much of an imposition to have typed:
> 
>      try:
>        test = any
>      except NameError:
>        def any(...):
>          ...
> 
> ? (Or any of the half dozen ways there must be to test the existence of
> a name.)

No, not much of an imposition. But then you might have your linter (if you
bother with one) complaining that you define "test" but never use it.

But regardless of whether it is an imposition or not, that's not how we do
it. It's a stylistic choice, that's all. We've explained the reasons for
it, but it ultimately comes down to a matter of taste.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#105880

On Sat, 26 Mar 2016 23:30:30 +0000, John Pote wrote:

> So I have sympathy with the OP, I would expect the compiler to pick this
> up

Why? The code is valid, the compiler knows how to generate the
appropriate bytecode for it.

The compiler isn't "lint".

Reporting code which is actually invalid is fairly straightforward.
When the parser attempts to match the next token against a parse rule
and finds that nothing matches (e.g. the "fnc next" example), it just
needs to raise a SyntaxError. The point at which the exception needs to be
raised naturally exists in the code.

But to identify code which is perfectly valid yet is "probably" a mistake
first requires someone to identify such cases, then someone needs to start
adding the appropriate tests to the compiler to distinguish such code from
the rest.

> It would be all to easy to write a series of lines just calling
> functions and forget the () on one of them. Not fun programming. It's
> also a good reminder that the meaning of a keyword in language A is not
> necessarily the same in language B (ie 'next', Python)

"next" isn't a keyword, it's a built-in function. It's perfectly valid to
re-use that name for your own variables or functions.

> So on this last point is this behaviour of Python defined somewhere in
> the docs? 

What behaviour? Evaluating a name?

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web