Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #98341 > unrolled thread

Question about math.pi is mutable

Back to article view | Back to comp.lang.python

Contents

  Question about math.pi is mutable "wangq@travelsky.com" <wangq@travelsky.com> - 2015-11-06 10:33 +0800
    Re: Question about math.pi is mutable Bartc <bc@freeuk.com> - 2015-11-06 12:30 +0000
      Re: Question about math.pi is mutable Lorenzo Sutton <lorenzofsutton@gmail.com> - 2015-11-06 17:12 +0100
        Re: Question about math.pi is mutable Bartc <bc@freeuk.com> - 2015-11-06 19:30 +0000
          Re: Question about math.pi is mutable Chris Angelico <rosuav@gmail.com> - 2015-11-07 06:40 +1100
            Re: Question about math.pi is mutable Christian Gollwitzer <auriocus@gmx.de> - 2015-11-06 20:59 +0100
            Re: Question about math.pi is mutable Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2015-11-06 23:19 +0100
              Re: Question about math.pi is mutable Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2015-11-07 00:48 +0100
              Re: Question about math.pi is mutable Steven D'Aprano <steve@pearwood.info> - 2015-11-07 13:00 +1100
          Re: Question about math.pi is mutable Ben Finney <ben+python@benfinney.id.au> - 2015-11-07 14:43 +1100
            Re: Question about math.pi is mutable Bartc <bc@freeuk.com> - 2015-11-07 11:23 +0000
              Re: Question about math.pi is mutable Ben Finney <ben+python@benfinney.id.au> - 2015-11-07 22:35 +1100
                Re: Question about math.pi is mutable Bartc <bc@freeuk.com> - 2015-11-07 11:57 +0000
                  Re: Question about math.pi is mutable Ben Finney <ben+python@benfinney.id.au> - 2015-11-07 23:15 +1100
                    Re: Question about math.pi is mutable Bartc <bc@freeuk.com> - 2015-11-07 13:00 +0000
                      Re: Question about math.pi is mutable Laura Creighton <lac@openend.se> - 2015-11-07 15:09 +0100
                      Re: Question about math.pi is mutable Marko Rauhamaa <marko@pacujo.net> - 2015-11-07 16:23 +0200
                        Re: Question about math.pi is mutable Marko Rauhamaa <marko@pacujo.net> - 2015-11-07 16:28 +0200
                          Re: Question about math.pi is mutable Bartc <bc@freeuk.com> - 2015-11-07 15:01 +0000
                            Re: Question about math.pi is mutable Marko Rauhamaa <marko@pacujo.net> - 2015-11-07 19:46 +0200
                              Re: Question about math.pi is mutable Steven D'Aprano <steve@pearwood.info> - 2015-11-08 14:07 +1100
                                Re: Question about math.pi is mutable Random832 <random832@fastmail.com> - 2015-11-08 01:29 -0500
                            Re: Question about math.pi is mutable Steven D'Aprano <steve@pearwood.info> - 2015-11-08 13:59 +1100
                              Re: Question about math.pi is mutable Bartc <bc@freeuk.com> - 2015-11-08 10:40 +0000
                                Re: Question about math.pi is mutable Marko Rauhamaa <marko@pacujo.net> - 2015-11-08 13:02 +0200
                                  Re: Question about math.pi is mutable BartC <bc@freeuk.com> - 2015-11-08 11:19 +0000
                                    Re: Question about math.pi is mutable Marko Rauhamaa <marko@pacujo.net> - 2015-11-08 13:50 +0200
                                      Re: Question about math.pi is mutable BartC <bc@freeuk.com> - 2015-11-08 12:39 +0000
                                        Re: Question about math.pi is mutable Marko Rauhamaa <marko@pacujo.net> - 2015-11-08 14:43 +0200
                                          Re: Question about math.pi is mutable BartC <bc@freeuk.com> - 2015-11-08 13:42 +0000
                                            Re: Question about math.pi is mutable Marko Rauhamaa <marko@pacujo.net> - 2015-11-08 16:33 +0200
                                            Re: Question about math.pi is mutable Chris Angelico <rosuav@gmail.com> - 2015-11-09 02:11 +1100
                                        Re: Question about math.pi is mutable Steven D'Aprano <steve@pearwood.info> - 2015-11-09 11:58 +1100
                                    Re: Question about math.pi is mutable Chris Angelico <rosuav@gmail.com> - 2015-11-08 23:28 +1100
                                      Re: Question about math.pi is mutable Steven D'Aprano <steve@pearwood.info> - 2015-11-09 11:35 +1100
                                    Re: Question about math.pi is mutable Michael Torrie <torriem@gmail.com> - 2015-11-08 08:59 -0700
                                      Re: Question about math.pi is mutable BartC <bc@freeuk.com> - 2015-11-08 17:54 +0000
                                        Re: Question about math.pi is mutable Chris Angelico <rosuav@gmail.com> - 2015-11-09 05:01 +1100
                                          Re: Question about math.pi is mutable BartC <bc@freeuk.com> - 2015-11-08 18:58 +0000
                                            Re: Question about math.pi is mutable Ian Kelly <ian.g.kelly@gmail.com> - 2015-11-08 12:51 -0700
                                            Re: Question about math.pi is mutable Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-11-08 18:13 -0500
                                              Re: Question about math.pi is mutable BartC <bc@freeuk.com> - 2015-11-08 23:54 +0000
                                                Re: Question about math.pi is mutable Ben Finney <ben+python@benfinney.id.au> - 2015-11-09 11:00 +1100
                                                  Re: Question about math.pi is mutable BartC <bc@freeuk.com> - 2015-11-09 00:11 +0000
                                                    Re: Question about math.pi is mutable Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-11-09 08:18 -0500
                                                Re: Question about math.pi is mutable Chris Angelico <rosuav@gmail.com> - 2015-11-09 11:04 +1100
                                                Re: Question about math.pi is mutable Ben Finney <ben+python@benfinney.id.au> - 2015-11-09 11:26 +1100
                                                  Re: Question about math.pi is mutable Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-11-09 21:29 +1300
                                                Re: Question about math.pi is mutable Chris Angelico <rosuav@gmail.com> - 2015-11-09 11:36 +1100
                                                Re: Question about math.pi is mutable Ben Finney <ben+python@benfinney.id.au> - 2015-11-09 11:50 +1100
                                                Re: Question about math.pi is mutable Chris Angelico <rosuav@gmail.com> - 2015-11-09 11:56 +1100
                                                Re: Question about math.pi is mutable Ben Finney <ben+python@benfinney.id.au> - 2015-11-09 12:04 +1100
                                                  Re: Question about math.pi is mutable Steven D'Aprano <steve@pearwood.info> - 2015-11-09 22:22 +1100
                                                    Re: Question about math.pi is mutable Random832 <random832@fastmail.com> - 2015-11-09 10:32 -0500
                                                    Re: Question about math.pi is mutable Ben Finney <ben+python@benfinney.id.au> - 2015-11-10 06:45 +1100
                                                      Re: Question about math.pi is mutable Steven D'Aprano <steve@pearwood.info> - 2015-11-10 13:37 +1100
                                                        Re: Question about math.pi is mutable Ben Finney <ben+python@benfinney.id.au> - 2015-11-10 17:10 +1100
                                                          Re: Question about math.pi is mutable Steven D'Aprano <steve@pearwood.info> - 2015-11-10 22:34 +1100
                                                            Re: Question about math.pi is mutable BartC <bc@freeuk.com> - 2015-11-10 13:26 +0000
                                                              Re: Question about math.pi is mutable Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-11-11 18:38 +1100
                                                                Re: Question about math.pi is mutable Marko Rauhamaa <marko@pacujo.net> - 2015-11-11 10:30 +0200
                                                                  Re: Question about math.pi is mutable Steven D'Aprano <steve@pearwood.info> - 2015-11-11 22:20 +1100
                                                        Re: Question about math.pi is mutable Terry Reedy <tjreedy@udel.edu> - 2015-11-10 03:30 -0500
                                                        Re: Question about math.pi is mutable Laura Creighton <lac@openend.se> - 2015-11-10 11:33 +0100
                                                        Re: Question about math.pi is mutable Ben Finney <ben+python@benfinney.id.au> - 2015-11-10 23:14 +1100
                                                          Re: Question about math.pi is mutable Steven D'Aprano <steve@pearwood.info> - 2015-11-11 12:10 +1100
                                                    Re: Question about math.pi is mutable Laura Creighton <lac@openend.se> - 2015-11-09 21:07 +0100
                                                    Re: Question about math.pi is mutable Ben Finney <ben+python@benfinney.id.au> - 2015-11-10 10:21 +1100
                                                  Re: Question about math.pi is mutable BartC <bc@freeuk.com> - 2015-11-09 16:54 +0000
                                                    Re: Question about math.pi is mutable Ben Finney <ben+python@benfinney.id.au> - 2015-11-10 06:52 +1100
                                    Re: Question about math.pi is mutable Ben Finney <ben+python@benfinney.id.au> - 2015-11-09 08:00 +1100
                                      Re: Question about math.pi is mutable BartC <bc@freeuk.com> - 2015-11-08 22:35 +0000
                                        Re: Question about math.pi is mutable Ben Finney <ben+python@benfinney.id.au> - 2015-11-09 09:54 +1100
                                        Re: Question about math.pi is mutable Steven D'Aprano <steve@pearwood.info> - 2015-11-09 13:23 +1100
                                          Re: Question about math.pi is mutable BartC <bc@freeuk.com> - 2015-11-09 12:22 +0000
                                            Re: Question about math.pi is mutable Chris Angelico <rosuav@gmail.com> - 2015-11-09 23:36 +1100
                                Re: Question about math.pi is mutable Steven D'Aprano <steve@pearwood.info> - 2015-11-08 22:30 +1100
                                  Re: Question about math.pi is mutable BartC <bc@freeuk.com> - 2015-11-08 12:24 +0000
                          Re: Question about math.pi is mutable Grant Edwards <invalid@invalid.invalid> - 2015-11-07 16:16 +0000
                            Re: Question about math.pi is mutable Marko Rauhamaa <marko@pacujo.net> - 2015-11-07 20:00 +0200
                              Re: Question about math.pi is mutable Grant Edwards <invalid@invalid.invalid> - 2015-11-08 03:31 +0000
                                Re: Question about math.pi is mutable Marko Rauhamaa <marko@pacujo.net> - 2015-11-08 09:45 +0200
                                  Re: Question about math.pi is mutable Christian Gollwitzer <auriocus@gmx.de> - 2015-11-08 09:00 +0100
                                  Re: Question about math.pi is mutable Paul Rubin <no.email@nospam.invalid> - 2015-11-08 00:08 -0800
                                    Re: Question about math.pi is mutable Marko Rauhamaa <marko@pacujo.net> - 2015-11-08 11:49 +0200
                                  Re: Question about math.pi is mutable Grant Edwards <invalid@invalid.invalid> - 2015-11-09 14:49 +0000
                        Re: Question about math.pi is mutable Bartc <bc@freeuk.com> - 2015-11-07 15:19 +0000
                        Re: Question about math.pi is mutable Steven D'Aprano <steve@pearwood.info> - 2015-11-08 14:50 +1100
                          Re: Question about math.pi is mutable Marko Rauhamaa <marko@pacujo.net> - 2015-11-08 11:44 +0200
                            Re: Question about math.pi is mutable Steven D'Aprano <steve@pearwood.info> - 2015-11-08 22:11 +1100
                              Re: Question about math.pi is mutable Marko Rauhamaa <marko@pacujo.net> - 2015-11-08 13:25 +0200
                          Re: Question about math.pi is mutable BartC <bc@freeuk.com> - 2015-11-08 11:06 +0000
          Re: Question about math.pi is mutable Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2015-11-09 15:49 +0100
          Re: Question about math.pi is mutable Ben Finney <ben+python@benfinney.id.au> - 2015-11-10 10:29 +1100
          Re: Question about math.pi is mutable Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2015-11-10 11:03 +0100
          Re: Question about math.pi is mutable Michael Torrie <torriem@gmail.com> - 2015-11-13 20:11 -0700
          Re: Question about math.pi is mutable Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2015-11-14 16:02 +0100
      Re: Question about math.pi is mutable Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2015-11-06 23:26 +0100
        Re: Question about math.pi is mutable Terry Reedy <tjreedy@udel.edu> - 2015-11-06 20:49 -0500
        Re: Question about math.pi is mutable Steven D'Aprano <steve@pearwood.info> - 2015-11-07 13:06 +1100
        Re: Question about math.pi is mutable Bartc <bc@freeuk.com> - 2015-11-07 23:02 +0000
          Re: Question about math.pi is mutable Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-11-07 23:27 +0000
          Re: Question about math.pi is mutable Ben Finney <ben+python@benfinney.id.au> - 2015-11-08 10:32 +1100
          Re: Question about math.pi is mutable Chris Angelico <rosuav@gmail.com> - 2015-11-08 10:52 +1100
            Re: Question about math.pi is mutable Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2015-11-12 21:40 +0100
              Re: Question about math.pi is mutable Steven D'Aprano <steve@pearwood.info> - 2015-11-13 09:04 +1100
                Re: Question about math.pi is mutable Denis McMahon <denismfmcmahon@gmail.com> - 2015-11-13 09:19 +0000
                  Re: Question about math.pi is mutable Larry Hudson <orgnut@yahoo.com> - 2015-11-13 18:30 -0800
              Re: Question about math.pi is mutable BartC <bc@freeuk.com> - 2015-11-12 22:19 +0000
                Re: Question about math.pi is mutable Chris Angelico <rosuav@gmail.com> - 2015-11-13 09:52 +1100

Page 2 of 6 — ← Prev page 1 [2] 3 4 5 6  Next page →

#98418

On Sun, 8 Nov 2015 04:46 am, Marko Rauhamaa wrote:

> Bartc <bc@freeuk.com>:
> 
>> (Yes, 'const' in C is a waste of time, and half the people using it
>> don't appear to know what it means.
> 
> It cannot mean anything meaningful; it's completely useless.
> 
> For example, what could one think of standard library functions whose
> prototypes are:
> 
>    char *strstr(const char *haystack, const char *needle);
> 
>    int execve(const char *filename, char *const argv[],
>               char *const envp[]);

Surely that is obvious? I don't speak much C, but I would expect that inside
the functions, const parameters can be read, but not assigned to. "*const"
is a mystery to me though.


> Ok, enough of C.

Amen to that.

If I had a dollar for every crime against computing committed because of C,
or enabled due to C's design decisions, I would be rich beyond the dreams
of avarice.

Fortunately, C is not the only other computer language, and we are by no
means limited to the two choices of the status quo (no constants in Python)
and what C does.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#98424

Steven D'Aprano <steve@pearwood.info> writes:
> Surely that is obvious? I don't speak much C, but I would expect that inside
> the functions, const parameters can be read, but not assigned to. "*const"
> is a mystery to me though.

There's a program (and now a website) called cdecl that can decode these:

char *const envp[]
declare envp as array of const pointer to char

In other words, the pointers in the array are const, but the array
itself (which is really a pointer, and it's generally regarded as
nonsense to make const first-level function arguments anyway, but enough
of that) is not, nor are the characters that the pointers point to.

There are various reasons I don't fully understand and are beyond the
scope of this argument regarding why it's not possible to assign a
pointer-to-pointer-to-char to a pointer-to-pointer-to-const-char, so the
argument isn't declared with the latter.

Unfortunately, while it handles simple function arguments, it (the
version on the website at least) apparently can't handle named or const
function arguments, otherwise you could paste the whole thing.

[toc] | [prev] | [next] | [standalone]

#98417

On Sun, 8 Nov 2015 02:01 am, Bartc wrote:

> Neither have the simplicity of concept of Pascal's 'const', which is
> just a named value. Not a variable that won't change once initialised,
> not a parameter that won't be changed nor any addressable location.)

Unfortunately the concept of "named value" doesn't match well with Python's
design. That implies a separate compilation step which doesn't fit well
with Python's runtime semantics. Very little happens at compile-time in
Python that *must* happen at compile-time.

I'm also not sure what difference you think there is between "named value"
and "variable that won't change once initialised".

Python has a convention for "constants" -- all UPPERCASE names. The fact
that the convention exists is enough to prove that the concept
of "constant" is a useful one. The difference between Python's
pseudo-constants and (say) Pascal's actual constants is that in Python, the
burden of ensuring that neither you, nor any of the libraries you call,
modifies the "constant" falls on you, the user, whereas in Pascal the
compiler or interpreter performs that checking for you.

Besides, the Python convention is honoured more in theory than in practice.
We have math.pi, not math.PI.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#98437

On 08/11/2015 02:59, Steven D'Aprano wrote:
> On Sun, 8 Nov 2015 02:01 am, Bartc wrote:
>
>> Neither have the simplicity of concept of Pascal's 'const', which is
>> just a named value. Not a variable that won't change once initialised,
>> not a parameter that won't be changed nor any addressable location.)
>
> Unfortunately the concept of "named value" doesn't match well with Python's
> design. That implies a separate compilation step which doesn't fit well
> with Python's runtime semantics. Very little happens at compile-time in
> Python that *must* happen at compile-time.
>
> I'm also not sure what difference you think there is between "named value"
> and "variable that won't change once initialised".

This is what I mean about people not understanding it!

In NASM terms, a named constant is like:

     daysinweek   equ   7           ; definition

     mov rax, daysinweek            ; using it, as immediate value
     mov rbx, daysinweek*2          ; an a 'compile-term' expression
;   mov daysinweek,8               ; can't be done! Illegal syntax

While a 'const' variable, C style, might be:

     segment .rodata
monthsinyear:                      ; definition
     dd 12

     mov rax,[monthsinyear]         ; using it, as memory access
;   mov rbx,[monthsinyear*10]      ; can't be done!
     mov [monthsinyear],6           ; can be done, but might give memory
                                    ; access errors if actually stored in
                                    ; protected memory. Usually in C,
                                    ; it isn't

So in native code, a named value is not much different to a literal such 
as 7, or 3.14159. (But unlike C's #define, the name is a proper symbol 
with normal scope rules, and a type).

The distinction at the machine level can be blurred with some 
instructions sets where there might not be an immediate data option for 
some data widths or types. Also where named constants are applied to 
things such as strings, which necessarily use storage.

In the language however, you will not be able to use the named constant 
as an lvalue, and you will usually be able to use it for compile-time 
constant folding and for dimensioning fixed-bound arrays and such.)

> Python has a convention for "constants" -- all UPPERCASE names. The fact
> that the convention exists is enough to prove that the concept
> of "constant" is a useful one. The difference between Python's
> pseudo-constants and (say) Pascal's actual constants is that in Python, the
> burden of ensuring that neither you, nor any of the libraries you call,
> modifies the "constant" falls on you, the user, whereas in Pascal the
> compiler or interpreter performs that checking for you.

With a real named constant the check can always be done at compile-time. 
Unless you have a pure interpreter or some more elaborate way of 
executing source code.

(In the case of Python, the main obstacle is that a constant name from 
inside an imported module is not visible when this module is compiled to 
byte-code. So it has to assume it can be anything.)

-- 
BartC

[toc] | [prev] | [next] | [standalone]

#98438

Bartc <bc@freeuk.com>:

> (In the case of Python, the main obstacle is that a constant name from
> inside an imported module is not visible when this module is compiled
> to byte-code. So it has to assume it can be anything.)

Which it can.

Optimizing for naive floating-point constants could be done but has
barely any significance in most Python code. Numeric computations are
carried out in separate native plugins.

Most Python code deals with encapsulated objects whose innards are
hidden and whose only semantics are defined by the access methods. New
implementations or derived objects can be plugged in transparently.

A case in point are the so-called file-like objects. They are a simple,
elegant concept. I can come up with new file-like objects without any
common pedigree with some predefined classes and incorporate them with a
lot of utility classes.

That elegant dynamism comes at a cost: method lookup is not a constant
memory offset. Rather, it is a dictionary lookup.


Marko

[toc] | [prev] | [next] | [standalone]

#98442

On 08/11/2015 11:02, Marko Rauhamaa wrote:
> Bartc <bc@freeuk.com>:
>
>> (In the case of Python, the main obstacle is that a constant name from
>> inside an imported module is not visible when this module is compiled
>> to byte-code. So it has to assume it can be anything.)
>
> Which it can.
....
> That elegant dynamism comes at a cost: method lookup is not a constant
> memory offset. Rather, it is a dictionary lookup.

I've never understood why this seems to be necessary in Python. Why do 
names have to be looked up? (I'm assuming this is searching by name in 
some sort of table.)

When a module is compiled, while the compiler can't see the definitions 
inside the imported modules, it /will/ know all the names that appear in 
this module, so it can organise them into fixed tables. Then the names 
can be referred to by index. (I think LOAD_FAST does this.)

Or is eval() the culprit here by making most optimisations impossible?

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#98445

BartC <bc@freeuk.com>:

> On 08/11/2015 11:02, Marko Rauhamaa wrote:
>> That elegant dynamism comes at a cost: method lookup is not a constant
>> memory offset. Rather, it is a dictionary lookup.
>
> I've never understood why this seems to be necessary in Python. Why do
> names have to be looked up? (I'm assuming this is searching by name in
> some sort of table.)
>
> When a module is compiled, while the compiler can't see the
> definitions inside the imported modules, it /will/ know all the names
> that appear in this module, so it can organise them into fixed tables.
> Then the names can be referred to by index. (I think LOAD_FAST does
> this.)

Modules are not the issue. Barely any functions are invoked from the
modules. Rather, almost all invocations are object methods. (Of course,
modules are objects, too.) Methods belong to objects that can be
literally anything.

Consider, for example,

    shutil.copyfileobj(src, dst[, length])

The shutil module has absolutely no idea what kind of objects src and
dst are. An example program:

========================================================================
import shutil

class Source:
    def __init__(self):
        self.remaining = "hello world"

    def read(self, count):
        if count <= 0:
            return ""
        chunk, self.remaining = self.remaining[:count], self.remaining[count:]
        return chunk

class Dest:
    def write(self, stuff):
        print("<{}>".format(stuff))

shutil.copyfileobj(Source(), Dest())
========================================================================


Marko

[toc] | [prev] | [next] | [standalone]

#98448

On 08/11/2015 11:50, Marko Rauhamaa wrote:
> BartC <bc@freeuk.com>:
>
>> On 08/11/2015 11:02, Marko Rauhamaa wrote:
>>> That elegant dynamism comes at a cost: method lookup is not a constant
>>> memory offset. Rather, it is a dictionary lookup.
>>
>> I've never understood why this seems to be necessary in Python. Why do
>> names have to be looked up? (I'm assuming this is searching by name in
>> some sort of table.)
>>
>> When a module is compiled, while the compiler can't see the
>> definitions inside the imported modules, it /will/ know all the names
>> that appear in this module, so it can organise them into fixed tables.
>> Then the names can be referred to by index. (I think LOAD_FAST does
>> this.)
>
> Modules are not the issue. Barely any functions are invoked from the
> modules. Rather, almost all invocations are object methods. (Of course,
> modules are objects, too.) Methods belong to objects that can be
> literally anything.
>
> Consider, for example,
>
>      shutil.copyfileobj(src, dst[, length])
>
> The shutil module has absolutely no idea what kind of objects src and
> dst are. An example program:
>
> ========================================================================
> import shutil
>
> class Source:
>      def __init__(self):
>          self.remaining = "hello world"
>
>      def read(self, count):
>          if count <= 0:
>              return ""
>          chunk, self.remaining = self.remaining[:count], self.remaining[count:]
>          return chunk
>
> class Dest:
>      def write(self, stuff):
>          print("<{}>".format(stuff))
>
> shutil.copyfileobj(Source(), Dest())
> ========================================================================

OK, so here, it is necessary to resolve "copyfileobj" by seeing if 
"shutil" is something that contains a name "copyfileobj" that happens to 
be a method.

Is this the lookup you're talking about, and is it done using the actual 
string "copyfileobj"? If so, does it need to be done every single time 
this line is executed? It would extraordinarily inefficient if that was 
the case.

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#98449

BartC <bc@freeuk.com>:

> On 08/11/2015 11:50, Marko Rauhamaa wrote:
>> ========================================================================
>> import shutil
>>
>> class Source:
>>      def __init__(self):
>>          self.remaining = "hello world"
>>
>>      def read(self, count):
>>          if count <= 0:
>>              return ""
>>          chunk, self.remaining = self.remaining[:count], self.remaining[count:]
>>          return chunk
>>
>> class Dest:
>>      def write(self, stuff):
>>          print("<{}>".format(stuff))
>>
>> shutil.copyfileobj(Source(), Dest())
>> ========================================================================
>
> OK, so here, it is necessary to resolve "copyfileobj" by seeing if
> "shutil" is something that contains a name "copyfileobj" that happens
> to be a method.
>
> Is this the lookup you're talking about, and is it done using the
> actual string "copyfileobj"? If so, does it need to be done every
> single time this line is executed? It would extraordinarily
> inefficient if that was the case.

No, what I'm talking about is that copyfileobj calls src.read() and
dst.write() without having any idea what kinds of objects src and dst
are. Thus, it will be difficult for a performance optimizer to get rid
of the dictionary lookups.


Marko

[toc] | [prev] | [next] | [standalone]

#98450

On 08/11/2015 12:43, Marko Rauhamaa wrote:
> BartC <bc@freeuk.com>:
>
>> On 08/11/2015 11:50, Marko Rauhamaa wrote:
>>> ========================================================================
>>> import shutil
>>>
>>> class Source:
>>>       def __init__(self):
>>>           self.remaining = "hello world"
>>>
>>>       def read(self, count):
>>>           if count <= 0:
>>>               return ""
>>>           chunk, self.remaining = self.remaining[:count], self.remaining[count:]
>>>           return chunk
>>>
>>> class Dest:
>>>       def write(self, stuff):
>>>           print("<{}>".format(stuff))
>>>
>>> shutil.copyfileobj(Source(), Dest())
>>> ========================================================================
>>
>> OK, so here, it is necessary to resolve "copyfileobj" by seeing if
>> "shutil" is something that contains a name "copyfileobj" that happens
>> to be a method.
>>
>> Is this the lookup you're talking about, and is it done using the
>> actual string "copyfileobj"? If so, does it need to be done every
>> single time this line is executed? It would extraordinarily
>> inefficient if that was the case.
>
> No, what I'm talking about is that copyfileobj calls src.read() and
> dst.write() without having any idea what kinds of objects src and dst
> are. Thus, it will be difficult for a performance optimizer to get rid
> of the dictionary lookups.

Sorry, you'll have to assume I'm very stupid.

What exactly is being looked up, and in what?

 From what I can understand in your example:

* You are calling shutil.copyfileobj with two arguments, which happen to 
be instances of classes Source and Dest.

* Let's say these are known as src and dst inside .copyfileobj.

* .copyfileobj then calls methods src.read() dst.write().

* Each of these method calls has the form A.B() which seems to me to be 
little different from shutil.copyfileobj().

So to get back to what I was saying, does this lookup involving 
searching for method B in object A, and if so, does it actually do a 
search by name?

(Someone mentioned a precalculated hash, of "A", or of "src" or "dst", 
use used, but still, it's looking something up in a table, and a hash 
table lookup I believe still requires an string compare to check if 
you've got the right entry.)

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#98451

BartC <bc@freeuk.com>:

>>> On 08/11/2015 11:50, Marko Rauhamaa wrote:
>>>> ========================================================================
>>>> import shutil
>>>>
>>>> class Source:
>>>>       def __init__(self):
>>>>           self.remaining = "hello world"
>>>>
>>>>       def read(self, count):
>>>>           if count <= 0:
>>>>               return ""
>>>>           chunk, self.remaining = self.remaining[:count], self.remaining[count:]
>>>>           return chunk
>>>>
>>>> class Dest:
>>>>       def write(self, stuff):
>>>>           print("<{}>".format(stuff))
>>>>
>>>> shutil.copyfileobj(Source(), Dest())
>>>> ========================================================================
>
> What exactly is being looked up, and in what?
>
> [...]
>
> So to get back to what I was saying, does this lookup involving
> searching for method B in object A, and if so, does it actually do a
> search by name?

Yes, finding Source.read and Dest.write involves hash table lookups. In
fact, it's even a bit more involved: each method fetch consists of *two*
hash table lookups. First you have to check if the object has a match in
its individual lookup table and next if there is a match in the class's
lookup table.


Marko

[toc] | [prev] | [next] | [standalone]

#98452

On Mon, Nov 9, 2015 at 12:42 AM, BartC <bc@freeuk.com> wrote:
> Sorry, you'll have to assume I'm very stupid.
>
> What exactly is being looked up, and in what?
>
> From what I can understand in your example:
>
> * You are calling shutil.copyfileobj with two arguments, which happen to be
> instances of classes Source and Dest.
>
> * Let's say these are known as src and dst inside .copyfileobj.
>
> * .copyfileobj then calls methods src.read() dst.write().
>
> * Each of these method calls has the form A.B() which seems to me to be
> little different from shutil.copyfileobj().
>
> So to get back to what I was saying, does this lookup involving searching
> for method B in object A, and if so, does it actually do a search by name?
>
> (Someone mentioned a precalculated hash, of "A", or of "src" or "dst", use
> used, but still, it's looking something up in a table, and a hash table
> lookup I believe still requires an string compare to check if you've got the
> right entry.)

The lookups I'm talking about happen pretty much anywhere. Firstly,
this function:

def f():
    shutil.copyfileobj(src, dst)

looks up the global name 'shutil' (that's one dict lookup - module
globals are a dictionary), then performs an attribute lookup on the
resulting object with the name 'copyfileobj', then looks up the two
global names 'src' and 'dst'.

In fact[1], those strings can be found in f.__code__.co_names, and the
byte-code identifies them by their indices in that tuple. These are
str objects (note that this means they can be any legal Unicode
string; "шутил.цопыфилеобй(срц, дст)" is just as valid), and their
hashes will normally be precomputed to save time. Now, things are a
bit different with function-locals; the compiler always knows which
names they are, and it can compile things differently. Compare:

def f(src):
    shutil.copyfileobj(src, dst)

Instead of looking up a global name 'src', this now looks up a local
name. They're identified by slot positions, so the compiler simply
knows that "src" is in slot #0, and instead of looking anything up,
simply says "load the thing in slot #0". There can also be other
levels of indirection, such as __getattr__ and __getattribute__, which
affect how stuff gets looked up. But otherwise, name lookups generally
involve poking a dict with a string and taking what comes back
(possibly replacing KeyError with NameError or AttributeError as the
case may be).

ChrisA

[1] Assuming we're using CPython, or something which uses the same
byte-code; this is NOT a language requirement - none of this is.

[toc] | [prev] | [next] | [standalone]

#98497

On Sun, 8 Nov 2015 11:39 pm, BartC wrote:


>> shutil.copyfileobj(Source(), Dest())
>> ========================================================================
> 
> OK, so here, it is necessary to resolve "copyfileobj" by seeing if
> "shutil" is something that contains a name "copyfileobj" that happens to
> be a method.

Correct.

> Is this the lookup you're talking about, and is it done using the actual
> string "copyfileobj"? If so, does it need to be done every single time
> this line is executed? It would extraordinarily inefficient if that was
> the case.

That's not what Marko was referring to, but you are correct, it is done
using the string "copyfileobj". And it has to be done every single time,
since you cannot be sure that the lookup will always return the same
object, or even whether it will always succeed. For example, here is a
self-destructing method:


class X:
    counter = 0
    def method(self):
        self.counter += 1
        if self.counter == 3:
            del X.method
        msg = "This method will self-destruct in %d calls."
        return msg % (3 - self.counter)


But you are wrong to describe this as "extraordinarily inefficient". It is
certainly efficient enough to be practical: inefficient or not, Python is
used for many thousands of practical applications. It is only about a
factor of 100 times slower than C or Java, and the PyPy optimizing compiler
averages ten times faster (and, occasionally, approaches or even exceeds
the speed of similar C or Java code). When performance is critical, you can
often locate the bottlenecks and turn that into C or Fortran code.

Ruby, which is even more flexible than Python in some ways, is even slower,
and yet even Ruby is still fast enough to be practical for many tasks.

And remember too that some of the speed of Java (and especially C) is gained
at the expense of:

- prohibiting the developer from doing certain things;
- restricting the range of values you can use;
- making significant assumptions about correctness;
- avoiding actually checking those assumptions;
- and in the case of C, by outright *ignoring the code you write* if you
inadvertently violate those assumptions.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#98447

On Sun, Nov 8, 2015 at 10:19 PM, BartC <bc@freeuk.com> wrote:
> I've never understood why this seems to be necessary in Python. Why do names
> have to be looked up? (I'm assuming this is searching by name in some sort
> of table.)

Yes, if by "searching" you include hash-table lookup. A CPython
dictionary is a *highly* optimized data structure, specifically
because it's used virtually everywhere (I understand Lua's "table"
type has similar optimizations for the same reason). In the common
case, where your names come from literal text in the module, the
strings used for the lookup will be interned constants, and their
hashes will have been precalculated and stored, so the lookup is
pretty easy. So it's a constant-time operation, and while that
constant may be larger than a simple offset-and-fetch, it's still
pretty fast in the overall scheme of things.

ChrisA

[toc] | [prev] | [next] | [standalone]

#98494

On Sun, 8 Nov 2015 11:28 pm, Chris Angelico wrote:

> On Sun, Nov 8, 2015 at 10:19 PM, BartC <bc@freeuk.com> wrote:
>> I've never understood why this seems to be necessary in Python. Why do
>> names have to be looked up? (I'm assuming this is searching by name in
>> some sort of table.)
> 
> Yes, if by "searching" you include hash-table lookup. A CPython
> dictionary is a *highly* optimized data structure, specifically
> because it's used virtually everywhere (I understand Lua's "table"
> type has similar optimizations for the same reason). In the common
> case, where your names come from literal text in the module, the
> strings used for the lookup will be interned constants, and their
> hashes will have been precalculated and stored, so the lookup is
> pretty easy. So it's a constant-time operation, and while that
> constant may be larger than a simple offset-and-fetch, it's still
> pretty fast in the overall scheme of things.

*Usually*.

There are pathological cases -- if you're unlucky enough, or malicious
enough, to have a whole lot of names with the same hash value, the look-up
degrades to a linear search.

More importantly, consider the case of a function referring to a variable
named "spam". In principle, at least, it may need to:

- search the local namespace;
- search the namespace of one or more enclosing functions;
- search the global namespace;
- search the built-in namespace;

before locating the variable's value. In practice, most Python
implementations will provide at least one optimization: local variables are
recognised and searched using an offset-and-fetch model.

Similarly for method calls: spam.eggs needs to:

- search the instance dict and slots;
- search the class dict;
- search the dict for each additional superclass;

where searching involves more than just a hash table lookup. It also
involves checking the classes for __getattribute__ and __getattr__ methods,
checking for descriptors, and more. The whole lookup process is quite
complicated, and little of it can be done at compile time.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#98460

On 11/08/2015 04:19 AM, BartC wrote:

>> That elegant dynamism comes at a cost: method lookup is not a constant
>> memory offset. Rather, it is a dictionary lookup.
> 
> I've never understood why this seems to be necessary in Python. Why do 
> names have to be looked up? (I'm assuming this is searching by name in 
> some sort of table.)
> 
> When a module is compiled, while the compiler can't see the definitions 
> inside the imported modules, it /will/ know all the names that appear in 
> this module, so it can organise them into fixed tables. Then the names 
> can be referred to by index. (I think LOAD_FAST does this.)
> 
> Or is eval() the culprit here by making most optimisations impossible?

Perhaps I'm misunderstanding what you're saying, here, but the reason a
lookup has to be performed is because python variables are not like C
variables. They aren't boxes per se.  They are names bound to objects.
so doing something like

a += 1

Actually assigns the name "a" to an entirely new object than the one it
had before, which, from the interpreters point of view, is in an
entirely different memory location.

Whereas in C, a variable is a box that's always at the same location, so
the name is simply not important, and at runtime variables have no
names.  Python is not like this.  names are important at runtime.  There
may be certain circumstances where the Python compiler could optimize
certain types of variable access (say if the variable was bound to
integers only) into C-style variables under the hood. But I don't think
the benefit would be that great.

[toc] | [prev] | [next] | [standalone]

#98469

On 08/11/2015 15:59, Michael Torrie wrote:
> On 11/08/2015 04:19 AM, BartC wrote:
>
>>> That elegant dynamism comes at a cost: method lookup is not a constant
>>> memory offset. Rather, it is a dictionary lookup.
>>
>> I've never understood why this seems to be necessary in Python. Why do
>> names have to be looked up? (I'm assuming this is searching by name in
>> some sort of table.)
>>
>> When a module is compiled, while the compiler can't see the definitions
>> inside the imported modules, it /will/ know all the names that appear in
>> this module, so it can organise them into fixed tables. Then the names
>> can be referred to by index. (I think LOAD_FAST does this.)
>>
>> Or is eval() the culprit here by making most optimisations impossible?
>
> Perhaps I'm misunderstanding what you're saying, here, but the reason a
> lookup has to be performed is because python variables are not like C
> variables. They aren't boxes per se.  They are names bound to objects.
> so doing something like
>
> a += 1
>
> Actually assigns the name "a" to an entirely new object than the one it
> had before, which, from the interpreters point of view, is in an
> entirely different memory location.

That would be true for locals as well. But as far as I can tell from the 
CPython source code, byte-codes uses an index to represent a local, 
which represents an entry into a linear table.

I can't quite see why that can't be done for global names and for 
attributes too (ie. the names that follow a ".").

Attributes are a more difficult because there can be multiple instances 
of each name, and Python won't know which one is meant, so it needs to 
be resolved at each use.

So in the case of A.B(), Python apparently looks up "B" in the names 
associated with object A (and also, someone said, with the class of 
which A is an instance if the first lookup fails).

I had in mind a different approach, where a table exists for each 
different attribute, which contains an entry for each owner class. 
Usually this table will be small, or will have just one entry, so is 
quick to search.

However, another feature of Python makes that impractical: the ability 
of any instance of a class to have arbitrary sets of attributes. There 
could be millions of such instances, all named "B".

Now, if methods could only be part of a class definition, then with 
A.B() rather than A.B, there will be a limited number of classes that 
could have a method "B", so the short table idea could work.

But then, A.B could be assigned the name of a function, which is then 
called using A.B(), the same as method-calling syntax!

So it looks like Python is stuck with its name lookups.

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#98470

On Mon, Nov 9, 2015 at 4:54 AM, BartC <bc@freeuk.com> wrote:
> That would be true for locals as well. But as far as I can tell from the
> CPython source code, byte-codes uses an index to represent a local, which
> represents an entry into a linear table.
>
> I can't quite see why that can't be done for global names and for attributes
> too (ie. the names that follow a ".").

At compilation time, the set of local names is locked in permanently.
That's not true of any other namespace (except nonlocals, which are
still locals, just not yours). Attributes and global names (which are
attributes of the current module) can be added and removed
dynamically, so they can't be assigned to slots; attributes can even
be simulated via __getattr__ and __getattribute__.

ChrisA

[toc] | [prev] | [next] | [standalone]

#98472

On 08/11/2015 18:01, Chris Angelico wrote:
> On Mon, Nov 9, 2015 at 4:54 AM, BartC <bc@freeuk.com> wrote:
>> That would be true for locals as well. But as far as I can tell from the
>> CPython source code, byte-codes uses an index to represent a local, which
>> represents an entry into a linear table.
>>
>> I can't quite see why that can't be done for global names and for attributes
>> too (ie. the names that follow a ".").
>
> At compilation time, the set of local names is locked in permanently.
> That's not true of any other namespace (except nonlocals, which are
> still locals, just not yours). Attributes and global names (which are
> attributes of the current module) can be added and removed
> dynamically, so they can't be assigned to slots; attributes can even
> be simulated via __getattr__ and __getattribute__.

Yes, that's yet another way that the language design makes things 
difficult (for those who want to streamline its implementation).

For most purposes, the set of attribute names that can follow a dot are 
limited and fixed, by the occurrences of such names in a source file. So 
they can be assigned a slot.

But then, you say that additional attributes, potentially millions of 
different ones, can be invented at runtime. Although I don't see how it 
can remove names that are part of the source code: if "A.B" is in the 
file, then surely "A" and "B" always have to be present in some table or 
other.

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#98474

On Nov 8, 2015 12:01 PM, "BartC" <bc@freeuk.com> wrote:
>
> But then, you say that additional attributes, potentially millions of
different ones, can be invented at runtime. Although I don't see how it can
remove names that are part of the source code: if "A.B" is in the file,
then surely "A" and "B" always have to be present in some table or other.

Gratuitous example:

def factory(name):
    class inner:
        # stuff
    inner.__name__ = name
    return inner

One = factory('One')
Two = factory('Two')

# factory is no longer needed, so remove it from globals.
del factory

[toc] | [prev] | [next] | [standalone]

Page 2 of 6 — ← Prev page 1 [2] 3 4 5 6  Next page →

Back to top | Article view | comp.lang.python

csiph-web