Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #90578 > unrolled thread

Building CPython

Back to article view | Back to comp.lang.python

Contents

  Building CPython BartC <bc@freeuk.com> - 2015-05-13 20:36 +0100
    Re: Building CPython Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-05-13 20:58 +0100
    Re: Building CPython Terry Reedy <tjreedy@udel.edu> - 2015-05-13 18:34 -0400
      Re: Building CPython BartC <bc@freeuk.com> - 2015-05-14 16:51 +0100
        Re: Building CPython Chris Angelico <rosuav@gmail.com> - 2015-05-15 02:09 +1000
          Re: Building CPython BartC <bc@freeuk.com> - 2015-05-14 18:02 +0100
            Re: Building CPython Dave Angel <davea@davea.name> - 2015-05-14 13:10 -0400
            Re: Building CPython Chris Angelico <rosuav@gmail.com> - 2015-05-15 03:11 +1000
              Re: Building CPython BartC <bc@freeuk.com> - 2015-05-14 18:32 +0100
                Re: Building CPython Chris Angelico <rosuav@gmail.com> - 2015-05-15 03:45 +1000
            Re: Building CPython Terry Reedy <tjreedy@udel.edu> - 2015-05-14 14:50 -0400
              Re: Building CPython Christian Gollwitzer <auriocus@gmx.de> - 2015-05-15 12:51 +0200
                Re: Building CPython Terry Reedy <tjreedy@udel.edu> - 2015-05-15 17:19 -0400
        Re: Building CPython Marko Rauhamaa <marko@pacujo.net> - 2015-05-14 19:29 +0300
          Re: Building CPython BartC <bc@freeuk.com> - 2015-05-14 22:55 +0100
            Re: Building CPython MRAB <python@mrabarnett.plus.com> - 2015-05-14 23:19 +0100
            Re: Building CPython BartC <bc@freeuk.com> - 2015-05-15 01:50 +0100
              Re: Building CPython Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-05-15 18:05 +1200
                Re: Building CPython Marko Rauhamaa <marko@pacujo.net> - 2015-05-15 11:59 +0300
                  Re: Building CPython wxjmfauth@gmail.com - 2015-05-15 02:07 -0700
                    Re: Building CPython Marko Rauhamaa <marko@pacujo.net> - 2015-05-15 12:20 +0300
                      Re: Building CPython wxjmfauth@gmail.com - 2015-05-15 02:51 -0700
                        Re: Building CPython Marko Rauhamaa <marko@pacujo.net> - 2015-05-15 13:52 +0300
                          Re: Building CPython Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-15 22:10 +1000
                            Re: Building CPython Chris Angelico <rosuav@gmail.com> - 2015-05-15 22:34 +1000
                            Re: Building CPython wxjmfauth@gmail.com - 2015-05-15 07:11 -0700
                          Re: Building CPython Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-05-15 13:41 +0100
                      Re: Building CPython Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-05-15 13:38 +0100
                  Re: Building CPython Chris Angelico <rosuav@gmail.com> - 2015-05-15 19:43 +1000
                    Re: Building CPython Marko Rauhamaa <marko@pacujo.net> - 2015-05-15 13:50 +0300
                      Re: Building CPython Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-15 22:43 +1000
                        Re: Building CPython Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-15 09:00 -0600
                        Re: Building CPython Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-15 09:04 -0600
                        Re: Building CPython Chris Angelico <rosuav@gmail.com> - 2015-05-16 01:06 +1000
                  Re: Building CPython Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-15 20:14 +1000
                    Re: Building CPython Chris Angelico <rosuav@gmail.com> - 2015-05-15 20:25 +1000
                  Re: Building CPython Terry Reedy <tjreedy@udel.edu> - 2015-05-15 17:05 -0400
                  Re: Building CPython BartC <bc@freeuk.com> - 2015-05-15 22:54 +0100
                    Re: Building CPython Marko Rauhamaa <marko@pacujo.net> - 2015-05-16 01:44 +0300
                      Re: Building CPython Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-05-16 00:27 +0100
                        Re: Building CPython Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-16 11:55 +1000
                          Re: Building CPython Chris Angelico <rosuav@gmail.com> - 2015-05-16 12:15 +1000
                          Re: Building CPython Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-05-16 03:17 +0100
                      Re: Building CPython BartC <bc@freeuk.com> - 2015-05-16 01:43 +0100
                        Re: Building CPython MRAB <python@mrabarnett.plus.com> - 2015-05-16 02:16 +0100
                        Re: Building CPython Marko Rauhamaa <marko@pacujo.net> - 2015-05-16 11:08 +0300
                          Re: Building CPython Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-16 19:40 +1000
                            Re: Building CPython Marko Rauhamaa <marko@pacujo.net> - 2015-05-16 16:59 +0300
                              Re: Building CPython Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-17 04:18 +1000
                                Re: Building CPython Marko Rauhamaa <marko@pacujo.net> - 2015-05-16 21:55 +0300
                                  Re: Building CPython Marko Rauhamaa <marko@pacujo.net> - 2015-05-17 01:51 +0300
                                    Re: Building CPython Marko Rauhamaa <marko@pacujo.net> - 2015-05-17 23:49 +0300
                    Re: Building CPython Terry Reedy <tjreedy@udel.edu> - 2015-05-15 19:54 -0400
                Re: Building CPython BartC <bc@freeuk.com> - 2015-05-15 10:32 +0100
                  Re: Building CPython Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-05-16 12:55 +1200
                    Re: Building CPython Jonas Wielicki <jonas@wielicki.name> - 2015-05-17 14:25 +0200
                      Re: Building CPython BartC <bc@freeuk.com> - 2015-05-17 14:41 +0100

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

#90658

wxjmfauth@gmail.com:

> Implement unicode correctly.

Did they reject your patch?


Marko

[toc] | [prev] | [next] | [standalone]

#90661

Le vendredi 15 mai 2015 11:20:25 UTC+2, Marko Rauhamaa a écrit :
> wxjmfauth@gmail.com:
> 
> > Implement unicode correctly.
> 
> Did they reject your patch?
> 
> 
> Marko

You can not patch something that is wrong by design.

I'm the first to agree, everybody has the right
to implement his own character encoding model.

Proposing a mechanism, which succeeds to do the
opposite [*] of what Unicode.org offers is a little
bit beyond my understanding.

What to say, if on top of this, it is sold as 
"unicode compliant".

[*] on the side of memory, on the side of perfomance (both
on the same time!) and on the side of very specific "unicode"
tasks.

jmf

PS The bugginess is only an expected and consequently logical
implementation detail.

[toc] | [prev] | [next] | [standalone]

#90666

wxjmfauth@gmail.com:

> Le vendredi 15 mai 2015 11:20:25 UTC+2, Marko Rauhamaa a écrit :
>> wxjmfauth@gmail.com:
>> 
>> > Implement unicode correctly.
>> Did they reject your patch?
>
> You can not patch something that is wrong by design.

Are you saying the Python language spec is unfixable or that the CPython
implementation is unfixable?

If CPython is unfixable, you can develop a better Python implementation.

If Python itself is unfixable, what brings you here?


Marko

[toc] | [prev] | [next] | [standalone]

#90670

On Fri, 15 May 2015 08:52 pm, Marko Rauhamaa wrote:

> wxjmfauth@gmail.com:
> 
>> Le vendredi 15 mai 2015 11:20:25 UTC+2, Marko Rauhamaa a écrit :
>>> wxjmfauth@gmail.com:
>>> 
>>> > Implement unicode correctly.
>>> Did they reject your patch?
>>
>> You can not patch something that is wrong by design.
> 
> Are you saying the Python language spec is unfixable or that the CPython
> implementation is unfixable?

JMF is obsessed with a trivial and artificial performance regression in the
handling of Unicode strings since Python 3.3, which introduced a
significant memory optimization for Unicode strings. Each individual string
uses a code unit no larger than necessary, thus if a string contains
nothing but ASCII or Latin 1 characters, it will use one byte per
character; if it fits into the Basic Multilingual Plane, two bytes per
character; and only use four bytes per character if there are "astral"
characters in the string.

(That is, Python strings select from a Latin-1, UCS-2 and UTF-32 encoded
form at creation time, according to the largest code point in the string.)

The benefit of this is that most strings will use 1/2 or 1/4 of the memory
that they otherwise would need, which gives an impressive memory saving.
That leads to demonstrable speed-ups in real-world code, however it is
possible to find artificial benchmarks that experience a slowdown compared
to Python 3.2.

JMF found one such artificial benchmark, involving creating and throwing
away many strings as fast as possible without doing any work with them, and
from this has built this fantasy in his head that Python is not compliant
with the Unicode spec and is logically, mathematically broken.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#90671

On Fri, May 15, 2015 at 10:10 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> The benefit of this is that most strings will use 1/2 or 1/4 of the memory
> that they otherwise would need, which gives an impressive memory saving.
> That leads to demonstrable speed-ups in real-world code, however it is
> possible to find artificial benchmarks that experience a slowdown compared
> to Python 3.2.

It's also possible to find a number of situations in which a narrow
build of 3.2 was faster than 3.3, due to the buggy handling of
surrogates. I've no idea whether jmf is still complaining against that
basis or not, as I don't see his posts.

ChrisA

[toc] | [prev] | [next] | [standalone]

#90677

Le vendredi 15 mai 2015 14:10:32 UTC+2, Steven D'Aprano a écrit :
> On Fri, 15 May 2015 08:52 pm, Marko Rauhamaa wrote:
> 
> > wxjmfauth@gmail.com:
> > 
> >> Le vendredi 15 mai 2015 11:20:25 UTC+2, Marko Rauhamaa a écrit :
> >>> wxjmfauth@gmail.com:
> >>> 
> >>> > Implement unicode correctly.
> >>> Did they reject your patch?
> >>
> >> You can not patch something that is wrong by design.
> > 
> > Are you saying the Python language spec is unfixable or that the CPython
> > implementation is unfixable?
> 
> JMF is obsessed with a trivial and artificial performance regression in the
> handling of Unicode strings since Python 3.3, which introduced a
> significant memory optimization for Unicode strings. Each individual string
> uses a code unit no larger than necessary, thus if a string contains
> nothing but ASCII or Latin 1 characters, it will use one byte per
> character; if it fits into the Basic Multilingual Plane, two bytes per
> character; and only use four bytes per character if there are "astral"
> characters in the string.
> 
> (That is, Python strings select from a Latin-1, UCS-2 and UTF-32 encoded
> form at creation time, according to the largest code point in the string.)
> 
> The benefit of this is that most strings will use 1/2 or 1/4 of the memory
> that they otherwise would need, which gives an impressive memory saving.
> That leads to demonstrable speed-ups in real-world code, however it is
> possible to find artificial benchmarks that experience a slowdown compared
> to Python 3.2.
> 
> JMF found one such artificial benchmark, involving creating and throwing
> away many strings as fast as possible without doing any work with them, and
> from this has built this fantasy in his head that Python is not compliant
> with the Unicode spec and is logically, mathematically broken.
> 
> 
> -- 
> Steven

It's time you start to understand the "coding of
characters" (can be unicode or other schemes).

Regards.

jmf

[toc] | [prev] | [next] | [standalone]

#90675

On 15/05/2015 11:52, Marko Rauhamaa wrote:
> wxjmfauth@gmail.com:
>
>> Le vendredi 15 mai 2015 11:20:25 UTC+2, Marko Rauhamaa a écrit :
>>> wxjmfauth@gmail.com:
>>>
>>>> Implement unicode correctly.
>>> Did they reject your patch?
>>
>> You can not patch something that is wrong by design.
>
> Are you saying the Python language spec is unfixable or that the CPython
> implementation is unfixable?
>
> If CPython is unfixable, you can develop a better Python implementation.
>
> If Python itself is unfixable, what brings you here?
>
> Marko
>

I forgot to mention earlier that I report all his rubbish as abuse on 
google groups.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#90673

On 15/05/2015 10:20, Marko Rauhamaa wrote:
> wxjmfauth@gmail.com:
>
>> Implement unicode correctly.
>
> Did they reject your patch?
>
> Marko
>

Please don't feed him, it's been obvious for years that he hasn't the 
faintest idea what he's talking about.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#90660

On Fri, May 15, 2015 at 6:59 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
> However, in some respects, Python might be going overboard with its
> dynamism; are all those dunder methods really needed?

Yes - at least, most of them. As regards operators, there are three
options: either you have magic methods for all of them (Python style),
or none of them (Java style, no operator overloading), or you
hybridize and permit just a handful of them (and then you have to
decide which). There's really no reason not to have them. The other
dunder methods are a mixed bag; some are to allow you to customize
object creation itself (a class's __new__ method could be considered
equivalent to an instance-specific __call__ method on the type
object), some let you pretend to be different types of number
(__int__, __index__, __complex__), which allows you to duck-type
integerness rather than having to subclass int and rely on magic; and
others let you customize the behaviour of well-known functions, such
as __len__ for len() and __repr__ for repr(). Without dunder methods
for all of these, it would be difficult to make an object "play
nicely" with the overall Python ecosystem. There are others, though,
which are less crucial (__getstate__ and so on for pickle), but you
can ignore them if you're not using those features.

> Must "false" be defined so broadly?

Different languages define true and false differently. REXX says that
"1" is true and "0" is false, and anything else is an error. Pike says
that the integer 0 is false and anything else is true; the philosophy
is that a "thing" is true and the absence of any thing is false.
Python says that an empty "thing" is false and a non-empty "thing" is
true; if next(iter(x)) raises StopIteration, x is probably false, and
vice versa. All three have their merits, all three have their
consequences. One consequence of Python's model is that a __bool__
method is needed on any object that might be empty (unless it defines
__len__, in which case that makes a fine fall-back); it's normal in
Python code to distinguish between "if x:" and "if x is not None:",
where the former sees if x has anything in it, but the latter sees if
there x even exists. (More or less.)

> Must a method lookup necessarily involve object creation?

Actually, no. Conceptually, this method call:

foo.bar(1, 2, 3)

involves looking up 'foo' in the current namespace, looking up
attribute 'bar' on it, treating the result as a function, and calling
it with three integer objects as its arguments. And the language
definition demands that this work even if the "foo.bar" part is broken
out:

def return_function():
    return foo.bar

def call_function():
    return_function()(1, 2, 3)

But a particular Python implementation is most welcome to notice the
extremely common situation of method calls and optimize it. I'm not
sure if PyPy does this, but I do remember reading about at least one
Python that does; CPython has an optimization for the actual memory
allocations involved, though I think it does actually construct some
sort of object for each one; as long as the resulting behaviour is
within spec, objects needn't be created just to be destroyed.

Dynamism doesn't have to be implemented naively, just as long as the
slow path is there if anyone needs it.

ChrisA

[toc] | [prev] | [next] | [standalone]

#90664

Chris Angelico <rosuav@gmail.com>:

> On Fri, May 15, 2015 at 6:59 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
>> Must a method lookup necessarily involve object creation?
>
> Actually, no.
> [...]
> a particular Python implementation is most welcome to notice the
> extremely common situation of method calls and optimize it.

I'm not sure that is feasible given the way it has been specified. You'd
have to prove the class attribute lookup produces the same outcome in
consecutive method references.

Also:

   >>> class X:
   ...   def f(self): pass
   ... 
   >>> x = X()
   >>> f = x.f
   >>> ff = x.f
   >>> f is ff
   False

Would a compliant Python implementation be allowed to respond "True"?

Maybe. At least method objects seem immutable:

   >>> f.__name__
   'f'
   >>> f.__name__ = 'g'
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   AttributeError: 'method' object has no attribute '__name__'
   >>> f.__eq__ = None
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   AttributeError: 'method' object attribute '__eq__' is read-only
   >>> f.xyz = 'xyz'
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   AttributeError: 'method' object has no attribute 'xyz'


Marko

[toc] | [prev] | [next] | [standalone]

#90674

On Fri, 15 May 2015 08:50 pm, Marko Rauhamaa wrote:

> Chris Angelico <rosuav@gmail.com>:
> 
>> On Fri, May 15, 2015 at 6:59 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
>>> Must a method lookup necessarily involve object creation?
>>
>> Actually, no.
>> [...]
>> a particular Python implementation is most welcome to notice the
>> extremely common situation of method calls and optimize it.
> 
> I'm not sure that is feasible given the way it has been specified. You'd
> have to prove the class attribute lookup produces the same outcome in
> consecutive method references.

Sure. But some implementations may have a more, um, flexible approach to
correctness, and offer more aggressive optimizations which break the letter
of Python's semantics but work for 90% of cases. Just because CPython
doesn't do so, doesn't mean that some new implementation might not offer a
series of aggressive optimizations which the caller (or maybe the module?)
can turn on as needed, e.g.:

- assume methods never change;
- assume classes are static;
- assume built-in names always refer to the known built-in;

etc. Such an optimized Python, when running with those optimizations turned
on, is not *strictly* Python, but "buyer beware" applies here. If the
optimizations break your code or make testing hard, don't use it.


> Also:
> 
>    >>> class X:
>    ...   def f(self): pass
>    ...
>    >>> x = X()
>    >>> f = x.f
>    >>> ff = x.f
>    >>> f is ff
>    False
> 
> Would a compliant Python implementation be allowed to respond "True"?

Certainly.

When you retrieve x.f, Python applies the usual "attribute lookup" code,
which simplified looks like this:

if 'f' in x.__dict__:
    attr = x.__dict__['f']
else:
    for K in type(x).__mro__:
        # Walk the parent classes of x in the method resolution order
        if 'f' in K.__dict__:
            attr = K.__dict__['f']
            break
    else:  # no break
        raise AttributeError
# if we get here, we know x.f exists and is bound to attr
# now apply the descriptor protocol (simplified)
if hasattr(attr, '__get__'):
    attr = attr.__get__(x, type(x))
# Finally we can call x.f()
return attr(x, *args)

Functions have a __get__ method which returns the method object! Imagine
they look something like this:

class FunctionType:
    def __call__(self, *args, **kwargs):
        # Actually call the code that does stuff

    def __get__(self, instance, cls):
        if cls is None:
            # Unbound instance
            return self
        return MethodType(self, instance)  # self is the function


This implementation creates a new method object every time you look it up.
But functions *could* do this:

    def __get__(self, instance, cls):
        if cls is None:
            # Unbound instance
            return self
        if self._method is None:
            self._method = MethodType(self, instance)  # Cache it.
        return self._method



What's more, a compliant implementation could reach the "if we get here"
point in the lookup procedure above, and do this:

# if we get here, we know attr exists
if type(attr) is FunctionType:  # Fast pointer comparison.
    return attr(x, *args)
else:
    # do the descriptor protocol thing, and then call attr


It can only do this if it knows that x.f is a real function, not some sort
of callable or function subclass, because in that case who knows what
side-effects the __get__ method might have.

How much time would it save? Probably very little. After all, unless the
method call itself did bugger-all work, the time to create the method
object is probably insignificant. But it's a possible optimization.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#90679

On Fri, May 15, 2015 at 6:43 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> How much time would it save? Probably very little. After all, unless the
> method call itself did bugger-all work, the time to create the method
> object is probably insignificant. But it's a possible optimization.

An interesting alternative (if it's not already being done) might be
to maintain a limited free-list of method objects, removing the need
to allocate memory for one before filling it in with data.

[toc] | [prev] | [next] | [standalone]

#90680

On Fri, May 15, 2015 at 9:00 AM, Ian Kelly <ian.g.kelly@gmail.com> wrote:
> On Fri, May 15, 2015 at 6:43 AM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> How much time would it save? Probably very little. After all, unless the
>> method call itself did bugger-all work, the time to create the method
>> object is probably insignificant. But it's a possible optimization.
>
> An interesting alternative (if it's not already being done) might be
> to maintain a limited free-list of method objects, removing the need
> to allocate memory for one before filling it in with data.

Looks like it is already being done:
https://hg.python.org/cpython/file/e7c7431f91b2/Objects/methodobject.c#l7

[toc] | [prev] | [next] | [standalone]

#90681

On Sat, May 16, 2015 at 1:00 AM, Ian Kelly <ian.g.kelly@gmail.com> wrote:
> On Fri, May 15, 2015 at 6:43 AM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> How much time would it save? Probably very little. After all, unless the
>> method call itself did bugger-all work, the time to create the method
>> object is probably insignificant. But it's a possible optimization.
>
> An interesting alternative (if it's not already being done) might be
> to maintain a limited free-list of method objects, removing the need
> to allocate memory for one before filling it in with data.

It is already done. Some stats were posted recently to python-dev, and
(if I read them correctly) method objects are among the free-list
types. So the actual memory (de)allocations are optimized, and all
that's left is setting a couple of pointers to select an object and a
function.

ChrisA

[toc] | [prev] | [next] | [standalone]

#90662

On Fri, 15 May 2015 06:59 pm, Marko Rauhamaa wrote:

> However, in some respects, Python might be going overboard with its
> dynamism; are all those dunder methods really needed? Must "false" be
> defined so broadly? Must a method lookup necessarily involve object
> creation?

Yes, what do you mean, and no.

(1) Yes, the dunder methods are necessary to support operator overloading
and various other protocols. The existence of dunder methods doesn't have
any runtime costs except for that related to memory usage. Fortunately, the
dunder methods only exist on the class itself, not each and every instance.

(2) What do you mean by false being defined so broadly?

The singleton instance False is not defined broadly at all. It is a built-in
constant, and there's only one of it. In Python 3, "False" is even a
keyword, so you cannot redefine the name. (It's not a keyword in Python 2
because of historical reasons.)

Perhaps you are talking about falsey (false-like) instances, e.g. 

    if []: ...
    else: ...

will run the else block. That's quite broad, in a sense, but it has no real
runtime cost over and above a more Pascal-ish language would force you to
have:

    if len([]) == 0: ...
    else: ...

Because True and False are singletons, the overhead of testing something in
a boolean context is no greater than the cost of testing the same condition
by hand. That is, it makes no difference whether you manually compare the
list's length to zero, or let its __nonzero__ or __bool__ method do the
same. (If anything, using an implicit boolean test will be faster than an
explicit manual test, because it doesn't have to call the len() global.)

(3) Method lookups don't *necessarily* have to involve object creation. The
only semantics which Python requires (so far as I understand it) are:

- Taking a reference to an unbound method:

    ref = str.upper

  may return the function object itself, which clearly already exists.

- Taking a reference to a bound method:

    ref = "some string".upper

  must return a method object, but it doesn't have to be recreated from
  scratch each and every time. It could cache it once, then always return
  that. That, I believe, is an implementation detail.

- Calling a method may bypass creating a method, and just use the 
  function object directly, provided Python knows that the descriptor
  has no side-effects.

For example, since FunctionType.__get__ has no side-effects, a Python
interpreter could special-case function objects and avoid calling the
descriptor protocol when it knows that the method is just going to be
called immediately.


But... having said all that... how do you know that these issues are
bottlenecks that eliminating them would speed up the code by any
significant amount?




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#90663

On Fri, May 15, 2015 at 8:14 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> (If anything, using an implicit boolean test will be faster than an
> explicit manual test, because it doesn't have to call the len() global.)

Even more so: Some objects may be capable of determining their own
lengths, but can ascertain their own emptiness more quickly. So len(x)
might have to chug chug chug to figure out exactly how many results
there are (imagine a database query or something), where bool(x)
merely has to see whether or not a single one exists (imagine a
database query with "LIMIT 1" tacked on).

ChrisA

[toc] | [prev] | [next] | [standalone]

#90692

On 5/15/2015 4:59 AM, Marko Rauhamaa wrote:
> Must a method lookup necessarily involve object creation?

Where is matters, inside loops, method lookup can be avoided after doing 
it once.

for i in range(1000000): ob.meth(i)

versus

meth = ob.meth
for i in range(1000000): meth(i)

For working with a single stack:

a = []
apush = stack.append
apop = stack.pop
# Now apush(x), apop(x), and test with 'if a:' as desired.
Being able to do this was part of my rational for adding list.pop, about 
15 years ago.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#90694

On 15/05/2015 09:59, Marko Rauhamaa wrote:

>> The path from decoding a bytecode to the C code that implements it can
>> be rather convoluted, but there are reasons for each of the
>> complications -- mainly to do with supporting the ability to override
>> operators with C and/or Python code.
>>
>> If you removed those abilities, the implemention would be simpler, and
>> possibly faster. But then the language wouldn't be Python any more.
>
> I agree that Python's raison-d'être is its dynamism and expressive
> power. It definitely shouldn't be sacrificed for performance.
>
> However, in some respects, Python might be going overboard with its
> dynamism; are all those dunder methods really needed? Must "false" be
> defined so broadly? Must a method lookup necessarily involve object
> creation?

What /is/ a method lookup? Is it when you have this:

  A.B()

and need to find whether the expression A (or its class or type) has a 
name B associated with it? (And it then needs to check whether B is 
something that can be called.)

If so, does that have to be done using Python's Dict mechanism? (Ie. 
searching for a key 'B' by name and seeing if the object associated with 
it is a method. That does not sound efficient.)

(And I guess Python's classes come into play so if B is not part of A's 
class then it might be part of some base-class. I can see that it can 
get complicated, but I don't use OO so can't speculate further.)

(In the language whose implementation I'm comparing with CPython's, it 
doesn't have classes. A.B() can still appear, but .B will need to be an 
attribute that the (bytecode) compiler already knows from a prior 
definition (usually, some struct or record if A is an expression).

If there is only one .B it knows, then a simple check that A is the 
correct type is all that is needed. Otherwise a runtime search through 
all .Bs (and a compact table will have been set up for this) is needed 
to find the .B that matches A's type. But this is all still pretty quick.

In this context, A.B will need to be some function pointer, and A.B() 
will call it.)

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#90696

BartC <bc@freeuk.com>:

> What /is/ a method lookup? Is it when you have this:
>
>  A.B()
>
> and need to find whether the expression A (or its class or type) has a
> name B associated with it? (And it then needs to check whether B is
> something that can be called.)
>
> If so, does that have to be done using Python's Dict mechanism? (Ie.
> searching for a key 'B' by name and seeing if the object associated
> with it is a method. That does not sound efficient.)

That is a general feature among high-level programming languages. In
Python, it is even more complicated:

 * first the object's dict is looked up for the method name

 * if the method is not found (it usually isn't), the dict of the
   object's class is consulted

 * if the method is found (it usually is), a function object is
   instantiated that delegates to the class's method and embeds a "self"
   reference to the object to the call

IOW, two dict lookups plus an object construction for each method call.


Marko

[toc] | [prev] | [next] | [standalone]

#90697

On 15/05/2015 23:44, Marko Rauhamaa wrote:
> BartC <bc@freeuk.com>:
>
>> What /is/ a method lookup? Is it when you have this:
>>
>>   A.B()
>>
>> and need to find whether the expression A (or its class or type) has a
>> name B associated with it? (And it then needs to check whether B is
>> something that can be called.)
>>
>> If so, does that have to be done using Python's Dict mechanism? (Ie.
>> searching for a key 'B' by name and seeing if the object associated
>> with it is a method. That does not sound efficient.)
>
> That is a general feature among high-level programming languages. In
> Python, it is even more complicated:
>
>   * first the object's dict is looked up for the method name
>
>   * if the method is not found (it usually isn't), the dict of the
>     object's class is consulted
>
>   * if the method is found (it usually is), a function object is
>     instantiated that delegates to the class's method and embeds a "self"
>     reference to the object to the call
>
> IOW, two dict lookups plus an object construction for each method call.
>
>
> Marko
>

As a picture paints a thousand words is anybody aware of a site or sites 
that show this diagramatically, as I think I and possibly others would 
find it far easier to grasp.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | comp.lang.python

csiph-web