Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #15438 > unrolled thread

all() is slow?

Started by"OKB (not okblacke)" <brenNOSPAMbarn@NObrenSPAMbarn.net>
First post2011-11-07 21:00 +0000
Last post2011-11-15 08:02 -0800
Articles 20 on this page of 43 — 18 participants

Back to article view | Back to comp.lang.python


Contents

  all() is slow? "OKB (not okblacke)" <brenNOSPAMbarn@NObrenSPAMbarn.net> - 2011-11-07 21:00 +0000
    Re: all() is slow? Chris Rebert <clp2@rebertia.com> - 2011-11-07 13:39 -0800
    Re: all() is slow? david vierra <codewarrior0@gmail.com> - 2011-11-07 13:46 -0800
      Re: all() is slow? Chris Angelico <rosuav@gmail.com> - 2011-11-08 09:06 +1100
        Re: all() is slow? Henrik Faber <hfaber@invalid.net> - 2011-11-08 13:09 +0100
          Re: all() is slow? Chris Angelico <rosuav@gmail.com> - 2011-11-08 23:14 +1100
          Re: all() is slow? John Posner <jjposner@optimum.net> - 2011-11-08 17:51 -0500
          Re: all() is slow? Tim Chase <python.list@tim.thechases.com> - 2011-11-08 17:56 -0600
      Re: all() is slow? Devin Jeanpierre <jeanpierreda@gmail.com> - 2011-11-08 19:44 -0500
        Re: all() is slow? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-11-09 02:47 +0000
          Re: all() is slow? Devin Jeanpierre <jeanpierreda@gmail.com> - 2011-11-09 18:01 -0500
            Re: all() is slow? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-11-09 23:11 +0000
              Re: all() is slow? Devin Jeanpierre <jeanpierreda@gmail.com> - 2011-11-09 20:26 -0500
                Re: all() is slow? alex23 <wuwei23@gmail.com> - 2011-11-09 19:50 -0800
                  Re: all() is slow? Devin Jeanpierre <jeanpierreda@gmail.com> - 2011-11-09 23:40 -0500
                  Re: all() is slow? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-11-10 07:35 +0000
                Re: all() is slow? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-11-10 07:48 +0000
                  Re: all() is slow? Devin Jeanpierre <jeanpierreda@gmail.com> - 2011-11-10 03:51 -0500
                  Re: all() is slow? gene heskett <gheskett@wdtv.com> - 2011-11-10 08:20 -0500
                  Re: all() is slow? Terry Reedy <tjreedy@udel.edu> - 2011-11-10 14:25 -0500
              Re: all() is slow? Devin Jeanpierre <jeanpierreda@gmail.com> - 2011-11-09 20:35 -0500
              Re: all() is slow? Ian Kelly <ian.g.kelly@gmail.com> - 2011-11-09 19:10 -0700
              Re: all() is slow? Ethan Furman <ethan@stoneleaf.us> - 2011-11-10 10:43 -0800
              Re: all() is slow? Devin Jeanpierre <jeanpierreda@gmail.com> - 2011-11-10 15:37 -0500
                Re: all() is slow? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-11-10 23:07 +0000
                  Re: all() is slow? Devin Jeanpierre <jeanpierreda@gmail.com> - 2011-11-10 23:35 -0500
                    Re: all() is slow? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-11-12 01:18 +0000
                      Re: all() is slow? Devin Jeanpierre <jeanpierreda@gmail.com> - 2011-11-13 01:28 -0500
                        Re: all() is slow? alex23 <wuwei23@gmail.com> - 2011-11-13 18:50 -0800
                          Re: all() is slow? Devin Jeanpierre <jeanpierreda@gmail.com> - 2011-11-13 23:48 -0500
                Re: all() is slow? "OKB (not okblacke)" <brenNOSPAMbarn@NObrenSPAMbarn.net> - 2011-11-11 05:40 +0000
              Re: all() is slow? Ian Kelly <ian.g.kelly@gmail.com> - 2011-11-10 14:19 -0700
                Re: all() is slow? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-11-10 22:56 +0000
              Re: all() is slow? Ethan Furman <ethan@stoneleaf.us> - 2011-11-10 13:47 -0800
          Re: all() is slow? Chris Angelico <rosuav@gmail.com> - 2011-11-10 10:15 +1100
      Re: all() is slow? Chris Angelico <rosuav@gmail.com> - 2011-11-09 12:19 +1100
      Re: all() is slow? Chris Rebert <clp2@rebertia.com> - 2011-11-08 17:30 -0800
        Re: all() is slow? Hans Mulder <hansmu@xs4all.nl> - 2011-11-09 16:41 +0100
          Re: all() is slow? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2011-11-09 09:07 -0800
    Re: all() is slow? John Nagle <nagle@animats.com> - 2011-11-09 14:16 -0800
      Re: all() is slow? alex23 <wuwei23@gmail.com> - 2011-11-09 19:52 -0800
      Re: all() is slow? "OKB (not okblacke)" <brenNOSPAMbarn@NObrenSPAMbarn.net> - 2011-11-10 19:35 +0000
    Re: all() is slow? BOOK-AZ <amalguseynov@gmail.com> - 2011-11-15 08:02 -0800

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →


#15521

FromDevin Jeanpierre <jeanpierreda@gmail.com>
Date2011-11-09 20:35 -0500
Message-ID<mailman.2581.1320888952.27778.python-list@python.org>
In reply to#15517
> Well. It reads fine in a certain sense, in that I can figure out
> what's going on (although I have some troubles figuring out why the
> heck certain things are in the code). The issue is that what's going
> on is otherworldly: this is not a Python pattern, this is not a normal
> approach. To me, that means it does not read fine.

Sorry for the double post, but I think I damaged my statement a
little in the edit process here. I have a hard time
reading the code, and the implications of using exec on how
initialization works is not obvious. I do believe that this solution
is significantly less readable than it would be in a language with
"real" macros, or using an alternate dict-based approach. I don't mean
to just say "it's different, therefore it's wrong" -- it takes more
than just being unusual (not that the unusualness helps).

Devin

On Wed, Nov 9, 2011 at 8:26 PM, Devin Jeanpierre <jeanpierreda@gmail.com> wrote:
>> Neither am I. I am less suspicious based on a reputation. Raymond is a
>> well-known, trusted senior Python developer who knows what he is doing.
>
> I don't really know anything about him or why people respect him, so I
> have no reason to share your faith.
>
>> It reads fine, and the justification is perfectly valid.
>
> Well. It reads fine in a certain sense, in that I can figure out
> what's going on (although I have some troubles figuring out why the
> heck certain things are in the code). The issue is that what's going
> on is otherworldly: this is not a Python pattern, this is not a normal
> approach. To me, that means it does not read fine.
>
> The use of exec also results in (seemingly) arbitrary constraints on
> the input. Like, why can't "--" be a name? Because exec? Is there some
> other reason?
>
> I don't like the use of exec, and I don't like the justification (it
> seems handwavy). I pointed this out in a thread full of people saying
> "never EVER use exec this way", so it's obviously not just me that
> thinks this is awful.
>
>> You're right to be cautious of exec. You're wrong to be phobic about it.
>> What do you think is going to happen?
>
> I think somebody will read it and think this is a good idea.
>
> Devin
>
>
> On Wed, Nov 9, 2011 at 6:11 PM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> On Wed, 09 Nov 2011 18:01:16 -0500, Devin Jeanpierre wrote:
>>
>>>> If it were someone other than Raymond Hettinger responsible for the use
>>>> of exec in namedtuple, I'd be a lot more suspicious of it.
>>>
>>> I'm not going to be less suspicious based on a name.
>>
>> Neither am I. I am less suspicious based on a reputation. Raymond is a
>> well-known, trusted senior Python developer who knows what he is doing.
>>
>>
>>> It reads like
>>> insanity, and the justification was terrible.
>>
>> It reads fine, and the justification is perfectly valid.
>>
>> You're right to be cautious of exec. You're wrong to be phobic about it.
>> What do you think is going to happen? The exec call inside namedtuple is
>> going to creep out of the module in the wee hours of the night,
>> contaminating other functions and modules while you sleep? Be serious. If
>> you have an actual concrete security vulnerability caused by the use of
>> exec inside namedtuple, or some other bug, then say so. Otherwise, your
>> paranoia is unjustified.
>>
>>
>>
>> --
>> Steven
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>>
>

[toc] | [prev] | [next] | [standalone]


#15523

FromIan Kelly <ian.g.kelly@gmail.com>
Date2011-11-09 19:10 -0700
Message-ID<mailman.2583.1320891059.27778.python-list@python.org>
In reply to#15517
On Wed, Nov 9, 2011 at 6:26 PM, Devin Jeanpierre <jeanpierreda@gmail.com> wrote:
> The use of exec also results in (seemingly) arbitrary constraints on
> the input. Like, why can't "--" be a name? Because exec? Is there some
> other reason?

That's by design, not because of exec.  The names are supposed to be
actual Python names, things that can used to designate keyword
arguments ("MyTuple(foo=42)") or to retrieve elements using attribute
lookup ("my_tuple.foo").  Using "--" as a name would be a syntax error
in either of those cases.

Cheers,
Ian

[toc] | [prev] | [next] | [standalone]


#15553

FromEthan Furman <ethan@stoneleaf.us>
Date2011-11-10 10:43 -0800
Message-ID<mailman.2611.1320951665.27778.python-list@python.org>
In reply to#15517
Devin Jeanpierre wrote:
> Well. It reads fine in a certain sense, in that I can figure out
> what's going on (although I have some troubles figuring out why the
> heck certain things are in the code). The issue is that what's going
> on is otherworldly: this is not a Python pattern, this is not a normal
> approach. To me, that means it does not read fine.

Certainly it's a Python pattern -- it's what you do to dynamically 
generate code.


> The use of exec also results in (seemingly) arbitrary constraints on
> the input. Like, why can't "--" be a name? Because exec? Is there some
> other reason?

'--' not being allowed for a name has *nothing* to do with exec, and 
everything to do with `--` not being a valid Python identifier.


 > '--' is a valid attribute name on virtually any object that supports
 > attribute setting (e.g. function objects). Of course, you need to use
 > setattr() and getattr(). Is this really the reason, or is it a
 > limitation caused primarily by the usage of exec and the need to
 > prevent code injection? If somebody added this feature later on, would
 > this create a security vulnerability in certain projects that used
 > namedtuple in certain ways?

So you think

     somevar = getattr(my_named_tuple, '--')

is more readable than

     somevar = my_named_tuple.spam

?

~Ethan~

[toc] | [prev] | [next] | [standalone]


#15559

FromDevin Jeanpierre <jeanpierreda@gmail.com>
Date2011-11-10 15:37 -0500
Message-ID<mailman.2621.1320957468.27778.python-list@python.org>
In reply to#15517
> '--' not being allowed for a name has *nothing* to do with exec, and
> everything to do with `--` not being a valid Python identifier.

The only reason valid python identifiers come into it at all is
because they get pasted into a string where identifiers would go, and
that string is passed to exec().

So really, does it have "nothing" to do with exec? Or does your
argument eventually boil down to the use of exec?

> is more readable than

Of course not. I do, however, think that it's conceivable that I'd
want to key a namedtuple by an invalid identifier, and to do that,
yes, I'd need to use getattr().

Devin

On Thu, Nov 10, 2011 at 1:43 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
> Devin Jeanpierre wrote:
>>
>> Well. It reads fine in a certain sense, in that I can figure out
>> what's going on (although I have some troubles figuring out why the
>> heck certain things are in the code). The issue is that what's going
>> on is otherworldly: this is not a Python pattern, this is not a normal
>> approach. To me, that means it does not read fine.
>
> Certainly it's a Python pattern -- it's what you do to dynamically generate
> code.
>
>
>> The use of exec also results in (seemingly) arbitrary constraints on
>> the input. Like, why can't "--" be a name? Because exec? Is there some
>> other reason?
>
> '--' not being allowed for a name has *nothing* to do with exec, and
> everything to do with `--` not being a valid Python identifier.
>
>
>> '--' is a valid attribute name on virtually any object that supports
>> attribute setting (e.g. function objects). Of course, you need to use
>> setattr() and getattr(). Is this really the reason, or is it a
>> limitation caused primarily by the usage of exec and the need to
>> prevent code injection? If somebody added this feature later on, would
>> this create a security vulnerability in certain projects that used
>> namedtuple in certain ways?
>
> So you think
>
>    somevar = getattr(my_named_tuple, '--')
>
> is more readable than
>
>    somevar = my_named_tuple.spam
>
> ?
>
> ~Ethan~
> --
> http://mail.python.org/mailman/listinfo/python-list
>

[toc] | [prev] | [next] | [standalone]


#15570

FromSteven D'Aprano <steve+comp.lang.python@pearwood.info>
Date2011-11-10 23:07 +0000
Message-ID<4ebc592b$0$29970$c3e8da3$5496439d@news.astraweb.com>
In reply to#15559
On Thu, 10 Nov 2011 15:37:05 -0500, Devin Jeanpierre wrote:

>> '--' not being allowed for a name has *nothing* to do with exec, and
>> everything to do with `--` not being a valid Python identifier.
> 
> The only reason valid python identifiers come into it at all is because
> they get pasted into a string where identifiers would go, and that
> string is passed to exec().

That is patently untrue. If you were implementing namedtuple without 
exec, you would still (or at least you *should*) prevent the user from 
passing invalid identifiers as attribute names. What's the point of 
allowing attribute names you can't actually *use* as attribute names?

You could remove the validation, allowing users to pass invalid field 
names, but that would be a lousy API. If you want field names that aren't 
valid identifiers, the right solution is a dict, not attributes.

Here's a re-implementation using a metaclass:

http://pastebin.com/AkG1gbGC

and a diff from the Python bug tracker removing exec from namedtuple:

http://bugs.python.org/file11608/new_namedtuples.diff


You will notice both of them keep the field name validation.


-- 
Steven

[toc] | [prev] | [next] | [standalone]


#15580

FromDevin Jeanpierre <jeanpierreda@gmail.com>
Date2011-11-10 23:35 -0500
Message-ID<mailman.2638.1320986176.27778.python-list@python.org>
In reply to#15570
> You will notice both of them keep the field name validation.

There are lots of reasons for that to be the case. To me, the most
likely one just seems to be that you don't want to remove more than
necessary when changing the way something works under the hood -- both
for compatibility reasons, and because you want the patch to get
accepted and want the least number of objectionable changes. I guess
you disagree.

> That is patently untrue. If you were implementing namedtuple without
> exec, you would still (or at least you *should*) prevent the user from
> passing invalid identifiers as attribute names. What's the point of
> allowing attribute names you can't actually *use* as attribute names?

Then why doesn't Python do this anywhere else? e.g. why can I
setattr(obj, 'a#b') when obj is any other mutable type?

I just don't believe this reasoning is what happened, I suppose. And
there's no way for you to convince me, or me to convince you, without
R. Hettinger stepping in here and verifying one side of the story.
This is subjective supposition, and that was supposed to be my most
objective opposition.

To go off on another tangent, though, I don't really understand how
you guys can think this is reasonable, though. I don't get this
philosophy of restricting inputs that would otherwise be perfectly
valid, just for concerns that you have that users might not share --
especially when it's valid everywhere else that the concept shows up.
It seems totally inconsistent with the attitude expressed above (by
you!) towards exec, which is that it's ok to be cautious of something,
but something else to forbid it outright.

Of course, as I said before, I don't agree with core python developers
on lots of things. I guess this is just another thing I just don't
understand.

Devin

On Thu, Nov 10, 2011 at 6:07 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> On Thu, 10 Nov 2011 15:37:05 -0500, Devin Jeanpierre wrote:
>
>>> '--' not being allowed for a name has *nothing* to do with exec, and
>>> everything to do with `--` not being a valid Python identifier.
>>
>> The only reason valid python identifiers come into it at all is because
>> they get pasted into a string where identifiers would go, and that
>> string is passed to exec().
>
> That is patently untrue. If you were implementing namedtuple without
> exec, you would still (or at least you *should*) prevent the user from
> passing invalid identifiers as attribute names. What's the point of
> allowing attribute names you can't actually *use* as attribute names?
>
> You could remove the validation, allowing users to pass invalid field
> names, but that would be a lousy API. If you want field names that aren't
> valid identifiers, the right solution is a dict, not attributes.
>
> Here's a re-implementation using a metaclass:
>
> http://pastebin.com/AkG1gbGC
>
> and a diff from the Python bug tracker removing exec from namedtuple:
>
> http://bugs.python.org/file11608/new_namedtuples.diff
>
>
> You will notice both of them keep the field name validation.
>
>
> --
> Steven
> --
> http://mail.python.org/mailman/listinfo/python-list
>

[toc] | [prev] | [next] | [standalone]


#15618

FromSteven D'Aprano <steve+comp.lang.python@pearwood.info>
Date2011-11-12 01:18 +0000
Message-ID<4ebdc961$0$29970$c3e8da3$5496439d@news.astraweb.com>
In reply to#15580
On Thu, 10 Nov 2011 23:35:32 -0500, Devin Jeanpierre wrote:

>> That is patently untrue. If you were implementing namedtuple without
>> exec, you would still (or at least you *should*) prevent the user from
>> passing invalid identifiers as attribute names. What's the point of
>> allowing attribute names you can't actually *use* as attribute names?
> 
> Then why doesn't Python do this anywhere else? e.g. why can I
> setattr(obj, 'a#b') when obj is any other mutable type?

That is implementation-specific behaviour and not documented behaviour 
for Python. If you try it in (say) IronPython or Jython, you may or may 
not see the same behaviour.

The docs for getattr state:

    getattr(x, 'foobar') is equivalent to x.foobar


which implies that getattr(x, 'a!b') should be equivalent to x.a!b which 
will give a syntax error. The fact that CPython does less validation is 
arguably a bug and not something that you should rely on: it is *not* a 
promise of the language.

As Terry Reedy already mentioned, the namespace used in classes and 
instances are ordinary generic dicts, which don't perform any name 
validation. That's done for speed. Other implementations may use 
namespaces that enforce legal names for attributes, and __slots__ already 
does:

>>> class X(object):
...     __slots__ = ['a', 'b!']
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Error when calling the metaclass bases
    __slots__ must be identifiers


[...]
> To go off on another tangent, though, I don't really understand how you
> guys can think this is reasonable, though. I don't get this philosophy
> of restricting inputs that would otherwise be perfectly valid

But they aren't perfectly valid. They are invalid inputs. Just because 
getattr and setattr in CPython allow you to create attributes with 
invalid names doesn't mean that everything else should be equally as 
slack.


-- 
Steven

[toc] | [prev] | [next] | [standalone]


#15635

FromDevin Jeanpierre <jeanpierreda@gmail.com>
Date2011-11-13 01:28 -0500
Message-ID<mailman.2677.1321165756.27778.python-list@python.org>
In reply to#15618
> which implies that getattr(x, 'a!b') should be equivalent to x.a!b

No, it does not. The documentation states equivalence for two
particular values, and there is no way to deduce truth for all cases
from that. In fact, if it _was_ trying to say it was true for any
attribute value, then your example would be proof that the
documentation is incorrect, since CPython breaks that equivalence.

Devin

On Fri, Nov 11, 2011 at 8:18 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> On Thu, 10 Nov 2011 23:35:32 -0500, Devin Jeanpierre wrote:
>
>>> That is patently untrue. If you were implementing namedtuple without
>>> exec, you would still (or at least you *should*) prevent the user from
>>> passing invalid identifiers as attribute names. What's the point of
>>> allowing attribute names you can't actually *use* as attribute names?
>>
>> Then why doesn't Python do this anywhere else? e.g. why can I
>> setattr(obj, 'a#b') when obj is any other mutable type?
>
> That is implementation-specific behaviour and not documented behaviour
> for Python. If you try it in (say) IronPython or Jython, you may or may
> not see the same behaviour.
>
> The docs for getattr state:
>
>    getattr(x, 'foobar') is equivalent to x.foobar
>
>
> which implies that getattr(x, 'a!b') should be equivalent to x.a!b which
> will give a syntax error. The fact that CPython does less validation is
> arguably a bug and not something that you should rely on: it is *not* a
> promise of the language.
>
> As Terry Reedy already mentioned, the namespace used in classes and
> instances are ordinary generic dicts, which don't perform any name
> validation. That's done for speed. Other implementations may use
> namespaces that enforce legal names for attributes, and __slots__ already
> does:
>
>>>> class X(object):
> ...     __slots__ = ['a', 'b!']
> ...
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> TypeError: Error when calling the metaclass bases
>    __slots__ must be identifiers
>
>
> [...]
>> To go off on another tangent, though, I don't really understand how you
>> guys can think this is reasonable, though. I don't get this philosophy
>> of restricting inputs that would otherwise be perfectly valid
>
> But they aren't perfectly valid. They are invalid inputs. Just because
> getattr and setattr in CPython allow you to create attributes with
> invalid names doesn't mean that everything else should be equally as
> slack.
>
>
> --
> Steven
> --
> http://mail.python.org/mailman/listinfo/python-list
>

[toc] | [prev] | [next] | [standalone]


#15655

Fromalex23 <wuwei23@gmail.com>
Date2011-11-13 18:50 -0800
Message-ID<2b46d46c-f66d-4a13-8c0b-af9b6c638863@h30g2000pro.googlegroups.com>
In reply to#15635
On Nov 13, 4:28 pm, Devin Jeanpierre <jeanpierr...@gmail.com> wrote:
> > which implies that getattr(x, 'a!b') should be equivalent to x.a!b
>
> No, it does not. The documentation states equivalence for two
> particular values

It states equivalence for two values _based on the name_.

"If the string is the name of one of the object’s attributes, the
result is the value of that attribute. For example, getattr(x,
'foobar') is equivalent to x.foobar."

The string 'a!b' is the name of the attribute, ergo getattr(x, 'a!b')
_is_ x.a!b. If x.a!b isn't valid CPython, then etc.

> CPython breaks that equivalence

So you're outright ignoring the comments that this behaviour is to
make CPython more performant?

[toc] | [prev] | [next] | [standalone]


#15656

FromDevin Jeanpierre <jeanpierreda@gmail.com>
Date2011-11-13 23:48 -0500
Message-ID<mailman.2691.1321246170.27778.python-list@python.org>
In reply to#15655
> It states equivalence for two values _based on the name_.

I don't know what you mean. "Based on the name" doesn't mean anything
in particular to me in this context.

> So you're outright ignoring the comments that this behaviour is to
> make CPython more performant?

I don't see how I'm ignoring the comment. Yes, breaking the spec
improves performance. Is that a reason to not fix the spec, or
something?

Devin

On Sun, Nov 13, 2011 at 9:50 PM, alex23 <wuwei23@gmail.com> wrote:
> On Nov 13, 4:28 pm, Devin Jeanpierre <jeanpierr...@gmail.com> wrote:
>> > which implies that getattr(x, 'a!b') should be equivalent to x.a!b
>>
>> No, it does not. The documentation states equivalence for two
>> particular values
>
> It states equivalence for two values _based on the name_.
>
> "If the string is the name of one of the object’s attributes, the
> result is the value of that attribute. For example, getattr(x,
> 'foobar') is equivalent to x.foobar."
>
> The string 'a!b' is the name of the attribute, ergo getattr(x, 'a!b')
> _is_ x.a!b. If x.a!b isn't valid CPython, then etc.
>
>> CPython breaks that equivalence
>
> So you're outright ignoring the comments that this behaviour is to
> make CPython more performant?
> --
> http://mail.python.org/mailman/listinfo/python-list
>

[toc] | [prev] | [next] | [standalone]


#15584

From"OKB (not okblacke)" <brenNOSPAMbarn@NObrenSPAMbarn.net>
Date2011-11-11 05:40 +0000
Message-ID<Xns9F99DC7F2A816OKB@88.198.244.100>
In reply to#15559
Devin Jeanpierre wrote:

>> '--' not being allowed for a name has *nothing* to do with exec, and
>> everything to do with `--` not being a valid Python identifier. 
> 
> The only reason valid python identifiers come into it at all is
> because they get pasted into a string where identifiers would go, and
> that string is passed to exec().

    	The whole point of named tuples is to be able to access the members 
via attribute access as in "obj.attr".  Things like "obj.--" are not 
valid Python syntax, so you can't use "--" as the name of a namedtuple 
field.  Yes, you can do "getattr(obj, '--')" if you want, but it's quite 
reasonable for namedtuple to refrain from catering to that sort of 
perverse usage.

-- 
--OKB (not okblacke)
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is
no path, and leave a trail."
	--author unknown

[toc] | [prev] | [next] | [standalone]


#15562

FromIan Kelly <ian.g.kelly@gmail.com>
Date2011-11-10 14:19 -0700
Message-ID<mailman.2624.1320959990.27778.python-list@python.org>
In reply to#15517
On Thu, Nov 10, 2011 at 1:37 PM, Devin Jeanpierre
<jeanpierreda@gmail.com> wrote:
> Of course not. I do, however, think that it's conceivable that I'd
> want to key a namedtuple by an invalid identifier, and to do that,
> yes, I'd need to use getattr().

Care to give a real use case?  You could even go a step further and
use, say, arbitrary ints as names if you're willing to give up
getattr() and use "ob.__class__.__dict__[42].__get__(ob,
ob.__class__)" everywhere instead.  The fact that somebody might
conceivably want to do this doesn't make it a good idea, though.

I do find it a bit funny that you're criticizing a somewhat smelly
implementation detail by complaining that it doesn't support an
equally smelly feature.

Cheers,
Ian

[toc] | [prev] | [next] | [standalone]


#15569

FromSteven D'Aprano <steve+comp.lang.python@pearwood.info>
Date2011-11-10 22:56 +0000
Message-ID<4ebc56a9$0$29970$c3e8da3$5496439d@news.astraweb.com>
In reply to#15562
On Thu, 10 Nov 2011 14:19:18 -0700, Ian Kelly wrote:

> On Thu, Nov 10, 2011 at 1:37 PM, Devin Jeanpierre
> <jeanpierreda@gmail.com> wrote:
>> Of course not. I do, however, think that it's conceivable that I'd want
>> to key a namedtuple by an invalid identifier, and to do that, yes, I'd
>> need to use getattr().
> 
> Care to give a real use case?  

A common use-case is for accessing fields from an external data source, 
using the same field names. For example, you might have a database with a 
field called "class", or a CSV file with columns "0-10", "11-20", etc.

Personally, I wouldn't bother using attributes to access fields, I'd use 
a dict, but some people think it's important to use attribute access.


> You could even go a step further and use,
> say, arbitrary ints as names if you're willing to give up getattr() and
> use "ob.__class__.__dict__[42].__get__(ob, ob.__class__)" everywhere
> instead.  The fact that somebody might conceivably want to do this
> doesn't make it a good idea, though.

Obviously you would write a helper function rather than repeat that mess 
in-line everywhere.


-- 
Steven

[toc] | [prev] | [next] | [standalone]


#15568

FromEthan Furman <ethan@stoneleaf.us>
Date2011-11-10 13:47 -0800
Message-ID<mailman.2628.1320965496.27778.python-list@python.org>
In reply to#15517
Devin Jeanpierre wrote:
> The only reason valid python identifiers come into it at all is
> because they get pasted into a string where identifiers would go, and
> that string is passed to exec().
> 
> So really, does it have "nothing" to do with exec? Or does your
> argument eventually boil down to the use of exec?

As I recall the big reason for namedtuples was things like

     sys.version_info[1]  # behind door number one is...

being much more readable as

     sys.version_info.minor

In other words, the tuple offsets are named -- hence, namedtuples.  And 
only valid identifiers will work.

So, no, it has nothing to do with 'exec', and everything to do with the 
problem namedtuple was designed to solve.

~Ethan~

[toc] | [prev] | [next] | [standalone]


#15518

FromChris Angelico <rosuav@gmail.com>
Date2011-11-10 10:15 +1100
Message-ID<mailman.2578.1320880510.27778.python-list@python.org>
In reply to#15493
On Thu, Nov 10, 2011 at 10:01 AM, Devin Jeanpierre
<jeanpierreda@gmail.com> wrote:
>> If it were someone other than Raymond Hettinger responsible for the use
>> of exec in namedtuple, I'd be a lot more suspicious of it.
>
> I'm not going to be less suspicious based on a name. It reads like
> insanity, and the justification was terrible.

It's said that code exists foremost for humans to read, and only
incidentally for computers to execute. I believe that this is inverted
for library code; as long as the _interface_ is clean, you can get
away with some messy internals, because it's going to be executed far
more often than actually read. Python programmers can use namedtuples
happily without knowing that the implementation uses exec.

The justification is, if I understand correctly, that the alternative
is worse. That's plenty of justification imho.

ChrisA

[toc] | [prev] | [next] | [standalone]


#15490

FromChris Angelico <rosuav@gmail.com>
Date2011-11-09 12:19 +1100
Message-ID<mailman.2560.1320801585.27778.python-list@python.org>
In reply to#15442
On Wed, Nov 9, 2011 at 11:44 AM, Devin Jeanpierre
<jeanpierreda@gmail.com> wrote:
> Clearly what we need is a modern hygienic macro system to avoid this exec mess!
>
> defmacro defall(name, cond):
>    def name(lst):
>        for a in lst:
>            if not cond:
>                return False
>        return True

#define defall(name,cond) def name(lst): \
    for a in lst: \
        if not cond: return False \
    return True

gcc -E myprog.pyi -o myprog.py

There's no code you can't make more opaque using the C preprocessor.

Python doesn't have inline functions? Ha! In your FACE, evil Python
development cabal! You can't tell ME what I can't do!

ChrisA
PS. Don't try this at home. We have years of experience with bad code.
Don't write code like this unless you have a life insurance policy
that covers angry mobs.

[toc] | [prev] | [next] | [standalone]


#15491

FromChris Rebert <clp2@rebertia.com>
Date2011-11-08 17:30 -0800
Message-ID<mailman.2561.1320802252.27778.python-list@python.org>
In reply to#15442
On Tue, Nov 8, 2011 at 5:19 PM, Chris Angelico <rosuav@gmail.com> wrote:
> On Wed, Nov 9, 2011 at 11:44 AM, Devin Jeanpierre
> <jeanpierreda@gmail.com> wrote:
>> Clearly what we need is a modern hygienic macro system to avoid this exec mess!
>>
>> defmacro defall(name, cond):
>>    def name(lst):
>>        for a in lst:
>>            if not cond:
>>                return False
>>        return True
>
> #define defall(name,cond) def name(lst): \
>    for a in lst: \
>        if not cond: return False \
>    return True
>
> gcc -E myprog.pyi -o myprog.py
>
> There's no code you can't make more opaque using the C preprocessor.
>
> Python doesn't have inline functions? Ha! In your FACE, evil Python
> development cabal! You can't tell ME what I can't do!
>
> ChrisA
> PS. Don't try this at home. We have years of experience with bad code.
> Don't write code like this unless you have a life insurance policy
> that covers angry mobs.

Burn him! Witch! Witch! Burn him!
His code turned me into a newt!
--
Sent nailed to a coconut carried by swallow.

[toc] | [prev] | [next] | [standalone]


#15498

FromHans Mulder <hansmu@xs4all.nl>
Date2011-11-09 16:41 +0100
Message-ID<4eba9f14$0$6987$e4fe514c@news2.news.xs4all.nl>
In reply to#15491
On 9/11/11 02:30:48, Chris Rebert wrote:

> Burn him! Witch! Witch! Burn him!
> His code turned me into a newt!
> --
> Sent nailed to a coconut carried by swallow.

Is that a European swallow or an African swallow?

-- HansM

[toc] | [prev] | [next] | [standalone]


#15501

FromDennis Lee Bieber <wlfraed@ix.netcom.com>
Date2011-11-09 09:07 -0800
Message-ID<mailman.2570.1320858482.27778.python-list@python.org>
In reply to#15498
	Feeling perverse..

On Wed, 09 Nov 2011 16:41:08 +0100, Hans Mulder <hansmu@xs4all.nl>
declaimed the following in gmane.comp.python.general:

> On 9/11/11 02:30:48, Chris Rebert wrote:
> 
> > Burn him! Witch! Witch! Burn him!
> > His code turned me into a newt!
> > --
> > Sent nailed to a coconut carried by swallow.
> 
> Is that a European swallow or an African swallow?
> 

	Both... International coconut carrying rules require the swallows to
hand over messages at the Strait of Gibraltar...
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
        wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]


#15515

FromJohn Nagle <nagle@animats.com>
Date2011-11-09 14:16 -0800
Message-ID<4ebafbb7$0$1724$742ec2ed@news.sonic.net>
In reply to#15438
On 11/7/2011 1:00 PM, OKB (not okblacke) wrote:
>      	I noticed this (Python 2.6.5 on Windows XP):

     CPython is slow. It's a naive interpreter.  There's
almost no optimization during compilation.  Try PyPy
or Shed Skin.

				John Nagle

[toc] | [prev] | [next] | [standalone]


Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | comp.lang.python


csiph-web