Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!weretis.net!feeder1.news.weretis.net!feeder.erje.net!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
From: gene heskett <gheskett@wdtv.com>
To: python-list@python.org
Subject: Re: all() is slow?
Date: Thu, 10 Nov 2011 08:20:54 -0500
User-Agent: KMail/1.13.7 (Linux/2.6.38.8-pclos2.pae.bfs; KDE/4.6.5; i686; ; )
References: <Xns9F9684761E3AOKB@88.198.244.100> <4ebb81e1$0$29988$c3e8da3$5496439d@news.astraweb.com> <CABicbJ+0EW1gNyFzVuhFkydAvHxrZCco-+CNK2h9LbxXbU4e6A@mail.gmail.com>
In-Reply-To: <CABicbJ+0EW1gNyFzVuhFkydAvHxrZCco-+CNK2h9LbxXbU4e6A@mail.gmail.com>
MIME-Version: 1.0
Content-Type: Text/Plain; charset="windows-1256"
Content-Transfer-Encoding: 7bit
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.2594.1320931266.27778.python-list@python.org>
Lines: 193
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:15535

On Thursday, November 10, 2011 08:13:13 AM Devin Jeanpierre did opine:

> > I don't expect you to take my word on it (and why should you, I could
> > be an idiot or a sock-puppet), but you could always try googling for
> > "Raymond Hettinger python" and see what comes up. He is not some
> > fly-by Python coder who snuck some dubious n00b code into the
> > standard library when no-one was looking :)
> 
> Alright, I know *something* about him. I knew he was a core developer,
> and that he was responsible for namedtuple. I also discovered (when I
> looked up his activestate profile) some other stuff he wrote. I don't
> really know anything about him outside of that -- i.e. I have no idea
> what parts of Python he's contributed things to in the past that could
> make me go, "oh, wow, _he_ did that?" and so on. I don't really feel
> like a few minutes research would give me the right feel, it generally
> has to come up organically.
> 
> Anyway, if we step back, for a trustworthy developer who wrote
> something seemingly-crazy, I should be willing to suspend judgement
> until I see the relevant facts about something that the developer
> might have and I don't. But he did give the facts,
> ( http://bugs.python.org/issue3974 again) , and I'm not convinced.
> 
> Things can go terribly wrong when abusing exec e.g.
> http://www.gossamer-threads.com/lists/python/bugs/568206 . That
> shouldn't ever happen with a function such as this. exec opens doors
> that should not be opened without a really good reason, and those
> reasons don't strike me that way.
 
If, in the sense that this python 'exec' essentially duplicates the bash 
version, then I have found it quite useful, it was taught to me several 
years ago by another teacher who was a long time Solaris fan, and if it 
were to go away, I have several bash scripts running here right now that 
would require major re-writes.

> > The mere fact that it was accepted into the standard library should
> > tell you that core Python developers consider it an acceptable
> > technique.

Well, its certainly not a new concept.  All the major 'shell interpreters' 
have it, why not python?

> I've seen core developers rail against the namedtuple source code. In
> fairness, I don't believe exec was the subject of the rant --
> nonetheless its presence isn't evidence of general support, and even
> if it were, my tastes have always differed from that of the core
> developers.
> 
> > That's not to say the technique is uncontroversial. But there are
> > still people who dislike "x if flag else y" and @decorator syntax --
> > controversy, in and of itself, isn't necessarily a reason to avoid
> > certain idioms.
> 
> I think there's somewhat a difference in magnitude of objections
> between using exec as a hacked-together macro system, and using "x if
> flag else y" when if statements would do.
> 
> If the exec trick is reasonable, we should normalize it in the form of
> a real, useful macro system, that can protect us against exec's many
> flaws (code injection, accidental syntax errors, etc.) and tell future
> programmers how to do this safely and in a semi-approvable way.
> 
> > I would agree that the use of exec is a code smell. But that doesn't
> > mean it is wrong or bad, merely that it needs a second look before
> > accepting it. There's a world of difference between "You MUST NOT use
> > exec" and "You SHOULD NOT use exec".
> 
> Do I really need a second look? I see exec, I wonder what it's doing.
> It isn't doing anything that couldn't be done subjectively better with
> e.g. a dict, so I disapprove of the usage of exec.
> 
> > There's nothing inside the template being exec'ed that couldn't be
> > found in non-exec code. So if you're having trouble figuring out
> > parts of the code, the presence of the exec is not the problem.
> 
> There's more overhead going back and forth to the template, and
> there's related things that I can't be sure are because of exec or
> because of design decisions, etc.  It makes code reading more
> challenging, even if it's still possible. That said, sure, some of
> these are problems with whatever else he's done.
> 
> > Having said that, dynamic code generation is well known for often
> > being harder to read than "ordinary" code. But then, pointers are
> > hard too.
> 
> And on the other other hand, Python lacks explicit support for both
> pointers and code generation (unless you count strings and ctypes).
> 
> > Because Python doesn't allow "--" to be an attribute name, and so
> > namedtuple doesn't let you try:
> > 
> > t = namedtuple("T", "foo -- bar")(1, 2, 3)
> > print(t.foo)
> > print(t.--)
> > print(t.bar)
> 
> '--' is a valid attribute name on virtually any object that supports
> attribute setting (e.g. function objects). Of course, you need to use
> setattr() and getattr(). Is this really the reason, or is it a
> limitation caused primarily by the usage of exec and the need to
> prevent code injection? If somebody added this feature later on, would
> this create a security vulnerability in certain projects that used
> namedtuple in certain ways?
> 
> Devin
> 
> On Thu, Nov 10, 2011 at 2:48 AM, Steven D'Aprano
> 
> <steve+comp.lang.python@pearwood.info> wrote:
> > On Wed, 09 Nov 2011 20:26:56 -0500, Devin Jeanpierre wrote:
> >>> Neither am I. I am less suspicious based on a reputation. Raymond is
> >>> a well-known, trusted senior Python developer who knows what he is
> >>> doing.
> >> 
> >> I don't really know anything about him or why people respect him, so
> >> I have no reason to share your faith.
> > 
> > That's fine.
> > 
> > I don't expect you to take my word on it (and why should you, I could
> > be an idiot or a sock-puppet), but you could always try googling for
> > "Raymond Hettinger python" and see what comes up. He is not some
> > fly-by Python coder who snuck some dubious n00b code into the
> > standard library when no-one was looking :)
> > 
> > The mere fact that it was accepted into the standard library should
> > tell you that core Python developers consider it an acceptable
> > technique. That's not to say the technique is uncontroversial. But
> > there are still people who dislike "x if flag else y" and @decorator
> > syntax -- controversy, in and of itself, isn't necessarily a reason
> > to avoid certain idioms.
> > 
> > 
> > Are you familiar with the idea of "code smell"?
> > 
> > http://www.codinghorror.com/blog/2006/05/code-smells.html
> > http://www.joelonsoftware.com/articles/Wrong.html
> > 
> > I would agree that the use of exec is a code smell. But that doesn't
> > mean it is wrong or bad, merely that it needs a second look before
> > accepting it. There's a world of difference between "You MUST NOT use
> > exec" and "You SHOULD NOT use exec".
> > 
> > See RFC 2119 if you are unclear on the difference:
> > 
> > http://www.ietf.org/rfc/rfc2119.txt
> > 
> >>> It reads fine, and the justification is perfectly valid.
> >> 
> >> Well. It reads fine in a certain sense, in that I can figure out
> >> what's going on (although I have some troubles figuring out why the
> >> heck certain things are in the code). The issue is that what's going
> >> on is otherworldly: this is not a Python pattern, this is not a
> >> normal approach. To me, that means it does not read fine.
> > 
> > There's nothing inside the template being exec'ed that couldn't be
> > found in non-exec code. So if you're having trouble figuring out
> > parts of the code, the presence of the exec is not the problem.
> > 
> > Having said that, dynamic code generation is well known for often
> > being harder to read than "ordinary" code. But then, pointers are
> > hard too.
> > 
> >> The use of exec also results in (seemingly) arbitrary constraints on
> >> the input. Like, why can't "--" be a name? Because exec? Is there
> >> some other reason?
> > 
> > Because Python doesn't allow "--" to be an attribute name, and so
> > namedtuple doesn't let you try:
> > 
> > t = namedtuple("T", "foo -- bar")(1, 2, 3)
> > print(t.foo)
> > print(t.--)
> > print(t.bar)
> > 
> > 
> > 
> > 
> > --
> > Steven
> > --
> > http://mail.python.org/mailman/listinfo/python-list


Cheers, Gene
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
My web page: <http://coyoteden.dyndns-free.com:85/gene>
>Ever heard of .cshrc?
That's a city in Bosnia.  Right?
(Discussion in comp.os.linux.misc on the intuitiveness of commands.)