Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #82076 > unrolled thread

Re: Python Iterables struggling using map() built-in

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Python Iterables struggling using map() built-in Chris Angelico <rosuav@gmail.com> - 2014-12-08 05:29 +1100
    Re: Python Iterables struggling using map() built-in Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-12-08 10:33 +1100
      Re: Python Iterables struggling using map() built-in Chris Angelico <rosuav@gmail.com> - 2014-12-08 10:50 +1100
        Re: Python Iterables struggling using map() built-in Roy Smith <roy@panix.com> - 2014-12-07 19:27 -0500
          Re: Python Iterables struggling using map() built-in Chris Angelico <rosuav@gmail.com> - 2014-12-08 11:35 +1100
            Re: Python Iterables struggling using map() built-in Steven D'Aprano <steve@pearwood.info> - 2014-12-08 09:40 +0000
              Re: Python Iterables struggling using map() built-in Chris Angelico <rosuav@gmail.com> - 2014-12-08 20:50 +1100
              Re: Python Iterables struggling using map() built-in Roy Smith <roy@panix.com> - 2014-12-08 19:02 -0500
      Re: Python Iterables struggling using map() built-in Roy Smith <roy@panix.com> - 2014-12-07 19:12 -0500
        Re: Python Iterables struggling using map() built-in Chris Angelico <rosuav@gmail.com> - 2014-12-08 11:34 +1100
          Re: Python Iterables struggling using map() built-in Roy Smith <roy@panix.com> - 2014-12-07 19:45 -0500
            Re: Python Iterables struggling using map() built-in Chris Angelico <rosuav@gmail.com> - 2014-12-08 12:00 +1100
            Re: Python Iterables struggling using map() built-in MRAB <python@mrabarnett.plus.com> - 2014-12-08 03:09 +0000
        Re: Python Iterables struggling using map() built-in Terry Reedy <tjreedy@udel.edu> - 2014-12-07 19:38 -0500
          Re: Python Iterables struggling using map() built-in Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-12-08 18:06 +1300
        Re: Python Iterables struggling using map() built-in Ned Batchelder <ned@nedbatchelder.com> - 2014-12-07 20:06 -0500
        Re: Python Iterables struggling using map() built-in Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-12-09 13:50 +1100
          Re: Python Iterables struggling using map() built-in Terry Reedy <tjreedy@udel.edu> - 2014-12-09 00:03 -0500
            Re: Python Iterables struggling using map() built-in Steven D'Aprano <steve@pearwood.info> - 2014-12-09 06:40 +0000
          Re: Python Iterables struggling using map() built-in Terry Reedy <tjreedy@udel.edu> - 2014-12-09 15:06 -0500
            Re: Python Iterables struggling using map() built-in Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-12-10 11:10 +1100
              Re: Python Iterables struggling using map() built-in Chris Angelico <rosuav@gmail.com> - 2014-12-10 11:20 +1100
              Re: Python Iterables struggling using map() built-in Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-12-10 00:24 +0000
              Re: Python Iterables struggling using map() built-in Roy Smith <roy@panix.com> - 2014-12-09 21:44 -0500
                Re: Python Iterables struggling using map() built-in Steven D'Aprano <steve@pearwood.info> - 2014-12-10 09:24 +0000
                  Re: Python Iterables struggling using map() built-in Chris Angelico <rosuav@gmail.com> - 2014-12-10 20:32 +1100
                  Re: Python Iterables struggling using map() built-in Marko Rauhamaa <marko@pacujo.net> - 2014-12-10 12:58 +0200
                  Re: Python Iterables struggling using map() built-in Ian Kelly <ian.g.kelly@gmail.com> - 2014-12-10 09:46 -0700
                    Re: Python Iterables struggling using map() built-in Steven D'Aprano <steve@pearwood.info> - 2014-12-11 04:01 +0000
                      Re: Python Iterables struggling using map() built-in Ian Kelly <ian.g.kelly@gmail.com> - 2014-12-11 15:00 -0700
                        Re: Python Iterables struggling using map() built-in Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-12-12 10:28 +1100
                          Re: Python Iterables struggling using map() built-in Ian Kelly <ian.g.kelly@gmail.com> - 2014-12-11 16:57 -0700
                            Re: Python Iterables struggling using map() built-in Roy Smith <roy@panix.com> - 2014-12-11 20:55 -0500
                              Re: Python Iterables struggling using map() built-in Ian Kelly <ian.g.kelly@gmail.com> - 2014-12-12 00:36 -0700
                          Re: Python Iterables struggling using map() built-in Chris Angelico <rosuav@gmail.com> - 2014-12-12 11:44 +1100
                  Re: Python Iterables struggling using map() built-in Terry Reedy <tjreedy@udel.edu> - 2014-12-10 12:48 -0500
                  Re: Python Iterables struggling using map() built-in Ian Kelly <ian.g.kelly@gmail.com> - 2014-12-10 13:32 -0700
                  Re: Python Iterables struggling using map() built-in Terry Reedy <tjreedy@udel.edu> - 2014-12-10 23:17 -0500
      Re: Python Iterables struggling using map() built-in Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-12-08 18:00 +1300

Page 2 of 2 — ← Prev page 1 [2]

#82244

Terry Reedy wrote:

> On 12/9/2014 12:03 AM, Terry Reedy wrote:
>>> Roy Smith wrote:
>>>
>>>> Chris Angelico wrote:
>>
>>>>> def myzip(*args):
>>>>>      iters = map(iter, args)
>>>>>      while iters:
>>>>>          res = [next(i) for i in iters]
>>>>>          yield tuple(res)
>>>>
>>>> Ugh.  When I see "while foo", my brain says, "OK, you're about to see a
>>>> loop which is controlled by the value of foo being changed inside the
>>>> loop".
> 
> What is nasty to me is that to understand the loop, one must do a whole
> program analysis to determine both that 'iters' is not rebound and that
> the list it is bound to is not mutated.

When people say "whole program analysis", they usually mean the entire
application including all its modules and libraries, not a four line
function, excluding the def header. (Or three lines if you get rid of the
unnecessary temporary variable 'res'.) Did it take you a long time to read
all two lines of the while loop to determine that iters is not modified or
rebound?

I really think you guys are trying too hard to make this function seem more
complicated than it is. If you find it so hard to understand a simple
function with four short lines, one wonders how you would possibly cope
with real code. Purely by coincidence, I have the source to the "pyclbr"
module from the standard library open in a text editor. I see a _readmodule
function that looks, in part, like this:


    try:
         for tokentype, token, start, _end, _line in g:
             if ...
                 while ...
             elif ...
                 while..
                 if ...
                 if ...
                     if ...
                 else ...
             elif ...
                 while ...
                 if ...
                 if ...
                     while ...
                         if ...
                             if ...
                             else ...
                                 if ...
                                     if ...
                                         if ...
                         if ...
                         elif ...
                             if ...
                         elif ...
                         elif ...


at which point I'm about halfway through the try block and I'm giving up.

https://hg.python.org/cpython/file/3.4/Lib/pyclbr.py

This, presumably, is good enough for the standard library, but the four line
version of zip is supposed to be too hard for mortal man to comprehend.
That's funny :-)


> To do the later, one must not 
> only read the loop body, but also preceding code to make sure the list
> is not aliased.

The preceding code is exactly *one* line, a single assignment binding the
name iters to the list. The while loop body is exactly two lines, one if
you dump the unnecessary 'res' temporary variable:

    yield tuple([next(i) for i in iters])


Quite frankly Terry, I do not believe for a second that somebody like you
who can successfully maintain IDLE is struggling to understand this myzip()
function.

Wait... is this like the Four Yorkshire Men sketch from Monty Python, only
instead of complaining about how hard you had it as children, you're all
trying to outdo each other about how difficult you find it to read this
function? If so, well done, you really had me for a while.



> Once the logic is clear and 'localized', even a simple compiler like
> CPython's can see that this is a loop-forever construct and that the
> loop test is unnecessary.  So it can be removed.

Ah, now that's nice. You're suggesting that by moving the loop condition
outside of the while statement, the compiler can generate more efficient
byte code. It only needs to test iters once, not at the start of every
loop.

That is the first interesting argument I've seen so far!

On the one hand, as micro-optimizations go, it will be pretty micro.
Particularly compared to the cost of starting and stopping a generator, I
doubt that will save any meaningful time, at least not enough to make up
for the extra effort in having to read and comprehend one more line of
code.

On the other hand, *premature optimization*. In general, one shouldn't write
more complex code so the compiler can optimize it, one should write simpler
code and have a smarter compiler. If *we* are capable of recognising that
iters is not modified in the body of the loop, then the compiler should be
capable of it too. (If it isn't, it is because nobody has bothered to give
the compiler sufficient smarts, not because it can't be done.) So a good
compiler should be able to compile "while iters" into "if iters: while
True" so long as iters is not modified in the body of the loop. 

Still, that's a nice observation: sometimes more complex source code can
lead to simpler byte code.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#82247

On Wed, Dec 10, 2014 at 11:10 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> On the other hand, *premature optimization*. In general, one shouldn't write
> more complex code so the compiler can optimize it, one should write simpler
> code and have a smarter compiler. If *we* are capable of recognising that
> iters is not modified in the body of the loop, then the compiler should be
> capable of it too. (If it isn't, it is because nobody has bothered to give
> the compiler sufficient smarts, not because it can't be done.) So a good
> compiler should be able to compile "while iters" into "if iters: while
> True" so long as iters is not modified in the body of the loop.

In general, one can't expect the boolification of a Python object to
be consistent, so the compiler can't optimize this. How can it be sure
the list will never become empty?

I'm still of the opinion that a while loop's header implies something
about the code; "while iters:" implies that iters might be able to
become false. Sure, you can verify easily enough that it never will...
but why should you have to verify at all?

ChrisA

[toc] | [prev] | [next] | [standalone]

#82248

On 10/12/2014 00:10, Steven D'Aprano wrote:
>
> Wait... is this like the Four Yorkshire Men sketch from Monty Python, only
> instead of complaining about how hard you had it as children, you're all
> trying to outdo each other about how difficult you find it to read this
> function? If so, well done, you really had me for a while.
>

The Four Yorkshiremen was not actually a Monty Python sketch.  Get it 
right, lad :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#82253

In article <54878f8a$0$13010$c3e8da3$5496439d@news.astraweb.com>,
 Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:

> I really think you guys are trying too hard to make this function seem more
> complicated than it is. If you find it so hard to understand a simple
> function with four short lines, one wonders how you would possibly cope
> with real code.

Well, look at it this way.  You've got several folks here (I count 
Terry, Ned, Chris, and myself) all saying, "We find this confusing", and 
you're saying, "nobody should find this confusing".  I suppose one 
possible explanation for this dichotomy is that we're all incapable of 
dealing with real code.  Yeah, that must be it.

[toc] | [prev] | [next] | [standalone]

#82274

On Tue, 09 Dec 2014 21:44:54 -0500, Roy Smith wrote:

> In article <54878f8a$0$13010$c3e8da3$5496439d@news.astraweb.com>,
>  Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:
> 
>> I really think you guys are trying too hard to make this function seem
>> more complicated than it is. If you find it so hard to understand a
>> simple function with four short lines, one wonders how you would
>> possibly cope with real code.
> 
> Well, look at it this way.  You've got several folks here (I count
> Terry, Ned, Chris, and myself) all saying, "We find this confusing", and
> you're saying, "nobody should find this confusing".  I suppose one
> possible explanation for this dichotomy is that we're all incapable of
> dealing with real code.  Yeah, that must be it.

I know you guys deal with *much* more complicated, obfuscated and 
difficult code. Just look at the stuff posted here by newbies :-) But you 
insist that you're confused by "while iters". I don't get it. I think 
you're protesting too much -- like the weightlifter who routinely bench-
presses 400 lbs, but then insists he can't take the trash out because 
it's too heavy. Or you're insisting on an unrealistic ideal of 
"readable", that if it takes *any time at all* to think about the code, 
that makes it bad.

I've noticed this deep-seated conservatism in Python programmers before. 
Parts of the language are deeply under-utilised, because there are simple 
idioms that people refuse to use because they're "confusing" even though 
they are a trivial generalisation of things that we use all the time.

Or at least, they should be common idioms. They're not common idioms 
because nobody uses them, and nobody uses them because they're not common 
idioms.

Example: In the statistics module in Python 3.4, I added a `median` 
function to calculate the median by the traditional schoolbook algorithm. 
But that's only one out of a number of ways to calculate medium, and 
inspired by similar syntax from R I proposed adding additional methods to 
that `median` object:

    median.low(alist)
    median.high(alist)

etc. We all know that functions are first class objects with attributes 
and methods, they have a __dict__ so you can attach per-function data to 
them. And there's the last line of the Zen, about namespaces being 
awesome. Every instance with a __dict__ is a namespace.

This ought to be a no-brainer, but apparently nobody has any experience 
with callable attributes attached to instances:

    math.sin(x)
    "some string".split(s)

We make it a point of pride that functions in Python are not just first 
class values but that they are objects with attributes and methods and a 
per-instance __dict__. But when somebody proposes making use of that, it 
is rejected. Not that I'm bitter :-)

But I digress.

I'm truly sorry that I have been unable to express in words just how 
elegant and beautiful the short version of the myzip() function is. It is 
vanishingly rare to have such a perfect match between an algorithm and 
real working code.

And if anyone has got the impression that I'm calling you a dummy because 
you don't see it my way, I'm not. I'm calling you nekulturny and somebody 
who can't recognise elegant code when it's staring you right in the 
face :-P



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#82275

On Wed, Dec 10, 2014 at 8:24 PM, Steven D'Aprano <steve@pearwood.info> wrote:
> And if anyone has got the impression that I'm calling you a dummy because
> you don't see it my way, I'm not. I'm calling you nekulturny and somebody
> who can't recognise elegant code when it's staring you right in the
> face :-P

I love it when a post sends me to a dictionary. Though in this case,
the word actually has Russian origin, so it ought perhaps to be
spelled некультурны. Good word, anyhow.

ChrisA

[toc] | [prev] | [next] | [standalone]

#82280

Steven D'Aprano <steve@pearwood.info>:

> I've noticed this deep-seated conservatism in Python programmers
> before. Parts of the language are deeply under-utilised, because there
> are simple idioms that people refuse to use because they're
> "confusing" even though they are a trivial generalisation of things
> that we use all the time.

IMO, large parts of the language are deservedly under-utilized. This
whole dunder thicket: __complex__(), __ifloordiv__(), __rxor__() etc. Or
this metaclass attribute. Or decorators.


Marko

[toc] | [prev] | [next] | [standalone]

#82298

[Multipart message — attachments visible in raw view] — view raw

On Wed, Dec 10, 2014 at 2:24 AM, Steven D'Aprano <steve@pearwood.info>
wrote:
>
> Example: In the statistics module in Python 3.4, I added a `median`
> function to calculate the median by the traditional schoolbook algorithm.
> But that's only one out of a number of ways to calculate medium, and
> inspired by similar syntax from R I proposed adding additional methods to
> that `median` object:
>
>     median.low(alist)
>     median.high(alist)
>
> etc. We all know that functions are first class objects with attributes
> and methods, they have a __dict__ so you can attach per-function data to
> them. And there's the last line of the Zen, about namespaces being
> awesome. Every instance with a __dict__ is a namespace.
>
> This ought to be a no-brainer, but apparently nobody has any experience
> with callable attributes attached to instances:
>
>     math.sin(x)
>     "some string".split(s)
>
> We make it a point of pride that functions in Python are not just first
> class values but that they are objects with attributes and methods and a
> per-instance __dict__. But when somebody proposes making use of that, it
> is rejected. Not that I'm bitter :-)

I don't particularly have a problem with functions having attributes, e.g.
I think itertools.chain.from_iterable is just peachy. There is a downside
though, which is that making those functions attributes of another function
rather than of the module defeats the dir() function for that module.
Likewise the generated help for the help() function, unless care is taken
to explicitly mention the existence of those functions in either the doc
string for the module or the standard median function. Thirdly, IDEs with
code completion features may simply fail to notice that these alternate
versions of the function exist. For these reasons I think that this is an
anti-pattern and is best avoided in the standard library. Having to type _
instead of . is only one extra key stroke.

Now if there were an established pattern to such function attributes, e.g.
if f.len(parrots) was a standard alternative to len(f(parrots)) for
functions that return sequences, then that could be a useful feature,
especially if such properties could be chained arbitrarily. But one-offs
like median.low would only serve to make the function more difficult to
find than necessary.

[toc] | [prev] | [next] | [standalone]

#82335

On Wed, 10 Dec 2014 09:46:55 -0700, Ian Kelly wrote:

> I don't particularly have a problem with functions having attributes,
> e.g. I think itertools.chain.from_iterable is just peachy. There is a
> downside though, which is that making those functions attributes of
> another function rather than of the module defeats the dir() function
> for that module. 

I think you are missing the point of namespaces :-)

When I call dir(os), I see os.path, but I don't see the names in os.path. 
That is working as designed.


> Likewise the generated help for the help() function,
> unless care is taken to explicitly mention the existence of those
> functions in either the doc string for the module or the standard median
> function. 

Yes.

I don't have a problem with saying "See also ``median.low``" in the 
docstring for median.

I would also have raised a feature request (and/or patch) for having 
help() automatically display functions and methods attached to a 
function, much as it already does for classes.

As the designer of the module I thought that median_low and median_high 
were important enough to include but not important enough to stick in the 
main module namespace. I wanted to delegate them to secondary status by 
putting them in a separate lightweight namespace, without going all the 
way to making statistics be a package. I even considered doing the same 
for the population variance and standard deviation, but decided against 
it as most people are familiar with them from scientific calculators and 
school.

Python gives us the functionality to include multiple namespaces in a 
single module. Despite the Zen of Python, hardly anyone makes use of the 
fact that functions are namespaces. I wish to rail against that :-)


> Thirdly, IDEs with code completion features may simply fail to
> notice that these alternate versions of the function exist.

Hmmm, that would be a bad bug in the IDE then. If I type

    foo.


I would expect the IDE to complete on whatever attributes foo has, 
regardless of whether it is a function or some other object.

I have tab completion enabled in Python 2.7, and it works for me:

py> def spam(): pass
... 
py> spam.
Display all 31 possibilities? (y or n)
spam.__call__(          spam.__globals__        spam.__str__(
spam.__class__(         spam.__hash__(          spam.__subclasshook__(
spam.__closure__        spam.__init__(          spam.func_closure
spam.__code__           spam.__module__         spam.func_code
spam.__defaults__       spam.__name__           spam.func_defaults
spam.__delattr__(       spam.__new__(           spam.func_dict
spam.__dict__           spam.__reduce__(        spam.func_doc
spam.__doc__            spam.__reduce_ex__(     spam.func_globals
spam.__format__(        spam.__repr__(          spam.func_name
spam.__get__(           spam.__setattr__(       
spam.__getattribute__(  spam.__sizeof__(        


I expect that most IDEs will Just Work in this case.

[...]
> Now if there were an established pattern to such function attributes,

But there's the rub. That's the conservatism I'm referring to. Somebody 
has to be the first to do such a thing, and if we reject it because 
nobody does it, nobody will do it because when they try they're always 
rejected.

We know namespaces are great and useful. We use namespaces in the form of 
modules, packages and classes, and the Zen says we should use more of 
them. But we don't, because we're trapped in a vicious circle where 
attempts to use more namespaces get dismissed because nobody uses them, 
and nobody uses them because when they try it gets dismissed :-(


Oh well, there's always my own non-stdlib modules :-)



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#82388

[Multipart message — attachments visible in raw view] — view raw

On Wed, Dec 10, 2014 at 9:01 PM, Steven D'Aprano <steve@pearwood.info>
wrote:
>
> On Wed, 10 Dec 2014 09:46:55 -0700, Ian Kelly wrote:
>
> > I don't particularly have a problem with functions having attributes,
> > e.g. I think itertools.chain.from_iterable is just peachy. There is a
> > downside though, which is that making those functions attributes of
> > another function rather than of the module defeats the dir() function
> > for that module.
>
> I think you are missing the point of namespaces :-)
>
> When I call dir(os), I see os.path, but I don't see the names in os.path.
> That is working as designed.

I don't think I'm missing the point. os.path is a module. That makes it
well suited for a namespace, because that's what modules are designed and
expected to provide: the implementation of namespaces. A function, on the
other hand, is not well suited to be a namespace, because it's not expected
to provide one. When I see a new function, I don't think "I should check
what attributes it has." Instead, I wonder about what parameters it takes
and what it returns.

I expect to be able to open up my web browser and go to the library
documentation for the os.path module. I don't expect to find a separate
page for statistics.median, because the docs are organized around modules,
not functions.  Would the median function itself be documented in the
statistics page, in the statistics.median page, or redundantly in both?

I expect to be able to import modules, and by extension, namespaces.
 "import os.path" works.  "import statistics.median.low" would return an
error, as would "from statistics.median import low as median", if that were
the only function I wanted. In the latter case I would have to do something
like "from statics import median; median = median.low". That's not just
ugly; it also demonstrates how the programmer needs to remember and
incorporate the detail that this namespace does not work like other
namespaces.

> As the designer of the module I thought that median_low and median_high
> were important enough to include but not important enough to stick in the
> main module namespace.

To me this sounds backward. Namespaces are a great idea, but at the same
time they also create cognitive burden. One creates a namespace because the
contents are important enough to be referenced as a group, not because they
aren't as important as the other things they're grouped with. The reason
the collections ABCs were moved into collections.abc wasn't because they
weren't important enough to be in the top-level collections module.

[toc] | [prev] | [next] | [standalone]

#82396

Ian Kelly wrote:

> A function, on the
> other hand, is not well suited to be a namespace, because it's not
> expected to provide one.


And that is exactly the point I am making about the inherent
conservativeness of Python developers.

Functions ARE namespaces, like instances of user-defined classes they have a
__dict__ and can carry per-instance state. (If you think that only modules
are namespaces, you are badly mistaken. Instances are namespaces too.) This
is *not* an accident of implementation, it is a deliberate design choice.
Contrast functions with builtin functions:

py> def f(): pass
...
py> vars(f)
{}
py> vars(len)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: vars() argument must have __dict__ attribute


Although functions are written in pure Python, their type is implemented in
C and could easily have been designed to be __dict__-less just like the
builtin_function_or_method type. But they weren't. It is a language feature
that functions are namespaces.

So why don't we use functions as namespaces?

We don't use functions as namespaces because nobody thinks of them as
namespaces, and we don't think of them as namespaces because nobody uses
them as namespaces -- and when somebody tries to break out of that vicious
circle, the conservativeness kicks in and the very idea is rejected.

I think that there is a legitimate debate to be had as to whether this
conservativeness and resistance to change is a good thing or a bad thing,
but I don't think that there should be any debate about the reality of the
Python community being strongly conservative, compared to the "anything
goes" attitude of (say) Ruby, Perl and Lisp programmers. We don't like code
that does anything we haven't seen a hundred times before.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#82397

[Multipart message — attachments visible in raw view] — view raw

On Thu, Dec 11, 2014 at 4:28 PM, Steven D'Aprano <
steve+comp.lang.python@pearwood.info> wrote:
>
> Ian Kelly wrote:
>
> > A function, on the
> > other hand, is not well suited to be a namespace, because it's not
> > expected to provide one.
>
>
> And that is exactly the point I am making about the inherent
> conservativeness of Python developers.
>
> Functions ARE namespaces, like instances of user-defined classes they
have a
> __dict__ and can carry per-instance state. (If you think that only modules
> are namespaces, you are badly mistaken. Instances are namespaces too.)

I never said that functions can't be used as namespaces. I said that
functions are *bad* namespaces, and I gave reasons why I think this is true.

[toc] | [prev] | [next] | [standalone]

#82401

In article <mailman.16880.1418342293.18130.python-list@python.org>,
 Ian Kelly <ian.g.kelly@gmail.com> wrote:

> I never said that functions can't be used as namespaces. I said that
> functions are *bad* namespaces, and I gave reasons why I think this is true.

An excellent example of functions acting as namespaces is nosetest's 
@attr() decorator.  We use this, for example, to tag certain test cases 
as being reliant on facebook being up(*):

    @attr('facebook', 'services')
    def test_some_facebook_thing():
        # whatever

this lets us turn all those tests on or off with a single switch.  The 
way @attr() is implemented, it sets attributes on the decorated 
function.  It's the most logical and obvious place to store a piece of 
information about a test case -- right on the test case itself.

Similarly, we've got tests that we annotate as being dependent on itunes 
being reachable, depending on certain data existing in the database(**), 
being too slow to want to run all the time(**), etc.

(*) and please don't tell me that tests shouldn't depend on external 
services.

(**) see first footnote

[toc] | [prev] | [next] | [standalone]

#82417

[Multipart message — attachments visible in raw view] — view raw

On Thu, Dec 11, 2014 at 6:55 PM, Roy Smith <roy@panix.com> wrote:
>
> In article <mailman.16880.1418342293.18130.python-list@python.org>,
>  Ian Kelly <ian.g.kelly@gmail.com> wrote:
>
> > I never said that functions can't be used as namespaces. I said that
> > functions are *bad* namespaces, and I gave reasons why I think this is
true.
>
> An excellent example of functions acting as namespaces is nosetest's
> @attr() decorator.  We use this, for example, to tag certain test cases
> as being reliant on facebook being up(*):
>
>     @attr('facebook', 'services')
>     def test_some_facebook_thing():
>         # whatever
>
> this lets us turn all those tests on or off with a single switch.  The
> way @attr() is implemented, it sets attributes on the decorated
> function.  It's the most logical and obvious place to store a piece of
> information about a test case -- right on the test case itself.

I agree this is a great use of function attributes, but this is an example
of tags or annotations, not namespaces. The purpose of namespaces is to
distinguish between different entities of the same name, e.g. 'pow' and
'math.pow' are two distinct functions that both happen to be named pow. We
can tell them apart because they're in separate namespaces.

In the nosetest example, the 'facebook' attribute always means the same
thing no matter which functions you apply it to. Conceptually speaking,
each instance of the attribute is the same annotation. The expression
test_some_facebook_thing.facebook doesn't perform the role of a namespace
because it's used to signify the presence of that annotation, not to
identify something.

A similar example from the standard library is the functools.lru_cache
decorator, which adds a cache_info function to the wrapped function.
However, the cache_info function always performs the same task in relation
to the particular function it decorates, so in this regard it acts more
like an object method than like a name in a namespace.

[toc] | [prev] | [next] | [standalone]

#82398

On Fri, Dec 12, 2014 at 10:28 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> I think that there is a legitimate debate to be had as to whether this
> conservativeness and resistance to change is a good thing or a bad thing,
> but I don't think that there should be any debate about the reality of the
> Python community being strongly conservative, compared to the "anything
> goes" attitude of (say) Ruby, Perl and Lisp programmers. We don't like code
> that does anything we haven't seen a hundred times before.

I agree that Python is a lot more conservative than many languages,
and personally, I think it's good. When it comes to monkeypatching,
less is *definitely* better - less places where you have to figure out
what import changed the meanings of your built-ins, and so on. (Case
in point: A logger replacement tampered with the
log.info()/log.error() family of functions, adding a keyword
parameter. Took me a fair amount of digging to find out exactly where
that happened, and it made REPL importing of the module's functions
rather awkward.) The same is true of obscure namespaces, specifically
because they're obscure; if I do a JSON decode of something, I'm not
going to check to see if numbers came through as a subclass of float
with a ".warnings" attribute to tell me about loss of numeric
precision, even though that would be perfectly possible. Since almost
nobody does this, nobody will expect it to be done, ergo nobody will
go looking for it, and the feature may as well not even have been
implemented if it's never seen. Popularity *is* sometimes a legitimate
argument.

ChrisA

[toc] | [prev] | [next] | [standalone]

#82304

On 12/10/2014 11:46 AM, Ian Kelly wrote:

> I don't particularly have a problem with functions having attributes,
> e.g. I think itertools.chain.from_iterable is just peachy. There is a
> downside though, which is that making those functions attributes of
> another function rather than of the module

Itertools.chain is a class, not a function.  Chain.from_iterable is an 
*alternate constructor* for instances of the class, just as 
dict.fromkeys is an alternate constructor for dict instances.

It is true that if itertools were written in python, chain and the other 
classes would likely be written as generator functions, so tha analogy 
would not apply.

> defeats the dir() function for that module.

And it is also true that dir(__builtins__) does not show dict.fromkeys.

> Likewise the generated help for the help() function,
> unless care is taken to explicitly mention the existence of those
> functions in either the doc string for the module

help(it.chain) lists
  |  from_iterable(...) from builtins.type
  |      chain.from_iterable(iterable) --> chain object
  |
  |      Alternate chain() contructor taking a single iterable argument
  |      that evaluates lazily.

help(dict)
  |  fromkeys(iterable, value=None, /) from builtins.type
  |      Returns a new dict with keys from iterable and values equal to 
value.

> Thirdly, IDEs with code completion features may simply fail to
> notice that these alternate versions of the function exist.

I don't know what you mean here.  In Idle
 >>> it.chain.  --> completion box with from_iterable
 >>> dict.   --> completion box that includes fromkeys

None of the above is to say that alternate constructors for classed are 
not a bit awkward.  I believe this was recognized when dict.fromkeys was 
added.  But it was considered less awkward than the alternatives: 1. do 
not add the functionality; 2. further overload the main (__init__) 
constructor; 3. add a independent function.  Anyway, they exist, are an 
accepted pattern among core devs, and will probably increase in number.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#82309

[Multipart message — attachments visible in raw view] — view raw

On Wed, Dec 10, 2014 at 10:48 AM, Terry Reedy <tjreedy@udel.edu> wrote:
>> Likewise the generated help for the help() function,
>> unless care is taken to explicitly mention the existence of those
>> functions in either the doc string for the module
>
>
> help(it.chain) lists
>  |  from_iterable(...) from builtins.type
>  |      chain.from_iterable(iterable) --> chain object
>  |
>  |      Alternate chain() contructor taking a single iterable argument
>  |      that evaluates lazily.

As you point out, chain is a class, not a function. When you invoke help()
on a function, it lists the members, of which a static alternate
constructor would be one. Try it with a function, though:

>>> import statistics
>>> statistics.median.low = statistics.median_low
>>> help(statistics.median)
Help on function median in module statistics:

median(data)
    Return the median (middle value) of numeric data.

    When the number of data points is odd, return the middle data point.
    When the number of data points is even, the median is interpolated by
    taking the average of the two middle values:

    >>> median([1, 3, 5])
    3
    >>> median([1, 3, 5, 7])
    4.0

It only prints out the doc string, nothing about the attributes. So the doc
string for median would need to explicitly document the existence of
median.low and median.high in addition to its own usage.

>> Thirdly, IDEs with code completion features may simply fail to
>> notice that these alternate versions of the function exist.
>
> I don't know what you mean here.  In Idle
> >>> it.chain.  --> completion box with from_iterable
> >>> dict.   --> completion box that includes fromkeys

So Idle gets it right. At least for static methods of classes, which isn't
very surprising. Does it complete a function attribute of a function? I
don't have it installed to test.

Having used Komodo IDE for a number of years and been occasionally
frustrated by its code completion, it would not surprise me in the least if
it failed to pull attributes of functions into its completion database.

[toc] | [prev] | [next] | [standalone]

#82336

On 12/10/2014 3:32 PM, Ian Kelly wrote:

> So Idle gets it right. At least for static methods of classes, which
> isn't very surprising. Does it complete a function attribute of a
> function?

 >>> def f(): pass

 >>> f.a='attr'
 >>> f.  <box with 'a'> as possible completion.

> Having used Komodo IDE for a number of years and been occasionally
> frustrated by its code completion, it would not surprise me in the least
> if it failed to pull attributes of functions into its completion database.

I do not think Idle has a static database.  But I have not looked at the 
completion code yet to see exactly how it works (and how it might be 
improved).

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#82104

Steven D'Aprano wrote:
> I do not believe that good code must be obviously right. It's okay for code
> to be subtly right.

If you write code as subtly as you can, you're not
subtle enough to debug it...

-- 
Greg

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.python

csiph-web