Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #110899 > unrolled thread

Namespaces are one honking great idea

Back to article view | Back to comp.lang.python

Contents

  Namespaces are one honking great idea Steven D'Aprano <steve@pearwood.info> - 2016-07-02 00:13 +1000
    Re: Namespaces are one honking great idea Random832 <random832@fastmail.com> - 2016-07-01 10:40 -0400
    Re: Namespaces are one honking great idea BartC <bc@freeuk.com> - 2016-07-01 15:49 +0100
      Re: Namespaces are one honking great idea Chris Angelico <rosuav@gmail.com> - 2016-07-02 03:13 +1000
      Re: Namespaces are one honking great idea Steven D'Aprano <steve@pearwood.info> - 2016-07-02 03:46 +1000
      Re: Namespaces are one honking great idea Rustom Mody <rustompmody@gmail.com> - 2016-07-01 19:26 -0700
    Re: Namespaces are one honking great idea Ethan Furman <ethan@stoneleaf.us> - 2016-07-01 09:00 -0700
      Re: Namespaces are one honking great idea Steven D'Aprano <steve@pearwood.info> - 2016-07-02 03:10 +1000
        Re: Namespaces are one honking great idea Ethan Furman <ethan@stoneleaf.us> - 2016-07-01 12:29 -0700
          Re: Namespaces are one honking great idea Steven D'Aprano <steve@pearwood.info> - 2016-07-02 11:07 +1000
            Re: Namespaces are one honking great idea Kevin Conway <kevinjacobconway@gmail.com> - 2016-07-02 01:50 +0000
              Re: Namespaces are one honking great idea Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-07-01 19:55 -0700
              Re: Namespaces are one honking great idea Steven D'Aprano <steve@pearwood.info> - 2016-07-03 13:22 +1000
            Re: Namespaces are one honking great idea Random832 <random832@fastmail.com> - 2016-07-02 00:59 -0400
            Re: Namespaces are one honking great idea Kevin Conway <kevinjacobconway@gmail.com> - 2016-07-02 15:34 +0000
              Re: Namespaces are one honking great idea Steven D'Aprano <steve@pearwood.info> - 2016-07-03 13:44 +1000
                Re: Namespaces are one honking great idea Ethan Furman <ethan@stoneleaf.us> - 2016-07-02 22:14 -0700
                Re: Namespaces are one honking great idea Kevin Conway <kevinjacobconway@gmail.com> - 2016-07-03 22:02 +0000
                  Re: Namespaces are one honking great idea Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-07-03 16:21 -0700
                    Re: Namespaces are one honking great idea BartC <bc@freeuk.com> - 2016-07-04 01:19 +0100
                Re: Namespaces are one honking great idea Ethan Furman <ethan@stoneleaf.us> - 2016-07-03 21:01 -0700
            Re: Namespaces are one honking great idea Ethan Furman <ethan@stoneleaf.us> - 2016-07-02 16:28 -0700
    Re: Namespaces are one honking great idea Chris Angelico <rosuav@gmail.com> - 2016-07-04 21:37 +1000
      Re: Namespaces are one honking great idea Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-07-04 15:09 -0700
        Re: Namespaces are one honking great idea Lawrence D’Oliveiro <lawrencedo99@gmail.com> - 2016-07-04 15:11 -0700
    Re: Namespaces are one honking great idea jmp <jeanmichel@sequans.com> - 2016-07-04 17:58 +0200
    Re: Namespaces are one honking great idea Chris Angelico <rosuav@gmail.com> - 2016-07-05 12:58 +1000
      Re: Namespaces are one honking great idea Steven D'Aprano <steve@pearwood.info> - 2016-07-05 13:35 +1000
        Re: Namespaces are one honking great idea Chris Angelico <rosuav@gmail.com> - 2016-07-05 13:47 +1000
          Re: Namespaces are one honking great idea Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2016-07-05 16:09 +1000
            A nestedmodule decorator (Re: Namespaces are one honking great idea) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2016-07-05 21:10 +1200
              Improved nestedmodule decorator implementation Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2016-07-05 21:31 +1200
                Re: Improved nestedmodule decorator implementation Steven D'Aprano <steve@pearwood.info> - 2016-07-05 20:54 +1000
    Re: Namespaces are one honking great idea Steven D'Aprano <steve@pearwood.info> - 2016-07-05 12:34 +1000
    Re: Namespaces are one honking great idea jmp <jeanmichel@sequans.com> - 2016-07-04 13:23 +0200
    Re: Namespaces are one honking great idea carlosjosepita@gmail.com - 2016-07-09 07:05 -0700

Page 1 of 2  [1] 2  Next page →

#110899 — Namespaces are one honking great idea

The Zen of Python says:

    Namespaces are one honking great idea -- let's do more of those!



Proposal
=========

Add a new "namespace" object to Python.


Rationale
==========

Sometimes we have a group of related functions and variables that belong
together, but are not sufficiently unrelated to the rest of the module that
we want to split them out into another file.

Python allows us to group related chunks of code in three levels of
grouping:

- the class, containing one of more variable and/or function (method);
- the module, containing one or more classes and/or functions;
- the package, containing one or more modules.

But classes are not like the others: they must be instantiated before they
can be used, and they are more than just a mere namespace grouping related
entities. Classes support inheritance. Classes should be used for "is-a"
relationships, not "has-a" relationships. Although classes (and instances)
are namespaces, they provide fundamentally different kind of behaviour than
modules and packages.

Sometimes we have a group of related functions which belongs together, but
they are more closely related to each other than they are to the rest of
the module, but not so distantly unrelated that they should be split off
into a separate module. Nor do they belong as a class -- there's nothing
analogous to an instance, and no need for inheritance.

Perhaps the functions have some name clashes, e.g. two different functions
that deserve the same name, or perhaps they're just conceptually related.

What is needed is a module inside a module, but without the necessity of
refactoring the code into a package and splitting the code out into a
separate file. In other words, something rather like C++ namespaces:

https://msdn.microsoft.com/en-us/library/5cb46ksf.aspx


Proof of concept
=================

Here's a proof of concept. I use a class with a custom metaclass like this:


# Python 3 version
class ExampleNS(metaclass=Namespace):
    x = 1
    y = []

    def spam(n):
        return 'spam'*n

    z = spam(3).upper()

    def ham(n):
        return spam(n).replace('sp', 'H')

    def test():
        global x
        x += 1
        y.append(1)
        return x, y


And a demonstration of its use:

py> Example.spam(5)
'spamspamspamspamspam'
py> Example.ham(5)
'HamHamHamHamHam'
py> Example.test()
(2, [1])
py> Example.test()
(3, [1, 1])
py> print(Example.x, Example.y, Example.z)
3 [1, 1] SPAMSPAMSPAM

Despite the "class" statement (a limitation of Python's lack of dedicated
syntax for namespaces), the Example namespace behaves like (in fact, *is*)
a module embedded inside a module.

Notice that the functions are not methods and don't require a "self"
parameter. Being functions, they can be used inside the body of the
namespace, including inside other functions.

Outside the namespace, access is via dotted attribute: Example.spam. But
inside it, functions can refer to each other without knowing the name of
their namespace.

Especially note that inside the namespace, the global statement makes a name
refer to the namespace scope.


Implementation
===============

Here's a simplified implementation:


from types import FunctionType, ModuleType

class NamespaceType(ModuleType):
    def __repr__(self):
        return 'namespace <%s>' % self.__name__

class Namespace(type):
    def __new__(meta, name, bases, ns):
        if bases != ():
            raise RuntimeError("namespaces cannot inherit")
        module = NamespaceType(name)
        globals_ = module.__dict__
        if '__builtins__' not in globals_:
            globals_['__builtins__'] = globals()['__builtins__']
        for name, obj in ns.items():
            if type(obj) is FunctionType:
                ns[name] = meta._copy_and_inject(obj, globals_)
        module.__dict__.update(ns)
        return module

    @classmethod
    def _copy_and_inject(meta, func, globals):
        """Return a new function object with the given globals."""
        assert type(func) is FunctionType
        f = FunctionType(
                func.__code__, globals, func.__name__,
                 func.__defaults__, func.__closure__)
        f.__dict__ = func.__dict__
        # Next two lines are Python 3 only.
        f.__kwdefaults__ = func.__kwdefaults__
        f.__annotations__ = func.__annotations__
        return f



The namespace object itself is actually a module (to be precise, a subclass
of ModuleType). The Namespace metaclass returns a new module object,
containing all the names I defined inside the class, after modifying any
function objects to treat the new module object as their globals.


Unresolved questions
=====================

Would a decorator be better than a metaclass?

    @namespace
    class Example:
        ...


How about a mechanism to exclude functions from having their globals
changed?


Some limitations
=================

The simplified implementation shown doesn't allow functions inside the
namespace to access names outside of the namespace easily. In practice, the
user might want the name resolution order inside the namespace to be:

 local variables
 nonlocal variables
 namespace globals
 module globals
 builtins


The biggest limitation is that I have to abuse the class statement to do
this. In an ideal world, there would be syntactic support and a keyword:

    namespace Example:
        x = 0
        y = []
        def test(n): ...

although others might argue that *not* needing a dedicated keyword is, in
fact, better.


Disadvantages
==============

Those unfamiliar with the namespace concept may be confused by a class that
doesn't get instantiated.


Alternatives
=============

At the moment, Python gives us four existing solutions:

(1) Split the functions into a separate file, whether they deserve it or
not. That's the smallest grouping Python supports so you're stuck with it.

The obvious disadvantage is that what is conceptually a single file must be
split into two, making more work for the developer.


(2) Don't split the functions into their own namespace, keep them in the
enclosing module namespace, but give them a "virtual namespace" by
prefixing their names:

    def foods_spam(a, b): ...

    def foods_eggs(a): ...

    def advertising_spam(a, b, c): ...

This requires the reader to see the "_" as a "virtual dot", rather than as a
word separator as underscores are normally used as. It also fails to
visually group the functions together, and requires functions to refer to
each other using their full "virtual" dotted name.


(3) Pretend you're in the Kingdom of Nouns[1]. Put the functions in a class,
turn them into methods, and create an instance just so you can use them:

    class Foods:
        def spam(self, a, b): ...
        def eggs(self, a): ...

    class Advertising:
        def spam(self, a, b, c): ...

    foods = Foods()
    advertising = Advertising()

optionally making each instance a singleton. Sometimes the class is deleted
to discourage subclassing.

This solution is often an anti-pattern, as it violates expectations about
classes. The class should not be subclassed, nor should it be instantiated
more than once. The class doesn't represent an "is-a" relationship, nor
does the instance represent a real-world object.


(4) Put them in a class, but use staticmethod (or classmethod) to avoid
needing an instance:

    class foods:
        @staticmethod
        def spam(a, b): ...
        @staticmethod
        def eggs(a): ...

    class advertising:
        @staticmethod
        def spam(a, b, c): ...


This has the disadvantage that functions in the class cannot call each other
directly, but must use the dotted names. This is the Python equivalent of
the "Java utility class" pattern, or arguably anti-pattern[2], where you
create a class with little or no state and populate it with a number of
static methods.







[1]
http://steve-yegge.blogspot.com/2006/03/execution-in-kingdom-of-nouns.html

[2]
http://www.yegor256.com/2014/05/05/oop-alternative-to-utility-classes.html

-- 
Steven

[toc] | [next] | [standalone]

#110903

On Fri, Jul 1, 2016, at 10:13, Steven D'Aprano wrote:
> The biggest limitation is that I have to abuse the class statement to do
> this. In an ideal world, there would be syntactic support and a keyword:
> 
>     namespace Example:
>         x = 0
>         y = []
>         def test(n): ...
> 
> although others might argue that *not* needing a dedicated keyword is, in
> fact, better.

What might be nice would be a single syntax that isn't specific to
classes or your "namespaces".

namespace(type) Name:
   "equivalent to class Name:"

namespace(whatever) Name:
   "equivalent to class Name(metaclass=whatever)"

[toc] | [prev] | [next] | [standalone]

#110904

On 01/07/2016 15:13, Steven D'Aprano wrote:

> Sometimes we have a group of related functions and variables that belong
> together, but are not sufficiently unrelated to the rest of the module that
> we want to split them out into another file.

> Here's a proof of concept. I use a class with a custom metaclass like this:
>
>
> # Python 3 version
> class ExampleNS(metaclass=Namespace):
>     x = 1
>     y = []
>
>     def spam(n):
>         return 'spam'*n

> py> Example.spam(5)
> 'spamspamspamspamspam'


Why not just extend the capabilities of a class? I actually thought this 
would work until I tried it and it didn't:

class C():
     def fn():
         print ("Hi!")

C.fn()

The error message suggests Python knows what's going on. So why not just 
make it work?

(The main issue would be, if an instance X of C /was/ created, then 
X.fn() wouldn't work as there is no 'self' parameter for fn.)

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#110910

On Sat, Jul 2, 2016 at 12:49 AM, BartC <bc@freeuk.com> wrote:
> Why not just extend the capabilities of a class? I actually thought this
> would work until I tried it and it didn't:
>
> class C():
>     def fn():
>         print ("Hi!")
>
> C.fn()
>
> The error message suggests Python knows what's going on. So why not just
> make it work?

rosuav@sikorsky:~$ python3
Python 3.6.0a2+ (default:57f3514564f6, Jun 29 2016, 16:27:34)
[GCC 5.3.1 20160528] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class C():
...     def fn():
...         print ("Hi!")
...
>>> C.fn()
Hi!
>>>

But if you have two such functions, they can't call each other without
the namespace marker.

ChrisA

[toc] | [prev] | [next] | [standalone]

#110912

On Sat, 2 Jul 2016 12:49 am, BartC wrote:

> On 01/07/2016 15:13, Steven D'Aprano wrote:
> 
>> Sometimes we have a group of related functions and variables that belong
>> together, but are not sufficiently unrelated to the rest of the module
>> that we want to split them out into another file.
> 
>> Here's a proof of concept. I use a class with a custom metaclass like
>> this:
>>
>>
>> # Python 3 version
>> class ExampleNS(metaclass=Namespace):
>>     x = 1
>>     y = []
>>
>>     def spam(n):
>>         return 'spam'*n
> 
>> py> Example.spam(5)
>> 'spamspamspamspamspam'
> 
> 
> Why not just extend the capabilities of a class? 

The usual ways to change the behaviour of classes from Python code is via
decorator, which lets you modify the class after it is created, a
metaclass, which lets you modify it before it is created, or by using
descriptors to customize attribute access.

I'm using a metaclass. When you write:

class K: ...

that is syntactic sugar for calling the default metaclass (`type`) with some
arguments, and `type` then returns the new class K. But if you use a
metaclass (usually, but not necessarily a subclass of `type`) you can
customize the creation of the new class, or even return a completely
different object altogether. That's what I'm doing.

Why am I returning a module instead of a class? Because (1) conceptually a
namespace (in the C++ sense) is like a module; (2) classes have a whole lot
of expectations that aren't relevant to namespaces (like inheritance,
instantiation, "is-a", etc); and (3) it's easy to use a module, which
already provides most of the behaviour I want.


> I actually thought this 
> would work until I tried it and it didn't:
> 
> class C():
>      def fn():
>          print ("Hi!")
> 
> C.fn()

That actually will work in Python 3. (I had forgotten that.) In Python 2,
you have to inherit C from object, AND decorate fn with staticmethod:

class C(object):
    @staticmethod
    def fn():
        print ("Hi!")


But for my use, that's not enough. I want this to work:

class Foo:
    def f():
        return g().upper()
    def g():
        return "hi!"


That is, Foo should behave as if it were a module imported from a file,
except without actually being imported from a file.




-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

[toc] | [prev] | [next] | [standalone]

#110927

On Friday, July 1, 2016 at 8:19:36 PM UTC+5:30, BartC wrote:
> On 01/07/2016 15:13, Steven D'Aprano wrote:
> 
> > Sometimes we have a group of related functions and variables that belong
> > together, but are not sufficiently unrelated to the rest of the module that
> > we want to split them out into another file.
> 
> > Here's a proof of concept. I use a class with a custom metaclass like this:
> >
> >
> > # Python 3 version
> > class ExampleNS(metaclass=Namespace):
> >     x = 1
> >     y = []
> >
> >     def spam(n):
> >         return 'spam'*n
> 
> > py> Example.spam(5)
> > 'spamspamspamspamspam'
> 
> 
> Why not just extend the capabilities of a class? I actually thought this 
> would work until I tried it and it didn't:

Well I also thought similarly -- sure a normal (instance) method cannot be
used without an instance but sure this is what class/static methods are for?
ie Steven's option 4.

Then I remembered that python's LEGB rule is bizarre:
Outer scopes are added outside inner scopes ... except for classes

[toc] | [prev] | [next] | [standalone]

#110907

On 07/01/2016 07:13 AM, Steven D'Aprano wrote:

I like the idea, but I have a couple questions about the design choices. 
  Comments below.

> The Zen of Python says:
>
>      Namespaces are one honking great idea -- let's do more of those!

> Proposal
> =========
>
> Add a new "namespace" object to Python.


> Proof of concept
> =================
>
> Here's a proof of concept. I use a class with a custom metaclass like this:
>
>
> # Python 3 version
> class ExampleNS(metaclass=Namespace):
>      x = 1
>      y = []
>
>      def spam(n):
>          return 'spam'*n
>
>      z = spam(3).upper()
>
>      def ham(n):
>          return spam(n).replace('sp', 'H')
>
>      def test():
>          global x
>          x += 1
>          y.append(1)
>          return x, y
>

> Despite the "class" statement (a limitation of Python's lack of dedicated
> syntax for namespaces), the Example namespace behaves like (in fact, *is*)
> a module embedded inside a module.

So the idea is to have several "mini-modules" inside a single file?

Can a mini-module function access a different mini-module's function?

Can a mini-module function access/interact with the module's global scope?

> Especially note that inside the namespace, the global statement makes a name
> refer to the namespace scope.

Interesting.  :)


> Unresolved questions
> =====================
>
> Would a decorator be better than a metaclass?

I think a decorator may provide more of a clue that something 
interesting is happening -- whether a decorator is feasible depends on 
whether you need the __prepare__ magic of metaclasses.

>      @namespace
>      class Example:
>          ...

I believe Python already has a NameSpace type, so a different name is 
probably better.  How about minimodule? ;)


> Some limitations
> =================
>
> The simplified implementation shown doesn't allow functions inside the
> namespace to access names outside of the namespace easily.

How not-easy is it?

> In practice, the
> user might want the name resolution order inside the namespace to be:
>
>   local variables
>   nonlocal variables
>   namespace globals
>   module globals
>   builtins

That would be ideal, I think.  How close can we get to that?

> The biggest limitation is that I have to abuse the class statement to do
> this. In an ideal world, there would be syntactic support and a keyword:
>
>      namespace Example:
>          x = 0
>          y = []
>          def test(n): ...
>
> although others might argue that *not* needing a dedicated keyword is, in
> fact, better.

Metaclasses for the win!

> Disadvantages
> ==============
>
> Those unfamiliar with the namespace concept may be confused by a class that
> doesn't get instantiated.

A well-named decorator/base-class should help with that.


Did you mean for this to go to -Ideas?

--
~Ethan~

[toc] | [prev] | [next] | [standalone]

#110909

On Sat, 2 Jul 2016 02:00 am, Ethan Furman wrote:

> On 07/01/2016 07:13 AM, Steven D'Aprano wrote:
> 
> I like the idea, but I have a couple questions about the design choices.

Thanks!

>   Comments below.


[...]
>> Despite the "class" statement (a limitation of Python's lack of dedicated
>> syntax for namespaces), the Example namespace behaves like (in fact,
>> *is*) a module embedded inside a module.
> 
> So the idea is to have several "mini-modules" inside a single file?

That would certainly be possible.


> Can a mini-module function access a different mini-module's function?

Yes, via the dotted attribute name, just as you might say "math.sin".

The only difference is that you don't use import to get access to the
(mini-)module. It's just there, defined in your own file.


> Can a mini-module function access/interact with the module's global scope?

Yes. I'm still experimenting with different implementations, but I have a
proof of concept working.


>> Especially note that inside the namespace, the global statement makes a
>> name refer to the namespace scope.
> 
> Interesting.  :)
> 
> 
>> Unresolved questions
>> =====================
>>
>> Would a decorator be better than a metaclass?
> 
> I think a decorator may provide more of a clue that something
> interesting is happening -- whether a decorator is feasible depends on
> whether you need the __prepare__ magic of metaclasses.

I don't think __prepare__ is needed.



>>      @namespace
>>      class Example:
>>          ...
> 
> I believe Python already has a NameSpace type, so a different name is
> probably better.  How about minimodule? ;)

I think you are thinking of "SimpleNamespace".

In any case, so long as my namespace lives in a different, er, namespace as
the other namespace, there's no conflict :-)

In the long term, I think my namespace (if accepted into the stdlib) would
belong in functools, collections or types. Perhaps even its own module. I
don't think this needs to be a built-in.



>> Some limitations
>> =================
>>
>> The simplified implementation shown doesn't allow functions inside the
>> namespace to access names outside of the namespace easily.
> 
> How not-easy is it?

Not that hard now. I have an implementation working. It actually turned out
to be easy. (Thanks Raymond for ChainMap!)



>> In practice, the
>> user might want the name resolution order inside the namespace to be:
>>
>>   local variables
>>   nonlocal variables
>>   namespace globals
>>   module globals
>>   builtins
> 
> That would be ideal, I think.  How close can we get to that?

I now have that working, for some definition of working. Its not extensively
tested, so there might be some bugs or issues I haven't thought of, but the
basic concept is working: inside the namespace/mini-module object,
functions see namespace globals ahead of module globals, which are ahead of
builtins.


[...]
> Did you mean for this to go to -Ideas?


Not yet. I wanted some initial feedback to see if anyone else liked the idea
before taking it to Bikeshedding Central :-)

Besides, I expect Python-Ideas will say it needs to be a package on PpPI
first. Although I am kinda considering sneaking this into the std lib as an
undocumented internal feature, like simplegeneric in pkgutil. (Don't tell
anyone I said that *wink* )





-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

[toc] | [prev] | [next] | [standalone]

#110919

On 07/01/2016 10:10 AM, Steven D'Aprano wrote:
> On Sat, 2 Jul 2016 02:00 am, Ethan Furman wrote:

>> Did you mean for this to go to -Ideas?
>
> Not yet. I wanted some initial feedback to see if anyone else liked the idea
> before taking it to Bikeshedding Central :-)
>
> Besides, I expect Python-Ideas will say it needs to be a package on PpPI
> first. Although I am kinda considering sneaking this into the std lib as an
> undocumented internal feature, like simplegeneric in pkgutil. (Don't tell
> anyone I said that *wink* )

Are there good use-cases for this in the stdlib?

--
~Ethan~

[toc] | [prev] | [next] | [standalone]

#110924

On Sat, 2 Jul 2016 05:29 am, Ethan Furman wrote:

> On 07/01/2016 10:10 AM, Steven D'Aprano wrote:
>> On Sat, 2 Jul 2016 02:00 am, Ethan Furman wrote:
> 
>>> Did you mean for this to go to -Ideas?
>>
>> Not yet. I wanted some initial feedback to see if anyone else liked the
>> idea before taking it to Bikeshedding Central :-)
>>
>> Besides, I expect Python-Ideas will say it needs to be a package on PpPI
>> first. Although I am kinda considering sneaking this into the std lib as
>> an undocumented internal feature, like simplegeneric in pkgutil. (Don't
>> tell anyone I said that *wink* )
> 
> Are there good use-cases for this in the stdlib?

I have at least one.



-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

[toc] | [prev] | [next] | [standalone]

#110925

I believe the namespace object you are referring to is exactly a class.
IIRC, classes came about as a "module in a module".

Regardless, all use cases you've listed are already satisfied by use of the
static and class method decorators. Methods decorated with these do not
require an instance initialization to use.

On Fri, Jul 1, 2016, 20:17 Steven D'Aprano <steve@pearwood.info> wrote:

> On Sat, 2 Jul 2016 05:29 am, Ethan Furman wrote:
>
> > On 07/01/2016 10:10 AM, Steven D'Aprano wrote:
> >> On Sat, 2 Jul 2016 02:00 am, Ethan Furman wrote:
> >
> >>> Did you mean for this to go to -Ideas?
> >>
> >> Not yet. I wanted some initial feedback to see if anyone else liked the
> >> idea before taking it to Bikeshedding Central :-)
> >>
> >> Besides, I expect Python-Ideas will say it needs to be a package on PpPI
> >> first. Although I am kinda considering sneaking this into the std lib as
> >> an undocumented internal feature, like simplegeneric in pkgutil. (Don't
> >> tell anyone I said that *wink* )
> >
> > Are there good use-cases for this in the stdlib?
>
> I have at least one.
>
>
>
> --
> Steven
> “Cheer up,” they said, “things could be worse.” So I cheered up, and sure
> enough, things got worse.
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

[toc] | [prev] | [next] | [standalone]

#110928

On Saturday, July 2, 2016 at 1:50:56 PM UTC+12, Kevin Conway wrote:
> Regardless, all use cases you've listed are already satisfied by use of the
> static and class method decorators.

Except for the need to decorate every such function inside the class. How about:

    import types

    def namespace(inclass) :
        "decorator which turns a class into a module."
        outclass = types.ModuleType \
          (
            name = inclass.__name__,
            doc = inclass.__doc__,
          )
        for attr in dir(inclass) :
            if not attr.startswith("__") or attr in ("__package__",) :
                setattr(outclass, attr, getattr(inclass, attr))
            #end if
        #end for
        return \
            outclass
    #end namespace

Example use:

    @namespace
    class try_it :
        "try it!"

        val = "some text"

        def func() :
            print("func got called")
        #end func

    #end try_it

[toc] | [prev] | [next] | [standalone]

#110970

On Sat, 2 Jul 2016 11:50 am, Kevin Conway wrote:

> I believe the namespace object you are referring to is exactly a class.

Yes, a namespace is exactly like a class, minus inheritance, instantiation,
the implied "is-a" relationship, and the whole Java "utility class"
anti-pattern.

In other words, exactly not like a class *wink*

Classes and modules are both namespaces: "an abstract container or
environment created to hold a logical grouping of unique identifiers or
symbols (i.e. names)", to quote Wikipedia:

https://en.wikipedia.org/wiki/Namespace

But classes provide much more functionality, over and above mere namespace:
inheritance, instantiation, etc. I'm not anti-classes or opposed to OOP,
these features are great when they are wanted. But when they aren't wanted,
they're a nuisance.

Modules also provide more than just an abstract namespace: they provide
scope, but that's exactly what I want for my Namespace entity: it should be
a namespace, and it should be a scope for functions. In other words, it
should be a module. I just don't want to have to place the source code in a
separate file, because sometimes that's annoying and overkill.

Modules are good for information hiding and encapsulation, just like
classes, but without inheritance etc. But because you cannot put more than
one module inside a single file, the developer has to make a choice between
information hiding and encapsulation:

(1) You can keep your functions encapsulated inside a single module/file,
but you cannot hide some of them from parts of the module that don't care
about them.

(2) You can hide those functions from the parts of the module that don't
care about them, but only by breaking encapsulation, moving them to another
file, and exposing another name in the file system namespace.


The "namespace" feature offers an alternative.

Namespaces in C#, C++ and PHP:

https://msdn.microsoft.com/en-us/library/z2kcy19k.aspx
https://msdn.microsoft.com/en-us/library/5cb46ksf.aspx
http://php.net/manual/en/language.namespaces.php



> IIRC, classes came about as a "module in a module".

I'm pretty sure they did not. Object oriented programming (and hence
classes) came about from simulating real world objects, hence the name:

http://www.exforsys.com/tutorials/oops-concepts/the-history-of-object-oriented-programming.html

In languages like Pascal, you can nest functions and procedures inside
others, providing something like a namespace functionality, except that
Pascal strictly enforces information hiding. There's no way for code to
peer inside a Pascal function and see nested functions.


> Regardless, all use cases you've listed are already satisfied by use of
> the static and class method decorators. Methods decorated with these do
> not require an instance initialization to use.

And are significantly less easy to use, as the functions MUST refer to each
other by their dotted names.




-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

[toc] | [prev] | [next] | [standalone]

#110939

On Fri, Jul 1, 2016, at 21:50, Kevin Conway wrote:
> I believe the namespace object you are referring to is exactly a
> class. IIRC, classes came about as a "module in a module".

No, because classes have instances. And conceptually they seem like they
*should* have instances. Just using the term "class" carries
expectations.

More to the point, what modules do that classes do not is provide a
global namespace for functions defined within them, so that variables
within them can be used (well, read - writing them requires a
declaration) by the functions without extra qualification.

> Regardless, all use cases you've listed are already satisfied by use
> of the static and class method decorators. Methods decorated with
> these do not require an instance initialization to use.

staticmethod isn't technically required to use a method through the
class (or subclasses), it simply provides the appropriate magic to allow
it to be called through instances.

[toc] | [prev] | [next] | [standalone]

#110960

> staticmethod isn't technically required to use a method through the class
(or subclasses), it simply provides the appropriate magic to allow it to be
called through instances.

For example, the following code covers all described use cases of the
proposed namespace. Methods are invoked without creating instances and
state is managed on the class object directly.

class CustomNs:

    stateful_data = 1

    @staticmethod
    def echo(text):
        print(text)

    @classmethod
    def mutate(cls):
        cls.stateful_data += 1
        print(cls.stateful_data)

CustomNs.echo("test")
print(CustomNs.stateful_data)
CustomNs.mutate()
print(CustomNs.stateful_data)

For the proponents of namespace, what is deficient in the above example
that necessitates a language change?

On Sat, Jul 2, 2016, 00:02 Random832 <random832@fastmail.com> wrote:

> On Fri, Jul 1, 2016, at 21:50, Kevin Conway wrote:
> > I believe the namespace object you are referring to is exactly a
> > class. IIRC, classes came about as a "module in a module".
>
> No, because classes have instances. And conceptually they seem like they
> *should* have instances. Just using the term "class" carries
> expectations.
>
> More to the point, what modules do that classes do not is provide a
> global namespace for functions defined within them, so that variables
> within them can be used (well, read - writing them requires a
> declaration) by the functions without extra qualification.
>
> > Regardless, all use cases you've listed are already satisfied by use
> > of the static and class method decorators. Methods decorated with
> > these do not require an instance initialization to use.
>
> staticmethod isn't technically required to use a method through the
> class (or subclasses), it simply provides the appropriate magic to allow
> it to be called through instances.
> --
> https://mail.python.org/mailman/listinfo/python-list
>

[toc] | [prev] | [next] | [standalone]

#110971

On Sun, 3 Jul 2016 01:34 am, Kevin Conway wrote:

> staticmethod isn't technically required to use a method through the class
> (or subclasses), it simply provides the appropriate magic to allow it to
> be called through instances.
> 
> For example, the following code covers all described use cases of the
> proposed namespace. 

It really doesn't. Here's your class:

> Methods are invoked without creating instances and 
> state is managed on the class object directly.
> 
> class CustomNs:
> 
>     stateful_data = 1
> 
>     @staticmethod
>     def echo(text):
>         print(text)
> 
>     @classmethod
>     def mutate(cls):
>         cls.stateful_data += 1
>         print(cls.stateful_data)


The class scope is not "first class" (pun intended), as you cannot refer to
class-level variables without prefixing them with the class, even from
within the class.

Although the class *may* be used without instantiating, this violates the
user's expectations and may be confusing.

I acknowledge that my own namespace has a similar problem, as it uses the
class keyword, but the use of an explicit Namespace decorator or metaclass
gives a hint that something is different. And if you try calling the
Namespace, as if to instantiate it, you get a TypeError.


[...]
> For the proponents of namespace, what is deficient in the above example
> that necessitates a language change?

It's not a language change. Although I'd like a new keyword, to avoid
the "but it looks like a class" problem, its not necessary.


Try getting this behaviour from within a class:


class Food(metaclass=Namespace):

    # (1) no special decorators required
    def spam(n):
        return ' '.join(['spam']*n)

    # (2) can call functions from inside the namespace
    breakfast = spam(5)

    # (3) no "cls" or "self" argument
    def lunch():
        # (4) can access variables using their undotted name
        return breakfast + ' and eggs'

    def supper():
        # (5) likewise functions (a special case of #4)
        return lunch() + ' and a fried slice of spam'

    def mutate(n):
        # global inside the namespace refers to the namespace,
        # not the surrounding module
        global breakfast
        breakfast = spam(5)



Everything written inside the namespace object could have started as top
level module code. I select the code, hit my editor's "Indent" command, and
insert a single line at the top to turn it into a namespace.

If I decide to move the code into a separate file, I just copy the block,
excluding the "class ... metaclass" header line, past into a new file,
select all, and Dedent. Done.

Can you do all of that with an ordinary class?





-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

[toc] | [prev] | [next] | [standalone]

#110975

On 07/02/2016 08:44 PM, Steven D'Aprano wrote:

> Try getting this behaviour from within a class:
>
>
> class Food(metaclass=Namespace):
>
>      # (1) no special decorators required
>      def spam(n):
>          return ' '.join(['spam']*n)
>
>      # (2) can call functions from inside the namespace
>      breakfast = spam(5)
>
>      # (3) no "cls" or "self" argument
>      def lunch():
>          # (4) can access variables using their undotted name
>          return breakfast + ' and eggs'
>
>      def supper():
>          # (5) likewise functions (a special case of #4)
>          return lunch() + ' and a fried slice of spam'
>
>      def mutate(n):
>          # global inside the namespace refers to the namespace,
>          # not the surrounding module
>          global breakfast
>          breakfast = spam(5)

> Can you do all of that with an ordinary class?

You can get #2 already, but not the rest (without your spiffy code ;) :

Python 3.5.1+ (3.5:f840608f79da, Apr 14 2016, 12:29:06)
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
 >>> class Huh:
...   def blah(text):
...     print('blah blah %s blah blah blah' % text)
...   blah('whatever')
...
blah blah whatever blah blah blah

--
~Ethan~

[toc] | [prev] | [next] | [standalone]

#111015

>> Regardless, all use cases you've listed are already satisfied by use of
>> the static and class method decorators. Methods decorated with these do
>> not require an instance initialization to use.

> And are significantly less easy to use, as the functions MUST refer to
each
> other by their dotted names.

My response to this may come off as a bit glib, but it isn't intended that
way. If the problem with using classes to satisfy the namespace need is
that it's unwieldy to use dot qualified paths then isn't that quite similar
to saying namespaces are unwieldy? Leveraging classes as a nested module
creates a de-facto internal namespace of "cls" for self reference and I'm
unsure of why that is unwanted but "module.namespace.blah" is wanted.

I suppose my issue is not so much that namespace objects are a bad idea as
that the proposal does little to express why the existing tools are
deficient enough to require a new concept.

> For the proponents of namespace, what is deficient in the above example
> that necessitates a language change?

>> It's not a language change.

Perhaps. My argument is that anything that introduces a new class-like
construct and set of lexical scoping rules is a language change. For
example, if this change went into 2.7.13 would Jython suddenly be broken
because it hasn't implemented the new scoping rules?

>> IIRC, classes came about as a "module in a module".

> I'm pretty sure they did not. Object oriented programming (and hence
> classes) came about from simulating real world objects, hence the
> name:

I realize my statement was ambiguous. I didn't mean to suggest that
classes, as an idea and OOP tool, were derived from the concept of a
namespace. I meant to say that the Python implementation of classes is
quite similar to the implementation of modules in the cPython code. The
commit messages from the earlier days (ref: ab5db02) don't carry much
detail so I'm mostly basing my belief of the implementation similarities
and anecdotes from the community. Right or wrong, I don't believe it has
much weight in the namespace discussion and I shouldn't have brought it up.

On Sun, Jul 3, 2016, 00:17 Ethan Furman <ethan@stoneleaf.us> wrote:

> On 07/02/2016 08:44 PM, Steven D'Aprano wrote:
>
> > Try getting this behaviour from within a class:
> >
> >
> > class Food(metaclass=Namespace):
> >
> >      # (1) no special decorators required
> >      def spam(n):
> >          return ' '.join(['spam']*n)
> >
> >      # (2) can call functions from inside the namespace
> >      breakfast = spam(5)
> >
> >      # (3) no "cls" or "self" argument
> >      def lunch():
> >          # (4) can access variables using their undotted name
> >          return breakfast + ' and eggs'
> >
> >      def supper():
> >          # (5) likewise functions (a special case of #4)
> >          return lunch() + ' and a fried slice of spam'
> >
> >      def mutate(n):
> >          # global inside the namespace refers to the namespace,
> >          # not the surrounding module
> >          global breakfast
> >          breakfast = spam(5)
>
> > Can you do all of that with an ordinary class?
>
> You can get #2 already, but not the rest (without your spiffy code ;) :
>
> Python 3.5.1+ (3.5:f840608f79da, Apr 14 2016, 12:29:06)
> [GCC 4.8.2] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> class Huh:
> ...   def blah(text):
> ...     print('blah blah %s blah blah blah' % text)
> ...   blah('whatever')
> ...
> blah blah whatever blah blah blah
>
> --
> ~Ethan~
> --
> https://mail.python.org/mailman/listinfo/python-list
>

[toc] | [prev] | [next] | [standalone]

#111018

On Monday, July 4, 2016 at 10:02:59 AM UTC+12, Kevin Conway wrote:
> If the problem with using classes to satisfy the namespace need is
> that it's unwieldy to use dot qualified paths then isn't that quite similar
> to saying namespaces are unwieldy?

Python has a simple solution to that:

    a = mod.a
    b = mod.b

etc

[toc] | [prev] | [next] | [standalone]

#111029

On 04/07/2016 00:21, Lawrence D’Oliveiro wrote:
> On Monday, July 4, 2016 at 10:02:59 AM UTC+12, Kevin Conway wrote:
>> If the problem with using classes to satisfy the namespace need is
>> that it's unwieldy to use dot qualified paths then isn't that quite similar
>> to saying namespaces are unwieldy?
>
> Python has a simple solution to that:
>
>     a = mod.a
>     b = mod.b

Except that then

      a = 0

doesn't do the same thing as:

      mod.a = 0

But it'll probably work for things that are not assigned to.

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.python

csiph-web