Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #51553 > unrolled thread

Python descriptor protocol (for more or less structured data)

Back to article view | Back to comp.lang.python

Contents

  Python descriptor protocol (for more or less structured data) CWr <christoph.wruck@gmail.com> - 2013-07-30 06:04 -0700
    Re: Python descriptor protocol (for more or less structured data) Peter Otten <__peter__@web.de> - 2013-07-30 16:35 +0200
      Re: Python descriptor protocol (for more or less structured data) CWr <christoph.wruck@gmail.com> - 2013-07-31 02:16 -0700
        Re: Python descriptor protocol (for more or less structured data) Terry Reedy <tjreedy@udel.edu> - 2013-07-31 13:44 -0400

#51553 — Python descriptor protocol (for more or less structured data)

Hi together,

Some years ago I started a small WSGI project at my university. Since then the project was grown up every year. Some classes have more than 600 lines of code with (incl. boiler-plates mostly in descriptors/properties). 

Many of these properties are similar or have depencies among themselves.
The idea is to grouping similar properties like:

new style:
----------
>>>m = MyClass(...)
>>>m.attr = 'some; complex:data#string'

>>>m.attr.value
'some'
>>>m.attr.extras
{'complex':('data','string')}

I wrote this descriptor:

class Descr:
    
    def __init__(self, value):
        self.attribute = self.__class__.__name__
        self.__set__(None, value)

    def __get__(self, obj, Type=None):
        return getattr(obj, self.attribute, self)
        
    def __set__(self, obj, value):
        if obj is None: # descripting yourself
            # do something here ...
            self.value = value
        else:
            if hasattr(obj, self.attribute):
                self.__get__(obj).__set__(None, value)
            else:
                setattr(obj, self.attribute, type(self)(value))

This works fine as long as the value attribute of Descr is read-only and the 
user have to use the descriptor interface e.g. __get__/__set__. Because
it's not guaranteed that the user gets a seperated instance of Descr which
will be living in obj.__dict__. If obj is None the descriptor will be returned
themselves.

But I would like that the user can use the following statement:

>>>m = MyClass(...)
>>>m.attr = 'some; complex:data#string'
>>>m.attr.value
'some'
>>>m.attr.value = 'some other'
>>>m.attr.value
'some other'

But this usage will be problematic. If the descriptor returned themselves (default case) and the user modified the value, he modified the default
value without to create a seperated instance attribute.

>>>class C:
>>>    def __init__(self, value):
>>>        if not value is None:
>>>            self.attr = value
>>>    attr = Descr('default value')

>>># explicit default usage (no problem): 
>>>C.attr.value
'default value'

>>>a = C()
>>>a.attr.value
'default value'

The following is the main Problem:

>>>a.attr.value = 'other'
>>>C.attr.value
'other'

The user could think that a new instance based value will be created. But it
isn't.

It will works fine only if I assign a value explicitly.

>>>m = MyClass(value='test')
>>>m.__dict__
>>>{'Descr':<object of ...>}

Has anyone had a similar problem in the past? Or I am on the wrong way.

Kind Regards,
Chris

Sorry for my terrible english ...





 

[toc] | [next] | [standalone]

#51561

CWr wrote:

> Some years ago I started a small WSGI project at my university. Since then
> the project was grown up every year. Some classes have more than 600 lines
> of code with (incl. boiler-plates mostly in descriptors/properties).
> 
> Many of these properties are similar or have depencies among themselves.
> The idea is to grouping similar properties like:
> 
> new style:
> ----------
>>>>m = MyClass(...)
>>>>m.attr = 'some; complex:data#string'
> 
>>>>m.attr.value
> 'some'
>>>>m.attr.extras
> {'complex':('data','string')}
> 
> I wrote this descriptor:
> 
> class Descr:
>     
>     def __init__(self, value):
>         self.attribute = self.__class__.__name__
>         self.__set__(None, value)
> 
>     def __get__(self, obj, Type=None):
>         return getattr(obj, self.attribute, self)
>         
>     def __set__(self, obj, value):
>         if obj is None: # descripting yourself
>             # do something here ...
>             self.value = value
>         else:
>             if hasattr(obj, self.attribute):
>                 self.__get__(obj).__set__(None, value)
>             else:
>                 setattr(obj, self.attribute, type(self)(value))

You must not store per-object data in the descriptor. I suggest a naming 
convention (the internal data for obj.attr is stored in obj._attr) together 
with a value class that handles breaking of the string into attributes of an 
instance of itself:

class StructuredAttribute:
    def __init__(self, name, make_default):
        self.name = name
        self.make_default = make_default

    def __get__(self, obj, type=None):
        if obj is None:
            return self
        _name = "_" + self.name
        try:
            return getattr(obj, _name)
        except AttributeError:
            setattr(obj, _name, self.make_default())
        return getattr(obj, _name)

    def __set__(self, obj, value):
        self.__get__(obj).update(value)


class Value:
    def __init__(self, value):
        self.update(value)
    def update(self, value):
        if isinstance(value, str):
            self.value, sep, rest = value.partition(";")
            self.extras = dict(item.partition("#")[::2] for item in 
rest.split())
        else:
            self.value = value.value
            self.extras = value.extras
    def __repr__(self):
        return repr("{}; {}".format(self.value, " ".join("{}:
{}".format(*item) for item in self.extras.items())))

def make_default_value():
    return Value("some; complex:data#string")

class A:
    attr = StructuredAttribute("alpha", make_default_value)

def show(obj):
    print("attr:", obj.attr)
    print("attr.value:", obj.attr.value)
    print("attr.extras:", obj.attr.extras)

a = A()
show(a)
newvalue = "whatever"
print("updating value to", newvalue)
a.attr.value = newvalue
show(a)

That's the general idea if you want "setattr polymorphism". Personally I 
would go with simpler standard attributes:

class A:
    def __init__(self):
        self.attr = Value(...)

a = A()
a.value = Value(...)
a.value.extras = ...

[toc] | [prev] | [next] | [standalone]

#51634

Peter, thanks for your response.
Sure, you are right when you say that's easier to use standard attribute assigning via __init__.

But my intention was:
- reducing the complexiticity of __init__
- avoiding boiler-plates (mostly property descriptors inside of the main class)
- creating instances (for complex data strings) only if they will be needed, otherwise use default instances (descriptors) 
- make it prossible that the data structure can be used in static context - like MyClass.attr - to get default values

Standard procedure:

>>>class C:
>>>    def __init__(self, one, two=None, three=None, four=None, five=None, ...):
>>>        if not two is None:
>>>            self.two = Value(two)
>>>        else:
>>>            self.two = Value(self.DEFAULT_4_TWO)
>>>        ...

vs:

>>>class C:
>>>
>>>    two = MyDescriptor('default for two')
>>>
>>>    def __init__(self, one, two=None, three=None, four=None, five=None, ...):
>>>        self.one = one
>>>        if not two is None:
>>>            self.two = two
>>>        ...

Probably it will be necessary to set the attribute at first access. Alternatively it may be possible to observe the descriptor til an attribute will be setted e.g. instance.attr.value = 'whatever'. At this point a new instance (like Value) should be created on obj.__dict__.

It's the procedure what I'm looking for. ;)

Kind Regards,
Chris

[toc] | [prev] | [next] | [standalone]

#51675

On 7/31/2013 5:16 AM, CWr wrote:
> Peter, thanks for your response.
> Sure, you are right when you say that's easier to use standard attribute assigning via __init__.
>
> But my intention was:
> - reducing the complexiticity of __init__
> - avoiding boiler-plates (mostly property descriptors inside of the main class)
> - creating instances (for complex data strings) only if they will be needed, otherwise use default instances (descriptors)
> - make it prossible that the data structure can be used in static context - like MyClass.attr - to get default values
>
> Standard procedure:
>
>>>> class C:

     DEFAULT_4_TWO = <somethint> # for following code to work

>>>>     def __init__(self, one, two=None, three=None, four=None, five=None, ...):
>>>>         if not two is None:

"if two is not None:"  reads better and is the preferred form.
'is not' is a single comparison operator, just like '!=' and 'not in'. 
The current CPython AST or peephole optimizer happens to notice that the 
'is' operator followed by the 'not' operator can be replaced by the 'is 
not' operator, but this is not guaranteed for all implementations.

>>>>             self.two = Value(two)
>>>>         else:
>>>>             self.two = Value(self.DEFAULT_4_TWO)

self.two = Value(two if two is not None else self.DEFAULT_4_TWO)

There is no need to introduce the new name DEFAULT_4_TWO. It is a 
symptom of using the wrong namespace to get the default.

class C:
     two = <default>
     def __init__(self, two=None):
         self.two = Value(two if two is not None else C.two)

-- 
Terry Jan Reedy

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web