Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #105368 > unrolled thread

Re: Static caching property

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Static caching property Ian Kelly <ian.g.kelly@gmail.com> - 2016-03-21 10:15 -0600
    Re: Static caching property Steven D'Aprano <steve@pearwood.info> - 2016-03-22 03:36 +1100
      Re: Static caching property "Joseph L. Casale" <jcasale@activenetwerx.com> - 2016-03-21 16:49 +0000
      Re: Static caching property Chris Angelico <rosuav@gmail.com> - 2016-03-22 03:54 +1100
      Re: Static caching property "Joseph L. Casale" <jcasale@activenetwerx.com> - 2016-03-21 17:03 +0000
      Re: Static caching property Ian Kelly <ian.g.kelly@gmail.com> - 2016-03-21 11:44 -0600
      Re: Static caching property Ethan Furman <ethan@stoneleaf.us> - 2016-03-21 10:45 -0700
      Re: Static caching property Ian Kelly <ian.g.kelly@gmail.com> - 2016-03-21 11:48 -0600
        Re: Static caching property Steven D'Aprano <steve@pearwood.info> - 2016-03-22 11:05 +1100
          Re: Static caching property Chris Angelico <rosuav@gmail.com> - 2016-03-22 11:15 +1100
          Re: Static caching property Ian Kelly <ian.g.kelly@gmail.com> - 2016-03-22 07:30 -0600

#105368 — Re: Static caching property

On Mon, Mar 21, 2016 at 9:38 AM, Joseph L. Casale
<jcasale@activenetwerx.com> wrote:
> With non static properties, you can use a decorator that overwrites the
> method on the instance with an attribute containing the methods return
> effectively caching it.

Can you give an example of what you mean?

> What technique for a static property can be used to accomplish what the
> descriptor protocol does?
>
> I need to cache the results of a method on a class across all instances.

Why not do the same thing but using a class attribute instead of an
instance attribute?

[toc] | [next] | [standalone]

#105370

On Tue, 22 Mar 2016 03:15 am, Ian Kelly wrote:

> On Mon, Mar 21, 2016 at 9:38 AM, Joseph L. Casale
> <jcasale@activenetwerx.com> wrote:
>> With non static properties, you can use a decorator that overwrites the
>> method on the instance with an attribute containing the methods return
>> effectively caching it.
> 
> Can you give an example of what you mean?

I think Joseph is using "static" in the Java sense of being associated with
the class rather than an instance. (In Java, members of classes must be
known at compile-time.)


>> What technique for a static property can be used to accomplish what the
>> descriptor protocol does?
>>
>> I need to cache the results of a method on a class across all instances.
> 
> Why not do the same thing but using a class attribute instead of an
> instance attribute?

Properties don't work when called from a class:

py> class Test(object):
...     @property
...     def x(self):
...             return 999
...
py> Test.x == 999
False
py> Test.x
<property object at 0xb7a148b4>



But what you can do is have the property refer to a class attribute:


py> class Test(object):
...     _private = 999
...     @property
...     def x(self):
...             return type(self)._private
...     @x.setter
...     def x(self, value):
...             type(self)._private = value
...
py> a = Test()
py> b = Test()
py> c = Test()
py> a.x
999
py> b.x = 50
py> c.x
50
py> a.x
50



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#105373

> I think Joseph is using "static" in the Java sense of being associated with
> the class rather than an instance. (In Java, members of classes must be
> known at compile-time.)

Yup, so a single value on the class itself, not instance specific.

> But what you can do is have the property refer to a class attribute:
> 
> 
> py> class Test(object):
> ...     _private = 999
> ...     @property
> ...     def x(self):
> ...             return type(self)._private
> ...     @x.setter
> ...     def x(self, value):
> ...             type(self)._private = value
> ...
> py> a = Test()
> py> b = Test()
> py> c = Test()
> py> a.x
> 999
> py> b.x = 50
> py> c.x
> 50
> py> a.x
> 50

Right, but _private refers to an api call that is expensive and may not even be accessed,
so while I may new up three instances of Test across a, b and c, if none of those end up
accessing var x, it shouldn't get fetched. Without some deferred execution, if the value
of _private is a callable whose return value is what I am interested in, it gets invoked the
moment the class is compiled.

In the non static sense this is trivial to accomplish with the descriptor protocol, I am just not
clear for the static sense.

Thanks Ian and Steven,
jlc

[toc] | [prev] | [next] | [standalone]

#105374

On Tue, Mar 22, 2016 at 3:49 AM, Joseph L. Casale
<jcasale@activenetwerx.com> wrote:
> Right, but _private refers to an api call that is expensive and may not even be accessed,
> so while I may new up three instances of Test across a, b and c, if none of those end up
> accessing var x, it shouldn't get fetched. Without some deferred execution, if the value
> of _private is a callable whose return value is what I am interested in, it gets invoked the
> moment the class is compiled.
>
> In the non static sense this is trivial to accomplish with the descriptor protocol, I am just not
> clear for the static sense.

One solution is to use descriptor protocol on the class, which means
using a metaclass. I'm not sure it's the best option, but it is an
option.

ChrisA

[toc] | [prev] | [next] | [standalone]

#105375

> One solution is to use descriptor protocol on the class, which means
> using a metaclass. I'm not sure it's the best option, but it is an
> option.

I will look at that, I wonder if however I am not over complicating it:

class Foo:
    _bar = None
    @property
    def expensive(self):
        if Foo._bar is None:
            import something
            Foo._bar = something.expensive()
        return Foo._bar

Somewhat naive, but a test with if is pretty cheap...
Thanks Chris,
jlc

[toc] | [prev] | [next] | [standalone]

#105380

On Mon, Mar 21, 2016 at 10:36 AM, Steven D'Aprano <steve@pearwood.info> wrote:
> On Tue, 22 Mar 2016 03:15 am, Ian Kelly wrote:
>> Why not do the same thing but using a class attribute instead of an
>> instance attribute?
>
> Properties don't work when called from a class:

Properties specifically do not, but descriptors in general do.

[toc] | [prev] | [next] | [standalone]

#105381

On 03/21/2016 10:03 AM, Joseph L. Casale wrote:

>> One solution is to use descriptor protocol on the class, which means
>> using a metaclass. I'm not sure it's the best option, but it is an
>> option.
>
> I will look at that, I wonder if however I am not over complicating it:
>
> class Foo:
>      _bar = None
>      @property
>      def expensive(self):
>          if Foo._bar is None:
>              import something
>              Foo._bar = something.expensive()
>          return Foo._bar
>
> Somewhat naive, but a test with if is pretty cheap...

A slightly cleaner approach (but only slightly):

   class Cache(object):
       _sentinal = object()
       def __init__(self, expensive_func):
           self.value = self._sentinal
           self.func = expensive_func
       def __get__(self, *args):
           if self.value is self._sentinal:
               self.value = self.func()
          return self.func()

The advantages:

- only one location in the class
- works correctly whether accessed via class or instance
- clue as to functionality in the name

--
~Ethan~

[toc] | [prev] | [next] | [standalone]

#105382

On Mon, Mar 21, 2016 at 10:54 AM, Chris Angelico <rosuav@gmail.com> wrote:
> On Tue, Mar 22, 2016 at 3:49 AM, Joseph L. Casale
> <jcasale@activenetwerx.com> wrote:
>> Right, but _private refers to an api call that is expensive and may not even be accessed,
>> so while I may new up three instances of Test across a, b and c, if none of those end up
>> accessing var x, it shouldn't get fetched. Without some deferred execution, if the value
>> of _private is a callable whose return value is what I am interested in, it gets invoked the
>> moment the class is compiled.
>>
>> In the non static sense this is trivial to accomplish with the descriptor protocol, I am just not
>> clear for the static sense.
>
> One solution is to use descriptor protocol on the class, which means
> using a metaclass. I'm not sure it's the best option, but it is an
> option.

You don't actually need a metaclass for this:

>>> class Desc:
...     def __get__(self, obj, type=None):
...         if not type._cached_value:
...             type._cached_value = compute_value(type)
...         return type._cached_value
...
>>> def compute_value(x): return 42
...
>>> class Test:
...     foo = Desc()
...     _cached_value = None
...
>>> Test._cached_value
>>> Test.foo
42
>>> Test._cached_value
42

[toc] | [prev] | [next] | [standalone]

#105416

On Tue, 22 Mar 2016 04:48 am, Ian Kelly wrote:

> You don't actually need a metaclass for this:
> 
>>>> class Desc:
> ...     def __get__(self, obj, type=None):
> ...         if not type._cached_value:
> ...             type._cached_value = compute_value(type)
> ...         return type._cached_value


This won't quite work. What if the cached value happens to be falsey, yet
still expensive to compute? You should compare it to a known sentinel which
the expensive function will never return (possibly None). Or use a hasattr
test: if not hasattr(type, '_cached_value').

But my favourite is to combine them:


class Desc:
    def __get__(self, obj, type):
        sentinel = object()  # guaranteed to be unique
        value = getattr(type, _cached_value, sentinel)
        if value is sentinel:
            value = type._cached_value = compute_value(type)
        return value



Also, you don't need the default type=None. The descriptor protocol should
never call __get__ without supplying the type. The obj may be None, but I
don't believe there are any circumstances where type will be None.

Which is good, because if it is:

AttributeError: 'NoneType' object has no attribute '_cached_value'



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#105419

On Tue, Mar 22, 2016 at 11:05 AM, Steven D'Aprano <steve@pearwood.info> wrote:
> But my favourite is to combine them:
>
>
> class Desc:
>     def __get__(self, obj, type):
>         sentinel = object()  # guaranteed to be unique
>         value = getattr(type, _cached_value, sentinel)
>         if value is sentinel:
>             value = type._cached_value = compute_value(type)
>         return value
>
>
>

That seems like overkill. Inside getattr is the equivalent of:

try: return type._cached_value
except AttributeError: return sentinel

So skip getattr/hasattr and just use try/except:

class Desc:
    def __get__(self, obj, type):
        try:
            return type._cached_value
        except AttributeError:
            value = type._cached_value = compute_value(type)
            return value

ChrisA

[toc] | [prev] | [next] | [standalone]

#105482

On Mon, Mar 21, 2016 at 6:05 PM, Steven D'Aprano <steve@pearwood.info> wrote:
> On Tue, 22 Mar 2016 04:48 am, Ian Kelly wrote:
>
>> You don't actually need a metaclass for this:
>>
>>>>> class Desc:
>> ...     def __get__(self, obj, type=None):
>> ...         if not type._cached_value:
>> ...             type._cached_value = compute_value(type)
>> ...         return type._cached_value
>
>
> This won't quite work. What if the cached value happens to be falsey, yet
> still expensive to compute? You should compare it to a known sentinel which
> the expensive function will never return (possibly None). Or use a hasattr
> test: if not hasattr(type, '_cached_value').

Sure. This was just a quick-and-dirty demonstration that I threw
together in 30 seconds.

> Also, you don't need the default type=None. The descriptor protocol should
> never call __get__ without supplying the type. The obj may be None, but I
> don't believe there are any circumstances where type will be None.

Why do the examples in the Python docs use type=None? I literally just
copy-pasted the method signature from there, since I can never
remember which argument comes first.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web