Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #6965 > unrolled thread

except KeyError, everywhere

Back to article view | Back to comp.lang.python

Contents

  except KeyError, everywhere Wilbert Berendsen <wbsoft@xs4all.nl> - 2011-06-03 22:08 +0200
    Re: except KeyError, everywhere Nobody <nobody@nowhere.com> - 2011-06-04 01:02 +0100
      Re: except KeyError, everywhere --> memoization Wilbert Berendsen <wbsoft@xs4all.nl> - 2011-06-05 00:27 +0200
      Re: except KeyError, everywhere "Gabriel Genellina" <gagsl-py2@yahoo.com.ar> - 2011-06-07 00:45 -0300
        Re: except KeyError, everywhere Ben Finney <ben+python@benfinney.id.au> - 2011-06-07 14:08 +1000
    Re: except KeyError, everywhere Ben Finney <ben+python@benfinney.id.au> - 2011-06-04 10:03 +1000
      Re: except KeyError, everywhere Ben Finney <ben+python@benfinney.id.au> - 2011-06-04 10:37 +1000

#6965 — except KeyError, everywhere

Hi,

I find myself all over the place associating objects with each other using 
dicts as caches:

something like this:

_cache = {}

def get_something(obj):
    """Returns the frobnicate-plugin for the specified object."""
    try:
        return _cache[obj]
    except KeyError:
        res = _cache[obj] = LargeClass(obj)
        return res


I would like a dict that would do this by itself, so that I could write:

_cache = somelazydict(LargeClass)

def get_something(obj):
    """Returns the frobnicate-plugin for the specified object."""
    return _cache[obj]

or even using the dict directly:

plugin_store = somelazydict(LargeClass)

It seems that I can't use defaultdict for this, because it calls the factory 
function without an argument, where I need to have the key as argument.

So I came up with this (using the __missing__ dict hook since Python 2.5):

class cachedict(dict):
    """A dict, but with a factory function as the first argument.
    
    On lookup, if the key is missing, the function is called with the key as
    argument. The returned value is also stored in the dict.
    
    """
    def __init__(self, func, *args, **kwargs):
        """Creates the dict, with the factory function as the first argument.
        
        All other arguments work just like dict.
        
        """
        dict.__init__(self, *args, **kwargs)
        self.func = func
    
    def __missing__(self, key):
        res = self[key] = self.func(key)
        return res

Are there other peoply using things like this? Is there a solution like this 
in the standard lib that I'm overlooking? Of course 'except KeyError' 
everywhere is not really a big deal...

With best regards,
Wilbert Berendsen

-- 
http://www.wilbertberendsen.nl/
"You must be the change you wish to see in the world."
        -- Mahatma Gandhi

[toc] | [next] | [standalone]

#6983

On Fri, 03 Jun 2011 22:08:16 +0200, Wilbert Berendsen wrote:

> I find myself all over the place associating objects with each other using 
> dicts as caches:

> Are there other peoply using things like this? Is there a solution like
> this in the standard lib that I'm overlooking?

The general concept is called "memoization". There isn't an implementation
in the standard library, but you'll find plenty of examples, e.g. (from
the first page of Google hits for "python memoization"):

http://wiki.python.org/moin/PythonDecoratorLibrary#Memoize
http://code.activestate.com/recipes/52201-memoizing-cacheing-function-return-values/
http://code.activestate.com/recipes/577219-minimalistic-memoization/

[toc] | [prev] | [next] | [standalone]

#7024 — Re: except KeyError, everywhere --> memoization

Hi,
Many thanks for everyone's explanations and pointers!
thanks!
Wilbert Berendsen

-- 
http://www.wilbertberendsen.nl/
"You must be the change you wish to see in the world."
        -- Mahatma Gandhi

[toc] | [prev] | [next] | [standalone]

#7131

En Fri, 03 Jun 2011 21:02:56 -0300, Nobody <nobody@nowhere.com> escribió:

> On Fri, 03 Jun 2011 22:08:16 +0200, Wilbert Berendsen wrote:
>
>> I find myself all over the place associating objects with each other  
>> using
>> dicts as caches:
>
> The general concept is called "memoization". There isn't an  
> implementation
> in the standard library

Yes, there is, in Python 3.2:
http://docs.python.org/py3k/library/functools.html#functools.lru_cache


-- 
Gabriel Genellina

[toc] | [prev] | [next] | [standalone]

#7135

"Gabriel Genellina" <gagsl-py2@yahoo.com.ar> writes:

> En Fri, 03 Jun 2011 21:02:56 -0300, Nobody <nobody@nowhere.com> escribió:
>
> > On Fri, 03 Jun 2011 22:08:16 +0200, Wilbert Berendsen wrote:
> >
> >> I find myself all over the place associating objects with each
> >> other using dicts as caches:
> >
> > The general concept is called "memoization". There isn't an
> > implementation in the standard library
>
> Yes, there is, in Python 3.2:
> http://docs.python.org/py3k/library/functools.html#functools.lru_cache

Beauty. Thanks!

-- 
 \         “If we don't believe in freedom of expression for people we |
  `\           despise, we don't believe in it at all.” —Noam Chomsky, |
_o__)                                                       1992-11-25 |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#6984

Wilbert Berendsen <wbsoft@xs4all.nl> writes:

> I find myself all over the place associating objects with each other using 
> dicts as caches:
>
> something like this:
>
> _cache = {}
>
> def get_something(obj):
>     """Returns the frobnicate-plugin for the specified object."""
>     try:
>         return _cache[obj]
>     except KeyError:
>         res = _cache[obj] = LargeClass(obj)
>         return res

You seem to be looking for the Memoize pattern
<URL:https://secure.wikimedia.org/wikipedia/en/wiki/Memoization>.

It's best to implement Memoize as a Python decorator in one place
<URL:http://wiki.python.org/moin/PythonDecoratorLibrary#Memoize>.

Once you have that decorator, apply it to any function you like::

    @memoized
    def get_something(obj):
        """ Returns the frobnicate-plugin for the specified object. """
        res = LargeClass(obj)
        return res

-- 
 \         “Faith may be defined briefly as an illogical belief in the |
  `\                  occurrence of the improbable.” —Henry L. Mencken |
_o__)                                                                  |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#6988

Ben Finney <ben+python@benfinney.id.au> writes:

> It's best to implement Memoize as a Python decorator in one place
> <URL:http://wiki.python.org/moin/PythonDecoratorLibrary#Memoize>.

Michele Simionato discusses a better implementation of a Memoize
decorator in the documentation for his useful ‘decorator’ library
<URL:http://micheles.googlecode.com/hg/decorator/documentation.html>.

-- 
 \                “Stop — Drive sideways.” —detour sign, Kyushu, Japan |
  `\                                                                   |
_o__)                                                                  |
Ben Finney

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web