Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #42617 > unrolled thread

Decorating functions without losing their signatures

Back to article view | Back to comp.lang.python

Contents

  Decorating functions without losing their signatures Rotwang <sg552@hotmail.co.uk> - 2013-04-03 02:05 +0100
    Re: Decorating functions without losing their signatures Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-04-03 04:15 +0000
      Re: Decorating functions without losing their signatures Rotwang <sg552@hotmail.co.uk> - 2013-04-04 02:53 +0100
    Re: Decorating functions without losing their signatures Jan Riechers <janpeterr@freenet.de> - 2013-04-03 08:06 +0300
    Re: Decorating functions without losing their signatures Michele Simionato <michele.simionato@gmail.com> - 2013-04-03 18:18 -0700
      Re: Decorating functions without losing their signatures Rotwang <sg552@hotmail.co.uk> - 2013-04-04 03:17 +0100
    Re: Decorating functions without losing their signatures Rotwang <sg552@hotmail.co.uk> - 2013-04-04 02:44 +0100

#42617 — Decorating functions without losing their signatures

Hi all,

Here's a Python problem I've come up against and my crappy solution. 
Hopefully someone here can suggest something better. I want to decorate 
a bunch of functions with different signatures; for example, I might 
want to add some keyword-only arguments to all functions that return 
instances of a particular class so that the caller can create instances 
with additional attributes. So I do something like this:

import functools

def mydecorator(f):
     @functools.wraps(f)
     def wrapped(*args, attribute = None, **kwargs):
         result = f(*args, **kwargs)
         result.attribute = attribute
         return result
     return wrapped

@mydecorator
def f(x, y = 1, *a, z = 2, **k):
	return something

The problem with this is, when I subsequently type 'f(' in IDLE, the 
signature prompt that appears is not very useful; it looks like this:

(*args, attribute=None, **kwargs)

whereas I'd like it to look like this:

(x, y=1, *a, z=2, attribute=None, **k)


After thinking about it for a while I've come up with the following 
abomination:

import inspect

def sigwrapper(sig):
   if not isinstance(sig, inspect.Signature):
     sig = inspect.signature(sig)
   def wrapper(f):
     ps = 'args = []\n\t\t'
     ks = 'kwargs = {}\n\t\t'
     for p in sig.parameters.values():
       if p.kind in (p.POSITIONAL_ONLY, p.POSITIONAL_OR_KEYWORD):
         ps = '%sargs.append(%s)\n\t\t' % (ps, p.name)
       elif p.kind == p.VAR_POSITIONAL:
         ps = '%sargs.extend(%s)\n\t\t' % (ps, p.name)
       elif p.kind == p.KEYWORD_ONLY:
         ks = '%skwargs[%r] = %s\n\t\t' % (ks, p.name, p.name)
       elif p.kind == p.VAR_KEYWORD:
         ks = '%skwargs.update(%s)\n\t\t' % (ks, p.name)
     loc = {'wrapped': f}
     defstring = ('def wrapouter(wrapped = wrapped):'
                  '\n\tdef wrapinner%s:'
                  '\n\t\t%s%sreturn wrapped(*args, **kwargs)'
                  '\n\treturn wrapinner' % (sig, ps, ks))
     exec(defstring, f.__globals__, loc)
     return loc['wrapouter']()
   return wrapper

The function sigwrapper() may be passed an inspect.Signature object sig 
(or function, if that function has the right signature) and returns a 
decorator that gives any function the signature sig. I can then replace 
my original decorator with something like

def mydecorator(f):
     sig = inspect.signature(f)
     sig = do_something(sig) # add an additional kw-only argument to sig
     @functools.wraps(f)
     @sigwrapper(sig)
     def wrapped(*args, attribute = None, **kwargs):
         result = f(*args, **kwargs)
         result.attribute = attribute
         return result
     return wrapped

It seems to work, but I don't like it. Does anyone know of a better way 
of doing the same thing?

[toc] | [next] | [standalone]

#42626

On Wed, 03 Apr 2013 02:05:31 +0100, Rotwang wrote:

> Hi all,
> 
> Here's a Python problem I've come up against and my crappy solution.
> Hopefully someone here can suggest something better. I want to decorate
> a bunch of functions with different signatures;
[...]
> After thinking about it for a while I've come up with the following
> abomination:
[...]
> It seems to work, but I don't like it. Does anyone know of a better way
> of doing the same thing?


Wait until Python 3.4 or 3.5 (or Python 4000?) when functools.wraps 
automatically preserves the function signature?

Alas, I think this is a hard problem to solve with current Python. You 
might like to compare your solution with that of Michele Simionato's 
"decorator" module:

http://micheles.googlecode.com/hg/decorator/documentation.html


See this for some other ideas:

http://numericalrecipes.wordpress.com/2009/05/25/signature-preserving-
function-decorators/



Good luck!




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#42724

On 03/04/2013 05:15, Steven D'Aprano wrote:
> On Wed, 03 Apr 2013 02:05:31 +0100, Rotwang wrote:
>
>> Hi all,
>>
>> Here's a Python problem I've come up against and my crappy solution.
>> Hopefully someone here can suggest something better. I want to decorate
>> a bunch of functions with different signatures;
> [...]
>> After thinking about it for a while I've come up with the following
>> abomination:
> [...]
>> It seems to work, but I don't like it. Does anyone know of a better way
>> of doing the same thing?
>
>
> Wait until Python 3.4 or 3.5 (or Python 4000?) when functools.wraps
> automatically preserves the function signature?
>
> Alas, I think this is a hard problem to solve with current Python. You
> might like to compare your solution with that of Michele Simionato's
> "decorator" module:
>
> http://micheles.googlecode.com/hg/decorator/documentation.html
>
>
> See this for some other ideas:
>
> http://numericalrecipes.wordpress.com/2009/05/25/signature-preserving-
> function-decorators/
>
>
>
> Good luck!

Thanks. It'll take me a while to fully absorb the links, but it looks 
like both are similarly based on abusing the exec function.

Thanks to Jan too.

[toc] | [prev] | [next] | [standalone]

#42628

On 03.04.2013 04:05, Rotwang wrote:
> Hi all,
>
> Here's a Python problem I've come up against and my crappy solution.
> Hopefully someone here can suggest something better. I want to decorate
> a bunch of functions with different signatures; for example, I might
> want to add some keyword-only arguments to all functions that return
> instances of a particular class so that the caller can create instances
> with additional attributes. So I do something like this:
[...]
> It seems to work, but I don't like it. Does anyone know of a better way
> of doing the same thing?

Hi,

I think you might want to check out that Pycon2013 Video about Metaclass 
Prgoramming of David Beazley:
http://www.pyvideo.org/video/1716/python-3-metaprogramming

He explains how to passing attributes, such creating custom classes on 
demand and returning there signatures even when wrapped.

I think that was what you wanted to archive?

Regards
Jan

[toc] | [prev] | [next] | [standalone]

#42720

On Wednesday, April 3, 2013 3:05:31 AM UTC+2, Rotwang wrote:
> After thinking about it for a while I've come up with the following 
> 
> abomination


Alas, there is actually no good way to implement this feature in pure Python without abominations. Internally the decorator module does something similar to what you are doing. However, instead of cooking up yourself your custom solution, it is probably better if you stick to the decorator module which has been used in production for several years and has hundreds of thousands of downloads. I am not claiming that it is bug free, but it is stable, bug reports come very rarely and it works for all versions of Python from 2.5 to 3.3.

[toc] | [prev] | [next] | [standalone]

#42728

On 04/04/2013 02:18, Michele Simionato wrote:
> On Wednesday, April 3, 2013 3:05:31 AM UTC+2, Rotwang wrote:
>> After thinking about it for a while I've come up with the following
>>
>> abomination
>
> Alas, there is actually no good way to implement this feature in pure
> Python without abominations. Internally the decorator module does
> something similar to what you are doing. However, instead of cooking up
> yourself your custom solution, it is probably better if you stick to
> the decorator module which has been used in production for several
> years and has hundreds of thousands of downloads. I am not claiming
> that it is bug free, but it is stable, bug reports come very rarely and
> it works for all versions of Python from 2.5 to 3.3.

Thanks, I'll check it out. Looking at the link Steven provided, I didn't 
see an easy way to add additional keyword-only arguments to a function's 
signature, though (but then I've yet to figure out how the FunctionMaker 
class works).

[toc] | [prev] | [next] | [standalone]

#42723

On 03/04/2013 02:05, Rotwang wrote:
> [...]
>
> After thinking about it for a while I've come up with the following
> abomination:
>
> import inspect
>
> def sigwrapper(sig):
>    if not isinstance(sig, inspect.Signature):
>      sig = inspect.signature(sig)
>    def wrapper(f):
>      ps = 'args = []\n\t\t'
>      ks = 'kwargs = {}\n\t\t'
>      for p in sig.parameters.values():
>        if p.kind in (p.POSITIONAL_ONLY, p.POSITIONAL_OR_KEYWORD):
>          ps = '%sargs.append(%s)\n\t\t' % (ps, p.name)
>        elif p.kind == p.VAR_POSITIONAL:
>          ps = '%sargs.extend(%s)\n\t\t' % (ps, p.name)
>        elif p.kind == p.KEYWORD_ONLY:
>          ks = '%skwargs[%r] = %s\n\t\t' % (ks, p.name, p.name)
>        elif p.kind == p.VAR_KEYWORD:
>          ks = '%skwargs.update(%s)\n\t\t' % (ks, p.name)
>      loc = {'wrapped': f}
>      defstring = ('def wrapouter(wrapped = wrapped):'
>                   '\n\tdef wrapinner%s:'
>                   '\n\t\t%s%sreturn wrapped(*args, **kwargs)'
>                   '\n\treturn wrapinner' % (sig, ps, ks))
>      exec(defstring, f.__globals__, loc)
>      return loc['wrapouter']()
>    return wrapper

Oops! Earlier I found out the hard way that this fails when the 
decorated function has arguments called 'args' or 'kwargs'. Here's a 
modified version that fixes said bug, but presumably not the many others 
I haven't noticed yet:

def sigwrapper(sig):
   if not isinstance(sig, inspect.Signature):
     sig = inspect.signature(sig)
   n = 0
   while True:
     pn = 'p_%i' % n
     kn = 'k_%i' % n
     if pn not in sig.parameters and kn not in sig.parameters:
       break
     n += 1
   ps = '%s = []\n\t\t' % pn
   ks = '%s = {}\n\t\t' % kn
   for p in sig.parameters.values():
     if p.kind in (p.POSITIONAL_ONLY, p.POSITIONAL_OR_KEYWORD):
       ps = '%s%s.append(%s)\n\t\t' % (ps, pn, p.name)
     elif p.kind == p.VAR_POSITIONAL:
       ps = '%s%s.extend(%s)\n\t\t' % (ps, pn, p.name)
     elif p.kind == p.KEYWORD_ONLY:
       ks = '%s%s[%r] = %s\n\t\t' % (ks, kn, p.name, p.name)
     elif p.kind == p.VAR_KEYWORD:
       ks = '%s%s.update(%s)\n\t\t' % (ks, kn, p.name)
   defstring = ('def wrapouter(wrapped = wrapped):'
                '\n\tdef wrapinner%s:'
                '\n\t\t%s%sreturn wrapped(*%s, **%s)'
                '\n\treturn wrapinner' % (sig, ps, ks, pn, kn))
   def wrapper(f):
     loc = {'wrapped': f}
     exec(defstring, f.__globals__, loc)
     return loc['wrapouter']()
   return wrapper

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web