Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #110646 > unrolled thread

#110646

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  #110646 
    #110650 
    #110671 
    #110798 
      #110801 
      #110825 
        #110828 
          #110829 
          #110832 
      #110888 
        #110920 
          #110926 
          #110947 
          #110950 
    #110799 
    #110921 
      #110942 
        #110946 
          #110973 
            #110976 
              #110992 
                #111010 
                  #111016 
                    #111019 
                      #111020 
                        #111022 
                          #111026 
                            #111031 
                      #111025 
                        #111027 
                          #111030 
      #110943 
      #110944 

Page 1 of 2  [1] 2  Next page →

#110646 — #110646

Howdy all,

I want an explicit replacement for a common decorator idiom.

There is a clever one-line decorator that has been copy-pasted without
explanation in many code bases for many years::

    decorator_with_args = lambda decorator: lambda *args, **kwargs: lambda func: decorator(func, *args, **kwargs)

My problem with this is precisely that it is clever: it explains nothing
about what it does, has many moving parts that are not named, it is
non-obvious and lacks expressiveness.

Even the widely-cited ActiveState recipe by Peter Hunt from 2005
<URL:http://code.activestate.com/recipes/465427-simple-decorator-with-arguments/>
gives no clue as to what this is doing internally nor what the names of
its parts should be.

I would like to see a more Pythonic, more explicit and expressive
replacement with its component parts easily understood.

-- 
 \     “[H]ow deep can a truth be — indeed, how true can it be — if it |
  `\             is not built from facts?” —Kathryn Schulz, 2015-10-19 |
_o__)                                                                  |
Ben Finney

[toc] | [next] | [standalone]

#110650 — #110650

Ben Finney <ben+python@benfinney.id.au> writes:
>     decorator_with_args = lambda decorator: lambda *args, **kwargs:
> lambda func: decorator(func, *args, **kwargs)
> I would like to see a more Pythonic, more explicit and expressive
> replacement with its component parts easily understood.

How's this:

    from functools import partial
    def dwa(decorator):
        def wrap(*args,**kwargs):
                return partial(decorator, *args, **kwargs)
        return wrap

[toc] | [prev] | [next] | [standalone]

#110671 — #110671

On Tuesday 28 June 2016 15:02, Ben Finney wrote:

> Howdy all,
> 
> I want an explicit replacement for a common decorator idiom.
> 
> There is a clever one-line decorator that has been copy-pasted without
> explanation in many code bases for many years::
> 
>     decorator_with_args = lambda decorator: lambda *args, **kwargs: lambda
>     func: decorator(func, *args, **kwargs)

I've never seen it before, and I'll admit I really had to twist my brain to 
understand it, but I came up with an example showing the traditional style of 
decorator-factory versus this meta-decorator.

# Standard idiom.

def chatty(name, age, verbose=True):  # the factory
    def decorator(func):  # the decorator returned by the factory
        if verbose:
            print("decorating function...")
        @functools.wraps(func)
        def inner(*args, **kwargs):
            print("Hi, my name is %s and I am %d years old!" % (name, age))
            return func(*args, **kwargs)
        return inner
    return decorator

@chatty("Bob", 99)
def calculate(x, y, z=1):
    return x+y-z


# Meta-decorator variant.

decorator_with_args = (lambda decorator: lambda *args, **kwargs: lambda func: 
decorator(func, *args, **kwargs))

@decorator_with_args
def chatty(func, name, age, verbose=True):
    if verbose:
        print("decorating function...")
    @functools.wraps(func)
    def inner(*args, **kwargs):
        print("Hi, my name is %s and I am %d years old!" % (name, age))
        return func(*args, **kwargs)
    return inner

@chatty("Bob", 99)
def calculate(x, y, z=1):
    return x+y-1



I agree -- it's very clever, and completely opaque in how it works. The easy 
part is expanding the lambdas:

def decorator_with_args(decorator):
    def inner(*args, **kwargs):
        def innermost(func):
            return decorator(func, *args, **kwargs)
        return innermost
    return inner


but I'm not closer to having good names for inner and innermost than you.


-- 
Steve

[toc] | [prev] | [next] | [standalone]

#110798 — #110798

On Tuesday, June 28, 2016 at 5:03:08 PM UTC+12, Ben Finney wrote:
> There is a clever one-line decorator that has been copy-pasted without
> explanation in many code bases for many years::
> 
>     decorator_with_args = lambda decorator: lambda *args, **kwargs: lambda func: decorator(func, *args, **kwargs)
> 
> My problem with this is precisely that it is clever: it explains nothing
> about what it does, has many moving parts that are not named, it is
> non-obvious and lacks expressiveness.

It is written in a somewhat roundabout way: why not just

    decorator_with_args = lambda decorator, *args, **kwargs : lambda func : decorator(func, *args, **kwargs)

? Could it be this was not valid in earlier versions of Python?

Anyway, it’s a function of a function returning a function. Example use (off the top of my head):

    my_decorator = decorator_with_args(actual_func_to_call)(... remaining args for actual_func_to_call)

(for the original version) or

    my_decorator = decorator_with_args(actual_func_to_call,... remaining args for actual_func_to_call)

(for the simpler version). Then an example use would be

    @my_decorator
    def some_func...

which would call “actual_func_to_call” with some_func plus all those extra args, and assign the result back to some_func.

> I would like to see a more Pythonic, more explicit and expressive
> replacement with its component parts easily understood.

I don’t know why this fear and suspicion of lambdas is so widespread among Python users ... former Java/C# programmers, perhaps?

[toc] | [prev] | [next] | [standalone]

#110801 — #110801

On Thu, 30 Jun 2016 12:43 pm, Lawrence D’Oliveiro wrote:

> On Tuesday, June 28, 2016 at 5:03:08 PM UTC+12, Ben Finney wrote:
>> There is a clever one-line decorator that has been copy-pasted without
>> explanation in many code bases for many years::
>> 
>>     decorator_with_args = lambda decorator: lambda *args, **kwargs:
>>     lambda func: decorator(func, *args, **kwargs)
>> 
>> My problem with this is precisely that it is clever: it explains nothing
>> about what it does, has many moving parts that are not named, it is
>> non-obvious and lacks expressiveness.
> 
> It is written in a somewhat roundabout way: why not just
> 
>     decorator_with_args = lambda decorator, *args, **kwargs : lambda func
>     : decorator(func, *args, **kwargs)
> 
> ? Could it be this was not valid in earlier versions of Python?

Your version has a much inferior API than the original. You can't wrap your
decorators ahead of time, you have to keep calling decorator_with_args each
time you want to use them. Contrast your version:


# LDO's version
import functools
decorator_with_args = (
        lambda decorator, *args, **kwargs : 
        lambda func : decorator(func, *args, **kwargs)
        )

def chatty(func, name, age, verbose=True):
    if verbose:
        print("decorating function...")
    @functools.wraps(func)
    def inner(*args, **kwargs):
        print("Hi, my name is %s and I am %d years old!" % (name, age))
        return func(*args, **kwargs)
    return inner

@decorator_with_args(chatty, "Bob", 99)
def calculate(x, y, z=1):
    return x+y-z

@decorator_with_args(chatty, "Sue", 35, False)
def spam(n):
    return ' '.join(['spam']*n)



Here's the original. It's a much better API, as the decorator "chatty" only
needs to be wrapped once, rather than repeatedly. For a library, it means
that now you can expose "chatty" as a public function, and keep
decorator_with_args as an internal detail, instead of needing to make them
both public:


# original meta decorator version

import functools
decorator_with_args = (
        lambda decorator: 
        lambda *args, **kwargs: 
        lambda func: decorator(func, *args, **kwargs)
        )

@decorator_with_args
def chatty(func, name, age, verbose=True):
    if verbose:
        print("decorating function...")
    @functools.wraps(func)
    def inner(*args, **kwargs):
        print("Hi, my name is %s and I am %d years old!" % (name, age))
        return func(*args, **kwargs)
    return inner

@chatty("Bob", 99)
def calculate(x, y, z=1):
    return x+y-z

@chatty("Sue", 35, False)
def spam(n):
    return ' '.join(['spam']*n)



Think of the use-case where you are the author of a library that provides
various decorators. `decorator_with_args` is an implementation detail of
your library: *you* say:

@decorator_with_args
def chatty(func, name, age, verbose=True): ...

@decorator_with_args
def logged(func, where_to_log): ...

@decorator_with_args
def memoise(func, cache_size): ...


and then offer chatty, logged and memoise as public functions to the users
of your library, who just write:

@memoise(1000)
def myfunc(arg): ...

etc. as needed. But with your version, you have to make decorator_with_args
a public part of the API, and require the user to write:

@decorator_with_args(memoise, 1000)
def myfunc(arg): ...


which I maintain is a much inferior API for the users of your library.



> I don’t know why this fear and suspicion of lambdas is so widespread among
> Python users ... former Java/C# programmers, perhaps?

Not so much fear as a little healthy respect for them, I think.




-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

[toc] | [prev] | [next] | [standalone]

#110825 — #110825

Lawrence D’Oliveiro wrote:

> On Tuesday, June 28, 2016 at 5:03:08 PM UTC+12, Ben Finney wrote:
> 
>> I would like to see a more Pythonic, more explicit and expressive
>> replacement with its component parts easily understood.
> 
> I don’t know why this fear and suspicion of lambdas is so widespread among
> Python users ... former Java/C# programmers, perhaps?

If there is "fear" I don't share it. However, for

foo = lambda <args>: <expr>

there is syntactic sugar in Python that allows you to write it as

def foo(<args>):
    return <expr>

with the nice side effects that it improves the readability of tracebacks 
and allows you to provide a docstring.

So yes, assigning a lambda to a name raises my "suspicion".

If a lambda is provided as an argument or as part of an expression I wonder 
how it is tested, and if even if it is trivial like

def reduce(items, func=lambda x, y: x + y): ...

the alternative

def reduce(items, func=add): ...

looks more readable in my eyes even though somewhere -- under the rug or in 
the operator module -- you need

def add(x, y):
   """
   >>> add(3, 4)
   7
   """
   return x + y

>>     decorator_with_args = lambda decorator: lambda *args, **kwargs: 
lambda func: decorator(func, *args, **kwargs)
> 
> Ah, I see why there are 3 lambdas, instead of 2. It’s so that you can 
write

I have a suspicion that there would not have been a correction "I see why 
there are three functions instead of two" ;)

[toc] | [prev] | [next] | [standalone]

#110828 — #110828

On Thursday, June 30, 2016 at 7:26:01 PM UTC+12, Peter Otten wrote:
> foo = lambda <args>: <expr>
> 
> there is syntactic sugar in Python that allows you to write it as
> 
> def foo(<args>):
>     return <expr>
> 
> with the nice side effects that it improves the readability of tracebacks 
> and allows you to provide a docstring.

True, but then again the original had three lambdas, so one line would have to become at least 3×2 = 6 lines, more if you want docstrings.

> def reduce(items, func=lambda x, y: x + y): ...

There was a reason why “reduce” was removed from being a builtin function in Python 2.x, to being banished to functools in Python 3.

> the alternative
> 
> def reduce(items, func=add): ...
> 
> looks more readable in my eyes even though somewhere ...

Just use “sum” in this case.

[toc] | [prev] | [next] | [standalone]

#110829 — #110829

On Thursday 30 June 2016 17:43, Lawrence D’Oliveiro wrote:

> On Thursday, June 30, 2016 at 7:26:01 PM UTC+12, Peter Otten wrote:
>> foo = lambda <args>: <expr>
>> 
>> there is syntactic sugar in Python that allows you to write it as
>> 
>> def foo(<args>):
>>     return <expr>
>> 
>> with the nice side effects that it improves the readability of tracebacks
>> and allows you to provide a docstring.
> 
> True, but then again the original had three lambdas, so one line would have
> to become at least 3×2 = 6 lines, more if you want docstrings.

I hear that we've passed "Peak Newlines" now, so adding extra lines will get 
more and more expensive. I guess languages like C and Java will soon have to be 
abandoned, and everyone will move to writing minified Javascript and Perl one-
liners.


>> def reduce(items, func=lambda x, y: x + y): ...
> 
> There was a reason why “reduce” was removed from being a builtin function in
> Python 2.x, to being banished to functools in Python 3.

Yes, and that reason is that Guido personally doesn't like reduce.



-- 
Steve

[toc] | [prev] | [next] | [standalone]

#110832 — #110832

Lawrence D’Oliveiro wrote:

> On Thursday, June 30, 2016 at 7:26:01 PM UTC+12, Peter Otten wrote:
>> foo = lambda <args>: <expr>
>> 
>> there is syntactic sugar in Python that allows you to write it as
>> 
>> def foo(<args>):
>>     return <expr>
>> 
>> with the nice side effects that it improves the readability of tracebacks
>> and allows you to provide a docstring.
> 
> True, but then again the original had three lambdas, so one line would
> have to become at least 3×2 = 6 lines, more if you want docstrings.
> 
>> def reduce(items, func=lambda x, y: x + y): ...
> 
> There was a reason why “reduce” was removed from being a builtin function
> in Python 2.x, to being banished to functools in Python 3.

You do understand what an example is?

 
>> the alternative
>> 
>> def reduce(items, func=add): ...
>> 
>> looks more readable in my eyes even though somewhere ...
> 
> Just use “sum” in this case.

Nah, the implementation is of course

def reduce(items, func=lambda x, y: x + y):
    """
    >>> list(reduce([1,2,3]))
    [1, 3, 6]

    >>> list(reduce([42]))
    [42]

    >>> reduce([])
    Traceback (most recent call last):
        ...
    TypeError: reduce() with empty sequence
    """
    items = iter(items)
    try:
        first = next(items)
    except StopIteration:
        raise TypeError("reduce() with empty sequence")

    def _reduce(accu=first):
        for item in items:
            yield accu
            accu = func(accu, item)
        yield accu

    return _reduce()

Yes, I'm kidding...

[toc] | [prev] | [next] | [standalone]

#110888 — #110888

Peter Otten <__peter__@web.de> writes:

> Lawrence D’Oliveiro wrote:
>
>> On Tuesday, June 28, 2016 at 5:03:08 PM UTC+12, Ben Finney wrote:
>> 
>>> I would like to see a more Pythonic, more explicit and expressive
>>> replacement with its component parts easily understood.
>> 
>> I don’t know why this fear and suspicion of lambdas is so widespread among
>> Python users ... former Java/C# programmers, perhaps?

Maybe, it's the name ("lambda").

In Javascript, anonymous functions are widespread (and extensively
used for e.g. callback definitions) - but it uses the much more familiar
"function" (rather than "lambda") for this purpose.

[toc] | [prev] | [next] | [standalone]

#110920 — #110920

dieter <dieter@handshake.de> writes:
<snip>
>> Lawrence D’Oliveiro wrote:
<snip>
>>> I don’t know why this fear and suspicion of lambdas is so widespread among
>>> Python users ... former Java/C# programmers, perhaps?

By replying I'm not accepting the premise -- I have no idea if there is
widespread fear and suspicion of lambdas among Python users but it seems
unlikely.

> Maybe, it's the name ("lambda").
>
> In Javascript, anonymous functions are widespread (and extensively
> used for e.g. callback definitions)

Yes, but in Python they are restricted to a single expression.  It's
therefore quite likely that programmers who want the semantics of an
anonymous function just define a named function and use that instead.
That saves you from having to decide, up front, if an expression is
going to be enough or from having to change it later if you find that it
isn't.

> - but it uses the much more familiar
> "function" (rather than "lambda") for this purpose.

... or the new arrow notation.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#110926 — #110926

Ben Bacarisse <ben.usenet@bsb.me.uk> writes:

> By replying I'm not accepting the premise -- I have no idea if there
> is widespread fear and suspicion of lambdas among Python users but it
> seems unlikely.

I can testify, as the person who started this thread, that there is no
fear or suspicion of lambda here. I use it quite frequently without
qualm for creating self-explanatory functions that need no name.

Rather, the motivation was that a complex thing, with many moving parts,
has an unexplained implementation: a nested set of functions without
names to explain their part in the pattern.

That these are then immediately bound to a name, to me defeats the
purpose of using lambda in the first place. If you want a named
function, lambda is not the tool to reach for; we have the ‘def’
statement for that.

But by using ‘lambda’ the author avoided one of the more important parts
of publishing the code: making it readable and self-explanatory. If
they'd chosen a name for each function, that would at least have
prompted them to explain what they're doing.

So ‘lambda’ is great, and I use it without worry for creating simple
self-explanatory nameless functions. But it's not the right tool for
this job: This is not self-explanatory, and the component functions
should not be nameless.

-- 
 \       “It is forbidden to steal hotel towels. Please if you are not |
  `\          person to do such is please not to read notice.” —hotel, |
_o__)                                               Kowloon, Hong Kong |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#110947 — #110947

Ben Finney <ben+python@benfinney.id.au> writes:
> ...
> Rather, the motivation was that a complex thing, with many moving parts,
> has an unexplained implementation: a nested set of functions without
> names to explain their part in the pattern.

In a previous reply, I have tried to explain (apparently without success)
that the "thing" is not "complex" at all but a simple signature
transform for a decorator definitition and that the nested function
implementation is natural for this purpose.

[toc] | [prev] | [next] | [standalone]

#110950 — #110950

dieter <dieter@handshake.de> writes:

> Ben Finney <ben+python@benfinney.id.au> writes:
> > ... Rather, the motivation was that a complex thing, with many
> > moving parts, has an unexplained implementation: a nested set of
> > functions without names to explain their part in the pattern.
>
> In a previous reply, I have tried to explain (apparently without
> success) that the "thing" is not "complex" at all

Your explanation was clear. I disagree with it; the code is not at all
obvious in its intention or operation.

Naming the parts descriptively, and writing a brief synopsis docstring
for each function, is a way to address that. Which is my motivation for
this thread.

> but a simple signature transform for a decorator definitition

You're making my case for me: To anyone who doesn't already know exactly
what's going on, that is not at all obvious from the code as presented
in the first message.

> and that the nested function implementation is natural for this
> purpose.

The nested function implementation is not the problem, as I hope I've
made clear elsewhere in this thread.

-- 
 \         “People's Front To Reunite Gondwanaland: Stop the Laurasian |
  `\              Separatist Movement!” —wiredog, http://kuro5hin.org/ |
_o__)                                                                  |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#110799 — #110799

On Tuesday, June 28, 2016 at 5:03:08 PM UTC+12, Ben Finney wrote:
>     decorator_with_args = lambda decorator: lambda *args, **kwargs: lambda func: decorator(func, *args, **kwargs)

Ah, I see why there are 3 lambdas, instead of 2. It’s so that you can write

    decorator_func = decorator_with_args(actual_func)

then you can use it like this:

    @decorator_func(...additional args for actual_func...)
    def some_func...

to invoke actual_func(some_func, ... additional args for actual_func...) on some_func, and assign the result back to some_func.

[toc] | [prev] | [next] | [standalone]

#110921 — #110921

On Tuesday, June 28, 2016 at 5:03:08 PM UTC+12, Ben Finney wrote:
> There is a clever one-line decorator that has been copy-pasted without
> explanation in many code bases for many years::
> 
>     decorator_with_args = lambda decorator: lambda *args, **kwargs: lambda func: decorator(func, *args, **kwargs)
> 

For those who want docstrings, I’ll give you docstrings:

    def decorator_with_args(decorator) :
        "given function decorator(func, *args, **kwargs), returns a decorator which," \
        " given func, returns the result of decorator(func, *args, **kwargs)."

        def decorate(*args, **kwargs) :

            def generated_decorator(func) :
                return \
                    decorator(func, *args, **kwargs)
            #end generated_decorator

        #begin decorate
            generated_decorator.__name__ = "decorator_{}".format(decorator.__name__)
            generated_decorator.__doc__ = "decorator which applies {} to the previously-specified arguments".format(decorator.__name__)
            return \
                generated_decorator
        #end decorate

    #begin decorator_with_args
        decorate.__name__ = "decorate_with_{}".format(decorator.__name__)
        decorate.__doc__ = "generates a decorator which applies {} to the given arguments".format(decorator.__name__)
        return \
            decorate
    #end decorator_with_args

[toc] | [prev] | [next] | [standalone]

#110942 — #110942

On Fri, Jul 1, 2016 at 4:08 PM, Lawrence D’Oliveiro
<lawrencedo99@gmail.com> wrote:
> On Tuesday, June 28, 2016 at 5:03:08 PM UTC+12, Ben Finney wrote:
>> There is a clever one-line decorator that has been copy-pasted without
>> explanation in many code bases for many years::
>>
>>     decorator_with_args = lambda decorator: lambda *args, **kwargs: lambda func: decorator(func, *args, **kwargs)
>>
>
> For those who want docstrings, I’ll give you docstrings:
>
>     def decorator_with_args(decorator) :
>         "given function decorator(func, *args, **kwargs), returns a decorator which," \
>         " given func, returns the result of decorator(func, *args, **kwargs)."
>
>         def decorate(*args, **kwargs) :
>
>             def generated_decorator(func) :
>                 return \
>                     decorator(func, *args, **kwargs)
>             #end generated_decorator
>
>         #begin decorate
>             generated_decorator.__name__ = "decorator_{}".format(decorator.__name__)
>             generated_decorator.__doc__ = "decorator which applies {} to the previously-specified arguments".format(decorator.__name__)
>             return \
>                 generated_decorator
>         #end decorate
>
>     #begin decorator_with_args
>         decorate.__name__ = "decorate_with_{}".format(decorator.__name__)
>         decorate.__doc__ = "generates a decorator which applies {} to the given arguments".format(decorator.__name__)

You should use functools.wraps instead of clobbering the decorated
function's name and docstring:

        @functools.wraps(decorator)
        def decorate(*args, **kwargs):
            ...

>         return \
>             decorate

Just to satisfy my own curiosity, do you have something against
putting the return keyword and the returned expression on the same
line?

[toc] | [prev] | [next] | [standalone]

#110946 — #110946

On Saturday, July 2, 2016 at 5:10:06 PM UTC+12, Ian wrote:
> You should use functools.wraps instead of clobbering the decorated
> function's name and docstring:

Where am I doing that?
> Just to satisfy my own curiosity, do you have something against
> putting the return keyword and the returned expression on the same
> line?

Yes.

[toc] | [prev] | [next] | [standalone]

#110973 — #110973

On Sat, Jul 2, 2016 at 12:40 AM, Lawrence D’Oliveiro
<lawrencedo99@gmail.com> wrote:
> On Saturday, July 2, 2016 at 5:10:06 PM UTC+12, Ian wrote:
>> You should use functools.wraps instead of clobbering the decorated
>> function's name and docstring:
>
> Where am I doing that?

Using the implementation you posted:

>>> @decorator_with_args
... def my_decorator(func, *args, **kwargs):
...     """Returns func unmodified."""
...     return func
...
>>> my_decorator.__doc__
'generates a decorator which applies my_decorator to the given arguments'

[toc] | [prev] | [next] | [standalone]

#110976 — #110976

On Sunday, July 3, 2016 at 4:49:15 PM UTC+12, Ian wrote:
>
> On Sat, Jul 2, 2016 at 12:40 AM, Lawrence D’Oliveiro wrote:
>>
>> On Saturday, July 2, 2016 at 5:10:06 PM UTC+12, Ian wrote:
>>>
>>> You should use functools.wraps instead of clobbering the decorated
>>> function's name and docstring:
>>
>> Where am I doing that?
> 
> Using the implementation you posted:
> 
>>>> @decorator_with_args
> ... def my_decorator(func, *args, **kwargs):
> ...     """Returns func unmodified."""
> ...     return func
> ...
>>>> my_decorator.__doc__
> 'generates a decorator which applies my_decorator to the given arguments'

That is a function that I am generating, so naturally I want to give it a useful docstring. Am I “clobbering” any objects passed in by the caller, or returned by the caller? No, I am not.

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.python

csiph-web