Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #78821 > unrolled thread

operator module functions

Back to article view | Back to comp.lang.python

Contents

  operator module functions Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-10-08 12:41 +1100
    Re: operator module functions Terry Reedy <tjreedy@udel.edu> - 2014-10-08 02:28 -0400
    Re: operator module functions marco.nawijn@colosso.nl - 2014-10-08 01:46 -0700
      Re: operator module functions Chris Angelico <rosuav@gmail.com> - 2014-10-08 20:32 +1100
        Re: operator module functions Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-10-09 10:37 +1300
          Re: operator module functions Chris Angelico <rosuav@gmail.com> - 2014-10-09 09:30 +1100
          Re: operator module functions Chris Kaynor <ckaynor@zindagigames.com> - 2014-10-08 15:40 -0700
      Re: operator module functions Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-10-08 21:57 +1100
        Re: operator module functions Terry Reedy <tjreedy@udel.edu> - 2014-10-08 15:09 -0400
        Re: operator module functions Ethan Furman <ethan@stoneleaf.us> - 2014-10-08 12:38 -0700
        Re: operator module functions random832@fastmail.us - 2014-10-08 15:49 -0400
        Re: operator module functions Gelonida N <gelonida@gmail.com> - 2014-10-08 23:05 +0200
        Re: operator module functions Ian Kelly <ian.g.kelly@gmail.com> - 2014-10-08 15:23 -0600
        Re: operator module functions Ethan Furman <ethan@stoneleaf.us> - 2014-10-08 14:49 -0700
        Re: operator module functions Chris Kaynor <ckaynor@zindagigames.com> - 2014-10-08 14:21 -0700
        Re: operator module functions Ben Finney <ben+python@benfinney.id.au> - 2014-10-09 08:57 +1100
        Re: operator module functions Terry Reedy <tjreedy@udel.edu> - 2014-10-08 18:46 -0400
        Re: operator module functions Ethan Furman <ethan@stoneleaf.us> - 2014-10-08 16:35 -0700
        Re: operator module functions Ned Batchelder <ned@nedbatchelder.com> - 2014-10-08 21:41 -0400

#78821 — operator module functions

Every Python operator has a function version in the operator module:

operator + has function operator.add;
operator - has function operator.sub;
operator * has function operator.mul;

and so forth. Only, that's not quite right... according to the
documentation, the "official" functions are actually:

operator.__add__;
operator.__sub__;
operator.__mul__;

etc., with the underscore-less versions being provided as a convenience.

Was anyone aware of this? It came as a surprise to me.

Is there anyone who uses or prefers the dunder versions?


-- 
Steven

[toc] | [next] | [standalone]

#78826

On 10/7/2014 9:41 PM, Steven D'Aprano wrote:
> Every Python operator has a function version in the operator module:
>
> operator + has function operator.add;
> operator - has function operator.sub;
> operator * has function operator.mul;
>
> and so forth. Only, that's not quite right... according to the
> documentation, the "official" functions are actually:
>
> operator.__add__;
> operator.__sub__;
> operator.__mul__;
>
> etc., with the underscore-less versions being provided as a convenience.
>
> Was anyone aware of this?

Of course. It has bugged me a bit for years.  Messy.

> Is there anyone who uses or prefers the dunder versions?

I don't and don't, and that seems to be true of other core devs: there 
are no non-test uses of the dunder methods in the stdlib.  Grepping 
/lib/*.py recursively for 'operator.' gives 540 hits, most for 
operator.someop.  Grepping the same for 'operator.__' returns 4 hits for 
operator.__name__ and .__doc__ (not operators) and these 6 tests:

C:\Programs\Python34\Lib\test\test_richcmp.py: 88:     "lt": (lambda 
a,b: a< b, operator.lt, operator.__lt__),
C:\Programs\Python34\Lib\test\test_richcmp.py: 89:     "le": (lambda 
a,b: a<=b, operator.le, operator.__le__),
C:\Programs\Python34\Lib\test\test_richcmp.py: 90:     "eq": (lambda 
a,b: a==b, operator.eq, operator.__eq__),
C:\Programs\Python34\Lib\test\test_richcmp.py: 91:     "ne": (lambda 
a,b: a!=b, operator.ne, operator.__ne__),
C:\Programs\Python34\Lib\test\test_richcmp.py: 92:     "gt": (lambda 
a,b: a> b, operator.gt, operator.__gt__),
C:\Programs\Python34\Lib\test\test_richcmp.py: 93:     "ge": (lambda 
a,b: a>=b, operator.ge, operator.__ge_

If there a back-compatibility excuse?  I cannot remember what operator 
had in 1.4 or whenever it was added, if after that.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#78829

For me it makes sense. operator.add should be used in a "global" context
(I don't know how to express it otherwise). So you provide it with the 
two values that you want to add. The .__add__ variants are bound to a
particular instance and you provide it with a single value that you want
to add. 

You also need the dunder versions when you want to implement addition for
user defined types. So although they are similar, I do believe they have
slightly different uses. As an example, you cannot use the dunder versions
for literals.

>> 2.__add__(3) # Oops, does not work
>> a = 2
>> a.__add__(3)
5
>> import operator
>> operator.add(2,3) # Fine

I also think operator.add versus .__add__ is equivalent to the global
getattr() and .__getattr__.

Marco

[toc] | [prev] | [next] | [standalone]

#78832

(You didn't include any context in your post. Please quote as much
text as would be helpful; it's the easiest way to show what you're
talking about.)

On Wed, Oct 8, 2014 at 7:46 PM,  <marco.nawijn@colosso.nl> wrote:
> For me it makes sense. operator.add should be used in a "global" context
> (I don't know how to express it otherwise). So you provide it with the
> two values that you want to add. The .__add__ variants are bound to a
> particular instance and you provide it with a single value that you want
> to add.

What Steven's talking about is this:

>>> operator.add is operator.__add__
True

It's the exact same function, just accessed with a different name.

> As an example, you cannot use the dunder versions for literals.
>
>>> 2.__add__(3) # Oops, does not work
>>> a = 2
>>> a.__add__(3)
> 5

That's actually just a syntactic issue with integers and the dot. It
works fine if you use any other form of literal, or if you put a space
between the digits and the dot, or use parentheses, or anything; this
is the case with all methods off integers, not just dunder ones.

>>> 2 .__add__(3)
5
>>> (2).__add__(3)
5
>>> 2.0.__add__(3.0)
5.0

The object is exactly the same whether you reference the literal '2'
or the name 'a' that you've bound to it, so its methods must by
definition all be there.

ChrisA

[toc] | [prev] | [next] | [standalone]

#78853

Chris Angelico wrote:

>>>>operator.add is operator.__add__
> True

That doesn't always seem to have been the case, however.
In Python 2.7 and 3.3, I get

 >>> operator.add is operator.__add__
False

-- 
Greg

[toc] | [prev] | [next] | [standalone]

#78858

On Thu, Oct 9, 2014 at 8:37 AM, Gregory Ewing
<greg.ewing@canterbury.ac.nz> wrote:
> Chris Angelico wrote:
>
>>>>> operator.add is operator.__add__
>>
>> True
>
>
> That doesn't always seem to have been the case, however.
> In Python 2.7 and 3.3, I get
>
>>>> operator.add is operator.__add__
> False

Huh. So it is.

rosuav@sikorsky:~$ python3
Python 3.5.0a0 (default:301b9a58021c, Oct  2 2014, 09:20:24)
[GCC 4.7.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import operator
>>> operator.add, operator.__add__
(<built-in function add>, <built-in function add>)
>>>
rosuav@sikorsky:~$ python
Python 2.7.3 (default, Mar 13 2014, 11:03:55)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import operator
>>> operator.add, operator.__add__
(<built-in function add>, <built-in function __add__>)
>>>

Presumably they have the same code behind them, just different
function names. But anyway, the fact that it doesn't throw back an
AttributeError proves that both functions do at least exist.

Learn something new every day!

ChrisA

[toc] | [prev] | [next] | [standalone]

#78859

[Multipart message — attachments visible in raw view] — view raw

On Wed, Oct 8, 2014 at 3:30 PM, Chris Angelico <rosuav@gmail.com> wrote:

> >
> >
> > That doesn't always seem to have been the case, however.
> > In Python 2.7 and 3.3, I get
> >
> >>>> operator.add is operator.__add__
> > False
>
> Huh. So it is.
>
> rosuav@sikorsky:~$ python3
> Python 3.5.0a0 (default:301b9a58021c, Oct  2 2014, 09:20:24)
> [GCC 4.7.2] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import operator
> >>> operator.add, operator.__add__
> (<built-in function add>, <built-in function add>)
> >>>
> rosuav@sikorsky:~$ python
> Python 2.7.3 (default, Mar 13 2014, 11:03:55)
> [GCC 4.7.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import operator
> >>> operator.add, operator.__add__
> (<built-in function add>, <built-in function __add__>)
> >>>
>
> Presumably they have the same code behind them, just different
> function names. But anyway, the fact that it doesn't throw back an
> AttributeError proves that both functions do at least exist.
>
> Learn something new every day!


I only tried it in Python 3.4, and I got true. Perhaps it was optimized
slightly after 3.3?

Chris

[toc] | [prev] | [next] | [standalone]

#78838

marco.nawijn@colosso.nl wrote:

> For me it makes sense. operator.add should be used in a "global" context
> (I don't know how to express it otherwise). So you provide it with the
> two values that you want to add. The .__add__ variants are bound to a
> particular instance and you provide it with a single value that you want
> to add.

That's not correct. You're misunderstanding me, I'm not talking about the
special "dunder" methods on classes, e.g. (2).__add__(3) -> 5. I'm talking
about *functions* in the operator module:

py> import operator
py> operator.add(2, 3)
5
py> operator.__add__(2, 3)
5

According to the documentation, operator.__add__ is the "official" function,
and operator.add is just there for convenience. Neither of them is the
special method __add__. But it seems that nobody uses the official
operator.__add__ function, and Terry Reedy says there are no tests for it.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#78845

On 10/8/2014 6:57 AM, Steven D'Aprano wrote:

> According to the documentation, operator.__add__ is the "official" function,
> and operator.add is just there for convenience.

You are paraphrasing "The function names are those used for special 
class methods; variants without leading and trailing __ are also 
provided for convenience."  But then there is the following:

10.3.1. Mapping Operators to Functions

This table shows how abstract operations correspond to operator symbols 
in the Python syntax and the functions in the operator module.
Operation 	Syntax 	Function
Addition 	a + b 	add(a, b)

etc, using the 'convenient' names. I would like to deprecate and 
eventually remove the dunder names.  To me, the duplication is not 
'convenient'.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#78848

On 10/08/2014 12:09 PM, Terry Reedy wrote:
> On 10/8/2014 6:57 AM, Steven D'Aprano wrote:
>
>> According to the documentation, operator.__add__ is the "official" function,
>> and operator.add is just there for convenience.
>
> You are paraphrasing "The function names are those used for special class methods; variants without leading and trailing
> __ are also provided for convenience."  But then there is the following:
>
> 10.3.1. Mapping Operators to Functions
>
> This table shows how abstract operations correspond to operator symbols in the Python syntax and the functions in the
> operator module.
> Operation     Syntax     Function
> Addition     a + b     add(a, b)
>
> etc, using the 'convenient' names. I would like to deprecate and eventually remove the dunder names.  To me, the
> duplication is not 'convenient'.

LOL, no kidding!  The main reason I bother using the operator module is for the readability of not seeing the dunders, 
and the writability of not having to type them.

--
~Ethan~

[toc] | [prev] | [next] | [standalone]

#78849

On Wed, Oct 8, 2014, at 15:38, Ethan Furman wrote:
> LOL, no kidding!  The main reason I bother using the operator module is
> for the readability of not seeing the dunders, 
> and the writability of not having to type them.

I'm not sure what situation you would have to type them (as opposed to
simply a + b) that the operator module would help with.

[toc] | [prev] | [next] | [standalone]

#78850

On 10/8/2014 9:09 PM, Terry Reedy wrote:
> On 10/8/2014 6:57 AM, Steven D'Aprano wrote:
>
>> According to the documentation, operator.__add__ is the "official"
>> function,
>> and operator.add is just there for convenience.
>
> You are paraphrasing "The function names are those used for special
> class methods; variants without leading and trailing __ are also
> provided for convenience."  But then there is the following:
>
> 10.3.1. Mapping Operators to Functions
>
> This table shows how abstract operations correspond to operator symbols
> in the Python syntax and the functions in the operator module.
> Operation     Syntax     Function
> Addition     a + b     add(a, b)
>
> etc, using the 'convenient' names. I would like to deprecate and
> eventually remove the dunder names.  To me, the duplication is not
> 'convenient'.
>


I'd be curious about a proposal to obsolete the double underscore 
functions and just keep operator.add or to just keep the operator +

For me they seem rather distinct, though they are interchangable in 
certain situations.

Perhaps I got something wrong, but all dunder functions are 'magic' 
functions. Many of them mostly should ne used when when overloading 
behaviour or when creating a class which should implement operators.

So for implemetation you use __add__ for using you use the operator +
if you need a

class MyClass(object):
         def __init__(self, x, y):
                 self.x = x
                 self.y = y

         # implements + for a certain type
         def __add__(self, other):
                 return MyClass(self.x + other.x, self.y + other.y)

         def __repr__(self):
             return "(%f,%f)" % (self.x, self.y)


a = MyClass(1,2)
b = MyClass(4,5)

print(a)
print(b)
print(a+b)

# operator.add is identical to the + operator ,
# BUT it is a function and can thuss be
# assigned to variables passed as parameter

def dosomething(op, val1, val2):
	return op(val1, val2)

print(dosomething(operator.add, 1, 2))
print(dosomething(operator.add, a, b))

[toc] | [prev] | [next] | [standalone]

#78851

On Wed, Oct 8, 2014 at 3:05 PM, Gelonida N <gelonida@gmail.com> wrote:
> On 10/8/2014 9:09 PM, Terry Reedy wrote:
>>
>> On 10/8/2014 6:57 AM, Steven D'Aprano wrote:
>>
>>> According to the documentation, operator.__add__ is the "official"
>>> function,
>>> and operator.add is just there for convenience.
>>
>>
>> You are paraphrasing "The function names are those used for special
>> class methods; variants without leading and trailing __ are also
>> provided for convenience."  But then there is the following:
>>
>> 10.3.1. Mapping Operators to Functions
>>
>> This table shows how abstract operations correspond to operator symbols
>> in the Python syntax and the functions in the operator module.
>> Operation     Syntax     Function
>> Addition     a + b     add(a, b)
>>
>> etc, using the 'convenient' names. I would like to deprecate and
>> eventually remove the dunder names.  To me, the duplication is not
>> 'convenient'.
>>
>
>
> I'd be curious about a proposal to obsolete the double underscore functions
> and just keep operator.add or to just keep the operator +
>
> For me they seem rather distinct, though they are interchangable in certain
> situations.
>
> Perhaps I got something wrong, but all dunder functions are 'magic'
> functions. Many of them mostly should ne used when when overloading
> behaviour or when creating a class which should implement operators.

This isn't about the dunder *method* names. It's about the naming of
the functions in the operator module, e.g. having both operator.add
and operator.__add__ that are two names for the same function.

[toc] | [prev] | [next] | [standalone]

#78854

[redirecting back to the list]

On 10/08/2014 02:23 PM, random832@fastmail.us wrote:
> On Wed, Oct 8, 2014, at 15:53, Ethan Furman wrote:
>> On 10/08/2014 12:49 PM, random832@fastmail.us wrote:
>>> On Wed, Oct 8, 2014, at 15:38, Ethan Furman wrote:
>>>>
>>>> LOL, no kidding!  The main reason I bother using the operator module is
>>>> for the readability of not seeing the dunders,
>>>> and the writability of not having to type them.
>>>
>>> I'm not sure what situation you would have to type them (as opposed to
>>> simply a + b) that the operator module would help with.
>>
>> unittest springs to mind:
>>
>>     self.assertRaises(TypeError, op.add, obj1, obj2)
>
> Er, my point is, that is not a situation where you would be able to use
> obj1.__add__ - you'd have to use the operator module *anyway*, and
> therefore aren't using it just to get the convenience of a non-dunder
> name.

  self.assertRaises(TypeError, lambda x, y: x+y, obj1, obj2)

--
~Ethan~

[toc] | [prev] | [next] | [standalone]

#78855

[Multipart message — attachments visible in raw view] — view raw

On Wed, Oct 8, 2014 at 2:05 PM, Gelonida N <gelonida@gmail.com> wrote:

> On 10/8/2014 9:09 PM, Terry Reedy wrote:
>
>> On 10/8/2014 6:57 AM, Steven D'Aprano wrote:
>>
>>  According to the documentation, operator.__add__ is the "official"
>>> function,
>>> and operator.add is just there for convenience.
>>>
>>
>> You are paraphrasing "The function names are those used for special
>> class methods; variants without leading and trailing __ are also
>> provided for convenience."  But then there is the following:
>>
>> 10.3.1. Mapping Operators to Functions
>>
>> This table shows how abstract operations correspond to operator symbols
>> in the Python syntax and the functions in the operator module.
>> Operation     Syntax     Function
>> Addition     a + b     add(a, b)
>>
>> etc, using the 'convenient' names. I would like to deprecate and
>> eventually remove the dunder names.  To me, the duplication is not
>> 'convenient'.
>>
>>
>
> I'd be curious about a proposal to obsolete the double underscore
> functions and just keep operator.add or to just keep the operator +
>
> For me they seem rather distinct, though they are interchangable in
> certain situations.
>
> Perhaps I got something wrong, but all dunder functions are 'magic'
> functions. Many of them mostly should ne used when when overloading
> behaviour or when creating a class which should implement operators.
>
> print(dosomething(operator.add, 1, 2))
> print(dosomething(operator.add, a, b))


The thing being purposed to remove, is that you can also do:

print(dosomething(operator.__add__, 1, 2))
print(dosomething(operator.__add__, a, b))

Not that fact that you can define __add__ in the class, or that
"operator.add(a, b)" is the same as "a + b" (all of which are quite useful).

Basically, the operator module has a "__add__" function in addition to a
"add" function, and they are both identical (in fact, they are the same
function object):
>>> import operator
>>> operator.add
<built-in function add>
>>> operator.__add__
<built-in function add>
>>> operator.__add__ is operator.add
True

[toc] | [prev] | [next] | [standalone]

#78856

random832@fastmail.us writes:

> On Wed, Oct 8, 2014, at 15:38, Ethan Furman wrote:
> > The main reason I bother using the operator module is for the
> > readability of not seeing the dunders, and the writability of not
> > having to type them.
>
> I'm not sure what situation you would have to type them (as opposed to
> simply a + b) that the operator module would help with.

Any situation where you need to refer to a function. ‘+’ is not a
function, whereas ‘operator.add’ is.

    import operator
    import functools

    add_three = functools.partial(operator.add, 3)

    foo = list(range(10))
    bar = map(add_three, foo)

The above ‘map’ invocation – which is useful because it allows any
arbitrary one-parameter function to be used – cannot be used with ‘+’,
which is not a function.

Likewise, ‘functools.partial’ requires a function, and ‘+’ is not a
function.

There are countless such uses for the functions in the ‘operator’
module.

-- 
 \      “It is the integrity of each individual human that is in final |
  `\        examination. On personal integrity hangs humanity's fate.” |
_o__)               —Richard Buckminster Fuller, _Critical Path_, 1981 |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#78861

On 10/8/2014 5:49 PM, Ethan Furman wrote:
> [redirecting back to the list]

>>>> I'm not sure what situation you would have to type them (as opposed to
>>>> simply a + b) that the operator module would help with.
>>>
>>> unittest springs to mind:
>>>
>>>     self.assertRaises(TypeError, op.add, obj1, obj2)
>>
>> Er, my point is, that is not a situation where you would be able to use
>> obj1.__add__ - you'd have to use the operator module *anyway*, and
>> therefore aren't using it just to get the convenience of a non-dunder
>> name.
>
>   self.assertRaises(TypeError, lambda x, y: x+y, obj1, obj2)

That works, but is a bit harder to type (given the import), more 
confusing to read (the extra ',' that does *not* delimit an argument) 
and adds an extra layer to the call stack with each call.  The extra 
layer *could* make a difference in recursion.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#78862

On 10/08/2014 03:46 PM, Terry Reedy wrote:
> On 10/8/2014 5:49 PM, Ethan Furman wrote:
>> [redirecting back to the list]
>
>>>>> I'm not sure what situation you would have to type them (as opposed to
>>>>> simply a + b) that the operator module would help with.
>>>>
>>>> unittest springs to mind:
>>>>
>>>>     self.assertRaises(TypeError, op.add, obj1, obj2)
>>>
>>> Er, my point is, that is not a situation where you would be able to use
>>> obj1.__add__ - you'd have to use the operator module *anyway*, and
>>> therefore aren't using it just to get the convenience of a non-dunder
>>> name.
>>
>>   self.assertRaises(TypeError, lambda x, y: x+y, obj1, obj2)
>
> That works, but is a bit harder to type (given the import), more confusing to read (the extra ',' that does *not*
> delimit an argument) and adds an extra layer to the call stack with each call.  The extra layer *could* make a
> difference in recursion.

Indeed, thanks for the corrections.  I was merely replying to the "have to use the operator module anyway" comment.

--
~Ethan~

[toc] | [prev] | [next] | [standalone]

#78867

On 10/8/14 5:49 PM, Ethan Furman wrote:
> [redirecting back to the list]
>
> On 10/08/2014 02:23 PM, random832@fastmail.us wrote:
>> On Wed, Oct 8, 2014, at 15:53, Ethan Furman wrote:
>>> On 10/08/2014 12:49 PM, random832@fastmail.us wrote:
>>>> On Wed, Oct 8, 2014, at 15:38, Ethan Furman wrote:
>>>>>
>>>>> LOL, no kidding!  The main reason I bother using the operator
>>>>> module is
>>>>> for the readability of not seeing the dunders,
>>>>> and the writability of not having to type them.
>>>>
>>>> I'm not sure what situation you would have to type them (as opposed to
>>>> simply a + b) that the operator module would help with.
>>>
>>> unittest springs to mind:
>>>
>>>     self.assertRaises(TypeError, op.add, obj1, obj2)
>>
>> Er, my point is, that is not a situation where you would be able to use
>> obj1.__add__ - you'd have to use the operator module *anyway*, and
>> therefore aren't using it just to get the convenience of a non-dunder
>> name.
>
>   self.assertRaises(TypeError, lambda x, y: x+y, obj1, obj2)

Don't forget, in 2.7 you can do:

     with self.assertRaises(TypeError):
         obj1 + obj2

-- 
Ned Batchelder, http://nedbatchelder.com

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web