Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #17630 > unrolled thread

Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python. Nathan Rice <nathan.alexander.rice@gmail.com> - 2011-12-20 19:24 -0500
    Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python. alex23 <wuwei23@gmail.com> - 2011-12-20 20:27 -0800
      Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python. Nathan Rice <nathan.alexander.rice@gmail.com> - 2011-12-21 10:15 -0500
      Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python. Robert Kern <robert.kern@gmail.com> - 2011-12-21 16:29 +0000
      Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python. Nathan Rice <nathan.alexander.rice@gmail.com> - 2011-12-21 11:43 -0500

#17630 — Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.

On Tue, Dec 20, 2011 at 7:03 PM, Ian Kelly <ian.g.kelly@gmail.com> wrote:
> On Tue, Dec 20, 2011 at 12:45 PM, Nathan Rice
> <nathan.alexander.rice@gmail.com> wrote:
>> There are still some issues with proper support of things like bool()
>> and int(), which refuse to return things that are not of the correct
>> type.
>
> And that's a good thing.  As type conversion functions, bool(x) and
> int(x) should *always* return bools and ints respectively (or raise an
> exception), no matter what you pass in for x.

I was hoping to have the proxy be completely transparent.  I don't
disagree with your statement though.

> If I do "list(efoo)", where efoo is an ElementwiseProxy object, should
> I expect to get the efoo collection converted to a list, or another
> ElementwiseProxy where each element has been converted to a list?  I
> would hope the former.

Iterators are how you go from an ElementwiseProxy back to a regular
collection.  Thus list/set/etc or anything that takes an iterator will
work.

>> This was developed as a proof of concept for expanding the role of
>> element-wise syntax in python, and to that end I welcome comments.
>
> The examples you gave are all numerical in nature.  If I might
> inquire, what might I use this for that I can't already do with numpy?

efoo2 = ElementwiseProxy(["one", "two", "three", "four"])

efoo_res = ((efoo2.capitalize() + " little indian").split("
").apply(reversed) * 2).apply("_".join) # note that you could do
reversed(...) instead, I just like to read left to right
efoo_res.parent.parent.parent # same as ((efoo2.capitalize() + "
little indian").split(" ") in case you need to debug something and
want to look at intermediate values

The idea is to provide a syntax that lets you do very complex things
on collections in a more readable manner, without having 5 or 6 lines
of generator expressions.

Nathan

[toc] | [next] | [standalone]

#17644

On Dec 21, 10:24 am, Nathan Rice <nathan.alexander.r...@gmail.com>
wrote:
> The idea is to provide a syntax that lets you do very complex things
> on collections in a more readable manner, without having 5 or 6 lines
> of generator expressions.

Have you seen PyLINQ? It has a similar approach to operating on
collections, returning a PyLINQ object after each call to facilitate
chaining. https://github.com/kalessin/PyLINQ/blob/master/pylinq/linq.py

This is a personal opinion on the code, but I'd move instantiating the
new ElementwiseProxy out of each method and into its own decorator:

    # declare this outside of the class
    def chainable(fn):
        def _(self, *args, **kwargs):
            return ElementwiseProxy(fn(self, *args, **kwargs), self)
        return _

This way, each method that is chainable is a little more obvious
without inspecting the code, and the method body itself is only doing
what the method says it does:

    @chainable
    def __add__(self, other):
        return (e + other for e in object.__getattribute__(self,
"iterable"))

Incidentally, displaying an ElementwiseProxy instance doesn't go down
well with iPython:

In [1]: from elementwise import *

In [2]: e = ElementwiseProxy(['one','two','three'])

In [3]: e
Out[3]: ERROR: An unexpected error occurred while tokenizing input
The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line statement', (6, 0))

[toc] | [prev] | [next] | [standalone]

#17667

> Have you seen PyLINQ? It has a similar approach to operating on
> collections, returning a PyLINQ object after each call to facilitate
> chaining. https://github.com/kalessin/PyLINQ/blob/master/pylinq/linq.py

I hadn't seen it previously.  I am a VERY heavy user of SQL Alchemy
though, and I am sure chaining generative ClauseElements/Queries/etc,
has burned some patterns into my subconscious at this point.

> This is a personal opinion on the code, but I'd move instantiating the
> new ElementwiseProxy out of each method and into its own decorator:
>
>    # declare this outside of the class
>    def chainable(fn):
>        def _(self, *args, **kwargs):
>            return ElementwiseProxy(fn(self, *args, **kwargs), self)
>        return _
>
> This way, each method that is chainable is a little more obvious
> without inspecting the code, and the method body itself is only doing
> what the method says it does:
>
>    @chainable
>    def __add__(self, other):
>        return (e + other for e in object.__getattribute__(self,
> "iterable"))

This is a reasonable suggestion and I will play with something along
those lines soon.

> Incidentally, displaying an ElementwiseProxy instance doesn't go down
> well with iPython:
>
> In [1]: from elementwise import *
>
> In [2]: e = ElementwiseProxy(['one','two','three'])
>
> In [3]: e
> Out[3]: ERROR: An unexpected error occurred while tokenizing input
> The following traceback may be corrupted or invalid
> The error message is: ('EOF in multi-line statement', (6, 0))

I love IPython, but it has had known problems with iterators for
years.  A long time ago, I agonized over what I thought was a bug in
my code where itertools.count would skip numbers in IPython, but my
unit tests all passed.  Everything should work fine if you tuple() it
first.

Nathan

[toc] | [prev] | [next] | [standalone]

#17672

On 12/21/11 3:15 PM, Nathan Rice wrote:

>> Incidentally, displaying an ElementwiseProxy instance doesn't go down
>> well with iPython:
>>
>> In [1]: from elementwise import *
>>
>> In [2]: e = ElementwiseProxy(['one','two','three'])
>>
>> In [3]: e
>> Out[3]: ERROR: An unexpected error occurred while tokenizing input
>> The following traceback may be corrupted or invalid
>> The error message is: ('EOF in multi-line statement', (6, 0))
>
> I love IPython, but it has had known problems with iterators for
> years.  A long time ago, I agonized over what I thought was a bug in
> my code where itertools.count would skip numbers in IPython, but my
> unit tests all passed.  Everything should work fine if you tuple() it
> first.

This is a different problem, actually. The problem is that the recently added 
(by me, actually) pretty-printing system tries to dispatch based on the type. In 
order to handle old-style classes, it checks for the type using .__class__ 
first. ElementProxy's __getattribute__() gets in the way here by returning a 
generator instead of the ElementProxy class.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

[toc] | [prev] | [next] | [standalone]

#17673

On Wed, Dec 21, 2011 at 11:29 AM, Robert Kern <robert.kern@gmail.com> wrote:
> On 12/21/11 3:15 PM, Nathan Rice wrote:
>
>>> Incidentally, displaying an ElementwiseProxy instance doesn't go down
>>> well with iPython:
>>>
>>> In [1]: from elementwise import *
>>>
>>> In [2]: e = ElementwiseProxy(['one','two','three'])
>>>
>>> In [3]: e
>>> Out[3]: ERROR: An unexpected error occurred while tokenizing input
>>> The following traceback may be corrupted or invalid
>>> The error message is: ('EOF in multi-line statement', (6, 0))
>>
>>
>> I love IPython, but it has had known problems with iterators for
>> years.  A long time ago, I agonized over what I thought was a bug in
>> my code where itertools.count would skip numbers in IPython, but my
>> unit tests all passed.  Everything should work fine if you tuple() it
>> first.
>
>
> This is a different problem, actually. The problem is that the recently
> added (by me, actually) pretty-printing system tries to dispatch based on
> the type. In order to handle old-style classes, it checks for the type using
> .__class__ first. ElementProxy's __getattribute__() gets in the way here by
> returning a generator instead of the ElementProxy class.

Thanks for the heads up Robert.  Given that IPython is awesome and
people should probably be doing an '.apply(type)' in that instance
anyhow, I will corner case that on __getattribute__.

Nathan

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web