Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #94286 > unrolled thread

Send data to asyncio coroutine

Back to article view | Back to comp.lang.python

Contents

  Send data to asyncio coroutine jcarmena@gmail.com - 2015-07-21 04:31 -0700
    Re: Send data to asyncio coroutine Ian Kelly <ian.g.kelly@gmail.com> - 2015-07-21 07:35 -0600
      Re: Send data to asyncio coroutine Javier <jcarmena@gmail.com> - 2015-07-28 14:17 -0700
        Re: Send data to asyncio coroutine Javier <jcarmena@gmail.com> - 2015-07-28 15:41 -0700
          Re: Send data to asyncio coroutine Ian Kelly <ian.g.kelly@gmail.com> - 2015-07-28 15:20 -0800
        Re: Send data to asyncio coroutine Ian Kelly <ian.g.kelly@gmail.com> - 2015-07-28 15:06 -0800
          Re: Send data to asyncio coroutine Rustom Mody <rustompmody@gmail.com> - 2015-07-28 19:52 -0700
          Re: Send data to asyncio coroutine Javier <jcarmena@gmail.com> - 2015-07-29 07:24 -0700
    Re: Send data to asyncio coroutine Javier <jcarmena@gmail.com> - 2015-08-01 09:07 -0700
      Re: Send data to asyncio coroutine Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-08-01 17:41 +0100
        Re: Send data to asyncio coroutine Javier <jcarmena@gmail.com> - 2015-08-01 11:22 -0700
          Re: Send data to asyncio coroutine Marko Rauhamaa <marko@pacujo.net> - 2015-08-01 21:38 +0300
            Re: Send data to asyncio coroutine Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-08-01 19:45 +0100
              Re: Send data to asyncio coroutine Javier <jcarmena@gmail.com> - 2015-08-01 12:07 -0700
                Re: Send data to asyncio coroutine Marko Rauhamaa <marko@pacujo.net> - 2015-08-01 22:14 +0300
                  Re: Send data to asyncio coroutine Javier <jcarmena@gmail.com> - 2015-08-01 17:50 -0700
                Re: Send data to asyncio coroutine Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-08-01 20:33 +0100
                Re: Send data to asyncio coroutine Chris Angelico <rosuav@gmail.com> - 2015-08-02 09:28 +1000
              Re: Send data to asyncio coroutine Marko Rauhamaa <marko@pacujo.net> - 2015-08-01 22:09 +0300
                Re: Send data to asyncio coroutine Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-08-01 20:29 +0100
      Re: Send data to asyncio coroutine Marko Rauhamaa <marko@pacujo.net> - 2015-08-01 20:18 +0300
        Re: Send data to asyncio coroutine Javier <jcarmena@gmail.com> - 2015-08-01 10:50 -0700
          Re: Send data to asyncio coroutine Marko Rauhamaa <marko@pacujo.net> - 2015-08-01 21:33 +0300

Page 1 of 2  [1] 2  Next page →

#94286 — Send data to asyncio coroutine

Hello, I'm trying to understand and link asyncio with ordinary coroutines. Now I just want to understand how to do this on asyncio:


def foo():
    data = yield 8
    print(data)
    yield "bye"
    
def bar():
    f = foo()
    n = f.next()
    print(n)
    message = f.send("hello")
    print(message)


What is the equivalent for coro.send("some data") in asyncio?

coro.send on an asyncio coroutine throws AssertionError: yield from wasn't used with future.


Thank you

[toc] | [next] | [standalone]

#94290

On Tue, Jul 21, 2015 at 5:31 AM,  <jcarmena@gmail.com> wrote:
> Hello, I'm trying to understand and link asyncio with ordinary coroutines. Now I just want to understand how to do this on asyncio:
>
>
> def foo():
>     data = yield 8
>     print(data)
>     yield "bye"
>
> def bar():
>     f = foo()
>     n = f.next()
>     print(n)
>     message = f.send("hello")
>     print(message)
>
>
> What is the equivalent for coro.send("some data") in asyncio?

I don't know of any reason why you couldn't do it just like the above.
However, the exchange would not be asynchronous, if that is your goal.

> coro.send on an asyncio coroutine throws AssertionError: yield from wasn't used with future.

So somehow a future got involved where it shouldn't have been. What
was the actual code that you tried to run?

Note that while "yield" and "yield from" look similar, they are quite
different, and you cannot send to a generator that is currently paused
at a "yield from".

If you want to emulate bidirectional communication similar to
coro.send asynchronously, I think you'll need to use Futures to
mediate, something like this (lightly tested):

@asyncio.coroutine
def foo(fut):
    data, fut = yield from send_to_future(8, fut)
    print("foo", data)
    fut.set_result("bye")

@asyncio.coroutine
def bar():
    n, fut = yield from start_coro(foo)
    print("bar", n)
    message = yield from send_to_future("hello", fut)
    print("bar", message)

def start_coro(coro):
    future = asyncio.Future()
    asyncio.async(coro(future))
    return future

def send_to_future(data, future):
    new_future = asyncio.Future()
    future.set_result((data, new_future))
    return new_future

[toc] | [prev] | [next] | [standalone]

#94706

El martes, 21 de julio de 2015, 15:42:47 (UTC+2), Ian  escribió:
> On Tue, Jul 21, 2015 at 5:31 AM,  <jcarmena@gmail.com> wrote:
> > Hello, I'm trying to understand and link asyncio with ordinary coroutines. Now I just want to understand how to do this on asyncio:
> >
> >
> > def foo():
> >     data = yield 8
> >     print(data)
> >     yield "bye"
> >
> > def bar():
> >     f = foo()
> >     n = f.next()
> >     print(n)
> >     message = f.send("hello")
> >     print(message)
> >
> >
> > What is the equivalent for coro.send("some data") in asyncio?
> 
> I don't know of any reason why you couldn't do it just like the above.
> However, the exchange would not be asynchronous, if that is your goal.
> 
> > coro.send on an asyncio coroutine throws AssertionError: yield from wasn't used with future.
> 
> So somehow a future got involved where it shouldn't have been. What
> was the actual code that you tried to run?
> 
> Note that while "yield" and "yield from" look similar, they are quite
> different, and you cannot send to a generator that is currently paused
> at a "yield from".
> 
> If you want to emulate bidirectional communication similar to
> coro.send asynchronously, I think you'll need to use Futures to
> mediate, something like this (lightly tested):
> 
> @asyncio.coroutine
> def foo(fut):
>     data, fut = yield from send_to_future(8, fut)
>     print("foo", data)
>     fut.set_result("bye")
> 
> @asyncio.coroutine
> def bar():
>     n, fut = yield from start_coro(foo)
>     print("bar", n)
>     message = yield from send_to_future("hello", fut)
>     print("bar", message)
> 
> def start_coro(coro):
>     future = asyncio.Future()
>     asyncio.async(coro(future))
>     return future
> 
> def send_to_future(data, future):
>     new_future = asyncio.Future()
>     future.set_result((data, new_future))
>     return new_future




Hello again. I have been investigating a bit your example. I don't understand why I can't write something like this:

--------

import asyncio

def foo():
    print("start foo")
    try:
        while True:
            val = yield
            print("foo:", val)
            yield from asyncio.sleep(3)
    except GeneratorExit:
        print("foo closed")
    print("exit foo")

def bar(next):
    print("start bar")
    next.send(None)
    try:
        while True:
            val = yield
            next.send("bar/"+val)
    except GeneratorExit:
        print("bar closed")
    print("exit bar")

def fun(next):
    next.send(None)
    for e in ["hello", "world", "I'm", "pythonist"]:
        next.send(e)

@asyncio.coroutine
def run():
    fun(bar(foo()))

loop = asyncio.get_event_loop()
loop.run_until_complete(run())
loop.close()

-------

The expected output is:

start bar
start foo
foo: bar/hello
foo: bar/world
foo: bar/I'm
foo: bar/phytonist
bar closed
exit bar
foo closed
exit foo

But the yield from asyncio.sleep(3) call raises AssertionError, however it's inside a Task!
I think this is a big flaw in python/asyncio design.

[toc] | [prev] | [next] | [standalone]

#94710

El martes, 28 de julio de 2015, 23:18:11 (UTC+2), Javier  escribió:
> El martes, 21 de julio de 2015, 15:42:47 (UTC+2), Ian  escribió:
> > On Tue, Jul 21, 2015 at 5:31 AM,  <jcarmena@gmail.com> wrote:
> > > Hello, I'm trying to understand and link asyncio with ordinary coroutines. Now I just want to understand how to do this on asyncio:
> > >
> > >
> > > def foo():
> > >     data = yield 8
> > >     print(data)
> > >     yield "bye"
> > >
> > > def bar():
> > >     f = foo()
> > >     n = f.next()
> > >     print(n)
> > >     message = f.send("hello")
> > >     print(message)
> > >
> > >
> > > What is the equivalent for coro.send("some data") in asyncio?
> > 
> > I don't know of any reason why you couldn't do it just like the above.
> > However, the exchange would not be asynchronous, if that is your goal.
> > 
> > > coro.send on an asyncio coroutine throws AssertionError: yield from wasn't used with future.
> > 
> > So somehow a future got involved where it shouldn't have been. What
> > was the actual code that you tried to run?
> > 
> > Note that while "yield" and "yield from" look similar, they are quite
> > different, and you cannot send to a generator that is currently paused
> > at a "yield from".
> > 
> > If you want to emulate bidirectional communication similar to
> > coro.send asynchronously, I think you'll need to use Futures to
> > mediate, something like this (lightly tested):
> > 
> > @asyncio.coroutine
> > def foo(fut):
> >     data, fut = yield from send_to_future(8, fut)
> >     print("foo", data)
> >     fut.set_result("bye")
> > 
> > @asyncio.coroutine
> > def bar():
> >     n, fut = yield from start_coro(foo)
> >     print("bar", n)
> >     message = yield from send_to_future("hello", fut)
> >     print("bar", message)
> > 
> > def start_coro(coro):
> >     future = asyncio.Future()
> >     asyncio.async(coro(future))
> >     return future
> > 
> > def send_to_future(data, future):
> >     new_future = asyncio.Future()
> >     future.set_result((data, new_future))
> >     return new_future
> 
> 
> 
> 
> Hello again. I have been investigating a bit your example. I don't understand why I can't write something like this:
> 
> --------
> 
> import asyncio
> 
> def foo():
>     print("start foo")
>     try:
>         while True:
>             val = yield
>             print("foo:", val)
>             yield from asyncio.sleep(3)
>     except GeneratorExit:
>         print("foo closed")
>     print("exit foo")
> 
> def bar(next):
>     print("start bar")
>     next.send(None)
>     try:
>         while True:
>             val = yield
>             next.send("bar/"+val)
>     except GeneratorExit:
>         print("bar closed")
>     print("exit bar")
> 
> def fun(next):
>     next.send(None)
>     for e in ["hello", "world", "I'm", "pythonist"]:
>         next.send(e)
> 
> @asyncio.coroutine
> def run():
>     fun(bar(foo()))
> 
> loop = asyncio.get_event_loop()
> loop.run_until_complete(run())
> loop.close()
> 
> -------
> 
> The expected output is:
> 
> start bar
> start foo
> foo: bar/hello
> foo: bar/world
> foo: bar/I'm
> foo: bar/phytonist
> bar closed
> exit bar
> foo closed
> exit foo
> 
> But the yield from asyncio.sleep(3) call raises AssertionError, however it's inside a Task!
> I think this is a big flaw in python/asyncio design.


I think that force the developer to 'yield from' all function calls to keep async capabilities is a big mistake, it should be more flexible, like this:

import asyncio

@asyncio.coroutine
fun non_blocking_io():
    """ Everybody knows I'm doing non blocking IO """
    ...

fun foo():
    """ I invoke functions that do IO stuff """
    data = yield from non_blocking_io()
    yield from store_data_db(data)
    ...

fun bar():
    """ I don't know what foo implementation does """
    foo()

asyncio.async(bar())


Does python 3.5 await/async solve this?

[toc] | [prev] | [next] | [standalone]

#94712

On Tue, Jul 28, 2015 at 2:41 PM, Javier <jcarmena@gmail.com> wrote:
> I think that force the developer to 'yield from' all function calls to keep async capabilities is a big mistake, it should be more flexible, like this:
>
> import asyncio
>
> @asyncio.coroutine
> fun non_blocking_io():
>     """ Everybody knows I'm doing non blocking IO """
>     ...
>
> fun foo():
>     """ I invoke functions that do IO stuff """
>     data = yield from non_blocking_io()
>     yield from store_data_db(data)
>     ...
>
> fun bar():
>     """ I don't know what foo implementation does """
>     foo()
>
> asyncio.async(bar())

Do you have a proposal for implementing this? What does "yield from"
actually do here? How does the event loop know that foo is waiting for
something and what it is waiting for?

> Does python 3.5 await/async solve this?

Yes and no. await is basically equivalent to yield from with extra
validation, so you still can't use them like what you have above. But
native coroutines are also clearly not generators as they lack
__next__ and __iter__ methods, so maybe there will be less temptation
to try to use them without using await / yield from.

[toc] | [prev] | [next] | [standalone]

#94711

On Tue, Jul 28, 2015 at 1:17 PM, Javier <jcarmena@gmail.com> wrote:
> Hello again. I have been investigating a bit your example. I don't understand why I can't write something like this:
>
> --------
>
> import asyncio
>
> def foo():
>     print("start foo")
>     try:
>         while True:
>             val = yield
>             print("foo:", val)
>             yield from asyncio.sleep(3)
>     except GeneratorExit:
>         print("foo closed")
>     print("exit foo")
>
> def bar(next):
>     print("start bar")
>     next.send(None)
>     try:
>         while True:
>             val = yield
>             next.send("bar/"+val)
>     except GeneratorExit:
>         print("bar closed")
>     print("exit bar")
>
> def fun(next):
>     next.send(None)
>     for e in ["hello", "world", "I'm", "pythonist"]:
>         next.send(e)
>
> @asyncio.coroutine
> def run():
>     fun(bar(foo()))
>
> loop = asyncio.get_event_loop()
> loop.run_until_complete(run())
> loop.close()

Because "yield from asyncio.sleep(3)" doesn't magically pause the
coroutine as you want it to. It yields a future, which is meant to be
yielded back up the coroutine chain to the event loop. The event loop
would then resume the coroutine once the future is done, which in the
case of asyncio.sleep will happen after the sleep timer completes.

In your example, the future never makes it back to the event loop.
asyncio.sleep yields the future to foo, and since foo is suspended by
a yield from, foo yields the future to bar, where it is the result of
the next.send call. The return value of next.send is ignored, so the
future just gets dropped on the floor at this point. bar yields to
fun, which sends bar the next string in its list, "world". bar sends
"world" to foo, and since foo is still suspended by a yield from, it
sends "world" on to the asyncio.sleep future. Not a new asyncio.sleep
future, but the same one that it's still yielding from. The future
then realizes that something is wrong, because its generator code is
running again but it doesn't have a result yet, so it throws that
AssertionError.

If you want the yield from in foo to work properly, then you need to
make sure that the future gets back to the event loop, and if you do
that in some tricky way other than a yield from chain, you'll also
need to make sure that you're not trying to send it more data before
foo has resumed.

> I think this is a big flaw in python/asyncio design.

I don't entirely disagree. I think that the implementation of async
coroutines on top of synchronous coroutines on top of generators is
overly clever and results in a somewhat leaky abstraction and a fair
amount of confusion.

[toc] | [prev] | [next] | [standalone]

#94718

On Wednesday, July 29, 2015 at 4:37:22 AM UTC+5:30, Ian wrote:
> I don't entirely disagree. I think that the implementation of async
> coroutines on top of synchronous coroutines on top of generators is
> overly clever and results in a somewhat leaky abstraction and a fair
> amount of confusion.

Hear Hear!

I think the 'overly clever' started with yield becoming an expression.
At an operational level coroutines and generators are very similar
At a conceptual level they are quite apart: coroutines generalize subroutines
whereas generators generalize lists to possibly infinite, possibly incomplete
lists

[toc] | [prev] | [next] | [standalone]

#94734

El miércoles, 29 de julio de 2015, 1:07:22 (UTC+2), Ian  escribió:
> On Tue, Jul 28, 2015 at 1:17 PM, Javier <jcarmena@gmail.com> wrote:
> > Hello again. I have been investigating a bit your example. I don't understand why I can't write something like this:
> >
> > --------
> >
> > import asyncio
> >
> > def foo():
> >     print("start foo")
> >     try:
> >         while True:
> >             val = yield
> >             print("foo:", val)
> >             yield from asyncio.sleep(3)
> >     except GeneratorExit:
> >         print("foo closed")
> >     print("exit foo")
> >
> > def bar(next):
> >     print("start bar")
> >     next.send(None)
> >     try:
> >         while True:
> >             val = yield
> >             next.send("bar/"+val)
> >     except GeneratorExit:
> >         print("bar closed")
> >     print("exit bar")
> >
> > def fun(next):
> >     next.send(None)
> >     for e in ["hello", "world", "I'm", "pythonist"]:
> >         next.send(e)
> >
> > @asyncio.coroutine
> > def run():
> >     fun(bar(foo()))
> >
> > loop = asyncio.get_event_loop()
> > loop.run_until_complete(run())
> > loop.close()
> 
> Because "yield from asyncio.sleep(3)" doesn't magically pause the
> coroutine as you want it to. It yields a future, which is meant to be
> yielded back up the coroutine chain to the event loop. The event loop
> would then resume the coroutine once the future is done, which in the
> case of asyncio.sleep will happen after the sleep timer completes.
> 
> In your example, the future never makes it back to the event loop.
> asyncio.sleep yields the future to foo, and since foo is suspended by
> a yield from, foo yields the future to bar, where it is the result of
> the next.send call. The return value of next.send is ignored, so the
> future just gets dropped on the floor at this point. bar yields to
> fun, which sends bar the next string in its list, "world". bar sends
> "world" to foo, and since foo is still suspended by a yield from, it
> sends "world" on to the asyncio.sleep future. Not a new asyncio.sleep
> future, but the same one that it's still yielding from. The future
> then realizes that something is wrong, because its generator code is
> running again but it doesn't have a result yet, so it throws that
> AssertionError.
> 
> If you want the yield from in foo to work properly, then you need to
> make sure that the future gets back to the event loop, and if you do
> that in some tricky way other than a yield from chain, you'll also
> need to make sure that you're not trying to send it more data before
> foo has resumed.
> 
> > I think this is a big flaw in python/asyncio design.
> 
> I don't entirely disagree. I think that the implementation of async
> coroutines on top of synchronous coroutines on top of generators is
> overly clever and results in a somewhat leaky abstraction and a fair
> amount of confusion.



I see, so I was wrong. I used to see event loop, and yield from as a language improvement but it's actually a library improvement that, as a side effect, prevents you from usign some language features like ordinary generator/coroutine communication. That is a pity. 

Thank you very much for your time and explanations, now I understand some points. I'll keep on learning asyncio.

[toc] | [prev] | [next] | [standalone]

#94839

El martes, 21 de julio de 2015, 13:31:56 (UTC+2), Javier  escribió:
> Hello, I'm trying to understand and link asyncio with ordinary coroutines. Now I just want to understand how to do this on asyncio:
> 
> 
> def foo():
>     data = yield 8
>     print(data)
>     yield "bye"
>     
> def bar():
>     f = foo()
>     n = f.next()
>     print(n)
>     message = f.send("hello")
>     print(message)
> 
> 
> What is the equivalent for coro.send("some data") in asyncio?
> 
> coro.send on an asyncio coroutine throws AssertionError: yield from wasn't used with future.
> 
> 
> Thank you




Asyncio is a crazy headache! I realized that I can't use asyncio tcp servers with pickle! Asyncio is good as a concept but bad in real life. 

I think python's non blocking I/O is far from being something useful for developers till non-async code can invoke async code transparently. Duplicating all code/libs when you realize that something not fits asyncio is not a solution and even less a pythonic solution.

[toc] | [prev] | [next] | [standalone]

#94844

On 01/08/2015 17:07, Javier wrote:
>
> Asyncio is a crazy headache! I realized that I can't use asyncio tcp servers with pickle! Asyncio is good as a concept but bad in real life.
>
> I think python's non blocking I/O is far from being something useful for developers till non-async code can invoke async code transparently. Duplicating all code/libs when you realize that something not fits asyncio is not a solution and even less a pythonic solution.
>

I'll keep my eyes open for your solution on pypi as clearly you know 
better than the Python core developers.  Fully documented and tested of 
course.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#94848

El sábado, 1 de agosto de 2015, 18:45:17 (UTC+2), Mark Lawrence  escribió:
> On 01/08/2015 17:07, Javier wrote:
> >
> > Asyncio is a crazy headache! I realized that I can't use asyncio tcp servers with pickle! Asyncio is good as a concept but bad in real life.
> >
> > I think python's non blocking I/O is far from being something useful for developers till non-async code can invoke async code transparently. Duplicating all code/libs when you realize that something not fits asyncio is not a solution and even less a pythonic solution.
> >
> 
> I'll keep my eyes open for your solution on pypi as clearly you know 
> better than the Python core developers.  Fully documented and tested of 
> course.
> 
> -- 
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
> 
> Mark Lawrence

Don't be rascal, friend. Nobody thinks that self is better than core developers, and personaly I don't think I am better than anybody, but, I have my own opinion.
The fact that the core team is better at python coding and compilers than me does not mean that her implementation fits my needs/opinion.

I dont like the current event loop implementation (plus yield from): 
- It forces me to use beta libraries for doing something that another production ready library already does, 
- I can't use one of the most awesome python features: yielding between generators and coroutines
- As you said, I'm not as smart as they are but I thought python is for everybody, not only for them. Should I ask them how can I use pickle with streamreader? I think it would be better if it would become trivial.


I read PEPs. I don't imagine a good users community if all of us would have to read the pythons source code to understand how to use new features or rewrite a new implementation proposal to be allowed to tell our opinion, as you claims, 

I would like that all the code would run on event loop, but till then I just believe that event loop should be a native feature, not a python hack. I believe that we should be able to call asyncio functions without doing "yield from" through all the stack. I believe that It's possible an implementation that allows generators for processing data streams and that allows to mix classic code with loop code.

And finally I claim a place for not super-level pythoners that want to talk about features improvement without being asked for a reference implementation, each of us have our own role.

:)

[toc] | [prev] | [next] | [standalone]

#94851

Javier <jcarmena@gmail.com>:

> El sábado, 1 de agosto de 2015, 18:45:17 (UTC+2), Mark Lawrence  escribió:
>> clearly you know better than the Python core developers
>
> Nobody thinks that self is better than core developers, and personaly
> I don't think I am better than anybody, but, I have my own opinion.

It is odd how an engineering forum like this one so often judges ideas
based on the pedigree of the participants rather than objective
technical arguments.


Marko

[toc] | [prev] | [next] | [standalone]

#94852

On 01/08/2015 19:38, Marko Rauhamaa wrote:
> Javier <jcarmena@gmail.com>:
>
>> El sábado, 1 de agosto de 2015, 18:45:17 (UTC+2), Mark Lawrence  escribió:
>>> clearly you know better than the Python core developers
>>
>> Nobody thinks that self is better than core developers, and personaly
>> I don't think I am better than anybody, but, I have my own opinion.
>
> It is odd how an engineering forum like this one so often judges ideas
> based on the pedigree of the participants rather than objective
> technical arguments.
>
>
> Marko
>

What I find odd is that the bleating and whinging comes long after the 
PEP process has finished and the code is already in production. 
Wouldn't it be easier for everybody all around to sort this out right at 
the start of the process?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#94854

El sábado, 1 de agosto de 2015, 20:46:49 (UTC+2), Mark Lawrence  escribió:
> On 01/08/2015 19:38, Marko Rauhamaa wrote:
> > Javier <jcarmena@gmail.com>:
> >
> >> El sábado, 1 de agosto de 2015, 18:45:17 (UTC+2), Mark Lawrence  escribió:
> >>> clearly you know better than the Python core developers
> >>
> >> Nobody thinks that self is better than core developers, and personaly
> >> I don't think I am better than anybody, but, I have my own opinion.
> >
> > It is odd how an engineering forum like this one so often judges ideas
> > based on the pedigree of the participants rather than objective
> > technical arguments.
> >
> >
> > Marko
> >
> 
> What I find odd is that the bleating and whinging comes long after the 
> PEP process has finished and the code is already in production. 
> Wouldn't it be easier for everybody all around to sort this out right at 
> the start of the process?
> 
> -- 
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
> 
> Mark Lawrence


I think so too, Mark, but I talk about it when I have used it. I came from blocking IO development. But isn't this a live language?

Well! let's forget all this and let's work with python 3.4 :)


My intention now is to use the asyncio.StreamReader passed as argument to the asyncio.start_server callback to read objects serialized with pickle. The problems are that pickle cant read from it (because dont yield from the full stack) and that I don't know the exact length of each serialized object, so I can't extract data manually.

Knows anybody the steps to follow?

Thanks

[toc] | [prev] | [next] | [standalone]

#94856

Javier <jcarmena@gmail.com>:

> My intention now is to use the asyncio.StreamReader passed as argument
> to the asyncio.start_server callback to read objects serialized with
> pickle. The problems are that pickle cant read from it (because dont
> yield from the full stack) and that I don't know the exact length of
> each serialized object, so I can't extract data manually.
>
> Knows anybody the steps to follow?

You probably need to frame your pickled encoding. A simple thing would
be to use struct.pack/unpack to encode the length of the pickle encoding
in front of the pickle encoding. When decoding, you first read out the
length, then the pickle encoding, and finally you unpickle the payload.


Marko

[toc] | [prev] | [next] | [standalone]

#94881

El sábado, 1 de agosto de 2015, 21:15:07 (UTC+2), Marko Rauhamaa  escribió:
> Javier <jcarmena@gmail.com>:
> 
> > My intention now is to use the asyncio.StreamReader passed as argument
> > to the asyncio.start_server callback to read objects serialized with
> > pickle. The problems are that pickle cant read from it (because dont
> > yield from the full stack) and that I don't know the exact length of
> > each serialized object, so I can't extract data manually.
> >
> > Knows anybody the steps to follow?
> 
> You probably need to frame your pickled encoding. A simple thing would
> be to use struct.pack/unpack to encode the length of the pickle encoding
> in front of the pickle encoding. When decoding, you first read out the
> length, then the pickle encoding, and finally you unpickle the payload.
> 
> 
> Marko


Great! it works! I could not find a way to send an integer with fixed length, struct.pack/unpack does the job perfectly.

Thanks!

[toc] | [prev] | [next] | [standalone]

#94858

On 01/08/2015 20:07, Javier wrote:
> El sábado, 1 de agosto de 2015, 20:46:49 (UTC+2), Mark Lawrence  escribió:
>
> Well! let's forget all this and let's work with python 3.4 :)
>

Please keep up, 3.5 is in beta and the current default will be 3.6 :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#94875

On Sun, Aug 2, 2015 at 5:33 AM, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
> Please keep up, 3.5 is in beta and the current default will be 3.6 :)

Not sure what you mean by "default", but I've been running 3.6.0a0 for
a while now :)

ChrisA

[toc] | [prev] | [next] | [standalone]

#94855

Mark Lawrence <breamoreboy@yahoo.co.uk>:

> On 01/08/2015 19:38, Marko Rauhamaa wrote:
>> It is odd how an engineering forum like this one so often judges
>> ideas based on the pedigree of the participants rather than objective
>> technical arguments.
>
> What I find odd is that the bleating and whinging comes long after the
> PEP process has finished and the code is already in production.
> Wouldn't it be easier for everybody all around to sort this out right
> at the start of the process?

Err...

 * You are free to say you don't like something, even aspects of Python.

 * Criticism can be well-founded even if the critic wasn't around when
   the developers put the spec together.

 * Most Python users (as in 99.999% of them) aren't Python developers.
   That's as it should be.

 * Even if critics had been around and voiced their criticism, asyncio
   probably would have been introduced.

 * Even if somebody doesn't like an idea, somebody else might love it.

Are you happy with asyncio, Mark?


Marko

[toc] | [prev] | [next] | [standalone]

#94857

On 01/08/2015 20:09, Marko Rauhamaa wrote:
> Mark Lawrence <breamoreboy@yahoo.co.uk>:
>
>> On 01/08/2015 19:38, Marko Rauhamaa wrote:
>>> It is odd how an engineering forum like this one so often judges
>>> ideas based on the pedigree of the participants rather than objective
>>> technical arguments.
>>
>> What I find odd is that the bleating and whinging comes long after the
>> PEP process has finished and the code is already in production.
>> Wouldn't it be easier for everybody all around to sort this out right
>> at the start of the process?
>
> Err...
>
>   * You are free to say you don't like something, even aspects of Python.
>
>   * Criticism can be well-founded even if the critic wasn't around when
>     the developers put the spec together.
>
>   * Most Python users (as in 99.999% of them) aren't Python developers.
>     That's as it should be.
>
>   * Even if critics had been around and voiced their criticism, asyncio
>     probably would have been introduced.
>
>   * Even if somebody doesn't like an idea, somebody else might love it.
>
> Are you happy with asyncio, Mark?
>
>
> Marko
>

Yes.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.python

csiph-web