Re: Question about asyncio and blocking operations

From	Ian Kelly <ian.g.kelly@gmail.com>
Newsgroups	comp.lang.python
Subject	Re: Question about asyncio and blocking operations
Date	2016-01-28 14:49 -0700
Message-ID	<mailman.66.1454017808.2338.python-list@python.org> (permalink)
References	(9 earlier) <n8ct18$fu4$1@ger.gmane.org> <CALwzidnGbz7kM=D7MKua2tA9-csFn9u0OHL0w-X5Bbixpcw4Ow@mail.gmail.com> <n8dgas$s1u$1@ger.gmane.org> <CALwzidn6NfT_O0cfHw1itWja81+MW3scHuEcADVCen3ix6z73w@mail.gmail.com> <CAGqiJR-kbF2HDTPRG=49yEDkyZZRQbUC-=3Dq_bApab2exsKAw@mail.gmail.com>

Show all headers | View raw

On Thu, Jan 28, 2016 at 2:23 PM, Maxime S <maxischmeii@gmail.com> wrote:
>
> 2016-01-28 17:53 GMT+01:00 Ian Kelly <ian.g.kelly@gmail.com>:
>>
>> On Thu, Jan 28, 2016 at 9:40 AM, Frank Millman <frank@chagford.com> wrote:
>>
>> > The caller requests some data from the database like this.
>> >
>> >    return_queue = asyncio.Queue()
>> >    sql = 'SELECT ...'
>> >    request_queue.put((return_queue, sql))
>>
>> Note that since this is a queue.Queue, the put call has the potential
>> to block your entire event loop.
>>
>
> Actually, I don't think you actually need an asyncio.Queue.
>
> You could use a simple deque as a buffer, and call fetchmany() when it is
> empty, like that (untested):

True. The asyncio Queue is really just a wrapper around a deque with
an interface designed for use with the producer-consumer pattern. If
the producer isn't a coroutine then it may not be appropriate.

This seems like a nice suggestion. Caution is advised if multiple
cursor methods are executed concurrently since they would be in
different threads and the underlying cursor may not be thread-safe.

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: Question about asyncio and blocking operations Ian Kelly <ian.g.kelly@gmail.com> - 2016-01-28 14:49 -0700

csiph-web