Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #103014

Re: asyncio - run coroutine in the background

Path csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From "Frank Millman" <frank@chagford.com>
Newsgroups comp.lang.python
Subject Re: asyncio - run coroutine in the background
Date Tue, 16 Feb 2016 17:21:04 +0200
Lines 37
Message-ID <mailman.173.1455636082.22075.python-list@python.org> (permalink)
References <mailman.131.1455518141.22075.python-list@python.org> <8737sumpjl.fsf@elektro.pacujo.net> <mailman.132.1455520607.22075.python-list@python.org> <87h9ha8lt0.fsf@jester.gateway.pace.com> <CAPTjJmoWUDfcOe3OCaMK7=zDtW3QzS+7hqUG6DoYZ2AbqzJBUw@mail.gmail.com> <mailman.166.1455628969.22075.python-list@python.org> <87d1rwpwo2.fsf@elektro.pacujo.net> <n9vc4j$n89$1@ger.gmane.org> <CAKF=+dhXZ=yax8STAWr_gjX3Tg8yUjPRJG-7yM2_bRV2Kxm3Jg@mail.gmail.com>
Mime-Version 1.0
Content-Type text/plain; format=flowed; charset="iso-8859-1"; reply-type=original
Content-Transfer-Encoding 7bit
X-Trace news.uni-berlin.de nQAfpTLbAFikpWLQr+74qAW8r8LC4ZrNuynRt6lGKAoQ==
Return-Path <python-python-list@m.gmane.org>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.005
X-Spam-Evidence '*H*': 0.99; '*S*': 0.00; '(using': 0.07; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'thread': 0.10; 'async': 0.16; 'background,': 0.16; 'processes.': 0.16; 'query,': 0.16; 'received:80.91.229.3': 0.16; 'received:io': 0.16; 'received:plane.gmane.org': 0.16; 'received:psf.io': 0.16; 'skip:n 70': 0.16; 'subject:run': 0.16; 'thread.': 0.16; 'threads': 0.16; 'driver': 0.18; "aren't": 0.22; 'seems': 0.23; 'wrote': 0.23; 'header:In-Reply-To:1': 0.24; 'header:X-Complaints-To:1': 0.26; 'switch': 0.27; "skip:' 10": 0.28; 'far,': 0.29; 'issues.': 0.29; 'occasionally': 0.29; 'handled': 0.29; 'code': 0.30; 'task': 0.30; 'seconds': 0.31; 'supposed': 0.31; 'users.': 0.31; 'another': 0.32; 'maybe': 0.33; 'run': 0.33; 'problem': 0.33; 'usually': 0.33; 'environment,': 0.33; 'running': 0.34; 'server': 0.34; 'something': 0.35; 'but': 0.36; 'to:addr:python-list': 0.36; 'subject:: ': 0.37; 'client': 0.37; 'received:org': 0.37; 'suggestion': 0.37; 'does': 0.39; 'subject:the': 0.39; 'takes': 0.39; 'well.': 0.40; 'to:addr:python.org': 0.40; 'where': 0.40; 'some': 0.40; 'waiting': 0.60; 'your': 0.60; 'hope': 0.61; 'frank': 0.72; 'away,': 0.84; 'networking.': 0.84; 'response,': 0.95
X-Injected-Via-Gmane http://gmane.org/
X-Gmane-NNTP-Posting-Host 197.89.68.71
In-Reply-To <CAKF=+dhXZ=yax8STAWr_gjX3Tg8yUjPRJG-7yM2_bRV2Kxm3Jg@mail.gmail.com>
X-MSMail-Priority Normal
Importance Normal
X-Newsreader Microsoft Windows Live Mail 15.4.3502.922
X-MimeOLE Produced By Microsoft MimeOLE V15.4.3502.922
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.21rc2
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Xref csiph.com comp.lang.python:103014

Show key headers only | View raw

"Kevin Conway"  wrote in message 
news:CAKF=+dhXZ=yax8STAWr_gjX3Tg8yUjPRJG-7yM2_bRV2Kxm3Jg@mail.gmail.com...
>
> > My background task does take a long time to run - about 10 seconds - but
> > most of that time is spent waiting for database responses, which is 
> > handled
> > in another thread.
>
> Something else to look into is an asyncio driver for your database
> connections. Threads aren't inherently harmful, but using them to achieve
> async networking when running asyncio is a definite code smell since that
> is precisely the problem asyncio is supposed to solve for.
>

Maybe I have not explained very well. I am not using threads to achieve 
async networking. I am using asyncio in a client server environment, and it 
works very well. If a client request involves a database query, I use a 
thread to perform that so that it does not slow down the other users. I 
usually want the originating client to block until I have a response, so I 
use 'await'. However, occasionally the request takes some time, and it is 
not necessary for the client to wait for the response, so I want to unblock 
the client straight away, run the task in the background, and then notify 
the client when the task is complete. This is where your suggestion of 
'ensure_future' does the job perfectly.

I would love to drive the database asynchronously, but of the three 
databases I use, only psycopg2 seems to have asyncio support. As my 
home-grown solution (using queues) seems to be working well so far, I am 
sticking with that until I start to experience responsiveness issues. If 
that happens, my first line of attack will be to switch from threads to 
processes.

Hope this makes sense.

Frank

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

asyncio - run coroutine in the background "Frank Millman" <frank@chagford.com> - 2016-02-15 08:35 +0200
  Re: asyncio - run coroutine in the background Marko Rauhamaa <marko@pacujo.net> - 2016-02-15 08:54 +0200
    Re: asyncio - run coroutine in the background "Frank Millman" <frank@chagford.com> - 2016-02-15 09:16 +0200
      Re: asyncio - run coroutine in the background Marko Rauhamaa <marko@pacujo.net> - 2016-02-15 09:34 +0200
      Re: asyncio - run coroutine in the background Paul Rubin <no.email@nospam.invalid> - 2016-02-14 23:39 -0800
        Re: asyncio - run coroutine in the background "Frank Millman" <frank@chagford.com> - 2016-02-15 10:17 +0200
        Re: asyncio - run coroutine in the background Marko Rauhamaa <marko@pacujo.net> - 2016-02-15 13:05 +0200
        Re: asyncio - run coroutine in the background Chris Angelico <rosuav@gmail.com> - 2016-02-16 16:51 +1100
        Re: asyncio - run coroutine in the background Kevin Conway <kevinjacobconway@gmail.com> - 2016-02-16 13:22 +0000
          Re: asyncio - run coroutine in the background Marko Rauhamaa <marko@pacujo.net> - 2016-02-16 16:17 +0200
            Re: asyncio - run coroutine in the background "Frank Millman" <frank@chagford.com> - 2016-02-16 16:36 +0200
            Re: asyncio - run coroutine in the background Kevin Conway <kevinjacobconway@gmail.com> - 2016-02-16 14:54 +0000
            Re: asyncio - run coroutine in the background Steven D'Aprano <steve@pearwood.info> - 2016-02-17 02:17 +1100
              Re: asyncio - run coroutine in the background Marko Rauhamaa <marko@pacujo.net> - 2016-02-16 19:12 +0200
                Re: asyncio - run coroutine in the background Paul Rubin <no.email@nospam.invalid> - 2016-02-17 20:38 -0800
                Re: asyncio - run coroutine in the background Marko Rauhamaa <marko@pacujo.net> - 2016-02-18 08:10 +0200
              Re: asyncio - run coroutine in the background Marko Rauhamaa <marko@pacujo.net> - 2016-02-16 19:13 +0200
              Re: asyncio - run coroutine in the background Marko Rauhamaa <marko@pacujo.net> - 2016-02-16 19:14 +0200
              Re: asyncio - run coroutine in the background Marko Rauhamaa <marko@pacujo.net> - 2016-02-16 19:15 +0200
                Re: asyncio - run coroutine in the background Marko Rauhamaa <marko@pacujo.net> - 2016-02-16 19:15 +0200
                Re: asyncio - run coroutine in the background Robin Becker <robin@reportlab.com> - 2016-02-16 17:52 +0000
            Re: asyncio - run coroutine in the background "Frank Millman" <frank@chagford.com> - 2016-02-16 17:21 +0200
              Re: asyncio - run coroutine in the background Marko Rauhamaa <marko@pacujo.net> - 2016-02-16 19:20 +0200
                Re: asyncio - run coroutine in the background Paul Rubin <no.email@nospam.invalid> - 2016-02-19 23:40 -0800
                Re: asyncio - run coroutine in the background Marko Rauhamaa <marko@pacujo.net> - 2016-02-20 10:13 +0200
                Re: asyncio - run coroutine in the background Paul Rubin <no.email@nospam.invalid> - 2016-02-20 00:37 -0800
                Re: asyncio - run coroutine in the background Chris Angelico <rosuav@gmail.com> - 2016-02-20 19:52 +1100
                Re: asyncio - run coroutine in the background Marko Rauhamaa <marko@pacujo.net> - 2016-02-20 10:59 +0200
                Re: asyncio - run coroutine in the background Chris Angelico <rosuav@gmail.com> - 2016-02-20 20:02 +1100
                Re: asyncio - run coroutine in the background Marko Rauhamaa <marko@pacujo.net> - 2016-02-20 11:28 +0200
                Re: asyncio - run coroutine in the background Kevin Conway <kevinjacobconway@gmail.com> - 2016-02-20 13:52 +0000
                Re: asyncio - run coroutine in the background "Martin A. Brown" <martin@linux-ip.net> - 2016-02-20 09:45 -0800
                Re: asyncio - run coroutine in the background Chris Angelico <rosuav@gmail.com> - 2016-02-21 08:47 +1100
            Re: asyncio - run coroutine in the background Chris Angelico <rosuav@gmail.com> - 2016-02-17 02:28 +1100
            Re: asyncio - run coroutine in the background "Frank Millman" <frank@chagford.com> - 2016-02-16 17:45 +0200
        Re: asyncio - run coroutine in the background "Frank Millman" <frank@chagford.com> - 2016-02-16 15:52 +0200

csiph-web