Path: csiph.com!feeder.erje.net!2.eu.feeder.erje.net!newsfeed.kamp.net!newsfeed.kamp.net!fu-berlin.de!uni-berlin.de!not-for-mail
From: Kevin Conway <kevinjacobconway@gmail.com>
Newsgroups: comp.lang.python
Subject: Re: asyncio - run coroutine in the background
Date: Tue, 16 Feb 2016 14:54:04 +0000
Lines: 60
Message-ID: <mailman.172.1455634489.22075.python-list@python.org>
References: <mailman.131.1455518141.22075.python-list@python.org> <8737sumpjl.fsf@elektro.pacujo.net> <mailman.132.1455520607.22075.python-list@python.org> <87h9ha8lt0.fsf@jester.gateway.pace.com> <CAPTjJmoWUDfcOe3OCaMK7=zDtW3QzS+7hqUG6DoYZ2AbqzJBUw@mail.gmail.com> <mailman.166.1455628969.22075.python-list@python.org> <87d1rwpwo2.fsf@elektro.pacujo.net> <n9vc4j$n89$1@ger.gmane.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
In-Reply-To: <n9vc4j$n89$1@ger.gmane.org>
Precedence: list
Xref: csiph.com comp.lang.python:103011

> Ok, yes, but those "background tasks" monopolize the CPU once they are
scheduled to run.

This is true if the coroutines are cpu bound. If that is the case then a
coroutine is likely the wrong choice for that code to begin with.
Coroutines, in asyncio land, are primarily designed for io bound work.

> My background task does take a long time to run - about 10 seconds - but
most of that time is spent waiting for database responses, which is handled
in another thread.

Something else to look into is an asyncio driver for your database
connections. Threads aren't inherently harmful, but using them to achieve
async networking when running asyncio is a definite code smell since that
is precisely the problem asyncio is supposed to solve for.

On Tue, Feb 16, 2016, 08:37 Frank Millman <frank@chagford.com> wrote:

> "Marko Rauhamaa"  wrote in message news:87d1rwpwo2.fsf@elektro.pacujo.net.
> ..
> >
> > Kevin Conway <kevinjacobconway@gmail.com>:
> >
> > > If you're handling coroutines there is an asyncio facility for
> > > "background tasks". The ensure_future [1] will take a coroutine,
> > > attach it to a Task, and return a future to you that resolves when the
> > > coroutine is complete.
> >
> > Ok, yes, but those "background tasks" monopolize the CPU once they are
> > scheduled to run.
> >
> > If your "background task" doesn't need a long time to run, just call the
> > function in the foreground and be done with it. If it does consume time,
> > you need to delegate it to a separate process so the other tasks remain
> > responsive.
> >
>
> I will explain my situation - perhaps you can tell me if it makes sense.
>
> My background task does take a long time to run - about 10 seconds - but
> most of that time is spent waiting for database responses, which is handled
> in another thread.
>
> You could argue that the database thread should rather be handled by
> another
> process, and that is definitely an option if I find that response times are
> affected.
>
> So far my response times have been very good, even with database activity
> in
> the background. However, I have not simulated a large number of concurrent
> users. That could throw up the kinds of problem that you are concerned
> about.
>
> Frank
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>