Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.python > #108283
| From | Michael Selik <michael.selik@gmail.com> |
|---|---|
| Newsgroups | comp.lang.python |
| Subject | Re: python - handling HTTP requests asynchronously |
| Date | 2016-05-07 19:04 +0000 |
| Message-ID | <mailman.462.1462647873.32212.python-list@python.org> (permalink) |
| References | <dc385b5f-a453-441c-9476-ea73515201ad@googlegroups.com> <CAGgTfkO4Gm-JkjY6Fu+O0yO6Se4RSAzA6XUxp4mF_qLnXGKAvA@mail.gmail.com> |
On Fri, May 6, 2016 at 3:01 AM <lordluke80@gmail.com> wrote: > The PDF is generated through an external API. Since currently is generated > on demand, this is handled synchronously via an HTTP request/response. Are you sending the request or are you receiving the request? If you are sending, you can just use threads as you are only doing IO. If you are receiving the requests and generating PDFs, you may want to use subprocesses if the PDF-generation is compute-intensive. > 3) multiprocessing module, with e.g. 10 as workers limit. > multiprocessing.Pool is an easy way to use subprocesses multiprocessing.pool.ThreadPool is an easy way to use threads. It's not well documented, but has the exact same interface as Pool. the goal is to use the API concurrently (e.g. send 10 or 100 http requests > simultaneously, depending on how many concurrent requests the API can > handle). > Sounds like you want to use threads. How does the API let you know you're hitting it too frequently? Perhaps you want to code an exponential backoff and retry wrapper for your API requests.
Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
python - handling HTTP requests asynchronously lordluke80@gmail.com - 2016-05-05 23:56 -0700
Re: python - handling HTTP requests asynchronously justin walters <walters.justin01@gmail.com> - 2016-05-06 08:23 -0700
Re: python - handling HTTP requests asynchronously Michael Selik <michael.selik@gmail.com> - 2016-05-07 19:04 +0000
Re: python - handling HTTP requests asynchronously lordluke80@gmail.com - 2016-05-07 14:42 -0700
Re: python - handling HTTP requests asynchronously Michael Selik <michael.selik@gmail.com> - 2016-05-10 23:54 +0000
Re: python - handling HTTP requests asynchronously edgargdcv@gmail.com - 2016-07-19 11:59 -0700
csiph-web