Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #68340

asyncio question

Show all headers | View raw

I have a portion of code I need to speed up, there are 3 api calls to an external system
where the first enumerates a large collection of objects I then loop through and perform
two additional api calls each. The first call is instant, the second and third per object are
very slow. Currently after accumulating all the data I write the relevant data into a database.

I have the ability to hold all this in memory and dump it once fully accumulated, so performing
the second and third call in parallel with fixed batches would be great,

I took a look at coroutines and some skeleton code worked fine, but I am not sure how to
perform the acquisition in fixed groups like I might for example with multiprocessing and
a pool of workers.

Anyone done something like this or have an opinion?

Thanks,
jlc

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

asyncio question "Joseph L. Casale" <jcasale@activenetwerx.com> - 2014-03-13 21:24 +0000

csiph-web