Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #71125

Re: Real-world use of concurrent.futures

Show all headers | View raw

Andrew McLean <lists@andros.org.uk>:

> That's fine, but I suspect that isn't a helpful pattern if I have a
> very large number of tasks. In my case I could run out of memory if I
> tried submitting all of the tasks to the executor before processing
> any of the results.

This is related to flow control. You'll need an object for each flow
(transaction). When new work comes in from the network, you'll have to
see if you are hitting the maximum number of pending transactions, and
not start another one before previous transactions have been processed.

Whenever a transaction is completed, you pull in more work.


Marko

Back to comp.lang.python | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Real-world use of concurrent.futures Andrew McLean <lists@andros.org.uk> - 2014-05-08 19:55 +0100
  Re: Real-world use of concurrent.futures Marko Rauhamaa <marko@pacujo.net> - 2014-05-08 22:56 +0300

csiph-web