Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail From: Marko Rauhamaa Newsgroups: comp.lang.python Subject: Re: Benefits of asyncio Date: Tue, 03 Jun 2014 13:08:23 +0300 Organization: A noiseless patient Spider Lines: 36 Message-ID: <87ppiquwyw.fsf@elektro.pacujo.net> References: <874n03t5t9.fsf@elektro.pacujo.net> <7x4n03dor0.fsf@ruckus.brouhaha.com> <878upe8poc.fsf@elektro.pacujo.net> <87wqcyuznv.fsf@elektro.pacujo.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Injection-Info: mx05.eternal-september.org; posting-host="ff5cf27ef3d5b31f034d3b72bdc27a41"; logging-data="20174"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX18Jj3nZXt/ISURpG9mxBVNN" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (gnu/linux) Cancel-Lock: sha1:J8aoe7aWCPOa6JO5MUCiMo8Yloo= sha1:cYmC4Uhw3TWqSawp6pocx2GnLu0= Xref: csiph.com comp.lang.python:72498 Chris Angelico : > Okay, but how do you handle two simultaneous requests going through > the processing that you see above? You *MUST* separate them onto two > transactions, otherwise one will commit half of the other's work. (Or > are you forgetting Databasing 101 - a transaction should be a logical > unit of work?) And since you can't, with most databases, have two > transactions on one connection, that means you need a separate > connection for each request. Given that the advantages of asyncio > include the ability to scale to arbitrary numbers of connections, it's > not really a good idea to then say "oh but you need that many > concurrent database connections". Most systems can probably handle a > few thousand threads without a problem, but a few million is going to > cause major issues; but most databases start getting inefficient at a > few thousand concurrent sessions. I will do whatever I have to. Pooling transaction contexts ("connections") is probably necessary. Point is, no task should ever block. I deal with analogous situations all the time, in fact, I'm dealing with one as we speak. > Alright. I'm throwing down the gauntlet. Write me a purely nonblocking > web site concept that can handle a million concurrent connections, > where each one requires one query against the database, and one in a > hundred of them require five queries which happen atomically. I can do > it with a thread pool and blocking database queries, and by matching > the thread pool size and the database concurrent connection limit, I > can manage memory usage fairly easily; how do you do it efficiently > with pure async I/O? Sorry, I'm going to pass. That doesn't look like a 5-liner. Marko