Path: csiph.com!x330-a1.tempe.blueboxinc.net!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!news.tele.dk!feed118.news.tele.dk!news.tele.dk!small.news.tele.dk!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <richardbp@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.066
X-Spam-Evidence: '*H*': 0.87; '*S*': 0.00; 'python': 0.08; 'async': 0.16; 'libraries?': 0.16; 'cc:addr:python-list': 0.16; 'received:74.125.82.44': 0.16; 'received:mail-ww0-f44.google.com': 0.16; 'cc:no real name:2**0': 0.21; 'header:In-Reply-To:1': 0.22; 'libraries': 0.24; 'scale': 0.25; 'cc:2**0': 0.26; 'fact': 0.27; 'message-id:@mail.gmail.com': 0.29; 'languages.': 0.29; 'cc:addr:python.org': 0.29; 'seem': 0.29; 'asynchronous': 0.30; 'thread': 0.32; 'there': 0.33; 'richard': 0.34; 'external': 0.35; 'received:74.125.82': 0.36; 'but': 0.37; 'received:google.com': 0.37; 'some': 0.38; 'received:74.125': 0.38; 'difficult': 0.39; 'header:Received:6': 0.61; 'processes,': 0.67; 'offer': 0.71; 'compete.': 0.84; 'received:10.180': 0.84
Received-SPF: pass (google.com: domain of richardbp@gmail.com designates 10.180.94.68 as permitted sender) client-ip=10.180.94.68;
Authentication-Results: mr.google.com; spf=pass (google.com: domain of richardbp@gmail.com designates 10.180.94.68 as permitted sender) smtp.mail=richardbp@gmail.com; dkim=pass header.i=richardbp@gmail.com
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=nppTUDhj4AUwAYGBGzgOaJMmOtrEEW3GL/9QOjJkmWA=; b=kidYTA7z0w+fJcbVjuAL9G6PhuW8RUnvwExAsXGl4FTh7vfqUyMCNiYKDdTHHLYg9C EK6T3QgfQdwyCYWoRS6md35nram1iQz+ruuSVI+RzB85FCjR3AL2uH9NpZ2docb8B8it GZh83HKazGjPjTNeQxM85clHHaR8sRFsVRN2E=
MIME-Version: 1.0
In-Reply-To: <4F462108.2000400@skippinet.com.au>
References: <33089103.45.1329980290357.JavaMail.geo-discussion-forums@pbne2> <4F462108.2000400@skippinet.com.au>
From: Richard Baron Penman <richardbp@gmail.com>
Date: Fri, 24 Feb 2012 00:50:34 +1100
Subject: Re: asynchronous downloading
To: Mark Hammond <mhammond@skippinet.com.au>
Content-Type: text/plain; charset=ISO-8859-1
Cc: python-list@python.org
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.74.1330005055.3037.python-list@python.org>
Lines: 16
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1330005055 news.xs4all.nl 6864 [2001:888:2000:d::a6]:35069
X-Complaints-To: abuse@xs4all.nl
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:20727

>> I want to download content asynchronously. This would be
>> straightforward to do threaded or across processes, but difficult
>> asynchronously so people seem to rely on external libraries (twisted
>> / gevent / eventlet).
>
>
> Exactly - the fact it's difficult is why those tools compete.

It is difficult in Python because the async libraries do not offer
much. Straightforward in some other languages.

Do you know why there is little support for asynchronous execution in
the standard libraries?
For large scale downloading I found thread pools do not scale well.

Richard