Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #20727

Re: asynchronous downloading

Path csiph.com!x330-a1.tempe.blueboxinc.net!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!news.tele.dk!feed118.news.tele.dk!news.tele.dk!small.news.tele.dk!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path <richardbp@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.066
X-Spam-Evidence '*H*': 0.87; '*S*': 0.00; 'python': 0.08; 'async': 0.16; 'libraries?': 0.16; 'cc:addr:python-list': 0.16; 'received:74.125.82.44': 0.16; 'received:mail-ww0-f44.google.com': 0.16; 'cc:no real name:2**0': 0.21; 'header:In-Reply-To:1': 0.22; 'libraries': 0.24; 'scale': 0.25; 'cc:2**0': 0.26; 'fact': 0.27; 'message-id:@mail.gmail.com': 0.29; 'languages.': 0.29; 'cc:addr:python.org': 0.29; 'seem': 0.29; 'asynchronous': 0.30; 'thread': 0.32; 'there': 0.33; 'richard': 0.34; 'external': 0.35; 'received:74.125.82': 0.36; 'but': 0.37; 'received:google.com': 0.37; 'some': 0.38; 'received:74.125': 0.38; 'difficult': 0.39; 'header:Received:6': 0.61; 'processes,': 0.67; 'offer': 0.71; 'compete.': 0.84; 'received:10.180': 0.84
Received-SPF pass (google.com: domain of richardbp@gmail.com designates 10.180.94.68 as permitted sender) client-ip=10.180.94.68;
Authentication-Results mr.google.com; spf=pass (google.com: domain of richardbp@gmail.com designates 10.180.94.68 as permitted sender) smtp.mail=richardbp@gmail.com; dkim=pass header.i=richardbp@gmail.com
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=nppTUDhj4AUwAYGBGzgOaJMmOtrEEW3GL/9QOjJkmWA=; b=kidYTA7z0w+fJcbVjuAL9G6PhuW8RUnvwExAsXGl4FTh7vfqUyMCNiYKDdTHHLYg9C EK6T3QgfQdwyCYWoRS6md35nram1iQz+ruuSVI+RzB85FCjR3AL2uH9NpZ2docb8B8it GZh83HKazGjPjTNeQxM85clHHaR8sRFsVRN2E=
MIME-Version 1.0
In-Reply-To <4F462108.2000400@skippinet.com.au>
References <33089103.45.1329980290357.JavaMail.geo-discussion-forums@pbne2> <4F462108.2000400@skippinet.com.au>
From Richard Baron Penman <richardbp@gmail.com>
Date Fri, 24 Feb 2012 00:50:34 +1100
Subject Re: asynchronous downloading
To Mark Hammond <mhammond@skippinet.com.au>
Content-Type text/plain; charset=ISO-8859-1
Cc python-list@python.org
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.12
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.74.1330005055.3037.python-list@python.org> (permalink)
Lines 16
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1330005055 news.xs4all.nl 6864 [2001:888:2000:d::a6]:35069
X-Complaints-To abuse@xs4all.nl
Xref x330-a1.tempe.blueboxinc.net comp.lang.python:20727

Show key headers only | View raw

>> I want to download content asynchronously. This would be
>> straightforward to do threaded or across processes, but difficult
>> asynchronously so people seem to rely on external libraries (twisted
>> / gevent / eventlet).
>
>
> Exactly - the fact it's difficult is why those tools compete.

It is difficult in Python because the async libraries do not offer
much. Straightforward in some other languages.

Do you know why there is little support for asynchronous execution in
the standard libraries?
For large scale downloading I found thread pools do not scale well.

Richard

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

asynchronous downloading Plumo <richardbp@gmail.com> - 2012-02-22 22:58 -0800
  Re: asynchronous downloading Justin Ezequiel <justin.mailinglists@gmail.com> - 2012-02-22 23:25 -0800
  Re: asynchronous downloading Mark Hammond <mhammond@skippinet.com.au> - 2012-02-23 22:20 +1100
  Re: asynchronous downloading Paul Rubin <no.email@nospam.invalid> - 2012-02-23 03:46 -0800
    Re: asynchronous downloading Plumo <richardbp@gmail.com> - 2012-02-23 16:28 -0800
  Re: asynchronous downloading Richard Baron Penman <richardbp@gmail.com> - 2012-02-24 00:50 +1100
  Re: asynchronous downloading Giampaolo Rodolà <g.rodola@gmail.com> - 2012-02-23 18:31 +0100
    Re: asynchronous downloading Plumo <richardbp@gmail.com> - 2012-02-23 17:10 -0800
    Re: asynchronous downloading Plumo <richardbp@gmail.com> - 2012-02-23 17:10 -0800
      Re: asynchronous downloading Fayaz Yusuf Khan <fayaz.yusuf.khan@gmail.com> - 2012-02-23 18:31 -0800
      Re: asynchronous downloading Giampaolo Rodolà <g.rodola@gmail.com> - 2012-02-24 10:03 +0100
      Re: asynchronous downloading Richard Baron Penman <richardbp@gmail.com> - 2012-02-26 08:44 +1100

csiph-web