Re: downloading from links within a webpage

Path	csiph.com!usenet.pasdenom.info!gegeweb.org!usenet-fr.net!nerim.net!novso.com!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path	<joel.goldstick@gmail.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.050
X-Spam-Evidence	'H': 0.90; 'S': 0.00; 'parser': 0.07; 'to)': 0.09; 'cc:addr:python-list': 0.11; 'python': 0.11; 'wrote': 0.14; 'idea:': 0.16; 'limiting': 0.16; 'modules.': 0.16; '(you': 0.16; 'wrote:': 0.18; 'wed,': 0.18; 'cc:addr:python.org': 0.22; 'cc:2**0': 0.24; '15,': 0.26; 'header:In-Reply-To:1': 0.27; 'chris': 0.29; 'external': 0.29; 'am,': 0.29; 'specified': 0.30; 'message-id:@mail.gmail.com': 0.30; 'url:mailman': 0.30; 'code': 0.31; 'easier': 0.31; 'requests': 0.31; 'depth': 0.31; 'url:python': 0.33; 'subject:from': 0.34; 'received:google.com': 0.35; '14,': 0.36; 'url:listinfo': 0.36; 'url:org': 0.36; 'searching': 0.37; 'does': 0.39; 'url:mail': 0.40; 'how': 0.40; 'first': 0.61; 'more': 0.64; 'here': 0.66; 'webpage': 0.68; 'limit': 0.70; '3.4': 0.84; 'easier,': 0.84; 'joel': 0.91; 'to:none': 0.92
DKIM-Signature	v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:cc :content-type; bh=ju+kNsVuH0v66WOWwv1UeWmaZm9+pOMrvcd/DPFkx/8=; b=MxZHiPzhuY+nPWDM7fz6aA3QxFoezfZEC/Rdqi8K22Jaf9MPNHVXF3IzUCXEZQSV/9 4Qm+x/N/PV1Rxct4gqO1SQvzyfmnFv/ezdRhfcsdQqBAaqNOB1fe4fTbNJ2dA51bvVur L0h00JqNHtIu2ZgRG86/sGQviOCVZT4Tz1RkZeyM2rFbosGK08Cn2FSKs42pBwXL5TX4 o3mUeRaQKYt1VPasxlxaV+PXaIj6EdHQRZBWjb5RPPRMcNz//Lx8SEdvfS7PD8qm2vtu PUxlKamIMXYieJXT1wjFN6oVLlcDj6GzV5syawDESsUM+2l8Bs25B6Wg0I7JCp/s7s7M 81ag==
MIME-Version	1.0
X-Received	by 10.224.167.208 with SMTP id r16mr7766122qay.42.1413298642217; Tue, 14 Oct 2014 07:57:22 -0700 (PDT)
In-Reply-To	<CAPTjJmpcu5zi1+r9nVHEWot0j7Xqxyms8_2_RRrKN=Hnu-7bHg@mail.gmail.com>
References	<loom.20141014T163517-784@post.gmane.org> <CAPTjJmpcu5zi1+r9nVHEWot0j7Xqxyms8_2_RRrKN=Hnu-7bHg@mail.gmail.com>
Date	Tue, 14 Oct 2014 10:57:22 -0400
Subject	Re: downloading from links within a webpage
From	Joel Goldstick <joel.goldstick@gmail.com>
Cc	"python-list@python.org" <python-list@python.org>
Content-Type	text/plain; charset=UTF-8
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.15
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.14855.1413298650.18130.python-list@python.org> (permalink)
Lines	26
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1413298650 news.xs4all.nl 2939 [2001:888:2000:d::a6]:57716
X-Complaints-To	abuse@xs4all.nl
Xref	csiph.com comp.lang.python:79103

Show key headers only | View raw

On Tue, Oct 14, 2014 at 10:54 AM, Chris Angelico <rosuav@gmail.com> wrote:
> On Wed, Oct 15, 2014 at 1:42 AM, Shiva
> <shivaji_tn@yahoo.com.dmarc.invalid> wrote:
>> Here is a small code that I wrote that downloads images from a webpage url
>> specified (you can limit to how many downloads you want). However, I am
>> looking at adding functionality and searching external links from this page
>> and downloading the same number of images from that page as well.(And
>> limiting the depth it can go to)
>>
>> Any ideas?  (I am using Python 3.4 & I am a beginner)
>
> First idea: Use wget, it does all this for you :)

You might look at Requests and BeautifulSoup python modules.  Requests
is easier for many things than urllib.  BS is an HTML parser that may
be easier, and more powerful than what you can do with regex
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
Joel Goldstick
http://joelgoldstick.com

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: downloading from links within a webpage Joel Goldstick <joel.goldstick@gmail.com> - 2014-10-14 10:57 -0400

csiph-web