download pages using a cookies.txt file

Path	csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder2.enfer-du-nord.net!feeder.news-service.com!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path	<jabba.laci@gmail.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.046
X-Spam-Evidence	'H': 0.91; 'S': 0.00; 'subject:using': 0.04; 'python?': 0.05; 'to:name:python mailing list': 0.09; 'subject:file': 0.13; 'cookies': 0.16; 'subject:download': 0.16; 'thanks,': 0.18; 'extract': 0.24; 'message-id:@mail.gmail.com': 0.29; 'hi,': 0.32; 'to:addr:python-list': 0.33; "i've": 0.34; 'external': 0.35; 'received:209.85.161': 0.35; 'post': 0.36; 'file': 0.36; 'question': 0.36; 'page': 0.37; 'received:google.com': 0.38; 'received:209.85': 0.38; 'possible,': 0.39; 'to:addr:python.org': 0.39; 'download': 0.60; 'url:bit': 0.64; 'here:': 0.65; 'making': 0.67; 'url:ly': 0.68
DKIM-Signature	v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:from:date:message-id:subject:to:content-type; bh=11w+mdq/3HbI99PihaqGNulCoSYN2dFkTK77AX6SyM4=; b=VXtteoHsL3OvjERHOU/7EUXg7HFb71qMJVzwuNLePxIJWh5bxMSxWuKyNhrJDdBAHk dOBwqHj+SFZQdB5uQ5XIGJXtx8ci4MvKKWJ3FhI2/JWlvpLNcDra8bWFno9PtHIj/yTG o9rDmckGi2ju+6jKInpoVHWNU1pCRRlvg0ms0=
MIME-Version	1.0
From	Jabba Laci <jabba.laci@gmail.com>
Date	Mon, 5 Sep 2011 21:36:46 -0400
Subject	download pages using a cookies.txt file
To	Python mailing list <python-list@python.org>
Content-Type	text/plain; charset=ISO-8859-1
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.12
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.787.1315273029.27778.python-list@python.org> (permalink)
Lines	15
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1315273029 news.xs4all.nl 2550 [2001:888:2000:d::a6]:53731
X-Complaints-To	abuse@xs4all.nl
Xref	x330-a1.tempe.blueboxinc.net comp.lang.python:12806

Show key headers only | View raw

Hi,

I would like to download a page that requires authentication with
cookies. I've already figured out how to do it with wget (blog post
here: http://bit.ly/pp25LP). In short:
(1) extract cookies from Firefox's cookies.sqlite and store them in a text file
(2) wget --cookies=on --load-cookies=cookies.txt
--keep-session-cookies "http://..."

My question is: how to do it from Python? If possible, I want to avoid
making external calls to wget.

Thanks,

Laszlo

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

download pages using a cookies.txt file Jabba Laci <jabba.laci@gmail.com> - 2011-09-05 21:36 -0400

csiph-web