Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #3555 > unrolled thread

Looking for a urllib(2) cookie handler

Back to article view | Back to comp.lang.python

Contents

  Looking for a urllib(2) cookie handler Mark Carter <alt.mcarter@gmail.com> - 2011-04-19 04:44 -0700
    Re: Looking for a urllib(2) cookie handler Chris Rebert <clp2@rebertia.com> - 2011-04-19 05:01 -0700
    Re: Looking for a urllib(2) cookie handler Mark Carter <alt.mcarter@gmail.com> - 2011-04-19 05:48 -0700
      Re: Looking for a urllib(2) cookie handler Chris Rebert <clp2@rebertia.com> - 2011-04-19 05:57 -0700

#3555 — Looking for a urllib(2) cookie handler

I'm in python 2.6.5, and have Firefox 3.6.13. I would like to download
some html from a site and scrape it programatically. The site requires
a cookie, which I have in Firefox.

Is there a simple python recipe I can use to read the contents of a
url and say "just use the cookie that I have in Firefox"?

[toc] | [next] | [standalone]

#3556

On Tue, Apr 19, 2011 at 4:44 AM, Mark Carter <alt.mcarter@gmail.com> wrote:
> I'm in python 2.6.5, and have Firefox 3.6.13. I would like to download
> some html from a site and scrape it programatically. The site requires
> a cookie, which I have in Firefox.
>
> Is there a simple python recipe I can use to read the contents of a
> url and say "just use the cookie that I have in Firefox"?

Untested (3rd Google hit for "import firefox cookies python"):
http://blog.mithis.net/archives/python/90-firefox3-cookies-in-python

At a minimum, I think you'll need to replace:
from pysqlite2 import dbapi2 as sqlite
With:
import sqlite3 as sqlite

You'll also have to figure out where Firefox's `cookies.sqlite` file
is located on your system.

Cheers,
Chris
--
My compiler is compiling, I swear!
http://blog.rebertia.com

[toc] | [prev] | [next] | [standalone]

#3557

On Apr 19, 12:44 pm, Mark Carter <alt.mcar...@gmail.com> wrote:

> url and say "just use the cookie that I have in Firefox"?

"mechanize" looks kinda like what I want, but i still can't get it to
work properly. So far I have:

import cookielib
import mechanize

cookiefile = "C:\\Users\\$ME\\AppData\\Roaming\\Mozilla\\Firefox\
\Profiles\\zl648qvt.default\\cookies.sqlite"
cookies = mechanize.MozillaCookieJar(filename = cookiefile,
delayload=True)
#cookies = cookielib.MozillaCookieJar()
#cookies = cookielib.MSIECookieJar()
#cookies.load_from_registry()  # finds cookie index file from registry
br = mechanize.Browser()
br.set_cookiejar(cookies)
br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-
US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]
url = "$URL"
r = br.open(url)

#print cj
#opener =
mechanize.build_opener(mechanize.HTTPCookieProcessor(cookies))


html = r.read()
print html

where $ME and $URL are replaced with suitable values. It doesn't
appear to acutally be "using" the cookies.

[toc] | [prev] | [next] | [standalone]

#3559

On Tue, Apr 19, 2011 at 5:48 AM, Mark Carter <alt.mcarter@gmail.com> wrote:
> On Apr 19, 12:44 pm, Mark Carter <alt.mcar...@gmail.com> wrote:
>
>> url and say "just use the cookie that I have in Firefox"?
>
> "mechanize" looks kinda like what I want, but i still can't get it to
> work properly. So far I have:
>
> import cookielib
> import mechanize
>
> cookiefile = "C:\\Users\\$ME\\AppData\\Roaming\\Mozilla\\Firefox\
> \Profiles\\zl648qvt.default\\cookies.sqlite"
> cookies = mechanize.MozillaCookieJar(filename = cookiefile,
> delayload=True)
<snip>
> where $ME and $URL are replaced with suitable values. It doesn't
> appear to acutally be "using" the cookies.

http://wwwsearch.sourceforge.net/mechanize/doc.html :
"Firefox since version 3 persists cookies in an sqlite database, which
is not supported by MozillaCookieJar."

Please see my prior response.

Cheers,
Chris
--
http://blog.rebertia.com

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web