Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #53109

Re: How to keep cookies when making http requests (Python 2.7)

Show key headers only | View raw

Luca Cerone <luca.cerone@gmail.com> writes:

> ...
> Ok so after reading the documentation for urllib2 and cookielib I came up with the following code:
>
> #START
> from urllib2 import urlopen , Request
> from cookielib import CookieJar
> import re
> regex = re.compile(r'<span class=\'x\'>\{(.*)\}<span class=\'x\'>')
>
> base_url = "http://quiz.gambitresearch.com"
> job_url  = base_url + "/job/"
>
> cookies = CookieJar()
> r = Request(base_url) #prepare the request object
> cookies.add_cookie_header(r) #allow to have cookies
> R = urlopen(r) #read the url
> cookies.extract_cookies(R,r) #take the cookies from the response R and adds #them to the request object 

"adds them to the request object" should be "adds them to the cookie jar".

> #build the new url
> t = R.read()
> v = str(eval(regex.findall(t)[0]))
> job_url = job_url + v
>
>
> # Here I create a new request to the url containing the email address
> r2 = Request(job_url)
> cookies.add_cookie_header(r2) #I prepare the request for cookies adding the cookies that I extracted before.
>
> #perform the request and print the page
> R2 = urlopen(r2)
> t2 = R2.read()
> print job_url
> print t2
> #END
>
> This still doesn't work, but I really can't understand why.
> As far as I have understood first I have to instantiate a Request object
> and allow it to receive and set cookies (I do this with r = Request() and cookies.add_cookie_header(r))
> Next I perform the request (urlopen),  save the cookies in the CookieJar (cookies.extract_cookies(R,r)).
>
> I evaluate the new address and I create a new Request for it (r2 = Request)
> I add the cookies stored in the cookiejar in my new request (cookies.add_cookie_header(r2))
> Then I perform the request (R2 = urlopen(r2)) and read the page (t2 = R2.read())
>
> What am I doing wrong?

With respect to cookie handling, you do everything right.

There may be other problems with the (wider) process.
Analysing the responses of your requests (reading the status codes,
the response headers and the response bodies) may provide hints
towards the problem.

>Do I misunderstand something in the process?

Not with respect to cookie handling.

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

How to keep cookies when making http requests (Python 2.7) Luca Cerone <luca.cerone@gmail.com> - 2013-08-20 15:24 -0700
  Re: How to keep cookies when making http requests (Python 2.7) dieter <dieter@handshake.de> - 2013-08-21 08:07 +0200
    Re: How to keep cookies when making http requests (Python 2.7) Luca Cerone <luca.cerone@gmail.com> - 2013-08-21 01:18 -0700
      Re: How to keep cookies when making http requests (Python 2.7) Fábio Santos <fabiosantosart@gmail.com> - 2013-08-21 10:15 +0100
      Re: How to keep cookies when making http requests (Python 2.7) dieter <dieter@handshake.de> - 2013-08-22 08:08 +0200
        Re: How to keep cookies when making http requests (Python 2.7) Luca Cerone <luca.cerone@gmail.com> - 2013-08-27 02:16 -0700
          Re: How to keep cookies when making http requests (Python 2.7) Luca Cerone <luca.cerone@gmail.com> - 2013-08-27 03:17 -0700
            Re: How to keep cookies when making http requests (Python 2.7) dieter <dieter@handshake.de> - 2013-08-28 06:52 +0200
              Re: How to keep cookies when making http requests (Python 2.7) Luca Cerone <luca.cerone@gmail.com> - 2013-08-30 05:01 -0700

csiph-web