Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #20068

Re: Issue with Scrapping Data from a webpage- Noob Question

Show all headers | View raw

> Hi Fellow Pythoners,
>
> I'm trying to collect table data from an authenticated webpage (Tool) to
> which I have access.
>
> I will have the required data after 'click'ing a submit button on the tool
> homepage.
> When I inspect the submit button i see
> <form action="/Tool/index.do" method="POST">
>
> Thus the tool's homepage is of the form www.example.com/Tool and on
> clicking the submit button the data I need will be at
> www.example.com/Tool/index.do
>
> The problem that I'm running into is in my below code is giving me the
> source of homepage(www.example.com/Tool) and not the of the submitted page(
> www.example.com/Tool/index.do)
>
> url="www.example.com/Tool/index.do"
> request = urllib2.Request(url, data, {'Authorization': "Basic " +
> base64.b64encode("%s:%s" % (username, password))})
> Response_Page=urllib2.urlopen(request).read()
>
> Is there a way I can access the source of the submitted page?
>
> PS: Sorry for laying out very tiny details on what I'm trying to do, I just
> wanted to explain myself clearly :)
>
> Thanks in advance for your time on this one.

Have you checked beautifulsoup?

Best,
anonhung


-- 
Viktor Orban Prime minister of Hungary
http://spreadingviktororban.weebly.com

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: Issue with Scrapping Data from a webpage- Noob Question anon hung <anonhung@gmail.com> - 2012-02-09 06:07 +0100

csiph-web