Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!news.albasani.net!feeder.erje.net!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <anonhung@gmail.com>
X-Original-To: Python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: UNSURE 0.270
X-Spam-Level: **
X-Spam-Evidence: '*H*': 0.48; '*S*': 0.02; 'page?': 0.07; 'subject:Issue': 0.15; 'advance': 0.17; 'subject:Question': 0.19; 'checked': 0.21; 'thus': 0.21; 'trying': 0.21; 'header:In-Reply- To:1': 0.22; 'do,': 0.25; 'code': 0.26; "i'm": 0.28; 'message- id:@mail.gmail.com': 0.29; 'problem': 0.29; 'skip:b 20': 0.29; 'source': 0.32; 'ps:': 0.32; 'thanks': 0.32; 'there': 0.33; 'received:209.85.212': 0.33; 'running': 0.34; 'homepage.': 0.34; 'to:addr:python-list': 0.35; 'to:name:python-list': 0.37; 'received:google.com': 0.37; 'subject:with': 0.37; 'received:209.85': 0.38; 'data': 0.38; 'clearly': 0.39; 'subject:from': 0.39; 'data,': 0.39; 'received:209': 0.39; 'tool': 0.39; 'to:addr:python.org': 0.40; 'your': 0.61; 'collect': 0.61; 'below': 0.62; 'skip:w 30': 0.63; 'details': 0.64; 'prime': 0.67; 'webpage': 0.67; 'clicking': 0.80; 'hungary': 0.84; 'url:weebly': 0.84; 'homepage': 0.93; 'minister': 0.93; 'subject:Data': 0.93
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=aP99hVGysLFCcdl9DAZ/rVQVWs9sTCQdqtAJ+c7EdpE=; b=ESBBZnZLXpq12s2rfROywKp6DlIxSWRYT5AOzmSrf2TvgDbk5TPbWxaYGLYlHtwnZH 9T/uzQ76WTAvAJl9yR1tv0lgltIWm+3ba4UCqZy5a1xTyKer33gvn8MZ3QyRHw8JJSKj FSJTrxe4Puystc+fzdcO0V9IO8nPrnNW+z+/U=
MIME-Version: 1.0
In-Reply-To: <CA+VcYvtyU9KFCk3APKtS4s2cd0JQuqRnoG2Qz4XxQbZSC3n1Lg@mail.gmail.com>
References: <CA+VcYvtyU9KFCk3APKtS4s2cd0JQuqRnoG2Qz4XxQbZSC3n1Lg@mail.gmail.com>
Date: Thu, 9 Feb 2012 06:07:42 +0100
Subject: Re: Issue with Scrapping Data from a webpage- Noob Question
From: anon hung <anonhung@gmail.com>
To: Python-list <Python-list@python.org>
Content-Type: text/plain; charset=ISO-8859-1
X-Mailman-Approved-At: Thu, 09 Feb 2012 12:36:36 +0100
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.5582.1328787397.27778.python-list@python.org>
Lines: 39
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1328787397 news.xs4all.nl 6939 [2001:888:2000:d::a6]:38532
X-Complaints-To: abuse@xs4all.nl
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:20068

> Hi Fellow Pythoners,
>
> I'm trying to collect table data from an authenticated webpage (Tool) to
> which I have access.
>
> I will have the required data after 'click'ing a submit button on the tool
> homepage.
> When I inspect the submit button i see
> <form action="/Tool/index.do" method="POST">
>
> Thus the tool's homepage is of the form www.example.com/Tool and on
> clicking the submit button the data I need will be at
> www.example.com/Tool/index.do
>
> The problem that I'm running into is in my below code is giving me the
> source of homepage(www.example.com/Tool) and not the of the submitted page(
> www.example.com/Tool/index.do)
>
> url="www.example.com/Tool/index.do"
> request = urllib2.Request(url, data, {'Authorization': "Basic " +
> base64.b64encode("%s:%s" % (username, password))})
> Response_Page=urllib2.urlopen(request).read()
>
> Is there a way I can access the source of the submitted page?
>
> PS: Sorry for laying out very tiny details on what I'm trying to do, I just
> wanted to explain myself clearly :)
>
> Thanks in advance for your time on this one.

Have you checked beautifulsoup?

Best,
anonhung


-- 
Viktor Orban Prime minister of Hungary
http://spreadingviktororban.weebly.com