Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!aioe.org!feeder.news-service.com!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!newsgate.cistron.nl!newsgate.news.xs4all.nl!194.109.133.85.MISMATCH!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.017 X-Spam-Evidence: '*H*': 0.97; '*S*': 0.00; 'parser': 0.05; '(right': 0.09; 'pages.': 0.09; 'sun,': 0.09; 'pm,': 0.11; 'this:': 0.11; 'def': 0.13; 'am,': 0.14; 'wrote:': 0.14; "'a'": 0.16; 'exemple': 0.16; 'from:addr:free.fr': 0.16; 'received:212.27': 0.16; 'received:212.27.42': 0.16; 'received:free.fr': 0.16; 'subject:Trying': 0.16; 'skip:m 30': 0.16; 'figure': 0.18; 'header :In-Reply-To:1': 0.22; 'trying': 0.23; 'extract': 0.25; 'script': 0.26; "i'm": 0.26; 'david': 0.27; "doesn't": 0.28; 'class': 0.29; 'this.': 0.30; 'edited': 0.31; 'html.': 0.31; 'import': 0.32; 'page.': 0.32; 'to:addr:python-list': 0.32; 'expression': 0.33; 'relatively': 0.33; 'created': 0.33; 'page': 0.33; 'test': 0.33; "isn't": 0.34; 'received:192': 0.34; 'skip:" 10': 0.34; 'regular': 0.34; 'got': 0.34; 'received:192.168.0': 0.35; 'print': 0.35; 'header:User-Agent:1': 0.35; 'closely': 0.35; 'fixing': 0.35; 'source,': 0.35; 'hello,': 0.36; 'none': 0.36; 'received:192.168': 0.37; 'andrew': 0.38; 'strings': 0.38; 'but': 0.38; 'returning': 0.39; 'to:addr:python.org': 0.39; 'solution': 0.40; "it's": 0.40; 'simple': 0.60; 'url:nl': 0.60; 'skip:h 20': 0.60; '2011': 0.62; 'link': 0.62; 'below': 0.63; 'links:': 0.68; 'following.': 0.84; 'tag,': 0.84; 'ultimately,': 0.84; 'url:dir': 0.84; 'page;': 0.91 Date: Mon, 16 May 2011 09:26:26 +0200 From: Karim User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110424 Thunderbird/3.1.10 MIME-Version: 1.0 To: python-list@python.org Subject: Re: Trying to understand html.parser.HTMLParser References: <4DD03B69.6050301@gmail.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 40 NNTP-Posting-Host: 82.94.164.166 X-Trace: 1305530799 news.xs4all.nl 41113 [::ffff:82.94.164.166]:49358 X-Complaints-To: abuse@xs4all.nl Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:5494 On 05/16/2011 03:06 AM, David Robinow wrote: > On Sun, May 15, 2011 at 4:45 PM, Andrew Berg wrote: >> I'm trying to understand why HMTLParser.feed() isn't returning the whole >> page. My test script is this: >> >> import urllib.request >> import html.parser >> class MyHTMLParser(html.parser.HTMLParser): >> def handle_starttag(self, tag, attrs): >> if tag == 'a' and attrs: >> print(tag,'-',attrs) >> >> url = 'http://x264.nl/x264/?dir=./64bit/8bit_depth' >> page = urllib.request.urlopen(url).read() >> parser = MyHTMLParser() >> parser.feed(str(page)) >> >> I can do print(page) and get the entire HTML source, but >> parser.feed(str(page)) only spits out the information for the top links >> and none of the "revisionxxxx" links. Ultimately, I just want to find >> the name of the first "revisionxxxx" link (right now it's >> "revision1995", when a new build is uploaded it will be "revision2000" >> or whatever). I figure this is a relatively simple page; once I >> understand all of this, I can move on to more complicated pages. > You've got bad HTML. Look closely and you'll see the there's no space > between the "revisionxxxx" strings and the style tag following. > The parser doesn't like this. I don't know a solution other than > fixing the html. > (I created a local copy, edited it and it worked.) Hello, Use regular expression for bad HTLM or beautifulSoup (google it), below a exemple to extract all html links: linksList = re.findall('.*?',htmlSource) for link in linksList: print link Cheers Karim