Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.fsmpi.rwth-aachen.de!news-1.dfn.de!news.dfn.de!news.informatik.hu-berlin.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: "Mel Smith" <med_cutout_syntel@aol.com>
Newsgroups: comp.lang.javascript
Subject: Re: OT (Probably ??) -- on Parsing a complex web page
Date: Thu, 2 May 2013 13:41:56 -0600
Lines: 55
Message-ID: <aufts8F6qiU1@mid.individual.net>
References: <aufgi5Frkj0U1@mid.individual.net> <kludct$m0v$4@dont-email.me>
X-Trace: individual.net GoIv7Wf9hNb2q+m6jCD5iQmcF2NqH7+UWo28m+oRyhx4QKxqhM
Cancel-Lock: sha1:ESWdh8L/dqq9lF1VSDb8o8Aqz1M=
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2900.5931
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157
X-RFC2646: Format=Flowed; Original
Xref: csiph.com comp.lang.javascript:19602

Denis said:

>
> Assuming this is the sort of job that you can leave going overnight, what
> you could do is grab the page, load it into a dom parser, and then use dom
> manipulation to pull out the data you want.
>
> Quick & dirty (and untested) example in php, assuming $urlList is an
> array of url strings, and you have some way to tell if a td's id matches
> your requirements:
>
> <?php
> $doc = new DOMDocument();
> foreach ( $urlList as $url ) {
>  $doc->loadHTMLFile( $url );
>  $cells = $doc->getElementsByTagName( "td" );
>  for ( $i = 0; $i < $cells->length; $i++ ) {
>    $cellId = $cells->item( $i )->attributes->getNamedItem( "id" );
>    if ( <some condition involving $cellId, eg preg_match> ) {
>      $cellValue = $cells->item( $i )->textContent;
>      // do something with $cellValue here!
>    }
>  }
> }
>
> If you have a way of deriving the specific ids you want from each
> document url, then instead of looping through the cells and testing every
> id, you could use something like:
>
> <?php
>
> function lookupIds( $url ) {
>  return arrayOfIdsForUrl;
> }
>
> $doc = new DOMDocument();
> foreach ( $urlList as $url ) {
>  $doc->loadHTMLFile( $url );
>  $idsInThisDoc = lookupIds( $url );
>  foreach ( $idsInThisDoc as $nextId ) {
>    $cell = $doc->getElementById( $nextId );
>    $cellValue = $cell->textContent;
>    // do something with $cellValue here!
>  }
> }
>


Than ks to you Denis and Scott and Evertjan for the info provided.

Now I'll have to put my head down and stasrt 'scribbling' !

-Mel