Groups > comp.lang.javascript > #19602

Re: OT (Probably ??) -- on Parsing a complex web page

Path	csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.fsmpi.rwth-aachen.de!news-1.dfn.de!news.dfn.de!news.informatik.hu-berlin.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From	"Mel Smith" <med_cutout_syntel@aol.com>
Newsgroups	comp.lang.javascript
Subject	Re: OT (Probably ??) -- on Parsing a complex web page
Date	Thu, 2 May 2013 13:41:56 -0600
Lines	55
Message-ID	<aufts8F6qiU1@mid.individual.net> (permalink)
References	<aufgi5Frkj0U1@mid.individual.net> <kludct$m0v$4@dont-email.me>
X-Trace	individual.net GoIv7Wf9hNb2q+m6jCD5iQmcF2NqH7+UWo28m+oRyhx4QKxqhM
Cancel-Lock	sha1:ESWdh8L/dqq9lF1VSDb8o8Aqz1M=
X-Priority	3
X-MSMail-Priority	Normal
X-Newsreader	Microsoft Outlook Express 6.00.2900.5931
X-MimeOLE	Produced By Microsoft MimeOLE V6.00.2900.6157
X-RFC2646	Format=Flowed; Original
Xref	csiph.com comp.lang.javascript:19602

Show key headers only | View raw

Denis said:

>
> Assuming this is the sort of job that you can leave going overnight, what
> you could do is grab the page, load it into a dom parser, and then use dom
> manipulation to pull out the data you want.
>
> Quick & dirty (and untested) example in php, assuming $urlList is an
> array of url strings, and you have some way to tell if a td's id matches
> your requirements:
>
> <?php
> $doc = new DOMDocument();
> foreach ( $urlList as $url ) {
>  $doc->loadHTMLFile( $url );
>  $cells = $doc->getElementsByTagName( "td" );
>  for ( $i = 0; $i < $cells->length; $i++ ) {
>    $cellId = $cells->item( $i )->attributes->getNamedItem( "id" );
>    if ( <some condition involving $cellId, eg preg_match> ) {
>      $cellValue = $cells->item( $i )->textContent;
>      // do something with $cellValue here!
>    }
>  }
> }
>
> If you have a way of deriving the specific ids you want from each
> document url, then instead of looping through the cells and testing every
> id, you could use something like:
>
> <?php
>
> function lookupIds( $url ) {
>  return arrayOfIdsForUrl;
> }
>
> $doc = new DOMDocument();
> foreach ( $urlList as $url ) {
>  $doc->loadHTMLFile( $url );
>  $idsInThisDoc = lookupIds( $url );
>  foreach ( $idsInThisDoc as $nextId ) {
>    $cell = $doc->getElementById( $nextId );
>    $cellValue = $cell->textContent;
>    // do something with $cellValue here!
>  }
> }
>


Than ks to you Denis and Scott and Evertjan for the info provided.

Now I'll have to put my head down and stasrt 'scribbling' !

-Mel

Back to comp.lang.javascript | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

OT (Probably ??) -- on Parsing a complex web page "Mel Smith" <med_cutout_syntel@aol.com> - 2013-05-02 09:54 -0600
  Re: OT (Probably ??) -- on Parsing a complex web page "Evertjan." <exxjxw.hannivoort@inter.nl.net> - 2013-05-02 18:19 +0200
    Re: OT (Probably ??) -- on Parsing a complex web page "Mel Smith" <med_cutout_syntel@aol.com> - 2013-05-02 13:39 -0600
  Re: OT (Probably ??) -- on Parsing a complex web page Scott Sauyet <scott.sauyet@gmail.com> - 2013-05-02 10:41 -0700
  Re: OT (Probably ??) -- on Parsing a complex web page Denis McMahon <denismfmcmahon@gmail.com> - 2013-05-02 19:05 +0000
    Re: OT (Probably ??) -- on Parsing a complex web page "Mel Smith" <med_cutout_syntel@aol.com> - 2013-05-02 13:41 -0600

csiph-web