Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.javascript > #19602

Re: OT (Probably ??) -- on Parsing a complex web page

Show all headers | View raw

Denis said:

>
> Assuming this is the sort of job that you can leave going overnight, what
> you could do is grab the page, load it into a dom parser, and then use dom
> manipulation to pull out the data you want.
>
> Quick & dirty (and untested) example in php, assuming $urlList is an
> array of url strings, and you have some way to tell if a td's id matches
> your requirements:
>
> <?php
> $doc = new DOMDocument();
> foreach ( $urlList as $url ) {
>  $doc->loadHTMLFile( $url );
>  $cells = $doc->getElementsByTagName( "td" );
>  for ( $i = 0; $i < $cells->length; $i++ ) {
>    $cellId = $cells->item( $i )->attributes->getNamedItem( "id" );
>    if ( <some condition involving $cellId, eg preg_match> ) {
>      $cellValue = $cells->item( $i )->textContent;
>      // do something with $cellValue here!
>    }
>  }
> }
>
> If you have a way of deriving the specific ids you want from each
> document url, then instead of looping through the cells and testing every
> id, you could use something like:
>
> <?php
>
> function lookupIds( $url ) {
>  return arrayOfIdsForUrl;
> }
>
> $doc = new DOMDocument();
> foreach ( $urlList as $url ) {
>  $doc->loadHTMLFile( $url );
>  $idsInThisDoc = lookupIds( $url );
>  foreach ( $idsInThisDoc as $nextId ) {
>    $cell = $doc->getElementById( $nextId );
>    $cellValue = $cell->textContent;
>    // do something with $cellValue here!
>  }
> }
>


Than ks to you Denis and Scott and Evertjan for the info provided.

Now I'll have to put my head down and stasrt 'scribbling' !

-Mel

Back to comp.lang.javascript | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

OT (Probably ??) -- on Parsing a complex web page "Mel Smith" <med_cutout_syntel@aol.com> - 2013-05-02 09:54 -0600
  Re: OT (Probably ??) -- on Parsing a complex web page "Evertjan." <exxjxw.hannivoort@inter.nl.net> - 2013-05-02 18:19 +0200
    Re: OT (Probably ??) -- on Parsing a complex web page "Mel Smith" <med_cutout_syntel@aol.com> - 2013-05-02 13:39 -0600
  Re: OT (Probably ??) -- on Parsing a complex web page Scott Sauyet <scott.sauyet@gmail.com> - 2013-05-02 10:41 -0700
  Re: OT (Probably ??) -- on Parsing a complex web page Denis McMahon <denismfmcmahon@gmail.com> - 2013-05-02 19:05 +0000
    Re: OT (Probably ??) -- on Parsing a complex web page "Mel Smith" <med_cutout_syntel@aol.com> - 2013-05-02 13:41 -0600

csiph-web