Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.php > #7613 > unrolled thread

Scrape dynamically generated hyperlinks

Started bystudent4lifer <student4lifer@gmail.com>
First post2012-11-17 13:46 -0600
Last post2012-11-18 05:23 +0100
Articles 5 — 3 participants

Back to article view | Back to comp.lang.php


Contents

  Scrape dynamically generated hyperlinks student4lifer <student4lifer@gmail.com> - 2012-11-17 13:46 -0600
    Re: Scrape dynamically generated hyperlinks Jerry Stuckle <jstucklex@attglobal.net> - 2012-11-17 17:05 -0500
    RE: Scrape dynamically generated hyperlinks student4lifer <student4lifer@gmail.com> - 2012-11-17 18:03 -0600
      Re: Scrape dynamically generated hyperlinks Jerry Stuckle <jstucklex@attglobal.net> - 2012-11-17 19:36 -0500
    Re: Scrape dynamically generated hyperlinks Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2012-11-18 05:23 +0100

#7613 — Scrape dynamically generated hyperlinks

Fromstudent4lifer <student4lifer@gmail.com>
Date2012-11-17 13:46 -0600
SubjectScrape dynamically generated hyperlinks
Message-ID<Wumdnf0ve_vndDrNnZ2dnUVZ_r6dnZ2d@giganews.com>
Could someone let me know if it is possible to scrape dynamically generated hyperlinks like those from the following site:

https://fox.taleo.net/careersection/fox_external_career_section/jobsearch.ftl?


TIA.

[toc] | [next] | [standalone]


#7615

FromJerry Stuckle <jstucklex@attglobal.net>
Date2012-11-17 17:05 -0500
Message-ID<k891n6$l4s$1@dont-email.me>
In reply to#7613
On 11/17/2012 2:46 PM, student4lifer wrote:
> Could someone let me know if it is possible to scrape dynamically generated hyperlinks like those from the following site:
>
> https://fox.taleo.net/careersection/fox_external_career_section/jobsearch.ftl?
>
>
> TIA.
>
>
It's always possible - the browser does it all the time.  All you need 
to do is load the iframe(s), parse and execute the javascript and make 
the appropriate calls back to the server based on the parameters you wish.

Now if you want to ask "Is it practical?", that's another story.

-- 
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attglobal.net
==================

[toc] | [prev] | [next] | [standalone]


#7616

Fromstudent4lifer <student4lifer@gmail.com>
Date2012-11-17 18:03 -0600
Message-ID<P8-dnVoelYFQuDXNnZ2dnUVZ_rOdnZ2d@giganews.com>
In reply to#7613
Thanks for your reply, Jerry. I tried to isolate a single hyperlink (from the parsed page with all its javasript) but clicking on it brought back the whole 'generic' starting page with many other hyperlinks as well. Only then, clicking on any of the hyperlinks truly brought back the detail record. Is it possible to manipulate/isolate the parsed page so that clicking on the hyperlink only once would return the detail record? Thanks again.

[toc] | [prev] | [next] | [standalone]


#7617

FromJerry Stuckle <jstucklex@attglobal.net>
Date2012-11-17 19:36 -0500
Message-ID<k89ai5$bcr$1@dont-email.me>
In reply to#7616
On 11/17/2012 7:03 PM, student4lifer wrote:
> Thanks for your reply, Jerry. I tried to isolate a single hyperlink (from the parsed page with all its javasript) but clicking on it brought back the whole 'generic' starting page with many other hyperlinks as well. Only then, clicking on any of the hyperlinks truly brought back the detail record. Is it possible to manipulate/isolate the parsed page so that clicking on the hyperlink only once would return the detail record? Thanks again.
>
>

It doesn't look like that option is available on the first page.  But 
then I haven't gone through all the html and javascript, either.

-- 
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attglobal.net
==================

[toc] | [prev] | [next] | [standalone]


#7621

FromThomas 'PointedEars' Lahn <PointedEars@web.de>
Date2012-11-18 05:23 +0100
Message-ID<1695906.f0pe0gaTuB@PointedEars.de>
In reply to#7613
student4lifer wrote:

> Could someone let me know if it is possible to scrape dynamically
> generated hyperlinks like those from the following site:
> 
> 
https://fox.taleo.net/careersection/fox_external_career_section/jobsearch.ftl?

It is possible.  You need an ECMAScript-capable browser emulation written in 
PHP.  Good luck.


PointedEars
-- 
Danny Goodman's books are out of date and teach practices that are
positively harmful for cross-browser scripting.
  -- Richard Cornford, cljs, <cife6q$253$1$8300dec7@news.demon.co.uk> (2004)

[toc] | [prev] | [standalone]


Back to top | Article view | comp.lang.php


csiph-web