Groups > comp.lang.java.programmer > #2588

JavaScript and Screenscraping

From	Roedy Green <see_website@mindprod.com.invalid>
Newsgroups	comp.lang.java.programmer
Subject	JavaScript and Screenscraping
Date	2011-03-30 06:51 -0700
Organization	Canadian Mind Products
Message-ID	<rvc6p6toumdlevjb48ohjnlf1gur128eqe@4ax.com> (permalink)

Show all headers | View raw

I am working on a screenscraping project that is turning out to much
more time-consuming that I thought it would be. I am trying to gather
a database of information about all the motherboards sold my major
manufacturers.  The idea is to eventually create a comparison shopper
to help you narrow down models that fit your needs.

Oddly motherboard manufacturers don't use a database and generate
their specification pages. These are all hand-compiled with theme and
a dozen variations on every field. This is can handle.

However, Asus decided to obfuscate their web pages with JavaScript.
There are no data on them. 

I wondered if there exists a tool that is like browser in that it will
read a page and render the JavaScript, but unlike a browser, it would
not show the information on the screen, just dump the generated HTML
or raw text and accept a script of pages to analyse.

-- 
Roedy Green Canadian Mind Products
http://mindprod.com
There are only two industries that refer to their customers as "users".
~ Edward Tufte

Back to comp.lang.java.programmer | Previous | Next — Next in thread | Find similar

Thread

JavaScript and Screenscraping Roedy Green <see_website@mindprod.com.invalid> - 2011-03-30 06:51 -0700
  Re: JavaScript and Screenscraping Michal Kleczek <kleku75@gmail.com> - 2011-03-30 16:27 +0200
    Re: JavaScript and Screenscraping Tom Anderson <twic@urchin.earth.li> - 2011-03-31 00:28 +0100
  Re: JavaScript and Screenscraping Peter Duniho <NpOeStPeAdM@NnOwSlPiAnMk.com> - 2011-03-30 07:40 -0700
    Re: JavaScript and Screenscraping Roedy Green <see_website@mindprod.com.invalid> - 2011-03-30 18:27 -0700
  Re: JavaScript and Screenscraping Dr J R Stockton <reply1113@merlyn.demon.co.uk> - 2011-04-01 23:39 +0100
    Re: JavaScript and Screenscraping Roedy Green <see_website@mindprod.com.invalid> - 2011-04-01 20:00 -0700
      Re: JavaScript and Screenscraping Dr J R Stockton <reply1113@merlyn.demon.co.uk> - 2011-04-03 17:27 +0100
  Re: JavaScript and Screenscraping RedGrittyBrick <RedGrittyBrick@spamweary.invalid> - 2011-04-05 17:15 +0100

csiph-web