Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #2589

Re: JavaScript and Screenscraping

Followups directed to: comp.lang.java.programmer

Show all headers | View raw

Roedy Green wrote:

> I am working on a screenscraping project that is turning out to much
> more time-consuming that I thought it would be. I am trying to gather
> a database of information about all the motherboards sold my major
> manufacturers.  The idea is to eventually create a comparison shopper
> to help you narrow down models that fit your needs.
> 
> Oddly motherboard manufacturers don't use a database and generate
> their specification pages. These are all hand-compiled with theme and
> a dozen variations on every field. This is can handle.
> 
> However, Asus decided to obfuscate their web pages with JavaScript.
> There are no data on them.
> 
> I wondered if there exists a tool that is like browser in that it will
> read a page and render the JavaScript, but unlike a browser, it would
> not show the information on the screen, just dump the generated HTML
> or raw text and accept a script of pages to analyse.
> 

http://htmlunit.sourceforge.net/

-- 
Michal

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

JavaScript and Screenscraping Roedy Green <see_website@mindprod.com.invalid> - 2011-03-30 06:51 -0700
  Re: JavaScript and Screenscraping Michal Kleczek <kleku75@gmail.com> - 2011-03-30 16:27 +0200
    Re: JavaScript and Screenscraping Tom Anderson <twic@urchin.earth.li> - 2011-03-31 00:28 +0100
  Re: JavaScript and Screenscraping Peter Duniho <NpOeStPeAdM@NnOwSlPiAnMk.com> - 2011-03-30 07:40 -0700
    Re: JavaScript and Screenscraping Roedy Green <see_website@mindprod.com.invalid> - 2011-03-30 18:27 -0700
  Re: JavaScript and Screenscraping Dr J R Stockton <reply1113@merlyn.demon.co.uk> - 2011-04-01 23:39 +0100
    Re: JavaScript and Screenscraping Roedy Green <see_website@mindprod.com.invalid> - 2011-04-01 20:00 -0700
      Re: JavaScript and Screenscraping Dr J R Stockton <reply1113@merlyn.demon.co.uk> - 2011-04-03 17:27 +0100
  Re: JavaScript and Screenscraping RedGrittyBrick <RedGrittyBrick@spamweary.invalid> - 2011-04-05 17:15 +0100

csiph-web