Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!news.dougwise.org!gegeweb.org!de-l.enfer-du-nord.net!feeder1.enfer-du-nord.net!feeder.news-service.com!xlned.com!feeder1.xlned.com!zen.net.uk!hamilton.zen.co.uk!shaftesbury.zen.co.uk.POSTED!not-for-mail Date: Tue, 05 Apr 2011 17:15:40 +0100 From: RedGrittyBrick User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.2.15) Gecko/20110303 Thunderbird/3.1.9 MIME-Version: 1.0 Newsgroups: comp.lang.java.programmer Subject: Re: JavaScript and Screenscraping References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Lines: 23 Message-ID: <4d9b400b$0$2525$da0feed9@news.zen.co.uk> Organization: Zen Internet NNTP-Posting-Host: 412157f0.news.zen.co.uk X-Trace: DXC=CgKO7QSFNUj0G3WFm3::nbnok4Z\3cB^f=jA8gPDHTT7?fOeVS7NWc0i` X-Complaints-To: abuse@zen.co.uk Xref: x330-a1.tempe.blueboxinc.net comp.lang.java.programmer:2887 On 30/03/2011 14:51, Roedy Green wrote: > I wondered if there exists a tool that is like browser in that it will > read a page and render the JavaScript, but unlike a browser, it would > not show the information on the screen, just dump the generated HTML > or raw text and accept a script of pages to analyse. > http://links.twibright.com/features.php: "Links runs in text mode (mouse optional) on UN*X console, ssh/telnet virtual terminal, vt100 terminal, xterm, and virtually any other text terminal. " Links2 supports Javascript. I haven't used it but it seems to have command line options, maybe, like Lynx, some of them allow you to save the HTML to a file? Open Source, so if the GPL is usable for your project, you can probably repurpose it. -- RGB