Groups > comp.lang.java.programmer > #31928

Re: how to capture full webpage in java

Path	csiph.com!optima2.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!us.feeder.erje.net!newsfeed.fsmpi.rwth-aachen.de!newsfeed.straub-nv.de!eternal-september.org!feeder.eternal-september.org!mx02.eternal-september.org!.POSTED!not-for-mail
From	Eternal Hope <laughingspam99@gmail.com>
Newsgroups	comp.lang.java.programmer
Subject	Re: how to capture full webpage in java
Date	Thu, 02 Jul 2015 20:50:52 +0100
Organization	A noiseless patient Spider
Lines	54
Message-ID	<mn44k2$o4i$2@dont-email.me> (permalink)
References	<b93849bb-23f7-4867-bd3e-81ba9157b5c6@googlegroups.com> <mn44cq$o4i$1@dont-email.me>
Mime-Version	1.0
Content-Type	text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding	7bit
Injection-Date	Thu, 2 Jul 2015 19:49:22 +0000 (UTC)
Injection-Info	mx02.eternal-september.org; posting-host="318ed0177f074561c4d3d4fc4dfe3634"; logging-data="24722"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/Di6jATDc3wyJ81dr5NFx47/cpaOwv01I="
User-Agent	Mozilla/5.0 (Windows NT 6.3; rv:31.0) Gecko/20100101 Thunderbird/31.7.0
In-Reply-To	<mn44cq$o4i$1@dont-email.me>
Cancel-Lock	sha1:41DIWCWEqnFKCiyTpNOCDqPaNOk=
Xref	csiph.com comp.lang.java.programmer:31928

Show key headers only | View raw

On 02/07/2015 20:46, Eternal Hope wrote:
> On 30/06/2015 10:13, Navneet Mathpal wrote:
>> Hi ,
>> Have you tried to capture the full webpage using java ?
>> I want to capture full webpage using java (ex:
>> http://stackoverflow.com) . I am able to capture the first screen but
>> can not scoll down till the web-page ends ?
>>
>> I tried to do it with Selenium , but could not do it(some jar problem
>> occuring all the time )
>>
>>
>> could you please help. ?
>
> package usenet.foo;
>
> import java.io.BufferedReader;
> import java.io.InputStreamReader;
> import java.net.HttpURLConnection;
> import java.net.URL;
>
> public class PageReader {
>
>      public static void main(String[] args){
>          URL url = null;
>          BufferedReader reader = null;
>          StringBuilder stringBuilder;
>
>          try{
>
>            url = new URL("http://stackoverflow.com");
>            HttpURLConnection connection = (HttpURLConnection)
> url.openConnection();
>            connection.setRequestMethod("GET");
>            connection.setReadTimeout(15*1000);
>            connection.connect();
>            reader = new BufferedReader(new
> InputStreamReader(connection.getInputStream()));
>            stringBuilder = new StringBuilder();
>
>            String line = null;
>            while ((line = reader.readLine()) != null) {
>              stringBuilder.append(line + "\n");
>            }
>            //parse or whatever
>            System.out.println(stringBuilder.toString());
>          }
>          catch (Exception e){
>            e.printStackTrace();
>        }
>      }
> }

That took 2 minutes to find, 2 minutes to modify and 30 seconds to test :-)

Thread

how to capture full webpage in java Navneet Mathpal <navneetmathpal93@gmail.com> - 2015-06-30 02:13 -0700
  Re: how to capture full webpage in java e.d.programmer@gmail.com - 2015-06-30 05:34 -0700
  Re: how to capture full webpage in java Eternal Hope <laughingspam99@gmail.com> - 2015-07-02 20:46 +0100
    Re: how to capture full webpage in java Eternal Hope <laughingspam99@gmail.com> - 2015-07-02 20:50 +0100
      Re: how to capture full webpage in java e.d.programmer@gmail.com - 2015-07-02 12:58 -0700

csiph-web