Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!gegeweb.42!gegeweb.eu!nntpfeed.proxad.net!proxad.net!feeder2-2.proxad.net!nx01.iad01.newshosting.com!newshosting.com!69.16.185.11.MISMATCH!npeer01.iad.highwinds-media.com!news.highwinds-media.com!feed-me.highwinds-media.com!post01.iad.highwinds-media.com!newsfe13.iad.POSTED!83aa503d!not-for-mail
From: Daniel Pitts <newsgroup.nospam@virtualinfinity.net>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:8.0) Gecko/20111105 Thunderbird/8.0
MIME-Version: 1.0
Newsgroups: comp.lang.java.help
Subject: Re: Failure of Java HTTP, some puzzles for those with not enough to do
References: <f7qad7h1a48tcaunk3u44jn8783hnutt2b@4ax.com>
In-Reply-To: <f7qad7h1a48tcaunk3u44jn8783hnutt2b@4ax.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Lines: 41
Message-ID: <Z4eBq.279$LO2.151@newsfe13.iad>
X-Complaints-To: abuse@newsrazor.net
NNTP-Posting-Date: Tue, 29 Nov 2011 23:41:45 UTC
Date: Tue, 29 Nov 2011 15:41:44 -0800
Xref: x330-a1.tempe.blueboxinc.net comp.lang.java.help:1407

On 11/29/11 3:28 PM, Roedy Green wrote:
> Here is a list of URLs that a browser can read but that Java's HTTP
> get/head cannot.
>
> http://ask.com/
> http://cgi.omroep.nl/cgi-bin/streams?/rnw/smac/2004/amsterdam_forum__chomsky_on_iraq_and_war_on_terror_20051216_low.rm
> http://www.akademika.no/
> http://www.amnesty.org/en/library/asset/AMR51/145/2004/en/b6ab0f58-d570-11dd-bb24-1fb85fe8fa05/amr511452004en.html
> http://www.bechtel.com/
> http://www.downloadplex.com/Submit-Software.html
> http://www.glish.com/css/7.asp
> http://www.microsoft.com/windows/windowsmedia/download/AllDownloads.aspx
> http://www.os2site.com/sw/internet/time/clock2.htm
> http://www.qantas.com.au/travel/airlines/home/au/en
> http://www.telegraph.co.uk/news/yourview/1562772/David-Cameron-answers-your-questions.html
> http://www.thefreedictionary.com/
> https://player.helixcommunity.org/2008/help/playerfaq.html
> https://tsa.aloaha.com/
> https://wush.net/svn/mindprod
> https://www.eecs.harvard.edu/mailman/listinfo/jopt-users
>
> Harvard is because they use their own SSL cert
> aloaha is a timestamp service
>
> Sometimes it is because they give some sort of failed response code
> but deliver the page anyway.
>
> Sometimes I sort out what screwy thing they are doing (e.g. circular
> redirects) and write, but they almost never correct the problem.
>
> In the days of the telephone, if you attached some device that was not
> behaving according to spec the telephone company would demand you
> disconnect it, and when you did they would reconnect service.  There
> seems to be no equivalent in the Internet world.

Have you tried spoofing a different User Agent value?

As far as SSL certs, it is possible to set up Java to accept specific certs.

Are you using the standard JDK URL classes, or have you tried using 
apache httpcore/httpclient?