Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.ruby > #7167

Re: HTTP response question.

Show key headers only | View raw

On 20.12.2015 09:58, Michael Uplawski wrote:

> In one program, I receive a Content-Length header and am able to resume
> partial downloads, in the second, the Content-Length header appears to
> always be *missing*.
> The program which is always successful is restricted to the retrieval of
> one type of data from two precise web-sites. Put another way: I do not
> know a lot about HTTP and learn much by trial and error (and some hints
> from Mr. Klemm on this newsgroup, of course). Alas, my trials with the
> second program are all futile, so far.
>
> Question: Is it me or is it the responding site? Can I do anything
> better to resume the previously interrupted download?

There is no guarantee that a server will send Content-Length header.  As 
simple as that.  You can check with curl:

HEAD request

$ url='http://www.fosshub.com/IrfanView.html/iview441_setup.exe'
$ curl --head -D headers.txt -o /dev/null "$url"
   % Total    % Received % Xferd  Average Speed   Time    Time     Time 
  Current
                                  Dload  Upload   Total   Spent    Left 
  Speed
   0     0    0     0    0     0      0      0 --:--:-- --:--:-- 
--:--:--     0
$ cat headers.txt
HTTP/1.1 200 OK
Date: Sun, 20 Dec 2015 10:57:56 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Set-Cookie: __cfduid=d5dc7775a3bc4742413be867fb87e18ae1450609075; 
expires=Mon, 19-Dec-16 10:57:55 GMT; path=/; domain=.fosshub.com; HttpOnly
Vary: Accept-Encoding
X-Powered-By: PHP/5.4.36
Age: 59104
Via: 1.1 varnish
X-Cache: HIT
X-Cache-Hits: 726
Server: cloudflare-nginx
CF-RAY: 257ad444c16b271a-FRA

GET request:

$ curl --get -D headers.txt -o /dev/null "$url"
   % Total    % Received % Xferd  Average Speed   Time    Time     Time 
  Current
                                  Dload  Upload   Total   Spent    Left 
  Speed
100 46352    0 46352    0     0   114k      0 --:--:-- --:--:-- --:--:-- 
  114k
$ cat headers.txt
HTTP/1.1 200 OK
Date: Sun, 20 Dec 2015 10:58:42 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: __cfduid=da523a3a1942bb307a1e7ed6f847ee36a1450609122; 
expires=Mon, 19-Dec-16 10:58:42 GMT; path=/; domain=.fosshub.com; HttpOnly
Vary: Accept-Encoding
X-Powered-By: PHP/5.4.36
Age: 59151
Via: 1.1 varnish
X-Cache: HIT
X-Cache-Hits: 727
Server: cloudflare-nginx
CF-RAY: 257ad5684e292684-FRA


Note, there is some line wrapping interfering but you can try out yourself.

Regarding your code: first thing you should probably do is to check the 
HTTP status code to avoid downloading a HTML error page for the binary 
that you are expecting.

You could also use Mechanize to take care of session and cookie handling.

Cheers

	robert


-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

Back to comp.lang.ruby | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

HTTP response question. Michael Uplawski <michael.uplawski@uplawski.eu> - 2015-12-20 09:58 +0100
  Re: HTTP response question. Robert Klemme <shortcutter@googlemail.com> - 2015-12-20 12:03 +0100
    Re: HTTP response question. Michael Uplawski <michael.uplawski@uplawski.eu> - 2015-12-24 08:46 +0100
  Re: HTTP response question. Michael Uplawski <michael.uplawski@uplawski.eu> - 2015-12-20 12:40 +0100
    Re: HTTP response question. Robert Klemme <shortcutter@googlemail.com> - 2015-12-20 14:12 +0100
      Re: HTTP response question. Michael Uplawski <michael.uplawski@uplawski.eu> - 2015-12-21 08:09 +0100
        Re: HTTP response question. Robert Klemme <shortcutter@googlemail.com> - 2015-12-22 11:55 +0100

csiph-web