Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #43482 > unrolled thread

SimpleHTTPRequestHandler used with HTTP/1.1 hangs after the second resource on a page.

Back to article view | Back to comp.lang.python

Contents

  SimpleHTTPRequestHandler used with HTTP/1.1 hangs after the second resource on a page. Piotr Dobrogost <p@google-groups-2013.dobrogost.net> - 2013-04-12 13:46 -0700
    Re: SimpleHTTPRequestHandler used with HTTP/1.1 hangs after the second resource on a page. Terry Jan Reedy <tjreedy@udel.edu> - 2013-04-12 18:21 -0400
      Re: SimpleHTTPRequestHandler used with HTTP/1.1 hangs after the second resource on a page. Piotr Dobrogost <p@google-groups-2013.dobrogost.net> - 2013-04-13 10:21 -0700
      Re: SimpleHTTPRequestHandler used with HTTP/1.1 hangs after the second resource on a page. Piotr Dobrogost <p@google-groups-2013.dobrogost.net> - 2013-04-13 10:21 -0700

#43482 — SimpleHTTPRequestHandler used with HTTP/1.1 hangs after the second resource on a page.

Hi!

I'd like to bring your attention to the question titled "Use HTTP/1.1 with SimpleHTTPRequestHandler" at http://stackoverflow.com/q/15839718/95735 which reads; "When I use HTTP/1.1 with SimpleHTTPRequestHandler, loading a page that pulls in other resources will hang after the second resource." and there's a tiny test showing the problem.
It's hard to believe that SimpleHTTPServer doesn't handle such a simple task, does it?
If I checked correctly code is stuck in handle_one_request() method at line 370 of server.py (http://hg.python.org/cpython/file/d9893d13c628/Lib/http/server.py#l370) but I can't see what's wrong.
Any ideas?


Best regards,
Piotr Dobrogost

[toc] | [next] | [standalone]

#43489

On 4/12/2013 4:46 PM, Piotr Dobrogost wrote:
> Hi!
>
> I'd like to bring your attention to the question titled "Use HTTP/1.1
> with SimpleHTTPRequestHandler" at
> http://stackoverflow.com/q/15839718/95735 which reads; "When I use

I find the doc slightly confusing. The SO code uses BaseHTTPServer. The 
doc says "Usually, this module isn’t used directly," On the other hand, 
SimpleHTTPServer only defines a request handler and not a server itself.

> HTTP/1.1 with SimpleHTTPRequestHandler, loading a page that pulls in
> other resources will hang after the second resource." and there's a

What does 'hang' mean? Does the page get displayed? If so, 
server.serve_forever is supposes to 'hang'.

> tiny test showing the problem. It's hard to believe that
> SimpleHTTPServer doesn't handle such a simple task, does it? If I
> checked correctly code is stuck in handle_one_request() method at
> line 370 of server.py
> (http://hg.python.org/cpython/file/d9893d13c628/Lib/http/server.py#l370)
> but I can't see what's wrong. Any ideas?

The SO post says

class MyRequestHandler(SimpleHTTPRequestHandler):
     #protocol_version = "HTTP/1.0"   # works
     protocol_version = "HTTP/1.1"   # hangs

so this should be some sort of clue. The code says

569 # The version of the HTTP protocol we support.
570 # Set this to HTTP/1.1 to enable automatic keepalive
571 protocol_version = "HTTP/1.0"

The only substantive code I found using protocol_version (in the CPython 
default branch you linked to) is

300 if version_number >= (1, 1) and self.protocol_version >= "HTTP/1.1":
301 self.close_connection = 0

331 elif (conntype.lower() == 'keep-alive' and
332 self.protocol_version >= "HTTP/1.1"):
333 self.close_connection = 0
334 # Examine the headers and look for an Expect directive
335 expect = self.headers.get('Expect', "")
336 if (expect.lower() == "100-continue" and
337 self.protocol_version >= "HTTP/1.1" and
338 self.request_version >= "HTTP/1.1"):
339 if not self.handle_expect_100():
340 return False

I would re-test with the recently released 3.3.1 (and/or 2.7.4) to make 
sure nothing has changed

[toc] | [prev] | [next] | [standalone]

#43529

On Saturday, April 13, 2013 12:21:33 AM UTC+2, Terry Jan Reedy wrote:
> I find the doc slightly confusing. The SO code uses BaseHTTPServer. The 
> doc says "Usually, this module isn’t used directly," On the other hand, 
> SimpleHTTPServer only defines a request handler and not a server itself.

Thanks for taking time to reply.

Well, I've seen presentations where BaseHTTPServer is a recommended way of running simple, ad-hoc web server. In addition the documentation of http.server module in Python 3.3 (which contains implementation of the same HTTPServer class as found in BaseHTTPServer module in Python 2.x) does not contain the statement you cited which I take to mean it can safely be used directly.

As Python 3.3.1 exhibits the same behavior let's focus on the following Python 3 version of the code from the original question:

from http.server import SimpleHTTPRequestHandler
from http.server import HTTPServer

class MyRequestHandler(SimpleHTTPRequestHandler):
    #protocol_version = "HTTP/1.0"   # works
    protocol_version = "HTTP/1.1"   # hangs

server = HTTPServer(("localhost", 7080), MyRequestHandler)
server.serve_forever()

> What does 'hang' mean? Does the page get displayed? If so, 
> server.serve_forever is supposes to 'hang'.

I increased the number of resources to 7 to be able to detect possible patterns. The page and some of the resources are being displayed almost instantly. However browser waits for the requests to get remaining resources to finish and so those resources naturally are not being displayed. When browser waits the code is stuck at the line I showed in my original post. What's interesting is that after some time the request to get the second resource (in case there are only 2 on the page) finishes. However the time spent waiting for the second resource is ridiculously long compared to time spend waiting for the first resource. For example in one of my tests Firebug showed waiting time for the first resource to be 4ms whereas the waiting time for the second resource to be 1m 55s.

> The SO post says
> 
> class MyRequestHandler(SimpleHTTPRequestHandler):
>      #protocol_version = "HTTP/1.0"   # works
>      protocol_version = "HTTP/1.1"   # hangs
> 
> so this should be some sort of clue. The code says
> 
> 569 # The version of the HTTP protocol we support.
> 570 # Set this to HTTP/1.1 to enable automatic keepalive
> 571 protocol_version = "HTTP/1.0"

Sure, in version 1.1 of http the connection is not closed by default unless 'Connection: Close' header was sent in request. Firefox sends all requests with 'Connection: keep-alive' header.

I don't really know how to debug this...

[toc] | [prev] | [next] | [standalone]

#43531

On Saturday, April 13, 2013 12:21:33 AM UTC+2, Terry Jan Reedy wrote:
> I find the doc slightly confusing. The SO code uses BaseHTTPServer. The 
> doc says "Usually, this module isn’t used directly," On the other hand, 
> SimpleHTTPServer only defines a request handler and not a server itself.

Thanks for taking time to reply.

Well, I've seen presentations where BaseHTTPServer is a recommended way of running simple, ad-hoc web server. In addition the documentation of http.server module in Python 3.3 (which contains implementation of the same HTTPServer class as found in BaseHTTPServer module in Python 2.x) does not contain the statement you cited which I take to mean it can safely be used directly.

As Python 3.3.1 exhibits the same behavior let's focus on the following Python 3 version of the code from the original question:

from http.server import SimpleHTTPRequestHandler
from http.server import HTTPServer

class MyRequestHandler(SimpleHTTPRequestHandler):
    #protocol_version = "HTTP/1.0"   # works
    protocol_version = "HTTP/1.1"   # hangs

server = HTTPServer(("localhost", 7080), MyRequestHandler)
server.serve_forever()

> What does 'hang' mean? Does the page get displayed? If so, 
> server.serve_forever is supposes to 'hang'.

I increased the number of resources to 7 to be able to detect possible patterns. The page and some of the resources are being displayed almost instantly. However browser waits for the requests to get remaining resources to finish and so those resources naturally are not being displayed. When browser waits the code is stuck at the line I showed in my original post. What's interesting is that after some time the request to get the second resource (in case there are only 2 on the page) finishes. However the time spent waiting for the second resource is ridiculously long compared to time spend waiting for the first resource. For example in one of my tests Firebug showed waiting time for the first resource to be 4ms whereas the waiting time for the second resource to be 1m 55s.

> The SO post says
> 
> class MyRequestHandler(SimpleHTTPRequestHandler):
>      #protocol_version = "HTTP/1.0"   # works
>      protocol_version = "HTTP/1.1"   # hangs
> 
> so this should be some sort of clue. The code says
> 
> 569 # The version of the HTTP protocol we support.
> 570 # Set this to HTTP/1.1 to enable automatic keepalive
> 571 protocol_version = "HTTP/1.0"

Sure, in version 1.1 of http the connection is not closed by default unless 'Connection: Close' header was sent in request. Firefox sends all requests with 'Connection: keep-alive' header.

I don't really know how to debug this...

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web