Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.albasani.net!news.stack.nl!newsfeed.xs4all.nl!newsfeed4a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.000 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'else:': 0.03; 'elif': 0.05; 'handler': 0.05; 'output': 0.05; 'filename:fname piece:py': 0.07; 'python3': 0.07; 'reason,': 0.07; 'chunk': 0.09; 'filename': 0.09; 'path)': 0.09; 'resp': 0.09; 'try:': 0.09; 'url:8001': 0.09; 'subject:How': 0.10; 'def': 0.12; 'wrote': 0.14; "%s'": 0.16; "'%s'": 0.16; "'/'": 0.16; "'rb')": 0.16; 'blocking': 0.16; 'blocks': 0.16; 'chunk:': 0.16; 'dirname': 0.16; 'doing,': 0.16; 'fetch': 0.16; 'filename)': 0.16; 'fine.': 0.16; 'mimetypes': 0.16; 'name)': 0.16; 'proxy,': 0.16; 'runs,': 0.16; 'skip:~ 20': 0.16; 'somewhere.': 0.16; 'subject:non': 0.16; 'index': 0.16; 'subject: ?': 0.16; 'fix': 0.17; 'wrote:': 0.18; 'app': 0.19; 'seems': 0.21; 'feb': 0.22; 'import': 0.22; '(in': 0.22; 'proxy': 0.24; 'skip:l 30': 0.24; 'source': 0.25; 'script': 0.25; 'to:2**1': 0.27; 'instruction': 0.29; 'raise': 0.29; 'to:no real name:2**1': 0.29; "doesn't": 0.30; 'message-id:@mail.gmail.com': 0.30; 'url:mailman': 0.30; 'code': 0.31; 'requests': 0.31; 'serve': 0.31; "skip:' 10": 0.31; 'that.': 0.31; 'url:127': 0.31; 'class': 0.32; 'skip:m 30': 0.32; 'url:python': 0.33; 'skip:# 10': 0.33; 'url:non-standard http port': 0.33; 'trouble': 0.34; 'maybe': 0.34; 'display': 0.35; 'except': 0.35; 'problem.': 0.35; 'skip:s 30': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'yield': 0.36; 'url:listinfo': 0.36; 'charset:us-ascii': 0.36; 'similar': 0.36; 'url:org': 0.36; 'application': 0.37; 'skip:o 20': 0.38; 'sometimes': 0.38; 'server': 0.38; 'thank': 0.38; 'handle': 0.38; 'to:addr:python-list': 0.38; 'little': 0.38; 'skip:& 20': 0.39; 'skip:b 40': 0.39; 'to:addr:python.org': 0.39; 'skip:p 20': 0.39; 'url:mail': 0.40; 'read': 0.60; 'skip:l 50': 0.60; 'break': 0.61; 'skip:a 30': 0.61; 'new': 0.61; 'you.': 0.62; 'back': 0.62; 'name': 0.63; 'skip:n 10': 0.64; 'more': 0.64; 'url:0': 0.67; 'hang': 0.67; 'skip:r 30': 0.69; 'limit': 0.70; 'acts': 0.74; '2015': 0.84; 'hanging': 0.84; 'interrupt': 0.84; 'refuses': 0.84; 'route': 0.84; 'subject:skip:S 20': 0.91 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=Kr6qq9Jc5mbAjZvYfGtpyEm+enC/AyDtVJXU4P/PCR0=; b=vcdrAFx6jJw/bZ3v0Dw+aWr5jIQay7lrq9IJ7v+shVuYe0pgs4YqYLM1LR0UTPjatD dvllSr4CO37Xt84/XK6T2DNYCHMgVZc0je0WJGAiJWTF9Wu2Eyjz1IJTZYFAhWtzBAmO uFg1ro+5YcB+ZDtjzDTBwF1Pk0b90nQ0tcVi3EPvKDD/bqNwSMliQYm+pvvCGC0gEE2N NsAIQbKDL/NtCaHLy8zkRGYydyyrepi88NqLSdWb+TRdr6Eh11McTgl6JY+TaYk+TmQX 2ZM66FJncHigHBRdG8lltIgH2Q1tijHpqhPTjvkgJrG9WE7MKBA3iU2gofplrDnAIp9S EwSA== X-Received: by 10.112.235.67 with SMTP id uk3mr18695015lbc.48.1422871626104; Mon, 02 Feb 2015 02:07:06 -0800 (PST) MIME-Version: 1.0 From: Amirouche Boubekki Date: Mon, 02 Feb 2015 10:07:05 +0000 Subject: Re: How to write a non blocking SimpleHTTPRequestHandler ? To: yacinechaouche@yahoo.com.dmarc.invalid, python-list@python.org Content-Type: multipart/mixed; boundary=001a11c3135ed81942050e181db6 X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 215 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1422871634 news.xs4all.nl 2966 [2001:888:2000:d::a6]:55274 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:85070 --001a11c3135ed81942050e181db6 Content-Type: multipart/alternative; boundary=001a11c3135ed8193d050e181db4 --001a11c3135ed8193d050e181db4 Content-Type: text/plain; charset=UTF-8 On Mon Feb 02 2015 at 10:55:26 AM wrote: > I wrote a little script that acts like a proxy, you just give it a URL and > it will fetch the content and display it back to you. > > For some reason, this proxy blocks sometimes and refuses to serve any new > queries. The script still runs, but it seems like it's stuck somewhere. > > When I strace it to see what it's doing, I find it hanging on this > instruction : > root@backup[10.10.10.21] ~/SCRIPTS/INFOMANIAK # strace -fp 6918 > Process 6918 attached - interrupt to quit > recvfrom(6, > ^CProcess 6918 detached > root@backup[10.10.10.21] ~/SCRIPTS/INFOMANIAK # > > I read in the SimpleHTTPServer source code that one can inherit from the > SocketServer.TrheadingMixIn mixin to enable a threaded server to handle > multiple requests at a time instead of just one (thinking maybe that's what > was blocking it). However, it seems like it has nothing to do with my > problem. What I need to do is not only handle multiple requests at a time, > but more importantly to make the request handler non-blocking. > > Any ideas ? here's come code : > > import SimpleHTTPServer > import BaseHTTPServer > import SocketServer > import requests > > class Handler(SocketServer.ThreadingMixIn,SimpleHTTPServer.SimpleH > TTPRequestHandler): > def do_GET(self): > self.send_response(200) > self.send_header('Content-Type', 'text/html') > self.end_headers() > # self.path will contain a URL to be fetched by my proxy > self.wfile.write(getFlux(self.path.lstrip("/"))) > > session = requests.Session() > IP,PORT = "MY_IP_HERE",8080 > > def getFlux(url): > response = session.get(url) > s = response.text > return s > > server = BaseHTTPServer.HTTPServer((IP,PORT),Handler) > server.serve_forever() > Your code seem perfectly fine. I had some trouble with py3's http.server with IE10 (in a virtualbox...), I put together a small server script similar to http.server that doesn't hang up on microsoft. It works with ayncio. It's not ready to serve big files, but hopefully you can fix that. HTH > Thank you. > -- > https://mail.python.org/mailman/listinfo/python-list > --001a11c3135ed8193d050e181db4 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On Mon Feb 02 2015 at 10:55:26 AM <yacinechao= uche@yahoo.com.dmarc.invalid> wrote:
I= wrote a little script that acts like a proxy, you just give it a URL and i= t will fetch the content and display it back to you.

For some reason, this proxy blocks sometimes and refuses to serve any new q= ueries. The script still runs, but it seems like it's stuck somewhere.<= br>
When I strace it to see what it's doing, I find it hanging on this inst= ruction :
root@backup[10.10.10.21] ~/SCRIPTS/INFOMANIAK # strace -fp 6918
Process 6918 attached - interrupt to quit
recvfrom(6,
^CProcess 6918 detached
root@backup[10.10.10.21] ~/SCRIPTS/INFOMANIAK #

I read in the SimpleHTTPServer source code that one can inherit from the So= cketServer.TrheadingMixIn mixin to enable a threaded server to handle multi= ple requests at a time instead of just one (thinking maybe that's what = was blocking it). However, it seems like it has nothing to do with my probl= em. What I need to do is not only handle multiple requests at a time, but m= ore importantly to make the request handler non-blocking.

Any ideas ? here's come code :

import SimpleHTTPServer
import BaseHTTPServer
import SocketServer
import requests

class Handler(SocketServer.ThreadingMixIn,SimpleHTTPSe= rver.SimpleHTTPRequestHandler):
=C2=A0 =C2=A0 def do_GET(self):
=C2=A0 =C2=A0 =C2=A0 =C2=A0 self.send_response(200)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 self.send_header('Content-Type', 'text/html')
=C2=A0 =C2=A0 =C2=A0 =C2=A0 self.end_headers()
=C2=A0 =C2=A0 =C2=A0 =C2=A0 # self.path will contain a URL to be fetched by= my proxy
=C2=A0 =C2=A0 =C2=A0 =C2=A0 self.wfile.write(getFlux(self.pat= h.lstrip("/")))

session =3D requests.Session()
IP,PORT =3D "MY_IP_HERE",8080

def getFlux(url):
=C2=A0 =C2=A0 response=C2=A0 =3D session.get(url)
=C2=A0 =C2=A0 s =3D response.text
=C2=A0 =C2=A0 return s

server =3D BaseHTTPServer.HTTPServer((IP,PORT),Handler)
server.serve_forever()


Y= our code seem perfectly fine. I had some trouble with py3's http.server= with IE10 (in a virtualbox...), I put together a small server script simil= ar to http.server that doesn't hang up on microsoft. It works with aync= io. It's not ready to serve big files, but hopefully you can fix that.<= br>


HTH

= =C2=A0
Thank you.
--
https://mail.python.org/mailman/listinfo/python-list<= /a>
--001a11c3135ed8193d050e181db4-- --001a11c3135ed81942050e181db6 Content-Type: text/x-python; charset=US-ASCII; name="server.py" Content-Disposition: attachment; filename="server.py" Content-Transfer-Encoding: base64 X-Attachment-Id: 14b49c1a7bcf43aa094 IyEvdXNyL2Jpbi9lbnYgcHl0aG9uMwppbXBvcnQgb3MKaW1wb3J0IG1pbWV0eXBlcwpmcm9tIGFz eW5jaW8gaW1wb3J0IGNvcm91dGluZQpmcm9tIGFzeW5jaW8gaW1wb3J0IGdldF9ldmVudF9sb29w Cgpmcm9tIGFpb2h0dHAud2ViIGltcG9ydCBBcHBsaWNhdGlvbgpmcm9tIGFpb2h0dHAud2ViIGlt cG9ydCBTdGF0aWNSb3V0ZQpmcm9tIGFpb2h0dHAud2ViIGltcG9ydCBIVFRQTm90Rm91bmQKZnJv bSBhaW9odHRwLndlYiBpbXBvcnQgSFRUUE1vdmVkUGVybWFuZW50bHkKZnJvbSBhaW9odHRwLndl YiBpbXBvcnQgU3RyZWFtUmVzcG9uc2UKCgpjbGFzcyBTdGF0aWNSb3V0ZVdpdGhJbmRleChTdGF0 aWNSb3V0ZSk6CgogICAgbGltaXQgPSA4MTkyCgogICAgQGNvcm91dGluZQogICAgZGVmIGhhbmRs ZShzZWxmLCByZXF1ZXN0KToKICAgICAgICByZXNwID0gU3RyZWFtUmVzcG9uc2UoKQogICAgICAg IGZpbGVuYW1lID0gcmVxdWVzdC5tYXRjaF9pbmZvWydmaWxlbmFtZSddCiAgICAgICAgZmlsZXBh dGggPSBvcy5wYXRoLmpvaW4oc2VsZi5fZGlyZWN0b3J5LCBmaWxlbmFtZSkKCiAgICAgICAgaWYg Jy4uJyBpbiBmaWxlbmFtZToKICAgICAgICAgICAgcHJpbnQoJ25vdCBmb3VuZCAlcycgJSBmaWxl cGF0aCkKICAgICAgICAgICAgcmFpc2UgSFRUUE5vdEZvdW5kKCkKICAgICAgICBpZiBub3Qgb3Mu cGF0aC5leGlzdHMoZmlsZXBhdGgpOgogICAgICAgICAgICBwcmludCgnbm90IGZvdW5kICVzJyAl IGZpbGVwYXRoKQogICAgICAgICAgICByYWlzZSBIVFRQTm90Rm91bmQoKQoKICAgICAgICBpZiBu b3Qgb3MucGF0aC5pc2ZpbGUoZmlsZXBhdGgpOgogICAgICAgICAgICBkaXJlY3RvcnkgPSBmaWxl cGF0aAogICAgICAgICAgICBmaWxlbmFtZSA9IE5vbmUKICAgICAgICAgICAgaWYgbm90IGZpbGVw YXRoLmVuZHN3aXRoKCcvJyk6CiAgICAgICAgICAgICAgICBwYXRoID0gZmlsZXBhdGggKyAnLycK ICAgICAgICAgICAgICAgIHBhdGggPSBwYXRoW2xlbihzZWxmLl9kaXJlY3RvcnkpOl0KICAgICAg ICAgICAgICAgIHJhaXNlIEhUVFBNb3ZlZFBlcm1hbmVudGx5KHBhdGgpCiAgICAgICAgICAgIGZv ciBpbmRleCBpbiAoJ2luZGV4Lmh0bWwnLCAnaW5kZXguaHRtJyk6CiAgICAgICAgICAgICAgICBw YXRoID0gb3MucGF0aC5qb2luKGRpcmVjdG9yeSwgaW5kZXgpCiAgICAgICAgICAgICAgICBpZiBv cy5wYXRoLmV4aXN0cyhwYXRoKToKICAgICAgICAgICAgICAgICAgICBmaWxlbmFtZSA9IGluZGV4 CiAgICAgICAgICAgICAgICAgICAgZmlsZXBhdGggPSBwYXRoCiAgICAgICAgICAgICAgICAgICAg YnJlYWsKCiAgICAgICAgaWYgbm90IGZpbGVuYW1lIGFuZCBvcy5wYXRoLmlzZGlyKGZpbGVwYXRo KToKICAgICAgICAgICAgaWYgbm90IGZpbGVwYXRoLmVuZHN3aXRoKCcvJyk6CiAgICAgICAgICAg ICAgICBmaWxlcGF0aCArPSAnLycKICAgICAgICAgICAgbmFtZXMgPSBvcy5saXN0ZGlyKGZpbGVw YXRoKQogICAgICAgICAgICBuYW1lcy5zb3J0KCkKICAgICAgICAgICAgb3V0cHV0ID0gJzx1bD4n CiAgICAgICAgICAgIGZvciBuYW1lIGluIG5hbWVzOgogICAgICAgICAgICAgICAgZGlybmFtZSA9 IG9zLnBhdGguam9pbihmaWxlcGF0aCwgbmFtZSkKICAgICAgICAgICAgICAgIGlmIG9zLnBhdGgu aXNkaXIoZGlybmFtZSk6CiAgICAgICAgICAgICAgICAgICAgZGlybmFtZSArPSAnLycKICAgICAg ICAgICAgICAgIHBhdGggPSBkaXJuYW1lW2xlbihzZWxmLl9kaXJlY3RvcnkpOl0KICAgICAgICAg ICAgICAgIGxpbmsgPSAnPGEgaHJlZj0iJXMiPiVzPC9hPicgJSAocGF0aCwgbmFtZSkKICAgICAg ICAgICAgICAgIG91dHB1dCArPSAnPGxpPicgKyBsaW5rICsgJzwvbGk+JwogICAgICAgICAgICBv dXRwdXQgKz0gJzwvdWw+JwogICAgICAgICAgICByZXNwLmNvbnRlbnRfdHlwZSA9ICd0ZXh0L2h0 bWwnCiAgICAgICAgICAgIHJlc3Auc3RhcnQocmVxdWVzdCkKICAgICAgICAgICAgcmVzcC53cml0 ZShvdXRwdXQuZW5jb2RlKCd1dGYtOCcpKQogICAgICAgICAgICByZXR1cm4gcmVzcAogICAgICAg IGVsaWYgbm90IGZpbGVuYW1lOgogICAgICAgICAgICBwcmludCgnbm90IGZvdW5kICVzJyAlIGZp bGVwYXRoKQogICAgICAgICAgICByYWlzZSBIVFRQTm90Rm91bmQoKQogICAgICAgIGVsc2U6CiAg ICAgICAgICAgIGN0ID0gbWltZXR5cGVzLmd1ZXNzX3R5cGUoZmlsZW5hbWUpWzBdCiAgICAgICAg ICAgIGlmIG5vdCBjdDoKICAgICAgICAgICAgICAgIGN0ID0gJ2FwcGxpY2F0aW9uL29jdGV0LXN0 cmVhbScKICAgICAgICAgICAgcmVzcC5jb250ZW50X3R5cGUgPSBjdAoKICAgICAgICAgICAgZmls ZV9zaXplID0gb3Muc3RhdChmaWxlcGF0aCkuc3Rfc2l6ZQogICAgICAgICAgICBzaW5nbGVfY2h1 bmsgPSBmaWxlX3NpemUgPCBzZWxmLmxpbWl0CgogICAgICAgICAgICBpZiBzaW5nbGVfY2h1bms6 CiAgICAgICAgICAgICAgICByZXNwLmNvbnRlbnRfbGVuZ3RoID0gZmlsZV9zaXplCiAgICAgICAg ICAgIHJlc3Auc3RhcnQocmVxdWVzdCkKCiAgICAgICAgICAgIHdpdGggb3BlbihmaWxlcGF0aCwg J3JiJykgYXMgZjoKICAgICAgICAgICAgICAgIGNodW5rID0gZi5yZWFkKHNlbGYubGltaXQpCiAg ICAgICAgICAgICAgICBpZiBzaW5nbGVfY2h1bms6CiAgICAgICAgICAgICAgICAgICAgcmVzcC53 cml0ZShjaHVuaykKICAgICAgICAgICAgICAgIGVsc2U6CiAgICAgICAgICAgICAgICAgICAgd2hp bGUgY2h1bms6CiAgICAgICAgICAgICAgICAgICAgICAgIHJlc3Aud3JpdGUoY2h1bmspCiAgICAg ICAgICAgICAgICAgICAgICAgIGNodW5rID0gZi5yZWFkKHNlbGYubGltaXQpCiAgICAgICAgICAg IHByaW50KCdvayAlcycgJSBmaWxlcGF0aCkKICAgICAgICAgICAgcmV0dXJuIHJlc3AKCgoKCkBj b3JvdXRpbmUKZGVmIGluaXQobG9vcCk6CiAgICBwYXRoID0gb3MucGF0aC5hYnNwYXRoKG9zLmN1 cmRpcikKICAgIGFwcCA9IEFwcGxpY2F0aW9uKGxvb3A9bG9vcCkKICAgIHJvdXRlID0gU3RhdGlj Um91dGVXaXRoSW5kZXgoTm9uZSwgJy8nLCBwYXRoKQogICAgYXBwLnJvdXRlci5fcmVnaXN0ZXJf ZW5kcG9pbnQocm91dGUpCiAgICBoYW5kbGVyID0gYXBwLm1ha2VfaGFuZGxlcigpCiAgICBzcnYg PSB5aWVsZCBmcm9tIGxvb3AuY3JlYXRlX3NlcnZlcihoYW5kbGVyLCAnMC4wLjAuMCcsIDgwMDEp CiAgICByZXR1cm4gc3J2LCBoYW5kbGVyCgoKbG9vcCA9IGdldF9ldmVudF9sb29wKCkKc3J2LCBo YW5kbGVyID0gbG9vcC5ydW5fdW50aWxfY29tcGxldGUoaW5pdChsb29wKSkKcHJpbnQoIlNlcnZl ciBzdGFydGVkIGF0IGh0dHA6Ly8xMjcuMC4wLjE6ODAwMSIpCnRyeToKICAgIGxvb3AucnVuX2Zv cmV2ZXIoKQpleGNlcHQgS2V5Ym9hcmRJbnRlcnJ1cHQ6CiAgICBsb29wLnJ1bl91bnRpbF9jb21w bGV0ZShoYW5kbGVyLmZpbmlzaF9jb25uZWN0aW9ucygpKQo= --001a11c3135ed81942050e181db6--