Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!goblin2!goblin.stu.neva.ru!newsfeed.xs4all.nl!newsfeed4a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.003 X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'favorable': 0.07; 'urllib2': 0.07; 'implements': 0.09; 'subject:while': 0.09; 'url:github': 0.09; 'posted': 0.15; '#python': 0.16; '(),': 0.16; 'blocking': 0.16; 'bug,': 0.16; 'concurrency,': 0.16; 'essentials': 0.16; 'imports': 0.16; 'subject: \n ': 0.16; 'subject:Thread': 0.16; 'subject:after': 0.16; 'subject:started': 0.16; 'twisted': 0.16; 'url:py': 0.16; 'looked': 0.18; 'bit': 0.19; 'module': 0.19; 'implementing': 0.19; 'meant': 0.20; 'import': 0.22; "haven't": 0.24; 'question': 0.24; 'point': 0.28; 'absolute': 0.30; 'message-id:@mail.gmail.com': 0.30; "i'm": 0.30; 'code': 0.31; 'that.': 0.31; 'url:wiki': 0.31; '(maybe': 0.31; 'bunch': 0.31; 'exceptions': 0.31; 'libraries': 0.31; 'this.': 0.32; 'subject:the': 0.34; 'subject:with': 0.35; "can't": 0.35; 'something': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'should': 0.36; 'too': 0.37; 'project': 0.37; 'implement': 0.38; 'handle': 0.38; 'needed': 0.38; 'to:addr:python-list': 0.38; 'little': 0.38; 'recent': 0.39; 'sure': 0.39; 'to:addr:python.org': 0.39; 'either': 0.39; 'even': 0.60; 'eventually': 0.60; 'back': 0.62; 'within': 0.65; 'due': 0.66; 'latest': 0.67; 'bare': 0.84; 'ratio': 0.91 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=BepErbJRY81rDKf0zokBAD7KliERl+u2d6tDFPMv7nw=; b=eW0kutYRkrAnsnrPST2AclByac0XS612LZWgHc6qQyBskWI/Ub9QvB+yxgbxS9pgVu ZxutBFLIoguWA4PllQhEBGet+kOAdHegKLARVfVi0T9oH32rtJymQn05OMDVQT5myYUx pLbTr6J0WOSunbicTvQ0f/vvzJ5RxXBW84TXLj2opgfmTsvBUOHPW0zyehoJNcghKQSz 2DamQcRvpoJL9CqgLwDrJwdp9Ev/Wldi3UwmD/vRql9DLNLa5wPFmG0VfVFhKLJXwksM r7q3tBd6S5CXVPFHqEjJAKX0If2tDwCh90tZ3kv+TFTh7lyo30i6mEi76e4q/s9Ur3nj /b0w== MIME-Version: 1.0 X-Received: by 10.224.172.133 with SMTP id l5mr22549734qaz.25.1394976285689; Sun, 16 Mar 2014 06:24:45 -0700 (PDT) Date: Sun, 16 Mar 2014 09:24:45 -0400 Subject: Thread is somehow interfering with a while loop called after the thread is started From: Dan McInerney To: python-list@python.org Content-Type: multipart/alternative; boundary=001a11c22f06fa8b3e04f4b939a3 X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 57 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1394976294 news.xs4all.nl 2944 [2001:888:2000:d::a6]:38666 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:68394 --001a11c22f06fa8b3e04f4b939a3 Content-Type: text/plain; charset=ISO-8859-1 Coming back to this a long time later. I figured it out a little bit after I posted this. I wasn't aware that q.try_run() within the nfqueue module was a blocking call https://www.wzdftpd.net/redmine/projects/nfqueue-bindings/wiki/Examples. I'm not sure I was even aware of what it meant to be blocking vs nonblocking. Either way, someone from #python looked at my code and told me nfqueue is a blocking call and I should look into something like twisted. Twisted is a complex module but I spent a bunch of time looking at it and eventually had some working code implementing the absolute bare essentials as I needed them to go asynchronous. This question eventually progressed into https://github.com/DanMcInerney/LANs.py. Speaking of network concurrency, my recent project has been from a web scraper point of view and I think gevents+requests or gevents+mechanize are the way to go for that somewhat-related topic. erequests won't import due to a bug, grequests has no way to handle exceptions in the fetching of responses, treq can't find and fill out form logins (maybe with a POST, haven't explored that route), and scrapy is way too big and complex for my latest project (https://github.com/DanMcInerney/shodan_pharmer), eventlets I'm sure CAN work with libraries other than urllib2 but it implements and imports its own urllib2 and it is not a favorable time:effort ratio for me to try and implement that. --001a11c22f06fa8b3e04f4b939a3 Content-Type: text/html; charset=ISO-8859-1
Coming back to this a long time later. I figured it out a little bit after I posted this. I wasn't aware that q.try_run() within the nfqueue module was a blocking call https://www.wzdftpd.net/redmine/projects/nfqueue-bindings/wiki/Examples. I'm not sure I was even aware of what it meant to be blocking vs nonblocking. Either way, someone from #python looked at my code and told me nfqueue is a blocking call and I should look into something like twisted.

Twisted is a complex module but I spent a bunch of time looking at it and eventually had some working code implementing the absolute bare essentials as I needed them to go asynchronous. This question eventually progressed into https://github.com/DanMcInerney/LANs.py.

Speaking of network concurrency, my recent project has been from a web scraper point of view and I think gevents+requests or gevents+mechanize are the way to go for that somewhat-related topic. erequests won't import due to a bug, grequests has no way to handle exceptions in the fetching of responses, treq can't find and fill out form logins (maybe with a POST, haven't explored that route), and scrapy is way too big and complex for my latest project (https://github.com/DanMcInerney/shodan_pharmer), eventlets I'm sure CAN work with libraries other than urllib2 but it implements and imports its own urllib2 and it is not a favorable time:effort ratio for me to try and implement that.
--001a11c22f06fa8b3e04f4b939a3--