Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.python > #68232
| From | Jim Gibson <jimsgibson@gmail.com> |
|---|---|
| Newsgroups | comp.lang.python |
| Subject | Re: querry on queue ( thread safe ) multithreading |
| Date | 2014-03-11 10:18 -0700 |
| Message-ID | <110320141018377289%jimsgibson@gmail.com> (permalink) |
| References | <de667c72-c1f7-449e-86cf-ce8c6818b87d@googlegroups.com> |
In article <de667c72-c1f7-449e-86cf-ce8c6818b87d@googlegroups.com>, Jaiprakash Singh <jaiprakash@wisepromo.com> wrote: > hey i am working on scraping a site , so i am using multi-threading concept. > i wrote a code based on queue (thread safe) but still my code block out after > sometime, please help , i have searched a lot but unable to resolve it. > please help i stuck here. Do you really want to subject the web server to 150 simultaneous requests? Some would consider that a denial-of-service attack. When I scrape a site, and I have been doing that occasionally of late, I put a 10-second sleep after each HTTP request. That makes my program more considerate of other people's resources and a better web citizen. It is also much easier to program. -- Jim Gibson
Back to comp.lang.python | Previous | Next — Previous in thread | Find similar | Unroll thread
querry on queue ( thread safe ) multithreading Jaiprakash Singh <jaiprakash@wisepromo.com> - 2014-03-10 22:52 -0700 Re: querry on queue ( thread safe ) multithreading Jim Gibson <jimsgibson@gmail.com> - 2014-03-11 10:18 -0700
csiph-web