Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #53100

Re: Improving the web page download code.

Show all headers | View raw

On 27/08/2013 21:53, mukesh tiwari wrote:
> On Wednesday, 28 August 2013 01:49:59 UTC+5:30, MRAB  wrote:
>> On 27/08/2013 20:41, mukesh tiwari wrote:
>>
[snip]
 >> > if __name__== '__main__':
 >> > 	u = Downloader()
 >> > 	signal.signal( signal.SIGINT , u.handleexception)
 >> > 	thread.start_new_thread ( u.createurl , () )
 >> > 	for i in xrange ( 5 ) :
 >> > 		thread.start_new_thread ( u.downloadurl , () )
 >> > 	while True : pass
 >> > 			
 >> >
 >> My preferred method when working with background threads is to put a
 >> sentinel such as None at the end and then when a worker gets an item
 >> from the queue and sees that it's the sentinel, it puts it back in
 >> the queue for the other workers to see, and then returns
 >> (terminates). The main thread can then call each worker thread's
 >> .join method to wait for it to finish. You currently have the main
 >> thread running in a 'busy loop', consuming processing time doing
 >> nothing!
 >
 > Hi MRAB,
 > Thank you for the reply. I wrote this while loop only because of
 > there is no thread.join in thread[1] library but I got your point. I
 > am simply running a while loop for doing nothing. So if somehow I can
 > block the main without too much computation then it will great.
 >
Why don't you use the 'threading' module instead?


creator = threading.Thread(target=u.createurl)

workers = []
for i in xrange(5):
	workers.append(threading.Thread(target=u.downloadurl))

creator.start()

for w in workers:
	w.start()

creator.join()

for w in workers:
	w.join()

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Improving the web page download code. mukesh tiwari <mukeshtiwari.iiitm@gmail.com> - 2013-08-27 12:41 -0700
  Re: Improving the web page download code. MRAB <python@mrabarnett.plus.com> - 2013-08-27 21:19 +0100
    Re: Improving the web page download code. mukesh tiwari <mukeshtiwari.iiitm@gmail.com> - 2013-08-27 13:53 -0700
      Re: Improving the web page download code. MRAB <python@mrabarnett.plus.com> - 2013-08-27 23:33 +0100
        Re: Improving the web page download code. mukesh tiwari <mukeshtiwari.iiitm@gmail.com> - 2013-08-27 23:23 -0700
          Re: Improving the web page download code. MRAB <python@mrabarnett.plus.com> - 2013-08-28 16:12 +0100
  Re: Improving the web page download code. Alister <alister.ware@ntlworld.com> - 2013-08-28 08:58 +0000

csiph-web