Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.python > #12983
| Path | csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!weretis.net!feeder1.news.weretis.net!feeder.erje.net!news2.arglkargh.de!news.musoftware.de!wum.musoftware.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail |
|---|---|
| From | Gregory Ewing <greg.ewing@canterbury.ac.nz> |
| Newsgroups | comp.lang.python |
| Subject | Re: Processing a file using multithreads |
| Date | Fri, 09 Sep 2011 12:03:45 +1200 |
| Lines | 20 |
| Message-ID | <9ct3f4FuvnU1@mid.individual.net> (permalink) |
| References | <mailman.885.1315522214.27778.python-list@python.org> |
| Mime-Version | 1.0 |
| Content-Type | text/plain; charset=UTF-8; format=flowed |
| Content-Transfer-Encoding | 7bit |
| X-Trace | individual.net rzijKiUEfKOdc/TUJTrtwwld2MHRcet7xTtGLCTrAmty0NESFG |
| Cancel-Lock | sha1:Y+kNVad+0fnET310vT/+UOnLFN8= |
| User-Agent | Mozilla Thunderbird 1.0.5 (Macintosh/20050711) |
| X-Accept-Language | en-us, en |
| In-Reply-To | <mailman.885.1315522214.27778.python-list@python.org> |
| Xref | x330-a1.tempe.blueboxinc.net comp.lang.python:12983 |
Show key headers only | View raw
Abhishek Pratap wrote: > 3. Each file handle is processed in by an individual thread using the > same function ( so total 10 cores are assumed to be available on the > machine) Are you expecting the processing to be CPU bound or I/O bound? If it's I/O bound, multiple cores won't help you, and neither will threading, because it's the disk doing the work, not the CPU. If it's CPU bound, multiple threads in one Python process won't help, because of the GIL. You'll have to fork multiple OS processes in order to get Python code running in parallel on different cores. -- Greg
Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Processing a file using multithreads Abhishek Pratap <abhishek.vit@gmail.com> - 2011-09-08 15:49 -0700
Re: Processing a file using multithreads Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2011-09-09 12:03 +1200
Re: Processing a file using multithreads aspineux <aspineux@gmail.com> - 2011-09-08 21:44 -0700
Re: Processing a file using multithreads Roy Smith <roy@panix.com> - 2011-09-09 09:19 -0400
Re: Processing a file using multithreads Abhishek Pratap <abhishek.vit@gmail.com> - 2011-09-09 10:07 -0700
Re: Processing a file using multithreads Tim Roberts <timr@probo.com> - 2011-09-09 22:43 -0700
csiph-web