Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.python > #45510
| Date | 2013-05-18 09:55 -0400 |
|---|---|
| From | Dave Angel <davea@davea.name> |
| Subject | Re: Please help with Threading |
| References | <7baacf5a-0c50-4935-ad5b-148c208d759b@googlegroups.com> |
| Newsgroups | comp.lang.python |
| Message-ID | <mailman.1802.1368885361.3114.python-list@python.org> (permalink) |
On 05/18/2013 04:58 AM, Jurgens de Bruin wrote:
> This is my first script where I want to use the python threading module. I have a large dataset which is a list of dict this can be as much as 200 dictionaries in the list. The final goal is a histogram for each dict 16 histograms on a page ( 4x4 ) - this already works.
> What I currently do is a create a nested list [ [ {} ], [ {} ] ] each inner list contains 16 dictionaries, thus each inner list is a single page of 16 histograms. Iterating over the outer-list and creating the graphs takes to long. So I would like multiple inner-list to be processes simultaneously and creating the graphs in "parallel".
> I am trying to use the python threading for this. I create 4 threads loop over the outer-list and send a inner-list to the thread. This seems to work if my nested lists only contains 2 elements - thus less elements than threads. Currently the scripts runs and then seems to get hung up. I monitor the resource on my mac and python starts off good using 80% and when the 4-thread is created the CPU usages drops to 0%.
>
> My thread creating is based on the following : http://www.tutorialspoint.com/python/python_multithreading.htm
>
> Any help would be create!!!
>
CPython, and apparently (all of?) the other current Python
implementations, uses a GIL to prevent multi-threaded applications from
shooting themselves in the foot.
However the practical effect of the GIL is that CPU-bound applications
do not multi-thread efficiently; the single-threaded version usually
runs faster.
The place where CPython programs gain from multithreading is where each
thread spends much of its time waiting for some external trigger.
(More specifically, if such a wait is inside well-written C code, it
releases the GIL so other threads can get useful work done. Example is
a thread waiting for internet activity, and blocks inside a system call)
--
DaveA
Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Please help with Threading Jurgens de Bruin <debruinjj@gmail.com> - 2013-05-18 01:58 -0700
Re: Please help with Threading Peter Otten <__peter__@web.de> - 2013-05-18 11:09 +0200
Re: Please help with Threading Jurgens de Bruin <debruinjj@gmail.com> - 2013-05-18 04:23 -0700
Re: Please help with Threading Peter Otten <__peter__@web.de> - 2013-05-18 14:01 +0200
Re: Please help with Threading Dave Angel <davea@davea.name> - 2013-05-18 09:55 -0400
Re: Please help with Threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-05-18 15:28 -0400
RE: Please help with Threading Carlos Nepomuceno <carlosnepomuceno@outlook.com> - 2013-05-19 03:02 +0300
Re: Please help with Threading Chris Angelico <rosuav@gmail.com> - 2013-05-19 10:38 +1000
Re: Please help with Threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-05-19 17:46 -0400
Re: Please help with Threading Chris Angelico <rosuav@gmail.com> - 2013-05-20 07:52 +1000
Re: Please help with Threading Dave Angel <davea@davea.name> - 2013-05-19 21:04 -0400
Re: Please help with Threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-05-19 22:58 -0400
Re: Please help with Threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-05-19 23:25 -0400
Re: Please help with Threading Fábio Santos <fabiosantosart@gmail.com> - 2013-05-20 07:25 +0100
Re: Please help with Threading Jurgens de Bruin <debruinjj@gmail.com> - 2013-06-02 18:47 -0700
csiph-web