Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #45510

Re: Please help with Threading

Path csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!rt.uk.eu.org!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path <davea@davea.name>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.003
X-Spam-Evidence '*H*': 0.99; '*S*': 0.00; 'scripts': 0.03; 'cpython': 0.05; '(all': 0.07; 'nested': 0.07; 'subject:help': 0.08; 'works.': 0.09; 'runs': 0.10; 'python': 0.11; 'thread': 0.14; 'blocks': 0.16; 'call)': 0.16; 'dict': 0.16; 'dictionaries': 0.16; 'iterating': 0.16; 'multi-thread': 0.16; 'shooting': 0.16; 'threads.': 0.16; 'elements': 0.16; 'prevent': 0.16; 'wrote:': 0.18; 'trying': 0.19; 'starts': 0.20; 'seems': 0.21; 'code,': 0.22; 'example': 0.22; 'creating': 0.23; 'header:User-Agent:1': 0.23; 'script': 0.25; 'header:In-Reply-To:1': 0.27; 'external': 0.29; 'am,': 0.29; 'thus': 0.29; 'usually': 0.31; 'apparently': 0.31; 'long.': 0.31; 'lists': 0.32; 'this.': 0.32; 'themselves': 0.32; 'up.': 0.33; 'url:python': 0.33; 'mac': 0.33; 'subject:with': 0.35; 'created': 0.35; 'done.': 0.35; 'version': 0.36; 'module.': 0.36; 'useful': 0.36; 'list': 0.37; 'list.': 0.37; 'received:10': 0.37; 'to:addr:python-list': 0.38; 'resource': 0.38; 'to:addr:python.org': 0.39; 'first': 0.61; 'such': 0.63; 'received:74.208': 0.68; 'internet': 0.71; 'url:htm': 0.73; 'goal': 0.75; 'gain': 0.79; '80%': 0.84; 'faster.': 0.84; 'received:74.208.4.194': 0.84; 'activity,': 0.91; 'drops': 0.91
Date Sat, 18 May 2013 09:55:56 -0400
From Dave Angel <davea@davea.name>
User-Agent Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Thunderbird/17.0.5
MIME-Version 1.0
To python-list@python.org
Subject Re: Please help with Threading
References <7baacf5a-0c50-4935-ad5b-148c208d759b@googlegroups.com>
In-Reply-To <7baacf5a-0c50-4935-ad5b-148c208d759b@googlegroups.com>
Content-Type text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding 7bit
X-Provags-ID V02:K0:3KWiM8xJ4sjQxv0gYlZqmBV1lwmj8k5b1PXFvtwASQX R+skk27SWzos3CAJylpPByrk5RaG0MAO/L5Fwmbdjw3V85p7hr bjXjSb+ifwQ7pR0OVUtxKQQP0LhaPMkyxRQyIGZ5Qs+DY2xeku nrOq6l1HRGSuS8Pz7EZC3c2qH8hHI23ud3f/Q/w/m0dn0L+VaZ r3Mw6L48gmDNEdd5Yzr18qtOjO3JkgOHy24DWO7pb1rPMMfxvV L5L38f+zfFqXsM7+ygsav6TB0HF0dVlfp3CxJH8DjGYQOxAwgP qnFw97koafreIeQk4Y79mEi02+U8wnqC2J1D01kD7qMjYFI48/ Mvn6ne+HVhTHafpEb7a8=
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.1802.1368885361.3114.python-list@python.org> (permalink)
Lines 28
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1368885361 news.xs4all.nl 15973 [2001:888:2000:d::a6]:56273
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:45510

Show key headers only | View raw

On 05/18/2013 04:58 AM, Jurgens de Bruin wrote:
> This is my first script where I want to use the python threading module. I have a large dataset which is a list of dict this can be as much as 200 dictionaries in the list. The final goal is a  histogram for each dict 16 histograms on a page ( 4x4 ) - this already works.
> What I currently do is a create a nested list [ [ {}  ], [ {} ] ] each inner list contains 16 dictionaries, thus each inner list is a single page of 16 histograms. Iterating over the outer-list  and creating the graphs takes to long. So I would like multiple inner-list to be processes simultaneously and creating the graphs in "parallel".
> I am trying to use the python threading for this. I create 4 threads loop over the outer-list and send a inner-list to the thread. This seems to work if my nested lists only contains 2 elements - thus less elements than threads. Currently the scripts runs and then seems to get hung up. I monitor the resource  on my mac and python starts off good using 80% and when the 4-thread is created the CPU usages drops to 0%.
>
> My thread creating is based on the following : http://www.tutorialspoint.com/python/python_multithreading.htm
>
> Any help would be create!!!
>

CPython, and apparently (all of?) the other current Python 
implementations, uses a GIL to prevent multi-threaded applications from 
shooting themselves in the foot.

However the practical effect of the GIL is that CPU-bound applications 
do not multi-thread efficiently;  the single-threaded version usually 
runs faster.

The place where CPython programs gain from multithreading is where each 
thread spends much of its time waiting for some external trigger.

(More specifically, if such a wait is inside well-written C code, it 
releases the GIL so other threads can get useful work done.  Example is 
a thread waiting for internet activity, and blocks inside a system call)


-- 
DaveA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Please help with Threading Jurgens de Bruin <debruinjj@gmail.com> - 2013-05-18 01:58 -0700
  Re: Please help with Threading Peter Otten <__peter__@web.de> - 2013-05-18 11:09 +0200
    Re: Please help with Threading Jurgens de Bruin <debruinjj@gmail.com> - 2013-05-18 04:23 -0700
      Re: Please help with Threading Peter Otten <__peter__@web.de> - 2013-05-18 14:01 +0200
  Re: Please help with Threading Dave Angel <davea@davea.name> - 2013-05-18 09:55 -0400
  Re: Please help with Threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-05-18 15:28 -0400
  RE: Please help with Threading Carlos Nepomuceno <carlosnepomuceno@outlook.com> - 2013-05-19 03:02 +0300
  Re: Please help with Threading Chris Angelico <rosuav@gmail.com> - 2013-05-19 10:38 +1000
  Re: Please help with Threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-05-19 17:46 -0400
  Re: Please help with Threading Chris Angelico <rosuav@gmail.com> - 2013-05-20 07:52 +1000
  Re: Please help with Threading Dave Angel <davea@davea.name> - 2013-05-19 21:04 -0400
  Re: Please help with Threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-05-19 22:58 -0400
  Re: Please help with Threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-05-19 23:25 -0400
  Re: Please help with Threading Fábio Santos <fabiosantosart@gmail.com> - 2013-05-20 07:25 +0100
  Re: Please help with Threading Jurgens de Bruin <debruinjj@gmail.com> - 2013-06-02 18:47 -0700

csiph-web