Path: csiph.com!weretis.net!feeder4.news.weretis.net!newsreader4.netcologne.de!news.netcologne.de!newsfeed.kamp.net!newsfeed.kamp.net!fu-berlin.de!uni-berlin.de!not-for-mail
From: MRAB <python@mrabarnett.plus.com>
Newsgroups: comp.lang.python
Subject: Re: Calling python from C with OpenMP
Date: Fri, 13 May 2016 18:12:08 +0100
Lines: 38
Message-ID: <mailman.637.1463159537.32212.python-list@python.org>
References: <8224bdd2-9afe-487b-804b-f3b88dee2028@googlegroups.com> <1288606789484790413.405054sturla.molden-gmail.com@news.gmane.org> <mailman.618.1463097878.32212.python-list@python.org> <91ea4bb0-2d34-4a27-8b18-640c79712f64@googlegroups.com> <4cb4d566-709d-1139-b515-2eb3cdbbbc9a@mrabarnett.plus.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: news.uni-berlin.de 5GiQf/MwsX2yAtuZwr1yrQ/2VdosJxt19pyXsGYP63ag==
Return-Path: <python@mrabarnett.plus.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.002
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'friday,': 0.07; 'continue;': 0.09; 'matched': 0.09; 'thread': 0.10; 'python': 0.10; 'subject:python': 0.14; '(int)': 0.16; '10;': 0.16; '2016': 0.16; 'api,': 0.16; 'from:addr:mrabarnett.plus.com': 0.16; 'from:addr:python': 0.16; 'from:name:mrab': 0.16; 'message- id:@mrabarnett.plus.com': 0.16; 'null.': 0.16; 'pyobject': 0.16; 'received:192.168.1.4': 0.16; 'received:84.93': 0.16; 'received:84.93.230': 0.16; 'received:io': 0.16; 'received:psf.io': 0.16; 'threads': 0.16; 'wrote:': 0.16; 'written,': 0.18; 'this:': 0.23; 'header:In-Reply-To:1': 0.24; "i've": 0.25; 'header:User-Agent:1': 0.26; "skip:' 10": 0.28; 'looks': 0.29; '13,': 0.29; 'calculated': 0.29; 'gil': 0.29; 'ret': 0.29; 'call.': 0.30; 'received:84': 0.32; 'run': 0.33; 'int': 0.33; 'lock': 0.33; 'right?': 0.33; 'safely': 0.33; 'done': 0.35; 'but': 0.36; 'success.': 0.36; 'to:addr:python-list': 0.36; 'subject:: ': 0.37; 'done.': 0.37; 'release': 0.37; 'creation': 0.38; "won't": 0.38; 'skip:p 20': 0.38; 'subject:from': 0.39; 'received:192': 0.39; 'to:addr:python.org': 0.40; 'subject:with': 0.40; 'still': 0.40; "you'll": 0.61; 'making': 0.62; 'grab': 0.64; 'was:': 0.66; 'here': 0.66; 'gain': 0.82; 'calls,': 0.84
X-CM-Score: 0.00
X-CNFS-Analysis: v=2.1 cv=bsGxfxui c=1 sm=1 tr=0 a=0nF1XD0wxitMEM03M9B4ZQ==:117 a=0nF1XD0wxitMEM03M9B4ZQ==:17 a=L9H7d07YOLsA:10 a=9cW_t1CCXrUA:10 a=s5jvgZ67dGcA:10 a=IkcTkHD0fZMA:10 a=ULAATZW4dwJLQUJvaaEA:9 a=_DbuXqHl7vudiYgz:21 a=zr2i25PLA7VswQRD:21 a=QEXdDO2ut3YA:10
X-AUTH: mrabarnett@:2500
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.0
In-Reply-To: <91ea4bb0-2d34-4a27-8b18-640c79712f64@googlegroups.com>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <4cb4d566-709d-1139-b515-2eb3cdbbbc9a@mrabarnett.plus.com>
X-Mailman-Original-References: <8224bdd2-9afe-487b-804b-f3b88dee2028@googlegroups.com> <1288606789484790413.405054sturla.molden-gmail.com@news.gmane.org> <mailman.618.1463097878.32212.python-list@python.org> <91ea4bb0-2d34-4a27-8b18-640c79712f64@googlegroups.com>
Xref: csiph.com comp.lang.python:108607

On 2016-05-13 17:22, Øystein Schønning-Johansen wrote:
> On Friday, May 13, 2016 at 2:04:53 AM UTC+2, Sturla Molden wrote:
>> You must own the GIL before you can safely use the Python C API, object
>> creation and refcounting in particular. Use the "Simplified GIL API" to
>> grab the GIL and release it when you are done.
>
> I've now read about the GIL and it looks like I am in deep problems.
>
> I've added the GILState lock to the threaded loop like this:
>
> #pragma omp parallel for
>     for( int i = 0; i < 10; i++ ){
>         PyGILState_STATE gstate;
>         gstate = PyGILState_Ensure();
>         PyObject *ret = PyObject_CallMethod( mult_obj, "do_multiply", "i", i );
>         if( !ret ){
>             printf("Cannot call 'do_multiply'\n");
>             continue;
>         }
>         printf("The value calculated in Python was: %3d\n", (int) PyLong_AsLong(ret));
>         Py_DECREF(ret);
>         PyGILState_Release(gstate);
>     }
>
> .... but still no success. Have I done it right?
>
> regs,
> -Øystein
>
Every PyGILState_Ensure call must be matched with a PyGILState_Release 
call. The way it's currently written, it won't call PyGILState_Release 
if ret is NULL.

However, I don't think you'll gain much here because you can gain from 
multi-threading only if the threads can run in parallel. You need to 
hold the GIL while making Python calls, and only 1 thread can hold the 
GIL at any time.