Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #45145 > unrolled thread

Re: Right way to initialize python embedded in a multi-threaded application

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Right way to initialize python embedded in a multi-threaded application francis.brosnan@gmail.com - 2013-05-11 11:23 -0700
    Re: Right way to initialize python embedded in a multi-threaded application francis.brosnan@gmail.com - 2013-05-11 13:00 -0700

#45145 — Re: Right way to initialize python embedded in a multi-threaded application

Hi,

Maybe you already fixed the issue, but for the record, I've got the same
problem and finally it turned out that I was calling PyEval_InitThreads twice
and also after fixing that, I also had to move the call to PyEval_ReleaseLock();  at the end of the entire initialization (not just after PyEval_initThreads).

The key thing there is to follow:

<<at initialization thread>>
Py_Initialize ();
PyEval_InitThreads(); 
/* now call here to initialize all python code by loading external files 
   or internal module loading (i.e. Py_InitModule3) */
/* ..and now, once no more Python C/API call is required, release
   the GIL so other threads can come into play */
PyEval_ReleaseLock ();

<<and now, from other threads, use>>
/* wait til gil acquired */
state  = PyGILState_Ensure();
/* your code */
/* release GIL */
PyGILState_Release (state);

Hope it helps, Cheers!


> I'm embedding python in a multi-threaded C application.
> 
> I've taken care to wrap every call to the Python C API with
> 
> 
> 
> gstate = PyGILState_Ensure();
> 
> // call python code
> 
> PyGILState_Release(gstate);
> 
> 
> 
> But I'm stumped with what to do in the initialization.
> 
> Right after the call to Py_IsInitialized() I've added a call:
> 
> 
> 
> PyEval_InitThreads();
> 
> 
> 
> The docs say that this function leaves the GIL locked when it returns.
> 
> I do some more initializations like importing modules and then I call
> 
> 
> 
> PyEval_ReleaseLock();
> 
> 
> 
> This seems to cause a problem since not long after a call to
> 
> PyGILState_Release(gstate) that's made in a different thread crashes.
> 
> with
> 
> 
> 
> "Fatal Python error: This thread state must be current when releasing"
> 
> 
> 
> If I don't do the call to PyEval_ReleaseLock() in the main thread
> 
> right after initialization, the GIL seems to be released
> 
> after the first PyGILState_Ensure() - PyGILState_Release() pair.
> 
> 
> 
> So what am I doing wrong here?
> 
> What is the correct way of initializing a multi-threaded application?

[toc] | [next] | [standalone]

#45152

Just clarify there's no problem about calling twice to PyEval_InitThreads ()
as indicated by Python's doc.

> Hi,
> 
> 
> 
> Maybe you already fixed the issue, but for the record, I've got the same
> 
> problem and finally it turned out that I was calling PyEval_InitThreads twice
> 
> and also after fixing that, I also had to move the call to PyEval_ReleaseLock();  at the end of the entire initialization (not just after PyEval_initThreads).
> 
> 
> 
> The key thing there is to follow:
> 
> 
> 
> <<at initialization thread>>
> 
> Py_Initialize ();
> 
> PyEval_InitThreads(); 
> 
> /* now call here to initialize all python code by loading external files 
> 
>    or internal module loading (i.e. Py_InitModule3) */
> 
> /* ..and now, once no more Python C/API call is required, release
> 
>    the GIL so other threads can come into play */
> 
> PyEval_ReleaseLock ();
> 
> 
> 
> <<and now, from other threads, use>>
> 
> /* wait til gil acquired */
> 
> state  = PyGILState_Ensure();
> 
> /* your code */
> 
> /* release GIL */
> 
> PyGILState_Release (state);
> 
> 
> 
> Hope it helps, Cheers!
> 
> 
> 
> 
> 
> > I'm embedding python in a multi-threaded C application.
> 
> > 
> 
> > I've taken care to wrap every call to the Python C API with
> 
> > 
> 
> > 
> 
> > 
> 
> > gstate = PyGILState_Ensure();
> 
> > 
> 
> > // call python code
> 
> > 
> 
> > PyGILState_Release(gstate);
> 
> > 
> 
> > 
> 
> > 
> 
> > But I'm stumped with what to do in the initialization.
> 
> > 
> 
> > Right after the call to Py_IsInitialized() I've added a call:
> 
> > 
> 
> > 
> 
> > 
> 
> > PyEval_InitThreads();
> 
> > 
> 
> > 
> 
> > 
> 
> > The docs say that this function leaves the GIL locked when it returns.
> 
> > 
> 
> > I do some more initializations like importing modules and then I call
> 
> > 
> 
> > 
> 
> > 
> 
> > PyEval_ReleaseLock();
> 
> > 
> 
> > 
> 
> > 
> 
> > This seems to cause a problem since not long after a call to
> 
> > 
> 
> > PyGILState_Release(gstate) that's made in a different thread crashes.
> 
> > 
> 
> > with
> 
> > 
> 
> > 
> 
> > 
> 
> > "Fatal Python error: This thread state must be current when releasing"
> 
> > 
> 
> > 
> 
> > 
> 
> > If I don't do the call to PyEval_ReleaseLock() in the main thread
> 
> > 
> 
> > right after initialization, the GIL seems to be released
> 
> > 
> 
> > after the first PyGILState_Ensure() - PyGILState_Release() pair.
> 
> > 
> 
> > 
> 
> > 
> 
> > So what am I doing wrong here?
> 
> > 
> 
> > What is the correct way of initializing a multi-threaded application?

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web