Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!rt.uk.eu.org!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <368aec88-ebe9-4da0-a537-92ff9b690647@googlegroups.com>
References: <368aec88-ebe9-4da0-a537-92ff9b690647@googlegroups.com>
From: Devin Jeanpierre <jeanpierreda@gmail.com>
Date: Fri, 30 May 2014 13:47:02 -0700
Subject: Re: daemon thread cleanup approach
To: Carl Banks <pavlovevidence@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Cc: "comp.lang.python" <python-list@python.org>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.10495.1401482865.18130.python-list@python.org>
Lines: 52
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:72318

Don't use daemon threads, they are inherently un-thread-safe: any
global access you do anywhere inside a daemon thread can fail, because
daemon threads are still potentially run during interpreter shutdown,
when globals are being deleted from every module. Most functions you
might call are not safe in a daemon thread at shutdown.

-- Devin

On Wed, May 28, 2014 at 6:20 PM, Carl Banks <pavlovevidence@gmail.com> wrot=
e:
> Ok, so I have an issue with cleaning up threads upon a unexpected exit.  =
I came up with a solution but I wanted to ask if anyone has any advice or w=
arnings.
>
> Basically I am writing a Python library to run certain tasks.  All of the=
 calls in the library start worker threads to do the actual work, and some =
of the worker threads are persistent, others not.  Most threads have cleanu=
p work to do (such as deleting temporary directories and killing spawned pr=
ocesses).
>
> For better or worse, one of the requirements is that the library can't ca=
use the program to hang no matter what, even if it means you have to forego=
 cleanup in the event of an unexpected exit.  Therefore all worker threads =
run as daemons.  Nevertheless, I feel like the worker threads should at lea=
st be given a fair opportunity to clean up; all threads can be communicated=
 with and asked to exit.
>
> One obvious solution is to ask users to put all library calls inside a wi=
th-statement that cleans up on exit, but I don't like it for various reason=
s.
> Using atexit doesn't work because it's called after the daemon threads ar=
e killed.
>
> Here's the solution I came up with: in the library's init function, it wi=
ll start a non-daemon thread that simply joins the main thread, and then as=
ks all existing worker threads to exit gracefully before timing out and lea=
ving them to be killed.  So if an exception ends the main thread, there is =
still a chance to clean up properly.
>
> Does anyone see a potential problem with this approach?  It it possible t=
hat this will cause the program to hang in any case?  We can assume that al=
l calls to the library will occur from the main thread, or at least from th=
e same thread.  (If that isn't the case, then the caller has taken responsi=
bility to ensure the program doesn't hang.)
>
> This is Python 2.7, and it's only ever going to run on Windows.
>
> Thanks for any advice/warnings.
>
> Carl Banks
> --
> https://mail.python.org/mailman/listinfo/python-list