X-Received: by 10.236.99.7 with SMTP id w7mr1547502yhf.4.1401326406349; Wed, 28 May 2014 18:20:06 -0700 (PDT)
X-Received: by 10.182.66.73 with SMTP id d9mr30391obt.10.1401326406180; Wed, 28 May 2014 18:20:06 -0700 (PDT)
Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.glorb.com!c1no19571870igq.0!news-out.google.com!qf4ni17234igc.0!nntp.google.com!c1no19571852igq.0!postnews.google.com!glegroupsg2000goo.googlegroups.com!not-for-mail
Newsgroups: comp.lang.python
Date: Wed, 28 May 2014 18:20:05 -0700 (PDT)
Complaints-To: groups-abuse@google.com
Injection-Info: glegroupsg2000goo.googlegroups.com; posting-host=134.223.230.157; posting-account=XFEWnwoAAADNO10m3Wcmq_SWdmyZuXff
NNTP-Posting-Host: 134.223.230.157
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <368aec88-ebe9-4da0-a537-92ff9b690647@googlegroups.com>
Subject: daemon thread cleanup approach
From: Carl Banks <pavlovevidence@gmail.com>
Injection-Date: Thu, 29 May 2014 01:20:06 +0000
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Xref: csiph.com comp.lang.python:72206

Ok, so I have an issue with cleaning up threads upon a unexpected exit.  I =
came up with a solution but I wanted to ask if anyone has any advice or war=
nings.

Basically I am writing a Python library to run certain tasks.  All of the c=
alls in the library start worker threads to do the actual work, and some of=
 the worker threads are persistent, others not.  Most threads have cleanup =
work to do (such as deleting temporary directories and killing spawned proc=
esses).

For better or worse, one of the requirements is that the library can't caus=
e the program to hang no matter what, even if it means you have to forego c=
leanup in the event of an unexpected exit.  Therefore all worker threads ru=
n as daemons.  Nevertheless, I feel like the worker threads should at least=
 be given a fair opportunity to clean up; all threads can be communicated w=
ith and asked to exit.

One obvious solution is to ask users to put all library calls inside a with=
-statement that cleans up on exit, but I don't like it for various reasons.
Using atexit doesn't work because it's called after the daemon threads are =
killed.

Here's the solution I came up with: in the library's init function, it will=
 start a non-daemon thread that simply joins the main thread, and then asks=
 all existing worker threads to exit gracefully before timing out and leavi=
ng them to be killed.  So if an exception ends the main thread, there is st=
ill a chance to clean up properly.

Does anyone see a potential problem with this approach?  It it possible tha=
t this will cause the program to hang in any case?  We can assume that all =
calls to the library will occur from the main thread, or at least from the =
same thread.  (If that isn't the case, then the caller has taken responsibi=
lity to ensure the program doesn't hang.)

This is Python 2.7, and it's only ever going to run on Windows.

Thanks for any advice/warnings.

Carl Banks