Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.mixmin.net!rt.uk.eu.org!newsfeed.xs4all.nl!newsfeed4a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <368aec88-ebe9-4da0-a537-92ff9b690647@googlegroups.com>
References: <368aec88-ebe9-4da0-a537-92ff9b690647@googlegroups.com>
Date: Thu, 29 May 2014 18:15:35 +1000
Subject: Re: daemon thread cleanup approach
From: Chris Angelico <rosuav@gmail.com>
Cc: "python-list@python.org" <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.10444.1401351344.18130.python-list@python.org>
Lines: 31
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:72228

On Thu, May 29, 2014 at 11:20 AM, Carl Banks <pavlovevidence@gmail.com> wrote:
> Most threads have cleanup work to do (such as deleting temporary directories and killing spawned processes).
>
> For better or worse, one of the requirements is that the library can't cause the program to hang no matter what...

This ma y be a fundamental problem. I don't know how Windows goes with
killing processes (can that ever hang?), but certainly you can get
unexpected delays deleting a temp dir, although it would probably
require some deliberate intervention, like putting your %temp% on a
remote drive and then bringing that server down. But believe you me,
if there is a stupid way to do something, someone WILL have done it.
(Have you ever thought what it'd be like to have your
swapfile/pagefile on a network drive? I mean, there's acres of room on
the server, why waste some of your precious local space?)

So you may want to organize this as a separate spin-off process that
does the cleaning up. That way, the main process has completely ended,
but the cleanup daemon is still busy. And if you're going to do that,
then the easiest way, IMO, would be to have your worker threads be
themselves in a separate process; your library passes work across to
this other process via a pipe or socket (this being Windows, that
would have to be a TCP socket, not a Unix domain socket, but a named
pipe would also work), and when the pipe/socket connection is broken,
the other end knows that it should clean up. That way, you get to
clean up perfectly even if the process terminates abruptly (segfault,
system kill, whatever), although possibly delayed until the system
notices that the other end is gone.

ChrisA
(sometimes I feel I suggest TCP/IP sockets the way Grant Imahara
suggests building a robot... enthusiastically and maaaaaybe too often)