Path: csiph.com!usenet.pasdenom.info!dedibox.gegeweb.org!gegeweb.eu!nntpfeed.proxad.net!proxad.net!feeder1-2.proxad.net!usenet-fr.net!nerim.net!novso.com!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <675725e3-38d2-4d81-bf64-f6d903d4a684@googlegroups.com>
References: <a087ae04-a87c-4e77-a7dd-2d883d30a6f0@googlegroups.com> <mailman.9495.1398434710.18130.python-list@python.org> <675725e3-38d2-4d81-bf64-f6d903d4a684@googlegroups.com>
Date: Mon, 28 Apr 2014 00:33:38 +1000
Subject: Re: MacOS 10.9.2: threading error using python.org 2.7.6 distribution
From: Chris Angelico <rosuav@gmail.com>
Cc: "python-list@python.org" <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.9531.1398609228.18130.python-list@python.org>
Lines: 50
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:70654

On Mon, Apr 28, 2014 at 12:16 AM, Matthew Pounsett
<matt.pounsett@gmail.com> wrote:
> On Friday, 25 April 2014 10:05:03 UTC-4, Chris Angelico  wrote:
>> First culprit I'd look at is the mixing of subprocess and threading.
>> It's entirely possible that something goes messy when you fork from a
>> thread.
>
> I liked the theory, but I've run some tests and can't reproduce the error=
 that way.  I'm using all the elements in my test code that the real code r=
uns, and I can't get the same error.  Even when I deliberately break things=
 I'm getting a proper exception with stack trace.
>

In most contexts, "thread unsafe" simply means that you can't use the
same facilities simultaneously from two threads (eg a lot of database
connection libraries are thread unsafe with regard to a single
connection, as they'll simply write to a pipe or socket and then read
a response from it). But processes and threads are, on many systems,
linked. Just the act of spinning off a new thread and then forking can
potentially cause problems. Those are the exact sorts of issues that
you'll see when you switch OSes, as it's the underlying thread/process
model that's significant. (Particularly of note is that Windows is
*very* different from Unix-based systems, in that subprocess
management is not done by forking. But not applicable here.)

You may want to have a look at subprocess32, which Ned pointed out. I
haven't checked, but I would guess that its API is identical to
subprocess's, so it should be a drop-in replacement ("import
subprocess32 as subprocess"). If that produces the exact same results,
then it's (probably) not thread-safety that's the problem.

>> Separately: You're attempting a very messy charset decode there. You
>> attempt to decode as UTF-8, errors ignored, and if that fails, you log
>> an error... and continue on with the original bytes. You're risking
>> shooting yourself in the foot there; I would recommend you have an
>> explicit fall-back (maybe re-decode as Latin-1??), so the next code is
>> guaranteed to be working with Unicode. Currently, it might get a
>> unicode or a str.
>
> Yeah, that was a logic error on my part that I hadn't got around to notic=
ing, since I'd been concentrating on the stuff that was actively breaking. =
 That should have been in an else: block on the end of the try.
>

Ah good. Keeping bytes versus text separate is something that becomes
particularly important in Python 3, so I always like to encourage
people to get them straight even in Py2. It'll save you some hassle
later on.

ChrisA