Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #108651

Re: intermittent ValueErrors from subprocess

Path csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From Chris Angelico <rosuav@gmail.com>
Newsgroups comp.lang.python
Subject Re: intermittent ValueErrors from subprocess
Date Mon, 16 May 2016 09:32:37 +1000
Lines 39
Message-ID <mailman.8.1463355159.19823.python-list@python.org> (permalink)
References <57389A9D.1010205@gmail.com> <CAPTjJmp1H99HKmEarskE1b-ia0=KFDBTpfU+cWt6vVFeJtBHZg@mail.gmail.com> <5738F8E2.1030403@gmail.com> <CAPTjJmrNLFnvWOr=0D=WT3FJuZ1btFuLsxxrRaAF5M2D7GYtXg@mail.gmail.com> <57390395.8070005@gmail.com> <CAPTjJmo8QCbtbFbhXPgHVO_nBSEYhpWk2Xfzvdzn8DA71HnrYQ@mail.gmail.com>
Mime-Version 1.0
Content-Type text/plain; charset=UTF-8
Content-Transfer-Encoding quoted-printable
X-Trace news.uni-berlin.de RHYOfx9oJ633eQygKxoXNAQUTQHPUMuXRRyHHh6aX4jA==
Return-Path <rosuav@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.001
X-Spam-Evidence '*H*': 1.00; '*S*': 0.00; '16,': 0.03; 'that?': 0.05; 'apis': 0.07; 'executed': 0.07; 'means,': 0.07; 'api': 0.09; 'cc:addr:python-list': 0.09; '"run': 0.09; 'exits': 0.09; 'mess': 0.09; 'messing': 0.09; 'yeah,': 0.09; 'exception': 0.13; '12:59': 0.16; '2016': 0.16; "can't.": 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'pipes': 0.16; 'received:io': 0.16; 'received:psf.io': 0.16; 'so;': 0.16; 'traceback.': 0.16; 'zero,': 0.16; 'wrote:': 0.16; 'cc:2**0': 0.20; 'cc:addr:python.org': 0.20; 'fix': 0.21; 'explicit': 0.22; 'simpler': 0.22; 'trace': 0.22; 'am,': 0.23; 'slightly': 0.23; 'somewhere': 0.24; 'header:In-Reply-To:1': 0.24; 'mon,': 0.24; 'chris': 0.26; 'least': 0.27; 'message-id:@mail.gmail.com': 0.27; 'occurred': 0.29; 'such.': 0.29; 'program,': 0.29; 'code': 0.30; "i'd": 0.31; 'post': 0.31; "can't": 0.32; 'says': 0.32; 'problem': 0.33; 'source': 0.33; 'usually': 0.33; 'case,': 0.34; 'received:google.com': 0.35; 'could': 0.35; 'instance': 0.35; 'something': 0.35; 'level': 0.35; 'received:74.125.82': 0.35; 'problem.': 0.35; 'but': 0.36; 'lines': 0.36; 'smaller': 0.36; 'subject:: ': 0.37; 'being': 0.37; 'there,': 0.37; 'client': 0.37; 'say': 0.37; 'times.': 0.38; 'several': 0.38; 'someone': 0.38; 'mean': 0.38; 'sure': 0.39; 'subject:from': 0.39; "didn't": 0.39; 'unable': 0.39; 'rather': 0.39; 'high': 0.60; 'your': 0.60; 'clients': 0.61; 'worth': 0.67; 'apart': 0.70; 'click': 0.76; 'demand': 0.79; 'chrisa': 0.84; "else's": 0.84; 'hard.': 0.84; 'received:74.125.82.47': 0.84; 'start.': 0.84; 'tease': 0.84; 'to:none': 0.91
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:cc :content-transfer-encoding; bh=VHp+K0j62uYew3IZBFydfSqHrp3qeQsxgshewmN8VnI=; b=RlbAuPgLpAtp+4OTqsVJzi3bQ1V1zH/UVVWcH1PZW0ERJ8xKFHYey6I2HdbcjtKOlH ACDAPjCNwqWbk4pAEsCANlPBHi8VGtxKrXbkoeW5Wk/Dh26RgdZB1BNeW3w4wIXFwLv3 keErnZO3A+vVC1fB5/Rm7znJG7jzb9yVX+wgVvMwbf3oxV54sw838fDhZVqf9oqauoPH JOUUVlrn02qbYbZYTFRtZPCjUq+SNqnl/JrxMyNrW+JTnM99gMZGY/k55FmiOnISHSS+ xzMWtzcekIVosgy/dGxxc0+4AKxoltMskfYQLMOcHRGNXEwIaMr8WJGJb3LHr6/cq0dz UHOQ==
X-Google-DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:cc:content-transfer-encoding; bh=VHp+K0j62uYew3IZBFydfSqHrp3qeQsxgshewmN8VnI=; b=kk+wOHKSLMxy+cUppZJufLyV7udYIKeFnBGUIxPE359UssMkfjRJX+8X3vFqyNiyNJ drJRC4nmDVjsDXpNLsRfrHhpeEuo81irdHgcuK2BOy8Hi6ErBoplE8JhSsW0bU7eu+TP qvt7qRUngffjhvzRVO0xAaOOkdi/cfL9YWthL+tD074qs47aMpVAhkY/CgtNZ9bTRueN x1IEC2Lxybp7LS9Qgk7rkH9yfAzDZ4F/GnEk3GhscJxilchnqcQjdCY8qJk9YdhY8Kmx zrw4mfnmW+f8wIQAMsXvmj51mlzoyzWUJagDMGtJiKsoeC6wadLn3d4u0MlhqPxHmWIc YlDA==
X-Gm-Message-State AOPr4FUsPKjKsJj3/5alqB32Mg+W3/76nrJogivgheC8AadvXLiPCzljsdSFZmyIxxFHRja8j8NeiAl8KzGrew==
X-Received by 10.194.233.97 with SMTP id tv1mr13059417wjc.30.1463355157784; Sun, 15 May 2016 16:32:37 -0700 (PDT)
In-Reply-To <57390395.8070005@gmail.com>
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.22
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID <CAPTjJmo8QCbtbFbhXPgHVO_nBSEYhpWk2Xfzvdzn8DA71HnrYQ@mail.gmail.com>
X-Mailman-Original-References <57389A9D.1010205@gmail.com> <CAPTjJmp1H99HKmEarskE1b-ia0=KFDBTpfU+cWt6vVFeJtBHZg@mail.gmail.com> <5738F8E2.1030403@gmail.com> <CAPTjJmrNLFnvWOr=0D=WT3FJuZ1btFuLsxxrRaAF5M2D7GYtXg@mail.gmail.com> <57390395.8070005@gmail.com>
Xref csiph.com comp.lang.python:108651

Show key headers only | View raw


On Mon, May 16, 2016 at 9:17 AM, Pavlos Parissis
<pavlos.parissis@gmail.com> wrote:
> On 16/05/2016 12:59 πμ, Chris Angelico wrote:
>> Ah. I didn't click that link in your original post - didn't know it
>> was the traceback. Better would have been to at least say so; best
>> would have been to include it inline.
>>
>
> I don't usually include traces as they have long lines and several
> e-mail clients mess with them in a way that makes the trace unreadable.

If your client can't send them, fix your client. If someone else's
client can't read them, that's not your problem.

>> If you can recreate the problem consistently,
>
> I can't. This is my main problem. This code has been executed ~100K and
> that exception has occurred only ~5 times.

Yeah, I figured. Makes it tough.

>> it would be worth
>> messing around with slightly lower level APIs - using subprocess.Popen
>> rather than check_output, for instance - and see what you can do
>> without the pipes.
>
> What do you mean by that?

You're currently using a very high level API that says "run this
program, make sure it exits zero, and give me its output". Somewhere
inside there, something's going wrong. So the first thing I'd do would
be to tease apart the job into smaller and simpler parts; that means,
in this case, going to something like subprocess.Popen, with explicit
use of pipes and such. You could look at the source code for
check_output for a start.

But being unable to recreate the problem on demand makes it hard.

ChrisA

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread


Thread

Re: intermittent ValueErrors from subprocess Chris Angelico <rosuav@gmail.com> - 2016-05-16 09:32 +1000

csiph-web