Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From: Chris Warrick <kwpolska@gmail.com>
Newsgroups: comp.lang.python
Subject: Re: Using subprocess to capture a progress line
Date: Wed, 11 Nov 2015 10:48:03 +0100
Lines: 59
Message-ID: <mailman.234.1447235293.16136.python-list@python.org>
References: <20151110224756.GA1944@mail.akwebsoft.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
In-Reply-To: <20151110224756.GA1944@mail.akwebsoft.com>
Precedence: list
Xref: csiph.com comp.lang.python:98634

On 10 November 2015 at 23:47, Tim Johnson <tim@akwebsoft.com> wrote:
> Using python 2.7.6 on ubuntu 14.04
> The application in question is run with bash and gnome-terminal :
>
> I've written a command-line "wrapper" for youtube-dl, executing
> youtube-dl as a subprocess.
>
> ------------------------------------------------------------------
> youtube-dl reports download progress on one line. I.E. the line is
> overwritten numerous times with no carriage return until the
> downloading is finished.
> ------------------------------------------------------------------
>
> The following code runs the youtube-dl command and reports each line
> as output by youtube-dl
> ###########
>     p =3D subprocess.Popen(list(args), stderr=3Dsubprocess.STDOUT,
>                          stdout=3Dsubprocess.PIPE)
>     while True:
>         line =3D p.stdout.readline()
>         if not line:
>             break
>         tmp =3D line.strip()
>         print tmp
> ###########
>
> However this method not does show the download progress _until_ the
> download is complete.

There is no \n character at the end =E2=80=94 which means that
p.stdout.readline() cannot return. In fact, if you printed repr() of
the line you read, you would get this:

b'\r[download]  54.9% of 2.73MiB at 26.73KiB/s ETA 00:47\r[download]
55.0% of 2.73MiB at 79.33KiB/s ETA 00:15\r=E2=80=A6snip=E2=80=A6\r[download=
] 100% of
2.73MiB in 00:01\n'

The download line is implemented using \r, which is the carriage
return character (return to the first character), and then by
overwriting characters that were already printed.

The solution? There are numerous. I=E2=80=99ll help you by obscuring the wo=
rst one.

(1) [recommended] figure out how to make youtube_dl work as a library,
read its main file to figure out the problem. Don=E2=80=99t mess with
subprocess.
(2) [don=E2=80=99t do it] do you need to intercept the lines? If you don=E2=
=80=99t set
stderr=3D and stdout=3D, things will print just fine.
(3) [DON=E2=80=99T DO IT] .ernq() punenpgre ol punenpgre naq znxr n zrff.

PS. Thank you for setting a sensible Reply-To header on your messages.
Which is something the list should be doing.

--=20
Chris Warrick <https://chriswarrick.com/>
PGP: 5EAAEA16