Groups > linux.debian.maint.python > #9746

Re: MBF for deprecating Python2 usage

Path	csiph.com!news.mixmin.net!aioe.org!bofh.it!news.nic.it!robomod
From	Geoffrey Thomas <geofft@ldpreload.com>
Newsgroups	linux.debian.maint.python
Subject	Re: MBF for deprecating Python2 usage
Date	Thu, 10 Aug 2017 17:40:03 +0200
Message-ID	<ucXJ1-7xc-23@gated-at.bofh.it> (permalink)
References	<uaBtf-1rM-9@gated-at.bofh.it> <uaLCi-867-15@gated-at.bofh.it> <ubYe6-6ih-19@gated-at.bofh.it> <ubZ0u-6St-15@gated-at.bofh.it> <ubZMR-7qT-11@gated-at.bofh.it> <ubZWy-7wz-3@gated-at.bofh.it> <uc1F0-zY-7@gated-at.bofh.it> <uc282-11e-7@gated-at.bofh.it> <uc2ro-1eJ-1@gated-at.bofh.it> <uc3dL-1UD-3@gated-at.bofh.it> <uc5fB-3gQ-31@gated-at.bofh.it>
X-Original-To	debian-python@lists.debian.org
X-Mailbox-Line	From debian-python-request@lists.debian.org Thu Aug 10 15:36:10 2017
Old-Return-Path	<geofft@ldpreload.com>
X-Amavis-Spam-Status	No, score=-2.6 tagged_above=-10000 required=5.3 tests=[BAYES_00=-2, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, MURPHY_SCAM1=0.1, RCVD_IN_DNSWL_LOW=-0.7] autolearn=no autolearn_force=no
X-Policyd-Weight	using cached result; rate: -6.1
Dkim-Signature	v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=date:from:to :subject:in-reply-to:message-id:references:mime-version :content-type; s=sasl; bh=wvBJAot53qXyDo8nwhbw2Zm0+yc=; b=erzuj7 c01z5fccnLhQuFCoMuVT0qk9Kbf0NU4BwB1ZDSx5tFBEhA8cmoFk86iN8VvkVrqr 3UA0q4o1WMXPq4yTSNBSnovt5mf+L/X1ri+A3WKbrJq7uU3Zn64GtejPik/v2mnH f45VTichkvEgFVNlt74p9b0EzUTR7iJ7YpwCc=
User-Agent	Alpine 2.11 (DEB 23 2013-08-11)
MIME-Version	1.0
Content-Type	TEXT/PLAIN; format=flowed; charset=US-ASCII
X-Pobox-Relay-ID	51BF2C7C-7DDF-11E7-8B7F-FE4B1A68708C-67873895!pb-smtp1.pobox.com
Dkim-Signature	v=1; a=rsa-sha256; c=relaxed; d=ldpreload.com; h=date:from:to:subject:in-reply-to:message-id:references:mime-version:content-type; s=mesmtp; bh=A/pFGXhwraX5Es/BYqdyZ2w+HBU5RllcyWqhP7ZG7Cg=; b=kanX4z942SXf8sq4qn2/OWE13u+tpWd+m+MA+VomaGi7InoI3ppvQqsGN+Imz6FKzTPxR+MfBiCB+OdfjvMIwiraQRWt6hJevJanNcrGnCrVuP7jfPKVvkFbl1/sevViiJlbiX375ITa/VJx8Xsjugh4vtEyDjdcNZdu+KNkh7c=
X-Mailing-List	<debian-python@lists.debian.org> archive/latest/15128
List-ID	<debian-python.lists.debian.org>
List-URL	<https://lists.debian.org/debian-python/>
List-Archive	https://lists.debian.org/msgid-search/alpine.DEB.2.11.1708101045290.1801@titan.ldpreload.com
Approved	robomod@news.nic.it
Lines	44
Organization	linux.* mail to news gateway
Sender	robomod@news.nic.it
X-Original-Date	Thu, 10 Aug 2017 11:19:35 -0400 (EDT)
X-Original-Message-ID	<alpine.DEB.2.11.1708101045290.1801@titan.ldpreload.com>
X-Original-References	<9614BFBC-E260-48A6-B722-DDD41E715A20@kitterman.com> <92E6FF73-391C-45FE-A213-6AAED284629D@debian.org> <1502142757.2351.35.camel@ghic.org> <690EAE7E-8005-4723-891E-C2E30C3B35AD@debian.org> <85shh3lzp1.fsf@benfinney.id.au> <1502149701.2351.60.camel@ghic.org> <85inhyn8zk.fsf@benfinney.id.au> <alpine.DEB.2.11.1708072135500.1801@titan.ldpreload.com> <854ltin6q1.fsf@benfinney.id.au> <1502161900.2351.65.camel@ghic.org> <20170808052035.vx2j76cr2wugln3i@virgil.dodds.net>
Xref	csiph.com linux.debian.maint.python:9746

Show key headers only | View raw

On Mon, 7 Aug 2017, Steve Langasek wrote:

> But I am concerned about possible implementation strategies here.  exec() is
> a very expensive syscall, and python is a frequently used interpreter.  If
> this were implemented as a wrapper that checked isatty(), printed a banner,
> and then re-execed the real python, that could have a measurable performance
> impact on some applications.  There is a reason /usr/bin/python,
> /usr/bin/gcc, etc. are always symlinks to the real interpreter on Debian,
> not wrapper scripts - other distributions have tried to do this as a wrapper
> script and the result wasn't pretty.

Python itself is not very fast to start up, though, so exec is not the 
dominating cost. Some ad-hoc testing with the current version of pythonmux 
(which uses the re-exec approach) shows that it takes about 13.8 ms to 
print("Hello world") with /usr/bin/python and 15.5 ms to print("Hello 
world") with pythonmux on my 2010-ish laptop.

For the sake of argument, I just pushed a branch of pythonmux that links 
libpython3.5 and calls Py_Main instead of execing python3.5 (that is, if 
the target interpreter is python3.5, it will not actually exec):

   https://github.com/geofft/pythonmux/commits/libpython

But it's quite a bit _slower_ with this approach (about 26.9 ms per run), 
regardless of whether I link libpython3.5m.so or libpython3.5m.a. This 
confuses me and I'm definitely curious to figure this out if that's worth 
doing, but I think this is getting into a lot of weird complexity to save 
1.7 ms on a mere hello world -- for comparison, if I add `import requests` 
to my hello world program, each run now takes about 342 ms.

By the way, I do think I made pythonmux do too much since I wanted to make 
it useful in the general case; in particular I don't think it's super 
helpful to support scripts that say "I can run on Python 3.2 but not any 
newer version". Given the way the discussion is going (both here and on 
linux-sig) and how close we are to Python 2's end-of-life, I think it's 
sufficient to check isatty(0) and the shebang line if one exists, call 
libpython3's Py_Main if that seems reasonable, print a warning and attempt 
to exec python2 otherwise, and print an error message about how to install 
python2 if the exec fails.

-- 
Geoffrey Thomas
https://ldpreload.com
geofft@ldpreload.com

Thread

MBF for deprecating Python2 usage Matthias Klose <doko@debian.org> - 2017-08-04 00:00 +0200
  Re: MBF for deprecating Python2 usage barry@debian.org - 2017-08-04 03:10 +0200
    Re: MBF for deprecating Python2 usage Scott Kitterman <debian@kitterman.com> - 2017-08-04 05:30 +0200
      Re: MBF for deprecating Python2 usage Piotr Ożarowski <piotr@debian.org> - 2017-08-04 08:10 +0200
      Re: MBF for deprecating Python2 usage barry@debian.org - 2017-08-04 16:20 +0200
        Re: MBF for deprecating Python2 usage Ole Streicher <olebole@debian.org> - 2017-08-04 17:10 +0200
          Re: MBF for deprecating Python2 usage Barry Warsaw <barry@debian.org> - 2017-08-04 17:50 +0200
        Re: MBF for deprecating Python2 usage Scott Kitterman <debian@kitterman.com> - 2017-08-04 23:10 +0200
          Re: MBF for deprecating Python2 usage Barry Warsaw <barry@debian.org> - 2017-08-05 03:40 +0200
            Re: MBF for deprecating Python2 usage Scott Kitterman <debian@kitterman.com> - 2017-08-05 04:10 +0200
          Re: MBF for deprecating Python2 usage Geoffrey Thomas <geofft@ldpreload.com> - 2017-08-08 00:50 +0200
        Re: MBF for deprecating Python2 usage Diane Trout <diane@ghic.org> - 2017-08-08 00:00 +0200
          Re: MBF for deprecating Python2 usage Barry Warsaw <barry@debian.org> - 2017-08-08 00:50 +0200
            Re: MBF for deprecating Python2 usage Ben Finney <bignose@debian.org> - 2017-08-08 01:40 +0200
              Re: MBF for deprecating Python2 usage Diane Trout <diane@ghic.org> - 2017-08-08 01:50 +0200
                Re: MBF for deprecating Python2 usage Ben Finney <bignose@debian.org> - 2017-08-08 03:40 +0200
                Re: MBF for deprecating Python2 usage Barry Warsaw <barry@debian.org> - 2017-08-08 04:00 +0200
                Re: MBF for deprecating Python2 usage Ben Finney <bignose@debian.org> - 2017-08-08 04:20 +0200
                Re: MBF for deprecating Python2 usage Diane Trout <diane@ghic.org> - 2017-08-08 05:10 +0200
                Re: MBF for deprecating Python2 usage Ben Finney <bignose@debian.org> - 2017-08-08 05:30 +0200
                Re: MBF for deprecating Python2 usage Diane Trout <diane@ghic.org> - 2017-08-08 05:40 +0200
                Re: MBF for deprecating Python2 usage Ben Finney <bignose@debian.org> - 2017-08-08 06:00 +0200
                Re: MBF for deprecating Python2 usage Barry Warsaw <barry@debian.org> - 2017-08-08 07:00 +0200
                Re: MBF for deprecating Python2 usage Geoffrey Thomas <geofft@ldpreload.com> - 2017-08-08 04:10 +0200
                Re: MBF for deprecating Python2 usage Ben Finney <bignose@debian.org> - 2017-08-08 04:30 +0200
                Re: MBF for deprecating Python2 usage Diane Trout <diane@ghic.org> - 2017-08-08 05:20 +0200
                Re: MBF for deprecating Python2 usage Ben Finney <bignose@debian.org> - 2017-08-08 05:40 +0200
                Re: MBF for deprecating Python2 usage Steve Langasek <vorlon@debian.org> - 2017-08-08 07:30 +0200
                Re: MBF for deprecating Python2 usage Ben Finney <bignose@debian.org> - 2017-08-08 08:20 +0200
                Re: MBF for deprecating Python2 usage Geoffrey Thomas <geofft@ldpreload.com> - 2017-08-10 17:40 +0200
                Re: MBF for deprecating Python2 usage Ole Streicher <olebole@debian.org> - 2017-08-08 21:20 +0200
                Re: MBF for deprecating Python2 usage Scott Kitterman <debian@kitterman.com> - 2017-08-08 21:30 +0200
    Re: MBF for deprecating Python2 usage Matthias Klose <doko@debian.org> - 2017-08-04 13:00 +0200
      Re: MBF for deprecating Python2 usage Scott Kitterman <debian@kitterman.com> - 2017-08-04 14:40 +0200
  Re: MBF for deprecating Python2 usage Adrian Bunk <bunk@debian.org> - 2017-08-07 00:00 +0200
  Re: MBF for deprecating Python2 usage Adrian Bunk <bunk@debian.org> - 2017-08-07 01:00 +0200
  Re: MBF for deprecating Python2 usage Christian Seiler <christian@iwakd.de> - 2017-08-07 01:10 +0200

csiph-web