Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #10850 > unrolled thread

Byte-compilation for pypy3

Back to article view | Back to linux.debian.maint.python

Contents

  Byte-compilation for pypy3 Stefano Rivera <stefanor@debian.org> - 2018-08-27 04:10 +0200
    Re: Byte-compilation for pypy3 teng zhaozhi <freet15@gmail.com> - 2018-08-27 07:50 +0200
    Re: Byte-compilation for pypy3 Piotr Ożarowski <piotr@debian.org> - 2018-08-27 11:10 +0200
      Re: Byte-compilation for pypy3 Piotr Ożarowski <piotr@debian.org> - 2018-08-27 11:20 +0200
        Re: Byte-compilation for pypy3 Piotr Ożarowski <piotr@debian.org> - 2018-08-27 11:20 +0200
          Re: Byte-compilation for pypy3 Stefano Rivera <stefanor@debian.org> - 2018-08-27 18:10 +0200
            Re: Byte-compilation for pypy3 Piotr Ożarowski <piotr@debian.org> - 2018-09-28 00:00 +0200

#10850 — Byte-compilation for pypy3

So, I've had some pypy3 packages sitting around for a while, and the
main thing that's stopped me from uploading, is figuring out a plan for
byte-compilation.

If it's sharing /usr/lib/python3/dist-packages with cPython (which it
currently is) then we should figure out a plat for byte-compiling for
it.

rtupdate and postinst / prerm hooks for python3 packages currently call
py3{compile,clean} which only operates on cpython3.

Options I can think of:
1. Ignore byte-compilation entirely.
2. Change py3{compile,clean} to call pypy3{compile,clean} too.
   And do some extra compiling / cleaning when pypy3 is installed and
   removed.
3. Change py3{compile,clean} to do the work of pypy3{compile,clean} too.
   And do some extra compiling / cleaning when pypy3 is installed and
   removed.
4. Change the snipets dh_python3 generates, to include
   pypy3{compile,clean}, when they exist. And again, do the extra
   compiling / cleanup.
5. Something else?

3 *seems* like the right option - py3{compile,clean} supports a range of
cpython versions, already. But I'm not really sure.

Here's what I've got:
https://salsa.debian.org/debian/pypy3
https://people.debian.org/~stefanor/pypy3/

Other TODOs that need some attention but possibly aren't blocking an
upload to unstable:
* test failures and errors
* import paths aren't quite right, yet, e.g.:
  - setup.py install installs to a /usr/local path that isn't on
    sys.path
  - plat-linux2 is on sys.path, but doesn't exist

SR

-- 
Stefano Rivera
  http://tumbleweed.org.za/
  +1 415 683 3272

[toc] | [next] | [standalone]

#10851

[Multipart message — attachments visible in raw view] — view raw

Unsubscribed

On Mon, Aug 27, 2018 at 10:04 AM Stefano Rivera <stefanor@debian.org> wrote:

> So, I've had some pypy3 packages sitting around for a while, and the
> main thing that's stopped me from uploading, is figuring out a plan for
> byte-compilation.
>
> If it's sharing /usr/lib/python3/dist-packages with cPython (which it
> currently is) then we should figure out a plat for byte-compiling for
> it.
>
> rtupdate and postinst / prerm hooks for python3 packages currently call
> py3{compile,clean} which only operates on cpython3.
>
> Options I can think of:
> 1. Ignore byte-compilation entirely.
> 2. Change py3{compile,clean} to call pypy3{compile,clean} too.
>    And do some extra compiling / cleaning when pypy3 is installed and
>    removed.
> 3. Change py3{compile,clean} to do the work of pypy3{compile,clean} too.
>    And do some extra compiling / cleaning when pypy3 is installed and
>    removed.
> 4. Change the snipets dh_python3 generates, to include
>    pypy3{compile,clean}, when they exist. And again, do the extra
>    compiling / cleanup.
> 5. Something else?
>
> 3 *seems* like the right option - py3{compile,clean} supports a range of
> cpython versions, already. But I'm not really sure.
>
> Here's what I've got:
> https://salsa.debian.org/debian/pypy3
> https://people.debian.org/~stefanor/pypy3/
>
> Other TODOs that need some attention but possibly aren't blocking an
> upload to unstable:
> * test failures and errors
> * import paths aren't quite right, yet, e.g.:
>   - setup.py install installs to a /usr/local path that isn't on
>     sys.path
>   - plat-linux2 is on sys.path, but doesn't exist
>
> SR
>
> --
> Stefano Rivera
>   http://tumbleweed.org.za/
>   +1 415 683 3272
>
>

-- 
Kind regards
滕召智 (Ben teng)
Cell:   13381221392
MSN:   freet15@hotmail.com
Skype: freet15

[toc] | [prev] | [next] | [standalone]

#10854

> rtupdate and postinst / prerm hooks for python3 packages currently call
> py3{compile,clean} which only operates on cpython3.
> 
> 5. Something else?

modify these files and ship (renamed ones) in pypy3 package:
/usr/share/python3/runtime.d/public_modules.rtinstall
/usr/share/python3/runtime.d/public_modules.rtremove
-- 
GPG: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645

[toc] | [prev] | [next] | [standalone]

#10855

[Piotr Ożarowski, 2018-08-27]
> > rtupdate and postinst / prerm hooks for python3 packages currently call
> > py3{compile,clean} which only operates on cpython3.
> > 
> > 5. Something else?
> 
> modify these files and ship (renamed ones) in pypy3 package:
> /usr/share/python3/runtime.d/public_modules.rtinstall
> /usr/share/python3/runtime.d/public_modules.rtremove


err, you meant for python3-foo packages, so I guess option 3 is the best
one.
-- 
GPG: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645

[toc] | [prev] | [next] | [standalone]

#10856

> err, you meant for python3-foo packages, so I guess option 3 is the best
> one.

err again (sorry, to many things at once), I meant option 4 (I can
change dh-python to generate additional checks for pypy{compile,clean})
-- 
GPG: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645

[toc] | [prev] | [next] | [standalone]

#10858

Hi Piotr (2018.08.27_11:15:05_+0200)
> > err, you meant for python3-foo packages

Yes, I did.

> err again (sorry, to many things at once), I meant option 4 (I can
> change dh-python to generate additional checks for pypy{compile,clean})

That'd be pypy3{compile,clean}

OK, so, now to figure out the "magic" part (byte-compiling already
installed modules when pypy3 is installed / removed.

Should I run the runtime.d directory? Currently everything in that will
only do things with cpython bytecode.

Apps that aren't in dist-packages probably don't need pypy3 bytecode,
unless they want to be run with pypy.

Anything that tries to determine the version (e.g
/usr/share/python3/runtime.d/public_modules.rtinstall and
/usr/share/python3/runtime.d/public_modules.rtremove) looks like they
would break if I told them the version was pypy3.

So, I'm somewhat tempted to ignore runtime.d entirely, and just
byte-compile /usr/lib/python3/dist-packages. That seems pretty hacky,
though :/

SR

-- 
Stefano Rivera
  http://tumbleweed.org.za/
  +1 415 683 3272

[toc] | [prev] | [next] | [standalone]

#10947

[Multipart message — attachments visible in raw view] — view raw

> > err again (sorry, to many things at once), I meant option 4 (I can
> > change dh-python to generate additional checks for pypy{compile,clean})
>
> That'd be pypy3{compile,clean}

this is now available in dh-python 3.20180927. Only postinst files are
checking for pypy3compile, though. py3clean in prerm will remove .pyc
generated by PyPy3 as well (so no need to call pypy3clean in there)

> OK, so, now to figure out the "magic" part (byte-compiling already
> installed modules when pypy3 is installed / removed.
>
> Should I run the runtime.d directory? Currently everything in that will
> only do things with cpython bytecode.
>
> Apps that aren't in dist-packages probably don't need pypy3 bytecode,
> unless they want to be run with pypy.
>
> Anything that tries to determine the version (e.g
> /usr/share/python3/runtime.d/public_modules.rtinstall and
> /usr/share/python3/runtime.d/public_modules.rtremove) looks like they
> would break if I told them the version was pypy3.
>
> So, I'm somewhat tempted to ignore runtime.d entirely, and just
> byte-compile /usr/lib/python3/dist-packages. That seems pretty hacky,
> though :/

I think we should create a separate runtime.d for pypy3 and a separate one
for bcep files as well while we're at it.

FTR: bcep files are used to describe files that py3compile should avoid.
F.e. files meant for Python >= 3.6 ('async' keyword, etc.) should not be
byte-compiled using python3.5 (see dh_python3's manpage for details or
python3-jinja2 and pylint3 binary packages for examples).

Since we will share dist-package directory, we'll need to describe
exceptions for pypy3compile. One way is to extend version range syntax
in current /usr/share/python3/bcep/ files but I'd rather have separate
/usr/share/pypy/bcep/ used by pypy3compile.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web