Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #11971 > unrolled thread

Python module packages that don't bytecompile on installation?

Back to article view | Back to linux.debian.maint.python

Contents

  Python module packages that don't bytecompile on installation? Paul Wise <pabs@debian.org> - 2019-11-02 04:30 +0100
    Re: Python module packages that don't bytecompile on installation? Matthias Klose <doko@debian.org> - 2019-11-02 11:10 +0100
      Re: Python module packages that don't bytecompile on installation? Paul Wise <pabs@debian.org> - 2019-11-03 02:30 +0100
        Re: Python module packages that don't bytecompile on installation? Matthias Klose <doko@debian.org> - 2019-11-03 10:30 +0100
        Re: Python module packages that don't bytecompile on installation? Ansgar <ansgar@43-1.org> - 2019-11-04 09:10 +0100

#11971 — Python module packages that don't bytecompile on installation?

[Multipart message — attachments visible in raw view] — view raw

Hi all,

I run adequate on my system, which means I notice when Python module
packages don't bytecompile when they are installed. So far I've just
been ignoring the warnings that adequate prints.

https://salsa.debian.org/debian/adequate

In addition I noticed:

 * some Python modules on my system weren't bytecompiled but did
   bytecompile when I reinstalled them (blinker for example)
 * many of the Python modules on my system don't bytecompile their test
   directories (django for example)
 * relatively few packages I have installed don't bytecompile when I
   ignore test paths (lib2to3, tk, uno, distutils)

Some questions:

Should all module packages bytecompile?

Should all module packages bytecompile all their directories?

What are the downsides when module packages don't bytecompile?

What are the downsides when module packages do bytecompile?

Do we need a lintian complaint about this issue?

Do we need any improvements in how module packages bytecompile?
For example using triggers instead of postinst scriptlets.

Any other thoughts?

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

[toc] | [next] | [standalone]

#11974

On 02.11.19 04:22, Paul Wise wrote:
> Hi all,
> 
> I run adequate on my system, which means I notice when Python module
> packages don't bytecompile when they are installed. So far I've just
> been ignoring the warnings that adequate prints.
> 
> https://salsa.debian.org/debian/adequate
> 
> In addition I noticed:
> 
>   * some Python modules on my system weren't bytecompiled but did
>     bytecompile when I reinstalled them (blinker for example)
>   * many of the Python modules on my system don't bytecompile their test
>     directories (django for example)
>   * relatively few packages I have installed don't bytecompile when I
>     ignore test paths (lib2to3, tk, uno, distutils)
> 
> Some questions:
> 
> Should all module packages bytecompile?

At this point I'd ignore any Python2 related package, and concentrate on Python3 
stuff only.

> Should all module packages bytecompile all their directories?
> 
> What are the downsides when module packages don't bytecompile?
> 
> What are the downsides when module packages do bytecompile?

Byte compilation is an optimization, speeding up a program start if 
byte-compiled files are ready.  Packages are smaller without bc, take a bit 
longer to install.  I think nobody is asking the question if .py files should be 
dropped in favor of .pyc files.

> Do we need a lintian complaint about this issue?
> 
> Do we need any improvements in how module packages bytecompile?
> For example using triggers instead of postinst scriptlets.
> 
> Any other thoughts?

I'd say, there are currently more pressing issues than that, like the Python2 
removal, or the introduction of Python 3.8.  3.8 also offers a central directory 
for bc files, so that's maybe another thing to look at, but not a priority now 
from my point of view.

Matthias

[toc] | [prev] | [next] | [standalone]

#11975

On Sat, Nov 2, 2019 at 6:04 PM Matthias Klose wrote:

> At this point I'd ignore any Python2 related package, and concentrate on Python3
> stuff only.

Yes, I was referring only to python3-* module packages.

> Byte compilation is an optimization, speeding up a program start if
> byte-compiled files are ready.  Packages are smaller without bc, take a bit
> longer to install.  I think nobody is asking the question if .py files should be
> dropped in favor of .pyc files.

As I understand it, the .pyc files are interpreter version specific
and doing a transition for each major Python update was deemed to be
too much work, so instead we offload building the .pyc files onto
user's systems.

> I'd say, there are currently more pressing issues than that, like the Python2
> removal, or the introduction of Python 3.8.  3.8 also offers a central directory
> for bc files, so that's maybe another thing to look at, but not a priority now
> from my point of view.

Agreed re Python 2 etc. Eventually switching to using a central
directory in /var would be nice, right now the .pyc files are dumped
cruftily into /usr subdirs.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise
https://bonedaddy.net/pabs3/

[toc] | [prev] | [next] | [standalone]

#11976

On 03.11.19 02:20, Paul Wise wrote:
> On Sat, Nov 2, 2019 at 6:04 PM Matthias Klose wrote:
> 
>> At this point I'd ignore any Python2 related package, and concentrate on Python3
>> stuff only.
> 
> Yes, I was referring only to python3-* module packages.
> 
>> Byte compilation is an optimization, speeding up a program start if
>> byte-compiled files are ready.  Packages are smaller without bc, take a bit
>> longer to install.  I think nobody is asking the question if .py files should be
>> dropped in favor of .pyc files.
> 
> As I understand it, the .pyc files are interpreter version specific
> and doing a transition for each major Python update was deemed to be
> too much work, so instead we offload building the .pyc files onto
> user's systems.

yes, that's another reason. Not having to touch binary indep packages during a 
transition.

>> I'd say, there are currently more pressing issues than that, like the Python2
>> removal, or the introduction of Python 3.8.  3.8 also offers a central directory
>> for bc files, so that's maybe another thing to look at, but not a priority now
>> from my point of view.
> 
> Agreed re Python 2 etc. Eventually switching to using a central
> directory in /var would be nice, right now the .pyc files are dumped
> cruftily into /usr subdirs.

yes, but that's something when 3.7 is removed.

Matthias

[toc] | [prev] | [next] | [standalone]

#11980

Paul Wise writes:
> On Sat, Nov 2, 2019 at 6:04 PM Matthias Klose wrote:
>> I'd say, there are currently more pressing issues than that, like the Python2
>> removal, or the introduction of Python 3.8.  3.8 also offers a central directory
>> for bc files, so that's maybe another thing to look at, but not a priority now
>> from my point of view.
>
> Agreed re Python 2 etc. Eventually switching to using a central
> directory in /var would be nice, right now the .pyc files are dumped
> cruftily into /usr subdirs.

But /var is for variable state data, i.e. data that is modified during
regular system operation.  This isn't the case for *.pyc which follow
the same rules as the *.py included directly in the packages.

So these files should stay in /usr.  (I even think /var/lib/dpkg/info
should really be in /usr for similar reasons.)

See also http://0pointer.net/blog/projects/stateless.html for some other
reasons why not using /var might be an advantage.

Ansgar

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web