Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #12276 > unrolled thread

Separate lib and executable packages?

Back to article view | Back to linux.debian.maint.python

Contents

  Separate lib and executable packages? Gordon Ball <gordon@chronitis.net> - 2020-02-07 12:50 +0100
    Re: Separate lib and executable packages? Ondrej Novy <novy@ondrej.org> - 2020-02-07 13:50 +0100
    Re: Separate lib and executable packages? Mattia Rizzolo <mattia@debian.org> - 2020-02-07 17:10 +0100
      Re: Separate lib and executable packages? Scott Kitterman <debian@kitterman.com> - 2020-02-08 04:20 +0100
        Re: Separate lib and executable packages? Ondrej Novy <novy@ondrej.org> - 2020-02-08 14:00 +0100
          Re: Separate lib and executable packages? Scott Kitterman <debian@kitterman.com> - 2020-02-08 15:30 +0100
        Re: Separate lib and executable packages? Gordon Ball <gordon@chronitis.net> - 2020-02-08 21:00 +0100
          Re: Separate lib and executable packages? Ondrej Novy <novy@ondrej.org> - 2020-02-08 21:20 +0100
            Re: Separate lib and executable packages? Gordon Ball <gordon@chronitis.net> - 2020-02-11 11:00 +0100
          Re: Separate lib and executable packages? Scott Kitterman <debian@kitterman.com> - 2020-02-09 04:10 +0100

#12276 — Separate lib and executable packages?

The library style guide [1] says under _Executables and library
packages_:

> Here are some recommendations. We do not have a standard (though maybe
> we should)

regarding whether a library with an associated executable should be
split across `foo` and `python3-foo` or just bundled into `python3-foo`.
Does anyone consider that there is now a standard either way?

In eg, ipython this means shipping `ipython3` (containing a ~300 byte
entrypoints script and a similarly sized man page) as a separate
package to the implementation in `python3-ipython`.

I wonder if this split really makes sense; it feels like adding the
overhead of an extra binary package to avoid not having a very small
file in /usr/bin if you're only planning to use the library.

Does it seem reasonable to drop the executable package and just make it
a Provides: of the library package? (and is there any potential breakage
here that I'm overlooking)

Gordon

[1]: https://wiki.debian.org/Python/LibraryStyleGuide

[toc] | [next] | [standalone]

#12277

[Multipart message — attachments visible in raw view] — view raw

Hi,

it depends :). My two cents:

If you are packaging application (PAPT team):
* user is using it in CLI / Xorg
* user don't need to know it's written in Python
* Python modules are typically private, not usable by other
applications/libraries
-> one binary package: foo

If you are packaging library (DPMT team):
* your package is typically not usable by user directly
* your package is typically dependency of other packages (Python
applications/libraries)
-> as many binary packages as "language branches" we have, e.g. python-foo,
python3-foo, pypy-foo

If you are packaging application with public library part (PAPT team):
* both of above
* user is using it in CLI / Xorg
* BUT package have library part, which can be used in other Python
applications/libraries
-> as many binary packages as "language branches" we have, e.g. python-foo,
python3-foo, pypy-foo + foo for application. "foo" package can contain more
than entrypoints, but icons, menu items, ...

-- 
Best regards
 Ondřej Nový

[toc] | [prev] | [next] | [standalone]

#12278

[Multipart message — attachments visible in raw view] — view raw

On Fri, Feb 07, 2020 at 11:36:00AM +0000, Gordon Ball wrote:
> I wonder if this split really makes sense; it feels like adding the
> overhead of an extra binary package to avoid not having a very small
> file in /usr/bin if you're only planning to use the library.
> 
> Does it seem reasonable to drop the executable package and just make it
> a Provides: of the library package? (and is there any potential breakage
> here that I'm overlooking)


Maybe not for ipython3, since that's very much tied to python3-ipython3.

BUT, as a user (even forgetting I'm also a DD) I was hurt many times by
executables in python-foo but wanting to use python3-foo instead, or by
executables that moved from python-foo to python3-foo and I had to fix
my own deployments, and whatnot.

We are not going to have a python4 anytime soon (hopefully never), but I
think that keeping a separate binary package makes sense for almost all
cases I can think of packages shipping executables under /usr/bin/.

-- 
regards,
                        Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540      .''`.
More about me:  https://mapreri.org                             : :'  :
Launchpad user: https://launchpad.net/~mapreri                  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-

[toc] | [prev] | [next] | [standalone]

#12279

On February 7, 2020 3:59:46 PM UTC, Mattia Rizzolo <mattia@debian.org> wrote:
>On Fri, Feb 07, 2020 at 11:36:00AM +0000, Gordon Ball wrote:
>> I wonder if this split really makes sense; it feels like adding the
>> overhead of an extra binary package to avoid not having a very small
>> file in /usr/bin if you're only planning to use the library.
>> 
>> Does it seem reasonable to drop the executable package and just make
>it
>> a Provides: of the library package? (and is there any potential
>breakage
>> here that I'm overlooking)
>
>
>Maybe not for ipython3, since that's very much tied to
>python3-ipython3.
>
>BUT, as a user (even forgetting I'm also a DD) I was hurt many times by
>executables in python-foo but wanting to use python3-foo instead, or by
>executables that moved from python-foo to python3-foo and I had to fix
>my own deployments, and whatnot.
>
>We are not going to have a python4 anytime soon (hopefully never), but
>I
>think that keeping a separate binary package makes sense for almost all
>cases I can think of packages shipping executables under /usr/bin/.

Except: Every time we add a binary to Debian it impacts everyone.  The packages file gets bigger and so updates get slower.  Although there's no firm rule, the FTP Masters discourage addition of trivially small packages and so your package might be rejected.

As an example, the dkimpy module that I maintain provides some scripts.  Originally they were provided in python-dkim.  Now they are provided in python3-dkim (and since python-dkim is gone in unstable/testing, there's no more chance of users being confused about which one to install to get the scripts).

I recall a few questions by people that didn't know where to find the scripts.  I could have had both packages provide it using update-alternatives, but the problem never seemed severe enough to be worth the work.

While splitting things out as suggested makes sense from the perspective of a single package, it has negative implications for the distro as a whole.  Maintainers: please be mindful of the cumulative impact of the addition many small packages that aren't strictly needed.

Scott K

[toc] | [prev] | [next] | [standalone]

#12280

[Multipart message — attachments visible in raw view] — view raw

Hi,

so 8. 2. 2020 v 4:11 odesílatel Scott Kitterman <debian@kitterman.com>
napsal:

> Except: Every time we add a binary to Debian it impacts everyone.  The
> packages file gets bigger and so updates get slower.  Although there's no
> firm rule, the FTP Masters discourage addition of trivially small packages
> and so your package might be rejected.
>

so let's fix infrastructure?

-- 
Best regards
 Ondřej Nový

[toc] | [prev] | [next] | [standalone]

#12281

On February 8, 2020 12:38:54 PM UTC, Ondrej Novy <novy@ondrej.org> wrote:
>Hi,
>
>so 8. 2. 2020 v 4:11 odesílatel Scott Kitterman <debian@kitterman.com>
>napsal:
>
>> Except: Every time we add a binary to Debian it impacts everyone. 
>The
>> packages file gets bigger and so updates get slower.  Although
>there's no
>> firm rule, the FTP Masters discourage addition of trivially small
>packages
>> and so your package might be rejected.
>>
>
>so let's fix infrastructure?

That's what everyone always suggests, but no one ever does.  AFAIK, there's no agreed design for secure package updates that doesn't require a local copy of the packages file.

If Debian could quit worrying about adding lots of little binaries, it would help in many areas.

Scott K

[toc] | [prev] | [next] | [standalone]

#12282

On Sat, Feb 08, 2020 at 02:42:11AM +0000, Scott Kitterman wrote:
> 
> 
> On February 7, 2020 3:59:46 PM UTC, Mattia Rizzolo <mattia@debian.org> wrote:
> >On Fri, Feb 07, 2020 at 11:36:00AM +0000, Gordon Ball wrote:
> >> I wonder if this split really makes sense; it feels like adding the
> >> overhead of an extra binary package to avoid not having a very small
> >> file in /usr/bin if you're only planning to use the library.
> >> 
> >> Does it seem reasonable to drop the executable package and just make
> >it
> >> a Provides: of the library package? (and is there any potential
> >breakage
> >> here that I'm overlooking)
> >
> >
> >Maybe not for ipython3, since that's very much tied to
> >python3-ipython3.
> >
> >BUT, as a user (even forgetting I'm also a DD) I was hurt many times by
> >executables in python-foo but wanting to use python3-foo instead, or by
> >executables that moved from python-foo to python3-foo and I had to fix
> >my own deployments, and whatnot.
> >
> >We are not going to have a python4 anytime soon (hopefully never), but
> >I
> >think that keeping a separate binary package makes sense for almost all
> >cases I can think of packages shipping executables under /usr/bin/.
> 
> Except: Every time we add a binary to Debian it impacts everyone.  The packages file gets bigger and so updates get slower.  Although there's no firm rule, the FTP Masters discourage addition of trivially small packages and so your package might be rejected.
> 

Perhaps this is worth making an explicit recommendation for new packages
of this type, given that anything new _should_ be python3-only and we
hope for at least some time before another python major version becomes
an issue. "Packages which are primarily libraries but include an
executable should avoid multiple binary packages without good reason"? -
At least in the above linked style guide, since it's not normative
policy in any case.

For packages already split (eg, ipython3, python3-ipython), having
thought a bit more I suspect it's probably more trouble than it is worth
to drop a handful of binary packages - I'm wondering about eg, what
happens if the executable package is manually installed and the library
hence flagged as automatically installed; if the former than becomes a
Provides: of the latter, is it immediately a candidate for autoremoval?

> As an example, the dkimpy module that I maintain provides some scripts.  Originally they were provided in python-dkim.  Now they are provided in python3-dkim (and since python-dkim is gone in unstable/testing, there's no more chance of users being confused about which one to install to get the scripts).
> 
> I recall a few questions by people that didn't know where to find the scripts.  I could have had both packages provide it using update-alternatives, but the problem never seemed severe enough to be worth the work.
> 
> While splitting things out as suggested makes sense from the perspective of a single package, it has negative implications for the distro as a whole.  Maintainers: please be mindful of the cumulative impact of the addition many small packages that aren't strictly needed.
> 
> Scott K
> 

[toc] | [prev] | [next] | [standalone]

#12283

[Multipart message — attachments visible in raw view] — view raw

Hi,

so 8. 2. 2020 v 20:51 odesílatel Gordon Ball <gordon@chronitis.net> napsal:

> Perhaps this is worth making an explicit recommendation for new packages
> of this type, given that anything new _should_ be python3-only and we
>

and what about pypy- prefix?

-- 
Best regards
 Ondřej Nový

[toc] | [prev] | [next] | [standalone]

#12293

On Sat, Feb 08, 2020 at 08:54:27PM +0100, Ondrej Novy wrote:
> Hi,
> 
> so 8. 2. 2020 v 20:51 odesílatel Gordon Ball <gordon@chronitis.net> napsal:
> 
> > Perhaps this is worth making an explicit recommendation for new packages
> > of this type, given that anything new _should_ be python3-only and we
> >
> 
> and what about pypy- prefix?

That's a good point.

Probably not many packages are likely to provide pypy{,3} invoking binaries
(ipython is probably actually a good candidate here) and so it probably
counts as an exceptional case which can reasonably bypass a recommendation.

I suppose you could:

 * Not ship any executables which invoke pypy. For pypy3, that appears
   to be the case today (nothing in /usr/bin using pypy3 except the
   interpreter itself).
 * Ship the library and python3 executable together (possibly with a
   Provides: for the executable), and the pypy3 executable in a
   separate package (since the library itself presumably
   doesn't want to depend on pypy3, and might need to depend on
   pypy-specific library packages). This saves 1 binary package but
   seems inconsistent. The actual implementation could be picked with
   update-alternatives.
 * Ship library, python3 executable and pypy3 executable separately.
   This seems more consistent but generates an extra binary package.

This is probably a good case to consider in this thread because ipython
and ipykernel are probably reasonable cases where a pypy-specialised
executable might make sense, and ipython at least depends only on
pure-python modules which work out-of-the-box with pypy.

> 
> -- 
> Best regards
>  Ondřej Nový

[toc] | [prev] | [next] | [standalone]

#12284

[Multipart message — attachments visible in raw view] — view raw

On Saturday, February 8, 2020 2:50:54 PM EST Gordon Ball wrote:
> On Sat, Feb 08, 2020 at 02:42:11AM +0000, Scott Kitterman wrote:
> > On February 7, 2020 3:59:46 PM UTC, Mattia Rizzolo <mattia@debian.org> 
wrote:
> > >On Fri, Feb 07, 2020 at 11:36:00AM +0000, Gordon Ball wrote:
> > >> I wonder if this split really makes sense; it feels like adding the
> > >> overhead of an extra binary package to avoid not having a very small
> > >> file in /usr/bin if you're only planning to use the library.
> > >> 
> > >> Does it seem reasonable to drop the executable package and just make
> > >
> > >it
> > >
> > >> a Provides: of the library package? (and is there any potential
> > >
> > >breakage
> > >
> > >> here that I'm overlooking)
> > >
> > >Maybe not for ipython3, since that's very much tied to
> > >python3-ipython3.
> > >
> > >BUT, as a user (even forgetting I'm also a DD) I was hurt many times by
> > >executables in python-foo but wanting to use python3-foo instead, or by
> > >executables that moved from python-foo to python3-foo and I had to fix
> > >my own deployments, and whatnot.
> > >
> > >We are not going to have a python4 anytime soon (hopefully never), but
> > >I
> > >think that keeping a separate binary package makes sense for almost all
> > >cases I can think of packages shipping executables under /usr/bin/.
> > 
> > Except: Every time we add a binary to Debian it impacts everyone.  The
> > packages file gets bigger and so updates get slower.  Although there's no
> > firm rule, the FTP Masters discourage addition of trivially small
> > packages and so your package might be rejected.
> Perhaps this is worth making an explicit recommendation for new packages
> of this type, given that anything new _should_ be python3-only and we
> hope for at least some time before another python major version becomes
> an issue. "Packages which are primarily libraries but include an
> executable should avoid multiple binary packages without good reason"? -
> At least in the above linked style guide, since it's not normative
> policy in any case.

Sounds reasonable to me.  It's a wiki, so I'd say go for it.

> For packages already split (eg, ipython3, python3-ipython), having
> thought a bit more I suspect it's probably more trouble than it is worth
> to drop a handful of binary packages - I'm wondering about eg, what
> happens if the executable package is manually installed and the library
> hence flagged as automatically installed; if the former than becomes a
> Provides: of the latter, is it immediately a candidate for autoremoval?

Yes.  Removing the existing cases where it might not really be needed can be 
problematic.  I think it generally makes sense to leave them be.

Scott K

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web