Path: csiph.com!goblin2!goblin.stu.neva.ru!gothmog.csi.it!bofh.it!news.nic.it!robomod
From: Donald Stufft <donald@stufft.io>
Newsgroups: linux.debian.maint.python
Subject: Re: Dealing with flit -- a simplified packaging of python  modules
Date: Sat, 26 Sep 2015 05:30:01 +0200
Message-ID: <qcOlr-4Di-1@gated-at.bofh.it>
References: <qakhP-29v-11@gated-at.bofh.it> <qaldT-3Av-3@gated-at.bofh.it> <qatl7-6Oz-1@gated-at.bofh.it> <qaKFj-6va-5@gated-at.bofh.it> <qcc6u-q6-13@gated-at.bofh.it> <qccJc-1op-5@gated-at.bofh.it> <qcKBc-7G0-9@gated-at.bofh.it> <qcKBc-7G0-15@gated-at.bofh.it> <qcLxf-zG-1@gated-at.bofh.it>
Old-Return-Path: <donald@stufft.io>
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
Approved: robomod@news.nic.it
Lines: 82
Organization: linux.* mail to news gateway
Sender: robomod@news.nic.it
Xref: csiph.com linux.debian.maint.python:7318

On September 25, 2015 at 8:21:39 PM, Ben =46inney (ben+debian=40benfinney=
.id.au) wrote:
> Donald Stufft writes:
> =20
> > On September 25, 2015 at 7:24:30 PM, Paul Wise (pabs=40debian.org) wr=
ote:
> > > Why are end users using source packages instead of binary packages
> > > and then complaining that the source tarballs aren't ready-to-run
> > > binary packages=3F
> >
> > Because the way Python packaging currently is and historically has
> > been, binary packages are not something that is widely available or
> > viable.
> =20
> So, commendable effort has gone toward making that problem reduce,
> perhaps eventually to go away. Congratulations are deserved for getting=

> us this far, and thanks for continuing to push for feasible binary
> distributions of Python packages on all supported OSen.
> =20
> It seems reasonable, then, to also put effort toward making source
> distributions more targeted toward being bundles of the released
> *source* for the project version, and reducing the justification for
> bundling non-source files in the source distribution.

I think distutils/setuptools is to blame for a lot of this, they don't ma=
ke it
easy to have generated files that aren't either compiled C code or py(c=7C=
o).
Ideally in the future distutils/setuptools will no longer be special (exc=
ept as
a historical and legacy fallback) and we'll have something far more plugg=
able
in the future. This will ideally make it a lot easier and sane to have cu=
stom
build steps that create other generated files as part of the building of =
a
package.

> =20
> Can we expect the value of =E2=80=9Cbut people expect non-source stuff =
bundled
> in the source distrubution=E2=80=9D to diminish, as some kind of object=
ion to
> making proper source distributions=3F

It's certainly a goal of mine that we move as much of the upstream Python=

packaging onto binary packages so that most people are not installing fro=
m
sdists or really, =22installing=22 from an sdist being an outdated concep=
t and
long term, the only thing we do with an sdist is turn it into a binary pa=
ckage,
even if that build step is taken on the end user's computer. I of course,=
 can't
speak to everyone so it would not surprise me if some people simply will =
never
be willing to not distributed generated files as part of their sdist.

I suspect this to be a long road though, and certainly the nature of *nix=
 makes
it harder on us, since it's hard to make a binary package that works for =
say,
all of Linux due to ABI problems and such. The more cases where =60=60pip=
 install=60=60
has end users pulling from a sdist, the more resistence I suspect you'll =
find
from authors to making sdist have more complicated build steps.

I suspect you'll have an easier time convinving authors of pure Python th=
ings
that include some generated files (translations, js, whatever) to rely en=
tirely
on wheels than you will have convincing authors of Python things that inc=
lude
some C or other compiled code that precludes universal(ish) Wheels.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DC=46A // 7C6B 7C5D 5E2B 6356 A926 =4604=46 6E3C BCE9 =
3372 DC=46A