Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #12913 > unrolled thread

How to watch pypi.org

Back to article view | Back to linux.debian.maint.python

Contents

  How to watch pypi.org Fioddor Superconcentrado <fioddor@gmail.com> - 2020-10-04 17:30 +0200
    Re: How to watch pypi.org Geert Stappers <stappers@stappers.nl> - 2020-10-04 17:50 +0200
      Re: How to watch pypi.org Fioddor Superconcentrado <fioddor@gmail.com> - 2020-10-30 15:10 +0100
    Re: How to watch pypi.org Andrey Rahmatullin <wrar@debian.org> - 2020-10-04 17:50 +0200
      Re: How to watch pypi.org Fioddor Superconcentrado <fioddor@gmail.com> - 2020-10-30 15:10 +0100
    Re: How to watch pypi.org Paul Wise <pabs@debian.org> - 2020-10-05 04:30 +0200
      Re: How to watch pypi.org Scott Kitterman <debian@kitterman.com> - 2020-10-05 05:10 +0200
        Re: How to watch pypi.org, example Geert Stappers <stappers@stappers.nl> - 2020-10-05 08:20 +0200
      Re: How to watch pypi.org Fioddor Superconcentrado <fioddor@gmail.com> - 2020-10-30 15:20 +0100
        Re: How to watch pypi.org Andrey Rahmatullin <wrar@debian.org> - 2020-10-30 15:30 +0100
        Re: How to watch pypi.org Paul Wise <pabs@debian.org> - 2020-10-31 02:40 +0100
          Re: How to watch pypi.org Jeremy Stanley <fungi@yuggoth.org> - 2020-10-31 03:40 +0100
            Re: How to watch pypi.org Paul Wise <pabs@debian.org> - 2020-10-31 05:30 +0100
        Re: How to watch pypi.org Thomas Goirand <zigo@debian.org> - 2020-10-31 12:10 +0100
          Re: How to watch pypi.org Jeremy Stanley <fungi@yuggoth.org> - 2020-10-31 13:20 +0100
            Re: How to watch pypi.org Thomas Goirand <zigo@debian.org> - 2020-11-01 20:30 +0100
              Re: How to watch pypi.org Jeremy Stanley <fungi@yuggoth.org> - 2020-11-01 21:10 +0100
          Re: How to watch pypi.org Andrey Rahmatullin <wrar@debian.org> - 2020-10-31 15:50 +0100
            Re: How to watch pypi.org Jeremy Stanley <fungi@yuggoth.org> - 2020-10-31 18:40 +0100

#12913 — How to watch pypi.org

[Multipart message — attachments visible in raw view] — view raw

I've packaged a project provided via https://pipi.org and I wanted to
create a debian/watch file but pipi.org publishes the tarball behind a
strange url like

https://files.pythonhosted.org/packages/d6/72/9848a2d631dad70d7ea582540f0619e1a7ecf31b3a117de9d9f2b6b28029/django-settings-export-1.2.1.tar.gz

I guess pypi.org is a common source of python code so perhaps there's
already a solution for this.

Thanks in advanced.

[toc] | [next] | [standalone]

#12917

On Sun, Oct 04, 2020 at 05:28:57PM +0200, Fioddor Superconcentrado wrote:
> I've packaged a project provided via https://pipi.org and I wanted to
> create a debian/watch file but pipi.org publishes the tarball behind a
> strange url like
> 
> https://files.pythonhosted.org/packages/d6/72/9848a2d631dad70d7ea582540f0619e1a7ecf31b3a117de9d9f2b6b28029/django-settings-export-1.2.1.tar.gz
> 
> I guess pypi.org is a common source of python code so perhaps there's
> already a solution for this.

Quoting the manual page of uscan.
(  uscan is the utility reads debian/watch for watching. )


   PyPI
       For PyPI based projects, pypi.debian.net runs a redirector
       which allows a simpler form of URL. The format below will
       automatically be rewritten to use the redirector with the watch
       file:

         version=4
         https://pypi.python.org/packages/source/<initial>/<project>/ \
             <tar-name>-(.+)\.tar\.gz debian uupdate

       For cfn-sphere, set the watch file as:

         version=4
         https://pypi.python.org/packages/source/c/cfn-sphere/ \
             cfn-sphere-([\d\.]+).tar.gz debian uupdate

       Please note, you can still use normal functionalities of uscan
       to set up a watch file for this site without using the
       redirector.

         version=4
         opts="pgpmode=none" \
             https://pypi.python.org/pypi/cfn-sphere/ \
             https://pypi.python.org/packages/.*/.*/.*/\
             cfn-sphere-([\d\.]+).tar.gz#.* debian uupdate


 
> Thanks in advanced.

Looking forward to something like "OK, that works"


Regards
Geert Stappers
-- 
Silence is hard to parse

[toc] | [prev] | [next] | [standalone]

#12983

[Multipart message — attachments visible in raw view] — view raw

Thank you as well, Geert.
And yes. It has taken me some time to come back to this issue but I've
tested and seems to work fine.

El dom., 4 oct. 2020 a las 17:44, Geert Stappers (<stappers@stappers.nl>)
escribió:

> On Sun, Oct 04, 2020 at 05:28:57PM +0200, Fioddor Superconcentrado wrote:
> > I've packaged a project provided via https://pipi.org and I wanted to
> > create a debian/watch file but pipi.org publishes the tarball behind a
> > strange url like
> >
> >
> https://files.pythonhosted.org/packages/d6/72/9848a2d631dad70d7ea582540f0619e1a7ecf31b3a117de9d9f2b6b28029/django-settings-export-1.2.1.tar.gz
> >
> > I guess pypi.org is a common source of python code so perhaps there's
> > already a solution for this.
>
> Quoting the manual page of uscan.
> (  uscan is the utility reads debian/watch for watching. )
>
>
>    PyPI
>        For PyPI based projects, pypi.debian.net runs a redirector
>        which allows a simpler form of URL. The format below will
>        automatically be rewritten to use the redirector with the watch
>        file:
>
>          version=4
>          https://pypi.python.org/packages/source/<initial>/<project>/ \
>              <tar-name>-(.+)\.tar\.gz debian uupdate
>
>        For cfn-sphere, set the watch file as:
>
>          version=4
>          https://pypi.python.org/packages/source/c/cfn-sphere/ \
>              cfn-sphere-([\d\.]+).tar.gz debian uupdate
>
>        Please note, you can still use normal functionalities of uscan
>        to set up a watch file for this site without using the
>        redirector.
>
>          version=4
>          opts="pgpmode=none" \
>              https://pypi.python.org/pypi/cfn-sphere/ \
>              https://pypi.python.org/packages/.*/.*/.*/\
>              cfn-sphere-([\d\.]+).tar.gz#.* debian uupdate
>
>
>
> > Thanks in advanced.
>
> Looking forward to something like "OK, that works"
>
>
> Regards
> Geert Stappers
> --
> Silence is hard to parse
>
>

[toc] | [prev] | [next] | [standalone]

#12918

[Multipart message — attachments visible in raw view] — view raw

On Sun, Oct 04, 2020 at 05:28:57PM +0200, Fioddor Superconcentrado wrote:
> I've packaged a project provided via https://pipi.org and I wanted to
> create a debian/watch file but pipi.org publishes the tarball behind a
> strange url like
> 
> https://files.pythonhosted.org/packages/d6/72/9848a2d631dad70d7ea582540f0619e1a7ecf31b3a117de9d9f2b6b28029/django-settings-export-1.2.1.tar.gz
> 
> I guess pypi.org is a common source of python code so perhaps there's
> already a solution for this.
The solution (that transparently uses pypi.debian.net) is described in
uscan(1).

-- 
WBR, wRAR

[toc] | [prev] | [next] | [standalone]

#12984

[Multipart message — attachments visible in raw view] — view raw

Thank you very much, Andrey.

El dom., 4 oct. 2020 a las 17:42, Andrey Rahmatullin (<wrar@debian.org>)
escribió:

> On Sun, Oct 04, 2020 at 05:28:57PM +0200, Fioddor Superconcentrado wrote:
> > I've packaged a project provided via https://pipi.org and I wanted to
> > create a debian/watch file but pipi.org publishes the tarball behind a
> > strange url like
> >
> >
> https://files.pythonhosted.org/packages/d6/72/9848a2d631dad70d7ea582540f0619e1a7ecf31b3a117de9d9f2b6b28029/django-settings-export-1.2.1.tar.gz
> >
> > I guess pypi.org is a common source of python code so perhaps there's
> > already a solution for this.
> The solution (that transparently uses pypi.debian.net) is described in
> uscan(1).
>
> --
> WBR, wRAR
>

[toc] | [prev] | [next] | [standalone]

#12924

On Sun, Oct 4, 2020 at 3:29 PM Fioddor Superconcentrado wrote:

> I've packaged a project provided via https://pipi.org and I wanted to create a debian/watch file but pipi.org publishes the tarball behind a strange url like

I would suggest using the upstream git repo instead of the PyPi tarballs.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

[toc] | [prev] | [next] | [standalone]

#12925

On Sunday, October 4, 2020 10:24:22 PM EDT Paul Wise wrote:
> On Sun, Oct 4, 2020 at 3:29 PM Fioddor Superconcentrado wrote:
> > I've packaged a project provided via https://pipi.org and I wanted to
> > create a debian/watch file but pipi.org publishes the tarball behind a
> > strange url like
> I would suggest using the upstream git repo instead of the PyPi tarballs.

I think that's a different argument.

There is a pypi redirector for Debian watch files.  Something like this works 
(this is from the pyspf package):

https://pypi.debian.net/pyspf/pyspf-([0-9][0-9t\.\-]*).tar.gz

This currently works, but no guarantee for how long:

https://pypi.python.org/packages/source/x/@PACKAGE@/
@PACKAGE@@ANY_VERSION@@ARCHIVE_EXT@

Scott K

[toc] | [prev] | [next] | [standalone]

#12926 — Re: How to watch pypi.org, example

On Sun, Oct 04, 2020 at 10:53:47PM -0400, Scott Kitterman wrote:
> On Sunday, October 4, 2020 10:24:22 PM EDT Paul Wise wrote:
> > On Sun, Oct 4, 2020 at 3:29 PM Fioddor Superconcentrado wrote:
> > > I've packaged a project provided via https://pipi.org and I wanted to
> > > create a debian/watch file but pipi.org publishes the tarball behind a
> > > strange url like
> > I would suggest using the upstream git repo instead of the PyPi tarballs.
> 
> I think that's a different argument.
> 
> There is a pypi redirector for Debian watch files.  Something like this works 
> (this is from the pyspf package):
> 
> https://pypi.debian.net/pyspf/pyspf-([0-9][0-9t\.\-]*).tar.gz
> 
> This currently works, but no guarantee for how long:
> 
> https://pypi.python.org/packages/source/x/@PACKAGE@/@PACKAGE@@ANY_VERSION@@ARCHIVE_EXT@
 https://pypi.python.org/packages/source/x/@PACKAGE@/@PACKAGE@-@ANY_VERSION@@ARCHIVE_EXT@



And learning is also learning from examples.


<screenshot>
stappers@paddy:~
$ cd packaging_dir/
stappers@paddy:packaging_dir
$ debcheckout pyspf
declared git repository at
https://salsa.debian.org/python-team/modules/pyspf.git
git clone https://salsa.debian.org/python-team/modules/pyspf.git pyspf
...
Cloning into 'pyspf'...
remote: Enumerating objects: 1039, done.
remote: Counting objects: 100% (1039/1039), done.
remote: Compressing objects: 100% (368/368), done.
remote: Total 1039 (delta 612), reused 943 (delta 551), pack-reused 0
Receiving objects: 100% (1039/1039), 237.12 KiB | 415.00 KiB/s, done.
Resolving deltas: 100% (612/612), done.
stappers@paddy:packaging_dir
$ cd pyspf
stappers@paddy:packaging_dir/pyspf
$ cat debian/watch 
version=3
https://pypi.debian.net/pyspf/pyspf-([0-9][0-9t\.\-]*).tar.gz
stappers@paddy:packaging_dir/pyspf
$ 
</screenshot>


Regards
Geert Stappers
-- 
Silence is hard to parse

[toc] | [prev] | [next] | [standalone]

#12985

[Multipart message — attachments visible in raw view] — view raw

As I said I'm very new to this and all (python) packages I'm using lately
use the usual python tools (pipy, setup.py, etc) and my first approach has
been to stick as close as possible to the upstream procedures. But I might
very likely be taking a wrong decision. What are the reasons to go for git
instead of pypi? I see that it is 'more upstream' but it seems that
everyone else is pointing to pypi as a distro-agnostic solution.

El lun., 5 oct. 2020 a las 4:24, Paul Wise (<pabs@debian.org>) escribió:

> On Sun, Oct 4, 2020 at 3:29 PM Fioddor Superconcentrado wrote:
>
> > I've packaged a project provided via https://pipi.org and I wanted to
> create a debian/watch file but pipi.org publishes the tarball behind a
> strange url like
>
> I would suggest using the upstream git repo instead of the PyPi tarballs.
>
> --
> bye,
> pabs
>
> https://wiki.debian.org/PaulWise
>
>

[toc] | [prev] | [next] | [standalone]

#12986

[Multipart message — attachments visible in raw view] — view raw

On Fri, Oct 30, 2020 at 03:18:58PM +0100, Fioddor Superconcentrado wrote:
> As I said I'm very new to this and all (python) packages I'm using lately
> use the usual python tools (pipy, setup.py, etc) and my first approach has
> been to stick as close as possible to the upstream procedures. But I might
> very likely be taking a wrong decision. What are the reasons to go for git
> instead of pypi? I see that it is 'more upstream' but it seems that
> everyone else is pointing to pypi as a distro-agnostic solution.
One reason is tarballs may be missing some files, including tests.

> El lun., 5 oct. 2020 a las 4:24, Paul Wise (<pabs@debian.org>) escribió:
> 
> > On Sun, Oct 4, 2020 at 3:29 PM Fioddor Superconcentrado wrote:
> >
> > > I've packaged a project provided via https://pipi.org and I wanted to
> > create a debian/watch file but pipi.org publishes the tarball behind a
> > strange url like
> >
> > I would suggest using the upstream git repo instead of the PyPi tarballs.
> >

-- 
WBR, wRAR

[toc] | [prev] | [next] | [standalone]

#12987

On Fri, Oct 30, 2020 at 2:19 PM Fioddor Superconcentrado wrote:

> As I said I'm very new to this and all (python) packages I'm using lately use the usual python tools (pipy, setup.py, etc) and my first approach has been to stick as close as possible to the upstream procedures. But I might very likely be taking a wrong decision. What are the reasons to go for git instead of pypi? I see that it is 'more upstream' but it seems that everyone else is pointing to pypi as a distro-agnostic solution.

As Andrey says, missing files is one issue, another is that tarballs
often contain extra generated files that should be built from source,
but if you use the tarball then they quite likely will not be built
from source.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

[toc] | [prev] | [next] | [standalone]

#12988

[Multipart message — attachments visible in raw view] — view raw

On 2020-10-31 01:33:36 +0000 (+0000), Paul Wise wrote:
> On Fri, Oct 30, 2020 at 2:19 PM Fioddor Superconcentrado wrote:
> > As I said I'm very new to this and all (python) packages I'm
> > using lately use the usual python tools (pipy, setup.py, etc)
> > and my first approach has been to stick as close as possible to
> > the upstream procedures. But I might very likely be taking a
> > wrong decision. What are the reasons to go for git instead of
> > pypi? I see that it is 'more upstream' but it seems that
> > everyone else is pointing to pypi as a distro-agnostic solution.
> 
> As Andrey says, missing files is one issue, another is that tarballs
> often contain extra generated files that should be built from source,
> but if you use the tarball then they quite likely will not be built
> from source.

I have to agree, though in the upstream projects with which I'm
involved, those generated files are basically a lossy re-encoding of
metadata from the Git repositories themselves: AUTHORS file
generated from committer headers, ChangeLog files from commit
subjects, version information from tag names, and so on. Some of
this information may be referenced from copyright licenses, so it's
important in those cases for package maintainers to generate it when
making their source packages if not using the sdist tarballs
published by the project.
-- 
Jeremy Stanley

[toc] | [prev] | [next] | [standalone]

#12989

On Sat, Oct 31, 2020 at 2:31 AM Jeremy Stanley wrote:

> I have to agree, though in the upstream projects with which I'm
> involved, those generated files are basically a lossy re-encoding of
> metadata from the Git repositories themselves: AUTHORS file
> generated from committer headers, ChangeLog files from commit
> subjects, version information from tag names, and so on. Some of
> this information may be referenced from copyright licenses, so it's
> important in those cases for package maintainers to generate it when
> making their source packages if not using the sdist tarballs
> published by the project.

As the maintainer of the autorevision package (which aims at dumping a
cache of VCS version metadata, for when exporting a tarball from a
VCS), I've been thinking about this for a while now. I've been
thinking about modifying automake (and other build tools) to have a
mode that basically does `git archive` instead of just calling tar and
also creates separate tarballs for all the other generated or embedded
files. One for the autorevision metadata, one for the autotools cruft,
one for a cache of the data needed for AUTHORS/ChangeLog/NEWS etc and
possibly other ones. Then Debian can use our multi-component quilt 3.0
format to take the git archive, the autorevision metadata, and the
cache of the VCS data, but leave the autotools cruft behind. Then we
can audit the git repo for generated files and embedded data/code
copies and when there are none, we can confidently build the configure
script from source knowing that everything required is available in
the build-deps. The same could apply to projects uploading to NPM or
PyPi, although I'm not sure if those support this sort of thing
though.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

[toc] | [prev] | [next] | [standalone]

#12990

On 10/30/20 3:18 PM, Fioddor Superconcentrado wrote:
> As I said I'm very new to this and all (python) packages I'm using
> lately use the usual python tools (pipy, setup.py, etc) and my first
> approach has been to stick as close as possible to the upstream
> procedures. But I might very likely be taking a wrong decision. What are
> the reasons to go for git instead of pypi? I see that it is 'more
> upstream' but it seems that everyone else is pointing to pypi as a
> distro-agnostic solution.

Pypi is often thought as a Python module source repository. It is *NOT*.
It is a repository for binaries to be consumed by pip. Therefore, the
tarballs may omit some artifacts, and add some more. For example, it's
preferable that the sources do not include the egg-info folder, or
pre-generated documentation. These must be built from source.

On 10/31/20 3:07 AM, Jeremy Stanley wrote:
> I have to agree, though in the upstream projects with which I'm
> involved, those generated files are basically a lossy re-encoding of
> metadata from the Git repositories themselves: AUTHORS file
> generated from committer headers, ChangeLog files from commit
> subjects, version information from tag names, and so on. Some of
> this information may be referenced from copyright licenses, so it's
> important in those cases for package maintainers to generate it when
> making their source packages if not using the sdist tarballs
> published by the project.

Unfortunately, the FTP masters do not agree with you. I've been told
that the OpenStack changelog is a way too big, and it's preferable to
not have it in the binary packages. Also, there's nothing in the Apache
license that mandates having an AUTHORS list as per what PBR builds. If
we are to care that much in OpenStack, then the license must be changed.

In Debian, quite the opposite, and like it or not (I personally don't
really feel the policy is right, but that's how it is), what Debian
cares is what's in the source code marked with "copyright (c) yeah,
copyright-holder", and this must already be in debian/copyright.

Cheers,

Thomas Goirand (zigo)

[toc] | [prev] | [next] | [standalone]

#12991

[Multipart message — attachments visible in raw view] — view raw

On 2020-10-31 12:03:50 +0100 (+0100), Thomas Goirand wrote:
[...]
> On 10/31/20 3:07 AM, Jeremy Stanley wrote:
> > I have to agree, though in the upstream projects with which I'm
> > involved, those generated files are basically a lossy re-encoding of
> > metadata from the Git repositories themselves: AUTHORS file
> > generated from committer headers, ChangeLog files from commit
> > subjects, version information from tag names, and so on. Some of
> > this information may be referenced from copyright licenses, so it's
> > important in those cases for package maintainers to generate it when
> > making their source packages if not using the sdist tarballs
> > published by the project.
> 
> Unfortunately, the FTP masters do not agree with you. I've been told
> that the OpenStack changelog is a way too big, and it's preferable to
> not have it in the binary packages.

PBR started creating much smaller changelogs years ago, after you
asked ftpmaster. I get that you see no value in changelog files, but
it seems like it would be worth revisiting.

> Also, there's nothing in the Apache license that mandates having
> an AUTHORS list as per what PBR builds. If we are to care that
> much in OpenStack, then the license must be changed.
[...]

I agree, it's not commonplace in OpenStack other than this possible
exception:

https://opendev.org/openstack/python-openstackclient/src/branch/master/doc/source/cli/man/openstack.rst#user-content-copyright

You do tend to find it in other Python projects however, for
example:

https://github.com/pygments/pygments/blob/master/LICENSE#L1

My point was that, in general, some Python projects do autogenerate
an AUTHORS file from commit metadata at dist time rather than
storing it directly in a file within their Git repositories, and
some projects (including Python projects) refer to AUTHORS from
copyright statements, so it's a good idea to build/keep it.
-- 
Jeremy Stanley

[toc] | [prev] | [next] | [standalone]

#12999

On 10/31/20 1:10 PM, Jeremy Stanley wrote:
> On 2020-10-31 12:03:50 +0100 (+0100), Thomas Goirand wrote:
> [...]
>> On 10/31/20 3:07 AM, Jeremy Stanley wrote:
>>> I have to agree, though in the upstream projects with which I'm
>>> involved, those generated files are basically a lossy re-encoding of
>>> metadata from the Git repositories themselves: AUTHORS file
>>> generated from committer headers, ChangeLog files from commit
>>> subjects, version information from tag names, and so on. Some of
>>> this information may be referenced from copyright licenses, so it's
>>> important in those cases for package maintainers to generate it when
>>> making their source packages if not using the sdist tarballs
>>> published by the project.
>>
>> Unfortunately, the FTP masters do not agree with you. I've been told
>> that the OpenStack changelog is a way too big, and it's preferable to
>> not have it in the binary packages.
> 
> PBR started creating much smaller changelogs years ago, after you
> asked ftpmaster. I get that you see no value in changelog files, but
> it seems like it would be worth revisiting.

Probably it'd be worth revisiting indeed, especially by finding a way to
push the logs only in the relevant -doc packages. Maybe by using the
same kind of trick which I want to do with the release notes (see
below), ie storing these files in the debian folder.

However, if I am to put more efforts on stuff like that, my priority
would be first on getting the reno release notes published in the Debian
package. I've been thinking about this for a long time, and haven't
figured out yet what would be the best way, with a reasonable workflow.

>From the Debian perspective, I'd have to:
- generate the release notes from the last version in Debian Stable, up
to what's in Sid. For example, I would include all the OpenStack release
notes for Stein, Train, Ussuri and Victoria in all packages uploaded for
Debian Bullseye, as this would concern anyone upgrading from Buster.
- make it so that release notes would be generated from Git, and maybe
stored in a debian/release-notes folder, so that it wouldn't generate a
diff with the original upstream tag.

The drawback would be that, on each upload of a new Debian revision, the
debian.tar.xz will contain the release notes, which could be of
significant size (especially if they include static objects like CSS,
JS, and all what makes a theme).

If you have any other suggestion concerning how to handle these release
notes, please let me know.

Cheers,

Thomas Goirand (zigo)

[toc] | [prev] | [next] | [standalone]

#13000

[Multipart message — attachments visible in raw view] — view raw

On 2020-11-01 20:23:20 +0100 (+0100), Thomas Goirand wrote:
[...]
> However, if I am to put more efforts on stuff like that, my priority
> would be first on getting the reno release notes published in the Debian
> package. I've been thinking about this for a long time, and haven't
> figured out yet what would be the best way, with a reasonable workflow.
> 
> From the Debian perspective, I'd have to:
> - generate the release notes from the last version in Debian Stable, up
> to what's in Sid. For example, I would include all the OpenStack release
> notes for Stein, Train, Ussuri and Victoria in all packages uploaded for
> Debian Bullseye, as this would concern anyone upgrading from Buster.
> - make it so that release notes would be generated from Git, and maybe
> stored in a debian/release-notes folder, so that it wouldn't generate a
> diff with the original upstream tag.
> 
> The drawback would be that, on each upload of a new Debian revision, the
> debian.tar.xz will contain the release notes, which could be of
> significant size (especially if they include static objects like CSS,
> JS, and all what makes a theme).
> 
> If you have any other suggestion concerning how to handle these release
> notes, please let me know.

It's likely I'm missing some subtle reason for the complexities you
outline above, but if you install python3-reno and then run `reno
report` in the upstream Git repository for any project with a
releasenotes tree (or pass the path to said Git repository in the
command line) it will generate a reStructuredText compilation of the
release notes contained therein. Very lightweight, no need for extra
files or anything. I'd think you could just dump that output into a
NEWS file or similar at binary package build time. This is basically
the same thing reno's Sphinx extension does under the covers.

Check out `reno report --help` for a number of flags you might want
to pass it to make the results more readable like omitting the
source filename comments, skipping notes earlier than a certain
version, collapsing pre-release notes, and so on. A quick test of
Nova's release notes indicates that even if you don't truncate them
though and include everything back to when the project started using
reno 5 years ago, that NEWS file would only increase the compressed
size of the nova-doc package by 1%.
-- 
Jeremy Stanley

[toc] | [prev] | [next] | [standalone]

#12992

[Multipart message — attachments visible in raw view] — view raw

On Sat, Oct 31, 2020 at 12:03:50PM +0100, Thomas Goirand wrote:
> Pypi is often thought as a Python module source repository. It is *NOT*.
> It is a repository for binaries to be consumed by pip.
Oooh, that's a very interesting thought I never considered.

-- 
WBR, wRAR

[toc] | [prev] | [next] | [standalone]

#12993

[Multipart message — attachments visible in raw view] — view raw

On 2020-10-31 19:48:29 +0500 (+0500), Andrey Rahmatullin wrote:
> On Sat, Oct 31, 2020 at 12:03:50PM +0100, Thomas Goirand wrote:
> > Pypi is often thought as a Python module source repository. It
> > is *NOT*. It is a repository for binaries to be consumed by pip.
> 
> Oooh, that's a very interesting thought I never considered.

It's not entirely accurate, however. These days there are two
remaining package formats commonly distributed through PyPI: sdists
and wheels. The sdist format is intended to be a
platform-independent "source distribution" (hence its name), perhaps
more analogous to Debian's own source package formats, and the
traditional setup.py in many sdists is akin to debian/rules and the
$PACKAGE.egg-info/ tree similar to other sorts of metadata you would
expect under the debian/ directory. Python package installation
tools like pip call libraries to do things like compile and link
included C extensions from the unpacked sdist before installing the
results into a usable location in the system (or more recently,
putting the results into a wheel package, then caching that and
installing its contents into the system).

When it comes to "binaries" this is definitely the domain of wheels.
A wheel is (usually, with the exception of toolchains like flit)
built from an sdist and may be platform-dependent, especially if it
contains compiled extensions. The wheel is much more akin to
Debian's binary package format.

The main operating system distribution package maintainer argument
against relying on sdists is that they may omit files from the
upstream revision control system which that upstream did not want
included in their official source distributions, or may include
extra generated files which upstream did want included but don't
exist (or at least don't exist as files in that form) within the
upstream revision control. This is perhaps not entirely dissimilar
from C/autotools based projects having a `make dist` target which
they use to prepare their source distribution tarballs. Whether it
actually represents a problem for downstream packaging likely varies
a bit from project to project.
-- 
Jeremy Stanley

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web