Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #13325 > unrolled thread

Build an initial Debian source package with py2dsp from a GitHub project

Back to article view | Back to linux.debian.maint.python

Contents

  Build an initial Debian source package with py2dsp from a GitHub project Sandro Tosi <morph@debian.org> - 2021-04-25 06:30 +0200
    Re: Build an initial Debian source package with py2dsp from a GitHub  project Antonio Terceiro <terceiro@debian.org> - 2021-04-25 14:40 +0200
      Re: Build an initial Debian source package with py2dsp from a GitHub project Sandro Tosi <morph@debian.org> - 2021-04-26 00:30 +0200
        Re: Build an initial Debian source package with py2dsp from a GitHub  project Antonio Terceiro <terceiro@debian.org> - 2021-04-26 01:20 +0200
          Re: Build an initial Debian source package with py2dsp from a GitHub project Sandro Tosi <morph@debian.org> - 2021-04-26 03:30 +0200
    Re: Build an initial Debian source package with py2dsp from a GitHub project Utkarsh Gupta <utkarsh@debian.org> - 2021-05-05 20:40 +0200
      Re: Build an initial Debian source package with py2dsp from a GitHub  project Andrey Rahmatullin <wrar@debian.org> - 2021-05-05 21:00 +0200
        Re: Build an initial Debian source package with py2dsp from a GitHub project Sandro Tosi <morph@debian.org> - 2021-05-05 21:10 +0200
          Re: Build an initial Debian source package with py2dsp from a GitHub project Utkarsh Gupta <utkarsh@debian.org> - 2021-05-05 21:50 +0200
            Re: Build an initial Debian source package with py2dsp from a GitHub project Utkarsh Gupta <utkarsh@debian.org> - 2021-05-05 22:00 +0200
              Re: Build an initial Debian source package with py2dsp from a GitHub project Sandro Tosi <morph@debian.org> - 2021-05-06 00:20 +0200
                Re: Build an initial Debian source package with py2dsp from a GitHub project Paul Wise <pabs@debian.org> - 2021-05-06 00:50 +0200
                  Re: Build an initial Debian source package with py2dsp from a GitHub project Sandro Tosi <morph@debian.org> - 2021-05-06 01:10 +0200
            Re: Build an initial Debian source package with py2dsp from a GitHub  project Andrey Rahmatullin <wrar@debian.org> - 2021-05-05 22:00 +0200
              Re: Build an initial Debian source package with py2dsp from a GitHub project Paul Wise <pabs@debian.org> - 2021-05-06 00:50 +0200
    Re: Build an initial Debian source package with py2dsp from a GitHub project Sandro Tosi <morph@debian.org> - 2021-10-29 06:50 +0200

#13325 — Build an initial Debian source package with py2dsp from a GitHub project

Hello,
recently i've been making some enhancements to py2dsp (part of
pypi2deb[1] ); for those who dont know what that is, py2dsp is a tool
that, given a PyPI project, will create an (initial) Debian source
package.

[1] https://packages.qa.debian.org/p/pypi2deb.html

I've just finished a patch that extend py2dsp to fetch the upstream
tarball from GitHub instead; nowadays this is my preferred source for
upstream tarballs, given it contains all the project files (not only
the one published on pypi via sdist, often missing important files
like tests, or doc sources, etc).

it's currently available at the git branch at [2] (there's a PR open at [3]):

[2] https://github.com/sandrotosi/pypi2deb/tree/morph
[3] https://github.com/p1otr/pypi2deb/pull/27

once you cloned/checkout that branch, you can run:

$ ./py2dsp --profile dpt --distribution unstable --revision 1 --github
https://github.com/USER/PROJECT

alternatively, you can specify an additional argument `<source name>`,
if PROJECT is not the source name you want to use in Debian:

$ ./py2dsp --profile dpt --distribution unstable --revision 1 --github
https://github.com/USER/PROJECT <source name>

and it will create the source package in the `result/` directory.

Let me know if you find this useful and if there are
issues/enhancement you'd like to see added/fixed.

Regards,
-- 
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
Twitter: https://twitter.com/sandrotosi

[toc] | [next] | [standalone]

#13326 — Re: Build an initial Debian source package with py2dsp from a GitHub project

[Multipart message — attachments visible in raw view] — view raw

On Sun, Apr 25, 2021 at 12:22:19AM -0400, Sandro Tosi wrote:
> Let me know if you find this useful and if there are
> issues/enhancement you'd like to see added/fixed.

That's very useful, thanks!

It would be useful if it could also be run against a tarball or git
repository (a directory) that one has already downloaded.

[toc] | [prev] | [next] | [standalone]

#13327

> It would be useful if it could also be run against a tarball

this is already supported (but in general by py2dsp and in the context
of --github), f.e.:

$ ./py2dsp --profile dpt --distribution unstable --revision 1 --gh
https://github.com/indygreg/python-zstandard
./zstandard_0.14.1.orig.tar.gz

uses the locally available zstandard_0.14.1.orig.tar.gz tarball (which
is not the latest available on gh) to create the source pkg with the
github customizations

> or git repository (a directory) that one has already downloaded.

i dont see how starting from a git repo is useful, can you expand?

Regards,
-- 
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
Twitter: https://twitter.com/sandrotosi

[toc] | [prev] | [next] | [standalone]

#13328 — Re: Build an initial Debian source package with py2dsp from a GitHub project

[Multipart message — attachments visible in raw view] — view raw

On Sun, Apr 25, 2021 at 06:19:18PM -0400, Sandro Tosi wrote:
> > It would be useful if it could also be run against a tarball
> 
> this is already supported (but in general by py2dsp and in the context
> of --github), f.e.:
> 
> $ ./py2dsp --profile dpt --distribution unstable --revision 1 --gh
> https://github.com/indygreg/python-zstandard
> ./zstandard_0.14.1.orig.tar.gz
> 
> uses the locally available zstandard_0.14.1.orig.tar.gz tarball (which
> is not the latest available on gh) to create the source pkg with the
> github customizations
> 
> > or git repository (a directory) that one has already downloaded.
> 
> i dont see how starting from a git repo is useful, can you expand?

instead of generating a .dsc first and then importing it into a git
repository, it's more logical to me to import an upstream tarball into a
git repository first (gbp import-orig), and then generate the debian
packaging on top of that.

[toc] | [prev] | [next] | [standalone]

#13329

> > > or git repository (a directory) that one has already downloaded.
> >
> > i dont see how starting from a git repo is useful, can you expand?
>
> instead of generating a .dsc first and then importing it into a git
> repository, it's more logical to me to import an upstream tarball into a
> git repository first (gbp import-orig), and then generate the debian
> packaging on top of that.

py2dsp does that for you: it creates a git repo, along with a source
package, with the DPT settings (note i used the `--profile dpt`) ready
to be pushed to salsa (the repo creation on salsa is still manual)

Regards,
-- 
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
Twitter: https://twitter.com/sandrotosi

[toc] | [prev] | [next] | [standalone]

#13330

Hi Sandro,

On Sun, Apr 25, 2021 at 9:53 AM Sandro Tosi <morph@debian.org> wrote:
> Let me know if you find this useful and if there are
> issues/enhancement you'd like to see added/fixed.

This looks great! Thank you!
However, I am running into an issue (or I guess I am just not doing it
correctly).
Whilst trying to package from the g/h source
(https://github.com/keylime/keylime), it fails like this:
http://paste.debian.net/1195339/

Am I missing something?


- u

[toc] | [prev] | [next] | [standalone]

#13331 — Re: Build an initial Debian source package with py2dsp from a GitHub project

[Multipart message — attachments visible in raw view] — view raw

On Thu, May 06, 2021 at 12:08:06AM +0530, Utkarsh Gupta wrote:
> However, I am running into an issue (or I guess I am just not doing it
> correctly).
> Whilst trying to package from the g/h source
> (https://github.com/keylime/keylime), it fails like this:
> http://paste.debian.net/1195339/
> 
> Am I missing something?
As far as I can see, the change is only about getting the tarball, it
still needs metadata from PyPI.

-- 
WBR, wRAR

[toc] | [prev] | [next] | [standalone]

#13332

On Wed, May 5, 2021 at 2:58 PM Andrey Rahmatullin <wrar@debian.org> wrote:
>
> On Thu, May 06, 2021 at 12:08:06AM +0530, Utkarsh Gupta wrote:
> > However, I am running into an issue (or I guess I am just not doing it
> > correctly).
> > Whilst trying to package from the g/h source
> > (https://github.com/keylime/keylime), it fails like this:
> > http://paste.debian.net/1195339/
> >
> > Am I missing something?
> As far as I can see, the change is only about getting the tarball, it
> still needs metadata from PyPI.

that's correct, the package still needs to be on PyPI, as that's the
place where py2dsp obtains most of the package metadata

-- 
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
Twitter: https://twitter.com/sandrotosi

[toc] | [prev] | [next] | [standalone]

#13333

Hello,

On Thu, May 6, 2021 at 12:33 AM Sandro Tosi <morph@debian.org> wrote:
> that's correct, the package still needs to be on PyPI, as that's the
> place where py2dsp obtains most of the package metadata

Can we change that or have a flag or something added so that it pulls
from g/h directly?


- u

[toc] | [prev] | [next] | [standalone]

#13334

On Thu, May 6, 2021 at 1:20 AM Andrey Rahmatullin <wrar@debian.org> wrote:
> But Github doesn't provide the metadata. You would need to get a tarball
> and run sdist or something like that.

Right and thus I am wondering if we could work through this, somehow?
That is, $something fetches the tarball, runs sdist or whatever, and
then the py2dsp magic.

P.S. I know this sounds a little ambitious but I believe this would
really help, too.


- u

[toc] | [prev] | [next] | [standalone]

#13336

> Right and thus I am wondering if we could work through this, somehow?
> That is, $something fetches the tarball, runs sdist or whatever, and
> then the py2dsp magic.
>
> P.S. I know this sounds a little ambitious but I believe this would
> really help, too.

i do not plan to implement such a feature.

this solution also underestimates the in-progress migration towards
poetry and pyproject.toml, where `python3 setup.py sdist` is not
available.

-- 
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
Twitter: https://twitter.com/sandrotosi

[toc] | [prev] | [next] | [standalone]

#13337

On Wed, May 5, 2021 at 10:14 PM Sandro Tosi wrote:

> this solution also underestimates the in-progress migration towards
> poetry and pyproject.toml, where `python3 setup.py sdist` is not
> available.

Where does the metadata come from for projects using these things?

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

[toc] | [prev] | [next] | [standalone]

#13339

> > this solution also underestimates the in-progress migration towards
> > poetry and pyproject.toml, where `python3 setup.py sdist` is not
> > available.
>
> Where does the metadata come from for projects using these things?

that'd be pyproject.toml AFAIUI

the point i wanted to make is that it would require to implement a
parser for setup.{py,cfg} and pyproject.toml, plus some way to decide
which one to use in case a project supports both (it happens) and how
to override the selection in case one system is available and outdated
(it happens).

All of this is done by PyPI already, and the fact a project exists on
GH but not on PyPI it's either because it's such a niche project or
it's still under heaving development, that working on a solution
*right now* is not a good use of our time.

-- 
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
Twitter: https://twitter.com/sandrotosi

[toc] | [prev] | [next] | [standalone]

#13335 — Re: Build an initial Debian source package with py2dsp from a GitHub project

[Multipart message — attachments visible in raw view] — view raw

On Thu, May 06, 2021 at 01:16:40AM +0530, Utkarsh Gupta wrote:
> Hello,
> 
> On Thu, May 6, 2021 at 12:33 AM Sandro Tosi <morph@debian.org> wrote:
> > that's correct, the package still needs to be on PyPI, as that's the
> > place where py2dsp obtains most of the package metadata
> 
> Can we change that or have a flag or something added so that it pulls
> from g/h directly?
But Github doesn't provide the metadata. You would need to get a tarball
and run sdist or something like that.

-- 
WBR, wRAR

[toc] | [prev] | [next] | [standalone]

#13338

On Wed, May 5, 2021 at 7:50 PM Andrey Rahmatullin wrote:

> But Github doesn't provide the metadata.

Usually the metadata is available in setup.cfg (or possibly setup.py),
both of which should be in GitHub, although data in setup.py would be
harder to extract.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

[toc] | [prev] | [next] | [standalone]

#13665

couple of features recently introduced that may interest people:

- rudimental support for autopkgtests: autodep8 + a stub for
developers to setup autopkgstests for unittests
- salsa repo creation, if SALSA_TOKEN is available in ~/.devscripts;
this also setup the KGB webhook to post in #d-p-changes

I'm definitely no expert in autopkgtests, so if there's something to
improve, lemme know.

On Sun, Apr 25, 2021 at 12:22 AM Sandro Tosi <morph@debian.org> wrote:
>
> Hello,
> recently i've been making some enhancements to py2dsp (part of
> pypi2deb[1] ); for those who dont know what that is, py2dsp is a tool
> that, given a PyPI project, will create an (initial) Debian source
> package.
>
> [1] https://packages.qa.debian.org/p/pypi2deb.html
>
> I've just finished a patch that extend py2dsp to fetch the upstream
> tarball from GitHub instead; nowadays this is my preferred source for
> upstream tarballs, given it contains all the project files (not only
> the one published on pypi via sdist, often missing important files
> like tests, or doc sources, etc).
>
> it's currently available at the git branch at [2] (there's a PR open at [3]):
>
> [2] https://github.com/sandrotosi/pypi2deb/tree/morph
> [3] https://github.com/p1otr/pypi2deb/pull/27
>
> once you cloned/checkout that branch, you can run:
>
> $ ./py2dsp --profile dpt --distribution unstable --revision 1 --github
> https://github.com/USER/PROJECT
>
> alternatively, you can specify an additional argument `<source name>`,
> if PROJECT is not the source name you want to use in Debian:
>
> $ ./py2dsp --profile dpt --distribution unstable --revision 1 --github
> https://github.com/USER/PROJECT <source name>
>
> and it will create the source package in the `result/` directory.
>
> Let me know if you find this useful and if there are
> issues/enhancement you'd like to see added/fixed.
>
> Regards,
> --
> Sandro "morph" Tosi
> My website: http://sandrotosi.me/
> Me at Debian: http://wiki.debian.org/SandroTosi
> Twitter: https://twitter.com/sandrotosi



-- 
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
Twitter: https://twitter.com/sandrotosi

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web