Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #15119 > unrolled thread

Re: pdm: Please replace python3-pep517 with python3-pyproject-hooks in Depends/Build-Depends

Back to article view | Back to linux.debian.maint.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: pdm: Please replace python3-pep517 with python3-pyproject-hooks in Depends/Build-Depends Scott Kitterman <debian@kitterman.com> - 2023-08-15 15:10 +0200
    Re: pdm: Please replace python3-pep517 with python3-pyproject-hooks in Depends/Build-Depends Scott Kitterman <debian@kitterman.com> - 2023-08-15 16:20 +0200
      Re: pdm: Please replace python3-pep517 with python3-pyproject-hooks  in Depends/Build-Depends Andreas Tille <andreas@an3as.eu> - 2023-08-18 15:40 +0200
        Re: pdm: Please replace python3-pep517 with python3-pyproject-hooks in Depends/Build-Depends Scott Kitterman <debian@kitterman.com> - 2023-08-18 16:30 +0200
    Re: pdm: Please replace python3-pep517 with python3-pyproject-hooks in Depends/Build-Depends Jeroen Dekkers <jeroen@dekkers.ch> - 2023-08-15 16:20 +0200

#15119 — Re: pdm: Please replace python3-pep517 with python3-pyproject-hooks in Depends/Build-Depends

[Multipart message — attachments visible in raw view] — view raw

On Tuesday, August 15, 2023 3:53:07 AM EDT Andreas Tille wrote:
> Hi Scott,
> 
> Am Mon, Aug 14, 2023 at 02:06:42PM +0000 schrieb Scott Kitterman:
> > >Before I upload I'd like to ask for reviewing this patch and opinions
> > >about the test suite errors.  While these possibly occure in previous
> > >versions (which I did not tested) we might consider ignoring just the
> > >failing tests.  I need to admit that I did not went through the list of
> > >single failures - may be there is a chance of easy fixes for some of
> > >them.  I simply wanted to discuss the reintroduction of the artifacts
> > >and my patch first.
> > 
> > With the exception of future_fstrings
> 
> I think there is also the souce for demo included.
> 
> > those are all binary without source.  That's a problem.
> 
> Given your role as ftpmaster you definitely have more experience than I
> in this field.  I personally was thinking more in the line of binary
> data we can not avoid in some cases.  This is a bit in the line with
> Rdata decision[1] where those binary data files are documented in
> debian/README.source.
> 
> My point is just: If we remove those data files (which are IMHO harmless
> since these are not executd but used as input in tests - please correct
> me if I'm wrong) we can not test the package.  Removing the files
> prevents testing and thus we can not know whether the package we deliver
> will work.  I've actually shown that not all tests are working but
> stopped investigating this further.

Even if they are used as data (I didn't check), they are, in fact, binary 
blobs of code by our definition and that requires the corresponding source.

> > It's probably okay if you add the corresponding source somewhere in the
> > package.
> We probably have some source packages inside Debian - may be in
> different versions.  I need to admit that I intended to do a "quick fix
> of a simple bug that affects some Debian Med packages" but realised that
> I'm possibly opening a can of worms.  The simplest solution to fulfill
> my needs would be probably to revert my change to run the tests and be
> done.  However, I'm not sure whetherr this is in the interest of the
> users of this package.  I'm absolutely not able time-wise to povide
> sources and reconstruct all those *.whl files *and* in addition track
> down the test suite errors.  This is a team package and if the team
> decides we should go without testing I can do an according upload.
> However, my prefered path would to document the issue of some binary
> data properly an test what upstream expects to be tested.

i think this is definitely more complicated than you want to take on casually.  
I don't think it's required to actually rebuild the wheels.  If you provide 
the source, the DFSG is happy.  You have to be able to rebuild it, but you 
aren't required to do it.

It might, however, be simpler in the long run to just depend on those packages 
and build wheels from those (a Debian binary package has enough in it 
generally to build a wheel from it).

I agree it'd be better in the long run to run the tests, but that may be more 
than you want to take on right now.

Scott K

[toc] | [next] | [standalone]

#15120

On August 15, 2023 1:51:54 PM UTC, Jeroen Dekkers <jeroen@dekkers.ch> wrote:
>On Tue, 15 Aug 2023 15:08:11 +0200,
>Scott Kitterman wrote:
>>
>> On Tuesday, August 15, 2023 3:53:07 AM EDT Andreas Tille wrote:
>> > Hi Scott,
>> >
>> > Am Mon, Aug 14, 2023 at 02:06:42PM +0000 schrieb Scott Kitterman:
>> >
>> > > those are all binary without source.  That's a problem.
>> >
>> > Given your role as ftpmaster you definitely have more experience than I
>> > in this field.  I personally was thinking more in the line of binary
>> > data we can not avoid in some cases.  This is a bit in the line with
>> > Rdata decision[1] where those binary data files are documented in
>> > debian/README.source.
>> >
>> > My point is just: If we remove those data files (which are IMHO harmless
>> > since these are not executd but used as input in tests - please correct
>> > me if I'm wrong) we can not test the package.  Removing the files
>> > prevents testing and thus we can not know whether the package we deliver
>> > will work.  I've actually shown that not all tests are working but
>> > stopped investigating this further.
>>
>> Even if they are used as data (I didn't check), they are, in fact, binary
>> blobs of code by our definition and that requires the corresponding source.
>
>They are zip files containing python source code. It is possible to include
>compiled C extensions in wheels, but I checked and these wheels are all pure
>python, so no binary blobs are included.

In Debian terms, it's not the preferred form for modification, so it's not source.  In this regard DFSG goes farther than some software licenses.

Scott K

[toc] | [prev] | [next] | [standalone]

#15130 — Re: pdm: Please replace python3-pep517 with python3-pyproject-hooks in Depends/Build-Depends

Hi Scott,

Am Fri, Aug 18, 2023 at 01:15:18PM +0000 schrieb Scott Kitterman:
> On August 18, 2023 1:04:26 PM UTC, Andreas Tille <andreas@an3as.eu> wrote:
> >> In Debian terms, it's not the preferred form for modification, so it's not source.  In this regard DFSG goes farther than some software licenses.
> >
> >I think the point Jeroen wanted to make is that these are actually
> >source file archives which "by chance" are featuring a .whl extension
> >rather than a .zip extension.
> 
> A wheel is not the preferred form for modification.  They're not wheels by chance at all.

Yes, thanks to Jeroen's hint I realised this as well and I agree that
this is a nasty way to hide the fact that the files are actually source
archives.

However, you confirmed yourself that future_fstrings is an exception and
comes with source and thus does not violate DFSG.  The only difference
I personally can see is that the archives are just hiding what they are.
We could simply add do some
   for whl in *.whl ; do ln -s $whl $(basename .whl).zip ; done
and we have source archives that are obviously what they are.
 
> From a DFSG perspective,

Hmmm, the only thing where I can draw a violation of the DFSG is that
there are no d/copyright entries for the source code that is hidden
inside these *.whl files.  Otherwise its "just" duplicated code (in most
cases) which is definitely not nice but IMHO not a violation of DFSG.

> the most straightforward approach is to build-depend on the relevant Debian packages and build any needed wheels from that.

Do avoid source code duplication I'm willing to do that.  Yes, I
perfectly agree that its pretty ugly (I'm just a bit unsure about
the DFSG violation).  I'm wondering whether a simple

   zip whl.zip /path/to/python/files ; mv whl.zip whl.whl

will be something that can replace the current packages.  I think
we also need to patch the tests to fit the version numbers (if
we do not want to cheat and simply use the version numbers of the
original whl files to avoid patching).

>  It won't necessarily get you the same version as upstream uses, but it's definitely DFSG compliant.

We also might symlink our resulting whl files with the version number
pdm upstream might expect in their tests.  The question is, whether all
this effort might break the tests in some way and we might end up with
endless patching by at the same time loosing the chance to discuss with
upstream.  But it might be time to discuss the issue with upstream
anyway.

Kind regards
    Andreas.

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#15133

[Multipart message — attachments visible in raw view] — view raw

On Friday, August 18, 2023 9:33:48 AM EDT Andreas Tille wrote:
> Hi Scott,
> 
> Am Fri, Aug 18, 2023 at 01:15:18PM +0000 schrieb Scott Kitterman:
> > On August 18, 2023 1:04:26 PM UTC, Andreas Tille <andreas@an3as.eu> wrote:
> > >> In Debian terms, it's not the preferred form for modification, so it's
> > >> not source.  In this regard DFSG goes farther than some software
> > >> licenses.> >
> > >I think the point Jeroen wanted to make is that these are actually
> > >source file archives which "by chance" are featuring a .whl extension
> > >rather than a .zip extension.
> > 
> > A wheel is not the preferred form for modification.  They're not wheels by
> > chance at all.
> Yes, thanks to Jeroen's hint I realised this as well and I agree that
> this is a nasty way to hide the fact that the files are actually source
> archives.

They aren't.

> However, you confirmed yourself that future_fstrings is an exception and
> comes with source and thus does not violate DFSG.  The only difference
> I personally can see is that the archives are just hiding what they are.
> We could simply add do some
>    for whl in *.whl ; do ln -s $whl $(basename .whl).zip ; done
> and we have source archives that are obviously what they are.
> 
> > From a DFSG perspective,
> 
> Hmmm, the only thing where I can draw a violation of the DFSG is that
> there are no d/copyright entries for the source code that is hidden
> inside these *.whl files.  Otherwise its "just" duplicated code (in most
> cases) which is definitely not nice but IMHO not a violation of DFSG.
> 
The disagreement here is that Python wheels aren't source.  DFSG #2 requires 
the source be present and these aren't it.  If you look at the WAF entry in 
the FTP team reject FAQ, this is similar.  The FTP team view has long been 
that DFSG #2 means the actual preferred form for modification.

> > the most straightforward approach is to build-depend on the relevant
> > Debian packages and build any needed wheels from that.
> Do avoid source code duplication I'm willing to do that.  Yes, I
> perfectly agree that its pretty ugly (I'm just a bit unsure about
> the DFSG violation).  I'm wondering whether a simple
> 
>    zip whl.zip /path/to/python/files ; mv whl.zip whl.whl
> 
> will be something that can replace the current packages.  I think
> we also need to patch the tests to fit the version numbers (if
> we do not want to cheat and simply use the version numbers of the
> original whl files to avoid patching).

Perhaps, but there are nuances to the wheel format.  Please use Wheel to 
generate them.

> >  It won't necessarily get you the same version as upstream uses, but it's
> >  definitely DFSG compliant.
> We also might symlink our resulting whl files with the version number
> pdm upstream might expect in their tests.  The question is, whether all
> this effort might break the tests in some way and we might end up with
> endless patching by at the same time loosing the chance to discuss with
> upstream.  But it might be time to discuss the issue with upstream
> anyway.

Perhaps.  If it were me, what I'd do is locate the missing tarballs and stash 
them in debian/missing-sources/ and worry about more complex solutions later.  
Once you're done that, you've met DSFG #2 and there's no need to strip the 
wheels from the binary.

It's not super maintainable, but it will allow you to focos on getting the 
tests working as upstream ships them without any Debian customizations that 
might cause Debian specific failures.

Scott K

[toc] | [prev] | [next] | [standalone]

#15121

On Tue, 15 Aug 2023 15:08:11 +0200,
Scott Kitterman wrote:
>
> On Tuesday, August 15, 2023 3:53:07 AM EDT Andreas Tille wrote:
> > Hi Scott,
> >
> > Am Mon, Aug 14, 2023 at 02:06:42PM +0000 schrieb Scott Kitterman:
> >
> > > those are all binary without source.  That's a problem.
> >
> > Given your role as ftpmaster you definitely have more experience than I
> > in this field.  I personally was thinking more in the line of binary
> > data we can not avoid in some cases.  This is a bit in the line with
> > Rdata decision[1] where those binary data files are documented in
> > debian/README.source.
> >
> > My point is just: If we remove those data files (which are IMHO harmless
> > since these are not executd but used as input in tests - please correct
> > me if I'm wrong) we can not test the package.  Removing the files
> > prevents testing and thus we can not know whether the package we deliver
> > will work.  I've actually shown that not all tests are working but
> > stopped investigating this further.
>
> Even if they are used as data (I didn't check), they are, in fact, binary
> blobs of code by our definition and that requires the corresponding source.

They are zip files containing python source code. It is possible to include
compiled C extensions in wheels, but I checked and these wheels are all pure
python, so no binary blobs are included.

Kind regards,

Jeroen Dekkers

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web