Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #9648 > unrolled thread

packaging problems with python3.5 and python 3.6

Back to article view | Back to linux.debian.maint.python

Contents

  packaging problems with python3.5 and python 3.6 Norbert Preining <norbert@preining.info> - 2017-07-30 02:10 +0200
    Re: packaging problems with python3.5 and python 3.6 Steve Langasek <vorlon@debian.org> - 2017-07-30 07:50 +0200
      Re: packaging problems with python3.5 and python 3.6 Norbert Preining <norbert@preining.info> - 2017-07-30 08:40 +0200
        Re: packaging problems with python3.5 and python 3.6 Steve Langasek <vorlon@debian.org> - 2017-07-30 08:50 +0200
          Re: packaging problems with python3.5 and python 3.6 Norbert Preining <norbert@preining.info> - 2017-07-30 09:20 +0200
          Re: packaging problems with python3.5 and python 3.6 barry@debian.org - 2017-08-02 18:20 +0200
            Re: packaging problems with python3.5 and python 3.6 Steve Langasek <vorlon@debian.org> - 2017-08-02 21:40 +0200
              Re: packaging problems with python3.5 and python 3.6 barry@debian.org - 2017-08-03 15:10 +0200

#9648 — packaging problems with python3.5 and python 3.6

Dear all

(please Cc)

to satisfy a dependency of the next Calibre package I have packaged
	html5-parser
	https://github.com/kovidgoyal/html5-parser
see 
Vcs-Browser: https://anonscm.debian.org/cgit/collab-maint/html5-parser.git
Vcs-Git: git://anonscm.debian.org/collab-maint/html5-parser.git

Unfortunately I have given up providing python3 modules, since it seems
that either python3.5 or python3.6 breaks when adding the respective 
line as explained in the python packaging guide.

For now I have uploaded the package with python2 support, only, as this also
the only one needed for Calibre (which is python2 application).

If you have any suggestion concerning the python3 packaging part, please
let me know how this should be done.  You can see what I have done wrt
python3 in the following commit
https://anonscm.debian.org/cgit/collab-maint/html5-parser.git/commit/?id=4a8e02b20698e35e577482f96efdeb17826d797f
which I reverted afterwards.

Thanks a lot and all the best

Norbert

--
PREINING Norbert                               http://www.preining.info
Accelia Inc.     +    JAIST     +    TeX Live     +    Debian Developer
GPG: 0x860CDC13   fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13

[toc] | [next] | [standalone]

#9649

[Multipart message — attachments visible in raw view] — view raw

On Sun, Jul 30, 2017 at 08:35:47AM +0900, Norbert Preining wrote:
> Dear all
> 
> (please Cc)
> 
> to satisfy a dependency of the next Calibre package I have packaged
> 	html5-parser
> 	https://github.com/kovidgoyal/html5-parser
> see 
> Vcs-Browser: https://anonscm.debian.org/cgit/collab-maint/html5-parser.git
> Vcs-Git: git://anonscm.debian.org/collab-maint/html5-parser.git

> Unfortunately I have given up providing python3 modules, since it seems
> that either python3.5 or python3.6 breaks when adding the respective 
> line as explained in the python packaging guide.

Which "python packaging guide" do you refer to?  A search for "debian python
packaging guide" does not return unambiguous results.

Your difficulties almost certainly stem from you having set wrong
build-dependencies.  The correct build-dependencies here are
'python-all-dev, python3-all-dev' - plus the python2 and python3 versions of
any other modules needed at build time.  You should, e.g., never be
build-depending on python2.7-dev without a very good reason, and if you
build-depend on libpython3.6-dev this will obviously not give you python3.5
support.

If there is a python packaging guide suggesting that these sort of build
dependencies are appropriate, we should certainly get that corrected.

Since this package is in collab-maint, I've gone ahead and pushed fixed
python3 support to the repo.

> For now I have uploaded the package with python2 support, only, as this also
> the only one needed for Calibre (which is python2 application).

> If you have any suggestion concerning the python3 packaging part, please
> let me know how this should be done.  You can see what I have done wrt
> python3 in the following commit
> https://anonscm.debian.org/cgit/collab-maint/html5-parser.git/commit/?id=4a8e02b20698e35e577482f96efdeb17826d797f
> which I reverted afterwards.

> Thanks a lot and all the best

Thanks,
-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org

[toc] | [prev] | [next] | [standalone]

#9650

Hi Steve,

Thanks for your answer and your fix!

> Which "python packaging guide" do you refer to?  A search for "debian python
> packaging guide" does not return unambiguous results.

https://wiki.debian.org/Python/LibraryStyleGuide where it states:
	You'll want to have at least the following build dependencies: 
	...
	python3-all
but the
	python[,3]-all-dev
are never mentioned. So sorry, I didn't know about them and searched
for the required header files, and added the respective packages.

> 'python-all-dev, python3-all-dev' - plus the python2 and python3 versions of

Thanks, maybe one with proper knowledge should fix the wiki, though.

> Since this package is in collab-maint, I've gone ahead and pushed fixed
> python3 support to the repo.

Thanks again. You added a dependency on python-lxml. I don't think this
is necessary as libxml is only used at compile time. At least building
in a clean chroot did work out without any problems.

BTW, does it make sense to upload another -1 version to ftp-master
to replace the one in the NEW queue?

All the best

Norbert

--
PREINING Norbert                               http://www.preining.info
Accelia Inc.     +    JAIST     +    TeX Live     +    Debian Developer
GPG: 0x860CDC13   fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13

[toc] | [prev] | [next] | [standalone]

#9651

[Multipart message — attachments visible in raw view] — view raw

On Sun, Jul 30, 2017 at 03:16:06PM +0900, Norbert Preining wrote:
> Hi Steve,

> Thanks for your answer and your fix!

> > Which "python packaging guide" do you refer to?  A search for "debian python
> > packaging guide" does not return unambiguous results.

> https://wiki.debian.org/Python/LibraryStyleGuide where it states:
> 	You'll want to have at least the following build dependencies: 
> 	...
> 	python3-all
> but the
> 	python[,3]-all-dev
> are never mentioned. So sorry, I didn't know about them and searched
> for the required header files, and added the respective packages.

> > 'python-all-dev, python3-all-dev' - plus the python2 and python3 versions of

> Thanks, maybe one with proper knowledge should fix the wiki, though.

Ok, that was the first hit I got but I discounted it because it spoke of
packaging "libraries", not modules or extensions.

Perhaps someone should make this wiki page a redirect to the python policy
manual?  https://www.debian.org/doc/packaging-manuals/python-policy/

> > Since this package is in collab-maint, I've gone ahead and pushed fixed
> > python3 support to the repo.

> Thanks again. You added a dependency on python-lxml. I don't think this
> is necessary as libxml is only used at compile time. At least building
> in a clean chroot did work out without any problems.

I added python3-lxml because without it, build-time tests failed for
python3.  If I also added python-lxml, that was purely a cut'n'paste error.

> BTW, does it make sense to upload another -1 version to ftp-master
> to replace the one in the NEW queue?

AFAIK you would need to get the ftpmasters to reject the existing -1 first.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org

[toc] | [prev] | [next] | [standalone]

#9652

Hi Steve,

thanks for your answer, very much appreciated.

> Ok, that was the first hit I got but I discounted it because it spoke of
> packaging "libraries", not modules or extensions.

Indeed, but I was choosing between "applications" and "libraries" and
the later one sounded more appropriate.

> I added python3-lxml because without it, build-time tests failed for
> python3.  If I also added python-lxml, that was purely a cut'n'paste error.

No, my fault, it is python*3*-lxml. Thanks.

> AFAIK you would need to get the ftpmasters to reject the existing -1 first.

Ok, thanks a lot.

All the best

Norbert

--
PREINING Norbert                               http://www.preining.info
Accelia Inc.     +    JAIST     +    TeX Live     +    Debian Developer
GPG: 0x860CDC13   fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13

[toc] | [prev] | [next] | [standalone]

#9679

[Multipart message — attachments visible in raw view] — view raw

On Jul 30, 2017, at 02:41, Steve Langasek <vorlon@debian.org> wrote:

>> https://wiki.debian.org/Python/LibraryStyleGuide where it states:
>> 	You'll want to have at least the following build dependencies:
>> 	...
>> 	python3-all
>> but the
>> 	python[,3]-all-dev
>> are never mentioned. So sorry, I didn't know about them and searched
>> for the required header files, and added the respective packages.

> Ok, that was the first hit I got but I discounted it because it spoke of
> packaging "libraries", not modules or extensions.

“Libraries” is typically the term we’ve used to describe things you import from other things.  In Python, everything you import is a module, and packages are just special kinds of modules, whereas extensions are module implemented in a compiled language instead of native Python.  It may not be the best nomenclature, but it’s what we’ve been using for a long time.

> Perhaps someone should make this wiki page a redirect to the python policy
> manual?  https://www.debian.org/doc/packaging-manuals/python-policy/

No, because LibraryStyleGuide *isn’t* policy, it’s an opinionated collection of best practices, and it is better suited for a wiki because those best practices do change over time.

Policy is what you MUST do, and the barrier for changing policy is — and needs to be — higher.

Cheers,
-Barry

[toc] | [prev] | [next] | [standalone]

#9680

[Multipart message — attachments visible in raw view] — view raw

On Wed, Aug 02, 2017 at 12:10:05PM -0400, barry@debian.org wrote:
> On Jul 30, 2017, at 02:41, Steve Langasek <vorlon@debian.org> wrote:

> >> https://wiki.debian.org/Python/LibraryStyleGuide where it states:
> >> 	You'll want to have at least the following build dependencies:
> >> 	...
> >> 	python3-all
> >> but the
> >> 	python[,3]-all-dev
> >> are never mentioned. So sorry, I didn't know about them and searched
> >> for the required header files, and added the respective packages.

> > Ok, that was the first hit I got but I discounted it because it spoke of
> > packaging "libraries", not modules or extensions.

> “Libraries” is typically the term we’ve used to describe things you import from other things.  In Python, everything you import is a module, and packages are just special kinds of modules, whereas extensions are module implemented in a compiled language instead of native Python.  It may not be the best nomenclature, but it’s what we’ve been using for a long time.

> > Perhaps someone should make this wiki page a redirect to the python policy
> > manual?  https://www.debian.org/doc/packaging-manuals/python-policy/

> No, because LibraryStyleGuide *isn’t* policy, it’s an opinionated
> collection of best practices, and it is better suited for a wiki because
> those best practices do change over time.

> Policy is what you MUST do, and the barrier for changing policy is — and
> needs to be — higher.

Except the python packaging policy already mentions build dependencies:

"The python3-all-dev should be used when building extensions for any or all
Python 3 versions.  The python-all-dev should be used when building
extensions for any or all Python 2 versions."

  https://www.debian.org/doc/packaging-manuals/python-policy/ap-build_dependencies.html

But this is then shadowed by the less precise/complete recommendation in the
wiki.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org

[toc] | [prev] | [next] | [standalone]

#9682

[Multipart message — attachments visible in raw view] — view raw

On Aug 2, 2017, at 15:34, Steve Langasek <vorlon@debian.org> wrote:
> 
> Except the python packaging policy already mentions build dependencies:
> 
> "The python3-all-dev should be used when building extensions for any or all
> Python 3 versions.  The python-all-dev should be used when building
> extensions for any or all Python 2 versions."
> 
>  https://www.debian.org/doc/packaging-manuals/python-policy/ap-build_dependencies.html
> 
> But this is then shadowed by the less precise/complete recommendation in the
> wiki.

Note that the LSG does link to the policy page and even says:

"It is not intended to supplant the Debian Python Policy and in fact, if you have not read that document, stop now and go read it.”

and goes on:

"Instead, this page captures in example form, best practices for simple Python libraries that you might find on the Python Package Index.”

Maybe packages containing extensions are simple enough to include on this page.  It’s a wiki so, 1) it’s a living document; 2) contributions welcome! :)

-Barry

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web