Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #8906 > unrolled thread

Upstream build system, Sphinx autodoc, Python import path

Back to article view | Back to linux.debian.maint.python

Contents

  Upstream build system, Sphinx autodoc, Python import path Ben Finney <ben+debian@benfinney.id.au> - 2016-09-27 10:50 +0200
    Re: Upstream build system, Sphinx autodoc, Python import path Thomas Goirand <zigo@debian.org> - 2016-09-29 23:30 +0200
      Re: Upstream build system, Sphinx autodoc, Python import path Dmitry Shachnev <mitya57@debian.org> - 2016-09-30 08:50 +0200
    Re: Upstream build system, Sphinx autodoc, Python import path Thomas Goirand <zigo@debian.org> - 2016-09-29 23:40 +0200
      Re: Upstream build system, Sphinx autodoc, Python import path Barry Warsaw <barry@debian.org> - 2016-09-30 00:00 +0200
      Re: Upstream build system, Sphinx autodoc, Python import path Nikolaus Rath <Nikolaus@rath.org> - 2016-09-30 05:00 +0200
        Re: Upstream build system, Sphinx autodoc, Python import path Florent Rougon <f.rougon@free.fr> - 2016-09-30 08:20 +0200
          Re: Upstream build system, Sphinx autodoc, Python import path Thomas Goirand <zigo@debian.org> - 2016-09-30 12:20 +0200
          Re: Upstream build system, Sphinx autodoc, Python import path Nikolaus Rath <Nikolaus@rath.org> - 2016-09-30 23:20 +0200

#8906 — Upstream build system, Sphinx autodoc, Python import path

Howdy all,

What advice can we give to upstream for their build system, so that
in a Debian build environment the Sphinx ‘autodoc’ can import upstream's
code modules?

Our wiki page on Library Style Guide advises the Debian packaging
override the Python import path:

    If you need to build the Sphinx documentation (usually from .rst or
    .md files), add:

        override_dh_auto_build:
            dh_auto_build
            PYTHONPATH=. sphinx-build -N -bhtml docs/ build/html # HTML generator
            PYTHONPATH=. sphinx-build -N -bman docs/ build/man # Manpage generator

    <URL:https://wiki.debian.org/Python/LibraryStyleGuide#Sphinx_documentation>

This is surely not ideal, though. Better would be to not need
‘PYTHONPATH’ override at all, and just call the build program.

What information do our build tools look for, and what variables are
set, when running upstream's build system?

What can we advise to upstream so their build system will work correctly
in our build environment *without* that override in every Sphinx-using
package?

-- 
 \         “For a sentimentalist is simply one who desires to have the |
  `\    luxury of an emotion without paying for it.” —Oscar Wilde, _De |
_o__)                                                 Profundis_, 1897 |
Ben Finney

[toc] | [next] | [standalone]

#8913

On 09/27/2016 10:48 AM, Ben Finney wrote:
> Our wiki page on Library Style Guide advises the Debian packaging
> override the Python import path:
> 
>     If you need to build the Sphinx documentation (usually from .rst or
>     .md files), add:
> 
>         override_dh_auto_build:
>             dh_auto_build
>             PYTHONPATH=. sphinx-build -N -bhtml docs/ build/html # HTML generator
>             PYTHONPATH=. sphinx-build -N -bman docs/ build/man # Manpage generator

If that's what the wiki advise, then it's quite wrong. The target which
should be "overrides" is dh_sphinxdoc. I usually do this:

override_dh_sphinxdoc:
ifeq (,$(findstring nodocs, $(DEB_BUILD_OPTIONS)))
        sphinx-build -b html doc/source \
		$(CURDIR)/debian/foo-doc/usr/share/doc/foo-doc/html
	dh_sphinxdoc
endif

And the same kind of thing for man page. It's IMO nice to have the
nodocs thing if it takes too long to build (probably useless if it's
quick). In such case, overriding the correct debian/rules target is key.

> This is surely not ideal, though. Better would be to not need
> ‘PYTHONPATH’ override at all, and just call the build program.

PYTHONPATH=. isn't *always* needed, therefore, I don't think it should
always be in debian/rules.

> What can we advise to upstream so their build system will work correctly
> in our build environment *without* that override in every Sphinx-using
> package?

I don't see why upstream requiring PYTHONPATH=. would be bad...

Cheers,

Thomas Goirand (zigo)

[toc] | [prev] | [next] | [standalone]

#8918

[Multipart message — attachments visible in raw view] — view raw

Hi Thomas,

On Thu, Sep 29, 2016 at 11:22:08PM +0200, Thomas Goirand wrote:
> On 09/27/2016 10:48 AM, Ben Finney wrote:
> > Our wiki page on Library Style Guide advises the Debian packaging
> > override the Python import path:
> >
> >     If you need to build the Sphinx documentation (usually from .rst or
> >     .md files), add:
> >
> >         override_dh_auto_build:
> >             dh_auto_build
> >             PYTHONPATH=. sphinx-build -N -bhtml docs/ build/html # HTML generator
> >             PYTHONPATH=. sphinx-build -N -bman docs/ build/man # Manpage generator
>
> If that's what the wiki advise, then it's quite wrong. The target which
> should be "overrides" is dh_sphinxdoc. I usually do this:
>
> override_dh_sphinxdoc:
> ifeq (,$(findstring nodocs, $(DEB_BUILD_OPTIONS)))
>         sphinx-build -b html doc/source \
> 		$(CURDIR)/debian/foo-doc/usr/share/doc/foo-doc/html
> 	dh_sphinxdoc
> endif
>
> And the same kind of thing for man page. It's IMO nice to have the
> nodocs thing if it takes too long to build (probably useless if it's
> quick). In such case, overriding the correct debian/rules target is key.

I think the modern way to disable docs is using DEB_BUILD_PROFILES=nodoc
rather than DEB_BUILD_OPTIONS (because that allows you to actually disable
building the foo-doc package, and also allows dropping sphinx build-dep).
See [1], [2] for details.

Anyway, for non-core Python packages there is usually no reason to care
about this, and in that case building the docs in override_dh_auto_build
is perfectly fine. And IMO it is semantically more correct to build stuff
in build target, rather than install target.

[1]: https://wiki.debian.org/BuildProfileSpec#Registered_profile_names
[2]: https://wiki.debian.org/DebianBootstrap#Documentation_loops

> I don't see why upstream requiring PYTHONPATH=. would be bad...

Agreed.

--
Dmitry Shachnev

[toc] | [prev] | [next] | [standalone]

#8914

On 09/27/2016 10:48 AM, Ben Finney wrote:
> https://wiki.debian.org/Python/LibraryStyleGuide#Sphinx_documentation

I just had a quick look to that page, not only it advises to override
the wrong dh sequence, but also it gives stupid advices for intersphinx:

"sphinx-build might try to access the Internet to fetch intersphinx
inventory files; http_proxy set to 127.0.0.1:9 will prevent that."

Instead of attempting circumvention of the effect of using intersphinx,
it's best to simply *DISABLE* intersphinx in the conf.py of the
documentation. Setting-up http_proxy / https_proxy to do that, really,
is the wrong way. Lots of my packages contain intersphinx disabling patches.

Cheers,

Thomas Goirand (zigo)

[toc] | [prev] | [next] | [standalone]

#8915

On Sep 29, 2016, at 11:30 PM, Thomas Goirand wrote:

>I just had a quick look to that page, not only it advises to override
>the wrong dh sequence, but also it gives stupid advices for intersphinx:

ObDIY: "It's a wiki". :)

Cheers,
-Barry

[toc] | [prev] | [next] | [standalone]

#8916

On Sep 29 2016, Thomas Goirand <zigo@debian.org> wrote:
> On 09/27/2016 10:48 AM, Ben Finney wrote:
>> https://wiki.debian.org/Python/LibraryStyleGuide#Sphinx_documentation
>
> I just had a quick look to that page, not only it advises to override
> the wrong dh sequence, but also it gives stupid advices for intersphinx:
>
> "sphinx-build might try to access the Internet to fetch intersphinx
> inventory files; http_proxy set to 127.0.0.1:9 will prevent that."
>
> Instead of attempting circumvention of the effect of using intersphinx,
> it's best to simply *DISABLE* intersphinx in the conf.py of the
> documentation. Setting-up http_proxy / https_proxy to do that, really,
> is the wrong way. Lots of my packages contain intersphinx disabling
> patches.

Even better than disabling intersphinx is to ship the required data in
the package, e.g:

$ cat debian/patches/use-local-intersphinx-inventory.patch 
>From 48e6c33f77106b9368e7db430d296ba6c31e47a6 Mon Sep 17 00:00:00 2001
From: Nikolaus Rath <Nikolaus@rath.org>
Date: Thu, 8 Oct 2015 12:24:34 -0700
Subject: Use local intersphinx inventory

Forwarded: not-needed
Last-Update: 2011-07-06
 Instead of downloading the Python intersphinx directory
 at build time, use the cached copy shipped in debian/.
Patch-Name: use-local-intersphinx-inventory.patch
---
 rst/conf.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/rst/conf.py b/rst/conf.py
index 2290d..f6425 100644
--- a/rst/conf.py
+++ b/rst/conf.py
@@ -27,7 +27,8 @@ extensions = ['sphinx.ext.autodoc', 'sphinx.ext.intersphinx',
               'sphinx_cython' ]
 
 # Link to Python standard library
-intersphinx_mapping = {'python': ('http://docs.python.org/3/', None) }
+intersphinx_mapping = {'python': ('http://docs.python.org/3/',
+                                  '../debian/python.inv')}
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']

$ grep intersphinx -C 1 debian/rules  

update_intersphinx:
	wget http://docs.python.org/3/objects.inv -O debian/python.inv



Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

[toc] | [prev] | [next] | [standalone]

#8917

Or use the following alternative, which lets the user browse the
documentation offline.

$ cat debian/patches/use-local-python3-doc-for-cross-references
>From f519a948173bfd4e9700a4ebc60bc54bf3455018 Mon Sep 17 00:00:00 2001
From: Florent Rougon <f.rougon@free.fr>
Date: Thu, 8 Oct 2015 13:28:35 -0700
Subject: Use local doc from python3-doc for cross references

Forwarded: not-needed
Last-Update: 2014-10-17

Patch-Name: use-local-python3-doc-for-cross-references
---
 doc/conf.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/doc/conf.py b/doc/conf.py
index 728397d..18461c9 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -35,7 +35,8 @@ extensions = [
     'sphinx.ext.viewcode',
 ]
 
-intersphinx_mapping = {'python': ('http://docs.python.org/3', None)}
+intersphinx_mapping = {'python': ('file:///usr/share/doc/python3/html',
+                                  '/usr/share/doc/python3/html/objects.inv')}
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']

Then put python3-doc in B-D and Suggests or so.

-- 
Florent

[toc] | [prev] | [next] | [standalone]

#8919

On 09/30/2016 08:13 AM, Florent Rougon wrote:
> Or use the following alternative, which lets the user browse the
> documentation offline.

I found it very nice to do that indeed, but probably too much time
consuming for the package maintainer. It'd be nice if we found a way to
make it more automated. Does anyone have an idea about how to write such
automation?

Cheers,

Thomas Goirand (zigo)

[toc] | [prev] | [next] | [standalone]

#8920

On Sep 30 2016, Florent Rougon <f.rougon@free.fr> wrote:
>>> Instead of attempting circumvention of the effect of using intersphinx,
>>> it's best to simply *DISABLE* intersphinx in the conf.py of the
>>> documentation.
>>
>> Even better than disabling intersphinx is to ship the required data in
>> the package, e.g:
>
> Or use the following alternative, which lets the user browse the
> documentation offline.

Which just goes to show that there is no solution that can't be further
improved :-). Thanks!


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web