Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #8648 > unrolled thread

Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping extension causes network access during package builds

Back to article view | Back to linux.debian.maint.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping extension causes network access during package builds Brian May <bam@debian.org> - 2016-07-10 03:40 +0200
    Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping extension causes network access during package builds Daniele Tricoli <eriol@mornie.org> - 2016-07-10 04:00 +0200
    Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping  extension causes network access during package builds Dmitry Shachnev <mitya57@debian.org> - 2016-07-11 20:10 +0200
      Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping extension causes network access during package builds Daniele Tricoli <eriol@mornie.org> - 2016-07-11 20:30 +0200
      Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping extension causes network access during package builds Brian May <bam@debian.org> - 2016-07-11 23:50 +0200
        Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping  extension causes network access during package builds Dmitry Shachnev <mitya57@debian.org> - 2016-07-12 00:10 +0200
          Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping extension causes network access during package builds Brian May <bam@debian.org> - 2016-07-17 03:50 +0200
            Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping extension causes network access during package builds Dmitry Shachnev <mitya57@debian.org> - 2016-07-18 10:30 +0200
    Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping  extension causes network access during package builds Thomas Goirand <zigo@debian.org> - 2016-07-12 16:10 +0200
      Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping  extension causes network access during package builds Barry Warsaw <barry@python.org> - 2016-07-12 17:20 +0200
    Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping  extension causes network access during package builds Andreas Tille <andreas@an3as.eu> - 2016-07-14 13:30 +0200
      Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping extension causes network access during package builds Sergio Durigan Junior <sergiodj@sergiodj.net> - 2016-07-14 13:40 +0200
      Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping  extension causes network access during package builds Piotr Ożarowski <piotr@debian.org> - 2016-07-14 14:00 +0200

#8648 — Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping extension causes network access during package builds

Moved to debian-python list from bug report.

Dmitry Shachnev <mitya57@debian.org> writes:

> You say that we can read some flag set by dh-python. However, dh-python
> (aka pybuild) already sets http{,s}_proxy environment variables which
> disable network access for Sphinx. So packages using dh-python are already
> safe.

Not sure I understand how dh-python / pybuild help with this problem.

I thought the recommended way to deal with building documentation was
something like this in debian/rules:

override_dh_auto_build:
        dh_auto_build
        PYTHONPATH=. sphinx-build -b html -N Doc/ Doc/.build/html

... in which case building the documentation happens outside
dh_auto_build.

Did I miss something here?
-- 
Brian May <bam@debian.org>

[toc] | [next] | [standalone]

#8649

[Multipart message — attachments visible in raw view] — view raw

Hello Brian,

On Sunday, July 10, 2016 11:34:53 AM Brian May wrote:
> ... in which case building the documentation happens outside
> dh_auto_build.
> 
> Did I miss something here?

I have the same issue with betamax, and IIUC #830186 we have to patch 
intersphinx to use the distro provided inventory files. See this reply of 
Dmitry Shachnev (at the end he point to few examples):

    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=830186#10

Cheers,

-- 
 Daniele Tricoli 'eriol'
 https://mornie.org

[toc] | [prev] | [next] | [standalone]

#8650 — Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping extension causes network access during package builds

[Multipart message — attachments visible in raw view] — view raw

Hi Brian,

On Sun, Jul 10, 2016 at 11:34:53AM +1000, Brian May wrote:
> Not sure I understand how dh-python / pybuild help with this problem.
>
> I thought the recommended way to deal with building documentation was
> something like this in debian/rules:
>
> override_dh_auto_build:
>         dh_auto_build
>         PYTHONPATH=. sphinx-build -b html -N Doc/ Doc/.build/html
>
> ... in which case building the documentation happens outside
> dh_auto_build.
>
> Did I miss something here?

Indeed my comment applies only if Sphinx is invoked by pybuild (for example
if it's called by setup.py build).

If it's invoked separately the only ways would be to export http(s)_proxy in
debian/rules or patch conf.py to get correct intersphinx mapping.

Unfortunately there is no easy way for Sphinx to detect if it's invoked as
part of the package build process, so the ways I can change it are limited.

-- 
Dmitry Shachnev

[toc] | [prev] | [next] | [standalone]

#8652

[Multipart message — attachments visible in raw view] — view raw

Hello Dmitry,

On Monday, July 11, 2016 09:06:57 PM Dmitry Shachnev wrote:
> If it's invoked separately the only ways would be to export http(s)_proxy in
> debian/rules or patch conf.py to get correct intersphinx mapping.

Thanks for the details.
I'm already exporting http_proxy inside betamax debian/rules, maybe patch 
conf.py is needed anyway due DNS queries: it seems confirmed looking at 
tcpdump log for #830543.

Thanks! Cheers,

-- 
 Daniele Tricoli 'eriol'
 https://mornie.org

[toc] | [prev] | [next] | [standalone]

#8653

Dmitry Shachnev <mitya57@debian.org> writes:

> Indeed my comment applies only if Sphinx is invoked by pybuild (for example
> if it's called by setup.py build).

How do you call Sphinx in setup.py? When I researched this before, the
best advice I could find seemed to be along the lines of "don't do
that".
-- 
Brian May <bam@debian.org>

[toc] | [prev] | [next] | [standalone]

#8654 — Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping extension causes network access during package builds

[Multipart message — attachments visible in raw view] — view raw

Hi Brian,

On Tue, Jul 12, 2016 at 07:45:01AM +1000, Brian May wrote:
> How do you call Sphinx in setup.py? When I researched this before, the
> best advice I could find seemed to be along the lines of "don't do
> that".

Setuptools provides a build_sphinx command out of the box. For packages
using distutils, there is a sphinx.setup_command module provided by
Sphinx.

Some upstreams call that command during build phase, though that’s not
common.

I agree that my comment was a bit confusing because it applies only to
this rare case (it was mainly an answer to Chris’ question which mentioned
pybuild).

-- 
Dmitry Shachnev

[toc] | [prev] | [next] | [standalone]

#8661

Dmitry Shachnev <mitya57@debian.org> writes:

> Setuptools provides a build_sphinx command out of the box. For packages
> using distutils, there is a sphinx.setup_command module provided by
> Sphinx.
>
> Some upstreams call that command during build phase, though that’s not
> common.

Is there any reason that this isn't more common? Should I be doing this
for my packages which I maintain upstream?
-- 
Brian May <bam@debian.org>

[toc] | [prev] | [next] | [standalone]

#8666

Hi Brian,

On July 17, 2016 3:44:29 AM GMT+02:00, Brian May <bam@debian.org> wrote:
> Dmitry Shachnev <mitya57@debian.org> writes:
> 
> > Setuptools provides a build_sphinx command out of the box. For
> packages
> > using distutils, there is a sphinx.setup_command module provided by
> > Sphinx.
> >
> > Some upstreams call that command during build phase, though that’s
> not
> > common.
> 
> Is there any reason that this isn't more common? Should I be doing
> this
> for my packages which I maintain upstream?

If you are both the upstream and the packager, then adding a few lines in debian/rules to build the documentation is probably easier than hacking the build command in setup.py.

--
Dmitry Shachnev

[toc] | [prev] | [next] | [standalone]

#8656 — Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping extension causes network access during package builds

On 07/10/2016 03:34 AM, Brian May wrote:
> Moved to debian-python list from bug report.
> 
> Dmitry Shachnev <mitya57@debian.org> writes:
> 
>> You say that we can read some flag set by dh-python. However, dh-python
>> (aka pybuild) already sets http{,s}_proxy environment variables which
>> disable network access for Sphinx. So packages using dh-python are already
>> safe.
> 
> Not sure I understand how dh-python / pybuild help with this problem.

It doesn't, it simply hides the issue.

Also, sometimes exporting http{s,}_proxy simply breaks unit testing (I
did have such case, yes...). So that's not really a fit-all solution either.

> I thought the recommended way to deal with building documentation was
> something like this in debian/rules:
> 
> override_dh_auto_build:
>         dh_auto_build
>         PYTHONPATH=. sphinx-build -b html -N Doc/ Doc/.build/html

I find this not the top-noch way. Here's what I do:

override_dh_sphinxdoc:
ifeq (,$(findstring nodocs, $(DEB_BUILD_OPTIONS)))
	PYTHONPATH=. sphinx-build -b html doc/source \
		debian/python-foo-doc/usr/share/doc/python-foo-doc/html
	dh_sphinxdoc
endif

IMO, that's better because you don't need to clean Doc/.build/html.
Also, the dh_auto_build isn't the correct target. Some derivative may
well choose not to implement dh_sphinxdoc (for example, for small
systems where they don't want docs), and here, you're making their lives
miserable.

Cheers,

Thomas Goirand (zigo)

[toc] | [prev] | [next] | [standalone]

#8657 — Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping extension causes network access during package builds

[Multipart message — attachments visible in raw view] — view raw

On Jul 12, 2016, at 04:07 PM, Thomas Goirand wrote:

>I find this not the top-noch way. Here's what I do:
>
>override_dh_sphinxdoc:
>ifeq (,$(findstring nodocs, $(DEB_BUILD_OPTIONS)))
>	PYTHONPATH=. sphinx-build -b html doc/source \
>		debian/python-foo-doc/usr/share/doc/python-foo-doc/html
>	dh_sphinxdoc
>endif

I'm going to do this in the next python-webob upload to close #830611.  It had
been building the docs in override_dh_installdocs.  I'll also need to patch
the conf.py file to use the python{,3}-doc files and add those packages as
build dependencies.

Cheers,
-Barry

[toc] | [prev] | [next] | [standalone]

#8658 — Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping extension causes network access during package builds

Hi,

On Sun, Jul 10, 2016 at 11:34:53AM +1000, Brian May wrote:
> Moved to debian-python list from bug report.

Great - so I can step in here with an equivalent problem. :-)
 
> Not sure I understand how dh-python / pybuild help with this problem.
> 
> I thought the recommended way to deal with building documentation was
> something like this in debian/rules:
> 
> override_dh_auto_build:
>         dh_auto_build
>         PYTHONPATH=. sphinx-build -b html -N Doc/ Doc/.build/html
> 
> ... in which case building the documentation happens outside
> dh_auto_build.

I'm using

override_dh_auto_build:
        # arch
        USE_CYTHON=true dh_auto_build
        # indep:
        PYTHONPATH=. http_proxy='127.0.0.1:9' sphinx-build -N -b html doc build/html
        PYTHONPATH=. http_proxy='127.0.0.1:9' sphinx-build -N -b man doc build/man

in python-biom-format[1] which results in #831239.  I was once advised
to specify the http_proxy that way (others here in this thread as well)
and I wonder how to solve this.  If I try to export http_proxy and do
not call sphinx-build manually at all no documentation will be created.

So how to solve this?

Kind regards

       Andreas.

[1] https://anonscm.debian.org/git/debian-med/python-biom-format.git

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#8659

On Thursday, July 14 2016, Andreas Tille wrote:

>> Not sure I understand how dh-python / pybuild help with this problem.
>> 
>> I thought the recommended way to deal with building documentation was
>> something like this in debian/rules:
>> 
>> override_dh_auto_build:
>>         dh_auto_build
>>         PYTHONPATH=. sphinx-build -b html -N Doc/ Doc/.build/html
>> 
>> ... in which case building the documentation happens outside
>> dh_auto_build.
>
> I'm using
>
> override_dh_auto_build:
>         # arch
>         USE_CYTHON=true dh_auto_build
>         # indep:
>         PYTHONPATH=. http_proxy='127.0.0.1:9' sphinx-build -N -b html doc build/html
>         PYTHONPATH=. http_proxy='127.0.0.1:9' sphinx-build -N -b man doc build/man
>
> in python-biom-format[1] which results in #831239.  I was once advised
> to specify the http_proxy that way (others here in this thread as well)
> and I wonder how to solve this.  If I try to export http_proxy and do
> not call sphinx-build manually at all no documentation will be created.

Hey Andreas,

You have to instruct debhelper to use dh_sphinxdoc.  I.e.:

  dh $@ --with python2,python3,bash-completion,sphinxdoc --buildsystem=pybuild

Cheers,

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
http://sergiodj.net/

[toc] | [prev] | [next] | [standalone]

#8660 — Re: [Python-modules-team] Bug#830186: sphinx: intersphinx mapping extension causes network access during package builds

[Andreas Tille, 2016-07-14]
> override_dh_auto_build:
>         # arch
>         USE_CYTHON=true dh_auto_build
>         # indep:

override_dh_auto_build-indep:

>         PYTHONPATH=. http_proxy='127.0.0.1:9' sphinx-build -N -b html doc build/html
>         PYTHONPATH=. http_proxy='127.0.0.1:9' sphinx-build -N -b man doc build/man
> 

(python-sphinx is only in Build-Depends-Indep hence you cannot use it in
override_dh_auto_build)
-- 
Piotr Ożarowski                         Debian GNU/Linux Developer
www.ozarowski.pl          www.griffith.cc           www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web