Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.kernel > #56838 > unrolled thread

Sphinx builds always write to the source directory

Back to article view | Back to linux.debian.kernel

Contents

  Sphinx builds always write to the source directory Ben Hutchings <ben@decadent.org.uk> - 2017-01-31 00:00 +0100
    Re: Sphinx builds always write to the source directory Jani Nikula <jani.nikula@linux.intel.com> - 2017-01-31 09:10 +0100
      Re: Sphinx builds always write to the source directory Jonathan Corbet <corbet@lwn.net> - 2017-01-31 18:30 +0100
        [PATCH RESEND] Documentation/sphinx: prevent generation of .pyc files in the source tree Jani Nikula <jani.nikula@intel.com> - 2017-01-31 19:40 +0100
          Re: [PATCH RESEND] Documentation/sphinx: prevent generation of .pyc  files in the source tree Jonathan Corbet <corbet@lwn.net> - 2017-02-01 21:30 +0100
            Re: [PATCH RESEND] Documentation/sphinx: prevent generation of .pyc files in the source tree Jani Nikula <jani.nikula@intel.com> - 2017-02-03 16:00 +0100

#56838 — Sphinx builds always write to the source directory

[Multipart message — attachments visible in raw view] — view raw

I'm having trouble with the Debian packaging of kernel documentation. 
We try to build everything in a separate output directory (underneath
debian/build), but the new Sphinx-based build system writes to the
source directory.

Firstly, Python creates bytecode files alongside the module sources. 
This could be avoided by setting environment variable
PYTHONDONTWRITEBYTECODE=1, at the expense of requiring recompilation
more often.

Secondly, starting with 4.10, Documentation/media/Makefile converts
from dot to SVG and from SVG to PDF in the source directory.  It seems
like it should be possible to put the converted files in the output
directory, but I don't know how to tell Sphinx to find them there.

Are these issues likely to be fixed?

Ben.

-- 
Ben Hutchings
Q.  Which is the greater problem in the world today, ignorance or
apathy?
A.  I don't know and I couldn't care less.

[toc] | [next] | [standalone]

#56843

On Tue, 31 Jan 2017, Ben Hutchings <ben@decadent.org.uk> wrote:
> I'm having trouble with the Debian packaging of kernel documentation. 
> We try to build everything in a separate output directory (underneath
> debian/build), but the new Sphinx-based build system writes to the
> source directory.
>
> Firstly, Python creates bytecode files alongside the module sources. 
> This could be avoided by setting environment variable
> PYTHONDONTWRITEBYTECODE=1, at the expense of requiring recompilation
> more often.

I've sent the patch a few times.

http://lkml.kernel.org/r/1479916579-17151-1-git-send-email-jani.nikula@intel.com

> Secondly, starting with 4.10, Documentation/media/Makefile converts
> from dot to SVG and from SVG to PDF in the source directory.  It seems
> like it should be possible to put the converted files in the output
> directory, but I don't know how to tell Sphinx to find them there.
>
> Are these issues likely to be fixed?
>
> Ben.

-- 
Jani Nikula, Intel Open Source Technology Center

[toc] | [prev] | [next] | [standalone]

#56845

On Tue, 31 Jan 2017 09:32:20 +0200
Jani Nikula <jani.nikula@linux.intel.com> wrote:

> On Tue, 31 Jan 2017, Ben Hutchings <ben@decadent.org.uk> wrote:
> > I'm having trouble with the Debian packaging of kernel documentation. 
> > We try to build everything in a separate output directory (underneath
> > debian/build), but the new Sphinx-based build system writes to the
> > source directory.
> >
> > Firstly, Python creates bytecode files alongside the module sources. 
> > This could be avoided by setting environment variable
> > PYTHONDONTWRITEBYTECODE=1, at the expense of requiring recompilation
> > more often.  
> 
> I've sent the patch a few times.
> 
> http://lkml.kernel.org/r/1479916579-17151-1-git-send-email-jani.nikula@intel.com

It doesn't appear that you sent it to me, though, and I don't recall
having seen it.  Care to resend, and I'll pick it up?

jon

[toc] | [prev] | [next] | [standalone]

#56848 — [PATCH RESEND] Documentation/sphinx: prevent generation of .pyc files in the source tree

Use PYTHONDONTWRITEBYTECODE=1 to prevent python from creating .pyc files
in the source tree. Python 3.2 has a __pycache__ scheme [1], but before
that the only alternative seems to be to copy the source files to the
build tree to ensure the .pyc files are created there too. Just prevent
.pyc file generation for simplicity.

Considering the small amount of python code to compile (assuming sphinx
itself has .pyc around), the impact on build is neglible.

[1] http://stackoverflow.com/questions/3522079/changing-the-directory-where-pyc-files-are-created

References: http://lkml.kernel.org/r/CAMuHMdVxqpH7-9XJ+YE_pgoA+-fe0969cSkOehYh3uubYcrhZA@mail.gmail.com
Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
References: http://lkml.kernel.org/r/1485816692.2900.17.camel@decadent.org.uk
Reported-by: Ben Hutchings <ben@decadent.org.uk>
Cc: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 Documentation/Makefile.sphinx | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Documentation/Makefile.sphinx b/Documentation/Makefile.sphinx
index 707c65337ebf..91f541a52884 100644
--- a/Documentation/Makefile.sphinx
+++ b/Documentation/Makefile.sphinx
@@ -55,6 +55,7 @@ loop_cmd = $(echo-cmd) $(cmd_$(1))
 
 quiet_cmd_sphinx = SPHINX  $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
       cmd_sphinx = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=Documentation/media $2;\
+	PYTHONDONTWRITEBYTECODE=1 \
 	BUILDDIR=$(abspath $(BUILDDIR)) SPHINX_CONF=$(abspath $(srctree)/$(src)/$5/$(SPHINX_CONF)) \
 	$(SPHINXBUILD) \
 	-b $2 \
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]

#56864 — Re: [PATCH RESEND] Documentation/sphinx: prevent generation of .pyc files in the source tree

On Tue, 31 Jan 2017 20:18:05 +0200
Jani Nikula <jani.nikula@intel.com> wrote:

> Considering the small amount of python code to compile (assuming sphinx
> itself has .pyc around), the impact on build is neglible.

Hey...don't you know that performance-impacting patches need benchmarks?

Sphinx-only htmldocs build before:

	real	2m26.063s
	user	2m19.184s
	sys	0m5.429s

After:

	real	2m26.342s
	user	2m20.445s
	sys	0m5.611s

You owe me 0.279 seconds :)

(Applied, thanks).

jon

[toc] | [prev] | [next] | [standalone]

#56889 — Re: [PATCH RESEND] Documentation/sphinx: prevent generation of .pyc files in the source tree

On Wed, 01 Feb 2017, Jonathan Corbet <corbet@lwn.net> wrote:
> On Tue, 31 Jan 2017 20:18:05 +0200
> Jani Nikula <jani.nikula@intel.com> wrote:
>
>> Considering the small amount of python code to compile (assuming sphinx
>> itself has .pyc around), the impact on build is neglible.
>
> Hey...don't you know that performance-impacting patches need benchmarks?

Heh, I was briefly worried at this point! :)

>
> Sphinx-only htmldocs build before:
>
> 	real	2m26.063s
> 	user	2m19.184s
> 	sys	0m5.429s
>
> After:
>
> 	real	2m26.342s
> 	user	2m20.445s
> 	sys	0m5.611s
>
> You owe me 0.279 seconds :)
>
> (Applied, thanks).
>
> jon

-- 
Jani Nikula, Intel Open Source Technology Center

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.kernel

csiph-web