Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1688546 > unrolled thread

[PATCH 0/5] Add a script to check for Sphinx install requirements

Back to article view | Back to linux.kernel

Contents

  [PATCH 0/5] Add a script to check for Sphinx install requirements Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-07-17 00:10 +0200
    [PATCH 4/5] sphinx.rst: document scripts/sphinx-pre-install script Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-07-17 00:10 +0200
    Re: [PATCH 0/5] Add a script to check for Sphinx install requirements Markus Heiser <markus.heiser@darmarit.de> - 2017-07-17 10:20 +0200
      Re: [PATCH 0/5] Add a script to check for Sphinx install  requirements Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-07-17 11:40 +0200
        Re: [PATCH 0/5] Add a script to check for Sphinx install  requirements Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-07-17 12:20 +0200
          Re: [PATCH 0/5] Add a script to check for Sphinx install requirements Markus Heiser <markus.heiser@darmarit.de> - 2017-07-17 12:40 +0200
          Re: [PATCH 0/5] Add a script to check for Sphinx install  requirements Mauro Carvalho Chehab <mchehab@s-opensource.com> - 2017-07-17 13:00 +0200
            Re: [PATCH 0/5] Add a script to check for Sphinx install requirements Markus Heiser <markus.heiser@darmarit.de> - 2017-07-18 11:00 +0200

#1688546 — [PATCH 0/5] Add a script to check for Sphinx install requirements

Sphinx installation is not trivial, as not all versions are supported,
and it requires a lot of stuff for math, images and PDF/LaTeX output
to work.

So, add a script that checks if everything is fine, providing
distro-specific hints about what's needed for it to work.

I posted already RFC patches for the first patch in this series along
this weekend at linux-doc ML. The current version of the script
will provide hints for some popular distributions. I tested myself
on a couple of release-based and rolling popular distributions:

            - Fedora 25 and 26;
            - Ubuntu 17.04;
            - OpenSuse Tumbleweed;
            - Arch Linux;
            - Gentoo.

I tested compilation with both htmldocs and pdfdocs targets[1].
Yet, tests (and fix patches) are welcomed :-)

[1] There's an unrelated bug with the Kernel's sphinx extension
kimage: when parsing GraphViz graphs, it uses "-Tpdf" argument,
in order to generate a PDF image. That doesn't work on some
distros, as GraphViz doesn't support PDF images. 

Mauro Carvalho Chehab (5):
  scripts/sphinx-pre-install: add a script to check Sphinx install
  sphinx-pre-install: detect an existing virtualenv
  sphinx-pre-install: use a requirements file
  sphinx.rst: document scripts/sphinx-pre-install script
  doc: Makefile: if sphinx is not found, run a check script

 Documentation/Makefile             |   2 +
 Documentation/doc-guide/sphinx.rst |  48 +++-
 scripts/sphinx-pre-install         | 524 +++++++++++++++++++++++++++++++++++++
 3 files changed, 570 insertions(+), 4 deletions(-)
 create mode 100755 scripts/sphinx-pre-install

-- 
2.13.3

[toc] | [next] | [standalone]

#1688547 — [PATCH 4/5] sphinx.rst: document scripts/sphinx-pre-install script

Now that we have a script to check for Sphinx dependencies,
document it.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 Documentation/doc-guide/sphinx.rst | 48 ++++++++++++++++++++++++++++++++++----
 1 file changed, 44 insertions(+), 4 deletions(-)

diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
index 5521213efa68..8faafb9b2d86 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -28,6 +28,9 @@ The ReST markups currently used by the Documentation/ files are meant to be
 built with ``Sphinx`` version 1.3 or upper. If you're desiring to build
 PDF outputs, it is recommended to use version 1.4.6 or upper.
 
+There's a script that checks for the Spinx requirements. Please see
+:ref:`sphinx-pre-install` for further details.
+
 Most distributions are shipped with Sphinx, but its toolchain is fragile,
 and it is not uncommon that upgrading it or some other Python packages
 on your machine would cause the documentation build to break.
@@ -47,13 +50,15 @@ or ``virtualenv``, depending on how your distribution packaged Python 3.
       on the Sphinx version, it should be installed  in separate,
       with ``pip install sphinx_rtd_theme``.
 
+   #) Some ReST pages contain math expressions. Due to the way Sphinx work,
+      those expressions are written using LaTeX notation. It needs texlive
+      installed with amdfonts and amsmath in order to evaluate them.
+
 In summary, if you want to install Sphinx version 1.4.9, you should do::
 
        $ virtualenv sphinx_1.4
        $ . sphinx_1.4/bin/activate
-       (sphinx_1.4) $ pip install 'docutils==0.12'
-       (sphinx_1.4) $ pip install 'Sphinx==1.4.9'
-       (sphinx_1.4) $ pip install sphinx_rtd_theme
+       (sphinx_1.4) $ pip install -r Documentation/sphinx/requirements.txt
 
 After running ``. sphinx_1.4/bin/activate``, the prompt will change,
 in order to indicate that you're using the new environment. If you
@@ -83,7 +88,42 @@ For PDF and LaTeX output, you'll also need ``XeLaTeX`` version 3.14159265.
 
 Depending on the distribution, you may also need to install a series of
 ``texlive`` packages that provide the minimal set of functionalities
-required for ``XeLaTex`` to work.
+required for ``XeLaTeX`` to work.
+
+.. _sphinx-pre-install:
+
+Checking for Sphinx dependencies
+--------------------------------
+
+There's a script that automatically check for Sphinx dependencies. If it can
+recognize your distribution, it will also give a hint about the install
+command line options for your distro::
+
+	$ ./scripts/sphinx-pre-install
+	Checking if the needed tools for Fedora release 26 (Twenty Six) are available
+	Warning: better to also install "texlive-luatex85".
+	You should run:
+
+		sudo dnf install -y texlive-luatex85
+		/usr/bin/virtualenv sphinx_1.4
+		. sphinx_1.4/bin/activate
+		pip install -r Documentation/sphinx/requirements.txt
+
+	Can't build as 1 mandatory dependency is missing at ./scripts/sphinx-pre-install line 468.
+
+By default, it checks all the requirements for both html and PDF, including
+the requirements for images, math expressions and LaTeX build, and assumes
+that a virtual Python environment will be used. The ones needed for html
+builds are assumed to be mandatory; the others to be optional.
+
+It supports two optional parameters:
+
+``--no-pdf``
+	Disable checks for PDF;
+
+``--no-virtualenv``
+	Use OS packaging for Sphinx instead of Python virtual environment.
+
 
 Sphinx Build
 ============
-- 
2.13.3

[toc] | [prev] | [next] | [standalone]

#1688755

> Am 17.07.2017 um 00:08 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> 
> [1] There's an unrelated bug with the Kernel's sphinx extension
> kimage: when parsing GraphViz graphs, it uses "-Tpdf" argument,
> in order to generate a PDF image. That doesn't work on some
> distros, as GraphViz doesn't support PDF images. 
> 

Hmm .. http://www.graphviz.org/content/output-formats#dpdf

Is it a problem of the GraphViz version?

See http://www.graphviz.org/News.php ..

  """New Release 2.32 (1 August 2013)

     ... In addition, if the poppler library is available, Graphviz
     can now use PDF files as images."""

I suppose that 2.32 is in your distro .. it's from 2013

It seems, that the graphiz homepage is not up to date. I found 
the repo at github. There is a issue about PDF

  https://github.com/ellson/graphviz/issues/1180

So I guess graphviz is compiled without HAVE_PANGOCAIRO in your
distros?

-- Markus --

[toc] | [prev] | [next] | [standalone]

#1688824 — Re: [PATCH 0/5] Add a script to check for Sphinx install requirements

Em Mon, 17 Jul 2017 10:15:26 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:

> > Am 17.07.2017 um 00:08 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> > 
> > [1] There's an unrelated bug with the Kernel's sphinx extension
> > kimage: when parsing GraphViz graphs, it uses "-Tpdf" argument,
> > in order to generate a PDF image. That doesn't work on some
> > distros, as GraphViz doesn't support PDF images. 
> >   
> 
> Hmm .. http://www.graphviz.org/content/output-formats#dpdf
> 
> Is it a problem of the GraphViz version?
> 
> See http://www.graphviz.org/News.php ..
> 
>   """New Release 2.32 (1 August 2013)
> 
>      ... In addition, if the poppler library is available, Graphviz
>      can now use PDF files as images."""
> 
> I suppose that 2.32 is in your distro .. it's from 2013

Fedora 26 is shipped with version 2.40:

	$ rpm -q graphviz
	graphviz-2.40.1-4.fc26.x86_64

Yet:

$ dot -Thelp|grep pdf
Format: "help" not recognized. Use one of: canon cmap cmapx cmapx_np dot dot_json eps fig gv imap imap_np ismap json json0 mp pic plain plain-ext pov ps ps2 svg svgz tk vdx vml vmlz xdot xdot1.2 xdot1.4 xdot_json

I actually noticed this issue first on OpenSuse, where GraphViz is
packaged on several packages, but I was unable to find one package
there with pdf suport. I'm pretty sure pdf used to work on F25.

> 
> It seems, that the graphiz homepage is not up to date. I found 
> the repo at github. There is a issue about PDF
> 
>   https://github.com/ellson/graphviz/issues/1180
> 
> So I guess graphviz is compiled without HAVE_PANGOCAIRO in your
> distros?


From Fedora source package, it is requiring both cairo and pango:

	BuildRequires:		cairo-devel >= 1.1.10, pango-devel, gmp-devel, lua-devel, gtk2-devel, libgnomeui-devel

Building it gets:

	checking for PANGOCAIRO... yes

Still, no PDF support. I'll try to investigate it further, as I
found myself useful to produce PDF directly from GraphViz.

Anyway, I guess we should modify kfigure.py to check if PDF is
available, falling back to SVG, using ImageMagick to convert
from SVG to PDF.


Thanks,
Mauro

[toc] | [prev] | [next] | [standalone]

#1688847 — Re: [PATCH 0/5] Add a script to check for Sphinx install requirements

Em Mon, 17 Jul 2017 06:33:03 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:

> Em Mon, 17 Jul 2017 10:15:26 +0200
> Markus Heiser <markus.heiser@darmarit.de> escreveu:
> 
> > > Am 17.07.2017 um 00:08 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> > > 
> > > [1] There's an unrelated bug with the Kernel's sphinx extension
> > > kimage: when parsing GraphViz graphs, it uses "-Tpdf" argument,
> > > in order to generate a PDF image. That doesn't work on some
> > > distros, as GraphViz doesn't support PDF images. 
> > >     
> > 
> > Hmm .. http://www.graphviz.org/content/output-formats#dpdf
> > 
> > Is it a problem of the GraphViz version?
> > 
> > See http://www.graphviz.org/News.php ..
> > 
> >   """New Release 2.32 (1 August 2013)
> > 
> >      ... In addition, if the poppler library is available, Graphviz
> >      can now use PDF files as images."""
> > 
> > I suppose that 2.32 is in your distro .. it's from 2013  
> 
> Fedora 26 is shipped with version 2.40:
> 
> 	$ rpm -q graphviz
> 	graphviz-2.40.1-4.fc26.x86_64
> 
> Yet:
> 
> $ dot -Thelp|grep pdf
> Format: "help" not recognized. Use one of: canon cmap cmapx cmapx_np dot dot_json eps fig gv imap imap_np ismap json json0 mp pic plain plain-ext pov ps ps2 svg svgz tk vdx vml vmlz xdot xdot1.2 xdot1.4 xdot_json
> 
> I actually noticed this issue first on OpenSuse, where GraphViz is
> packaged on several packages, but I was unable to find one package
> there with pdf suport. I'm pretty sure pdf used to work on F25.
> 
> > 
> > It seems, that the graphiz homepage is not up to date. I found 
> > the repo at github. There is a issue about PDF
> > 
> >   https://github.com/ellson/graphviz/issues/1180
> > 
> > So I guess graphviz is compiled without HAVE_PANGOCAIRO in your
> > distros?  
> 
> 
> From Fedora source package, it is requiring both cairo and pango:
> 
> 	BuildRequires:		cairo-devel >= 1.1.10, pango-devel, gmp-devel, lua-devel, gtk2-devel, libgnomeui-devel
> 
> Building it gets:
> 
> 	checking for PANGOCAIRO... yes
> 
> Still, no PDF support. I'll try to investigate it further, as I
> found myself useful to produce PDF directly from GraphViz.
> 
> Anyway, I guess we should modify kfigure.py to check if PDF is
> available, falling back to SVG, using ImageMagick to convert
> from SVG to PDF.

Ok, I discovered that, on Fedora, support for pdf, png and some other
formats are packaged on a separate package: graphviz-gd.x86_64

With it installed, it now shows "pdf":

$ dot -Thelp
Format: "help" not recognized. Use one of: bmp canon cmap cmapx cmapx_np dot dot_json eps fig gd gd2 gif gtk gv ico imap imap_np ismap jpe jpeg jpg json json0 mp pdf pic plain plain-ext png pov ps ps2 svg svgz tif tiff tk vdx vml vmlz vrml wbmp x11 xdot xdot1.2 xdot1.4 xdot_json xlib

Thanks,
Mauro

[toc] | [prev] | [next] | [standalone]

#1688879

> Am 17.07.2017 um 12:09 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> 
>> 
>> Anyway, I guess we should modify kfigure.py to check if PDF is
>> available, falling back to SVG, using ImageMagick to convert
>> from SVG to PDF.
> 
> Ok, I discovered that, on Fedora, support for pdf, png and some other
> formats are packaged on a separate package: graphviz-gd.x86_64
> 
> With it installed, it now shows "pdf":
> 

Ah, OK .. thanks a lot for clarifying.

-- Markus --

[toc] | [prev] | [next] | [standalone]

#1688924 — Re: [PATCH 0/5] Add a script to check for Sphinx install requirements

Em Mon, 17 Jul 2017 07:09:46 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:

> Em Mon, 17 Jul 2017 06:33:03 -0300
> Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:
> 
> > Em Mon, 17 Jul 2017 10:15:26 +0200
> > Markus Heiser <markus.heiser@darmarit.de> escreveu:
> >   
> > > > Am 17.07.2017 um 00:08 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> > > > 
> > > > [1] There's an unrelated bug with the Kernel's sphinx extension
> > > > kimage: when parsing GraphViz graphs, it uses "-Tpdf" argument,
> > > > in order to generate a PDF image. That doesn't work on some
> > > > distros, as GraphViz doesn't support PDF images. 
> > > >       
> > > 
> > > Hmm .. http://www.graphviz.org/content/output-formats#dpdf
> > > 
> > > Is it a problem of the GraphViz version?
> > > 
> > > See http://www.graphviz.org/News.php ..
> > > 
> > >   """New Release 2.32 (1 August 2013)
> > > 
> > >      ... In addition, if the poppler library is available, Graphviz
> > >      can now use PDF files as images."""
> > > 
> > > I suppose that 2.32 is in your distro .. it's from 2013    
> > 
> > Fedora 26 is shipped with version 2.40:
> > 
> > 	$ rpm -q graphviz
> > 	graphviz-2.40.1-4.fc26.x86_64
> > 
> > Yet:
> > 
> > $ dot -Thelp|grep pdf
> > Format: "help" not recognized. Use one of: canon cmap cmapx cmapx_np dot dot_json eps fig gv imap imap_np ismap json json0 mp pic plain plain-ext pov ps ps2 svg svgz tk vdx vml vmlz xdot xdot1.2 xdot1.4 xdot_json
> > 
> > I actually noticed this issue first on OpenSuse, where GraphViz is
> > packaged on several packages, but I was unable to find one package
> > there with pdf suport. I'm pretty sure pdf used to work on F25.
> >   
> > > 
> > > It seems, that the graphiz homepage is not up to date. I found 
> > > the repo at github. There is a issue about PDF
> > > 
> > >   https://github.com/ellson/graphviz/issues/1180
> > > 
> > > So I guess graphviz is compiled without HAVE_PANGOCAIRO in your
> > > distros?    
> > 
> > 
> > From Fedora source package, it is requiring both cairo and pango:
> > 
> > 	BuildRequires:		cairo-devel >= 1.1.10, pango-devel, gmp-devel, lua-devel, gtk2-devel, libgnomeui-devel
> > 
> > Building it gets:
> > 
> > 	checking for PANGOCAIRO... yes
> > 
> > Still, no PDF support. I'll try to investigate it further, as I
> > found myself useful to produce PDF directly from GraphViz.
> > 
> > Anyway, I guess we should modify kfigure.py to check if PDF is
> > available, falling back to SVG, using ImageMagick to convert
> > from SVG to PDF.  
> 
> Ok, I discovered that, on Fedora, support for pdf, png and some other
> formats are packaged on a separate package: graphviz-gd.x86_
> 
> With it installed, it now shows "pdf":
> 
> $ dot -Thelp
> Format: "help" not recognized. Use one of: bmp canon cmap cmapx cmapx_np dot dot_json eps fig gd gd2 gif gtk gv ico imap imap_np ismap jpe jpeg jpg json json0 mp pdf pic plain plain-ext png pov ps ps2 svg svgz tif tiff tk vdx vml vmlz vrml wbmp x11 xdot xdot1.2 xdot1.4 xdot_json xlib

Double-checked on all VMs I used to test this pachset:

- on Fedora 25, Debian 17.04, Gentoo and ArchLinux, just graphviz
  is enough for PDF support.

- on Fedora 26, graphviz-gd is needed for PDF support;

- on OpenSuse Tumbleweed, I was unable to discover a package that
  would make GraphViz to support pdf. There, it uses graphviz
  version 2.38, with is the same version as used on Fedora 25.

That is with just graphviz:

	mchehab@OpenSuse:~/docs> dot -Thelp
	Format: "help" not recognized. Use one of: canon cmap cmapx cmapx_np dot eps fig gv imap imap_np ismap pic plain plain-ext pov ps ps2 svg svgz tk vml vmlz xdot xdot1.2 xdot1.4

And that is with graphviz-gd:

	mchehab@OpenSuse:~/docs> dot -Thelp
	Format: "help" not recognized. Use one of: canon cmap cmapx cmapx_np dot eps fig gd gd2 gif gv imap imap_np ismap jpe jpeg jpg pic plain plain-ext png pov ps ps2 svg svgz tk vml vmlz vrml wbmp xdot xdot1.2 xdot1.4

So, we need a way for kfigure to fallback when distros don't
have such feature.

Regards,
Mauro

[toc] | [prev] | [next] | [standalone]

#1689887

> Am 17.07.2017 um 12:57 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> 
> So, we need a way for kfigure to fallback when distros don't
> have such feature.

Hm .. I'am not very happy to implement where distros packaging fail.
But .. if there is really a need for, I will do so.

Lets see how this series goes upstream and feel free to remind
on my words if I completely suppressed the idea after upstream. :) 

-- Markus --

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web