Groups > linux.kernel > #1393462 > unrolled thread

Re: Kernel docs: muddying the waters a bit

Started by	Daniel Vetter <daniel.vetter@ffwll.ch>
First post	2016-05-03 16:40 +0200
Last post	2016-05-06 19:10 +0200
Articles	13 on this page of 33 — 7 participants

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

  Re: Kernel docs: muddying the waters a bit Daniel Vetter <daniel.vetter@ffwll.ch> - 2016-05-03 16:40 +0200
    Re: Kernel docs: muddying the waters a bit Keith Packard <keithp@keithp.com> - 2016-05-03 18:10 +0200
    Re: Kernel docs: muddying the waters a bit Markus Heiser <markus.heiser@darmarit.de> - 2016-05-04 11:40 +0200
      Re: Kernel docs: muddying the waters a bit Jani Nikula <jani.nikula@intel.com> - 2016-05-04 12:00 +0200
        Re: Kernel docs: muddying the waters a bit Markus Heiser <markus.heiser@darmarit.de> - 2016-05-04 14:50 +0200
          Re: Kernel docs: muddying the waters a bit Jani Nikula <jani.nikula@intel.com> - 2016-05-04 15:50 +0200
            Re: Kernel docs: muddying the waters a bit Jonathan Corbet <corbet@lwn.net> - 2016-05-04 17:10 +0200
          Re: Kernel docs: muddying the waters a bit Daniel Vetter <daniel@ffwll.ch> - 2016-05-04 15:50 +0200
            Re: Kernel docs: muddying the waters a bit Daniel Vetter <daniel.vetter@ffwll.ch> - 2016-05-04 16:20 +0200
              Re: Kernel docs: muddying the waters a bit Jonathan Corbet <corbet@lwn.net> - 2016-05-04 17:00 +0200
                Re: Kernel docs: muddying the waters a bit Daniel Vetter <daniel@ffwll.ch> - 2016-05-04 17:10 +0200
                  Re: Kernel docs: muddying the waters a bit Daniel Vetter <daniel.vetter@ffwll.ch> - 2016-05-04 18:40 +0200
                Re: Kernel docs: muddying the waters a bit Jani Nikula <jani.nikula@intel.com> - 2016-05-04 17:50 +0200
                Re: Kernel docs: muddying the waters a bit Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-05-04 18:40 +0200
            Re: Kernel docs: muddying the waters a bit Markus Heiser <markus.heiser@darmarit.de> - 2016-05-04 18:00 +0200
              Re: Kernel docs: muddying the waters a bit Jani Nikula <jani.nikula@intel.com> - 2016-05-04 18:20 +0200
                Re: Kernel docs: muddying the waters a bit Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-05-04 19:00 +0200
                  Re: Kernel docs: muddying the waters a bit Jonathan Corbet <corbet@lwn.net> - 2016-05-04 19:00 +0200
                    Re: Kernel docs: muddying the waters a bit Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-05-04 20:00 +0200
                      Re: Kernel docs: muddying the waters a bit Jonathan Corbet <corbet@lwn.net> - 2016-05-05 15:10 +0200
                        Re: Kernel docs: muddying the waters a bit Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-05-05 15:30 +0200
                Re: Kernel docs: muddying the waters a bit Markus Heiser <markus.heiser@darmarit.de> - 2016-05-06 13:30 +0200
                  Re: Kernel docs: muddying the waters a bit Markus Heiser <markus.heiser@darmarit.de> - 2016-05-06 13:50 +0200
                  Re: Kernel docs: muddying the waters a bit Markus Heiser <markus.heiser@darmarit.de> - 2016-05-06 16:30 +0200
                    Re: Kernel docs: muddying the waters a bit Jani Nikula <jani.nikula@intel.com> - 2016-05-06 17:10 +0200
                      Re: Kernel docs: muddying the waters a bit Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-05-06 17:30 +0200
                      Re: Kernel docs: muddying the waters a bit Markus Heiser <markus.heiser@darmarit.de> - 2016-05-06 17:40 +0200
                        Re: Kernel docs: muddying the waters a bit Jani Nikula <jani.nikula@intel.com> - 2016-05-06 18:00 +0200
                    Re: Kernel docs: muddying the waters a bit Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-05-06 17:20 +0200
      Re: Kernel docs: muddying the waters a bit Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-05-04 18:20 +0200
        Re: Kernel docs: muddying the waters a bit Markus Heiser <markus.heiser@darmarit.de> - 2016-05-06 12:10 +0200
          Re: Kernel docs: muddying the waters a bit Markus Heiser <markus.heiser@darmarit.de> - 2016-05-06 18:40 +0200
            Re: Kernel docs: muddying the waters a bit Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-05-06 19:10 +0200

Page 2 of 2 — ← Prev page 1 [2]

#1395118

From	Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Date	2016-05-05 15:30 +0200
Message-ID	<rvrvR-MH-45@gated-at.bofh.it>
In reply to	#1395094

Em Thu, 5 May 2016 07:02:10 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> On Wed, 4 May 2016 14:57:38 -0300
> Mauro Carvalho Chehab <mchehab@osg.samsung.com> wrote:
> 
> > Also, media documentation is not just one more documentation. It is
> > the biggest one we have, and that has more changes than any other
> > documentation under Documentation/DocBook:
> > 
> > $ git lg --since 01/01/2015 ` ls *.tmpl|grep -v media`|wc -l
> > 116
> > $ git lg --since 01/01/2015 ` ls *.tmpl|grep media` `find media/ -type f`|wc -l
> > 179
> > 
> > It also is more than twice the size of the other DocBook docs:
> > 
> > $ wc -l $(ls *.tmpl|grep  media) `find media/ -type f`|tail -1
> >   82275 total
> > $ wc -l $(ls *.tmpl|grep -v media)|tail -1
> >   29568 total
> > 
> > E. g. media corresponds to 60% of the number of patches and 73% of
> > the DocBook stuff.  
> 
> These numbers are not entirely representative, I have to say.  You're
> ignoring the kerneldoc comments - which is what much of the "DocBook"
> documents are made of, and which is the focus of much of this activity.  If
> you could find a way to count those, you'd get a different picture.

Yeah, if we take the big picture, I'm pretty sure that there are more
stuff written using kerneldoc. However, what is written at the 
kerneldoc comments don't use DocBook markup, but its own markup
language.

Assuming that we'll keep using kerneldoc script to convert from the
Kerneldoc NANO markup, it means that any changes at the kerneldoc
backend to use DocBook/Markdown/asciidoc/reST/... won't change anything
for the developers nor will require them to actually use the new markups.
So, they can gradually start using new markups as they wish/learn, if
they want to improve the documentation. Lazy developers could just ignore
reST if they want, as everything will work as before.

However, for the files written directly in DocBook, there's no option:
at the moment the doc is converted, the developer will need to submit
patches using the new markup language. So, we need to be sure that
such transition will happen in a way that won't cause more harm than
needed.  I would prefer to convert all of them at one, on a single Kernel
release (or two), in order to minimize the impact for the developers.

> 
> But I don't think that really matters; there doesn't seem to be *that* much
> disagreement here.
> 
> The media book is important; we want it to be a part of the overall kernel
> documentation suite and not stuck in some DocBook ghetto.  I agree that we
> should have an idea for a plausible path for *all* of our documentation.
> But I'm also concerned about delaying this work yet again; we have
> developers trying to push forward with improved documentation, and they've
> had to wait a year for this stuff - so far.

Yeah, I understand.

Regards,
Mauro

[toc] | [prev] | [next] | [standalone]

#1395781

From	Markus Heiser <markus.heiser@darmarit.de>
Date	2016-05-06 13:30 +0200
Message-ID	<rvM7f-3YW-1@gated-at.bofh.it>
In reply to	#1394503

Hy Jani,

Am 04.05.2016 um 18:13 schrieb Jani Nikula <jani.nikula@intel.com>:

>> Am 04.05.2016 um 17:09 schrieb Jonathan Corbet <corbet@lwn.net>:
>> 
>>> I think all of this makes sense.  It would be really nice to have the
>>> directives in the native sphinx language like that.  I *don't* think we
>>> need to aim for that at the outset; the docproc approach works until we can
>>> properly get rid of it.  What would be *really* nice would be to get
>>> support for the kernel-doc directive into the sphinx upstream.
>> 
>> No need for kernel-doc directive in sphinx upstream, later it will be 
>> an extension which could be installed by a simple command like 
>> "pip install kernel-doc-extensions" or similar.
>> 
>> I develop these required extension (and more) within my proof of concept
>> on github ... this takes time ... if I finished all my tests and all is
>> well, I will build the *kernel-doc-extensions* package and deploy it
>> on https://pypi.python.org/pypi from where everyone could install this 
>> with "pip".
> 
> I think we should go for vanilla sphinx at first, to make the setup step
> as easy as possible for everyone. Even if it means still doing that ugly
> docproc step to call kernel-doc. We can improve from there, and I
> definitely appreciate your work on making this work with sphinx
> extensions.

+1 

> That said, how would it work to include the kernel-doc extension in the
> kernel source tree? Having things just work if sphinx is installed is
> preferred over requiring installation of something extra from pypi. (I
> know this may sound backwards for a lot of projects, but for kernel I'm
> pretty sure this is how it should be done.)

Thats all right. Lets talk about the extension infrastructure by example:

First we have to chose a folder where we place all the *sphinx-documentation*
I recommending:

 /share/linux/Documentation/sphinx

Next we have to chose a folder where reST-extensions should take place, I
would prefer ... or similar:
 
 /share/linux/Documentation/sphinx/extensions

Lets say, you wan't to get in use of the "flat-table" extension.

Copy (only) the rstFlatTable.py file from my POC extension folder (ignore
other extensions which might be there) ...

 https://github.com/return42/sphkerneldoc/tree/master/doc/extensions

Now lets say you are writing on a gpu book, it wold be placed in the folder:

 /share/linux/Documentation/sphinx/gpu

In this gpu-folder you have to place the conf.py config file, needed to
setup the sphinx build environment.

 /share/linux/Documentation/sphinx/gpu/conf.py

In this conf.py you have to *register* your folder with the extensions.

<SNIP conf.py> --------

    import os.path, sys

    EXT_PATH  = "../extensions"  # the path of extension folder relative to the conf.py file
    sys.path.insert(0, os.path.join(os.path.dirname(__file__), EXT_PATH)))

    # now import the "flat-table" extension, it will be self-registering to docutils

    import rstFlatTable

<SNIP conf.py> --------

Thats all, you can run your sphinx-build command and the flat-tables in your
reST sources should be handled as common tables.

ASIDE: 

You will find similar parts in your conf.py which you have created 
with the sphinx-quickstart command. There, you will also find a block 
looks like ...

extensions = [
    'sphinx.ext.autodoc'
....
]

Don't try to add flat-table extension to this list. This list is a list
of sphinx extensions, we will use it later for other *real* sphinx 
extensions.

A few words about the flat-table extension and a (future) kernel-doc one:

The flat-table is a pure docutils (the layer below sphinx) extension which
is not application specific, so I will ask for moving it to the docutils 
upstream. 

The kernel-doc extension on the other side is a very (very) kernel specific
application, this would never go to sphinx nor docutils upstream.

--Markus--

[toc] | [prev] | [next] | [standalone]

#1395802

From	Markus Heiser <markus.heiser@darmarit.de>
Date	2016-05-06 13:50 +0200
Message-ID	<rvMqC-4be-5@gated-at.bofh.it>
In reply to	#1395781

Hi Jani,

I forget to mentioning, with a local copy of my kernel-doc script:

https://github.com/return42/sphkerneldoc/blob/master/scripts/kernel-doc

You could do reST markup in the source code comments and extract them.

This might be a interim workaround which helps you not to edit source 
code comments twice.

-- Markus--



Am 06.05.2016 um 13:23 schrieb Markus Heiser <markus.heiser@darmarIT.de>:

> 
> Hy Jani,
> 
> Am 04.05.2016 um 18:13 schrieb Jani Nikula <jani.nikula@intel.com>:
> 
>>> Am 04.05.2016 um 17:09 schrieb Jonathan Corbet <corbet@lwn.net>:
>>> 
>>>> I think all of this makes sense.  It would be really nice to have the
>>>> directives in the native sphinx language like that.  I *don't* think we
>>>> need to aim for that at the outset; the docproc approach works until we can
>>>> properly get rid of it.  What would be *really* nice would be to get
>>>> support for the kernel-doc directive into the sphinx upstream.
>>> 
>>> No need for kernel-doc directive in sphinx upstream, later it will be 
>>> an extension which could be installed by a simple command like 
>>> "pip install kernel-doc-extensions" or similar.
>>> 
>>> I develop these required extension (and more) within my proof of concept
>>> on github ... this takes time ... if I finished all my tests and all is
>>> well, I will build the *kernel-doc-extensions* package and deploy it
>>> on https://pypi.python.org/pypi from where everyone could install this 
>>> with "pip".
>> 
>> I think we should go for vanilla sphinx at first, to make the setup step
>> as easy as possible for everyone. Even if it means still doing that ugly
>> docproc step to call kernel-doc. We can improve from there, and I
>> definitely appreciate your work on making this work with sphinx
>> extensions.
> 
> +1 
> 
>> That said, how would it work to include the kernel-doc extension in the
>> kernel source tree? Having things just work if sphinx is installed is
>> preferred over requiring installation of something extra from pypi. (I
>> know this may sound backwards for a lot of projects, but for kernel I'm
>> pretty sure this is how it should be done.)
> 
> Thats all right. Lets talk about the extension infrastructure by example:
> 
> First we have to chose a folder where we place all the *sphinx-documentation*
> I recommending:
> 
> /share/linux/Documentation/sphinx
> 
> Next we have to chose a folder where reST-extensions should take place, I
> would prefer ... or similar:
> 
> /share/linux/Documentation/sphinx/extensions
> 
> Lets say, you wan't to get in use of the "flat-table" extension.
> 
> Copy (only) the rstFlatTable.py file from my POC extension folder (ignore
> other extensions which might be there) ...
> 
> https://github.com/return42/sphkerneldoc/tree/master/doc/extensions
> 
> Now lets say you are writing on a gpu book, it wold be placed in the folder:
> 
> /share/linux/Documentation/sphinx/gpu
> 
> In this gpu-folder you have to place the conf.py config file, needed to
> setup the sphinx build environment.
> 
> /share/linux/Documentation/sphinx/gpu/conf.py
> 
> In this conf.py you have to *register* your folder with the extensions.
> 
> <SNIP conf.py> --------
> 
>    import os.path, sys
> 
>    EXT_PATH  = "../extensions"  # the path of extension folder relative to the conf.py file
>    sys.path.insert(0, os.path.join(os.path.dirname(__file__), EXT_PATH)))
> 
>    # now import the "flat-table" extension, it will be self-registering to docutils
> 
>    import rstFlatTable
> 
> <SNIP conf.py> --------
> 
> Thats all, you can run your sphinx-build command and the flat-tables in your
> reST sources should be handled as common tables.
> 
> ASIDE: 
> 
> You will find similar parts in your conf.py which you have created 
> with the sphinx-quickstart command. There, you will also find a block 
> looks like ...
> 
> extensions = [
>    'sphinx.ext.autodoc'
> ....
> ]
> 
> Don't try to add flat-table extension to this list. This list is a list
> of sphinx extensions, we will use it later for other *real* sphinx 
> extensions.
> 
> A few words about the flat-table extension and a (future) kernel-doc one:
> 
> The flat-table is a pure docutils (the layer below sphinx) extension which
> is not application specific, so I will ask for moving it to the docutils 
> upstream. 
> 
> The kernel-doc extension on the other side is a very (very) kernel specific
> application, this would never go to sphinx nor docutils upstream.
> 
> --Markus--
>

[toc] | [prev] | [next] | [standalone]

#1395896

From	Markus Heiser <markus.heiser@darmarit.de>
Date	2016-05-06 16:30 +0200
Message-ID	<rvOVr-6Tb-7@gated-at.bofh.it>
In reply to	#1395781

Hi all, hi Jonathan,

Am 06.05.2016 um 15:42 schrieb Mauro Carvalho Chehab <mchehab@osg.samsung.com>:

> Em Fri, 6 May 2016 15:32:35 +0200
> Markus Heiser <markus.heiser@darmarit.de> escreveu:
> 
>> Hi Mauro,
>> 
>> Am 06.05.2016 um 13:35 schrieb Mauro Carvalho Chehab <mchehab@osg.samsung.com>:
>> 
>>> Markus,
>>> 
>>> Em Fri, 6 May 2016 13:23:06 +0200
>>> Markus Heiser <markus.heiser@darmarit.de> escreveu:
>>> 
>>>> 
>>>> In this conf.py you have to *register* your folder with the extensions.
>>>> A few words about the flat-table extension and a (future) kernel-doc one:  
>>> 
>>> ...
>>> 
>>>> The flat-table is a pure docutils (the layer below sphinx) extension which
>>>> is not application specific, so I will ask for moving it to the docutils 
>>>> upstream.   
>>> 
>>> So, if I understood well, your proposal is to have a conf.py and the
>>> flat-table (plus other extensions) at the Kernel tree.  
>> 
>> Each book (better call it root-node) is a Sphinx-project, each 
>> Sphinx-project need a conf.py file (the build configuration file)
>> in its reST-source tree.
>> 
>> * http://www.sphinx-doc.org/en/stable/config.html
>> 
>>> Assuming that docutils upstream receives the flat-table extension
>>> (and eventually modifies it), while the new version doesn't arrive
>>> all distros, we'll end by having some developers using the newer
>>> docutils with the extension, plus others using the in-tree one.
>>> 
>>> Is there a way to specify at the conf.py what extension variant
>>> should it use, in case of both the in-tree or the docutils have
>>> the same?  
>> 
>> The build configuration file is a regular python file, you can 
>> implement conditions whatever you want/need.
>> 
>>> This could be trickier if they end by modifying the extension,
>>> but we can always backport the latest version, if they change the
>>> API.  
>> 
>> As far as i know, the docutils API is stable since 2002. In the
>> meantime there has been so many application build on it that
>> it is not realistic, you will see a not backward compatibly 
>> change.
>> 
>> The docutils project is conservative, very conservative, IMO to
>> conservative.
>> 
>> Today I'am doubtful if it isn't better I would merge it sphinx
>> upstream. I have to discuss this with some maintainers, but 
>> before I have to persuade myself, that all aspects are covered
>> and the implementations are robust. We are at the beginning and
>> we should not fear about every bit which could happen in the future.
>> 
>> The sphinx / docutils bottom plate gives us a number of degrees 
>> of freedom to find answers to question we have not yet asked. ;-)
> 
> Ok. So, from what I understand, once Sphinx support is added at
> Kernel upstream, we could convert the media docbook to
> reST+flat-table extension, adding such extension either on a shared
> place or only for the media DocBook build, together with its
> conf.py.
> 

Yes, in your media-conf you could decide to use a extension.

> Once it reaches upstream (either sphinx or docutils), we can
> work to make it integrate better with upstream as needed.
> 
> Right?

yes, right :-)

> If so, I'm ok with merging it as soon as possible.

If we advice a merge of the flat-table directive we should 
bundle this with the (to implement) "kernel-doc" directive ...

> In reST the directive might look like:
> 
> <reST-SNIP> -----
> Device Instance and Driver Handling
> ===================================
> 
> .. kernel-doc::  drivers/gpu/drm/drm_drv.c
>   :doc:      driver instance overview
>   :exported:
> 
> <reST-SNAP> -----

and the patches from my kernel-doc perl script to produce
reST from source code comments.

With this bundle within the kernel tree we have a good starting
point to compose reST documents from scratch and to migrate book
by book from DocBook to reST.

I insist to migrate book by book, because there are some
broken books. Broken by that, that some sources have changed
but not the corresponding documentation which use the comments
of these sources ... grap "Ooops" in the builded (xml or rst) docs.

E.g. I greped the .rst file and found the following Oops in the migrated books:

./books/mtdnand/pubfunctions-000-012.rst:13:Oops
./books/scsi/mid_layer-000-001-016-003.rst:13:Oops
./books/s390-drivers/ccw-000-004-003.rst:13:Oops
./books/device-drivers/devdrivers-000-003-048.rst:13:Oops
./books/device-drivers/devdrivers-000-003-050.rst:13:Oops
./books/device-drivers/Basics-000-001-002.rst:13:Oops
./books/device-drivers/devdrivers-000-003-031.rst:13:Oops
./books/device-drivers/Basics-000-009-032.rst:13:Oops
./books/kernel-api/kernel-lib-000-004-008.rst:13:Oops
./books/gadget/api-000-011-005.rst:13:Oops
./books/gadget/api-000-011-009.rst:13:Oops
./books/gadget/api-000-011-007.rst:13:Oops
./books/gadget/api-000-011-003.rst:13:Oops
./books/gadget/api-000-011-011.rst:13:Oops
./books/genericirq/intfunctions-000-009.rst:13:Oops

----- Summarize ----

@Jonathan: what do you think? Should I prepare a patch
with a basic reST (sphinx) build infrastructure, including

* a folder for sphinx docs:

  ./Documentation/sphinx/

* flat-table & kernel-doc extension at

  ./Documentation/sphinx/extensions

* a patch with rst-Output for the kernel-doc perl script at

  ./scripts/kernel-doc

* An example document "HowTo document with reST" at

  ./Documentation/sphinx/kernel-doc-rst-howto

  which at minimum describes the "flat-table" and "kernel-doc" 
  directive and the requirements for a building docs.

* a make file which fit into the kernel Makefile infrastructure (not 
  the one created by sphinx-quickstart).

-- Markus --

[toc] | [prev] | [next] | [standalone]

#1395916

From	Jani Nikula <jani.nikula@intel.com>
Date	2016-05-06 17:10 +0200
Message-ID	<rvPya-7xW-5@gated-at.bofh.it>
In reply to	#1395896

On Fri, 06 May 2016, Markus Heiser <markus.heiser@darmarit.de> wrote:
> @Jonathan: what do you think? Should I prepare a patch
> with a basic reST (sphinx) build infrastructure, including
>
> * a folder for sphinx docs:
>
>   ./Documentation/sphinx/

I'm already working on a patch series taking a different approach. I
don't think we should hide the documentation under an extra folder named
after a tool. Actually, I'm strongly opposed to that.

Instead, we should place the Sphinx stuff directly under Documentation,
and have Sphinx recursively pick up all the *.rst files. We should
promote gradually switching to lightweight markup and integration of the
documents into one system. This process should be as little disruptive
as possible.

If someone wants to convert a .txt document to .rst and get it processed
by Sphinx, it should be as simple as renaming the file, doing the
necessary edits, and adding it to a toctree. Imagine gradually
converting the files under, say, Documentation/kbuild. Why should the
.rst files be moved under another directory? They should stay alongside
the .txt files under the same directory. There's bound to be a lot of
people who'll never use Sphinx, and will expect to find the good old
plain text files (albeit with some markup) where they always were.

BR,
Jani.

-- 
Jani Nikula, Intel Open Source Technology Center

[toc] | [prev] | [next] | [standalone]

#1395936

From	Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Date	2016-05-06 17:30 +0200
Message-ID	<rvPRw-7I4-1@gated-at.bofh.it>
In reply to	#1395916

Em Fri, 6 May 2016 18:06:49 +0300
Jani Nikula <jani.nikula@intel.com> escreveu:

> On Fri, 06 May 2016, Markus Heiser <markus.heiser@darmarit.de> wrote:
> > @Jonathan: what do you think? Should I prepare a patch
> > with a basic reST (sphinx) build infrastructure, including
> >
> > * a folder for sphinx docs:
> >
> >   ./Documentation/sphinx/  
> 
> I'm already working on a patch series taking a different approach. I
> don't think we should hide the documentation under an extra folder named
> after a tool. Actually, I'm strongly opposed to that.
> 
> Instead, we should place the Sphinx stuff directly under Documentation,
> and have Sphinx recursively pick up all the *.rst files. We should
> promote gradually switching to lightweight markup and integration of the
> documents into one system. This process should be as little disruptive
> as possible.

We won't avoid the need of moving things among directories, as we
have lots of stuff under DocBook/ dir (btw, named after the toolchain).

Ok, if we put the .rst files at Documentation, we very likely reduce
the number of renames, but we'll increase the Makefile complexity,
and the risk of breakages. One alternative would be to put the sphinx
stuff on a separate Makefile, but using multiple makefiles on a single
dir is not standard at the Kernel.

> If someone wants to convert a .txt document to .rst and get it processed
> by Sphinx, it should be as simple as renaming the file, doing the
> necessary edits, and adding it to a toctree. Imagine gradually
> converting the files under, say, Documentation/kbuild. Why should the
> .rst files be moved under another directory? They should stay alongside
> the .txt files under the same directory. There's bound to be a lot of
> people who'll never use Sphinx, and will expect to find the good old
> plain text files (albeit with some markup) where they always were.

Well, git will show the change as a rename, no matter if the
directory name changes or not (except if we keep the rst files
with .txt extension), but I agree with you that people will expect
to see text files at Documentation, and most will just read it without
caring to run Sphinx.

-- 
Thanks,
Mauro

[toc] | [prev] | [next] | [standalone]

#1395946

From	Markus Heiser <markus.heiser@darmarit.de>
Date	2016-05-06 17:40 +0200
Message-ID	<rvQ1c-7PI-15@gated-at.bofh.it>
In reply to	#1395916

Am 06.05.2016 um 17:06 schrieb Jani Nikula <jani.nikula@intel.com>:

> On Fri, 06 May 2016, Markus Heiser <markus.heiser@darmarit.de> wrote:
>> @Jonathan: what do you think? Should I prepare a patch
>> with a basic reST (sphinx) build infrastructure, including
>> 
>> * a folder for sphinx docs:
>> 
>>  ./Documentation/sphinx/
> 
> I'm already working on a patch series taking a different approach. I
> don't think we should hide the documentation under an extra folder named
> after a tool. Actually, I'm strongly opposed to that.

Could you post a link to a repo? / thanks 

There is no need for concurrency, let's work together on your repo. 
Within my POC I realized similar building processes we will need in the
kernel sources ... where you have cascading configuration. A base 
configuration which fits for all common cases and (if needed) a 
*per-book* configuration.

At the end, when it comes to generate pdf books/articles, man pages 
and e.g. texinfo files out of a sphinx-project you will need a build
infrastructure like this.

> Instead, we should place the Sphinx stuff directly under Documentation,
> and have Sphinx recursively pick up all the *.rst files. We should
> promote gradually switching to lightweight markup and integration of the
> documents into one system. This process should be as little disruptive
> as possible.
> 
> If someone wants to convert a .txt document to .rst and get it processed
> by Sphinx, it should be as simple as renaming the file, doing the
> necessary edits, and adding it to a toctree. Imagine gradually
> converting the files under, say, Documentation/kbuild. Why should the
> .rst files be moved under another directory? They should stay alongside
> the .txt files under the same directory. There's bound to be a lot of
> people who'll never use Sphinx, and will expect to find the good old
> plain text files (albeit with some markup) where they always were.
> 

Ok, I agree with you in the fact that a additional "sphinx" folder
is unrewarding.

This means (e.g.) a migrated Documentation/DocBook/gpu book should placed
in Documentation/gpu ... but don' try to merge all (Doc-)Books and .txt-files
into one sphinx project!

You will need on sphinx-project for each DocBook and one single 
sphinx-project where you collect the .txt to .rst migrated files.

Am 06.05.2016 um 17:23 schrieb Mauro Carvalho Chehab <mchehab@osg.samsung.com>:

> We won't avoid the need of moving things among directories, as we
> have lots of stuff under DocBook/ dir (btw, named after the toolchain).

Yes, it is named by the toolchain, but no one reads xml-files. Reading
text files is common.

> Ok, if we put the .rst files at Documentation, we very likely reduce
> the number of renames, but we'll increase the Makefile complexity,
> and the risk of breakages.

I don't see a great potential of breakages ...  if we place every book
in a separated folder and have one project which collects the .txt files
(see above).

--Markus--

[toc] | [prev] | [next] | [standalone]

#1395954

From	Jani Nikula <jani.nikula@intel.com>
Date	2016-05-06 18:00 +0200
Message-ID	<rvQkz-7YH-15@gated-at.bofh.it>
In reply to	#1395946

On Fri, 06 May 2016, Markus Heiser <markus.heiser@darmarit.de> wrote:
> Am 06.05.2016 um 17:06 schrieb Jani Nikula <jani.nikula@intel.com>:
>
>> On Fri, 06 May 2016, Markus Heiser <markus.heiser@darmarit.de> wrote:
>>> @Jonathan: what do you think? Should I prepare a patch
>>> with a basic reST (sphinx) build infrastructure, including
>>> 
>>> * a folder for sphinx docs:
>>> 
>>>  ./Documentation/sphinx/
>> 
>> I'm already working on a patch series taking a different approach. I
>> don't think we should hide the documentation under an extra folder named
>> after a tool. Actually, I'm strongly opposed to that.
>
> Could you post a link to a repo? / thanks

Very much a work-in-progress
https://cgit.freedesktop.org/~jani/drm/log/?h=sphinx

I was hoping to polish it a bit more before showing it to the world.

> There is no need for concurrency, let's work together on your repo. 
> Within my POC I realized similar building processes we will need in the
> kernel sources ... where you have cascading configuration. A base 
> configuration which fits for all common cases and (if needed) a 
> *per-book* configuration.
>
> At the end, when it comes to generate pdf books/articles, man pages 
> and e.g. texinfo files out of a sphinx-project you will need a build
> infrastructure like this.

...

> You will need on sphinx-project for each DocBook and one single 
> sphinx-project where you collect the .txt to .rst migrated files.

Surely you know more about Sphinx than I do, but I specifically would
like to include e.g. gpu documentation in the main build. I'm really
hoping we can have *additional* configuration files for special cases
(only) as needed.

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Technology Center

[toc] | [prev] | [next] | [standalone]

#1395930

From	Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Date	2016-05-06 17:20 +0200
Message-ID	<rvPHQ-7CY-11@gated-at.bofh.it>
In reply to	#1395896

Em Fri, 6 May 2016 16:27:21 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:

> Hi all, hi Jonathan,
> 
> Am 06.05.2016 um 15:42 schrieb Mauro Carvalho Chehab <mchehab@osg.samsung.com>:
> 
> > Em Fri, 6 May 2016 15:32:35 +0200
> > Markus Heiser <markus.heiser@darmarit.de> escreveu:
> >   
> >> Hi Mauro,
> >> 
> >> Am 06.05.2016 um 13:35 schrieb Mauro Carvalho Chehab <mchehab@osg.samsung.com>:
> >>   
> >>> Markus,
> >>> 
> >>> Em Fri, 6 May 2016 13:23:06 +0200
> >>> Markus Heiser <markus.heiser@darmarit.de> escreveu:
> >>>   
> >>>> 
> >>>> In this conf.py you have to *register* your folder with the extensions.
> >>>> A few words about the flat-table extension and a (future) kernel-doc one:    
> >>> 
> >>> ...
> >>>   
> >>>> The flat-table is a pure docutils (the layer below sphinx) extension which
> >>>> is not application specific, so I will ask for moving it to the docutils 
> >>>> upstream.     
> >>> 
> >>> So, if I understood well, your proposal is to have a conf.py and the
> >>> flat-table (plus other extensions) at the Kernel tree.    
> >> 
> >> Each book (better call it root-node) is a Sphinx-project, each 
> >> Sphinx-project need a conf.py file (the build configuration file)
> >> in its reST-source tree.
> >> 
> >> * http://www.sphinx-doc.org/en/stable/config.html
> >>   
> >>> Assuming that docutils upstream receives the flat-table extension
> >>> (and eventually modifies it), while the new version doesn't arrive
> >>> all distros, we'll end by having some developers using the newer
> >>> docutils with the extension, plus others using the in-tree one.
> >>> 
> >>> Is there a way to specify at the conf.py what extension variant
> >>> should it use, in case of both the in-tree or the docutils have
> >>> the same?    
> >> 
> >> The build configuration file is a regular python file, you can 
> >> implement conditions whatever you want/need.
> >>   
> >>> This could be trickier if they end by modifying the extension,
> >>> but we can always backport the latest version, if they change the
> >>> API.    
> >> 
> >> As far as i know, the docutils API is stable since 2002. In the
> >> meantime there has been so many application build on it that
> >> it is not realistic, you will see a not backward compatibly 
> >> change.
> >> 
> >> The docutils project is conservative, very conservative, IMO to
> >> conservative.
> >> 
> >> Today I'am doubtful if it isn't better I would merge it sphinx
> >> upstream. I have to discuss this with some maintainers, but 
> >> before I have to persuade myself, that all aspects are covered
> >> and the implementations are robust. We are at the beginning and
> >> we should not fear about every bit which could happen in the future.
> >> 
> >> The sphinx / docutils bottom plate gives us a number of degrees 
> >> of freedom to find answers to question we have not yet asked. ;-)  
> > 
> > Ok. So, from what I understand, once Sphinx support is added at
> > Kernel upstream, we could convert the media docbook to
> > reST+flat-table extension, adding such extension either on a shared
> > place or only for the media DocBook build, together with its
> > conf.py.
> >   
> 
> Yes, in your media-conf you could decide to use a extension.
> 
> > Once it reaches upstream (either sphinx or docutils), we can
> > work to make it integrate better with upstream as needed.
> > 
> > Right?  
> 
> yes, right :-)
> 
> > If so, I'm ok with merging it as soon as possible.  
> 
> If we advice a merge of the flat-table directive we should 
> bundle this with the (to implement) "kernel-doc" directive ...
> 
> > In reST the directive might look like:
> > 
> > <reST-SNIP> -----
> > Device Instance and Driver Handling
> > ===================================
> > 
> > .. kernel-doc::  drivers/gpu/drm/drm_drv.c
> >   :doc:      driver instance overview
> >   :exported:
> > 
> > <reST-SNAP> -----  
> 
> and the patches from my kernel-doc perl script to produce
> reST from source code comments.
> 
> With this bundle within the kernel tree we have a good starting
> point to compose reST documents from scratch and to migrate book
> by book from DocBook to reST.
> 
> I insist to migrate book by book, because there are some
> broken books. Broken by that, that some sources have changed
> but not the corresponding documentation which use the comments
> of these sources ... grap "Ooops" in the builded (xml or rst) docs.
> 
> E.g. I greped the .rst file and found the following Oops in the migrated books:
> 
> ./books/mtdnand/pubfunctions-000-012.rst:13:Oops
> ./books/scsi/mid_layer-000-001-016-003.rst:13:Oops
> ./books/s390-drivers/ccw-000-004-003.rst:13:Oops
> ./books/device-drivers/devdrivers-000-003-048.rst:13:Oops
> ./books/device-drivers/devdrivers-000-003-050.rst:13:Oops
> ./books/device-drivers/Basics-000-001-002.rst:13:Oops
> ./books/device-drivers/devdrivers-000-003-031.rst:13:Oops
> ./books/device-drivers/Basics-000-009-032.rst:13:Oops
> ./books/kernel-api/kernel-lib-000-004-008.rst:13:Oops
> ./books/gadget/api-000-011-005.rst:13:Oops
> ./books/gadget/api-000-011-009.rst:13:Oops
> ./books/gadget/api-000-011-007.rst:13:Oops
> ./books/gadget/api-000-011-003.rst:13:Oops
> ./books/gadget/api-000-011-011.rst:13:Oops
> ./books/genericirq/intfunctions-000-009.rst:13:Oops
> 
> ----- Summarize ----
> 
> @Jonathan: what do you think? Should I prepare a patch
> with a basic reST (sphinx) build infrastructure, including
> 
> * a folder for sphinx docs:
> 
>   ./Documentation/sphinx/
> 
> * flat-table & kernel-doc extension at
> 
>   ./Documentation/sphinx/extensions
> 
> * a patch with rst-Output for the kernel-doc perl script at
> 
>   ./scripts/kernel-doc

Maybe the best here is to add a kernel-doc-rest script, avoiding
the risk of regressions while we don't migrate all the books.

> * An example document "HowTo document with reST" at
> 
>   ./Documentation/sphinx/kernel-doc-rst-howto
> 
>   which at minimum describes the "flat-table" and "kernel-doc" 
>   directive and the requirements for a building docs.
> 
> * a make file which fit into the kernel Makefile infrastructure (not 
>   the one created by sphinx-quickstart).

Works for me. Btw, if we're following the above, as the sphinx
build will be independent and won't affect the current documentation
build, then perhaps we could merge it on 4.7 merge window.

While Documentation for things like device-drivers.tmpl should of be
handled via the documentation tree, if we apply the needed
infrastructure, each subsystem maintainer would be freed to work
on the docs that are specific to their own subsystem. So, in my case,
I can handle the needed patches for the media subsystem DocBook via the
media development tree (as I currently do for
Documentation/DocBook/media).

Regards,
Mauro

[toc] | [prev] | [next] | [standalone]

#1394501

From	Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Date	2016-05-04 18:20 +0200
Message-ID	<rv7GO-7iY-21@gated-at.bofh.it>
In reply to	#1394116

Em Wed, 4 May 2016 11:34:08 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:

> Hi all, (hi Jonathan, please take note of my offer below)
> 
> Am 03.05.2016 um 16:31 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>:
> 
> > Hi all,
> > 
> > So sounds like moving ahead with rst/sphinx is the option that should
> > allow us to address everyone's concerns eventually? Of course the
> > first one won't have it all (media seems really tricky), ...  
> 
> BTW: Mauro mentioned that ASCII-art tables are not diff-friendly ... 
> 
> Am 18.04.2016 um 13:16 schrieb Mauro Carvalho Chehab <mchehab@osg.samsung.com>:
> 
> > With that sense, the "List tables" format is also not good, as
> > one row addition would generate several hunks (one for each column
> > of the table), making harder to review the patch by just looking at
> > the diff.  
> 
> For this, I wrote the "flat-table" reST-directive, which adds 
> missing cells automatically:
> 
> doc:    http://return42.github.io/sphkerneldoc/articles/table_concerns.html#flat-table
> source: https://github.com/return42/sphkerneldoc/blob/master/doc/extensions/rstFlatTable.py

Yeah, this should address the lack of a proper way to markup cell/row
spans, providing the additional bits for the tables we have at media.

Yet, there are some issues with table conversions. See below.

> 
> I used "flat-table" to migrate all DocBook-XML documents to reST. With this
> directive, I also managed to migrate the complete media book (no more TODOs)
> incl. the large tables like them from subdev-formats:
> 
> https://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/subdev-formats.html
> 
> (Rendering large tables is a general discussion which should not take place in this MT)

Some tables, like the one here:
	https://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/control.html

are truncated (tested with Mozilla and Chrome), and part of the information is
lost due to that.

Regards,
Mauro

[toc] | [prev] | [next] | [standalone]

#1395753

From	Markus Heiser <markus.heiser@darmarit.de>
Date	2016-05-06 12:10 +0200
Message-ID	<rvKRR-32q-37@gated-at.bofh.it>
In reply to	#1394501

Hi Mauro,

Am 04.05.2016 um 18:15 schrieb Mauro Carvalho Chehab <mchehab@osg.samsung.com>:

> Em Wed, 4 May 2016 11:34:08 +0200
> Markus Heiser <markus.heiser@darmarit.de> escreveu:
> 
>> Hi all, (hi Jonathan, please take note of my offer below)
>> 
>> Am 03.05.2016 um 16:31 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>:
>> 
>>> Hi all,
>>> 
>>> So sounds like moving ahead with rst/sphinx is the option that should
>>> allow us to address everyone's concerns eventually? Of course the
>>> first one won't have it all (media seems really tricky), ...  
>> 
>> BTW: Mauro mentioned that ASCII-art tables are not diff-friendly ... 
>> For this, I wrote the "flat-table" reST-directive, which adds 
>> missing cells automatically:
>> 
>> doc:    http://return42.github.io/sphkerneldoc/articles/table_concerns.html#flat-table
>> source: https://github.com/return42/sphkerneldoc/blob/master/doc/extensions/rstFlatTable.py
> 
> Yeah, this should address the lack of a proper way to markup cell/row
> spans, providing the additional bits for the tables we have at media.
> 
> Yet, there are some issues with table conversions. See below.
> Some tables, like the one here:
> 	https://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/control.html
> 
> are truncated (tested with Mozilla and Chrome), and part of the information is
> lost due to that.

Not a problem of rendering. This was a bug in the magration from DocBook to reST. 
You might remember that we have discussed, that some of the tables are better
marked-up as definition lists. This was (the last) one I forgot to convert to 
a definition list ... I hope it was the last one, if not and you find more 
or other broken parts, please inform me (on the linux-tv mailing, or direct).

--Markus--

[toc] | [prev] | [next] | [standalone]

#1395973

From	Markus Heiser <markus.heiser@darmarit.de>
Date	2016-05-06 18:40 +0200
Message-ID	<rvQXf-j2-9@gated-at.bofh.it>
In reply to	#1395753

Hi Mauro,

Am 06.05.2016 um 13:03 schrieb Mauro Carvalho Chehab <mchehab@osg.samsung.com>:
> Yeah, it looks better, however table truncation seem to be
> happening also on other parts, like the tables on this page:
> 
> 	https://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/pixfmt-packed-rgb.html
> 	(original table: https://linuxtv.org/downloads/v4l-dvb-apis/packed-rgb.html)
> This table should contain 32 bits, but only the first 7 bits are shown
> 
> and those (among others):
> 	https://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/pixfmt-y41p.html
> 	https://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/dev-sliced-vbi.html
> 	https://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/subdev-formats.html
> 
> Hmm... after looking more carefully, it added a horizontal scroll bar.
> That looks ugly, IMHO, and makes harder to understand its contents. The
> last one, in particular (https://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/subdev-formats.html),
> is a big table on both horiz and vert dimensions. Try to read how the
> bits are packed on a random line in the middle of the table, like
> MEDIA_BUS_FMT_BGR565_2X8_LE and you'll understand what I mean.

I know what you mean ;-) ... I'am also unhappy, but I will address this point
later when it goes to finish the layout.

Currently lets focus on contend and (the two)extensions.

> The table here looks weird (although it is correct):
> 	https://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/pixfmt-srggb10p.html
> 	(original table: https://linuxtv.org/downloads/v4l-dvb-apis/pixfmt-srggb10p.html)
> 

I validated this and the other tables you mentioned above ... these are 
all correct migrated ... it is 1:1 translated from DocBook ... they
might be show different to this what you know from your docbook
toolchain, because in the docbook-html you have no table grids and 
wrong cellspans are not clear ... sometimes, like in the last example 
you gave:

	    <tgroup cols="5" align="center">
	      <colspec align="left" colwidth="2*" />
	      <tbody valign="top">

a colspec might ambiguous ... so there is no clear role to migrate.

> It seems that Sphinx is assuming something like "A4 portrait" for
> the margins, while those big tables would only fit (in PDF) as
> "A4 landscape".

No, no, no ;-)

Sphinx assumes nothing about the layout, sphinx and the underlying
docutils mostly juggling with nodes and the writers in e.g. the
html-writer, outputs a clear HTML without any style but with classified 
HTML tags. Styling is done in the presentation layer, in HTML, it is 
done in CSS.

> I guess the better would be to not limit the right
> margin or to change it where those big tables happen, in order to
> allow PDF generation.

Generating PDF has nothing to do with generating HTML. To generate
PDF there is a other writer, the latex2e writer, which produce 
LaTeX markup from which you build PDF or other printed-like medias.
 
> 
> There are also some tables that went wrong. See the Color Sample
> Location table at:
> 	https://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/pixfmt-yuv444m.html
> I'm pretty sure we'll need to fix some cases like that manually.
> I didn't check, but perhaps, in this case, the DocBook were using 
> empty columns just to make the table bigger. If so, this is not a
> problem with the conversion, and should be manually fixed later.

+1

Yes, lets do it manually later ... what I have in my POC is a automated
process, it is hard to consider individuals in an automatic process.
Making details *nicer* and making ambiguous markups clear is manually
done in minutes where I need hours to implement this in a automated
process.

Aside, from:  http://docutils.sourceforge.net/docs/peps/pep-0258.html

Docutils Project Model -- Project components and data flow:

                 +---------------------------+
                 |        Docutils:          |
                 | docutils.core.Publisher,  |
                 | docutils.core.publish_*() |
                 +---------------------------+
                  /            |            \
                 /             |             \
        1,3,5   /        6     |              \ 7
       +--------+       +-------------+       +--------+
       | READER | ----> | TRANSFORMER | ====> | WRITER |
       +--------+       +-------------+       +--------+
        /     \\                                  |
       /       \\                                 |
 2    /      4  \\                             8  |
+-------+   +--------+                        +--------+
| INPUT |   | PARSER |                        | OUTPUT |
+-------+   +--------+                        +--------+


This is a bit simplified, because we use sphinx, which 
has "builders" and sits on top of this architecture.
But it might help to see, that processes like reading,
transforming and writing are discrete.

In short: readers (the reST file reader) are creating node trees,
which are transformed by a transformer (e.g. a HTML transformer),
the writer only writes the output to a file (and copies some files
like CSS files).

If I say "HTML-writer" I address the unity off the HTML-transformer
plus the HTML-writer. In Sphinx terminus/architecture, replace the
word writer with the word "builder" ... there you have (e.g.) a 
"HTML builder" and a "LaTeX builder".

--Markus--

[toc] | [prev] | [next] | [standalone]

#1395980

From	Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Date	2016-05-06 19:10 +0200
Message-ID	<rvRqh-Q6-5@gated-at.bofh.it>
In reply to	#1395973

Em Fri, 6 May 2016 18:26:10 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:

> Hi Mauro,
> 
> Am 06.05.2016 um 13:03 schrieb Mauro Carvalho Chehab <mchehab@osg.samsung.com>:
> > Yeah, it looks better, however table truncation seem to be
> > happening also on other parts, like the tables on this page:
> > 
> > 	https://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/pixfmt-packed-rgb.html
> > 	(original table: https://linuxtv.org/downloads/v4l-dvb-apis/packed-rgb.html)
> > This table should contain 32 bits, but only the first 7 bits are shown
> > 
> > and those (among others):
> > 	https://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/pixfmt-y41p.html
> > 	https://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/dev-sliced-vbi.html
> > 	https://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/subdev-formats.html
> > 
> > Hmm... after looking more carefully, it added a horizontal scroll bar.
> > That looks ugly, IMHO, and makes harder to understand its contents. The
> > last one, in particular (https://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/subdev-formats.html),
> > is a big table on both horiz and vert dimensions. Try to read how the
> > bits are packed on a random line in the middle of the table, like
> > MEDIA_BUS_FMT_BGR565_2X8_LE and you'll understand what I mean.  
> 
> I know what you mean ;-) ... I'am also unhappy, but I will address this point
> later when it goes to finish the layout.
> 
> Currently lets focus on contend and (the two)extensions.

OK.

> > The table here looks weird (although it is correct):
> > 	https://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/pixfmt-srggb10p.html
> > 	(original table: https://linuxtv.org/downloads/v4l-dvb-apis/pixfmt-srggb10p.html)
> >   
> 
> I validated this and the other tables you mentioned above ... these are 
> all correct migrated ... it is 1:1 translated from DocBook ... they
> might be show different to this what you know from your docbook
> toolchain, because in the docbook-html you have no table grids and 
> wrong cellspans are not clear ... sometimes, like in the last example 
> you gave:
> 
> 	    <tgroup cols="5" align="center">
> 	      <colspec align="left" colwidth="2*" />
> 	      <tbody valign="top">
> 
> a colspec might ambiguous ... so there is no clear role to migrate.

Ok, that's what I was thinking. Ok, this can be fixed later manually,
where needed. Of course one way would be to disable grids on those
tables, but I would instead fix it.

> 
> > It seems that Sphinx is assuming something like "A4 portrait" for
> > the margins, while those big tables would only fit (in PDF) as
> > "A4 landscape".  
> 
> No, no, no ;-)
> 
> Sphinx assumes nothing about the layout, sphinx and the underlying
> docutils mostly juggling with nodes and the writers in e.g. the
> html-writer, outputs a clear HTML without any style but with classified 
> HTML tags. Styling is done in the presentation layer, in HTML, it is 
> done in CSS.

Hmm... Then there's something deadly wrong at CSS template, as it is
shown texts only at half of my horizontal res (1920).

Probably this is the culpit:

      .container { margin: 50px auto 40px auto; width: 600px; text-align: center; }

width is set to 600px, instead of using a percentage, like 100%
(or 90%).

> 
> > I guess the better would be to not limit the right
> > margin or to change it where those big tables happen, in order to
> > allow PDF generation.  
> 
> Generating PDF has nothing to do with generating HTML. To generate
> PDF there is a other writer, the latex2e writer, which produce 
> LaTeX markup from which you build PDF or other printed-like medias.

Ok.

>  
> > 
> > There are also some tables that went wrong. See the Color Sample
> > Location table at:
> > 	https://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/pixfmt-yuv444m.html
> > I'm pretty sure we'll need to fix some cases like that manually.
> > I didn't check, but perhaps, in this case, the DocBook were using 
> > empty columns just to make the table bigger. If so, this is not a
> > problem with the conversion, and should be manually fixed later.  
> 
> +1
> 
> Yes, lets do it manually later ... what I have in my POC is a automated
> process, it is hard to consider individuals in an automatic process.
> Making details *nicer* and making ambiguous markups clear is manually
> done in minutes where I need hours to implement this in a automated
> process.

Yeah, we should not try to fix everything via auto-scripts, and
spending time right now with manual fixes will be wasted, as we need
to run it at the latest media documentation, as changes might have
happened upstream.

> 
> Aside, from:  http://docutils.sourceforge.net/docs/peps/pep-0258.html
> 
> Docutils Project Model -- Project components and data flow:
> 
>                  +---------------------------+
>                  |        Docutils:          |
>                  | docutils.core.Publisher,  |
>                  | docutils.core.publish_*() |
>                  +---------------------------+
>                   /            |            \
>                  /             |             \
>         1,3,5   /        6     |              \ 7
>        +--------+       +-------------+       +--------+
>        | READER | ----> | TRANSFORMER | ====> | WRITER |
>        +--------+       +-------------+       +--------+
>         /     \\                                  |
>        /       \\                                 |
>  2    /      4  \\                             8  |
> +-------+   +--------+                        +--------+
> | INPUT |   | PARSER |                        | OUTPUT |
> +-------+   +--------+                        +--------+
> 
> 
> This is a bit simplified, because we use sphinx, which 
> has "builders" and sits on top of this architecture.
> But it might help to see, that processes like reading,
> transforming and writing are discrete.
> 
> In short: readers (the reST file reader) are creating node trees,
> which are transformed by a transformer (e.g. a HTML transformer),
> the writer only writes the output to a file (and copies some files
> like CSS files).
> 
> If I say "HTML-writer" I address the unity off the HTML-transformer
> plus the HTML-writer. In Sphinx terminus/architecture, replace the
> word writer with the word "builder" ... there you have (e.g.) a 
> "HTML builder" and a "LaTeX builder".
> 
> --Markus--
> 
> 
> 
> 
> 


-- 
Thanks,
Mauro

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

csiph-web

Re: Kernel docs: muddying the waters a bit

Contents

#1395118

#1395781

#1395802

#1395896

#1395916

#1395936

#1395946

#1395954

#1395930

#1394501

#1395753

#1395973

#1395980