Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1434485 > unrolled thread

[PATCH] doc: flat-table directive

Back to article view | Back to linux.kernel

Contents

  [PATCH] doc: flat-table directive Markus Heiser <markus.heiser@darmarIT.de> - 2016-06-30 14:10 +0200
    [PATCH] doc-rst: flat-table directive - initial implementation Markus Heiser <markus.heiser@darmarIT.de> - 2016-06-30 14:10 +0200
    Re: [PATCH] doc: flat-table directive Jonathan Corbet <corbet@lwn.net> - 2016-06-30 21:10 +0200
      Re: [PATCH] doc: flat-table directive Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-06-30 21:40 +0200
      Re: [PATCH] doc: flat-table directive Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-06-30 22:00 +0200
        Re: [PATCH] doc: flat-table directive Markus Heiser <markus.heiser@darmarit.de> - 2016-07-01 10:50 +0200
          Re: [PATCH] doc: flat-table directive Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-07-01 12:00 +0200
            Re: [PATCH] doc: flat-table directive Jani Nikula <jani.nikula@intel.com> - 2016-07-01 12:50 +0200
              Re: [PATCH] doc: flat-table directive Markus Heiser <markus.heiser@darmarit.de> - 2016-07-01 13:20 +0200
                Re: [PATCH] doc: flat-table directive Jani Nikula <jani.nikula@intel.com> - 2016-07-01 14:00 +0200
                  Re: [PATCH] doc: flat-table directive Markus Heiser <markus.heiser@darmarit.de> - 2016-07-01 14:10 +0200
                    Re: [PATCH] doc: flat-table directive Jani Nikula <jani.nikula@intel.com> - 2016-07-01 14:30 +0200
                      Re: [PATCH] doc: flat-table directive Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-07-01 15:20 +0200
                  [docs-next PATCH] Documentation: add cleanmediadocs to the documentation targets Jani Nikula <jani.nikula@intel.com> - 2016-07-01 14:10 +0200
                    [docs-next PATCH] Documentation/sphinx: skip build if user requested specific DOCBOOKS Jani Nikula <jani.nikula@intel.com> - 2016-07-01 14:30 +0200
                      Re: [docs-next PATCH] Documentation/sphinx: skip build if user requested specific DOCBOOKS Markus Heiser <markus.heiser@darmarit.de> - 2016-07-01 14:40 +0200
                      Re: [docs-next PATCH] Documentation/sphinx: skip build if user  requested specific DOCBOOKS Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-07-01 15:30 +0200
                        Re: [docs-next PATCH] Documentation/sphinx: skip build if user requested specific DOCBOOKS Jani Nikula <jani.nikula@intel.com> - 2016-07-01 15:40 +0200
                          Re: [docs-next PATCH] Documentation/sphinx: skip build if user  requested specific DOCBOOKS Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-07-01 17:10 +0200
              Re: [PATCH] doc: flat-table directive Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-07-01 15:00 +0200
              Re: [PATCH] doc: flat-table directive Jonathan Corbet <corbet@lwn.net> - 2016-07-01 15:30 +0200
                Re: [PATCH] doc: flat-table directive Markus Heiser <markus.heiser@darmarit.de> - 2016-07-01 16:50 +0200
                  Re: [PATCH] doc: flat-table directive Jani Nikula <jani.nikula@intel.com> - 2016-07-01 19:30 +0200
                    Re: [PATCH] doc: flat-table directive Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-07-01 20:20 +0200
                      Re: [PATCH] doc: flat-table directive Jani Nikula <jani.nikula@intel.com> - 2016-07-01 20:50 +0200
            Re: [PATCH] doc: flat-table directive Markus Heiser <markus.heiser@darmarit.de> - 2016-07-01 14:20 +0200
              Re: [PATCH] doc: flat-table directive Jani Nikula <jani.nikula@intel.com> - 2016-07-01 15:20 +0200
                Re: [PATCH] doc: flat-table directive Markus Heiser <markus.heiser@darmarit.de> - 2016-07-01 16:20 +0200
                  Re: [PATCH] doc: flat-table directive Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-07-01 17:10 +0200
                  Re: [PATCH] doc: flat-table directive Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-07-01 17:10 +0200
                  Re: [PATCH] doc: flat-table directive Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-07-07 01:00 +0200
              Re: [PATCH] doc: flat-table directive Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-07-01 15:20 +0200
        Re: [PATCH] doc: flat-table directive Jani Nikula <jani.nikula@intel.com> - 2016-07-01 14:30 +0200
      Re: [PATCH] doc: flat-table directive Markus Heiser <markus.heiser@darmarit.de> - 2016-07-01 08:40 +0200
    Captions numbering support Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-07-01 20:30 +0200

Page 1 of 2  [1] 2  Next page →

#1434485 — [PATCH] doc: flat-table directive

Hi Jonathan,

this is my flat-table patch on top of your docs-next branch / we discussed on
the ML[1]

[1] http://mid.gmane.org/573D454C-2F55-4DD7-9C16-00B3897AEFB0@darmarit.de

Markus Heiser (1):
  doc-rst: flat-table directive - initial implementation

 Documentation/conf.py                  |   2 +-
 Documentation/kernel-documentation.rst |  85 ++++++++
 Documentation/sphinx/rstFlatTable.py   | 365 +++++++++++++++++++++++++++++++++
 3 files changed, 451 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/sphinx/rstFlatTable.py

-- 
sphinx-4.8

[toc] | [next] | [standalone]

#1434486 — [PATCH] doc-rst: flat-table directive - initial implementation

Implements the reST flat-table directive.

The ``flat-table`` is a double-stage list similar to the ``list-table`` with
some additional features:

* column-span: with the role ``cspan`` a cell can be extended through
  additional columns

* row-span: with the role ``rspan`` a cell can be extended through
  additional rows

* auto span rightmost cell of a table row over the missing cells on the right
  side of that table-row.  With Option ``:fill-cells:`` this behavior can
  changed from *auto span* to *auto fill*, which automaticly inserts (empty)

list tables

  The *list tables* formats are double stage lists. Compared to the
  ASCII-art they migth be less comfortable for readers of the
  text-files. Their advantage is, that they are easy to create/modify
  and that the diff of a modification is much more meaningfull, because
  it is limited to the modified content.

The initial implementation was taken from the sphkerneldoc project [1]

[1] https://github.com/return42/sphkerneldoc/commits/master/scripts/site-python/linuxdoc/rstFlatTable.py

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
---
 Documentation/conf.py                  |   2 +-
 Documentation/kernel-documentation.rst |  85 ++++++++
 Documentation/sphinx/rstFlatTable.py   | 365 +++++++++++++++++++++++++++++++++
 3 files changed, 451 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/sphinx/rstFlatTable.py

diff --git a/Documentation/conf.py b/Documentation/conf.py
index 6cc41a0..792b633 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -28,7 +28,7 @@ sys.path.insert(0, os.path.abspath('sphinx'))
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = ['kernel-doc']
+extensions = ['kernel-doc', 'rstFlatTable']
 
 # Gracefully handle missing rst2pdf.
 try:
diff --git a/Documentation/kernel-documentation.rst b/Documentation/kernel-documentation.rst
index ad3506c..6066c74 100644
--- a/Documentation/kernel-documentation.rst
+++ b/Documentation/kernel-documentation.rst
@@ -107,6 +107,91 @@ Here are some specific guidelines for the kernel documentation:
   the order as encountered."), having the higher levels the same overall makes
   it easier to follow the documents.
 
+list tables
+-----------
+
+We recommend to use *list table* formats. The *list table* formats are double
+stage lists. Compared to the ASCII-art they migth not be as comfortable for
+readers of the text-files. Their advantage is, that they are easy to
+create/modify and that the diff of a modification is much more meaningfull,
+because it is limited to the modified content.
+
+The ``flat-table`` is a double-stage list similar to the ``list-table`` with
+some additional features:
+
+* column-span: with the role ``cspan`` a cell can be extended through
+  additional columns
+
+* row-span: with the role ``rspan`` a cell can be extended through
+  additional rows
+
+* auto span rightmost cell of a table row over the missing cells on the right
+  side of that table-row.  With Option ``:fill-cells:`` this behavior can
+  changed from *auto span* to *auto fill*, which automaticly inserts (empty)
+  cells instead of spanning the last cell.
+
+options:
+
+* ``:header-rows:``   [int] count of header rows
+* ``:stub-columns:``  [int] count of stub columns
+* ``:widths:``        [[int] [int] ... ] widths of columns
+* ``:fill-cells:``    instead of autospann missing cells, insert missing cells
+
+roles:
+
+* ``:cspan:`` [int] additionale columns (*morecols*)
+* ``:rspan:`` [int] additionale rows (*morerows*)
+
+The example below shows how to use this markup.  The first level of the staged
+list is the *table-row*. In the *table-row* there is only one markup allowed,
+the list of the cells in this *table-row*. Exception are *comments* ( ``..`` )
+and *targets* (e.g. a ref to ``:ref:`last row <last row>``` / :ref:`last row
+<last row>`).
+
+.. code-block:: rst
+
+   .. flat-table:: table title
+      :widths: 2 1 1 3
+
+      * - head col 1
+        - head col 2
+        - head col 3
+        - head col 4
+
+      * - column 1
+        - field 1.1
+        - field 1.2 with autospan
+
+      * - column 2
+        - field 2.1
+        - :rspan:`1` :cspan:`1` field 2.2 - 3.3
+
+      * .. _`last row`:
+
+        - column 3
+
+Rendered as:
+
+   .. flat-table:: table title
+      :widths: 2 1 1 3
+
+      * - head col 1
+        - head col 2
+        - head col 3
+        - head col 4
+
+      * - column 1
+        - field 1.1
+        - field 1.2 with autospan
+
+      * - column 2
+        - field 2.1
+        - :rspan:`1` :cspan:`1` field 2.2 - 3.3
+
+      * .. _`last row`:
+
+        - column 3
+
 
 Including kernel-doc comments
 =============================
diff --git a/Documentation/sphinx/rstFlatTable.py b/Documentation/sphinx/rstFlatTable.py
new file mode 100644
index 0000000..26db852
--- /dev/null
+++ b/Documentation/sphinx/rstFlatTable.py
@@ -0,0 +1,365 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8; mode: python -*-
+# pylint: disable=C0330, R0903, R0912
+
+u"""
+    flat-table
+    ~~~~~~~~~~
+
+    Implementation of the ``flat-table`` reST-directive.
+
+    :copyright:  Copyright (C) 2016  Markus Heiser
+    :license:    GPL Version 2, June 1991 see linux/COPYING for details.
+
+    The ``flat-table`` (:py:class:`FlatTable`) is a double-stage list similar to
+    the ``list-table`` with some additional features:
+
+    * *column-span*: with the role ``cspan`` a cell can be extended through
+      additional columns
+
+    * *row-span*: with the role ``rspan`` a cell can be extended through
+      additional rows
+
+    * *auto span* rightmost cell of a table row over the missing cells on the
+      right side of that table-row.  With Option ``:fill-cells:`` this behavior
+      can changed from *auto span* to *auto fill*, which automaticly inserts
+      (empty) cells instead of spanning the last cell.
+
+    Options:
+
+    * header-rows:   [int] count of header rows
+    * stub-columns:  [int] count of stub columns
+    * widths:        [[int] [int] ... ] widths of columns
+    * fill-cells:    instead of autospann missing cells, insert missing cells
+
+    roles:
+
+    * cspan: [int] additionale columns (*morecols*)
+    * rspan: [int] additionale rows (*morerows*)
+"""
+
+# ==============================================================================
+# imports
+# ==============================================================================
+
+import sys
+
+from docutils import nodes
+from docutils.parsers.rst import directives, roles
+from docutils.parsers.rst.directives.tables import Table
+from docutils.utils import SystemMessagePropagation
+
+# ==============================================================================
+# common globals
+# ==============================================================================
+
+# The version numbering follows numbering of the specification
+# (Documentation/books/kernel-doc-HOWTO).
+__version__  = '1.0'
+
+PY3 = sys.version_info[0] == 3
+PY2 = sys.version_info[0] == 2
+
+if PY3:
+    # pylint: disable=C0103, W0622
+    unicode     = str
+    basestring  = str
+
+# ==============================================================================
+def setup(app):
+# ==============================================================================
+
+    app.add_directive("flat-table", FlatTable)
+    roles.register_local_role('cspan', c_span)
+    roles.register_local_role('rspan', r_span)
+
+# ==============================================================================
+def c_span(name, rawtext, text, lineno, inliner, options=None, content=None):
+# ==============================================================================
+    # pylint: disable=W0613
+
+    options  = options if options is not None else {}
+    content  = content if content is not None else []
+    nodelist = [colSpan(span=int(text))]
+    msglist  = []
+    return nodelist, msglist
+
+# ==============================================================================
+def r_span(name, rawtext, text, lineno, inliner, options=None, content=None):
+# ==============================================================================
+    # pylint: disable=W0613
+
+    options  = options if options is not None else {}
+    content  = content if content is not None else []
+    nodelist = [rowSpan(span=int(text))]
+    msglist  = []
+    return nodelist, msglist
+
+
+# ==============================================================================
+class rowSpan(nodes.General, nodes.Element): pass # pylint: disable=C0103,C0321
+class colSpan(nodes.General, nodes.Element): pass # pylint: disable=C0103,C0321
+# ==============================================================================
+
+# ==============================================================================
+class FlatTable(Table):
+# ==============================================================================
+
+    u"""FlatTable (``flat-table``) directive"""
+
+    option_spec = {
+        'name': directives.unchanged
+        , 'class': directives.class_option
+        , 'header-rows': directives.nonnegative_int
+        , 'stub-columns': directives.nonnegative_int
+        , 'widths': directives.positive_int_list
+        , 'fill-cells' : directives.flag }
+
+    def run(self):
+
+        if not self.content:
+            error = self.state_machine.reporter.error(
+                'The "%s" directive is empty; content required.' % self.name,
+                nodes.literal_block(self.block_text, self.block_text),
+                line=self.lineno)
+            return [error]
+
+        title, messages = self.make_title()
+        node = nodes.Element()          # anonymous container for parsing
+        self.state.nested_parse(self.content, self.content_offset, node)
+
+        tableBuilder = ListTableBuilder(self)
+        tableBuilder.parseFlatTableNode(node)
+        tableNode = tableBuilder.buildTableNode()
+        # SDK.CONSOLE()  # print --> tableNode.asdom().toprettyxml()
+        if title:
+            tableNode.insert(0, title)
+        return [tableNode] + messages
+
+
+# ==============================================================================
+class ListTableBuilder(object):
+# ==============================================================================
+
+    u"""Builds a table from a double-stage list"""
+
+    def __init__(self, directive):
+        self.directive = directive
+        self.rows      = []
+        self.max_cols  = 0
+
+    def buildTableNode(self):
+
+        colwidths    = self.directive.get_column_widths(self.max_cols)
+        stub_columns = self.directive.options.get('stub-columns', 0)
+        header_rows  = self.directive.options.get('header-rows', 0)
+
+        table = nodes.table()
+        tgroup = nodes.tgroup(cols=len(colwidths))
+        table += tgroup
+
+
+        for colwidth in colwidths:
+            colspec = nodes.colspec(colwidth=colwidth)
+            # FIXME: It seems, that the stub method only works well in the
+            # absence of rowspan (observed by the html buidler, the docutils-xml
+            # build seems OK).  This is not extraordinary, because there exists
+            # no table directive (except *this* flat-table) which allows to
+            # define coexistent of rowspan and stubs (there was no use-case
+            # before flat-table). This should be reviewed (later).
+            if stub_columns:
+                colspec.attributes['stub'] = 1
+                stub_columns -= 1
+            tgroup += colspec
+        stub_columns = self.directive.options.get('stub-columns', 0)
+
+        if header_rows:
+            thead = nodes.thead()
+            tgroup += thead
+            for row in self.rows[:header_rows]:
+                thead += self.buildTableRowNode(row)
+
+        tbody = nodes.tbody()
+        tgroup += tbody
+
+        for row in self.rows[header_rows:]:
+            tbody += self.buildTableRowNode(row)
+        return table
+
+    def buildTableRowNode(self, row_data, classes=None):
+        classes = [] if classes is None else classes
+        row = nodes.row()
+        for cell in row_data:
+            if cell is None:
+                continue
+            cspan, rspan, cellElements = cell
+
+            attributes = {"classes" : classes}
+            if rspan:
+                attributes['morerows'] = rspan
+            if cspan:
+                attributes['morecols'] = cspan
+            entry = nodes.entry(**attributes)
+            entry.extend(cellElements)
+            row += entry
+        return row
+
+    def raiseError(self, msg):
+        error =  self.directive.state_machine.reporter.error(
+            msg
+            , nodes.literal_block(self.directive.block_text
+                                  , self.directive.block_text)
+            , line = self.directive.lineno )
+        raise SystemMessagePropagation(error)
+
+    def parseFlatTableNode(self, node):
+        u"""parses the node from a :py:class:`FlatTable` directive's body"""
+
+        if len(node) != 1 or not isinstance(node[0], nodes.bullet_list):
+            self.raiseError(
+                'Error parsing content block for the "%s" directive: '
+                'exactly one bullet list expected.' % self.directive.name )
+
+        for rowNum, rowItem in enumerate(node[0]):
+            row = self.parseRowItem(rowItem, rowNum)
+            self.rows.append(row)
+        self.roundOffTableDefinition()
+
+    def roundOffTableDefinition(self):
+        u"""Round off the table definition.
+
+        This method rounds off the table definition in :py:member:`rows`.
+
+        * This method inserts the needed ``None`` values for the missing cells
+        arising from spanning cells over rows and/or columns.
+
+        * recount the :py:member:`max_cols`
+
+        * Autospan or fill (option ``fill-cells``) missing cells on the right
+          side of the table-row
+        """
+
+        y = 0
+        while y < len(self.rows):
+            x = 0
+
+            while x < len(self.rows[y]):
+                cell = self.rows[y][x]
+                if cell is None:
+                    x += 1
+                    continue
+                cspan, rspan = cell[:2]
+                # handle colspan in current row
+                for c in range(cspan):
+                    try:
+                        self.rows[y].insert(x+c+1, None)
+                    except: # pylint: disable=W0702
+                        # the user sets ambiguous rowspans
+                        pass # SDK.CONSOLE()
+                # handle colspan in spanned rows
+                for r in range(rspan):
+                    for c in range(cspan + 1):
+                        try:
+                            self.rows[y+r+1].insert(x+c, None)
+                        except: # pylint: disable=W0702
+                            # the user sets ambiguous rowspans
+                            pass # SDK.CONSOLE()
+                x += 1
+            y += 1
+
+        # Insert the missing cells on the right side. For this, first
+        # re-calculate the max columns.
+
+        for row in self.rows:
+            if self.max_cols < len(row):
+                self.max_cols = len(row)
+
+        # fill with empty cells or cellspan?
+
+        fill_cells = False
+        if 'fill-cells' in self.directive.options:
+            fill_cells = True
+
+        for row in self.rows:
+            x =  self.max_cols - len(row)
+            if x and not fill_cells:
+                if row[-1] is None:
+                    row.append( ( x - 1, 0, []) )
+                else:
+                    cspan, rspan, content = row[-1]
+                    row[-1] = (cspan + x, rspan, content)
+            elif x and fill_cells:
+                for i in range(x):
+                    row.append( (0, 0, nodes.comment()) )
+
+    def pprint(self):
+        # for debugging
+        retVal = "[   "
+        for row in self.rows:
+            retVal += "[ "
+            for col in row:
+                if col is None:
+                    retVal += ('%r' % col)
+                    retVal += "\n    , "
+                else:
+                    content = col[2][0].astext()
+                    if len (content) > 30:
+                        content = content[:30] + "..."
+                    retVal += ('(cspan=%s, rspan=%s, %r)'
+                               % (col[0], col[1], content))
+                    retVal += "]\n    , "
+            retVal = retVal[:-2]
+            retVal += "]\n  , "
+        retVal = retVal[:-2]
+        return retVal + "]"
+
+    def parseRowItem(self, rowItem, rowNum):
+        row = []
+        childNo = 0
+        error   = False
+        cell    = None
+        target  = None
+
+        for child in rowItem:
+            if (isinstance(child , nodes.comment)
+                or isinstance(child, nodes.system_message)):
+                pass
+            elif isinstance(child , nodes.target):
+                target = child
+            elif isinstance(child, nodes.bullet_list):
+                childNo += 1
+                cell = child
+            else:
+                error = True
+                break
+
+        if childNo != 1 or error:
+            self.raiseError(
+                'Error parsing content block for the "%s" directive: '
+                'two-level bullet list expected, but row %s does not '
+                'contain a second-level bullet list.'
+                % (self.directive.name, rowNum + 1))
+
+        for cellItem in cell:
+            cspan, rspan, cellElements = self.parseCellItem(cellItem)
+            if target is not None:
+                cellElements.insert(0, target)
+            row.append( (cspan, rspan, cellElements) )
+        return row
+
+    def parseCellItem(self, cellItem):
+        # search and remove cspan, rspan colspec from the first element in
+        # this listItem (field).
+        cspan = rspan = 0
+        if not len(cellItem):
+            return cspan, rspan, []
+        for elem in cellItem[0]:
+            if isinstance(elem, colSpan):
+                cspan = elem.get("span")
+                elem.parent.remove(elem)
+                continue
+            if isinstance(elem, rowSpan):
+                rspan = elem.get("span")
+                elem.parent.remove(elem)
+                continue
+        return cspan, rspan, cellItem[:]
-- 
sphinx-4.8

[toc] | [prev] | [next] | [standalone]

#1434793

On Thu, 30 Jun 2016 14:00:21 +0200
Markus Heiser <markus.heiser@darmarIT.de> wrote:

> this is my flat-table patch on top of your docs-next branch / we discussed on
> the ML

Hmm...  we don't have an official kernel coding style for Python, but if
we did, I'd sure like it to be a lot more like the usual Python
conventions.

The added documentation describes the "widths" parameter, but does not
tell the reader which units are used.

There were also a lot of typos and misspellings in the documentation
addition.

Anyway, I don't want to delay this work, so I have gone ahead and applied
it; I fixed the spelling errors on the way.  When you get a chance, can I
get a doc tweak for :widths:?

Thanks for separating this work out,

jon

[toc] | [prev] | [next] | [standalone]

#1434812

Hi Markus/Jon,

Em Thu, 30 Jun 2016 13:05:11 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> On Thu, 30 Jun 2016 14:00:21 +0200
> Markus Heiser <markus.heiser@darmarIT.de> wrote:
> 
> > this is my flat-table patch on top of your docs-next branch / we discussed on
> > the ML  
> 
> Hmm...  we don't have an official kernel coding style for Python, but if
> we did, I'd sure like it to be a lot more like the usual Python
> conventions.
> 
> The added documentation describes the "widths" parameter, but does not
> tell the reader which units are used.
> 
> There were also a lot of typos and misspellings in the documentation
> addition.
> 
> Anyway, I don't want to delay this work, so I have gone ahead and applied
> it; I fixed the spelling errors on the way. 

Thank you!

I rebased my tree to apply on the top of the master branch of
the media tree and to use the latest changeset from Jon's tree.

I kept it at the same place:

https://git.linuxtv.org/mchehab/experimental.git/log/?h=docs-next

I'm starting to manually edit and fix it right away. I'll later merge
it as one additional topic branch on my main tree, once I'm confident
that it is ok.

Markus,

Thank you very much for your hard work!

> When you get a chance, can I
> get a doc tweak for :widths:?
> 
> Thanks for separating this work out,
> 
> jon


-- 
Thanks,
Mauro

[toc] | [prev] | [next] | [standalone]

#1434821

Em Thu, 30 Jun 2016 13:05:11 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Anyway, I don't want to delay this work, so I have gone ahead and applied
> it; 

Got already one issue... Maybe on Jeni's changes to the makefiles...

I want to be able to compile just the book I'm working. Using the usual
syntax to build just one book worked: the build from the *.rst files
succeeded. Yet, it tried to compile it also as DocBook, causing an
error at the end.

See the logs below:

$ make DOCBOOKS=linux_tv htmldocs
  SPHINX  htmldocs
Running Sphinx v1.3.1
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 1 source files that are out of date
updating environment: 0 added, 1 changed, 0 removed
reading sources... [100%] linux_tv/media/dvb/dvbapi                             
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] linux_tv/media/dvb/dvbapi                              
generating indices... genindex
writing additional pages... search
copying images... [100%] linux_tv/media_api_files/typical_media_device.svg      
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded.
make[1]: *** No rule to make target 'Documentation/DocBook/linux_tv', needed by 'htmldocs'.  Pare.
Makefile:1420: recipe for target 'htmldocs' failed
make: *** [htmldocs] Error 2


-- 
Thanks,
Mauro

[toc] | [prev] | [next] | [standalone]

#1435165

Am 30.06.2016 um 21:44 schrieb Mauro Carvalho Chehab <mchehab@osg.samsung.com>:

> Em Thu, 30 Jun 2016 13:05:11 -0600
> Jonathan Corbet <corbet@lwn.net> escreveu:
> 
>> Anyway, I don't want to delay this work, so I have gone ahead and applied
>> it; 
> 
> Got already one issue... Maybe on Jeni's changes to the makefiles...
> 
> I want to be able to compile just the book I'm working.

This feature does not exists (yet) for the reST content. Jani and I
discussed it on the ML, but for the start he didn't want to have
books (aka sphinx sub-projects). Anyway, I don't want to restart
the discussion again, just the fact for information: no books
in reST, there is only one HTML site which includes all reST content.

> Using the usual
> syntax to build just one book worked: the build from the *.rst files
> succeeded. Yet, it tried to compile it also as DocBook, causing an
> error at the end.
> 
> See the logs below:
> 
> $ make DOCBOOKS=linux_tv htmldocs
>  SPHINX  htmldocs

As far as I know, there is only the one html-target "htmldocs" building
all HTML from reST & DocBook content. Selecting DocBook-XML books with the
environment "DOCBOOKS=..." is new for me. Did this feature exists in the 
past? I tried it with v4.6 but no success.

> Running Sphinx v1.3.1
> loading pickled environment... done
> building [mo]: targets for 0 po files that are out of date
> building [html]: targets for 1 source files that are out of date
> updating environment: 0 added, 1 changed, 0 removed
> reading sources... [100%] linux_tv/media/dvb/dvbapi                             
> looking for now-outdated files... none found
> pickling environment... done
> checking consistency... done
> preparing documents... done
> writing output... [100%] linux_tv/media/dvb/dvbapi                              
> generating indices... genindex
> writing additional pages... search
> copying images... [100%] linux_tv/media_api_files/typical_media_device.svg      
> copying static files... done
> copying extra files... done
> dumping search index in English (code: en) ... done
> dumping object inventory... done
> build succeeded.
> make[1]: *** No rule to make target 'Documentation/DocBook/linux_tv', needed by 'htmldocs'.  Pare.
> Makefile:1420: recipe for target 'htmldocs' failed
> make: *** [htmldocs] Error 2
> 
> -- 
> Thanks,
> Mauro


-- Markus --

[toc] | [prev] | [next] | [standalone]

#1435223

Em Fri, 01 Jul 2016 10:44:27 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:

> Am 30.06.2016 um 21:44 schrieb Mauro Carvalho Chehab <mchehab@osg.samsung.com>:
> 
> > Em Thu, 30 Jun 2016 13:05:11 -0600
> > Jonathan Corbet <corbet@lwn.net> escreveu:
> >   
> >> Anyway, I don't want to delay this work, so I have gone ahead and applied
> >> it;   
> > 
> > Got already one issue... Maybe on Jeni's changes to the makefiles...
> > 
> > I want to be able to compile just the book I'm working.  
> 
> This feature does not exists (yet) for the reST content. Jani and I
> discussed it on the ML, but for the start he didn't want to have
> books (aka sphinx sub-projects). Anyway, I don't want to restart
> the discussion again, just the fact for information: no books
> in reST, there is only one HTML site which includes all reST content.
> 
> > Using the usual
> > syntax to build just one book worked: the build from the *.rst files
> > succeeded. Yet, it tried to compile it also as DocBook, causing an
> > error at the end.
> > 
> > See the logs below:
> > 
> > $ make DOCBOOKS=linux_tv htmldocs
> >  SPHINX  htmldocs  
> 
> As far as I know, there is only the one html-target "htmldocs" building
> all HTML from reST & DocBook content. Selecting DocBook-XML books with the
> environment "DOCBOOKS=..." is new for me. Did this feature exists in the 
> past? I tried it with v4.6 but no success.

It exists, and I rely on such feature when merging patches.
Basically, for each patch I merge on my tree, I compile it.
if make returns an error, I play a buzz audio and I reject the
patch. I do the same for documentation: if the patch touches a file
listed at device_drivers.xml, I run this script:

	rm Documentation/DocBook/device-drivers.aux.xml  Documentation/DocBook/device-drivers.xml
	echo "Building device-drivers.xml"
	make DOCBOOKS=device-drivers.xml htmldocs 2>&1
	if [ "$?" != "0" ]; then
		echo -en '\e[0;31m'
		echo -e "***********\n***ERROR***\n***********\n\n"
		echo -en '\e[0;37m'
		play ~/sounds/pipe.wav 2>/dev/null >/dev/null
		exit -1 
	fi

And, if it touches on any file under Documentation/DocBook/media,
I run this script:

	make cleanmediadocs
	make DOCBOOKS=media_api.xml htmldocs 2>&1 | grep -v "element.*: validity error : ID .* already defined"
	echo
	echo "Do some pedantic checks and generate DocBook/media/media_api.html without chunks"
	echo
	xmllint --noent --postvalid "$PWD/Documentation/DocBook/media_api.xml" >/tmp/x.xml 2>/dev/null
	xmllint --noent --postvalid --noout /tmp/x.xml
	xmlto html-nochunks -m ./Documentation/DocBook/stylesheet.xsl -o Documentation/DocBook/media Documentation/DocBook/media_api.xml >/dev/null 2>&1
	if [ "$?" != "0" ]; then
		echo -en '\e[0;31m'
		echo -e "***********\n***ERROR***\n***********\n\n"
		echo -en '\e[0;37m'
		play ~/sounds/pipe.wav 2>/dev/null >/dev/null
		exit -1 
	fi

In the case of the DocBook, as we have all header files using references
to the code (and such references are dynamically created when make htmldocs
run), if someone adds a new API and forgets to send documentation, or if
it broke the DocBook, I'll also hear an annoying beep, and will reject
the patch.

Not being able to compile just one docbook is a regression and breaks
my process. This needs to be fixed.

Btw, yesterday, I tried to add references to a C code, at video.rst,
just like we did with DocBook:

.. code-block:: c
    :caption: Example 2: Switching to the first video input

    int index;

    index = 0;

    if (-1 == ioctl(fd, :ref:`VIDIOC_S_INPUT <vidioc-s-input>`, &index)) {
        perror("VIDIOC_S_INPUT");
        exit(EXIT_FAILURE);
    }

But Sphinx didn't allow doing it. I was unable to find any syntax
on it that would make Sphinx use a monospaced font but still parse
the references at the code. While for those small examples this would
be ok, This is something that we do want for the header files that
we put at V4L and DVB annexes. For DocBook, we do a lot of things like
this at the DocBook/media/Makefile:
	sed -e "s/\(enum *\)v4l2_mpeg_cx2341x_video_\([a-z]*_spatial_filter_type\)/\1<link linkend=\"\2\">v4l2_mpeg_cx2341x_video_\2<\/link>/g" videodev2.h

The actual code is a way more complex, but basically the idea is that
it escapes anything that DocBook might interpret as a command, and adds
<link> tags for every enum, typedef, ioctl, struct, syscall and define
it founds at the header files. the xmllint will produce errors when
links are not solved, and we'll be able to detect that the API is not
fully documented.

We need to do a similar process with Sphinx. A side effect is that, if
someone looks at the header files in the anexes, it can click on any
symbol at the API and see the full documentation.

--

Btw, in the case of the above example, I had to manually number it as
"Example 2", because I was unable to find a way with Sphinx to auto
numerate code-block captions. This is also something we want to fix,
as it is very hard to manually number things on a 600+ page document.

Regards,
Mauro

[toc] | [prev] | [next] | [standalone]

#1435253

On Fri, 01 Jul 2016, Mauro Carvalho Chehab <mchehab@osg.samsung.com> wrote:
> Not being able to compile just one docbook is a regression and breaks
> my process. This needs to be fixed.

Do you have a regression with *DocBook XML* on docs-next now? If yes,
clearly that must be fixed ASAP.

As to Sphinx, I fear your DocBook workflows are so elaborate that we are
bound to break some of them when switching over. Personally, I don't
think that's all bad. It's an opportunity to make things better. But we
can't make all of that happen overnight for media. This is why we have
Sphinx and DocBook side by side for now.

We specifically wanted to make the switch for e.g. GPU documentation
fast, act as guinea pigs, hit the issues, and make it easier for others
in the end. We'd also love to get some of those validation things you
have, for everyone, but using Sphinx instead of a bunch of scripts and
sed on the input files.

This is also one of the reasons why I so much want to keep everything
behind one configuration file, and build everything in the Sphinx
toolchain. To keep it all more uniform, to not duplicate stuff, and not
deviate to some silos like we've done in the past. I think when we have
things working, we can add dedicated config files for the select few
things that have additional special needs. Media is probably one of
them. But that said, I think we should be able to keep including that to
the main documentation build too.

Also, many thanks for digging into this and reporting issues that you do
face.


BR,
Jani.

-- 
Jani Nikula, Intel Open Source Technology Center

[toc] | [prev] | [next] | [standalone]

#1435260

Am 01.07.2016 um 12:44 schrieb Jani Nikula <jani.nikula@intel.com>:

> On Fri, 01 Jul 2016, Mauro Carvalho Chehab <mchehab@osg.samsung.com> wrote:
>> Not being able to compile just one docbook is a regression and breaks
>> my process. This needs to be fixed.
> 
> Do you have a regression with *DocBook XML* on docs-next now? If yes,
> clearly that must be fixed ASAP.

Now I see the problem ...

It seems that we didn't neglected that the DocBook Makefile 
builds single books, e.g.

  make DOCBOOKS=device-drivers.xml htmldocs

Since the Makefile.sphinx does not care the environment DOCBOOKS,
the XML is build and the reST also, where only the XML content
should be build.

Jani, may could you apply a small fix on the Makefile.sphinx which
takes into account, not to run the sphinx-build if the DOCBOOK 
environment is set .. is it a solution for a start?

-- Markus --

> As to Sphinx, I fear your DocBook workflows are so elaborate that we are
> bound to break some of them when switching over. Personally, I don't
> think that's all bad. It's an opportunity to make things better. But we
> can't make all of that happen overnight for media. This is why we have
> Sphinx and DocBook side by side for now.
> 
> We specifically wanted to make the switch for e.g. GPU documentation
> fast, act as guinea pigs, hit the issues, and make it easier for others
> in the end. We'd also love to get some of those validation things you
> have, for everyone, but using Sphinx instead of a bunch of scripts and
> sed on the input files.
> 
> This is also one of the reasons why I so much want to keep everything
> behind one configuration file, and build everything in the Sphinx
> toolchain. To keep it all more uniform, to not duplicate stuff, and not
> deviate to some silos like we've done in the past. I think when we have
> things working, we can add dedicated config files for the select few
> things that have additional special needs. Media is probably one of
> them. But that said, I think we should be able to keep including that to
> the main documentation build too.
> 
> Also, many thanks for digging into this and reporting issues that you do
> face.
> 
> 
> BR,
> Jani.
> 
> -- 
> Jani Nikula, Intel Open Source Technology Center

[toc] | [prev] | [next] | [standalone]

#1435277

On Fri, 01 Jul 2016, Markus Heiser <markus.heiser@darmarit.de> wrote:
> Am 01.07.2016 um 12:44 schrieb Jani Nikula <jani.nikula@intel.com>:
>
>> On Fri, 01 Jul 2016, Mauro Carvalho Chehab <mchehab@osg.samsung.com> wrote:
>>> Not being able to compile just one docbook is a regression and breaks
>>> my process. This needs to be fixed.
>> 
>> Do you have a regression with *DocBook XML* on docs-next now? If yes,
>> clearly that must be fixed ASAP.
>
> Now I see the problem ...
>
> It seems that we didn't neglected that the DocBook Makefile 
> builds single books, e.g.
>
>   make DOCBOOKS=device-drivers.xml htmldocs
>
> Since the Makefile.sphinx does not care the environment DOCBOOKS,
> the XML is build and the reST also, where only the XML content
> should be build.
>
> Jani, may could you apply a small fix on the Makefile.sphinx which
> takes into account, not to run the sphinx-build if the DOCBOOK 
> environment is set .. is it a solution for a start?

That is not the problem. Makefile.sphinx happily ignores DOCBOOKS. For
example 'make DOCBOOKS=kernel-api.xml htmldocs' on docs-next works just
fine for me. Sphinx build ignores DOCBOOKS, the DocBook build works just
like it always did.

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Technology Center

[toc] | [prev] | [next] | [standalone]

#1435280

Am 01.07.2016 um 13:56 schrieb Jani Nikula <jani.nikula@intel.com>:

> On Fri, 01 Jul 2016, Markus Heiser <markus.heiser@darmarit.de> wrote:
>> Am 01.07.2016 um 12:44 schrieb Jani Nikula <jani.nikula@intel.com>:
>> 
>>> On Fri, 01 Jul 2016, Mauro Carvalho Chehab <mchehab@osg.samsung.com> wrote:
>>>> Not being able to compile just one docbook is a regression and breaks
>>>> my process. This needs to be fixed.
>>> 
>>> Do you have a regression with *DocBook XML* on docs-next now? If yes,
>>> clearly that must be fixed ASAP.
>> 
>> Now I see the problem ...
>> 
>> It seems that we didn't neglected that the DocBook Makefile 
>> builds single books, e.g.
>> 
>>  make DOCBOOKS=device-drivers.xml htmldocs
>> 
>> Since the Makefile.sphinx does not care the environment DOCBOOKS,
>> the XML is build and the reST also, where only the XML content
>> should be build.
>> 
>> Jani, may could you apply a small fix on the Makefile.sphinx which
>> takes into account, not to run the sphinx-build if the DOCBOOK 
>> environment is set .. is it a solution for a start?
> 
> That is not the problem. Makefile.sphinx happily ignores DOCBOOKS. For
> example 'make DOCBOOKS=kernel-api.xml htmldocs' on docs-next works just
> fine for me. Sphinx build ignores DOCBOOKS, the DocBook build works just
> like it always did.

Are you sure .. may I do something wrong::

$ make DOCBOOKS=kernel-api.xml htmldocs
  SPHINX  htmldocs
Running Sphinx v1.4.4
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 0 source files that are out of date
updating environment: 0 added, 0 changed, 0 removed
looking for now-outdated files... none found
no targets are out of date.
build succeeded.
  DOCPROC Documentation/DocBook/kernel-api.xml
.//lib/crc32.c:148: warning: No description found for parameter 'tab)[256]'
.//lib/crc32.c:148: warning: Excess function parameter 'tab' description in 'crc32_le_generic'
.//lib/crc32.c:293: warning: No description found for parameter 'tab)[256]'
.//lib/crc32.c:293: warning: Excess function parameter 'tab' description in 'crc32_be_generic'
.//lib/crc32.c:1: warning: no structured comments found
.//mm/memory.c:2881: warning: No description found for parameter 'old'
  XMLREF  Documentation/DocBook/kernel-api.aux.xml
  HTML    Documentation/DocBook/kernel-api.html
  ....

In this example, the sphinx build is started as a side effect
even if it is not rebuild in this example, because nothing
in reST content has changed: "looking for now-outdated files... none found".

As far as I can see, this is done **because** Makefile.sphinx
ignores DOCBOOK.

Thanks for your feedback

 -- Markus --



> BR,
> Jani.
> 
> 
> -- 
> Jani Nikula, Intel Open Source Technology Center

[toc] | [prev] | [next] | [standalone]

#1435289

On Fri, 01 Jul 2016, Markus Heiser <markus.heiser@darmarit.de> wrote:
> Am 01.07.2016 um 13:56 schrieb Jani Nikula <jani.nikula@intel.com>:
>
>> On Fri, 01 Jul 2016, Markus Heiser <markus.heiser@darmarit.de> wrote:
>>> Am 01.07.2016 um 12:44 schrieb Jani Nikula <jani.nikula@intel.com>:
>>> 
>>>> On Fri, 01 Jul 2016, Mauro Carvalho Chehab <mchehab@osg.samsung.com> wrote:
>>>>> Not being able to compile just one docbook is a regression and breaks
>>>>> my process. This needs to be fixed.
>>>> 
>>>> Do you have a regression with *DocBook XML* on docs-next now? If yes,
>>>> clearly that must be fixed ASAP.
>>> 
>>> Now I see the problem ...
>>> 
>>> It seems that we didn't neglected that the DocBook Makefile 
>>> builds single books, e.g.
>>> 
>>>  make DOCBOOKS=device-drivers.xml htmldocs
>>> 
>>> Since the Makefile.sphinx does not care the environment DOCBOOKS,
>>> the XML is build and the reST also, where only the XML content
>>> should be build.
>>> 
>>> Jani, may could you apply a small fix on the Makefile.sphinx which
>>> takes into account, not to run the sphinx-build if the DOCBOOK 
>>> environment is set .. is it a solution for a start?
>> 
>> That is not the problem. Makefile.sphinx happily ignores DOCBOOKS. For
>> example 'make DOCBOOKS=kernel-api.xml htmldocs' on docs-next works just
>> fine for me. Sphinx build ignores DOCBOOKS, the DocBook build works just
>> like it always did.
>
> Are you sure .. may I do something wrong::
>
> $ make DOCBOOKS=kernel-api.xml htmldocs
>   SPHINX  htmldocs
> Running Sphinx v1.4.4
> loading pickled environment... done
> building [mo]: targets for 0 po files that are out of date
> building [html]: targets for 0 source files that are out of date
> updating environment: 0 added, 0 changed, 0 removed
> looking for now-outdated files... none found
> no targets are out of date.
> build succeeded.
>   DOCPROC Documentation/DocBook/kernel-api.xml
> .//lib/crc32.c:148: warning: No description found for parameter 'tab)[256]'
> .//lib/crc32.c:148: warning: Excess function parameter 'tab' description in 'crc32_le_generic'
> .//lib/crc32.c:293: warning: No description found for parameter 'tab)[256]'
> .//lib/crc32.c:293: warning: Excess function parameter 'tab' description in 'crc32_be_generic'
> .//lib/crc32.c:1: warning: no structured comments found
> .//mm/memory.c:2881: warning: No description found for parameter 'old'
>   XMLREF  Documentation/DocBook/kernel-api.aux.xml
>   HTML    Documentation/DocBook/kernel-api.html
>   ....
>
> In this example, the sphinx build is started as a side effect
> even if it is not rebuild in this example, because nothing
> in reST content has changed: "looking for now-outdated files... none found".
>
> As far as I can see, this is done **because** Makefile.sphinx
> ignores DOCBOOK.

I was referring to the original problem Mauro reported where the build
failed. The above is a different issue, but an issue nonetheless. Patch
sent.

BR,
Jani.

-- 
Jani Nikula, Intel Open Source Technology Center

[toc] | [prev] | [next] | [standalone]

#1435324

Em Fri, 01 Jul 2016 15:26:45 +0300
Jani Nikula <jani.nikula@intel.com> escreveu:

> > In this example, the sphinx build is started as a side effect
> > even if it is not rebuild in this example, because nothing
> > in reST content has changed: "looking for now-outdated files... none found".
> >
> > As far as I can see, this is done **because** Makefile.sphinx
> > ignores DOCBOOK.  
> 
> I was referring to the original problem Mauro reported where the build
> failed. The above is a different issue, but an issue nonetheless. Patch
> sent.

Patches look ok, but that's not what I wanted ;) what I want is to
have something that Makefile.sphinx will look into and build just
the book described there. E. g. something like:

make BOOK="linux_tv" htmldocs

Would make sphinx to only build the book under the
Documentation/linux_tv directory.

-- 
Thanks,
Mauro

[toc] | [prev] | [next] | [standalone]

#1435281 — [docs-next PATCH] Documentation: add cleanmediadocs to the documentation targets

This was broken when updating the documentation targets for the Sphinx
build, and moving from %docs target pattern to explicitly listed
targets.

Cc: Markus Heiser <markus.heiser@darmarit.de>
Cc: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Fixes: 22cba31bae9d ("Documentation/sphinx: add basic working Sphinx configuration and build")
Signed-off-by: Jani Nikula <jani.nikula@intel.com>

---

This fixes 'make cleanmediadocs' on docs-next, but I don't think this is
the problem Mauro is seeing.
---
 Documentation/Makefile.sphinx | 1 +
 Makefile                      | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/Makefile.sphinx b/Documentation/Makefile.sphinx
index 810c11f0d37f..6c203745b6c6 100644
--- a/Documentation/Makefile.sphinx
+++ b/Documentation/Makefile.sphinx
@@ -56,6 +56,7 @@ sgmldocs:
 psdocs:
 mandocs:
 installmandocs:
+cleanmediadocs:
 
 cleandocs:
 	$(Q)rm -rf $(BUILDDIR)
diff --git a/Makefile b/Makefile
index f7f393c40886..34bc4e22ec0b 100644
--- a/Makefile
+++ b/Makefile
@@ -1414,7 +1414,7 @@ $(help-board-dirs): help-%:
 
 # Documentation targets
 # ---------------------------------------------------------------------------
-DOC_TARGETS := xmldocs sgmldocs psdocs pdfdocs htmldocs mandocs installmandocs epubdocs cleandocs
+DOC_TARGETS := xmldocs sgmldocs psdocs pdfdocs htmldocs mandocs installmandocs epubdocs cleandocs cleanmediadocs
 PHONY += $(DOC_TARGETS)
 $(DOC_TARGETS): scripts_basic FORCE
 	$(Q)$(MAKE) $(build)=scripts build_docproc build_check-lc_ctype
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]

#1435288 — [docs-next PATCH] Documentation/sphinx: skip build if user requested specific DOCBOOKS

If the user requested specific DocBooks to be built using 'make
DOCBOOKS=foo.xml htmldocs', assume no Sphinx build is desired. This
check is transitional, and can be removed once we drop the DocBook
build.

Cc: Markus Heiser <markus.heiser@darmarit.de>
Cc: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Fixes: 22cba31bae9d ("Documentation/sphinx: add basic working Sphinx configuration and build")
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 Documentation/Makefile.sphinx | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/Documentation/Makefile.sphinx b/Documentation/Makefile.sphinx
index 6c203745b6c6..d8d13c92a178 100644
--- a/Documentation/Makefile.sphinx
+++ b/Documentation/Makefile.sphinx
@@ -17,6 +17,12 @@ ifeq ($(HAVE_SPHINX),0)
 	$(warning The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed and in PATH, or set the SPHINXBUILD make variable to point to the full path of the '$(SPHINXBUILD)' executable.)
 	@echo "  SKIP    Sphinx $@ target."
 
+else ifneq ($(DOCBOOKS),)
+
+# Skip Sphinx build if the user explicitly requested DOCBOOKS.
+.DEFAULT:
+	@echo "  SKIP    Sphinx $@ target (DOCBOOKS specified)."
+
 else # HAVE_SPHINX
 
 # User-friendly check for rst2pdf
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]

#1435293 — Re: [docs-next PATCH] Documentation/sphinx: skip build if user requested specific DOCBOOKS

Hi Jani,

tested ... it works, thanks a lot.

-- Markus --

Am 01.07.2016 um 14:24 schrieb Jani Nikula <jani.nikula@intel.com>:

> If the user requested specific DocBooks to be built using 'make
> DOCBOOKS=foo.xml htmldocs', assume no Sphinx build is desired. This
> check is transitional, and can be removed once we drop the DocBook
> build.
> 
> Cc: Markus Heiser <markus.heiser@darmarit.de>
> Cc: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
> Fixes: 22cba31bae9d ("Documentation/sphinx: add basic working Sphinx configuration and build")
> Signed-off-by: Jani Nikula <jani.nikula@intel.com>
> ---
> Documentation/Makefile.sphinx | 6 ++++++
> 1 file changed, 6 insertions(+)
> 
> diff --git a/Documentation/Makefile.sphinx b/Documentation/Makefile.sphinx
> index 6c203745b6c6..d8d13c92a178 100644
> --- a/Documentation/Makefile.sphinx
> +++ b/Documentation/Makefile.sphinx
> @@ -17,6 +17,12 @@ ifeq ($(HAVE_SPHINX),0)
> 	$(warning The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed and in PATH, or set the SPHINXBUILD make variable to point to the full path of the '$(SPHINXBUILD)' executable.)
> 	@echo "  SKIP    Sphinx $@ target."
> 
> +else ifneq ($(DOCBOOKS),)
> +
> +# Skip Sphinx build if the user explicitly requested DOCBOOKS.
> +.DEFAULT:
> +	@echo "  SKIP    Sphinx $@ target (DOCBOOKS specified)."
> +
> else # HAVE_SPHINX
> 
> # User-friendly check for rst2pdf
> -- 
> 2.1.4
> 

[toc] | [prev] | [next] | [standalone]

#1435336 — Re: [docs-next PATCH] Documentation/sphinx: skip build if user requested specific DOCBOOKS

Em Fri,  1 Jul 2016 15:24:44 +0300
Jani Nikula <jani.nikula@intel.com> escreveu:

> If the user requested specific DocBooks to be built using 'make
> DOCBOOKS=foo.xml htmldocs', assume no Sphinx build is desired. This
> check is transitional, and can be removed once we drop the DocBook
> build.

I guess this is actually a wrong assumption. I mean, it is valid to
build multiple documents at the same time. Sometimes I do things
like:
	make DOCBOOKS="media_api.xml device-drivers.xml" htmldocs

When I want both docs to be compiled.

What I would be expecting is that Sphinx would be looking into
the DOCBOOKS targets and see if (some) of them belongs to it.

Alternatively, we could add a separate makefile var for the
Sphinx targets, but the logic would be more complex, as it
should:

not ignore:
	make htmldocs

ignore:
	make DOCBOOKS="device-drivers.xml" htmldocs

not ignore:
	make SPHINXDOCS="linux_tv" DOCBOOKS="device-drivers.xml" htmldocs

not ignore:
	make SPHINXDOCS="linux_tv" htmldocs

Regards,
Mauro

> 
> Cc: Markus Heiser <markus.heiser@darmarit.de>
> Cc: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
> Fixes: 22cba31bae9d ("Documentation/sphinx: add basic working Sphinx configuration and build")
> Signed-off-by: Jani Nikula <jani.nikula@intel.com>
> ---
>  Documentation/Makefile.sphinx | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/Documentation/Makefile.sphinx b/Documentation/Makefile.sphinx
> index 6c203745b6c6..d8d13c92a178 100644
> --- a/Documentation/Makefile.sphinx
> +++ b/Documentation/Makefile.sphinx
> @@ -17,6 +17,12 @@ ifeq ($(HAVE_SPHINX),0)
>  	$(warning The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed and in PATH, or set the SPHINXBUILD make variable to point to the full path of the '$(SPHINXBUILD)' executable.)
>  	@echo "  SKIP    Sphinx $@ target."
>  
> +else ifneq ($(DOCBOOKS),)
> +
> +# Skip Sphinx build if the user explicitly requested DOCBOOKS.
> +.DEFAULT:
> +	@echo "  SKIP    Sphinx $@ target (DOCBOOKS specified)."
> +
>  else # HAVE_SPHINX
>  
>  # User-friendly check for rst2pdf


-- 
Thanks,
Mauro

[toc] | [prev] | [next] | [standalone]

#1435340 — Re: [docs-next PATCH] Documentation/sphinx: skip build if user requested specific DOCBOOKS

On Fri, 01 Jul 2016, Mauro Carvalho Chehab <mchehab@osg.samsung.com> wrote:
> Em Fri,  1 Jul 2016 15:24:44 +0300
> Jani Nikula <jani.nikula@intel.com> escreveu:
>
>> If the user requested specific DocBooks to be built using 'make
>> DOCBOOKS=foo.xml htmldocs', assume no Sphinx build is desired. This
>> check is transitional, and can be removed once we drop the DocBook
>> build.
>
> I guess this is actually a wrong assumption. I mean, it is valid to
> build multiple documents at the same time. Sometimes I do things
> like:
> 	make DOCBOOKS="media_api.xml device-drivers.xml" htmldocs
>
> When I want both docs to be compiled.
>
> What I would be expecting is that Sphinx would be looking into
> the DOCBOOKS targets and see if (some) of them belongs to it.
>
> Alternatively, we could add a separate makefile var for the
> Sphinx targets, but the logic would be more complex, as it
> should:

Please let's not conflate DOCBOOKS to mean something other than
DocBooks. I think it'll be easier that way.

So I guess we'll need a way to build just a subset of the Sphinx
documentation. I would like that to be a somewhat generic thing, not
requiring a separate conf file for each subset. Is the granularity of a
directory enough? I've been meaning to look into passing different
<sourcedir> and <filenames> to sphinx-build for this, but I don't have
the time now.

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Technology Center

[toc] | [prev] | [next] | [standalone]

#1435403 — Re: [docs-next PATCH] Documentation/sphinx: skip build if user requested specific DOCBOOKS

Em Fri, 01 Jul 2016 16:31:14 +0300
Jani Nikula <jani.nikula@intel.com> escreveu:

> On Fri, 01 Jul 2016, Mauro Carvalho Chehab <mchehab@osg.samsung.com> wrote:
> > Em Fri,  1 Jul 2016 15:24:44 +0300
> > Jani Nikula <jani.nikula@intel.com> escreveu:
> >  
> >> If the user requested specific DocBooks to be built using 'make
> >> DOCBOOKS=foo.xml htmldocs', assume no Sphinx build is desired. This
> >> check is transitional, and can be removed once we drop the DocBook
> >> build.  
> >
> > I guess this is actually a wrong assumption. I mean, it is valid to
> > build multiple documents at the same time. Sometimes I do things
> > like:
> > 	make DOCBOOKS="media_api.xml device-drivers.xml" htmldocs
> >
> > When I want both docs to be compiled.
> >
> > What I would be expecting is that Sphinx would be looking into
> > the DOCBOOKS targets and see if (some) of them belongs to it.
> >
> > Alternatively, we could add a separate makefile var for the
> > Sphinx targets, but the logic would be more complex, as it
> > should:  
> 
> Please let's not conflate DOCBOOKS to mean something other than
> DocBooks. I think it'll be easier that way.

OK. Works for me.

> So I guess we'll need a way to build just a subset of the Sphinx
> documentation. I would like that to be a somewhat generic thing, not
> requiring a separate conf file for each subset. Is the granularity of a
> directory enough? I've been meaning to look into passing different
> <sourcedir> and <filenames> to sphinx-build for this, but I don't have
> the time now.

In the case of the media doc, it is actually a matter of building
one rst file (linux_tv/index.rst). For that, I'm equally happy
if we specify it via <sourcedir> or via <filename>.

I suspect we'll end by needing filenames granularity in some future,
but, if you find easier to just handle <sorcedir> for now, that's
OK. 

Regards,
Mauro

[toc] | [prev] | [next] | [standalone]

#1435301

Hi Jani,

Em Fri, 01 Jul 2016 13:44:17 +0300
Jani Nikula <jani.nikula@intel.com> escreveu:

> On Fri, 01 Jul 2016, Mauro Carvalho Chehab <mchehab@osg.samsung.com> wrote:
> > Not being able to compile just one docbook is a regression and breaks
> > my process. This needs to be fixed.  
> 
> Do you have a regression with *DocBook XML* on docs-next now? If yes,
> clearly that must be fixed ASAP.

No, but as we'll be switching from DocBook to Sphinx, it should be
possible for the maintainers to use the same or similar workflows.

The one requirement from myside, as the media maintainer, is to be
able to compile *just* the media Sphinx book. As I do it patch by
patch, and we usually merge ~500 patches per kernel version, the
build should be fast, and shouldn't be affected by any breakages
on other docs.

> As to Sphinx, I fear your DocBook workflows are so elaborate that we are
> bound to break some of them when switching over. Personally, I don't
> think that's all bad. 

Well, you're not the one that is responsible for fixing the media mess.
this workflow is there since 2006 at the V4L2 side, and has been improved
over the years. Thanks to it, our documentation always reflect the API
supported by the Kernel. At DVB, we didn't have anything like that, and
we had a *huge* gap between documentation and code. So, last year, we
fixed the gap and the DVB APIs that are more relevant are also in good
shape.

We should not go back to the dark ages, where gaps happened and nobody
detected, until several years after the code has been introduced, and
being hard to identify what the hack some obscure API bits really mean.

> It's an opportunity to make things better.

I'm all for making things better, but let's not make them worse.

> But we
> can't make all of that happen overnight for media. This is why we have
> Sphinx and DocBook side by side for now.

My plan is to swith over the media docbook in one Kernel cycle. Maintaining
two documentations side by side, with the high volume of changes they suffer
on each Kernel cycle is not feasible.

> We specifically wanted to make the switch for e.g. GPU documentation
> fast, act as guinea pigs, hit the issues, and make it easier for others
> in the end. We'd also love to get some of those validation things you
> have, for everyone, but using Sphinx instead of a bunch of scripts and
> sed on the input files.

Sphinx doesn't seem to be able to cross-reference the source code to
identify documentation gaps at the public APIs. So, we'll need an external
script to parse the code and get the API identifiers from the headers.

Yeah, the logic we use to parse the media headers could be more
generic and be used on other subsystems, and I'm all for it, but
semantic parsers are tricky. In order to make them generic, it would
very likely need to use a real lexical analyzer, like flex (or some
perl/python module that would increase the toolchain dependencies to build
the Kernel). Also, right now, the logic handles some special cases,
that are specific to the subsystem. By making it generic, we'll need
to either get rid of that or add per-book quirk configuration files.

> This is also one of the reasons why I so much want to keep everything
> behind one configuration file, and build everything in the Sphinx
> toolchain. To keep it all more uniform, to not duplicate stuff, and not
> deviate to some silos like we've done in the past. I think when we have
> things working, we can add dedicated config files for the select few
> things that have additional special needs. Media is probably one of
> them. But that said, I think we should be able to keep including that to
> the main documentation build too.

I agree with the general principle of building everything at the Sphinx
toolchain. But this requires a deep look, and it will take a while for
this goal to happen.

Yet, if you're talking about creating an unique huge document with
everything inside, I don't believe at an "one solution fits all" kind
of approach.

We have at least 3 different sets of documents right now:

set 1) Kernel-doc Kernel internal API documentation

This is actually split into 27 different DocBook templates. Among this set, 
3 of them documents also the uAPI: alsa-driver-api.tmpl, iio.tmpl and
networking.tmpl.

I guess it should be possible to group all of them altogether, as you
seem to want. Yet, I'm not sure if this is the best thing to do. What 
would be the advantage of grouping them? What would be the disadvantage?

Looking at the positive side, they all document APIs and they all share
about the same "DocumentationStyle", with is enforced via
kernel-doc-nano-HOWTO.txt. So, gluing them together won't look too much
like a Frankenstein monster.

set 2) "Pure" DocBooks, each of them organized as a separate book and that
don't use Kernel-doc:
	1. kernel-hacking.tmpl
	2. lsm.tmpl
	3. Media DocBook (media_api.tmpl, media-entities.tmpl, media-indices.tmpl)
	4. uio-howto.tmpl
	5. writing-an-alsa-driver.tmpl
	6. writing_musb_glue_layer.tmpl
	7. writing_usb_driver.tmpl

Each was written on a different time, by a different set of authors
and following their own documentation style. Some work has to be
done before being able to merge them, if ever desired, in order to make
them look as a hole. Yet, they deal different subjects. I suspect 
they'll be better alone than grouped into a single document.

Also, media is clearly big and complex enough to be a separate 
document, and has its own documentation style. Except perhaps
for ALSA, all the other 5 documents handle completely different
matters. I can't think why someone would want them grouped.

Also, I'm almost sure that, at least for what's currently at the
Media API DocBook, we'll need to be able to use a different CSS
template, because we have very long tables there, so we want to
use a different layout to use a "landscape" like style.

set 3) Pure text documents, under Documentation/.

In this set, there are also several different subjects, from very
technical stuff to things like CodingStyle and CodeOfConduct.

There are also documents there for the Kernel end-users, like:
kernel-parameters.txt. 

So, IMHO, we should either keep them alone, or create some
some sets like:
	- API documentation;
	- Userspace documentation;
	- Compilation documentation;
	- Kernel programming best practices.

Looking at the media documentation on set 3, there is a mix of updated
and really old docs under video4linux/ and dvb/. Those require extra
care: some could probably be trashed, others should be merged with the
media kernel-doc stuff. There are things there that won't fit on
either one of the categories, like video4linux/CARDLIST.*,
as they're meant to be a sort of end-users documentation.

In summary, grouping all those things into a single document
seems wrong.

Putting everything altoghether would be like gluing all your C
programming language books with your English books and dictionaries,
just because they all were found on the same library. Let's separate
the books into shelves, as it makes easier for the user to find
the right documentation he needs.

> Also, many thanks for digging into this and reporting issues that you do
> face.

Anytime. I'll keep looking into the issues I'm finding, as the end
goal for all of us is to have a better documented Kernel.

-- 
Thanks,
Mauro

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | linux.kernel

csiph-web