Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.kernel > #1413696 > unrolled thread

[PATCH v2 00/38] Documentation/sphinx

Started byJani Nikula <jani.nikula@intel.com>
First post2016-06-04 13:40 +0200
Last post2016-06-10 22:50 +0200
Articles 20 on this page of 43 — 5 participants

Back to article view | Back to linux.kernel


Contents

  [PATCH v2 00/38] Documentation/sphinx Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:40 +0200
    [PATCH v2 10/38] Documentation/sphinx: nicer referencing of struct in docbook->rst conversion Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:40 +0200
    [PATCH v2 01/38] kernel-doc/rst: fix use of uninitialized value Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:40 +0200
    [PATCH v2 13/38] kernel-doc/rst: do not output DOC: section titles for requested ones Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:40 +0200
    [PATCH v2 14/38] kernel-doc/rst: reference functions according to C domain spec Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:40 +0200
    [PATCH v2 08/38] sphinx: cheesy script to convert .tmpl files Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:40 +0200
    [PATCH v2 02/38] kernel-doc: support printing exported and non-exported symbols Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:40 +0200
    [PATCH v2 12/38] kernel-doc: add names for output selection Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:40 +0200
    [PATCH v2 15/38] kernel-doc/rst: &foo references are more universal than structs Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:40 +0200
    [PATCH v2 05/38] Documentation/sphinx: add Sphinx kernel-doc directive extension Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:40 +0200
    [PATCH v2 03/38] Documentation/sphinx: add basic working Sphinx configuration and build Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:40 +0200
    [PATCH v2 28/38] kernel-doc/rst: remove fixme comment Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 24/38] kernel-doc/rst: change the output layout Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 22/38] kernel-doc/rst: blank lines in output are not needed Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 37/38] scripts/kernel-doc: Add option to inject line numbers Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 07/38] Documentation/sphinx: set version and release properly Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 32/38] Documentation/sphinx: fix kernel-doc extension on python3 Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 06/38] Documentation/sphinx: configure the kernel-doc extension Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 20/38] kernel-doc: do not regard $, %, or & prefixes as special in section names Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 19/38] kernel-doc/rst: highlight function/struct/enum purpose lines too Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 23/38] kernel-doc: strip leading blank lines from inline doc comments Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 26/38] kernel-doc: strip leading whitespace from continued param descs Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 38/38] doc/sphinx: Track line-number of starting blocks Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 17/38] kernel-doc/rst: add support for struct/union/enum member references Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 31/38] kernel-doc: reset contents and section harder Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 30/38] kernel-doc: concatenate contents of colliding sections Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 29/38] kernel-doc: limit the "section header:" detection to a select few Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
      Re: [PATCH v2 29/38] kernel-doc: limit the "section header:"  detection to a select few Jonathan Corbet <corbet@lwn.net> - 2016-06-09 17:10 +0200
        Re: [PATCH v2 29/38] kernel-doc: limit the "section header:" detection to a select few Jani Nikula <jani.nikula@intel.com> - 2016-06-09 18:50 +0200
    [PATCH v2 36/38] scripts/kernel-doc: Also give functions symbolic names Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 21/38] kernel-doc: fix wrong code indentation Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 34/38] scripts/kernel-doc: Remove duplicated DOC: start handling Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 18/38] kernel-doc/rst: drop redundant unescape in highlighting Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 33/38] doc/sphinx: Pass right filename as source Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 16/38] kernel-doc/rst: add support for &union foo and &typedef foo references Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 25/38] kernel-doc: improve handling of whitespace on the first line param description Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 35/38] doc/sphinx: Stop touching state_machine internals Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 27/38] kernel-doc/rst: use *undescribed* instead of _undescribed_ Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    [PATCH v2 09/38] sphinx: update docbook->rst conversion script match C domain spec Jani Nikula <jani.nikula@intel.com> - 2016-06-04 13:50 +0200
    Re: [PATCH v2 00/38] Documentation/sphinx Daniel Vetter <daniel@ffwll.ch> - 2016-06-04 14:20 +0200
    Re: [PATCH v2 00/38] Documentation/sphinx Jonathan Corbet <corbet@lwn.net> - 2016-06-09 22:00 +0200
      Re: [PATCH v2 00/38] Documentation/sphinx Daniel Vetter <daniel.vetter@ffwll.ch> - 2016-06-10 20:20 +0200
        Re: [PATCH v2 00/38] Documentation/sphinx Dave Airlie <airlied@gmail.com> - 2016-06-10 22:50 +0200

Page 1 of 3  [1] 2 3  Next page →


#1413696 — [PATCH v2 00/38] Documentation/sphinx

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:40 +0200
Subject[PATCH v2 00/38] Documentation/sphinx
Message-ID<rGi5P-8on-3@gated-at.bofh.it>
Jon, this is v2 of [1] and [2], with a considerable amount of polish and
fixes added. We started dogfooding this within drm-intel, and Daniel has
reviewed the lot and contributed a number of fixes, most notably
accurate file and line number references from Sphinx build
errors/warnings to the kernel-doc comments in source code.

We believe this is now in good shape for merging for v4.8. It's all in
my sphinx-for-docs-next branch that you've already looked at; pull
details below.

When this lands in docs-next and we can backmerge to drm, we'll plunge
ahead and convert gpu.tmpl to rst, and have that ready for v4.8. We
think it's best to contribute that via the drm tree, as it'll involve
splitting up the documentation and likely numerous updates to kernel-doc
comments.

I plan to update Documentation/kernel-doc-nano-HOWTO.txt for Sphinx and
rst, obviously converting it to rst while at it.

BR,
Jani.



[1] http://mid.gmane.org/cover.1463748027.git.jani.nikula@intel.com
[2] http://mid.gmane.org/877fec7gfm.fsf@intel.com

The following changes since commit 1a695a905c18548062509178b98bc91e67510864:

  Linux 4.7-rc1 (2016-05-29 09:29:24 -0700)

are available in the git repository at:

  git://people.freedesktop.org/~jani/drm sphinx-for-docs-next

for you to fetch changes up to d90368f2fa7ded7c56d214aef087e88bba5199e7:

  doc/sphinx: Track line-number of starting blocks (2016-06-04 11:35:59 +0300)

----------------------------------------------------------------
Daniel Vetter (6):
      doc/sphinx: Pass right filename as source
      scripts/kernel-doc: Remove duplicated DOC: start handling
      doc/sphinx: Stop touching state_machine internals
      scripts/kernel-doc: Also give functions symbolic names
      scripts/kernel-doc: Add option to inject line numbers
      doc/sphinx: Track line-number of starting blocks

Jani Nikula (31):
      kernel-doc/rst: fix use of uninitialized value
      kernel-doc: support printing exported and non-exported symbols
      Documentation/sphinx: add basic working Sphinx configuration and build
      Documentation: add .gitignore
      Documentation/sphinx: add Sphinx kernel-doc directive extension
      Documentation/sphinx: configure the kernel-doc extension
      Documentation/sphinx: set version and release properly
      sphinx: update docbook->rst conversion script match C domain spec
      Documentation/sphinx: nicer referencing of struct in docbook->rst conversion
      kernel-doc: add names for states and substates
      kernel-doc: add names for output selection
      kernel-doc/rst: do not output DOC: section titles for requested ones
      kernel-doc/rst: reference functions according to C domain spec
      kernel-doc/rst: &foo references are more universal than structs
      kernel-doc/rst: add support for &union foo and &typedef foo references
      kernel-doc/rst: add support for struct/union/enum member references
      kernel-doc/rst: drop redundant unescape in highlighting
      kernel-doc/rst: highlight function/struct/enum purpose lines too
      kernel-doc: do not regard $, %, or & prefixes as special in section names
      kernel-doc: fix wrong code indentation
      kernel-doc/rst: blank lines in output are not needed
      kernel-doc: strip leading blank lines from inline doc comments
      kernel-doc/rst: change the output layout
      kernel-doc: improve handling of whitespace on the first line param description
      kernel-doc: strip leading whitespace from continued param descs
      kernel-doc/rst: use *undescribed* instead of _undescribed_
      kernel-doc/rst: remove fixme comment
      kernel-doc: limit the "section header:" detection to a select few
      kernel-doc: concatenate contents of colliding sections
      kernel-doc: reset contents and section harder
      Documentation/sphinx: fix kernel-doc extension on python3

Jonathan Corbet (1):
      sphinx: cheesy script to convert .tmpl files

 Documentation/.gitignore                  |   1 +
 Documentation/DocBook/Makefile            |   7 +-
 Documentation/Makefile.sphinx             |  63 +++++
 Documentation/conf.py                     | 414 ++++++++++++++++++++++++++++++
 Documentation/index.rst                   |  23 ++
 Documentation/sphinx/convert_template.sed |  18 ++
 Documentation/sphinx/kernel-doc.py        | 127 +++++++++
 Documentation/sphinx/post_convert.sed     |  23 ++
 Documentation/sphinx/tmplcvt              |  19 ++
 Makefile                                  |   5 +-
 scripts/kernel-doc                        | 392 ++++++++++++++++++----------
 11 files changed, 948 insertions(+), 144 deletions(-)
 create mode 100644 Documentation/.gitignore
 create mode 100644 Documentation/Makefile.sphinx
 create mode 100644 Documentation/conf.py
 create mode 100644 Documentation/index.rst
 create mode 100644 Documentation/sphinx/convert_template.sed
 create mode 100644 Documentation/sphinx/kernel-doc.py
 create mode 100644 Documentation/sphinx/post_convert.sed
 create mode 100755 Documentation/sphinx/tmplcvt

-- 
2.1.4

[toc] | [next] | [standalone]


#1413698 — [PATCH v2 10/38] Documentation/sphinx: nicer referencing of struct in docbook->rst conversion

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:40 +0200
Subject[PATCH v2 10/38] Documentation/sphinx: nicer referencing of struct in docbook->rst conversion
Message-ID<rGi5Q-8on-13@gated-at.bofh.it>
In reply to#1413696
Add "struct" in the label of the reference.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 Documentation/sphinx/convert_template.sed | 10 +++++++---
 Documentation/sphinx/post_convert.sed     | 10 +++++++---
 2 files changed, 14 insertions(+), 6 deletions(-)

diff --git a/Documentation/sphinx/convert_template.sed b/Documentation/sphinx/convert_template.sed
index 0553eabeac79..c1503fcca4ec 100644
--- a/Documentation/sphinx/convert_template.sed
+++ b/Documentation/sphinx/convert_template.sed
@@ -2,12 +2,16 @@
 # Pandoc doesn't grok <function> or <structname>, so convert them
 # ahead of time.
 #
-# Use "$bq" instead of "`" so that pandoc won't mess with it.
+# Use the following escapes to pass through pandoc:
+#	$bq = "`"
+#	$lt = "<"
+#	$gt = ">"
 #
 s%<function>\([^<(]\+\)()</function>%:c:func:$bq\1()$bq%g
 s%<function>\([^<(]\+\)</function>%:c:func:$bq\1()$bq%g
-s%<structname>struct *\([^<]\+\)</structname>%:c:type:$bq\1$bq%g
-s%<structname>\([^<]\+\)</structname>%:c:type:$bq\1$bq%g
+s%<structname>struct *\([^<]\+\)</structname>%:c:type:$bqstruct \1 $lt\1$gt$bq%g
+s%struct <structname>\([^<]\+\)</structname>%:c:type:$bqstruct \1 $lt\1$gt$bq%g
+s%<structname>\([^<]\+\)</structname>%:c:type:$bqstruct \1 $lt\1$gt$bq%g
 #
 # Wrap docproc directives in para and code blocks.
 #
diff --git a/Documentation/sphinx/post_convert.sed b/Documentation/sphinx/post_convert.sed
index ce17049f6a8e..392770bac53b 100644
--- a/Documentation/sphinx/post_convert.sed
+++ b/Documentation/sphinx/post_convert.sed
@@ -1,9 +1,13 @@
 #
-# pandoc thinks that both "_" needs to be escaped.  Remove the extra
-# backslashes.  Also put in proper backquotes now that pandoc won't quote
-# them.
+# Unescape.
 #
 s/$bq/`/g
+s/$lt/</g
+s/$gt/>/g
+#
+# pandoc thinks that both "_" needs to be escaped.  Remove the extra
+# backslashes.
+#
 s/\\_/_/g
 #
 # Unwrap docproc directives.
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413701 — [PATCH v2 01/38] kernel-doc/rst: fix use of uninitialized value

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:40 +0200
Subject[PATCH v2 01/38] kernel-doc/rst: fix use of uninitialized value
Message-ID<rGi5Q-8on-17@gated-at.bofh.it>
In reply to#1413696
I'm not quite sure why the errors below are happening, but this fixes
them.

Use of uninitialized value in string ne at ./scripts/kernel-doc line 1819, <IN> line 6494.
Use of uninitialized value $_[0] in join or string at ./scripts/kernel-doc line 1759, <IN> line 6494.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 2fc8fad5195e..babb374c043d 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1803,7 +1803,8 @@ sub output_function_rst(%) {
 	} else {
 	    print "      ``$parameter``\n";
 	}
-	if ($args{'parameterdescs'}{$parameter_name} ne $undescribed) {
+	if (defined($args{'parameterdescs'}{$parameter_name}) &&
+	    $args{'parameterdescs'}{$parameter_name} ne $undescribed) {
 	    my $oldprefix = $lineprefix;
 	    $lineprefix = "        ";
 	    output_highlight_rst($args{'parameterdescs'}{$parameter_name});
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413702 — [PATCH v2 13/38] kernel-doc/rst: do not output DOC: section titles for requested ones

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:40 +0200
Subject[PATCH v2 13/38] kernel-doc/rst: do not output DOC: section titles for requested ones
Message-ID<rGi5Q-8on-19@gated-at.bofh.it>
In reply to#1413696
If the user requests a specific DOC: section by name, do not output its
section title. In these cases, the surrounding context already has a
heading, and the DOC: section title is only used as an identifier and a
heading for clarity in the source file.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index dd08944b0a6f..659d529b99d8 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1764,7 +1764,9 @@ sub output_blockhead_rst(%) {
     my ($parameter, $section);
 
     foreach $section (@{$args{'sectionlist'}}) {
-	print "**$section**\n\n";
+	if ($output_selection != OUTPUT_INCLUDE) {
+	    print "**$section**\n\n";
+	}
 	output_highlight_rst($args{'sections'}{$section});
 	print "\n";
     }
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413703 — [PATCH v2 14/38] kernel-doc/rst: reference functions according to C domain spec

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:40 +0200
Subject[PATCH v2 14/38] kernel-doc/rst: reference functions according to C domain spec
Message-ID<rGi5Q-8on-23@gated-at.bofh.it>
In reply to#1413696
The Sphinx C domain spec says function references should include the
parens ().

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 659d529b99d8..e8651d7cf1cd 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -280,7 +280,7 @@ my $blankline_text = "";
 # rst-mode
 my @highlights_rst = (
                        [$type_constant, "``\$1``"],
-                       [$type_func, "\\:c\\:func\\:`\$1`"],
+                       [$type_func, "\\:c\\:func\\:`\$1()`"],
                        [$type_struct_full, "\\:c\\:type\\:`\$1 \$2 <\$2>`"],
                        [$type_enum_full, "\\:c\\:type\\:`\$1 \$2 <\$2>`"],
                        [$type_struct, "\\:c\\:type\\:`struct \$1 <\$1>`"],
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413704 — [PATCH v2 08/38] sphinx: cheesy script to convert .tmpl files

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:40 +0200
Subject[PATCH v2 08/38] sphinx: cheesy script to convert .tmpl files
Message-ID<rGi5Q-8on-21@gated-at.bofh.it>
In reply to#1413696
From: Jonathan Corbet <corbet@lwn.net>

This script uses pandoc to convert existing DocBook template files to RST
templates.  A couple of sed scripts are need to massage things both before
and after the conversion, but the result is then usable with no hand
editing.

[Jani: Change usage to tmplcvt <in> <out>. Fix escaping for docproc
directives. Add support the new kernel-doc extension.]

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 Documentation/sphinx/convert_template.sed | 14 ++++++++++++++
 Documentation/sphinx/post_convert.sed     | 19 +++++++++++++++++++
 Documentation/sphinx/tmplcvt              | 19 +++++++++++++++++++
 3 files changed, 52 insertions(+)
 create mode 100644 Documentation/sphinx/convert_template.sed
 create mode 100644 Documentation/sphinx/post_convert.sed
 create mode 100755 Documentation/sphinx/tmplcvt

diff --git a/Documentation/sphinx/convert_template.sed b/Documentation/sphinx/convert_template.sed
new file mode 100644
index 000000000000..d53bb8220a26
--- /dev/null
+++ b/Documentation/sphinx/convert_template.sed
@@ -0,0 +1,14 @@
+#
+# Pandoc doesn't grok <function> or <structname>, so convert them
+# ahead of time.
+#
+# Use "$bq" instead of "`" so that pandoc won't mess with it.
+#
+s%<function>\([^<(]\+\)()</function>%:c:func:$bq\1$bq%g
+s%<function>\([^<(]\+\)</function>%:c:func:$bq\1$bq%g
+s%<structname>struct *\([^<]\+\)</structname>%:ref:$bqstruct \1$bq%g
+s%<structname>\([^<]\+\)</structname>%:ref:$bqstruct \1$bq%g
+#
+# Wrap docproc directives in para and code blocks.
+#
+s%^\(!.*\)$%<para><code>DOCPROC: \1</code></para>%
diff --git a/Documentation/sphinx/post_convert.sed b/Documentation/sphinx/post_convert.sed
new file mode 100644
index 000000000000..ce17049f6a8e
--- /dev/null
+++ b/Documentation/sphinx/post_convert.sed
@@ -0,0 +1,19 @@
+#
+# pandoc thinks that both "_" needs to be escaped.  Remove the extra
+# backslashes.  Also put in proper backquotes now that pandoc won't quote
+# them.
+#
+s/$bq/`/g
+s/\\_/_/g
+#
+# Unwrap docproc directives.
+#
+s/^``DOCPROC: !E\(.*\)``$/.. kernel-doc:: \1\n   :export:/
+s/^``DOCPROC: !I\(.*\)``$/.. kernel-doc:: \1\n   :internal:/
+s/^``DOCPROC: !F\([^ ]*\) \(.*\)``$/.. kernel-doc:: \1\n   :functions: \2/
+s/^``DOCPROC: !P\([^ ]*\) \(.*\)``$/.. kernel-doc:: \1\n   :doc: \2/
+s/^``DOCPROC: \(!.*\)``$/.. WARNING: DOCPROC directive not supported: \1/
+#
+# Trim trailing whitespace.
+#
+s/[[:space:]]*$//
diff --git a/Documentation/sphinx/tmplcvt b/Documentation/sphinx/tmplcvt
new file mode 100755
index 000000000000..909a73065e0a
--- /dev/null
+++ b/Documentation/sphinx/tmplcvt
@@ -0,0 +1,19 @@
+#!/bin/bash
+#
+# Convert a template file into something like RST
+#
+# fix <function>
+# feed to pandoc
+# fix \_
+# title line?
+#
+
+in=$1
+rst=$2
+tmp=$rst.tmp
+
+cp $in $tmp
+sed --in-place -f convert_template.sed $tmp
+pandoc -s -S -f docbook -t rst -o $rst $tmp
+sed --in-place -f post_convert.sed $rst
+rm $tmp
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413705 — [PATCH v2 02/38] kernel-doc: support printing exported and non-exported symbols

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:40 +0200
Subject[PATCH v2 02/38] kernel-doc: support printing exported and non-exported symbols
Message-ID<rGi5Q-8on-29@gated-at.bofh.it>
In reply to#1413696
Currently we use docproc to figure out which symbols are exported, and
then docproc calls kernel-doc on specific functions, to get
documentation on exported functions. According to git blame and docproc
comments, this is due to historical reasons, as functions and their
corresponding EXPORT_SYMBOL* may have been in different files. However
for more than ten years the recommendation in CodingStyle has been to
place the EXPORT_SYMBOL* immediately after the closing function brace
line.

Additionally, the kernel-doc comments for functions are generally placed
above the function definition in the .c files (i.e. where the
EXPORT_SYMBOL* is) rather than above the declaration in the .h
files. There are some exceptions to this, but AFAICT none of these are
included in DocBook documentation using the "!E" docproc directive.

Therefore, assuming the EXPORT_SYMBOL* and kernel-doc are with the
function definition, kernel-doc can extract the exported vs. not
information by making two passes on the input file. Add support for that
via the new -export and -internal parameters.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 29 +++++++++++++++++++++++++++--
 1 file changed, 27 insertions(+), 2 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index babb374c043d..3ad54abe0989 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -59,6 +59,12 @@ Output format selection (mutually exclusive):
   -text			Output plain text format.
 
 Output selection (mutually exclusive):
+  -export		Only output documentation for symbols that have been
+			exported using EXPORT_SYMBOL() or EXPORT_SYMBOL_GPL()
+			in the same FILE.
+  -internal		Only output documentation for symbols that have NOT been
+			exported using EXPORT_SYMBOL() or EXPORT_SYMBOL_GPL()
+			in the same FILE.
   -function NAME	Only output documentation for the given function(s)
 			or DOC: section title(s). All other functions and DOC:
 			sections are ignored. May be specified multiple times.
@@ -380,6 +386,7 @@ my $doc_block = $doc_com . 'DOC:\s*(.*)?';
 my $doc_split_start = '^\s*/\*\*\s*$';
 my $doc_split_sect = '\s*\*\s*(@[\w\s]+):(.*)';
 my $doc_split_end = '^\s*\*/\s*$';
+my $export_symbol = '^\s*EXPORT_SYMBOL(_GPL)?\s*\(\s*(\w+)\s*\)\s*;';
 
 my %constants;
 my %parameterdescs;
@@ -444,6 +451,12 @@ while ($ARGV[0] =~ m/^-(.*)/) {
 	$function_only = 2;
 	$function = shift @ARGV;
 	$function_table{$function} = 1;
+    } elsif ($cmd eq "-export") { # only exported symbols
+	$function_only = 3;
+	%function_table = ()
+    } elsif ($cmd eq "-internal") { # only non-exported symbols
+	$function_only = 4;
+	%function_table = ()
     } elsif ($cmd eq "-v") {
 	$verbose = 1;
     } elsif (($cmd eq "-h") || ($cmd eq "--help")) {
@@ -1971,8 +1984,10 @@ sub output_declaration {
     my $functype = shift;
     my $func = "output_${functype}_$output_mode";
     if (($function_only==0) ||
-	( $function_only == 1 && defined($function_table{$name})) ||
-	( $function_only == 2 && !($functype eq "function" && defined($function_table{$name}))))
+	( ($function_only == 1 || $function_only == 3) &&
+	  defined($function_table{$name})) ||
+	( ($function_only == 2 || $function_only == 4) &&
+	  !($functype eq "function" && defined($function_table{$name}))))
     {
 	&$func(@_);
 	$section_counter++;
@@ -2675,6 +2690,16 @@ sub process_file($) {
 	return;
     }
 
+    # two passes for -export and -internal
+    if ($function_only == 3 || $function_only == 4) {
+	while (<IN>) {
+	    if (/$export_symbol/o) {
+		$function_table{$2} = 1;
+	    }
+	}
+	seek(IN, 0, 0);
+    }
+
     $. = 1;
 
     $section_counter = 0;
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413706 — [PATCH v2 12/38] kernel-doc: add names for output selection

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:40 +0200
Subject[PATCH v2 12/38] kernel-doc: add names for output selection
Message-ID<rGi5Q-8on-31@gated-at.bofh.it>
In reply to#1413696
Make the output selection a bit more readable by adding constants for
the various types of output selection. While at it, actually call the
variable for choosing what to output $output_selection.

No functional changes.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 47 ++++++++++++++++++++++++++++++-----------------
 1 file changed, 30 insertions(+), 17 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index cb5fd248ac57..dd08944b0a6f 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -312,7 +312,15 @@ my $no_doc_sections = 0;
 my @highlights = @highlights_man;
 my $blankline = $blankline_man;
 my $modulename = "Kernel API";
-my $function_only = 0;
+
+use constant {
+    OUTPUT_ALL          => 0, # output all symbols and doc sections
+    OUTPUT_INCLUDE      => 1, # output only specified symbols
+    OUTPUT_EXCLUDE      => 2, # output everything except specified symbols
+    OUTPUT_EXPORTED     => 3, # output exported symbols
+    OUTPUT_INTERNAL     => 4, # output non-exported symbols
+};
+my $output_selection = OUTPUT_ALL;
 my $show_not_found = 0;
 
 my @build_time;
@@ -449,18 +457,18 @@ while ($ARGV[0] =~ m/^-(.*)/) {
     } elsif ($cmd eq "-module") { # not needed for XML, inherits from calling document
 	$modulename = shift @ARGV;
     } elsif ($cmd eq "-function") { # to only output specific functions
-	$function_only = 1;
+	$output_selection = OUTPUT_INCLUDE;
 	$function = shift @ARGV;
 	$function_table{$function} = 1;
-    } elsif ($cmd eq "-nofunction") { # to only output specific functions
-	$function_only = 2;
+    } elsif ($cmd eq "-nofunction") { # output all except specific functions
+	$output_selection = OUTPUT_EXCLUDE;
 	$function = shift @ARGV;
 	$function_table{$function} = 1;
     } elsif ($cmd eq "-export") { # only exported symbols
-	$function_only = 3;
+	$output_selection = OUTPUT_EXPORTED;
 	%function_table = ()
     } elsif ($cmd eq "-internal") { # only non-exported symbols
-	$function_only = 4;
+	$output_selection = OUTPUT_INTERNAL;
 	%function_table = ()
     } elsif ($cmd eq "-v") {
 	$verbose = 1;
@@ -530,15 +538,17 @@ sub dump_doc_section {
         return;
     }
 
-    if (($function_only == 0) ||
-	( $function_only == 1 && defined($function_table{$name})) ||
-	( $function_only == 2 && !defined($function_table{$name})))
+    if (($output_selection == OUTPUT_ALL) ||
+	($output_selection == OUTPUT_INCLUDE &&
+	 defined($function_table{$name})) ||
+	($output_selection == OUTPUT_EXCLUDE &&
+	 !defined($function_table{$name})))
     {
 	dump_section($file, $name, $contents);
 	output_blockhead({'sectionlist' => \@sectionlist,
 			  'sections' => \%sections,
 			  'module' => $modulename,
-			  'content-only' => ($function_only != 0), });
+			  'content-only' => ($output_selection != OUTPUT_ALL), });
     }
 }
 
@@ -1988,11 +1998,13 @@ sub output_declaration {
     my $name = shift;
     my $functype = shift;
     my $func = "output_${functype}_$output_mode";
-    if (($function_only==0) ||
-	( ($function_only == 1 || $function_only == 3) &&
-	  defined($function_table{$name})) ||
-	( ($function_only == 2 || $function_only == 4) &&
-	  !($functype eq "function" && defined($function_table{$name}))))
+    if (($output_selection == OUTPUT_ALL) ||
+	(($output_selection == OUTPUT_INCLUDE ||
+	  $output_selection == OUTPUT_EXPORTED) &&
+	 defined($function_table{$name})) ||
+	(($output_selection == OUTPUT_EXCLUDE ||
+	  $output_selection == OUTPUT_INTERNAL) &&
+	 !($functype eq "function" && defined($function_table{$name}))))
     {
 	&$func(@_);
 	$section_counter++;
@@ -2696,7 +2708,8 @@ sub process_file($) {
     }
 
     # two passes for -export and -internal
-    if ($function_only == 3 || $function_only == 4) {
+    if ($output_selection == OUTPUT_EXPORTED ||
+	$output_selection == OUTPUT_INTERNAL) {
 	while (<IN>) {
 	    if (/$export_symbol/o) {
 		$function_table{$2} = 1;
@@ -2929,7 +2942,7 @@ sub process_file($) {
     }
     if ($initial_section_counter == $section_counter) {
 	print STDERR "${file}:1: warning: no structured comments found\n";
-	if (($function_only == 1) && ($show_not_found == 1)) {
+	if (($output_selection == OUTPUT_INCLUDE) && ($show_not_found == 1)) {
 	    print STDERR "    Was looking for '$_'.\n" for keys %function_table;
 	}
 	if ($output_mode eq "xml") {
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413707 — [PATCH v2 15/38] kernel-doc/rst: &foo references are more universal than structs

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:40 +0200
Subject[PATCH v2 15/38] kernel-doc/rst: &foo references are more universal than structs
Message-ID<rGi5Q-8on-35@gated-at.bofh.it>
In reply to#1413696
It's possible to use &foo to reference structs, enums, typedefs, etc. in
the Sphinx C domain. Thus do not prefix the links with "struct".

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index e8651d7cf1cd..e7aa792e7f1b 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -283,7 +283,8 @@ my @highlights_rst = (
                        [$type_func, "\\:c\\:func\\:`\$1()`"],
                        [$type_struct_full, "\\:c\\:type\\:`\$1 \$2 <\$2>`"],
                        [$type_enum_full, "\\:c\\:type\\:`\$1 \$2 <\$2>`"],
-                       [$type_struct, "\\:c\\:type\\:`struct \$1 <\$1>`"],
+                       # in rst this can refer to any type
+                       [$type_struct, "\\:c\\:type\\:`\$1`"],
                        [$type_param, "**\$1**"]
 		      );
 my $blankline_rst = "\n";
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413708 — [PATCH v2 05/38] Documentation/sphinx: add Sphinx kernel-doc directive extension

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:40 +0200
Subject[PATCH v2 05/38] Documentation/sphinx: add Sphinx kernel-doc directive extension
Message-ID<rGi5Q-8on-25@gated-at.bofh.it>
In reply to#1413696
Add an extension to handle kernel-doc directives, to call kernel-doc
according to the arguments and parameters given to the reStructuredText
directive.

The syntax for the kernel-doc directive is:

.. kernel-doc:: FILENAME
   :export:
   :internal:
   :functions: FUNCTION [FUNCTION ...]
   :doc: SECTION TITLE

Of the directive options export, internal, functions, and doc, currently
only one option may be given at a time.

The FILENAME is relative from the kernel source tree root.

The extension notifies Sphinx about the document dependency on FILENAME,
causing the document to be rebuilt when the file has been changed.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 Documentation/sphinx/kernel-doc.py | 102 +++++++++++++++++++++++++++++++++++++
 1 file changed, 102 insertions(+)
 create mode 100644 Documentation/sphinx/kernel-doc.py

diff --git a/Documentation/sphinx/kernel-doc.py b/Documentation/sphinx/kernel-doc.py
new file mode 100644
index 000000000000..87a1332fe934
--- /dev/null
+++ b/Documentation/sphinx/kernel-doc.py
@@ -0,0 +1,102 @@
+# coding=utf-8
+#
+# Copyright © 2016 Intel Corporation
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+#
+# Authors:
+#    Jani Nikula <jani.nikula@intel.com>
+
+import os
+import subprocess
+import sys
+
+from docutils import nodes, statemachine
+from docutils.parsers.rst import directives
+from sphinx.util.compat import Directive
+
+class KernelDocDirective(Directive):
+    """Extract kernel-doc comments from the specified file"""
+    required_argument = 1
+    optional_arguments = 4
+    option_spec = {
+        'doc': directives.unchanged_required,
+        'functions': directives.unchanged_required,
+        'export': directives.flag,
+        'internal': directives.flag,
+    }
+    has_content = False
+
+    def run(self):
+        env = self.state.document.settings.env
+        cmd = [env.config.kerneldoc_bin, '-rst']
+
+        filename = env.config.kerneldoc_srctree + '/' + self.arguments[0]
+
+        # Tell sphinx of the dependency
+        env.note_dependency(os.path.abspath(filename))
+
+        tab_width = self.options.get('tab-width', self.state.document.settings.tab_width)
+        source = self.state_machine.input_lines.source(self.lineno - self.state_machine.input_offset - 1)
+
+        # FIXME: make this nicer and more robust against errors
+        if 'export' in self.options:
+            cmd += ['-export']
+        elif 'internal' in self.options:
+            cmd += ['-internal']
+        elif 'doc' in self.options:
+            cmd += ['-function', str(self.options.get('doc'))]
+        elif 'functions' in self.options:
+            for f in str(self.options.get('functions')).split(' '):
+                cmd += ['-function', f]
+
+        cmd += [filename]
+
+        try:
+            env.app.verbose('calling kernel-doc \'%s\'' % (" ".join(cmd)))
+
+            p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
+            out, err = p.communicate()
+
+            # assume the kernel sources are utf-8
+            out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8')
+
+            if p.returncode != 0:
+                sys.stderr.write(err)
+
+                env.app.warn('kernel-doc \'%s\' failed with return code %d' % (" ".join(cmd), p.returncode))
+                return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
+            elif env.config.kerneldoc_verbosity > 0:
+                sys.stderr.write(err)
+
+            lines = statemachine.string2lines(out, tab_width, convert_whitespace=True)
+            self.state_machine.insert_input(lines, source)
+            return []
+        except Exception as e:
+            env.app.warn('kernel-doc \'%s\' processing failed with: %s' %
+                         (" ".join(cmd), str(e)))
+            return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
+
+def setup(app):
+    app.add_config_value('kerneldoc_bin', None, 'env')
+    app.add_config_value('kerneldoc_srctree', None, 'env')
+    app.add_config_value('kerneldoc_verbosity', 1, 'env')
+
+    app.add_directive('kernel-doc', KernelDocDirective)
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413709 — [PATCH v2 03/38] Documentation/sphinx: add basic working Sphinx configuration and build

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:40 +0200
Subject[PATCH v2 03/38] Documentation/sphinx: add basic working Sphinx configuration and build
Message-ID<rGi5R-8on-37@gated-at.bofh.it>
In reply to#1413696
Add basic configuration and makefile to build documentation from any
.rst files under Documentation using Sphinx. For starters, there's just
the placeholder index.rst.

At the top level Makefile, hook Sphinx documentation targets alongside
(but independent of) the DocBook toolchain, having both be run on the
various 'make *docs' targets.

All Sphinx processing is placed into Documentation/Makefile.sphinx. Both
that and the Documentation/DocBook/Makefile are now expected to handle
all the documentation targets, explicitly ignoring them if they're not
relevant for that particular toolchain. The changes to the existing
DocBook Makefile are kept minimal.

There is graceful handling of missing Sphinx and rst2pdf (which is
needed for pdf output) by checking for the tool and python module,
respectively, with informative messages to the user.

If the Read the Docs theme (sphinx_rtd_theme) is available, use it, but
otherwise gracefully fall back to the Sphinx default theme, with an
informative message to the user, and slightly less pretty HTML output.

Sphinx can now handle htmldocs, pdfdocs (if rst2pdf is available),
epubdocs and xmldocs targets. The output documents are written into per
output type subdirectories under Documentation/output.

Finally, you can pass options to sphinx-build using the SPHINXBUILD make
variable. For example, 'make SPHINXOPTS=-v htmldocs' for more verbose
output from Sphinx.

This is based on the original work by Jonathan Corbet, but he probably
wouldn't recognize this as his own anymore.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 Documentation/DocBook/Makefile |   7 +-
 Documentation/Makefile.sphinx  |  61 +++++++
 Documentation/conf.py          | 387 +++++++++++++++++++++++++++++++++++++++++
 Documentation/index.rst        |  23 +++
 Makefile                       |   5 +-
 5 files changed, 478 insertions(+), 5 deletions(-)
 create mode 100644 Documentation/Makefile.sphinx
 create mode 100644 Documentation/conf.py
 create mode 100644 Documentation/index.rst

diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile
index d70f9b68174e..e0c7e1e0590b 100644
--- a/Documentation/DocBook/Makefile
+++ b/Documentation/DocBook/Makefile
@@ -33,10 +33,6 @@ PDF_METHOD	= $(prefer-db2x)
 PS_METHOD	= $(prefer-db2x)
 
 
-###
-# The targets that may be used.
-PHONY += xmldocs sgmldocs psdocs pdfdocs htmldocs mandocs installmandocs cleandocs
-
 targets += $(DOCBOOKS)
 BOOKS := $(addprefix $(obj)/,$(DOCBOOKS))
 xmldocs: $(BOOKS)
@@ -63,6 +59,9 @@ installmandocs: mandocs
 		sort -k 2 -k 1 | uniq -f 1 | sed -e 's: :/:' | \
 		xargs install -m 644 -t /usr/local/man/man9/
 
+# no-op for the DocBook toolchain
+epubdocs:
+
 ###
 #External programs used
 KERNELDOCXMLREF = $(srctree)/scripts/kernel-doc-xml-ref
diff --git a/Documentation/Makefile.sphinx b/Documentation/Makefile.sphinx
new file mode 100644
index 000000000000..25a7970bfe5c
--- /dev/null
+++ b/Documentation/Makefile.sphinx
@@ -0,0 +1,61 @@
+# -*- makefile -*-
+# Makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SPHINXBUILD   = sphinx-build
+SPHINXOPTS    =
+PAPER         =
+BUILDDIR      = $(obj)/output
+
+# User-friendly check for sphinx-build
+HAVE_SPHINX := $(shell if which $(SPHINXBUILD) >/dev/null 2>&1; then echo 1; else echo 0; fi)
+
+ifeq ($(HAVE_SPHINX),0)
+
+.DEFAULT:
+	$(warning The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed and in PATH, or set the SPHINXBUILD make variable to point to the full path of the '$(SPHINXBUILD)' executable.)
+	@echo "  SKIP    Sphinx $@ target."
+
+else # HAVE_SPHINX
+
+# User-friendly check for rst2pdf
+HAVE_RST2PDF := $(shell if python -c "import rst2pdf" >/dev/null 2>&1; then echo 1; else echo 0; fi)
+
+# Internal variables.
+PAPEROPT_a4     = -D latex_paper_size=a4
+PAPEROPT_letter = -D latex_paper_size=letter
+ALLSPHINXOPTS   = -d $(BUILDDIR)/.doctrees $(PAPEROPT_$(PAPER)) -c $(srctree)/$(src) $(SPHINXOPTS) $(srctree)/$(src)
+# the i18n builder cannot share the environment and doctrees with the others
+I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
+
+quiet_cmd_sphinx = SPHINX  $@
+      cmd_sphinx = $(SPHINXBUILD) -b $2 $(ALLSPHINXOPTS) $(BUILDDIR)/$2
+
+htmldocs:
+	$(call cmd,sphinx,html)
+
+pdfdocs:
+ifeq ($(HAVE_RST2PDF),0)
+	$(warning The Python 'rst2pdf' module was not found. Make sure you have the module installed to produce PDF output.)
+	@echo "  SKIP    Sphinx $@ target."
+else # HAVE_RST2PDF
+	$(call cmd,sphinx,pdf)
+endif # HAVE_RST2PDF
+
+epubdocs:
+	$(call cmd,sphinx,epub)
+
+xmldocs:
+	$(call cmd,sphinx,xml)
+
+# no-ops for the Sphinx toolchain
+sgmldocs:
+psdocs:
+mandocs:
+installmandocs:
+
+cleandocs:
+	$(Q)rm -rf $(BUILDDIR)
+
+endif # HAVE_SPHINX
diff --git a/Documentation/conf.py b/Documentation/conf.py
new file mode 100644
index 000000000000..0f0e20757bb1
--- /dev/null
+++ b/Documentation/conf.py
@@ -0,0 +1,387 @@
+# -*- coding: utf-8 -*-
+#
+# The Linux Kernel documentation build configuration file, created by
+# sphinx-quickstart on Fri Feb 12 13:51:46 2016.
+#
+# This file is execfile()d with the current directory set to its
+# containing dir.
+#
+# Note that not all possible configuration values are present in this
+# autogenerated file.
+#
+# All configuration values have a default; values that are commented out
+# serve to show the default.
+
+import sys
+import os
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#sys.path.insert(0, os.path.abspath('.'))
+
+# -- General configuration ------------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+#needs_sphinx = '1.0'
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = []
+
+# Gracefully handle missing rst2pdf.
+try:
+    import rst2pdf
+    extensions += ['rst2pdf.pdfbuilder']
+except ImportError:
+    pass
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+# source_suffix = ['.rst', '.md']
+source_suffix = '.rst'
+
+# The encoding of source files.
+#source_encoding = 'utf-8-sig'
+
+# The master toctree document.
+master_doc = 'index'
+
+# General information about the project.
+project = 'The Linux Kernel'
+copyright = '2016, The kernel development community'
+author = 'The kernel development community'
+
+# The version info for the project you're documenting, acts as replacement for
+# |version| and |release|, also used in various other places throughout the
+# built documents.
+#
+# The short X.Y version.
+version = '4.5'
+# The full version, including alpha/beta/rc tags.
+release = '4.5'
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#
+# This is also used if you do content translation via gettext catalogs.
+# Usually you set "language" from the command line for these cases.
+language = None
+
+# There are two options for replacing |today|: either, you set today to some
+# non-false value, then it is used:
+#today = ''
+# Else, today_fmt is used as the format for a strftime call.
+#today_fmt = '%B %d, %Y'
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+exclude_patterns = ['output']
+
+# The reST default role (used for this markup: `text`) to use for all
+# documents.
+#default_role = None
+
+# If true, '()' will be appended to :func: etc. cross-reference text.
+#add_function_parentheses = True
+
+# If true, the current module name will be prepended to all description
+# unit titles (such as .. function::).
+#add_module_names = True
+
+# If true, sectionauthor and moduleauthor directives will be shown in the
+# output. They are ignored by default.
+#show_authors = False
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = 'sphinx'
+
+# A list of ignored prefixes for module index sorting.
+#modindex_common_prefix = []
+
+# If true, keep warnings as "system message" paragraphs in the built documents.
+#keep_warnings = False
+
+# If true, `todo` and `todoList` produce output, else they produce nothing.
+todo_include_todos = False
+
+primary_domain = 'C'
+highlight_language = 'C'
+
+# -- Options for HTML output ----------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+
+# The Read the Docs theme is available from
+# - https://github.com/snide/sphinx_rtd_theme
+# - https://pypi.python.org/pypi/sphinx_rtd_theme
+# - python-sphinx-rtd-theme package (on Debian)
+try:
+    import sphinx_rtd_theme
+    html_theme = 'sphinx_rtd_theme'
+    html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
+except ImportError:
+    sys.stderr.write('Warning: The Sphinx \'sphinx_rtd_theme\' HTML theme was not found. Make sure you have the theme installed to produce pretty HTML output. Falling back to the default theme.\n')
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further.  For a list of options available for each theme, see the
+# documentation.
+#html_theme_options = {}
+
+# Add any paths that contain custom themes here, relative to this directory.
+#html_theme_path = []
+
+# The name for this set of Sphinx documents.  If None, it defaults to
+# "<project> v<release> documentation".
+#html_title = None
+
+# A shorter title for the navigation bar.  Default is the same as html_title.
+#html_short_title = None
+
+# The name of an image file (relative to this directory) to place at the top
+# of the sidebar.
+#html_logo = None
+
+# The name of an image file (within the static path) to use as favicon of the
+# docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
+# pixels large.
+#html_favicon = None
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+#html_static_path = ['_static']
+
+# Add any extra paths that contain custom files (such as robots.txt or
+# .htaccess) here, relative to this directory. These files are copied
+# directly to the root of the documentation.
+#html_extra_path = []
+
+# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
+# using the given strftime format.
+#html_last_updated_fmt = '%b %d, %Y'
+
+# If true, SmartyPants will be used to convert quotes and dashes to
+# typographically correct entities.
+#html_use_smartypants = True
+
+# Custom sidebar templates, maps document names to template names.
+#html_sidebars = {}
+
+# Additional templates that should be rendered to pages, maps page names to
+# template names.
+#html_additional_pages = {}
+
+# If false, no module index is generated.
+#html_domain_indices = True
+
+# If false, no index is generated.
+#html_use_index = True
+
+# If true, the index is split into individual pages for each letter.
+#html_split_index = False
+
+# If true, links to the reST sources are added to the pages.
+#html_show_sourcelink = True
+
+# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
+#html_show_sphinx = True
+
+# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
+#html_show_copyright = True
+
+# If true, an OpenSearch description file will be output, and all pages will
+# contain a <link> tag referring to it.  The value of this option must be the
+# base URL from which the finished HTML is served.
+#html_use_opensearch = ''
+
+# This is the file name suffix for HTML files (e.g. ".xhtml").
+#html_file_suffix = None
+
+# Language to be used for generating the HTML full-text search index.
+# Sphinx supports the following languages:
+#   'da', 'de', 'en', 'es', 'fi', 'fr', 'h', 'it', 'ja'
+#   'nl', 'no', 'pt', 'ro', 'r', 'sv', 'tr'
+#html_search_language = 'en'
+
+# A dictionary with options for the search language support, empty by default.
+# Now only 'ja' uses this config value
+#html_search_options = {'type': 'default'}
+
+# The name of a javascript file (relative to the configuration directory) that
+# implements a search results scorer. If empty, the default will be used.
+#html_search_scorer = 'scorer.js'
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'TheLinuxKerneldoc'
+
+# -- Options for LaTeX output ---------------------------------------------
+
+latex_elements = {
+# The paper size ('letterpaper' or 'a4paper').
+#'papersize': 'letterpaper',
+
+# The font size ('10pt', '11pt' or '12pt').
+#'pointsize': '10pt',
+
+# Additional stuff for the LaTeX preamble.
+#'preamble': '',
+
+# Latex figure (float) alignment
+#'figure_align': 'htbp',
+}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+#  author, documentclass [howto, manual, or own class]).
+latex_documents = [
+    (master_doc, 'TheLinuxKernel.tex', 'The Linux Kernel Documentation',
+     'The kernel development community', 'manual'),
+]
+
+# The name of an image file (relative to this directory) to place at the top of
+# the title page.
+#latex_logo = None
+
+# For "manual" documents, if this is true, then toplevel headings are parts,
+# not chapters.
+#latex_use_parts = False
+
+# If true, show page references after internal links.
+#latex_show_pagerefs = False
+
+# If true, show URL addresses after external links.
+#latex_show_urls = False
+
+# Documents to append as an appendix to all manuals.
+#latex_appendices = []
+
+# If false, no module index is generated.
+#latex_domain_indices = True
+
+
+# -- Options for manual page output ---------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [
+    (master_doc, 'thelinuxkernel', 'The Linux Kernel Documentation',
+     [author], 1)
+]
+
+# If true, show URL addresses after external links.
+#man_show_urls = False
+
+
+# -- Options for Texinfo output -------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+#  dir menu entry, description, category)
+texinfo_documents = [
+    (master_doc, 'TheLinuxKernel', 'The Linux Kernel Documentation',
+     author, 'TheLinuxKernel', 'One line description of project.',
+     'Miscellaneous'),
+]
+
+# Documents to append as an appendix to all manuals.
+#texinfo_appendices = []
+
+# If false, no module index is generated.
+#texinfo_domain_indices = True
+
+# How to display URL addresses: 'footnote', 'no', or 'inline'.
+#texinfo_show_urls = 'footnote'
+
+# If true, do not generate a @detailmenu in the "Top" node's menu.
+#texinfo_no_detailmenu = False
+
+
+# -- Options for Epub output ----------------------------------------------
+
+# Bibliographic Dublin Core info.
+epub_title = project
+epub_author = author
+epub_publisher = author
+epub_copyright = copyright
+
+# The basename for the epub file. It defaults to the project name.
+#epub_basename = project
+
+# The HTML theme for the epub output. Since the default themes are not
+# optimized for small screen space, using the same theme for HTML and epub
+# output is usually not wise. This defaults to 'epub', a theme designed to save
+# visual space.
+#epub_theme = 'epub'
+
+# The language of the text. It defaults to the language option
+# or 'en' if the language is not set.
+#epub_language = ''
+
+# The scheme of the identifier. Typical schemes are ISBN or URL.
+#epub_scheme = ''
+
+# The unique identifier of the text. This can be a ISBN number
+# or the project homepage.
+#epub_identifier = ''
+
+# A unique identification for the text.
+#epub_uid = ''
+
+# A tuple containing the cover image and cover page html template filenames.
+#epub_cover = ()
+
+# A sequence of (type, uri, title) tuples for the guide element of content.opf.
+#epub_guide = ()
+
+# HTML files that should be inserted before the pages created by sphinx.
+# The format is a list of tuples containing the path and title.
+#epub_pre_files = []
+
+# HTML files that should be inserted after the pages created by sphinx.
+# The format is a list of tuples containing the path and title.
+#epub_post_files = []
+
+# A list of files that should not be packed into the epub file.
+epub_exclude_files = ['search.html']
+
+# The depth of the table of contents in toc.ncx.
+#epub_tocdepth = 3
+
+# Allow duplicate toc entries.
+#epub_tocdup = True
+
+# Choose between 'default' and 'includehidden'.
+#epub_tocscope = 'default'
+
+# Fix unsupported image types using the Pillow.
+#epub_fix_images = False
+
+# Scale large images.
+#epub_max_image_width = 0
+
+# How to display URL addresses: 'footnote', 'no', or 'inline'.
+#epub_show_urls = 'inline'
+
+# If false, no index is generated.
+#epub_use_index = True
+
+#=======
+# rst2pdf
+#
+# Grouping the document tree into PDF files. List of tuples
+# (source start file, target name, title, author, options).
+#
+# See the Sphinx chapter of http://ralsina.me/static/manual.pdf
+#
+# FIXME: Do not add the index file here; the result will be too big. Adding
+# multiple PDF files here actually tries to get the cross-referencing right
+# *between* PDF files.
+pdf_documents = [
+    ('index', u'Kernel', u'Kernel', u'J. Random Bozo'),
+]
diff --git a/Documentation/index.rst b/Documentation/index.rst
new file mode 100644
index 000000000000..71a276f34c7f
--- /dev/null
+++ b/Documentation/index.rst
@@ -0,0 +1,23 @@
+.. The Linux Kernel documentation master file, created by
+   sphinx-quickstart on Fri Feb 12 13:51:46 2016.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+Welcome to The Linux Kernel's documentation!
+============================================
+
+Nothing for you to see here *yet*. Please move along.
+
+Contents:
+
+.. toctree::
+   :maxdepth: 2
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`
+
diff --git a/Makefile b/Makefile
index 0f70de63cfdb..8908a51b895a 100644
--- a/Makefile
+++ b/Makefile
@@ -1412,8 +1412,11 @@ $(help-board-dirs): help-%:
 
 # Documentation targets
 # ---------------------------------------------------------------------------
-%docs: scripts_basic FORCE
+DOC_TARGETS := xmldocs sgmldocs psdocs pdfdocs htmldocs mandocs installmandocs epubdocs cleandocs
+PHONY += $(DOC_TARGETS)
+$(DOC_TARGETS): scripts_basic FORCE
 	$(Q)$(MAKE) $(build)=scripts build_docproc build_check-lc_ctype
+	$(Q)$(MAKE) $(build)=Documentation -f $(srctree)/Documentation/Makefile.sphinx $@
 	$(Q)$(MAKE) $(build)=Documentation/DocBook $@
 
 else # KBUILD_EXTMOD
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413710 — [PATCH v2 28/38] kernel-doc/rst: remove fixme comment

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:50 +0200
Subject[PATCH v2 28/38] kernel-doc/rst: remove fixme comment
Message-ID<rGifv-8tB-1@gated-at.bofh.it>
In reply to#1413696
Yes, for our purposes the type should contain typedef.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 1 -
 1 file changed, 1 deletion(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 19cee0cd53a3..425a94be04f6 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1897,7 +1897,6 @@ sub output_typedef_rst(%) {
     my $oldprefix = $lineprefix;
     my $name = "typedef " . $args{'typedef'};
 
-    ### FIXME: should the name below contain "typedef" or not?
     print "\n\n.. c:type:: " . $name . "\n\n";
     $lineprefix = "   ";
     output_highlight_rst($args{'purpose'});
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413711 — [PATCH v2 24/38] kernel-doc/rst: change the output layout

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:50 +0200
Subject[PATCH v2 24/38] kernel-doc/rst: change the output layout
Message-ID<rGifw-8tB-3@gated-at.bofh.it>
In reply to#1413696
Move away from field lists, and simply use **strong emphasis** for
section headings on lines of their own. Do not use rst section headings,
because their nesting depth depends on the surrounding context, which
kernel-doc has no knowledge of. Also, they do not need to end up in any
table of contexts or indexes.

There are two related immediate benefits. Field lists are typically
rendered in two columns, while the new style uses the horizontal width
better. With no extra indent on the left, there's no need to be as fussy
about it. Field lists are more susceptible to indentation problems than
the new style.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 34 +++++++++++++++++-----------------
 1 file changed, 17 insertions(+), 17 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index e8ea295567a3..4f559de8b173 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1823,23 +1823,23 @@ sub output_function_rst(%) {
     output_highlight_rst($args{'purpose'});
     print "\n";
 
-    print ":Parameters:\n\n";
-    $lineprefix = "        ";
+    print "**Parameters**\n\n";
+    $lineprefix = "  ";
     foreach $parameter (@{$args{'parameterlist'}}) {
 	my $parameter_name = $parameter;
 	#$parameter_name =~ s/\[.*//;
 	$type = $args{'parametertypes'}{$parameter};
 
 	if ($type ne "") {
-	    print "      ``$type $parameter``\n";
+	    print "``$type $parameter``\n";
 	} else {
-	    print "      ``$parameter``\n";
+	    print "``$parameter``\n";
 	}
 	if (defined($args{'parameterdescs'}{$parameter_name}) &&
 	    $args{'parameterdescs'}{$parameter_name} ne $undescribed) {
 	    output_highlight_rst($args{'parameterdescs'}{$parameter_name});
 	} else {
-	    print "\n        _undescribed_\n";
+	    print "  _undescribed_\n";
 	}
 	print "\n";
     }
@@ -1852,10 +1852,10 @@ sub output_section_rst(%) {
     my %args = %{$_[0]};
     my $section;
     my $oldprefix = $lineprefix;
-    $lineprefix = "        ";
+    $lineprefix = "";
 
     foreach $section (@{$args{'sectionlist'}}) {
-	print ":$section:\n\n";
+	print "**$section**\n\n";
 	output_highlight_rst($args{'sections'}{$section});
 	print "\n";
     }
@@ -1875,14 +1875,14 @@ sub output_enum_rst(%) {
     output_highlight_rst($args{'purpose'});
     print "\n";
 
-    print "..\n\n:Constants:\n\n";
-    $lineprefix = "    ";
+    print "**Constants**\n\n";
+    $lineprefix = "  ";
     foreach $parameter (@{$args{'parameterlist'}}) {
-	print "  `$parameter`\n";
+	print "``$parameter``\n";
 	if ($args{'parameterdescs'}{$parameter} ne $undescribed) {
 	    output_highlight_rst($args{'parameterdescs'}{$parameter});
 	} else {
-	    print "    undescribed\n";
+	    print "  _undescribed_\n";
 	}
 	print "\n";
     }
@@ -1918,12 +1918,12 @@ sub output_struct_rst(%) {
     output_highlight_rst($args{'purpose'});
     print "\n";
 
-    print ":Definition:\n\n";
-    print " ::\n\n";
+    print "**Definition**\n\n";
+    print "::\n\n";
     print "  " . $args{'type'} . " " . $args{'struct'} . " {\n";
     foreach $parameter (@{$args{'parameterlist'}}) {
 	if ($parameter =~ /^#/) {
-	    print "    " . "$parameter\n";
+	    print "  " . "$parameter\n";
 	    next;
 	}
 
@@ -1944,8 +1944,8 @@ sub output_struct_rst(%) {
     }
     print "  };\n\n";
 
-    print ":Members:\n\n";
-    $lineprefix = "        ";
+    print "**Members**\n\n";
+    $lineprefix = "  ";
     foreach $parameter (@{$args{'parameterlist'}}) {
 	($parameter =~ /^#/) && next;
 
@@ -1954,7 +1954,7 @@ sub output_struct_rst(%) {
 
 	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
 	$type = $args{'parametertypes'}{$parameter};
-	print "      `$type $parameter`" . "\n";
+	print "``$type $parameter``\n";
 	output_highlight_rst($args{'parameterdescs'}{$parameter_name});
 	print "\n";
     }
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413712 — [PATCH v2 22/38] kernel-doc/rst: blank lines in output are not needed

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:50 +0200
Subject[PATCH v2 22/38] kernel-doc/rst: blank lines in output are not needed
Message-ID<rGifw-8tB-5@gated-at.bofh.it>
In reply to#1413696
Current approach leads to two blank lines, while one is enough.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index c154c3205df1..a89ff3ca366c 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1786,12 +1786,7 @@ sub output_highlight_rst {
     die $@ if $@;
 
     foreach $line (split "\n", $contents) {
-	if ($line eq "") {
-	    print $lineprefix, $blankline;
-	} else {
-	    print $lineprefix, $line;
-	}
-	print "\n";
+	print $lineprefix . $line . "\n";
     }
 }
 
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413713 — [PATCH v2 37/38] scripts/kernel-doc: Add option to inject line numbers

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:50 +0200
Subject[PATCH v2 37/38] scripts/kernel-doc: Add option to inject line numbers
Message-ID<rGifw-8tB-7@gated-at.bofh.it>
In reply to#1413696
From: Daniel Vetter <daniel.vetter@ffwll.ch>

Opt-in since this wreaks the rst output and must be removed
by consumers again. This is useful to adjust the linenumbers
for included kernel-doc snippets in shinx. With that sphinx
error message will be accurate when there's issues with the
rst-ness of the kernel-doc comments.

Especially when transitioning a new docbook .tmpl to .rst this
is extremely useful, since you can just use your editors compilation
quickfix list to accurately jump from error to error.

v2:
- Also make sure that we filter the LINENO for purpose/at declaration
  start so it only shows for selected blocks, not all of them (Jani).
  While at it make it a notch more accurate.
- Avoid undefined $lineno issues. I tried filtering these out at the
  callsite, but Jani spotted more when linting the entire kernel.
  Unamed unions and similar things aren't stored consistently and end
  up with an undefined line number (but also no kernel-doc text, just
  the parameter type). Simplify things and filter undefined line
  numbers in print_lineno() to catch them all.

v3: Fix LINENO 0 issue for kernel-doc comments without @param: lines
or any other special sections that directly jump to the description
after the "name - purpose" line. Only really possible for functions
without parameters. Noticed by Jani.

Cc: Jani Nikula <jani.nikula@intel.com>
Cc: linux-doc@vger.kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 41 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 4da6f952d18b..5192213c5005 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -74,6 +74,8 @@ Output selection (mutually exclusive):
 
 Output selection modifiers:
   -no-doc-sections	Do not output DOC: sections.
+  -enable-lineno        Enable output of #define LINENO lines. Only works with
+                        reStructuredText format.
 
 Other parameters:
   -v			Verbose output, more warnings and other information.
@@ -319,6 +321,7 @@ my $verbose = 0;
 my $output_mode = "man";
 my $output_preformatted = 0;
 my $no_doc_sections = 0;
+my $enable_lineno = 0;
 my @highlights = @highlights_man;
 my $blankline = $blankline_man;
 my $modulename = "Kernel API";
@@ -351,6 +354,7 @@ my $man_date = ('January', 'February', 'March', 'April', 'May', 'June',
 # CAVEAT EMPTOR!  Some of the others I localised may not want to be, which
 # could cause "use of undefined value" or other bugs.
 my ($function, %function_table, %parametertypes, $declaration_purpose);
+my $declaration_start_line;
 my ($type, $declaration_name, $return_type);
 my ($newsection, $newcontents, $prototype, $brcount, %source_map);
 
@@ -411,13 +415,16 @@ my $doc_inline_end = '^\s*\*/\s*$';
 my $export_symbol = '^\s*EXPORT_SYMBOL(_GPL)?\s*\(\s*(\w+)\s*\)\s*;';
 
 my %parameterdescs;
+my %parameterdesc_start_lines;
 my @parameterlist;
 my %sections;
 my @sectionlist;
+my %section_start_lines;
 my $sectcheck;
 my $struct_actual;
 
 my $contents = "";
+my $new_start_line = 0;
 
 # the canonical section names. see also $doc_sect above.
 my $section_default = "Description";	# default section
@@ -486,6 +493,8 @@ while ($ARGV[0] =~ m/^-(.*)/) {
 	usage();
     } elsif ($cmd eq '-no-doc-sections') {
 	    $no_doc_sections = 1;
+    } elsif ($cmd eq '-enable-lineno') {
+	    $enable_lineno = 1;
     } elsif ($cmd eq '-show-not-found') {
 	$show_not_found = 1;
     }
@@ -503,6 +512,13 @@ sub get_kernel_version() {
     return $version;
 }
 
+#
+sub print_lineno {
+    my $lineno = shift;
+    if ($enable_lineno && defined($lineno)) {
+        print "#define LINENO " . $lineno . "\n";
+    }
+}
 ##
 # dumps section contents to arrays/hashes intended for that purpose.
 #
@@ -516,11 +532,15 @@ sub dump_section {
 	$name = $1;
 	$parameterdescs{$name} = $contents;
 	$sectcheck = $sectcheck . $name . " ";
+        $parameterdesc_start_lines{$name} = $new_start_line;
+        $new_start_line = 0;
     } elsif ($name eq "@\.\.\.") {
 #	print STDERR "parameter def '...' = '$contents'\n";
 	$name = "...";
 	$parameterdescs{$name} = $contents;
 	$sectcheck = $sectcheck . $name . " ";
+        $parameterdesc_start_lines{$name} = $new_start_line;
+        $new_start_line = 0;
     } else {
 #	print STDERR "other section '$name' = '$contents'\n";
 	if (defined($sections{$name}) && ($sections{$name} ne "")) {
@@ -530,6 +550,8 @@ sub dump_section {
 	} else {
 	    $sections{$name} = $contents;
 	    push @sectionlist, $name;
+            $section_start_lines{$name} = $new_start_line;
+            $new_start_line = 0;
 	}
     }
 }
@@ -1775,6 +1797,7 @@ sub output_blockhead_rst(%) {
 	if ($output_selection != OUTPUT_INCLUDE) {
 	    print "**$section**\n\n";
 	}
+        print_lineno($section_start_lines{$section});
 	output_highlight_rst($args{'sections'}{$section});
 	print "\n";
     }
@@ -1824,6 +1847,7 @@ sub output_function_rst(%) {
 	}
     }
     print ")\n\n";
+    print_lineno($declaration_start_line);
     $lineprefix = "   ";
     output_highlight_rst($args{'purpose'});
     print "\n";
@@ -1840,6 +1864,9 @@ sub output_function_rst(%) {
 	} else {
 	    print "``$parameter``\n";
 	}
+
+        print_lineno($parameterdesc_start_lines{$parameter_name});
+
 	if (defined($args{'parameterdescs'}{$parameter_name}) &&
 	    $args{'parameterdescs'}{$parameter_name} ne $undescribed) {
 	    output_highlight_rst($args{'parameterdescs'}{$parameter_name});
@@ -1861,6 +1888,7 @@ sub output_section_rst(%) {
 
     foreach $section (@{$args{'sectionlist'}}) {
 	print "**$section**\n\n";
+        print_lineno($section_start_lines{$section});
 	output_highlight_rst($args{'sections'}{$section});
 	print "\n";
     }
@@ -1876,6 +1904,7 @@ sub output_enum_rst(%) {
     my $name = "enum " . $args{'enum'};
 
     print "\n\n.. c:type:: " . $name . "\n\n";
+    print_lineno($declaration_start_line);
     $lineprefix = "   ";
     output_highlight_rst($args{'purpose'});
     print "\n";
@@ -1903,6 +1932,7 @@ sub output_typedef_rst(%) {
     my $name = "typedef " . $args{'typedef'};
 
     print "\n\n.. c:type:: " . $name . "\n\n";
+    print_lineno($declaration_start_line);
     $lineprefix = "   ";
     output_highlight_rst($args{'purpose'});
     print "\n";
@@ -1918,6 +1948,7 @@ sub output_struct_rst(%) {
     my $name = $args{'type'} . " " . $args{'struct'};
 
     print "\n\n.. c:type:: " . $name . "\n\n";
+    print_lineno($declaration_start_line);
     $lineprefix = "   ";
     output_highlight_rst($args{'purpose'});
     print "\n";
@@ -1958,6 +1989,7 @@ sub output_struct_rst(%) {
 
 	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
 	$type = $args{'parametertypes'}{$parameter};
+        print_lineno($parameterdesc_start_lines{$parameter_name});
 	print "``$type $parameter``\n";
 	output_highlight_rst($args{'parameterdescs'}{$parameter_name});
 	print "\n";
@@ -2745,11 +2777,14 @@ sub process_file($) {
 	    if (/$doc_start/o) {
 		$state = STATE_NAME;	# next line is always the function name
 		$in_doc_sect = 0;
+		$declaration_start_line = $. + 1;
 	    }
 	} elsif ($state == STATE_NAME) {# this line is the function name (always)
 	    if (/$doc_block/o) {
 		$state = STATE_DOCBLOCK;
 		$contents = "";
+                $new_start_line = $. + 1;
+
 		if ( $1 eq "" ) {
 			$section = $section_intro;
 		} else {
@@ -2763,8 +2798,11 @@ sub process_file($) {
 		}
 
 		$state = STATE_FIELD;
+		# if there's no @param blocks need to set up default section
+		# here
 		$contents = "";
 		$section = $section_default;
+		$new_start_line = $. + 1;
 		if (/-(.*)/) {
 		    # strip leading/trailing/multiple spaces
 		    $descr= $1;
@@ -2833,6 +2871,7 @@ sub process_file($) {
 		$in_doc_sect = 1;
 		$in_purpose = 0;
 		$contents = $newcontents;
+                $new_start_line = $.;
 		while ((substr($contents, 0, 1) eq " ") ||
 		       substr($contents, 0, 1) eq "\t") {
 		    $contents = substr($contents, 1);
@@ -2866,6 +2905,7 @@ sub process_file($) {
 			dump_section($file, $section, xml_escape($contents));
 			$section = $section_default;
 			$contents = "";
+                        $new_start_line = $.;
 		    } else {
 			$contents .= "\n";
 		    }
@@ -2900,6 +2940,7 @@ sub process_file($) {
 	    if ($inline_doc_state == STATE_INLINE_NAME && /$doc_inline_sect/o) {
 		$section = $1;
 		$contents = $2;
+                $new_start_line = $.;
 		if ($contents ne "") {
 		    while ((substr($contents, 0, 1) eq " ") ||
 		           substr($contents, 0, 1) eq "\t") {
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413714 — [PATCH v2 07/38] Documentation/sphinx: set version and release properly

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:50 +0200
Subject[PATCH v2 07/38] Documentation/sphinx: set version and release properly
Message-ID<rGifw-8tB-9@gated-at.bofh.it>
In reply to#1413696
Read the version and release from the top level Makefile (for use when
Sphinx is invoked directly, by e.g. Read the Docs), but override them
via Sphinx command line arguments in a normal documentation build.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 Documentation/Makefile.sphinx |  2 +-
 Documentation/conf.py         | 29 +++++++++++++++++++++++++----
 2 files changed, 26 insertions(+), 5 deletions(-)

diff --git a/Documentation/Makefile.sphinx b/Documentation/Makefile.sphinx
index 8a662be9bd87..addf32309bc3 100644
--- a/Documentation/Makefile.sphinx
+++ b/Documentation/Makefile.sphinx
@@ -27,7 +27,7 @@ PAPEROPT_a4     = -D latex_paper_size=a4
 PAPEROPT_letter = -D latex_paper_size=letter
 KERNELDOC       = $(srctree)/scripts/kernel-doc
 KERNELDOC_CONF  = -D kerneldoc_srctree=$(srctree) -D kerneldoc_bin=$(KERNELDOC)
-ALLSPHINXOPTS   = -d $(BUILDDIR)/.doctrees $(KERNELDOC_CONF) $(PAPEROPT_$(PAPER)) -c $(srctree)/$(src) $(SPHINXOPTS) $(srctree)/$(src)
+ALLSPHINXOPTS   = -D version=$(KERNELVERSION) -D release=$(KERNELRELEASE) -d $(BUILDDIR)/.doctrees $(KERNELDOC_CONF) $(PAPEROPT_$(PAPER)) -c $(srctree)/$(src) $(SPHINXOPTS) $(srctree)/$(src)
 # the i18n builder cannot share the environment and doctrees with the others
 I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
 
diff --git a/Documentation/conf.py b/Documentation/conf.py
index 8b72577f38f5..6cc41a0555a3 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -60,10 +60,31 @@ author = 'The kernel development community'
 # |version| and |release|, also used in various other places throughout the
 # built documents.
 #
-# The short X.Y version.
-version = '4.5'
-# The full version, including alpha/beta/rc tags.
-release = '4.5'
+# In a normal build, version and release are are set to KERNELVERSION and
+# KERNELRELEASE, respectively, from the Makefile via Sphinx command line
+# arguments.
+#
+# The following code tries to extract the information by reading the Makefile,
+# when Sphinx is run directly (e.g. by Read the Docs).
+try:
+    makefile_version = None
+    makefile_patchlevel = None
+    for line in open('../Makefile'):
+        key, val = [x.strip() for x in line.split('=', 2)]
+        if key == 'VERSION':
+            makefile_version = val
+        elif key == 'PATCHLEVEL':
+            makefile_patchlevel = val
+        if makefile_version and makefile_patchlevel:
+            break
+except:
+    pass
+finally:
+    if makefile_version and makefile_patchlevel:
+        version = release = makefile_version + '.' + makefile_patchlevel
+    else:
+        sys.stderr.write('Warning: Could not extract kernel version\n')
+        version = release = "unknown version"
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413715 — [PATCH v2 32/38] Documentation/sphinx: fix kernel-doc extension on python3

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:50 +0200
Subject[PATCH v2 32/38] Documentation/sphinx: fix kernel-doc extension on python3
Message-ID<rGifw-8tB-11@gated-at.bofh.it>
In reply to#1413696
Reconcile differences between python2 and python3 on dealing with
stdout, stderr from Popen. This fixes "name 'unicode' is not defined"
errors on python3. We'll need to try to keep the extension working on
both python-sphinx and python3-sphinx so we don't need two copies.

Reported-and-tested-by: Marius Vlad <marius.c.vlad@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 Documentation/sphinx/kernel-doc.py | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/Documentation/sphinx/kernel-doc.py b/Documentation/sphinx/kernel-doc.py
index 87a1332fe934..a6e170872c8a 100644
--- a/Documentation/sphinx/kernel-doc.py
+++ b/Documentation/sphinx/kernel-doc.py
@@ -23,6 +23,9 @@
 #
 # Authors:
 #    Jani Nikula <jani.nikula@intel.com>
+#
+# Please make sure this works on both python2 and python3.
+#
 
 import os
 import subprocess
@@ -75,8 +78,10 @@ class KernelDocDirective(Directive):
             p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
             out, err = p.communicate()
 
-            # assume the kernel sources are utf-8
-            out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8')
+            # python2 needs conversion to unicode.
+            # python3 with universal_newlines=True returns strings.
+            if sys.version_info.major < 3:
+                out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8')
 
             if p.returncode != 0:
                 sys.stderr.write(err)
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413716 — [PATCH v2 06/38] Documentation/sphinx: configure the kernel-doc extension

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:50 +0200
Subject[PATCH v2 06/38] Documentation/sphinx: configure the kernel-doc extension
Message-ID<rGifw-8tB-13@gated-at.bofh.it>
In reply to#1413696
Tell Sphinx where to find the extension, and pass on the kernel src tree
and kernel-doc paths to the extension.

With this, any .rst files under Documentation may contain the kernel-doc
rst directive to include kernel-doc documentation from any source file.

While building, it may be handy to pass kernel-doc extension
configuration on the command line. For example, 'make SPHINXOPTS="-D
kerneldoc_verbosity=0" htmldocs' silences all stderr output from
kernel-doc when the kernel-doc exit code is 0. (The stderr will be
logged unconditionally when the exit code is non-zero.)

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 Documentation/Makefile.sphinx |  4 +++-
 Documentation/conf.py         | 10 ++++++++--
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/Documentation/Makefile.sphinx b/Documentation/Makefile.sphinx
index 25a7970bfe5c..8a662be9bd87 100644
--- a/Documentation/Makefile.sphinx
+++ b/Documentation/Makefile.sphinx
@@ -25,7 +25,9 @@ HAVE_RST2PDF := $(shell if python -c "import rst2pdf" >/dev/null 2>&1; then echo
 # Internal variables.
 PAPEROPT_a4     = -D latex_paper_size=a4
 PAPEROPT_letter = -D latex_paper_size=letter
-ALLSPHINXOPTS   = -d $(BUILDDIR)/.doctrees $(PAPEROPT_$(PAPER)) -c $(srctree)/$(src) $(SPHINXOPTS) $(srctree)/$(src)
+KERNELDOC       = $(srctree)/scripts/kernel-doc
+KERNELDOC_CONF  = -D kerneldoc_srctree=$(srctree) -D kerneldoc_bin=$(KERNELDOC)
+ALLSPHINXOPTS   = -d $(BUILDDIR)/.doctrees $(KERNELDOC_CONF) $(PAPEROPT_$(PAPER)) -c $(srctree)/$(src) $(SPHINXOPTS) $(srctree)/$(src)
 # the i18n builder cannot share the environment and doctrees with the others
 I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
 
diff --git a/Documentation/conf.py b/Documentation/conf.py
index 0f0e20757bb1..8b72577f38f5 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -18,7 +18,7 @@ import os
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
-#sys.path.insert(0, os.path.abspath('.'))
+sys.path.insert(0, os.path.abspath('sphinx'))
 
 # -- General configuration ------------------------------------------------
 
@@ -28,7 +28,7 @@ import os
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = []
+extensions = ['kernel-doc']
 
 # Gracefully handle missing rst2pdf.
 try:
@@ -385,3 +385,9 @@ epub_exclude_files = ['search.html']
 pdf_documents = [
     ('index', u'Kernel', u'Kernel', u'J. Random Bozo'),
 ]
+
+# kernel-doc extension configuration for running Sphinx directly (e.g. by Read
+# the Docs). In a normal build, these are supplied from the Makefile via command
+# line arguments.
+kerneldoc_bin = '../scripts/kernel-doc'
+kerneldoc_srctree = '..'
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413717 — [PATCH v2 20/38] kernel-doc: do not regard $, %, or & prefixes as special in section names

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:50 +0200
Subject[PATCH v2 20/38] kernel-doc: do not regard $, %, or & prefixes as special in section names
Message-ID<rGifw-8tB-15@gated-at.bofh.it>
In reply to#1413696
The use of these is confusing in the script, and per this grep, they're
not used anywhere anyway:

$ git grep " \* [%$&][a-zA-Z0-9_]*:" -- *.[ch] | grep -v "\$\(Id\|Revision\|Date\)"

While at it, throw out the constants array, nothing is ever put there
again.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 14 ++------------
 1 file changed, 2 insertions(+), 12 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 76bad55c031e..f795660dfc7b 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -396,14 +396,12 @@ my $inline_doc_state;
 # 'function', 'struct', 'union', 'enum', 'typedef'
 my $decl_type;
 
-my $doc_special = "\@\%\$\&";
-
 my $doc_start = '^/\*\*\s*$'; # Allow whitespace at end of comment start.
 my $doc_end = '\*/';
 my $doc_com = '\s*\*\s*';
 my $doc_com_body = '\s*\* ?';
 my $doc_decl = $doc_com . '(\w+)';
-my $doc_sect = $doc_com . '([' . $doc_special . ']?[\w\s]+):(.*)';
+my $doc_sect = $doc_com . '(\@?[\w\s]+):(.*)';
 my $doc_content = $doc_com_body . '(.*)';
 my $doc_block = $doc_com . 'DOC:\s*(.*)?';
 my $doc_inline_start = '^\s*/\*\*\s*$';
@@ -411,7 +409,6 @@ my $doc_inline_sect = '\s*\*\s*(@[\w\s]+):(.*)';
 my $doc_inline_end = '^\s*\*/\s*$';
 my $export_symbol = '^\s*EXPORT_SYMBOL(_GPL)?\s*\(\s*(\w+)\s*\)\s*;';
 
-my %constants;
 my %parameterdescs;
 my @parameterlist;
 my %sections;
@@ -511,11 +508,7 @@ sub dump_section {
     my $name = shift;
     my $contents = join "\n", @_;
 
-    if ($name =~ m/$type_constant/) {
-	$name = $1;
-#	print STDERR "constant section '$1' = '$contents'\n";
-	$constants{$name} = $contents;
-    } elsif ($name =~ m/$type_param/) {
+    if ($name =~ m/$type_param/) {
 #	print STDERR "parameter def '$1' = '$contents'\n";
 	$name = $1;
 	$parameterdescs{$name} = $contents;
@@ -2528,7 +2521,6 @@ sub dump_function($$) {
 
 sub reset_state {
     $function = "";
-    %constants = ();
     %parameterdescs = ();
     %parametertypes = ();
     @parameterlist = ();
@@ -2924,7 +2916,6 @@ sub process_file($) {
 			dump_doc_section($file, $section, xml_escape($contents));
 			$contents = "";
 			$function = "";
-			%constants = ();
 			%parameterdescs = ();
 			%parametertypes = ();
 			@parameterlist = ();
@@ -2942,7 +2933,6 @@ sub process_file($) {
 			dump_doc_section($file, $section, xml_escape($contents));
 			$contents = "";
 			$function = "";
-			%constants = ();
 			%parameterdescs = ();
 			%parametertypes = ();
 			@parameterlist = ();
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


#1413718 — [PATCH v2 19/38] kernel-doc/rst: highlight function/struct/enum purpose lines too

FromJani Nikula <jani.nikula@intel.com>
Date2016-06-04 13:50 +0200
Subject[PATCH v2 19/38] kernel-doc/rst: highlight function/struct/enum purpose lines too
Message-ID<rGifw-8tB-17@gated-at.bofh.it>
In reply to#1413696
Let the user use @foo, &bar, %baz, etc. in the first kernel-doc purpose
line too.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 37 +++++++++++++++++++++++++------------
 1 file changed, 25 insertions(+), 12 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 8f9eac509377..76bad55c031e 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1805,6 +1805,7 @@ sub output_highlight_rst {
 sub output_function_rst(%) {
     my %args = %{$_[0]};
     my ($parameter, $section);
+    my $oldprefix = $lineprefix;
     my $start;
 
     print ".. c:function:: ";
@@ -1829,9 +1830,13 @@ sub output_function_rst(%) {
 	    print $type . " " . $parameter;
 	}
     }
-    print ")\n\n    " . $args{'purpose'} . "\n\n";
+    print ")\n\n";
+    $lineprefix = "   ";
+    output_highlight_rst($args{'purpose'});
+    print "\n";
 
     print ":Parameters:\n\n";
+    $lineprefix = "        ";
     foreach $parameter (@{$args{'parameterlist'}}) {
 	my $parameter_name = $parameter;
 	#$parameter_name =~ s/\[.*//;
@@ -1844,15 +1849,14 @@ sub output_function_rst(%) {
 	}
 	if (defined($args{'parameterdescs'}{$parameter_name}) &&
 	    $args{'parameterdescs'}{$parameter_name} ne $undescribed) {
-	    my $oldprefix = $lineprefix;
-	    $lineprefix = "        ";
 	    output_highlight_rst($args{'parameterdescs'}{$parameter_name});
-	    $lineprefix = $oldprefix;
 	} else {
 	    print "\n        _undescribed_\n";
 	}
 	print "\n";
     }
+
+    $lineprefix = $oldprefix;
     output_section_rst(@_);
 }
 
@@ -1874,14 +1878,16 @@ sub output_section_rst(%) {
 sub output_enum_rst(%) {
     my %args = %{$_[0]};
     my ($parameter);
+    my $oldprefix = $lineprefix;
     my $count;
     my $name = "enum " . $args{'enum'};
 
     print "\n\n.. c:type:: " . $name . "\n\n";
-    print "    " . $args{'purpose'} . "\n\n";
+    $lineprefix = "   ";
+    output_highlight_rst($args{'purpose'});
+    print "\n";
 
     print "..\n\n:Constants:\n\n";
-    my $oldprefix = $lineprefix;
     $lineprefix = "    ";
     foreach $parameter (@{$args{'parameterlist'}}) {
 	print "  `$parameter`\n";
@@ -1892,6 +1898,7 @@ sub output_enum_rst(%) {
 	}
 	print "\n";
     }
+
     $lineprefix = $oldprefix;
     output_section_rst(@_);
 }
@@ -1899,23 +1906,29 @@ sub output_enum_rst(%) {
 sub output_typedef_rst(%) {
     my %args = %{$_[0]};
     my ($parameter);
-    my $count;
+    my $oldprefix = $lineprefix;
     my $name = "typedef " . $args{'typedef'};
 
     ### FIXME: should the name below contain "typedef" or not?
     print "\n\n.. c:type:: " . $name . "\n\n";
-    print "    " . $args{'purpose'} . "\n\n";
+    $lineprefix = "   ";
+    output_highlight_rst($args{'purpose'});
+    print "\n";
 
+    $lineprefix = $oldprefix;
     output_section_rst(@_);
 }
 
 sub output_struct_rst(%) {
     my %args = %{$_[0]};
     my ($parameter);
+    my $oldprefix = $lineprefix;
     my $name = $args{'type'} . " " . $args{'struct'};
 
     print "\n\n.. c:type:: " . $name . "\n\n";
-    print "    " . $args{'purpose'} . "\n\n";
+    $lineprefix = "   ";
+    output_highlight_rst($args{'purpose'});
+    print "\n";
 
     print ":Definition:\n\n";
     print " ::\n\n";
@@ -1944,6 +1957,7 @@ sub output_struct_rst(%) {
     print "  };\n\n";
 
     print ":Members:\n\n";
+    $lineprefix = "        ";
     foreach $parameter (@{$args{'parameterlist'}}) {
 	($parameter =~ /^#/) && next;
 
@@ -1953,13 +1967,12 @@ sub output_struct_rst(%) {
 	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
 	$type = $args{'parametertypes'}{$parameter};
 	print "      `$type $parameter`" . "\n";
-	my $oldprefix = $lineprefix;
-	$lineprefix = "        ";
 	output_highlight_rst($args{'parameterdescs'}{$parameter_name});
-	$lineprefix = $oldprefix;
 	print "\n";
     }
     print "\n";
+
+    $lineprefix = $oldprefix;
     output_section_rst(@_);
 }
 
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]


Page 1 of 3  [1] 2 3  Next page →

Back to top | Article view | linux.kernel


csiph-web