Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1419460 > unrolled thread

[docs-next PATCH 0/9] Documentation/sphinx follow-up

Back to article view | Back to linux.kernel

Contents

  [docs-next PATCH 0/9] Documentation/sphinx follow-up Jani Nikula <jani.nikula@intel.com> - 2016-06-10 16:20 +0200
    [docs-next PATCH 3/9] kernel-doc: add missing semi-colons in option parsing Jani Nikula <jani.nikula@intel.com> - 2016-06-10 16:20 +0200
    [docs-next PATCH 4/9] kernel-doc: abstract filename mapping Jani Nikula <jani.nikula@intel.com> - 2016-06-10 16:20 +0200
    [docs-next PATCH 2/9] kernel-doc: do not warn about duplicate default section names Jani Nikula <jani.nikula@intel.com> - 2016-06-10 16:20 +0200
    [docs-next PATCH 7/9] Documentation/sphinx: remove unnecessary temporary variable Jani Nikula <jani.nikula@intel.com> - 2016-06-10 16:20 +0200
    [docs-next PATCH 1/9] kernel-doc: remove old debug cruft from dump_section() Jani Nikula <jani.nikula@intel.com> - 2016-06-10 16:20 +0200
    [docs-next PATCH 9/9] Documentation/sphinx: add support for specifying extra export files Jani Nikula <jani.nikula@intel.com> - 2016-06-10 16:20 +0200
    [docs-next PATCH 8/9] Documentation/sphinx: use a more sensible string split in kernel-doc extension Jani Nikula <jani.nikula@intel.com> - 2016-06-10 16:20 +0200

#1419460 — [docs-next PATCH 0/9] Documentation/sphinx follow-up

Hi Jon -

Thanks for merging the main Sphinx series! I greet you with another set
of patches on top. ;)

The main things here are reducing the noise on duplicate sections and
adding better support for extracting exported symbols when
EXPORT_SYMBOLs and kernel-doc are in separate files. And then there are
some drive-by cleanups and fixes on top.

I'm still working on rewriting Documentation/kernel-doc-nano-HOWTO.txt
to reflect all the changes, but didn't want to let these wait for that.

BR,
Jani.


PS. One wrinkle I spotted is that in the kernel-doc directive,
:functions: and :internal: also match structs and other types. This is
just kernel-doc the script shining its awesomeness through to the upper
layer; I just totally missed this earlier. I added :doc: to make the
distinction for documents, even though internally that uses kernel-doc
-function parameter as well.

Should we rename :functions: to, mmh, maybe :symbols: (for want of a
better word) while we still can, or leave that in as a historical
curiosity? I'm not sure there's much value in adding a separate :types:
(or something) for non-functions either? And should :internal: keep
returning non-functions as well? Or make that return just functions and
have another argument to get the types? Thoughts?


Jani Nikula (9):
  kernel-doc: remove old debug cruft from dump_section()
  kernel-doc: do not warn about duplicate default section names
  kernel-doc: add missing semi-colons in option parsing
  kernel-doc: abstract filename mapping
  kernel-doc: add support for specifying extra files for EXPORT_SYMBOLs
  kernel-doc: unify all EXPORT_SYMBOL scanning to one place
  Documentation/sphinx: remove unnecessary temporary variable
  Documentation/sphinx: use a more sensible string split in kernel-doc
    extension
  Documentation/sphinx: add support for specifying extra export files

 Documentation/sphinx/kernel-doc.py | 18 +++++---
 scripts/kernel-doc                 | 88 +++++++++++++++++++++++++++-----------
 2 files changed, 75 insertions(+), 31 deletions(-)

-- 
2.1.4

[toc] | [next] | [standalone]

#1419462 — [docs-next PATCH 3/9] kernel-doc: add missing semi-colons in option parsing

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 710615f3a4ff..a6f82c812c15 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -484,10 +484,10 @@ while ($ARGV[0] =~ m/^-(.*)/) {
 	$function_table{$function} = 1;
     } elsif ($cmd eq "-export") { # only exported symbols
 	$output_selection = OUTPUT_EXPORTED;
-	%function_table = ()
+	%function_table = ();
     } elsif ($cmd eq "-internal") { # only non-exported symbols
 	$output_selection = OUTPUT_INTERNAL;
-	%function_table = ()
+	%function_table = ();
     } elsif ($cmd eq "-v") {
 	$verbose = 1;
     } elsif (($cmd eq "-h") || ($cmd eq "--help")) {
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]

#1419464 — [docs-next PATCH 4/9] kernel-doc: abstract filename mapping

Reduce duplication in follow-up work. No functional changes.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 27 ++++++++++++++++++---------
 1 file changed, 18 insertions(+), 9 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index a6f82c812c15..516d95fcefb7 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -2730,26 +2730,35 @@ sub local_unescape($) {
 	return $text;
 }
 
-sub process_file($) {
+sub map_filename($) {
     my $file;
-    my $identifier;
-    my $func;
-    my $descr;
-    my $in_purpose = 0;
-    my $initial_section_counter = $section_counter;
     my ($orig_file) = @_;
-    my $leading_space;
 
     if (defined($ENV{'SRCTREE'})) {
 	$file = "$ENV{'SRCTREE'}" . "/" . $orig_file;
-    }
-    else {
+    } else {
 	$file = $orig_file;
     }
+
     if (defined($source_map{$file})) {
 	$file = $source_map{$file};
     }
 
+    return $file;
+}
+
+sub process_file($) {
+    my $file;
+    my $identifier;
+    my $func;
+    my $descr;
+    my $in_purpose = 0;
+    my $initial_section_counter = $section_counter;
+    my ($orig_file) = @_;
+    my $leading_space;
+
+    $file = map_filename($orig_file);
+
     if (!open(IN,"<$file")) {
 	print STDERR "Error: Cannot open file $file\n";
 	++$errors;
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]

#1419465 — [docs-next PATCH 2/9] kernel-doc: do not warn about duplicate default section names

Since

commit 32217761ee9db0215350dfe1ca4e66f312fb8c54
Author: Jani Nikula <jani.nikula@intel.com>
Date:   Sun May 29 09:40:44 2016 +0300

    kernel-doc: concatenate contents of colliding sections

we started getting (more) errors on duplicate section names, especially
on the default section name "Description":

include/net/mac80211.h:3174: warning: duplicate section name 'Description'

This is usually caused by a slightly unorthodox placement of parameter
descriptions, like in the above case, and kernel-doc resetting back to
the default section more than once within a kernel-doc comment.

Ignore warnings on the duplicate section name automatically assigned by
kernel-doc, and only consider explicitly user assigned duplicate section
names an issue.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index ac18eb5ed776..710615f3a4ff 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -542,8 +542,11 @@ sub dump_section {
         $new_start_line = 0;
     } else {
 	if (defined($sections{$name}) && ($sections{$name} ne "")) {
-	    print STDERR "${file}:$.: warning: duplicate section name '$name'\n";
-	    ++$warnings;
+	    # Only warn on user specified duplicate section names.
+	    if ($name ne $section_default) {
+		print STDERR "${file}:$.: warning: duplicate section name '$name'\n";
+		++$warnings;
+	    }
 	    $sections{$name} .= $contents;
 	} else {
 	    $sections{$name} = $contents;
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]

#1419467 — [docs-next PATCH 7/9] Documentation/sphinx: remove unnecessary temporary variable

Leftover cruft. No functional changes.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 Documentation/sphinx/kernel-doc.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/Documentation/sphinx/kernel-doc.py b/Documentation/sphinx/kernel-doc.py
index 4adfb0e91ecc..2856376cc62c 100644
--- a/Documentation/sphinx/kernel-doc.py
+++ b/Documentation/sphinx/kernel-doc.py
@@ -59,7 +59,6 @@ class KernelDocDirective(Directive):
         env.note_dependency(os.path.abspath(filename))
 
         tab_width = self.options.get('tab-width', self.state.document.settings.tab_width)
-        source = filename
 
         # FIXME: make this nicer and more robust against errors
         if 'export' in self.options:
@@ -105,7 +104,7 @@ class KernelDocDirective(Directive):
                     lineoffset = int(match.group(1)) - 1
                     # we must eat our comments since the upset the markup
                 else:
-                    result.append(line, source, lineoffset)
+                    result.append(line, filename, lineoffset)
                     lineoffset += 1
 
             node = nodes.section()
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]

#1419469 — [docs-next PATCH 1/9] kernel-doc: remove old debug cruft from dump_section()

No functional changes.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 27757c21551a..ac18eb5ed776 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -529,21 +529,18 @@ sub dump_section {
     my $contents = join "\n", @_;
 
     if ($name =~ m/$type_param/) {
-#	print STDERR "parameter def '$1' = '$contents'\n";
 	$name = $1;
 	$parameterdescs{$name} = $contents;
 	$sectcheck = $sectcheck . $name . " ";
         $parameterdesc_start_lines{$name} = $new_start_line;
         $new_start_line = 0;
     } elsif ($name eq "@\.\.\.") {
-#	print STDERR "parameter def '...' = '$contents'\n";
 	$name = "...";
 	$parameterdescs{$name} = $contents;
 	$sectcheck = $sectcheck . $name . " ";
         $parameterdesc_start_lines{$name} = $new_start_line;
         $new_start_line = 0;
     } else {
-#	print STDERR "other section '$name' = '$contents'\n";
 	if (defined($sections{$name}) && ($sections{$name} ne "")) {
 	    print STDERR "${file}:$.: warning: duplicate section name '$name'\n";
 	    ++$warnings;
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]

#1419470 — [docs-next PATCH 9/9] Documentation/sphinx: add support for specifying extra export files

Let the user specify file patterns where to look for the EXPORT_SYMBOLs
in addition to the file with kernel-doc comments. This is directly based
on the -export-file FILE option added to kernel-doc in "kernel-doc: add
support for specifying extra files for EXPORT_SYMBOLs", but we extend
that with globbing patterns in the Sphinx extension.

The file patterns are added as options to the :export: and :internal:
arguments of the kernel-doc directive. For example, to extract the
documentation of exported functions from include/net/mac80211.h:

.. kernel-doc:: include/net/mac80211.h
   :export: net/mac80211/*.c

Without the file pattern, no exported functions would be found, as the
EXPORT_SYMBOLs are placed in the various source files under
net/mac80211.

The matched files are also added as dependencies on the document in
Sphinx, as they may affect the output. This is one of the reasons to do
the globbing in the Sphinx extension instead of in scripts/kernel-doc.

The file pattern remains optional, and is not needed if the kernel-doc
comments and EXPORT_SYMBOLs are placed in the source file passed in as
the main argument to the kernel-doc directive. This is the most common
case across the kernel source tree.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 Documentation/sphinx/kernel-doc.py | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/Documentation/sphinx/kernel-doc.py b/Documentation/sphinx/kernel-doc.py
index d6a76f9a0ba5..dedb24e3bb7a 100644
--- a/Documentation/sphinx/kernel-doc.py
+++ b/Documentation/sphinx/kernel-doc.py
@@ -31,6 +31,7 @@ import os
 import subprocess
 import sys
 import re
+import glob
 
 from docutils import nodes, statemachine
 from docutils.statemachine import ViewList
@@ -44,8 +45,8 @@ class KernelDocDirective(Directive):
     option_spec = {
         'doc': directives.unchanged_required,
         'functions': directives.unchanged_required,
-        'export': directives.flag,
-        'internal': directives.flag,
+        'export': directives.unchanged,
+        'internal': directives.unchanged,
     }
     has_content = False
 
@@ -54,6 +55,7 @@ class KernelDocDirective(Directive):
         cmd = [env.config.kerneldoc_bin, '-rst', '-enable-lineno']
 
         filename = env.config.kerneldoc_srctree + '/' + self.arguments[0]
+        export_file_patterns = []
 
         # Tell sphinx of the dependency
         env.note_dependency(os.path.abspath(filename))
@@ -63,14 +65,21 @@ class KernelDocDirective(Directive):
         # FIXME: make this nicer and more robust against errors
         if 'export' in self.options:
             cmd += ['-export']
+            export_file_patterns = str(self.options.get('export')).split()
         elif 'internal' in self.options:
             cmd += ['-internal']
+            export_file_patterns = str(self.options.get('internal')).split()
         elif 'doc' in self.options:
             cmd += ['-function', str(self.options.get('doc'))]
         elif 'functions' in self.options:
             for f in str(self.options.get('functions')).split():
                 cmd += ['-function', f]
 
+        for pattern in export_file_patterns:
+            for f in glob.glob(env.config.kerneldoc_srctree + '/' + pattern):
+                env.note_dependency(os.path.abspath(f))
+                cmd += ['-export-file', f]
+
         cmd += [filename]
 
         try:
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]

#1419471 — [docs-next PATCH 8/9] Documentation/sphinx: use a more sensible string split in kernel-doc extension

Using the default str.split doesn't return empty strings like the
current version does.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 Documentation/sphinx/kernel-doc.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/sphinx/kernel-doc.py b/Documentation/sphinx/kernel-doc.py
index 2856376cc62c..d6a76f9a0ba5 100644
--- a/Documentation/sphinx/kernel-doc.py
+++ b/Documentation/sphinx/kernel-doc.py
@@ -68,7 +68,7 @@ class KernelDocDirective(Directive):
         elif 'doc' in self.options:
             cmd += ['-function', str(self.options.get('doc'))]
         elif 'functions' in self.options:
-            for f in str(self.options.get('functions')).split(' '):
+            for f in str(self.options.get('functions')).split():
                 cmd += ['-function', f]
 
         cmd += [filename]
-- 
2.1.4

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web