Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1330781 > unrolled thread

Re: [RFC] A first shot at asciidoc-based formatted docs

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [RFC] A first shot at asciidoc-based formatted docs Jonathan Corbet <corbet@lwn.net> - 2016-02-10 01:10 +0100
    Re: [RFC] A first shot at asciidoc-based formatted docs Daniel Vetter <daniel.vetter@ffwll.ch> - 2016-02-10 09:10 +0100
      Re: [RFC] A first shot at asciidoc-based formatted docs Jani Nikula <jani.nikula@intel.com> - 2016-02-10 15:10 +0100
        Re: [RFC] A first shot at asciidoc-based formatted docs Jani Nikula <jani.nikula@intel.com> - 2016-02-10 17:20 +0100
        Re: [RFC] A first shot at asciidoc-based formatted docs Jonathan Corbet <corbet@lwn.net> - 2016-02-10 22:00 +0100
          Re: [RFC] A first shot at asciidoc-based formatted docs Keith Packard <keithp@keithp.com> - 2016-02-11 16:20 +0100
      Re: [RFC] A first shot at asciidoc-based formatted docs Jonathan Corbet <corbet@lwn.net> - 2016-02-10 21:50 +0100
    Re: [RFC] A first shot at asciidoc-based formatted docs Keith Packard <keithp@keithp.com> - 2016-02-11 00:10 +0100
      Re: [RFC] A first shot at asciidoc-based formatted docs Jani Nikula <jani.nikula@intel.com> - 2016-02-11 14:50 +0100
        Re: [RFC] A first shot at asciidoc-based formatted docs Keith Packard <keithp@keithp.com> - 2016-02-11 16:30 +0100
      Re: [RFC] A first shot at asciidoc-based formatted docs Keith Packard <keithp@keithp.com> - 2016-02-13 04:30 +0100

#1330781 — Re: [RFC] A first shot at asciidoc-based formatted docs

On Tue, 26 Jan 2016 14:08:45 +0200
Jani Nikula <jani.nikula@intel.com> wrote:

> I'm afraid we've done some overlapping work in the mean time, but I'm
> happy we've both looked at the tool chain, and can have a more
> meaningful conversation now.

[Adding Keith since you said you wanted to be a part of this - let us know
when you've had enough!]

So I've spent a bit of time looking at this, and quite a bit more time
talking with various folks at LCA.  There is pretty much universal
agreement that this is interesting work and the direction we'd like to
go.  My current hope is that we can merge some version of it for 4.6 and
see where it goes from there.

So naturally I have some thoughts on the whole thing...

 - I would like to format directly to HTML if at all possible.  It seems
   it should be possible to get a table of contents into the files, and
   the feedback I got was that a TOC would be enough for navigation - it
   would not be necessary to split the files at that point.  We might
   still want to try to figure that out too, though.  In any case, this
   isn't a show stopper, in that we can change it anytime if a better way
   shows up.  But I'd like to have it in mind.

 - Asciidoc templates and processing should happen in a new directory
   (perhaps imaginatively called "asciidoc"); having them in a directory
   called "DocBook" seems a little weird.  More importantly, though, I'd
   like to separate them out as a fresh start, and not mess with the
   existing DocBook templates until we decide we don't need them anymore.
   If we could end up with a cleaner, simpler makefile in the process,
   that would be a bonus.

 - I'm not sold on the new inclusion mechanism.  Creating thousands of
   little files and tracking them for dependencies and such doesn't seem
   like a simplification or a path toward better performance.  I would
   like to at least consider keeping the direct-from-source inclusion.  

 - Insisting on EXPORT_SYMBOL being in the same file doesn't seem like
   it's going to work for now; that could maybe change after Al's work
   goes in, which could be fairly soon.

Please let me know your thoughts on the above.  Do you think you can find
some time over the next month for this?  I'll try to shake loose some time
too, but, well, $EXCUSES...

Many thanks for doing this work!

jon

[toc] | [next] | [standalone]

#1330939

On Wed, Feb 10, 2016 at 1:09 AM, Jonathan Corbet <corbet@lwn.net> wrote:
> On Tue, 26 Jan 2016 14:08:45 +0200
> Jani Nikula <jani.nikula@intel.com> wrote:
>
>> I'm afraid we've done some overlapping work in the mean time, but I'm
>> happy we've both looked at the tool chain, and can have a more
>> meaningful conversation now.
>
> [Adding Keith since you said you wanted to be a part of this - let us know
> when you've had enough!]
>
> So I've spent a bit of time looking at this, and quite a bit more time
> talking with various folks at LCA.  There is pretty much universal
> agreement that this is interesting work and the direction we'd like to
> go.  My current hope is that we can merge some version of it for 4.6 and
> see where it goes from there.
>
> So naturally I have some thoughts on the whole thing...
>
>  - I would like to format directly to HTML if at all possible.  It seems
>    it should be possible to get a table of contents into the files, and
>    the feedback I got was that a TOC would be enough for navigation - it
>    would not be necessary to split the files at that point.  We might
>    still want to try to figure that out too, though.  In any case, this
>    isn't a show stopper, in that we can change it anytime if a better way
>    shows up.  But I'd like to have it in mind.

I think for 4.6 it'd be best to go with the hybrid asciidoc->docbook
toolchain, since that's less disruptive. And with that we can also
fully concentrating on the frontend, and how it'll look and behave.

Once that's solid we can look into the icing on the cake for later
kernels I think.

>  - Asciidoc templates and processing should happen in a new directory
>    (perhaps imaginatively called "asciidoc"); having them in a directory
>    called "DocBook" seems a little weird.  More importantly, though, I'd
>    like to separate them out as a fresh start, and not mess with the
>    existing DocBook templates until we decide we don't need them anymore.
>    If we could end up with a cleaner, simpler makefile in the process,
>    that would be a bonus.

For the long term dream plan of including other .txt files from the
existing pile of unstructured docs, do we really want a separate
asciidoc directory? Or just .asciidoc as a special extension?

>  - I'm not sold on the new inclusion mechanism.  Creating thousands of
>    little files and tracking them for dependencies and such doesn't seem
>    like a simplification or a path toward better performance.  I would
>    like to at least consider keeping the direct-from-source inclusion.

The motivation behind the new inclusion mechanism isn't the speed-up
due to parallelization, but being able to use native asciidoc
includes. With those you can pass options to e.g. shift the hierarchy.
With that you can do subheadings in DOC: sections and then seamlessly
include them. Or similar stuff.

The speed-up due to parallelization is just a small bonus.

Also generating thousands of files is totally not unheard of in the kernel:

$ find include/config | wc -l
2623

None of those are in git.

>  - Insisting on EXPORT_SYMBOL being in the same file doesn't seem like
>    it's going to work for now; that could maybe change after Al's work
>    goes in, which could be fairly soon.

Hm, assuming Al gets his stuff into 4.6 could we just assume this? It
holds true for gpu docs already I think, and most other subsystems.
The trouble iirc is all around asm and similar stuff, and we can't
kerneldoc asm afaik.

> Please let me know your thoughts on the above.  Do you think you can find
> some time over the next month for this?  I'll try to shake loose some time
> too, but, well, $EXCUSES...

One more thing we discussed: Did you ping kbuild folks already? Or
want to get some agreement on the overall build process first?

Cheers, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

[toc] | [prev] | [next] | [standalone]

#1331187

[Sorry this turned out a long email, I didn't have the time to write a
short one.]

On Wed, 10 Feb 2016, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> On Wed, Feb 10, 2016 at 1:09 AM, Jonathan Corbet <corbet@lwn.net> wrote:
>> On Tue, 26 Jan 2016 14:08:45 +0200
>> Jani Nikula <jani.nikula@intel.com> wrote:
>>
>>> I'm afraid we've done some overlapping work in the mean time, but I'm
>>> happy we've both looked at the tool chain, and can have a more
>>> meaningful conversation now.
>>
>> [Adding Keith since you said you wanted to be a part of this - let us know
>> when you've had enough!]
>>
>> So I've spent a bit of time looking at this, and quite a bit more time
>> talking with various folks at LCA.  There is pretty much universal
>> agreement that this is interesting work and the direction we'd like to
>> go.  My current hope is that we can merge some version of it for 4.6 and
>> see where it goes from there.
>>
>> So naturally I have some thoughts on the whole thing...
>>
>>  - I would like to format directly to HTML if at all possible.  It seems
>>    it should be possible to get a table of contents into the files, and
>>    the feedback I got was that a TOC would be enough for navigation - it
>>    would not be necessary to split the files at that point.  We might
>>    still want to try to figure that out too, though.  In any case, this
>>    isn't a show stopper, in that we can change it anytime if a better way
>>    shows up.  But I'd like to have it in mind.
>
> I think for 4.6 it'd be best to go with the hybrid asciidoc->docbook
> toolchain, since that's less disruptive. And with that we can also
> fully concentrating on the frontend, and how it'll look and behave.

I'd like to clarify the end goal a bit more before deciding what to do
next. In particular, is the aim to have asciidoc->HTML only or dual
asciidoc->HTML and asciidoc->XML->whatever? Or independent
asciidoc->HTML first, with the existing DocBook on the side until
everything's converted? Something else?

Direct asciidoc->HTML has the problem I mentioned that there is no
chunked output. If the source is big (as-is or via asciidoc includes)
the output is big. The current gpu.tmpl turned way too big. We could
alleviate that by splitting the source documents into smaller pieces (in
gpu.tmpl case it's desirable no matter what), and tying them together
via cross-references and TOC rather than asciidoc includes.

The problem with this, in turn, is that I don't really know how
automatic cross-referencing between kernel-doc comments would turn out
then (e.g. i915 kernel-doc references a symbol in drm core kernel-doc
after gpu.tmpl split) as asciidoc would process the files
independently. A kernel-doc comment writer shouldn't have to know which
document the referenced symbol is in... We could do post-processing I
guess, but I'd really like to get rid of the homebrew aspects here.

Is it acceptable to have dead links when referencing symbols outside of
the document in question, for the time being, until someone figures out
a nice way to do this?

> Once that's solid we can look into the icing on the cake for later
> kernels I think.
>
>>  - Asciidoc templates and processing should happen in a new directory
>>    (perhaps imaginatively called "asciidoc"); having them in a directory
>>    called "DocBook" seems a little weird.  More importantly, though, I'd
>>    like to separate them out as a fresh start, and not mess with the
>>    existing DocBook templates until we decide we don't need them anymore.
>>    If we could end up with a cleaner, simpler makefile in the process,
>>    that would be a bonus.
>
> For the long term dream plan of including other .txt files from the
> existing pile of unstructured docs, do we really want a separate
> asciidoc directory? Or just .asciidoc as a special extension?

Also in my dream world you could have asciidoc files anywhere in the
Documentation tree, with a Makefile per directory identifying which ones
should be processed as asciidoc. I might even name them all .txt, and
you wouldn't have to rename existing "almost markup" plain text files to
have them processed, just fix the markup and update the Makefile. (FWIW
asciidoc suggests .txt extension, though asciidoctor suggests .adoc or
.asciidoc.) I think this would better promote a gradual transition to
lightweight markup, with easier to review patches. Also you mentioned
there's no structure under Documentation. Allowing asciidoc files
anywhere would, I think, help gradual restructuring.

The output could be a subdirectory (one per output format?) under
Documentation.

>>  - I'm not sold on the new inclusion mechanism.  Creating thousands of
>>    little files and tracking them for dependencies and such doesn't seem
>>    like a simplification or a path toward better performance.  I would
>>    like to at least consider keeping the direct-from-source inclusion.
>
> The motivation behind the new inclusion mechanism isn't the speed-up
> due to parallelization, but being able to use native asciidoc
> includes. With those you can pass options to e.g. shift the hierarchy.
> With that you can do subheadings in DOC: sections and then seamlessly
> include them. Or similar stuff.
>
> The speed-up due to parallelization is just a small bonus.
>
> Also generating thousands of files is totally not unheard of in the kernel:
>
> $ find include/config | wc -l
> 2623
>
> None of those are in git.

Yes, my main motivation here was to get rid of the preprocessing step
(currently tmpl->xml). I wanted to have the source documents in pure
markup which could be directly processed by asciidoc. I wanted to have
the editor markup helpers and syntax highlighting just work, with no
extra non-markup cruft to confuse it. (For example, emacs tells me the
current tmpl files are invalid XML because of the docproc directives.)
This ties back to the dream above; just have .txt files with no
preprocessing step, IMO it's less confusing for actually writing the
docs.

I didn't think there'd be anything weird about having thousands of
intermediate files generated from source files, with dependencies set
and working, just like we have .o files.

Sure, the mechanism is a proof-of-concept, rough around the edges, and
needs to stow away the intermediate files better, but I still think it's
a conceptually better approach than adding a layer of homebrew when we
have a chance to break away from that. And there's the bonus of getting
parallelization, which I think just backs the concept.

I did try to make asciidoc filters and plugins work for including
kernel-doc, which might have been a better match to what Jon wants, but
without docproc in between. I didn't quite manage to make that work, and
there's the problem they're both incompatible with asciidoctor.

>>  - Insisting on EXPORT_SYMBOL being in the same file doesn't seem like
>>    it's going to work for now; that could maybe change after Al's work
>>    goes in, which could be fairly soon.
>
> Hm, assuming Al gets his stuff into 4.6 could we just assume this? It
> holds true for gpu docs already I think, and most other subsystems.
> The trouble iirc is all around asm and similar stuff, and we can't
> kerneldoc asm afaik.

I'd turn this around. IMO the problem isn't insisting EXPORT_SYMBOL is
in the same file as the definition of the symbol. The problem is
insisting that the kernel-doc comment is in the same file as the
EXPORT_SYMBOL and the definition. Particularly include/media has plenty
of kernel-doc in headers with the declarations.

If we can't insist on that, we could teach kernel-doc to scan a list of
other files for the EXPORT_SYMBOLs, instead of having that logic
externally in docproc. This should be trivial, especially if you know
perl. (Unfortunately this might get a little tricky with the include
syntax.)

This was mostly driven by the desire to get rid of the docproc
preprocessing step.

>> Please let me know your thoughts on the above.  Do you think you can find
>> some time over the next month for this?  I'll try to shake loose some time
>> too, but, well, $EXCUSES...

If we can come up with a plan where I can be reasonably sure the
polished effort isn't going down the drain... ;)

> One more thing we discussed: Did you ping kbuild folks already? Or
> want to get some agreement on the overall build process first?

I think CONFIG_BUILD_DOCSRC vs. having documentation targets directly in
Documentation/Makefile (instead of top level make issuing recursive make
in Documentation/DocBook/Makefile) should be reconciliated
somehow. Frankly, I find it odd that the hostprog targets under
Documentation seem to be better class citizens than documentation
targets. Not saying they can't both be there, but they should coexist.

BR,
Jani.



-- 
Jani Nikula, Intel Open Source Technology Center

[toc] | [prev] | [next] | [standalone]

#1331300

On Wed, 10 Feb 2016, Jani Nikula <jani.nikula@intel.com> wrote:
>> On Wed, Feb 10, 2016 at 1:09 AM, Jonathan Corbet <corbet@lwn.net> wrote:
>>>  - I'm not sold on the new inclusion mechanism.  Creating thousands of
>>>    little files and tracking them for dependencies and such doesn't seem
>>>    like a simplification or a path toward better performance.  I would
>>>    like to at least consider keeping the direct-from-source inclusion.

...

> Yes, my main motivation here was to get rid of the preprocessing step
> (currently tmpl->xml). I wanted to have the source documents in pure
> markup which could be directly processed by asciidoc. I wanted to have
> the editor markup helpers and syntax highlighting just work, with no
> extra non-markup cruft to confuse it. (For example, emacs tells me the
> current tmpl files are invalid XML because of the docproc directives.)
> This ties back to the dream above; just have .txt files with no
> preprocessing step, IMO it's less confusing for actually writing the
> docs.

I suppose a compromise could be to put the docproc directives in
asciidoc comments to keep the files pure asciidoc and to hide the
preprocessing step from the document writers, i.e. call them asciidoc
and name them .txt instead of .tmpl or something. While I'm not thrilled
about the idea of keeping docproc around, this would be progress, would
avoid the EXPORT_SYMBOL problem for now, and, most importantly, wouldn't
block us from doing what I suggested as a future iteration.

BR,
Jani.

-- 
Jani Nikula, Intel Open Source Technology Center

[toc] | [prev] | [next] | [standalone]

#1331482

On Wed, 10 Feb 2016 16:03:38 +0200
Jani Nikula <jani.nikula@intel.com> wrote:

> I'd like to clarify the end goal a bit more before deciding what to do
> next. In particular, is the aim to have asciidoc->HTML only or dual
> asciidoc->HTML and asciidoc->XML->whatever? Or independent
> asciidoc->HTML first, with the existing DocBook on the side until
> everything's converted? Something else?

asciidoc->HTML on its own isn't viable, I think; we do have people wanting
other formats.  Though one might well ask when somebody last successfully
generated PDF...maybe it's not worth the trouble.  I would like epub
someday...

There's also people who actually use the man-page output.  I don't think
that should require the xml step; getting rid of that might make it
possible to do "make mandocs" and have it finish before the next merge
window opens...

> Direct asciidoc->HTML has the problem I mentioned that there is no
> chunked output. If the source is big (as-is or via asciidoc includes)
> the output is big. The current gpu.tmpl turned way too big. We could
> alleviate that by splitting the source documents into smaller pieces (in
> gpu.tmpl case it's desirable no matter what), and tying them together
> via cross-references and TOC rather than asciidoc includes.

We talked about that a bit in Geelong; the short-term idea was to generate
a TOC and use CSS to place it correctly.  Daniel, if I heard you correctly,
you thought that would be a fine solution that would remove the need for
chunked output.  Keith seemed interested in looking into this too.

I would still like to look into splitting up the output.  One would *think*
we ought to be able to do that without the whole docbook infrastructure,
but, then, I'm known to be a naive optimist...

> The problem with this, in turn, is that I don't really know how
> automatic cross-referencing between kernel-doc comments would turn out
> then (e.g. i915 kernel-doc references a symbol in drm core kernel-doc
> after gpu.tmpl split) as asciidoc would process the files
> independently. A kernel-doc comment writer shouldn't have to know which
> document the referenced symbol is in... We could do post-processing I
> guess, but I'd really like to get rid of the homebrew aspects here.
> 
> Is it acceptable to have dead links when referencing symbols outside of
> the document in question, for the time being, until someone figures out
> a nice way to do this?

Short-term *maybe*, but I think we'd want to figure that one out quickly. 

> Also in my dream world you could have asciidoc files anywhere in the
> Documentation tree, with a Makefile per directory identifying which ones
> should be processed as asciidoc. I might even name them all .txt, and
> you wouldn't have to rename existing "almost markup" plain text files to
> have them processed, just fix the markup and update the Makefile. (FWIW
> asciidoc suggests .txt extension, though asciidoctor suggests .adoc or
> .asciidoc.) I think this would better promote a gradual transition to
> lightweight markup, with easier to review patches. Also you mentioned
> there's no structure under Documentation. Allowing asciidoc files
> anywhere would, I think, help gradual restructuring.

I agree with all of this, but I still think that, for the short term while
we're figuring out how all this works, it's better to concentrate it in one
place where people can actually find it...

> I'd turn this around. IMO the problem isn't insisting EXPORT_SYMBOL is
> in the same file as the definition of the symbol. The problem is
> insisting that the kernel-doc comment is in the same file as the
> EXPORT_SYMBOL and the definition. Particularly include/media has plenty
> of kernel-doc in headers with the declarations.
> 
> If we can't insist on that, we could teach kernel-doc to scan a list of
> other files for the EXPORT_SYMBOLs, instead of having that logic
> externally in docproc. This should be trivial, especially if you know
> perl. (Unfortunately this might get a little tricky with the include
> syntax.)
> 
> This was mostly driven by the desire to get rid of the docproc
> preprocessing step.

...and that's a worthy goal.  In an ideal world, it's all found together.
I think we should probably proceed with the idea that the EXPORT_SYMBOL
issue can be dealt with.

> >> Please let me know your thoughts on the above.  Do you think you can find
> >> some time over the next month for this?  I'll try to shake loose some time
> >> too, but, well, $EXCUSES...  
> 
> If we can come up with a plan where I can be reasonably sure the
> polished effort isn't going down the drain... ;)

Seems we should be able to do that.  We want this stuff, even I'm not so
dumb as to send it down the drain when things are so close...:)

jon

[toc] | [prev] | [next] | [standalone]

#1332144

[Multipart message — attachments visible in raw view] — view raw

Jonathan Corbet <corbet@lwn.net> writes:

> asciidoc->HTML on its own isn't viable, I think; we do have people wanting
> other formats.  Though one might well ask when somebody last successfully
> generated PDF...maybe it's not worth the trouble.  I would like epub
> someday...

I'm hopeful that I can hack up asciidoc to generate usable HTML
directly. Once we've got HTML, we've got epub.

If we don't want to use docbook for pdf, asciidoc has a native latex
backend. That's in about the same shape as the html backend. Would that
be better than docbook?

> There's also people who actually use the man-page output.  I don't think
> that should require the xml step; getting rid of that might make it
> possible to do "make mandocs" and have it finish before the next merge
> window opens...

Adding a troff backend to asciidoc would be simple enough; I'm not sure
what other method you'd suggest here.

> We talked about that a bit in Geelong; the short-term idea was to generate
> a TOC and use CSS to place it correctly.  Daniel, if I heard you correctly,
> you thought that would be a fine solution that would remove the need for
> chunked output.  Keith seemed interested in looking into this too.

Here's an example that takes the docbook output with some simple CSS
hacks to place the TOC alongside the document in a separate scrolling
list. With a small bit of javascript, I'm pretty sure that could have
collapsible entries.

-- 
-keith

[toc] | [prev] | [next] | [standalone]

#1331478

On Wed, 10 Feb 2016 09:07:22 +0100
Daniel Vetter <daniel.vetter@ffwll.ch> wrote:

> I think for 4.6 it'd be best to go with the hybrid asciidoc->docbook
> toolchain, since that's less disruptive. And with that we can also
> fully concentrating on the frontend, and how it'll look and behave.

That can be fine, I'd just like to have the end goal in mind.  For the near
future we should go with what actually works, definitely.

> >  - Asciidoc templates and processing should happen in a new directory
> >    (perhaps imaginatively called "asciidoc"); having them in a directory
> >    called "DocBook" seems a little weird.  More importantly, though, I'd
> >    like to separate them out as a fresh start, and not mess with the
> >    existing DocBook templates until we decide we don't need them anymore.
> >    If we could end up with a cleaner, simpler makefile in the process,
> >    that would be a bonus.  
> 
> For the long term dream plan of including other .txt files from the
> existing pile of unstructured docs, do we really want a separate
> asciidoc directory? Or just .asciidoc as a special extension?

That's a good question.  I can certainly see the value of mixing the
templates with the rest, but that's a longer-term thing.  I'd sure like to
clean up the main Documentation/ directory before we start scattering
asciidoc stuff around there.  For the moment, I think my preference is
still to focus this work in one place where it's easily found and played
with.  Straightening out this directory is going to involve a fair amount
of moving stuff around, adding a few template files to that debt isn't
going to change the situation much.

> >  - I'm not sold on the new inclusion mechanism.  Creating thousands of
> >    little files and tracking them for dependencies and such doesn't seem
> >    like a simplification or a path toward better performance.  I would
> >    like to at least consider keeping the direct-from-source inclusion.  
> 
> The motivation behind the new inclusion mechanism isn't the speed-up
> due to parallelization, but being able to use native asciidoc
> includes. With those you can pass options to e.g. shift the hierarchy.
> With that you can do subheadings in DOC: sections and then seamlessly
> include them. Or similar stuff.
> 
> The speed-up due to parallelization is just a small bonus.
> 
> Also generating thousands of files is totally not unheard of in the kernel:
> 
> $ find include/config | wc -l
> 2623
> 
> None of those are in git.

Well, we are talking about an order of magnitude more files...  Still, I
said "not sold on" rather than "violently opposed to".  It does seem that
there are some good reasons for doing things this way, including, as Jani
said, getting rid of docproc and not mixing in a weird alien include
syntax.  I'm still not 100% sold, but I'll not hold up a working patch on
this point.

> >  - Insisting on EXPORT_SYMBOL being in the same file doesn't seem like
> >    it's going to work for now; that could maybe change after Al's work
> >    goes in, which could be fairly soon.  
> 
> Hm, assuming Al gets his stuff into 4.6 could we just assume this? It
> holds true for gpu docs already I think, and most other subsystems.
> The trouble iirc is all around asm and similar stuff, and we can't
> kerneldoc asm afaik.

asm stuff and things built into libraries.  But it does seem that this is
well on the way toward being fixed.

> One more thing we discussed: Did you ping kbuild folks already? Or
> want to get some agreement on the overall build process first?

Not yet.  It's worth doing...it would be nice, someday, if docs makefiles
could just have lines like:

      adoc-y += drm.txt ...

but I think we should work out how the pieces fit together before we get
too worried about the details of the build system.  What we have now isn't
particularly well integrated, we're not likely to make it all that much
worse...

Thanks,

jon

[toc] | [prev] | [next] | [standalone]

#1331571

[Multipart message — attachments visible in raw view] — view raw

Jonathan Corbet <corbet@lwn.net> writes:

> [Adding Keith since you said you wanted to be a part of this - let us know
> when you've had enough!]

Thanks.

>  - I would like to format directly to HTML if at all possible.

Agreed. asciidoc's docbook path seems to only increase the amount of
software involved.

>    It seems it should be possible to get a table of contents into the
>    files, and the feedback I got was that a TOC would be enough for
>    navigation

I spent a few hours on the flight home reading asciidoc source code, and
it is a stream processor with a stack. The output is generated with some
simple templates, one for each backend. Here's the xhtml1.1 template for
section level sections: (sect1 in the .conf file):

        [sect1]
        <div class="sect1{style? {style}}{role? {role}}">
        <h2{id? id="{id}"}>{numbered?{sectnum} }{title}</h2>
        <div class="sectionbody">
        |
        </div>
        </div>

The contents of the section get inserted at the |; it's nesting, so
[sect2] bits would get expanded while being processed.

Each asciidoc backend has dramatically different functionality. It's
pretty clear to me that the 'docbook' backend has the best support for
larger documents as that provides 'book-scale' processing bits. I've
recently written a book in asciidoc using the docbook backend, and the
html and pdf results are quite comparable. Using the html backend from
asciidoc yields a significantly different result.

I think it should be pretty easy to hack asciidoc to add diversions to
hold TOC contents while generating the rest of the doc and then replay
the diversion into the final document. Something like:

        [sect1]
        <div class="sect1{style? {style}}{role? {role}}">
        <h2{id? id="{id}"}>{numbered?{sectnum} }{title}</h2>
        |>"table-of-contents"<dl class="toc">
                <dt>
                        <span class="section">
                                <a href="{id}">{numbered?{sectnum} {title}</a>
                        </span>
                </dt>
        </dl>
        <div class="sectionbody">
        |
        </div>
        </div>

At the end of the document, we'd have some way of wrapping the diversion
in suitable additional bits to  complete the TOC, which would then be
formatted by CSS.

This same technique could be used to create lists of figures and tables.

The goal would be to create an html document which could be used without
javascript, and that would work without css as well.

-- 
-keith

[toc] | [prev] | [next] | [standalone]

#1331949

On Thu, 11 Feb 2016, Keith Packard <keithp@keithp.com> wrote:
> I think it should be pretty easy to hack asciidoc to add diversions to
> hold TOC contents while generating the rest of the doc and then replay
> the diversion into the final document.

One of the chief complaints with the current pipeline (and some of the
proposals) has been the need to install lots of tools with lots of
dependencies. I would like to avoid the need to install bleeding edge
tools and stick to what's already widely available in distros. Thus I
would like to avoid hacking asciidoc for our needs.

Also, I'd really like to not have to decide between asciidoc and
asciidoctor, and only use features supported by both. Let the users pick
which one suits them better.

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Technology Center

[toc] | [prev] | [next] | [standalone]

#1332151

[Multipart message — attachments visible in raw view] — view raw

Jani Nikula <jani.nikula@intel.com> writes:

> One of the chief complaints with the current pipeline (and some of the
> proposals) has been the need to install lots of tools with lots of
> dependencies. I would like to avoid the need to install bleeding edge
> tools and stick to what's already widely available in distros. Thus I
> would like to avoid hacking asciidoc for our needs.

Agreed. That means using docbook for now; the native html output from
asciidoc is simply not usable for anything more complicated than a short
web page. However, getting ready to collapse the pipeline by eliminating
docbook seems like a good medium-term goal.

> Also, I'd really like to not have to decide between asciidoc and
> asciidoctor, and only use features supported by both. Let the users pick
> which one suits them better.

That's harder; you'll have much different output from the two
processors. I'd encourage the selection of one of these two tools
instead of trying to support both. I've settled on using only asciidoc
for my other projects because it doesn't require the installation of a
whole new language environment.

-- 
-keith

[toc] | [prev] | [next] | [standalone]

#1333259

[Multipart message — attachments visible in raw view] — view raw

Keith Packard <keithp@keithp.com> writes:

> The goal would be to create an html document which could be used without
> javascript, and that would work without css as well.

I've managed to hack up asciidoc to generate the TOC within the
document, rather than requiring javascript. The changes are fairly
minor, and seem to add a nice generalization to the asciidoc environment
which should be useful in other contexts.

The changes consist of two bits -- the first is to allow the diversion
of some text from .conf file sections, the second is to postpone some
attribute processing to a second pass over the document so that the TOC
can be inserted in the desired location, instead of requiring that it be
placed at the bottom.

I've sent these changes upstream, and also pushed them to a personal
asciidoc git repository at :

        git clone git://keithp.com/git/asciidoc

-- 
-keith

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web