Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1433896 > unrolled thread

Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's  docs-next Jonathan Corbet <corbet@lwn.net> - 2016-06-29 18:30 +0200
    Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next Markus Heiser <markus.heiser@darmarit.de> - 2016-06-29 19:40 +0200
      Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's  docs-next Jonathan Corbet <corbet@lwn.net> - 2016-06-29 20:00 +0200
        Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's  docs-next Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-06-29 23:00 +0200
          Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next Markus Heiser <markus.heiser@darmarit.de> - 2016-06-30 11:40 +0200
            Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's  docs-next Mauro Carvalho Chehab <mchehab@osg.samsung.com> - 2016-06-30 12:30 +0200
        Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next Markus Heiser <markus.heiser@darmarit.de> - 2016-06-30 11:10 +0200

#1433896 — Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next

Hi, Markus,

I was glad to hear from you, but I have to agree with Jani: this is not
how things are done.  Consider this one line:

> 706 files changed, 123369 insertions(+), 752 deletions(-)

Something like that will be a huge red flag to any kernel maintainer!

In the kernel community, we have spent the last 25 years figuring out a
development model that is based on gradual, incremental changes, each of
which can be reviewed on its own merits.  It does *not* encompass
wholesale replacements of existing code — even in situations where that
code was not just merged after a year of discussions, negotiations, and
false starts.

I simply cannot accept this pull request.

Markus, we all very much want your help in this work.  You have expertise
and energy that could really push the documentation effort forward.  But
it needs to be done the way kernel developers do it: cooperatively,
incrementally, and always mindful of the entire community's needs.

I would love it if you would take the flat-table and man-page work,
separate them out, and make them work with the *existing* Sphinx-based
scheme.  If you can do it soon, we can maybe get it into 4.8.  Can you
focus on that for now, please?

As for the rest, what we have now is certainly far from perfect; we're
figuring a lot of this out as we go.  Incremental improvements are
welcome, and each will be evaluated independently.  Please help us to
make the kernel's documentation better that way.

Thanks,

jon

[toc] | [next] | [standalone]

#1433925 — Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next

Hi Jonathan,

Am 29.06.2016 um 18:24 schrieb Jonathan Corbet <corbet@lwn.net>:

> Hi, Markus,
> 
> I was glad to hear from you, but I have to agree with Jani: this is not
> how things are done.  Consider this one line:
> 
>> 706 files changed, 123369 insertions(+), 752 deletions(-)
> 
> Something like that will be a huge red flag to any kernel maintainer!
> 
> In the kernel community, we have spent the last 25 years figuring out a
> development model that is based on gradual, incremental changes, each of
> which can be reviewed on its own merits.  It does *not* encompass
> wholesale replacements of existing code — even in situations where that
> code was not just merged after a year of discussions, negotiations, and
> false starts.

Yes, my mistake, as I wrote to Jani.

> I simply cannot accept this pull request.
> 
> Markus, we all very much want your help in this work.  You have expertise
> and energy that could really push the documentation effort forward.  But
> it needs to be done the way kernel developers do it: cooperatively,
> incrementally, and always mindful of the entire community's needs.
> 
> I would love it if you would take the flat-table and man-page work,
> separate them out, and make them work with the *existing* Sphinx-based
> scheme.  If you can do it soon, we can maybe get it into 4.8.  Can you
> focus on that for now, please?

Yes, I will send you flat-table request in the next days.

> As for the rest, what we have now is certainly far from perfect; we're
> figuring a lot of this out as we go.  Incremental improvements are
> welcome, and each will be evaluated independently.  Please help us to
> make the kernel's documentation better that way.

I'am willing to do so, but I need some help / suggestions:

1. I have this extensions in the scripts/site-python/linuxdoc.
   What do you recommend, how could I split this up in a patch
   series which is more evaluated.

.. I wrote to Jani, that my approach was chaotic in the past
and I'am sorry for this. But now I'am sitting in front of this
bulk of source and I'am bit helpless how to split ... I will
try to make it more elaborate, but it will be helpfull if 
you point me the right direction ... 

2. What is the best way to ship these migrations

or better I asked, what is your recommendation for a
migration strategy. Jani says, that this better belongs
to authors, but I have a doubt that we end with the
migration in the next years, if we wait about every author.
I think, supporting both infrastructures - the xml and the
reST - over a long period is not the best option. What is
your recommendation on this? 

Regards

  -- Markus --
> 
> Thanks,
> 
> jon

[toc] | [prev] | [next] | [standalone]

#1433931

On Wed, 29 Jun 2016 19:35:46 +0200
Markus Heiser <markus.heiser@darmarit.de> wrote:

> > I would love it if you would take the flat-table and man-page work,
> > separate them out, and make them work with the *existing* Sphinx-based
> > scheme.  If you can do it soon, we can maybe get it into 4.8.  Can you
> > focus on that for now, please?  
> 
> Yes, I will send you flat-table request in the next days.

I'm glad to hear that.  One request: please post it as a patch, rather than
as a pull request; that makes it easier for everybody to review it.

> > As for the rest, what we have now is certainly far from perfect; we're
> > figuring a lot of this out as we go.  Incremental improvements are
> > welcome, and each will be evaluated independently.  Please help us to
> > make the kernel's documentation better that way.  
> 
> I'am willing to do so, but I need some help / suggestions:
> 
> 1. I have this extensions in the scripts/site-python/linuxdoc.
>    What do you recommend, how could I split this up in a patch
>    series which is more evaluated.
> 
> .. I wrote to Jani, that my approach was chaotic in the past
> and I'am sorry for this. But now I'am sitting in front of this
> bulk of source and I'am bit helpless how to split ... I will
> try to make it more elaborate, but it will be helpfull if 
> you point me the right direction ... 

Try to break it down as much as possible so that each patch represents a
single logical change.  Each bit that you can break out reduces the problem
space a bit, and often helps with the rest.  If possible, I'd like to
suggest starting with the man-page generation, since that's a hole in the
current system.  I'll have to fill it if you don't :)

Please note that I'd really like to see this stuff done without big changes
like the wholesale replacement of kernel-doc with a version in a different
language.  Someday we might want to make a change like that, but one step
at a time.

> 2. What is the best way to ship these migrations
> 
> or better I asked, what is your recommendation for a
> migration strategy. Jani says, that this better belongs
> to authors, but I have a doubt that we end with the
> migration in the next years, if we wait about every author.
> I think, supporting both infrastructures - the xml and the
> reST - over a long period is not the best option. What is
> your recommendation on this?

I think we need to give maintainers the first shot at doing the conversion;
in any case, I don't think we can just force it through without their
cooperation.  And, honestly, while we're still groping around in this
space, I think it's fine if we don't have lots of conversions right away.
The ones that go more slowly will benefit from what we learn with the easy
ones.

You could certainly talk to maintainers and see if they would like
assistance with specific books.  Helping Mauro to get his tables done
without going totally nuts would be a great first step, IMO.

That said, if you're wanting to convert documents, there is a set of older
ones in the docbook directory that have no current maintainer and will
never move over on their own.  kernel-hacking.tmpl is an obvious example.
The problem with these, of course, is that they are *way* out of date in
general, and really need attention beyond just a format conversion.  I
won't say one has to happen before the other, but I am unsure that we will
really benefit from convert-and-forget-again efforts.

Thanks,

jon

[toc] | [prev] | [next] | [standalone]

#1434048

Em Wed, 29 Jun 2016 11:52:09 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> > 2. What is the best way to ship these migrations
> > 
> > or better I asked, what is your recommendation for a
> > migration strategy. Jani says, that this better belongs
> > to authors, but I have a doubt that we end with the
> > migration in the next years, if we wait about every author.
> > I think, supporting both infrastructures - the xml and the
> > reST - over a long period is not the best option. What is
> > your recommendation on this?  
> 
> I think we need to give maintainers the first shot at doing the conversion;
> in any case, I don't think we can just force it through without their
> cooperation.  And, honestly, while we're still groping around in this
> space, I think it's fine if we don't have lots of conversions right away.
> The ones that go more slowly will benefit from what we learn with the easy
> ones.
> 
> You could certainly talk to maintainers and see if they would like
> assistance with specific books.  Helping Mauro to get his tables done
> without going totally nuts would be a great first step, IMO.

Yeah, going to each maintainer is the best way, as we're the ones
that will be bothered if something goes wrong on the documentation
(and code) that we maintain.

In the specific case of media stuff, the first step is to get the
flat-tree support and to put the migrated documentation on a separate
directory. Unfortunately, I have two sets of topic branches for 4.8,
and they both touch at the media docbook (and nothing prevents that
similar things to happen on 4.9 or any other version).

So, my plan is to keep both reST and docbook on Kernel for a while,
until be sure that everything is properly migrated.

Yet, I'm not sure if we should keep the migration scripts at the
Kernel tree, or if the best is to keep them in separate, as I
intend to not take more than one Kernel version to finish the
conversion. At least for media docbook, this will be a one-time
conversion.

So, I really appreciate if you could send me the patches with
the converted media documentation that you did. I'll merge it
after Jonathan applies the flat-tree patch on his tree, and
start reviewing and fixing the documentation over the main
branch. I'll need to track the topic branch changes at the
Docbook, to apply them again at the master tree, once they
gets merged during the 4.8 window. Thankfully, there aren't
complex elements on such changes (as far as I remember).

If everything goes right, by -rc2 or -rc3 I'll likely be dropping
the DocBook/media from my tree.

--  
Thanks,
Mauro

[toc] | [prev] | [next] | [standalone]

#1434413 — Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next

Am 29.06.2016 um 22:57 schrieb Mauro Carvalho Chehab <mchehab@osg.samsung.com>:

> Em Wed, 29 Jun 2016 11:52:09 -0600
> Jonathan Corbet <corbet@lwn.net> escreveu:
> 
>>> 2. What is the best way to ship these migrations
>>> 
>>> or better I asked, what is your recommendation for a
>>> migration strategy. Jani says, that this better belongs
>>> to authors, but I have a doubt that we end with the
>>> migration in the next years, if we wait about every author.
>>> I think, supporting both infrastructures - the xml and the
>>> reST - over a long period is not the best option. What is
>>> your recommendation on this?  
>> 
>> I think we need to give maintainers the first shot at doing the conversion;
>> in any case, I don't think we can just force it through without their
>> cooperation.  And, honestly, while we're still groping around in this
>> space, I think it's fine if we don't have lots of conversions right away.
>> The ones that go more slowly will benefit from what we learn with the easy
>> ones.
>> 
>> You could certainly talk to maintainers and see if they would like
>> assistance with specific books.  Helping Mauro to get his tables done
>> without going totally nuts would be a great first step, IMO.
> 
> Yeah, going to each maintainer is the best way, as we're the ones
> that will be bothered if something goes wrong on the documentation
> (and code) that we maintain.

Later if I have more time, may I will pick up those books who need
only small correction e.g. "debugobjects" .. but when the time comes
I will contact the maintainer and Jon' first.

> In the specific case of media stuff, the first step is to get the
> flat-tree support and to put the migrated documentation on a separate
> directory. Unfortunately, I have two sets of topic branches for 4.8,
> and they both touch at the media docbook (and nothing prevents that
> similar things to happen on 4.9 or any other version).
> 
> So, my plan is to keep both reST and docbook on Kernel for a while,
> until be sure that everything is properly migrated.
> 
> Yet, I'm not sure if we should keep the migration scripts at the
> Kernel tree, or if the best is to keep them in separate, as I
> intend to not take more than one Kernel version to finish the
> conversion. At least for media docbook, this will be a one-time
> conversion.

If I understood you right: the scripts, which are made the
migration itself should not committed to the kernel tree (IMHO).
Currently the scripts are available at my POC, may I could
separate them into a python package which could be installed
via python's pip. So if one wants to use this procedure of
migration, he could install the package locally, make the
conversion and after finishing, he uninstall the package. 

> So, I really appreciate if you could send me the patches with
> the converted media documentation that you did. I'll merge it
> after Jonathan applies the flat-tree patch on his tree, and
> start reviewing and fixing the documentation over the main
> branch.

Yes, first flat-table to Jon', then the converted media book
to Mauro, after this the man-page builder to Jon ... as long
as there are no man-pages in media it should be smoothly.

But I need a bit time, hopefully, end next week you got the
media patch from me.

-- Markus --

> I'll need to track the topic branch changes at the
> Docbook, to apply them again at the master tree, once they
> gets merged during the 4.8 window. Thankfully, there aren't
> complex elements on such changes (as far as I remember).
> 
> If everything goes right, by -rc2 or -rc3 I'll likely be dropping
> the DocBook/media from my tree.
> 
> --  
> Thanks,
> Mauro

[toc] | [prev] | [next] | [standalone]

#1434433

Em Thu, 30 Jun 2016 11:25:11 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:

> Am 29.06.2016 um 22:57 schrieb Mauro Carvalho Chehab <mchehab@osg.samsung.com>:
> 
> > Em Wed, 29 Jun 2016 11:52:09 -0600
> > Jonathan Corbet <corbet@lwn.net> escreveu:
> >   
> >>> 2. What is the best way to ship these migrations
> >>> 
> >>> or better I asked, what is your recommendation for a
> >>> migration strategy. Jani says, that this better belongs
> >>> to authors, but I have a doubt that we end with the
> >>> migration in the next years, if we wait about every author.
> >>> I think, supporting both infrastructures - the xml and the
> >>> reST - over a long period is not the best option. What is
> >>> your recommendation on this?    
> >> 
> >> I think we need to give maintainers the first shot at doing the conversion;
> >> in any case, I don't think we can just force it through without their
> >> cooperation.  And, honestly, while we're still groping around in this
> >> space, I think it's fine if we don't have lots of conversions right away.
> >> The ones that go more slowly will benefit from what we learn with the easy
> >> ones.
> >> 
> >> You could certainly talk to maintainers and see if they would like
> >> assistance with specific books.  Helping Mauro to get his tables done
> >> without going totally nuts would be a great first step, IMO.  
> > 
> > Yeah, going to each maintainer is the best way, as we're the ones
> > that will be bothered if something goes wrong on the documentation
> > (and code) that we maintain.  
> 
> Later if I have more time, may I will pick up those books who need
> only small correction e.g. "debugobjects" .. but when the time comes
> I will contact the maintainer and Jon' first.
> 
> > In the specific case of media stuff, the first step is to get the
> > flat-tree support and to put the migrated documentation on a separate
> > directory. Unfortunately, I have two sets of topic branches for 4.8,
> > and they both touch at the media docbook (and nothing prevents that
> > similar things to happen on 4.9 or any other version).
> > 
> > So, my plan is to keep both reST and docbook on Kernel for a while,
> > until be sure that everything is properly migrated.
> > 
> > Yet, I'm not sure if we should keep the migration scripts at the
> > Kernel tree, or if the best is to keep them in separate, as I
> > intend to not take more than one Kernel version to finish the
> > conversion. At least for media docbook, this will be a one-time
> > conversion.  
> 
> If I understood you right: the scripts, which are made the
> migration itself should not committed to the kernel tree (IMHO).
> Currently the scripts are available at my POC, may I could
> separate them into a python package which could be installed
> via python's pip. So if one wants to use this procedure of
> migration, he could install the package locally, make the
> conversion and after finishing, he uninstall the package. 

Sounds like a plan.

> 
> > So, I really appreciate if you could send me the patches with
> > the converted media documentation that you did. I'll merge it
> > after Jonathan applies the flat-tree patch on his tree, and
> > start reviewing and fixing the documentation over the main
> > branch.  
> 
> Yes, first flat-table to Jon', then the converted media book
> to Mauro, after this the man-page builder to Jon ... as long
> as there are no man-pages in media it should be smoothly.

OK.

> But I need a bit time, hopefully, end next week you got the
> media patch from me.

Ok, but please notice that the end of the next week is probably too late
for 4.8. Next week, we'll be on -rc6, and most maintainers freeze their
trees during -rc7, except for bug fixes. Ok, as this is just documentation
and should not cause regressions, I may open an exception, if it won't
cause any troubles to Jon.

> 
> -- Markus --
> 
> > I'll need to track the topic branch changes at the
> > Docbook, to apply them again at the master tree, once they
> > gets merged during the 4.8 window. Thankfully, there aren't
> > complex elements on such changes (as far as I remember).
> > 
> > If everything goes right, by -rc2 or -rc3 I'll likely be dropping
> > the DocBook/media from my tree.
> > 
> > --  
> > Thanks,
> > Mauro  
> 


-- 
Thanks,
Mauro

[toc] | [prev] | [next] | [standalone]

#1434392 — Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next

Hi Jonathan,

Am 29.06.2016 um 19:52 schrieb Jonathan Corbet <corbet@lwn.net>:

> On Wed, 29 Jun 2016 19:35:46 +0200
> Markus Heiser <markus.heiser@darmarit.de> wrote:
> 
>>> I would love it if you would take the flat-table and man-page work,
>>> separate them out, and make them work with the *existing* Sphinx-based
>>> scheme.  If you can do it soon, we can maybe get it into 4.8.  Can you
>>> focus on that for now, please?  
>> 
>> Yes, I will send you flat-table request in the next days.
> 
> I'm glad to hear that.  One request: please post it as a patch, rather than
> as a pull request; that makes it easier for everybody to review it.
> 
>>> As for the rest, what we have now is certainly far from perfect; we're
>>> figuring a lot of this out as we go.  Incremental improvements are
>>> welcome, and each will be evaluated independently.  Please help us to
>>> make the kernel's documentation better that way.  
>> 
>> I'am willing to do so, but I need some help / suggestions:
>> 
>> 1. I have this extensions in the scripts/site-python/linuxdoc.
>>   What do you recommend, how could I split this up in a patch
>>   series which is more evaluated.
>> 
>> .. I wrote to Jani, that my approach was chaotic in the past
>> and I'am sorry for this. But now I'am sitting in front of this
>> bulk of source and I'am bit helpless how to split ... I will
>> try to make it more elaborate, but it will be helpfull if 
>> you point me the right direction ... 
> 
> Try to break it down as much as possible so that each patch represents a
> single logical change.  Each bit that you can break out reduces the problem
> space a bit, and often helps with the rest.  If possible, I'd like to
> suggest starting with the man-page generation, since that's a hole in the
> current system.  I'll have to fill it if you don't :)

Give me a bit time, I will do it. At first flat-table, then man-page.


> Please note that I'd really like to see this stuff done without big changes
> like the wholesale replacement of kernel-doc with a version in a different
> language.  Someday we might want to make a change like that, but one step
> at a time.

mmh, OK ... it will be "the long run" for me ... I will take it (again). The 
replacement makes many things much easier and has this big features; to parse 
only once (not on every kernel-doc directive / one error log, not n times same 
error messages) and a rich interface to control the reST output fine grained (
and Snippets, and a NullTranslator as a lint for free and .. and) ... it is a
good working base ... no need for breadcrumbs or other tricky workarounds ... 

OK, I will start improving the perl script insofar it is needed (the reST out
has to be more structural, you will see it / if it comes to the man-page builder)
... may be later I could persuade you, that it is a "dead end street" ... if the
language python is the problem, I could maintain these modules (15 years practice).

Regards 

  --Markus--


>> 2. What is the best way to ship these migrations
>> 
>> or better I asked, what is your recommendation for a
>> migration strategy. Jani says, that this better belongs
>> to authors, but I have a doubt that we end with the
>> migration in the next years, if we wait about every author.
>> I think, supporting both infrastructures - the xml and the
>> reST - over a long period is not the best option. What is
>> your recommendation on this?
> 
> I think we need to give maintainers the first shot at doing the conversion;
> in any case, I don't think we can just force it through without their
> cooperation.  And, honestly, while we're still groping around in this
> space, I think it's fine if we don't have lots of conversions right away.
> The ones that go more slowly will benefit from what we learn with the easy
> ones.
> 
> You could certainly talk to maintainers and see if they would like
> assistance with specific books.  Helping Mauro to get his tables done
> without going totally nuts would be a great first step, IMO.
> 
> That said, if you're wanting to convert documents, there is a set of older
> ones in the docbook directory that have no current maintainer and will
> never move over on their own.  kernel-hacking.tmpl is an obvious example.
> The problem with these, of course, is that they are *way* out of date in
> general, and really need attention beyond just a format conversion.  I
> won't say one has to happen before the other, but I am unsure that we will
> really benefit from convert-and-forget-again efforts.
> 
> Thanks,
> 
> jon

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web