Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.bugs.dist > #1021985 > unrolled thread

Bug#956041: libmaxminddb: FTBFS (BD-Uninstallable): Build-Depends-Arch on pandoc

Back to article view | Back to linux.debian.bugs.dist

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Bug#956041: libmaxminddb: FTBFS (BD-Uninstallable): Build-Depends-Arch on pandoc Faidon Liambotis <paravoid@debian.org> - 2020-08-18 18:00 +0200
    Bug#956041: libmaxminddb: FTBFS (BD-Uninstallable): Build-Depends-Arch on pandoc Thorsten Glaser <t.glaser@tarent.de> - 2020-08-18 19:10 +0200
    Bug#956041: libmaxminddb: FTBFS (BD-Uninstallable): Build-Depends-Arch on pandoc Faidon Liambotis <paravoid@debian.org> - 2020-08-18 20:40 +0200
      Bug#956041: libmaxminddb: FTBFS (BD-Uninstallable): Build-Depends-Arch on pandoc Thorsten Glaser <t.glaser@tarent.de> - 2020-08-18 21:00 +0200
      Bug#956041: libmaxminddb: FTBFS (BD-Uninstallable): Build-Depends-Arch on pandoc Helmut Grohne <helmut@subdivi.de> - 2020-08-18 22:30 +0200
        Bug#956041: libmaxminddb: FTBFS (BD-Uninstallable): Build-Depends-Arch on pandoc Thorsten Glaser <t.glaser@tarent.de> - 2020-08-18 23:00 +0200
          Bug#956041: libmaxminddb: FTBFS (BD-Uninstallable): Build-Depends-Arch on pandoc Helmut Grohne <helmut@subdivi.de> - 2020-08-19 07:10 +0200
            Bug#956041: libmaxminddb: FTBFS (BD-Uninstallable): Build-Depends-Arch on pandoc Thorsten Glaser <t.glaser@tarent.de> - 2020-08-19 17:10 +0200
        Bug#956041: libmaxminddb: FTBFS (BD-Uninstallable): Build-Depends-Arch on pandoc Faidon Liambotis <paravoid@debian.org> - 2020-08-19 01:40 +0200

#1021985 — Bug#956041: libmaxminddb: FTBFS (BD-Uninstallable): Build-Depends-Arch on pandoc

Hi there,

On Tue, Aug 18, 2020 at 04:17:53PM +0200, John Paul Adrian Glaubitz wrote:
> On 8/17/20 11:36 AM, John Paul Adrian Glaubitz wrote:
> > Therefore, in order to keep Debian bootstappable, please try to get rid of this
> > arch-any build dependency and move it to arch-all.
> 
> Just had a look at the package and I'm wondering what prevents us from building the
> manpages in binary-indep? Documentation should move into libmaxmindb-common anyway.

I don't think two (very small) manpages warrant the creation of a new
-common package. It would also be both against a policy "should", about
including the man page in "the same package" (12.1), as well as common
practice and user expectations.

If you're looking for ways to help, there is an upstream bug about the
pandoc dependency -- I think upstream would appreciate any contributions
folks may have on this particular issue.

Regards,
Faidon

[toc] | [next] | [standalone]

#1021996

On Tue, 18 Aug 2020, Faidon Liambotis wrote:

> If you're looking for ways to help, there is an upstream bug about the
> pandoc dependency -- I think upstream would appreciate any contributions
> folks may have on this particular issue.

Unfortunately, they’re not so willing. All they’d agree to do is to
check in the generated files into version control, and we are supposed
to just ship these, and patch the generated file if we patch the source.
This is… probably untenable.

bye,
//mirabilos
-- 
Sometimes they [people] care too much: pretty printers [and syntax highligh-
ting, d.A.] mechanically produce pretty output that accentuates irrelevant
detail in the program, which is as sensible as putting all the prepositions
in English text in bold font.	-- Rob Pike in "Notes on Programming in C"

[toc] | [prev] | [next] | [standalone]

#1022004

On Tue, Aug 18, 2020 at 05:41:26PM +0200, John Paul Adrian Glaubitz wrote:
> As I explained, it's a matter of bootstrappability. Your package is not a leaf
> package which is why this particular problem cannot be ignored. Helmut Grohne
> (CC'ed) who is also working on keeping Debian bootstrappable will also be
> able to explain this problem to you.
>
> Keeping Debian bootstrappable is important as it allows the whole system to be
> bootstrapped from scratch with merely the sources available. Since pandoc is
> a package which requires a large number of packages to be built before it
> can be built itself, your package becomes a blocker when bootstrapping Debian
> which is something that needs to be avoided.

I understand the overall problem and I do think that Debian being
bootstrappable is a worthy and meaningful goal for what it's worth!

I'm not sure whether the package not being a leaf has that much to do
with it. Honestly, I feel like blaming libmaxminddb for bootstrappability
issues is a bit odd. The chain I think is:
  isc-dhcp-client(?) -> bind9 -> libmaxminddb -> pandoc -> <bunch of Haskell>
with Haskell being particularly hard to bootstrap. Did I get that right?

All that said, I don't want to be an obstacle here and would be happy to
help, as long as what we end up doing makes overall sense :)

> As for 12.1, there is already git-man which contains the packages for the manpages
> for git. And since 12.1 is a recommendation and not a requirement, I do not think
> how your argument holds up. Especially, since this simple change unbreaks an important
> requirement in Debian.

git-man has 171 manpages and is multiple megabytes in size. That's not
comparable in the slightest with libmaxminddb which 2 manpages that are
10KB combined. I disagree that splitting off its two manpages into a
separate package is the reasonable thing to do here, I'm afraid.

Regards,
Faidon

[toc] | [prev] | [next] | [standalone]

#1022008

On Tue, 18 Aug 2020, Faidon Liambotis wrote:

> with Haskell being particularly hard to bootstrap. Did I get that right?

That, and unportable, exactly.

> 10KB combined. I disagree that splitting off its two manpages into a
> separate package is the reasonable thing to do here, I'm afraid.

This is a loss-loss situation. We will have to either look that over,
or add the pre-compiled manpages (and ignore the policy that everything
should be compiled during building) or deviate from upstream in conver‐
ting manpages or add a fragile home-grown script to convert them during
the build (with the chance of breaking, although I’d be willing to fix/
update it as needed)… or “convince” upstream to switch to that although
since they refused to do that, “convincing” might need undue force…

bye,
//mirabilos
-- 
«MyISAM tables -will- get corrupted eventually. This is a fact of life. »
“mysql is about as much database as ms access” – “MSSQL at least descends
from a database” “it's a rebranded SyBase” “MySQL however was born from a
flatfile and went downhill from there” – “at least jetDB doesn’t claim to
be a database”	(#nosec)    ‣‣‣ Please let MySQL and MariaDB finally die!

[toc] | [prev] | [next] | [standalone]

#1022021

On Tue, Aug 18, 2020 at 09:38:17PM +0300, Faidon Liambotis wrote:
> I understand the overall problem and I do think that Debian being
> bootstrappable is a worthy and meaningful goal for what it's worth!

Thank you.

> I'm not sure whether the package not being a leaf has that much to do
> with it. Honestly, I feel like blaming libmaxminddb for bootstrappability
> issues is a bit odd. The chain I think is:
>   isc-dhcp-client(?) -> bind9 -> libmaxminddb -> pandoc -> <bunch of Haskell>
> with Haskell being particularly hard to bootstrap. Did I get that right?

The cycle seems wrong to me. isc-dhcp-client Build-Depends on
libbind-export-dev, which is built from src:bind9-libs and doesn't
depend on something built from libmaxminddb. So this particular cycle is
already solved at the bind level by splitting the bind source package.

This is not to say that there would be no cycle. I haven't checked, but
it isn't this cycle.

I also note that cycles involving pandoc doc not affect cross bootstrap
in any way, because pandoc is marked Multi-Arch: foreign. Therefore, the
problem at hand can only be about native bootstraps, which are pretty
badly supported anyway.

That said, if you want to break a cycle using cross compilation (which
is not applicable to all bootstrap scenarios), libmaxminddb would have
to be cross buildable, which presently isn't the case. This is due to
all the perl-ish dependencies. Do I understand correctly, that you only
need them for running unit tests? If yes, please annotate them
<!nocheck>.

> All that said, I don't want to be an obstacle here and would be happy to
> help, as long as what we end up doing makes overall sense :)

Speaking of sense: Having the libmaxminddb.3 manual page in the
libmaxminddb0 packages certainly does not make sense. You added the
developer documentation to the shared library package. A much better
place would be libmaxminddb-dev. Unfortunately, that doesn't help us in
any way with solving the bootstrapping problem at hand.

> git-man has 171 manpages and is multiple megabytes in size. That's not
> comparable in the slightest with libmaxminddb which 2 manpages that are
> 10KB combined. I disagree that splitting off its two manpages into a
> separate package is the reasonable thing to do here, I'm afraid.

It actually is kinda worse here. The manual pages presently reside in
different packages. Splitting them off would add two binary packages,
not just one. Indeed this looks like quite a large hammer to me.

So let's talk about alternatives.

1. nodoc. There is a build profile/option called nodoc. I really hate
   it, because it makes things unreproducible and I like things to be
   reproducible. However, making the manual pages optional via a nodoc
   profile certainly solves the bootstrap cycle. It's just inconvenient
   for the bootstrapper and requires manual work. I don't like it, but
   it really is a solution.

2. Try a different converter. ronn (written in ruby) is such a
   converter. I'm not sure whether ruby cycles with libmaxminddb in some
   other way.

So maybe this is less about making bootstraps possible, but more about
making bootstraps convenient (i.e. not requiring manual work). This is
an important property as well, as we already need to fiddle with way too
many aspects to make bootstrapping possible. In this case, autobuilding
is stuck and manual work is required.

Thorsten, can you tell more as to what precisely the problem is? I
looked into https://buildd.debian.org/status/package.php?p=libmaxminddb.
It doesn't look bd-uninstallable anywhere.

Helmut

[toc] | [prev] | [next] | [standalone]

#1022031

On Tue, 18 Aug 2020, Helmut Grohne wrote:

> Thorsten, can you tell more as to what precisely the problem is? I
> looked into https://buildd.debian.org/status/package.php?p=libmaxminddb.

It requires pandoc, which is rather unportable.

> It doesn't look bd-uninstallable anywhere.

That’s because I did a massive hack using versions of
Haskell from snapshot.d.o and some self-compiled intermediate
(read: old) versions to make pandoc installable-ish on x32.
I did the unspeakable there thus…

This state will not last, and it’s unclear if it even works right.

bye,
//mirabilos
-- 
15:41⎜<Lo-lan-do:#fusionforge> Somebody write a testsuite for helloworld :-)

[toc] | [prev] | [next] | [standalone]

#1022056

Hi Thorsten,

On Tue, Aug 18, 2020 at 10:51:23PM +0200, Thorsten Glaser wrote:
> That’s because I did a massive hack using versions of
> Haskell from snapshot.d.o and some self-compiled intermediate
> (read: old) versions to make pandoc installable-ish on x32.
> I did the unspeakable there thus…

This is the bit I was somehow missing in the discussion. Thanks for your
effort here.

> This state will not last, and it’s unclear if it even works right.

Yes.

I don't think this is only a topic of bootstrapping. We actually don't
bootstrap that often and requiring a bit of effort seems reasonable. It
really is a topic of keeping the archive up to date with reasonable
effort. If a package becomes bd-uninstallable repeatedly and requires
this bootstrap-ish effort (per architecture), then something is wrong.
It appears to me that this is the case here and this is what Adrian and
Thorsten complain about. I also looked into
https://bootstrap.debian.net/botch-native/amd64/stats.html and couldn't
find a cycle there. Maybe we should remove the bootstrapping argument
from the discussion?

This is not black and white. We're ultimately talking about trade-offs.
We're making this easier on one side and more difficult on the other or
vice versa.

What Adrian and Thorsten have proposed is the solution that makes it
effortless on the side of keeping the archive up to date. (It still
requires effort, but not necessarily with libmaxminddb.) The status quo
is maximizing features at the expense of porter time.

I think a key takeaway is that we will not have pandoc for all
architectures. Getting it built everywhere is beyond our capacity as
porters and package maintainers. It'll exist for release architectures,
but not for all ports. The wisdom that you should only list pandoc in
B-D-I certainly isn't new as a general rule of thumb and I think
Thorsten explained again why we have it.

Any solution where we still have pandoc in B-D-A, is a solution that
requires some effort from porters. This is why Adrian and Thorsten push
back here. They rightly do. For that reason, my nodoc proposal looks
like a non-solution to me. It is something that Adrian has already done
manually to work around the issue.

I've also attempted to convert the manual pages using ronn and the
output is rather unpleasant.

So let's look at the costs for the solution where we split off manual
pages. We'd add a binary package, yes. It contains two small manual
pages. libmaxminddb-dev also contains 14 manpage links, that would
likely move to the new package. This is still a small package, but the
cost of a small package to the archive is relatively small. Let me give
more examples where a technical matter resulted in a package split:
 * qt5-qmake -> qt5-qmake-bin was split for cross building.
 * libglib2.0-dev -> libglib2.0-dev-bin was split for cross building.
 * dropbear was split into dropbear-bin, dropbear-initramfs,
   dropbear-run to make the init and initramfs integration optional.
 * cryptsetup was similarly split.

This really is a common thing. Putting the .service/init.d files in a
different package is very similar to putting manual pages in a different
package to me.

I must admit that I find the trade-off proposed by Adrian and Thorsten
quite reasonable. I've also looked for more options, but none came to my
mind.

Faidon, can you elaborate on the costs of the split from your point of
view? It would seem quite cheap on the maintenance side to me.

Helmut

[toc] | [prev] | [next] | [standalone]

#1022103

On Wed, 19 Aug 2020, Helmut Grohne wrote:

> I must admit that I find the trade-off proposed by Adrian and Thorsten
> quite reasonable. I've also looked for more options, but none came to my
> mind.

I’m actually in favour of one of the other solutions,
not the package split.

But thanks for the good summary!

bye,
//mirabilos
-- 
15:41⎜<Lo-lan-do:#fusionforge> Somebody write a testsuite for helloworld :-)

[toc] | [prev] | [next] | [standalone]

#1022040

On Tue, Aug 18, 2020 at 10:12:07PM +0200, Helmut Grohne wrote:
> > I'm not sure whether the package not being a leaf has that much to do
> > with it. Honestly, I feel like blaming libmaxminddb for bootstrappability
> > issues is a bit odd. The chain I think is:
> >   isc-dhcp-client(?) -> bind9 -> libmaxminddb -> pandoc -> <bunch of Haskell>
> > with Haskell being particularly hard to bootstrap. Did I get that right?
> 
> The cycle seems wrong to me. isc-dhcp-client Build-Depends on
> libbind-export-dev, which is built from src:bind9-libs and doesn't
> depend on something built from libmaxminddb. So this particular cycle is
> already solved at the bind level by splitting the bind source package.
> 
> This is not to say that there would be no cycle. I haven't checked, but
> it isn't this cycle.

Ah, apologies then. Not sure what the cycle is in that case :)

> That said, if you want to break a cycle using cross compilation (which
> is not applicable to all bootstrap scenarios), libmaxminddb would have
> to be cross buildable, which presently isn't the case. This is due to
> all the perl-ish dependencies. Do I understand correctly, that you only
> need them for running unit tests? If yes, please annotate them
> <!nocheck>.

Not all, but from what I saw in the bug you filed, you figured that out
already :) Thanks for that bug report, I applied that patch in git.

> > All that said, I don't want to be an obstacle here and would be happy to
> > help, as long as what we end up doing makes overall sense :)
> 
> Speaking of sense: Having the libmaxminddb.3 manual page in the
> libmaxminddb0 packages certainly does not make sense. You added the
> developer documentation to the shared library package. A much better
> place would be libmaxminddb-dev. Unfortunately, that doesn't help us in
> any way with solving the bootstrapping problem at hand.

Whoops! Thanks for spotting that! Fixed that in git as well.

> So let's talk about alternatives.
> 
> 1. nodoc. There is a build profile/option called nodoc. I really hate
>    it, because it makes things unreproducible and I like things to be
>    reproducible. However, making the manual pages optional via a nodoc
>    profile certainly solves the bootstrap cycle. It's just inconvenient
>    for the bootstrapper and requires manual work. I don't like it, but
>    it really is a solution.
> 
> 2. Try a different converter. ronn (written in ruby) is such a
>    converter. I'm not sure whether ruby cycles with libmaxminddb in some
>    other way.

Hmmm alright. That's helpful! 

I gave ronn a look just now, attempting to convert the mmdblookup
manpage. The output is broken, apparently due to apjanke/ronn-ng#57
(very new), apjanke/ronn-ng#33 (old), as well as the requirement to use
section 2 markdown headers instead of section 1 (the latter being
relatively easy to workaround in the Perl script, though). I also did
not check for current or potential bootstrap cycles.

Best,
Faidon

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.bugs.dist

csiph-web