Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.devel > #112131

Re: Mandatory LC_ALL=C.UTF-8 during package building

Show all headers | View raw

Hi,

On 6/11/24 00:26, Simon McVittie wrote:

>> Reproducibility outside of sterile environments is however a problem for us
>> as a distribution, because it affects how well people are able to contribute
>> to packages they are not directly maintaining

> If our package-building entry point sets up aspects of its desired
> normalized (or "sterile") environment itself, surely it's equally easy
> for those contributors to build every package in that way, whether they
> maintain this particular package or not?

Yes, but building the package is not the hard part in making a useful 
contribution -- anything but trivial changes will need iterative 
modifications and testing, and the package building entrypoint is 
limited to "clean and build entire package" and "build package without 
cleaning first", with the latter being untested and likely broken for a 
lot of packages -- both meson and cmake utterly dislike being asked to 
configure an existing build directory as if it were new.

For my own packages, I roughly know how far I can deviate from the clean 
environment and still get meaningful test results, but for anything 
else, I will still need to deep-dive into the build system to get 
something that is capable of incremental builds.

>> if my package is not required to work outside
>> of a very controlled environment, that is also an impediment to
>> co-maintenance

> I'm not sure that follows. If the only thing we require is that it works
> in a controlled environment, and the controlled environment is documented
> and easy to achieve, then surely every prospective co-maintainer is in
> an equally good position to be contributing? That seems better, to me, than
> having unwritten rules about what environment is "close enough" and what
> environment doesn't actually work.

I will need to deviate from the clean environment, because the clean 
environment does not have vim installed. Someone else might need to 
deviate further and have a graphical environment and a lot of dbus 
services available because their preferred editor requires it.

Adding a global expectation about the environment that a package build 
can rely on *creates* an unwritten per-package rule whether it is 
permissible to deviate from this expectation during development.

I expect that pretty much no one uses the C.UTF-8 locale for their 
normal login session, so adding this as a requirement to the build 
environment creates a pretty onerous rule: "if you want to test your 
changes, you need to remember to call make/ninja with LC_ALL=C.UTF-8."

Of course we know that this rule is bullshit, because the majority of 
packages will build and test fine without it, but this knowledge is 
precisely one of the "unwritten rules" that we're trying to avoid here.

> We already do expect maintainers to be building in a specified
> environment: Debian unstable (not stable, and not Ubuntu, for example).

I develop mostly on Debian or Devuan stable, then do a pbuilder build 
right before upload to make sure it also builds in a clean unstable 
environment. The original requirement was mostly about uploading binary 
packages, which we (almost) don't do anymore.

> (I also do agree that it is an anti-pattern if we have a specified
> environment where tests or QA will be run, and serious consequences for
> failures in that environment, without it being reasonably straightforward
> for contributors to repeat the testing in a sufficiently similar
> controlled environment that they have a decent chance at being able to
> reproduce the failure. But, again, that isn't this thread.)

This is largely what I think is this thread -- narrowing the environment 
where builds, tests and QA will be run, and narrowing what will be 
considered a bug.

>> Indeed -- however this class of bugs has already been solved because
>> reproducible-builds.org have filed bugs wherever this happened, and
>> maintainers have added workarounds where it was impossible to fix.

> Someone (actually, quite a lot of someones) had to do that testing,
> and those fixes or workarounds. Who did it benefit, and would they have
> received the same benefit if we had said "building in a locale other than
> C.UTF-8 is unsupported", or in some cases "building in a non-UTF-8 locale
> is unsupported", and made it straightforward to build in the supported
> locales?

I'd say that developers who don't have English as their first language 
have directly benefited from this, and would not have benefited if it 
was not seen as a problem if a package didn't build on their machines 
without the use of a controlled environment.

I also think that we have indirectly benefited from better test coverage.

>> Turning this workaround into boilerplate code was a mistake already, so the
>> answer to the complaint about having to copy boilerplate code that should be
>> moved into the framework is "do not copy boilerplate code."

> If you don't want package-specific code to be responsible for forcing
> a "reasonable" locale where necessary, then what layer do you want to
> be responsible for it?

I want this to be package-specific, but applied only when necessary.

The original complaint was that having to copy this boilerplate code to 
fix reproducibility issues to each new package was a waste of 
maintainers' time and that it should be centralized into some framework, 
and my response to that is to stop copying unnecessary code into 
packages that don't need it.

At best, it does nothing because the package isn't broken, at worst it 
manifests additional bugs while someone is modifying the package to fix 
another problem.

If we are to move this into a framework, then this should take a 
declarative form, like "Rules-Requires-Locale: C.UTF-8", and it should 
be a goal to minimize use of this.

    Simon

Back to linux.debian.devel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Mandatory LC_ALL=C.UTF-8 during package building Gioele Barabucci <gioele@svario.it> - 2024-06-06 08:20 +0200
  Re: Mandatory LC_ALL=C.UTF-8 during package building Luca Boccassi <bluca@debian.org> - 2024-06-06 11:20 +0200
  Re: Mandatory LC_ALL=C.UTF-8 during package building Simon Richter <sjr@debian.org> - 2024-06-06 11:50 +0200
    Re: Mandatory LC_ALL=C.UTF-8 during package building Johannes Schauer Marin Rodrigues <josch@debian.org> - 2024-06-06 12:10 +0200
      Re: Mandatory LC_ALL=C.UTF-8 during package building Johannes Schauer Marin Rodrigues <josch@debian.org> - 2024-06-06 13:00 +0200
        Re: Mandatory LC_ALL=C.UTF-8 during package building Simon Richter <sjr@debian.org> - 2024-06-06 13:40 +0200
        Re: Mandatory LC_ALL=C.UTF-8 during package building Simon McVittie <smcv@debian.org> - 2024-06-06 17:00 +0200
      Re: Mandatory LC_ALL=C.UTF-8 during package building Simon McVittie <smcv@debian.org> - 2024-06-06 16:40 +0200
        Re: Mandatory LC_ALL=C.UTF-8 during package building Jeremy Bícha <jeremy.bicha@canonical.com> - 2024-06-06 16:50 +0200
        Re: Mandatory LC_ALL=C.UTF-8 during package building Guillem Jover <guillem@debian.org> - 2024-06-07 14:40 +0200
          Re: Mandatory LC_ALL=C.UTF-8 during package building Holger Levsen <holger@layer-acht.org> - 2024-06-07 15:40 +0200
            Re: Mandatory LC_ALL=C.UTF-8 during package building Alexandre Detiste <alexandre.detiste@gmail.com> - 2024-06-07 15:50 +0200
              Re: Mandatory LC_ALL=C.UTF-8 during package building Simon McVittie <smcv@debian.org> - 2024-06-07 17:50 +0200
                Re: Mandatory LC_ALL=C.UTF-8 during package building Simon McVittie <smcv@debian.org> - 2024-06-10 17:30 +0200
                Re: Mandatory LC_ALL=C.UTF-8 during package building Simon Richter <sjr@debian.org> - 2024-06-11 11:30 +0200
              Re: Mandatory LC_ALL=C.UTF-8 during package building Guillem Jover <guillem@debian.org> - 2024-07-02 03:50 +0200
                Re: Mandatory LC_ALL=C.UTF-8 during package building Simon McVittie <smcv@debian.org> - 2024-07-02 11:00 +0200
                Re: Mandatory LC_ALL=C.UTF-8 during package building Guillem Jover <guillem@debian.org> - 2024-07-02 14:40 +0200
                Re: Mandatory LC_ALL=C.UTF-8 during package building Alexandre Detiste <alexandre.detiste@gmail.com> - 2024-07-06 13:20 +0200
                Re: Mandatory LC_ALL=C.UTF-8 during package building Guillem Jover <guillem@debian.org> - 2024-08-03 19:20 +0200
                Re: Mandatory LC_ALL=C.UTF-8 during package building Simon McVittie <smcv@debian.org> - 2024-08-03 20:20 +0200
                Re: Mandatory LC_ALL=C.UTF-8 during package building Chris Hofstaedtler <zeha@debian.org> - 2025-02-23 13:50 +0100
          Re: Mandatory LC_ALL=C.UTF-8 during package building Gioele Barabucci <gioele@svario.it> - 2024-06-07 16:20 +0200
          Re: Mandatory LC_ALL=C.UTF-8 during package building Simon McVittie <smcv@debian.org> - 2024-06-07 18:30 +0200
            Re: Mandatory LC_ALL=C.UTF-8 during package building Guillem Jover <guillem@debian.org> - 2024-08-03 15:30 +0200
      Re: Mandatory LC_ALL=C.UTF-8 during package building Marco d'Itri <md@Linux.IT> - 2024-06-06 18:50 +0200
      Re: Mandatory LC_ALL=C.UTF-8 during package building Andrey Rakhmatullin <wrar@debian.org> - 2024-06-06 19:00 +0200
        Re: Mandatory LC_ALL=C.UTF-8 during package building Holger Levsen <holger@layer-acht.org> - 2024-06-07 10:40 +0200
    Locale sanitizer (Was: Mandatory LC_ALL=C.UTF-8 during package  building) Daniel Gröber <dxld@darkboxed.org> - 2024-06-06 14:50 +0200
      Re: Locale sanitizer (Was: Mandatory LC_ALL=C.UTF-8 during package  building) Simon McVittie <smcv@debian.org> - 2024-06-06 15:40 +0200
  Re: Mandatory LC_ALL=C.UTF-8 during package building Adrian Bunk <bunk@debian.org> - 2024-06-18 00:40 +0200

csiph-web