Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.bugs.dist > #842844 > unrolled thread

Bug#844431: Revised patch: seeking seconds

Back to article view | Back to linux.debian.bugs.dist

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Bug#844431: Revised patch: seeking seconds Sean Whitton <spwhitton@spwhitton.name> - 2017-08-12 20:30 +0200
    Bug#844431: Revised patch: seeking seconds Holger Levsen <holger@layer-acht.org> - 2017-08-12 21:00 +0200
    Bug#844431: Revised patch: seeking seconds Ondrej Novy <novy@ondrej.org> - 2017-08-12 21:00 +0200
    Bug#844431: Revised patch: seeking seconds Russ Allbery <rra@debian.org> - 2017-08-12 21:30 +0200
    Bug#844431: Revised patch: seeking seconds Ximin Luo <infinity0@debian.org> - 2017-08-12 22:10 +0200
      Bug#844431: Revised patch: seeking seconds Russ Allbery <rra@debian.org> - 2017-08-12 22:30 +0200
        Bug#844431: Revised patch: seeking seconds Holger Levsen <holger@layer-acht.org> - 2017-08-12 22:40 +0200
    Bug#844431: Revised patch: seeking seconds Johannes Schauer <josch@debian.org> - 2017-08-12 22:50 +0200
      Bug#844431: Revised patch: seeking seconds Russ Allbery <rra@debian.org> - 2017-08-12 23:10 +0200
        Bug#844431: Revised patch: seeking seconds Sean Whitton <spwhitton@spwhitton.name> - 2017-08-13 00:40 +0200
          Bug#844431: Revised patch: seeking seconds Ximin Luo <infinity0@debian.org> - 2017-08-13 01:20 +0200
            Bug#844431: Revised patch: seeking seconds Sean Whitton <spwhitton@spwhitton.name> - 2017-08-13 14:30 +0200
          Bug#844431: Revised patch: seeking seconds Holger Levsen <holger@layer-acht.org> - 2017-08-13 15:40 +0200
          Bug#844431: Revised patch: seeking seconds gregor herrmann <gregoa@debian.org> - 2017-08-13 17:00 +0200
          Bug#844431: Revised patch: seeking seconds Adrian Bunk <bunk@debian.org> - 2017-08-15 20:10 +0200
            Bug#844431: Revised patch: seeking seconds Holger Levsen <holger@layer-acht.org> - 2017-08-15 22:10 +0200
          Bug#844431: Revised patch: Oppose Adrian Bunk <bunk@stusta.de> - 2017-08-16 17:40 +0200
            Bug#844431: Revised patch: Oppose Russ Allbery <rra@debian.org> - 2017-08-16 18:40 +0200
              Bug#844431: Revised patch: Oppose Adrian Bunk <bunk@debian.org> - 2017-08-16 20:30 +0200
                Bug#844431: Revised patch: Oppose Russ Allbery <rra@debian.org> - 2017-08-16 20:40 +0200
              Bug#844431: Revised patch: Oppose Bill Allombert <ballombe@debian.org> - 2017-08-16 20:50 +0200
                Bug#844431: Revised patch: Oppose Russ Allbery <rra@debian.org> - 2017-08-16 21:20 +0200
                  Bug#844431: Revised patch: Oppose Bill Allombert <ballombe@debian.org> - 2017-08-16 23:50 +0200
                    Bug#844431: Revised patch: Oppose Russ Allbery <rra@debian.org> - 2017-08-17 00:10 +0200
                  Bug#844431: Revised patch: Oppose Ximin Luo <infinity0@debian.org> - 2017-08-21 16:30 +0200
                Bug#844431: Revised patch: Oppose Don Armstrong <don@debian.org> - 2017-08-16 21:40 +0200
                  Bug#844431: Revised patch: Oppose Bill Allombert <ballombe@debian.org> - 2017-08-17 00:00 +0200
    Bug#844431: Revised patch: seeking seconds Adrian Bunk <bunk@debian.org> - 2017-08-15 20:00 +0200
      Bug#844431: Revised patch: seeking seconds Russ Allbery <rra@debian.org> - 2017-08-15 21:00 +0200
        Bug#844431: Revised patch: seeking seconds Adrian Bunk <bunk@debian.org> - 2017-08-15 21:20 +0200
          Bug#844431: Revised patch: seeking seconds Holger Levsen <holger@layer-acht.org> - 2017-08-15 22:00 +0200
            Bug#844431: Revised patch: seeking seconds Bill Allombert <ballombe@debian.org> - 2017-08-15 23:20 +0200
              Bug#844431: Revised patch: seeking seconds Ximin Luo <infinity0@debian.org> - 2017-08-16 18:40 +0200
          Bug#844431: Revised patch: seeking seconds Russ Allbery <rra@debian.org> - 2017-08-15 22:10 +0200
            Bug#844431: Revised patch: seeking seconds Sean Whitton <spwhitton@spwhitton.name> - 2017-08-15 22:20 +0200
            Bug#844431: Revised patch: seeking seconds Adrian Bunk <bunk@debian.org> - 2017-08-15 23:10 +0200
              Bug#844431: Revised patch: seeking seconds Russ Allbery <rra@debian.org> - 2017-08-15 23:40 +0200
              Bug#844431: Revised patch: seeking seconds Sean Whitton <spwhitton@spwhitton.name> - 2017-08-16 01:10 +0200
                Bug#844431: Revised patch: seeking seconds Adrian Bunk <bunk@debian.org> - 2017-08-16 07:40 +0200
              Bug#844431: Revised patch: seeking seconds Mattia Rizzolo <mattia@debian.org> - 2017-08-16 12:30 +0200
                Bug#844431: Revised patch: seeking seconds Adrian Bunk <bunk@debian.org> - 2017-08-16 13:30 +0200
                  Bug#844431: Revised patch: seeking seconds Ximin Luo <infinity0@debian.org> - 2017-08-16 13:50 +0200
                    Bug#844431: Revised patch: seeking seconds Adrian Bunk <bunk@debian.org> - 2017-08-16 17:10 +0200
                      Bug#844431: Revised patch: seeking seconds Ximin Luo <infinity0@debian.org> - 2017-08-16 17:50 +0200
                        Bug#844431: Revised patch: seeking seconds Adrian Bunk <bunk@debian.org> - 2017-08-16 18:20 +0200
                          Bug#844431: Revised patch: seeking seconds Ximin Luo <infinity0@debian.org> - 2017-08-16 18:40 +0200
                    Bug#844431: Revised patch: seeking seconds Russ Allbery <rra@debian.org> - 2017-08-16 18:50 +0200
                      Bug#844431: Revised patch: seeking seconds Ximin Luo <infinity0@debian.org> - 2017-08-16 20:30 +0200
            Bug#844431: Revised patch: seeking seconds Bill Allombert <ballombe@debian.org> - 2017-08-15 23:40 +0200
              Bug#844431: Revised patch: seeking seconds Ximin Luo <infinity0@debian.org> - 2017-08-16 13:00 +0200
              Bug#844431: Revised patch: seeking seconds Russ Allbery <rra@debian.org> - 2017-08-16 18:40 +0200
                Bug#844431: Revised patch: seeking seconds Bill Allombert <ballombe@debian.org> - 2017-08-16 20:00 +0200
                  Bug#844431: Revised patch: seeking seconds Russ Allbery <rra@debian.org> - 2017-08-16 20:00 +0200
                  Bug#844431: Revised patch: seeking seconds Chris Lamb <lamby@debian.org> - 2017-08-17 02:50 +0200
                    Bug#844431: Revised patch: seeking seconds Bill Allombert <ballombe@debian.org> - 2017-08-20 21:30 +0200
                      Bug#844431: Revised patch: seeking seconds Chris Lamb <lamby@debian.org> - 2017-08-21 02:40 +0200
        Bug#872288: debian-policy: document .buildinfo files Holger Levsen <holger@layer-acht.org> - 2017-08-15 21:50 +0200

Page 1 of 3  [1] 2 3  Next page →

#842844 — Bug#844431: Revised patch: seeking seconds

[Multipart message — attachments visible in raw view] — view raw

control: tag -1 +patch

This patch incorporates the feedback given on the proposal I sent
yesterday, both in this bug and in person from Russ and Holger (thank
you to all).

I am seeking formal seconds for this patch, from any DD.

In particular:

- for now, we only require reproducibility when the set of environment
  variable values set is exactly the same

  This is because

  - the reproducible builds team aren't yet totally clear on the
    variables that they think may be allowed to vary

  - we should wait until .buildinfo is properly documented in policy,
    and then we can refer to that file

- we don't require reproducibility when build paths vary

  This is because

  - since there is not a consensus on whether we should require this,
    and there is strong consensus on the requirement of reproducibility
    if the path does /not/ vary, this issue should not block this change.
    We should open a separate bug against debian-policy

diff --git a/policy/ch-source.rst b/policy/ch-source.rst
index 127b125..cc4b020 100644
--- a/policy/ch-source.rst
+++ b/policy/ch-source.rst
@@ -661,6 +661,22 @@ particularly complex or unintuitive source layout or build system (for
 example, a package that builds the same source multiple times to
 generate different binary packages).
 
+Reproducibility
+---------------
+
+Packages should build reproducibly, which for the purposes of this
+document [#]_ means that given
+
+- a version of a source package unpacked at a given path;
+- a set of versions of installed build dependencies;
+- a set of environment variable values; and
+- a build architecture,
+
+repeatedly building the source package on any machine of the same
+architecture with those versions of the build dependencies installed
+and exactly those environment variable values set will produce
+bit-for-bit identical binary packages.
+
 .. [#]
    See the file ``upgrading-checklist`` for information about policy
    which has changed between different versions of this document.
@@ -790,3 +806,7 @@ generate different binary packages).
    often creates either static linking or shared library conflicts, and,
    most importantly, increases the difficulty of handling security
    vulnerabilities in the duplicated code.
+
+.. [#]
+   This is Debian's precisification of the `reproducible-builds.org
+   definition <https://reproducible-builds.org/docs/definition/>`_.

-- 
Sean Whitton

[toc] | [next] | [standalone]

#842849

[Multipart message — attachments visible in raw view] — view raw

On Sat, Aug 12, 2017 at 11:23:14AM -0700, Sean Whitton wrote:
> I am seeking formal seconds for this patch, from any DD.
> 
> In particular:
> 
> - for now, we only require reproducibility when the set of environment
>   variable values set is exactly the same
> 
>   This is because
> 
>   - the reproducible builds team aren't yet totally clear on the
>     variables that they think may be allowed to vary
> 
>   - we should wait until .buildinfo is properly documented in policy,
>     and then we can refer to that file
> 
> - we don't require reproducibility when build paths vary
> 
>   This is because
> 
>   - since there is not a consensus on whether we should require this,
>     and there is strong consensus on the requirement of reproducibility
>     if the path does /not/ vary, this issue should not block this change.
>     We should open a separate bug against debian-policy
> 
> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index 127b125..cc4b020 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -661,6 +661,22 @@ particularly complex or unintuitive source layout or build system (for
>  example, a package that builds the same source multiple times to
>  generate different binary packages).
>  
> +Reproducibility
> +---------------
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values; and
> +- a build architecture,
> +
> +repeatedly building the source package on any machine of the same
> +architecture with those versions of the build dependencies installed
> +and exactly those environment variable values set will produce
> +bit-for-bit identical binary packages.
> +
>  .. [#]
>     See the file ``upgrading-checklist`` for information about policy
>     which has changed between different versions of this document.
> @@ -790,3 +806,7 @@ generate different binary packages).
>     often creates either static linking or shared library conflicts, and,
>     most importantly, increases the difficulty of handling security
>     vulnerabilities in the duplicated code.
> +
> +.. [#]
> +   This is Debian's precisification of the `reproducible-builds.org
> +   definition <https://reproducible-builds.org/docs/definition/>`_.

very happily seconded, many thanks to everyone who has contributed to this bug
directly or "indirectly" (I'm thinking specifically about Lunar here).


-- 
cheers,
	Holger (who watched http://meetings-archive.debian.net/pub/debian-meetings/2017/debconf17/reproducible-builds-status-update.vp8.webm today and was equally happy when seeing the whole audience agreeing this should be in policy - and the applause after Russ's closing statement was also very very nice…!)

[toc] | [prev] | [next] | [standalone]

#842850

[Multipart message — attachments visible in raw view] — view raw

Hi,

2017-08-12 14:23 GMT-04:00 Sean Whitton <spwhitton@spwhitton.name>:

> control: tag -1 +patch
>
> This patch incorporates the feedback given on the proposal I sent
> yesterday, both in this bug and in person from Russ and Holger (thank
> you to all).
>

seconded, thanks for working on this.

-- 
Best regards
 Ondřej Nový

Email: novy@ondrej.org
PGP: 3D98 3C52 EB85 980C 46A5  6090 3573 1255 9D1E 064B

[toc] | [prev] | [next] | [standalone]

#842854

Sean Whitton <spwhitton@spwhitton.name> writes:

> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index 127b125..cc4b020 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -661,6 +661,22 @@ particularly complex or unintuitive source layout or build system (for
>  example, a package that builds the same source multiple times to
>  generate different binary packages).
>  
> +Reproducibility
> +---------------
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values; and
> +- a build architecture,
> +
> +repeatedly building the source package on any machine of the same
> +architecture with those versions of the build dependencies installed
> +and exactly those environment variable values set will produce
> +bit-for-bit identical binary packages.
> +
>  .. [#]
>     See the file ``upgrading-checklist`` for information about policy
>     which has changed between different versions of this document.
> @@ -790,3 +806,7 @@ generate different binary packages).
>     often creates either static linking or shared library conflicts, and,
>     most importantly, increases the difficulty of handling security
>     vulnerabilities in the duplicated code.
> +
> +.. [#]
> +   This is Debian's precisification of the `reproducible-builds.org
> +   definition <https://reproducible-builds.org/docs/definition/>`_.

Seconded.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>

[toc] | [prev] | [next] | [standalone]

#842865

Sean Whitton:
> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index 127b125..cc4b020 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -661,6 +661,22 @@ particularly complex or unintuitive source layout or build system (for
>  example, a package that builds the same source multiple times to
>  generate different binary packages).
>  
> +Reproducibility
> +---------------
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values; and
> +- a build architecture,
> +
> +repeatedly building the source package on any machine of the same
> +architecture with those versions of the build dependencies installed
> +and exactly those environment variable values set will produce
> +bit-for-bit identical binary packages.
> +

To echo dkg and others' comments, it would be nice if we could add here:

+Packages are encouraged to produce bit-for-bit identical binary packages even
+if most environment variables and build paths are varied. This is technically
+more difficult at the time of writing, but it is intended that this stricter
+definition would replace the above one, when appropriate in the future.

If this type of "intent" wording is not appropriate for Policy then disregard what I'm saying, I don't wish to block this patch for this reason.

>  .. [#]
>     See the file ``upgrading-checklist`` for information about policy
>     which has changed between different versions of this document.
> @@ -790,3 +806,7 @@ generate different binary packages).
>     often creates either static linking or shared library conflicts, and,
>     most importantly, increases the difficulty of handling security
>     vulnerabilities in the duplicated code.
> +
> +.. [#]
> +   This is Debian's precisification of the `reproducible-builds.org
> +   definition <https://reproducible-builds.org/docs/definition/>`_.
> 

"precisification" -> "more precise version"

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

[toc] | [prev] | [next] | [standalone]

#842872

Ximin Luo <infinity0@debian.org> writes:

> To echo dkg and others' comments, it would be nice if we could add here:

> +Packages are encouraged to produce bit-for-bit identical binary packages even
> +if most environment variables and build paths are varied. This is technically
> +more difficult at the time of writing, but it is intended that this stricter
> +definition would replace the above one, when appropriate in the future.

> If this type of "intent" wording is not appropriate for Policy then
> disregard what I'm saying, I don't wish to block this patch for this
> reason.

Oh, that's a good way to capture that.  This seems fine to me, and I have
no objections to adding this advice.  Seconded the original with or
without this addition.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>

[toc] | [prev] | [next] | [standalone]

#842873

[Multipart message — attachments visible in raw view] — view raw

On Sat, Aug 12, 2017 at 01:18:23PM -0700, Russ Allbery wrote:
> > +Packages are encouraged to produce bit-for-bit identical binary packages even
> > +if most environment variables and build paths are varied. This is technically
> > +more difficult at the time of writing, but it is intended that this stricter
> > +definition would replace the above one, when appropriate in the future.
> 
> > If this type of "intent" wording is not appropriate for Policy then
> > disregard what I'm saying, I don't wish to block this patch for this
> > reason.
> 
> Oh, that's a good way to capture that.  This seems fine to me, and I have
> no objections to adding this advice.  Seconded the original with or
> without this addition.
 
I'm also seconding the original with or without this addition.


-- 
cheers,
	Holger

[toc] | [prev] | [next] | [standalone]

#842877

[Multipart message — attachments visible in raw view] — view raw

Hi,

Quoting Sean Whitton (2017-08-13 03:23:14)
> +Reproducibility
> +---------------
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values; and
> +- a build architecture,

Policy §4.9 defines "build architecture" in the context of dpkg-architecture
already and I think what you mean here is either "host architecture" or at
least "build and host architecture" or you need to mention that you are only
talking about native builds where build and host architecture are equal.

Thanks!

cheers, josch

[toc] | [prev] | [next] | [standalone]

#842886

Johannes Schauer <josch@debian.org> writes:

> Policy §4.9 defines "build architecture" in the context of
> dpkg-architecture already and I think what you mean here is either "host
> architecture" or at least "build and host architecture" or you need to
> mention that you are only talking about native builds where build and
> host architecture are equal.

I suspect we want to say build and host architecture for right now.
(Maybe we can later aspire to making the build architecture not matter.)

Thanks, good catch!

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>

[toc] | [prev] | [next] | [standalone]

#842904

[Multipart message — attachments visible in raw view] — view raw

Hello,

On Sat, Aug 12 2017, Russ Allbery wrote:

> I suspect we want to say build and host architecture for right now.
> (Maybe we can later aspire to making the build architecture not
> matter.)

On Sat, Aug 12 2017, Ximin Luo wrote:

> To echo dkg and others' comments, it would be nice if we could add
> here:
>
> +Packages are encouraged to produce bit-for-bit identical binary
> packages even +if most environment variables and build paths are
> varied. This is technically +more difficult at the time of writing,
> but it is intended that this stricter +definition would replace the
> above one, when appropriate in the future.

Here is an updated patch addressing these.  I reworded it to use
'recommended' and changed the tone to better suit policy.

Thank you Ximin, Russ and Johannes!

> "precisification" -> "more precise version"

Our definition is not actually a /version/ of the
reproducible-builds.org definition -- that would imply that our
definition could replace the reproducible-builds.org definition, like
upgrading a package.

'precisification' means roughly "filling out the missing specification
when it is appropriate to fill it out", which is what the r-p.org
definition instructs distributors to do.

diff --git a/policy/ch-source.rst b/policy/ch-source.rst
index 127b125..6e32870 100644
--- a/policy/ch-source.rst
+++ b/policy/ch-source.rst
@@ -661,6 +661,28 @@ particularly complex or unintuitive source layout or build system (for
 example, a package that builds the same source multiple times to
 generate different binary packages).
 
+Reproducibility
+---------------
+
+Packages should build reproducibly, which for the purposes of this
+document [#]_ means that given
+
+- a version of a source package unpacked at a given path;
+- a set of versions of installed build dependencies;
+- a set of environment variable values;
+- a build architecture; and
+- a host architecture,
+
+repeatedly building the source package for the build architecture on
+any machine of the host architecture with those versions of the build
+dependencies installed and exactly those environment variable values
+set will produce bit-for-bit identical binary packages.
+
+It is recommended that packages produce bit-for-bit identical binaries
+even if most environment variables and build paths are varied.  It is
+intended for this stricter standard to replace the above when it is
+easier for packages to meet it.
+
 .. [#]
    See the file ``upgrading-checklist`` for information about policy
    which has changed between different versions of this document.
@@ -790,3 +812,7 @@ generate different binary packages).
    often creates either static linking or shared library conflicts, and,
    most importantly, increases the difficulty of handling security
    vulnerabilities in the duplicated code.
+
+.. [#]
+   This is Debian's precisification of the `reproducible-builds.org
+   definition <https://reproducible-builds.org/docs/definition/>`_.

-- 
Sean Whitton

[toc] | [prev] | [next] | [standalone]

#842906

Sean Whitton:
> [..]
> 
> Here is an updated patch addressing these.  I reworded it to use
> 'recommended' and changed the tone to better suit policy.
> 
> Thank you Ximin, Russ and Johannes!
> 
>> "precisification" -> "more precise version"
> 
> Our definition is not actually a /version/ of the
> reproducible-builds.org definition -- that would imply that our
> definition could replace the reproducible-builds.org definition, like
> upgrading a package.
> 
> 'precisification' means roughly "filling out the missing specification
> when it is appropriate to fill it out", which is what the r-p.org
> definition instructs distributors to do.
> 
> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index 127b125..6e32870 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -661,6 +661,28 @@ particularly complex or unintuitive source layout or build system (for
>  example, a package that builds the same source multiple times to
>  generate different binary packages).
>  
> +Reproducibility
> +---------------
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values;
> +- a build architecture; and
> +- a host architecture,
> +
> +repeatedly building the source package for the build architecture on
> +any machine of the host architecture with those versions of the build
> +dependencies installed and exactly those environment variable values
> +set will produce bit-for-bit identical binary packages.
> +
> +It is recommended that packages produce bit-for-bit identical binaries
> +even if most environment variables and build paths are varied.  It is
> +intended for this stricter standard to replace the above when it is
> +easier for packages to meet it.
> +
>  .. [#]
>     See the file ``upgrading-checklist`` for information about policy
>     which has changed between different versions of this document.
> @@ -790,3 +812,7 @@ generate different binary packages).
>     often creates either static linking or shared library conflicts, and,
>     most importantly, increases the difficulty of handling security
>     vulnerabilities in the duplicated code.
> +
> +.. [#]
> +   This is Debian's precisification of the `reproducible-builds.org
> +   definition <https://reproducible-builds.org/docs/definition/>`_.
> 
> 

Thanks! Seconded.

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

[toc] | [prev] | [next] | [standalone]

#842977

On Sat, Aug 12 2017, Ximin Luo wrote:

> Thanks! Seconded.

Just to be clear, we are waiting on one more second for the version
that refers to build and target architecture.

-- 
Sean Whitton

[toc] | [prev] | [next] | [standalone]

#843004

[Multipart message — attachments visible in raw view] — view raw

On Sat, Aug 12, 2017 at 03:34:35PM -0700, Sean Whitton wrote:
> Here is an updated patch addressing these.  I reworded it to use
> 'recommended' and changed the tone to better suit policy.
> 
> Thank you Ximin, Russ and Johannes!
> 
> > "precisification" -> "more precise version"
> 
> Our definition is not actually a /version/ of the
> reproducible-builds.org definition -- that would imply that our
> definition could replace the reproducible-builds.org definition, like
> upgrading a package.
> 
> 'precisification' means roughly "filling out the missing specification
> when it is appropriate to fill it out", which is what the r-p.org
> definition instructs distributors to do.
> 
> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index 127b125..6e32870 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -661,6 +661,28 @@ particularly complex or unintuitive source layout or build system (for
>  example, a package that builds the same source multiple times to
>  generate different binary packages).
>  
> +Reproducibility
> +---------------
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values;
> +- a build architecture; and
> +- a host architecture,
> +
> +repeatedly building the source package for the build architecture on
> +any machine of the host architecture with those versions of the build
> +dependencies installed and exactly those environment variable values
> +set will produce bit-for-bit identical binary packages.
> +
> +It is recommended that packages produce bit-for-bit identical binaries
> +even if most environment variables and build paths are varied.  It is
> +intended for this stricter standard to replace the above when it is
> +easier for packages to meet it.
> +
>  .. [#]
>     See the file ``upgrading-checklist`` for information about policy
>     which has changed between different versions of this document.
> @@ -790,3 +812,7 @@ generate different binary packages).
>     often creates either static linking or shared library conflicts, and,
>     most importantly, increases the difficulty of handling security
>     vulnerabilities in the duplicated code.
> +
> +.. [#]
> +   This is Debian's precisification of the `reproducible-builds.org
> +   definition <https://reproducible-builds.org/docs/definition/>`_.

seconded & thanks for these improvements!


-- 
cheers,
	Holger

[toc] | [prev] | [next] | [standalone]

#843020

[Multipart message — attachments visible in raw view] — view raw

On Sat, 12 Aug 2017 15:34:35 -0700, Sean Whitton wrote:

> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index 127b125..6e32870 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -661,6 +661,28 @@ particularly complex or unintuitive source layout or build system (for
>  example, a package that builds the same source multiple times to
>  generate different binary packages).
>  
> +Reproducibility
> +---------------
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values;
> +- a build architecture; and
> +- a host architecture,
> +
> +repeatedly building the source package for the build architecture on
> +any machine of the host architecture with those versions of the build
> +dependencies installed and exactly those environment variable values
> +set will produce bit-for-bit identical binary packages.
> +
> +It is recommended that packages produce bit-for-bit identical binaries
> +even if most environment variables and build paths are varied.  It is
> +intended for this stricter standard to replace the above when it is
> +easier for packages to meet it.
> +
>  .. [#]
>     See the file ``upgrading-checklist`` for information about policy
>     which has changed between different versions of this document.
> @@ -790,3 +812,7 @@ generate different binary packages).
>     often creates either static linking or shared library conflicts, and,
>     most importantly, increases the difficulty of handling security
>     vulnerabilities in the duplicated code.
> +
> +.. [#]
> +   This is Debian's precisification of the `reproducible-builds.org
> +   definition <https://reproducible-builds.org/docs/definition/>`_.


Seconded.

Thanks to everyone for their work on this.


Cheers,
gregor

-- 
 .''`.  https://info.comodo.priv.at/ - Debian Developer https://www.debian.org
 : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D  85FA BB3A 6801 8649 AA06
 `. `'  Member of VIBE!AT & SPI, fellow of the Free Software Foundation Europe
   `-   

[toc] | [prev] | [next] | [standalone]

#843392

On Sat, Aug 12, 2017 at 03:34:35PM -0700, Sean Whitton wrote:
>...
> +Reproducibility
> +---------------
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values;
> +- a build architecture; and
> +- a host architecture,
>...

Is identical building on any kernel required (and tested)?

Examples:

A self-compiled kernel with CONFIG_IPV6=n

Imagine the next time Linus changes the kernel versioning,
he chooses <year>.<month>.<revision>
Will every reproducible package in buster build identical on the
bullseye+1 kernel 2022.11.321 ? [1]

> Sean Whitton

cu
Adrian

[1] the wheezy LTS updates are now built on buildds running stretch
    kernels, and in buster we will have the similar situation that
    nearly everyting in the initial release will be built on stretch
    kernels while post-release updates will be built on buster,
    bullseye and bullseye+1 kernels

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

[toc] | [prev] | [next] | [standalone]

#843423

[Multipart message — attachments visible in raw view] — view raw

On Tue, Aug 15, 2017 at 09:05:29PM +0300, Adrian Bunk wrote:
> Is identical building on any kernel required (and tested)?

no and no.

it's only required that the results is reproducible, that is bit by bit identical…
 
> Will every reproducible package in buster build identical on the
> bullseye+1 kernel 2022.11.321 ? [1]

my crystal ball is broken, sorry…
 
> [1] the wheezy LTS updates are now built on buildds running stretch
>     kernels, and in buster we will have the similar situation that
>     nearly everyting in the initial release will be built on stretch
>     kernels while post-release updates will be built on buster,
>     bullseye and bullseye+1 kernels
 
there surely are situations where different kernels (much like different
libraries) will cause variations in the build artifacts. many times it
will not matter and sometimes it will and we don't really have much experience
with that *yet*.

I'm inclined to file a bug report against dpkg to document the kernel 
used in the .buildinfo files and (hereby) am asking the reproducible builds team
for comments / advice on this.

And probably we should amend debian-policy for this too, but I also 
think we'd rather do this via a new bug report at some later point in time.

Thanks, Adrian, for making sure we don't forget (this pretty old aspect).


-- 
cheers,
	Holger

[toc] | [prev] | [next] | [standalone]

#843575 — Bug#844431: Revised patch: Oppose

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On Sat, Aug 12, 2017 at 03:34:35PM -0700, Sean Whitton wrote:
>...
> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index 127b125..6e32870 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -661,6 +661,28 @@ particularly complex or unintuitive source layout or build system (for
>  example, a package that builds the same source multiple times to
>  generate different binary packages).
>  
> +Reproducibility
> +---------------
> +
> +Packages should build reproducibly, which for the purposes of this
> +document [#]_ means that given
> +
> +- a version of a source package unpacked at a given path;
> +- a set of versions of installed build dependencies;
> +- a set of environment variable values;
> +- a build architecture; and
> +- a host architecture,
> +
> +repeatedly building the source package for the build architecture on
> +any machine of the host architecture with those versions of the build
> +dependencies installed and exactly those environment variable values
> +set will produce bit-for-bit identical binary packages.
> +
> +It is recommended that packages produce bit-for-bit identical binaries
> +even if most environment variables and build paths are varied.  It is
> +intended for this stricter standard to replace the above when it is
> +easier for packages to meet it.
> +
>  .. [#]
>     See the file ``upgrading-checklist`` for information about policy
>     which has changed between different versions of this document.
> @@ -790,3 +812,7 @@ generate different binary packages).
>     often creates either static linking or shared library conflicts, and,
>     most importantly, increases the difficulty of handling security
>     vulnerabilities in the duplicated code.
> +
> +.. [#]
> +   This is Debian's precisification of the `reproducible-builds.org
> +   definition <https://reproducible-builds.org/docs/definition/>`_.

I hereby oppose the addition of this to policy.

It is not true that this would be "Debian's precisification"
of reproducible builds. 

The definition does not match any past, present or future practice in Debian.

Including the people who want this change to policy, there seems to be 
noone intending to use this definition of reproducibility.

Adding this to policy would do more harm than good.

E.g. tracker.d.o saying "Does not build reproducibly during testing" 
based on a definition of reproducibility that is quite different from 
the official "Debian precisification" would only create confusion.

> Sean Whitton

cu
Adrian

- -- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEOvp1f6xuoR0v9F3wiNJCh6LYmLEFAlmUZAsACgkQiNJCh6LY
mLG9RhAAjr0dgpxSv9lnqM3+4AR3JeWwTaj9J118Efsr4qmSbgK9gE3HE3bL7zXG
OJHE5AqGZidx/Oyw/+TVLq3cHEi+6WfgJcwNzFeRAa7fAv+BKSJJ4T9dhOBYvmfs
YN/BfIhU8j4bQppVFtsduprdxooBx9bHWO/lFzCLl/cZOZ7RPOCya7iXcgEgWuA2
SAo96bcDeL3h5I/qM7fBLcm4Yvca219u8RoD7HqQNcmEI53CKS5qIW1cy0wkNbUy
Pqgovee2GpW7WkgqdG92E770/m2tcxdQQywVf5IeLHiSfJ0VP9dGFOoQCsnXZgvg
4GGstXzTJ2OEKMQ2QK1938Tne0S1WIG5o2zLEzOpHqw11Z9TsRg94CRm0/f/tfNt
ym35/N3qNdjERzozTQckbz4ZKCyLKJU3AIxGOH1U1caIjSNBbWY+nGAu62SzY9fb
IVdmKBkqL+c0MT4AW4yRUjFQ/EZYQNkWrh9USPAlgtWdIfjP4ERJ+60RJcRSgYvz
cJJw8DfDKYTNI6sgu0W++rhv89J4eAFdBKDmBazO5gLnFYBacgrFXW9HvwkxCcSZ
WJUlcuEalDpZrtPKGYO5arQp/vWWqXsVBzZeUphi6UbUjmCw+1M4emJh9Zk41jU3
BeTKcjh/hr0tihUvXhZKAJ85HmSkVLjPqZfY/DNiDecr9q+ZdvQ=
=i6V/
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#843585 — Bug#844431: Revised patch: Oppose

Adrian Bunk <bunk@stusta.de> writes:

> I hereby oppose the addition of this to policy.

> It is not true that this would be "Debian's precisification" of
> reproducible builds.

> The definition does not match any past, present or future practice in
> Debian.

> Including the people who want this change to policy, there seems to be 
> noone intending to use this definition of reproducibility.

> Adding this to policy would do more harm than good.

Let me get the formal part of this out of the way first:

As Policy Editor (a delegated position), based on my read of project
consensus including in-person verification of that consensus at DebConf
17, I am formally declaring that I believe this change has consensus
despite your opposition.  We will therefore include this change in the
next release of Policy.

If you disagree, your choices of action are appealing to the Technical
Committee under section 6.1 of the Constitution (I'm fine with using that
section and letting the TC take a majority vote), or propose a GR under
section 4.1.3.

Okay, now, why I'm taking that stance:

This text is a formalization and simplification of existing practice that
we worked out in conjuction with the reproducible builds team and that
strikes a balance between attempting to enumerate all the causes of
nonreproducibility (which would be quite difficult to do) and providing
some clear guidance to maintainers about what types of output variance
they *don't* have to worry about (since obviously packages can't be
reproducible under all circumstances and in all environments).  The
intention is to set a minimum bar that packages should be trying to meet,
and to lay the groundwork for future work.

This is directly in the center of Policy's normal role of standardizing
and documenting best practice that has been developed elsewhere in the
project.  The project is already comfortable with filing bugs against
packages for being non-reproducible under this criteria of
non-reproducibility, and we already have put significant work into
establishing a baseline and have a firm understanding of how close we're
currently coming to meeting that baseline.  This meets the bar for
maturity of work that we look for in major Policy changes.  As with many
other things in Debian, we hope to improve further later on, and to raise
the Policy bar over time as we develop better tools and better
understanding, but this bar is one that we can start recognizing with
normal-priority bugs right now.

The bug severity was specifically chosen to not kick any packages out of
Debian and to not make reproducible builds mandatory, simply to recognize
them as bugs that we as a project want to see fixed.  This feels like the
right balance to strike at this point.  More work would be required to
make them RC.

The definition is not decoupled from current practice.  It is roughly
equivalent to the information currently captured in *.buildinfo files
while being easily comprehensible to people who haven't studied
*.buildinfo files.  More precision will be possible in the future, but we
don't have to wait for that to set the simpler bar.

On the consensus side, there was a rare opportunity here to get a measure
of consensus from a large section of the project.  Holger specifically
asked for a show of hands and a show of objections (there were none) for
including this standard in Policy, and we had various discussions with
other people over the course of DebConf.  We have a much better read on
project consensus for this than we have for many other things in Debian.

Finally, Policy in no way constrains people from filing bugs or reporting
issues (via whatever means, such as tracker.debian.org) in packages about
things that are not spelled out in Policy.  This is a core principle of
Policy maintenance that we have held to for the more than a decade I've
been involed in Policy maintenance.  Policy is not an exhaustive list of
the possible bugs in packages, and never will be, and Policy will never
prevent people from filing bugs against packages at the severity that they
think is appropriate.  The definition of reproducibility is no exception
to this general rule.

Rather, the general project stance has been that Policy spells out the
things that are less open to debate, and bugs filed on the basis of things
that aren't in Policy are more at the maintainer's discretion (assuming
obvious common sense is applied).  And that's what I would expect for any
bugs filed about reproducible builds failing criteria more strict than
those stated in Policy (such as differing build paths) until such time as
project consensus builds that we want to hold all packages to that
stricter standard regardless of maintainer preference.

To be clear, the above discussion is intended as an explanation for this
decision, not a continuation of debate.  If you disagree with the above,
you should probably address those objections to the Technical Committee; I
feel like I have a pretty complete understanding of the issues here, and
it's highly unlikely that further elaborations or rephrasings of your
current arguments are going to change my mind.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>

[toc] | [prev] | [next] | [standalone]

#843617 — Bug#844431: Revised patch: Oppose

On Wed, Aug 16, 2017 at 09:30:23AM -0700, Russ Allbery wrote:
>...
> This text is a formalization and simplification of existing practice that
> we worked out in conjuction with the reproducible builds team and that
> strikes a balance between attempting to enumerate all the causes of
> nonreproducibility (which would be quite difficult to do) and providing
> some clear guidance to maintainers about what types of output variance
> they *don't* have to worry about (since obviously packages can't be
> reproducible under all circumstances and in all environments).  The
> intention is to set a minimum bar that packages should be trying to meet,
>...

The definition of reproducibility in policy does not match any past, 
present or future practice in Debian.

And no current or currently planned reproducible testing does test
or is intended to test whether packages meet this minimum bar.

> This is directly in the center of Policy's normal role of standardizing
> and documenting best practice that has been developed elsewhere in the
> project.
>...

If it would actually standardize what is considered reproducible
in Debian everything would be fine.

> The definition is not decoupled from current practice.  It is roughly
> equivalent to the information currently captured in *.buildinfo files
> while being easily comprehensible to people who haven't studied
> *.buildinfo files.
>...

2 of the 5 items in policy require changes to .buildinfo, and for a 
third I cannot easily comprehend whether it would require changes to 
.buildinfo since it is unclear what it is supposed to mean:

- a set of environment variable values;

.buildinfo currently records only some environment variables.
If all or different ones are allowed to vary that is a change.
I am actually surprised that the latest set of suggested permitted
variations does not seem to be based on the existing list currently
used for .buildinfo

- a version of a source package unpacked at a given path;

The path is currently not in .buildinfo

- a build architecture;

What is the intended purpose of this, especially what is this supposed 
to output for i386 builds on amd64 kernel?
.buildinfo currently follows dpkg-architecture, and outputs i386.
i386/amd64 kernels is a build variation in the reproducible builds
infrastructure that does result in packages being built differently,
which makes it unclear whether this difference was supposed to be
addressed here.

> Finally, Policy in no way constrains people from filing bugs or reporting
> issues (via whatever means, such as tracker.debian.org) in packages about
> things that are not spelled out in Policy.
>...

https://tracker.debian.org/pkg/hsqldb1.8.0
"Does not build reproducibly during testing"

This statement in tracker is automatically generated based on results 
from the reproducible builds infrastructure.

Is it acceptable to claim in tracker that a package is not reproducible, 
when that package might actually be reproducible based on the definition 
of reproducibility spelled out in Policy? [1]

cu
Adrian

[1] as explained earlier, it is not obvious whether or not this
    specific package is reproducible according to Policy

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

[toc] | [prev] | [next] | [standalone]

#843621 — Bug#844431: Revised patch: Oppose

Just to be completely, 100% clear: I will not be responding further to
this line of argument in this bug.  If you disagree with my decision as a
project delegate, I've spelled out your possible next steps under Debian's
governance process.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>

[toc] | [prev] | [next] | [standalone]

Page 1 of 3  [1] 2 3  Next page →

Back to top | Article view | linux.debian.bugs.dist

csiph-web