Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.kernel > #54671 > unrolled thread

Bug#830268: linux: please make the build reproducible

Back to article view | Back to linux.debian.kernel

Contents

  Bug#830268: linux: please make the build reproducible Reiner Herrmann <reiner@reiner-h.de> - 2016-07-07 21:00 +0200
    Bug#830268: linux: please make the build reproducible Ian Campbell <ijc@debian.org> - 2016-07-07 21:30 +0200
      Bug#830268: linux: please make the build reproducible Reiner Herrmann <reiner@reiner-h.de> - 2016-07-07 21:40 +0200
    Bug#830268: linux: please make the build reproducible Ben Hutchings <ben@decadent.org.uk> - 2016-08-04 01:20 +0200

#54671 — Bug#830268: linux: please make the build reproducible

[Multipart message — attachments visible in raw view] — view raw

Source: linux
Version: 4.6.3-1
Severity: wishlist
User: reproducible-builds@lists.alioth.debian.org
Usertags: environment randomness
X-Debbugs-Cc: reproducible-builds@lists.alioth.debian.org

Hi!

While working on the "reproducible builds" effort [1], we have noticed
that linux could not be built reproducibly.
Since we started varying the shell used for /bin/sh (bash vs. dash),
linux no longer builds reproducibly.

The following differences are in linux-headers packages:

── ./usr/src/linux-headers-4.6.0-1-686/arch/x86/include/generated/asm/.syscalls_32.h.cmd
 @@ -1 +1 @@
 -cmd_arch/x86/entry/syscalls/../../include/generated/asm/syscalls_32.h := /bin/bash '/build/linux-4.6.3/arch/x86/entry/syscalls/syscalltbl.sh' /build/linux-4.6.3/arch/x86/entry/syscalls/syscall_32.tbl arch/x86/entry/syscalls/../../include/generated/asm/syscalls_32.h
 +cmd_arch/x86/entry/syscalls/../../include/generated/asm/syscalls_32.h := /bin/sh '/build/linux-4.6.3/arch/x86/entry/syscalls/syscalltbl.sh' /build/linux-4.6.3/arch/x86/entry/syscalls/syscall_32.tbl arch/x86/entry/syscalls/../../include/generated/asm/syscalls_32.h
── ./usr/src/linux-headers-4.6.0-1-686/arch/x86/include/generated/uapi/asm/.unistd_32.h.cmd
 @@ -1 +1 @@
 -cmd_arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_32.h := /bin/bash '/build/linux-4.6.3/arch/x86/entry/syscalls/syscallhdr.sh' '/build/linux-4.6.3/arch/x86/entry/syscalls/syscall_32.tbl' 'arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_32.h' 'i386' '' ''
 +cmd_arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_32.h := /bin/sh '/build/linux-4.6.3/arch/x86/entry/syscalls/syscallhdr.sh' '/build/linux-4.6.3/arch/x86/entry/syscalls/syscall_32.tbl' 'arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_32.h' 'i386' '' ''
── ./usr/src/linux-headers-4.6.0-1-686/arch/x86/include/generated/uapi/asm/.unistd_64.h.cmd
 @@ -1 +1 @@
 -cmd_arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_64.h := /bin/bash '/build/linux-4.6.3/arch/x86/entry/syscalls/syscallhdr.sh' '/build/linux-4.6.3/arch/x86/entry/syscalls/syscall_64.tbl' 'arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_64.h' 'common,64' '' ''
 +cmd_arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_64.h := /bin/sh '/build/linux-4.6.3/arch/x86/entry/syscalls/syscallhdr.sh' '/build/linux-4.6.3/arch/x86/entry/syscalls/syscall_64.tbl' 'arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_64.h' 'common,64' '' ''
── ./usr/src/linux-headers-4.6.0-1-686/arch/x86/include/generated/uapi/asm/.unistd_x32.h.cmd
 @@ -1 +1 @@
 -cmd_arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_x32.h := /bin/bash '/build/linux-4.6.3/arch/x86/entry/syscalls/syscallhdr.sh' '/build/linux-4.6.3/arch/x86/entry/syscalls/syscall_64.tbl' 'arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_x32.h' 'common,x32' '' '__X32_SYSCALL_BIT'
 +cmd_arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_x32.h := /bin/sh '/build/linux-4.6.3/arch/x86/entry/syscalls/syscallhdr.sh' '/build/linux-4.6.3/arch/x86/entry/syscalls/syscall_64.tbl' 'arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_x32.h' 'common,x32' '' '__X32_SYSCALL_BIT'

Are those .cmd files actually needed in the packages?
They are caused by the SYSTBL and SYSHDR commands if I undertand it
correctly. If they should be kept in the package, this can probably be
solved by setting CONFIG_SHELL to a static value (/bin/bash).

And the linux-source package has differences (randomly orderes elements)
in the file usr/src/linux-source-4.6/tools/usb/usbip/autom4te.cache/requests
(which I think doesn't need to be included either).

Regards,
 Reiner

[1]: https://wiki.debian.org/ReproducibleBuilds

[toc] | [next] | [standalone]

#54672

On Thu, 2016-07-07 at 20:52 +0200, Reiner Herrmann wrote:
> While working on the "reproducible builds" effort [1], we have noticed
> that linux could not be built reproducibly.
> Since we started varying the shell used for /bin/sh (bash vs. dash),
> linux no longer builds reproducibly.

OOI what is the motivation for varying the build environment in this
way?

Obviously a package built with $SHELL should build reproducibly with
the same $SHELL no matter which $SHELL is chosen so long as it is
consistent, but from the diff that doesn't seem to be what the goal is
here, but rather to build with $SHELL_A and then rebuild with a
different $SHELL_B.

I thought part of the reproducible builds effort included ensuring a
reproducible build environment too (through .buildinfo etc). Is
changing the shell different to changing the compiler or some library
build dep?

Ian.

[toc] | [prev] | [next] | [standalone]

#54673

[Multipart message — attachments visible in raw view] — view raw

On Thu, Jul 07, 2016 at 08:22:24PM +0100, Ian Campbell wrote:
> On Thu, 2016-07-07 at 20:52 +0200, Reiner Herrmann wrote:
> > While working on the "reproducible builds" effort [1], we have noticed
> > that linux could not be built reproducibly.
> > Since we started varying the shell used for /bin/sh (bash vs. dash),
> > linux no longer builds reproducibly.
> 
> OOI what is the motivation for varying the build environment in this
> way?
> 
> Obviously a package built with $SHELL should build reproducibly with
> the same $SHELL no matter which $SHELL is chosen so long as it is
> consistent, but from the diff that doesn't seem to be what the goal is
> here, but rather to build with $SHELL_A and then rebuild with a
> different $SHELL_B.
> 
> I thought part of the reproducible builds effort included ensuring a
> reproducible build environment too (through .buildinfo etc). Is
> changing the shell different to changing the compiler or some library
> build dep?

Different users have different shells configured, like they have
different locales, different home directories, different file systems etc.
The installed environment is actually the same (dash and bash are
installed), but it is configured differently.
We want packages not only building reproducibly in clean environments
like chroots/VMs etc., but also when a user builds it with plain
dpkg-buildpackage (assuming he has the matching dependencies installed).

[toc] | [prev] | [next] | [standalone]

#54824

[Multipart message — attachments visible in raw view] — view raw

On Thu, 2016-07-07 at 20:52 +0200, Reiner Herrmann wrote:
> Source: linux
> Version: 4.6.3-1
> Severity: wishlist
> User: reproducible-builds@lists.alioth.debian.org
> Usertags: environment randomness
> X-Debbugs-Cc: reproducible-builds@lists.alioth.debian.org
> 
> Hi!
> 
> While working on the "reproducible builds" effort [1], we have noticed
> that linux could not be built reproducibly.
> Since we started varying the shell used for /bin/sh (bash vs. dash),
> linux no longer builds reproducibly.
> 
> The following differences are in linux-headers packages:
> 
> ── ./usr/src/linux-headers-4.6.0-1-686/arch/x86/include/generated/asm/.syscalls_32.h.cmd
>  @@ -1 +1 @@
>  -cmd_arch/x86/entry/syscalls/../../include/generated/asm/syscalls_32.h := /bin/bash '/build/linux-4.6.3/arch/x86/entry/syscalls/syscalltbl.sh' /build/linux-4.6.3/arch/x86/entry/syscalls/syscall_32.tbl arch/x86/entry/syscalls/../../include/generated/asm/syscalls_32.h
>  +cmd_arch/x86/entry/syscalls/../../include/generated/asm/syscalls_32.h := /bin/sh '/build/linux-4.6.3/arch/x86/entry/syscalls/syscalltbl.sh' /build/linux-4.6.3/arch/x86/entry/syscalls/syscall_32.tbl arch/x86/entry/syscalls/../../include/generated/asm/syscalls_32.h
> ── ./usr/src/linux-headers-4.6.0-1-686/arch/x86/include/generated/uapi/asm/.unistd_32.h.cmd
>  @@ -1 +1 @@
>  -cmd_arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_32.h := /bin/bash '/build/linux-4.6.3/arch/x86/entry/syscalls/syscallhdr.sh' '/build/linux-4.6.3/arch/x86/entry/syscalls/syscall_32.tbl' 'arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_32.h' 'i386' '' ''
>  +cmd_arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_32.h := /bin/sh '/build/linux-4.6.3/arch/x86/entry/syscalls/syscallhdr.sh' '/build/linux-4.6.3/arch/x86/entry/syscalls/syscall_32.tbl' 'arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_32.h' 'i386' '' ''
> ── ./usr/src/linux-headers-4.6.0-1-686/arch/x86/include/generated/uapi/asm/.unistd_64.h.cmd
>  @@ -1 +1 @@
>  -cmd_arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_64.h := /bin/bash '/build/linux-4.6.3/arch/x86/entry/syscalls/syscallhdr.sh' '/build/linux-4.6.3/arch/x86/entry/syscalls/syscall_64.tbl' 'arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_64.h' 'common,64' '' ''
>  +cmd_arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_64.h := /bin/sh '/build/linux-4.6.3/arch/x86/entry/syscalls/syscallhdr.sh' '/build/linux-4.6.3/arch/x86/entry/syscalls/syscall_64.tbl' 'arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_64.h' 'common,64' '' ''
> ── ./usr/src/linux-headers-4.6.0-1-686/arch/x86/include/generated/uapi/asm/.unistd_x32.h.cmd
>  @@ -1 +1 @@
>  -cmd_arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_x32.h := /bin/bash '/build/linux-4.6.3/arch/x86/entry/syscalls/syscallhdr.sh' '/build/linux-4.6.3/arch/x86/entry/syscalls/syscall_64.tbl' 'arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_x32.h' 'common,x32' '' '__X32_SYSCALL_BIT'
>  +cmd_arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_x32.h := /bin/sh '/build/linux-4.6.3/arch/x86/entry/syscalls/syscallhdr.sh' '/build/linux-4.6.3/arch/x86/entry/syscalls/syscall_64.tbl' 'arch/x86/entry/syscalls/../../include/generated/uapi/asm/unistd_x32.h' 'common,x32' '' '__X32_SYSCALL_BIT'
> 
> Are those .cmd files actually needed in the packages?

I don't think so, but I'm not certain.

> They are caused by the SYSTBL and SYSHDR commands if I undertand it
> correctly. If they should be kept in the package, this can probably be
> solved by setting CONFIG_SHELL to a static value (/bin/bash).
> 
> And the linux-source package has differences (randomly orderes elements)
> in the file usr/src/linux-source-4.6/tools/usb/usbip/autom4te.cache/requests
> (which I think doesn't need to be included either).

It certainly doesn't.  We try to build everything in suubdirectories of
debian/build/ and never modify any files outside of the debian/
directory, so linux-source-<version> should be clean   Unfortunately we
need to run autotools for the usbip userland which creates lots of
files in the source directory.  We'll have to make a temporary copy of
the source.

Ben.

-- 

Ben Hutchings
Sturgeon's Law: Ninety percent of everything is crap.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.kernel

csiph-web