Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.kernel > #78636 > unrolled thread

Bug#1033663: linux: reproducible-builds: Embedded build path in various binaries

Back to article view | Back to linux.debian.kernel

Contents

  Bug#1033663: linux: reproducible-builds: Embedded build path in various binaries Vagrant Cascadian <vagrant@reproducible-builds.org> - 2023-03-29 19:10 +0200
    Bug#1033663: linux: reproducible-builds: Embedded build path in various binaries Vagrant Cascadian <vagrant@reproducible-builds.org> - 2023-07-14 09:00 +0200
      Bug#1033663: linux: reproducible-builds: Embedded build path in various binaries Bastian Blank <waldi@debian.org> - 2023-07-14 09:40 +0200
    Bug#1033663: linux: reproducible-builds: Embedded build path in various binaries Ben Hutchings <ben@decadent.org.uk> - 2023-07-14 15:10 +0200
      Bug#1033663: linux: reproducible-builds: Embedded build path in various binaries Vagrant Cascadian <vagrant@reproducible-builds.org> - 2023-07-14 16:30 +0200
    Processed: "Debian Bug Tracking System" <owner@bugs.debian.org> - 2024-03-12 23:20 +0100
    Bug#1033663:  James Addison <jay@jp-hosting.net> - 2024-03-12 23:20 +0100
    Bug#1033663: marked as done (linux: reproducible-builds: Embedded  build path in various binaries) "Debian Bug Tracking System" <owner@bugs.debian.org> - 2024-10-06 19:20 +0200

#78636 — Bug#1033663: linux: reproducible-builds: Embedded build path in various binaries

[Multipart message — attachments visible in raw view] — view raw

Source: linux
Severity: normal
Tags: patch
User: reproducible-builds@lists.alioth.debian.org
Usertags: buildpath
X-Debbugs-Cc: reproducible-bugs@lists.alioth.debian.org

The build path is embedded in various binaries:

  https://tests.reproducible-builds.org/debian/rb-pkg/unstable/i386/diffoscope-results/linux.html

  /usr/sbin/bpftool

  /build/1st/linux-6.1.20/tools/lib/bpf/libbpf.c:805
  vs.
  /build/2/linux-6.1.20/2nd/tools/lib/bpf/libbpf.c:805  

This *seems* like a regression from earlier versions, though I have not
tracked down when the tools started embedding the build paths...

The attached patches fix this for some of the binary packages and
corresponding dbgsym packages (bpftool, hyperv-daemons, *cpupower*) by
explicitly passing the -ffile-prefix-map argument (to avoid embedding
the build path) in the CFLAGS or EXTRA_CFLAGS variables in
debian/rules.d/.

I unsuccessfully tried applying similar patches for other tools packages
(rtla, perf, usbip) but was not able to convince the tools to take the
passed flags...

It would, of course, be better to figure out a way to get the various
tools to respect the default CFLAGS, as -ffile-prefix-map is included in
the default CFLAGS.

Unfortunately, these patches do not fix all reproducibility issues, but
applying these patches could reduce some of the noise and make it easier
to debug the remaining issues.

Thanks for maintaining linux!

live well,
  vagrant

[toc] | [next] | [standalone]

#79590

[Multipart message — attachments visible in raw view] — view raw

On 2023-03-29, Vagrant Cascadian wrote:
> The build path is embedded in various binaries:
>
>   https://tests.reproducible-builds.org/debian/rb-pkg/unstable/i386/diffoscope-results/linux.html
>
>   /usr/sbin/bpftool
>
>   /build/1st/linux-6.1.20/tools/lib/bpf/libbpf.c:805
>   vs.
>   /build/2/linux-6.1.20/2nd/tools/lib/bpf/libbpf.c:805  
>
> This *seems* like a regression from earlier versions, though I have not
> tracked down when the tools started embedding the build paths...
>
> The attached patches fix this for some of the binary packages and
> corresponding dbgsym packages (bpftool, hyperv-daemons, *cpupower*) by
> explicitly passing the -ffile-prefix-map argument (to avoid embedding
> the build path) in the CFLAGS or EXTRA_CFLAGS variables in
> debian/rules.d/.

Any objections to me merging these patches? Prefer if they remain
separate or squash them into a single patch?


> From 7dcacfb3415c5ec4caea8933dd2c12fb401bea52 Mon Sep 17 00:00:00 2001
> From: Vagrant Cascadian <vagrant@reproducible-builds.org>
> Date: Tue, 28 Mar 2023 14:49:01 -0700
> Subject: [PATCH 1/4] debian/rules.d/tools/bpf/bpftool/Makefile: Pass
>  -ffile-prefix-map via EXTRA_CFLAGS.
>
> ---
>  debian/rules.d/tools/bpf/bpftool/Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/debian/rules.d/tools/bpf/bpftool/Makefile b/debian/rules.d/tools/bpf/bpftool/Makefile
> index 17c27c104..9d3017e42 100644
> --- a/debian/rules.d/tools/bpf/bpftool/Makefile
> +++ b/debian/rules.d/tools/bpf/bpftool/Makefile
> @@ -5,7 +5,7 @@ MAKE_BPFTOOL += prefix=/usr
>  MAKE_BPFTOOL += mandir=/usr/share/man
>  MAKE_BPFTOOL += V=1
>  MAKE_BPFTOOL += ARCH=$(KERNEL_ARCH)
> -MAKE_BPFTOOL += EXTRA_CFLAGS='$(CFLAGS) $(CPPFLAGS)'
> +MAKE_BPFTOOL += EXTRA_CFLAGS='$(CFLAGS) $(CPPFLAGS) -ffile-prefix-map=$(top_srcdir)=.'
>  MAKE_BPFTOOL += EXTRA_LDFLAGS='$(LDFLAGS)'
>  
>  # dynamically linking with libbfd is not allowed in Debian
> -- 
> 2.39.2
>
> From cb11c490a41a0afbdeb00f272fa8a71cc3c69be1 Mon Sep 17 00:00:00 2001
> From: Vagrant Cascadian <vagrant@reproducible-builds.org>
> Date: Tue, 28 Mar 2023 15:48:05 -0700
> Subject: [PATCH 2/4] debian/rules.d/tools/power/cpupower/Makefile: Pass
>  -ffile-prefix-map via MAKE_CPUPOWER CFLAGS.
>
> ---
>  debian/rules.d/tools/power/cpupower/Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/debian/rules.d/tools/power/cpupower/Makefile b/debian/rules.d/tools/power/cpupower/Makefile
> index e4bd5202d..28ccf7004 100644
> --- a/debian/rules.d/tools/power/cpupower/Makefile
> +++ b/debian/rules.d/tools/power/cpupower/Makefile
> @@ -1,6 +1,6 @@
>  include $(top_rulesdir)/Makefile.inc
>  
> -MAKE_CPUPOWER := CFLAGS='$(CFLAGS) $(CPPFLAGS)' LDFLAGS='$(LDFLAGS)' $(MAKE) O=$(CURDIR) CPUFREQ_BENCH=false V=true mandir=/usr/share/man
> +MAKE_CPUPOWER := CFLAGS='$(CFLAGS) -ffile-prefix-map=$(top_srcdir)=. $(CPPFLAGS)' LDFLAGS='$(LDFLAGS)' $(MAKE) O=$(CURDIR) CPUFREQ_BENCH=false V=true mandir=/usr/share/man
>  
>  MAKE_CPUPOWER += DEBUG=$(if $(filter noopt,$(DEB_BUILD_OPTIONS)),true,)
>  
> -- 
> 2.39.2
>
> From 2cf67982526879524548e37df0efac5f8b37e0d3 Mon Sep 17 00:00:00 2001
> From: Vagrant Cascadian <vagrant@reproducible-builds.org>
> Date: Tue, 28 Mar 2023 16:57:01 -0700
> Subject: [PATCH 3/4] debian/rules.d/tools/power/x86/*/Makefile: Add
>  -ffile-prefix-map to CFLAGS.
>
> ---
>  debian/rules.d/tools/power/x86/turbostat/Makefile              | 2 ++
>  debian/rules.d/tools/power/x86/x86_energy_perf_policy/Makefile | 2 ++
>  2 files changed, 4 insertions(+)
>
> diff --git a/debian/rules.d/tools/power/x86/turbostat/Makefile b/debian/rules.d/tools/power/x86/turbostat/Makefile
> index eb5124d3a..019fcbde6 100644
> --- a/debian/rules.d/tools/power/x86/turbostat/Makefile
> +++ b/debian/rules.d/tools/power/x86/turbostat/Makefile
> @@ -7,3 +7,5 @@ include $(top_rulesdir)/Makefile.inc
>  CPPFLAGS += -I"$(top_srcdir)/tools/include" -DMSRHEADER='"$(top_srcdir)/arch/x86/include/asm/msr-index.h"' -DINTEL_FAMILY_HEADER='"$(top_srcdir)/arch/x86/include/asm/intel-family.h"'
>  
>  LDLIBS += -lcap -lrt
> +
> +CFLAGS += -ffile-prefix-map=$(top_srcdir)=.
> diff --git a/debian/rules.d/tools/power/x86/x86_energy_perf_policy/Makefile b/debian/rules.d/tools/power/x86/x86_energy_perf_policy/Makefile
> index b9ec56c89..a86da0f1e 100644
> --- a/debian/rules.d/tools/power/x86/x86_energy_perf_policy/Makefile
> +++ b/debian/rules.d/tools/power/x86/x86_energy_perf_policy/Makefile
> @@ -5,3 +5,5 @@ installdir = /usr/sbin
>  include $(top_rulesdir)/Makefile.inc
>  
>  CPPFLAGS += -I"$(top_srcdir)/tools/include" -DMSRHEADER='"$(top_srcdir)/arch/x86/include/asm/msr-index.h"'
> +
> +CFLAGS += -ffile-prefix-map=$(top_srcdir)=.
> -- 
> 2.39.2
>
> From 36e49dee5adf4ba4b566cb87bea35dd0158f8ff3 Mon Sep 17 00:00:00 2001
> From: Vagrant Cascadian <vagrant@reproducible-builds.org>
> Date: Tue, 28 Mar 2023 21:44:05 -0700
> Subject: [PATCH 4/4] debian/rules.d/tools/hv/Makefile: Add -ffile-prefix-map
>  to CFLAGS.
>
> ---
>  debian/rules.d/tools/hv/Makefile | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/debian/rules.d/tools/hv/Makefile b/debian/rules.d/tools/hv/Makefile
> index 5f00f5513..ed8ed967d 100644
> --- a/debian/rules.d/tools/hv/Makefile
> +++ b/debian/rules.d/tools/hv/Makefile
> @@ -14,4 +14,6 @@ installdir = /usr/sbin
>  
>  include $(top_rulesdir)/Makefile.inc
>  
> +CFLAGS += -ffile-prefix-map=$(top_srcdir)=.
> +
>  endif
> -- 
> 2.39.2

[toc] | [prev] | [next] | [standalone]

#79591

On Thu, Jul 13, 2023 at 11:53:40PM -0700, Vagrant Cascadian wrote:
> Any objections to me merging these patches? Prefer if they remain
> separate or squash them into a single patch?

Hmm, I have to check, but dpkg-buildflags should already provide this
flag and all the modified locations are for userspace tooling.

Maybe you could look into that as well.

Regards,
Bastian

-- 
It is a human characteristic to love little animals, especially if
they're attractive in some way.
		-- McCoy, "The Trouble with Tribbles", stardate 4525.6

[toc] | [prev] | [next] | [standalone]

#79593

[Multipart message — attachments visible in raw view] — view raw

On Wed, 2023-03-29 at 09:58 -0700, Vagrant Cascadian wrote:
> Source: linux
> Severity: normal
> Tags: patch
> User: reproducible-builds@lists.alioth.debian.org
> Usertags: buildpath
> X-Debbugs-Cc: reproducible-bugs@lists.alioth.debian.org
> 
> The build path is embedded in various binaries:
> 
>   https://tests.reproducible-builds.org/debian/rb-pkg/unstable/i386/diffoscope-results/linux.html
> 
>   /usr/sbin/bpftool
> 
>   /build/1st/linux-6.1.20/tools/lib/bpf/libbpf.c:805
>   vs.
>   /build/2/linux-6.1.20/2nd/tools/lib/bpf/libbpf.c:805  
> 
> This *seems* like a regression from earlier versions, though I have not
> tracked down when the tools started embedding the build paths...
[...]

Please see
<https://salsa.debian.org/kernel-team/linux/-/merge_requests/741>.

The embedded paths are mostly fixed by:
<https://salsa.debian.org/kernel-team/linux/-/merge_requests/741/diffs?commit_id=7293fda0eef37c07214d1e0c4e5ff1d1fa3e2e60>.

Perhaps I should finalise that MR even though it doesn't fix
everything?

Ben.

-- 
Ben Hutchings
Hoare's Law of Large Problems:
   Inside every large problem is a small problem struggling to get out.

[toc] | [prev] | [next] | [standalone]

#79597

[Multipart message — attachments visible in raw view] — view raw

On 2023-07-14, Ben Hutchings wrote:
> On Wed, 2023-03-29 at 09:58 -0700, Vagrant Cascadian wrote:
>> The build path is embedded in various binaries:
>> 
>>   https://tests.reproducible-builds.org/debian/rb-pkg/unstable/i386/diffoscope-results/linux.html
>> 
>>   /usr/sbin/bpftool
>> 
>>   /build/1st/linux-6.1.20/tools/lib/bpf/libbpf.c:805
>>   vs.
>>   /build/2/linux-6.1.20/2nd/tools/lib/bpf/libbpf.c:805  
>> 
>> This *seems* like a regression from earlier versions, though I have not
>> tracked down when the tools started embedding the build paths...
> [...]
>
> Please see
> <https://salsa.debian.org/kernel-team/linux/-/merge_requests/741>.
>
> The embedded paths are mostly fixed by:
> <https://salsa.debian.org/kernel-team/linux/-/merge_requests/741/diffs?commit_id=7293fda0eef37c07214d1e0c4e5ff1d1fa3e2e60>.
>
> Perhaps I should finalise that MR even though it doesn't fix
> everything?

Very glad to see that merge request, thanks!

With my reproducible builds hat on, I would welcome partial fixes at
this point, especially for such core packages! It helps to troubleshoot
outstanding issues and also avoids duplication of work.

live well,
  vagrant

[toc] | [prev] | [next] | [standalone]

#82076 — Processed:

Processing control commands:

> severity -1 wishlist
Bug #1033663 [src:linux] linux: reproducible-builds: Embedded build path in various binaries
Severity set to 'wishlist' from 'normal'

-- 
1033663: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1033663
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems

[toc] | [prev] | [next] | [standalone]

#82077 — Bug#1033663:

Control: severity -1 wishlist

Dear Maintainer,

Because Debian builds packages from a fixed build path, neither the 'reprotest'
utility in Salsa-CI, nor the Reproducible Builds team's package test
infrastructure for Debian[1] currently check for equivalent binary package
output from differing source package build paths.

This means that your package will pass current reproducibility tests; however
we believe that source code and/or build steps still embed the build path into
the binary package output, making it more difficult than necessary for
independent consumers to check the integrity of binary packages by recompiling
them themselves.

As a result, this bugreport will remain open and be re-assigned the 'wishlist'
severity[2].

For more information about build paths and how they can affect reproducibility,
please refer to: https://reproducible-builds.org/docs/build-path/

Thanks,
James

[1] - https://tests.reproducible-builds.org/debian/reproducible.html

[2] - https://www.debian.org/Bugs/Developer#severities

[toc] | [prev] | [next] | [standalone]

#84217 — Bug#1033663: marked as done (linux: reproducible-builds: Embedded build path in various binaries)

[Multipart message — attachments visible in raw view] — view raw

Your message dated Sun, 06 Oct 2024 17:10:12 +0000
with message-id <E1sxUlo-00D3Al-Uv@fasolo.debian.org>
and subject line Bug#1033663: fixed in linux 6.11.2-1
has caused the Debian Bug report #1033663,
regarding linux: reproducible-builds: Embedded build path in various binaries
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)


-- 
1033663: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1033663
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.kernel

csiph-web