Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.kernel > #58670 > unrolled thread

Bug#872263: linux-image-4.11.0-1-amd64-dbg: file overwrite error upgrading from stretch-backports

Back to article view | Back to linux.debian.kernel

Contents

  Bug#872263: linux-image-4.11.0-1-amd64-dbg: file overwrite error upgrading from stretch-backports Andreas Beckmann <anbe@debian.org> - 2017-08-15 15:10 +0200
    Bug#872263: linux-image-4.11.0-1-amd64-dbg: file overwrite error upgrading from stretch-backports Ben Hutchings <ben@decadent.org.uk> - 2017-08-16 01:40 +0200
    Bug#872263: [rb-general] Bug#872263: linux-image-4.11.0-1-amd64-dbg: file overwrite error upgrading from stretch-backports Daniel Shahaf <danielsh@apache.org> - 2017-08-16 20:10 +0200
      Bug#872263: [rb-general] Bug#872263: linux-image-4.11.0-1-amd64-dbg: file overwrite error upgrading from stretch-backports Ben Hutchings <ben@decadent.org.uk> - 2017-08-16 23:10 +0200
        Bug#872263: [rb-general] Bug#872263: linux-image-4.11.0-1-amd64-dbg: file overwrite error upgrading from stretch-backports Ximin Luo <infinity0@debian.org> - 2017-09-06 14:00 +0200

#58670 — Bug#872263: linux-image-4.11.0-1-amd64-dbg: file overwrite error upgrading from stretch-backports

[Multipart message — attachments visible in raw view] — view raw

Package: linux-image-4.11.0-1-amd64-dbg
Version: 4.11.6-1
Severity: serious
User: debian-qa@lists.debian.org
Usertags: piuparts
Control: affects -1 + linux-image-amd64-dbg

Hi,

during a test with piuparts I noticed your package fails to upgrade from
'stretch-backports'.
It installed fine in 'stretch-backports', then the upgrade to 'buster' fails
because it tries to overwrite other packages files without declaring a
Breaks+Replaces relation.

See policy 7.6 at
https://www.debian.org/doc/debian-policy/ch-relationships.html#s-replaces

>From the attached log (scroll to the bottom...):

  Selecting previously unselected package linux-image-4.11.0-1-amd64-dbg.
  Preparing to unpack .../linux-image-4.11.0-1-amd64-dbg_4.11.6-1_amd64.deb ...
  Unpacking linux-image-4.11.0-1-amd64-dbg (4.11.6-1) ...
  dpkg: error processing archive /var/cache/apt/archives/linux-image-4.11.0-1-amd64-dbg_4.11.6-1_amd64.deb (--unpack):
   trying to overwrite '/usr/lib/debug/.build-id/25/99e5c063eeb45c3b0068e66a8251a66f313afa.debug', which is also in package linux-image-4.11.0-0.bpo.1-amd64-dbg 4.11.6-1~bpo9+1
  Preparing to unpack .../linux-image-amd64-dbg_4.11+82_amd64.deb ...
  Unpacking linux-image-amd64-dbg (4.11+82) over (4.11+82~bpo9+1) ...
  Errors were encountered while processing:
   /var/cache/apt/archives/linux-image-4.11.0-1-amd64-dbg_4.11.6-1_amd64.deb


cheers,

Andreas

[toc] | [next] | [standalone]

#58672

[Multipart message — attachments visible in raw view] — view raw

On Tue, 2017-08-15 at 14:59 +0200, Andreas Beckmann wrote:
> Package: linux-image-4.11.0-1-amd64-dbg
> Version: 4.11.6-1
> Severity: serious
> User: debian-qa@lists.debian.org
> Usertags: piuparts
> Control: affects -1 + linux-image-amd64-dbg
> 
> Hi,
> 
> during a test with piuparts I noticed your package fails to upgrade from
> 'stretch-backports'.
> It installed fine in 'stretch-backports', then the upgrade to 'buster' fails
> because it tries to overwrite other packages files without declaring a
> Breaks+Replaces relation.
> 
> See policy 7.6 at
> https://www.debian.org/doc/debian-policy/ch-relationships.html#s-replaces
> 
> > From the attached log (scroll to the bottom...):
> 
>   Selecting previously unselected package linux-image-4.11.0-1-amd64-dbg.
>   Preparing to unpack .../linux-image-4.11.0-1-amd64-dbg_4.11.6-1_amd64.deb ...
>   Unpacking linux-image-4.11.0-1-amd64-dbg (4.11.6-1) ...
>   dpkg: error processing archive /var/cache/apt/archives/linux-image-4.11.0-1-amd64-dbg_4.11.6-1_amd64.deb (--unpack):
>    trying to overwrite '/usr/lib/debug/.build-id/25/99e5c063eeb45c3b0068e66a8251a66f313afa.debug', which is also in package linux-image-4.11.0-0.bpo.1-amd64-dbg 4.11.6-1~bpo9+1

This filename is a hash of file contents, so this indicates that at
least one file was identical for the two builds.  Binary
reproducibility has bitten us in the ass!

We only create .build-id links for user-space code (vDSOs), so let's
compare those:

[linux-image-4.11.0-1-amd64-dbg_4.11.6-1_amd64.deb]
lrwxrwxrwx 1 ben ben 47 Jun 20 00:25 foo/usr/lib/debug/.build-id/22/1234d10e2a0342be17ce0b885f6fb4d8ea3330.debug -> ../../lib/modules/4.11.0-1-amd64/vdso/vdso32.so
lrwxrwxrwx 1 ben ben 47 Jun 20 00:25 foo/usr/lib/debug/.build-id/25/99e5c063eeb45c3b0068e66a8251a66f313afa.debug -> ../../lib/modules/4.11.0-1-amd64/vdso/vdso64.so
lrwxrwxrwx 1 ben ben 48 Jun 20 00:25 foo/usr/lib/debug/.build-id/66/ff5d513f55db31f2802e037593555810452e02.debug -> ../../lib/modules/4.11.0-1-amd64/vdso/vdsox32.so
4cdc4e5658ed934b8d3cfe22ddfe4aa1474acfddfc8a18a8a2c282f5bb0431be  foo/usr/lib/debug/lib/modules/4.11.0-1-amd64/vdso/vdso32.so
6e763e4f0677224b323164719495119c6d834a287a18c580a05eb2a3b5a2c1cf  foo/usr/lib/debug/lib/modules/4.11.0-1-amd64/vdso/vdso64.so
b8374d512446748311b333701bca11c5c8ed4c6140579d9c0de4e9d3b1a67821  foo/usr/lib/debug/lib/modules/4.11.0-1-amd64/vdso/vdsox32.so

[linux-image-4.11.0-0.bpo.1-amd64-dbg_4.11.6-1~bpo9+1_amd64.deb]
lrwxrwxrwx 1 ben ben 53 Jul  9 19:22 bar/usr/lib/debug/.build-id/25/99e5c063eeb45c3b0068e66a8251a66f313afa.debug -> ../../lib/modules/4.11.0-0.bpo.1-amd64/vdso/vdso64.so
lrwxrwxrwx 1 ben ben 54 Jul  9 19:22 bar/usr/lib/debug/.build-id/66/ff5d513f55db31f2802e037593555810452e02.debug -> ../../lib/modules/4.11.0-0.bpo.1-amd64/vdso/vdsox32.so
lrwxrwxrwx 1 ben ben 53 Jul  9 19:22 bar/usr/lib/debug/.build-id/6f/cb3e7c92bce418d52f5f5d6710222ae5377723.debug -> ../../lib/modules/4.11.0-0.bpo.1-amd64/vdso/vdso32.so
6dcde68bba7f9939cbc521eb262842011ed512ab473848fe94ea7d2639c6d839  bar/usr/lib/debug/lib/modules/4.11.0-0.bpo.1-amd64/vdso/vdso32.so
6e763e4f0677224b323164719495119c6d834a287a18c580a05eb2a3b5a2c1cf  bar/usr/lib/debug/lib/modules/4.11.0-0.bpo.1-amd64/vdso/vdso64.so
b8374d512446748311b333701bca11c5c8ed4c6140579d9c0de4e9d3b1a67821  bar/usr/lib/debug/lib/modules/4.11.0-0.bpo.1-amd64/vdso/vdsox32.so

So the amd64 vDSO and x32 compat vDSO are identical when built in
unstable and stretch-backports, but the i386 compat vDSO is not (for
some reason the debug filename mapping isn't being applied there).

>   Preparing to unpack .../linux-image-amd64-dbg_4.11+82_amd64.deb ...
>   Unpacking linux-image-amd64-dbg (4.11+82) over (4.11+82~bpo9+1) ...
>   Errors were encountered while processing:
>    /var/cache/apt/archives/linux-image-4.11.0-1-amd64-dbg_4.11.6-1_amd64.deb

We can probably work around this in src:linux by including the kernel
release string in the vDSO, so the backports build will always be
different.

Still, it seems like there is a wider problem here: if the exact same
code is ever built in two unrelated packages then their debug info
packages will conflict even if the regular binary packages don't.

Ben.

-- 
Ben Hutchings
Nothing is ever a complete failure; it can always serve as a bad
example.

[toc] | [prev] | [next] | [standalone]

#58684 — Bug#872263: [rb-general] Bug#872263: linux-image-4.11.0-1-amd64-dbg: file overwrite error upgrading from stretch-backports

Chris Lamb wrote on Wed, 16 Aug 2017 07:54 -0700:
> > Still, it seems like there is a wider problem here: if the exact same
> > code is ever built in two unrelated packages then their debug info
> > packages will conflict even if the regular binary packages don't.
> 
> I've seen this outside of reproducibility where I was shipping the exact
> same binary in the redis-server and redis-sentinel packages (it changes
> behaviour based on argv[0]).
> 
> The -dbgsym packages then conflicted for the same reason.

Stupid question, but why _do_ the packages conflict?  Couldn't the
package manager notice that the file versions that would be installed by
each package are equivalent [= same name, chmod, and bit-by-bit
contents], and keep the file existing so long as _either_ package is
installed?

[toc] | [prev] | [next] | [standalone]

#58685 — Bug#872263: [rb-general] Bug#872263: linux-image-4.11.0-1-amd64-dbg: file overwrite error upgrading from stretch-backports

[Multipart message — attachments visible in raw view] — view raw

On Wed, 2017-08-16 at 17:51 +0000, Daniel Shahaf wrote:
> Chris Lamb wrote on Wed, 16 Aug 2017 07:54 -0700:
> > > Still, it seems like there is a wider problem here: if the exact same
> > > code is ever built in two unrelated packages then their debug info
> > > packages will conflict even if the regular binary packages don't.
> > 
> > I've seen this outside of reproducibility where I was shipping the exact
> > same binary in the redis-server and redis-sentinel packages (it changes
> > behaviour based on argv[0]).
> > 
> > The -dbgsym packages then conflicted for the same reason.
> 
> Stupid question, but why _do_ the packages conflict?  Couldn't the
> package manager notice that the file versions that would be installed by
> each package are equivalent [= same name, chmod, and bit-by-bit
> contents], and keep the file existing so long as _either_ package is
> installed?

In the case of the kernel packages, the identical binaries (vDSOs) are
emebedded in kernel images with different filenames.  The identical
debug info is installed with different filenames.  But the symlinks to
them underneath /usr/lib/debug/.build-id therefore have the same (hash-
based) name and *different* content.

Ben.

-- 
Ben Hutchings
The generation of random numbers is too important to be left to chance.
                                                       - Robert Coveyou

[toc] | [prev] | [next] | [standalone]

#58863 — Bug#872263: [rb-general] Bug#872263: linux-image-4.11.0-1-amd64-dbg: file overwrite error upgrading from stretch-backports

Ben Hutchings:
> On Wed, 2017-08-16 at 17:51 +0000, Daniel Shahaf wrote:
>> Chris Lamb wrote on Wed, 16 Aug 2017 07:54 -0700:
>>>> Still, it seems like there is a wider problem here: if the exact same
>>>> code is ever built in two unrelated packages then their debug info
>>>> packages will conflict even if the regular binary packages don't.
>>>
>>> I've seen this outside of reproducibility where I was shipping the exact
>>> same binary in the redis-server and redis-sentinel packages (it changes
>>> behaviour based on argv[0]).
>>>
>>> The -dbgsym packages then conflicted for the same reason.
>>
>> Stupid question, but why _do_ the packages conflict?  Couldn't the
>> package manager notice that the file versions that would be installed by
>> each package are equivalent [= same name, chmod, and bit-by-bit
>> contents], and keep the file existing so long as _either_ package is
>> installed?
> 
> In the case of the kernel packages, the identical binaries (vDSOs) are
> emebedded in kernel images with different filenames.  The identical
> debug info is installed with different filenames.  But the symlinks to
> them underneath /usr/lib/debug/.build-id therefore have the same (hash-
> based) name and *different* content.
> 

Could you reverse the symlink? So the target (the actual file) is in /usr/lib/debug and so both packages will have the same file installed at that path.

Then perhaps we could ask dpkg to add an exception for this Breaks/Replaces annotation for files under /usr/lib/debug when the file contents are the same (or just blanket-allow this case for all paths).

When the policy was originally written it would have taken extra technical effort to refcount which packages referred to the same file+filepath, but (IIRC) this logic has since been added in order to deal gracefully with Multi-Arch. Maybe it's time to extend it to more things?

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.kernel

csiph-web