Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.debian.security > #5173 > unrolled thread
| Started by | Adam Warner <lists@consulting.net.nz> |
|---|---|
| First post | 2017-01-28 14:20 +0100 |
| Last post | 2017-01-28 17:20 +0100 |
| Articles | 8 on this page of 28 — 15 participants |
Back to article view | Back to linux.debian.security
Some Debian package upgrades are corrupting rsync "quick check" backups Adam Warner <lists@consulting.net.nz> - 2017-01-28 14:20 +0100
Re: Some Debian package upgrades are corrupting rsync "quick check" backups Christoph Biedl <debian.axhn@manchmal.in-ulm.de> - 2017-01-28 15:00 +0100
Re: Some Debian package upgrades are corrupting rsync "quick check" backups Daniel Reichelt <debian@nachtgeist.net> - 2017-01-28 15:50 +0100
Re: Some Debian package upgrades are corrupting rsync "quick check" backups Holger Levsen <holger@layer-acht.org> - 2017-01-28 16:00 +0100
Re: Some Debian package upgrades are corrupting rsync "quick check" backups Daniel Reichelt <debian@nachtgeist.net> - 2017-01-28 16:40 +0100
Re: Some Debian package upgrades are corrupting rsync "quick check" backups Lupe Christoph <lupe@lupe-christoph.de> - 2017-01-28 23:10 +0100
Re: Some Debian package upgrades are corrupting rsync "quick check" backups "Jonathan Hutchins" <hutchins@tarcanfel.org> - 2017-01-29 00:10 +0100
Re: Some Debian package upgrades are corrupting rsync "quick check" backups Santiago Vila <sanvila@unex.es> - 2017-01-29 20:30 +0100
Re: Some Debian package upgrades are corrupting rsync "quick check" backups Russell Coker <russell@coker.com.au> - 2017-01-30 10:40 +0100
Re: Some Debian package upgrades are corrupting rsync "quick check" backups Jonathan Wiltshire <jmw@debian.org> - 2017-01-29 21:20 +0100
Re: Some Debian package upgrades are corrupting rsync "quick check" backups Henrik Ahlgren <pablo@seestieto.com> - 2017-01-30 13:10 +0100
Re: Some Debian package upgrades are corrupting rsync "quick check" backups Mattia Rizzolo <mattia@debian.org> - 2017-01-30 13:20 +0100
Re: Some Debian package upgrades are corrupting rsync "quick check" backups Holger Levsen <holger@layer-acht.org> - 2017-01-30 14:30 +0100
Re: Some Debian package upgrades are corrupting rsync "quick check" backups Johannes Schauer <josch@debian.org> - 2017-01-30 15:00 +0100
Re: Some Debian package upgrades are corrupting rsync "quick check" backups Holger Levsen <holger@layer-acht.org> - 2017-01-30 15:00 +0100
Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups Johannes Schauer <josch@debian.org> - 2017-01-30 16:30 +0100
Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups Chris Lamb <lamby@debian.org> - 2017-05-13 16:50 +0200
Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups Holger Levsen <holger@layer-acht.org> - 2017-05-13 18:00 +0200
Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups Mattia Rizzolo <mattia@debian.org> - 2017-05-13 18:00 +0200
Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups Aurelien Jarno <aurelien@aurel32.net> - 2017-05-13 21:40 +0200
Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups Aurelien Jarno <aurelien@aurel32.net> - 2017-05-13 22:50 +0200
Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups Holger Levsen <holger@layer-acht.org> - 2017-05-13 23:00 +0200
Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups Adam Warner <lists@consulting.net.nz> - 2017-08-28 08:20 +0200
Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups Aurelien Jarno <aurelien@aurel32.net> - 2017-08-28 12:40 +0200
Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups Aurelien Jarno <aurelien@aurel32.net> - 2017-08-28 12:50 +0200
Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups Aurelien Jarno <aurelien@aurel32.net> - 2017-08-28 13:00 +0200
Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups Adam Warner <lists@consulting.net.nz> - 2017-08-28 14:20 +0200
Re: Some Debian package upgrades are corrupting rsync "quick check" backups Bob Weber <bobrweber@gmail.com> - 2017-01-28 17:20 +0100
Page 2 of 2 — ← Prev page 1 [2]
| From | Aurelien Jarno <aurelien@aurel32.net> |
|---|---|
| Date | 2017-05-13 22:50 +0200 |
| Subject | Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups |
| Message-ID | <tGM9b-g8-11@gated-at.bofh.it> |
| In reply to | #5232 |
[Multipart message — attachments visible in raw view] — view raw
On 2017-05-13 21:34, Aurelien Jarno wrote: > On 2017-05-13 17:52, Mattia Rizzolo wrote: > > On Sat, May 13, 2017 at 03:44:57PM +0100, Chris Lamb wrote: > > > a) Has anything changed in the meantime? > > > > Yes: sbuild stopped repeating the changelog time taking it from the last > > entry, and will instead generate a new timestamp based on the current > > time: > > > > * For binNMUs, instead of copying the timestamp of the last changelog entry, > > generate a new one (closes: #843773) > > > > In version 0.73.0-1. > > And I am glad that after all that months with people talking about the > issue, I finally got a detailed description of the issue and a pointer > to the commit to backport. I'll work on that. The above change should now be deployed on most jessie based buildds, it's only missing on the buildds that are currently down. -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurelien@aurel32.net http://www.aurel32.net
[toc] | [prev] | [next] | [standalone]
| From | Holger Levsen <holger@layer-acht.org> |
|---|---|
| Date | 2017-05-13 23:00 +0200 |
| Subject | Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups |
| Message-ID | <tGMiR-jQ-3@gated-at.bofh.it> |
| In reply to | #5233 |
[Multipart message — attachments visible in raw view] — view raw
On Sat, May 13, 2017 at 10:48:18PM +0200, Aurelien Jarno wrote: > The above change should now be deployed on most jessie based buildds, > it's only missing on the buildds that are currently down. cool, thank you! -- cheers, Holger
[toc] | [prev] | [next] | [standalone]
| From | Adam Warner <lists@consulting.net.nz> |
|---|---|
| Date | 2017-08-28 08:20 +0200 |
| Subject | Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups |
| Message-ID | <ujlyW-1YD-11@gated-at.bofh.it> |
| In reply to | #5233 |
On Sat, 2017-05-13 at 22:48 +0200, Aurelien Jarno wrote: > On 2017-05-13 21:34, Aurelien Jarno wrote: > > On 2017-05-13 17:52, Mattia Rizzolo wrote: > > > On Sat, May 13, 2017 at 03:44:57PM +0100, Chris Lamb wrote: > > > > a) Has anything changed in the meantime? > > > > > > Yes: sbuild stopped repeating the changelog time taking it from > > > the last > > > entry, and will instead generate a new timestamp based on the > > > current > > > time: > > > > > > * For binNMUs, instead of copying the timestamp of the last > > > changelog entry, > > > generate a new one (closes: #843773) > > > > > > In version 0.73.0-1. > > > > And I am glad that after all that months with people talking about > > the > > issue, I finally got a detailed description of the issue and a > > pointer > > to the commit to backport. I'll work on that. > > The above change should now be deployed on most jessie based buildds, > it's only missing on the buildds that are currently down. Original thread author here reporting to beware that some rsync data corruption can still become apparent after all this time. It's been a while since I did a full rsync checksum test. Decided to do one after a recent upgrade that includes clang 3.9 related files. These are Debian systems that default to unstable BUT include all debian apt sources including experimental/unstable/testing/stable/oldstable. I found these four corrupted files in my rsync backups: var/lib/dpkg/info/clang-3.9.md5sums var/lib/dpkg/info/libclang-common-3.9-dev.md5sums var/lib/dpkg/info/libclang1-3.9:amd64.md5sums var/lib/dpkg/info/libllvm3.9:amd64.md5sums The latest packages were installed from this repository: Get:17 https://cdn-aws.deb.debian.org/debian unstable/main amd64 clang-3.9 amd64 1:3.9.1-11 [37.3 MB] Get:18 https://cdn-aws.deb.debian.org/debian unstable/main amd64 libclang-common-3.9-dev amd64 1:3.9.1-11 [2,587 kB] Get:19 https://cdn-aws.deb.debian.org/debian unstable/main amd64 libllvm3.9 amd64 1:3.9.1-11 [11.4 MB] Get:20 https://cdn-aws.deb.debian.org/debian unstable/main amd64 libclang1-3.9 amd64 1:3.9.1-11 [5,896 kB] Here are the mtimes of the newly install files: # ls -la /var/lib/dpkg/info/clang-3.9.md5sums /var/lib/dpkg/info/libclang-common-3.9-dev.md5sums /var/lib/dpkg/info/libclang1-3.9:amd64.md5sums /var/lib/dpkg/info/libllvm3.9:amd64.md5sums -rw-r--r-- 1 root root 10855 Jun 19 05:12 /var/lib/dpkg/info/clang-3.9.md5sums -rw-r--r-- 1 root root 384 Jun 19 05:12 /var/lib/dpkg/info/libclang1-3.9:amd64.md5sums -rw-r--r-- 1 root root 18320 Jun 19 05:12 /var/lib/dpkg/info/libclang-common-3.9-dev.md5sums -rw-r--r-- 1 root root 371 Jun 19 05:12 /var/lib/dpkg/info/libllvm3.9:amd64.md5sums Here's a diff of /var/lib/dpkg/info/clang-3.9.md5sums (rsync backup vs current file): 2,11c2,11 < 3cbbedf0c71035cd3029dfe1b052e581 usr/lib/llvm-3.9/bin/c-index-test < 0fdde07673439d60670efed27456169a usr/lib/llvm-3.9/bin/clang < aa1260a7741de9cdf0c748f8b7462c1c usr/lib/llvm-3.9/bin/clang-apply-replacements < 996c76e1c1ab4075d016d4a860a616a2 usr/lib/llvm-3.9/bin/clang-check < 8021eba7391ad79567cf0d2cc71bacfa usr/lib/llvm-3.9/bin/clang-include-fixer < 048cc918522e9099d7085d734e7356cc usr/lib/llvm-3.9/bin/clang-query < 7dae8990cdbf679b19e370d62fea05e1 usr/lib/llvm-3.9/bin/clang-rename < 3f7c675b0be7c94c88e6b27f665236d2 usr/lib/llvm-3.9/bin/find-all-symbols < a2920ac65d5085773747078cdd099f87 usr/lib/llvm-3.9/bin/modularize < dd36711af0597df2c78f3583fb914728 usr/lib/llvm-3.9/bin/sancov --- > e48899b8ed6e12602edaec4be4cf5ba8 usr/lib/llvm-3.9/bin/c-index-test > c523712273fb46731e4ad88de23a00a6 usr/lib/llvm-3.9/bin/clang > 1d59562b0f0167b20b3f47af8da65bb2 usr/lib/llvm-3.9/bin/clang-apply-replacements > de8b1949a36135e7f5eac0016e99c813 usr/lib/llvm-3.9/bin/clang-check > 458274598da467da470be2404374432e usr/lib/llvm-3.9/bin/clang-include-fixer > cca520673bf5586037e66b8384944576 usr/lib/llvm-3.9/bin/clang-query > 3d3b408633a6ffe416c28a281082ffde usr/lib/llvm-3.9/bin/clang-rename > 288d8965a7fa9389c37233d8751683a9 usr/lib/llvm-3.9/bin/find-all-symbols > ec0b482ab1af827ea958b51a87a096f8 usr/lib/llvm-3.9/bin/modularize > 21c34b02091b53c37a90306f1dea5d6a usr/lib/llvm-3.9/bin/sancov 109c109 < 32db7068b426f045c64d8642c1e6c022 usr/share/doc/clang-3.9/changelog.Debian.gz --- > 2ec59c559a5c304cacddea2f02d74697 usr/share/doc/clang-3.9/changelog.Debian.gz 116,123c116,123 < 3a8deaf272eff95b110efb05ebc9ee9d usr/share/man/man1/clang-apply-replacements-3.9.1.gz < efe297e129ca7d6823ac46f5b7a58644 usr/share/man/man1/clang-check-3.9.1.gz < 114d671d66872a188832cad20062cd7b usr/share/man/man1/clang-include-fixer-3.9.1.gz < 8992eb9feffb6b90c015a28785d0b33b usr/share/man/man1/clang-query-3.9.1.gz < c69a128b81404ecc2eac7e759608fd15 usr/share/man/man1/clang-rename-3.9.1.gz < 14f265858cf398647aa0be918e65dd6b usr/share/man/man1/find-all-symbols-3.9.1.gz < e54fd298f2fea34b272d44b7c8c372a2 usr/share/man/man1/modularize-3.9.1.gz < c062a1192879288d88211d8804c740c4 usr/share/man/man1/sancov-3.9.1.gz --- > 184960744b144151add1be27f96e5a2f usr/share/man/man1/clang-apply-replacements-3.9.1.gz > 68480b6ce324eae9ce6771eb30ae0093 usr/share/man/man1/clang-check-3.9.1.gz > 1f16d5c2d51a41aa723ae820817b3b5b usr/share/man/man1/clang-include-fixer-3.9.1.gz > 2da1698364052c490c02bae9a9c81907 usr/share/man/man1/clang-query-3.9.1.gz > cae329a9d569ef1cdf1d13754c872131 usr/share/man/man1/clang-rename-3.9.1.gz > 7c29374033ef542c082fa437ff158765 usr/share/man/man1/find-all-symbols-3.9.1.gz > a1522b2d84d25ca07c5fc195b0a1ffcf usr/share/man/man1/modularize-3.9.1.gz > 41fb9f4cfb33c8a49bac4f88dc5a762f usr/share/man/man1/sancov-3.9.1.gz It appears the rsync backups were corrupted by new packages recently build for unstable that were source-modified in June: <http://metadata.ftp-master.debian.org/changelogs/main/l/llvm-toolchain-3.9/llvm-toolchain-3.9_3.9.1-11_changelog> Regards, Adam Warner
[toc] | [prev] | [next] | [standalone]
| From | Aurelien Jarno <aurelien@aurel32.net> |
|---|---|
| Date | 2017-08-28 12:40 +0200 |
| Subject | Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups |
| Message-ID | <ujpCx-4si-11@gated-at.bofh.it> |
| In reply to | #5266 |
[Multipart message — attachments visible in raw view] — view raw
On 2017-08-28 18:06, Adam Warner wrote: > On Sat, 2017-05-13 at 22:48 +0200, Aurelien Jarno wrote: > > On 2017-05-13 21:34, Aurelien Jarno wrote: > > > On 2017-05-13 17:52, Mattia Rizzolo wrote: > > > > On Sat, May 13, 2017 at 03:44:57PM +0100, Chris Lamb wrote: > > > > > a) Has anything changed in the meantime? > > > > > > > > Yes: sbuild stopped repeating the changelog time taking it from > > > > the last > > > > entry, and will instead generate a new timestamp based on the > > > > current > > > > time: > > > > > > > > * For binNMUs, instead of copying the timestamp of the last > > > > changelog entry, > > > > generate a new one (closes: #843773) > > > > > > > > In version 0.73.0-1. > > > > > > And I am glad that after all that months with people talking about > > > the > > > issue, I finally got a detailed description of the issue and a > > > pointer > > > to the commit to backport. I'll work on that. > > > > The above change should now be deployed on most jessie based buildds, > > it's only missing on the buildds that are currently down. > > Original thread author here reporting to beware that some rsync data > corruption can still become apparent after all this time. > > It's been a while since I did a full rsync checksum test. Decided to do > one after a recent upgrade that includes clang 3.9 related files. These > are Debian systems that default to unstable BUT include all debian apt > sources including experimental/unstable/testing/stable/oldstable. > > I found these four corrupted files in my rsync backups: > > var/lib/dpkg/info/clang-3.9.md5sums > var/lib/dpkg/info/libclang-common-3.9-dev.md5sums > var/lib/dpkg/info/libclang1-3.9:amd64.md5sums > var/lib/dpkg/info/libllvm3.9:amd64.md5sums > > The latest packages were installed from this repository: > > Get:17 https://cdn-aws.deb.debian.org/debian unstable/main amd64 clang-3.9 amd64 1:3.9.1-11 [37.3 MB] > Get:18 https://cdn-aws.deb.debian.org/debian unstable/main amd64 libclang-common-3.9-dev amd64 1:3.9.1-11 [2,587 kB] > Get:19 https://cdn-aws.deb.debian.org/debian unstable/main amd64 libllvm3.9 amd64 1:3.9.1-11 [11.4 MB] > Get:20 https://cdn-aws.deb.debian.org/debian unstable/main amd64 libclang1-3.9 amd64 1:3.9.1-11 [5,896 kB] These files haven't been built on a build daemon, but instead have been uploaded by the maintainer [1]. This is therefore not a buildd issue, the issue has been fixed there already with the upgrade to stretch. Aurelien [1] https://tracker.debian.org/news/866006 -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurelien@aurel32.net http://www.aurel32.net
[toc] | [prev] | [next] | [standalone]
| From | Aurelien Jarno <aurelien@aurel32.net> |
|---|---|
| Date | 2017-08-28 12:50 +0200 |
| Subject | Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups |
| Message-ID | <ujpMe-4xh-11@gated-at.bofh.it> |
| In reply to | #5267 |
[Multipart message — attachments visible in raw view] — view raw
On 2017-08-28 12:33, Aurelien Jarno wrote: > On 2017-08-28 18:06, Adam Warner wrote: > > On Sat, 2017-05-13 at 22:48 +0200, Aurelien Jarno wrote: > > > On 2017-05-13 21:34, Aurelien Jarno wrote: > > > > On 2017-05-13 17:52, Mattia Rizzolo wrote: > > > > > On Sat, May 13, 2017 at 03:44:57PM +0100, Chris Lamb wrote: > > > > > > a) Has anything changed in the meantime? > > > > > > > > > > Yes: sbuild stopped repeating the changelog time taking it from > > > > > the last > > > > > entry, and will instead generate a new timestamp based on the > > > > > current > > > > > time: > > > > > > > > > > * For binNMUs, instead of copying the timestamp of the last > > > > > changelog entry, > > > > > generate a new one (closes: #843773) > > > > > > > > > > In version 0.73.0-1. > > > > > > > > And I am glad that after all that months with people talking about > > > > the > > > > issue, I finally got a detailed description of the issue and a > > > > pointer > > > > to the commit to backport. I'll work on that. > > > > > > The above change should now be deployed on most jessie based buildds, > > > it's only missing on the buildds that are currently down. > > > > Original thread author here reporting to beware that some rsync data > > corruption can still become apparent after all this time. > > > > It's been a while since I did a full rsync checksum test. Decided to do > > one after a recent upgrade that includes clang 3.9 related files. These > > are Debian systems that default to unstable BUT include all debian apt > > sources including experimental/unstable/testing/stable/oldstable. > > > > I found these four corrupted files in my rsync backups: > > > > var/lib/dpkg/info/clang-3.9.md5sums > > var/lib/dpkg/info/libclang-common-3.9-dev.md5sums > > var/lib/dpkg/info/libclang1-3.9:amd64.md5sums > > var/lib/dpkg/info/libllvm3.9:amd64.md5sums > > > > The latest packages were installed from this repository: > > > > Get:17 https://cdn-aws.deb.debian.org/debian unstable/main amd64 clang-3.9 amd64 1:3.9.1-11 [37.3 MB] > > Get:18 https://cdn-aws.deb.debian.org/debian unstable/main amd64 libclang-common-3.9-dev amd64 1:3.9.1-11 [2,587 kB] > > Get:19 https://cdn-aws.deb.debian.org/debian unstable/main amd64 libllvm3.9 amd64 1:3.9.1-11 [11.4 MB] > > Get:20 https://cdn-aws.deb.debian.org/debian unstable/main amd64 libclang1-3.9 amd64 1:3.9.1-11 [5,896 kB] > > These files haven't been built on a build daemon, but instead have > been uploaded by the maintainer [1]. This is therefore not a buildd > issue, the issue has been fixed there already with the upgrade to > stretch. More precisely the two latest changelog entries have the same date: | llvm-toolchain-3.9 (1:3.9.1-11) unstable; urgency=medium | | [ Sylvestre Ledru ] | * Remove the --no-discard-stderr option from help2man calls | * Also add a missing include in ftfbs-gcc.diff to fix a ftbfs | with gcc 7 | * clang was producing unusable binaries on armv5tel (Closes #873304) | Thanks to Adrian Bunk for the patch | * Disable -gsplit-dwarf when using gcc 7 for causing a linking issue | See https://bugs.llvm.org/show_bug.cgi?id=34140 (Closes: #853524) | | [ Gianfranco Costamagna, John Paul Adrian Glaubitz ] | * Add powerpcspe to latomic archs | | [ Katsuhiko Nishimra ] | * Ensure /usr/bin/g++-$(GCC_VERSION) exists (Closes: #871591) | | -- Sylvestre Ledru <sylvestre@debian.org> Sun, 18 Jun 2017 19:12:15 +0200 | | llvm-toolchain-3.9 (1:3.9.1-10) unstable; urgency=medium | | * Now that strech has been released, upload in unstable! | This is necessary for rust in unstable | * Try to fix some PATH_MAX on hurd | * Enable the verbose mode when trying to build libfuzzer | to detect potential issues in the path search | | -- Sylvestre Ledru <sylvestre@debian.org> Sun, 18 Jun 2017 19:12:15 +0200 That's the reason why the files ended-up with the same date but different content. Aurelien -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurelien@aurel32.net http://www.aurel32.net
[toc] | [prev] | [next] | [standalone]
| From | Aurelien Jarno <aurelien@aurel32.net> |
|---|---|
| Date | 2017-08-28 13:00 +0200 |
| Subject | Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups |
| Message-ID | <ujpVU-4AQ-11@gated-at.bofh.it> |
| In reply to | #5268 |
[Multipart message — attachments visible in raw view] — view raw
On 2017-08-28 12:42, Aurelien Jarno wrote: > On 2017-08-28 12:33, Aurelien Jarno wrote: > > On 2017-08-28 18:06, Adam Warner wrote: > > > On Sat, 2017-05-13 at 22:48 +0200, Aurelien Jarno wrote: > > > > On 2017-05-13 21:34, Aurelien Jarno wrote: > > > > > On 2017-05-13 17:52, Mattia Rizzolo wrote: > > > > > > On Sat, May 13, 2017 at 03:44:57PM +0100, Chris Lamb wrote: > > > > > > > a) Has anything changed in the meantime? > > > > > > > > > > > > Yes: sbuild stopped repeating the changelog time taking it from > > > > > > the last > > > > > > entry, and will instead generate a new timestamp based on the > > > > > > current > > > > > > time: > > > > > > > > > > > > * For binNMUs, instead of copying the timestamp of the last > > > > > > changelog entry, > > > > > > generate a new one (closes: #843773) > > > > > > > > > > > > In version 0.73.0-1. > > > > > > > > > > And I am glad that after all that months with people talking about > > > > > the > > > > > issue, I finally got a detailed description of the issue and a > > > > > pointer > > > > > to the commit to backport. I'll work on that. > > > > > > > > The above change should now be deployed on most jessie based buildds, > > > > it's only missing on the buildds that are currently down. > > > > > > Original thread author here reporting to beware that some rsync data > > > corruption can still become apparent after all this time. > > > > > > It's been a while since I did a full rsync checksum test. Decided to do > > > one after a recent upgrade that includes clang 3.9 related files. These > > > are Debian systems that default to unstable BUT include all debian apt > > > sources including experimental/unstable/testing/stable/oldstable. > > > > > > I found these four corrupted files in my rsync backups: > > > > > > var/lib/dpkg/info/clang-3.9.md5sums > > > var/lib/dpkg/info/libclang-common-3.9-dev.md5sums > > > var/lib/dpkg/info/libclang1-3.9:amd64.md5sums > > > var/lib/dpkg/info/libllvm3.9:amd64.md5sums > > > > > > The latest packages were installed from this repository: > > > > > > Get:17 https://cdn-aws.deb.debian.org/debian unstable/main amd64 clang-3.9 amd64 1:3.9.1-11 [37.3 MB] > > > Get:18 https://cdn-aws.deb.debian.org/debian unstable/main amd64 libclang-common-3.9-dev amd64 1:3.9.1-11 [2,587 kB] > > > Get:19 https://cdn-aws.deb.debian.org/debian unstable/main amd64 libllvm3.9 amd64 1:3.9.1-11 [11.4 MB] > > > Get:20 https://cdn-aws.deb.debian.org/debian unstable/main amd64 libclang1-3.9 amd64 1:3.9.1-11 [5,896 kB] > > > > These files haven't been built on a build daemon, but instead have > > been uploaded by the maintainer [1]. This is therefore not a buildd > > issue, the issue has been fixed there already with the upgrade to > > stretch. > > More precisely the two latest changelog entries have the same date: > I have just filled a bug against ftp.debian.org for such packages to be rejected. See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=873489 Aurelien -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurelien@aurel32.net http://www.aurel32.net
[toc] | [prev] | [next] | [standalone]
| From | Adam Warner <lists@consulting.net.nz> |
|---|---|
| Date | 2017-08-28 14:20 +0200 |
| Subject | Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups |
| Message-ID | <ujrbk-5ug-21@gated-at.bofh.it> |
| In reply to | #5269 |
On Mon, 2017-08-28 at 12:58 +0200, Aurelien Jarno wrote: [...] > > > These files haven't been built on a build daemon, but instead have > > > been uploaded by the maintainer [1]. This is therefore not a buildd > > > issue, the issue has been fixed there already with the upgrade to > > > stretch. > > > > More precisely the two latest changelog entries have the same date: > > I have just filled a bug against ftp.debian.org for such packages to be > rejected. See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=873489 > > Aurelien Thank you for your prompt and accurate analysis of the situation and your bug submission to the FTP Masters suggesting that Lintian is used to reject maintainer uploads with a non-subsequent changelog date. Regards, Adam
[toc] | [prev] | [next] | [standalone]
| From | Bob Weber <bobrweber@gmail.com> |
|---|---|
| Date | 2017-01-28 17:20 +0100 |
| Message-ID | <t4DTk-3I7-11@gated-at.bofh.it> |
| In reply to | #5173 |
[Multipart message — attachments visible in raw view] — view raw
As a user I have run into this with X11 files myself. I use rsnapshot to backup the root partition to a location on /home mounted on its own much larger partition before I do upgrades. A while back when xorg was crashing a lot I had to restore from this backup. I routinely use "debsums -ca" after an upgrade to check OS files against their hashes and I discovered this strange occurrence with files having the same date/time and size but different hashes. Luckily I have a server running apt-cacher-ng so I had the older deb files to re-install to fix the problem. So as a user this a BIG pain in the rear. I also tried the rsync checksum option but as you noted it is much more time consuming. Are these files actually equivalent but with their code segments scrambled for security? I was actually running xorg with the "bad" files for a while before I found them with debsums. From a user point of view this is totally inexcusable. I hope that a solution can be found to fix this ... even if it is just incrementing the second by one. *...Bob* On 01/28/2017 08:11 AM, Adam Warner wrote: > Hi all, > > rsync typically detects file modifications by comparing exact > modification time and size of each identically named file in the source > and destination locations. > > An rsync backup can be surreptitiously corrupted by modifying a source > file's content while keeping its size and modification time the same. > > If some program is doing this then one way for rsync to detect the > modification is by appending the --checksum option. This requires every > file in the source and destination to be fully read. This can be many > orders of magnitude slower than a "quick check". > > If you have been using rsync without --checksum to back up your Debian > partitions then your backups are likely corrupted. > > Some packages are being generated with files containing exactly the > same size and timestamps even though the contents of those files are > different. This is how the data corruption arises: > > 1. You use rsync to back up your OS partition. > 2. You perform package upgrades. Some files are replaced with different > content but have the same size and modification time. > 3. You use rsync to back up your OS partition to the same destination. > The modified files with the same size and modification time are > skipped. Your backup is now corrupt (you have old files mixed with new > files). > > Here's a recent example of a package upgrade causing this: > > <https://packages.debian.org/testing/amd64/libqt5concurrent5/download> > <https://packages.debian.org/sid/amd64/libqt5concurrent5/download> > > If you're tracking unstable you may have recently upgraded from > libqt5concurrent5_5.7.1+dfsg-3_amd64.deb to > libqt5concurrent5_5.7.1+dfsg-3+b1_amd64.deb (a subsequent binary non- > maintainer upload). > > Here are the contents of both packages: > $ dpkg -c libqt5concurrent5_5.7.1+dfsg-3_amd64.deb > drwxr-xr-x root/root 0 2017-01-12 04:14 ./ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/lib/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/ > -rw-r--r-- root/root 27352 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/doc/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/doc/libqt5concurrent5/ > -rw-r--r-- root/root 1196 2016-12-01 21:17 ./usr/share/doc/libqt5concurrent5/LGPL_EXCEPTION.txt > -rw-r--r-- root/root 18232 2017-01-12 04:14 ./usr/share/doc/libqt5concurrent5/changelog.Debian.gz > -rw-r--r-- root/root 3792 2016-12-01 21:17 ./usr/share/doc/libqt5concurrent5/changelog.gz > -rw-r--r-- root/root 103466 2017-01-12 04:14 ./usr/share/doc/libqt5concurrent5/copyright > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/lintian/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/lintian/overrides/ > -rw-r--r-- root/root 230 2017-01-12 04:14 ./usr/share/lintian/overrides/libqt5concurrent5 > lrwxrwxrwx root/root 0 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5 -> libQt5Concurrent.so.5.7.1 > lrwxrwxrwx root/root 0 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7 -> libQt5Concurrent.so.5.7.1 > > $ dpkg -c libqt5concurrent5_5.7.1+dfsg-3+b1_amd64.deb > drwxr-xr-x root/root 0 2017-01-12 04:14 ./ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/lib/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/ > -rw-r--r-- root/root 27352 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/doc/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/doc/libqt5concurrent5/ > -rw-r--r-- root/root 1196 2016-12-01 21:17 ./usr/share/doc/libqt5concurrent5/LGPL_EXCEPTION.txt > -rw-r--r-- root/root 252 2017-01-12 04:14 ./usr/share/doc/libqt5concurrent5/changelog.Debian.amd64.gz > -rw-r--r-- root/root 18232 2017-01-12 04:14 ./usr/share/doc/libqt5concurrent5/changelog.Debian.gz > -rw-r--r-- root/root 3792 2016-12-01 21:17 ./usr/share/doc/libqt5concurrent5/changelog.gz > -rw-r--r-- root/root 103466 2017-01-12 04:14 ./usr/share/doc/libqt5concurrent5/copyright > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/lintian/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/lintian/overrides/ > -rw-r--r-- root/root 230 2017-01-12 04:14 ./usr/share/lintian/overrides/libqt5concurrent5 > lrwxrwxrwx root/root 0 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5 -> libQt5Concurrent.so.5.7.1 > lrwxrwxrwx root/root 0 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7 -> libQt5Concurrent.so.5.7.1 > > In particular notice the shared library > /usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 has an identical > size and timestamp. > > However: > (testing) $ sha256sum ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 > aef7b3d108ba5c686279fa2dbd63bb7916b1bb79ec69faeab55ea8f3b85725df ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 > (sid) $ sha256sum ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 > 052a748ebc2ab3e65e8b25cab3515a73762450b7c84308dad157c1f677e21b0a ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 > > Why is a 27 January recompilation of the source package purporting to > have the same modification time as the original binary package > distributed 16 days earlier? > > It is now also clear I have previously encountered this with some X > shared libraries. > > I suggest an rsync --checksum test on your backups (using > --itemize-changes and --dry-run to check for modifications without > making changes to the destination). > > Regards, > Adam Warner > >
[toc] | [prev] | [standalone]
Page 2 of 2 — ← Prev page 1 [2]
Back to top | Article view | linux.debian.security
csiph-web