Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #266433

Re: Disk corruption and performance issue.

Show all headers | View raw

On 1/21/24 02:45, Tim Woodall wrote:
> On Sat, 20 Jan 2024, David Christensen wrote:
>> On 1/20/24 08:25, Tim Woodall wrote:
>>> Some time ago I wrote about a data corruption issue. I've still not
>>> managed to track it down ...
>>
>> Please post a console session that demonstrates, or at least 
>> documents, the data corruption.
> 
> Console session is difficult - this is a script that takes around 6
> hours to run - but a typical example of corruption is something like
> this:
> 
> Preparing to unpack .../03-libperl5.34_5.34.0-5_arm64.deb ...
> Unpacking libperl5.34:arm64 (5.34.0-5) ...
> dpkg-deb (subprocess): decompressing archive 
> '/tmp/apt-dpkg-install-zqY3js/03-libperl5.34_5.34.0-5_arm64.deb' 
> (size=4015516) member 'data.tar': lzma error: compressed data is corrupt
> dpkg-deb: error: <decompress> subprocess returned error exit status 2
> dpkg: error processing archive 
> /tmp/apt-dpkg-install-zqY3js/03-libperl5.34_5.34.0-5_arm64.deb (--unpack):
>   cannot copy extracted data for 
> './usr/lib/aarch64-linux-gnu/libperl.so.5.34.0' to 
> '/usr/lib/aarch64-linux-gnu/libperl.so.5.34.0.dpkg-new': unexpected end 
> of file or stream
> 
> The checksum will have been verified by apt during the download but when
> it comes to read the downloaded deb to unpack and install it doesn't get
> the same data. The corruption can happen at both the writing (the file
> on disk is corrupted) and the reading (the file on disk has the correct
> checksum)


Suggestions:

1.  Use the -e (errexit), -u (nounset), and/or -x (xtrace) options; or 
their equivalents, if you are not using Bourne shell.

2.  Add printf's to dump progress and debugging information to a file 
while the script runs.

3.  Add assertions to check for disk corruption, performance problems, 
and any other else that concerns you; now or in the past.  If any 
assertion fails, the assertion should identify itself, halt the script, 
and dump the relevant debugging information.

4.  Refactor your code into a hierarchy (directed acyclic graph).  Start 
your debugging/ validation at the bottom (leaf nodes; functions, 
commands) and work your way up (root node; the 6 hour script).

5.  Make the script idempotent, so that when it fails and you run it 
again the script will skip over previously completed steps.


David

Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Disk corruption and performance issue. Tim Woodall <debianuser@woodall.me.uk> - 2024-01-20 17:30 +0100
  Re: Disk corruption and performance issue. David Christensen <dpchrist@holgerdanske.com> - 2024-01-20 22:20 +0100
    Re: Disk corruption and performance issue. Tim Woodall <debianuser@woodall.me.uk> - 2024-01-21 11:50 +0100
      Re: Disk corruption and performance issue. gene heskett <gheskett@shentel.net> - 2024-01-21 19:00 +0100
      Re: Disk corruption and performance issue. David Christensen <dpchrist@holgerdanske.com> - 2024-01-22 00:10 +0100
      SOLVED Re: Disk corruption and performance issue. Tim Woodall <debianuser@woodall.me.uk> - 2024-02-26 19:30 +0100
        Re: SOLVED Re: Disk corruption and performance issue. Gremlin <scott-andrews@columbus.rr.com> - 2024-02-26 20:10 +0100
          Re: SOLVED Re: Disk corruption and performance issue. Tim Woodall <debianuser@woodall.me.uk> - 2024-02-26 20:50 +0100
            Re: SOLVED Re: Disk corruption and performance issue. Gremlin <scott-andrews@columbus.rr.com> - 2024-02-26 22:00 +0100
              Re: SOLVED Re: Disk corruption and performance issue. Tim Woodall <debianuser@woodall.me.uk> - 2024-02-26 22:40 +0100
                Re: SOLVED Re: Disk corruption and performance issue. Gremlin <scott-andrews@columbus.rr.com> - 2024-02-26 22:50 +0100
              Re: SOLVED Re: Disk corruption and performance issue. Stefan Monnier <monnier@iro.umontreal.ca> - 2024-02-26 22:50 +0100
        Re: SOLVED Re: Disk corruption and performance issue. Andy Smith <andy@strugglers.net> - 2024-02-27 00:50 +0100
          Re: SOLVED Re: Disk corruption and performance issue. Tim Woodall <debianuser@woodall.me.uk> - 2024-02-27 05:40 +0100

csiph-web