Re: Useless use of "dd"

From	David Christensen <dpchrist@holgerdanske.com>
Newsgroups	linux.debian.user
Subject	Re: Useless use of "dd"
Date	2021-07-02 21:40 +0200
Message-ID	<Cwxb3-8oy-7@gated-at.bofh.it> (permalink)
References	<Cv0TT-1C0-3@gated-at.bofh.it> <Cv6cV-4QX-1@gated-at.bofh.it> <Cwhzj-715-1@gated-at.bofh.it> <CwilH-7y5-1@gated-at.bofh.it> <CwkdP-rs-3@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

On 7/1/21 10:40 PM, Teemu Likonen wrote:
> * 2021-07-01 20:43:09-0700, David Christensen wrote:
> 
>> To "take an image", the script invokes dd(1) and pipes the output to
>> gzip(1), copying raw device octets to a file. To "restore an image",
>> the process is reversed.
> 
> Sounds like the classic "useless use of dd". For general educational
> purposes I remind (not necessarily you) that "dd" is just a file copying
> (and conversion) tool, almost like "cat". By default "dd" is a slow
> version of "cat" because it uses smaller buffers. "dd" can be made
> faster by increasing its buffer size but that will not make it more than
> just "cat".
> 
> So "dd" is not a special tool for accessing device files. Device files
> don't need special tools. They are files. The following command:
> 
>      dd if=/dev/sdX | gzip >image.gz
> 
> is slower but functionally the same as either of these:
> 
>      gzip </dev/sdX >image.gz
>      gzip --to-stdout /dev/sdX >image.gz
> 
> Usually we don't need "dd" for anything but there are some options which
> are useful sometimes. For example, "dd" can report progress to standard
> error stream and it can seek and limit read to some blocks or bytes.

My description was a high-level overview.  As always, "the devil is in 
the details".  dd(1) has features that would require extra work with 
cat(1) or gzip(1).  The script provides all of the options to dd(1) that 
you mention, and more.  'bs=1048576' gives good performance, makes math 
easy, and dovetails with 'parted ... u MiB'.  I also omitted 
functionality like MD5 and SHA256 checksum files, byte-for-byte 
verification, Bash code generation from Perl, and opportunities for 
concurrency (yet to be implemented).  It's all good stuff that you 
encounter when you've been doing imaging by hand and decide to "up your 
game" with scripting.

David

Thread

Whole Disk Encryption + SSD piorunz <piorunz@gmx.com> - 2021-06-28 17:00 +0200
  Re: Whole Disk Encryption + SSD Jochen Spieker <ml@well-adjusted.de> - 2021-06-28 18:00 +0200
  Re: Whole Disk Encryption + SSD David Christensen <dpchrist@holgerdanske.com> - 2021-06-28 22:40 +0200
    Re: Whole Disk Encryption + SSD David Christensen <dpchrist@holgerdanske.com> - 2021-06-29 00:00 +0200
    Re: Whole Disk Encryption + SSD Stefan Monnier <monnier@iro.umontreal.ca> - 2021-06-29 02:00 +0200
      Re: Whole Disk Encryption + SSD <tomas@tuxteam.de> - 2021-06-29 09:50 +0200
        Re: Whole Disk Encryption + SSD Stefan Monnier <monnier@iro.umontreal.ca> - 2021-06-30 00:50 +0200
        Re: Whole Disk Encryption + SSD David Christensen <dpchrist@holgerdanske.com> - 2021-06-30 01:00 +0200
    Re: Whole Disk Encryption + SSD Jonathan Dowland <jon+debian-user@dow.land> - 2021-06-29 11:30 +0200
    Re: Whole Disk Encryption + SSD piorunz <piorunz@gmx.com> - 2021-06-29 14:10 +0200
      Re: Whole Disk Encryption + SSD David Christensen <dpchrist@holgerdanske.com> - 2021-06-30 02:20 +0200
    Re: Whole Disk Encryption + SSD David Wright <deblis@lionunicorn.co.uk> - 2021-07-02 05:00 +0200
      Re: Whole Disk Encryption + SSD David Christensen <dpchrist@holgerdanske.com> - 2021-07-02 05:50 +0200
        Useless use of "dd" Teemu Likonen <tlikonen@iki.fi> - 2021-07-02 07:50 +0200
          Useful use of dd [was: Useless use of "dd"] <tomas@tuxteam.de> - 2021-07-02 09:50 +0200
            Re: Useful use of dd [was: Useless use of "dd"] Teemu Likonen <tlikonen@iki.fi> - 2021-07-02 12:30 +0200
              Re: Useful use of dd "Thomas Schmitt" <scdbackup@gmx.net> - 2021-07-02 13:00 +0200
                Re: Useful use of dd <tomas@tuxteam.de> - 2021-07-02 13:20 +0200
                Re: Useful use of dd Teemu Likonen <tlikonen@iki.fi> - 2021-07-02 13:30 +0200
                Re: Useful use of dd David Christensen <dpchrist@holgerdanske.com> - 2021-07-02 23:50 +0200
              Re: Useful use of dd [was: Useless use of "dd"] <tomas@tuxteam.de> - 2021-07-02 13:20 +0200
              Re: Useful use of dd Michael Stone <mstone@debian.org> - 2021-07-02 14:50 +0200
              Re: Useful use of dd [was: Useless use of "dd"] Kushal Kumaran <kushal@locationd.net> - 2021-07-02 21:10 +0200
                Creation of empty files [was: Useful use of dd] David Christensen <dpchrist@holgerdanske.com> - 2021-07-02 23:40 +0200
                Re: Creation of empty files [was: Useful use of dd] David Christensen <dpchrist@holgerdanske.com> - 2021-07-03 00:00 +0200
                Re: Creation of empty files [was: Useful use of dd] Michael Stone <mstone@debian.org> - 2021-07-03 15:50 +0200
                Re: Creation of empty files [was: Useful use of dd] David Christensen <dpchrist@holgerdanske.com> - 2021-07-03 23:40 +0200
                Re: Creation of empty files [was: Useful use of dd] Michael Stone <mstone@debian.org> - 2021-07-04 06:00 +0200
            Re: Useful use of dd [was: Useless use of "dd"] Greg Wooledge <greg@wooledge.org> - 2021-07-02 13:40 +0200
              Re: Useful use of dd [was: Useless use of "dd"] <tomas@tuxteam.de> - 2021-07-02 13:40 +0200
                Re: Useful use of dd [was: Useless use of "dd"] David Wright <deblis@lionunicorn.co.uk> - 2021-07-02 17:10 +0200
                Re: Useful use of dd [was: Useless use of "dd"] <tomas@tuxteam.de> - 2021-07-02 17:40 +0200
                Re: Useful use of dd [was: Useless use of "dd"] Gunnar Gervin <dofeelok@gmail.com> - 2021-07-03 09:20 +0200
                Re: Useful use of dd [was: Useless use of "dd"] <tomas@tuxteam.de> - 2021-07-03 09:30 +0200
            Useless use of shell pipelines [was: Useful use of dd] David Christensen <dpchrist@holgerdanske.com> - 2021-07-02 23:40 +0200
              Re: Useless use of shell pipelines [was: Useful use of dd] l0f4r0@tuta.io - 2021-07-03 10:30 +0200
                Re: Useless use of shell pipelines [was: Useful use of dd] Greg Wooledge <greg@wooledge.org> - 2021-07-03 15:40 +0200
          Re: Useless use of "dd" David Christensen <dpchrist@holgerdanske.com> - 2021-07-02 21:40 +0200
        Re: Whole Disk Encryption + SSD David Wright <deblis@lionunicorn.co.uk> - 2021-07-02 17:10 +0200
          Re: Whole Disk Encryption + SSD Michael Stone <mstone@debian.org> - 2021-07-02 17:30 +0200
          Re: Whole Disk Encryption + SSD David Christensen <dpchrist@holgerdanske.com> - 2021-07-02 20:40 +0200

csiph-web