Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #230267

Re: recommendations for supported, affordable hardware raid controller.

Show all headers | View raw

On 2021-01-02 03:24, Andrei POPESCU wrote:

> http://www.unixsheikh.com/articles/battle-testing-data-integrity-verification-with-zfs-btrfs-and-mdadm-dm-integrity.html

That looks interesting.  Thanks for the link.  :-)


On 2021-01-02 08:08, Richard Hector wrote:
> On 3/01/21 12:24 am, Andrei POPESCU wrote:
>> In case of data corruption (system crash, power outage, user error,
>> or even just a HDD "hiccup") plain md without the dm-integrity
>> layer won't even be able to tell which is the good data and will
>> overwrite your good data with bad data. Silently.

> I've had crashes and power outages and never noticed any problems,
> not that that means they won't happen (or even that they haven't
> happened). Does a journalling filesystem on top not cover that?

AIUI a journaling filesystem provides a two-step process to achieve 
atomic writes of multiple sectors to disk -- e.g. a process wants to put 
some data into a block here (say, a file), a block there (say, a 
directory), etc., and consistency of the on-disk data structures must be 
preserved.  The journal provides a two-step process whereby everything 
is written to the journal, then everything is written to disk.  If 
either step is interrupted, the filesystem driver will detect the 
failure and respond.  When done, either all of the blocks have been 
updated on disk or none of the blocks on disk have been changed.


Integrity checking addresses different failure modes by applying 
checksums to data blocks and metadata blocks.  If the contents of a 
block become corrupt, either in memory, in transit, on disk, etc., the 
driver will detect the failure and respond.  If redundant data is 
available, such as via RAID, the driver will correct the data and 
operations continue.  If no redundant data is available, the driver will 
generate an error.  File system layering features in the Linux kernel 
allow you to add the dm-integrity device mapper layer into a storage 
stack as desired:

https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/dm-integrity.html


On a related note, it is wise to have ECC memory to protect against data 
corruption in memory:

http://www.openoid.net/will-zfs-and-non-ecc-ram-kill-your-data/


More failure modes exist (potentially, an infinite number).  It's a 
question of what failure modes and effects concern you, and how much 
time and money you want to spend to mitigate risks.


David

Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

recommendations for supported, affordable hardware raid controller. Steven Mainor <steve@degga.net> - 2021-01-01 19:10 +0100
  Re: recommendations for supported, affordable hardware raid  controller. "Alexander V. Makartsev" <avbetev@gmail.com> - 2021-01-01 20:00 +0100
  Re: recommendations for supported, affordable hardware raid  controller. Dan Ritter <dsr@randomstring.org> - 2021-01-01 20:00 +0100
    Re: recommendations for supported, affordable hardware raid controller. Sven Hartge <sven@svenhartge.de> - 2021-01-01 20:40 +0100
  Re: recommendations for supported, affordable hardware raid  controller. David Christensen <dpchrist@holgerdanske.com> - 2021-01-01 21:10 +0100
    Re: recommendations for supported, affordable hardware raid  controller. Steven Mainor <steve@degga.net> - 2021-01-02 09:30 +0100
      Re: recommendations for supported, affordable hardware raid  controller. Steven Mainor <steve@degga.net> - 2021-01-02 09:50 +0100
  Re: recommendations for supported, affordable hardware raid  controller. Michael Stone <mstone@debian.org> - 2021-01-01 21:20 +0100
  Re: recommendations for supported, affordable hardware raid  controller. Steven Mainor <steve@degga.net> - 2021-01-02 09:20 +0100
    Re: recommendations for supported, affordable hardware raid  controller. David Christensen <dpchrist@holgerdanske.com> - 2021-01-02 10:50 +0100
      Re: recommendations for supported, affordable hardware raid  controller. Andrei POPESCU <andreimpopescu@gmail.com> - 2021-01-02 12:30 +0100
        Re: recommendations for supported, affordable hardware raid  controller. Richard Hector <richard@walnut.gen.nz> - 2021-01-02 17:10 +0100
          Re: recommendations for supported, affordable hardware raid controller. Gene Heskett <gheskett@shentel.net> - 2021-01-02 17:50 +0100
        Re: recommendations for supported, affordable hardware raid  controller. David Christensen <dpchrist@holgerdanske.com> - 2021-01-02 22:40 +0100
          Re: recommendations for supported, affordable hardware raid  controller. Andrei POPESCU <andreimpopescu@gmail.com> - 2021-01-03 10:30 +0100
            Re: recommendations for supported, affordable hardware raid  controller. David Christensen <dpchrist@holgerdanske.com> - 2021-01-03 22:50 +0100
              Re: recommendations for supported, affordable hardware raid  controller. Andrei POPESCU <andreimpopescu@gmail.com> - 2021-01-04 08:30 +0100
            Re: recommendations for supported, affordable hardware raid controller. Stefan Monnier <monnier@iro.umontreal.ca> - 2021-01-03 23:20 +0100
            Re: recommendations for supported, affordable hardware raid  controller. Michael Stone <mstone@debian.org> - 2021-01-04 02:00 +0100
              Re: recommendations for supported, affordable hardware raid  controller. Andrei POPESCU <andreimpopescu@gmail.com> - 2021-01-04 08:40 +0100
                Re: recommendations for supported, affordable hardware raid  controller. Michael Stone <mstone@debian.org> - 2021-01-04 14:40 +0100
    Re: recommendations for supported, affordable hardware raid controller. Sven Hartge <sven@svenhartge.de> - 2021-01-02 12:50 +0100
      Re: recommendations for supported, affordable hardware raid controller. Nicholas Geovanis <nickgeovanis@gmail.com> - 2021-01-02 16:30 +0100
        Re: recommendations for supported, affordable hardware raid controller. Sven Hartge <sven@svenhartge.de> - 2021-01-02 17:30 +0100
        Re: recommendations for supported, affordable hardware raid  controller. Alain D D Williams <addw@phcomp.co.uk> - 2021-01-02 18:40 +0100
  Re: recommendations for supported, affordable hardware raid controller. deloptes <deloptes@gmail.com> - 2021-01-02 10:00 +0100

csiph-web