Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #188832

Re: Sync two disks and hot swap

Show all headers | View raw

[Multipart message — attachments visible in raw view] - view raw

On 11/9/17 2:01 AM, David Christensen wrote:
> On 11/08/17 17:44, Bob Weber wrote:
>> On 11/8/17 5:59 PM, David Christensen wrote:
>>> I have read articles about building a RAID 1 with three drives, migrating in
>>> data, pulling one drive and placing it off-site, operating in degraded mode on
>>> two drives, and then periodically re-installing the third drive, resilvering,
>>> pulling one drive and placing it off-site, and returning to degraded
>>> operations on two drives.  But STFW just now, I see a lot of posts with titles
>>> indicating this is a bad idea.
> ...
>>> But what I really want is some form of snapshot technology (rsync/hard link,
>>> LVM, btrfs, ZFS) with all the goodies -- realtime compression, realtime
>>> de-duplication, and encryption.  I need a more powerful backup server (many
>>> core processor with AES-NI, 16+ GB RAM, SSD caches, etc.).
>
>> I have used raid 1 to make a drive I can take off site for backup.  You just
>> grow the raid 1 array by one disk and add the disk you want to take out (even on
>> a usb/sata connection ... but slow).   Of course the disk or partition(s) need
>> to be the same size as the array.  Let it sync and then boot to a live cd and
>> you can fail and remove that drive.   Or just power down and remove the drive.
>> That way the embedded file system will be unmounted correctly.  I have then
>> taken that one drive and connected it to another system and been able to run the
>> raid 1 in degraded mode and mount the embedded file system(s) to get to the
>> files.  To make the original raid happy just grow the array again setting the
>> number of drives back to what it was originally (you can grow to a smaller
>> number).  The syncing can be slow since every byte on the drive needs to be
>> "synced" instead of just the space the files take up.
>
> Okay.  What RAID technology were you using -- LVM, mdadm, btrfs, ZFS, other?
>
I use software raid with mdadm.  Its pretty forgiving with powering down and
removing a drive (after sync) and growing the array back down to the original
size.  I mainly do this on my backup server running backuppc.  The files are
compressed and hard linked between backups if they have not changed.  This makes
any other type of offsite backup pretty hard ... rsync just ran out of memory. 
So adding a drive to the raid 1 and syncing is easy in comparison.  Having grub
make the drive bootable  (since the os is also on a raid 1 partition) makes the
drive very easy to just install in new hardware and get going again (assuming
the original backup system was destroyed).

-- 


*...Bob*

Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Sync two disks and hot swap Dominik George <nik@naturalnet.de> - 2017-11-08 12:00 +0100
  Re: Sync two disks and hot swap Michael Stone <mstone@debian.org> - 2017-11-08 18:10 +0100
    Re: Sync two disks and hot swap Dominik George <nik@naturalnet.de> - 2017-11-08 20:00 +0100
      Re: Sync two disks and hot swap Michael Stone <mstone@debian.org> - 2017-11-08 20:20 +0100
        Re: Sync two disks and hot swap Dominik George <nik@naturalnet.de> - 2017-11-09 10:20 +0100
          Re: Sync two disks and hot swap Michael Stone <mstone@debian.org> - 2017-11-09 13:40 +0100
            Re: Sync two disks and hot swap David Christensen <dpchrist@holgerdanske.com> - 2017-11-09 20:00 +0100
              Re: Sync two disks and hot swap Michael Stone <mstone@debian.org> - 2017-11-09 20:40 +0100
                Re: Sync two disks and hot swap Curt <curty@free.fr> - 2017-11-10 11:00 +0100
                Re: Sync two disks and hot swap Curt <curty@free.fr> - 2017-11-10 11:10 +0100
  Re: Sync two disks and hot swap David Christensen <dpchrist@holgerdanske.com> - 2017-11-09 00:10 +0100
    Re: Sync two disks and hot swap Bob Weber <bobrweber@gmail.com> - 2017-11-09 02:50 +0100
      Re: Sync two disks and hot swap David Christensen <dpchrist@holgerdanske.com> - 2017-11-09 08:10 +0100
        Re: Sync two disks and hot swap Bob Weber <bobrweber@gmail.com> - 2017-11-09 23:50 +0100
          Re: Sync two disks and hot swap David Christensen <dpchrist@holgerdanske.com> - 2017-11-10 00:10 +0100

csiph-web