More RAID weirdness: external RAID over network

From	Nicolas George <george@nsup.org>
Newsgroups	linux.debian.user
Subject	More RAID weirdness: external RAID over network
Date	2023-03-17 11:10 +0100
Message-ID	<Gafc5-d5co-1@gated-at.bofh.it> (permalink)
Organization	linux.* mail to news gateway

Show all headers | View raw

Hi.

Is this possible: ?

I have a block device on the local host /dev/something with data on it.

I have a block device on the remote host remote:/dev/something with the
exact same size but no relevant data on it.

I have room on the local host in /data or an extra block device (could
be loop).

What I want to do:

1. Stop programs and umount /dev/something

2. mdadm --create /dev/md0 --level=mirror --force --raid-devices=1 \
  --metadata-file /data/raid_something /dev/something

→ Now I have /dev/md0 that is an exact image of /dev/something, with
changes on it synced instantaneously.

3. mount /dev/md0 and restart programs.

→ Now I have everything running again completely normally after a very
short service interruption. But behind the scenes files operations go
through /dev/md0 before reaching /dev/something. If I want to go back, I
de-configure /dev/md0 and can start using /dev/something directly again.

4. mdadm --add /dev/md0 remote:/dev/something && mdadm --grow /dev/md0 --raid-devices=2

→ The kernel starts syncing the contents of /dev/md0=/dev/something onto
remote:/dev/something.

5. Wait a long time.

→ When the sync is done, /dev/something and remote:/dev/something are
exact copies of each other, with /dev/md0 a view of both, and every
write to /dev/md0 is almost instantaneously synced onto them.

6. umount /dev/md0 && mdadm --stop /dev/md0 && stop services

7. ssh remote → mount /dev/something and start programs.

→ Now the programs are running on remote instead of local.


The points of all this:

- The service interruption is only between 1 and 3 and 6 and 7, very
  short. The blocking step is 5 while the RAID synchronizes, but the
  system is working normally during that time, just with slightly
  reduced performance due to the syncing.

- The system keeps track of the sync, I do not have to start again or
  figure out where it stopped like I would have to if I just copied
  /dev/something over SSH.


To say it differently, what I would like:

- A kind-of-RAID1 (it does not have to actually be mdadm), where all
  devices are mirror of each other.

- The metadata and bitmaps of the RAID resides separately.

- The contents of the underlying devices are exactly identical and the
  RAID device shows it exactly.

- When creating the RAID, one of the underlying is considered already
  valid and its data is preserved.

- Underlying devices can be on the network; they can be write-only.


Does anything like this exist?

Regards,

-- 
  Nicolas George

Back to linux.debian.user | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

More RAID weirdness: external RAID over network Nicolas George <george@nsup.org> - 2023-03-17 11:10 +0100
  Re: More RAID weirdness: external RAID over network Reco <recoverym4n@enotuniq.net> - 2023-03-17 12:00 +0100
    Re: More RAID weirdness: external RAID over network Nicolas George <george@nsup.org> - 2023-03-17 14:00 +0100
      Re: More RAID weirdness: external RAID over network Reco <recoverym4n@enotuniq.net> - 2023-03-17 15:40 +0100
        Re: More RAID weirdness: external RAID over network Nicolas George <george@nsup.org> - 2023-03-17 15:50 +0100
          Re: More RAID weirdness: external RAID over network Reco <recoverym4n@enotuniq.net> - 2023-03-17 16:20 +0100
            Re: More RAID weirdness: external RAID over network Nicolas George <george@nsup.org> - 2023-03-17 16:30 +0100
              Re: More RAID weirdness: external RAID over network Dan Ritter <dsr@randomstring.org> - 2023-03-17 17:10 +0100
                Re: More RAID weirdness: external RAID over network Nicolas George <george@nsup.org> - 2023-03-17 17:10 +0100
                Re: More RAID weirdness: external RAID over network Tim Woodall <debianuser@woodall.me.uk> - 2023-03-17 18:00 +0100
                Re: More RAID weirdness: external RAID over network Greg Wooledge <greg@wooledge.org> - 2023-03-17 20:10 +0100
                Re: More RAID weirdness: external RAID over network Nicolas George <george@nsup.org> - 2023-03-17 21:50 +0100
                Re: More RAID weirdness: external RAID over network Nicolas George <george@nsup.org> - 2023-03-17 22:00 +0100
                Re: More RAID weirdness: external RAID over network Tim Woodall <debianuser@woodall.me.uk> - 2023-03-17 23:50 +0100
                Re: More RAID weirdness: external RAID over network Nicolas George <george@nsup.org> - 2023-03-20 11:50 +0100
            RAID1 + iSCSI as backup (was Re: More RAID weirdness: external RAID  over network) Gregory Seidman <gsslist+debian@anthropohedron.net> - 2023-03-17 21:00 +0100
              Re: RAID1 + iSCSI as backup (was Re: More RAID weirdness: external  RAID over network) Dan Ritter <dsr@randomstring.org> - 2023-03-17 22:10 +0100
              Re: RAID1 + iSCSI as backup (was Re: More RAID weirdness: external  RAID over network) David Christensen <dpchrist@holgerdanske.com> - 2023-03-18 02:10 +0100
                Re: RAID1 + iSCSI as backup (was Re: More RAID weirdness: external  RAID over network) Gregory Seidman <gsslist+debian@anthropohedron.net> - 2023-03-18 03:30 +0100
                Re: RAID1 + iSCSI as backup (was Re: More RAID weirdness: external  RAID over network) David Christensen <dpchrist@holgerdanske.com> - 2023-03-18 09:10 +0100
                Re: RAID1 + iSCSI as backup (was Re: More RAID weirdness: external  RAID over network) Dan Ritter <dsr@randomstring.org> - 2023-03-19 02:40 +0100

csiph-web