Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #239879 > unrolled thread

debian-installer RAID question

Back to article view | Back to linux.debian.user

Contents

  debian-installer RAID question Felix Natter <fnatter@gmx.net> - 2021-09-11 19:00 +0200
    Re: debian-installer RAID question Andy Smith <andy@strugglers.net> - 2021-09-11 19:10 +0200
      Re: debian-installer RAID question Felix Natter <fnatter@gmx.net> - 2021-09-18 12:00 +0200
    Re: debian-installer RAID question Reco <recoverym4n@enotuniq.net> - 2021-09-11 19:20 +0200
      Re: debian-installer RAID question Felix Natter <fnatter@gmx.net> - 2021-09-18 12:10 +0200
    Re: debian-installer RAID question Sven Hartge <sven@svenhartge.de> - 2021-09-11 19:30 +0200
      Re: debian-installer RAID question Felix Natter <fnatter@gmx.net> - 2021-09-18 12:00 +0200
    Re: debian-installer RAID question Andrei POPESCU <andreimpopescu@gmail.com> - 2021-09-12 12:20 +0200
      Re: debian-installer RAID question Felix Natter <fnatter@gmx.net> - 2021-09-18 11:50 +0200
    Re: debian-installer RAID question David Christensen <dpchrist@holgerdanske.com> - 2021-09-12 21:10 +0200
    Re: debian-installer RAID question Darac Marjal <mailinglist@darac.org.uk> - 2021-09-12 23:00 +0200
      Re: debian-installer RAID question Felix Natter <fnatter@gmx.net> - 2021-09-18 11:50 +0200

#239879 — debian-installer RAID question

hello fellow Debian users,

I have an SSD for the root filesystem, and two HDDs using RAID1 for
/storage running Debian10. Now I need a plan B in case the upgrade
fails.

So I made an experiment with a VM and rougly the same setup (disk-wise),
and found out that when reinstalling Debian11, the d-i does recognize
the RAID1 (/storage) and can reuse it while keeping the data.

My question is: How does d-i know how the individual HDDs were combined
into a RAID1? For all that "sudo fdisk -l" shows, the disks are
"Linux raid autodetect". For all I see, it could be a RAIDX, X!=1 or
two different RAIDs .... Are there RAID headers on the partitions?

The same thing applies when I boot a GNU/Linux rescue system: I think I
can mount the RAID1 if I know the member partitions and type of RAID
using mdadm?

Many Thanks and Best Regards,
-- 
Felix Natter
debian/rules!

[toc] | [next] | [standalone]

#239880

Hello,

On Sat, Sep 11, 2021 at 06:55:56PM +0200, Felix Natter wrote:
> How does d-i know how the individual HDDs were combined into a
> RAID1?

d-i doesn't as such. In Linux MD, member devices contain metadata to
allow assembly by udev. You can examine the data on an MD member
device like this:

$ sudo mdadm --examine /dev/sda1
/dev/sda1:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : ae2dfc21:c73ccb97:24ca9476:e67c0532
           Name : arran:0  (local to host arran)
  Creation Time : Sat Dec  2 09:29:21 2017
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 3901440 (1905.00 MiB 1997.54 MB)
     Array Size : 1950720 (1905.00 MiB 1997.54 MB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
   Unused Space : before=1968 sectors, after=0 sectors
          State : clean
    Device UUID : 0bfc69e4:8e62ef0b:b9c16b32:d7cee4f1

Internal Bitmap : 8 sectors from superblock
    Update Time : Wed Sep  8 23:28:20 2021
       Checksum : 3daa61a0 - correct
         Events : 283


   Device Role : Active device 0
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)

This is how any system with Linux MD support can assemble and
manipulate your arrays given just the devices. Typically udev sees a
block device with MD metadata and calls the mdadm binary to
incrementally assemble it. Once enough devices have been seen the
array is functional.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [prev] | [next] | [standalone]

#240102

Many Thanks for the very helpful reply Andy!
-- 
Felix Natter
debian/rules!

[toc] | [prev] | [next] | [standalone]

#239881

	Hi.

On Sat, Sep 11, 2021 at 06:55:56PM +0200, Felix Natter wrote:
> My question is: How does d-i know how the individual HDDs were combined
> into a RAID1?

mdraid stores its metadata on each drive that belongs to the RAID.
Whenever it's the beginning of the drive, or the end of it - depends on
mdraid metadata version - mdadm(8).

At least three things are stored on each drive:
- mdraid metadata version
- UUID of RAID array itself
- hostname that was used to create an array.

So, answering your question - d-i does not have to know all this.
It's kernel's job to detect your drives, and userspace's (mdadm) one to
search for mdraid metadata and assemble an appropriate array from the
detected drives.
RAID array assembly merely signals the kernel to consider a set of
drives an array.


tl;dr version. Each time you run "mdadm --detail --scan" all parts of
the result are taken from the metadata that's stored on each drive.


> The same thing applies when I boot a GNU/Linux rescue system: I think I
> can mount the RAID1 if I know the member partitions and type of RAID
> using mdadm?

The general answer is - you have to assemble (mdadm -A) your array
first. Before the assembly you have to know which arrays you have, hence
the need of "mdadm --detail --scan", or /etc/mdadm.conf.
Any rescue system worthy of its title should perform mdraid assembly for you.

Reco

[toc] | [prev] | [next] | [standalone]

#240105

Many Thanks for the very helpful reply, Reco!
-- 
Felix Natter
debian/rules!

[toc] | [prev] | [next] | [standalone]

#239882

Felix Natter <fnatter@gmx.net> wrote:

> My question is: How does d-i know how the individual HDDs were combined
> into a RAID1? For all that "sudo fdisk -l" shows, the disks are
> "Linux raid autodetect". For all I see, it could be a RAIDX, X!=1 or
> two different RAIDs .... Are there RAID headers on the partitions?

Yes, there is metadata telling the code exactly what type of RAID it is
and the IDs of the componenet devices it comprises of are.

S°

-- 
Sigmentation fault. Core dumped.

[toc] | [prev] | [next] | [standalone]

#240103

Thanks Sven!
-- 
Felix Natter
debian/rules!

[toc] | [prev] | [next] | [standalone]

#239910

[Multipart message — attachments visible in raw view] — view raw

On Sb, 11 sep 21, 18:55:56, Felix Natter wrote:
> hello fellow Debian users,
> 
> I have an SSD for the root filesystem, and two HDDs using RAID1 for
> /storage running Debian10. Now I need a plan B in case the upgrade
> fails.
> 
> So I made an experiment with a VM and rougly the same setup (disk-wise),
> and found out that when reinstalling Debian11, the d-i does recognize
> the RAID1 (/storage) and can reuse it while keeping the data.

Careful here! You can tell d-i to not format an existing partition, but 
if you want to use it as the root partition the new system will probably 
be installed in the same file structure as your other data.

Is this what you're considering?

Provided you have sufficient space on the RAID you should rather create 
a dedicated partition for the backup install, even if you need to shrink 
another partition (or several). The installer should be able to do that 
as well, though backing up your data is recommended[1].

On the other hand, as plan B in case of upgrade failures you can also 
use the rescue mode of the Debian Installer or, if you need a full 
system, install Debian to a USB stick. The speed might not be great[2], 
but it should be sufficient to get you out of trouble.

If you go this route make sure the stick still boots and works as 
expected even if you disconnect all other storage from the system.

[1] Just in case it's not obvious, the RAID won't help in case the 
resizing goes wrong (it shouldn't, but...).

[2] The speed will depend on the particular stick (look for one with 
consistent good write speed) and ports (USB 3 or better if possible, but 
USB 2 will also work). Avoid Desktop Environments if possible, or at 
least choose a light one like LXDE. 

Kind regards,
Andrei
-- 
http://wiki.debian.org/FAQsFromDebianUser

[toc] | [prev] | [next] | [standalone]

#240101

hi Andrei,

Andrei POPESCU <andreimpopescu@gmail.com> writes:

thank you for your answer.

> On Sb, 11 sep 21, 18:55:56, Felix Natter wrote:
>> hello fellow Debian users,
>> 
>> I have an SSD for the root filesystem, and two HDDs using RAID1 for
>> /storage running Debian10. Now I need a plan B in case the upgrade
>> fails.
>> 
>> So I made an experiment with a VM and rougly the same setup (disk-wise),
>> and found out that when reinstalling Debian11, the d-i does recognize
>> the RAID1 (/storage) and can reuse it while keeping the data.
>
> Careful here! You can tell d-i to not format an existing partition, but 
> if you want to use it as the root partition the new system will probably 
> be installed in the same file structure as your other data.
>
> Is this what you're considering?

No, the RAID1 is for /storage, the root filesystem (which I intend to
format) is on a separate SSD.

Thanks and Best Regards,
Felix
-- 
Felix Natter
debian/rules!

[toc] | [prev] | [next] | [standalone]

#239932

On 9/11/21 9:55 AM, Felix Natter wrote:
> hello fellow Debian users,
> 
> I have an SSD for the root filesystem, and two HDDs using RAID1 for
> /storage running Debian10. Now I need a plan B in case the upgrade
> fails.
> 
> So I made an experiment with a VM and rougly the same setup (disk-wise),
> and found out that when reinstalling Debian11, the d-i does recognize
> the RAID1 (/storage) and can reuse it while keeping the data.
> 
> My question is: How does d-i know how the individual HDDs were combined
> into a RAID1? For all that "sudo fdisk -l" shows, the disks are
> "Linux raid autodetect". For all I see, it could be a RAIDX, X!=1 or
> two different RAIDs .... Are there RAID headers on the partitions?
> 
> The same thing applies when I boot a GNU/Linux rescue system: I think I
> can mount the RAID1 if I know the member partitions and type of RAID
> using mdadm?
> 
> Many Thanks and Best Regards,
> 

I would backup everything, unmount the RAID, shutdown, disconnect the 
RAID devices, remove the SSD, insert a blank SSD, boot the Debian 11 
installer, do a fresh install, configure, do some basic testing, 
shutdown, connect the RAID devices, boot, mount the RAID, and finish 
configuring/ testing.


If Debian 11 turns out to be bad, revert to the Debian 10 SSD.


If the RAID gets trashed, restore from backup.


David

[toc] | [prev] | [next] | [standalone]

#239933

[Multipart message — attachments visible in raw view] — view raw

On 11/09/2021 17:55, Felix Natter wrote:
> hello fellow Debian users,
>
> I have an SSD for the root filesystem, and two HDDs using RAID1 for
> /storage running Debian10. Now I need a plan B in case the upgrade
> fails.

Just want to check that you've not missed something obvious here. You
don't *need* to involve Debian-Installer in an upgrade. Debian fully
supports in-place upgrades. The process is documented here:
https://www.debian.org/releases/bullseye/amd64/release-notes/ch-upgrading.en.html


>
> So I made an experiment with a VM and rougly the same setup (disk-wise),
> and found out that when reinstalling Debian11, the d-i does recognize
> the RAID1 (/storage) and can reuse it while keeping the data.
>
> My question is: How does d-i know how the individual HDDs were combined
> into a RAID1? For all that "sudo fdisk -l" shows, the disks are
> "Linux raid autodetect". For all I see, it could be a RAIDX, X!=1 or
> two different RAIDs .... Are there RAID headers on the partitions?
>
> The same thing applies when I boot a GNU/Linux rescue system: I think I
> can mount the RAID1 if I know the member partitions and type of RAID
> using mdadm?
>
> Many Thanks and Best Regards,

[toc] | [prev] | [next] | [standalone]

#240100

Darac Marjal <mailinglist@darac.org.uk> writes:

> On 11/09/2021 17:55, Felix Natter wrote:
>> hello fellow Debian users,
>>
>> I have an SSD for the root filesystem, and two HDDs using RAID1 for
>> /storage running Debian10. Now I need a plan B in case the upgrade
>> fails.
>
> Just want to check that you've not missed something obvious here. You
> don't *need* to involve Debian-Installer in an upgrade. Debian fully
> supports in-place upgrades. The process is documented here:
> https://www.debian.org/releases/bullseye/amd64/release-notes/ch-upgrading.en.html

I wrote "Now I need a plan B in case the upgrade fails" ;-)

Thanks and Best Regards,

Felix
-- 
Felix Natter
debian/rules!

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web