Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #230982 > unrolled thread

Raid 1

Back to article view | Back to linux.debian.user

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Raid 1 mick crane <mick.crane@gmail.com> - 2021-01-22 23:30 +0100
    Re: Raid 1 Linux-Fan <Ma_Sys.ma@web.de> - 2021-01-23 00:10 +0100
    Re: Raid 1 David Christensen <dpchrist@holgerdanske.com> - 2021-01-23 00:20 +0100
    Re: Raid 1 David Christensen <dpchrist@holgerdanske.com> - 2021-01-23 00:20 +0100
    Re: Raid 1 Andrei POPESCU <andreimpopescu@gmail.com> - 2021-01-23 13:30 +0100
      Re: Raid 1 Linux-Fan <Ma_Sys.ma@web.de> - 2021-01-23 18:20 +0100
        Re: Raid 1 Linux-Fan <Ma_Sys.ma@web.de> - 2021-01-23 19:00 +0100
          Re: Raid 1 Pankaj Jangid <pankaj@codeisgreat.org> - 2021-01-25 07:00 +0100
            Re: Raid 1 Andy Smith <andy@strugglers.net> - 2021-01-25 08:20 +0100
              Re: Raid 1 Linux-Fan <Ma_Sys.ma@web.de> - 2021-01-25 12:10 +0100
                Re: Raid 1 Pankaj Jangid <pankaj@codeisgreat.org> - 2021-01-25 13:50 +0100
      Re: Raid 1 David Christensen <dpchrist@holgerdanske.com> - 2021-01-23 23:10 +0100
        Re: Raid 1 Andrei POPESCU <andreimpopescu@gmail.com> - 2021-01-24 18:40 +0100
          Re: Raid 1 Linux-Fan <Ma_Sys.ma@web.de> - 2021-01-24 23:30 +0100
            Re: Raid 1 Andrei POPESCU <andreimpopescu@gmail.com> - 2021-01-25 07:40 +0100
        Re: Raid 1 Andy Smith <andy@strugglers.net> - 2021-01-24 19:00 +0100
          Re: Raid 1 Andrei POPESCU <andreimpopescu@gmail.com> - 2021-01-24 21:20 +0100
        Re: Raid 1 David Christensen <dpchrist@holgerdanske.com> - 2021-01-24 21:20 +0100
          Re: Raid 1 deloptes <deloptes@gmail.com> - 2021-01-25 09:50 +0100
          Re: Raid 1 David Christensen <dpchrist@holgerdanske.com> - 2021-01-25 19:00 +0100

#230982 — Raid 1

hello,
I want to tidy things up as suggested.
Have one old PC that I'll put 2 disks in and tidy everything up so 
what's scattered about is on the running disks and this new/old one is 
just backup for them.
Can I assume that Debian installer in some expert mode will sort out the 
raid or do I need to install to one disk and then mirror it manually 
before invoking the raid thing ?

mick

-- 
Key ID    4BFEBB31

[toc] | [next] | [standalone]

#230985

[Multipart message — attachments visible in raw view] — view raw

mick crane writes:

> hello,
> I want to tidy things up as suggested.
> Have one old PC that I'll put 2 disks in and tidy everything up so what's  
> scattered about is on the running disks and this new/old one is just backup  
> for them.
> Can I assume that Debian installer in some expert mode will sort out the  
> raid or do I need to install to one disk and then mirror it manually before  
> invoking the raid thing ?

Debian Installer can create MDADM RAID volumes even in non-expert mode.

You need to explicitly select the right options in the installer  
partitioning screen i.e. create the partitions, then create MDADM RAID 1  
devices on top of them and finally let them be formatted with  
ext4/filesystem of choice and be the installation target.

AFAIK "Guided" installation modes do not automatically create RAID, i.e. I  
recommend using the manual partitioning mode.

In the few RAID installs I did, it worked out all of the time.

Only thing to do afterwards is to ensure that GRUB is installed on both of  
the respective devices (dpkg-reconfigure grub-pc assuming BIOS mode).

HTH
Linux-Fan

öö

[toc] | [prev] | [next] | [standalone]

#230987

On 2021-01-22 15:10, David Christensen wrote:

> A key issue with storage is bit rot.

I should have said "bit rot protection".


David

[toc] | [prev] | [next] | [standalone]

#230988

On 2021-01-22 14:26, mick crane wrote:
> hello,
> I want to tidy things up as suggested.
> Have one old PC that I'll put 2 disks in and tidy everything up so 
> what's scattered about is on the running disks and this new/old one is 
> just backup for them.
> Can I assume that Debian installer in some expert mode will sort out the 
> raid or do I need to install to one disk and then mirror it manually 
> before invoking the raid thing ?


I would install a small SSD and do a fresh install of the OS onto that.


I would then install the two HDD's and set up a mirror (RAID 1).  Linux 
options include Multiple Device md(4), Linux Volume Manager lvm(8), and 
ZFS zfs(8).


A key issue with storage is bit rot.  btrfs and ZFS have it. 
dm-integrity (man page?) can provide it for Linux solutions without. 
btrfs requires maintenance.  I did not do it, and my disks suffered. 
ZFS does not require maintenance and has many killer features.  I have 
not tried dm-integrity, but would be interested in reading a HOWTO for 
Debian.


Due to CDDL and GPL licensing conflicts, ZFS is not fully integrated 
into Debian.  ZFS can be installed and used on Debian, but ZFS-on-root 
is not supported by the Debian installer.


The CDDL and BSD licenses are compatible.  So, ZFS is fully integrated 
on FreeBSD, and the FreeBSD installer can do ZFS-on-root.  FreeBSD has 
other features I like.  I use FreeBSD and ZFS on my servers; including 
storage (Samba).


David

[toc] | [prev] | [next] | [standalone]

#231021

[Multipart message — attachments visible in raw view] — view raw

On Vi, 22 ian 21, 22:26:46, mick crane wrote:
> hello,
> I want to tidy things up as suggested.
> Have one old PC that I'll put 2 disks in and tidy everything up so what's
> scattered about is on the running disks and this new/old one is just backup
> for them.
> Can I assume that Debian installer in some expert mode will sort out the
> raid or do I need to install to one disk and then mirror it manually before
> invoking the raid thing ?

The "raid thing" is a separate layer below the partitions and file 
systems.

Technically it is possible to create the mirror with just one device (I 
believe mdadm calls this "degraded"), partition the md mirror device, 
install, copy data to it, etc., add the second device later and let md 
synchronize the two drives.

Because Linux RAID is a separate layer with no knowledge of the data 
"above" it has to copy every single bit to the other drive as well 
(similar to a dd device-to-device copy), regardless if actually needed 
or not.

If you are really strapped for space and must do this ZFS can do it much 
more efficiently, because it controls the entire "stack" and knows 
exactly which blocks to copy (besides many other advantages over Linux 
RAID).

Unfortunately ZFS is slightly more complicated from the packaging side, 
and installing Debian on a ZFS root is difficult.

It still makes an excelent choice to manage your storage drives, 
especially on a stable system, where there is less hassle with the dkms 
module and it's amazingly simple to use once you familiarise yourself 
with the basics.

Kind regards,
Andrei
-- 
http://wiki.debian.org/FAQsFromDebianUser

[toc] | [prev] | [next] | [standalone]

#231028

[Multipart message — attachments visible in raw view] — view raw

mick crane writes:

> On 2021-01-23 12:20, Andrei POPESCU wrote:
>> On Vi, 22 ian 21, 22:26:46, mick crane wrote:
>>> hello,
>>> I want to tidy things up as suggested.
>>> Have one old PC that I'll put 2 disks in and tidy everything up so what's
>>> scattered about is on the running disks and this new/old one is just backup
>>> for them.
>>> Can I assume that Debian installer in some expert mode will sort out the
>>> raid or do I need to install to one disk and then mirror it manually before
>>> invoking the raid thing ?

[...]

> Sigh, OK I take advice and have a go.
> Really I just want to get on and do some drawings or something but I think  
> I'll thank myself later if I get proper backup in place.
> If after having a quick look am I understanding anything?
> Partition and install minimal Debian with no X or anything on just one disk.
> install headers and zfs-utils.
> Add other disk and then what ? To make it a mirror pool (like raid1) does  
> zfs take care of the partitions.
> Do I want to delete all partitions on other disk first or make like for like  
> partitions?

If I get your scenario correctly you want to install Debian (without ZFS  
i.e. not "root on ZFS") and then create a ZFS mirror?

If yes, then as a preparation you need either (a) two entire devices of  
~ same size to use with ZFS or (b) two partitions to use with ZFS.

Say you install as follows:

 * sda1: OS
 * sda2: Swap
 * sda : XX GiB free

 * sdb: XX+ Gib free

Then prepare two unformatted partitions:

 * sda3: XX GiB "for ZFS"
 * sdb1: XX GiB "for ZFS"

and use these devices for ZFS.

> If that's done and I've made a zpool called  "backup" from then on the ZFS  
> is nothing to do with the kernel ?
> I ask kernel make a directory "my_pc1"
> then
> "zfs create -o mountpoint=/my_pc1 backup/my_pc1"

You can specifiy a mountpoint and it will be created automatically. No need  
to pre-create the Directory as with other file systems.

> I ask kernel make a directory "my_pc2"
> then
> "zfs create -o mountpoint=/my_pc2 backup/my_pc2"
>
> So then I can copy files from other PC (pc1) to "my_backup_pc/backup/my_pc1"  
> and ZFS mirrors the data to other disk in pool ?

Yes. In case you are unsure check the output of `zpool status` to see the  
structure as understood by ZFS.

> If that's how it works I'll just need something on the backup_pc and the  
> other PCs to automate the backing up.
> Is that backup Ninja or something ?

I have never used backup Ninja. Depending on your use case anything from  
simple rsync to borgbackup may serve :)

Please note that "root on ZFS" is possible but quite complicated:
https://openzfs.github.io/openzfs-docs/Getting%20Started/Debian/Debian%20Buster%20Root%20on%20ZFS.html

For my current system I actually used mdadm RAID 1 for OS+Swap and ZFS  
mirrors for the actual data. This way, I can use the Debian Installer for  
installation purposes and benefit from the bit rot protection for the  
acutally important data while maintaining basic redundancy for the OS  
installation. YMMV.

Here are my notes on essential ZFS commands (in case they might be of help):
https://masysma.lima-city.de/37/zfs_commands_shortref.xhtml

HTH
Linux-Fan

öö

[toc] | [prev] | [next] | [standalone]

#231029

[Multipart message — attachments visible in raw view] — view raw

mick crane writes:

> On 2021-01-23 17:11, Linux-Fan wrote:
>> mick crane writes:

[...]

>> Please note that "root on ZFS" is possible but quite complicated:
>> https://openzfs.github.io/openzfs-docs/Getting%20Started/Debian/Debian 
>> %20Buster%20Root%20on%20ZFS.html
>>
>> For my current system I actually used mdadm RAID 1 for OS+Swap and ZFS
>> mirrors for the actual data. This way, I can use the Debian Installer
>> for installation purposes and benefit from the bit rot protection for
>> the acutally important data while maintaining basic redundancy for the
>> OS installation. YMMV.
>>
>> Here are my notes on essential ZFS commands (in case they might be of help):
>> https://masysma.lima-city.de/37/zfs_commands_shortref.xhtml

[...]

> link is not currently available.
> what you seem to be doing there is backing up the data with ZFS but not  
> backing up the OS, so I guess your raid is the backup for the OS ?
> mick

Both open fine here, which of the links fails for you?

RAID is not Backup! Hence I have entirely separate programs for backup. The  
RAID is only the "quickest" layer -- solely responsible for catching  
problems with randomly failing HDDs and -- for non-OS-data -- bit rot.

My system works as follows

 * OS data bitrot is not covered, but OS single HDD failure is.
   I achieve this by having OS and Swap on MDADM RAID 1
   i.e. mirrored but without ZFS.

 * Actual data bitrot is covered, as is single HDD failure by
   means of ZFS mirrors for all data.

 * Backups are separate. For instance, important data is copied to a
   separate computer upon shutdown. Less important data is part of
   manually-invoked backup tasks which use multiple programs to cope
   with different types of data...

HTH
Linux-Fan

öö

[toc] | [prev] | [next] | [standalone]

#231065

Linux-Fan <Ma_Sys.ma@web.de> writes:

> * OS data bitrot is not covered, but OS single HDD failure is.
>   I achieve this by having OS and Swap on MDADM RAID 1
>   i.e. mirrored but without ZFS.

I am still learning.

1. By "by having OS and Swap on MDADM", did you mean the /boot partition
   and swap.

2. Why did you put Swap on RAID? What is the advantage?

   - I understood that RAID is used to detect disk failures early. How
     do you decide which partition to cover and which not?

[toc] | [prev] | [next] | [standalone]

#231074

Hi Pankaj,

Not wishing to put words in Linux-Fan's mouth, but my own views
are…

On Mon, Jan 25, 2021 at 11:04:09AM +0530, Pankaj Jangid wrote:
> Linux-Fan <Ma_Sys.ma@web.de> writes:
> 
> > * OS data bitrot is not covered, but OS single HDD failure is.
> >   I achieve this by having OS and Swap on MDADM RAID 1
> >   i.e. mirrored but without ZFS.
> 
> I am still learning.
> 
> 1. By "by having OS and Swap on MDADM", did you mean the /boot partition
>    and swap.

When people say, "I put OS and Swap on MDADM" they typically mean
the entire installed system before user/service data is put on it.
So that's / and all its usual sub-directories, and swap, possibly
with things later split off after install.

> 2. Why did you put Swap on RAID? What is the advantage?

If you have swap used, and the device behind it goes away, your
system will likely crash.

The point of RAID is to increase availability. If you have the OS
itself in RAID and you have swap, the swap should be in RAID too.

There are use cases where the software itself provides the
availability. For example, there is Ceph, which typically uses
simple block devices from multiple hosts and distributes the data
around.

A valid setup for Ceph is to have the OS in a small RAID just so
that a device failure doesn't take down a machine entirely, but then
have the data devices stand alone as Ceph itself will handle a
failure of those. Small boot+OS devices are cheap and it's so simple
to RAID them.

Normally Ceph is set up so that an entire host can be lost. If host
reinstallation is automatic and quick and there's so many hosts that
losing any one of them is a fairly minor occurrence then it could be
valid to not even put the OS+swap in RAID. Though for me it still
sounds like a lot more hassle than just replacing a dead drive in a
running machine, so I wouldn't do it personally.

>    - I understood that RAID is used to detect disk failures early.

Not really. Although with RAID or ZFS or the like it is typical to
have a periodic (weekly, monthly, etc) scrub that reads all data and
may uncover drive problems like unreadable sectors, usually failures
happen when they will happen. The difference is that a copy of the
data still exists somewhere else, so that can be used and the
failure does not have to propagate to the application.

> How do you decide which partition to cover and which not?

For each of the storage devices in your system, ask yourself:

- Would your system still run if that device suddenly went away?

- Would your application(s) still run if that device suddenly went
  away?

- Could finding a replacement device and restoring your data from
  backups be done in a time span that you consider reasonable?

If the answer to those questions are not what you could tolerate,
add some redundancy in order to reduce unavailability. If you decide
you can tolerate the possible unavailability then so be it.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [prev] | [next] | [standalone]

#231084

[Multipart message — attachments visible in raw view] — view raw

Andy Smith writes:

> Hi Pankaj,
>
> Not wishing to put words in Linux-Fan's mouth, but my own views
> are…
>
> On Mon, Jan 25, 2021 at 11:04:09AM +0530, Pankaj Jangid wrote:
> > Linux-Fan <Ma_Sys.ma@web.de> writes:
> >
> > > * OS data bitrot is not covered, but OS single HDD failure is.
> > >   I achieve this by having OS and Swap on MDADM RAID 1
> > >   i.e. mirrored but without ZFS.
> >
> > I am still learning.
> >
> > 1. By "by having OS and Swap on MDADM", did you mean the /boot partition
> >    and swap.
>
> When people say, "I put OS and Swap on MDADM" they typically mean
> the entire installed system before user/service data is put on it.
> So that's / and all its usual sub-directories, and swap, possibly
> with things later split off after install.

Yes, that is exactly how I meant it :)

My current setup has two disks each partitioned as follows:

 * first   partition ESP          for /boot/efi (does not support RAID)
 * sencond partition MDADM RAID 1 for / (including /boot and /home)
 * third   partition MDADM RAID 1 for swap
 * fourth  partition ZFS mirror   for virtual machines and containers

Some may like to have /home separately. I personally prefer to store all my  
user-created data outside of the /home tree because many programs are using  
/home structures for cache and configuration files that are automatically  
generated and should (IMHO) not be mixed with what I consider important data.

> > 2. Why did you put Swap on RAID? What is the advantage?
>
> If you have swap used, and the device behind it goes away, your
> system will likely crash.
>
> The point of RAID is to increase availability. If you have the OS
> itself in RAID and you have swap, the swap should be in RAID too.

That was exactly my reasoning, too. I can add that I did not use a ZFS  
volume for the swap mostly because of
https://github.com/openzfs/zfs/issues/7734
and I did not use it for the OS (/, /boot, /home) mainly because I wanted to  
avoid getting a non-booting system in case anything fails with the ZFS  
module DKMS build. The added benefit was a less complex installation  
procedure i.e. using Debian installer was possible and all ZFS stuff could  
be done from the installed and running system.

I would advise against replicating my setup for first-time RAID users  
because restore after a failed disk will require invoking the respective  
restoration procedures of both technologies.

> There are use cases where the software itself provides the
> availability. For example, there is Ceph, which typically uses
> simple block devices from multiple hosts and distributes the data
> around.

Yes.

[...]

> > How do you decide which partition to cover and which not?
>
> For each of the storage devices in your system, ask yourself:
>
> - Would your system still run if that device suddenly went away?
>
> - Would your application(s) still run if that device suddenly went
>   away?
>
> - Could finding a replacement device and restoring your data from
>   backups be done in a time span that you consider reasonable?
>
> If the answer to those questions are not what you could tolerate,
> add some redundancy in order to reduce unavailability. If you decide
> you can tolerate the possible unavailability then so be it.

[...]

My rule of thumb: RAID 1 whenever possible i.e. on all actively relied-upon  
computers that are not laptops or other special form factors with tightly  
limited HDD/SSD options.

The replacement drive considerations are important for RAID setups, too. I  
used to have a "cold spare" HDD but given the rate at which the  
capacity/price ratio rises I thought it to be overly cautious/expensive to  
keep that scheme.

HTH
Linux-Fan

öö

[toc] | [prev] | [next] | [standalone]

#231090

Thanks Andy and Linux-Fan, for the detailed reply.

[toc] | [prev] | [next] | [standalone]

#231034

On 2021-01-23 07:01, mick crane wrote:
> On 2021-01-23 12:20, Andrei POPESCU wrote:
>> On Vi, 22 ian 21, 22:26:46, mick crane wrote:
>>> hello,
>>> I want to tidy things up as suggested.
>>> Have one old PC that I'll put 2 disks in and tidy everything up so 
>>> what's
>>> scattered about is on the running disks and this new/old one is just 
>>> backup
>>> for them.
>>> Can I assume that Debian installer in some expert mode will sort out the
>>> raid or do I need to install to one disk and then mirror it manually 
>>> before
>>> invoking the raid thing ?
>>
>> The "raid thing" is a separate layer below the partitions and file
>> systems.
>>
>> Technically it is possible to create the mirror with just one device (I
>> believe mdadm calls this "degraded"), partition the md mirror device,
>> install, copy data to it, etc., add the second device later and let md
>> synchronize the two drives.
>>
>> Because Linux RAID is a separate layer with no knowledge of the data
>> "above" it has to copy every single bit to the other drive as well
>> (similar to a dd device-to-device copy), regardless if actually needed
>> or not.
>>
>> If you are really strapped for space and must do this ZFS can do it much
>> more efficiently, because it controls the entire "stack" and knows
>> exactly which blocks to copy (besides many other advantages over Linux
>> RAID).
>>
>> Unfortunately ZFS is slightly more complicated from the packaging side,
>> and installing Debian on a ZFS root is difficult.
>>
>> It still makes an excelent choice to manage your storage drives,
>> especially on a stable system, where there is less hassle with the dkms
>> module and it's amazingly simple to use once you familiarise yourself
>> with the basics.
>>
>> Kind regards,
>> Andrei
> 
> Sigh, OK I take advice and have a go.
> Really I just want to get on and do some drawings or something but I 
> think I'll thank myself later if I get proper backup in place.
> If after having a quick look am I understanding anything?
> Partition and install minimal Debian with no X or anything on just one 
> disk.
> install headers and zfs-utils.
> Add other disk and then what ? To make it a mirror pool (like raid1) 
> does zfs take care of the partitions.
> Do I want to delete all partitions on other disk first or make like for 
> like partitions?
> 
> If that's done and I've made a zpool called  "backup" from then on the 
> ZFS is nothing to do with the kernel ?
> I ask kernel make a directory "my_pc1"
> then
> "zfs create -o mountpoint=/my_pc1 backup/my_pc1"
> 
> I ask kernel make a directory "my_pc2"
> then
> "zfs create -o mountpoint=/my_pc2 backup/my_pc2"
> 
> So then I can copy files from other PC (pc1) to 
> "my_backup_pc/backup/my_pc1" and ZFS mirrors the data to other disk in 
> pool ?
> 
> If that's how it works I'll just need something on the backup_pc and the 
> other PCs to automate the backing up.
> Is that backup Ninja or something ?


RAID protects against storage device sectors going bad and against 
entire storage devices going bad -- e.g. hard disk drives, solid state 
drives, etc..


Backups protect against filesystem contents going bad -- e.g. files, 
directories, metadata, etc..


While putting an operating system and backups within a single RAID can 
be done, this will complicate creation of a ZFS pool and will complicate 
disaster preparedness/ recovery procedures.  The following instructions 
assume your OS is on one device and that you will dedicate two HDD's to ZFS.


See "Creating a Mirrored Storage Pool":

https://docs.oracle.com/cd/E19253-01/819-5461/gaynr/index.html


The above URL is good for concepts, but the virtual device names 
('c1d0', 'c2d0') are for Solaris.  For Debian, you will want to 
zero-fill both HDD's with dd(1) and then create the pool with zpool(8) 
using device identity nodes:

     /dev/disk/by-id/ata-...


Be extremely careful that you specify the correct devices!


ZFS will mark the drives and create a ZFS pool named 'tank' mounted at 
'/tank'.  Note the parallel namespaces -- 'tank'is ZFS namespace and has 
no leading slash, while '/tank' is a Unix absolute path.


'/tank' is a ZFS filesystem that can do everything a normal Unix 
directory can do.  So, you could create a directory for backups and 
create directories for specific machines:

# mkdir /tank/backup

# mkdir /tank/backup/pc1

# mkdir /tank/backup/pc2


Or, you could create a ZFS filesystem for backups and create ZFS 
filesystems for specific machines:

# zfs create tank/backup

# zfs create tank/backup/pc1

# zfs create tank/backup/pc2


Both will give you directories that you can put your backups into using 
whatever tools you choose, but the latter will give you additional ZFS 
capabilities.


David

[toc] | [prev] | [next] | [standalone]

#231043

[Multipart message — attachments visible in raw view] — view raw

On Du, 24 ian 21, 11:36:09, mick crane wrote:
> 
> I know I'm a bit thick about these things, what I'm blocked about is where
> is the OS.
> Let's say I have one PC and 2 unpartitioned disks.
> Put one disk in PC and install Debian on it.

Ok

> Install headers and ZFS-utils.
> I put other disk in PC, PC boots from first disk.

Ok.

> "zpool create tank mirror disk1 disk2"

This will destroy all data already existing on disk1 and disk2 (though I 
strongly suspect zpool will simply refuse to use disk1). Same with Linux 
RAID.

Creating the RAID (Linux or ZFS) will overwrite any data already 
existing on the disks / partitions used for the RAID.

If you want to have the OS on RAID it's probably easiest to let the 
installer configure that for you. This implies *both* disks are 
available during install (unless the installer can create a "degraded" 
RAID).

Installing Debian on ZFS involves manual steps anyway, so it's basically 
create the pool with just one disk, install Debian and then 'attach' the 
other disk to the first one.

If you want to combine Linux RAID and ZFS on just two drives you could 
partition the drives (e.g. two partitions on each drive), use the first 
partition on each drive for Linux RAID, install Debian (others will have 
to confirm whether the installer supports creating RAID from partitions) 
and then use the other partitions for the ZFS pool.

You might want to experiment with this in a VM first. For testing 
purposes you can also experiment with ZFS on files instead of real 
devices / partitions (probably with Linux RAID as well).

Kind regards,
Andrei
-- 
http://wiki.debian.org/FAQsFromDebianUser

[toc] | [prev] | [next] | [standalone]

#231058

[Multipart message — attachments visible in raw view] — view raw

mick crane writes:

> On 2021-01-24 17:37, Andrei POPESCU wrote:

[...]

>> If you want to combine Linux RAID and ZFS on just two drives you could
>> partition the drives (e.g. two partitions on each drive), use the first
>> partition on each drive for Linux RAID, install Debian (others will have
>> to confirm whether the installer supports creating RAID from partitions)
>> and then use the other partitions for the ZFS pool.

I can confirm that this works. In fact, I always thought that to be the  
"best practice" for MDADM: To use individual partitions rather than whole  
devices. OTOH for ZFS, best practice seems to be to use entire devices. I am  
not an expert on this, though :)

>> You might want to experiment with this in a VM first. For testing
>> purposes you can also experiment with ZFS on files instead of real
>> devices / partitions (probably with Linux RAID as well).
>>
>> Kind regards,
>> Andrei
>
> This is my problem "where is the OS to be running the ZFS to put Debian on ?"

You could use a live system, for instance. Beware that this route is  
complicated. I linked to the guide in a previous mail but am not sure if you  
were finally able to check it (you mentioned at least one of my links not  
being accessible, but not which one...).

> All I want to do is back up PCs to another and have that have redundancy  
> with 2 disks so if one gets borked I can still use the other and put things  
> back together.
> How do I do that ?

My recommendation would be to keep it simple stupid: Let the installer setup  
RAID 1 MDADM for OS, swap and data and be done with it, avoid ZFS unless  
there is some reason to need it :)

For sure MDADM lacks the bit rot protection, but it is easier to setup  
especially for the OS and you can mitigate the bit rot (to some extent)  
by running periodic backup integrity checks which your software hopefully  
supports.

HTH
Linux-Fan

öö

[...]

[toc] | [prev] | [next] | [standalone]

#231071

[Multipart message — attachments visible in raw view] — view raw

On Du, 24 ian 21, 23:21:38, Linux-Fan wrote:
> mick crane writes:
> 
> > On 2021-01-24 17:37, Andrei POPESCU wrote:
> 
> [...]
> 
> > > If you want to combine Linux RAID and ZFS on just two drives you could
> > > partition the drives (e.g. two partitions on each drive), use the first
> > > partition on each drive for Linux RAID, install Debian (others will have
> > > to confirm whether the installer supports creating RAID from partitions)
> > > and then use the other partitions for the ZFS pool.
> 
> I can confirm that this works. In fact, I always thought that to be the
> "best practice" for MDADM: To use individual partitions rather than whole
> devices. OTOH for ZFS, best practice seems to be to use entire devices. I am
> not an expert on this, though :)

ZFS is actually using GPT partitions and also automatically creates a 
"reserve" 8 MiB partition, just in case a replacement disk is not 
exactly the same size as the other disk(s) in a VDEV.

So far I haven't found a way around it (not that I care, as I prefer to 
partition manually and identify physical devices by partition label)

Kind regards,
Andrei
-- 
http://wiki.debian.org/FAQsFromDebianUser

[toc] | [prev] | [next] | [standalone]

#231044

Hi Mick,

On Sun, Jan 24, 2021 at 11:36:09AM +0000, mick crane wrote:
> I know I'm a bit thick about these things, what I'm blocked about is where
> is the OS.

Wherever you installed it.

> Let's say I have one PC and 2 unpartitioned disks.
> Put one disk in PC and install Debian on it.

I think you are fundamentally going about this the wrong way.

There are several concerns and I think you are mixing them up. If I
understand you correctly, you concerns are:

1. Your data and OS should be backed up.
2. Your data and OS should be available even if a disk dies

Concern #1 is totally separate from concern #2 and is achieved by
setting up a backup system, has very little to do with whether you
use RAID or ZFS or whatever. It is worth a separate thread because
it's separate project.

For concern #2, that being *availability* of data and OS, there's
many ways to do it. You seem to have settled upon ZFS for your data,
and OS separately by some other means. That's fine.

A ZFS mirror vdev is going to need two identically-sized devices.
And you want to keep your OS separate. This suggests that each of
your disks should have two partitions. The first one would be for
the OS, and the second one would be for ZFS.

If you are going to keep your OS separate, I don't see any reason
not to use mdadm RAID-1 for the OS even if you're going to use zfs
for your data. Yes you could just install the OS onto a single
partition of a single disk, but you have two disks so why not use
RAID-1? If a disk breaks, your computer carries on working, what's
not to like?

So personally I would just do the install of Debian with both disks
inside the machine, manual partitioning, create a single partition
big enough for your OS on the first disk and then another one the
same on the second disk. Mark them as RAID members, set them to
RAID-1, install on that.

Once it's up and running you can then go and create a second
partition that spans the rest of each disk, and then when you are
ready to create your zfs pool:

> "zpool create tank mirror disk1 disk2"

# zpool create tank mirror /dev/disk/by-id/ata-DISK1MODEL-SERIAL-part2 /dev/disk/by-id/ata-DISK2MODEL-SERIAL-part2

The DISK1MODEL-SERIAL bits will be different for you based on what
the model and serial numbers are of your disks. Point is it's a pair
of devices that are partition 2 of each disk.

> Can I then remove disk1 and PC will boot Debian from disk2 ?

This is only going to work if you have gone to the effort of
installing your OS on RAID. The easiest way to achieve that is to
have both disks in the machine when you install it and to properly
tell it that the first partition of each is a RAID member, create
them as a RAID-1 and tell the installer to install onto that.

As other mentioned, after it's installed you do have to manually
install the grub bootloader to the second device as well as by
default it only gets installed on the first one.

A word of warning: RAID is quite a big topic for the uninitiated and
so is ZFS. You are proposing to take on both at once. You have some
learning to do. You may make mistakes, and this data seems precious
to you. I advise you to sort out the backups first. You might need
them sooner than you'd hoped.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [prev] | [next] | [standalone]

#231051

[Multipart message — attachments visible in raw view] — view raw

On Du, 24 ian 21, 17:50:06, Andy Smith wrote:
> 
> Once it's up and running you can then go and create a second
> partition that spans the rest of each disk, and then when you are
> ready to create your zfs pool:
> 
> > "zpool create tank mirror disk1 disk2"
> 
> # zpool create tank mirror /dev/disk/by-id/ata-DISK1MODEL-SERIAL-part2 /dev/disk/by-id/ata-DISK2MODEL-SERIAL-part2
> 
> The DISK1MODEL-SERIAL bits will be different for you based on what
> the model and serial numbers are of your disks. Point is it's a pair
> of devices that are partition 2 of each disk.

At this point I'd recommend to use GPT partition labels instead (not to 
be confused with file system labels). Assuming labels datapart1 and 
datapart2 the create becomes:

    # zpool create tank mirror /dev/disk/by-partlabel/datapart1 /dev/disk/by-partlabel/datapart2

Now the output of 'zpool status' and all other commands will show the 
human-friendly labels instead of the device ID.


Kind regards,
Andrei
-- 
http://wiki.debian.org/FAQsFromDebianUser

[toc] | [prev] | [next] | [standalone]

#231050

On 2021-01-24 03:36, mick crane wrote:

> Let's say I have one PC and 2 unpartitioned disks.

Please tell us why you must put the OS and the backup images on the same 
RAID mirror of two HDD's, and why you cannot add one (or two?) more 
devices for the OS.


David

[toc] | [prev] | [next] | [standalone]

#231076

mick crane wrote:

> I think I'll go with the first and last suggestion to just have 2 disks
> in raid1.
> It seems that properly you'd want 2 disks in raid for the OS, 2 at least
> for the pool and maybe 1 for the cache.
> Don't have anything big enough I could put 5 disks in.
> I could probably get 3 disks in. Install the OS on one and then dd that
> to another and put that in a drawer and have another 2 disks as the zfs
> pool. I might have a fiddle about and see what goes on.

Hi,
I have not followed this thread closely, but my advise is keep it as simple
as possible.
Very often people here overcomplicate things - geeks and freaks - in the
good sense - but still if you do not know ZFS or can not afford the
infrastructure for that, just leave it.

In my usecase I came with following solution:

md0 - boot disk (ext3)
md1 - root disk (ext4)
md2 - swap
md3 - LVM for user data (encrypted + xfs)

I have this on two disks that were replaced and "grown" from 200GB to 1TB
over the past 18y. Some of the Seagates I used in the beginning died and
RAID1 payed off.

Planning to move to GPT next md0 will be converted to EFI disk (FAT32) or I
will just create one additional partition on each disk for the EFI stuff.
I'm not sure if I need it at all, so I must be really bored to touch this.

[toc] | [prev] | [next] | [standalone]

#231120

On 2021-01-24 21:23, mick crane wrote:
> On 2021-01-24 20:10, David Christensen wrote:

>> Please tell us why you must put the OS and the backup images on the
>> same RAID mirror of two HDD's, and why you cannot add one (or two?)
>> more devices for the OS.

> I think I'll go with the first and last suggestion to just have 2 disks 
> in raid1.
> It seems that properly you'd want 2 disks in raid for the OS, 2 at least 
> for the pool and maybe 1 for the cache.
> Don't have anything big enough I could put 5 disks in.
> I could probably get 3 disks in. Install the OS on one and then dd that 
> to another and put that in a drawer and have another 2 disks as the zfs 
> pool. I might have a fiddle about and see what goes on.


If you are short on hardware or money, one option is to install Debian 
onto a USB flash drive.   I ran desktop hardware as servers on USB flash 
drives for many years, and still keep a Debian 9 system on USB flash for 
maintenance purposes.  I have yet to wear one out.  If you feel the need 
for RAID, use two USB flash drives.


David

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web