Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #232258 > unrolled thread

Deb10 installer can't install grub

Back to article view | Back to linux.debian.user

Contents

  Deb10 installer can't install grub Dave Sherohman <dave@sherohman.org> - 2021-03-02 13:30 +0100
    Re: Deb10 installer can't install grub Dan Ritter <dsr@randomstring.org> - 2021-03-02 14:00 +0100
      Re: Deb10 installer can't install grub The Wanderer <wanderer@fastmail.fm> - 2021-03-02 14:30 +0100
        Re: Deb10 installer can't install grub Dan Ritter <dsr@randomstring.org> - 2021-03-02 16:00 +0100
          Re: Deb10 installer can't install grub The Wanderer <wanderer@fastmail.fm> - 2021-03-02 16:00 +0100
          Re: Deb10 installer can't install grub Charles Curley <charlescurley@charlescurley.com> - 2021-03-02 17:20 +0100
      Re: Deb10 installer can't install grub Dave Sherohman <dave@sherohman.org> - 2021-03-02 15:10 +0100
        Re: Deb10 installer can't install grub The Wanderer <wanderer@fastmail.fm> - 2021-03-02 15:20 +0100
          Re: Deb10 installer can't install grub The Wanderer <wanderer@fastmail.fm> - 2021-03-02 15:30 +0100
            Re: Deb10 installer can't install grub Dave Sherohman <dave@sherohman.org> - 2021-03-02 15:40 +0100
          Re: Deb10 installer can't install grub Dave Sherohman <dave@sherohman.org> - 2021-03-02 15:30 +0100
      Re: Deb10 installer can't install grub Felix Miata <mrmazda@earthlink.net> - 2021-03-02 20:10 +0100
        Re: Deb10 installer can't install grub Andrei POPESCU <andreimpopescu@gmail.com> - 2021-03-03 17:00 +0100
          Re: Deb10 installer can't install grub Felix Miata <mrmazda@earthlink.net> - 2021-03-03 23:20 +0100
            Re: Deb10 installer can't install grub Dan Ritter <dsr@randomstring.org> - 2021-03-03 23:50 +0100
              Re: Deb10 installer can't install grub <tomas@tuxteam.de> - 2021-03-04 10:10 +0100
                Re: Deb10 installer can't install grub Greg Wooledge <greg@wooledge.org> - 2021-03-04 14:20 +0100
                  Re: Deb10 installer can't install grub <tomas@tuxteam.de> - 2021-03-04 14:30 +0100
                  Re: Deb10 installer can't install grub Anssi Saari <as@sci.fi> - 2021-03-05 10:30 +0100
                    Re: Deb10 installer can't install grub Darac Marjal <mailinglist@darac.org.uk> - 2021-03-05 11:10 +0100
                      Re: Deb10 installer can't install grub Andy Smith <andy@strugglers.net> - 2021-03-05 13:00 +0100
            Re: Deb10 installer can't install grub <tomas@tuxteam.de> - 2021-03-04 10:10 +0100
            Re: Deb10 installer can't install grub Andrei POPESCU <andreimpopescu@gmail.com> - 2021-03-05 14:50 +0100
    Re: Deb10 installer can't install grub "Andrew M.A. Cater" <amacater@einval.com> - 2021-03-02 22:00 +0100
    Re: Deb10 installer can't install grub Dave Sherohman <dave@sherohman.org> - 2021-03-03 10:00 +0100
      SOLVED - Re: Deb10 installer can't install grub Dave Sherohman <dave@sherohman.org> - 2021-03-03 12:30 +0100
        Re: SOLVED - Re: Deb10 installer can't install grub Marc Auslander <marcausl@gmail.com> - 2021-03-03 16:50 +0100
          Re: SOLVED - Re: Deb10 installer can't install grub Dave Sherohman <dave@sherohman.org> - 2021-03-07 14:10 +0100
            Re: SOLVED - Re: Deb10 installer can't install grub Nicholas Geovanis <nickgeovanis@gmail.com> - 2021-03-07 15:10 +0100
      Re: Deb10 installer can't install grub Anssi Saari <as@sci.fi> - 2021-03-04 10:10 +0100

Page 1 of 2  [1] 2  Next page →

#232258 — Deb10 installer can't install grub

I've got a new server and am currently fighting with the Debian 10
installer (build 20190702) in my attempts to get it up and running.
After much wailing and gnashing of teeth, I managed to get it to stop
complaining about being unble to mount /boot/efi and complete the
"Partition disks" step successfully, but now I'm completely stuck on the
"Install GRUB" step.

The GUI installer shows the error:

Unable to install GRUB in dummy
Executing 'grub-install dummy' failed.

Checking the syslog output on virtual console 4 shows a bunch of
os-prober activity (as expected), then finally:

Installing for x64_64-efi platform.
grub-install: error: failed to get canonical path of `/dev/nvme0n1p1`.
error: Running 'grub-install  --force "dummy"' failed.

I assume that the "canonical path" it's looking for is a /dev/sda-type
device name, but I have no idea how to assign one of those to an nvme
drive.  (And I thought that kind of name was supposed to have been
banished in favor of "predictable" names by now anyhow.)

Several of the old-style names are already in use; the installer is
showing sda for the USB stick that the installation was booted from and
sdb-sdi for eight large disks, but the operating system is to boot and
run from the drives /dev/nvme0n1 and /dev/nvme1n1.

What do I need to do to get this working?

Also, would that solution also work for md devices as well as for nvme
devices?  I had previously tried putting UEFI onto a RAID1 mirror
between the two nvme drives, but got a similar error from grub about not
being able to find the canonical path of /dev/md1.

-- 
Dave Sherohman

[toc] | [next] | [standalone]

#232260

Dave Sherohman wrote: 
> I've got a new server and am currently fighting with the Debian 10
> installer (build 20190702) in my attempts to get it up and running.
> After much wailing and gnashing of teeth, I managed to get it to stop
> complaining about being unble to mount /boot/efi and complete the
> "Partition disks" step successfully, but now I'm completely stuck on the
> "Install GRUB" step.
> 
> The GUI installer shows the error:
> 
> Unable to install GRUB in dummy
> Executing 'grub-install dummy' failed.
> 
> Checking the syslog output on virtual console 4 shows a bunch of
> os-prober activity (as expected), then finally:
> 
> Installing for x64_64-efi platform.
> grub-install: error: failed to get canonical path of `/dev/nvme0n1p1`.
> error: Running 'grub-install  --force "dummy"' failed.
> 
> I assume that the "canonical path" it's looking for is a /dev/sda-type
> device name, but I have no idea how to assign one of those to an nvme
> drive.  (And I thought that kind of name was supposed to have been
> banished in favor of "predictable" names by now anyhow.)
> 
> Several of the old-style names are already in use; the installer is
> showing sda for the USB stick that the installation was booted from and
> sdb-sdi for eight large disks, but the operating system is to boot and
> run from the drives /dev/nvme0n1 and /dev/nvme1n1.
> 
> What do I need to do to get this working?
> 
> Also, would that solution also work for md devices as well as for nvme
> devices?  I had previously tried putting UEFI onto a RAID1 mirror
> between the two nvme drives, but got a similar error from grub about not
> being able to find the canonical path of /dev/md1.

/dev/nvme0n1 is your first NVMe drive; /dev/nvme1n1 is your
second. Add a p1 to each to reference the first partition.

If you want to install to an mdadm RAID, that's do-able with
grub as well. 

In the installer, you want expert mode. Does the disk
partitioner recognize both nvme0n1 and nvme1n1 ? 

You'll want to create the following partitions on each,
identically:

1 efi - type efi
2 boot (or boot/root) - type MDADM volume
3 root, if using separate boot - type MDADM volume
4 swap - type MDADM volume

Then you go to the mdadm setup and create MDADM RAID1 devices
out of each pair of boot, root and swap.

After install to those RAID1 devices, grub will want to install.
Do that to the boot sector of /dev/nvme0n1, and then again to
the boot sector of /dev/nvme1n1.

After a reboot into the new OS, you should set up your storage
disks. 

If this doesn't work, tell us where in the process it stops, and
what the error is there.

-dsr-

[toc] | [prev] | [next] | [standalone]

#232261

[Multipart message — attachments visible in raw view] — view raw

On 2021-03-02 at 07:54, Dan Ritter wrote:

> You'll want to create the following partitions on each,
> identically:
> 
> 1 efi - type efi
> 2 boot (or boot/root) - type MDADM volume
> 3 root, if using separate boot - type MDADM volume
> 4 swap - type MDADM volume
> 
> Then you go to the mdadm setup and create MDADM RAID1 devices
> out of each pair of boot, root and swap.

Out of interest, how would you modify this for someone who intends to
run LVM on top of the RAID-1 being defined here, and wants to define as
many of the partitions as possible (but at least /) at the LVM level
rather than the mdadm level?

I'm guessing that the answer would be to define the EFI partition here
directly, then define one further partition as an appropriate type, set
up LVM inside that, and define the further partitions from there.

I'm not positive that that's the correct approach, however, especially
given that I suspect that /boot will need to be visible to GRUB and may
thus need to be outside the LVM.


This isn't purely academic interest; I'm not moving forward right now
because I can't currently afford all the components, but I'm planning a
new computer build as soon as that changes, and one part of the design
I've decided on is a pair of M.2 SSDs in RAID-1.

I remember having had enough trouble getting GRUB installed and booting
with a pair of SATA SSDs in RAID-1 when I built my current machine,
which doesn't use GPT or EFI boot, and I'd prefer to avoid having to
juggle that on the fly again - especially not given that the need to
cater to the partitioning requirements of EFI boot is likely to make
things more complicated.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man.         -- George Bernard Shaw

[toc] | [prev] | [next] | [standalone]

#232267

The Wanderer wrote: 
> On 2021-03-02 at 07:54, Dan Ritter wrote:
> 
> > You'll want to create the following partitions on each,
> > identically:
> > 
> > 1 efi - type efi
> > 2 boot (or boot/root) - type MDADM volume
> > 3 root, if using separate boot - type MDADM volume
> > 4 swap - type MDADM volume
> > 
> > Then you go to the mdadm setup and create MDADM RAID1 devices
> > out of each pair of boot, root and swap.
> 
> Out of interest, how would you modify this for someone who intends to
> run LVM on top of the RAID-1 being defined here, and wants to define as
> many of the partitions as possible (but at least /) at the LVM level
> rather than the mdadm level?
> 
> I'm guessing that the answer would be to define the EFI partition here
> directly, then define one further partition as an appropriate type, set
> up LVM inside that, and define the further partitions from there.
> 
> I'm not positive that that's the correct approach, however, especially
> given that I suspect that /boot will need to be visible to GRUB and may
> thus need to be outside the LVM.

grub can find a root in LVM, but needs /boot to be outside.
/boot in an mdadm RAID1 is a good idea there.

-dsr-

[toc] | [prev] | [next] | [standalone]

#232268

[Multipart message — attachments visible in raw view] — view raw

On 2021-03-02 at 09:50, Dan Ritter wrote:

> The Wanderer wrote: 
>
>> On 2021-03-02 at 07:54, Dan Ritter wrote:
>> 
>>> You'll want to create the following partitions on each,
>>> identically:
>>> 
>>> 1 efi - type efi
>>> 2 boot (or boot/root) - type MDADM volume
>>> 3 root, if using separate boot - type MDADM volume
>>> 4 swap - type MDADM volume
>>> 
>>> Then you go to the mdadm setup and create MDADM RAID1 devices
>>> out of each pair of boot, root and swap.
>> 
>> Out of interest, how would you modify this for someone who intends to
>> run LVM on top of the RAID-1 being defined here, and wants to define as
>> many of the partitions as possible (but at least /) at the LVM level
>> rather than the mdadm level?
>> 
>> I'm guessing that the answer would be to define the EFI partition here
>> directly, then define one further partition as an appropriate type, set
>> up LVM inside that, and define the further partitions from there.
>> 
>> I'm not positive that that's the correct approach, however, especially
>> given that I suspect that /boot will need to be visible to GRUB and may
>> thus need to be outside the LVM.
> 
> grub can find a root in LVM, but needs /boot to be outside.
> /boot in an mdadm RAID1 is a good idea there.

So, three partitions, then?

1. efi
2. /boot
3. LVM

And define everything else inside the LVM. (In practice, "everything
else" here will be /, /var, /tmp, and possibly swap. /home will be on a
separate RAID array.)

It'll still probably be interesting figuring out what to specify as the
destination for the GRUB install, but if I'm lucky it might Just Work.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man.         -- George Bernard Shaw

[toc] | [prev] | [next] | [standalone]

#232272

On Tue, 2 Mar 2021 09:50:22 -0500
Dan Ritter <dsr@randomstring.org> wrote:

> > I'm not positive that that's the correct approach, however,
> > especially given that I suspect that /boot will need to be visible
> > to GRUB and may thus need to be outside the LVM.  
> 
> grub can find a root in LVM, but needs /boot to be outside.
> /boot in an mdadm RAID1 is a good idea there.

I'm not sure you need a separate /boot partition. I am using Bullseye
weekly builds, so I may have a more recent d-i and grub than you do.
Also this is on i386 on a small (60GB) drive, and a DOS type MBR.

I have one LVM partition, with swap and root in that. The LVM partition
starts at sector 2048, so I expect grub resides on the first 2048
sectors.

I suspect one can get an idea of things grub can do by
listing /boot/grub/i386-pc/.

-- 
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

[toc] | [prev] | [next] | [standalone]

#232262

On Tue, Mar 02, 2021 at 07:54:01AM -0500, Dan Ritter wrote:
> In the installer, you want expert mode. Does the disk
> partitioner recognize both nvme0n1 and nvme1n1 ? 

Yes.  The list of disks shown in the partitioner is currently:

RAID device #1
RAID device #2
/dev/nvme0n1
/dev/nvme1n1
SCSI2 (0,0,0) (sda) - the USB stick I booted from
SCSI1 (0,0,0) (sdb) - the first storage drive
... the other seven storage drives ...

> You'll want to create the following partitions on each,
> identically:
> 
> 1 efi - type efi
> 2 boot (or boot/root) - type MDADM volume
> 3 root, if using separate boot - type MDADM volume
> 4 swap - type MDADM volume

Partitioning is identical on both nvme drives, consisting of:

1.0   MB  free space
2.0   GB  ESP (bootable)
8.0   GB  swap
1.9   TB  RAID physical device
331.8 kB  free space

> Then you go to the mdadm setup and create MDADM RAID1 devices
> out of each pair of boot, root and swap.

RAID Device #1 is 1.9 TB, ext4fs, and set to mount on /
RAID Device #2 is 2.0 GB, ESP, and bootable

> After install to those RAID1 devices, grub will want to install.
> Do that to the boot sector of /dev/nvme0n1, and then again to
> the boot sector of /dev/nvme1n1.

...and this is where it dies.  I'm now in the text-mode "expert
install", and there is no option to manually specify a partition to
install GRUB on, only a yes/no "Force GRUB installation to the EFI
removable media path?" prompt.  Regardless of which answer I choose, I
get "Unable to install GRUB in dummy" and syslog records "failed to get
canonical path of /dev/md2".

I also tried dropping to a shell and manually running `grub-installer
/dev/nvme0n1` and got the error "ls: /dev/nvme0n1/proc: Not a directory"
Manually running grub-installer for /dev/md2 and /dev/nvme1n1
(unsurprisingly) produced similar results.

-- 
Dave Sherohman

[toc] | [prev] | [next] | [standalone]

#232263

[Multipart message — attachments visible in raw view] — view raw

On 2021-03-02 at 09:01, Dave Sherohman wrote:

> On Tue, Mar 02, 2021 at 07:54:01AM -0500, Dan Ritter wrote:

>> Then you go to the mdadm setup and create MDADM RAID1 devices
>> out of each pair of boot, root and swap.
> 
> RAID Device #1 is 1.9 TB, ext4fs, and set to mount on /
> RAID Device #2 is 2.0 GB, ESP, and bootable

So the EFI partitions themselves are also RAIDed? I wouldn't have
expected that to work.

>> After install to those RAID1 devices, grub will want to install.
>> Do that to the boot sector of /dev/nvme0n1, and then again to
>> the boot sector of /dev/nvme1n1.
> 
> ...and this is where it dies.  I'm now in the text-mode "expert
> install", and there is no option to manually specify a partition to
> install GRUB on, only a yes/no "Force GRUB installation to the EFI
> removable media path?" prompt.  Regardless of which answer I choose, I
> get "Unable to install GRUB in dummy" and syslog records "failed to get
> canonical path of /dev/md2".
> 
> I also tried dropping to a shell and manually running `grub-installer
> /dev/nvme0n1` and got the error "ls: /dev/nvme0n1/proc: Not a directory"
> Manually running grub-installer for /dev/md2 and /dev/nvme1n1
> (unsurprisingly) produced similar results.

Is grub-installer specific to the Debian installer environment? I don't
find it at a glance in my installed system, or with 'apt-file search'; I
do find grub-install, which is a binary (where the error you quote seems
to indicate a script), and is what I seem to recall having used for the
manual installation of GRUB that finally got my current machine working
with a RAIDed boot-drive pair all those years ago. (Albeit not with EFI
boot partitioning, so that may not be directly comparable.)

If grub-installer is a script and is the correct thing to use, the first
thing I'd be inclined to do in your place is to examine that script and
see what it's trying to do at the point of the error, so that I can
figure out what it expects as input and how I can potentially try to
provide that.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man.         -- George Bernard Shaw

[toc] | [prev] | [next] | [standalone]

#232264

[Multipart message — attachments visible in raw view] — view raw

On 2021-03-02 at 09:20, Dave Sherohman wrote:

> On Tue, Mar 02, 2021 at 09:09:52AM -0500, The Wanderer wrote:
> 
>> On 2021-03-02 at 09:01, Dave Sherohman wrote:

>>> RAID Device #1 is 1.9 TB, ext4fs, and set to mount on /
>>> RAID Device #2 is 2.0 GB, ESP, and bootable
>> 
>> So the EFI partitions themselves are also RAIDed? I wouldn't have
>> expected that to work.
> 
> That was my expectation as well, but Dan said it should work, so I went
> with it.

I didn't parse what he wrote that way. He said:

>>>> You'll want to create the following partitions on each,
>>>> identically:
>>>> 
>>>> 1 efi - type efi
>>>> 2 boot (or boot/root) - type MDADM volume
>>>> 3 root, if using separate boot - type MDADM volume
>>>> 4 swap - type MDADM volume
>>>> 
>>>> Then you go to the mdadm setup and create MDADM RAID1 devices
>>>> out of each pair of boot, root and swap.

Note that he specified to RAID the three listed MDADM partitions, but
did not mention the same for the EFI partition.

> Of course, I've also tried it with then not RAIDed and the results
> are identical, aside from the "canonoical path" error message in 
> syslog changing from /dev/md2 to /dev/nvme0n1p1.

Not terribly surprising, but it was worth a check.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man.         -- George Bernard Shaw

[toc] | [prev] | [next] | [standalone]

#232266

On Tue, Mar 02, 2021 at 09:26:21AM -0500, The Wanderer wrote:
> I didn't parse what he wrote that way. He said:
> 
> >>>> You'll want to create the following partitions on each,
> >>>> identically:
> >>>> 
> >>>> 1 efi - type efi
> >>>> 2 boot (or boot/root) - type MDADM volume
> >>>> 3 root, if using separate boot - type MDADM volume
> >>>> 4 swap - type MDADM volume
> >>>> 
> >>>> Then you go to the mdadm setup and create MDADM RAID1 devices
> >>>> out of each pair of boot, root and swap.
> 
> Note that he specified to RAID the three listed MDADM partitions, but
> did not mention the same for the EFI partition.

Ah, right.  When he mentioned RAIDing the boot, I took that to mean the
EFI partitions.  Forgot that he as using a separate /boot.



So I took a look at the grub-installer script, and guess what I found as
a comment just before the `ls` that kills it?

---
# XXX cjwatson 2019-03-25: This is all far too complicated and fragile, and
# should be replaced with in-target or similar.

# Ensure proc is mounted in all the $chroot calls;
# needed for RAID+LVM for example
initial_proc_contents="$(ls $ROOT/proc)"
---

"Far too complicated and fragile", indeed!

-- 
Dave Sherohman

[toc] | [prev] | [next] | [standalone]

#232265

On Tue, Mar 02, 2021 at 09:09:52AM -0500, The Wanderer wrote:
> On 2021-03-02 at 09:01, Dave Sherohman wrote:
> 
> > On Tue, Mar 02, 2021 at 07:54:01AM -0500, Dan Ritter wrote:
> 
> >> Then you go to the mdadm setup and create MDADM RAID1 devices
> >> out of each pair of boot, root and swap.
> > 
> > RAID Device #1 is 1.9 TB, ext4fs, and set to mount on /
> > RAID Device #2 is 2.0 GB, ESP, and bootable
> 
> So the EFI partitions themselves are also RAIDed? I wouldn't have
> expected that to work.

That was my expectation as well, but Dan said it should work, so I went
with it.  Of course, I've also tried it with then not RAIDed and the
results are identical, aside from the "canonoical path" error message in
syslog changing from /dev/md2 to /dev/nvme0n1p1.

> Is grub-installer specific to the Debian installer environment? I don't
> find it at a glance in my installed system, or with 'apt-file search'; I
> do find grub-install, which is a binary (where the error you quote seems
> to indicate a script), and is what I seem to recall having used for the
> manual installation of GRUB that finally got my current machine working
> with a RAIDed boot-drive pair all those years ago. (Albeit not with EFI
> boot partitioning, so that may not be directly comparable.)

Yes, it seems to be unique to the installer environment.  I made that
same trip in reverse, first trying to run grub-install, and then having
to go looking when that didn't exist.

> If grub-installer is a script and is the correct thing to use, the first
> thing I'd be inclined to do in your place is to examine that script and
> see what it's trying to do at the point of the error, so that I can
> figure out what it expects as input and how I can potentially try to
> provide that.

Good thought.  It's a shell script, so I'll see if I can make sense of
what it's doing in [filesystem]/proc.

-- 
Dave Sherohman

[toc] | [prev] | [next] | [standalone]

#232276

Dan Ritter composed on 2021-03-02 07:54 (UTC-0500):

> Dave Sherohman wrote: 

>> I've got a new server...

> You'll want to create the following partitions on each,
> identically:

> 1 efi - type efi
										
Not RAID I presume?

> 2 boot (or boot/root) - type MDADM volume
										
I never have /boot on RAID, but that's just me I suppose.

> 3 root, if using separate boot - type MDADM volume
										
Agree.

> 4 swap - type MDADM volume
										
Think about this:

To start with, RAID1 is marginally slower than ordinary filesystems on partitions.

Does a server use swap to hibernate?

Does a server /want/ to use swap for anything?

Does anyone want swap on any type of SSD?

If I was to put swap on NVME, I would not use a RAID, just like on my old RAID
systems on HDDs. If I had 32G or more RAM, I'd just let the kernel swap to file
instead of devoting discrete partition space to swap.
-- 
Evolution as taught in public schools, like religion,
	is based on faith, not on science.

 Team OS/2 ** Reg. Linux User #211409 ** a11y rocks!

Felix Miata  ***  http://fm.no-ip.com/

[toc] | [prev] | [next] | [standalone]

#232316

[Multipart message — attachments visible in raw view] — view raw

On Ma, 02 mar 21, 14:01:52, Felix Miata wrote:
> 
> To start with, RAID1 is marginally slower than ordinary filesystems on partitions.

This is true for some workloads, for others it can be significantly 
faster.

https://arstechnica.com/information-technology/2020/04/understanding-raid-how-performance-scales-from-one-disk-to-eight/


Kind regards,
Andrei
-- 
http://wiki.debian.org/FAQsFromDebianUser

[toc] | [prev] | [next] | [standalone]

#232330

Andrei POPESCU composed on 2021-03-03 17:50 (UTC+0200):

> Felix Miata wrote:

>> To start with, RAID1 is marginally slower than ordinary filesystems on partitions.

> This is true for some workloads, for others it can be significantly 
> faster.

> https://arstechnica.com/information-technology/2020/04/understanding-raid-how-performance-scales-from-one-disk-to-eight/
										
I wrote not RAID, but RAID1, very purposely. I found no mention of RAID1 in any of
the graphs there, and the subject of RAID1 barely touched, basically describing
its purely mirror topology and little else. I can't recall any graph there that
described performance of anything other than single disk, RAID6, and RAID10. Do
you know of, or can you provide a reference to, any way RAID1 performance can be
better than single disk?
-- 
Evolution as taught in public schools, like religion,
	is based on faith, not on science.

 Team OS/2 ** Reg. Linux User #211409 ** a11y rocks!

Felix Miata  ***  http://fm.no-ip.com/

[toc] | [prev] | [next] | [standalone]

#232331

Felix Miata wrote: 
> Andrei POPESCU composed on 2021-03-03 17:50 (UTC+0200):
> 
> > Felix Miata wrote:
> 
> >> To start with, RAID1 is marginally slower than ordinary filesystems on partitions.
> 
> > This is true for some workloads, for others it can be significantly 
> > faster.
> 
> > https://arstechnica.com/information-technology/2020/04/understanding-raid-how-performance-scales-from-one-disk-to-eight/
> 										
> I wrote not RAID, but RAID1, very purposely. I found no mention of RAID1 in any of
> the graphs there, and the subject of RAID1 barely touched, basically describing
> its purely mirror topology and little else. I can't recall any graph there that
> described performance of anything other than single disk, RAID6, and RAID10. Do
> you know of, or can you provide a reference to, any way RAID1 performance can be
> better than single disk?

Sure.

https://calomel.org/zfs_raid_speed_capacity.html

First two lines of the first table of results:

1x 4TB, single drive,          3.7 TB,  w=108MB/s , rw=50MB/s  , r=204MB/s 
2x 4TB, mirror (raid1),        3.7 TB,  w=106MB/s , rw=50MB/s , r=488MB/s 

Which is as theory predicts: slight reduction in write
performance (write being dominated by disk cache -> disk, not
system -> disk cache), and approximate doubling in read
performance.

-dsr-

[toc] | [prev] | [next] | [standalone]

#232343

[Multipart message — attachments visible in raw view] — view raw

On Wed, Mar 03, 2021 at 05:40:39PM -0500, Dan Ritter wrote:
> Felix Miata wrote: 
> > Andrei POPESCU composed on 2021-03-03 17:50 (UTC+0200):
> > 
> > > Felix Miata wrote:
> > [...] Do
> > you know of, or can you provide a reference to, any way RAID1 performance can be
> > better than single disk?
> 
> Sure.
> 
> https://calomel.org/zfs_raid_speed_capacity.html
> 
> First two lines of the first table of results:
> 
> 1x 4TB, single drive,          3.7 TB,  w=108MB/s , rw=50MB/s  , r=204MB/s 
> 2x 4TB, mirror (raid1),        3.7 TB,  w=106MB/s , rw=50MB/s , r=488MB/s 
> 
> Which is as theory predicts: slight reduction in write
> performance (write being dominated by disk cache -> disk, not
> system -> disk cache), and approximate doubling in read
> performance.

Thanks. Real data :)

The doubling in read throughput is somewhat surprising to me. Some
learning to do, it seems.

Cheers
 - t

[toc] | [prev] | [next] | [standalone]

#232352

tomas@tuxteam.de (tomas@tuxteam.de) wrote:
> > 1x 4TB, single drive,          3.7 TB,  w=108MB/s , rw=50MB/s  , r=204MB/s 
> > 2x 4TB, mirror (raid1),        3.7 TB,  w=106MB/s , rw=50MB/s , r=488MB/s 

> Thanks. Real data :)
> 
> The doubling in read throughput is somewhat surprising to me. Some
> learning to do, it seems.

There are two copies of everything.  So, if you need to read a big
file, you can read half of it from disk 1 and the other half from disk 2,
roughly simultaneously.

[toc] | [prev] | [next] | [standalone]

#232354

[Multipart message — attachments visible in raw view] — view raw

On Thu, Mar 04, 2021 at 08:09:46AM -0500, Greg Wooledge wrote:
> tomas@tuxteam.de (tomas@tuxteam.de) wrote:
> > > 1x 4TB, single drive,          3.7 TB,  w=108MB/s , rw=50MB/s  , r=204MB/s 
> > > 2x 4TB, mirror (raid1),        3.7 TB,  w=106MB/s , rw=50MB/s , r=488MB/s 
> 
> > Thanks. Real data :)
> > 
> > The doubling in read throughput is somewhat surprising to me. Some
> > learning to do, it seems.
> 
> There are two copies of everything.  So, if you need to read a big
> file, you can read half of it from disk 1 and the other half from disk 2,
> roughly simultaneously.

Makes sense, thanks.

Cheers
 - t

[toc] | [prev] | [next] | [standalone]

#232388

Greg Wooledge <greg@wooledge.org> writes:

> tomas@tuxteam.de (tomas@tuxteam.de) wrote:
>> > 1x 4TB, single drive,          3.7 TB,  w=108MB/s , rw=50MB/s  , r=204MB/s 
>> > 2x 4TB, mirror (raid1),        3.7 TB,  w=106MB/s , rw=50MB/s , r=488MB/s 
>
>> Thanks. Real data :)
>> 
>> The doubling in read throughput is somewhat surprising to me. Some
>> learning to do, it seems.
>
> There are two copies of everything.  So, if you need to read a big
> file, you can read half of it from disk 1 and the other half from disk 2,
> roughly simultaneously.

I have a vague memory of reading once that mdraid doesn't stripe reads
to a single file. I.e. with raid1 you get the double speed if you read
different files (or different parts of disk) but reading a single file,
no.

So is this different in zfs? I guess I could test this myself. The way
Bonnie++ was used for those results above is definitely parallel.

[toc] | [prev] | [next] | [standalone]

#232392

[Multipart message — attachments visible in raw view] — view raw

On 05/03/2021 09:25, Anssi Saari wrote:
> Greg Wooledge <greg@wooledge.org> writes:
>
>> tomas@tuxteam.de (tomas@tuxteam.de) wrote:
>>>> 1x 4TB, single drive,          3.7 TB,  w=108MB/s , rw=50MB/s  , r=204MB/s 
>>>> 2x 4TB, mirror (raid1),        3.7 TB,  w=106MB/s , rw=50MB/s , r=488MB/s 
>>> Thanks. Real data :)
>>>
>>> The doubling in read throughput is somewhat surprising to me. Some
>>> learning to do, it seems.
>> There are two copies of everything.  So, if you need to read a big
>> file, you can read half of it from disk 1 and the other half from disk 2,
>> roughly simultaneously.
> I have a vague memory of reading once that mdraid doesn't stripe reads
> to a single file. I.e. with raid1 you get the double speed if you read
> different files (or different parts of disk) but reading a single file,
> no.
RAID won't know what's a file or not. It works below the file system
layer, so only concerns itself with blocks. For any particular block, a
RAID1 array has N copies of that block to read from. It makes no sense
to read all of those copies - imagine if you were really paranoid and
had a 10-disk RAID array. Instead, the driver will choose one of the
constituent drives and read the block from that drive. So, if your file
is small, then yes, you won't see any performance benefit. But if your
file is larger than a block, or if you want to access more than one file
at once, then RAID can read the second block from a different drive.
Remember that  the I in RAID stands for Inexpensive. RAID assumes that
the disk is slow, the bus is faster and the applications are impatient  :)
>
> So is this different in zfs? I guess I could test this myself. The way
> Bonnie++ was used for those results above is definitely parallel.
ZFS will work in much the same manner, except when performing a "scrub".
During a scrub, all copies of all blocks are read and compared. If a
block differs across the mirrors, then ZFS has a decision to make: if
there are more identical copies than different copies, ZFS assumes that
the identical copies are good (hence why having a three-way mirror is
recommended). Alternatively, ZFS uses checksums on stored data. If the
block doesn't match the checksum, then ZFS can also recover that data.
More details here
https://blogs.oracle.com/bonwick/zfs-end-to-end-data-integrity
>

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | linux.debian.user

csiph-web