Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1226600 > unrolled thread

Failover root devices

Back to article view | Back to linux.kernel

Contents

  Failover root devices Drew DeVault <sir@cmpwn.com> - 2015-09-17 02:30 +0200
    Re: Failover root devices Austin S Hemmelgarn <ahferroin7@gmail.com> - 2015-09-17 18:10 +0200
      Re: Failover root devices Drew DeVault <sir@cmpwn.com> - 2015-09-17 19:40 +0200
        Re: Failover root devices Austin S Hemmelgarn <ahferroin7@gmail.com> - 2015-09-18 16:40 +0200
          Re: Failover root devices Drew DeVault <sir@cmpwn.com> - 2015-09-18 16:50 +0200

#1226600 — Failover root devices

I would like to see Linux support multiple root devices, so that it can
attempt one and move on to the next if it is not present. I've reviewed
the relevant code during boot-up and it seems like a good place for me
to submit my first patch, but I want to bring it up for discussion here
on LKML first.

The design I had in mind is something like this:

root=device;device;device;...

Where 'device' follows the current format (/dev/sdX, UUIDs, and so on,
via name_to_dev_t). I would modify prepare_namespace to iterate through
each offered root device until one works.

My use-case for this feature is that I would like to be able to change
the hardware of my machine and boot up differently based on what's
present. In my case, I would like to install my system normally, with
/boot on its own partition, and keep a seperate userspace on a flash
drive. Then, during boot-up, if the flash drive is present, it would be
used as the root device. If it's not present, a partition on disk would
be selected.

The only potential roadblock with this feature that comes to mind is
figuring out how to handle time-outs between root devices. I think it
would be wise to choose a sensible default value, and provide another
cmdline parameter to tweak it. The prepare_namespace flow might end up
looking something like this:

1. Wait rootdelay seconds
2. Check 1st device, not present
3. Recheck 1st device until rootfailoverdelay seconds has passed
4. Move on to 2nd device, present -> boot

Or:

1. Wait rootdelay seconds
2. Check 1st device, not present
3. Recheck 1st device until rootfailoverdelay seconds has passed
4. Move on to 2nd device, not present
5. Recheck 2st device until rootfailoverdelay seconds has passed
6. GOTO 2

And so on.

I also need to research how the various init systems interact with this
part of the boot process. I suspect systemd probably does something
silly wrt waiting for the root device. Since this feature would (of
course) be backwards compatible, it might be wise to just implement it
here and let the init systems add support for the feature themselves.

Advice? Who should I send my patches to when they're ready? Please CC
me, I do not subscribe to LKML.

--
Drew DeVault
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1227177

[Multipart message — attachments visible in raw view] — view raw

On 2015-09-16 20:16, Drew DeVault wrote:
> I would like to see Linux support multiple root devices, so that it can
> attempt one and move on to the next if it is not present. I've reviewed
> the relevant code during boot-up and it seems like a good place for me
> to submit my first patch, but I want to bring it up for discussion here
> on LKML first.
>
> The design I had in mind is something like this:
>
> root=device;device;device;...
>
> Where 'device' follows the current format (/dev/sdX, UUIDs, and so on,
> via name_to_dev_t). I would modify prepare_namespace to iterate through
> each offered root device until one works.
>
> My use-case for this feature is that I would like to be able to change
> the hardware of my machine and boot up differently based on what's
> present. In my case, I would like to install my system normally, with
> /boot on its own partition, and keep a seperate userspace on a flash
> drive. Then, during boot-up, if the flash drive is present, it would be
> used as the root device. If it's not present, a partition on disk would
> be selected.
I think this is an excellent idea, in addition to the above use-case, it 
would allow for distros to automatically launch a recovery image if the 
main root device has failed for some reason.

That said, using the term failover for this is probably not the best 
idea, many people associate it almost exclusively with online failover 
and high-availability setups, and trying to do something like that with 
the root file system is just asking for trouble (I'll be happy to go 
into specifics as to why if someone asks).
> The only potential roadblock with this feature that comes to mind is
> figuring out how to handle time-outs between root devices. I think it
> would be wise to choose a sensible default value, and provide another
> cmdline parameter to tweak it. The prepare_namespace flow might end up
> looking something like this:
>
> 1. Wait rootdelay seconds
> 2. Check 1st device, not present
> 3. Recheck 1st device until rootfailoverdelay seconds has passed
> 4. Move on to 2nd device, present -> boot
>
> Or:
>
> 1. Wait rootdelay seconds
> 2. Check 1st device, not present
> 3. Recheck 1st device until rootfailoverdelay seconds has passed
> 4. Move on to 2nd device, not present
> 5. Recheck 2st device until rootfailoverdelay seconds has passed
> 6. GOTO 2
>
> And so on.
As for this, I'd say default to the first method, and then provide an 
option to switch to the second (both have practical uses).
> I also need to research how the various init systems interact with this
> part of the boot process. I suspect systemd probably does something
> silly wrt waiting for the root device. Since this feature would (of
> course) be backwards compatible, it might be wise to just implement it
> here and let the init systems add support for the feature themselves.
If you're using an initramfs (which is a requirement from what I 
understand for using systemd), then this could be done entirely in the 
initramfs.  The issue with that is that there is no standard syntax for 
doing it, and no way to do it without an initramfs (both of which would 
be nice to have).
> Advice? Who should I send my patches to when they're ready? Please CC
> me, I do not subscribe to LKML.
Use scripts/getmaintainer.pl (or just check the MAINTAINERS file 
directly) to determine this, but make sure to Cc at least LKML for the 
changes as well.

[toc] | [prev] | [next] | [standalone]

#1227245

> That said, using the term failover for this is probably not the best
> idea, many people associate it almost exclusively with online failover
> and high-availability setups, and trying to do something like that with
> the root file system is just asking for trouble (I'll be happy to go
> into specifics as to why if someone asks).

Do you have a suggestion for another name for this feature? Maybe we can 
just call it "multiple root devices". The issue comes with the 
associated command line options, like "rootfailoverdelay". Perhaps it 
could be called "rootcycledelay". "rootdelay" is the obvious one, but 
it's taken for another feature.

>> 1. Wait rootdelay seconds
>> 2. Check 1st device, not present
>> 3. Recheck 1st device until rootfailoverdelay seconds has passed
>> 4. Move on to 2nd device, present -> boot
>>
>> Or:
>>
>> 1. Wait rootdelay seconds
>> 2. Check 1st device, not present
>> 3. Recheck 1st device until rootfailoverdelay seconds has passed
>> 4. Move on to 2nd device, not present
>> 5. Recheck 2st device until rootfailoverdelay seconds has passed
>> 6. GOTO 2
>>
>> And so on.
> As for this, I'd say default to the first method, and then provide an
> option to switch to the second (both have practical uses).

Sorry to cause confusion - these are actually the same method, but 
handling different scenarios. The first is dealing with the first device 
being nonexistent, and the second device existing. The second is dealing 
with both being nonexistent, and cycling between them until one of them 
shows up. After further thought, though, I think the best solution is a 
bit different: a new command line option called "rootmultiwait" or 
similar, which is a maximum amount of time to wait for the user's first 
choice of root device to become available, then testing all devices 
until that time runs out or the first choice becomes available.

--
Drew DeVault
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1227921

[Multipart message — attachments visible in raw view] — view raw

On 2015-09-17 13:30, Drew DeVault wrote:
>> That said, using the term failover for this is probably not the best
>> idea, many people associate it almost exclusively with online failover
>> and high-availability setups, and trying to do something like that with
>> the root file system is just asking for trouble (I'll be happy to go
>> into specifics as to why if someone asks).
>
> Do you have a suggestion for another name for this feature? Maybe we can
> just call it "multiple root devices". The issue comes with the
> associated command line options, like "rootfailoverdelay". Perhaps it
> could be called "rootcycledelay". "rootdelay" is the obvious one, but
> it's taken for another feature.
Possibly 'multirootdelay'?

However, is there any case you can think of for wanting the values to be 
different between rootdelay and the per-device scan delay other than 
having the per-device scan delay be 0 and rootdelay be >0?

The way I'd probably write it would be:
1. Wait rootdelay seconds
2. Check for 1st device
3. If first device is not there, check for 2nd
4. If second device is not there, check next one
5. Repeat 4 until all devices are checked.
6. If a device wasn't found, check if we were told to loop until one is 
found, and if so, start at 1 again.
And then add an option to tell it to wait 'rootdelay' seconds between 
checking each device.
>
>>> 1. Wait rootdelay seconds
>>> 2. Check 1st device, not present
>>> 3. Recheck 1st device until rootfailoverdelay seconds has passed
>>> 4. Move on to 2nd device, present -> boot
>>>
>>> Or:
>>>
>>> 1. Wait rootdelay seconds
>>> 2. Check 1st device, not present
>>> 3. Recheck 1st device until rootfailoverdelay seconds has passed
>>> 4. Move on to 2nd device, not present
>>> 5. Recheck 2st device until rootfailoverdelay seconds has passed
>>> 6. GOTO 2
>>>
>>> And so on.
>> As for this, I'd say default to the first method, and then provide an
>> option to switch to the second (both have practical uses).
>
> Sorry to cause confusion - these are actually the same method, but
> handling different scenarios. The first is dealing with the first device
> being nonexistent, and the second device existing. The second is dealing
> with both being nonexistent, and cycling between them until one of them
> shows up. After further thought, though, I think the best solution is a
> bit different: a new command line option called "rootmultiwait" or
> similar, which is a maximum amount of time to wait for the user's first
> choice of root device to become available, then testing all devices
> until that time runs out or the first choice becomes available.
I think there's value in being able to tell it to go through each one 
exactly once, and halt like it does now if it can't find the filesystem 
on any of them.  That should probably be the default behavior in fact, 
as it's more similar to what's done now.

Secondarily, I've been thinking more about this, and I think it would be 
wonderful to have such functionality in the nfsroot code as well (and 
for that matter, also in any other built-in networked root filesystem 
support).

[toc] | [prev] | [next] | [standalone]

#1227931

> Possibly 'multirootdelay'?

I had the same thought, but wanted to avoid using any prefix other than 
root*= since it would break tradition for this part of the kernel.

> However, is there any case you can think of for wanting the values to be
> different between rootdelay and the per-device scan delay other than
> having the per-device scan delay be 0 and rootdelay be >0?

rootdelay is not really a part of this flow, to be honest. It's a number 
of seconds that it just blocks before trying any devices at all. It 
might not be necessary to make them seperate, though. What's the 
"per-device scan delay" you're speaking of?

> The way I'd probably write it would be:
> 1. Wait rootdelay seconds
> 2. Check for 1st device
> 3. If first device is not there, check for 2nd
> 4. If second device is not there, check next one
> 5. Repeat 4 until all devices are checked.
> 6. If a device wasn't found, check if we were told to loop until one is
> found, and if so, start at 1 again.
> And then add an option to tell it to wait 'rootdelay' seconds between
> checking each device.

-snip-

 > I think there's value in being able to tell it to go through each one
 > exactly once, and halt like it does now if it can't find the filesystem
 > on any of them.  That should probably be the default behavior in fact,
 > as it's more similar to what's done now.

What is the behavior if we weren't told to loop, but reach the end of 
the list? I don't want to try one device for a while and move on - I'd 
prefer to try all devices, then loop. With the other strategy, what 
happens if you try the first device for a while, then move on to the 
second, and the first device comes up while you're waiting on the 
second? The way the user would want that to play out is to boot the 
first device (since it is their preference, after all), but the actual 
behavior will be to boot the second if it comes up during this time.

> Secondarily, I've been thinking more about this, and I think it would be
> wonderful to have such functionality in the nfsroot code as well (and
> for that matter, also in any other built-in networked root filesystem
> support).

I agree that it would be great, but there be dragons. I'm nervous to 
attempt that in my first patch (or even my first few).

--
Drew DeVault
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web