Re: linux raid vs hw raid

From	Tim Watts <tw@dionic.net>
Newsgroups	comp.os.linux.misc
Subject	Re: linux raid vs hw raid
Followup-To	comp.os.linux.misc
Date	2011-04-06 08:01 +0100
Organization	A noiseless patient Spider
Message-ID	<dmft68-2q8.ln1@squidward.dionic.net> (permalink)
References	<fc0t68x5ci.ln2@goaway.wombat.san-francisco.ca.us>

Followups directed to: comp.os.linux.misc

Show all headers | View raw

Keith Keller wrote:

> Hi all,
> 
> I am attempting to build a snapshot server for a ~15TB fileserver with
> old fileserver hardware I have on hand.  My initial plan was to use the
> hardware card in the old fileserver in a RAID50 (the card is old enough
> that it doesn't support RAID6 natively) using new 2TB enterprise hard
> drives.  But, as you probably know, these drives are reasonably
> expensive.  So, since this machine will not be used by end-users very
> much, I was contemplating using linux software raid instead, exporting
> desktop-class drives as JBODs and using mdadm to RAID them.
> 
> The obvious advantage to this is cost: I can save almost 40% of my
> original estimate by using desktop drives instead, thus fulfilling the
> original meaning of the I of the RAID acronym.  There are other
> advantages, as well, including being able to build a RAID6, which I
> slightly prefer over a RAID50, and having more flexibility later on if I
> want to move to bigger disks.  (Yes, I have seen the documentation
> warning against too-large RAID arrays resulting in a failure during a
> rebuild.)  A tertiary advantage would be that I would learn how to work
> with linux software RAID, a skill I haven't yet acquired.
> 
> The disadvantages I can think of are: higher probability of disk
> failures, resulting in more work for me in swapping out and RMAing
> failed drives; potential degradation in performance, due both to RAID in
> software and slower disks; a learning curve for linux RAID; and a
> configuration less likely to be supported by the hardware RAID vendor.

Hi,

Highly dependant on your server and RAID card of course, but you may find MD 
software raid is quicker.

Even and older server has far more CPU horsepower available compared to a 
mediocre RAID card (and by mediocre, I mean anything costing less than 100's 
pounds.

> My counters to most of the disadvantages would be that performance only
> has to be decent, not great, on this box; the learning curve shouldn't
> be too bad; and this configuration shouldn't require support from the
> hardware RAID vendor anyway.  The disk failures would be the only issue
> I couldn't counter, except by trying to determine if my labor costs
> would end up being more than the savings in moving to cheaper disks.

The learning curve is fairly easy with mdadm - furthermore, linux MD is now 
more functionally complete than all but the better end *modern* hardware 
RAID systems. Specifically, some things linux will do that a lot of 
older/cheaper HW RAID won't:

1) Attempt to rewrite a disck block that has failed to read <- triggers a 
bad block remap on most drives.

2) If you run the monitor daemon, linux will alert you if stuff goes bad, eg 
failed disk (OK, a crappy HW raid knos this, but can it alert you by email 
or just sit there with a falshing red LED?)

3) Perform a full sweep and parity verify on demand?

There are more, but those are what I consider most useful.

> My questions:
> 
> 1) Has anyone done this before, and if so, what were the results?  Was
> performance acceptable in this configuration?  Are there any gotchas to
> an otherwise workable configuration?

Yep - been running SW raid 5 at home on 1.5TB total for 3 years. I have used 
a lot of mid range RAID controllers too (Chaparrel, Infotrend, ARECA, 
Eurologic)

> 2) From what I've read so far, using desktop-class disks with linux
> software RAID should not be a major problem, unlike using them on a true
> hardware RAID card.  Is this reasonably accurate?  If not, are there
> links that describe the difficulties?

Yep - desktop are fine. Enterprise class or "RAID Edition" may be better 
quality and/or quicker. Quicker is usually related to RPM and at least is 
checkable in the specifications. "Well built" is more abstract. I prefer to 
use a mixture of makes in the same server, eg Hitachi, Seagate, Fujitsu, WD) 
- that way, you lessen the risk of the "Maxtor Deathstar" whole buch failing 
at once syndrome.

> 3) Suppose that my RAID6 starts out using 12 2TB disks, with three free
> drive bays (one would be a hot spare).  Later on, I want to seamlessly
> replace the 2TB disks with 3TB or larger disks.  Can mdadm grow an array
> like this if, say, I replace one drive, rebuild, and repeat until I've
> replaced all 12 disks with larger ones?  Or will the new 3TB disks only
> be used up to 2TB, the size of the original disks?

RAID5/6 need to be spread over identically sized partitions. So you can't 
add a 3TB drive to a 2TB disk based array. You can partition and make a new 
RAID across the 1TB partition. This is where ZFS gets clever, but that's not 
really an option for linux (BTRFS will probably get there one day).

> Thanks for any advice or pointers you can provide!

One thing, whichever system you go for: set it up and do some speed and 
breakage tests to make sure it all works correctly - pull a disk out live, 
be sure you know how to put the disk back and bring the array back to fault 
tolerant and stuff like that.

It's good fun, enjoy :)

Cheers

Tim

> --keith
> 

-- 
Tim Watts

Back to comp.os.linux.misc | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

linux raid vs hw raid Keith Keller <kkeller-usenet@wombat.san-francisco.ca.us> - 2011-04-05 19:39 -0700
  Re: linux raid vs hw raid Tim Watts <tw@dionic.net> - 2011-04-06 08:01 +0100
    Re: linux raid vs hw raid David Brown <david@westcontrol.removethisbit.com> - 2011-04-06 10:03 +0200
      Re: linux raid vs hw raid Keith Keller <kkeller-usenet@wombat.san-francisco.ca.us> - 2011-04-06 14:00 -0700
        Re: linux raid vs hw raid David Brown <david.brown@removethis.hesbynett.no> - 2011-04-06 23:42 +0200
        Re: linux raid vs hw raid Grant <omg@grrr.id.au> - 2011-04-08 10:45 +1000
          Re: linux raid vs hw raid David Brown <david@westcontrol.removethisbit.com> - 2011-04-08 11:12 +0200
            Re: linux raid vs hw raid Keith Keller <kkeller-usenet@wombat.san-francisco.ca.us> - 2011-04-08 08:22 -0700
              Re: linux raid vs hw raid Grant <omg@grrr.id.au> - 2011-04-09 09:51 +1000
                Re: linux raid vs hw raid Keith Keller <kkeller-usenet@wombat.san-francisco.ca.us> - 2011-04-08 17:10 -0700
                Re: linux raid vs hw raid David Brown <david.brown@removethis.hesbynett.no> - 2011-04-09 13:14 +0200
            Re: linux raid vs hw raid Grant <omg@grrr.id.au> - 2011-04-09 09:47 +1000
              Re: linux raid vs hw raid David Brown <david.brown@removethis.hesbynett.no> - 2011-04-09 13:55 +0200
          Re: linux raid vs hw raid Tris Orendorff <triso@remove-me.cogeco.ca> - 2011-04-12 18:04 +0000
            Re: linux raid vs hw raid Keith Keller <kkeller-usenet@wombat.san-francisco.ca.us> - 2011-04-12 11:34 -0700
              Re: linux raid vs hw raid The Natural Philosopher <tnp@invalid.invalid> - 2011-04-12 21:13 +0100
                Re: linux raid vs hw raid David Brown <david@westcontrol.removethisbit.com> - 2011-04-13 09:45 +0200
                Re: linux raid vs hw raid Grant <omg@grrr.id.au> - 2011-04-14 13:42 +1000
                Re: linux raid vs hw raid David Brown <david@westcontrol.removethisbit.com> - 2011-04-14 09:15 +0200
                Re: linux raid vs hw raid Grant <omg@grrr.id.au> - 2011-04-15 08:03 +1000
                Re: linux raid vs hw raid Tim Watts <tw@dionic.net> - 2011-04-15 07:22 +0100
                Re: linux raid vs hw raid David Brown <david@westcontrol.removethisbit.com> - 2011-04-15 09:28 +0200
                Re: linux raid vs hw raid Grant <omg@grrr.id.au> - 2011-04-19 11:20 +1000
              Re: linux raid vs hw raid Grant <omg@grrr.id.au> - 2011-04-14 13:38 +1000
                Re: linux raid vs hw raid Keith Keller <kkeller-usenet@wombat.san-francisco.ca.us> - 2011-04-13 21:49 -0700
            Re: linux raid vs hw raid Grant <omg@grrr.id.au> - 2011-04-14 13:34 +1000
              Re: linux raid vs hw raid Tris Orendorff <triso@remove-me.cogeco.ca> - 2011-04-15 21:59 +0000
                Re: linux raid vs hw raid "Peter J. Holzer" <hjp-usenet2@hjp.at> - 2011-04-16 00:56 +0200
                Re: linux raid vs hw raid The Natural Philosopher <tnp@invalid.invalid> - 2011-04-16 01:32 +0100
  Re: linux raid vs hw raid Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2011-04-08 21:38 +0300
    Re: linux raid vs hw raid Grant <omg@grrr.id.au> - 2011-04-09 09:53 +1000
  Re: linux raid vs hw raid KR <kristian.rasmussen@broadpark.no.spam.com> - 2011-04-09 11:56 +0200
    Re: linux raid vs hw raid Keith Keller <kkeller-usenet@wombat.san-francisco.ca.us> - 2011-04-09 10:32 -0700
      Re: linux raid vs hw raid Grant <omg@grrr.id.au> - 2011-04-10 11:12 +1000
        Re: linux raid vs hw raid Keith Keller <kkeller-usenet@wombat.san-francisco.ca.us> - 2011-04-09 18:59 -0700
          Re: linux raid vs hw raid KR <kristian.rasmussen@broadpark.no.spam.com> - 2011-04-10 04:32 +0200
            Re: linux raid vs hw raid Grant <omg@grrr.id.au> - 2011-04-10 12:46 +1000
              Re: linux raid vs hw raid Keith Keller <kkeller-usenet@wombat.san-francisco.ca.us> - 2011-04-09 20:39 -0700
    Re: linux raid vs hw raid Robert Riches <spamtrap42@jacob21819.net> - 2011-04-10 03:47 +0000
      Re: linux raid vs hw raid Balwinder S Dheeman <bsd.SANSPAM@anu.homelinux.net> - 2011-04-10 11:11 +0530
        Re: linux raid vs hw raid Keith Keller <kkeller-usenet@wombat.san-francisco.ca.us> - 2011-04-09 23:29 -0700
          Re: linux raid vs hw raid Balwinder S Dheeman <bsd.SANSPAM@anu.homelinux.net> - 2011-04-10 14:05 +0530
        Re: linux raid vs hw raid Grant <omg@grrr.id.au> - 2011-04-10 20:16 +1000
          Re: linux raid vs hw raid Tim Watts <tw@dionic.net> - 2011-04-10 11:28 +0100
          Re: linux raid vs hw raid Balwinder S Dheeman <bsd.SANSPAM@anu.homelinux.net> - 2011-04-10 19:43 +0530
        Re: linux raid vs hw raid Robert Riches <spamtrap42@jacob21819.net> - 2011-04-12 03:44 +0000
          Re: linux raid vs hw raid Balwinder S Dheeman <bsd.SANSPAM@anu.homelinux.net> - 2011-04-12 13:56 +0530
      Re: linux raid vs hw raid Grant <omg@grrr.id.au> - 2011-04-10 20:09 +1000
        Re: linux raid vs hw raid Robert Riches <spamtrap42@jacob21819.net> - 2011-04-12 03:37 +0000

csiph-web