Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #190325 > unrolled thread

Experiences with BTRFS -- is it mature enough for enterprise use?

Back to article view | Back to linux.debian.user

Contents

  Experiences with BTRFS -- is it mature enough for enterprise use? Rick Thomas <rbthomas@pobox.com> - 2017-12-26 20:50 +0100
    Re: Experiences with BTRFS -- is it mature enough for enterprise use? Eero Volotinen <eero.volotinen@iki.fi> - 2017-12-26 20:50 +0100
      Re: Experiences with BTRFS -- is it mature enough for enterprise use? Roberto C. Sánchez <roberto@debian.org> - 2017-12-26 23:30 +0100
        Re: Experiences with BTRFS -- is it mature enough for enterprise use? Sven Hartge <sven@svenhartge.de> - 2017-12-27 00:00 +0100
        Re: Experiences with BTRFS -- is it mature enough for enterprise use? Michael Stone <mstone@debian.org> - 2017-12-27 00:40 +0100
    Re: Experiences with BTRFS -- is it mature enough for enterprise use? Jan Vales <jan@jvales.net> - 2017-12-26 21:30 +0100
    Re: Experiences with BTRFS -- is it mature enough for enterprise use? Andy Smith <andy@strugglers.net> - 2017-12-26 22:40 +0100
    Re: Experiences with BTRFS -- is it mature enough for enterprise use? David Christensen <dpchrist@holgerdanske.com> - 2017-12-27 04:10 +0100
      Re: Experiences with BTRFS -- is it mature enough for enterprise use? Matthew Crews <mailinglists@mattcrews.com> - 2017-12-27 13:00 +0100
        Re: Experiences with BTRFS -- is it mature enough for enterprise use? Tom Dial <tddial@comcast.net> - 2017-12-27 23:10 +0100
          Re: Experiences with BTRFS -- is it mature enough for enterprise use? Rick Thomas <rick.thomas@pobox.com> - 2017-12-28 02:10 +0100
            Re: Experiences with BTRFS -- is it mature enough for enterprise use? deloptes <deloptes@gmail.com> - 2017-12-28 02:20 +0100
              Re: Experiences with BTRFS -- is it mature enough for enterprise use? Eero Volotinen <eero.volotinen@iki.fi> - 2017-12-28 07:10 +0100
                Re: Experiences with BTRFS -- is it mature enough for enterprise use? Dan Ritter <dsr@randomstring.org> - 2017-12-28 17:00 +0100
                  Re: Experiences with BTRFS -- is it mature enough for enterprise use? Eero Volotinen <eero.volotinen@iki.fi> - 2017-12-28 18:30 +0100
                    Re: Experiences with BTRFS -- is it mature enough for enterprise use? Glenn English <ghe2001@gmail.com> - 2017-12-28 20:40 +0100
                      Re: Experiences with BTRFS -- is it mature enough for enterprise use? Matthew Crews <mailinglists@mattcrews.com> - 2017-12-28 20:50 +0100
                      Re: Experiences with BTRFS -- is it mature enough for enterprise use? Andy Smith <andy@strugglers.net> - 2017-12-29 01:00 +0100
                        Re: Experiences with BTRFS -- is it mature enough for enterprise use? Jan Vales <jan@jvales.net> - 2017-12-30 00:50 +0100
                          Re: Experiences with BTRFS -- is it mature enough for enterprise use? Matthew Crews <mailinglists@mattcrews.com> - 2017-12-30 01:30 +0100
                            Re: Experiences with BTRFS -- is it mature enough for enterprise use? Jan Vales <jan@jvales.net> - 2017-12-30 01:40 +0100
                              Re: Experiences with BTRFS -- is it mature enough for enterprise use? Matthew Crews <mailinglists@mattcrews.com> - 2017-12-30 23:40 +0100
                                Re: Experiences with BTRFS -- is it mature enough for enterprise use? David Christensen <dpchrist@holgerdanske.com> - 2017-12-31 07:20 +0100
                                  Re: Experiences with BTRFS -- is it mature enough for enterprise use? Sven Hartge <sven@svenhartge.de> - 2017-12-31 18:50 +0100
                                    Re: Experiences with BTRFS -- is it mature enough for enterprise use? David Christensen <dpchrist@holgerdanske.com> - 2017-12-31 23:10 +0100
                                      Re: Experiences with BTRFS -- is it mature enough for enterprise use? Sven Hartge <sven@svenhartge.de> - 2017-12-31 23:50 +0100
                                        Re: Experiences with BTRFS -- is it mature enough for enterprise use? David Christensen <dpchrist@holgerdanske.com> - 2018-01-02 03:10 +0100
                                          Re: Experiences with BTRFS -- is it mature enough for enterprise use? <tomas@tuxteam.de> - 2018-01-02 11:30 +0100
                                            Re: Experiences with BTRFS -- is it mature enough for enterprise use? David Christensen <dpchrist@holgerdanske.com> - 2018-01-02 17:40 +0100
                                              Re: Experiences with BTRFS -- is it mature enough for enterprise use? deloptes <deloptes@gmail.com> - 2018-01-02 19:10 +0100
                                                Re: Experiences with BTRFS -- is it mature enough for enterprise use? Pascal Hambourg <pascal@plouf.fr.eu.org> - 2018-01-02 20:10 +0100
                                                  Re: Experiences with BTRFS -- is it mature enough for enterprise use? deloptes <deloptes@gmail.com> - 2018-01-03 01:00 +0100
                                                    Re: Experiences with BTRFS -- is it mature enough for enterprise use? Pascal Hambourg <pascal@plouf.fr.eu.org> - 2018-01-04 00:00 +0100
                                                      Re: Experiences with BTRFS -- is it mature enough for enterprise use? deloptes <deloptes@gmail.com> - 2018-01-04 09:00 +0100
                                                        Re: Experiences with BTRFS -- is it mature enough for enterprise use? Pascal Hambourg <pascal@plouf.fr.eu.org> - 2018-01-04 20:20 +0100
                                                          Re: Experiences with BTRFS -- is it mature enough for enterprise use? deloptes <deloptes@gmail.com> - 2018-01-04 22:00 +0100
                                                      Re: Experiences with BTRFS -- is it mature enough for enterprise use? deloptes <deloptes@gmail.com> - 2018-01-04 12:00 +0100
                                                Re: Experiences with BTRFS -- is it mature enough for enterprise use? David Christensen <dpchrist@holgerdanske.com> - 2018-01-03 00:10 +0100
                                                  Re: Experiences with BTRFS -- is it mature enough for enterprise use? Darac Marjal <mailinglist@darac.org.uk> - 2018-01-03 00:50 +0100
                                                    Re: Experiences with BTRFS -- is it mature enough for enterprise use? Gene Heskett <gheskett@shentel.net> - 2018-01-03 02:10 +0100
                                                      Re: Experiences with BTRFS -- is it mature enough for enterprise use? Richard Hector <richard@walnut.gen.nz> - 2018-01-06 13:30 +0100
                                                    Re: Experiences with BTRFS -- is it mature enough for enterprise use? David Christensen <dpchrist@holgerdanske.com> - 2018-01-03 08:40 +0100
                                    Re: Experiences with BTRFS -- is it mature enough for enterprise use? Joel Wirāmu Pauling <joel@aenertia.net> - 2017-12-31 23:20 +0100
                                      Re: Experiences with BTRFS -- is it mature enough for enterprise use? Andy Smith <andy@strugglers.net> - 2018-01-03 04:40 +0100
                          Re: Experiences with BTRFS -- is it mature enough for enterprise use? Pascal Hambourg <pascal@plouf.fr.eu.org> - 2017-12-30 10:30 +0100
                    Re: Experiences with BTRFS -- is it mature enough for enterprise use? Dan Ritter <dsr@randomstring.org> - 2017-12-28 21:10 +0100
                Re: Experiences with BTRFS -- is it mature enough for enterprise use? deloptes <deloptes@gmail.com> - 2017-12-28 18:00 +0100
    Re: Experiences with BTRFS -- is it mature enough for enterprise use? Igor Cicimov <icicimov@gmail.com> - 2017-12-29 05:00 +0100

Page 1 of 3  [1] 2 3  Next page →

#190325 — Experiences with BTRFS -- is it mature enough for enterprise use?

Is btrfs mature enough to use in enterprise applications?

If you are using it, I’d like to hear from you about your experiences — good or bad.

My proposed application is for a small community radio station music library.
We currently have about 5TB of data in a RAID10 using four 3TB drives, with ext4 over the RAID.  So we’re about 75% full, growing at the rate of about 1TB/year, so we’ll run out of space by the end of 2018.

I’m proposing to go to three 6TB drives in a btrfs/RAID5 configuration.  This would give us 12TB of usable data space and hold off til the end of 2024 before needing the next upgrade.

Will it work?  Would I be safer with ext4 over RAID5?

Thanks in advance!
Rick

[toc] | [next] | [standalone]

#190326

[Multipart message — attachments visible in raw view] — view raw

use XFS, it's mature and suitable for big storage. (or gluster or cehp?)


Eero

26.12.2017 21.45 "Rick Thomas" <rbthomas@pobox.com> kirjoitti:

>
> Is btrfs mature enough to use in enterprise applications?
>
> If you are using it, I’d like to hear from you about your experiences —
> good or bad.
>
> My proposed application is for a small community radio station music
> library.
> We currently have about 5TB of data in a RAID10 using four 3TB drives,
> with ext4 over the RAID.  So we’re about 75% full, growing at the rate of
> about 1TB/year, so we’ll run out of space by the end of 2018.
>
> I’m proposing to go to three 6TB drives in a btrfs/RAID5 configuration.
> This would give us 12TB of usable data space and hold off til the end of
> 2024 before needing the next upgrade.
>
> Will it work?  Would I be safer with ext4 over RAID5?
>
> Thanks in advance!
> Rick
>

[toc] | [prev] | [next] | [standalone]

#190329

On Tue, Dec 26, 2017 at 09:48:09PM +0200, Eero Volotinen wrote:
>    use XFS, it's mature and suitable for big storage. (or gluster or cehp?)

I second XFS for your application.  Another to consider might be JFS.
Either one of those would be very mature and suitable for the
enterprise.  I don't have any real experience with JFS, but I used XFS
for many years starting in 2002 or 2003.  These days the sorts of
applications I work with don't really call for delving deeply enough to
warrant worrying about the filesystem, but it sounds like something that
merits evaluation in your case.

In one of the early applications that I dealt with we had very large
files (on the order of 10s to 100s of GB, which was rather large in
those days.  On ext3, a file removal could take many minutes.  The
performance for ext2 was a bit better, but of course you take on added
risk with a non-journaled filesystem.  XFS, on the otherhand, could
delete a multi-100 GB file just quickly as it could delete a 1 KB file.

Regards,

-Roberto

-- 
Roberto C. Sánchez

[toc] | [prev] | [next] | [standalone]

#190330

Roberto C. Sánchez <roberto@debian.org> wrote:
> On Tue, Dec 26, 2017 at 09:48:09PM +0200, Eero Volotinen wrote:

>>    use XFS, it's mature and suitable for big storage. (or gluster or
>>    cehp?)

> I second XFS for your application.  Another to consider might be JFS.

I believe development of JFS has stopped several years ago and I don't
think it is suitable for new deployments into production.

Grüße,
Sven.

-- 
Sigmentation fault. Core dumped.

[toc] | [prev] | [next] | [standalone]

#190331

On Tue, Dec 26, 2017 at 05:21:10PM -0500, Roberto C. Sánchez wrote:
>I second XFS for your application.  Another to consider might be JFS.
>Either one of those would be very mature and suitable for the
>enterprise.  I don't have any real experience with JFS

I can't think of any reason to go with JFS. Among other things, XFS is 
actively maintained...

Mike Stone

[toc] | [prev] | [next] | [standalone]

#190327

[Multipart message — attachments visible in raw view] — view raw

tl;dr:
save yourself the hassle and dont. go for md-raid5/6 + (luks +) XFS.

long version:

Just last week we migrated our soon to be production server (6 disks)
btrfs-raid10 to md-raid6+XFS, after btrfs managed to die twice in december.

So cool btrfs-raid/filesystem-level-raid sounds, so broken it seems atm :(
Not only did they not manage to use "common" definitions of raid - as
in: md-raid being de-facto-standard, they get to define certain aspects
of raid and others have to live by that ...

READ THEIR DOCS + MAILING LIST!

For example: when people talk about raid1 with say 4 disks, they expect
that all disks have the same contents... Therefore expecting that losing
3 disks means no data loss.
Well... btrfs decided to stick with some ancient definition of raid1
which is imho more like md-raid10, than md-raid1, as there are only 2
copies of data and you cant make btrfs to store more than 2 copies.
Therefore losing 2/4 disks -> btrfs irrecoverably broken.

btrfs-raid1 and btrfs-raid10 seem to be basically the same thing, with
some imho performance optimizations (not striping vs striping), which
should imho not even be user-settable as it makes no sense to go for
suboptimal performance, but then again, maybe i missed some of their
nearly inexistent documentation pointing out the good part of
btrfs-raid1 ...

Also btrfs-raid5/6 are broken and didnt even survive our in-vm-testing,
before getting a chance on bare-metal.


Also it seems way easier to go "full-raid-encryption" with luks + md-raid.
(Our intention is to be able to send in disks to get them replaced
without having to worry, that data could be easily extracted; there is
an usb-stick with the luks key sticking out of every machine...)

If you need some btrfs-features (snapshots <3), consider going
md-raid+luks+btrfs.
* Unsure if it still holds that btrfs will fail horribly if it sees its
uuid on more than one disk, possibly making the
full-disk-encryption-layer mandatory.

br,
Jan


On 12/26/17 20:37, Rick Thomas wrote:
> 
> Is btrfs mature enough to use in enterprise applications?
> 
> If you are using it, I’d like to hear from you about your experiences — good or bad.
> 
> My proposed application is for a small community radio station music library.
> We currently have about 5TB of data in a RAID10 using four 3TB drives, with ext4 over the RAID.  So we’re about 75% full, growing at the rate of about 1TB/year, so we’ll run out of space by the end of 2018.
> 
> I’m proposing to go to three 6TB drives in a btrfs/RAID5 configuration.  This would give us 12TB of usable data space and hold off til the end of 2024 before needing the next upgrade.
> 
> Will it work?  Would I be safer with ext4 over RAID5?
> 
> Thanks in advance!
> Rick
> 


-- 
lg
Jan Vales
--
I only read plaintext emails.

Someone @ irc://irc.fsinf.at:6667/tuwien
webIRC: https://frost.fsinf.at/iris/

[toc] | [prev] | [next] | [standalone]

#190328

Hi Rick,

On Tue, Dec 26, 2017 at 11:37:32AM -0800, Rick Thomas wrote:
> Is btrfs mature enough to use in enterprise applications?

Not in my opinion. I've dabbled with it at home and based on those
experiences I will not be using it professionally any time soon.

> If you are using it, I’d like to hear from you about your experiences — good or bad.

During the time of Debian wheezy I made use of btrfs for my home
fileserver, which is an HP Microserver with 4x 3.5" SATA drives and
an 8 bay disk chassis with 6 more 3.5" SATA HDDs in it, connected by
eSATA. It had previously been using LVM on top of Linux MD without
issue, but I'd become mindful of the amount of storage that was
being used without consistency checks (except for a weekly MD
scrub).

In order to do this I required a backports kernel and btrfs-tools
from git. I went for one of the more simple btrfs configurations
which is a raid1.

Over the next few years I didn't lose any data, but I did
experience:

- Out of space errors even when there was plenty of space

- Filesystems that went read-only on a device failure even though
  there was enough redundancy

- Filesystems that couldn't be remounted read-write after failed
  device replacement, even though the hardware is hot-swap, due to
  bugs in btrfs which required a kernel upgrade to fix (therefore a
  reboot, despite having the redundancy otherwise).

In summary, btrfs lowered the availability of the system due to
being buggy even in a relatively unexciting configuration. I could
not recommend it for serious use yet.

I am sure there will be plenty of people who've used it for years
without experiencing any issue. I am still subscribed to the btrfs
mailing list though, and unfortunately I still see people on there
reporting serious issues including data loss.

> My proposed application is for a small community radio station
> music library. We currently have about 5TB of data in a RAID10
> using four 3TB drives, with ext4 over the RAID.  So we’re about
> 75% full, growing at the rate of about 1TB/year, so we’ll run out
> of space by the end of 2018.

A year to solve this problem is nice to have, though. :)

> I’m proposing to go to three 6TB drives in a btrfs/RAID5 configuration.

If I were you I'd think very very carefully before using RAID-5 for
anything, btrfs or not.

- Four spindles to three means reduced performance.

- RAID-5 means parity means reduced performance.

- Lose one device and you're operating on just two spindles and
  recalculating parity. It will run like a dog and any further error
  means data loss, and array failure, which can be stressful and
  nerve-wracking to repair.

- I wouldn't like to chance my arm finding previously-unknown bad
  areas on two 6TB devices. When you have a three device RAID-5 and
  one device dies, you REQUIRE every sector on both the other
  devices to be readable in order to reconstruct the data onto the
  new device.

Personally I would not risk 3x6TB in any kind of RAID-5. HDDs are
pretty cheap so I really have to question the wisdom of cutting down
the number of spindles this far, and then using RAID-5.

RAID-10: okay, it "wastes" the most capacity in exchange for better
write performance. If this is a media library I get that maybe you
don't need the write performance (have you benchmarked your current
system??). RAID-10 is what you use when you can afford it, but if
you can't then compromises have to be made. In that case consider 4
or more spindles RAID-6. At least you stand a chance of being able
to replace a failed device before coming across a bad area on
another device.

Spread the extra cost of whatever you need to make it to four
devices in RAID-6 across the expected lifetime of your system (~6
years?) and does it still seem too much to pay?

Finally, the parity RAID levels in btrfs are newer than RAID-1 and
-10 and have seen a lot more bugs. Including really bad data loss
bugs.

One look at https://btrfs.wiki.kernel.org/index.php/RAID56 should be
enough.

> Would I be safer with ext4 over RAID5?

It's a bit of a frying pan / ground zero nuclear blast situation,
really.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

"I remember the first time I made love.  Perhaps it was not love exactly but I
 made it and it still works." — The League Against Tedium

[toc] | [prev] | [next] | [standalone]

#190332

On 12/26/17 11:37, Rick Thomas wrote:
> Is btrfs mature enough to use in enterprise applications?
> 
> If you are using it, I’d like to hear from you about your experiences — good or bad.
> 
> My proposed application is for a small community radio station music library.
> We currently have about 5TB of data in a RAID10 using four 3TB drives, with ext4 over the RAID.  So we’re about 75% full, growing at the rate of about 1TB/year, so we’ll run out of space by the end of 2018.
> 
> I’m proposing to go to three 6TB drives in a btrfs/RAID5 configuration.  This would give us 12TB of usable data space and hold off til the end of 2024 before needing the next upgrade.
> 
> Will it work?  Would I be safer with ext4 over RAID5?

Take a look at ZFS before you decide -- either ZFS on Linux or FreeBSD:

https://packages.debian.org/stretch/zfs-dkms

https://www.freebsd.org/


I would suggest building a ZFS pool with two 8 TB mirrored drives.  When 
you want more space, add another mirrored pair.


Live compression and live de-duplication are useful features of ZFS, but 
the killer features are live snapshots and live replication.


David

[toc] | [prev] | [next] | [standalone]

#190337

I wouldn't trust BTRFS in an enterprise environment, but I have good experience in a personal environment. Make sure you are using modern kernels though (I wouldn't use anything earlier than 4.4, and realistically I would use 4.9 or 4.13 or higher), and I definitely would not use RAID5/6.

For an enterprise environment, ZFS wins, hands down.

[toc] | [prev] | [next] | [standalone]

#190365

On 12/27/2017 04:57 AM, Matthew Crews wrote:
> I wouldn't trust BTRFS in an enterprise environment, but I have good experience in a personal environment. Make sure you are using modern kernels though (I wouldn't use anything earlier than 4.4, and realistically I would use 4.9 or 4.13 or higher), and I definitely would not use RAID5/6.
> 
> For an enterprise environment, ZFS wins, hands down.

Based on prior experience with ZFS under Solaris and FreeBSD, I've been
considering the possibility of using it with Debian now that it is
pretty much a first class file system.

Reports of actual use, pointers, and gotchas, if any, would be useful to
me and probably others.

Thanks,
Tom Dial
> 

[toc] | [prev] | [next] | [standalone]

#190370

On Wed, Dec 27, 2017, at 1:46 PM, Tom Dial wrote:
> 
> 
> On 12/27/2017 04:57 AM, Matthew Crews wrote:
> > I wouldn't trust BTRFS in an enterprise environment, but I have good experience in a personal environment. Make sure you are using modern kernels though (I wouldn't use anything earlier than 4.4, and realistically I would use 4.9 or 4.13 or higher), and I definitely would not use RAID5/6.
> > 
> > For an enterprise environment, ZFS wins, hands down.
> 
> Based on prior experience with ZFS under Solaris and FreeBSD, I've been
> considering the possibility of using it with Debian now that it is
> pretty much a first class file system.
> 
> Reports of actual use, pointers, and gotchas, if any, would be useful to
> me and probably others.
> 
> Thanks,
> Tom Dial

Since it doesn't look like I'll be using BTRFS for my application, I too would appreciate hearing about experiences with ZFS as an alternative.  Unfortunately, the application we're using is only available for CentOS-6, so we'll have to pressure the developer to release his CentOS-7 code, but we've got a year to do it, so it's probably do-able.

Thanks in advance!
Rick

[toc] | [prev] | [next] | [standalone]

#190371

Rick Thomas wrote:

> Since it doesn't look like I'll be using BTRFS for my application, I too
> would appreciate hearing about experiences with ZFS as an alternative. 
> Unfortunately, the application we're using is only available for CentOS-6,
> so we'll have to pressure the developer to release his CentOS-7 code, but
> we've got a year to do it, so it's probably do-able.

+1 for ZFS

[toc] | [prev] | [next] | [standalone]

#190374

[Multipart message — attachments visible in raw view] — view raw

Are you really using it in production?



Eero

28.12.2017 3.12 "deloptes" <deloptes@gmail.com> kirjoitti:

> Rick Thomas wrote:
>
> > Since it doesn't look like I'll be using BTRFS for my application, I too
> > would appreciate hearing about experiences with ZFS as an alternative.
> > Unfortunately, the application we're using is only available for
> CentOS-6,
> > so we'll have to pressure the developer to release his CentOS-7 code, but
> > we've got a year to do it, so it's probably do-able.
>
> +1 for ZFS
>
>

[toc] | [prev] | [next] | [standalone]

#190405

On Thu, Dec 28, 2017 at 08:01:44AM +0200, Eero Volotinen wrote:
> Are you really using it in production?
> 

I'm using ZFS at home (3 pools, including my main server's
/home and two backup pools) and at work (my desktop machine's
root and /home, some of the backup servers).

It is much more stable than btrfs, which I used to use at home.

I stick to RAID1 and RAID10 layouts everywhere.

-dsr-

[toc] | [prev] | [next] | [standalone]

#190415

[Multipart message — attachments visible in raw view] — view raw

That really doesn't sound like critical production use.

I really cannot recommend zfs on linux for production use. It works better
on FreeBSD and it's not included in standard dist due to licence issues.

--
Eero

2017-12-28 17:53 GMT+02:00 Dan Ritter <dsr@randomstring.org>:

> On Thu, Dec 28, 2017 at 08:01:44AM +0200, Eero Volotinen wrote:
> > Are you really using it in production?
> >
>
> I'm using ZFS at home (3 pools, including my main server's
> /home and two backup pools) and at work (my desktop machine's
> root and /home, some of the backup servers).
>
> It is much more stable than btrfs, which I used to use at home.
>
> I stick to RAID1 and RAID10 layouts everywhere.
>
> -dsr-
>

[toc] | [prev] | [next] | [standalone]

#190423

On Thu, Dec 28, 2017 at 5:29 PM, Eero Volotinen <eero.volotinen@iki.fi> wrote:
> That really doesn't sound like critical production use.
>
> I really cannot recommend zfs on linux for production use. It works better
> on FreeBSD and it's not included in standard dist due to licence issues.

Is there something wrong with ext4 in a RAID1?

--
Glenn English

[toc] | [prev] | [next] | [standalone]

#190424

>-------- Original Message --------
>From: ghe2001@gmail.com
>On Thu, Dec 28, 2017 at 5:29 PM, Eero Volotinen eero.volotinen@iki.fi wrote:
>>That really doesn't sound like critical production use.
>>I really cannot recommend zfs on linux for production use. It works better
>> on FreeBSD and it's not included in standard dist due to licence issues.
>>
> Is there something wrong with ext4 in a RAID1?

Nothing wrong per se, just that ZFS or BTRFS are better in many cases.

[toc] | [prev] | [next] | [standalone]

#190432

Hello,

On Thu, Dec 28, 2017 at 07:35:27PM +0000, Glenn English wrote:
> Is there something wrong with ext4 in a RAID1?

Not if you don't need any of the features of ZFS that ext4 lacks,
no. But if you do, then ext4 is not an option.

The killer feature of ZFS is its checksumming of all data and
metadata to protect against bitrot and other forms of data
corruption. The only other filesystem offering this on Linux is
btrfs, hence the many mentions of ZFS in this thread. Putting the
filesystem under MD RAID (or hardware RAID) and scrubbing it will
detect corruption but cannot fix it.

But like everything else, ZFS has its downsides too, so it is a
matter of requirements.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [prev] | [next] | [standalone]

#190459

[Multipart message — attachments visible in raw view] — view raw

On 12/29/17 00:55, Andy Smith wrote:
> The killer feature of ZFS is its checksumming of all data and
> metadata to protect against bitrot and other forms of data
> corruption. The only other filesystem offering this on Linux is
> btrfs, hence the many mentions of ZFS in this thread. Putting the
> filesystem under MD RAID (or hardware RAID) and scrubbing it will
> detect corruption but cannot fix it.
> 

md-raid6 can fix most few-byte issues online.

# dd if=/dev/zero of=/dev/sdb1 bs=1 count=1 seek=1234
md-raid6 scrub will fix that byte.
Remember to always flush disc caches when testing!

But unlike btrfs-raid10, md-raid6 cannot recover a whole disk or
partition getting fully zero-dd'd - with or without a reboot.

I didnt actually test how much of a disk must be zeroed or random'ed
before md-raid6 scrub starts to fail. A few calls to the above dd with
diffrent seek= and then scrubbing will fix the corrupted bytes every time.
A full zero-dd will not.
-> drive will be "failed" and you need to re-add it.

Which was the reason why we initially gave btrfs-raid10 a try...
... it would be a really cool FS, if it was as stable as it is on my
laptop (and I really dislike btrfs-raid definitions)

You still can go md-raid + btrfs, if you want some btrfs features.
Snapshots (and send/receive) are what I really love on my laptop and
could not live without anymore.
(fulldisk encryption may be mandatory, as btrfs at least some time ago,
had the tendency to brick itself, if it sees its uuid on multiple disks
at the same time (md-raid1))

br,
Jan Vales
--
I only read plaintext emails.

[toc] | [prev] | [next] | [standalone]

#190461

>-------- Original Message --------
>Subject: Re: Experiences with BTRFS -- is it mature enough for enterprise use?
>Local Time: December 29, 2017 4:48 PM
>UTC Time: December 29, 2017 11:48 PM
>From: jan@jvales.net

> You still can go md-raid + btrfs, if you want some btrfs features.

If you're using Raid1 or Raid10, md-raid + btrfs is probably worse than native btrfs Raid1 + Raid10.

[toc] | [prev] | [next] | [standalone]

Page 1 of 3  [1] 2 3  Next page →

Back to top | Article view | linux.debian.user

csiph-web