Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.hardware > #3250

Re: Speed ups for a disk IO bound machine

Show all headers | View raw

On 2016-12-11 12:54, Peter Chant wrote:
[...]
> Thanks.  I've done some reading and there is more to do plus some
> experimentation.

Experimentation is good.
It is the only way to have so idea of the
different peculiarities.

> I understand that mdadm is used to create the raid arrays, it is not
> part of lvm itself?

LVM (dmraid) and md share a lot of code, but I'm
not sure about this RAID-10. Maybe it is only md.

[...]
> OK.  Simple to set up with the SSDs as one is blank and the other has
> free space.

Be careful, it is easy to destroy data.
Maybe you can practice with loop devices.
There are some howtos around.

> With RAID-10 far 2 am I correct in assuming that the available capacity
> for a two device array of 3TB disks would be 3TB (two copies of data)?

Yep.

>> Part of the RAID-10 with SSD can be used as cache.
>>
> 
> OK, so having done some reading up but not carried out any testing I
> think the following is a possible setup.  Note I am using LUKS so I will
> add this extra layer to the mix.  I have not noticed any significant
> performance difference with and without it.

OK, you'll have to decide at which layer LUKS fits.
There are pro and cons for each case.

> SSDs:
> Create empty partition on larger system SSD.
> Add smaller empty SSD.
> 
> Create raid 10 far 2 raid array with mdadm for example (need to check
> what metadata means!):
> mdadm --create /dev/md-ssd --level=10 --metadata=0.90 --raid-devices=2
> --layout=f2 /dev/sda4 /dev/sdb

Metadata 1.0, 1.1 and 1.2 are the new ones, use these,
not the 0.90, which have less features.

Maybe, not really sure, but partitioning /dev/sdb might
be better.

> Use crypt setup to create an encrypted block device on top of this raid
> 10 far 2 device.
> 
> Use LVM to create a volume group on top of the encrypted raid 10 far 2
> device.

Or the other way around.
I'm not sure which is better, maybe your proposal.

> Create a logical volume on top of the above to use as the cache device.

Yep, if you use lvmcache, maybe bcache can do as well.

> HDD:
> Similar to the SSD case above except that the logical volume will be for
> the slow disks doing the bulk of the work.

OK, I think.

> Create a cached device:
> Use lvmcache to create a device using the the two logical volumes
> created above (bcache would also work).

Seems good to me.

> Create a file system on top of the cached device.  If using btrfs (what
> I do now) use single for data as the raid is occuring a couple of levels
> down.

Well, it might be btrfs has already RAID-10.
Again, code is shared between this and md too.

> Of course, I could have got completely the wrong idea above and invented
> some horrid monster!

If I understood it right, it sounds OK to me.

I would, in any case, strongly suggest to experiment,
maybe, as mentioned above, with loop devices.
Not for performances, but for practising possible
combinations and layouts.

Then there is the story of the caching, which has
different scope and performances between bcache
and lvmcache.
In your specific case, I cannot judge which is better,
but lvmcache seems to me easier.

> It looks like I could do this without the LVM layer between the LUKs and
> cache parts.  However, this does give me the flexability to create
> logical volumes that are on HDD RAID10, SDD RAID10 soley as well as
> cached.  Or different cache options.  I think I could have put another
> LVM on top of the cached item I was creating but I think that was over
> doing it.

Probably it is.
I would try to use not more than one component at time.
So, 1 md, 1 LUKS, 1 LVM, at maximum.
If possible less.

It would be also possible to create two PVs out of the two
RAID-10 and a single VG with both.
Then the LV can be fitted in one or the other PV.
LUKS can be at RAID level or on top of LV.

This type of "generic" setup has some advantages in case
of hardware updates (easy to add / remove storage devices,
by using pvmove).

bye,

-- 

piergiorgio

Back to comp.os.linux.hardware | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-07 22:41 +0000
  Re: Speed ups for a disk IO bound machine Roger Blake <rogblake@iname.invalid> - 2016-12-08 16:44 +0000
    Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-10 11:24 +0000
      Re: Speed ups for a disk IO bound machine Roger Blake <rogblake@iname.invalid> - 2016-12-11 01:34 +0000
        Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-11 22:07 +0000
  Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-10 12:43 +0100
    Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-10 20:44 +0000
      Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-10 23:05 +0100
        Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-11 11:54 +0000
          Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-11 14:10 +0100
            Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-11 22:46 +0000
              Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-12 19:22 +0100
            Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-13 10:17 +0100
              Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-14 20:03 +0000
                Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-15 12:55 +0100
                Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-15 23:18 +0000
                Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-16 09:34 +0100
        Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-13 09:57 +0100
  Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-12 09:11 +0100
    Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-12 23:08 +0000
      Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-13 09:29 +0100
        Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-14 23:16 +0000
          Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-15 13:15 +0100
            Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-15 22:50 +0000
              Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-16 09:39 +0100

csiph-web