Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.hardware > #3252

Re: Speed ups for a disk IO bound machine

Show all headers | View raw

On 12/11/2016 01:10 PM, Piergiorgio Sartor wrote:

> Be careful, it is easy to destroy data.
> Maybe you can practice with loop devices.
> There are some howtos around.
> 

Thanks.  I'm reasonably careful.  However, I'd probally image the device
before trying gparted on it.

>> With RAID-10 far 2 am I correct in assuming that the available capacity
>> for a two device array of 3TB disks would be 3TB (two copies of data)?
> 
> Yep.

So I have the same space as I have now with btrfs, but with striping and
no loss of redundancy.  Cool.




>> SSDs:
>> Create empty partition on larger system SSD.
>> Add smaller empty SSD.
>>
>> Create raid 10 far 2 raid array with mdadm for example (need to check
>> what metadata means!):
>> mdadm --create /dev/md-ssd --level=10 --metadata=0.90 --raid-devices=2
>> --layout=f2 /dev/sda4 /dev/sdb
> 
> Metadata 1.0, 1.1 and 1.2 are the new ones, use these,
> not the 0.90, which have less features.

I've not looked into that.  Perhaps the default is sensible if I don't
pick one.  Must do reading.

> 
> Maybe, not really sure, but partitioning /dev/sdb might
> be better.

I've been using btrfs for a couple of years now, where it is quite
normal not to partition disks.  So maybe partition.

> 
>> Use crypt setup to create an encrypted block device on top of this raid
>> 10 far 2 device.
>>
>> Use LVM to create a volume group on top of the encrypted raid 10 far 2
>> device.
> 
> Or the other way around.
> I'm not sure which is better, maybe your proposal.
> 


I though on top of the raid as otherwise there are four lots of LUKS
with passwords to manage (careful use of keyfiles would undoubtedly
help).  Logically ought to be less overhead doing it twice rather than
four times, but that is a guess.

>> Create a logical volume on top of the above to use as the cache device.
> 
> Yep, if you use lvmcache, maybe bcache can do as well.
> 
>> HDD:
>> Similar to the SSD case above except that the logical volume will be for
>> the slow disks doing the bulk of the work.
> 
> OK, I think.
> 
>> Create a cached device:
>> Use lvmcache to create a device using the the two logical volumes
>> created above (bcache would also work).
> 
> Seems good to me.
> 
>> Create a file system on top of the cached device.  If using btrfs (what
>> I do now) use single for data as the raid is occuring a couple of levels
>> down.
> 
> Well, it might be btrfs has already RAID-10.
> Again, code is shared between this and md too.
> 

Btrfs does indeed support RAID10.  However, it is fairly new and does
not appear to support far 2.  So mdadm is the way to achieve that.

>> Of course, I could have got completely the wrong idea above and invented
>> some horrid monster!
> 
> If I understood it right, it sounds OK to me.


Thanks for your help.

> 
> I would, in any case, strongly suggest to experiment,
> maybe, as mentioned above, with loop devices.
> Not for performances, but for practising possible
> combinations and layouts.
> 

Any for ensuring I can get a sensible workable system. :-)

> Then there is the story of the caching, which has
> different scope and performances between bcache
> and lvmcache.
> In your specific case, I cannot judge which is better,
> but lvmcache seems to me easier.
> 

Really I'm not sure.  But it seems worthwhile giving a cache system a go.

>> It looks like I could do this without the LVM layer between the LUKs and
>> cache parts.  However, this does give me the flexability to create
>> logical volumes that are on HDD RAID10, SDD RAID10 soley as well as
>> cached.  Or different cache options.  I think I could have put another
>> LVM on top of the cached item I was creating but I think that was over
>> doing it.
> 
> Probably it is.
> I would try to use not more than one component at time.
> So, 1 md, 1 LUKS, 1 LVM, at maximum.
> If possible less.

I'd not expect it to work if I set it up in one go.
> 
> It would be also possible to create two PVs out of the two
> RAID-10 and a single VG with both.
> Then the LV can be fitted in one or the other PV.
> LUKS can be at RAID level or on top of LV.

If I understand correctly:  One mdadm raid10 far 2 from SSDs, one from
HDDs and then add them BOTH to one single VG, and then build the cached
file system from that one volume group? I did not think it worked like
that, as you'd have to control which disks the cache was on.

> 
> This type of "generic" setup has some advantages in case
> of hardware updates (easy to add / remove storage devices,
> by using pvmove).
> 
> bye,
> 

Back to comp.os.linux.hardware | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-07 22:41 +0000
  Re: Speed ups for a disk IO bound machine Roger Blake <rogblake@iname.invalid> - 2016-12-08 16:44 +0000
    Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-10 11:24 +0000
      Re: Speed ups for a disk IO bound machine Roger Blake <rogblake@iname.invalid> - 2016-12-11 01:34 +0000
        Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-11 22:07 +0000
  Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-10 12:43 +0100
    Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-10 20:44 +0000
      Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-10 23:05 +0100
        Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-11 11:54 +0000
          Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-11 14:10 +0100
            Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-11 22:46 +0000
              Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-12 19:22 +0100
            Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-13 10:17 +0100
              Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-14 20:03 +0000
                Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-15 12:55 +0100
                Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-15 23:18 +0000
                Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-16 09:34 +0100
        Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-13 09:57 +0100
  Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-12 09:11 +0100
    Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-12 23:08 +0000
      Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-13 09:29 +0100
        Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-14 23:16 +0000
          Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-15 13:15 +0100
            Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-15 22:50 +0000
              Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-16 09:39 +0100

csiph-web