Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.setup > #2666 > unrolled thread

Pointers on LVM and Partitioning.

Back to article view | Back to comp.os.linux.setup

Contents

  Pointers on  LVM and Partitioning. Harry <simonsharry@gmail.com> - 2012-04-17 00:01 -0700
    Re: Pointers on  LVM and Partitioning. David Brown <david@westcontrol.removethisbit.com> - 2012-04-17 09:23 +0200
      Re: Pointers on  LVM and Partitioning. Harry <simonsharry@gmail.com> - 2012-04-17 07:38 -0700
        Re: Pointers on  LVM and Partitioning. David Brown <david.brown@removethis.hesbynett.no> - 2012-04-17 22:36 +0200
          Re: Pointers on  LVM and Partitioning. Harry <simonsharry@gmail.com> - 2012-04-19 07:14 -0700
          Re: Pointers on  LVM and Partitioning. Doug Freyburger <dfreybur@yahoo.com> - 2012-04-19 16:51 +0000
    Re: Pointers on  LVM and Partitioning. Richard Kettlewell <rjk@greenend.org.uk> - 2012-04-17 12:50 +0100
    Re: Pointers on  LVM and Partitioning. Doug Freyburger <dfreybur@yahoo.com> - 2012-04-17 17:15 +0000
    Re: Pointers on  LVM and Partitioning. GangGreene <GangGreene@invalid.com> - 2012-04-17 18:05 -0400

#2666 — Pointers on LVM and Partitioning.

Hello,

I know next to nothing about this topic as I write this, but would like to learn *everything* about it. By 'everything', I mean: I should be able to not just install and set up systems but also troubleshoot and repair, should it become necessary.

I don't have loads of hardware (CPUs or disks) with me to play with; just a dual-core Fedora 16 desktop with a 250 GB disk, 4 GB RAM, and may be a couple of USB 2GB+ sticks. Thus, for the most part, I'll have to rely on some virtual machine environment to make up for the necessary physical hardware.

This self-learning exercise could last several months and, so, before I proceeded with it, I thought I could use some helpful, Getting Started tips from you forum members, like:
 * if there are any good books that cover this topic thoroughly;
 * what kind of virtual machine env I should setup (VirtualBox, Xen, or...) ?
 * how did *you* learn this topic yourself (any helpful tips?)

Would the virtual env be enough to learn and try out all concepts that fall under the related topics of Partitioning, LVM & (software) Raid? If not, can I get a list of things that I will *not* be able to learn in such a virtual setup no matter what... so that I know about them upfront?

I have only a very vague idea right now of what all I will possibly be doing. A partial list would be: setting up virtual machines, creating lots of small-sized  virtual disks (say, each <= 1MB), losetup's, dmsetup's, and other man Section 8 commands.

Many thanks in advance.
/HS

[toc] | [next] | [standalone]

#2667

On 17/04/2012 09:01, Harry wrote:
> Hello,
>
> I know next to nothing about this topic as I write this, but would
> like to learn *everything* about it. By 'everything', I mean: I
> should be able to not just install and set up systems but also
> troubleshoot and repair, should it become necessary.
>
> I don't have loads of hardware (CPUs or disks) with me to play with;
> just a dual-core Fedora 16 desktop with a 250 GB disk, 4 GB RAM, and
> may be a couple of USB 2GB+ sticks. Thus, for the most part, I'll
> have to rely on some virtual machine environment to make up for the
> necessary physical hardware.
>
> This self-learning exercise could last several months and, so, before
> I proceeded with it, I thought I could use some helpful, Getting
> Started tips from you forum members, like: * if there are any good
> books that cover this topic thoroughly; * what kind of virtual
> machine env I should setup (VirtualBox, Xen, or...) ? * how did *you*
> learn this topic yourself (any helpful tips?)
>
> Would the virtual env be enough to learn and try out all concepts
> that fall under the related topics of Partitioning, LVM&  (software)
> Raid? If not, can I get a list of things that I will *not* be able to
> learn in such a virtual setup no matter what... so that I know about
> them upfront?
>
> I have only a very vague idea right now of what all I will possibly
> be doing. A partial list would be: setting up virtual machines,
> creating lots of small-sized  virtual disks (say, each<= 1MB),
> losetup's, dmsetup's, and other man Section 8 commands.
>
> Many thanks in advance. /HS

Use VirtualBox - it's the easiest, fastest, and most powerful 
virtualisation software for desktops for general use.  KVM and Xen have 
some other benefits, and are perhaps a better choice for servers, but 
VBox is definitely my first choice for desktops.

You can create multiple virtual disks and play with raid in virtual 
machines.  I'd recommend getting a second real disk - you'll value the 
extra space when you make lots of experimental virtual machines, and you 
can also put your virtual disks on two physical disks and see that raid 
makes a speed difference.

You'll want your virtual disks a little bigger than 1MB - you need to be 
able to put something on them to test.  Something like 50MB or 100MB 
would be better.

Another trick is to use loopback files.  Make some files of the size you 
want, and set them up as loopback devices - then you can pretend they 
are disks without having to do any virtualisation.  For extra points, 
put the files on a tmpfs file system - then they are not even on the disk.

[toc] | [prev] | [next] | [standalone]

#2669

On Tuesday, April 17, 2012 12:53:12 PM UTC+5:30, David Brown wrote:
> On 17/04/2012 09:01, Harry wrote:
> > Hello,
> >
> > I know next to nothing about this topic as I write this, but would
> > like to learn *everything* about it. By 'everything', I mean: I
> > should be able to not just install and set up systems but also
> > troubleshoot and repair, should it become necessary.
> >
> > I don't have loads of hardware (CPUs or disks) with me to play with;
> > just a dual-core Fedora 16 desktop with a 250 GB disk, 4 GB RAM, and
> > may be a couple of USB 2GB+ sticks. Thus, for the most part, I'll
> > have to rely on some virtual machine environment to make up for the
> > necessary physical hardware.
> >
> > This self-learning exercise could last several months and, so, before
> > I proceeded with it, I thought I could use some helpful, Getting
> > Started tips from you forum members, like: * if there are any good
> > books that cover this topic thoroughly; * what kind of virtual
> > machine env I should setup (VirtualBox, Xen, or...) ? * how did *you*
> > learn this topic yourself (any helpful tips?)
> >
> > Would the virtual env be enough to learn and try out all concepts
> > that fall under the related topics of Partitioning, LVM&  (software)
> > Raid? If not, can I get a list of things that I will *not* be able to
> > learn in such a virtual setup no matter what... so that I know about
> > them upfront?
> >
> > I have only a very vague idea right now of what all I will possibly
> > be doing. A partial list would be: setting up virtual machines,
> > creating lots of small-sized  virtual disks (say, each<= 1MB),
> > losetup's, dmsetup's, and other man Section 8 commands.
> >
> > Many thanks in advance. /HS
> 
> Use VirtualBox - it's the easiest, fastest, and most powerful 
> virtualisation software for desktops for general use.  KVM and Xen have 
> some other benefits, and are perhaps a better choice for servers, but 
> VBox is definitely my first choice for desktops.

Would you like to elaborate just a little bit more on the benefits of KVM and Xen, esp on servers? It's possible that, on my own, I may not immediately notice such benefits, or any server vs client vm differences. Any why VBox for desktops: why not KVM/Xen for desktops as well?
 
> You can create multiple virtual disks and play with raid in virtual 
> machines.  I'd recommend getting a second real disk - you'll value the 
> extra space when you make lots of experimental virtual machines, and you 
> can also put your virtual disks on two physical disks and see that raid 
> makes a speed difference.

That's a very good point.

> Another trick is to use loopback files.  Make some files of the size you 
> want, and set them up as loopback devices - then you can pretend they 
> are disks without having to do any virtualisation.  For extra points, 
> put the files on a tmpfs file system - then they are not even on the disk.

I'm simply dying to play with losetup and dmsetup, just don't know what *all* practical applications to use them for. It would be wonderful if you/sb could share some realworld applications where you have put them to good use. Here's one that I've found so far: http://linuxgazette.net/114/kapil.html

[toc] | [prev] | [next] | [standalone]

#2676

On 17/04/12 16:38, Harry wrote:
> On Tuesday, April 17, 2012 12:53:12 PM UTC+5:30, David Brown wrote:
>> On 17/04/2012 09:01, Harry wrote:
>>> Hello,
>>>
>>> I know next to nothing about this topic as I write this, but
>>> would like to learn *everything* about it. By 'everything', I
>>> mean: I should be able to not just install and set up systems but
>>> also troubleshoot and repair, should it become necessary.
>>>
>>> I don't have loads of hardware (CPUs or disks) with me to play
>>> with; just a dual-core Fedora 16 desktop with a 250 GB disk, 4 GB
>>> RAM, and may be a couple of USB 2GB+ sticks. Thus, for the most
>>> part, I'll have to rely on some virtual machine environment to
>>> make up for the necessary physical hardware.
>>>
>>> This self-learning exercise could last several months and, so,
>>> before I proceeded with it, I thought I could use some helpful,
>>> Getting Started tips from you forum members, like: * if there are
>>> any good books that cover this topic thoroughly; * what kind of
>>> virtual machine env I should setup (VirtualBox, Xen, or...) ? *
>>> how did *you* learn this topic yourself (any helpful tips?)
>>>
>>> Would the virtual env be enough to learn and try out all
>>> concepts that fall under the related topics of Partitioning, LVM&
>>> (software) Raid? If not, can I get a list of things that I will
>>> *not* be able to learn in such a virtual setup no matter what...
>>> so that I know about them upfront?
>>>
>>> I have only a very vague idea right now of what all I will
>>> possibly be doing. A partial list would be: setting up virtual
>>> machines, creating lots of small-sized  virtual disks (say,
>>> each<= 1MB), losetup's, dmsetup's, and other man Section 8
>>> commands.
>>>
>>> Many thanks in advance. /HS
>>
>> Use VirtualBox - it's the easiest, fastest, and most powerful
>> virtualisation software for desktops for general use.  KVM and Xen
>> have some other benefits, and are perhaps a better choice for
>> servers, but VBox is definitely my first choice for desktops.
>
> Would you like to elaborate just a little bit more on the benefits of
> KVM and Xen, esp on servers? It's possible that, on my own, I may not
> immediately notice such benefits, or any server vs client vm
> differences. Any why VBox for desktops: why not KVM/Xen for desktops
> as well?
>

With Xen, you have a minimal OS on the bottom, rather than having quite 
a substantial Linux host as needed for VBox.  I think Xen also makes it 
easier to attach PCI cards and other hardware to particular virtual 
machines (though VBox has experimental support for that sort of thing). 
  KVM is built into the kernel, and thus is probably slightly more 
efficient.  It also has support for merging duplicate pages, which can 
be a benefit when you have lots of virtual machines.

VirtualBox, however, beats them both on ease of use on a desktop.  And 
because it is installed as an application, rather than part of the 
kernel or a low-level hypervisor, it is something that can be added at 
any time to any machine.  It is also cross-platform, so that you can run 
it on Windows and Mac hosts too, if that is of interest.

>> You can create multiple virtual disks and play with raid in
>> virtual machines.  I'd recommend getting a second real disk -
>> you'll value the extra space when you make lots of experimental
>> virtual machines, and you can also put your virtual disks on two
>> physical disks and see that raid makes a speed difference.
>
> That's a very good point.
>
>> Another trick is to use loopback files.  Make some files of the
>> size you want, and set them up as loopback devices - then you can
>> pretend they are disks without having to do any virtualisation.
>> For extra points, put the files on a tmpfs file system - then they
>> are not even on the disk.
>
> I'm simply dying to play with losetup and dmsetup, just don't know
> what *all* practical applications to use them for. It would be
> wonderful if you/sb could share some realworld applications where you
> have put them to good use. Here's one that I've found so far:
> http://linuxgazette.net/114/kapil.html

I have had three practical applications for loopback devices.  One is a 
common situation - mounting an iso file using a loopback device (using 
"mount -o loop" rather than explicitly creating the device).  That's 
quite a common situation.

Another was to create a small file in a tmpfs directory, export it as a 
network block device, then from a small embedded Linux system I 
connected to it as a network block device and used it as a swap drive. 
It was a little slow doing swap over the network, but it gave the 
embedded system more virtual memory.

Finally, and more relevantly, I used a set of loopbacks on tmpfs files 
to learn more about md raid and mdadm.  In particular, I played around 
with safely resizing layered raids (i.e., resizing while always having 
redundancy).  You can read the thread here:

<http://marc.info/?l=linux-raid&m=130255832215796&w=2>

[toc] | [prev] | [next] | [standalone]

#2680

On Wednesday, April 18, 2012 2:06:33 AM UTC+5:30, David Brown wrote:
> On 17/04/12 16:38, Harry wrote:
> > On Tuesday, April 17, 2012 12:53:12 PM UTC+5:30, David Brown wrote:
> >> On 17/04/2012 09:01, Harry wrote:
> >>> Hello,
> >>>
> >>> I know next to nothing about this topic as I write this, but
> >>> would like to learn *everything* about it. By 'everything', I
> >>> mean: I should be able to not just install and set up systems but
> >>> also troubleshoot and repair, should it become necessary.
> >>>
> >>> I don't have loads of hardware (CPUs or disks) with me to play
> >>> with; just a dual-core Fedora 16 desktop with a 250 GB disk, 4 GB
> >>> RAM, and may be a couple of USB 2GB+ sticks. Thus, for the most
> >>> part, I'll have to rely on some virtual machine environment to
> >>> make up for the necessary physical hardware.
> >>>
> >>> This self-learning exercise could last several months and, so,
> >>> before I proceeded with it, I thought I could use some helpful,
> >>> Getting Started tips from you forum members, like: * if there are
> >>> any good books that cover this topic thoroughly; * what kind of
> >>> virtual machine env I should setup (VirtualBox, Xen, or...) ? *
> >>> how did *you* learn this topic yourself (any helpful tips?)
> >>>
> >>> Would the virtual env be enough to learn and try out all
> >>> concepts that fall under the related topics of Partitioning, LVM&
> >>> (software) Raid? If not, can I get a list of things that I will
> >>> *not* be able to learn in such a virtual setup no matter what...
> >>> so that I know about them upfront?
> >>>
> >>> I have only a very vague idea right now of what all I will
> >>> possibly be doing. A partial list would be: setting up virtual
> >>> machines, creating lots of small-sized  virtual disks (say,
> >>> each<= 1MB), losetup's, dmsetup's, and other man Section 8
> >>> commands.
> >>>
> >>> Many thanks in advance. /HS
> >>
> >> Use VirtualBox - it's the easiest, fastest, and most powerful
> >> virtualisation software for desktops for general use.  KVM and Xen
> >> have some other benefits, and are perhaps a better choice for
> >> servers, but VBox is definitely my first choice for desktops.
> >
> > Would you like to elaborate just a little bit more on the benefits of
> > KVM and Xen, esp on servers? It's possible that, on my own, I may not
> > immediately notice such benefits, or any server vs client vm
> > differences. Any why VBox for desktops: why not KVM/Xen for desktops
> > as well?
> >
> 
> With Xen, you have a minimal OS on the bottom, rather than having quite 
> a substantial Linux host as needed for VBox.  I think Xen also makes it 
> easier to attach PCI cards and other hardware to particular virtual 
> machines (though VBox has experimental support for that sort of thing). 
>   KVM is built into the kernel, and thus is probably slightly more 
> efficient.  It also has support for merging duplicate pages, which can 
> be a benefit when you have lots of virtual machines.
> 
> VirtualBox, however, beats them both on ease of use on a desktop.  And 
> because it is installed as an application, rather than part of the 
> kernel or a low-level hypervisor, it is something that can be added at 
> any time to any machine.  It is also cross-platform, so that you can run 
> it on Windows and Mac hosts too, if that is of interest.

That was quite a useful info. 

> >> You can create multiple virtual disks and play with raid in
> >> virtual machines.  I'd recommend getting a second real disk -
> >> you'll value the extra space when you make lots of experimental
> >> virtual machines, and you can also put your virtual disks on two
> >> physical disks and see that raid makes a speed difference.
> >
> > That's a very good point.
> >
> >> Another trick is to use loopback files.  Make some files of the
> >> size you want, and set them up as loopback devices - then you can
> >> pretend they are disks without having to do any virtualisation.
> >> For extra points, put the files on a tmpfs file system - then they
> >> are not even on the disk.
> >
> > I'm simply dying to play with losetup and dmsetup, just don't know
> > what *all* practical applications to use them for. It would be
> > wonderful if you/sb could share some realworld applications where you
> > have put them to good use. Here's one that I've found so far:
> > http://linuxgazette.net/114/kapil.html
> 
> I have had three practical applications for loopback devices.  One is a 
> common situation - mounting an iso file using a loopback device (using 
> "mount -o loop" rather than explicitly creating the device).  That's 
> quite a common situation.

Yep, I have used it many times myself.

> Another was to create a small file in a tmpfs directory, export it as a 
> network block device, then from a small embedded Linux system I 
> connected to it as a network block device and used it as a swap drive. 
> It was a little slow doing swap over the network, but it gave the 
> embedded system more virtual memory.

Cool! I definitely intend to try that out with my desktop as the embedded system, and a coworker's desktop as the additional virtual memory source.

(You deserve some sort of a prize for that! Also, the Unix toolbox continues to astound me.)

> Finally, and more relevantly, I used a set of loopbacks on tmpfs files 
> to learn more about md raid and mdadm.  In particular, I played around 
> with safely resizing layered raids (i.e., resizing while always having 
> redundancy).  You can read the thread here:
> 
> <http://marc.info/?l=linux-raid&m=130255832215796&w=2>

I'm not very literate with Raids just yet, so will better be able appreciate this application (of loopbacks) may be few weeks/months down the road.

Thanks again, David!

[toc] | [prev] | [next] | [standalone]

#2684

David Brown wrote:
>
> With Xen, you have a minimal OS on the bottom, rather than having quite 
> a substantial Linux host as needed for VBox.  I think Xen also makes it 
> easier to attach PCI cards and other hardware to particular virtual 
> machines (though VBox has experimental support for that sort of thing). 
>   KVM is built into the kernel, and thus is probably slightly more 
> efficient.  It also has support for merging duplicate pages, which can 
> be a benefit when you have lots of virtual machines.

Xen comes with an assortment of complexities because it is intended as
the supervisor host for virtual images.  If you don't intend to run
virtual images I suggest not using the Xen kernel.

One client recently took a host to 64 GB memory.  Common enough for a
database production server.  It only acknowledged 32 GB when it booted. 
They called me to ask why.  I started digging.  "Hey look, you're
running a Xen kernel, any idea why?"  "No that's just how the previous
folks built it."  "We'll need to schedule another outage to switch to a
vanilla kernel.  When would you like."  "Saturday in three weeks might
be good.  We'll get back to you to schedule it."

The previous consultant had proposed using that box to host a web server
farm each image serving out interfaces for a different application.  Nce
idea but they eventually bought some physical boxes and built some
images under VMware to do that and forgot the build details of that host.

[toc] | [prev] | [next] | [standalone]

#2668

Harry <simonsharry@gmail.com> writes:
> I know next to nothing about this topic as I write this, but would
> like to learn *everything* about it. By 'everything', I mean: I should
> be able to not just install and set up systems but also troubleshoot
> and repair, should it become necessary.
>
> I don't have loads of hardware (CPUs or disks) with me to play with;
> just a dual-core Fedora 16 desktop with a 250 GB disk, 4 GB RAM, and
> may be a couple of USB 2GB+ sticks. Thus, for the most part, I'll have
> to rely on some virtual machine environment to make up for the
> necessary physical hardware.
>
> This self-learning exercise could last several months and, so, before
> I proceeded with it, I thought I could use some helpful, Getting
> Started tips from you forum members, like:
>  * if there are any good books that cover this topic thoroughly;
>  * what kind of virtual machine env I should setup (VirtualBox, Xen,
>    or...) ?

I've got on well with KVM, which I suspect you'll find is natively
supported by Fedora.  libvirt and its GUI virt-manager make it pretty
easy to set things up.

>  * how did *you* learn this topic yourself (any helpful tips?)

I read the man pages and started using the tools.

-- 
http://www.greenend.org.uk/rjk/

[toc] | [prev] | [next] | [standalone]

#2671

Harry wrote:
>
>  * how did *you* learn this topic yourself (any helpful tips?)

AIX led the way in logical volume management and as usual with IBM
failed to evangelize it to the rest of UNIX.  Their system is simple
once you get used to it and it shares concepts with what ended up as
Linux LVM2.

Veritas on Solaris led the way on popularizing LVM systems to other
UNIX-like systems.  It is extent based not block based so instead of
having many fixed side extents in a pool it slices subdisks directly to
size.  Cleaner in the implementation while being dirtier in the ideas.

HPUX followed AIX done the LVM path.  It's extent based system has been
emulated almost exactly for Linux.  I learned HPUX LVM as it evolved
from HPUX 9 on.  HPUX 8 failed to deal with partitions on disks the way
other systems did so HPUX had a much more powerful driver to push
maturity in LVM systems.

Having over a decade of experience on LVM gave me plenty of time to let
the concepts simplify.  I learned it the same way I learned virtual
memory and eventually realized the concepts overlap.  By the time I saw
it on Linux I recognized it as the HPUX system almost without change.

[toc] | [prev] | [next] | [standalone]

#2677

On Tue, 17 Apr 2012 00:01:09 -0700, Harry wrote:

> Hello,
> 
> I know next to nothing about this topic as I write this, but would like
> to learn *everything* about it. By 'everything', I mean: I should be
> able to not just install and set up systems but also troubleshoot and
> repair, should it become necessary.
> 


https://wiki.archlinux.org/index.php/Lvm

[toc] | [prev] | [standalone]

Back to top | Article view | comp.os.linux.setup

csiph-web