Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.networking > #3733 > unrolled thread

Lightweight nodes for learning hands-on TCP/IP networking and administration

Back to article view | Back to comp.os.linux.networking

Contents

  Lightweight nodes for learning hands-on TCP/IP networking and administration HS <simonsharry@gmail.com> - 2016-05-09 19:57 -0700
    Re: Lightweight nodes for learning hands-on TCP/IP networking and administration Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2016-05-10 09:06 +0300
      Re: Lightweight nodes for learning hands-on TCP/IP networking and administration HS <simonsharry@gmail.com> - 2016-05-12 04:28 -0700
    Re: Lightweight nodes for learning hands-on TCP/IP networking and administration "Carlos E. R." <robin_listas@invalid.es> - 2016-05-11 02:21 +0200
    Re: Lightweight nodes for learning hands-on TCP/IP networking and administration Stefan Monnier <monnier@iro.umontreal.ca> - 2016-05-11 08:53 -0400
      Re: Lightweight nodes for learning hands-on TCP/IP networking and administration Stefan Monnier <monnier@iro.umontreal.ca> - 2016-05-11 09:23 -0400
        Re: Lightweight nodes for learning hands-on TCP/IP networking and administration HS <simonsharry@gmail.com> - 2016-05-12 04:34 -0700
        Re: Lightweight nodes for learning hands-on TCP/IP networking and administration HS <simonsharry@gmail.com> - 2016-05-12 21:41 -0700
          Re: Lightweight nodes for learning hands-on TCP/IP networking and administration Stefan Monnier <monnier@iro.umontreal.ca> - 2016-05-13 08:06 -0400
      Re: Lightweight nodes for learning hands-on TCP/IP networking and administration HS <simonsharry@gmail.com> - 2016-05-12 04:39 -0700
        Re: Lightweight nodes for learning hands-on TCP/IP networking and administration Stefan Monnier <monnier@iro.umontreal.ca> - 2016-05-12 08:17 -0400

#3733 — Lightweight nodes for learning hands-on TCP/IP networking and administration

Hello,

I know enough of the basics of TCP/IP theory and would now like to self-learn the hands-on aspects of it as well as you typically would in a training class for aspiring network-/system administrators. Topics including but not limited to: setting up routes, firewalls, proxy servers, and services like mail, DHCP etc; basic network troubleshooting and some Linux security related topics as well. 

In this hands-on exercise I'd need to have root access to multiple Linux nodes/hosts. However, all I have got with me is a single Ubuntu 16 workstation!

I could use VirtualBox-based VMs (for my nodes) but each VM node would take a heavy toll on my disk, RAM, and CPU performance! 

I have looked at LxC also. It can surely cut down the RAM footprint and boot-up times of my node but I'd still need to have a *copy* of the VM image/rootfs for each of the nodes on the disk (each disk copy being several 100 MBs). The reason for this *copy* is: I'm using ext4fs which does not support efficient cloning of LxC disk images. Reformatting my existing ext4fs to use, say, zfs or btrfs (that both support cloning), would be very disruptive/painful!

So, staying within ext4fs, what are my options for having disk-, RAM-, and CPU-friendly nodes for learning TCP/IP network and system administration, and some security topics as well? 

Thanks in advance,
/HS

[toc] | [next] | [standalone]

#3734

On 10.5.16 05:57, HS wrote:
> Hello,
>
> I know enough of the basics of TCP/IP theory and would now like to self-learn the hands-on aspects of it as well as you typically would in a training class for aspiring network-/system administrators. Topics including but not limited to: setting up routes, firewalls, proxy servers, and services like mail, DHCP etc; basic network troubleshooting and some Linux security related topics as well.
>
> In this hands-on exercise I'd need to have root access to multiple Linux nodes/hosts. However, all I have got with me is a single Ubuntu 16 workstation!
>
> I could use VirtualBox-based VMs (for my nodes) but each VM node would take a heavy toll on my disk, RAM, and CPU performance!
>
> I have looked at LxC also. It can surely cut down the RAM footprint and boot-up times of my node but I'd still need to have a *copy* of the VM image/rootfs for each of the nodes on the disk (each disk copy being several 100 MBs). The reason for this *copy* is: I'm using ext4fs which does not support efficient cloning of LxC disk images. Reformatting my existing ext4fs to use, say, zfs or btrfs (that both support cloning), would be very disruptive/painful!
>
> So, staying within ext4fs, what are my options for having disk-, RAM-, and CPU-friendly nodes for learning TCP/IP network and system administration, and some security topics as well?
>
> Thanks in advance,
> /HS


Buy a bunch of Raspberry Pi's and a switch.

-- 

-TV

[toc] | [prev] | [next] | [standalone]

#3749

On Tuesday, May 10, 2016 at 11:36:47 AM UTC+5:30, Tauno Voipio wrote:
> On 10.5.16 05:57, HS wrote:
> > Hello,
> >
> > I know enough of the basics of TCP/IP theory and would now like to self-learn the hands-on aspects of it as well as you typically would in a training class for aspiring network-/system administrators. Topics including but not limited to: setting up routes, firewalls, proxy servers, and services like mail, DHCP etc; basic network troubleshooting and some Linux security related topics as well.
> >
> > In this hands-on exercise I'd need to have root access to multiple Linux nodes/hosts. However, all I have got with me is a single Ubuntu 16 workstation!
> >
> > I could use VirtualBox-based VMs (for my nodes) but each VM node would take a heavy toll on my disk, RAM, and CPU performance!
> >
> > I have looked at LxC also. It can surely cut down the RAM footprint and boot-up times of my node but I'd still need to have a *copy* of the VM image/rootfs for each of the nodes on the disk (each disk copy being several 100 MBs). The reason for this *copy* is: I'm using ext4fs which does not support efficient cloning of LxC disk images. Reformatting my existing ext4fs to use, say, zfs or btrfs (that both support cloning), would be very disruptive/painful!
> >
> > So, staying within ext4fs, what are my options for having disk-, RAM-, and CPU-friendly nodes for learning TCP/IP network and system administration, and some security topics as well?
> >
> > Thanks in advance,
> > /HS
> 
> 
> Buy a bunch of Raspberry Pi's and a switch.
> 
> -- 
> 
> -TV

That is a very interesting option, surely. +1.

[toc] | [prev] | [next] | [standalone]

#3741

On 2016-05-10 04:57, HS wrote:
> In this hands-on exercise I'd need to have root access to multiple Linux nodes/hosts. However, all I have got with me is a single Ubuntu 16 workstation!
> 
> I could use VirtualBox-based VMs (for my nodes) but each VM node would take a heavy toll on my disk, RAM, and CPU performance!

What do you call heavy? You need less than 8 GB "disk" for each, and
maybe 512 MB of ram, or less in text mode. You don't need to run heavy
processes on them, do you? Just don't install a desktop on each, or
install a lightweight one. Perhaps XFCE, but there are even lighters.

-- 
Cheers,
       Carlos E.R.

[toc] | [prev] | [next] | [standalone]

#3743

> I have looked at LxC also. It can surely cut down the RAM footprint and
> boot-up times of my node but I'd still need to have a *copy* of the VM
> image/rootfs for each of the nodes on the disk (each disk copy being several
> 100 MBs). The reason for this *copy* is: I'm using ext4fs which does not
> support efficient cloning of LxC disk images.

Not sure I understand your remark about ext4, but here's how I think I'd
do it: I'd use lxc, and more specifically, I'd create an LVM volume for
an initial lxc container.  Once I have this container working in
a satisfactory way, I'd make N "LVM snapshots" of this volume.
From then on, you don't want to write to the original LVM volume any
more (more specifically, you can, but it costs too much, so make sure
you either don't mount this volume any more or only mount it read-only)
The snapshots are writable and you can use them for your
N lxc containers.  Of course, you don't have to know N beforehand, you
can create new snapshots as the need arises.

See http://tldp.org/HOWTO/LVM-HOWTO/snapshotintro.html


        Stefan

[toc] | [prev] | [next] | [standalone]

#3744

> a satisfactory way, I'd make N "LVM snapshots" of this volume.

Another option of course is to use overlayfs.  IIUC lxc-clone even
provides builtin support for that via the "-B overlayfs" argument.


        Stefan

[toc] | [prev] | [next] | [standalone]

#3750

On Wednesday, May 11, 2016 at 6:53:45 PM UTC+5:30, Stefan Monnier wrote:
> > a satisfactory way, I'd make N "LVM snapshots" of this volume.
> 
> Another option of course is to use overlayfs.  IIUC lxc-clone even
> provides builtin support for that via the "-B overlayfs" argument.
> 
> 
>         Stefan

Will check that out, thanks!

[toc] | [prev] | [next] | [standalone]

#3754

On Wednesday, May 11, 2016 at 6:53:45 PM UTC+5:30, Stefan Monnier wrote:
> > a satisfactory way, I'd make N "LVM snapshots" of this volume.
> 
> Another option of course is to use overlayfs.  IIUC lxc-clone even
> provides builtin support for that via the "-B overlayfs" argument.
> 
> 
>         Stefan

lxc-clone - This command is deprecated now in favor of lxc-copy.

With lxc-copy, this is what I've found so far:

1. This creates u2 as a physical copy of u1, and so, just as big as u1. Kinda expected it also. 

Not what I wanted; I wanted a ufs-like disk footprint.
  
  $ lxc-copy -n u1 -N u2 

2. This creates u2 as a lightweight snapshot of u1. Hoever, subsequent changes made to u1's filesystem get reflected in u2's as well - which is not what I want. Apparently, snapshotting has the semantics of 'marking' to which you could revert later.

Not what I wanted; I wanted both u1 and u2 to be independent.
  
  $ lxc-copy -n u1 -N u2 -B overlayfs -s

3. Without the -s snapshot option, the command doesn't even run:

  $ lxc-copy -n u1 -N u2 -B overlayfs
  lxc-copy: bdev/lxcoverlay.c: ovl_clonepaths: 58 overlayfs is only for snapshot clones
  lxc-copy: bdev/bdev.c: bdev_copy: 388 failed getting pathnames for cloned storage: /var/lib/lxc/u1/rootfs
  lxc-copy: lxccontainer.c: copy_storage: 2844 Error copying storage.
  clone failed


/HS

[toc] | [prev] | [next] | [standalone]

#3755

> 2. This creates u2 as a lightweight snapshot of u1. Hoever, subsequent
> changes made to u1's filesystem get reflected in u2's as well - which is not
> what I want. Apparently, snapshotting has the semantics of 'marking' to
> which you could revert later.
>
> Not what I wanted; I wanted both u1 and u2 to be independent.
>  
>   $ lxc-copy -n u1 -N u2 -B overlayfs -s

If you want 2 independent ones, then you do:

   $ lxc-copy -n u1 -N u2 -B overlayfs -s
   $ lxc-copy -n u1 -N u3 -B overlayfs -s

and then u2 and u3 are independent.  Better not touch u1 after that.


        Stefan

[toc] | [prev] | [next] | [standalone]

#3751

On Wednesday, May 11, 2016 at 6:23:14 PM UTC+5:30, Stefan Monnier wrote:
> > I have looked at LxC also. It can surely cut down the RAM footprint and
> > boot-up times of my node but I'd still need to have a *copy* of the VM
> > image/rootfs for each of the nodes on the disk (each disk copy being several
> > 100 MBs). The reason for this *copy* is: I'm using ext4fs which does not
> > support efficient cloning of LxC disk images.
> 
> Not sure I understand your remark about ext4, but here's how I think I'd
> do it: I'd use lxc, and more specifically, I'd create an LVM volume for
> an initial lxc container.  Once I have this container working in
> a satisfactory way, I'd make N "LVM snapshots" of this volume.
> From then on, you don't want to write to the original LVM volume any
> more (more specifically, you can, but it costs too much, so make sure
> you either don't mount this volume any more or only mount it read-only)
> The snapshots are writable and you can use them for your
> N lxc containers.  Of course, you don't have to know N beforehand, you
> can create new snapshots as the need arises.
> 
> See http://tldp.org/HOWTO/LVM-HOWTO/snapshotintro.html
> 
> 
>         Stefan

Thanks Stefan. I've also started looking at Docker since it uses aufs. The standard Docker Ubuntu 16 image with some additional (networking related) packages installed is coming to < 250 MB! My guess so far is, a Docker node will be much easier and lighterweight (on disk) relative to an LxC based node. But will still like to try out what you suggest above.

[toc] | [prev] | [next] | [standalone]

#3752

> Thanks Stefan. I've also started looking at Docker since it uses aufs. The
> standard Docker Ubuntu 16 image with some additional (networking related)
> packages installed is coming to < 250 MB! My guess so far is, a Docker node
> will be much easier and lighterweight (on disk) relative to an LxC based
> node. But will still like to try out what you suggest above.

LXC (with overlayfs) and Docker use basically the same underlying tools
in basically the same way, AFAIK, so in terms of resource usage, it
should be a wash.


        Stefan

[toc] | [prev] | [standalone]

Back to top | Article view | comp.os.linux.networking

csiph-web