Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #159581

Re: Backup solutions without reinventig the wheel these days

Show all headers | View raw

[Multipart message — attachments visible in raw view] - view raw

I could try to request a RAM upgrade (not super easy in uni environment,
but doable), but it goes against my understanding of what a storage server
is. In my book, a storage server needs fast IO capabilities (storage drives
and NIC) and possibly CPU for compression. Deduplication is the only reason
I could think of higher RAM requirements on a storage server.

Sarunas, did you have good experiences with actually using the backups for
restoring?

On Wed, Oct 21, 2015 at 4:28 PM, Sarunas Burdulis <
sarunas@math.dartmouth.edu> wrote:

> On 10/21/2015 05:32 AM, Ondřej Grover wrote:
> > [...]
> > Your btrfs setup sounds interesting, Sarunas. Are you running those
> > btrfs volumes on Jessie? Any stability issues? Do you think it could
> > work with just 1GB RAM + 1 GB swap? I was thinking about doing
> > out-of-band deduplication to conserve RAM. Do you use both compression
> > and deduplication (file or block level)? I heard they don't play nicely
> > together. Could you share your mount options?
>
> Our backup server is actually running Linux 3.13.0-66-generic, Btrfs
> v3.12 — Ubuntu 14.04.3 LTS, which is behind Jessie in software versions.
> It has been very stable — for the past too years it's running as a
> headless box in an off-site location.
>
> We don't use compression or deduplication. In-band dedup is not in btrfs
> 3.*, and I haven't tried any out-of-band tools. Nothing particular in
> Btrfs mount, the only non-default options in 'noatime':
>
> LABEL=BACKUPS /backups btrfs noatime 0 0
>
> 1GB RAM sounds low by today's habits. That doesn't mean it won't work,
> but just begs for RAM upgrade.
>
> Sarunas
>
> > On Tue, Oct 20, 2015 at 8:35 PM, Sarunas Burdulis
> > <sarunas@math.dartmouth.edu <mailto:sarunas@math.dartmouth.edu>> wrote:
> >
> > On 10/20/2015 12:57 PM, Ondřej Grover wrote:
> >> Hello,
> >
> >> I'm looking for recommendations for backup solutions that don't reinvent
> >> the wheel and are reliable and used. I want to backup two servers to a
> >> backup server. The main data content is several hundred GB in many very
> >> small files.
> >
> >> [...]
> >
> >> I've also looked at the new kids on the block like obnam, attick and
> >> borgbackup. They look interesting, but I prefer time-tested SW for
> backups.
> >> After realizing that these new backup programs pretty much try to
> >> replicate features of btrfs or ZFS (incremental snapshots, block-level
> >> compression and deduplication) I started thinking that I could perhaps
> >> just send the data to the backup server via rsync and save them to a
> >> btrfs or ZFS (but the backup server may not have enough RAM for ZFS) and
> >> create daily snaphosts on the server. If memory will permit (if I
> >> optimize it), I'd go with ZFS as it should be more reliable. Does
> >> anybody use such a solution?
> >
> > Yes, we do use pretty much the setup you describe, i.e. rsync push to
> > backup server, btrfs volume with daily snapshot subvolumes on the
> > server. This particular setup has been in use for more than two years by
> > now, and is working well with 8-10 servers (totals for all backed-up
> > filesystems is about 1.2TiB/2.5M files of varying size; transferred
> > daily delta is of course just a fraction of this). The backup server is
> > a rather modest self-built Mini-ITX Core i5 with 8GiB RAM and 6TiB
> > hardware (3ware) RAID-10 storage volume.
> >
> > Using btrfs volume in btrfs-raid configuration (raid-1 for data and
> > metadata) would provide protection against data bit-rot. We use several
> > such volumes on some of our severs, though not in combination with btrfs
> > snapshot subvolumes.
> >
> > --
> > Sarunas Burdulis
> > http://math.dartmouth.edu/~sarunas
>
>

Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Backup solutions without reinventig the wheel these days Ondřej Grover <ondrej.grover@gmail.com> - 2015-10-20 19:00 +0200
  Re: Backup solutions without reinventig the wheel these days Mario Castelán Castro  <marioxcc.MT@yandex.com> - 2015-10-20 19:50 +0200
    Re: Backup solutions without reinventig the wheel these days Teemu Likonen <tlikonen@iki.fi> - 2015-10-20 20:20 +0200
      Re: Backup solutions without reinventig the wheel these days Jonathan Dowland <jmtd@debian.org> - 2015-10-27 12:40 +0100
        Re: Backup solutions without reinventig the wheel these days Teemu Likonen <tlikonen@iki.fi> - 2015-10-27 16:40 +0100
  Re: Backup solutions without reinventig the wheel these days Sarunas Burdulis <sarunas@math.dartmouth.edu> - 2015-10-20 21:10 +0200
    Re: Backup solutions without reinventig the wheel these days Ondřej Grover <ondrej.grover@gmail.com> - 2015-10-21 11:40 +0200
      Re: Backup solutions without reinventig the wheel these days mourik jan heupink <heupink@merit.unu.edu> - 2015-10-21 13:30 +0200
        Re: Backup solutions without reinventig the wheel these days mourik jan heupink <heupink@merit.unu.edu> - 2015-10-21 13:40 +0200
      Re: Backup solutions without reinventig the wheel these days Eduardo M KALINOWSKI <eduardo@kalinowski.com.br> - 2015-10-21 13:30 +0200
      Re: Backup solutions without reinventig the wheel these days Dan Ritter <dsr@randomstring.org> - 2015-10-21 16:10 +0200
      Re: Backup solutions without reinventig the wheel these days Sarunas Burdulis <sarunas@math.dartmouth.edu> - 2015-10-21 16:30 +0200
        Re: Backup solutions without reinventig the wheel these days Ondřej Grover <ondrej.grover@gmail.com> - 2015-10-21 16:50 +0200
          Re: Backup solutions without reinventig the wheel these days Sarunas Burdulis <sarunas@math.dartmouth.edu> - 2015-10-21 17:10 +0200
            Re: Backup solutions without reinventig the wheel these days Ondřej Grover <ondrej.grover@gmail.com> - 2015-10-22 08:50 +0200
              Re: Backup solutions without reinventig the wheel these days Sarunas Burdulis <sarunas@math.dartmouth.edu> - 2015-10-22 16:10 +0200
  Re: Backup solutions without reinventig the wheel these days Glenn English <ghe@slsware.net> - 2015-10-22 18:30 +0200

csiph-web