Re: Homebuilt NAS Advice

From	Leslie Rhorer <lesrhorer@att.net>
Newsgroups	linux.debian.user
Subject	Re: Homebuilt NAS Advice
Date	2020-08-06 03:50 +0200
Message-ID	<AADcB-4b4-1@gated-at.bofh.it> (permalink)
References	(3 earlier) <AzRUl-LM-3@gated-at.bofh.it> <AAhlM-7WC-1@gated-at.bofh.it> <AAlSp-2j1-1@gated-at.bofh.it> <AAnrb-34D-5@gated-at.bofh.it> <AAwNQ-gO-3@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw


On 8/5/2020 1:51 PM, deloptes wrote:

> Imagine you have classical backup: daily incrementals, full weekly and full
> monthly.

	This is not required with DAR.  One full backup is all that is 
required.  Anything else is a waste of time and space.  One can employ 
either differential or decremental backups, keeping a full history of 
all activity on the system.

  Imagine you have retention for the full weekly 3 (until end of
> month) and for full monthly 12 (until end of year).
> You have to maintain 15 full backups and the 6 daily incrementals. How much
> space is it, that you need for your backup storage?
> This is why the question what is your active size. No imagine I have 2TB of
> data, even if I compress this data - lets say with avg. 60% ratio it is
> 800GB per full backup. 15 copies + means 12TB+. Of course if you have
> video/audio like mp3, it is already compressed and ratio for the backup
> compression goes down and space needed up.
> Now here comes the trick with deduplication. The backup system makes one
> full backup (800GB) and then keeps track of the bits that changed (it is
> not that simple, but for the example). Only they are being backuped. Some
> systems provide ratio of 90%. So to keep your 15+ copies with deduplication
> ratio of ~80% you need about 3TB.

	You are talking about differential archiving at the binary level. DAR 
can employ delta binaries for both incremental and decremental backups, 
which means only the changes to the binary data, if any, is stored in 
subsequent backups, not the entire file.  There is no need for multiple 
full backups.  A full backup of my main array to hard drive media will 
take a minimum of 15 days.  Transferring across the 10G link to my 
backup server would take a minimum of 2.8 days.  Although full backups 
of smaller systems will normally take less time, the fact is full 
backups on a regular basis are just not necessary, and especially not 
when using DAR.


>> volume size is 46.9 Terabytes (43.6 Teribytes) after formatting. ??The
>> main server currently has 22 Terabytes of data on it. ??The backup server
>> is effectively full.
>>
> 
> So you have a perfect candidate for deduplication :) because I guess you can
> keep only few copies of that size on the backup server.

	A few copies of what size?  The backup server is an exact mirror of the 
main server, plus several T of additional files I don't need on the main 
server.  My DAR backups include new files (typically a few hundred GB) 
plus changes to files (typically a few MB).  Borg isn't really offering 
anything DAR does not, except DAR is not server based.  If I wanted to 
run DAR on my backup server, I could, but I don't want to.  I use DAR 
for offline archival storage.  Thew backup array is a mirror, not an 
archive.

	In the case of a catastrophic failure of the main server or its array, 
I simply bring up the backup server as a replacement until such time - 
days later - as the main server is rebuilt.  In the case of an 
accidentally lost or corrupted file(s), I just do a cp from the backup 
to the main server, restoring the files to the state they were in as of 
04:00 this morning.  Typically, this takes a few seconds.

Thread

Homebuilt NAS Advice Patrick Bartek <patrick.bartek@gmail.com> - 2020-07-29 22:00 +0200
  Re: Homebuilt NAS Advice Dan Ritter <dsr@randomstring.org> - 2020-07-29 22:20 +0200
    Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-07-29 22:30 +0200
      Re: Homebuilt NAS Advice Dan Ritter <dsr@randomstring.org> - 2020-07-29 23:40 +0200
    Re: Homebuilt NAS Advice Patrick Bartek <patrick.bartek@gmail.com> - 2020-07-30 02:50 +0200
      Re: Homebuilt NAS Advice Tom Dial <tddial@comcast.net> - 2020-07-30 22:50 +0200
        Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-02 01:40 +0200
          Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-02 03:10 +0200
            Re: Homebuilt NAS Advice Keith Bainbridge <ke1th3216@gmail.com> - 2020-08-02 04:50 +0200
              Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-02 06:10 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-02 21:30 +0200
            Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-02 22:10 +0200
          Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-03 06:10 +0200
            Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-03 17:30 +0200
              Re: Homebuilt NAS Advice Dan Ritter <dsr@randomstring.org> - 2020-08-03 17:40 +0200
              Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-03 18:40 +0200
                Re: Homebuilt NAS Advice <tomas@tuxteam.de> - 2020-08-03 19:10 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-03 22:00 +0200
                Re: Homebuilt NAS Advice <tomas@tuxteam.de> - 2020-08-03 22:20 +0200
                Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-03 23:50 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-04 00:50 +0200
            Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-04 00:40 +0200
    Re: Homebuilt NAS Advice Anssi Saari <as@sci.fi> - 2020-07-30 10:00 +0200
      Re: Homebuilt NAS Advice rhkramer@gmail.com - 2020-07-30 13:20 +0200
  Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-07-30 01:50 +0200
    Re: Homebuilt NAS Advice Patrick Bartek <nemommxiv@gmail.com> - 2020-07-30 23:40 +0200
    Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-08 01:30 +0200
      Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-08 03:30 +0200
        Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-09 14:40 +0200
  Re: Homebuilt NAS Advice Peter Ehlert <peter@sdi-baja.com> - 2020-07-30 19:00 +0200
    Re: Homebuilt NAS Advice Charles Curley <charlescurley@charlescurley.com> - 2020-07-30 21:50 +0200
      Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-07-31 13:10 +0200
    Re: Homebuilt NAS Advice Patrick Bartek <nemommxiv@gmail.com> - 2020-07-30 23:30 +0200
      Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-02 01:50 +0200
        Re: Homebuilt NAS Advice Patrick Bartek <nemommxiv@gmail.com> - 2020-08-02 17:50 +0200
  Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-03 17:20 +0200
    Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-04 00:10 +0200
      Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-04 01:20 +0200
        Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-04 10:40 +0200
          Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-04 11:30 +0200
            Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-04 16:00 +0200
              Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-04 16:30 +0200
              Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-05 05:50 +0200
                Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-05 16:20 +0200
                Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-05 17:00 +0200
                Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-05 17:40 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 12:10 +0200
                Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-06 16:10 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 17:10 +0200
                Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-06 18:10 +0200
                Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-07 18:30 +0200
                Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-07 19:40 +0200
                Re: Homebuilt NAS Advice rhkramer@gmail.com - 2020-08-08 00:10 +0200
                Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-09 04:30 +0200
                Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-09 11:10 +0200
                Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-09 17:00 +0200
                Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-09 18:30 +0200
                Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-09 19:20 +0200
                Re: Homebuilt NAS Advice rhkramer@gmail.com - 2020-08-09 23:50 +0200
                Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-10 00:00 +0200
                Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-10 00:30 +0200
                Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-10 00:50 +0200
                Re: Homebuilt NAS Advice rhkramer@gmail.com - 2020-08-10 00:30 +0200
                Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-10 00:30 +0200
                Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-09 18:40 +0200
                Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-09 23:00 +0200
                Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-06 18:20 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 05:30 +0200
                Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-07 18:30 +0200
                Re: Homebuilt NAS Advice rhkramer@gmail.com - 2020-08-06 17:20 +0200
                Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-06 18:20 +0200
                Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-06 18:20 +0200
        Re: Homebuilt NAS Advice Andrei POPESCU <andreimpopescu@gmail.com> - 2020-08-04 17:20 +0200
          Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-05 06:20 +0200
        Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-05 04:30 +0200
          Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-05 09:20 +0200
            Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-05 11:00 +0200
              Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-05 21:00 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 03:50 +0200
                Re: Homebuilt NAS Advice Kushal Kumaran <kushal@locationd.net> - 2020-08-06 06:20 +0200
                Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-06 18:20 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 04:10 +0200
                Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-07 05:50 +0200
            Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-05 12:30 +0200
      Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-05 12:30 +0200
        Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-05 21:10 +0200
          Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-05 23:30 +0200
            Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-06 00:30 +0200
              Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 01:20 +0200
                Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-08 09:10 +0200
              Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-06 04:30 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 04:50 +0200
                Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-06 06:20 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 07:10 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 07:30 +0200
                Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-06 18:10 +0200
                Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-06 18:30 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 03:00 +0200
                Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-07 03:20 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 05:00 +0200
                Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-07 05:40 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 08:00 +0200
                Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-07 08:50 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 09:10 +0200
                Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-07 17:50 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-08 01:40 +0200
                Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-08 04:50 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-08 22:30 +0200
                Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-09 00:20 +0200
                Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-09 01:30 +0200
                Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-09 22:40 +0200
                Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-10 00:30 +0200
                Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-10 01:30 +0200
    Re: Homebuilt NAS Advice Patrick Bartek <nemommxiv@gmail.com> - 2020-08-04 06:30 +0200

csiph-web