Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #173016 > unrolled thread

Recommendation: Backup system

Back to article view | Back to linux.debian.user

Contents

  Recommendation: Backup system mo <mo777@gmx.net> - 2016-10-01 11:40 +0200
    Re: Recommendation: Backup system Mark Fletcher <mark27q1@gmail.com> - 2016-10-01 14:50 +0200
      Re: Recommendation: Backup system mo <mo777@gmx.net> - 2016-10-01 15:10 +0200
      Re: Recommendation: Backup system Gene Heskett <gheskett@shentel.net> - 2016-10-01 18:30 +0200
        Re: Recommendation: Backup system Clive Menzies <clive@clivemenzies.co.uk> - 2016-10-01 18:50 +0200
          Re: Recommendation: Backup system Gene Heskett <gheskett@shentel.net> - 2016-10-01 19:50 +0200
            Re: Recommendation: Backup system Clive Menzies <clive@clivemenzies.co.uk> - 2016-10-01 20:00 +0200
              Re: Recommendation: Backup system Glenn English <ghe2001@gmail.com> - 2016-10-01 22:10 +0200
              Re: Recommendation: Backup system Gene Heskett <gheskett@shentel.net> - 2016-10-02 06:20 +0200
        Re: Recommendation: Backup system Glenn English <ghe2001@gmail.com> - 2016-10-01 22:00 +0200
          Re: Recommendation: Backup system mo <mo777@gmx.net> - 2016-10-01 22:50 +0200
        Re: Recommendation: Backup system mo <mo777@gmx.net> - 2016-10-01 22:40 +0200
    Re: Recommendation: Backup system Bob Weber <bobrweber@gmail.com> - 2016-10-01 16:50 +0200
      Re: Recommendation: Backup system mo <mo777@gmx.net> - 2016-10-01 17:10 +0200
      Re: Recommendation: Backup system Mark Fletcher <mark27q1@gmail.com> - 2016-10-01 17:40 +0200
    Re: Recommendation: Backup system mo <mo777@gmx.net> - 2016-10-01 17:20 +0200
    Re: Recommendation: Backup system Dan Purgert <dan@djph.net> - 2016-10-01 19:10 +0200
      Re: Recommendation: Backup system mo <mo777@gmx.net> - 2016-10-01 22:50 +0200
    Re: Recommendation: Backup system mo <mo777@gmx.net> - 2016-10-01 22:50 +0200
      Re: Recommendation: Backup system Bob Weber <bobrweber@gmail.com> - 2016-10-01 23:10 +0200
        Re: Recommendation: Backup system mo <mo777@gmx.net> - 2016-10-02 10:50 +0200
          Re: Recommendation: Backup system rhkramer@gmail.com - 2016-10-02 15:00 +0200
            Re: Recommendation: Backup system Bob Weber <bobrweber@gmail.com> - 2016-10-03 01:20 +0200
      Re: Recommendation: Backup system Dan Purgert <dan@djph.net> - 2016-10-02 03:00 +0200
        Re: Recommendation: Backup system mo <mo777@gmx.net> - 2016-10-02 11:00 +0200
          Re: Recommendation: Backup system Dan Purgert <dan@djph.net> - 2016-10-02 13:10 +0200
            Re: Recommendation: Backup system mo <mo777@gmx.net> - 2016-10-02 14:40 +0200
    Re: Recommendation: Backup system Jonathan Dowland <jmtd@debian.org> - 2016-10-03 11:50 +0200
      Re: Recommendation: Backup system Teemu Likonen <tlikonen@iki.fi> - 2016-10-03 14:40 +0200
        Re: Recommendation: Backup system Markus Grunwald <markus@the-grue.de> - 2016-10-03 14:50 +0200
          Re: Recommendation: Backup system Richard Hector <richard@walnut.gen.nz> - 2016-10-05 04:10 +0200
            Re: Recommendation: Backup system Daniel Bareiro <daniel-listas@gmx.net> - 2016-10-05 05:10 +0200
              Re: Recommendation: Backup system Richard Hector <richard@walnut.gen.nz> - 2016-10-05 23:40 +0200
            Re: Recommendation: Backup system Jonathan Dowland <jmtd@debian.org> - 2016-10-07 12:20 +0200
              Re: Recommendation: Backup system Richard Hector <richard@walnut.gen.nz> - 2016-10-12 12:10 +0200
        Re: Recommendation: Backup system Jonathan Dowland <jmtd@debian.org> - 2016-10-03 17:10 +0200
      Re: Recommendation: Backup system Celejar <celejar@gmail.com> - 2016-10-10 19:30 +0200
        Re: Recommendation: Backup system Jonathan Dowland <jmtd@debian.org> - 2016-10-11 00:00 +0200
          Re: Recommendation: Backup system Dave Thayer <debian1628420.dmthayer@recursor.net> - 2016-10-11 05:40 +0200
    Re: Recommendation: Backup system Jarle Aase <jgaa@jgaa.com> - 2016-10-04 20:50 +0200

Page 1 of 2  [1] 2  Next page →

#173016 — Recommendation: Backup system

Hi Debian users :)

Information:
Distributor ID:	Debian
Description:	Debian GNU/Linux 8.6 (jessie)
Release:	8.6
Codename:	jessie

As the title say i'm in search for a backup application/system.
Currently i manage my backups with a little script that i wrote... but 
it does not really serve my needs anymore.
I want to be able to make backups on my main PC and also on my server, 
the backups i would then store on my NAS.

Make a long story short:
Have you guys a recommendation for me?
Is there a specific application you use for your backups guys?

Btw: I dont mind configuring or playing around with new applications, 
every recommendation is welcome ;)


Here is my current backup script (Which is run by cron daily):
#!/bin/bash

TO_BACKUP="/home /etc /var/log"
BACKUP_DIR="/var/backup"
BACKUP_ARCHIVE="backup-`date +%d_%m_%Y-%H:%M`.tar"
TAR_OPTIONS='-cpf'

delete_old_backup() {
	if [ -f ${BACKUP_DIR}/backup*.tar ]; then
		rm -rf $BACKUP_DIR/backup*
	fi
}

create_new_backup() {
	tar $TAR_OPTIONS ${BACKUP_DIR}/$BACKUP_ARCHIVE $TO_BACKUP
}

main() {
	delete_old_backup
	create_new_backup
}

main

Greets
mo

[toc] | [next] | [standalone]

#173037

On Sat, Oct 01, 2016 at 11:37:31AM +0200, mo wrote:
> Hi Debian users :)
> 
> Information:
> Distributor ID:	Debian
> Description:	Debian GNU/Linux 8.6 (jessie)
> Release:	8.6
> Codename:	jessie
> 
> As the title say i'm in search for a backup application/system.
> Currently i manage my backups with a little script that i wrote... but it
> does not really serve my needs anymore.
> I want to be able to make backups on my main PC and also on my server, the
> backups i would then store on my NAS.
> 
> Make a long story short:
> Have you guys a recommendation for me?
> Is there a specific application you use for your backups guys?

I know Gene is a fan of Amanda, I have it on my list to try it out 
myself based on positive remarks he has made about it in the past.

Mark

[toc] | [prev] | [next] | [standalone]

#173042

Am 01.10.2016 um 14:40 schrieb Mark Fletcher:
> On Sat, Oct 01, 2016 at 11:37:31AM +0200, mo wrote:
>> Hi Debian users :)
>>
>> Information:
>> Distributor ID:	Debian
>> Description:	Debian GNU/Linux 8.6 (jessie)
>> Release:	8.6
>> Codename:	jessie
>>
>> As the title say i'm in search for a backup application/system.
>> Currently i manage my backups with a little script that i wrote... but it
>> does not really serve my needs anymore.
>> I want to be able to make backups on my main PC and also on my server, the
>> backups i would then store on my NAS.
>>
>> Make a long story short:
>> Have you guys a recommendation for me?
>> Is there a specific application you use for your backups guys?
>
> I know Gene is a fan of Amanda, I have it on my list to try it out
> myself based on positive remarks he has made about it in the past.

Sounds good to me :) - I'll set that on the todo list myself to (Just 
checked the webpage of Amanda and it does sound pretty good!).

Gene, can you elaborate on Amanda? :) (If Gene does read this mail)

> Mark
>

Greets

mo

[toc] | [prev] | [next] | [standalone]

#173064

On Saturday 01 October 2016 08:40:35 Mark Fletcher wrote:

> On Sat, Oct 01, 2016 at 11:37:31AM +0200, mo wrote:
> > Hi Debian users :)
> >
> > Information:
> > Distributor ID:	Debian
> > Description:	Debian GNU/Linux 8.6 (jessie)
> > Release:	8.6
> > Codename:	jessie
> >
> > As the title say i'm in search for a backup application/system.
> > Currently i manage my backups with a little script that i wrote...
> > but it does not really serve my needs anymore.
> > I want to be able to make backups on my main PC and also on my
> > server, the backups i would then store on my NAS.
> >
> > Make a long story short:
> > Have you guys a recommendation for me?
> > Is there a specific application you use for your backups guys?
>
> I know Gene is a fan of Amanda, I have it on my list to try it out
> myself based on positive remarks he has made about it in the past.
>
> Mark

Yeppers! It runs in the wee hours of the night here, for an hour or so. 
Currently backing up this machine, and 3 more on my little home network 
here, using its own unique, distribute the nightly load to equalize as 
much a it can given its list of what to back up with nightly backups 
totaling 11 to 14 Gb per night.

When I first started using it, I had a DDS2 changer, but it wasn't very 
dependable, and at only 4Gb per tape, limited what I could do.  About 
the time 500Gb Hd's came out, amanda had worked out a way to use virtual 
tapes on a hard drive, so I converted. That has the advantage that 
should I need to recover something, its random access instead of 
sequential, so I can get back anything I need in just a few minutes, 
most of which is spent studying the recovery docs because I've forgotten 
how to do it. ;-)

With tapes, I could easily be a half a day recovering the same file 
because each level of a backup has to be read from the start of the tape 
until its found.  If I need something whose last good backup was a level 
3, it would have to back up and find the level0, which is the last full 
backup, then find the level 1 and merge any changes, wash, rinse, and 
repeat.  A hard drive based system can do all that in seconds.

And HD's have become much more dependable than tape, along with the 
methods of warning the user that the drive is failing, and that alone 
beats tape all the way into the trash bin.

I was rather worried about the drive I use for amanda's v-tapes when I 
saw almost 3 years ago that smartctl said it had 25 Reallocated sectors.  
It still says 25 10 seconds ago.  That drive, now a 1Tb drive 
as /dev/sdc, now has 58357 Power up hours on it. I don't care what you 
may have paid for a tape library, it cannot survive that long, when this 
HD has done that for a $100 bill at the time I bought it.  And I can put 
this HD in a shirt pocket.  The tape library would need a refrigerator 
rated 2 wheel dolly to move it, and a similar second dolly to move its 
tapes.

Whats not to like?

I've had far more trouble dealing with tar changes as its been updated 
over the years than I've had with amanda itself. All have been fixable 
in a day or two once you can post the breakage on the tar list. amanda 
uses tar to do the bare metal work.  A wrapper for tar in that sense, 
and I then wrap amanda with a bash script that fixes the always a day 
late record keeping that you can add to the v-tape image by making a 
copy of amanda's database and writing it to the V-tape amanda just used, 
so I can lose the main drive and have to start with a new install on a 
fresh drive. With amanda I would install from the repo's on the new 
drive. Its 2 or 3 steps, but in an hours time I can have this working 
wheezy system with all its dross, put back on a new drive.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#173065

On 01/10/16 17:22, Gene Heskett wrote
> Yeppers! It runs in the wee hours of the night here, for an hour or so.
> Currently backing up this machine, and 3 more on my little home network
> here, using its own unique, distribute the nightly load to equalize as
> much a it can given its list of what to back up with nightly backups
> totaling 11 to 14 Gb per night.
>
Hi Gene

Quick question. Are your backups incremental or complete every night?

Regards

Clive

-- 
Clive Menzies
http://freecriticalthinking.org

[toc] | [prev] | [next] | [standalone]

#173067

On Saturday 01 October 2016 12:39:58 Clive Menzies wrote:

> On 01/10/16 17:22, Gene Heskett wrote
>
> > Yeppers! It runs in the wee hours of the night here, for an hour or
> > so. Currently backing up this machine, and 3 more on my little home
> > network here, using its own unique, distribute the nightly load to
> > equalize as much a it can given its list of what to back up with
> > nightly backups totaling 11 to 14 Gb per night.
>
> Hi Gene
>
> Quick question. Are your backups incremental or complete every night?
>
> Regards
>
> Clive

This is probably better explained in the manpages. Amanda has the concept 
of doing a full backup of everything in its disklist according to the 
days you give it. Amanda will then shuffle the schedule such that those 
full backups are done at random in that cycle, with an eye toward 
equalizing, as much as possible, the amount of data saved during each 
run. It does this by advanceing the the level 0's as it will not let a 
given file go beyond that cycle before a full copy is made again.
Level 0 is a full copy of a file, level 1 is whats been changed in that 
file since the last level 0. Level 2 is whats been changed since level 
1, ad infinitum but most useage never gets past level 2.  I have the 
drive I use for amanda setup as 30 virtual tapes, using one a day, then 
recycle. With 4 machines feeding amanda, that 1Tb drive stays at around 
46% used:

gene@coyote:~$ df /amandatapes
Filesystem     1K-blocks      Used Available Use% Mounted on
/dev/sdc2      960798056 410555684 501429932  46% /amandatapes

I stayed on the devel branch of amanda, playing the part of the canary in 
the coal mine for years while it was being heavily developed, but 
apparently that grant ran out so not a lot has changed in around 5 
years. So while I have a self made, slightly newer version on this 
machine as master, v3.3.7p1, the slaves are all running 3.3.1 clients 
from the wheezy repo.  And it Just Works(TM).

I would be the first to point out that my way is NOT for archival storage 
due to this 30 day and recycle setup. I could extend it to 60 days on 
this drive I suppose, but this is not a business.

For a business, I would include the price of the drive as a CODB, fill it 
up and put it on the shelf at a remote location so it doesn't all go up 
in smoke when the place burns, thereby giving me the ability to recover 
something 5+ years old, or for however long one has had that setup 
running.

That $100 or less a month for a new commodity drive is far less of a CODB 
than the archival storage of tapes would be over a 10 year period. And 
you would have to add in that the tape drive(s) would be out of service 
for about a month each annually while they spend the holidays in 
Oklahoma City getting fresh heads and rubber at about a kilobuck each 
for the rebuild.  Thats been the track record here in my usage of tapes.  
The hard drives all have been 10x (or) more dependable.

And all it takes is getting rid of the idea that one must do a full 
backup on Friday nights. Yes, amanda can do that, but do it as a 
separate configuration else you will drive the poor girl out of her mind 
when she finds out all her carefully worked out plans have all gone 
aglay.

And don't forget that in ones long term business plans, the technology 
changes with time and there will come a time when you will have nothing 
in the house that can read todays 1Tb sata hard drive.  So having a 
storage location to save the old tech that can read those drives should 
be part of that long term plan also.  And be damned hard headed about 
it.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#173068

On 01/10/16 18:40, Gene Heskett wrote:
> On Saturday 01 October 2016 12:39:58 Clive Menzies wrote:
>
>> Quick question. Are your backups incremental or complete every night?
> This is probably better explained in the manpages. Amanda has the concept
> of doing a full backup of everything in its disklist according to the
> days you give it. Amanda will then shuffle the schedule such that those
> full backups are done at random in that cycle, with an eye toward
> equalizing, as much as possible, the amount of data saved during each
> run. It does this by advanceing the the level 0's as it will not let a
> given file go beyond that cycle before a full copy is made again.
> Level 0 is a full copy of a file, level 1 is whats been changed in that
> file since the last level 0. Level 2 is whats been changed since level
> 1, ad infinitum but most useage never gets past level 2.  I have the
> drive I use for amanda setup as 30 virtual tapes, using one a day, then
> recycle. With 4 machines feeding amanda, that 1Tb drive stays at around
> 46% used:
>
> gene@coyote:~$ df /amandatapes
> Filesystem     1K-blocks      Used Available Use% Mounted on
> /dev/sdc2      960798056 410555684 501429932  46% /amandatapes
>
> I stayed on the devel branch of amanda, playing the part of the canary in
> the coal mine for years while it was being heavily developed, but
> apparently that grant ran out so not a lot has changed in around 5
> years. So while I have a self made, slightly newer version on this
> machine as master, v3.3.7p1, the slaves are all running 3.3.1 clients
> from the wheezy repo.  And it Just Works(TM).
>
> I would be the first to point out that my way is NOT for archival storage
> due to this 30 day and recycle setup. I could extend it to 60 days on
> this drive I suppose, but this is not a business.
>
> For a business, I would include the price of the drive as a CODB, fill it
> up and put it on the shelf at a remote location so it doesn't all go up
> in smoke when the place burns, thereby giving me the ability to recover
> something 5+ years old, or for however long one has had that setup
> running.
>
> That $100 or less a month for a new commodity drive is far less of a CODB
> than the archival storage of tapes would be over a 10 year period. And
> you would have to add in that the tape drive(s) would be out of service
> for about a month each annually while they spend the holidays in
> Oklahoma City getting fresh heads and rubber at about a kilobuck each
> for the rebuild.  Thats been the track record here in my usage of tapes.
> The hard drives all have been 10x (or) more dependable.
>
> And all it takes is getting rid of the idea that one must do a full
> backup on Friday nights. Yes, amanda can do that, but do it as a
> separate configuration else you will drive the poor girl out of her mind
> when she finds out all her carefully worked out plans have all gone
> aglay.
>
> And don't forget that in ones long term business plans, the technology
> changes with time and there will come a time when you will have nothing
> in the house that can read todays 1Tb sata hard drive.  So having a
> storage location to save the old tech that can read those drives should
> be part of that long term plan also.  And be damned hard headed about
> it.
>
Thanks Gene

A dozen years ago, we found a couple posts on incremental backups using 
rsync and adapted it to provide 6 months of rolling incremental backups. 
We've been running this setup ever since - automated nightly, 
incremental backups. I posted our notes on the installation earlier in 
this thread.

All important stuff is kept on the servers but it would be good to 
backup the laptops/workstations too and maybe Amanda is the answer :-) 
We don't install GUIs on our servers; can this be managed from 
individual workstations?

Regards

Clive

-- 
Clive Menzies
http://freecriticalthinking.org

[toc] | [prev] | [next] | [standalone]

#173078

> On Oct 1, 2016, at 11:54 AM, Clive Menzies <clive@clivemenzies.co.uk> wrote:
> 
> We don't install GUIs on our servers; can this be managed from individual workstations?

I'm not sure, but I think Amanda was written before GUIs existed :-)

-- 
Glenn English

[toc] | [prev] | [next] | [standalone]

#173100

On Saturday 01 October 2016 13:54:29 Clive Menzies wrote:

> On 01/10/16 18:40, Gene Heskett wrote:
> > On Saturday 01 October 2016 12:39:58 Clive Menzies wrote:
> >> Quick question. Are your backups incremental or complete every
> >> night?
> >
> > This is probably better explained in the manpages. Amanda has the
> > concept of doing a full backup of everything in its disklist
> > according to the days you give it. Amanda will then shuffle the
> > schedule such that those full backups are done at random in that
> > cycle, with an eye toward equalizing, as much as possible, the
> > amount of data saved during each run. It does this by advanceing the
> > the level 0's as it will not let a given file go beyond that cycle
> > before a full copy is made again. Level 0 is a full copy of a file,
> > level 1 is whats been changed in that file since the last level 0.
> > Level 2 is whats been changed since level 1, ad infinitum but most
> > useage never gets past level 2.  I have the drive I use for amanda
> > setup as 30 virtual tapes, using one a day, then recycle. With 4
> > machines feeding amanda, that 1Tb drive stays at around 46% used:
> >
> > gene@coyote:~$ df /amandatapes
> > Filesystem     1K-blocks      Used Available Use% Mounted on
> > /dev/sdc2      960798056 410555684 501429932  46% /amandatapes
> >
> > I stayed on the devel branch of amanda, playing the part of the
> > canary in the coal mine for years while it was being heavily
> > developed, but apparently that grant ran out so not a lot has
> > changed in around 5 years. So while I have a self made, slightly
> > newer version on this machine as master, v3.3.7p1, the slaves are
> > all running 3.3.1 clients from the wheezy repo.  And it Just
> > Works(TM).
> >
> > I would be the first to point out that my way is NOT for archival
> > storage due to this 30 day and recycle setup. I could extend it to
> > 60 days on this drive I suppose, but this is not a business.
> >
> > For a business, I would include the price of the drive as a CODB,
> > fill it up and put it on the shelf at a remote location so it
> > doesn't all go up in smoke when the place burns, thereby giving me
> > the ability to recover something 5+ years old, or for however long
> > one has had that setup running.
> >
> > That $100 or less a month for a new commodity drive is far less of a
> > CODB than the archival storage of tapes would be over a 10 year
> > period. And you would have to add in that the tape drive(s) would be
> > out of service for about a month each annually while they spend the
> > holidays in Oklahoma City getting fresh heads and rubber at about a
> > kilobuck each for the rebuild.  Thats been the track record here in
> > my usage of tapes. The hard drives all have been 10x (or) more
> > dependable.
> >
> > And all it takes is getting rid of the idea that one must do a full
> > backup on Friday nights. Yes, amanda can do that, but do it as a
> > separate configuration else you will drive the poor girl out of her
> > mind when she finds out all her carefully worked out plans have all
> > gone aglay.
> >
> > And don't forget that in ones long term business plans, the
> > technology changes with time and there will come a time when you
> > will have nothing in the house that can read todays 1Tb sata hard
> > drive.  So having a storage location to save the old tech that can
> > read those drives should be part of that long term plan also.  And
> > be damned hard headed about it.
>
> Thanks Gene
>
> A dozen years ago, we found a couple posts on incremental backups
> using rsync and adapted it to provide 6 months of rolling incremental
> backups. We've been running this setup ever since - automated nightly,
> incremental backups. I posted our notes on the installation earlier in
> this thread.
>
> All important stuff is kept on the servers but it would be good to
> backup the laptops/workstations too and maybe Amanda is the answer :-)
> We don't install GUIs on our servers; can this be managed from
> individual workstations?
>
> Regards
>
> Clive

The closest you'll get to a gui is using your fav text editor to 
configure it. If building your own, the example directory has copies of 
all that, very liberally commented.  That may also be in the server deb, 
but if I looked, I've forgotten what I found. Heck most of the time I 
can't remember if I had breakfast this morning...  Synaptic, once the 
amanda server has been installed, will be more than happy to show you 
what is available, and where it was put.  There is also a mailing list, 
at amanda.org with very knowledgable people to explain the fine details. 

As for the other machines here that I backup every night, I only worry 
about the directories that contain linuxcnc or rtapi stuff, all the rest 
of it is no farther away than the install dvd from 
www.linuxcnc.org/downloads if I can't find the written dvd.  The amount 
of data from each of those machines is maybe 25 megabytes each worst 
case per nightly run, typically just 1 or 2 megs.

As for the gcode that drives LinuxCNC, I write the huge majority of what 
I use, and I'm a huge fan of incremental loops. I tossed a thing 
together in about 45 minutes this morning, about 50 LOC, which ran for 
about 5 hours, digging the beginnings of a t-slot trench about 1.25" 
wide, 3.5" long, and .7 inches deep, in a block of cast iron that will 
eventually be the toolpost pad which mounts a quick change tool post on 
an elderly Sheldon metal lathe.  Once that was done, I drove it from the 
keyboard to finish the upside down "T" up and the toolpost is actually 
sitting bolted in place on it right now.  Some folks call me a 
j.o.a.t. ;-)  And some call me other lesser terms.  Shrug...  At 82 in 3 
more days, I am used to it.
 
Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#173077

> On Oct 1, 2016, at 10:22 AM, Gene Heskett <gheskett@shentel.net> wrote:
> 
> On Saturday 01 October 2016 08:40:35 Mark Fletcher wrote:
> 
>> I know Gene is a fan of Amanda, I have it on my list to try it out
>> myself based on positive remarks he has made about it in the past.

Yeah. Amanda's a good solution. I use it with tape.

There are a couple advantages to this:

Amanda's been around for a very long time (in computer years). So, like Debian stable, the bugs tend to have been worked out.

Amanda does its backups using plain old standard *nix software, dump or tar. So if you have a really nasty data loss, you can restore from a backup without dealing with data in a format built by the backup program. It's not a trivial job, I'm told, but if you're really stuck...

The tapes aren't disks, and every backup is done to a different and simple storage device with few moving parts. (The reliability of a disk RAID1 reduces this advantage significantly.)

OK, there are disadvantages too: Amanda's not easy to configure, and tapes and tape drives are very expensive and very slow. 

But I like the advantages more than the disadvantages. It does a really good and reliable job.

-- 
Glenn English

[toc] | [prev] | [next] | [standalone]

#173084

Am 01.10.2016 um 21:37 schrieb Glenn English:
>
>> On Oct 1, 2016, at 10:22 AM, Gene Heskett <gheskett@shentel.net> wrote:
>>
>> On Saturday 01 October 2016 08:40:35 Mark Fletcher wrote:
>>
>>> I know Gene is a fan of Amanda, I have it on my list to try it out
>>> myself based on positive remarks he has made about it in the past.
>
> Yeah. Amanda's a good solution. I use it with tape.
>
> There are a couple advantages to this:
>
> Amanda's been around for a very long time (in computer years). So, like Debian stable, the bugs tend to have been worked out.
>
> Amanda does its backups using plain old standard *nix software, dump or tar. So if you have a really nasty data loss, you can restore from a backup without dealing with data in a format built by the backup program. It's not a trivial job, I'm told, but if you're really stuck...
>
> The tapes aren't disks, and every backup is done to a different and simple storage device with few moving parts. (The reliability of a disk RAID1 reduces this advantage significantly.)
>
> OK, there are disadvantages too: Amanda's not easy to configure, and tapes and tape drives are very expensive and very slow.
>
> But I like the advantages more than the disadvantages. It does a really good and reliable job.
>

Amanda is beginning to be more and more interesting to me :)

[toc] | [prev] | [next] | [standalone]

#173079

Am 01.10.2016 um 18:22 schrieb Gene Heskett:
> On Saturday 01 October 2016 08:40:35 Mark Fletcher wrote:
>
>> On Sat, Oct 01, 2016 at 11:37:31AM +0200, mo wrote:
>>> Hi Debian users :)
>>>
>>> Information:
>>> Distributor ID:	Debian
>>> Description:	Debian GNU/Linux 8.6 (jessie)
>>> Release:	8.6
>>> Codename:	jessie
>>>
>>> As the title say i'm in search for a backup application/system.
>>> Currently i manage my backups with a little script that i wrote...
>>> but it does not really serve my needs anymore.
>>> I want to be able to make backups on my main PC and also on my
>>> server, the backups i would then store on my NAS.
>>>
>>> Make a long story short:
>>> Have you guys a recommendation for me?
>>> Is there a specific application you use for your backups guys?
>>
>> I know Gene is a fan of Amanda, I have it on my list to try it out
>> myself based on positive remarks he has made about it in the past.
>>
>> Mark
>
> Yeppers! It runs in the wee hours of the night here, for an hour or so.
> Currently backing up this machine, and 3 more on my little home network
> here, using its own unique, distribute the nightly load to equalize as
> much a it can given its list of what to back up with nightly backups
> totaling 11 to 14 Gb per night.
>
> When I first started using it, I had a DDS2 changer, but it wasn't very
> dependable, and at only 4Gb per tape, limited what I could do.  About
> the time 500Gb Hd's came out, amanda had worked out a way to use virtual
> tapes on a hard drive, so I converted. That has the advantage that
> should I need to recover something, its random access instead of
> sequential, so I can get back anything I need in just a few minutes,
> most of which is spent studying the recovery docs because I've forgotten
> how to do it. ;-)
>
> With tapes, I could easily be a half a day recovering the same file
> because each level of a backup has to be read from the start of the tape
> until its found.  If I need something whose last good backup was a level
> 3, it would have to back up and find the level0, which is the last full
> backup, then find the level 1 and merge any changes, wash, rinse, and
> repeat.  A hard drive based system can do all that in seconds.

I guess i'm too young for tapes... or let's put it that way: I've never 
used them. (Why would you need tapes at home anyway :D )

> And HD's have become much more dependable than tape, along with the
> methods of warning the user that the drive is failing, and that alone
> beats tape all the way into the trash bin.

True, the oldest HDD here (which is a 500GB drive) is running since 7 
years straight without a problem.

> I was rather worried about the drive I use for amanda's v-tapes when I
> saw almost 3 years ago that smartctl said it had 25 Reallocated sectors.
> It still says 25 10 seconds ago.  That drive, now a 1Tb drive
> as /dev/sdc, now has 58357 Power up hours on it. I don't care what you
> may have paid for a tape library, it cannot survive that long, when this
> HD has done that for a $100 bill at the time I bought it.  And I can put
> this HD in a shirt pocket.  The tape library would need a refrigerator
> rated 2 wheel dolly to move it, and a similar second dolly to move its
> tapes.
>
> Whats not to like?
>
> I've had far more trouble dealing with tar changes as its been updated
> over the years than I've had with amanda itself. All have been fixable
> in a day or two once you can post the breakage on the tar list. amanda
> uses tar to do the bare metal work.  A wrapper for tar in that sense,
> and I then wrap amanda with a bash script that fixes the always a day
> late record keeping that you can add to the v-tape image by making a
> copy of amanda's database and writing it to the V-tape amanda just used,
> so I can lose the main drive and have to start with a new install on a
> fresh drive. With amanda I would install from the repo's on the new
> drive. Its 2 or 3 steps, but in an hours time I can have this working
> wheezy system with all its dross, put back on a new drive.
>
> Cheers, Gene Heskett
>

Thanks for the info Gene ;)

[toc] | [prev] | [next] | [standalone]

#173058

[Multipart message — attachments visible in raw view] — view raw

I use backuppc.  It is web browser based setup and usage.  It takes incremental
and full backups that can remain as long as you want or have space for.  It can
browse files by name or in a version mode where you can see the date where a
file changed and restore an earlier version if you want (or to a separate 
download directory).  It compresses files for space and only keeps one copy of a
file's data if it is located in different directories or servers (using hard
links as needed).  It can even backup user data for windows users (samba).  I
use the rsync transfer for Linux machines and even with windows running Cygwin.

I currently backup 8 computers going back almost 1 year.  I even backup a vm at
digital ocean.  Backuppc reports this:

144 full backups of total size 8951.56GB (prior to pooling and compression),
57 incr backups of total size 57.13GB (prior to pooling and compression).
Pool is 358.94GB comprising 1903010 files and 4369 directories (as of 10/1 01:09).

So 8951GB is compressed or pooled into just 358 GB! 

*...Bob*
On 10/01/2016 05:37 AM, mo wrote:
> Hi Debian users :)
>
> Information:
> Distributor ID:    Debian
> Description:    Debian GNU/Linux 8.6 (jessie)
> Release:    8.6
> Codename:    jessie
>
> As the title say i'm in search for a backup application/system.
> Currently i manage my backups with a little script that i wrote... but it does
> not really serve my needs anymore.
> I want to be able to make backups on my main PC and also on my server, the
> backups i would then store on my NAS.
>
> Make a long story short:
> Have you guys a recommendation for me?
> Is there a specific application you use for your backups guys?
>
> Btw: I dont mind configuring or playing around with new applications, every
> recommendation is welcome ;)
>
>
> Here is my current backup script (Which is run by cron daily):
> #!/bin/bash
>
> TO_BACKUP="/home /etc /var/log"
> BACKUP_DIR="/var/backup"
> BACKUP_ARCHIVE="backup-`date +%d_%m_%Y-%H:%M`.tar"
> TAR_OPTIONS='-cpf'
>
> delete_old_backup() {
>     if [ -f ${BACKUP_DIR}/backup*.tar ]; then
>         rm -rf $BACKUP_DIR/backup*
>     fi
> }
>
> create_new_backup() {
>     tar $TAR_OPTIONS ${BACKUP_DIR}/$BACKUP_ARCHIVE $TO_BACKUP
> }
>
> main() {
>     delete_old_backup
>     create_new_backup
> }
>
> main
>
> Greets
> mo
>
>

[toc] | [prev] | [next] | [standalone]

#173059

Am 01.10.2016 um 16:46 schrieb Bob Weber:
> I use backuppc.  It is web browser based setup and usage.  It takes
> incremental and full backups that can remain as long as you want or have
> space for.  It can browse files by name or in a version mode where you
> can see the date where a file changed and restore an earlier version if
> you want (or to a separate  download directory).  It compresses files
> for space and only keeps one copy of a file's data if it is located in
> different directories or servers (using hard links as needed).  It can
> even backup user data for windows users (samba).  I use the rsync
> transfer for Linux machines and even with windows running Cygwin.

I like the web interface part i have to say (Just browsed their website :) )
I will look into it ;)
(I have currently no windows machines here, but that might change in the 
future since i plan to deplay Windows Server 2012 R2)

Thanks for the hint ;)

> I currently backup 8 computers going back almost 1 year.  I even backup
> a vm at digital ocean.  Backuppc reports this:
>
> 144 full backups of total size 8951.56GB (prior to pooling and compression),
> 57 incr backups of total size 57.13GB (prior to pooling and compression).
> Pool is 358.94GB comprising 1903010 files and 4369 directories (as of
> 10/1 01:09).
>
> So 8951GB is compressed or pooled into just 358 GB!

The compression is quite nice, i might have at least 100GB to backup 
here. (Maybe a little more depending on what i want to save)

> *...Bob*

Greets

mo

[toc] | [prev] | [next] | [standalone]

#173061

On Sat, Oct 01, 2016 at 10:46:07AM -0400, Bob Weber wrote:
> I use backuppc.  It is web browser based setup and usage.  It takes incremental
> and full backups that can remain as long as you want or have space for.  It can
> browse files by name or in a version mode where you can see the date where a
> file changed and restore an earlier version if you want (or to a separate 
> download directory).  It compresses files for space and only keeps one copy of a
> file's data if it is located in different directories or servers (using hard
> links as needed).  It can even backup user data for windows users (samba).  I
> use the rsync transfer for Linux machines and even with windows running Cygwin.
> 
> I currently backup 8 computers going back almost 1 year.  I even backup a vm at
> digital ocean.  Backuppc reports this:
> 
> 144 full backups of total size 8951.56GB (prior to pooling and compression),
> 57 incr backups of total size 57.13GB (prior to pooling and compression).
> Pool is 358.94GB comprising 1903010 files and 4369 directories (as of 10/1 01:09).
> 
> So 8951GB is compressed or pooled into just 358 GB! 

Wow, that is impressive!

Mark

[toc] | [prev] | [next] | [standalone]

#173060

Has someone experience with Bacula?
I heard good things about it, although i never looked into it... maybe 
someone has and can give me his report on it :)

[toc] | [prev] | [next] | [standalone]

#173066

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

mo wrote:
> As the title say i'm in search for a backup application/system.
> Currently i manage my backups with a little script that i wrote... but 
> it does not really serve my needs anymore.
> I want to be able to make backups on my main PC and also on my server, 
> the backups i would then store on my NAS.

There's always rsync from the hosts to the NAS box.

Only thing that I see "wrong" with your script is that:
 - you're deleting the backups (what happens when you delete something,
   and don't catch it til after the next backup run?)
 - you're backing up everything, even if it hasn't changed.



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJX76m+AAoJEI4R3fMSeaKBwWQH/RiG+HH83WcyGT5wOhGM344j
UTOXN+VS7UC0jOsZG+MYrNJkAE8U9qwAYahi/63X+TFmSa2rFRmw6rv0pfvfBeiP
mItlVsjqyeR00tFc+zmAPWKnOZgela6kO6sdM7w2/GZTTThDL/CF/a+BP7YuOyid
QQEM5HLcSqbF1fiIStKkP2bTBZq0C8aSioy69Tg0nWES3vou7WxkXESUyP5ZEjp7
k9WQZj8JHZvJsJlfzRfJWSswkoR5Rj6OjsEjbbgFmoIHjndQ+ivhlKoJH+/6oGIM
txTwx8LzlucuKeYNr9bFQkV74ws8VGQBQPi1JBqvhXQEzW7qlbGg6ri3ADMrfq0=
=jx9W
-----END PGP SIGNATURE-----

-- 
|_|O|_| Registered Linux user #585947
|_|_|O| Github: https://github.com/dpurgert
|O|O|O| PGP: 05CA 9A50 3F2E 1335 4DC5  4AEE 8E11 DDF3 1279 A281

[toc] | [prev] | [next] | [standalone]

#173081

Am 01.10.2016 um 14:20 schrieb Dan Purgert:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> mo wrote:
>> As the title say i'm in search for a backup application/system.
>> Currently i manage my backups with a little script that i wrote... but
>> it does not really serve my needs anymore.
>> I want to be able to make backups on my main PC and also on my server,
>> the backups i would then store on my NAS.
>
> There's always rsync from the hosts to the NAS box.

rsync is quite handy for that, indeed ;)
(I actually do that atm - i transfer the data over via rsync to my NAS)

> Only thing that I see "wrong" with your script is that:
>  - you're deleting the backups (what happens when you delete something,
>    and don't catch it til after the next backup run?)

I forgot to mention that i have a script which transfers the current 
backup over if the given backup is not yet saved, sorry, i kind left 
that out. :P

>  - you're backing up everything, even if it hasn't changed.

That's indeed a problem atm... That's mainly why i'm on the search for a 
more professional approach so to speak. Honestly, i have not yet planned 
a specific strategy...

>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1
>
> iQEcBAEBAgAGBQJX76m+AAoJEI4R3fMSeaKBwWQH/RiG+HH83WcyGT5wOhGM344j
> UTOXN+VS7UC0jOsZG+MYrNJkAE8U9qwAYahi/63X+TFmSa2rFRmw6rv0pfvfBeiP
> mItlVsjqyeR00tFc+zmAPWKnOZgela6kO6sdM7w2/GZTTThDL/CF/a+BP7YuOyid
> QQEM5HLcSqbF1fiIStKkP2bTBZq0C8aSioy69Tg0nWES3vou7WxkXESUyP5ZEjp7
> k9WQZj8JHZvJsJlfzRfJWSswkoR5Rj6OjsEjbbgFmoIHjndQ+ivhlKoJH+/6oGIM
> txTwx8LzlucuKeYNr9bFQkV74ws8VGQBQPi1JBqvhXQEzW7qlbGg6ri3ADMrfq0=
> =jx9W
> -----END PGP SIGNATURE-----
>

Thanks Dan ;)

Greets

mo

[toc] | [prev] | [next] | [standalone]

#173080

Maybe this is a little OT, but what kind of backup strategy would you 
guys recommend? (Any advice Gene? :) )

[toc] | [prev] | [next] | [standalone]

#173085

[Multipart message — attachments visible in raw view] — view raw

Like I said backuppc uses incremental and full backups.  The web interface lets
you browse any backup (inc or full) and you see all the files backed up.  I set
the incremental for each day up to a week.  So I have up to 7 of them.  The full
can kept for for however long you want.  I currently keep 12 weekly, 8 bi-weekly
and 4 monthly full backups so that covers almost a year.

There is another solution you might like called rsnapshot.  I use it to backup
just my root directory on my desktop before I do updates.  That way if something
goes wrong I can boot into a rescue cd and restore the system to the state
before the update.  I just can't afford to have my desktop to break.  rsnapshot
uses rsync so it can backup any computer that has rsync.  It uses hard links so
duplicate files are only stored once.  You specify how many backups you want to
keep and rsnapshot deletes older ones over that max before adding the new one. 
That way you always have backups (assuming you set the count greater that 1)
that will be there even if there is a transfer error.  This is similar to your
script but is very versatile.


*...Bob*
On 10/01/2016 04:42 PM, mo wrote:
> Maybe this is a little OT, but what kind of backup strategy would you guys
> recommend? (Any advice Gene? :) )
>
>

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | linux.debian.user

csiph-web