Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #177834 > unrolled thread

User-oriented backup tools

Back to article view | Back to linux.debian.user

Contents

  User-oriented backup tools Francesco Porro <frapox@gmail.com> - 2017-02-15 23:40 +0100
    Re: User-oriented backup tools Greg Wooledge <wooledg@eeg.ccf.org> - 2017-02-15 23:50 +0100
      Re: User-oriented backup tools Francesco Porro <frapox@gmail.com> - 2017-02-16 11:50 +0100
        Re: User-oriented backup tools Glenn English <ghe2001@gmail.com> - 2017-02-16 18:30 +0100
          Re: User-oriented backup tools Francesco Porro <frapox@gmail.com> - 2017-02-16 19:10 +0100
            Re: User-oriented backup tools Glenn English <ghe2001@gmail.com> - 2017-02-16 21:30 +0100
              Re: User-oriented backup tools Francesco Porro <frapox@gmail.com> - 2017-02-17 00:50 +0100
    Re: User-oriented backup tools Gene Heskett <gheskett@shentel.net> - 2017-02-16 01:10 +0100
      Re: User-oriented backup tools "Martin McCormick" <martin.m@suddenlink.net> - 2017-02-16 03:50 +0100
        Re: User-oriented backup tools Francesco Porro <frapox@gmail.com> - 2017-02-16 11:50 +0100
          Re: User-oriented backup tools Jonathan Dowland <jmtd@debian.org> - 2017-02-16 22:50 +0100
    Re: User-oriented backup tools solitone <solitone@mail.com> - 2017-02-16 06:50 +0100
      Re: User-oriented backup tools Francesco Porro <frapox@gmail.com> - 2017-02-16 11:50 +0100
    Re: User-oriented backup tools <tomas@tuxteam.de> - 2017-02-16 09:20 +0100
      Re: User-oriented backup tools Francesco Porro <frapox@gmail.com> - 2017-02-16 11:50 +0100
        Re: User-oriented backup tools <tomas@tuxteam.de> - 2017-02-16 12:10 +0100
          Re: User-oriented backup tools Francesco Porro <frapox@gmail.com> - 2017-02-16 14:30 +0100
            Re: User-oriented backup tools <tomas@tuxteam.de> - 2017-02-16 15:00 +0100
    Re: User-oriented backup tools Francesco Porro <frapox@gmail.com> - 2017-02-16 12:00 +0100
    Re: User-oriented backup tools Daniel Bareiro <daniel-listas@gmx.net> - 2017-02-16 12:20 +0100
      Re: User-oriented backup tools Hans <hans.ullrich@loop.de> - 2017-02-16 12:30 +0100
        Re: User-oriented backup tools Francesco Porro <frapox@gmail.com> - 2017-02-16 14:30 +0100
          Re: User-oriented backup tools Hans <hans.ullrich@loop.de> - 2017-02-16 15:00 +0100
            Re: User-oriented backup tools Francesco Porro <frapox@gmail.com> - 2017-02-16 17:40 +0100
              Re: User-oriented backup tools Glenn Holmer <cenbe@kolabnow.com> - 2017-02-16 17:40 +0100
                Re: User-oriented backup tools Chen Wei <weichen302@icloud.com> - 2017-02-17 09:40 +0100
      Re: User-oriented backup tools Francesco Porro <frapox@gmail.com> - 2017-02-16 14:50 +0100
    Re: User-oriented backup tools Boyan Penkov <boyan.penkov@gmail.com> - 2017-02-16 13:20 +0100
      Re: User-oriented backup tools Francesco Porro <frapox@gmail.com> - 2017-02-16 14:40 +0100
        Re: User-oriented backup tools Boyan Penkov <boyan.penkov@gmail.com> - 2017-02-16 14:50 +0100
          Re: User-oriented backup tools Francesco Porro <frapox@gmail.com> - 2017-02-16 15:10 +0100
            Re: User-oriented backup tools Francesco Porro <frapox@gmail.com> - 2017-02-16 17:30 +0100
              Re: User-oriented backup tools Francesco Porro <frapox@gmail.com> - 2017-02-16 19:00 +0100
            Re: User-oriented backup tools Erwan David <erwan@rail.eu.org> - 2017-02-16 17:40 +0100
              Re: User-oriented backup tools Francesco Porro <frapox@gmail.com> - 2017-02-16 19:10 +0100

Page 1 of 2  [1] 2  Next page →

#177834 — User-oriented backup tools

[Multipart message — attachments visible in raw view] — view raw

Hi guys, It's the first time I'm posting here. I'm both a Fedora and
Debian user.

Which backup tool do you use?
At this time I feel comfortable with rsync, which I use to sync my home
to an external Usb drive. No automation, no scheduling for now. I just
launch rsync from the command line and let it backup all the stuff when
I need it.

Now I'd like to move to more powerful utility. It should be reliable and
very user-friendly. A friend on Irc suggested me borg [1], which also
feature decuplication, encryption and seems easy enough to use.

Otherwise I discovered a scripting hot-to to make some kind of snapshots
via rsync (and hardlinking) [2], but I haven't tryed it out yet.

So... What's your opinion about user-oriented backup tools? ^^

Thanks in advance.

[1] https://borgbackup.readthedocs.io/en/stable/
[2] http://www.mikerubel.org/computers/rsync_snapshots/

-- 
fp
pgp: 0x45399C26

[toc] | [next] | [standalone]

#177836

On Wed, Feb 15, 2017 at 11:37:40PM +0100, Francesco Porro wrote:
> At this time I feel comfortable with rsync, which I use to sync my home
> to an external Usb drive. No automation, no scheduling for now. I just
> launch rsync from the command line and let it backup all the stuff when
> I need it.

This is what I do, too.  Except I also rsync changes from my VPS, and
also make a second copy of parts of the data and burn that to a DVD+RW.

> Now I'd like to move to more powerful utility.

Why?  What do you want to do that your existing solution doesn't do?
Just swapping out a working solution for something more complex without
some compelling reason is not a good idea.

[toc] | [prev] | [next] | [standalone]

#177856

Hi Greg!

On 15/02/2017 23:42, Greg Wooledge wrote:
> Why?  What do you want to do that your existing solution doesn't do?
> Just swapping out a working solution for something more complex without
> some compelling reason is not a good idea.

Two reason:
1) In my home environment I'd like to test something else, maybe just to
look for something new and more powerful, or maybe just for curiosity. :)
2) rsync itself doesn't support multiple level of backups (as it happens
for incremental o differential backup tools) without additional
scripting. This is a feature that i'm missing quite a bit.

That's it.

When I was on Mac OS X, several years ago, I used to use CCC (carbon
copy cloner) which I liked a lot, and never found any similar on PC
(Win/Linux).

-- 
fp
pgp: 0x45399C26

[toc] | [prev] | [next] | [standalone]

#177887

[Multipart message — attachments visible in raw view] — view raw

On Thu, Feb 16, 2017 at 3:42 AM, Francesco Porro <frapox@gmail.com> wrote:

>
> When I was on Mac OS X, several years ago, I used to use CCC (carbon
> copy cloner) which I liked a lot, and never found any similar on PC
> (Win/Linux).
>

That would be dd (CLI). I believe that's what's under the CCC GUI on OS-X.

-- 
Glenn English

[toc] | [prev] | [next] | [standalone]

#177890

On 16/02/2017 18:27, Glenn English wrote:
> 
> 
> On Thu, Feb 16, 2017 at 3:42 AM, Francesco Porro <frapox@gmail.com
> <mailto:frapox@gmail.com>> wrote:
> 
> 
>     When I was on Mac OS X, several years ago, I used to use CCC (carbon
>     copy cloner) which I liked a lot, and never found any similar on PC
>     (Win/Linux).
> 
> 
> That would be dd (CLI). I believe that's what's under the CCC GUI on OS-X.
> 
> -- 
> Glenn English

Are you sure? Actually I thought it was (and is) based on rsync with a
solid and very flexible GUI. I don't think it's based on dd.

-- 
fp
pgp: 0x45399C26

[toc] | [prev] | [next] | [standalone]

#177892

[Multipart message — attachments visible in raw view] — view raw

On Thu, Feb 16, 2017 at 11:06 AM, Francesco Porro <frapox@gmail.com> wrote:

>
> Are you sure?


No. I've never used it, but a friend of mine does. And it takes forever for
him to do a backup. So I assumed it was dd.  Maybe it's that other OS X
disk clone 'backup.'

But when he's done, he doesn't have an incremental backup, just a clone.
Better than nothing, but not wonderful for programmers and lawyers who have
to retrieve last week's version.

Amanda, IMHO, is the proper backup. But 'simple', 'user friendly', and
'amanda' should never appear in the same paragraph. There's significant
futzing required to get it going. At least mine did, back in the DLT days...

-- 
Glenn English

[toc] | [prev] | [next] | [standalone]

#177899

On 16/02/2017 21:26, Glenn English wrote:
> No. I've never used it, but a friend of mine does. And it takes forever
> for him to do a backup. So I assumed it was dd.  Maybe it's that other
> OS X disk clone 'backup.'
> 
> But when he's done, he doesn't have an incremental backup, just a clone.
> Better than nothing, but not wonderful for programmers and lawyers who
> have to retrieve last week's version.

CCC has either cloning feature and backup feature. It simply let you
choose during the initial steps. I've always used it in the backup
"mode", but sincerely i don't think it relies on dd even for cloning.

dd is for low-level, byte-to-byte, copying, dumping.


> Amanda, IMHO, is the proper backup. But 'simple', 'user friendly', and
> 'amanda' should never appear in the same paragraph. There's significant
> futzing required to get it going. At least mine did, back in the DLT days...

I never saw those DLT days... :)

-- 
fp
pgp: 0x45399C26

[toc] | [prev] | [next] | [standalone]

#177839

On Wednesday 15 February 2017 17:37:40 Francesco Porro wrote:

> Hi guys, It's the first time I'm posting here. I'm both a Fedora and
> Debian user.
>
> Which backup tool do you use?
> At this time I feel comfortable with rsync, which I use to sync my
> home to an external Usb drive. No automation, no scheduling for now. I
> just launch rsync from the command line and let it backup all the
> stuff when I need it.
>
> Now I'd like to move to more powerful utility. It should be reliable
> and very user-friendly. A friend on Irc suggested me borg [1], which
> also feature decuplication, encryption and seems easy enough to use.
>
> Otherwise I discovered a scripting hot-to to make some kind of
> snapshots via rsync (and hardlinking) [2], but I haven't tryed it out
> yet.
>
> So... What's your opinion about user-oriented backup tools? ^^
>
Very simple: amanda

> Thanks in advance.
>
> [1] https://borgbackup.readthedocs.io/en/stable/
> [2] http://www.mikerubel.org/computers/rsync_snapshots/


Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#177842

	I use rsnapshot. I have 4 128 GB thumb drives that are
combined in to 1 large drive with the idea of backing up the
system I do most tinkering on on a daily basis for a year and
then I swap out the drives with other drives so that the year
just completed gets stored and the new drives start a new year.
At the end of the new year, I will store those drives, erase last
year's backups and re-use those older drives for the upcoming new year.

	Your mileage will vary if you must backup mysql or other
data bases so you need to make sure the backup tool you want to
use safely does those backups.

Martin

[toc] | [prev] | [next] | [standalone]

#177857

On 16/02/2017 03:39, Martin McCormick wrote:
> 	I use rsnapshot. 

Very nice one! This has what I was looking for, indeed there's a link in
the main page to the article that I posted before. I'll give it a try,
for sure!

Thank you Martin!

-- 
fp
pgp: 0x45399C26

[toc] | [prev] | [next] | [standalone]

#177896

[Multipart message — attachments visible in raw view] — view raw

On Thu, Feb 16, 2017 at 11:43:27AM +0100, Francesco Porro wrote:
> On 16/02/2017 03:39, Martin McCormick wrote:
> > 	I use rsnapshot. 
> 
> Very nice one! This has what I was looking for, indeed there's a link in
> the main page to the article that I posted before. I'll give it a try,
> for sure!

I used to use rsnapshot, which is not much of a conceptual step up from
rsync, and thus is an easy tool to use, but the architectural decision to
keep hard link trees for each increment caused me a lot of problems for
large mailboxes (lots and lots of small files; high ratio of filesystem
metadata to actual data). Deleting old increments was very computationally
expensive.

So instead I use rdiff-backup. It's a very similar tool but it uses a more
clever scheme for storing increments. The most recent increment is available
as simple files, and most restores are of the last backup. The scheme it uses
for increments is quite simple, well documented and third party tools exist
to manipulate it, which gives me confidence.

-- 
Jonathan Dowland
Please do not CC me, I am subscribed to the list.

[toc] | [prev] | [next] | [standalone]

#177844

On Wednesday, 15 February 2017 23:37:40 CET Francesco Porro wrote:
> Which backup tool do you use?

I'm happy with backup2l [1]. It's very simple to configure, supports 
incremental backups, and it's fully automated. Here you can find a short guide 
[2]. 

I also like it because it is simple to extract specific files you may need from 
a backup. For instance I had to recover just my old thunderbolt emails, and it 
was quick and easy with backup2l.

[1] http://backup2l.sourceforge.net
[2] http://www.debianhelp.co.uk/backup2l.htm

[toc] | [prev] | [next] | [standalone]

#177855

On 16/02/2017 06:44, solitone wrote:
> I'm happy with backup2l [1]. It's very simple to configure, supports 
> incremental backups, and it's fully automated.

A bit of configuration is required but it seems ok and easy enough!

> I also like it because it is simple to extract specific files you may need from 
> a backup.

Yes, extracting specific files from a backup should also be a
non-trouble task for me and users in general. That's why I've used rsync
until now (all files and dirs are saved as they originally are, in the
destination folder -- very easy to locate something, just navigating
through the filesystem :)).

Thanks.

-- 
fp
pgp: 0x45399C26

[toc] | [prev] | [next] | [standalone]

#177845

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, Feb 15, 2017 at 11:37:40PM +0100, Francesco Porro wrote:
> Hi guys, It's the first time I'm posting here. I'm both a Fedora and
> Debian user.
> 
> Which backup tool do you use?
> At this time I feel comfortable with rsync [...]

rsync.

To a LUKS-encrypted USB drive (my laptop's drive is LUKS, so it seemed
to be a majot weak point to have all on an unencrpted thumb I could lose
anytime ;-)

The command is wrapped up in some 8 lines of bash making sure home exists
and does basically:

  rsync -av --delete --filter="merge $home/.backup/filter" $home/ $backup/

deferring the filter to file ~/.backup/filter, which says:

  - /.cache/
  - /.config/
  [...other stuff to exclude...]
  dir-merge .backup-filter

The nice tidbit is the "dir-merge": this way, I can drop a filter in each
directory I don't want to back up completely, excluding all or part of
its files/subdirs (e.g. the build subdir of some biggish thing or a
typical junk download dir).

I'm so happy with that set up that I don't even think of changing that.

*Should* I some time be very bored, my next step would be to have a
live/install Debian partition on the stick (which I would somehow keep
up to date) to be able to do "bare metal" recovery, should my main
machine go down in flames.

KISS (although: rsync is smart, but simple? ;-)

regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlilX7UACgkQBcgs9XrR2kZ87QCeN02KSr18o/U8W5CB8BeagWJT
bFsAn3hydJq9QH7rQlsVB+5PvEKjPn1m
=Hlvh
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#177854

On 16/02/2017 09:15, tomas@tuxteam.de wrote:
> rsync.
>
> To a LUKS-encrypted USB drive (my laptop's drive is LUKS, so it seemed
> to be a majot weak point to have all on an unencrpted thumb I could lose
> anytime ;-)
> [cut]

that's very similar to my current backup-style, I use rsync with a lot
of options as for eg "--exclude-from=exclude.txt --> this file contains
all the folder I do not want to copy to the destination dir.

the drawback of all this is the lack of a snapshot/multi-level backup, a
feature that would be very useful to me.

Anyway, I didn't know the .backup-filter stuff and it seems a very smart
and simple tool. Thank you Tomás for sharing the whole procedure!

> regards
> -- tomás

regards

-- 
fp
pgp: 0x45399C26

[toc] | [prev] | [next] | [standalone]

#177859

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, Feb 16, 2017 at 11:44:32AM +0100, Francesco Porro wrote:
> 
> 
> On 16/02/2017 09:15, tomas@tuxteam.de wrote:
> > rsync.
> >
> > To a LUKS-encrypted USB drive (my laptop's drive is LUKS, so it seemed
> > to be a majot weak point to have all on an unencrpted thumb I could lose
> > anytime ;-)
> > [cut]
> 
> that's very similar to my current backup-style, I use rsync with a lot
> of options as for eg "--exclude-from=exclude.txt --> this file contains
> all the folder I do not want to copy to the destination dir.
> 
> the drawback of all this is the lack of a snapshot/multi-level backup, a
> feature that would be very useful to me.

That depends on what you mean by "snapshot" and "multi-level". If what you
want is full backups without repeating all the stuff, check out the
- --compare-dest and --link-dest options With this, you can make a backup
and the files which didn't change are just hard linked to their old version.

Since it's all hard links, you can "garbage collect" by just rm-ing old
stuff.

I had such a contraption running at a customer's (many moons ago) which made
hourly backups and kept two week's worth of backups: just at top level there
was a directory structure indexed by date/time and below a whole tree of
their data. I exported it (read only!) via Samba, and from there on, they
"Just Knew" what they had to do when they inadvertently busted That Important
File.

Now if you are talking about "Real Snapshots" (i.e. files don't change
during backup and stuff), then you've got to look into either LVM
(they say performance isn't the best) or into special file systems
(btrfs, zfs) which are really magic in this respect.

With rsync you'll always have some skew (i.e. the world is changing
while the backup is running). Don't back up a database's binary image:
always back up its dumps, or better: use the database facilities
for that.

> Anyway, I didn't know the .backup-filter stuff and it seems a very smart
> and simple tool. Thank you Tomás for sharing the whole procedure!

Yeah: the hardest (but most beautiful) part about rsync is reading its
man page. I still discover things, after years of use.

regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlilh1sACgkQBcgs9XrR2kaPIACcC1/n49YZFsYIeQWyk4YtNpS3
RNgAn107ILUEZZ5n1ML2gdgitHI11rFd
=zGSr
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#177869

On 16/02/2017 12:04, tomas@tuxteam.de wrote:
> Now if you are talking about "Real Snapshots" (i.e. files don't change
> during backup and stuff)

> With rsync you'll always have some skew (i.e. the world is changing
> while the backup is running).

Well, with the word "snapshot" I meant a "dump" of all my personal data
by daily basis. Currently I don't need to backup any additional db,
except for Thunderbird's and Firefox's ones. It should be ok to backup
those with no hassles via rsync or it can lead to any problems? in other
words: can rsync make a good backup of all my '~', including databases
of application like FF and TB (also if they're running)?

> I had such a contraption running at a customer's (many moons ago) which made
> hourly backups and kept two week's worth of backups: just at top level there
> was a directory structure indexed by date/time and below a whole tree of
> their data. I exported it (read only!) via Samba, and from there on, they
> "Just Knew" what they had to do when they inadvertently busted That Important
> File.

Yes, this is a useful method I can think about, and it is almost the
same thing described in mike rubel's article linked in my first message.
It should be ok, maybe with some scripting and scheduling via cron.

Ps. is it ok to pgp sign messages on a mailinglist? I'm asking just
because I did so in the first post, but don't know if it's ok or
compliant with m/l policy.

-- 
fp
pgp: 0x45399C26

[toc] | [prev] | [next] | [standalone]

#177875

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, Feb 16, 2017 at 02:24:11PM +0100, Francesco Porro wrote:
> On 16/02/2017 12:04, tomas@tuxteam.de wrote:
> > Now if you are talking about "Real Snapshots" (i.e. files don't change
> > during backup and stuff)
> 
> > With rsync you'll always have some skew (i.e. the world is changing
> > while the backup is running).
> 
> Well, with the word "snapshot" I meant a "dump" of all my personal data
> by daily basis. Currently I don't need to backup any additional db,
> except for Thunderbird's and Firefox's ones. It should be ok to backup
> those with no hassles via rsync or it can lead to any problems? in other
> words: can rsync make a good backup of all my '~', including databases
> of application like FF and TB (also if they're running)?

It depends :-)

I wouldn't count on FF (and perhaps TB) to get their state "rescued" if
you backup while running them: apart from there being tons of files
in there, there are "database-like" files, like several Berkeley DB and
several sqlite. While the biggest bulk is cache (and thus kind-of
recoverable [1]), the whole mess might crawl along while backup is
going through it.

Depends on how much value you attach to it. If losing it on a blue moon
is an option, don't worry: it will work "mostly". If not, you'd have to
complement your rsync thing with taking dumps of the important things
(e.g. bookmarks, whatever you care about) which are quiescent at rsync
time (akin to taking a dump of an SQL database which gets backed up).

For an SQL database it's a good thing to do anyway, since you get a
backup which is hopefully portable across DBMS software versions.

> > I had such a contraption running at a customer's (many moons ago) [...]

> Yes, this is a useful method I can think about, and it is almost the
> same thing described in mike rubel's article linked in my first message.
> It should be ok, maybe with some scripting and scheduling via cron.

Customer was happy :-)

> Ps. is it ok to pgp sign messages on a mailinglist? I'm asking just
> because I did so in the first post, but don't know if it's ok or
> compliant with m/l policy.

I've been signing mine all the time and nobody scolded me for that
(for other things, though... ;-)

regards
- -- t
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlilrtIACgkQBcgs9XrR2kaJCQCfbIRBbl75tnUo+qGWuCPoghpF
owEAnilGOQYWQnOBcAATpEAuYWKbufeE
=Lniw
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#177858

Sorry guys, I'm new to mailings list so I made a little mess sending
replies first to you mailbox and than to the correct mailinglist
address, resulting in two copies...

-- 
fp
pgp: 0x45399C26

[toc] | [prev] | [next] | [standalone]

#177860

[Multipart message — attachments visible in raw view] — view raw

Hi, Francesco

On 15/02/17 19:37, Francesco Porro wrote:

> Hi guys, It's the first time I'm posting here. I'm both a Fedora and
> Debian user.
> 
> Which backup tool do you use?
> At this time I feel comfortable with rsync, which I use to sync my home
> to an external Usb drive. No automation, no scheduling for now. I just
> launch rsync from the command line and let it backup all the stuff when
> I need it.
> 
> Now I'd like to move to more powerful utility. It should be reliable and
> very user-friendly. A friend on Irc suggested me borg [1], which also
> feature decuplication, encryption and seems easy enough to use.
> 
> Otherwise I discovered a scripting hot-to to make some kind of snapshots
> via rsync (and hardlinking) [2], but I haven't tryed it out yet.
> 
> So... What's your opinion about user-oriented backup tools? ^^

Another alternative could be to use Dirvish.


Kind regards,
Daniel

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | linux.debian.user

csiph-web