Re: Live Fille System Backup

From	<tomas@tuxteam.de>
Newsgroups	linux.debian.user
Subject	Re: Live Fille System Backup
Date	2017-05-06 21:50 +0200
Message-ID	<tEdSh-3MB-17@gated-at.bofh.it> (permalink)
References	(4 earlier) <tDSb8-6HT-11@gated-at.bofh.it> <tE0V3-44Y-1@gated-at.bofh.it> <tE5rH-6Vw-9@gated-at.bofh.it> <tE8pA-kL-37@gated-at.bofh.it> <tEbx7-2qh-3@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sat, May 06, 2017 at 02:18:47PM -0300, Henrique de Moraes Holschuh wrote:
> On Sat, 06 May 2017, tomas@tuxteam.de wrote:
> > Hm. That just means that the file will change after being copied.
> 
> There can also exist busy file *sets*, which need to be atomically
> snapshotted (this is, of course, application specific).

Yes, this is what I meant by "skew": the point in time files are
copied varies. Even worse, some applications might have essential
parts of their state in RAM, not committed to file at all.

That's why I mentioned snapshotting (LVM, or overlayfs, or a file
system which can do the trick, as zfs or btrfs). If you go the whole
way of zfs or btrfs, those file systems can do better than rsync
anyway.

For a database worth its salt there are other possibilities (PostgreSQL
can save a consistent state on-disk) -- after all they're in the
transaction business.

> "busy", as far as backups go, means:
> 
> 1. a file changes [either in memory/buffer cache, or in the backend
>    storage] *while* being copied (so it is internally inconsistent) --
>    backup will have sections of the old file, and sections of the new
>    file.

Rsync will notice and warn on that. Of course, at the next attempt
that can happen again :)

> 2. one or more components of a set of several files change *during*
>    backup (files may be internally consistent, but the set of files
>    itself isn't).

Only the application can know that. But this is a realm where snapshots
won't help either, without the application's help.

> In some cases, (1) and (2) can manifest as files missing from the
> backup, or being zero-sized.

If I understood you correctly (2) won't produce empty or missing files,
just inconsistent application state.

cheers
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlkOJ/sACgkQBcgs9XrR2kZ9DQCfX6V5a9UdO2pixunhFMyuM0ug
GFoAn1R/RLYL2KeRIkfBsL7BqrJJb1/V
=nRRb
-----END PGP SIGNATURE-----

Thread

Live Fille System Backup Sergei G <sergeig.public@gmail.com> - 2017-05-05 01:20 +0200
  Re: Live Fille System Backup Anders Andersson <pipatron@gmail.com> - 2017-05-05 03:50 +0200
    Re: Live Fille System Backup Sergei G <sergeig.public@gmail.com> - 2017-05-05 06:50 +0200
      Re: Live Fille System Backup Dejan Jocic <jodejka@gmail.com> - 2017-05-05 09:00 +0200
        Re: Live Fille System Backup Sergei G <sergeig.public@gmail.com> - 2017-05-05 22:40 +0200
          Re: Live Fille System Backup Sergei G <sergeig.public@gmail.com> - 2017-05-05 23:20 +0200
            Re: Live Fille System Backup Henrique de Moraes Holschuh <hmh@debian.org> - 2017-05-06 00:00 +0200
              Re: Live Fille System Backup conover@rahul.net (John Conover) - 2017-05-06 01:30 +0200
                Re: Live Fille System Backup Henrique de Moraes Holschuh <hmh@debian.org> - 2017-05-06 02:30 +0200
          Re: Live Fille System Backup Henrique de Moraes Holschuh <hmh@debian.org> - 2017-05-06 00:00 +0200
          Re: Live Fille System Backup <tomas@tuxteam.de> - 2017-05-06 08:00 +0200
            Re: Live Fille System Backup Dan Ritter <dsr@randomstring.org> - 2017-05-06 12:50 +0200
              Re: Live Fille System Backup tomas@tuxteam.de - 2017-05-06 16:00 +0200
                Re: Live Fille System Backup Henrique de Moraes Holschuh <hmh@debian.org> - 2017-05-06 19:20 +0200
                Re: Live Fille System Backup <tomas@tuxteam.de> - 2017-05-06 21:50 +0200
                Re: Live Fille System Backup Henrique de Moraes Holschuh <hmh@debian.org> - 2017-05-06 23:10 +0200
                Re: Live Fille System Backup Ben Caradoc-Davies <ben@transient.nz> - 2017-05-07 01:00 +0200
                Re: Live Fille System Backup <tomas@tuxteam.de> - 2017-05-07 12:00 +0200
                Re: Live Fille System Backup rhkramer@gmail.com - 2017-05-07 14:50 +0200
                Re: Live Fille System Backup <tomas@tuxteam.de> - 2017-05-07 17:00 +0200
                Re: Live Fille System Backup Sergei G <sergeig.public@gmail.com> - 2017-05-09 00:50 +0200
                Re: Live Fille System Backup Sergei G <sergeig.public@gmail.com> - 2017-05-09 01:10 +0200
                Re: Live Fille System Backup tomas@tuxteam.de - 2017-05-09 10:00 +0200
                Re: Live Fille System Backup Greg Wooledge <wooledg@eeg.ccf.org> - 2017-05-09 14:10 +0200
                Re: Live Fille System Backup <tomas@tuxteam.de> - 2017-05-09 14:30 +0200
                Re: Live Fille System Backup Curt <curty@free.fr> - 2017-05-09 15:20 +0200
                Re: Live Fille System Backup Curt <curty@free.fr> - 2017-05-09 14:10 +0200
                Re: Live Fille System Backup <tomas@tuxteam.de> - 2017-05-09 14:30 +0200
  Re: Live Fille System Backup <tomas@tuxteam.de> - 2017-05-05 09:20 +0200
  Re: Live Fille System Backup Bob Weber <bobrweber@gmail.com> - 2017-05-06 01:00 +0200
  Re: Live Fille System Backup Stefan Monnier <monnier@iro.umontreal.ca> - 2017-05-12 15:00 +0200
    Re: Live Fille System Backup Sergei G <sergeig.public@gmail.com> - 2017-05-25 05:40 +0200

csiph-web