Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.debian.user > #225505 > unrolled thread
| Started by | Patrick Bartek <patrick.bartek@gmail.com> |
|---|---|
| First post | 2020-07-29 22:00 +0200 |
| Last post | 2020-08-04 06:30 +0200 |
| Articles | 20 on this page of 114 — 18 participants |
Back to article view | Back to linux.debian.user
Homebuilt NAS Advice Patrick Bartek <patrick.bartek@gmail.com> - 2020-07-29 22:00 +0200
Re: Homebuilt NAS Advice Dan Ritter <dsr@randomstring.org> - 2020-07-29 22:20 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-07-29 22:30 +0200
Re: Homebuilt NAS Advice Dan Ritter <dsr@randomstring.org> - 2020-07-29 23:40 +0200
Re: Homebuilt NAS Advice Patrick Bartek <patrick.bartek@gmail.com> - 2020-07-30 02:50 +0200
Re: Homebuilt NAS Advice Tom Dial <tddial@comcast.net> - 2020-07-30 22:50 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-02 01:40 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-02 03:10 +0200
Re: Homebuilt NAS Advice Keith Bainbridge <ke1th3216@gmail.com> - 2020-08-02 04:50 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-02 06:10 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-02 21:30 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-02 22:10 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-03 06:10 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-03 17:30 +0200
Re: Homebuilt NAS Advice Dan Ritter <dsr@randomstring.org> - 2020-08-03 17:40 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-03 18:40 +0200
Re: Homebuilt NAS Advice <tomas@tuxteam.de> - 2020-08-03 19:10 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-03 22:00 +0200
Re: Homebuilt NAS Advice <tomas@tuxteam.de> - 2020-08-03 22:20 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-03 23:50 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-04 00:50 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-04 00:40 +0200
Re: Homebuilt NAS Advice Anssi Saari <as@sci.fi> - 2020-07-30 10:00 +0200
Re: Homebuilt NAS Advice rhkramer@gmail.com - 2020-07-30 13:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-07-30 01:50 +0200
Re: Homebuilt NAS Advice Patrick Bartek <nemommxiv@gmail.com> - 2020-07-30 23:40 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-08 01:30 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-08 03:30 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-09 14:40 +0200
Re: Homebuilt NAS Advice Peter Ehlert <peter@sdi-baja.com> - 2020-07-30 19:00 +0200
Re: Homebuilt NAS Advice Charles Curley <charlescurley@charlescurley.com> - 2020-07-30 21:50 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-07-31 13:10 +0200
Re: Homebuilt NAS Advice Patrick Bartek <nemommxiv@gmail.com> - 2020-07-30 23:30 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-02 01:50 +0200
Re: Homebuilt NAS Advice Patrick Bartek <nemommxiv@gmail.com> - 2020-08-02 17:50 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-03 17:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-04 00:10 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-04 01:20 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-04 10:40 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-04 11:30 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-04 16:00 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-04 16:30 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-05 05:50 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-05 16:20 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-05 17:00 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-05 17:40 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 12:10 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-06 16:10 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 17:10 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-06 18:10 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-07 18:30 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-07 19:40 +0200
Re: Homebuilt NAS Advice rhkramer@gmail.com - 2020-08-08 00:10 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-09 04:30 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-09 11:10 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-09 17:00 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-09 18:30 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-09 19:20 +0200
Re: Homebuilt NAS Advice rhkramer@gmail.com - 2020-08-09 23:50 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-10 00:00 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-10 00:30 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-10 00:50 +0200
Re: Homebuilt NAS Advice rhkramer@gmail.com - 2020-08-10 00:30 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-10 00:30 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-09 18:40 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-09 23:00 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-06 18:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 05:30 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-07 18:30 +0200
Re: Homebuilt NAS Advice rhkramer@gmail.com - 2020-08-06 17:20 +0200
Re: Homebuilt NAS Advice Celejar <celejar@gmail.com> - 2020-08-06 18:20 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-06 18:20 +0200
Re: Homebuilt NAS Advice Andrei POPESCU <andreimpopescu@gmail.com> - 2020-08-04 17:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-05 06:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-05 04:30 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-05 09:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-05 11:00 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-05 21:00 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 03:50 +0200
Re: Homebuilt NAS Advice Kushal Kumaran <kushal@locationd.net> - 2020-08-06 06:20 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-06 18:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 04:10 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-07 05:50 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-05 12:30 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-05 12:30 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-05 21:10 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-05 23:30 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-06 00:30 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 01:20 +0200
Re: Homebuilt NAS Advice Jonathan Dowland <jon+debian-user@dow.land> - 2020-08-08 09:10 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-06 04:30 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 04:50 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-06 06:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 07:10 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-06 07:30 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-06 18:10 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-06 18:30 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 03:00 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-07 03:20 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 05:00 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-07 05:40 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 08:00 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-07 08:50 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-07 09:10 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-07 17:50 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-08 01:40 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-08 04:50 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-08 22:30 +0200
Re: Homebuilt NAS Advice Stefan Monnier <monnier@iro.umontreal.ca> - 2020-08-09 00:20 +0200
Re: Homebuilt NAS Advice deloptes <deloptes@gmail.com> - 2020-08-09 01:30 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-09 22:40 +0200
Re: Homebuilt NAS Advice David Christensen <dpchrist@holgerdanske.com> - 2020-08-10 00:30 +0200
Re: Homebuilt NAS Advice Leslie Rhorer <lesrhorer@att.net> - 2020-08-10 01:30 +0200
Re: Homebuilt NAS Advice Patrick Bartek <nemommxiv@gmail.com> - 2020-08-04 06:30 +0200
Page 3 of 6 — ← Prev page 1 2 [3] 4 5 6 Next page →
| From | Celejar <celejar@gmail.com> |
|---|---|
| Date | 2020-08-04 16:00 +0200 |
| Message-ID | <AA5DY-Cu-5@gated-at.bofh.it> |
| In reply to | #225817 |
On Tue, 4 Aug 2020 10:26:32 +0100 Jonathan Dowland <jon+debian-user@dow.land> wrote: > On Tue, Aug 04, 2020 at 01:32:24AM -0700, David Christensen wrote: > >jdupes looks interesting, and should work on any file system that > >supports hard links. I expect BorgBackup either calls jdupes or > >implements similar functionality: > > > >https://linuxcommandlibrary.com/man/jdupes.html > > I'm fairly sure Borg stores files by a hash of their contents and > maintains a separate index of names→hashes for each generation. Indeed: "To actually perform the repository-wide deduplication, a hash of each chunk is checked against the chunks cache, which is a hash-table of all chunks that already exist." https://borgbackup.readthedocs.io/en/stable/internals.html Celejar
[toc] | [prev] | [next] | [standalone]
| From | Stefan Monnier <monnier@iro.umontreal.ca> |
|---|---|
| Date | 2020-08-04 16:30 +0200 |
| Message-ID | <AA671-129-13@gated-at.bofh.it> |
| In reply to | #225828 |
> "To actually perform the repository-wide deduplication, a hash of each
> chunk is checked against the chunks cache, which is a hash-table of all
> chunks that already exist."
>
> https://borgbackup.readthedocs.io/en/stable/internals.html
Same approach as used in `bup` (which all come from `rsync`, IIUC).
Stefan
[toc] | [prev] | [next] | [standalone]
| From | Leslie Rhorer <lesrhorer@att.net> |
|---|---|
| Date | 2020-08-05 05:50 +0200 |
| Message-ID | <AAiBb-a8-1@gated-at.bofh.it> |
| In reply to | #225828 |
On 8/4/2020 8:52 AM, Celejar wrote: > On Tue, 4 Aug 2020 10:26:32 +0100 > Jonathan Dowland <jon+debian-user@dow.land> wrote: > >> On Tue, Aug 04, 2020 at 01:32:24AM -0700, David Christensen wrote: >>> jdupes looks interesting, and should work on any file system that >>> supports hard links. I expect BorgBackup either calls jdupes or >>> implements similar functionality: >>> >>> https://linuxcommandlibrary.com/man/jdupes.html >> >> I'm fairly sure Borg stores files by a hash of their contents and >> maintains a separate index of names→hashes for each generation. > > Indeed: > > "To actually perform the repository-wide deduplication, a hash of each > chunk is checked against the chunks cache, which is a hash-table of all > chunks that already exist." > > https://borgbackup.readthedocs.io/en/stable/internals.html I prefer DAR for several reasons. First of all, as I mentioned before, DAR is the only backup solution of which I am aware that can restore not only deleted or corrupted files, but which can also restore deletions. This means DAR can restore any or all files one chooses in a batch, but then if requested can go back and delete files which were deleted at a later time but prior to additional DAR backups. As a simple example, if one backs up an entire system, deletes several files, then does another backup, DAR can restore the system to the exact state of the last backup without losing any files in the archive, yet not restoring the deleted files. This eliminates the need to ever "refresh" the main backup, keeping every file that was ever on the system, yet eliminating the need to delete files from a restore performed at a later time. DAR is also designed to work with removable media. In this respect, it is similar to the venerable TAR utility, but while TAR is not designed to work with random access media, DAR is. In fact, the name DAR is an acronym for Disk ARchive. Packages are avail;able for Windows, MacOS, FreeBSD, and nearly all Linux distros, including Debian. http://dar.linux.free.fr/doc/Features.html http://dar.linux.free.fr/
[toc] | [prev] | [next] | [standalone]
| From | Stefan Monnier <monnier@iro.umontreal.ca> |
|---|---|
| Date | 2020-08-05 16:20 +0200 |
| Message-ID | <AAsqR-6fO-1@gated-at.bofh.it> |
| In reply to | #225866 |
> I prefer DAR for several reasons. First of all, as I mentioned
> before, DAR is the only backup solution of which I am aware that can
> restore not only deleted or corrupted files, but which can also
> restore deletions. This means DAR can restore any or all files one
> chooses in a batch, but then if requested can go back and delete
> files which were deleted at a later time but prior to additional
> DAR backups.
I find it hard to believe that it's a rare feature.
The system I use (`bup`) does support that as well (every backup is
(more or less) a Git commit, so it doesn't distinguish full-backups from
incrementals), but it doesn't bother to mention it probably because it's
very basic.
Stefan
[toc] | [prev] | [next] | [standalone]
| From | Jonathan Dowland <jon+debian-user@dow.land> |
|---|---|
| Date | 2020-08-05 17:00 +0200 |
| Message-ID | <AAt3A-6sU-9@gated-at.bofh.it> |
| In reply to | #225892 |
On Wed, Aug 05, 2020 at 10:11:51AM -0400, Stefan Monnier wrote: >The system I use (`bup`) does support that as well (every backup is >(more or less) a Git commit, so it doesn't distinguish full-backups from >incrementals), but it doesn't bother to mention it probably because it's >very basic. Can bup expire old backups yet (and remove the corresponding objects)? When I looked at it a long time ago, it couldn't, which meant backup repositories could only ever grow in size. -- Please do not CC me, I am subscribed to the list. 👱🏻 Jonathan Dowland ✎ jmtd@debian.org 🔗 https://jmtd.net
[toc] | [prev] | [next] | [standalone]
| From | Stefan Monnier <monnier@iro.umontreal.ca> |
|---|---|
| Date | 2020-08-05 17:40 +0200 |
| Message-ID | <AAtGh-6Vd-1@gated-at.bofh.it> |
| In reply to | #225894 |
> Can bup expire old backups yet (and remove the corresponding objects)?
Yes ;-)
Stefan
[toc] | [prev] | [next] | [standalone]
| From | Leslie Rhorer <lesrhorer@att.net> |
|---|---|
| Date | 2020-08-06 12:10 +0200 |
| Message-ID | <AAL0A-KM-129@gated-at.bofh.it> |
| In reply to | #225892 |
On 8/5/2020 9:11 AM, Stefan Monnier wrote: >> I prefer DAR for several reasons. First of all, as I mentioned >> before, DAR is the only backup solution of which I am aware that can >> restore not only deleted or corrupted files, but which can also >> restore deletions. This means DAR can restore any or all files one >> chooses in a batch, but then if requested can go back and delete >> files which were deleted at a later time but prior to additional >> DAR backups. > > I find it hard to believe that it's a rare feature. > > The system I use (`bup`) does support that as well (every backup is > (more or less) a Git commit, so it doesn't distinguish full-backups from > incrementals), but it doesn't bother to mention it probably because it's > very basic. Uh-uh. Full, incremenal, and decremental backups are all completely different than deletion restoral. A GIT commit is somewhat similar, as is a VM snapshot, but they also are not quite the same. All I can say is, "I suggest you read all the features - it is quite a long list - and try it."
[toc] | [prev] | [next] | [standalone]
| From | Celejar <celejar@gmail.com> |
|---|---|
| Date | 2020-08-06 16:10 +0200 |
| Message-ID | <AAOKJ-342-3@gated-at.bofh.it> |
| In reply to | #225930 |
On Thu, 6 Aug 2020 05:02:17 -0500 Leslie Rhorer <lesrhorer@att.net> wrote: > On 8/5/2020 9:11 AM, Stefan Monnier wrote: > >> I prefer DAR for several reasons. First of all, as I mentioned > >> before, DAR is the only backup solution of which I am aware that can > >> restore not only deleted or corrupted files, but which can also > >> restore deletions. This means DAR can restore any or all files one > >> chooses in a batch, but then if requested can go back and delete > >> files which were deleted at a later time but prior to additional > >> DAR backups. > > > > I find it hard to believe that it's a rare feature. > > > > The system I use (`bup`) does support that as well (every backup is > > (more or less) a Git commit, so it doesn't distinguish full-backups from > > incrementals), but it doesn't bother to mention it probably because it's > > very basic. > > Uh-uh. Full, incremenal, and decremental backups are all completely > different than deletion restoral. A GIT commit is somewhat similar, as > is a VM snapshot, but they also are not quite the same. > > All I can say is, "I suggest you read all the features - it is quite a > long list - and try it." I (and I suspect others here) are still pretty confused about what you are trying to explain to us. Here's what I do, and do not, understand: * Incremental and differential backups are backups of the delta between the last full backup and the current system state (either individually [differential] or collectively [incremental]) * I have no idea (nor do Google, DuckDuckGo, or Wikipedia) what a "decremental backup" is. * Borg doesn't fit the "full / differential / incremental" paradigm neatly, but it certainly has some of the advantages of differential / incremental backups (plus others that classic differential / incremental ones do not have, such as deduplication - i.e., if the backup source contains multiple copies of some data, that dataonly needs to be stored once in the backup target). * I don't know exactly what you mean by "deletion restoral", but Borg (and, I assume, many other good backup solutions) offers a flexible variety of methods to restore deleted files, and / or to restore a snapshot of the backup source as it existed at a given point in time: https://docs.borgbase.com/restore/ If you are willing to explain further, that would be appreciated. Thanks! Celejar
[toc] | [prev] | [next] | [standalone]
| From | Leslie Rhorer <lesrhorer@att.net> |
|---|---|
| Date | 2020-08-06 17:10 +0200 |
| Message-ID | <AAPGN-3D7-9@gated-at.bofh.it> |
| In reply to | #225937 |
On 8/6/2020 9:07 AM, Celejar wrote: > On Thu, 6 Aug 2020 05:02:17 -0500 > Leslie Rhorer <lesrhorer@att.net> wrote: > * Incremental and differential backups are backups of the delta between > the last full backup and the current system state (either individually > [differential] or collectively [incremental]) > > * I have no idea (nor do Google, DuckDuckGo, or Wikipedia) what a > "decremental backup" is. I suggest reading the DAR features list. I will post it again: http://dar.linux.free.fr/doc/Features.html Meanwhile, a decremental backup is defines as: As opposed to incremental backups, where the older one is a full backup and each subsequent backup contains only the changes from the previous backup, decremental backup let the full backup be the more recent while the older ones only contain changes compared to the just more recent one. This has the advantage of providing a single archive to use to restore a whole system in its latest known state, while reducing the overall amount of data to retain older versions of files (same amount required as with differential backup). It has also the advantage to not have to keep several set of backup as you just need to delete the oldest backup when you need storage space. However it has the default to require at each new cycle the creation of a full backup, then the transformation of the previous full backup into a so-called decremental backup. Yes, everything has a cost! > * Borg doesn't fit the "full / differential / incremental" paradigm > neatly, but it certainly has some of the advantages of differential / > incremental backups (plus others that classic differential / > incremental ones do not have, such as deduplication - i.e., if the > backup source contains multiple copies of some data, that dataonly > needs to be stored once in the backup target). And how useful is that? There are very few duplicate files on my systems, because I use applications to eliminate duplicates. Eliminating duplicates in a live data repository is far more important than doing it on backup media. > * I don't know exactly what you mean by "deletion restoral", but Borg > (and, I assume, many other good backup solutions) offers a > flexible variety of methods to restore deleted files No. It is the opposite of that. It is the option to not restore files that have been deleted since the backup, or to actively delete files during a restore. It is similar - but not the same as - directing the restore not to restore files that do not exist on the restore target. , and / or to > restore a snapshot of the backup source as it existed at a given point > in time: https://docs.borgbase.com/restore/ > > If you are willing to explain further, that would be appreciated. > Thanks! > > Celejar >
[toc] | [prev] | [next] | [standalone]
| From | Celejar <celejar@gmail.com> |
|---|---|
| Date | 2020-08-06 18:10 +0200 |
| Message-ID | <AAQCS-4cZ-13@gated-at.bofh.it> |
| In reply to | #225938 |
On Thu, 6 Aug 2020 10:04:28 -0500 Leslie Rhorer <lesrhorer@att.net> wrote: > On 8/6/2020 9:07 AM, Celejar wrote: > > On Thu, 6 Aug 2020 05:02:17 -0500 > > Leslie Rhorer <lesrhorer@att.net> wrote: > > > * Incremental and differential backups are backups of the delta between > > the last full backup and the current system state (either individually > > [differential] or collectively [incremental]) > > > > * I have no idea (nor do Google, DuckDuckGo, or Wikipedia) what a > > "decremental backup" is. > > I suggest reading the DAR features list. I will post it again: > > http://dar.linux.free.fr/doc/Features.html > > Meanwhile, a decremental backup is defines as: > > As opposed to incremental backups, where the older one is a full backup > and each subsequent backup contains only the changes from the previous > backup, decremental backup let the full backup be the more recent while > the older ones only contain changes compared to the just more recent > one. This has the advantage of providing a single archive to use to > restore a whole system in its latest known state, while reducing the > overall amount of data to retain older versions of files (same amount > required as with differential backup). It has also the advantage to not > have to keep several set of backup as you just need to delete the oldest > backup when you need storage space. However it has the default to > require at each new cycle the creation of a full backup, then the > transformation of the previous full backup into a so-called decremental > backup. Yes, everything has a cost! Ah, okay. So IIUC, each time you backup you do a full backup, and you then convert the previous backups into the reverse of the more common incremental / differential backups. I certainly don't see much advantage to this over a typical incremental / differential backup scheme, and why it's worth paying the cost that you acknowledge, but if that's what you want, go for it. > > * Borg doesn't fit the "full / differential / incremental" paradigm > > neatly, but it certainly has some of the advantages of differential / > > incremental backups (plus others that classic differential / > > incremental ones do not have, such as deduplication - i.e., if the > > backup source contains multiple copies of some data, that dataonly > > needs to be stored once in the backup target). > > And how useful is that? There are very few duplicate files on my > systems, because I use applications to eliminate duplicates. > Eliminating duplicates in a live data repository is far more important > than doing it on backup media. Why do you consider it far more important? And do your applications cover the case where multiple non-identical files share some data chunks? > > * I don't know exactly what you mean by "deletion restoral", but Borg > > (and, I assume, many other good backup solutions) offers a > > flexible variety of methods to restore deleted files > > No. It is the opposite of that. It is the option to not restore files > that have been deleted since the backup, or to actively delete files > during a restore. It is similar - but not the same as - directing the > restore not to restore files that do not exist on the restore target. Okay, but I'm not really getting the use case for this. You want to restore files, but not files that have been deleted - so that means you only want to restore files that still exist (but have been altered)? Or are you saying that there's some way to distinguish between accidental and deliberate deletion? The way I see it, if I'm restoring an entire system snapshot, I want things back the way they were. If I want a specific file back, I know what I want, whether I deleted it or altered it and want the original back. If a disk (or disk array) fails, is there any way to know what files on it may have been deleted since the last backup? In other words, in what real world scenario is it useful to tell the restore program "restore everything except for files that have been deleted since the backup"? If a disk fails, I want everything back from the last backup, and I have no way of knowing (except for my rather unreliable memory) what exactly has changed or been deleted since that backup. If I have specific files that I've deleted or altered and I want the originals back, I just restore those files manually. Celejar
[toc] | [prev] | [next] | [standalone]
| From | Jonathan Dowland <jon+debian-user@dow.land> |
|---|---|
| Date | 2020-08-07 18:30 +0200 |
| Message-ID | <ABdpM-166-21@gated-at.bofh.it> |
| In reply to | #225943 |
On Thu, Aug 06, 2020 at 12:03:56PM -0400, Celejar wrote: >Ah, okay. So IIUC, each time you backup you do a full backup, and you then >convert the previous backups into the reverse of the more common >incremental / differential backups. It's an implementation detail: rdiff-backup does it too, I suspect it's commonly done. From what I recall many version control systems do something similar. -- Please do not CC me, I am subscribed to the list. 👱🏻 Jonathan Dowland ✎ jmtd@debian.org 🔗 https://jmtd.net
[toc] | [prev] | [next] | [standalone]
| From | Celejar <celejar@gmail.com> |
|---|---|
| Date | 2020-08-07 19:40 +0200 |
| Message-ID | <ABevw-1IH-21@gated-at.bofh.it> |
| In reply to | #225985 |
On Fri, 7 Aug 2020 17:21:44 +0100 Jonathan Dowland <jon+debian-user@dow.land> wrote: > On Thu, Aug 06, 2020 at 12:03:56PM -0400, Celejar wrote: > >Ah, okay. So IIUC, each time you backup you do a full backup, and you then > >convert the previous backups into the reverse of the more common > >incremental / differential backups. > > It's an implementation detail: rdiff-backup does it too, I suspect it's > commonly done. From what I recall many version control systems do > something similar. Thanks. But what's the advantage? Isn't it more work on each backup (of which there are typically many), in exchange for less work on restores (which are typically quite rare)? What am I missing? Celejar
[toc] | [prev] | [next] | [standalone]
| From | rhkramer@gmail.com |
|---|---|
| Date | 2020-08-08 00:10 +0200 |
| Message-ID | <ABiIN-4p2-5@gated-at.bofh.it> |
| In reply to | #225989 |
[Multipart message — attachments visible in raw view] — view raw
On Friday, August 07, 2020 01:36:21 PM Celejar wrote: > On Fri, 7 Aug 2020 17:21:44 +0100 > > Jonathan Dowland <jon+debian-user@dow.land> wrote: > > On Thu, Aug 06, 2020 at 12:03:56PM -0400, Celejar wrote: > > >Ah, okay. So IIUC, each time you backup you do a full backup, and you > > >then convert the previous backups into the reverse of the more common > > >incremental / differential backups. > > > > It's an implementation detail: rdiff-backup does it too, I suspect it's > > commonly done. From what I recall many version control systems do > > something similar. > > Thanks. But what's the advantage? Isn't it more work on each backup (of > which there are typically many), in exchange for less work on restores > (which are typically quite rare)? What am I missing? Ok, I went back and did a little research. RCS, one of the early version control systems, used reverse deltas for the main line of development -- forward deltas for branches, it's a long story ;-). The intent of the reverse deltas in RCS (according to the Tichy paper) was (is) to allow fast checkout of the most recent version. * [[https://docs.lib.purdue.edu/cgi/viewcontent.cgi?article=1393&context=cstech] [RCS: A System for Version Control - Purdue e-Pubs]]
[toc] | [prev] | [next] | [standalone]
| From | Celejar <celejar@gmail.com> |
|---|---|
| Date | 2020-08-09 04:30 +0200 |
| Message-ID | <ABJfX-3Bn-5@gated-at.bofh.it> |
| In reply to | #225992 |
On Fri, 7 Aug 2020 18:02:11 -0400 rhkramer@gmail.com wrote: > On Friday, August 07, 2020 01:36:21 PM Celejar wrote: > > On Fri, 7 Aug 2020 17:21:44 +0100 > > > > Jonathan Dowland <jon+debian-user@dow.land> wrote: > > > On Thu, Aug 06, 2020 at 12:03:56PM -0400, Celejar wrote: > > > >Ah, okay. So IIUC, each time you backup you do a full backup, and you > > > >then convert the previous backups into the reverse of the more common > > > >incremental / differential backups. > > > > > > It's an implementation detail: rdiff-backup does it too, I suspect it's > > > commonly done. From what I recall many version control systems do > > > something similar. > > > > Thanks. But what's the advantage? Isn't it more work on each backup (of > > which there are typically many), in exchange for less work on restores > > (which are typically quite rare)? What am I missing? > > Ok, I went back and did a little research. RCS, one of the early version > control systems, used reverse deltas for the main line of development -- > forward deltas for branches, it's a long story ;-). > > The intent of the reverse deltas in RCS (according to the Tichy paper) was > (is) to allow fast checkout of the most recent version. Fair enough. But less applicable in the case of backups, since restores are quite rare, as I've been pointing out. > [[https://docs.lib.purdue.edu/cgi/viewcontent.cgi?article=1393&context=cstech] > [RCS: A System for Version Control - Purdue e-Pubs]] Celejar
[toc] | [prev] | [next] | [standalone]
| From | Jonathan Dowland <jon+debian-user@dow.land> |
|---|---|
| Date | 2020-08-09 11:10 +0200 |
| Message-ID | <ABPv3-7t8-5@gated-at.bofh.it> |
| In reply to | #226029 |
On Sat, Aug 08, 2020 at 10:22:51PM -0400, Celejar wrote: >Fair enough. But less applicable in the case of backups, since restores >are quite rare, as I've been pointing out. I'd agree they're probably rare, and certainly less common than backups. But in my experience when you do perform a restore it's almost always the latest version you want. They should not be as rare as they are. We should all be in the habit of regularly performing a restore to test our backups are working. One advantage of the reverse-delta storage mechanism: With rdiff-backup, if I want to restore the latest version I can simply copy the backup repository (ignoring/excluding "rdiff-backup-increments" directory), I do not need to use rdiff-backup itself. Which could be useful if the backup tool breaks, or you can't easily install it on a restore target for whatever reason (It requires Python 2 for example and that becomes harder to get). Or if a catastrophic bug in the backup software means the incremental storage scheme is corrupted in some way. -- Jonathan Dowland
[toc] | [prev] | [next] | [standalone]
| From | Stefan Monnier <monnier@iro.umontreal.ca> |
|---|---|
| Date | 2020-08-09 17:00 +0200 |
| Message-ID | <ABUXM-29u-3@gated-at.bofh.it> |
| In reply to | #226033 |
>>Fair enough. But less applicable in the case of backups, since restores
>>are quite rare, as I've been pointing out.
>
> I'd agree they're probably rare, and certainly less common than backups.
> But in my experience when you do perform a restore it's almost always
> the latest version you want. They should not be as rare as they are. We
> should all be in the habit of regularly performing a restore to test our
> backups are working.
Also, it's convenient to make your latest backup readily accessible so
it's very easy to get back yesterday's version of a file in case your
fingers fumbled (which might be much more frequent than drive failures).
> One advantage of the reverse-delta storage mechanism: With rdiff-backup, if
> I want to restore the latest version I can simply copy the backup repository
> (ignoring/excluding "rdiff-backup-increments" directory), I do not need to
> use rdiff-backup itself.
Another advantage is that it lets you access a host's files even when
that host is down (as long as the backup server is up, which is likely
to always be the case).
Stefan
[toc] | [prev] | [next] | [standalone]
| From | Celejar <celejar@gmail.com> |
|---|---|
| Date | 2020-08-09 18:30 +0200 |
| Message-ID | <ABWmT-382-7@gated-at.bofh.it> |
| In reply to | #226039 |
On Sun, 09 Aug 2020 10:53:30 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: > >>Fair enough. But less applicable in the case of backups, since restores > >>are quite rare, as I've been pointing out. > > > > I'd agree they're probably rare, and certainly less common than backups. > > But in my experience when you do perform a restore it's almost always > > the latest version you want. They should not be as rare as they are. We > > should all be in the habit of regularly performing a restore to test our > > backups are working. > > Also, it's convenient to make your latest backup readily accessible so > it's very easy to get back yesterday's version of a file in case your > fingers fumbled (which might be much more frequent than drive failures). Understood. I have indeed done that many times ... > > One advantage of the reverse-delta storage mechanism: With rdiff-backup, if > > I want to restore the latest version I can simply copy the backup repository > > (ignoring/excluding "rdiff-backup-increments" directory), I do not need to > > use rdiff-backup itself. > > Another advantage is that it lets you access a host's files even when > that host is down (as long as the backup server is up, which is likely > to always be the case). Perhaps I'm misunderstanding something here, but why can't you do that with ordinary forward deltas, as long as all the necessary information is stored with the backups? Are you saying that classic forward-delta based systems don't store backup metadata (or whatever the right term is) on the backup server, but on the host? What happens, then, if the host's storage is unrecoverable? Celejar
[toc] | [prev] | [next] | [standalone]
| From | Stefan Monnier <monnier@iro.umontreal.ca> |
|---|---|
| Date | 2020-08-09 19:20 +0200 |
| Message-ID | <ABX9f-3Ec-5@gated-at.bofh.it> |
| In reply to | #226040 |
>> Another advantage is that it lets you access a host's files even when
>> that host is down (as long as the backup server is up, which is likely
>> to always be the case).
>
> Perhaps I'm misunderstanding something here, but why can't you do that
> with ordinary forward deltas, as long as all the necessary information
> is stored with the backups?
I was talking specifically about a system like `rdiff-backup` where the
latest version of the backed up files are readily available "in their
normal form". Wit forward deltas you'd either have to settle for
searching through some old backup, or have to deal with the deltas
somehow. Of course, some software might hide all of that for you to
some extent, but my point was that with `rdiff-backup`, I can trivially
log onto the backup server, go to a given hosts's backup area and
I immediately see that hosts's files (in an almost-uptodate form).
Stefan
[toc] | [prev] | [next] | [standalone]
| From | rhkramer@gmail.com |
|---|---|
| Date | 2020-08-09 23:50 +0200 |
| Message-ID | <AC1my-63z-7@gated-at.bofh.it> |
| In reply to | #226040 |
On Sunday, August 09, 2020 12:22:16 PM Celejar wrote: > On Sun, 09 Aug 2020 10:53:30 -0400 > Stefan Monnier <monnier@iro.umontreal.ca> wrote: > > Another advantage is that it lets you access a host's files even when > > that host is down (as long as the backup server is up, which is likely > > to always be the case). > > Perhaps I'm misunderstanding something here, but why can't you do that > with ordinary forward deltas, as long as all the necessary information > is stored with the backups? > Are you saying that classic forward-delta > based systems don't store backup metadata (or whatever the right term > is) on the backup server, but on the host? I don't think he is saying that (but I could be wrong) (and I think that is pretty clear from subsequent comments, but I wanted to perhaps make it a little more clear), iiuc, he's just pointing out things like the overhead on recovery when you have to take the base file and apply the forward deltas to it. > What happens, then, if the > host's storage is unrecoverable?
[toc] | [prev] | [next] | [standalone]
| From | Celejar <celejar@gmail.com> |
|---|---|
| Date | 2020-08-10 00:00 +0200 |
| Message-ID | <AC1we-66O-1@gated-at.bofh.it> |
| In reply to | #226047 |
On Sun, 9 Aug 2020 17:42:43 -0400 rhkramer@gmail.com wrote: > On Sunday, August 09, 2020 12:22:16 PM Celejar wrote: > > On Sun, 09 Aug 2020 10:53:30 -0400 > > Stefan Monnier <monnier@iro.umontreal.ca> wrote: > > > > Another advantage is that it lets you access a host's files even when > > > that host is down (as long as the backup server is up, which is likely > > > to always be the case). > > > > Perhaps I'm misunderstanding something here, but why can't you do that > > with ordinary forward deltas, as long as all the necessary information > > is stored with the backups? > > > Are you saying that classic forward-delta > > based systems don't store backup metadata (or whatever the right term > > is) on the backup server, but on the host? > > I don't think he is saying that (but I could be wrong) (and I think that is > pretty clear from subsequent comments, but I wanted to perhaps make it a > little more clear), iiuc, he's just pointing out things like the overhead on > recovery when you have to take the base file and apply the forward deltas to > it. Perhaps, but that has nothing to do with whether the host is down or up - any time you're doing a recovery, you're going to run into this issue, insofar as it's an issue. Celejar
[toc] | [prev] | [next] | [standalone]
Page 3 of 6 — ← Prev page 1 2 [3] 4 5 6 Next page →
Back to top | Article view | linux.debian.user
csiph-web