Re: freshly installed SW not found

From	William Unruh <unruh@invalid.ca>
Newsgroups	alt.os.linux
Subject	Re: freshly installed SW not found
Date	2019-03-01 23:38 +0000
Organization	A noiseless patient Spider
Message-ID	<q5cfp2$rfr$1@dont-email.me> (permalink)
References	(2 earlier) <q548td$tth$1@dont-email.me> <q5bhll$vgu$1@dont-email.me> <op.zxzcy6dla3w0dxdave@hodgins.homeip.net> <q5bua9$du9$1@dont-email.me> <op.zxznlbyza3w0dxdave@hodgins.homeip.net>

Show all headers | View raw

On 2019-03-01, David W. Hodgins <dwhodgins@nomail.afraid.org> wrote:
> On Fri, 01 Mar 2019 13:40:10 -0500, William Unruh <unruh@invalid.ca> wrote:
>
>> On 2019-03-01, David W. Hodgins <dwhodgins@nomail.afraid.org> wrote:
>>> Two things it doesn't have builtin
>>> - compression
>>> - hardlinking files in the snapshots that have not changed
>>>
>>> Both of these can be handled in post task scripts, which it makes it
>>> easy to add. A simple loop over the destination tree could be used to
>>
>> Unfortunately no. If you compress, then it becomes impossible to test if
>> the next day's file is the same as the previous day's. without
>> uncompressing the previous day's file. They are not the same.
>> And if you uncompress, then you have the problem that rsync's default
>> test for "the same" -- date and size-- no longer works either, making it
>> very slow.
>
> What I was thinking about is to run luckybackup creating a new snapshot,
> which is a full copy of the files/directories selected for backing up.
> No compression, no hard linking of duplicates.
>
> Then compress each file in the newly created snapshot.
>
> Then use freedups to remove one copy of the compressed copies of the
> file, when it's exactly the same as the previously created compressed
> copy of the file.
>
> Time consuming, still requires the full space for the initial creation
> of each snapshot, but reduces the total space needed when multiple
> snapshots are wanted. Compressing each individual file allows keeping
> only one copy of files that do not change between snapshots, at the
> expense of compression being more efficient when the files are put into
> a tar archive prior to compression.

This is really slow. You would have to use MD5 hash to decide if the two
compressed copies are the same, which is slow (rsync by default uses the
date and time of the file to decide if they are the same.)
If you have thousands of hard links ( which Soviet implied he had) that
will take forever. 
>
> It really depends on what percentage of the files tend to change between
> snapshots, and how well they compress, as to whether or not it's worth
> doing the extra steps.

But you have to compress the new file, compare it, using md5 to the old
file to decide if they are the same. If you have a large system, you are
entering the territory where the saving takes longer than the time
between backups.

Memory is cheap. Just buy more disks and do not compress them. You are
going to regret compressing them.

>
> It also depends on whether multiple snapshots will be kept in one file
> system, or, for example, archived to optical disc after creation.

In which case you cannot hard link them. 

>
> Regards, Dave Hodgins
>

Thread

freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-26 19:47 +0100
  Re: freshly installed SW not found "J.O. Aho" <user@example.net> - 2019-02-26 20:28 +0100
    Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-26 21:51 +0100
      Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-26 21:46 +0000
        Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-27 12:52 +0100
          Re: freshly installed SW not found Aragorn <thorongil@telenet.be> - 2019-02-27 12:44 +0000
            Re: freshly installed SW not found Dan Purgert <dan@djph.net> - 2019-02-27 13:27 +0000
            Re: freshly installed SW not found Richard Kettlewell <invalid@invalid.invalid> - 2019-02-27 15:16 +0000
            Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-27 16:59 +0100
              Re: freshly installed SW not found Dan Purgert <dan@djph.net> - 2019-02-27 16:02 +0000
                Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-27 20:46 +0100
                Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-27 20:18 +0000
                Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-28 12:47 +0100
                Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-28 18:32 +0000
                Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-03-01 01:14 +0100
                Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-03-01 01:54 +0100
                Re: freshly installed SW not found Jasen Betts <jasen@xnet.co.nz> - 2019-03-01 03:09 +0000
                Re: freshly installed SW not found Aragorn <thorongil@telenet.be> - 2019-02-27 20:21 +0000
                Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-27 23:02 +0000
                Re: freshly installed SW not found Richard Kettlewell <invalid@invalid.invalid> - 2019-02-27 23:24 +0000
                Re: freshly installed SW not found Jasen Betts <jasen@xnet.co.nz> - 2019-02-28 04:46 +0000
                Re: freshly installed SW not found Dan Purgert <dan@djph.net> - 2019-02-28 10:59 +0000
                Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-28 18:51 +0100
                Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-28 18:24 +0000
                Re: freshly installed SW not found Dan Purgert <dan@djph.net> - 2019-03-01 13:49 +0000
                Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-03-01 18:22 +0100
                Re: freshly installed SW not found Aragorn <thorongil@telenet.be> - 2019-03-01 19:06 +0000
                Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-03-03 01:08 +0100
                Re: freshly installed SW not found Aragorn <thorongil@telenet.be> - 2019-03-03 00:14 +0000
                Re: freshly installed SW not found Dan Purgert <dan@djph.net> - 2019-03-03 13:00 +0000
                Re: freshly installed SW not found Dan Purgert <dan@djph.net> - 2019-03-03 12:46 +0000
                Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-28 18:15 +0000
              Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-27 16:38 +0000
                Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-27 20:51 +0100
              Re: freshly installed SW not found Jasen Betts <jasen@xnet.co.nz> - 2019-02-28 04:39 +0000
        Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-02-27 14:30 +0100
      Re: freshly installed SW not found "J.O. Aho" <user@example.net> - 2019-02-27 07:05 +0100
      Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-02-27 14:29 +0100
        Re: freshly installed SW not found Paul <nospam@needed.invalid> - 2019-02-27 08:50 -0500
          Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-02-28 00:05 +0100
        Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-27 16:56 +0100
        Re: freshly installed SW not found Chris <ithinkiam@gmail.com> - 2019-03-01 14:58 +0000
      Re: freshly installed SW not found Chris <ithinkiam@gmail.com> - 2019-03-01 15:04 +0000
        Re: freshly installed SW not found "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2019-03-01 10:50 -0500
          Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-03-01 18:40 +0000
            Re: freshly installed SW not found "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2019-03-01 14:40 -0500
              Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-03-01 23:38 +0000
                Re: freshly installed SW not found "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2019-03-01 19:09 -0500
                Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-03-02 03:47 +0000
                Re: freshly installed SW not found "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2019-03-01 22:55 -0500
                Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-03-02 07:08 +0000
                Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-03-02 11:27 +0100
                Re: freshly installed SW not found Paul <nospam@needed.invalid> - 2019-03-02 04:10 -0500
                Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-03-02 16:29 +0000
                Re: freshly installed SW not found Paul <nospam@needed.invalid> - 2019-03-02 11:54 -0500
                Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-03-02 11:24 +0100
                Re: freshly installed SW not found Jasen Betts <jasen@xnet.co.nz> - 2019-03-02 00:28 +0000
                Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-03-02 11:31 +0100
                Re: freshly installed SW not found Aragorn <thorongil@telenet.be> - 2019-03-02 11:31 +0000
                Re: freshly installed SW not found Melzzzzz <Melzzzzz@zzzzz.com> - 2019-03-02 11:34 +0000
                Re: freshly installed SW not found Aragorn <thorongil@telenet.be> - 2019-03-02 11:48 +0000
                Re: freshly installed SW not found Melzzzzz <Melzzzzz@zzzzz.com> - 2019-03-02 12:15 +0000
                Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-03-02 23:15 +0100
                Re: freshly installed SW not found Chris Elvidge <chris@mshome.net> - 2019-03-03 00:33 +0000
                Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-03-03 01:38 +0100
                Compressed filesystems [Was: Re: freshly installed SW not found] "Carlos E.R." <robin_listas@es.invalid> - 2019-03-03 13:11 +0100
                Re: Compressed filesystems [Was: Re: freshly installed SW not found] Soviet_Mario <SovietMario@CCCP.MIR> - 2019-03-03 19:49 +0100
                Re: Compressed filesystems [Was: Re: freshly installed SW not found] "Carlos E.R." <robin_listas@es.invalid> - 2019-03-03 21:17 +0100
                Re: freshly installed SW not found Dan Purgert <dan@djph.net> - 2019-03-03 12:49 +0000
                Re: freshly installed SW not found Melzzzzz <Melzzzzz@zzzzz.com> - 2019-03-03 12:35 +0000
            Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-03-02 11:22 +0100
  Re: freshly installed SW not found Wildman <best_lay@yahoo.com> - 2019-02-26 13:31 -0600
    Re: freshly installed SW not found Wildman <best_lay@yahoo.com> - 2019-02-26 15:42 -0600
      Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-26 21:51 +0000
  Re: freshly installed SW not found "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2019-02-26 14:36 -0500
    Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-26 21:58 +0100
      Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-26 21:49 +0000
        Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-27 12:54 +0100
  Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-26 19:51 +0000

csiph-web