Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > alt.os.linux > #54222 > unrolled thread
| Started by | Soviet_Mario <SovietMario@CCCP.MIR> |
|---|---|
| First post | 2019-02-26 19:47 +0100 |
| Last post | 2019-02-26 19:51 +0000 |
| Articles | 20 on this page of 79 — 14 participants |
Back to article view | Back to alt.os.linux
freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-26 19:47 +0100
Re: freshly installed SW not found "J.O. Aho" <user@example.net> - 2019-02-26 20:28 +0100
Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-26 21:51 +0100
Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-26 21:46 +0000
Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-27 12:52 +0100
Re: freshly installed SW not found Aragorn <thorongil@telenet.be> - 2019-02-27 12:44 +0000
Re: freshly installed SW not found Dan Purgert <dan@djph.net> - 2019-02-27 13:27 +0000
Re: freshly installed SW not found Richard Kettlewell <invalid@invalid.invalid> - 2019-02-27 15:16 +0000
Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-27 16:59 +0100
Re: freshly installed SW not found Dan Purgert <dan@djph.net> - 2019-02-27 16:02 +0000
Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-27 20:46 +0100
Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-27 20:18 +0000
Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-28 12:47 +0100
Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-28 18:32 +0000
Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-03-01 01:14 +0100
Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-03-01 01:54 +0100
Re: freshly installed SW not found Jasen Betts <jasen@xnet.co.nz> - 2019-03-01 03:09 +0000
Re: freshly installed SW not found Aragorn <thorongil@telenet.be> - 2019-02-27 20:21 +0000
Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-27 23:02 +0000
Re: freshly installed SW not found Richard Kettlewell <invalid@invalid.invalid> - 2019-02-27 23:24 +0000
Re: freshly installed SW not found Jasen Betts <jasen@xnet.co.nz> - 2019-02-28 04:46 +0000
Re: freshly installed SW not found Dan Purgert <dan@djph.net> - 2019-02-28 10:59 +0000
Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-28 18:51 +0100
Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-28 18:24 +0000
Re: freshly installed SW not found Dan Purgert <dan@djph.net> - 2019-03-01 13:49 +0000
Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-03-01 18:22 +0100
Re: freshly installed SW not found Aragorn <thorongil@telenet.be> - 2019-03-01 19:06 +0000
Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-03-03 01:08 +0100
Re: freshly installed SW not found Aragorn <thorongil@telenet.be> - 2019-03-03 00:14 +0000
Re: freshly installed SW not found Dan Purgert <dan@djph.net> - 2019-03-03 13:00 +0000
Re: freshly installed SW not found Dan Purgert <dan@djph.net> - 2019-03-03 12:46 +0000
Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-28 18:15 +0000
Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-27 16:38 +0000
Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-27 20:51 +0100
Re: freshly installed SW not found Jasen Betts <jasen@xnet.co.nz> - 2019-02-28 04:39 +0000
Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-02-27 14:30 +0100
Re: freshly installed SW not found "J.O. Aho" <user@example.net> - 2019-02-27 07:05 +0100
Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-02-27 14:29 +0100
Re: freshly installed SW not found Paul <nospam@needed.invalid> - 2019-02-27 08:50 -0500
Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-02-28 00:05 +0100
Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-27 16:56 +0100
Re: freshly installed SW not found Chris <ithinkiam@gmail.com> - 2019-03-01 14:58 +0000
Re: freshly installed SW not found Chris <ithinkiam@gmail.com> - 2019-03-01 15:04 +0000
Re: freshly installed SW not found "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2019-03-01 10:50 -0500
Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-03-01 18:40 +0000
Re: freshly installed SW not found "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2019-03-01 14:40 -0500
Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-03-01 23:38 +0000
Re: freshly installed SW not found "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2019-03-01 19:09 -0500
Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-03-02 03:47 +0000
Re: freshly installed SW not found "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2019-03-01 22:55 -0500
Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-03-02 07:08 +0000
Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-03-02 11:27 +0100
Re: freshly installed SW not found Paul <nospam@needed.invalid> - 2019-03-02 04:10 -0500
Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-03-02 16:29 +0000
Re: freshly installed SW not found Paul <nospam@needed.invalid> - 2019-03-02 11:54 -0500
Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-03-02 11:24 +0100
Re: freshly installed SW not found Jasen Betts <jasen@xnet.co.nz> - 2019-03-02 00:28 +0000
Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-03-02 11:31 +0100
Re: freshly installed SW not found Aragorn <thorongil@telenet.be> - 2019-03-02 11:31 +0000
Re: freshly installed SW not found Melzzzzz <Melzzzzz@zzzzz.com> - 2019-03-02 11:34 +0000
Re: freshly installed SW not found Aragorn <thorongil@telenet.be> - 2019-03-02 11:48 +0000
Re: freshly installed SW not found Melzzzzz <Melzzzzz@zzzzz.com> - 2019-03-02 12:15 +0000
Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-03-02 23:15 +0100
Re: freshly installed SW not found Chris Elvidge <chris@mshome.net> - 2019-03-03 00:33 +0000
Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-03-03 01:38 +0100
Compressed filesystems [Was: Re: freshly installed SW not found] "Carlos E.R." <robin_listas@es.invalid> - 2019-03-03 13:11 +0100
Re: Compressed filesystems [Was: Re: freshly installed SW not found] Soviet_Mario <SovietMario@CCCP.MIR> - 2019-03-03 19:49 +0100
Re: Compressed filesystems [Was: Re: freshly installed SW not found] "Carlos E.R." <robin_listas@es.invalid> - 2019-03-03 21:17 +0100
Re: freshly installed SW not found Dan Purgert <dan@djph.net> - 2019-03-03 12:49 +0000
Re: freshly installed SW not found Melzzzzz <Melzzzzz@zzzzz.com> - 2019-03-03 12:35 +0000
Re: freshly installed SW not found "Carlos E.R." <robin_listas@es.invalid> - 2019-03-02 11:22 +0100
Re: freshly installed SW not found Wildman <best_lay@yahoo.com> - 2019-02-26 13:31 -0600
Re: freshly installed SW not found Wildman <best_lay@yahoo.com> - 2019-02-26 15:42 -0600
Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-26 21:51 +0000
Re: freshly installed SW not found "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2019-02-26 14:36 -0500
Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-26 21:58 +0100
Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-26 21:49 +0000
Re: freshly installed SW not found Soviet_Mario <SovietMario@CCCP.MIR> - 2019-02-27 12:54 +0100
Re: freshly installed SW not found William Unruh <unruh@invalid.ca> - 2019-02-26 19:51 +0000
Page 1 of 4 [1] 2 3 4 Next page →
| From | Soviet_Mario <SovietMario@CCCP.MIR> |
|---|---|
| Date | 2019-02-26 19:47 +0100 |
| Subject | freshly installed SW not found |
| Message-ID | <q541lh$bqc$1@dont-email.me> |
Hi, I came across a program claiming to be able to do exactly what I was looking for (a backup on a per-Directory basis, maintaining the hierarchy structure). The SW is called : awit-dbackup (Site : https://wiki.dbackup.org/) I installed it from Synaptic. But It neither shows up in any menu nor it is elsewhere on the paths (from shell no cmd line executes). Where is it and how to run it ? It seemed a GUI sw from the descriptio, but in case of doubt how to assess that ? TY (ah, I use Debian Stretch 9) -- 1) Resistere, resistere, resistere. 2) Se tutti pagano le tasse, le tasse le pagano tutti Soviet_Mario - (aka Gatto_Vizzato)
[toc] | [next] | [standalone]
| From | "J.O. Aho" <user@example.net> |
|---|---|
| Date | 2019-02-26 20:28 +0100 |
| Message-ID | <gdlialFffa1U1@mid.individual.net> |
| In reply to | #54222 |
On 2/26/19 7:47 PM, Soviet_Mario wrote: > Hi, I came across a program claiming to be able to do exactly what I was > looking for (a backup on a per-Directory basis, maintaining the > hierarchy structure). > > The SW is called : awit-dbackup (Site : https://wiki.dbackup.org/) Binary name should be dbackup see http://manpages.ubuntu.com/manpages/xenial/man8/dbackup.8.html > I installed it from Synaptic. > But It neither shows up in any menu nor it is elsewhere on the paths > (from shell no cmd line executes). Could be that it's located in /sbin /usr/sbin, those directories do not be part of a normal users $PATH, but would be the case if you were a root user. > Where is it and how to run it ? Location on a debian based system would be: /usr/sbin/dbackup see https://packages.debian.org/sid/all/awit-dbackup/filelist > It seemed a GUI sw from the descriptio, but in case of doubt how to > assess that ? From what I seen, no it's not a GUI based tool, take a look at the man page and you see how to use it. You may be able to install some kind of third party frontend for it, but as I never heard about this one until today, so I have no clue. -- //Aho
[toc] | [prev] | [next] | [standalone]
| From | Soviet_Mario <SovietMario@CCCP.MIR> |
|---|---|
| Date | 2019-02-26 21:51 +0100 |
| Message-ID | <q548td$tth$1@dont-email.me> |
| In reply to | #54223 |
On 26/02/19 20:28, J.O. Aho wrote: > On 2/26/19 7:47 PM, Soviet_Mario wrote: >> Hi, I came across a program claiming to be able to do >> exactly what I was looking for (a backup on a >> per-Directory basis, maintaining the hierarchy structure). >> >> The SW is called : awit-dbackup (Site : >> https://wiki.dbackup.org/) > > Binary name should be dbackup > see > http://manpages.ubuntu.com/manpages/xenial/man8/dbackup.8.html > > >> I installed it from Synaptic. >> But It neither shows up in any menu nor it is elsewhere on >> the paths (from shell no cmd line executes). > > Could be that it's located in /sbin /usr/sbin, those > directories do not be part of a normal users $PATH, but > would be the case if you were a root user. > > >> Where is it and how to run it ? > > Location on a debian based system would be: /usr/sbin/dbackup > see https://packages.debian.org/sid/all/awit-dbackup/filelist > > >> It seemed a GUI sw from the descriptio, but in case of >> doubt how to assess that ? > > From what I seen, no it's not a GUI based tool, take a look > at the man page and you see how to use it. > You may be able to install some kind of third party frontend > for it, but as I never heard about this one until today, so > I have no clue. > > yes, tnx. I located them. I'm using it but there is a trick to take into account if one (like me) wants to make a full backup : the /MEDIA (lowercase) folder must be excluded. As I was unsure i used both the --exclude-path and --exclude-dir Otherwise, and i slammed the face on it, an infinite recursion takes place. It was very vierd experience to see this sort of self inclusion and reinclusion. Anyway (using gzip) the programs is darn thunderbolt FAST. By now (one third of progress) It has grinded (or grounded ?) 150 GB of compressed data size, since 20:12 to 21:48. I'd really like some front end ... but well, it works and it does what I needed. I notice the "compressed contentS of each folder is named dbackup0.tar.gz I'm hoping the sw would have some hidden feature as to make incremental/differential backup, generating updates like dbackup1.tar.gz, dbackup2.tar.gz etc on different runs. It'd be too good to be true perhaps :) -- 1) Resistere, resistere, resistere. 2) Se tutti pagano le tasse, le tasse le pagano tutti Soviet_Mario - (aka Gatto_Vizzato)
[toc] | [prev] | [next] | [standalone]
| From | William Unruh <unruh@invalid.ca> |
|---|---|
| Date | 2019-02-26 21:46 +0000 |
| Message-ID | <q54c3a$fap$1@dont-email.me> |
| In reply to | #54227 |
On 2019-02-26, Soviet_Mario <SovietMario@CCCP.MIR> wrote: > On 26/02/19 20:28, J.O. Aho wrote: >> On 2/26/19 7:47 PM, Soviet_Mario wrote: >>> Hi, I came across a program claiming to be able to do >>> exactly what I was looking for (a backup on a >>> per-Directory basis, maintaining the hierarchy structure). >>> >>> The SW is called : awit-dbackup (Site : >>> https://wiki.dbackup.org/) >> >> Binary name should be dbackup >> see >> http://manpages.ubuntu.com/manpages/xenial/man8/dbackup.8.html >> >> >>> I installed it from Synaptic. >>> But It neither shows up in any menu nor it is elsewhere on >>> the paths (from shell no cmd line executes). >> >> Could be that it's located in /sbin /usr/sbin, those >> directories do not be part of a normal users $PATH, but >> would be the case if you were a root user. >> >> >>> Where is it and how to run it ? >> >> Location on a debian based system would be: /usr/sbin/dbackup >> see https://packages.debian.org/sid/all/awit-dbackup/filelist >> >> >>> It seemed a GUI sw from the descriptio, but in case of >>> doubt how to assess that ? >> >> From what I seen, no it's not a GUI based tool, take a look >> at the man page and you see how to use it. >> You may be able to install some kind of third party frontend >> for it, but as I never heard about this one until today, so >> I have no clue. >> >> > > yes, tnx. I located them. > > I'm using it but there is a trick to take into account if > one (like me) wants to make a full backup : the /MEDIA > (lowercase) folder must be excluded. > As I was unsure i used both the --exclude-path and --exclude-dir > > Otherwise, and i slammed the face on it, an infinite > recursion takes place. It was very vierd experience to see > this sort of self inclusion and reinclusion. > > Anyway (using gzip) the programs is darn thunderbolt FAST. > > By now (one third of progress) It has grinded (or grounded > ?) 150 GB of compressed data size, since 20:12 to 21:48. That would be about 1GB/min or 16MB/s which is not terribly fast. Are those "rust" drives or SSD? > > I'd really like some front end ... but well, it works and it > does what I needed. I notice the "compressed contentS of > each folder is named dbackup0.tar.gz Despite the compression, that is a lot of storage space they take up. You cannot keep too many backups. Something like rsnapshot (using rsync) will store about 30 backups of 200GB of data in half a TB. (it does differential backups so only files which changed get backed up). Not tarring or compressing means that the backup data is easy to get at which is usually what you want. > > I'm hoping the sw would have some hidden feature as to make > incremental/differential backup, generating updates like > dbackup1.tar.gz, dbackup2.tar.gz etc on different runs. rsnapshot makes hard links for files that have not changed so there is only one copy but retrieval is trivial, since you do not have to manually repeel all of the differences. > > It'd be too good to be true perhaps :) >
[toc] | [prev] | [next] | [standalone]
| From | Soviet_Mario <SovietMario@CCCP.MIR> |
|---|---|
| Date | 2019-02-27 12:52 +0100 |
| Message-ID | <q55tl5$bp2$1@dont-email.me> |
| In reply to | #54230 |
On 26/02/19 22:46, William Unruh wrote: > On 2019-02-26, Soviet_Mario <SovietMario@CCCP.MIR> wrote: >> On 26/02/19 20:28, J.O. Aho wrote: >>> On 2/26/19 7:47 PM, Soviet_Mario wrote: >>>> Hi, I came across a program claiming to be able to do >>>> exactly what I was looking for (a backup on a >>>> per-Directory basis, maintaining the hierarchy structure). >>>> >>>> The SW is called : awit-dbackup (Site : >>>> https://wiki.dbackup.org/) >>> >>> Binary name should be dbackup >>> see >>> http://manpages.ubuntu.com/manpages/xenial/man8/dbackup.8.html >>> >>> >>>> I installed it from Synaptic. >>>> But It neither shows up in any menu nor it is elsewhere on >>>> the paths (from shell no cmd line executes). >>> >>> Could be that it's located in /sbin /usr/sbin, those >>> directories do not be part of a normal users $PATH, but >>> would be the case if you were a root user. >>> >>> >>>> Where is it and how to run it ? >>> >>> Location on a debian based system would be: /usr/sbin/dbackup >>> see https://packages.debian.org/sid/all/awit-dbackup/filelist >>> >>> >>>> It seemed a GUI sw from the descriptio, but in case of >>>> doubt how to assess that ? >>> >>> From what I seen, no it's not a GUI based tool, take a look >>> at the man page and you see how to use it. >>> You may be able to install some kind of third party frontend >>> for it, but as I never heard about this one until today, so >>> I have no clue. >>> >>> >> >> yes, tnx. I located them. >> >> I'm using it but there is a trick to take into account if >> one (like me) wants to make a full backup : the /MEDIA >> (lowercase) folder must be excluded. >> As I was unsure i used both the --exclude-path and --exclude-dir >> >> Otherwise, and i slammed the face on it, an infinite >> recursion takes place. It was very vierd experience to see >> this sort of self inclusion and reinclusion. >> >> Anyway (using gzip) the programs is darn thunderbolt FAST. >> >> By now (one third of progress) It has grinded (or grounded >> ?) 150 GB of compressed data size, since 20:12 to 21:48. > > That would be about 1GB/min or 16MB/s which is not terribly fast. Are > those "rust" drives or SSD? old age Hdd as source (and with huge number of small files), and external 4 TB maxtor connected to USB 2.0 as target Not very favourable settings I deem >> >> I'd really like some front end ... but well, it works and it >> does what I needed. I notice the "compressed contentS of >> each folder is named dbackup0.tar.gz > > Despite the compression, that is a lot of storage space they take up. > You cannot keep too many backups. I'll not be doing always FULL backups. I'll discover how to do incremental or differential, in case by hand. > Something like rsnapshot (using rsync) will store about 30 backups of > 200GB of data in half a TB. (it does differential backups so only files > which changed get backed up). > > Not tarring or compressing means that the backup data is easy to get at > which is usually what you want. I call this inode space pollution, a thing that I don't find desirable on the backup target (even if I have to accept it in the userspace). It slows down searches, takes up space in itself just to store a huge number of inodes :\ > > > >> >> I'm hoping the sw would have some hidden feature as to make >> incremental/differential backup, generating updates like >> dbackup1.tar.gz, dbackup2.tar.gz etc on different runs. > > rsnapshot makes hard links for files that have not changed so there is > only one copy but retrieval is trivial, fast and space efficient, but UNSAFE (I mean in any unpleasant situation) I tried backintime and timeshift before, but were not a solution very suited for me > since you do not have to > manually repeel all of the differences. > >> >> It'd be too good to be true perhaps :) >> -- 1) Resistere, resistere, resistere. 2) Se tutti pagano le tasse, le tasse le pagano tutti Soviet_Mario - (aka Gatto_Vizzato)
[toc] | [prev] | [next] | [standalone]
| From | Aragorn <thorongil@telenet.be> |
|---|---|
| Date | 2019-02-27 12:44 +0000 |
| Message-ID | <q560n2$7vs$1@dont-email.me> |
| In reply to | #54237 |
On Wed, 27 Feb 2019 12:52:04 +0100, Soviet_Mario scribbled: > On 26/02/19 22:46, William Unruh wrote: > >> Something like rsnapshot (using rsync) will store about 30 backups of >> 200GB of data in half a TB. (it does differential backups so only files >> which changed get backed up). >> >> Not tarring or compressing means that the backup data is easy to get at >> which is usually what you want. > > I call this inode space pollution, a thing that I don't find desirable > on the backup target (even if I have to accept it in the userspace). > It slows down searches, takes up space in itself just to store a huge > number of inodes :\ rsnapshot uses hard-linking for files that haven't changed since the last backup. Hard links don't use up any inodes. -- With respect, = Aragorn =
[toc] | [prev] | [next] | [standalone]
| From | Dan Purgert <dan@djph.net> |
|---|---|
| Date | 2019-02-27 13:27 +0000 |
| Message-ID | <slrnq7d41m.3t0.dan@djph.net> |
| In reply to | #54239 |
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Aragorn wrote: > On Wed, 27 Feb 2019 12:52:04 +0100, Soviet_Mario scribbled: > >> On 26/02/19 22:46, William Unruh wrote: >> >>> Something like rsnapshot (using rsync) will store about 30 backups of >>> 200GB of data in half a TB. (it does differential backups so only files >>> which changed get backed up). >>> >>> Not tarring or compressing means that the backup data is easy to get at >>> which is usually what you want. >> >> I call this inode space pollution, a thing that I don't find desirable >> on the backup target (even if I have to accept it in the userspace). >> It slows down searches, takes up space in itself just to store a huge >> number of inodes :\ > > rsnapshot uses hard-linking for files that haven't changed since the last > backup. Hard links don't use up any inodes. Yup, though I use rsync and just stuff it all in one directory, since it doesn't change what hasn't changed. Then at some later point stuff it all into a tarball, and put it on more permanent storage. -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEBcqaUD8uEzVNxUrujhHd8xJ5ooEFAlx2kDYACgkQjhHd8xJ5 ooGJuQf+OkcJMa2wIAHrsYB1RKR8LfH8Dujy3101VqTCJ/03cduzKQWm4tu3r21u FZnOJR9jl1S1qkWaL35N+jNCsQsHwu7FTC0iwl4DyEunpKZeC0mufw7ppoSDf8mH +PP6tVOGBBGho6lsGrFzPR8LmmYsiFD9ayh6qlz4wp/8MBUkcCoVVfEE5wxaBS8S jNVsj0j0C5mPxkFfqv7xhuFxVC7uxaSmK6phi2zAmZ4y+GcMygrDare6sVE/IahJ 7k2KH2s8u5L5BANW4l5v+l9zSFxLDGlG+WnSsMH90wSNS1PCliVZvlAFxkrmHmvD Eoavis7xd1jV4kPDjmMpmzdIsqr5pw== =tYgT -----END PGP SIGNATURE----- -- |_|O|_| |_|_|O| Github: https://github.com/dpurgert |O|O|O| PGP: 05CA 9A50 3F2E 1335 4DC5 4AEE 8E11 DDF3 1279 A281
[toc] | [prev] | [next] | [standalone]
| From | Richard Kettlewell <invalid@invalid.invalid> |
|---|---|
| Date | 2019-02-27 15:16 +0000 |
| Message-ID | <87k1hlxnvh.fsf@LkoBDZeT.terraraq.uk> |
| In reply to | #54239 |
Aragorn <thorongil@telenet.be> writes: > On Wed, 27 Feb 2019 12:52:04 +0100, Soviet_Mario scribbled: >> On 26/02/19 22:46, William Unruh wrote: >>> Something like rsnapshot (using rsync) will store about 30 backups of >>> 200GB of data in half a TB. (it does differential backups so only files >>> which changed get backed up). >>> >>> Not tarring or compressing means that the backup data is easy to get at >>> which is usually what you want. My backup tool uses the same strategy. >> I call this inode space pollution, a thing that I don't find desirable >> on the backup target (even if I have to accept it in the userspace). >> It slows down searches, takes up space in itself just to store a huge >> number of inodes :\ > > rsnapshot uses hard-linking for files that haven't changed since the > last backup. Hard links don't use up any inodes. You do need a fresh copy of each directory for each incremental, though; you can’t hard-link directories in Linux even if the contents are identical (and I don’t think littering backup media with bind mounts is a realistic alternative l-) I’ve not observed a practical problem arising from this, though. -- https://www.greenend.org.uk/rjk/
[toc] | [prev] | [next] | [standalone]
| From | Soviet_Mario <SovietMario@CCCP.MIR> |
|---|---|
| Date | 2019-02-27 16:59 +0100 |
| Message-ID | <q56c4p$25f$2@dont-email.me> |
| In reply to | #54239 |
On 27/02/19 13:44, Aragorn wrote: > On Wed, 27 Feb 2019 12:52:04 +0100, Soviet_Mario scribbled: > >> On 26/02/19 22:46, William Unruh wrote: >> >>> Something like rsnapshot (using rsync) will store about 30 backups of >>> 200GB of data in half a TB. (it does differential backups so only files >>> which changed get backed up). >>> >>> Not tarring or compressing means that the backup data is easy to get at >>> which is usually what you want. >> >> I call this inode space pollution, a thing that I don't find desirable >> on the backup target (even if I have to accept it in the userspace). >> It slows down searches, takes up space in itself just to store a huge >> number of inodes :\ > > rsnapshot uses hard-linking for files that haven't changed since the last > backup. Hard links don't use up any inodes. > mmm, then I must have chosen the wrong terms. But some other kind of disk space must be used to add a link to somewhere. And this huge list of items (be them inode or else) really slows down directory listing and scan, for example when I perform searches or get properties (counts, stats etc) -- 1) Resistere, resistere, resistere. 2) Se tutti pagano le tasse, le tasse le pagano tutti Soviet_Mario - (aka Gatto_Vizzato)
[toc] | [prev] | [next] | [standalone]
| From | Dan Purgert <dan@djph.net> |
|---|---|
| Date | 2019-02-27 16:02 +0000 |
| Message-ID | <slrnq7dd48.3t0.dan@djph.net> |
| In reply to | #54246 |
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Soviet_Mario wrote: > On 27/02/19 13:44, Aragorn wrote: >> On Wed, 27 Feb 2019 12:52:04 +0100, Soviet_Mario scribbled: >> >>> On 26/02/19 22:46, William Unruh wrote: >>> >>>> Something like rsnapshot (using rsync) will store about 30 backups of >>>> 200GB of data in half a TB. (it does differential backups so only files >>>> which changed get backed up). >>>> >>>> Not tarring or compressing means that the backup data is easy to get at >>>> which is usually what you want. >>> >>> I call this inode space pollution, a thing that I don't find desirable >>> on the backup target (even if I have to accept it in the userspace). >>> It slows down searches, takes up space in itself just to store a huge >>> number of inodes :\ >> >> rsnapshot uses hard-linking for files that haven't changed since the last >> backup. Hard links don't use up any inodes. >> > > mmm, then I must have chosen the wrong terms. > But some other kind of disk space must be used to add a link Nope. > to somewhere. And this huge list of items (be them inode or > else) really slows down directory listing and scan, for > example when I perform searches or get properties (counts, > stats etc) Which'll be as equally slow, if not slower, when you try doing it on a tarball. > -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEBcqaUD8uEzVNxUrujhHd8xJ5ooEFAlx2tIcACgkQjhHd8xJ5 ooHDKgf/VM4oQseTinWNvAvJbPnO5ipuFGBOy4uNARSdhB7nxgIH7u7yCHDhcXB9 wc86oN8Niahv6e+k8MR37o56rMukvr4VJ/3ZkqPIDDImTIZpGcBFyPzSMxWmTETR 16iv0s8dnWKRyOXJb0K+43uCxY+LU/PL/aCrCOGXsUm4Bb7eYyvQJrU1X720VsvN G6Z+NQMefyINCklDqWjXwKEyB69Ux3teeZSp/SaqLQl68zL0EZQoWoZCeoUEgjZ5 4/EDYL7Lhrokwwb1OI+RGnhkY8xBny05bw5aP6aTKz+DtXZq4ALodFVs9y56gX0J A4YGoaATSdCI94KigZbPaqwjB9sLTQ== =KZ0n -----END PGP SIGNATURE----- -- |_|O|_| |_|_|O| Github: https://github.com/dpurgert |O|O|O| PGP: 05CA 9A50 3F2E 1335 4DC5 4AEE 8E11 DDF3 1279 A281
[toc] | [prev] | [next] | [standalone]
| From | Soviet_Mario <SovietMario@CCCP.MIR> |
|---|---|
| Date | 2019-02-27 20:46 +0100 |
| Message-ID | <q56pf6$1l5$1@dont-email.me> |
| In reply to | #54247 |
On 27/02/19 17:02, Dan Purgert wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Soviet_Mario wrote: >> On 27/02/19 13:44, Aragorn wrote: >>> On Wed, 27 Feb 2019 12:52:04 +0100, Soviet_Mario scribbled: >>> >>>> On 26/02/19 22:46, William Unruh wrote: >>>> >>>>> Something like rsnapshot (using rsync) will store about 30 backups of >>>>> 200GB of data in half a TB. (it does differential backups so only files >>>>> which changed get backed up). >>>>> >>>>> Not tarring or compressing means that the backup data is easy to get at >>>>> which is usually what you want. >>>> >>>> I call this inode space pollution, a thing that I don't find desirable >>>> on the backup target (even if I have to accept it in the userspace). >>>> It slows down searches, takes up space in itself just to store a huge >>>> number of inodes :\ >>> >>> rsnapshot uses hard-linking for files that haven't changed since the last >>> backup. Hard links don't use up any inodes. >>> >> >> mmm, then I must have chosen the wrong terms. >> But some other kind of disk space must be used to add a link > > Nope. so the link information is stored where ? I'm not saying the content need to be stored, but the link to it for sure : in as many places as there are the links This does not duplicate the info of Inode (Aragorns taught), but then the link is a reference to be dereferenced. So when the file manager scans the disk, re-read the same inode many times or at best looks-up some hash table to find whether or not the info is just present. > >> to somewhere. And this huge list of items (be them inode or >> else) really slows down directory listing and scan, for >> example when I perform searches or get properties (counts, >> stats etc) > > Which'll be as equally slow, if not slower, when you try doing it on a > tarball. If one has just a huge one, for sure. That's why I wanted tarbal:folder=1:1 > >> > > > -----BEGIN PGP SIGNATURE----- > > iQEzBAEBCAAdFiEEBcqaUD8uEzVNxUrujhHd8xJ5ooEFAlx2tIcACgkQjhHd8xJ5 > ooHDKgf/VM4oQseTinWNvAvJbPnO5ipuFGBOy4uNARSdhB7nxgIH7u7yCHDhcXB9 > wc86oN8Niahv6e+k8MR37o56rMukvr4VJ/3ZkqPIDDImTIZpGcBFyPzSMxWmTETR > 16iv0s8dnWKRyOXJb0K+43uCxY+LU/PL/aCrCOGXsUm4Bb7eYyvQJrU1X720VsvN > G6Z+NQMefyINCklDqWjXwKEyB69Ux3teeZSp/SaqLQl68zL0EZQoWoZCeoUEgjZ5 > 4/EDYL7Lhrokwwb1OI+RGnhkY8xBny05bw5aP6aTKz+DtXZq4ALodFVs9y56gX0J > A4YGoaATSdCI94KigZbPaqwjB9sLTQ== > =KZ0n > -----END PGP SIGNATURE----- > -- 1) Resistere, resistere, resistere. 2) Se tutti pagano le tasse, le tasse le pagano tutti Soviet_Mario - (aka Gatto_Vizzato)
[toc] | [prev] | [next] | [standalone]
| From | William Unruh <unruh@invalid.ca> |
|---|---|
| Date | 2019-02-27 20:18 +0000 |
| Message-ID | <q56rag$bd9$3@dont-email.me> |
| In reply to | #54250 |
On 2019-02-27, Soviet_Mario <SovietMario@CCCP.MIR> wrote: > On 27/02/19 17:02, Dan Purgert wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA256 >> >> Soviet_Mario wrote: >>> On 27/02/19 13:44, Aragorn wrote: >>>> On Wed, 27 Feb 2019 12:52:04 +0100, Soviet_Mario scribbled: >>>> >>>>> On 26/02/19 22:46, William Unruh wrote: >>>>> >>>>>> Something like rsnapshot (using rsync) will store about 30 backups of >>>>>> 200GB of data in half a TB. (it does differential backups so only files >>>>>> which changed get backed up). >>>>>> >>>>>> Not tarring or compressing means that the backup data is easy to get at >>>>>> which is usually what you want. >>>>> >>>>> I call this inode space pollution, a thing that I don't find desirable >>>>> on the backup target (even if I have to accept it in the userspace). >>>>> It slows down searches, takes up space in itself just to store a huge >>>>> number of inodes :\ >>>> >>>> rsnapshot uses hard-linking for files that haven't changed since the last >>>> backup. Hard links don't use up any inodes. >>>> >>> >>> mmm, then I must have chosen the wrong terms. >>> But some other kind of disk space must be used to add a link >> >> Nope. > > so the link information is stored where ? > I'm not saying the content need to be stored, but the link > to it for sure : in as many places as there are the links In the directory entry for that file. The directory entry contains the inode. That file simply points to that inode. That it is the same inode that some other entry points to as well, well,that is what a hard link means (in the inode structure is a number which says how many directory entries point to it. > > This does not duplicate the info of Inode (Aragorns taught), > but then the link is a reference to be dereferenced. So when That is a soft link. Here you just have an ordinary directory entry which points to the intial inode. > the file manager scans the disk, re-read the same inode many > times or at best looks-up some hash table to find whether or > not the info is just present. Reading an inode entry is fast. > > >> >>> to somewhere. And this huge list of items (be them inode or >>> else) really slows down directory listing and scan, for >>> example when I perform searches or get properties (counts, >>> stats etc) >> >> Which'll be as equally slow, if not slower, when you try doing it on a >> tarball. > > If one has just a huge one, for sure. That's why I wanted > tarbal:folder=1:1 NO. You have to uncompress the tarball, then search through the huge contents of that tarball to find the entry you want. That is slow. Very slow. > > >> >>> >> >> >> -----BEGIN PGP SIGNATURE----- >> >> iQEzBAEBCAAdFiEEBcqaUD8uEzVNxUrujhHd8xJ5ooEFAlx2tIcACgkQjhHd8xJ5 >> ooHDKgf/VM4oQseTinWNvAvJbPnO5ipuFGBOy4uNARSdhB7nxgIH7u7yCHDhcXB9 >> wc86oN8Niahv6e+k8MR37o56rMukvr4VJ/3ZkqPIDDImTIZpGcBFyPzSMxWmTETR >> 16iv0s8dnWKRyOXJb0K+43uCxY+LU/PL/aCrCOGXsUm4Bb7eYyvQJrU1X720VsvN >> G6Z+NQMefyINCklDqWjXwKEyB69Ux3teeZSp/SaqLQl68zL0EZQoWoZCeoUEgjZ5 >> 4/EDYL7Lhrokwwb1OI+RGnhkY8xBny05bw5aP6aTKz+DtXZq4ALodFVs9y56gX0J >> A4YGoaATSdCI94KigZbPaqwjB9sLTQ== >> =KZ0n >> -----END PGP SIGNATURE----- >> > >
[toc] | [prev] | [next] | [standalone]
| From | Soviet_Mario <SovietMario@CCCP.MIR> |
|---|---|
| Date | 2019-02-28 12:47 +0100 |
| Message-ID | <q58ho5$62k$1@dont-email.me> |
| In reply to | #54252 |
On 27/02/19 21:18, William Unruh wrote: > On 2019-02-27, Soviet_Mario <SovietMario@CCCP.MIR> wrote: >> On 27/02/19 17:02, Dan Purgert wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA256 >>> >>> Soviet_Mario wrote: >>>> On 27/02/19 13:44, Aragorn wrote: >>>>> On Wed, 27 Feb 2019 12:52:04 +0100, Soviet_Mario scribbled: >>>>> >>>>>> On 26/02/19 22:46, William Unruh wrote: >>>>>> >>>>>>> Something like rsnapshot (using rsync) will store about 30 backups of >>>>>>> 200GB of data in half a TB. (it does differential backups so only files >>>>>>> which changed get backed up). >>>>>>> >>>>>>> Not tarring or compressing means that the backup data is easy to get at >>>>>>> which is usually what you want. >>>>>> >>>>>> I call this inode space pollution, a thing that I don't find desirable >>>>>> on the backup target (even if I have to accept it in the userspace). >>>>>> It slows down searches, takes up space in itself just to store a huge >>>>>> number of inodes :\ >>>>> >>>>> rsnapshot uses hard-linking for files that haven't changed since the last >>>>> backup. Hard links don't use up any inodes. >>>>> >>>> >>>> mmm, then I must have chosen the wrong terms. >>>> But some other kind of disk space must be used to add a link >>> >>> Nope. >> >> so the link information is stored where ? >> I'm not saying the content need to be stored, but the link >> to it for sure : in as many places as there are the links > > In the directory entry for that file. The directory entry contains the > inode. That file simply points to that inode. That it is the same inode > that some other entry points to as well, well,that is what a hard link > means (in the inode structure is a number which says how many directory > entries point to it. > >> >> This does not duplicate the info of Inode (Aragorns taught), >> but then the link is a reference to be dereferenced. So when > > That is a soft link. Here you just have an ordinary directory entry > which points to the intial inode. No you have as many ordinary directory entries as the number of hardlinks, which is what I refer to as name pollution. It's a situation somewhat similar (a metaphore obv.) to enclosing names in a program in namespaces to avoid global name space pollution (in the example an archive hides the names inside it and result in a single entry) Let me precise I'm not absolutely saying it is faster to lookup for a name inside it instead of a regular entry ! I'm just saying that wide range operations (such as getting the disk usage maps and enumerating trees elements like folders) are. Also copying / moving whole large top level branch becomes faster as (apart from reduced compressed size) the number of files is shrinked. If one looks for a single file and knows where it is in advance, the regular hardlink style is faster > >> the file manager scans the disk, re-read the same inode many >> times or at best looks-up some hash table to find whether or >> not the info is just present. Reading an inode entry is fast. > >> >> >>> >>>> to somewhere. And this huge list of items (be them inode or >>>> else) really slows down directory listing and scan, for >>>> example when I perform searches or get properties (counts, >>>> stats etc) >>> >>> Which'll be as equally slow, if not slower, when you try doing it on a >>> tarball. >> >> If one has just a huge one, for sure. That's why I wanted >> tarbal:folder=1:1 > > NO. You have to uncompress the tarball, then search through the huge it's not so huge. But yes, in case I search for a particular item it is slower, I didn't mean that > contents of that tarball to find the entry you want. That is slow. Very > slow. > >> >> >>> >>>> >>> >>> >>> -----BEGIN PGP SIGNATURE----- >>> >>> iQEzBAEBCAAdFiEEBcqaUD8uEzVNxUrujhHd8xJ5ooEFAlx2tIcACgkQjhHd8xJ5 >>> ooHDKgf/VM4oQseTinWNvAvJbPnO5ipuFGBOy4uNARSdhB7nxgIH7u7yCHDhcXB9 >>> wc86oN8Niahv6e+k8MR37o56rMukvr4VJ/3ZkqPIDDImTIZpGcBFyPzSMxWmTETR >>> 16iv0s8dnWKRyOXJb0K+43uCxY+LU/PL/aCrCOGXsUm4Bb7eYyvQJrU1X720VsvN >>> G6Z+NQMefyINCklDqWjXwKEyB69Ux3teeZSp/SaqLQl68zL0EZQoWoZCeoUEgjZ5 >>> 4/EDYL7Lhrokwwb1OI+RGnhkY8xBny05bw5aP6aTKz+DtXZq4ALodFVs9y56gX0J >>> A4YGoaATSdCI94KigZbPaqwjB9sLTQ== >>> =KZ0n >>> -----END PGP SIGNATURE----- >>> >> >> -- 1) Resistere, resistere, resistere. 2) Se tutti pagano le tasse, le tasse le pagano tutti Soviet_Mario - (aka Gatto_Vizzato)
[toc] | [prev] | [next] | [standalone]
| From | William Unruh <unruh@invalid.ca> |
|---|---|
| Date | 2019-02-28 18:32 +0000 |
| Message-ID | <q599fd$nm9$3@dont-email.me> |
| In reply to | #54260 |
On 2019-02-28, Soviet_Mario <SovietMario@CCCP.MIR> wrote: > On 27/02/19 21:18, William Unruh wrote: >> On 2019-02-27, Soviet_Mario <SovietMario@CCCP.MIR> wrote: >>> On 27/02/19 17:02, Dan Purgert wrote: >>>> -----BEGIN PGP SIGNED MESSAGE----- >>>> Hash: SHA256 >>>> >>>> Soviet_Mario wrote: >>>>> On 27/02/19 13:44, Aragorn wrote: >>>>>> On Wed, 27 Feb 2019 12:52:04 +0100, Soviet_Mario scribbled: >>>>>> >>>>>>> On 26/02/19 22:46, William Unruh wrote: >>>>>>> >>>>>>>> Something like rsnapshot (using rsync) will store about 30 backups of >>>>>>>> 200GB of data in half a TB. (it does differential backups so only files >>>>>>>> which changed get backed up). >>>>>>>> >>>>>>>> Not tarring or compressing means that the backup data is easy to get at >>>>>>>> which is usually what you want. >>>>>>> >>>>>>> I call this inode space pollution, a thing that I don't find desirable >>>>>>> on the backup target (even if I have to accept it in the userspace). >>>>>>> It slows down searches, takes up space in itself just to store a huge >>>>>>> number of inodes :\ >>>>>> >>>>>> rsnapshot uses hard-linking for files that haven't changed since the last >>>>>> backup. Hard links don't use up any inodes. >>>>>> >>>>> >>>>> mmm, then I must have chosen the wrong terms. >>>>> But some other kind of disk space must be used to add a link >>>> >>>> Nope. >>> >>> so the link information is stored where ? >>> I'm not saying the content need to be stored, but the link >>> to it for sure : in as many places as there are the links >> >> In the directory entry for that file. The directory entry contains the >> inode. That file simply points to that inode. That it is the same inode >> that some other entry points to as well, well,that is what a hard link >> means (in the inode structure is a number which says how many directory >> entries point to it. >> >>> >>> This does not duplicate the info of Inode (Aragorns taught), >>> but then the link is a reference to be dereferenced. So when >> >> That is a soft link. Here you just have an ordinary directory entry >> which points to the intial inode. > > No you have as many ordinary directory entries as the number > of hardlinks, which is what I refer to as name pollution. Yes. In different directories. > > It's a situation somewhat similar (a metaphore obv.) to > enclosing names in a program in namespaces to avoid global > name space pollution (in the example an archive hides the > names inside it and result in a single entry) No idea what this analogy is supposed to be sorry. > > Let me precise I'm not absolutely saying it is faster to > lookup for a name inside it instead of a regular entry ! > I'm just saying that wide range operations (such as getting > the disk usage maps and enumerating trees elements like > folders) are. Nope. > > Also copying / moving whole large top level branch becomes > faster as (apart from reduced compressed size) the number of > files is shrinked. Nope. You just move the large top level branch. and if you have an intelligent move operation, like rsync -H it moves every inode only once. If you have a stupid one, like mv, then it will dereference the hard links and move them (across filesystem boundaries that is) separately. On the same filesystem all mv does is change the directory entries of the top level of the branch. > > If one looks for a single file and knows where it is in > advance, the regular hardlink style is faster And if one does the looking is an intelligent way (eg lists the inode entries of those with multiple hard links) one only needs to read each inode once. > >> >>> the file manager scans the disk, re-read the same inode many >>> times or at best looks-up some hash table to find whether or >>> not the info is just present. Reading an inode entry is fast. >> >>> >>> >>>> >>>>> to somewhere. And this huge list of items (be them inode or >>>>> else) really slows down directory listing and scan, for >>>>> example when I perform searches or get properties (counts, >>>>> stats etc) >>>> >>>> Which'll be as equally slow, if not slower, when you try doing it on a >>>> tarball. >>> >>> If one has just a huge one, for sure. That's why I wanted >>> tarbal:folder=1:1 >> >> NO. You have to uncompress the tarball, then search through the huge > > it's not so huge. But yes, in case I search for a particular > item it is slower, I didn't mean that Sorry, up above you were complaining about the hard links being slower and not you do not meanthat? What do you mean? > >> contents of that tarball to find the entry you want. That is slow. Very >> slow. >> >>> >>> >>>> >>>>> >>>> >>>> >>>> -----BEGIN PGP SIGNATURE----- >>>> >>>> iQEzBAEBCAAdFiEEBcqaUD8uEzVNxUrujhHd8xJ5ooEFAlx2tIcACgkQjhHd8xJ5 >>>> ooHDKgf/VM4oQseTinWNvAvJbPnO5ipuFGBOy4uNARSdhB7nxgIH7u7yCHDhcXB9 >>>> wc86oN8Niahv6e+k8MR37o56rMukvr4VJ/3ZkqPIDDImTIZpGcBFyPzSMxWmTETR >>>> 16iv0s8dnWKRyOXJb0K+43uCxY+LU/PL/aCrCOGXsUm4Bb7eYyvQJrU1X720VsvN >>>> G6Z+NQMefyINCklDqWjXwKEyB69Ux3teeZSp/SaqLQl68zL0EZQoWoZCeoUEgjZ5 >>>> 4/EDYL7Lhrokwwb1OI+RGnhkY8xBny05bw5aP6aTKz+DtXZq4ALodFVs9y56gX0J >>>> A4YGoaATSdCI94KigZbPaqwjB9sLTQ== >>>> =KZ0n >>>> -----END PGP SIGNATURE----- >>>> >>> >>> > >
[toc] | [prev] | [next] | [standalone]
| From | Soviet_Mario <SovietMario@CCCP.MIR> |
|---|---|
| Date | 2019-03-01 01:14 +0100 |
| Message-ID | <q59tht$r40$1@dont-email.me> |
| In reply to | #54264 |
On 28/02/19 19:32, William Unruh wrote: > On 2019-02-28, Soviet_Mario <SovietMario@CCCP.MIR> wrote: >> On 27/02/19 21:18, William Unruh wrote: >>> On 2019-02-27, Soviet_Mario <SovietMario@CCCP.MIR> wrote: >>>> On 27/02/19 17:02, Dan Purgert wrote: >>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>> Hash: SHA256 >>>>> >>>>> Soviet_Mario wrote: >>>>>> On 27/02/19 13:44, Aragorn wrote: >>>>>>> On Wed, 27 Feb 2019 12:52:04 +0100, Soviet_Mario scribbled: >>>>>>> >>>>>>>> On 26/02/19 22:46, William Unruh wrote: >>>>>>>> >>>>>>>>> Something like rsnapshot (using rsync) will store about 30 backups of >>>>>>>>> 200GB of data in half a TB. (it does differential backups so only files >>>>>>>>> which changed get backed up). >>>>>>>>> >>>>>>>>> Not tarring or compressing means that the backup data is easy to get at >>>>>>>>> which is usually what you want. >>>>>>>> >>>>>>>> I call this inode space pollution, a thing that I don't find desirable >>>>>>>> on the backup target (even if I have to accept it in the userspace). >>>>>>>> It slows down searches, takes up space in itself just to store a huge >>>>>>>> number of inodes :\ >>>>>>> >>>>>>> rsnapshot uses hard-linking for files that haven't changed since the last >>>>>>> backup. Hard links don't use up any inodes. >>>>>>> >>>>>> >>>>>> mmm, then I must have chosen the wrong terms. >>>>>> But some other kind of disk space must be used to add a link >>>>> >>>>> Nope. >>>> >>>> so the link information is stored where ? >>>> I'm not saying the content need to be stored, but the link >>>> to it for sure : in as many places as there are the links >>> >>> In the directory entry for that file. The directory entry contains the >>> inode. That file simply points to that inode. That it is the same inode >>> that some other entry points to as well, well,that is what a hard link >>> means (in the inode structure is a number which says how many directory >>> entries point to it. >>> >>>> >>>> This does not duplicate the info of Inode (Aragorns taught), >>>> but then the link is a reference to be dereferenced. So when >>> >>> That is a soft link. Here you just have an ordinary directory entry >>> which points to the intial inode. >> >> No you have as many ordinary directory entries as the number >> of hardlinks, which is what I refer to as name pollution. > > Yes. In different directories. >> >> It's a situation somewhat similar (a metaphore obv.) to >> enclosing names in a program in namespaces to avoid global >> name space pollution (in the example an archive hides the >> names inside it and result in a single entry) > > No idea what this analogy is supposed to be sorry. > >> >> Let me precise I'm not absolutely saying it is faster to >> lookup for a name inside it instead of a regular entry ! >> I'm just saying that wide range operations (such as getting >> the disk usage maps and enumerating trees elements like >> folders) are. > > Nope. > >> >> Also copying / moving whole large top level branch becomes >> faster as (apart from reduced compressed size) the number of >> files is shrinked. > > Nope. You just move the large top level branch. and if you have an > intelligent move operation, like rsync -H it moves every inode only > once. If you have a stupid one, like mv, then it will dereference the > hard links and move them (across filesystem boundaries that is) > separately. On the same filesystem all mv does is change the directory > entries of the top level of the branch. > i had taken for granted that the move operation were between different physical disks (as for safe backups). I have empiric proof that with every OS, moving 100 G consisting of few files or millions of files takes up much different time > >> >> If one looks for a single file and knows where it is in >> advance, the regular hardlink style is faster > > And if one does the looking is an intelligent way (eg lists the inode > entries of those with multiple hard links) one only needs to read each > inode once. > > > >> >>> >>>> the file manager scans the disk, re-read the same inode many >>>> times or at best looks-up some hash table to find whether or >>>> not the info is just present. Reading an inode entry is fast. >>> >>>> >>>> >>>>> >>>>>> to somewhere. And this huge list of items (be them inode or >>>>>> else) really slows down directory listing and scan, for >>>>>> example when I perform searches or get properties (counts, >>>>>> stats etc) >>>>> >>>>> Which'll be as equally slow, if not slower, when you try doing it on a >>>>> tarball. >>>> >>>> If one has just a huge one, for sure. That's why I wanted >>>> tarbal:folder=1:1 >>> >>> NO. You have to uncompress the tarball, then search through the huge >> >> it's not so huge. But yes, in case I search for a particular >> item it is slower, I didn't mean that > > Sorry, up above you were complaining about the hard links being slower > and not you do not meanthat? What do you mean? > >> >>> contents of that tarball to find the entry you want. That is slow. Very >>> slow. >>> >>>> >>>> >>>>> >>>>>> >>>>> >>>>> >>>>> -----BEGIN PGP SIGNATURE----- >>>>> >>>>> iQEzBAEBCAAdFiEEBcqaUD8uEzVNxUrujhHd8xJ5ooEFAlx2tIcACgkQjhHd8xJ5 >>>>> ooHDKgf/VM4oQseTinWNvAvJbPnO5ipuFGBOy4uNARSdhB7nxgIH7u7yCHDhcXB9 >>>>> wc86oN8Niahv6e+k8MR37o56rMukvr4VJ/3ZkqPIDDImTIZpGcBFyPzSMxWmTETR >>>>> 16iv0s8dnWKRyOXJb0K+43uCxY+LU/PL/aCrCOGXsUm4Bb7eYyvQJrU1X720VsvN >>>>> G6Z+NQMefyINCklDqWjXwKEyB69Ux3teeZSp/SaqLQl68zL0EZQoWoZCeoUEgjZ5 >>>>> 4/EDYL7Lhrokwwb1OI+RGnhkY8xBny05bw5aP6aTKz+DtXZq4ALodFVs9y56gX0J >>>>> A4YGoaATSdCI94KigZbPaqwjB9sLTQ== >>>>> =KZ0n >>>>> -----END PGP SIGNATURE----- >>>>> >>>> >>>> >> >> -- 1) Resistere, resistere, resistere. 2) Se tutti pagano le tasse, le tasse le pagano tutti Soviet_Mario - (aka Gatto_Vizzato)
[toc] | [prev] | [next] | [standalone]
| From | "Carlos E.R." <robin_listas@es.invalid> |
|---|---|
| Date | 2019-03-01 01:54 +0100 |
| Message-ID | <jfcmkf-cgj.ln1@Telcontar.valinor> |
| In reply to | #54265 |
On 01/03/2019 01.14, Soviet_Mario wrote: > On 28/02/19 19:32, William Unruh wrote: >>> Also copying / moving whole large top level branch becomes >>> faster as (apart from reduced compressed size) the number of >>> files is shrinked. >>> >> Nope. You just move the large top level branch. and if you have an >> intelligent move operation, like rsync -H it moves every inode only >> once. If you have a stupid one, like mv, then it will dereference the >> hard links and move them (across filesystem boundaries that is) >> separately. On the same filesystem all mv does is change the directory >> entries of the top level of the branch. >>> > > i had taken for granted that the move operation were between different > physical disks (as for safe backups). > > I have empiric proof that with every OS, moving 100 G consisting of few > files or millions of files takes up much different time Moving files across different filesystems, which in Linux can mean different disks or different partitions (same thing), is basically a file copy operation. The original file is *copied* to the new destination, which entails finding a location, updating directory entry, and filing the inode, plus copying the actual data. Finally, the original file is removed. Of course, moving many files is slower because many directory entries and inodes have to be written, not only the file contents. This is many head movements. This is very different from moving files inside the same filesystem. In that case only directory entries are changed, the inode data is the same. It can be done by creating a hard link of the file at the destination, then deleting the original "hard link". -- Cheers, Carlos.
[toc] | [prev] | [next] | [standalone]
| From | Jasen Betts <jasen@xnet.co.nz> |
|---|---|
| Date | 2019-03-01 03:09 +0000 |
| Message-ID | <q5a7pc$lsi$2@gonzo.revmaps.no-ip.org> |
| In reply to | #54260 |
On 2019-02-28, Soviet_Mario <SovietMario@CCCP.MIR> wrote: > On 27/02/19 21:18, William Unruh wrote: >> On 2019-02-27, Soviet_Mario <SovietMario@CCCP.MIR> wrote: >>> On 27/02/19 17:02, Dan Purgert wrote: >>>> -----BEGIN PGP SIGNED MESSAGE----- >>>> Hash: SHA256 >>> This does not duplicate the info of Inode (Aragorns taught), >>> but then the link is a reference to be dereferenced. So when >> >> That is a soft link. Here you just have an ordinary directory entry >> which points to the intial inode. > > No you have as many ordinary directory entries as the number > of hardlinks, which is what I refer to as name pollution. > > It's a situation somewhat similar (a metaphore obv.) to > enclosing names in a program in namespaces to avoid global > name space pollution (in the example an archive hides the > names inside it and result in a single entry) > > Let me precise I'm not absolutely saying it is faster to > lookup for a name inside it instead of a regular entry ! > I'm just saying that wide range operations (such as getting > the disk usage maps and enumerating trees elements like > folders) are. yeah, those things don't work well. > Also copying / moving whole large top level branch becomes > faster as (apart from reduced compressed size) the number of > files is shrinked. yeah if there's ever a reason to do that. > If one looks for a single file and knows where it is in > advance, the regular hardlink style is faster yeah, and you don't have to guess which partial backup it might be in. you get a kind of compression because the tuesday files overlap the monday files etc, so only the differences need to be added to the disk. Once you have a full set you can re-use the directory holding the expiring backup to hold the new backup. this makes the backup process even faster as files that have not changed since the expiring backup don't need to be touched at all. (files have changed but that match the previous backup only need a hard link which is also fast) -- When I tried casting out nines I made a hash of it.
[toc] | [prev] | [next] | [standalone]
| From | Aragorn <thorongil@telenet.be> |
|---|---|
| Date | 2019-02-27 20:21 +0000 |
| Message-ID | <q56rgs$eab$1@dont-email.me> |
| In reply to | #54250 |
On Wed, 27 Feb 2019 20:46:45 +0100, Soviet_Mario scribbled: > On 27/02/19 17:02, Dan Purgert wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA256 >> >> Soviet_Mario wrote: >>> On 27/02/19 13:44, Aragorn wrote: >>>> On Wed, 27 Feb 2019 12:52:04 +0100, Soviet_Mario scribbled: >>>> >>>>> On 26/02/19 22:46, William Unruh wrote: >>>>> >>>>>> Something like rsnapshot (using rsync) will store about 30 backups >>>>>> of 200GB of data in half a TB. (it does differential backups so >>>>>> only files which changed get backed up). >>>>>> >>>>>> Not tarring or compressing means that the backup data is easy to >>>>>> get at which is usually what you want. >>>>> >>>>> I call this inode space pollution, a thing that I don't find >>>>> desirable on the backup target (even if I have to accept it in the >>>>> userspace). It slows down searches, takes up space in itself just to >>>>> store a huge number of inodes :\ >>>> >>>> rsnapshot uses hard-linking for files that haven't changed since the >>>> last backup. Hard links don't use up any inodes. >>>> >>>> >>> mmm, then I must have chosen the wrong terms. >>> But some other kind of disk space must be used to add a link >> >> Nope. > > so the link information is stored where ? In the inode itself. > I'm not saying the content need to be stored, but the link to it for > sure : in as many places as there are the links > > This does not duplicate the info of Inode (Aragorns taught), > but then the link is a reference to be dereferenced. No more or no less than the original link. > So when the file manager scans the disk, re-read the same inode many > times or at best looks-up some hash table to find whether or not the > info is just present. Nothing of that sort. The link is stored in the inode's link counter itself. When the link is zero, the file's extents are released for reuse by other files, i.e. the file is considered deleted. A single file can have a hundred hard links, and it still wouldn't take any more time to read than if it had only one link. -- With respect, = Aragorn =
[toc] | [prev] | [next] | [standalone]
| From | William Unruh <unruh@invalid.ca> |
|---|---|
| Date | 2019-02-27 23:02 +0000 |
| Message-ID | <q574v2$8h7$1@dont-email.me> |
| In reply to | #54253 |
On 2019-02-27, Aragorn <thorongil@telenet.be> wrote: > On Wed, 27 Feb 2019 20:46:45 +0100, Soviet_Mario scribbled: > >> On 27/02/19 17:02, Dan Purgert wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA256 >>> >>> Soviet_Mario wrote: >>>> On 27/02/19 13:44, Aragorn wrote: >>>>> On Wed, 27 Feb 2019 12:52:04 +0100, Soviet_Mario scribbled: >>>>> >>>>>> On 26/02/19 22:46, William Unruh wrote: >>>>>> >>>>>>> Something like rsnapshot (using rsync) will store about 30 backups >>>>>>> of 200GB of data in half a TB. (it does differential backups so >>>>>>> only files which changed get backed up). >>>>>>> >>>>>>> Not tarring or compressing means that the backup data is easy to >>>>>>> get at which is usually what you want. >>>>>> >>>>>> I call this inode space pollution, a thing that I don't find >>>>>> desirable on the backup target (even if I have to accept it in the >>>>>> userspace). It slows down searches, takes up space in itself just to >>>>>> store a huge number of inodes :\ >>>>> >>>>> rsnapshot uses hard-linking for files that haven't changed since the >>>>> last backup. Hard links don't use up any inodes. >>>>> >>>>> >>>> mmm, then I must have chosen the wrong terms. >>>> But some other kind of disk space must be used to add a link >>> >>> Nope. >> >> so the link information is stored where ? > > In the inode itself. > >> I'm not saying the content need to be stored, but the link to it for >> sure : in as many places as there are the links >> >> This does not duplicate the info of Inode (Aragorns taught), >> but then the link is a reference to be dereferenced. > > No more or no less than the original link. > >> So when the file manager scans the disk, re-read the same inode many >> times or at best looks-up some hash table to find whether or not the >> info is just present. > > Nothing of that sort. The link is stored in the inode's link counter > itself. When the link is zero, the file's extents are released for reuse > by other files, i.e. the file is considered deleted. A single file can > have a hundred hard links, and it still wouldn't take any more time to > read than if it had only one link. Of course if you sequentially read the hundred files which all point to the same inode, it will take 100 times as long as reading from one of those files. But that would also be true if you read the same file 100 times, because that is what you are doing. (I am neglecting buffering, etc) It has never been entirely clear to me if the inode saves a reference to the files pointing to it. Ie, if you know the inode, can you determine all of the files that are hard linked to it, without haveing to scan the whole directory structure to fine those files that point to it. For example, rsync can preserve hard links. How does it know that file B is a hardlink to a file already read as file A, if it does? >
[toc] | [prev] | [next] | [standalone]
| From | Richard Kettlewell <invalid@invalid.invalid> |
|---|---|
| Date | 2019-02-27 23:24 +0000 |
| Message-ID | <878sy0yfvz.fsf@LkoBDZeT.terraraq.uk> |
| In reply to | #54254 |
William Unruh <unruh@invalid.ca> writes: > It has never been entirely clear to me if the inode saves a reference > to the files pointing to it. Ie, if you know the inode, can you > determine all of the files that are hard linked to it, without haveing > to scan the whole directory structure to fine those files that point > to it. No, the inode->path mapping is not stored on disk. > For example, rsync can preserve hard links. How does it know that file > B is a hardlink to a file already read as file A, if it does? I’ve not looked at this bit of rsync but the usual approach is to remember (dev,inode,path) for any file you encounter with a link count above 1; if you see a matching (dev,inode) for some subsequent path then you link instead of copying. -- https://www.greenend.org.uk/rjk/
[toc] | [prev] | [next] | [standalone]
Page 1 of 4 [1] 2 3 4 Next page →
Back to top | Article view | alt.os.linux
csiph-web