Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.misc > #27148 > unrolled thread

tip: sweet grep recurse feature

Back to article view | Back to comp.os.linux.misc

Contents

  tip: sweet grep recurse feature T <T@invalid.invalid> - 2019-06-13 03:15 -0700
    Re: tip: sweet grep recurse feature Javier <invalid@invalid.invalid> - 2019-06-13 08:46 -0500
      Re: tip: sweet grep recurse feature Jean-David Beyer <jeandavid8@verizon.net> - 2019-06-13 09:51 -0400
        Re: tip: sweet grep recurse feature Grant Taylor <gtaylor@tnetconsulting.net> - 2019-06-13 09:17 -0600
      Re: tip: sweet grep recurse feature Dan Espen <dan1espen@gmail.com> - 2019-06-13 11:22 -0400
        Re: tip: sweet grep recurse feature Grant Taylor <gtaylor@tnetconsulting.net> - 2019-06-13 09:56 -0600
        Re: tip: sweet grep recurse feature Richard Kettlewell <invalid@invalid.invalid> - 2019-06-13 17:29 +0100
          Re: tip: sweet grep recurse feature Dan Espen <dan1espen@gmail.com> - 2019-06-13 15:38 -0400
            Re: tip: sweet grep recurse feature Richard Kettlewell <invalid@invalid.invalid> - 2019-06-14 09:49 +0100
              Re: tip: sweet grep recurse feature Javier <invalid@invalid.invalid> - 2019-06-14 04:39 -0500
                Re: tip: sweet grep recurse feature "Carlos E.R." <robin_listas@es.invalid> - 2019-06-14 12:11 +0200
                  Re: tip: sweet grep recurse feature Javier <invalid@invalid.invalid> - 2019-06-14 07:11 -0500
                    Re: tip: sweet grep recurse feature Jean-David Beyer <jeandavid8@verizon.net> - 2019-06-14 09:07 -0400
                    Re: tip: sweet grep recurse feature Grant Taylor <gtaylor@tnetconsulting.net> - 2019-06-14 12:44 -0600
                      Re: tip: sweet grep recurse feature Robert Heller <heller@deepsoft.com> - 2019-06-14 14:20 -0500
                        Re: tip: sweet grep recurse feature Eli the Bearded <*@eli.users.panix.com> - 2019-06-15 03:18 +0000
                          Re: tip: sweet grep recurse feature Robert Heller <heller@deepsoft.com> - 2019-06-14 23:05 -0500
                          Re: tip: sweet grep recurse feature Anssi Saari <as@sci.fi> - 2019-06-17 09:44 +0300
                            Re: tip: sweet grep recurse feature Fenris <fenris@invalid.invalid> - 2019-06-17 10:21 +0200
                  Re: tip: sweet grep recurse feature Grant Taylor <gtaylor@tnetconsulting.net> - 2019-06-14 12:50 -0600
                    Re: tip: sweet grep recurse feature "Carlos E.R." <robin_listas@es.invalid> - 2019-06-15 13:37 +0200
                Re: tip: sweet grep recurse feature Dan Espen <dan1espen@gmail.com> - 2019-06-14 08:01 -0400
                  Re: tip: sweet grep recurse feature Grant Taylor <gtaylor@tnetconsulting.net> - 2019-06-14 12:55 -0600
                    Re: tip: sweet grep recurse feature Dan Espen <dan1espen@gmail.com> - 2019-06-14 17:54 -0400
                      Re: tip: sweet grep recurse feature Grant Taylor <gtaylor@tnetconsulting.net> - 2019-06-14 20:20 -0600
                  Re: tip: sweet grep recurse feature "Carlos E.R." <robin_listas@es.invalid> - 2019-06-15 13:42 +0200
                    Re: tip: sweet grep recurse feature Grant Taylor <gtaylor@tnetconsulting.net> - 2019-06-15 11:09 -0600
                      Re: tip: sweet grep recurse feature "Carlos E.R." <robin_listas@es.invalid> - 2019-06-17 12:37 +0200
                        Re: tip: sweet grep recurse feature Aragorn <thorongil@telenet.be> - 2019-06-17 12:51 +0200
                          Re: tip: sweet grep recurse feature "Carlos E.R." <robin_listas@es.invalid> - 2019-06-17 12:57 +0200
                            Re: tip: sweet grep recurse feature Aragorn <thorongil@telenet.be> - 2019-06-17 14:20 +0200
                            Re: tip: sweet grep recurse feature Grant Taylor <gtaylor@tnetconsulting.net> - 2019-06-17 08:51 -0600
                              Re: tip: sweet grep recurse feature Aragorn <thorongil@telenet.be> - 2019-06-17 19:43 +0200
                                Re: tip: sweet grep recurse feature "Carlos E.R." <robin_listas@es.invalid> - 2019-06-17 20:38 +0200
                                Re: tip: sweet grep recurse feature Grant Taylor <gtaylor@tnetconsulting.net> - 2019-06-17 13:07 -0600
                                  Re: tip: sweet grep recurse feature Aragorn <thorongil@telenet.be> - 2019-06-17 21:56 +0200
                                    Re: tip: sweet grep recurse feature Georges <georges@prout.com> - 2019-06-17 22:30 +0200
                                    Re: tip: sweet grep recurse feature "Carlos E.R." <robin_listas@es.invalid> - 2019-06-17 22:19 +0200
                                    Re: tip: sweet grep recurse feature Grant Taylor <gtaylor@tnetconsulting.net> - 2019-06-17 19:43 -0600
                                      Re: tip: sweet grep recurse feature Aragorn <thorongil@telenet.be> - 2019-06-18 10:26 +0200
                                      Re: tip: sweet grep recurse feature Charlie Gibbs <cgibbs@kltpzyxm.invalid> - 2019-06-18 23:58 +0000
              Re: tip: sweet grep recurse feature Dan Espen <dan1espen@gmail.com> - 2019-06-14 08:07 -0400
        Re: tip: sweet grep recurse feature The Natural Philosopher <tnp@invalid.invalid> - 2019-06-13 19:42 +0100
    Re: tip: sweet grep recurse feature Andreas Kohlbach <ank@spamfence.net> - 2019-06-13 14:58 -0400
      Re: tip: sweet grep recurse feature Grant Taylor <gtaylor@tnetconsulting.net> - 2019-06-13 14:29 -0600
        Re: tip: sweet grep recurse feature Andreas Kohlbach <ank@spamfence.net> - 2019-06-13 16:38 -0400
    Re: tip: sweet grep recurse feature Eli the Bearded <*@eli.users.panix.com> - 2019-06-15 03:10 +0000
      Re: tip: sweet grep recurse feature "Carlos E.R." <robin_listas@es.invalid> - 2019-06-15 13:46 +0200
        Re: tip: sweet grep recurse feature Richard Kettlewell <invalid@invalid.invalid> - 2019-06-15 13:16 +0100
          Re: tip: sweet grep recurse feature gazelle@shell.xmission.com (Kenny McCormack) - 2019-06-15 13:16 +0000
            Re: tip: sweet grep recurse feature Grant Taylor <gtaylor@tnetconsulting.net> - 2019-06-15 11:12 -0600
              Re: tip: sweet grep recurse feature Richard Kettlewell <invalid@invalid.invalid> - 2019-06-15 22:26 +0100
            Re: tip: sweet grep recurse feature Eli the Bearded <*@eli.users.panix.com> - 2019-06-15 20:09 +0000
      Re: tip: sweet grep recurse feature T <T@invalid.invalid> - 2019-06-16 14:29 -0700

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

#27176

On 14/06/2019 20.50, Grant Taylor wrote:
> On 6/14/19 4:11 AM, Carlos E.R. wrote:
>> But even so I still prefer that to having to use a dozen rsync program
>> variations like rsync1, rsync2, rsync3 or whatever name.
> 
> I think there is something to be said for code reuse / overlap.
> 
> What percentage of a pair of programs that do similar but slightly
> different things can re-use the same code?  If the reuse / overlap is a
> large enough percentage, then chances are good that the variant could
> qualify as an option to one program.
> 
>> I never know what *grep variation to use for what.
> 
> I think of it this way (using aliases as a demonstration):
> 
> grep
> alias egrep='grep -e'
> alias fgrep='grep -f'
> 
> grep by itself is sort of a default and does a number of different things.
> 
> egrep is grep in a mode expecting / optimized for regular expressions.
> 
> fgrep is grep in a mode expecting fixed text patterns.
> 
> I use different greps depending on what I'm doing and looking for.
> 
> I usually use egrep / fgrep instead of the grep -e / grep -f variants
> because I try to minimize key presses.

I used a lot a variant called cgrep, coming from Lucent (Bell Labs). It
is now open source, although I think it is not the original one. It does
context finding and fuzzy search.

-- 
Cheers, Carlos.

[toc] | [prev] | [next] | [standalone]

#27163

Javier <invalid@invalid.invalid> writes:

> Richard Kettlewell <invalid@invalid.invalid> wrote:
>> Dan Espen <dan1espen@gmail.com> writes:
>>> Richard Kettlewell <invalid@invalid.invalid> writes:
>>>> Dan Espen <dan1espen@gmail.com> writes:
>>>>> Javier <invalid@invalid.invalid> writes:
>>>>>> Brian W. Kernighan does not agree.
>>>>>>
>>>>>> http://harmful.cat-v.org/cat-v/unix_prog_design.pdf
>>>>>>
>>>>>> Programs should have as as few options as possible.  Do one thing and
>>>>>> do it well.  For recursing through dirs you already have find(1).
>>>>>
>>>>> I thought we'd already discovered that BWK was wrong.
>>>>> Perhaps he can explain how having 2 grep programs,
>>>>> perhaps "grep" and "grepr" (for grep recursive) is superior
>>>>> to "grep -r".
>>>>
>>>> I don’t think you can have read the paper, since he doesn’t argue that
>>>> there should be a separate program for each use case.
>>>
>>> I read the paper.
>>> At least last time this came up.
>>> Okay, just scanned the thing again.
>>>
>>> Basically the whole paper is an admonishment about keeping programs
>>> simple,  mainly by not adding options.
>>>
>>> When it finally gets down to it, he concedes that sometimes you add
>>> options:
>>>
>>>   Separate programs  are not  always better than  wider options;  which is
>>>   better depends  on the problem.  Whenever  one needs a way  to perform a
>>>   new function,  one faces the  choice of whether to  add a new  option or
>>>   write a new program...The guiding principle for making the choice should
>>>   be that each  program does one thing. Options are  appropriately added
>>>   to a  program that already has  the right functionality. If  there is no
>>>   such program, then a new program is called for.
>>>
>>> So, he's  really giving us only  the vaguest idea when  we add options
>>> and  when we  write new  programs.  We  find out,  it "depends",  but we
>>> should "do one thing", and you can add options when the program
>>> "already has the right functionality".
>>>
>>> I find the "do one thing" rule impossible to interpret.
>>> How can any program do one thing if it has options?
>>>
>>> BWK addresses program elegance and never mentions user expectations.
>> 
>> By focussing narrowly on options you appear to have completely missed
>> the thrust of the argument, which is (the claim that) loosely coupled
>> composition of logically separate functionality is a better design than
>> combining multiple unrelated functionalities into single programs.
>
> Exactly, that is what BWK advocates.  Coupling simple tools through pipes.
> For the usage case of the OP:
>
>     find /etc/dnf/ \! -type d | xargs grep xclude

Sure looks to me like the perfect proof that adding -r to grep is the
right choice.

> By adding less options the program source code is simpler and its
> manpage thinner.  Otherwise you end up with behemoths like rsync.
> Look at its manpage:
>
> $ man rsync | wc -l
> 3870

rsync still doesn't have enough options.
It needs more options when it can't do it's job with existing options.

I use rsync for backup of $HOME.
When I delete a file in $HOME, I want it deleted from the backup.
But suppose I deleted the file by accident.  I want rsync to wait
a while before it deletes the backup, in my case I want to wait 30
days so I can discover whether I deleted something I need.

So, I recently solved this problem by running rsync a second time
as a dry-run with the delete option.  Then I make a pass through all
the file names returned from this second pass looking at time stamps
and deleting files when they are 30 days old.  This works but would
be so much cleaner if I could just code:

  --delete 30

on the first pass.

Size of the module and smallness of the man page are irrelevant when
you can't do the job.


-- 
Dan Espen

[toc] | [prev] | [next] | [standalone]

#27169

On 6/14/19 6:01 AM, Dan Espen wrote:
> When I delete a file in $HOME, I want it deleted from the backup. 
> But suppose I deleted the file by accident.  I want rsync to wait 
> a while before it deletes the backup, in my case I want to wait 30 
> days so I can discover whether I deleted something I need.

Take a look at BackupPC.

It uses rsync under the hood.  But it does some very interesting things 
on the server.  It will keep different copies for each backup run 
(usually daily) and you can easily go back multiple days.  (You define 
how long.)

BackupPC will hard link (some) files that are identical so that they 
take up less disk space.

I say some because BackupPC has an option for how many files to hard 
link together.  The idea being that no more than that number of backups 
will be compromised if the common hard linked file gets corrupted.  So 
if you have 30 backups, and one file is messed up, it only impacts a 
small number of the backups.  I think it defaults to either 3 or 5.

Seriously, spend a few minutes and check out BackupPC.



-- 
Grant. . . .
unix || die

[toc] | [prev] | [next] | [standalone]

#27171

Grant Taylor <gtaylor@tnetconsulting.net> writes:

> On 6/14/19 6:01 AM, Dan Espen wrote:
>> When I delete a file in $HOME, I want it deleted from the
>> backup. But suppose I deleted the file by accident.  I want rsync to
>> wait a while before it deletes the backup, in my case I want to wait
>> 30 days so I can discover whether I deleted something I need.
>
> Take a look at BackupPC.
>
> It uses rsync under the hood.  But it does some very interesting
> things on the server.  It will keep different copies for each backup
> run (usually daily) and you can easily go back multiple days.  (You
> define how long.)
>
> BackupPC will hard link (some) files that are identical so that they
> take up less disk space.
>
> I say some because BackupPC has an option for how many files to hard
> link together.  The idea being that no more than that number of
> backups will be compromised if the common hard linked file gets
> corrupted.  So if you have 30 backups, and one file is messed up, it
> only impacts a small number of the backups.  I think it defaults to
> either 3 or 5.
>
> Seriously, spend a few minutes and check out BackupPC.

Very interesting, thanks, but it's a layer on top of rsync, so even though it's
touted as "high performance" it won't give me any speed ups.

I don't see any options for a delayed delete of backed up deleted files,
I suppose I'd have to handle that with multiple backups specifying one
that was longer term.  Since my backups are working pretty well,
I'm not inclined to switch.

This AM I got email showing my backup (to a USB stick) failed with
backup file system full.  That seemed odd, since I knew I had plenty
of room on the USB stick.  So, the USB backup directory was
/back-home.  Somehow, /back-home got unmounted so the backup script
was really writing to the "/" file system.  There is no way I have
enough room to copy /home over to "/".  So, now I added this to the
backup script:

blkid -L BACK-HOME >/dev/null 2>&1
if [[ $? != 0 ]] ; then
    echo "USB stick BACK-HOME is not mounted, exiting"
    exit 1
fi

I see I can also look at:

/dev/disk/by-label/BACK-HOME

but blkid seems good enough.

Right now, I'm backing up /home to 128Gib USB stick formatted as ext3.
I also need to back up 7K flac music files, that's just over 128Gib so I need to
invest in a larger USB stick or SD card.  Still trying to decide.

I'd like the music backup to go to FAT for maximum ability to restore.
At first my backups to FAT worked, except it kept trying to back up
files that were already backed up.  A little Google searching told me I
need to mount FAT with the options:

shortname=mixed,iocharset=utf8

and then run rsync with the option:

--modify-window=1

Not sure what BackupPC does with that.


-- 
Dan Espen

[toc] | [prev] | [next] | [standalone]

#27172

On 6/14/19 3:54 PM, Dan Espen wrote:
> Very interesting, thanks, but it's a layer on top of rsync, so 
> even though it's touted as "high performance" it won't give me any 
> speed ups.

I think the "high performance" that it's referring to is rsync vs 
something less intelligent.

> I don't see any options for a delayed delete of backed up deleted 
> files, I suppose I'd have to handle that with multiple backups 
> specifying one that was longer term.  Since my backups are working 
> pretty well, I'm not inclined to switch.

That's where the multiple backups that it manages for you comes into play.

I currently have the nightly backup job that does a standard rsync, 
including removing files.  But each nights backup is a different version 
in BackupPC.  I can easily browse to a directory in the BackupPC web 
interface, say that I want to see the different versions (backups) and 
be presented with a view of what version of a file existed in which backup.

I can trivially select the backup that has the file I deleted and 
restore it.

Since BackupPC manages the different versions for me so that I don't 
need to worry with time shifting my backup.  I just back up what's there 
currently and get historic copies / versions from BackupPC.  If that 
means I get a copy of a file that no longer exists, but BackupPC has an 
old copy, that's cool.

> This AM I got email showing my backup (to a USB stick) failed with 
> backup file system full.  That seemed odd, since I knew I had plenty of 
> room on the USB stick.  So, the USB backup directory was /back-home. 
> Somehow, /back-home got unmounted so the backup script was really 
> writing to the "/" file system.  There is no way I have enough room 
> to copy /home over to "/".  So, now I added this to the backup script:

oops

> Right now, I'm backing up /home to 128Gib USB stick formatted as ext3.
> I also need to back up 7K flac music files, that's just over 128Gib so I need to
> invest in a larger USB stick or SD card.  Still trying to decide.
> 
> I'd like the music backup to go to FAT for maximum ability to restore.
> At first my backups to FAT worked, except it kept trying to back up
> files that were already backed up.  A little Google searching told me I
> need to mount FAT with the options:
> 
> shortname=mixed,iocharset=utf8
> 
> and then run rsync with the option:
> 
> --modify-window=1
> 
> Not sure what BackupPC does with that.

You can specify options to the rsync command in the BackupPC interface, 
both BackupPC wide and / or per system.



-- 
Grant. . . .
unix || die

[toc] | [prev] | [next] | [standalone]

#27177

On 14/06/2019 14.01, Dan Espen wrote:
> Javier <invalid@invalid.invalid> writes:
> 
>> Richard Kettlewell <invalid@invalid.invalid> wrote:
>>> Dan Espen <dan1espen@gmail.com> writes:
>>>> Richard Kettlewell <invalid@invalid.invalid> writes:
>>>>> Dan Espen <dan1espen@gmail.com> writes:
>>>>>> Javier <invalid@invalid.invalid> writes:
>>>>>>> Brian W. Kernighan does not agree.
>>>>>>>
>>>>>>> http://harmful.cat-v.org/cat-v/unix_prog_design.pdf
>>>>>>>
>>>>>>> Programs should have as as few options as possible.  Do one thing and
>>>>>>> do it well.  For recursing through dirs you already have find(1).
>>>>>>
>>>>>> I thought we'd already discovered that BWK was wrong.
>>>>>> Perhaps he can explain how having 2 grep programs,
>>>>>> perhaps "grep" and "grepr" (for grep recursive) is superior
>>>>>> to "grep -r".
>>>>>
>>>>> I don’t think you can have read the paper, since he doesn’t argue that
>>>>> there should be a separate program for each use case.
>>>>
>>>> I read the paper.
>>>> At least last time this came up.
>>>> Okay, just scanned the thing again.
>>>>
>>>> Basically the whole paper is an admonishment about keeping programs
>>>> simple,  mainly by not adding options.
>>>>
>>>> When it finally gets down to it, he concedes that sometimes you add
>>>> options:
>>>>
>>>>   Separate programs  are not  always better than  wider options;  which is
>>>>   better depends  on the problem.  Whenever  one needs a way  to perform a
>>>>   new function,  one faces the  choice of whether to  add a new  option or
>>>>   write a new program...The guiding principle for making the choice should
>>>>   be that each  program does one thing. Options are  appropriately added
>>>>   to a  program that already has  the right functionality. If  there is no
>>>>   such program, then a new program is called for.
>>>>
>>>> So, he's  really giving us only  the vaguest idea when  we add options
>>>> and  when we  write new  programs.  We  find out,  it "depends",  but we
>>>> should "do one thing", and you can add options when the program
>>>> "already has the right functionality".
>>>>
>>>> I find the "do one thing" rule impossible to interpret.
>>>> How can any program do one thing if it has options?
>>>>
>>>> BWK addresses program elegance and never mentions user expectations.
>>>
>>> By focussing narrowly on options you appear to have completely missed
>>> the thrust of the argument, which is (the claim that) loosely coupled
>>> composition of logically separate functionality is a better design than
>>> combining multiple unrelated functionalities into single programs.
>>
>> Exactly, that is what BWK advocates.  Coupling simple tools through pipes.
>> For the usage case of the OP:
>>
>>     find /etc/dnf/ \! -type d | xargs grep xclude
> 
> Sure looks to me like the perfect proof that adding -r to grep is the
> right choice.
> 
>> By adding less options the program source code is simpler and its
>> manpage thinner.  Otherwise you end up with behemoths like rsync.
>> Look at its manpage:
>>
>> $ man rsync | wc -l
>> 3870
> 
> rsync still doesn't have enough options.
> It needs more options when it can't do it's job with existing options.
> 
> I use rsync for backup of $HOME.
> When I delete a file in $HOME, I want it deleted from the backup.
> But suppose I deleted the file by accident.  I want rsync to wait
> a while before it deletes the backup, in my case I want to wait 30
> days so I can discover whether I deleted something I need.
> 
> So, I recently solved this problem by running rsync a second time
> as a dry-run with the delete option.  Then I make a pass through all
> the file names returned from this second pass looking at time stamps
> and deleting files when they are 30 days old.  This works but would
> be so much cleaner if I could just code:
> 
>   --delete 30
> 
> on the first pass.
> 
> Size of the module and smallness of the man page are irrelevant when
> you can't do the job.

Interesting.

There is another way. You do a "new" backup with hardlinks to older
backups. The backup for today does not have copies of files that were
deleted, but the backup for "yesterday" has it. And unchanged files are
not copied again, wasting space, but are links to the first appeareance
on the backup. The other step needed is to purge backups older than 30 days.


What I miss is storing the files compressed.

-- 
Cheers, Carlos.

[toc] | [prev] | [next] | [standalone]

#27181

On 6/15/19 5:42 AM, Carlos E.R. wrote:
> There is another way. You do a "new" backup with hardlinks to older 
> backups. The backup for today does not have copies of files that 
> were deleted, but the backup for "yesterday" has it. And unchanged 
> files are not copied again, wasting space, but are links to the first 
> appeareance on the backup. The other step needed is to purge backups 
> older than 30 days.

Yep.

That's a technical description of what BackupPC does under the hood.

> What I miss is storing the files compressed.

Storing the BackupPC pool on a ZFS pool with compression enabled does 
that transparently.  ;-)



-- 
Grant. . . .
unix || die

[toc] | [prev] | [next] | [standalone]

#27205

On 15/06/2019 19.09, Grant Taylor wrote:
> On 6/15/19 5:42 AM, Carlos E.R. wrote:
>> There is another way. You do a "new" backup with hardlinks to older
>> backups. The backup for today does not have copies of files that were
>> deleted, but the backup for "yesterday" has it. And unchanged files
>> are not copied again, wasting space, but are links to the first
>> appeareance on the backup. The other step needed is to purge backups
>> older than 30 days.
> 
> Yep.
> 
> That's a technical description of what BackupPC does under the hood.
> 
>> What I miss is storing the files compressed.
> 
> Storing the BackupPC pool on a ZFS pool with compression enabled does
> that transparently.  ;-)

ZFS?  I'll have to think about that.


-- 
Cheers, Carlos.

[toc] | [prev] | [next] | [standalone]

#27206

On 17.06.2019 at 12:37, Carlos E.R. scribbled:

> On 15/06/2019 19.09, Grant Taylor wrote:
> > On 6/15/19 5:42 AM, Carlos E.R. wrote:  
> >> There is another way. You do a "new" backup with hardlinks to older
> >> backups. The backup for today does not have copies of files that
> >> were deleted, but the backup for "yesterday" has it. And unchanged
> >> files are not copied again, wasting space, but are links to the
> >> first appeareance on the backup. The other step needed is to purge
> >> backups older than 30 days.  
> > 
> > Yep.
> > 
> > That's a technical description of what BackupPC does under the hood.
> >   
> >> What I miss is storing the files compressed.  
> > 
> > Storing the BackupPC pool on a ZFS pool with compression enabled
> > does that transparently.  ;-)  
> 
> ZFS?  I'll have to think about that.
 
btrfs also has built-in compression, as well as auto-defragmentation. ;)

-- 
With respect,
= Aragorn =

[toc] | [prev] | [next] | [standalone]

#27207

On 17/06/2019 12.51, Aragorn wrote:
> On 17.06.2019 at 12:37, Carlos E.R. scribbled:
> 
>> On 15/06/2019 19.09, Grant Taylor wrote:
>>> On 6/15/19 5:42 AM, Carlos E.R. wrote:  
>>>> There is another way. You do a "new" backup with hardlinks to older
>>>> backups. The backup for today does not have copies of files that
>>>> were deleted, but the backup for "yesterday" has it. And unchanged
>>>> files are not copied again, wasting space, but are links to the
>>>> first appeareance on the backup. The other step needed is to purge
>>>> backups older than 30 days.  
>>>
>>> Yep.
>>>
>>> That's a technical description of what BackupPC does under the hood.
>>>   
>>>> What I miss is storing the files compressed.  
>>>
>>> Storing the BackupPC pool on a ZFS pool with compression enabled
>>> does that transparently.  ;-)  
>>
>> ZFS?  I'll have to think about that.
>  
> btrfs also has built-in compression, as well as auto-defragmentation. ;)
> 

That one I heard. But I don't know if it is considered an stable
feature. Two years ago it wasn't. And I have heard of so many people
with corrupt btrfs that it scares me.

-- 
Cheers, Carlos.

[toc] | [prev] | [next] | [standalone]

#27208

On 17.06.2019 at 12:57, Carlos E.R. scribbled:

> On 17/06/2019 12.51, Aragorn wrote:
> > On 17.06.2019 at 12:37, Carlos E.R. scribbled:
> >   
> >> On 15/06/2019 19.09, Grant Taylor wrote:  
> >>> On 6/15/19 5:42 AM, Carlos E.R. wrote:    
> >>>> There is another way. You do a "new" backup with hardlinks to
> >>>> older backups. The backup for today does not have copies of
> >>>> files that were deleted, but the backup for "yesterday" has it.
> >>>> And unchanged files are not copied again, wasting space, but are
> >>>> links to the first appeareance on the backup. The other step
> >>>> needed is to purge backups older than 30 days.    
> >>>
> >>> Yep.
> >>>
> >>> That's a technical description of what BackupPC does under the
> >>> hood. 
> >>>> What I miss is storing the files compressed.    
> >>>
> >>> Storing the BackupPC pool on a ZFS pool with compression enabled
> >>> does that transparently.  ;-)    
> >>
> >> ZFS?  I'll have to think about that.  
> >  
> > btrfs also has built-in compression, as well as
> > auto-defragmentation. ;) 
> 
> That one I heard. But I don't know if it is considered an stable
> feature. Two years ago it wasn't. And I have heard of so many people
> with corrupt btrfs that it scares me.

The on-disk format of btrfs has been stable for many years already.
The RAID-5/6 functionality is still not stable, and I've heard of some
problems with luks-encrypted volumes, but just about everything else in
the feature set is pretty robust.

I am using compression here myself, and I even have auto-defrag
enabled on the spinning rust disk — not on the SSD, for obvious
reasons. ;)



-- 
With respect,
= Aragorn =

[toc] | [prev] | [next] | [standalone]

#27210

On 6/17/19 4:57 AM, Carlos E.R. wrote:
> That one I heard. But I don't know if it is considered an stable 
> feature. Two years ago it wasn't. And I have heard of so many people 
> with corrupt btrfs that it scares me.

I thought I had heard through the grapevine that Red Hat was 
discouraging use of Btrfs.  Or that they were favoring XFS over Btrfs. 
Something, ¯\_(ツ)_/¯, that made it sound as if the fate of Btrfs was not 
promising.



-- 
Grant. . . .
unix || die

[toc] | [prev] | [next] | [standalone]

#27211

On 17.06.2019 at 08:51, Grant Taylor scribbled:

> On 6/17/19 4:57 AM, Carlos E.R. wrote:
> > That one I heard. But I don't know if it is considered an stable 
> > feature. Two years ago it wasn't. And I have heard of so many
> > people with corrupt btrfs that it scares me.  
> 
> I thought I had heard through the grapevine that Red Hat was 
> discouraging use of Btrfs.  Or that they were favoring XFS over
> Btrfs. Something, ¯\_(ツ)_/¯, that made it sound as if the fate of
> Btrfs was not promising.

RedHat has always been self-righteous about their choices.  A couple of
years ago you couldn't force the system to install on XFS filesystems,
because it HAD to be ext3 — trust me, I tried.  And now XFS is their
chosen filesystem and they reject everything else.

SUSE and openSUSE on the other hand use btrfs by default.

-- 
With respect,
= Aragorn =

[toc] | [prev] | [next] | [standalone]

#27212

On 17/06/2019 19.43, Aragorn wrote:
> On 17.06.2019 at 08:51, Grant Taylor scribbled:
> 
>> On 6/17/19 4:57 AM, Carlos E.R. wrote:
>>> That one I heard. But I don't know if it is considered an stable 
>>> feature. Two years ago it wasn't. And I have heard of so many
>>> people with corrupt btrfs that it scares me.  
>>
>> I thought I had heard through the grapevine that Red Hat was 
>> discouraging use of Btrfs.  Or that they were favoring XFS over
>> Btrfs. Something, ¯\_(ツ)_/¯, that made it sound as if the fate of
>> Btrfs was not promising.
> 
> RedHat has always been self-righteous about their choices.  A couple of
> years ago you couldn't force the system to install on XFS filesystems,
> because it HAD to be ext3 — trust me, I tried.  And now XFS is their
> chosen filesystem and they reject everything else.
> 
> SUSE and openSUSE on the other hand use btrfs by default.

Yes, for the system. The default for /home is XFS.

-- 
Cheers, Carlos.

[toc] | [prev] | [next] | [standalone]

#27213

On 6/17/19 11:43 AM, Aragorn wrote:
> RedHat has always been self-righteous about their choices.

You might not like the source of the data.  But that doesn't mean that 
it's not data.



-- 
Grant. . . .
unix || die

[toc] | [prev] | [next] | [standalone]

#27216

On 17.06.2019 at 13:07, Grant Taylor scribbled:

> On 6/17/19 11:43 AM, Aragorn wrote:
>
> > RedHat has always been self-righteous about their choices.  
> 
> You might not like the source of the data.  But that doesn't mean
> that it's not data.

You conveniently skipped the rest of that paragraph, which was meant to
put things into perspective.

XFS has already long been a mature filesystem, whether it was in IRIX
or in GNU/Linux, and yet up until at least version 6.x of RHEL and
CentOS — which was, what, 2005, 2006? — the installer refused to let you
install any part of the system on XFS partitions, and your only choice
was to accept ext3, which didn't even have its hashed trees enabled by
default (as is the case in ext4).

So the data may be whatever it is.  Just because RedHat doesn't approve
of btrfs doesn't mean that it's worthless.  RedHat has never approved
of KDE either [*], even though they've supplied it as an option until
recently.  But if you realize that RedHat is where GNOME and GTK are
being developed, then things suddenly start looking very different,
don't they?

RedHat is a corporation, and corporations do what's good for them and
for their business.  Facts are not as important to them as the support
from their shareholders.  Furthermore, RedHat wants to be the dominant
player in the enterprise GNU/Linux market.  Appearing steadfast in
their recommendations is part of their corporate propaganda, and
perhaps the fact that btrfs is being developed by Oracle — a competitor
— would also be playing into that.

How many times have you not heard Bill Gates or Steve Ballmer say that
GNU/Linux was insecure, unstable, and the operating system of choice of
terrorists?  Did you believe THAT too?  And their "data" about how the
TCO of Windows was lower than that of GNU/Linux?  

What their "data" of course did not mention was the fact that they were
comparing a Windows installation on a decently priced commodity PC to a
GNU/Linux installation running in a virtual machine on a Sun mainframe.
So yeah, mainframes tend to be a little costlier than commodity PCs,
indeed. <rolling my eyes>

Whom the data comes from and what their agenda is, is just as important
as the data themselves.


[*] They even had a genuine hate campaign going on against KDE for
    several years because KDE 1.x was built upon the then still not
    freely licensed QT libraries, but even though this was changed for
    QT2 and KDE2, RedHat continued to boycott and libel KDE for several
    more years — all throughout the lifetime of KDE2 and the early
    years of KDE3 — which earned them the moniker HatRed.

-- 
With respect,
= Aragorn =

[toc] | [prev] | [next] | [standalone]

#27217

Le 17/06/2019 à 21:56, Aragorn a écrit :
> On 17.06.2019 at 13:07, Grant Taylor scribbled:
> 
>> On 6/17/19 11:43 AM, Aragorn wrote:
>>
>>> RedHat has always been self-righteous about their choices.  
>>
>> You might not like the source of the data.  But that doesn't mean
>> that it's not data.
> 
> You conveniently skipped the rest of that paragraph, which was meant to
> put things into perspective.
> 
> XFS has already long been a mature filesystem, whether it was in IRIX
> or in GNU/Linux, and yet up until at least version 6.x of RHEL and
> CentOS — which was, what, 2005, 2006? — the installer refused to let you
> install any part of the system on XFS partitions, and your only choice
> was to accept ext3, which didn't even have its hashed trees enabled by
> default (as is the case in ext4).
> 
> So the data may be whatever it is.  Just because RedHat doesn't approve
> of btrfs doesn't mean that it's worthless.  RedHat has never approved
> of KDE either [*], even though they've supplied it as an option until
> recently.  But if you realize that RedHat is where GNOME and GTK are
> being developed, then things suddenly start looking very different,
> don't they?
> 
> RedHat is a corporation, and corporations do what's good for them and
> for their business.  Facts are not as important to them as the support
> from their shareholders.  Furthermore, RedHat wants to be the dominant
> player in the enterprise GNU/Linux market.  Appearing steadfast in
> their recommendations is part of their corporate propaganda, and
> perhaps the fact that btrfs is being developed by Oracle — a competitor
> — would also be playing into that.
> 
> How many times have you not heard Bill Gates or Steve Ballmer say that
> GNU/Linux was insecure, unstable, and the operating system of choice of
> terrorists?  Did you believe THAT too?  And their "data" about how the
> TCO of Windows was lower than that of GNU/Linux?  
> 
> What their "data" of course did not mention was the fact that they were
> comparing a Windows installation on a decently priced commodity PC to a
> GNU/Linux installation running in a virtual machine on a Sun mainframe.
> So yeah, mainframes tend to be a little costlier than commodity PCs,
> indeed. <rolling my eyes>
> 
> Whom the data comes from and what their agenda is, is just as important
> as the data themselves.
> 
> 
> [*] They even had a genuine hate campaign going on against KDE for
>     several years because KDE 1.x was built upon the then still not
>     freely licensed QT libraries, but even though this was changed for
>     QT2 and KDE2, RedHat continued to boycott and libel KDE for several
>     more years — all throughout the lifetime of KDE2 and the early
>     years of KDE3 — which earned them the moniker HatRed.
> 

Hi, thank you Aragorn for writing that clever post ;)

-- 
()
/\

[toc] | [prev] | [next] | [standalone]

#27220

On 17/06/2019 21.56, Aragorn wrote:
> On 17.06.2019 at 13:07, Grant Taylor scribbled:
> 
>> On 6/17/19 11:43 AM, Aragorn wrote:
>>
>>> RedHat has always been self-righteous about their choices.  
>>
>> You might not like the source of the data.  But that doesn't mean
>> that it's not data.
> 
> You conveniently skipped the rest of that paragraph, which was meant to
> put things into perspective.
> 
> XFS has already long been a mature filesystem, whether it was in IRIX
> or in GNU/Linux, and yet up until at least version 6.x of RHEL and
> CentOS — which was, what, 2005, 2006? — the installer refused to let you
> install any part of the system on XFS partitions, and your only choice
> was to accept ext3, which didn't even have its hashed trees enabled by
> default (as is the case in ext4).
> 
> So the data may be whatever it is.  Just because RedHat doesn't approve
> of btrfs doesn't mean that it's worthless.  RedHat has never approved
> of KDE either [*], even though they've supplied it as an option until
> recently.  But if you realize that RedHat is where GNOME and GTK are
> being developed, then things suddenly start looking very different,
> don't they?
> 
> RedHat is a corporation, and corporations do what's good for them and
> for their business.  Facts are not as important to them as the support
> from their shareholders.  Furthermore, RedHat wants to be the dominant
> player in the enterprise GNU/Linux market.  Appearing steadfast in
> their recommendations is part of their corporate propaganda, and
> perhaps the fact that btrfs is being developed by Oracle — a competitor
> — would also be playing into that.
> 
> How many times have you not heard Bill Gates or Steve Ballmer say that
> GNU/Linux was insecure, unstable, and the operating system of choice of
> terrorists?  Did you believe THAT too?  And their "data" about how the
> TCO of Windows was lower than that of GNU/Linux?  
> 
> What their "data" of course did not mention was the fact that they were
> comparing a Windows installation on a decently priced commodity PC to a
> GNU/Linux installation running in a virtual machine on a Sun mainframe.
> So yeah, mainframes tend to be a little costlier than commodity PCs,
> indeed. <rolling my eyes>
> 
> Whom the data comes from and what their agenda is, is just as important
> as the data themselves.

One Debian user despised my system because I use openSUSE, and blamed a
usb of his/her not working on "you surely have done something to the
system" and me not running the precious Debian. Did not permit me to
investigate the broken filesystem on the usb stick.

So, possibly the attitude of them pervades to some of its users... :-(

> 
> 
> [*] They even had a genuine hate campaign going on against KDE for
>     several years because KDE 1.x was built upon the then still not
>     freely licensed QT libraries, but even though this was changed for
>     QT2 and KDE2, RedHat continued to boycott and libel KDE for several
>     more years — all throughout the lifetime of KDE2 and the early
>     years of KDE3 — which earned them the moniker HatRed.
> 


-- 
Cheers, Carlos.

[toc] | [prev] | [next] | [standalone]

#27228

On 6/17/19 1:56 PM, Aragorn wrote:
> XFS has already long been a mature filesystem, whether it was in IRIX 
> or in GNU/Linux, and yet up until at least version 6.x of RHEL and 
> CentOS — which was, what, 2005, 2006? — the installer refused to 
> let you install any part of the system on XFS partitions, and your 
> only choice was to accept ext3, which didn't even have its hashed 
> trees enabled by default (as is the case in ext4).

I had a colleague that was running RHEL 6.<something> installed to XFS.

There is a big difference in what the installer provided options to do 
vs what could and couldn't be done.

My colleague did similar to me.  We let the installer format what ever 
it wanted to, went out to a shell, unmounted, changed file systems, 
remounted, and proceeded with the install.  He used XFS and I used 
ReiserFS.  Naturally we had to tweak /etc/fstab afterwards.

But we were able to install onto our preferred file systems.

> So the data may be whatever it is.  Just because RedHat doesn't approve 
> of btrfs doesn't mean that it's worthless.

I don't recall saying that Btrfs was worthless.  I simply indicated that 
it has been outmoded by Red Hat.

> RedHat has never approved of KDE either [*], even though they've 
> supplied it as an option until recently.  But if you realize that 
> RedHat is where GNOME and GTK are being developed, then things suddenly 
> start looking very different, don't they?

No.  Not to me, they don't.  If I care to, I'm usually able to separate 
and ignore marketing.

> RedHat is a corporation, and corporations do what's good for them and 
> for their business.

Agreed.

> Facts are not as important to them as the support from their 
> shareholders.  Furthermore, RedHat wants to be the dominant player 
> in the enterprise GNU/Linux market.  Appearing steadfast in their 
> recommendations is part of their corporate propaganda, and perhaps 
> the fact that btrfs is being developed by Oracle — a competitor 
> — would also be playing into that.

The circles that I traveled in, Btrfs was viewed as less table and more 
likely to corrupt data, particularly compared to ZFS.  Especially 
considering that Btrfs could (still can't to the best of my knowledge) 
reliably implement RAID 5 or 6.

> How many times have you not heard Bill Gates or Steve Ballmer say 
> that GNU/Linux was insecure, unstable, and the operating system of 
> choice of terrorists?  Did you believe THAT too?  And their "data" 
> about how the TCO of Windows was lower than that of GNU/Linux?

I can't think of a single time that I did hear them say that.  I tended 
to ignore them.  I did hear tell of them saying that, but I dismissed it 
as largely ignorance or propaganda.

> What their "data" of course did not mention was the fact that they were 
> comparing a Windows installation on a decently priced commodity PC to a 
> GNU/Linux installation running in a virtual machine on a Sun mainframe. 
> So yeah, mainframes tend to be a little costlier than commodity PCs, 
> indeed. <rolling my eyes>

Odd.  Most of the articles that I ever paid attention to installed both 
Windows and Linux (of one distro or another) on the same hardware.

Also, a Sun is not a mainframe.  Sun(s) are / can be extremely large 
mini-computers.

> Whom the data comes from and what their agenda is, is just as important 
> as the data themselves.

It's quite possible to extract useful data from just about anybody, even 
the shiniest prettiest P.R. marketing slicks.

> [*] They even had a genuine hate campaign going on against KDE for 
> several years because KDE 1.x was built upon the then still not freely 
> licensed QT libraries, but even though this was changed for QT2 and 
> KDE2, RedHat continued to boycott and libel KDE for several more years 
> — all throughout the lifetime of KDE2 and the early years of KDE3 
> — which earned them the moniker HatRed.

Shame on them.



-- 
Grant. . . .
unix || die

[toc] | [prev] | [next] | [standalone]

#27232

On 17.06.2019 at 19:43, Grant Taylor scribbled:

> On 6/17/19 1:56 PM, Aragorn wrote:
>
> > [...]
> >
> > How many times have you not heard Bill Gates or Steve Ballmer say 
> > that GNU/Linux was insecure, unstable, and the operating system of 
> > choice of terrorists?  Did you believe THAT too?  And their "data" 
> > about how the TCO of Windows was lower than that of GNU/Linux?  
> 
> I can't think of a single time that I did hear them say that.  I
> tended to ignore them.  I did hear tell of them saying that, but I
> dismissed it as largely ignorance or propaganda.

It was at Microsoft's own website, in a section about Windows 2000.  It
was taken down again after a few months.

> > What their "data" of course did not mention was the fact that they
> > were comparing a Windows installation on a decently priced
> > commodity PC to a GNU/Linux installation running in a virtual
> > machine on a Sun mainframe. So yeah, mainframes tend to be a little
> > costlier than commodity PCs, indeed. <rolling my eyes>  
> 
> Odd.  Most of the articles that I ever paid attention to installed
> both Windows and Linux (of one distro or another) on the same
> hardware.

This was not an article in a computer magazine, but an official
Microsoft PR document regarding the TCO of Windows versus GNU/Linux.

> Also, a Sun is not a mainframe.  Sun(s) are / can be extremely large 
> mini-computers.

Well, it was a very large machine — that I do remember.  And it may have
been an IBM instead of a Sun.

-- 
With respect,
= Aragorn =

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | comp.os.linux.misc

csiph-web