Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.misc > #70174 > unrolled thread

Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late

Back to article view | Back to comp.os.linux.misc

Contents

  Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late Lawrence D'Oliveiro <ldo@nz.invalid> - 2025-07-31 00:26 +0000
    Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late c186282 <c186282@nnada.net> - 2025-07-30 23:02 -0400
    Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late "Carlos E.R." <robin_listas@es.invalid> - 2025-07-31 15:21 +0200
      Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late The Natural Philosopher <tnp@invalid.invalid> - 2025-07-31 14:39 +0100
      Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late Lawrence D'Oliveiro <ldo@nz.invalid> - 2025-08-01 01:08 +0000
        Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late "Carlos E.R." <robin_listas@es.invalid> - 2025-08-01 14:03 +0200
      Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late Marc Haber <mh+usenetspam1118@zugschl.us> - 2025-08-01 10:05 +0200
        Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late "Carlos E.R." <robin_listas@es.invalid> - 2025-08-01 14:04 +0200
          Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late Marc Haber <mh+usenetspam1118@zugschl.us> - 2025-08-01 14:31 +0200
            Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late "Carlos E.R." <robin_listas@es.invalid> - 2025-08-01 14:40 +0200
              Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late Marc Haber <mh+usenetspam1118@zugschl.us> - 2025-08-01 18:16 +0200
            Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late Stéphane CARPENTIER <sc@fiat-linux.fr> - 2025-08-01 19:29 +0000
              Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late c186282 <c186282@nnada.net> - 2025-08-02 02:15 -0400
              Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late Marc Haber <mh+usenetspam1118@zugschl.us> - 2025-08-02 10:05 +0200
            Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late Lawrence D'Oliveiro <ldo@nz.invalid> - 2025-08-01 23:55 +0000
              Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late "Carlos E.R." <robin_listas@es.invalid> - 2025-08-02 03:39 +0200
                Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late Lawrence D'Oliveiro <ldo@nz.invalid> - 2025-08-02 03:24 +0000
                  Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late c186282 <c186282@nnada.net> - 2025-08-02 02:04 -0400
                  Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late "Carlos E.R." <robin_listas@es.invalid> - 2025-08-02 13:59 +0200
                Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late c186282 <c186282@nnada.net> - 2025-08-02 03:03 -0400
                Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late candycanearter07 <candycanearter07@candycanearter07.nomail.afraid> - 2025-08-05 19:20 +0000
                  Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late Nuno Silva <nunojsilva@invalid.invalid> - 2025-08-05 23:04 +0100
                    Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late c186282 <c186282@nnada.net> - 2025-08-05 19:54 -0400
                      Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late rbowman <bowman@montana.com> - 2025-08-06 06:37 +0000
                        Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late c186282 <c186282@nnada.net> - 2025-08-06 04:03 -0400
                          Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late not@telling.you.invalid (Computer Nerd Kev) - 2025-08-07 07:14 +1000
                            Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late c186282 <c186282@nnada.net> - 2025-08-06 22:16 -0400
                              Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late Richard Kettlewell <invalid@invalid.invalid> - 2025-08-07 18:20 +0100
                                Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late The Natural Philosopher <tnp@invalid.invalid> - 2025-08-07 18:22 +0100
                              Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late not@telling.you.invalid (Computer Nerd Kev) - 2025-08-08 09:10 +1000
                        Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late Bobbie Sellers <bliss-sf4ever@dslextreme.com> - 2025-08-06 07:39 -0700
                    Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late candycanearter07 <candycanearter07@candycanearter07.nomail.afraid> - 2025-08-11 19:40 +0000
              Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late Marc Haber <mh+usenetspam1118@zugschl.us> - 2025-08-02 10:07 +0200
                Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late Lawrence D'Oliveiro <ldo@nz.invalid> - 2025-08-02 23:32 +0000
                  Re: Linux PC Acting Up? How To Check For Bad Blocks On A Hard Drive - Before It's Too Late c186282 <c186282@nnada.net> - 2025-08-02 21:36 -0400

Page 2 of 2 — ← Prev page 1 [2]

#70396

Carlos E.R. <robin_listas@es.invalid> wrote at 01:39 this Saturday (GMT):
> On 2025-08-02 01:55, Lawrence D'Oliveiro wrote:
>> On Fri, 01 Aug 2025 14:31:54 +0200, Marc Haber wrote:
>> 
>>> I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
>>> backup was invalid the hard way.
>> 
>> My backups are done with rsync, or script wrappers around rsync. They are
>> effectively mirror filesystems, not in any special “backup” format. These
>> are the easiest to verify, and restoration can be done with the same file-
>> manipulation commands I use every day.
>> 
>> In particular, that means that, in a moment of crisis (which is what
>> happens when you find you’ve lost data), I am doing the restoration using
>> familiar commands with which I am (hopefully) less likely to make
>> mistakes.
>
> Once an rsync backup deleted the original.
>
> I was using the --delete parameter, which is intended to delete files 
> that are gone from the source, and delete them in the destination (when 
> you overwrite a backup). Unfortunately, the paths were messed up, and it 
> deleted the files on the source of another different backup.


That's why you have backups of backups.
-- 
user <candycane> is generated from /dev/urandom

[toc] | [prev] | [next] | [standalone]

#70405

On 2025-08-05, candycanearter07 wrote:

> Carlos E.R. <robin_listas@es.invalid> wrote at 01:39 this Saturday (GMT):
>> On 2025-08-02 01:55, Lawrence D'Oliveiro wrote:
>>> On Fri, 01 Aug 2025 14:31:54 +0200, Marc Haber wrote:
>>> 
>>>> I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
>>>> backup was invalid the hard way.
>>> 
>>> My backups are done with rsync, or script wrappers around rsync. They are
>>> effectively mirror filesystems, not in any special “backup” format. These
>>> are the easiest to verify, and restoration can be done with the same file-
>>> manipulation commands I use every day.
>>> 
>>> In particular, that means that, in a moment of crisis (which is what
>>> happens when you find you’ve lost data), I am doing the restoration using
>>> familiar commands with which I am (hopefully) less likely to make
>>> mistakes.
>>
>> Once an rsync backup deleted the original.
>>
>> I was using the --delete parameter, which is intended to delete files 
>> that are gone from the source, and delete them in the destination (when 
>> you overwrite a backup). Unfortunately, the paths were messed up, and it 
>> deleted the files on the source of another different backup.
>
>
> That's why you have backups of backups.


But... what if these fail too!?

10 PRINT "That's why you have"
20 PRINT "backups of backups of"
30 GOTO 20

(Ok, in reality, a second level probably already drives the likelihood
down a lot. But, of course, this gets to be an engineering problem, and
one gets to ask what could possibly go wrong outside of these
likelihoods.

And one could do things such as not using the same
approach/scripts/software for both levels. Just like NASA had BFS
developed separately from PASS.)

-- 
Nuno Silva

[toc] | [prev] | [next] | [standalone]

#70414

On 8/5/25 6:04 PM, Nuno Silva wrote:
> On 2025-08-05, candycanearter07 wrote:
> 
>> Carlos E.R. <robin_listas@es.invalid> wrote at 01:39 this Saturday (GMT):
>>> On 2025-08-02 01:55, Lawrence D'Oliveiro wrote:
>>>> On Fri, 01 Aug 2025 14:31:54 +0200, Marc Haber wrote:
>>>>
>>>>> I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
>>>>> backup was invalid the hard way.
>>>>
>>>> My backups are done with rsync, or script wrappers around rsync. They are
>>>> effectively mirror filesystems, not in any special “backup” format. These
>>>> are the easiest to verify, and restoration can be done with the same file-
>>>> manipulation commands I use every day.
>>>>
>>>> In particular, that means that, in a moment of crisis (which is what
>>>> happens when you find you’ve lost data), I am doing the restoration using
>>>> familiar commands with which I am (hopefully) less likely to make
>>>> mistakes.
>>>
>>> Once an rsync backup deleted the original.
>>>
>>> I was using the --delete parameter, which is intended to delete files
>>> that are gone from the source, and delete them in the destination (when
>>> you overwrite a backup). Unfortunately, the paths were messed up, and it
>>> deleted the files on the source of another different backup.
>>
>>
>> That's why you have backups of backups.
> 
> 
> But... what if these fail too!?
> 
> 10 PRINT "That's why you have"
> 20 PRINT "backups of backups of"
> 30 GOTO 20
> 
> (Ok, in reality, a second level probably already drives the likelihood
> down a lot. But, of course, this gets to be an engineering problem, and
> one gets to ask what could possibly go wrong outside of these
> likelihoods.
> 
> And one could do things such as not using the same
> approach/scripts/software for both levels. Just like NASA had BFS
> developed separately from PASS.)

   Best modern plan = TWO local copies, on different
   machines/media, and one dup (maybe just the 'important
   stuff') to 'cloud' storage. Not too expensive. Just
   automate it all so it works overnight.

   I wrote a number of programs that were ultimately
   smart front-ends for rsync ... which did the hard
   work as efficiently as possible.

[toc] | [prev] | [next] | [standalone]

#70437

On Tue, 5 Aug 2025 19:54:56 -0400, c186282 wrote:

>    Best modern plan = TWO local copies, on different machines/media, and
>    one dup (maybe just the 'important stuff') to 'cloud' storage. Not
>    too expensive. Just automate it all so it works overnight.

We used  portable drives that were off premises. That was part of the 
formal backups the build guy did. Informally everyone checked out the 
entire source tree in Subversion and kept it updated so unless the 
building was nuked there would be a backup. 

For my works in progress I'd have the code on two or three machines plus 
the corporate OneDrive.

My personal projects at home usually are copied to two or three machines. 
I don't bother with the cloud since there's nothing I couldn't recreate. 
Docmentation, SDKs, IDEs and so forth are safely stored wherever I got 
them from in the first place, and probably newer versions to boot.

I do have some mp3s on a WD Passport and assorted thumb drives because no 
way in hell do I want to rip a shelf full of CDs again. At least they're 
easier than cassettes.

[toc] | [prev] | [next] | [standalone]

#70440

On 8/6/25 2:37 AM, rbowman wrote:
> On Tue, 5 Aug 2025 19:54:56 -0400, c186282 wrote:
> 
>>     Best modern plan = TWO local copies, on different machines/media, and
>>     one dup (maybe just the 'important stuff') to 'cloud' storage. Not
>>     too expensive. Just automate it all so it works overnight.
> 
> We used  portable drives that were off premises. That was part of the
> formal backups the build guy did. Informally everyone checked out the
> entire source tree in Subversion and kept it updated so unless the
> building was nuked there would be a backup.

   My last place, there were a couple of out-buildings.
   One backup to the main building, one to an out-
   building. In case of fire/tornado, best approach.

   Bought a simple DropBox paid acct ... kept that
   "most important" stuff there. Cheap and effective.

   SOME cloud offer FTP, which would maybe have been
   better. But made do with DropBox.

   Note - ALWAYS pre-encrypted before sending to cloud.
   Encrypt local, random name, copy to DBx, Rename
   re-date and whatever else is possible afterwards.
   OpenSSL can do most of the security stuff. I just
   do NOT believe claims of their own 'security'.
   Send 'em binary garbage. IF they bitch it means
   they ARE trying to spy/sell.

   I wrote at least three big 'front-ends' for rsync
   to do all the backups. Made it smart. Two in Python,
   one in Pascal. Yes, I DO still love Pascal  :-)

> For my works in progress I'd have the code on two or three machines plus
> the corporate OneDrive.
> 
> My personal projects at home usually are copied to two or three machines.
> I don't bother with the cloud since there's nothing I couldn't recreate.
> Docmentation, SDKs, IDEs and so forth are safely stored wherever I got
> them from in the first place, and probably newer versions to boot.
> 
> I do have some mp3s on a WD Passport and assorted thumb drives because no
> way in hell do I want to rip a shelf full of CDs again. At least they're
> easier than cassettes.

   For a biz, a suitably robust backup scheme is
   absolutely imperative. Individuals really should
   do the same.

   With the clear cyberwar with China/Russia/NK in
   bloom, DO cover your ass !!!

   M$ 'solutions' DO beware, they'll be amongst
   the most-targeted. EXPECT all their 'cloud'
   stuff to be obliterated, including their
   backups. They WILL have inside agents ...

[toc] | [prev] | [next] | [standalone]

#70507

c186282 <c186282@nnada.net> wrote:
>   Note - ALWAYS pre-encrypted before sending to cloud.
>   Encrypt local, random name, copy to DBx, Rename
>   re-date and whatever else is possible afterwards.
>   OpenSSL can do most of the security stuff. I just
>   do NOT believe claims of their own 'security'.
>   Send 'em binary garbage.

Sure, although keep in mind that one day in the future they might
be able to decrypt even your old self-encrypted files easily, like
encrypted files from the 1990s can be today. Even if they say that
data is deleted, there's no way to be sure that someone didn't
keep/steal a copy.

-- 
__          __
#_ < |\| |< _#

[toc] | [prev] | [next] | [standalone]

#70520

On 8/6/25 5:14 PM, Computer Nerd Kev wrote:
> c186282 <c186282@nnada.net> wrote:
>>    Note - ALWAYS pre-encrypted before sending to cloud.
>>    Encrypt local, random name, copy to DBx, Rename
>>    re-date and whatever else is possible afterwards.
>>    OpenSSL can do most of the security stuff. I just
>>    do NOT believe claims of their own 'security'.
>>    Send 'em binary garbage.
> 
> Sure, although keep in mind that one day in the future they might
> be able to decrypt even your old self-encrypted files easily, like
> encrypted files from the 1990s can be today. Even if they say that
> data is deleted, there's no way to be sure that someone didn't
> keep/steal a copy.

   Well, 'relevance' tends to fall off over time.
   Ten or twenty year old bank/personnel records
   and such ... not very interesting.

   Even now I'm sure the spooks can crack AES256 and
   likely most of the others. Quantum makes that much
   faster. Have seen allegedly 'quantum-resistant'
   ciphers, but not sure how well they live up to
   the label yet. There's also the issue of govts
   demanding backdoors - ciphers ARE still classed
   as 'munitions'.

   First saw "NSA BACKDOOR" in the Win2k registry ...

   For my personal stuff now I tend to use PGP/GPG or
   openssl with Camilla cipher. Kept hearing about
   some bug in the AES cipher. One note though, for
   BIG backups, openssl is MUCH faster. GPG seems
   to have to be started, initialized, then do the
   job, then it shuts down and you have to repeat
   for the next (250,000) files. Openssl works
   99% the same in Linux/Unix and Winders ... have
   used it on all platforms for fast backups.

   And ANY encryption is better than NONE out in
   the Cloud. Unless you're a drug cartel the
   spooks are NOT going to be the ones looking
   at your files. It's gonna be opportunistic
   little hacks, probably doing quick text
   searches on your files.

   I wrote a little utility some years back that
   was not 'encryption' per-se but an 'obfuscation'
   app. It did a few of the old cheap tricks. We
   used it for confidential in-house stuff. It
   was very quick and would work on binaries. The
   idea with obfuscation/scrambling is that if you
   can see how the pgm works there IS a path to
   unscrambling if you lose the key. Might take
   half a day, but you can do it.

[toc] | [prev] | [next] | [standalone]

#70558

c186282 <c186282@nnada.net> writes:
> On 8/6/25 5:14 PM, Computer Nerd Kev wrote:
>> c186282 <c186282@nnada.net> wrote:
>>>    Note - ALWAYS pre-encrypted before sending to cloud.
>>>    Encrypt local, random name, copy to DBx, Rename
>>>    re-date and whatever else is possible afterwards.
>>>    OpenSSL can do most of the security stuff. I just
>>>    do NOT believe claims of their own 'security'.
>>>    Send 'em binary garbage.
>> Sure, although keep in mind that one day in the future they might be
>> able to decrypt even your old self-encrypted files easily, like
>> encrypted files from the 1990s can be today. Even if they say that
>> data is deleted, there's no way to be sure that someone didn't
>> keep/steal a copy.
>
>   Well, 'relevance' tends to fall off over time.  Ten or twenty year
>   old bank/personnel records and such ... not very interesting.
>
>   Even now I'm sure the spooks can crack AES256 and likely most of the
>   others. Quantum makes that much faster.  Have seen allegedly
>   'quantum-resistant' ciphers, but not sure how well they live up to
>   the label yet. There's also the issue of govts demanding backdoors -
>   ciphers ARE still classed as 'munitions'.

AES-256 (properly implemented and deployed) has stood up to decades of
analysis by a community which has successfully broken multiple other
primitives: RC4, MD5, SHA1, SIKE, FF3. Never say never, but by this
stage, I think the reason AES has not been added to this list is because
there is nothing to find.

As for quantum computers, if AES is the first thing you worry about then
you really do not know what you’re talking about.

A (currently hypothetical) quantum computer of sufficient size would in
principle reduce AES-256 key search to 2^128 time complexity (with a
rather larger constant factor than a classical computer) - far beyond
what human civilization can achieve. Adding more quantum computers
doesn’t help as much as you might imagine since (AIUI) Grover’s
algorithm scales sublinearly.

The algorithms which will be meaningfully vulnerable to nontrivial
quantum computers (at such time as they actually built) are the
classical asymmetric algorithms, based on factorization problems and
various forms of discrete logarithm problem: RSA, (EC)DSA, (EC)DH. The
concern over the relative novelty of their replacements (MLDSA etc)
motivates the widespread (although not universal) preference for hybrids
instead of a pure-PQC solution.

-- 
https://www.greenend.org.uk/rjk/

[toc] | [prev] | [next] | [standalone]

#70559

On 07/08/2025 18:20, Richard Kettlewell wrote:
> A (currently hypothetical) quantum computer of sufficient size would in
> principle reduce AES-256 key search to 2^128  time complexity (with a
> rather larger constant factor than a classical computer) - far beyond
> what human civilization can achieve. Adding more quantum computers
> doesn’t help as much as you might imagine since (AIUI) Grover’s
> algorithm scales sublinearly.

You mathematicians...

I'll take your word for it :-)

-- 
"I am inclined to tell the truth and dislike people who lie consistently.
This makes me unfit for the company of people of a Left persuasion, and 
all women"

[toc] | [prev] | [next] | [standalone]

#70584

c186282 <c186282@nnada.net> wrote:
> On 8/6/25 5:14 PM, Computer Nerd Kev wrote:
>> c186282 <c186282@nnada.net> wrote:
>>>    Note - ALWAYS pre-encrypted before sending to cloud.
>>>    Encrypt local, random name, copy to DBx, Rename
>>>    re-date and whatever else is possible afterwards.
>>>    OpenSSL can do most of the security stuff. I just
>>>    do NOT believe claims of their own 'security'.
>>>    Send 'em binary garbage.
>> 
>> Sure, although keep in mind that one day in the future they might
>> be able to decrypt even your old self-encrypted files easily, like
>> encrypted files from the 1990s can be today. Even if they say that
>> data is deleted, there's no way to be sure that someone didn't
>> keep/steal a copy.
> 
>   Well, 'relevance' tends to fall off over time.
>   Ten or twenty year old bank/personnel records
>   and such ... not very interesting.

Potentially still useful for fraudsters though.

-- 
__          __
#_ < |\| |< _#

[toc] | [prev] | [next] | [standalone]

#70483

On 8/5/25 23:37, rbowman wrote:
> On Tue, 5 Aug 2025 19:54:56 -0400, c186282 wrote:
> 
>>     Best modern plan = TWO local copies, on different machines/media, and
>>     one dup (maybe just the 'important stuff') to 'cloud' storage. Not
>>     too expensive. Just automate it all so it works overnight.
> 
> We used  portable drives that were off premises. That was part of the
> formal backups the build guy did. Informally everyone checked out the
> entire source tree in Subversion and kept it updated so unless the
> building was nuked there would be a backup.
> 
> For my works in progress I'd have the code on two or three machines plus
> the corporate OneDrive.
> 
> My personal projects at home usually are copied to two or three machines.
> I don't bother with the cloud since there's nothing I couldn't recreate.

	Not the best practice except for you developers.  Recently the computers
housing the PCLinuxOS forum were caught in a fire.  With the cloud backup
the Forum was back  online in less than a week.


> Documentation, SDKs, IDEs and so forth are safely stored wherever I got
> them from in the first place, and probably newer versions to boot.
> 
> I do have some mp3s on a WD Passport and assorted thumb drives because no
> way in hell do I want to rip a shelf full of CDs again. At least they're
> easier than cassettes.

	Good luck and they are faster than CDs or floppies.

bliss- Dell Precision 7730- PCLOS 2025.06- Linux 6.12.41-pclos1- KDE 
Plasma 5.27.11

[toc] | [prev] | [next] | [standalone]

#70798

Nuno Silva <nunojsilva@invalid.invalid> wrote at 22:04 this Tuesday (GMT):
> On 2025-08-05, candycanearter07 wrote:
>
>> Carlos E.R. <robin_listas@es.invalid> wrote at 01:39 this Saturday (GMT):
>>> On 2025-08-02 01:55, Lawrence D'Oliveiro wrote:
>>>> On Fri, 01 Aug 2025 14:31:54 +0200, Marc Haber wrote:
>>>> 
>>>>> I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
>>>>> backup was invalid the hard way.
>>>> 
>>>> My backups are done with rsync, or script wrappers around rsync. They are
>>>> effectively mirror filesystems, not in any special “backup” format. These
>>>> are the easiest to verify, and restoration can be done with the same file-
>>>> manipulation commands I use every day.
>>>> 
>>>> In particular, that means that, in a moment of crisis (which is what
>>>> happens when you find you’ve lost data), I am doing the restoration using
>>>> familiar commands with which I am (hopefully) less likely to make
>>>> mistakes.
>>>
>>> Once an rsync backup deleted the original.
>>>
>>> I was using the --delete parameter, which is intended to delete files 
>>> that are gone from the source, and delete them in the destination (when 
>>> you overwrite a backup). Unfortunately, the paths were messed up, and it 
>>> deleted the files on the source of another different backup.
>>
>>
>> That's why you have backups of backups.
>
>
> But... what if these fail too!?
>
> 10 PRINT "That's why you have"
> 20 PRINT "backups of backups of"
> 30 GOTO 20
>
> (Ok, in reality, a second level probably already drives the likelihood
> down a lot. But, of course, this gets to be an engineering problem, and
> one gets to ask what could possibly go wrong outside of these
> likelihoods.

Yeah, diminishing returns and all.

> And one could do things such as not using the same
> approach/scripts/software for both levels. Just like NASA had BFS
> developed separately from PASS.)


I'll keep that in mind when/if i set up another backup system.
-- 
user <candycane> is generated from /dev/urandom

[toc] | [prev] | [next] | [standalone]

#70255

Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>On Fri, 01 Aug 2025 14:31:54 +0200, Marc Haber wrote:
>> I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
>> backup was invalid the hard way.
>
>My backups are done with rsync, or script wrappers around rsync.

You're comparing 1998 with 2025. Do you think that is fair?

The OS in question wasn't even unix back then.

Btw, script wrappers around rsync do not scale to even a small fleet.

Greetings
Marc
-- 
----------------------------------------------------------------------------
Marc Haber         |   " Questions are the         | Mailadresse im Header
Rhein-Neckar, DE   |     Beginning of Wisdom "     | 
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 6224 1600402

[toc] | [prev] | [next] | [standalone]

#70268

On Sat, 02 Aug 2025 10:07:02 +0200, Marc Haber wrote:

> Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>>
>>On Fri, 01 Aug 2025 14:31:54 +0200, Marc Haber wrote:
>>>
>>> I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
>>> backup was invalid the hard way.
>>
>>My backups are done with rsync, or script wrappers around rsync.
> 
> You're comparing 1998 with 2025. Do you think that is fair?

Andrew Tridgell submitted his PhD thesis, in which he developed rsync’s 
algorithm for comparing files across the network without having to copy 
the entirety of either side onto the other, in 1998, as I recall.

At that time (and for a few years after), people would have been doing 
backups using commands like “cp -u”. Which work well enough, as far as 
they go.

> The OS in question wasn't even unix back then.

The clunkiness of such compared to *nix systems was already becoming 
apparent.

> Btw, script wrappers around rsync do not scale to even a small fleet.

They do, actually. Remember, rsync itself is doing all the grunt work, and 
that can happily handle hundreds of gigabytes, or terabytes of data. (Just 
speaking from personal experience.)

And given that CERN, for example, is a heavy Linux user, I imagine they 
use rsync to deal with their petabytes of data from the LHC and other 
projects as well.

How do I know this? Because being a bunch of scientists, if they had to 
develop a more robust tool for that purpose, they would have announced and 
open-sourced it by now.

[toc] | [prev] | [next] | [standalone]

#70274

On 8/2/25 7:32 PM, Lawrence D'Oliveiro wrote:
> On Sat, 02 Aug 2025 10:07:02 +0200, Marc Haber wrote:
> 
>> Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>>>
>>> On Fri, 01 Aug 2025 14:31:54 +0200, Marc Haber wrote:
>>>>
>>>> I lost 540 MB from a failed Fujitsu drive in 1998 and found out the
>>>> backup was invalid the hard way.
>>>
>>> My backups are done with rsync, or script wrappers around rsync.
>>
>> You're comparing 1998 with 2025. Do you think that is fair?
> 
> Andrew Tridgell submitted his PhD thesis, in which he developed rsync’s
> algorithm for comparing files across the network without having to copy
> the entirety of either side onto the other, in 1998, as I recall.
> 
> At that time (and for a few years after), people would have been doing
> backups using commands like “cp -u”. Which work well enough, as far as
> they go.
> 
>> The OS in question wasn't even unix back then.
> 
> The clunkiness of such compared to *nix systems was already becoming
> apparent.
> 
>> Btw, script wrappers around rsync do not scale to even a small fleet.
> 
> They do, actually. Remember, rsync itself is doing all the grunt work, and
> that can happily handle hundreds of gigabytes, or terabytes of data. (Just
> speaking from personal experience.)
> 
> And given that CERN, for example, is a heavy Linux user, I imagine they
> use rsync to deal with their petabytes of data from the LHC and other
> projects as well.
> 
> How do I know this? Because being a bunch of scientists, if they had to
> develop a more robust tool for that purpose, they would have announced and
> open-sourced it by now.


   'cp' works ... but it's dumb as dirt.

   Kind of odd that rsync has never really been
   ported to Winders. The far more excessive
   ownership info used in Winders files now
   would make it a bit of a pain.

   IF you can find good doc, rsync has options that
   deal with almost any situation you can imagine.
   Great utility !

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.os.linux.misc

csiph-web