Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > sci.electronics.design > #453203
| From | Don Y <blockedofcourse@foo.invalid> |
|---|---|
| Newsgroups | sci.electronics.design |
| Subject | Re: big drives |
| Date | 2017-03-22 23:21 -0700 |
| Organization | A noiseless patient Spider |
| Message-ID | <oavpbi$ac6$1@dont-email.me> (permalink) |
| References | (1 earlier) <060b95d5-f7d7-4f30-bf4f-1c2f28768790@googlegroups.com> <ppadncEiOrmhc0_FnZ2dnUU7-Y2dnZ2d@giganews.com> <hq06dcd2kq40eh0tpg4s8pee977cnr48h4@4ax.com> <oav0p0$m1m$2@dont-email.me> <6o26dct4ihbqtg8l94crijgnou7hmapao0@4ax.com> |
On 3/22/2017 4:45 PM, John Larkin wrote:
> On Wed, 22 Mar 2017 16:21:45 -0700, Don Y
> <blockedofcourse@foo.invalid> wrote:
>
>> On 3/22/2017 4:07 PM, John Larkin wrote:
>>> On Wed, 22 Mar 2017 16:37:02 -0500, Jon Elson <jmelson@wustl.edu>
>>> wrote:
>>>
>>>> tabbypurr@gmail.com wrote:
>>>>
>>>>> On Tuesday, 21 March 2017 17:21:23 UTC, John Larkin wrote:
>>>>>> I just happened to open my recycle bin and there was 42 Gbytes of
>>>>>> junk.
>>>>>>
>>>>>> I remember 256 Kbyte hard drives. A PC with a 10M drive was top-end.
>>>>>>
>>>>>> There's no point in reducing jpeg sizes, or even deleting stuff, any
>>>>>> more. I could never take enough pics, or create enough docs, to keep
>>>>>> up with the progress in hard drives. I suppose some people could
>>>>>> collect a large number of HD movies and use a terabyte somehow.
>>>>>>
>>>>>> It's impossible to crack one-time-pad encryption. Giant hard drives
>>>>>> and memory sticks make OTP files easy to physically hand off to
>>>>>> correspondents.
>>>>>
>>>>> I estimated the payback of deleting data some time last year. It came in
>>>>> at something like a penny or 2 an hour.
>>>> Well, the problem is I want physical, off-line backup of important stuff
>>>> (that's what I've created, not the OS, utilities, etc.) preferably located
>>>> in a couple places. So, getting rid of excess junk and redundant on-line
>>>> backups makes some sense, in thaty it makes those CDs, then DVDs, now Blu-
>>>> Ray backups fit on available media.
>>>>
>>>> jon
>>>
>>> We back up to USB terabyte hard drives and use them as write-once
>>> media.
>>
>> And what ongoing *policy* do you have to verify their contents are still
>> accessible (i.e., that the disk hasn't developed a flaw) *and* intact?
>
> Massive redundancy. If 90% of the off-site drives die, we'll still be
> OK.
I've a friend who thought copying all of his ROM images, etc. onto his
PC and routinely backing up his PC was a great way to ensure he could
always recreate ROMs.
Some years later, he was patching a ROM and botched the job. No way to
"click undo" and recreate the original ROM. *But*, hey, the image of
the ROM is present on his PC, right?
Imagine his dismay to discover that the image on the PC couldn't
possibly correlate with even the *bulk* of the botched ROM that he
had before him! (locations that were "burned to 0" in the image but
NOT burned in the actual ROM)
Obviously, at some point in the past, he had copied the file incorrectly
(either by some manual operation on his part *or* some screwup in the
backup/restore program). So, all of those backups over the years had
successfully backed up a defective copy of the ROM image!
Had he some OTHER (summary) indication of the intended content of the
image file, the change to the file could have been noticed when it
occurred: "File hash mismatch for ROM123.hex."
[I.e., the hash acts as another check on the contents: original, original
hash, copy, computed hash of copy]
> So far, the ones that I've checked were all OK.
How did you "check" them? Did you do a bytewise compare between the "copy"
and the "original"? How long after this check was performed can you guarantee
that the "original" has not changed in the normal course of evolution? (i.e.,
is no longer an instance against which you can verify ALL of the files in the
"copy")? How often will you reexamine the "copy" to ensure that:
- it still spins up
- it is accessible, in its entirety
- its contents agree with what they *were* when the volume was created
- it is operating reliably (i.e., not likely to fail any time soon)
My scheme regularly examines every object in the archive to verify
that the media is still operational (i.e., reads don't throw lots of
CORRECTED errors) and accessible. And, that the contents haven't changed
(a file hasn't disappeared) or been corrupted (a file hasn't been altered...
perhaps by an accidental overwrite that wasn't reconciled with the intended
contents of the archive: "Yeah, here's the revised copy of FOO but you haven't
told the database that you updated it so it can't tell you which OTHER
instances also need to be updated -- IF that is what you actually intended!")
I do so opportunistically: when a volume is on-line because something on
the volume needs to be accessed (read or write), then take advantage of that
to resume checking the files that you hadn't FINISHED checking the last time
the volume was mounted. And, make a note of the progress you've made *as*
you do this (in the database! last_verified = 201703222310) so the process
can resume from that point regardless of how long the volume remains
"spinning", now.
As the RDBMS *server* is up 24/7/365 (because I use it for many different
"databases" as well as issuing queries to locate "files of interest"),
it can periodically be queried to determine which volumes haven't been
checked "often enough" and I get an email alerting me to the fact:
"Mount volume X as it is N days overdue for its M day verification cycle"
or:
"Mount optical medium 'tax history' to ensure it isn't suffering bit rot"
etc.
For volumes that don't happen to see "frequent" access (due to the nature of
their contents), this ensures that everything gets "verified" peridodically
BEFORE an access attempt turns up a read error, etc.
[A *scrubbing* RAID array can do essentially the same thing -- at a much higher
cost per terabyte -- without requiring the "human intervention" to mount
volumes "as requested"]
Back to sci.electronics.design | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-21 10:21 -0700
Re: big drives bitrex <bitrex@de.lete.earthlink.net> - 2017-03-21 13:47 -0400
Re: big drives George Herold <gherold@teachspin.com> - 2017-03-21 12:42 -0700
Re: big drives dagmargoodboat@yahoo.com - 2017-03-21 12:55 -0700
Re: big drives George Herold <gherold@teachspin.com> - 2017-03-21 13:37 -0700
Re: big drives bill.sloman@ieee.org - 2017-03-21 21:51 -0700
Re: big drives George Herold <gherold@teachspin.com> - 2017-03-22 07:26 -0700
Re: big drives Winfield Hill <hill@rowland.harvard.edu> - 2017-03-22 03:38 -0700
Re: big drives Don Kuenz <g@crcomp.net> - 2017-03-22 14:23 +0000
Re: big drives David Brown <david.brown@hesbynett.no> - 2017-03-22 16:43 +0100
Re: big drives Don Kuenz <g@crcomp.net> - 2017-03-22 18:35 +0000
Re: big drives David Brown <david.brown@hesbynett.no> - 2017-03-23 10:24 +0100
Re: big drives Don Kuenz <g@crcomp.net> - 2017-03-23 16:11 +0000
Re: big drives krw@notreal.com - 2017-03-23 13:05 -0400
Re: big drives Don Kuenz <g@crcomp.net> - 2017-03-23 17:39 +0000
Re: big drives krw@notreal.com - 2017-03-23 21:05 -0400
Re: big drives Don Kuenz <g@crcomp.net> - 2017-03-24 03:01 +0000
Re: big drives krw@notreal.com - 2017-03-23 23:09 -0400
Re: big drives Don Kuenz <g@crcomp.net> - 2017-03-24 04:15 +0000
Re: big drives krw@notreal.com - 2017-03-24 22:34 -0400
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-22 12:25 -0700
Re: big drives dplatt@coop.radagast.org (Dave Platt) - 2017-03-22 13:57 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-22 16:03 -0700
Re: big drives krw@notreal.com - 2017-03-22 20:29 -0400
Re: big drives tabbypurr@gmail.com - 2017-03-22 04:36 -0700
Re: big drives Spehro Pefhany <speffSNIP@interlogDOTyou.knowwhat> - 2017-03-22 08:57 -0400
Re: big drives Phil Hobbs <pcdhSpamMeSenseless@electrooptical.net> - 2017-03-22 11:59 -0400
Re: big drives krw@notreal.com - 2017-03-22 13:23 -0400
Re: big drives Jon Elson <jmelson@wustl.edu> - 2017-03-22 16:38 -0500
Re: big drives krw@notreal.com - 2017-03-22 20:31 -0400
Re: big drives gray_wolf <g_wolf@nospam.com> - 2017-03-23 20:59 -0500
Re: big drives gray_wolf <g_wolf@nospam.com> - 2017-03-23 21:00 -0500
Re: big drives Robert Baer <robertbaer@localnet.com> - 2017-03-23 22:21 -0800
Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-22 07:51 -0700
Re: big drives krw@notreal.com - 2017-03-22 13:23 -0400
Re: big drives Robert Baer <robertbaer@localnet.com> - 2017-03-23 22:19 -0800
Re: big drives tabbypurr@gmail.com - 2017-03-24 04:26 -0700
Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-24 08:38 -0700
Re: big drives Jon Elson <jmelson@wustl.edu> - 2017-03-22 16:37 -0500
Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-22 16:07 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-22 16:21 -0700
Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-22 16:45 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-22 23:21 -0700
Re: big drives John Larkin <jjlarkinxyxy@highlandtechnology.com> - 2017-03-24 13:06 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 16:56 -0700
Re: big drives John Larkin <jjlarkinxyxy@highlandtechnology.com> - 2017-03-24 17:35 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 21:47 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-22 16:19 -0700
Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-22 16:50 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-22 17:58 -0700
Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-22 21:00 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-22 22:50 -0700
Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-23 08:27 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-23 12:25 -0700
Re: big drives tabbypurr@gmail.com - 2017-03-24 04:31 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 06:38 -0700
Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-24 08:41 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 11:38 -0700
Re: big drives John Larkin <jjlarkinxyxy@highlandtechnology.com> - 2017-03-24 13:55 -0700
Re: big drives tabbypurr@gmail.com - 2017-03-24 17:14 -0700
Re: big drives John Larkin <jjlarkinxyxy@highlandtechnology.com> - 2017-03-24 17:41 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 18:00 -0700
Re: big drives Joerg <news@analogconsultants.com> - 2017-03-23 09:35 -0700
Re: big drives "Michael A. Terrell" <mike.terrell@earthlink.net> - 2017-03-23 18:38 -0400
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-23 16:18 -0700
Re: big drives Phil Hobbs <pcdhSpamMeSenseless@electrooptical.net> - 2017-03-23 18:29 -0400
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-23 18:38 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-23 18:54 -0700
Re: big drives pcdhobbs@gmail.com - 2017-03-23 19:16 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-23 19:28 -0700
Re: big drives pcdhobbs@gmail.com - 2017-03-23 19:55 -0700
Re: big drives krw@notreal.com - 2017-03-23 23:04 -0400
Re: big drives pcdhobbs@gmail.com - 2017-03-23 22:00 -0700
Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-24 08:46 -0700
Re: big drives krw@notreal.com - 2017-03-24 22:38 -0400
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-23 22:22 -0700
Re: big drives pcdhobbs@gmail.com - 2017-03-23 23:08 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 00:21 -0700
Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-24 09:00 -0700
Re: big drives David Brown <david.brown@hesbynett.no> - 2017-03-24 17:14 +0100
Re: big drives tabbypurr@gmail.com - 2017-03-24 09:19 -0700
Re: big drives David Brown <david.brown@hesbynett.no> - 2017-03-24 17:47 +0100
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 12:01 -0700
Re: big drives Jasen Betts <jasen@xnet.co.nz> - 2017-03-24 20:30 +0000
Re: big drives John Larkin <jjlarkinxyxy@highlandtechnology.com> - 2017-03-24 14:06 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 11:55 -0700
Re: big drives "Michael A. Terrell" <mike.terrell@earthlink.net> - 2017-03-25 04:00 -0400
Re: big drives upsidedown@downunder.com - 2017-03-24 22:18 +0200
Re: big drives John Larkin <jjlarkinxyxy@highlandtechnology.com> - 2017-03-24 14:15 -0700
Re: big drives Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2017-03-25 13:21 +0200
Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-25 07:54 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-25 10:53 -0700
Re: big drives upsidedown@downunder.com - 2017-03-25 23:11 +0200
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-25 15:14 -0700
Re: big drives upsidedown@downunder.com - 2017-03-26 12:18 +0300
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-26 03:33 -0700
Re: big drives Phil Hobbs <pcdhSpamMeSenseless@electrooptical.net> - 2017-03-26 16:36 -0400
Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-26 14:13 -0700
Re: big drives Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2017-03-25 21:04 +0200
Re: big drives pcdhobbs@gmail.com - 2017-03-24 19:03 -0700
Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-24 08:53 -0700
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 12:12 -0700
Re: big drives John Larkin <jjlarkinxyxy@highlandtechnology.com> - 2017-03-24 14:20 -0700
Re: big drives krw@notreal.com - 2017-03-24 22:43 -0400
Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-24 20:10 -0700
Re: big drives krw@notreal.com - 2017-03-24 23:23 -0400
Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-24 22:25 -0700
Re: big drives Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2017-03-25 13:23 +0200
Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-23 19:36 -0700
Re: big drives upsidedown@downunder.com - 2017-03-24 10:14 +0200
Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 01:31 -0700
Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-23 16:21 -0700
Re: big drives "Michael A. Terrell" <mike.terrell@earthlink.net> - 2017-03-24 08:54 -0400
csiph-web