Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > sci.electronics.design > #452905 > unrolled thread

big drives

Started byJohn Larkin <jjlarkin@highlandtechnology.com>
First post2017-03-21 10:21 -0700
Last post2017-03-24 08:54 -0400
Articles 20 on this page of 113 — 24 participants

Back to article view | Back to sci.electronics.design


Contents

  big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-21 10:21 -0700
    Re: big drives bitrex <bitrex@de.lete.earthlink.net> - 2017-03-21 13:47 -0400
      Re: big drives George Herold <gherold@teachspin.com> - 2017-03-21 12:42 -0700
        Re: big drives dagmargoodboat@yahoo.com - 2017-03-21 12:55 -0700
          Re: big drives George Herold <gherold@teachspin.com> - 2017-03-21 13:37 -0700
            Re: big drives bill.sloman@ieee.org - 2017-03-21 21:51 -0700
              Re: big drives George Herold <gherold@teachspin.com> - 2017-03-22 07:26 -0700
    Re: big drives Winfield Hill <hill@rowland.harvard.edu> - 2017-03-22 03:38 -0700
      Re: big drives Don Kuenz <g@crcomp.net> - 2017-03-22 14:23 +0000
        Re: big drives David Brown <david.brown@hesbynett.no> - 2017-03-22 16:43 +0100
          Re: big drives Don Kuenz <g@crcomp.net> - 2017-03-22 18:35 +0000
            Re: big drives David Brown <david.brown@hesbynett.no> - 2017-03-23 10:24 +0100
              Re: big drives Don Kuenz <g@crcomp.net> - 2017-03-23 16:11 +0000
                Re: big drives krw@notreal.com - 2017-03-23 13:05 -0400
                  Re: big drives Don Kuenz <g@crcomp.net> - 2017-03-23 17:39 +0000
                    Re: big drives krw@notreal.com - 2017-03-23 21:05 -0400
                      Re: big drives Don Kuenz <g@crcomp.net> - 2017-03-24 03:01 +0000
                        Re: big drives krw@notreal.com - 2017-03-23 23:09 -0400
                          Re: big drives Don Kuenz <g@crcomp.net> - 2017-03-24 04:15 +0000
                            Re: big drives krw@notreal.com - 2017-03-24 22:34 -0400
        Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-22 12:25 -0700
          Re: big drives dplatt@coop.radagast.org (Dave Platt) - 2017-03-22 13:57 -0700
            Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-22 16:03 -0700
          Re: big drives krw@notreal.com - 2017-03-22 20:29 -0400
    Re: big drives tabbypurr@gmail.com - 2017-03-22 04:36 -0700
      Re: big drives Spehro Pefhany <speffSNIP@interlogDOTyou.knowwhat> - 2017-03-22 08:57 -0400
        Re: big drives Phil Hobbs <pcdhSpamMeSenseless@electrooptical.net> - 2017-03-22 11:59 -0400
          Re: big drives krw@notreal.com - 2017-03-22 13:23 -0400
            Re: big drives Jon Elson <jmelson@wustl.edu> - 2017-03-22 16:38 -0500
              Re: big drives krw@notreal.com - 2017-03-22 20:31 -0400
              Re: big drives gray_wolf <g_wolf@nospam.com> - 2017-03-23 20:59 -0500
              Re: big drives gray_wolf <g_wolf@nospam.com> - 2017-03-23 21:00 -0500
          Re: big drives Robert Baer <robertbaer@localnet.com> - 2017-03-23 22:21 -0800
      Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-22 07:51 -0700
        Re: big drives krw@notreal.com - 2017-03-22 13:23 -0400
        Re: big drives Robert Baer <robertbaer@localnet.com> - 2017-03-23 22:19 -0800
        Re: big drives tabbypurr@gmail.com - 2017-03-24 04:26 -0700
          Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-24 08:38 -0700
      Re: big drives Jon Elson <jmelson@wustl.edu> - 2017-03-22 16:37 -0500
        Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-22 16:07 -0700
          Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-22 16:21 -0700
            Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-22 16:45 -0700
              Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-22 23:21 -0700
                Re: big drives John Larkin <jjlarkinxyxy@highlandtechnology.com> - 2017-03-24 13:06 -0700
                  Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 16:56 -0700
                    Re: big drives John Larkin <jjlarkinxyxy@highlandtechnology.com> - 2017-03-24 17:35 -0700
                      Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 21:47 -0700
        Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-22 16:19 -0700
          Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-22 16:50 -0700
            Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-22 17:58 -0700
              Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-22 21:00 -0700
                Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-22 22:50 -0700
                  Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-23 08:27 -0700
                    Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-23 12:25 -0700
        Re: big drives tabbypurr@gmail.com - 2017-03-24 04:31 -0700
          Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 06:38 -0700
            Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-24 08:41 -0700
              Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 11:38 -0700
                Re: big drives John Larkin <jjlarkinxyxy@highlandtechnology.com> - 2017-03-24 13:55 -0700
                Re: big drives tabbypurr@gmail.com - 2017-03-24 17:14 -0700
                  Re: big drives John Larkin <jjlarkinxyxy@highlandtechnology.com> - 2017-03-24 17:41 -0700
                  Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 18:00 -0700
    Re: big drives Joerg <news@analogconsultants.com> - 2017-03-23 09:35 -0700
    Re: big drives "Michael A. Terrell" <mike.terrell@earthlink.net> - 2017-03-23 18:38 -0400
      Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-23 16:18 -0700
        Re: big drives Phil Hobbs <pcdhSpamMeSenseless@electrooptical.net> - 2017-03-23 18:29 -0400
          Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-23 18:38 -0700
            Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-23 18:54 -0700
              Re: big drives pcdhobbs@gmail.com - 2017-03-23 19:16 -0700
                Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-23 19:28 -0700
                  Re: big drives pcdhobbs@gmail.com - 2017-03-23 19:55 -0700
                    Re: big drives krw@notreal.com - 2017-03-23 23:04 -0400
                      Re: big drives pcdhobbs@gmail.com - 2017-03-23 22:00 -0700
                        Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-24 08:46 -0700
                        Re: big drives krw@notreal.com - 2017-03-24 22:38 -0400
                    Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-23 22:22 -0700
                      Re: big drives pcdhobbs@gmail.com - 2017-03-23 23:08 -0700
                        Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 00:21 -0700
                          Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-24 09:00 -0700
                            Re: big drives David Brown <david.brown@hesbynett.no> - 2017-03-24 17:14 +0100
                            Re: big drives tabbypurr@gmail.com - 2017-03-24 09:19 -0700
                              Re: big drives David Brown <david.brown@hesbynett.no> - 2017-03-24 17:47 +0100
                              Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 12:01 -0700
                                Re: big drives Jasen Betts <jasen@xnet.co.nz> - 2017-03-24 20:30 +0000
                              Re: big drives John Larkin <jjlarkinxyxy@highlandtechnology.com> - 2017-03-24 14:06 -0700
                            Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 11:55 -0700
                              Re: big drives "Michael A. Terrell" <mike.terrell@earthlink.net> - 2017-03-25 04:00 -0400
                            Re: big drives upsidedown@downunder.com - 2017-03-24 22:18 +0200
                              Re: big drives John Larkin <jjlarkinxyxy@highlandtechnology.com> - 2017-03-24 14:15 -0700
                                Re: big drives Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2017-03-25 13:21 +0200
                                  Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-25 07:54 -0700
                                    Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-25 10:53 -0700
                                      Re: big drives upsidedown@downunder.com - 2017-03-25 23:11 +0200
                                        Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-25 15:14 -0700
                                          Re: big drives upsidedown@downunder.com - 2017-03-26 12:18 +0300
                                            Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-26 03:33 -0700
                                          Re: big drives Phil Hobbs <pcdhSpamMeSenseless@electrooptical.net> - 2017-03-26 16:36 -0400
                                            Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-26 14:13 -0700
                                    Re: big drives Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2017-03-25 21:04 +0200
                              Re: big drives pcdhobbs@gmail.com - 2017-03-24 19:03 -0700
                      Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-24 08:53 -0700
                        Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 12:12 -0700
                          Re: big drives John Larkin <jjlarkinxyxy@highlandtechnology.com> - 2017-03-24 14:20 -0700
                        Re: big drives krw@notreal.com - 2017-03-24 22:43 -0400
                          Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-24 20:10 -0700
                            Re: big drives krw@notreal.com - 2017-03-24 23:23 -0400
                              Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-24 22:25 -0700
                          Re: big drives Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2017-03-25 13:23 +0200
          Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-23 19:36 -0700
        Re: big drives upsidedown@downunder.com - 2017-03-24 10:14 +0200
          Re: big drives Don Y <blockedofcourse@foo.invalid> - 2017-03-24 01:31 -0700
      Re: big drives John Larkin <jjlarkin@highlandtechnology.com> - 2017-03-23 16:21 -0700
        Re: big drives "Michael A. Terrell" <mike.terrell@earthlink.net> - 2017-03-24 08:54 -0400

Page 3 of 6 — ← Prev page 1 2 [3] 4 5 6  Next page →


#453149

FromDon Y <blockedofcourse@foo.invalid>
Date2017-03-22 16:21 -0700
Message-ID<oav0p0$m1m$2@dont-email.me>
In reply to#453147
On 3/22/2017 4:07 PM, John Larkin wrote:
> On Wed, 22 Mar 2017 16:37:02 -0500, Jon Elson <jmelson@wustl.edu>
> wrote:
>
>> tabbypurr@gmail.com wrote:
>>
>>> On Tuesday, 21 March 2017 17:21:23 UTC, John Larkin  wrote:
>>>> I just happened to open my recycle bin and there was 42 Gbytes of
>>>> junk.
>>>>
>>>> I remember 256 Kbyte hard drives. A PC with a 10M drive was top-end.
>>>>
>>>> There's no point in reducing jpeg sizes, or even deleting stuff, any
>>>> more. I could never take enough pics, or create enough docs, to keep
>>>> up with the progress in hard drives. I suppose some people could
>>>> collect a large number of HD movies and use a terabyte somehow.
>>>>
>>>> It's impossible to crack one-time-pad encryption. Giant hard drives
>>>> and memory sticks make OTP files easy to physically hand off to
>>>> correspondents.
>>>
>>> I estimated the payback of deleting data some time last year. It came in
>>> at something like a penny or 2 an hour.
>> Well, the problem is I want physical, off-line backup of important stuff
>> (that's what I've created, not the OS, utilities, etc.) preferably located
>> in a couple places.  So, getting rid of excess junk and redundant on-line
>> backups makes some sense, in thaty it makes those CDs, then DVDs, now Blu-
>> Ray backups fit on available media.
>>
>> jon
>
> We back up to USB terabyte hard drives and use them as write-once
> media.

And what ongoing *policy* do you have to verify their contents are still
accessible (i.e., that the disk hasn't developed a flaw) *and* intact?
Do you checksum the entire drive?  All of its contents?  Hope keeping TWO
copies of that drive gives you *one* that is still intact, over time?

People have been making backups for decades.  And, being *disappointed* by
their backups, eventually, because they didn't invest in the entire solution,
just a portion of it.

[toc] | [prev] | [next] | [standalone]


#453150

FromJohn Larkin <jjlarkin@highlandtechnology.com>
Date2017-03-22 16:45 -0700
Message-ID<6o26dct4ihbqtg8l94crijgnou7hmapao0@4ax.com>
In reply to#453149
On Wed, 22 Mar 2017 16:21:45 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>On 3/22/2017 4:07 PM, John Larkin wrote:
>> On Wed, 22 Mar 2017 16:37:02 -0500, Jon Elson <jmelson@wustl.edu>
>> wrote:
>>
>>> tabbypurr@gmail.com wrote:
>>>
>>>> On Tuesday, 21 March 2017 17:21:23 UTC, John Larkin  wrote:
>>>>> I just happened to open my recycle bin and there was 42 Gbytes of
>>>>> junk.
>>>>>
>>>>> I remember 256 Kbyte hard drives. A PC with a 10M drive was top-end.
>>>>>
>>>>> There's no point in reducing jpeg sizes, or even deleting stuff, any
>>>>> more. I could never take enough pics, or create enough docs, to keep
>>>>> up with the progress in hard drives. I suppose some people could
>>>>> collect a large number of HD movies and use a terabyte somehow.
>>>>>
>>>>> It's impossible to crack one-time-pad encryption. Giant hard drives
>>>>> and memory sticks make OTP files easy to physically hand off to
>>>>> correspondents.
>>>>
>>>> I estimated the payback of deleting data some time last year. It came in
>>>> at something like a penny or 2 an hour.
>>> Well, the problem is I want physical, off-line backup of important stuff
>>> (that's what I've created, not the OS, utilities, etc.) preferably located
>>> in a couple places.  So, getting rid of excess junk and redundant on-line
>>> backups makes some sense, in thaty it makes those CDs, then DVDs, now Blu-
>>> Ray backups fit on available media.
>>>
>>> jon
>>
>> We back up to USB terabyte hard drives and use them as write-once
>> media.
>
>And what ongoing *policy* do you have to verify their contents are still
>accessible (i.e., that the disk hasn't developed a flaw) *and* intact?

Massive redundancy. If 90% of the off-site drives die, we'll still be
OK.

So far, the ones that I've checked were all OK.

>Do you checksum the entire drive?  All of its contents?  Hope keeping TWO
>copies of that drive gives you *one* that is still intact, over time?

Keeping dozens of monthly backup drives around the state is pretty
good insurance. We have a lot of rolling daily on-site backups, too.

>
>People have been making backups for decades.  And, being *disappointed* by
>their backups, eventually, because they didn't invest in the entire solution,
>just a portion of it.

I think we should do the frequent backups to Dropbox, in addition to
or instead of on-site, and still make the USB hard drive backups,
once a month, to scatter about California.


-- 

John Larkin         Highland Technology, Inc

jlarkin att highlandtechnology dott com
http://www.highlandtechnology.com

[toc] | [prev] | [next] | [standalone]


#453203

FromDon Y <blockedofcourse@foo.invalid>
Date2017-03-22 23:21 -0700
Message-ID<oavpbi$ac6$1@dont-email.me>
In reply to#453150
On 3/22/2017 4:45 PM, John Larkin wrote:
> On Wed, 22 Mar 2017 16:21:45 -0700, Don Y
> <blockedofcourse@foo.invalid> wrote:
>
>> On 3/22/2017 4:07 PM, John Larkin wrote:
>>> On Wed, 22 Mar 2017 16:37:02 -0500, Jon Elson <jmelson@wustl.edu>
>>> wrote:
>>>
>>>> tabbypurr@gmail.com wrote:
>>>>
>>>>> On Tuesday, 21 March 2017 17:21:23 UTC, John Larkin  wrote:
>>>>>> I just happened to open my recycle bin and there was 42 Gbytes of
>>>>>> junk.
>>>>>>
>>>>>> I remember 256 Kbyte hard drives. A PC with a 10M drive was top-end.
>>>>>>
>>>>>> There's no point in reducing jpeg sizes, or even deleting stuff, any
>>>>>> more. I could never take enough pics, or create enough docs, to keep
>>>>>> up with the progress in hard drives. I suppose some people could
>>>>>> collect a large number of HD movies and use a terabyte somehow.
>>>>>>
>>>>>> It's impossible to crack one-time-pad encryption. Giant hard drives
>>>>>> and memory sticks make OTP files easy to physically hand off to
>>>>>> correspondents.
>>>>>
>>>>> I estimated the payback of deleting data some time last year. It came in
>>>>> at something like a penny or 2 an hour.
>>>> Well, the problem is I want physical, off-line backup of important stuff
>>>> (that's what I've created, not the OS, utilities, etc.) preferably located
>>>> in a couple places.  So, getting rid of excess junk and redundant on-line
>>>> backups makes some sense, in thaty it makes those CDs, then DVDs, now Blu-
>>>> Ray backups fit on available media.
>>>>
>>>> jon
>>>
>>> We back up to USB terabyte hard drives and use them as write-once
>>> media.
>>
>> And what ongoing *policy* do you have to verify their contents are still
>> accessible (i.e., that the disk hasn't developed a flaw) *and* intact?
>
> Massive redundancy. If 90% of the off-site drives die, we'll still be
> OK.

I've a friend who thought copying all of his ROM images, etc. onto his
PC and routinely backing up his PC was a great way to ensure he could
always recreate ROMs.

Some years later, he was patching a ROM and botched the job.  No way to
"click undo" and recreate the original ROM.  *But*, hey, the image of
the ROM is present on his PC, right?

Imagine his dismay to discover that the image on the PC couldn't
possibly correlate with even the *bulk* of the botched ROM that he
had before him!  (locations that were "burned to 0" in the image but
NOT burned in the actual ROM)

Obviously, at some point in the past, he had copied the file incorrectly
(either by some manual operation on his part *or* some screwup in the
backup/restore program).  So, all of those backups over the years had
successfully backed up a defective copy of the ROM image!

Had he some OTHER (summary) indication of the intended content of the
image file, the change to the file could have been noticed when it
occurred:  "File hash mismatch for ROM123.hex."

[I.e., the hash acts as another check on the contents:  original, original
hash, copy, computed hash of copy]

> So far, the ones that I've checked were all OK.

How did you "check" them?  Did you do a bytewise compare between the "copy"
and the "original"?  How long after this check was performed can you guarantee
that the "original" has not changed in the normal course of evolution?  (i.e.,
is no longer an instance against which you can verify ALL of the files in the
"copy")?  How often will you reexamine the "copy" to ensure that:
- it still spins up
- it is accessible, in its entirety
- its contents agree with what they *were* when the volume was created
- it is operating reliably (i.e., not likely to fail any time soon)

My scheme regularly examines every object in the archive to verify
that the media is still operational (i.e., reads don't throw lots of
CORRECTED errors) and accessible.  And, that the contents haven't changed
(a file hasn't disappeared) or been corrupted (a file hasn't been altered...
perhaps by an accidental overwrite that wasn't reconciled with the intended
contents of the archive:  "Yeah, here's the revised copy of FOO but you haven't
told the database that you updated it so it can't tell you which OTHER
instances also need to be updated -- IF that is what you actually intended!")

I do so opportunistically:  when a volume is on-line because something on
the volume needs to be accessed (read or write), then take advantage of that
to resume checking the files that you hadn't FINISHED checking the last time
the volume was mounted.  And, make a note of the progress you've made *as*
you do this (in the database!  last_verified = 201703222310) so the process
can resume from that point regardless of how long the volume remains
"spinning", now.

As the RDBMS *server* is up 24/7/365 (because I use it for many different
"databases" as well as issuing queries to locate "files of interest"),
it can periodically be queried to determine which volumes haven't been
checked "often enough" and I get an email alerting me to the fact:
    "Mount volume X as it is N days overdue for its M day verification cycle"
or:
    "Mount optical medium 'tax history' to ensure it isn't suffering bit rot"
etc.

For volumes that don't happen to see "frequent" access (due to the nature of
their contents), this ensures that everything gets "verified" peridodically
BEFORE an access attempt turns up a read error, etc.

[A *scrubbing* RAID array can do essentially the same thing -- at a much higher
cost per terabyte -- without requiring the "human intervention" to mount
volumes "as requested"]

[toc] | [prev] | [next] | [standalone]


#453435

FromJohn Larkin <jjlarkinxyxy@highlandtechnology.com>
Date2017-03-24 13:06 -0700
Message-ID<mtuadcd89esq21qacd2kovqebckmcbchib@4ax.com>
In reply to#453203
On Wed, 22 Mar 2017 23:21:15 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>On 3/22/2017 4:45 PM, John Larkin wrote:
>> On Wed, 22 Mar 2017 16:21:45 -0700, Don Y
>> <blockedofcourse@foo.invalid> wrote:
>>
>>> On 3/22/2017 4:07 PM, John Larkin wrote:
>>>> On Wed, 22 Mar 2017 16:37:02 -0500, Jon Elson <jmelson@wustl.edu>
>>>> wrote:
>>>>
>>>>> tabbypurr@gmail.com wrote:
>>>>>
>>>>>> On Tuesday, 21 March 2017 17:21:23 UTC, John Larkin  wrote:
>>>>>>> I just happened to open my recycle bin and there was 42 Gbytes of
>>>>>>> junk.
>>>>>>>
>>>>>>> I remember 256 Kbyte hard drives. A PC with a 10M drive was top-end.
>>>>>>>
>>>>>>> There's no point in reducing jpeg sizes, or even deleting stuff, any
>>>>>>> more. I could never take enough pics, or create enough docs, to keep
>>>>>>> up with the progress in hard drives. I suppose some people could
>>>>>>> collect a large number of HD movies and use a terabyte somehow.
>>>>>>>
>>>>>>> It's impossible to crack one-time-pad encryption. Giant hard drives
>>>>>>> and memory sticks make OTP files easy to physically hand off to
>>>>>>> correspondents.
>>>>>>
>>>>>> I estimated the payback of deleting data some time last year. It came in
>>>>>> at something like a penny or 2 an hour.
>>>>> Well, the problem is I want physical, off-line backup of important stuff
>>>>> (that's what I've created, not the OS, utilities, etc.) preferably located
>>>>> in a couple places.  So, getting rid of excess junk and redundant on-line
>>>>> backups makes some sense, in thaty it makes those CDs, then DVDs, now Blu-
>>>>> Ray backups fit on available media.
>>>>>
>>>>> jon
>>>>
>>>> We back up to USB terabyte hard drives and use them as write-once
>>>> media.
>>>
>>> And what ongoing *policy* do you have to verify their contents are still
>>> accessible (i.e., that the disk hasn't developed a flaw) *and* intact?
>>
>> Massive redundancy. If 90% of the off-site drives die, we'll still be
>> OK.
>
>I've a friend who thought copying all of his ROM images, etc. onto his
>PC and routinely backing up his PC was a great way to ensure he could
>always recreate ROMs.
>
>Some years later, he was patching a ROM and botched the job.  No way to
>"click undo" and recreate the original ROM.  *But*, hey, the image of
>the ROM is present on his PC, right?
>
>Imagine his dismay to discover that the image on the PC couldn't
>possibly correlate with even the *bulk* of the botched ROM that he
>had before him!  (locations that were "burned to 0" in the image but
>NOT burned in the actual ROM)
>
>Obviously, at some point in the past, he had copied the file incorrectly
>(either by some manual operation on his part *or* some screwup in the
>backup/restore program).  So, all of those backups over the years had
>successfully backed up a defective copy of the ROM image!

That's the reason to often use write-once backup media, take then off
site, store them in ziplox bags in a cool dry place. If a file gets
corrupted, some old back will have the original version. It doesn't
make sense to resuse or ever write over backup drives.



-- 

John Larkin         Highland Technology, Inc
picosecond timing   precision measurement 

jlarkin att highlandtechnology dott com
http://www.highlandtechnology.com

[toc] | [prev] | [next] | [standalone]


#453472

FromDon Y <blockedofcourse@foo.invalid>
Date2017-03-24 16:56 -0700
Message-ID<ob4bih$sg$1@dont-email.me>
In reply to#453435
On 3/24/2017 1:06 PM, John Larkin wrote:
> On Wed, 22 Mar 2017 23:21:15 -0700, Don Y
> <blockedofcourse@foo.invalid> wrote:
>
>> On 3/22/2017 4:45 PM, John Larkin wrote:
>>> On Wed, 22 Mar 2017 16:21:45 -0700, Don Y
>>> <blockedofcourse@foo.invalid> wrote:
>>>
>>>> On 3/22/2017 4:07 PM, John Larkin wrote:
>>>>> On Wed, 22 Mar 2017 16:37:02 -0500, Jon Elson <jmelson@wustl.edu>
>>>>> wrote:
>>>>>
>>>>>> tabbypurr@gmail.com wrote:
>>>>>>
>>>>>>> On Tuesday, 21 March 2017 17:21:23 UTC, John Larkin  wrote:
>>>>>>>> I just happened to open my recycle bin and there was 42 Gbytes of
>>>>>>>> junk.
>>>>>>>>
>>>>>>>> I remember 256 Kbyte hard drives. A PC with a 10M drive was top-end.
>>>>>>>>
>>>>>>>> There's no point in reducing jpeg sizes, or even deleting stuff, any
>>>>>>>> more. I could never take enough pics, or create enough docs, to keep
>>>>>>>> up with the progress in hard drives. I suppose some people could
>>>>>>>> collect a large number of HD movies and use a terabyte somehow.
>>>>>>>>
>>>>>>>> It's impossible to crack one-time-pad encryption. Giant hard drives
>>>>>>>> and memory sticks make OTP files easy to physically hand off to
>>>>>>>> correspondents.
>>>>>>>
>>>>>>> I estimated the payback of deleting data some time last year. It came in
>>>>>>> at something like a penny or 2 an hour.
>>>>>> Well, the problem is I want physical, off-line backup of important stuff
>>>>>> (that's what I've created, not the OS, utilities, etc.) preferably located
>>>>>> in a couple places.  So, getting rid of excess junk and redundant on-line
>>>>>> backups makes some sense, in thaty it makes those CDs, then DVDs, now Blu-
>>>>>> Ray backups fit on available media.
>>>>>>
>>>>>> jon
>>>>>
>>>>> We back up to USB terabyte hard drives and use them as write-once
>>>>> media.
>>>>
>>>> And what ongoing *policy* do you have to verify their contents are still
>>>> accessible (i.e., that the disk hasn't developed a flaw) *and* intact?
>>>
>>> Massive redundancy. If 90% of the off-site drives die, we'll still be
>>> OK.
>>
>> I've a friend who thought copying all of his ROM images, etc. onto his
>> PC and routinely backing up his PC was a great way to ensure he could
>> always recreate ROMs.
>>
>> Some years later, he was patching a ROM and botched the job.  No way to
>> "click undo" and recreate the original ROM.  *But*, hey, the image of
>> the ROM is present on his PC, right?
>>
>> Imagine his dismay to discover that the image on the PC couldn't
>> possibly correlate with even the *bulk* of the botched ROM that he
>> had before him!  (locations that were "burned to 0" in the image but
>> NOT burned in the actual ROM)
>>
>> Obviously, at some point in the past, he had copied the file incorrectly
>> (either by some manual operation on his part *or* some screwup in the
>> backup/restore program).  So, all of those backups over the years had
>> successfully backed up a defective copy of the ROM image!
>
> That's the reason to often use write-once backup media, take then off
> site, store them in ziplox bags in a cool dry place. If a file gets
> corrupted, some old back will have the original version. It doesn't
> make sense to resuse or ever write over backup drives.

That only works if you can find a backup copy that is intact.  And,
have some way of proving (to yourself) that it is intact.

In my approach, that proof comes in the form of preserving the copy and
its hash (in separate places).

In my friend's case, having a ROM and an image that can't possibly correspond
with that ROM (i.e., proof that it is corrupt).

I have everything that I've ever created in electronic form (plus lots of
things that I've converted *to* electronic forms) in my career (and a bit
before).  But, I don't have every *backup* instance of those things -- just
the "originals" and copies thereof.

I make a daily backup of my "development database" (about 20GB) on alternating
drives, replacing the oldest version that exists on the drive (so, I can go
back a few months to see what changed).  But, only keep checkpointed snapshots
very infrequently -- in the VCS (changes since the most recent snapshot aren't
formally documented; they exist solely to let me move back and forth in time
*informally* to see what I did "the day before yesterday")

I contend that the flaw in most backup strategies is the lack of positive
confirmation that the "copies" are true representations of the contents at
the time they were created (i.e., bytewise compare to the originals), still
intact (haven't been altered, media hasn't degraded, etc.) and recoverable
(i.e., you know HOW to get the copy back despite LIKELY being in a state
of near panic when that need arises!)

Had my friend followed a similar approach as mine, he'd have known the instant
the ROM image(s) had been corrupted and would/could have taken corrective
action at that time.  Instead, he kept backing up "bad data" (10 copies of bad
data don't compensate for one *good* datum!)

[toc] | [prev] | [next] | [standalone]


#453476

FromJohn Larkin <jjlarkinxyxy@highlandtechnology.com>
Date2017-03-24 17:35 -0700
Message-ID<2fdbdc9bblir7fjuo1eoohs1nepm0ovq76@4ax.com>
In reply to#453472
On Fri, 24 Mar 2017 16:56:44 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>On 3/24/2017 1:06 PM, John Larkin wrote:
>> On Wed, 22 Mar 2017 23:21:15 -0700, Don Y
>> <blockedofcourse@foo.invalid> wrote:
>>
>>> On 3/22/2017 4:45 PM, John Larkin wrote:
>>>> On Wed, 22 Mar 2017 16:21:45 -0700, Don Y
>>>> <blockedofcourse@foo.invalid> wrote:
>>>>
>>>>> On 3/22/2017 4:07 PM, John Larkin wrote:
>>>>>> On Wed, 22 Mar 2017 16:37:02 -0500, Jon Elson <jmelson@wustl.edu>
>>>>>> wrote:
>>>>>>
>>>>>>> tabbypurr@gmail.com wrote:
>>>>>>>
>>>>>>>> On Tuesday, 21 March 2017 17:21:23 UTC, John Larkin  wrote:
>>>>>>>>> I just happened to open my recycle bin and there was 42 Gbytes of
>>>>>>>>> junk.
>>>>>>>>>
>>>>>>>>> I remember 256 Kbyte hard drives. A PC with a 10M drive was top-end.
>>>>>>>>>
>>>>>>>>> There's no point in reducing jpeg sizes, or even deleting stuff, any
>>>>>>>>> more. I could never take enough pics, or create enough docs, to keep
>>>>>>>>> up with the progress in hard drives. I suppose some people could
>>>>>>>>> collect a large number of HD movies and use a terabyte somehow.
>>>>>>>>>
>>>>>>>>> It's impossible to crack one-time-pad encryption. Giant hard drives
>>>>>>>>> and memory sticks make OTP files easy to physically hand off to
>>>>>>>>> correspondents.
>>>>>>>>
>>>>>>>> I estimated the payback of deleting data some time last year. It came in
>>>>>>>> at something like a penny or 2 an hour.
>>>>>>> Well, the problem is I want physical, off-line backup of important stuff
>>>>>>> (that's what I've created, not the OS, utilities, etc.) preferably located
>>>>>>> in a couple places.  So, getting rid of excess junk and redundant on-line
>>>>>>> backups makes some sense, in thaty it makes those CDs, then DVDs, now Blu-
>>>>>>> Ray backups fit on available media.
>>>>>>>
>>>>>>> jon
>>>>>>
>>>>>> We back up to USB terabyte hard drives and use them as write-once
>>>>>> media.
>>>>>
>>>>> And what ongoing *policy* do you have to verify their contents are still
>>>>> accessible (i.e., that the disk hasn't developed a flaw) *and* intact?
>>>>
>>>> Massive redundancy. If 90% of the off-site drives die, we'll still be
>>>> OK.
>>>
>>> I've a friend who thought copying all of his ROM images, etc. onto his
>>> PC and routinely backing up his PC was a great way to ensure he could
>>> always recreate ROMs.
>>>
>>> Some years later, he was patching a ROM and botched the job.  No way to
>>> "click undo" and recreate the original ROM.  *But*, hey, the image of
>>> the ROM is present on his PC, right?
>>>
>>> Imagine his dismay to discover that the image on the PC couldn't
>>> possibly correlate with even the *bulk* of the botched ROM that he
>>> had before him!  (locations that were "burned to 0" in the image but
>>> NOT burned in the actual ROM)
>>>
>>> Obviously, at some point in the past, he had copied the file incorrectly
>>> (either by some manual operation on his part *or* some screwup in the
>>> backup/restore program).  So, all of those backups over the years had
>>> successfully backed up a defective copy of the ROM image!
>>
>> That's the reason to often use write-once backup media, take then off
>> site, store them in ziplox bags in a cool dry place. If a file gets
>> corrupted, some old back will have the original version. It doesn't
>> make sense to resuse or ever write over backup drives.
>
>That only works if you can find a backup copy that is intact.  And,
>have some way of proving (to yourself) that it is intact.

I typically have dozens to try. We do a full off-site backup every
month.

It's like RAID, but the drives are scattered all over California.

If I can read the file that I'm after, it must be intact, as-written.
I could always compare it to other backups if I suspect anything might
be wrong, such as it being an accurate backup of a corrupted file. If
it is, I look at older backup drives to see the history. It's not hard
to spot files that were corrupted before being backed up. I'm only
looking at them because someone thinks that the in-house file IS
corrupted.


>
>In my approach, that proof comes in the form of preserving the copy and
>its hash (in separate places).

How many off-site backups do you keep?

Do you ever write to a backup drive once it's been taken off site?



>
>In my friend's case, having a ROM and an image that can't possibly correspond
>with that ROM (i.e., proof that it is corrupt).
>
>I have everything that I've ever created in electronic form (plus lots of
>things that I've converted *to* electronic forms) in my career (and a bit
>before).  But, I don't have every *backup* instance of those things -- just
>the "originals" and copies thereof.
>
>I make a daily backup of my "development database" (about 20GB) on alternating
>drives, replacing the oldest version that exists on the drive (so, I can go
>back a few months to see what changed).  But, only keep checkpointed snapshots
>very infrequently -- in the VCS (changes since the most recent snapshot aren't
>formally documented; they exist solely to let me move back and forth in time
>*informally* to see what I did "the day before yesterday")

Sounds like you over-write backup files. We never do that. We write a
full company backup to a USB drive, then take it off site for storage.

I must dig into the backups to recover a file, on average, once a year
maybe.




-- 

John Larkin         Highland Technology, Inc
picosecond timing   precision measurement 

jlarkin att highlandtechnology dott com
http://www.highlandtechnology.com

[toc] | [prev] | [next] | [standalone]


#453512

FromDon Y <blockedofcourse@foo.invalid>
Date2017-03-24 21:47 -0700
Message-ID<ob4sj1$1bs$1@dont-email.me>
In reply to#453476
On 3/24/2017 5:35 PM, John Larkin wrote:
>> In my approach, that proof comes in the form of preserving the copy and
>> its hash (in separate places).
>
> How many off-site backups do you keep?

Depends on *what* is being backed up.  E.g., I have *one* backup of my
music archive.  It's here with the original -- and the CD's from which
it was ripped.  If the house burns to the ground, I'll lose all of it.
And, given the enormity of a house fire, worrying about lost music
doesn't even rate on the "Oh-that's-too-bad" scale!  :>

OTOH, there are multiple copies of all our financial records, old
taxes, etc. on different *types* of media (magnetic disk, MO, optical)
along with the most current backup (i.e., every time we pay a bill)
on a thumb drive (so we can put it in our pocket as we "bug out").

"Bulk" offsites are in the safe deposit box.

Every copy of a file has an associated "preciousness".  E.g., the copy
of my 2016 taxes on my workstation is far less precious than any of the
copies in the archive.  The archive is less precious than the copy on the
bug-out thumb drive.  The thumb drive copy is less precious than the
safe deposit box copy.

Preciousness is roughly inversely proportional to convenience of access.
I.e., I don't need to expend much effort to get at the copy of my tax
return that's spinning in the office, as I write this (I'd have to get
up and walk into the room).  To access any of the archive copies, I'd
have to query the database to figure out which volumes hold copies,
spin up those volumes and pull the file over to a workstation.  To access
the offsite, I'd have to wait for the bank to open, drive there, wait
to be escorted into the vault, fetch the medium, drag it out to the car,
mount it on the laptop, extract the copy, wander back into the bank, etc.

Offsites have historically included all "business" work -- i.e., copies
of "originals" that CLIENTS should be maintaining on their site(s).
The beauty of the database catalog approach is that it lets me quickly
determine how many instances of a file exist in my "collections"
and where it is located -- regardless of its filename by relying
on (hash, size) as a crude "identifier" for each "unique" file.

So, when I have every medium added to the database (e.g., including
CD's, DVD's, MO's, tapes, etc.) I can figure out what is redundant
and targeted for deletion (I've alerted clients that I'll no longer be
maintaining copies of past work efforts for them).  Then, reclaim the media
involved (e.g., if I have a DVD image of a client's work that contains
nothing *else* that I might want to retain, I can shred the DVD knowing
that I'm not "losing anything".

> Do you ever write to a backup drive once it's been taken off site?

Yes.  I swap the offsite drive(s) for newer versions of the same.
But, typically only when some "signficant" design stage is reached.
Note that my archive is measured in dozens of terabytes.  Keeping
multiple offsites in addition to multiple local copies just becomes
a file management nightmare.

I rely on local backups for more frequent updates being willing
to trade convenience (the effort of building a new COMPLETE backup
and swapping it for the stuff in the safe deposit box) for risk
control (no one will lose their job/livelihood -- even if the house,
all my equipment/prototypes *AND* bank vault burn to the ground!).

[toc] | [prev] | [next] | [standalone]


#453148

FromDon Y <blockedofcourse@foo.invalid>
Date2017-03-22 16:19 -0700
Message-ID<oav0jt$m1m$1@dont-email.me>
In reply to#453134
On 3/22/2017 2:37 PM, Jon Elson wrote:
> tabbypurr@gmail.com wrote:
>
>> On Tuesday, 21 March 2017 17:21:23 UTC, John Larkin  wrote:
>>> I just happened to open my recycle bin and there was 42 Gbytes of
>>> junk.
>>>
>>> I remember 256 Kbyte hard drives. A PC with a 10M drive was top-end.
>>>
>>> There's no point in reducing jpeg sizes, or even deleting stuff, any
>>> more. I could never take enough pics, or create enough docs, to keep
>>> up with the progress in hard drives. I suppose some people could
>>> collect a large number of HD movies and use a terabyte somehow.
>>>
>>> It's impossible to crack one-time-pad encryption. Giant hard drives
>>> and memory sticks make OTP files easy to physically hand off to
>>> correspondents.
>>
>> I estimated the payback of deleting data some time last year. It came in
>> at something like a penny or 2 an hour.
> Well, the problem is I want physical, off-line backup of important stuff
> (that's what I've created, not the OS, utilities, etc.) preferably located
> in a couple places.  So, getting rid of excess junk and redundant on-line
> backups makes some sense, in thaty it makes those CDs, then DVDs, now Blu-
> Ray backups fit on available media.

I've hacked together a "system" (not a "device") whereby I track "files"
(for want of a better term as it can mean finer grained things, as well)
and their "locations" (volume, directory/container, etc.) in a relational
database tagged with appropriate metadata (whatever *I* deem important
so I can add notes like "this is a list of important telephone numbers")
including a hash of the object's content.

The "location" can be any identifiable medium (e.g., a spinning magnetic disk,
CD, DVD, MO, etc.).  The hash and size let a script rummage through the
"manifest" to identify likely duplicates of "files".  A second pass can
further confirm this ("well, the size and hash are the same but the files'
contents differ so no match")

So, I can automatically notice that <volumeX>/Taxes/1978/Federal/MyTaxes
is actually the same file as <volumeY>/Business/Taxes/Federal/1978.

This lets me query the RDBMS to determine *where* something is likely
to reside ("There's a copy on <volumeQ> and a CD labeled 'MyStuff'")
so I can decide which copy I might want to access (if <volumeQ> happens
to be spinning on a machine right now, then it would obviously be the
easiest thing to access!).

Additionally, it lets me regularly verify the accessibility and
integrity of every cataloged object:  "The hash of this object SHOULD BE
as indicated in the RDBMS.  Is it?"  And, if a discrepancy is found,
alert me to the fact that an object has been corrupted and advise me
as to where I can find a copy of it that could be used to recover/restore
its contents (assuming the medium itself isn't suspect).

*And*, I don't have to keep all of this "on-line" (as would be the case
with a RAID configuration).

[toc] | [prev] | [next] | [standalone]


#453151

FromJohn Larkin <jjlarkin@highlandtechnology.com>
Date2017-03-22 16:50 -0700
Message-ID<m536dcl3u2tkkrd6qv5mnu534bpim00uqb@4ax.com>
In reply to#453148
On Wed, 22 Mar 2017 16:19:01 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>On 3/22/2017 2:37 PM, Jon Elson wrote:
>> tabbypurr@gmail.com wrote:
>>
>>> On Tuesday, 21 March 2017 17:21:23 UTC, John Larkin  wrote:
>>>> I just happened to open my recycle bin and there was 42 Gbytes of
>>>> junk.
>>>>
>>>> I remember 256 Kbyte hard drives. A PC with a 10M drive was top-end.
>>>>
>>>> There's no point in reducing jpeg sizes, or even deleting stuff, any
>>>> more. I could never take enough pics, or create enough docs, to keep
>>>> up with the progress in hard drives. I suppose some people could
>>>> collect a large number of HD movies and use a terabyte somehow.
>>>>
>>>> It's impossible to crack one-time-pad encryption. Giant hard drives
>>>> and memory sticks make OTP files easy to physically hand off to
>>>> correspondents.
>>>
>>> I estimated the payback of deleting data some time last year. It came in
>>> at something like a penny or 2 an hour.
>> Well, the problem is I want physical, off-line backup of important stuff
>> (that's what I've created, not the OS, utilities, etc.) preferably located
>> in a couple places.  So, getting rid of excess junk and redundant on-line
>> backups makes some sense, in thaty it makes those CDs, then DVDs, now Blu-
>> Ray backups fit on available media.
>
>I've hacked together a "system" (not a "device") whereby I track "files"
>(for want of a better term as it can mean finer grained things, as well)
>and their "locations" (volume, directory/container, etc.) in a relational
>database tagged with appropriate metadata (whatever *I* deem important
>so I can add notes like "this is a list of important telephone numbers")
>including a hash of the object's content.

I always thought that any file system ought to have a notes field in
each file header block. Like "what is this?"

I see so many files - Spice, schematics, READMEs, tech notes, pics,
source code - that has zero context. No author, no date, no rev ID, no
statement of WTF this IS.


-- 

John Larkin         Highland Technology, Inc

jlarkin att highlandtechnology dott com
http://www.highlandtechnology.com

[toc] | [prev] | [next] | [standalone]


#453161

FromDon Y <blockedofcourse@foo.invalid>
Date2017-03-22 17:58 -0700
Message-ID<oav6et$5nr$1@dont-email.me>
In reply to#453151
On 3/22/2017 4:50 PM, John Larkin wrote:
> On Wed, 22 Mar 2017 16:19:01 -0700, Don Y
> <blockedofcourse@foo.invalid> wrote:
>
>> On 3/22/2017 2:37 PM, Jon Elson wrote:
>>> tabbypurr@gmail.com wrote:
>>>
>>>> On Tuesday, 21 March 2017 17:21:23 UTC, John Larkin  wrote:
>>>>> I just happened to open my recycle bin and there was 42 Gbytes of
>>>>> junk.
>>>>>
>>>>> I remember 256 Kbyte hard drives. A PC with a 10M drive was top-end.
>>>>>
>>>>> There's no point in reducing jpeg sizes, or even deleting stuff, any
>>>>> more. I could never take enough pics, or create enough docs, to keep
>>>>> up with the progress in hard drives. I suppose some people could
>>>>> collect a large number of HD movies and use a terabyte somehow.
>>>>>
>>>>> It's impossible to crack one-time-pad encryption. Giant hard drives
>>>>> and memory sticks make OTP files easy to physically hand off to
>>>>> correspondents.
>>>>
>>>> I estimated the payback of deleting data some time last year. It came in
>>>> at something like a penny or 2 an hour.
>>> Well, the problem is I want physical, off-line backup of important stuff
>>> (that's what I've created, not the OS, utilities, etc.) preferably located
>>> in a couple places.  So, getting rid of excess junk and redundant on-line
>>> backups makes some sense, in thaty it makes those CDs, then DVDs, now Blu-
>>> Ray backups fit on available media.
>>
>> I've hacked together a "system" (not a "device") whereby I track "files"
>> (for want of a better term as it can mean finer grained things, as well)
>> and their "locations" (volume, directory/container, etc.) in a relational
>> database tagged with appropriate metadata (whatever *I* deem important
>> so I can add notes like "this is a list of important telephone numbers")
>> including a hash of the object's content.
>
> I always thought that any file system ought to have a notes field in
> each file header block. Like "what is this?"
>
> I see so many files - Spice, schematics, READMEs, tech notes, pics,
> source code - that has zero context. No author, no date, no rev ID, no
> statement of WTF this IS.

It goes far beyond just adding "comments".

Once you have a "catalog" (database) of all your files, you can assign them
unique identifiers (the RDBMS does it *for* you).  And, you can then exploit
the relational capabilities of the RDBMS to create *new* "catalogs"/collections
based on whatever YOU think to be appropriate!

So, I can note that objects 23, 3049, 345345, 5563, 456456, 30302221 are
"songs".  And, that songs have attributes like "artist", "duration", "genre",
etc.  I can then build a catalog (table) of "songs" wherein the first
"column" is an "object identifier" -- that links directly to the OID in the
"main FILE catalog".  The other "columns" can be whatever I deem appropriate
FOR SONGS!  (it would be silly to add a "duration" column to the main file
catalog as a file that isn't a song wouldn't have any notion of "duration"!)

Files that happen to be photographs (e.g., 48846, 858473, 99223 and 7483002)
can be *referenced* in a "photographs" catalog (table) -- accompanied by
attributes (columns) like "date taken", "dimensions", "location", "camera",
etc.

Artwork for circuit boards can be collected in another table that carries
annotations (attributes) for the associated "schematic" file, BoM file,
revision, etc.

I now more aggressively document the evolution of my designs by leveraging this
ability; I can retrieve earlier versions of code/schematics/layouts and see
what I found lacking in each.

Etc.

[E.g., my baking/cooking recipes are all cataloged.  There are many versions
of each of them.  Some have simple annotations:  "This is the style that Joe
likes!" or "No tomato products for D&J!".  Others are more cryptic/technical:
"This builds on the XYZ version but replaces the baking powder with soda and
tartar" (many of my recipes are in a CONSTANT state of flux -- e.g., my
biscotti Rx probably exists in more than 100 variations!).  This lets me see
what I've tried in the past as well as what I found "lacking" in it:  "This is
the *BEST* version but it is considerably more involved to produce; use only if
you are really after a primo result!"]

So, you can access the catalog that is of most interest to you and use *it*
to give you handles on the appropriate information (attributes) that you
can then chase down to find the file(s) of interest in the archive.

"Show me all artwork for project FOO..."
"Show me all songs by artist BAZ..."
"Show me the oldest revision of the specification for BLAH..."

[Searching free-form comments is tedious and usually ineffective.  It
requires a lot of discipline in the way that you describe things *or*
a lot of smarts in your search engine (e.g., are "project FOO", "proj FOO"
and "FOO" all suitable matches for the first of these examples above?
What if "FOO" was "APPLE"?  Would it conflict with a comment describing a
photograph of an *apple*?)]

[toc] | [prev] | [next] | [standalone]


#453184

FromJohn Larkin <jjlarkin@highlandtechnology.com>
Date2017-03-22 21:00 -0700
Message-ID<slh6dctp51cj813bmi4lqrvlu8pqr7qqfo@4ax.com>
In reply to#453161
On Wed, 22 Mar 2017 17:58:45 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>On 3/22/2017 4:50 PM, John Larkin wrote:
>> On Wed, 22 Mar 2017 16:19:01 -0700, Don Y
>> <blockedofcourse@foo.invalid> wrote:
>>
>>> On 3/22/2017 2:37 PM, Jon Elson wrote:
>>>> tabbypurr@gmail.com wrote:
>>>>
>>>>> On Tuesday, 21 March 2017 17:21:23 UTC, John Larkin  wrote:
>>>>>> I just happened to open my recycle bin and there was 42 Gbytes of
>>>>>> junk.
>>>>>>
>>>>>> I remember 256 Kbyte hard drives. A PC with a 10M drive was top-end.
>>>>>>
>>>>>> There's no point in reducing jpeg sizes, or even deleting stuff, any
>>>>>> more. I could never take enough pics, or create enough docs, to keep
>>>>>> up with the progress in hard drives. I suppose some people could
>>>>>> collect a large number of HD movies and use a terabyte somehow.
>>>>>>
>>>>>> It's impossible to crack one-time-pad encryption. Giant hard drives
>>>>>> and memory sticks make OTP files easy to physically hand off to
>>>>>> correspondents.
>>>>>
>>>>> I estimated the payback of deleting data some time last year. It came in
>>>>> at something like a penny or 2 an hour.
>>>> Well, the problem is I want physical, off-line backup of important stuff
>>>> (that's what I've created, not the OS, utilities, etc.) preferably located
>>>> in a couple places.  So, getting rid of excess junk and redundant on-line
>>>> backups makes some sense, in thaty it makes those CDs, then DVDs, now Blu-
>>>> Ray backups fit on available media.
>>>
>>> I've hacked together a "system" (not a "device") whereby I track "files"
>>> (for want of a better term as it can mean finer grained things, as well)
>>> and their "locations" (volume, directory/container, etc.) in a relational
>>> database tagged with appropriate metadata (whatever *I* deem important
>>> so I can add notes like "this is a list of important telephone numbers")
>>> including a hash of the object's content.
>>
>> I always thought that any file system ought to have a notes field in
>> each file header block. Like "what is this?"
>>
>> I see so many files - Spice, schematics, READMEs, tech notes, pics,
>> source code - that has zero context. No author, no date, no rev ID, no
>> statement of WTF this IS.
>
>It goes far beyond just adding "comments".
>
>Once you have a "catalog" (database) of all your files, you can assign them
>unique identifiers (the RDBMS does it *for* you).  And, you can then exploit
>the relational capabilities of the RDBMS to create *new* "catalogs"/collections
>based on whatever YOU think to be appropriate!
>
>So, I can note that objects 23, 3049, 345345, 5563, 456456, 30302221 are
>"songs".  And, that songs have attributes like "artist", "duration", "genre",
>etc.  I can then build a catalog (table) of "songs" wherein the first
>"column" is an "object identifier" -- that links directly to the OID in the
>"main FILE catalog".  The other "columns" can be whatever I deem appropriate
>FOR SONGS!  (it would be silly to add a "duration" column to the main file
>catalog as a file that isn't a song wouldn't have any notion of "duration"!)
>
>Files that happen to be photographs (e.g., 48846, 858473, 99223 and 7483002)
>can be *referenced* in a "photographs" catalog (table) -- accompanied by
>attributes (columns) like "date taken", "dimensions", "location", "camera",
>etc.
>
>Artwork for circuit boards can be collected in another table that carries
>annotations (attributes) for the associated "schematic" file, BoM file,
>revision, etc.
>
>I now more aggressively document the evolution of my designs by leveraging this
>ability; I can retrieve earlier versions of code/schematics/layouts and see
>what I found lacking in each.
>
>Etc.
>
>[E.g., my baking/cooking recipes are all cataloged.  There are many versions
>of each of them.  Some have simple annotations:  "This is the style that Joe
>likes!" or "No tomato products for D&J!".  Others are more cryptic/technical:
>"This builds on the XYZ version but replaces the baking powder with soda and
>tartar" (many of my recipes are in a CONSTANT state of flux -- e.g., my
>biscotti Rx probably exists in more than 100 variations!).  This lets me see
>what I've tried in the past as well as what I found "lacking" in it:  "This is
>the *BEST* version but it is considerably more involved to produce; use only if
>you are really after a primo result!"]
>
>So, you can access the catalog that is of most interest to you and use *it*
>to give you handles on the appropriate information (attributes) that you
>can then chase down to find the file(s) of interest in the archive.
>
>"Show me all artwork for project FOO..."
>"Show me all songs by artist BAZ..."
>"Show me the oldest revision of the specification for BLAH..."
>
>[Searching free-form comments is tedious and usually ineffective.  It
>requires a lot of discipline in the way that you describe things *or*
>a lot of smarts in your search engine (e.g., are "project FOO", "proj FOO"
>and "FOO" all suitable matches for the first of these examples above?
>What if "FOO" was "APPLE"?  Would it conflict with a comment describing a
>photograph of an *apple*?)]

What happens when you email a file to someone else? Or put together a
separate project folder, like to a contract manufacturer or a
consultant? You can't include the database.

I include the basics in every possible file; title, author, rev, date,
description, maybe a scope/introduction if that makes sense. Often a
rev history.

Databases too often become toys. And lots of people absolutely hate
comments.




-- 

John Larkin         Highland Technology, Inc

jlarkin att highlandtechnology dott com
http://www.highlandtechnology.com

[toc] | [prev] | [next] | [standalone]


#453200

FromDon Y <blockedofcourse@foo.invalid>
Date2017-03-22 22:50 -0700
Message-ID<oavni1$6d0$1@dont-email.me>
In reply to#453184
On 3/22/2017 9:00 PM, John Larkin wrote:
> On Wed, 22 Mar 2017 17:58:45 -0700, Don Y
> <blockedofcourse@foo.invalid> wrote:
>
>> On 3/22/2017 4:50 PM, John Larkin wrote:
>>> On Wed, 22 Mar 2017 16:19:01 -0700, Don Y
>>> <blockedofcourse@foo.invalid> wrote:
>>>
>>>> On 3/22/2017 2:37 PM, Jon Elson wrote:
>>>>> tabbypurr@gmail.com wrote:
>>>>>
>>>>>> On Tuesday, 21 March 2017 17:21:23 UTC, John Larkin  wrote:
>>>>>>> I just happened to open my recycle bin and there was 42 Gbytes of
>>>>>>> junk.
>>>>>>>
>>>>>>> I remember 256 Kbyte hard drives. A PC with a 10M drive was top-end.
>>>>>>>
>>>>>>> There's no point in reducing jpeg sizes, or even deleting stuff, any
>>>>>>> more. I could never take enough pics, or create enough docs, to keep
>>>>>>> up with the progress in hard drives. I suppose some people could
>>>>>>> collect a large number of HD movies and use a terabyte somehow.
>>>>>>>
>>>>>>> It's impossible to crack one-time-pad encryption. Giant hard drives
>>>>>>> and memory sticks make OTP files easy to physically hand off to
>>>>>>> correspondents.
>>>>>>
>>>>>> I estimated the payback of deleting data some time last year. It came in
>>>>>> at something like a penny or 2 an hour.
>>>>> Well, the problem is I want physical, off-line backup of important stuff
>>>>> (that's what I've created, not the OS, utilities, etc.) preferably located
>>>>> in a couple places.  So, getting rid of excess junk and redundant on-line
>>>>> backups makes some sense, in thaty it makes those CDs, then DVDs, now Blu-
>>>>> Ray backups fit on available media.
>>>>
>>>> I've hacked together a "system" (not a "device") whereby I track "files"
>>>> (for want of a better term as it can mean finer grained things, as well)
>>>> and their "locations" (volume, directory/container, etc.) in a relational
>>>> database tagged with appropriate metadata (whatever *I* deem important
>>>> so I can add notes like "this is a list of important telephone numbers")
>>>> including a hash of the object's content.
>>>
>>> I always thought that any file system ought to have a notes field in
>>> each file header block. Like "what is this?"
>>>
>>> I see so many files - Spice, schematics, READMEs, tech notes, pics,
>>> source code - that has zero context. No author, no date, no rev ID, no
>>> statement of WTF this IS.
>>
>> It goes far beyond just adding "comments".
>>
>> Once you have a "catalog" (database) of all your files, you can assign them
>> unique identifiers (the RDBMS does it *for* you).  And, you can then exploit
>> the relational capabilities of the RDBMS to create *new* "catalogs"/collections
>> based on whatever YOU think to be appropriate!
>>
>> So, I can note that objects 23, 3049, 345345, 5563, 456456, 30302221 are
>> "songs".  And, that songs have attributes like "artist", "duration", "genre",
>> etc.  I can then build a catalog (table) of "songs" wherein the first
>> "column" is an "object identifier" -- that links directly to the OID in the
>> "main FILE catalog".  The other "columns" can be whatever I deem appropriate
>> FOR SONGS!  (it would be silly to add a "duration" column to the main file
>> catalog as a file that isn't a song wouldn't have any notion of "duration"!)
>>
>> Files that happen to be photographs (e.g., 48846, 858473, 99223 and 7483002)
>> can be *referenced* in a "photographs" catalog (table) -- accompanied by
>> attributes (columns) like "date taken", "dimensions", "location", "camera",
>> etc.
>>
>> Artwork for circuit boards can be collected in another table that carries
>> annotations (attributes) for the associated "schematic" file, BoM file,
>> revision, etc.
>>
>> I now more aggressively document the evolution of my designs by leveraging this
>> ability; I can retrieve earlier versions of code/schematics/layouts and see
>> what I found lacking in each.
>>
>> Etc.
>>
>> [E.g., my baking/cooking recipes are all cataloged.  There are many versions
>> of each of them.  Some have simple annotations:  "This is the style that Joe
>> likes!" or "No tomato products for D&J!".  Others are more cryptic/technical:
>> "This builds on the XYZ version but replaces the baking powder with soda and
>> tartar" (many of my recipes are in a CONSTANT state of flux -- e.g., my
>> biscotti Rx probably exists in more than 100 variations!).  This lets me see
>> what I've tried in the past as well as what I found "lacking" in it:  "This is
>> the *BEST* version but it is considerably more involved to produce; use only if
>> you are really after a primo result!"]
>>
>> So, you can access the catalog that is of most interest to you and use *it*
>> to give you handles on the appropriate information (attributes) that you
>> can then chase down to find the file(s) of interest in the archive.
>>
>> "Show me all artwork for project FOO..."
>> "Show me all songs by artist BAZ..."
>> "Show me the oldest revision of the specification for BLAH..."
>>
>> [Searching free-form comments is tedious and usually ineffective.  It
>> requires a lot of discipline in the way that you describe things *or*
>> a lot of smarts in your search engine (e.g., are "project FOO", "proj FOO"
>> and "FOO" all suitable matches for the first of these examples above?
>> What if "FOO" was "APPLE"?  Would it conflict with a comment describing a
>> photograph of an *apple*?)]
>
> What happens when you email a file to someone else? Or put together a
> separate project folder, like to a contract manufacturer or a
> consultant? You can't include the database.

Will that contract manufacturer be able to recognize your "augmented"
file system?

That's why you include a MANIFEST file in any package of electronic objects.
There's no portable way to examine files of varying formats for "comments"
embedded within.  How do you examine the revision, date, author, etc. of
a photograph?  Of an AutoCAD 3D model?  Of a DSO capture?  etc.

[I.e., you require specialty "viewers" for each of those file types/formats.]

> I include the basics in every possible file; title, author, rev, date,
> description, maybe a scope/introduction if that makes sense. Often a
> rev history.
>
> Databases too often become toys. And lots of people absolutely hate
> comments.

Do you have a database that tracks part numbers, quantities, stock room
locations, "where used", etc.?  Is *it* a toy??

People hate *creating* comments -- often because they need some form of
guidance as to WHAT it should "include", the form it should take, etc.
The advantage of adding tags (attributes) to a table in a database is
that the "fields" that are important (as defined by whomever created the
table for that type of file object) have already been defined.  It becomes
a fill-in-the-blank exercise.  Much more than "add some comment to this file"

[toc] | [prev] | [next] | [standalone]


#453231

FromJohn Larkin <jjlarkin@highlandtechnology.com>
Date2017-03-23 08:27 -0700
Message-ID<ajp7dct3fku9ffums7nc03pulmhhguef0s@4ax.com>
In reply to#453200
On Wed, 22 Mar 2017 22:50:33 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>> What happens when you email a file to someone else? Or put together a
>> separate project folder, like to a contract manufacturer or a
>> consultant? You can't include the database.
>
>Will that contract manufacturer be able to recognize your "augmented"
>file system?

It's not a special file system; it's properly documented files. If
it's a readable file, open it and read. If it's not a human-readable
format, open the README file in the same folder.

>
>That's why you include a MANIFEST file in any package of electronic objects.
>There's no portable way to examine files of varying formats for "comments"
>embedded within.  How do you examine the revision, date, author, etc. of
>a photograph?  Of an AutoCAD 3D model?  Of a DSO capture?  etc.

You'd send a manifest file to a CM or consultant? How do you generate
that? What does it look like? How does it reference or name individual
files?

I recent sent design files to a CM, for a product that we license to
another company. It's a ZIP file, and the first thing you see is a
nice README.TXT file, with everything explained in direct English. I
doubt that any automatic database app can write clear English.


>
>[I.e., you require specialty "viewers" for each of those file types/formats.]
>
>> I include the basics in every possible file; title, author, rev, date,
>> description, maybe a scope/introduction if that makes sense. Often a
>> rev history.
>>
>> Databases too often become toys. And lots of people absolutely hate
>> comments.
>
>Do you have a database that tracks part numbers, quantities, stock room
>locations, "where used", etc.?  Is *it* a toy??

No. It does exactly what it's supposed to do. I wrote it.

https://dl.dropboxusercontent.com/u/53724080/Software/MAX2.jpg

>
>People hate *creating* comments -- often because they need some form of
>guidance as to WHAT it should "include", the form it should take, etc.
>The advantage of adding tags (attributes) to a table in a database is
>that the "fields" that are important (as defined by whomever created the
>table for that type of file object) have already been defined.  It becomes
>a fill-in-the-blank exercise.  Much more than "add some comment to this file"

I let my people know what's expected.

I saw some Windows source code. Each procedure has a fill-in-the
blanks form inside, author/date/etc.

FUNCTION OF MODULE :   what is says



-- 

John Larkin         Highland Technology, Inc

jlarkin att highlandtechnology dott com
http://www.highlandtechnology.com

[toc] | [prev] | [next] | [standalone]


#453254

FromDon Y <blockedofcourse@foo.invalid>
Date2017-03-23 12:25 -0700
Message-ID<ob17ag$ars$1@dont-email.me>
In reply to#453231
On 3/23/2017 8:27 AM, John Larkin wrote:
> On Wed, 22 Mar 2017 22:50:33 -0700, Don Y
> <blockedofcourse@foo.invalid> wrote:
>
>>> What happens when you email a file to someone else? Or put together a
>>> separate project folder, like to a contract manufacturer or a
>>> consultant? You can't include the database.
>>
>> Will that contract manufacturer be able to recognize your "augmented"
>> file system?
>
> It's not a special file system; it's properly documented files. If
> it's a readable file, open it and read. If it's not a human-readable
> format, open the README file in the same folder.

See below

>> That's why you include a MANIFEST file in any package of electronic objects.
>> There's no portable way to examine files of varying formats for "comments"
>> embedded within.  How do you examine the revision, date, author, etc. of
>> a photograph?  Of an AutoCAD 3D model?  Of a DSO capture?  etc.
>
> You'd send a manifest file to a CM or consultant? How do you generate
> that? What does it look like? How does it reference or name individual
> files?
>
> I recent sent design files to a CM, for a product that we license to
> another company. It's a ZIP file, and the first thing you see is a
> nice README.TXT file, with everything explained in direct English. I
> doubt that any automatic database app can write clear English.

A file called "MANIFEST" contains ASCII TEXT describing the contents of the
"document package" of which it is a part.  It doesn't say anything about
how the package is to be used, etc. but, rather, just describes the
contents.

    manifest, (noun):
       + a list of the cargo carried by a ship, made for the use of various
         agents and officials at the ports of destination.
       + a list or invoice of goods transported by truck or train.

A "README" can have more varied uses and content.  A MANIFEST only describes
the contents of the "document package":
     32322.abc   Schematic DASH-Strides 6 format
     43243.lmn   PCB layout in OrCAD Layout 9 format
     55555.*     Artwork in Gerber Photoplot layers
     12345.xyz   Bill of Material as ASCII text
     94949.rrr   Firmware image in binary form
     60945.qqq   Assembly instructions as ASCII text
etc.

[File naming conventions obviously vary so the recipient need not expect
a "foo.doc" file to be a MSWord file, etc.]

How *I* would generate it is by creating a report from the appropriate
fields associated with the file objects as named in the database.

E.g., if I was preparing a brochure, the MANIFEST might include:
     abcdefg.raw    Cover image, Adobe DNG format
     434534.300     verso page imagery, JPEG2000 format
     brochure.txt   Instructions, ASCII text
     contact.inf    contact information, ASCII text

[toc] | [prev] | [next] | [standalone]


#453366

Fromtabbypurr@gmail.com
Date2017-03-24 04:31 -0700
Message-ID<ac8100f9-8d9d-4fd8-8566-2d22669c180c@googlegroups.com>
In reply to#453134
On Wednesday, 22 March 2017 21:36:04 UTC, Jon Elson  wrote:
> tabbypurr wrote:
> > On Tuesday, 21 March 2017 17:21:23 UTC, John Larkin  wrote:
> >> I just happened to open my recycle bin and there was 42 Gbytes of
> >> junk.
> >> 
> >> I remember 256 Kbyte hard drives. A PC with a 10M drive was top-end.
> >> 
> >> There's no point in reducing jpeg sizes, or even deleting stuff, any
> >> more. I could never take enough pics, or create enough docs, to keep
> >> up with the progress in hard drives. I suppose some people could
> >> collect a large number of HD movies and use a terabyte somehow.
> >> 
> >> It's impossible to crack one-time-pad encryption. Giant hard drives
> >> and memory sticks make OTP files easy to physically hand off to
> >> correspondents.
> > 
> > I estimated the payback of deleting data some time last year. It came in
> > at something like a penny or 2 an hour.

> Well, the problem is I want physical, off-line backup of important stuff 
> (that's what I've created, not the OS, utilities, etc.) preferably located 
> in a couple places.  So, getting rid of excess junk and redundant on-line 
> backups makes some sense, in thaty it makes those CDs, then DVDs, now Blu-
> Ray backups fit on available media.
> 
> jon

I did that at one time, but quickly found buying more HDD storage made far more sense. Optical discs stopped being practical here long ago. Far too small & too unreliable.


NT

[toc] | [prev] | [next] | [standalone]


#453383

FromDon Y <blockedofcourse@foo.invalid>
Date2017-03-24 06:38 -0700
Message-ID<ob37bd$t67$1@dont-email.me>
In reply to#453366
On 3/24/2017 4:31 AM, tabbypurr@gmail.com wrote:

>> Well, the problem is I want physical, off-line backup of important stuff
>> (that's what I've created, not the OS, utilities, etc.) preferably located
>>  in a couple places.  So, getting rid of excess junk and redundant on-line
>>  backups makes some sense, in thaty it makes those CDs, then DVDs, now
>> Blu- Ray backups fit on available media.
>
> I did that at one time, but quickly found buying more HDD storage made far
> more sense. Optical discs stopped being practical here long ago. Far too
> small & too unreliable.

Use HDD's as the "off-line" backups.  Buy a SATA dock (or FW or
<yourfavoriteIF>) and a bunch of bare drives...

[toc] | [prev] | [next] | [standalone]


#453403

FromJohn Larkin <jjlarkin@highlandtechnology.com>
Date2017-03-24 08:41 -0700
Message-ID<ldfadc5iijh55i6q7gil2uli9ildvff36b@4ax.com>
In reply to#453383
On Fri, 24 Mar 2017 06:38:32 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>On 3/24/2017 4:31 AM, tabbypurr@gmail.com wrote:
>
>>> Well, the problem is I want physical, off-line backup of important stuff
>>> (that's what I've created, not the OS, utilities, etc.) preferably located
>>>  in a couple places.  So, getting rid of excess junk and redundant on-line
>>>  backups makes some sense, in thaty it makes those CDs, then DVDs, now
>>> Blu- Ray backups fit on available media.
>>
>> I did that at one time, but quickly found buying more HDD storage made far
>> more sense. Optical discs stopped being practical here long ago. Far too
>> small & too unreliable.
>
>Use HDD's as the "off-line" backups.  Buy a SATA dock (or FW or
><yourfavoriteIF>) and a bunch of bare drives...

USB terabyte drives are cheap and great for physical backup. Any PC,
laptop even, can read the files later.

As long as USB is supported.


-- 

John Larkin         Highland Technology, Inc

lunatic fringe electronics 

[toc] | [prev] | [next] | [standalone]


#453424

FromDon Y <blockedofcourse@foo.invalid>
Date2017-03-24 11:38 -0700
Message-ID<ob3oui$ud6$1@dont-email.me>
In reply to#453403
On 3/24/2017 8:41 AM, John Larkin wrote:
> On Fri, 24 Mar 2017 06:38:32 -0700, Don Y
> <blockedofcourse@foo.invalid> wrote:
>
>> On 3/24/2017 4:31 AM, tabbypurr@gmail.com wrote:
>>
>>>> Well, the problem is I want physical, off-line backup of important stuff
>>>> (that's what I've created, not the OS, utilities, etc.) preferably located
>>>>  in a couple places.  So, getting rid of excess junk and redundant on-line
>>>>  backups makes some sense, in thaty it makes those CDs, then DVDs, now
>>>> Blu- Ray backups fit on available media.
>>>
>>> I did that at one time, but quickly found buying more HDD storage made far
>>> more sense. Optical discs stopped being practical here long ago. Far too
>>> small & too unreliable.
>>
>> Use HDD's as the "off-line" backups.  Buy a SATA dock (or FW or
>> <yourfavoriteIF>) and a bunch of bare drives...
>
> USB terabyte drives are cheap and great for physical backup. Any PC,
> laptop even, can read the files later.

Yes, but this opens up a vulnerability:  that <something> can accidentally
corrupt/alter/discard a "backup".  Most of my "archive" resides on large
consumer drives (lower cost per byte, easy to acquire a second/third/fourth
copy of the data, etc. -- without requiring it to be part of a spinning RAID
array) so I can hand-carry a volume to any machine that I want and access it
"locally" (i.e., like being able to pull a drive *out* of a RAID array and
access it on ANY machine).

But, this means there is the chance that the medium can be altered by that
other system.  And, no way of *knowing* that this has occurred.  Hence the
reason I "scrub" (verify) drives whenever they are mounted in the "system"
("Ah, I see volume127 has been mounted on machine Pinky.  Let me see where
I was in my verification activity when last I encountered that drive and
continue where I left off...")

[Of course, this means the file system chosen for the medium must be
supported on any machines that want to access the contents "locally".
By accessing it through some other protocol (e.g., SMB/NFS share),
you can hide some of this from the client.]

The other problem with USB drives is they tend to be physically larger than
"bare" drives AND you have to keep track of a particular power supply for
that particular drive enclosure (if you have lots of bare drives, none of
them need power supplies -- just the *dock* does!)

> As long as USB is supported.
>
>

[toc] | [prev] | [next] | [standalone]


#453444

FromJohn Larkin <jjlarkinxyxy@highlandtechnology.com>
Date2017-03-24 13:55 -0700
Message-ID<pc1bdctvfk175bofqdfnn3pf4cp07fu85f@4ax.com>
In reply to#453424
On Fri, 24 Mar 2017 11:38:52 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>On 3/24/2017 8:41 AM, John Larkin wrote:
>> On Fri, 24 Mar 2017 06:38:32 -0700, Don Y
>> <blockedofcourse@foo.invalid> wrote:
>>
>>> On 3/24/2017 4:31 AM, tabbypurr@gmail.com wrote:
>>>
>>>>> Well, the problem is I want physical, off-line backup of important stuff
>>>>> (that's what I've created, not the OS, utilities, etc.) preferably located
>>>>>  in a couple places.  So, getting rid of excess junk and redundant on-line
>>>>>  backups makes some sense, in thaty it makes those CDs, then DVDs, now
>>>>> Blu- Ray backups fit on available media.
>>>>
>>>> I did that at one time, but quickly found buying more HDD storage made far
>>>> more sense. Optical discs stopped being practical here long ago. Far too
>>>> small & too unreliable.
>>>
>>> Use HDD's as the "off-line" backups.  Buy a SATA dock (or FW or
>>> <yourfavoriteIF>) and a bunch of bare drives...
>>
>> USB terabyte drives are cheap and great for physical backup. Any PC,
>> laptop even, can read the files later.
>
>Yes, but this opens up a vulnerability:  that <something> can accidentally
>corrupt/alter/discard a "backup". 

That's why we have so many of them. A one-year-old file has been
off-site backed up, to different drives, 12 times.

A 5-year old file, 60 times.

So far, when I've pulled a drive from storage and tried to access
backup files, it has worked every time. 


 Most of my "archive" resides on large
>consumer drives (lower cost per byte, easy to acquire a second/third/fourth
>copy of the data, etc. -- without requiring it to be part of a spinning RAID
>array) so I can hand-carry a volume to any machine that I want and access it
>"locally" (i.e., like being able to pull a drive *out* of a RAID array and
>access it on ANY machine).
>
>But, this means there is the chance that the medium can be altered by that
>other system.  And, no way of *knowing* that this has occurred.  Hence the
>reason I "scrub" (verify) drives whenever they are mounted in the "system"
>("Ah, I see volume127 has been mounted on machine Pinky.  Let me see where
>I was in my verification activity when last I encountered that drive and
>continue where I left off...")
>
>[Of course, this means the file system chosen for the medium must be
>supported on any machines that want to access the contents "locally".
>By accessing it through some other protocol (e.g., SMB/NFS share),
>you can hide some of this from the client.]
>
>The other problem with USB drives is they tend to be physically larger than
>"bare" drives AND you have to keep track of a particular power supply for
>that particular drive enclosure (if you have lots of bare drives, none of
>them need power supplies -- just the *dock* does!)

We use tiny terabyte USB drives with a captive USB pigtail cable. They
don't need extra power supplies; just plug them into a USB port on any
PC. I'm sure speed suffers from using a single USB port for power, but
that doesn't matter.


-- 

John Larkin         Highland Technology, Inc
picosecond timing   precision measurement 

jlarkin att highlandtechnology dott com
http://www.highlandtechnology.com

[toc] | [prev] | [next] | [standalone]


#453474

Fromtabbypurr@gmail.com
Date2017-03-24 17:14 -0700
Message-ID<a109fe81-9447-4e4a-9579-bf2b2f9fa055@googlegroups.com>
In reply to#453424
On Friday, 24 March 2017 18:39:02 UTC, Don Y  wrote:

> The other problem with USB drives is they tend to be physically larger than
> "bare" drives AND you have to keep track of a particular power supply for
> that particular drive enclosure (if you have lots of bare drives, none of
> them need power supplies -- just the *dock* does!)

All mine run on 12v.


NT

[toc] | [prev] | [next] | [standalone]


Page 3 of 6 — ← Prev page 1 2 [3] 4 5 6  Next page →

Back to top | Article view | sci.electronics.design


csiph-web