Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.arch.embedded > #25285

Re: Checking large numbers of files

Show all headers | View raw

On 30.10.2016 г. 05:06, Clifford Heath wrote:
> On 30/10/16 13:05, Dimiter_Popoff wrote:
>> On 30.10.2016 г. 03:00, Clifford Heath wrote:
>>> On 30/10/16 10:19, Don Y wrote:
>>>> On 10/29/2016 3:47 PM, Clifford Heath wrote:
>>>>> On 30/10/16 07:15, Don Y wrote:
>>>>>> On 10/29/2016 5:17 AM, kalvin.news@gmail.com wrote:
>>>>>>> Disclaimer: I am not an expert on the data storage, but here are my
>>>>>>> thoughts:
>>>>>>> - Use a RAID system which has enough redundancy to detect and
>>>>>>> correct
>>>>>> I started out using RAID5 "hardware" arrays (and other, more
>>>>>> sophisticated
>>>>>> configurations).  This requires you to have multiple spindles running
>>>>>> concurrently (else the array can't *detect* discrepancies).  Compact
>>>>>> arrays also tend to suffer from cooling problems.
>>>>>>
>>>>>> It also ties you to an often proprietary (or, at least, poorly
>>>>>> documented)
>>>>>> implementation ("volume format").  So, a failure of the hardware can
>>>>>> leave
>>>>>> you with possibly intact volume(s) and no straightforward means of
>>>>>> accessing their contents.
>>>>>
>>>>> No. Almost all commercial RAID systems (Synology, Qnap, etc)
>>>>> use Linux RAID support. If the hardware fails, you can mount
>>>>> the drives on any Linux system that has enough SATA ports.
>>>>
>>>> Really?  So my Sun RAID systems run Linux??
>>>
>>> Ahh, my apologies. I didn't realize you were running a computer antiques
>>> museum. I thought you were trying to ensure the availability of some
>>> data.
>>
>> So what is he supposed to do with data which are useless outside of
>> a particular environment. Apparently he has plenty of past projects
>> spanning a few decades back to maintain - how do you suggest he will
>> separate movable from non-movable data. Then the sheer amount of data
>> he is talking is just huge.
>
> No, his amount of data is trivial compared to the petabytes
> being accumulated daily and managed by internet-scale companies,
> who almost all use *commodity* hardware running *open source*
> software.
>
>> I strongly suspect Don is not an office secretary running an office
>> archive and I also strongly suspect he knows what he is doing.
>
> Don is a "special flower". He faces problems that no-one else
> has ever faced, and he analyses them with an insight that no-one
> else has ever had, requiring solutions that no-one else has ever
> found.
>
> None of these descriptions are true, of course.
>
> For what it's worth, data management is a global profession with
> a hundred major specialties and a thousand vendors, who all have
> experts with more experience than Don has in his toenails. I've
> just returned from a paid global speaking trip where I addressed
> some of them. "Availability" isn't my specialty, I just rely on
> commodity equipment to manage my measly few dozen terabytes. But
> what do I know, I don't work with special problems like Don's.


Well,

 >> Linux/foss may be a huge part of life but it is restricted to the
 >> needs of the general public which feeds it directly. There are
 >> other aspects of tech life which are simply beyond the scope of
 >> foss/linux, beyond its capabilities as they are.

But I know many of the foss/linux crowd think otherwise - which is
OK, everyone sees the walls of their own room. No need to be grumpy
about other people wanting to talk about other things though.

Dimiter

Back to comp.arch.embedded | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Checking large numbers of files Don Y <blockedofcourse@foo.invalid> - 2016-10-27 15:23 -0700
  Re: Checking large numbers of files Clifford Heath <no.spam@please.net> - 2016-10-28 09:48 +1100
    Re: Checking large numbers of files Don Y <blockedofcourse@foo.invalid> - 2016-10-27 16:48 -0700
      Re: Checking large numbers of files Dimiter_Popoff <dp@tgi-sci.com> - 2016-10-28 03:17 +0300
        Re: Checking large numbers of files Don Y <blockedofcourse@foo.invalid> - 2016-10-27 18:50 -0700
          Re: Checking large numbers of files Dimiter_Popoff <dp@tgi-sci.com> - 2016-10-28 05:38 +0300
            Re: Checking large numbers of files Don Y <blockedofcourse@foo.invalid> - 2016-10-27 21:33 -0700
      Re: Checking large numbers of files Clifford Heath <no.spam@please.net> - 2016-10-28 11:40 +1100
        Re: Checking large numbers of files Don Y <blockedofcourse@foo.invalid> - 2016-10-27 18:52 -0700
  Re: Checking large numbers of files kalvin.news@gmail.com - 2016-10-28 03:50 -0700
    Re: Checking large numbers of files Don Y <blockedofcourse@foo.invalid> - 2016-10-28 10:49 -0700
      Re: Checking large numbers of files kalvin.news@gmail.com - 2016-10-29 05:17 -0700
        Re: Checking large numbers of files Don Y <blockedofcourse@foo.invalid> - 2016-10-29 13:15 -0700
          Re: Checking large numbers of files Clifford Heath <no.spam@please.net> - 2016-10-30 09:47 +1100
            Re: Checking large numbers of files Don Y <blockedofcourse@foo.invalid> - 2016-10-29 16:19 -0700
              Re: Checking large numbers of files Don Y <blockedofcourse@foo.invalid> - 2016-10-29 16:36 -0700
              Re: Checking large numbers of files Clifford Heath <no.spam@please.net> - 2016-10-30 12:00 +1100
                Re: Checking large numbers of files Dimiter_Popoff <dp@tgi-sci.com> - 2016-10-30 04:05 +0200
                Re: Checking large numbers of files Clifford Heath <no.spam@please.net> - 2016-10-30 14:06 +1100
                Re: Checking large numbers of files Dimiter_Popoff <dp@tgi-sci.com> - 2016-10-30 05:48 +0200
                Re: Checking large numbers of files Don Y <blockedofcourse@foo.invalid> - 2016-10-29 22:33 -0700
                Re: Checking large numbers of files Clifford Heath <no.spam@please.net> - 2016-10-30 17:00 +1100
                Re: Checking large numbers of files (long) Don Y <blockedofcourse@foo.invalid> - 2016-10-31 04:43 -0700
                Re: Checking large numbers of files (long) David Brown <david.brown@hesbynett.no> - 2016-10-31 14:20 +0100
                Re: Checking large numbers of files (long) Don Y <blockedofcourse@foo.invalid> - 2016-10-31 08:02 -0700
                Re: Checking large numbers of files (long) David Brown <david.brown@hesbynett.no> - 2016-11-01 09:47 +0100
                Re: Checking large numbers of files (long) Don Y <blockedofcourse@foo.invalid> - 2016-11-01 04:36 -0700
                Re: Checking large numbers of files (long) David Brown <david.brown@hesbynett.no> - 2016-11-01 14:29 +0100
                Re: Checking large numbers of files (long) David Brown <david.brown@hesbynett.no> - 2016-11-01 15:16 +0100
                Re: Checking large numbers of files (long) Don Y <blockedofcourse@foo.invalid> - 2016-11-02 11:42 -0700
                Re: Checking large numbers of files Don Y <blockedofcourse@foo.invalid> - 2016-10-29 22:15 -0700
                Re: Checking large numbers of files Don Y <blockedofcourse@foo.invalid> - 2016-10-29 21:50 -0700
                Re: Checking large numbers of files Clifford Heath <no.spam@please.net> - 2016-10-30 17:14 +1100
                Re: Checking large numbers of files (long) Don Y <blockedofcourse@foo.invalid> - 2016-10-31 03:36 -0700
                Re: Checking large numbers of files (long) Clifford Heath <no.spam@please.net> - 2016-11-01 09:20 +1100
                Re: Checking large numbers of files (long) Don Y <blockedofcourse@foo.invalid> - 2016-11-01 02:23 -0700
                Re: Checking large numbers of files Don Y <blockedofcourse@foo.invalid> - 2016-10-29 21:20 -0700
                Re: Checking large numbers of files Clifford Heath <no.spam@please.net> - 2016-10-30 17:25 +1100
          Re: Checking large numbers of files Robert Wessel <robertwessel2@yahoo.com> - 2016-10-30 00:14 -0500
            Re: Checking large numbers of files Don Y <blockedofcourse@foo.invalid> - 2016-10-29 23:02 -0700
          Re: Checking large numbers of files David Brown <david.brown@hesbynett.no> - 2016-10-31 12:19 +0100
            Re: Checking large numbers of files Don Y <blockedofcourse@foo.invalid> - 2016-10-31 07:00 -0700
  Re: Checking large numbers of files Ali <abdulrazaq@gmail.com> - 2016-10-30 15:45 -0700
    Re: Checking large numbers of files Don Y <blockedofcourse@foo.invalid> - 2016-10-31 03:45 -0700

csiph-web