Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.development.system > #618 > unrolled thread

shred or scrub

Back to article view | Back to comp.os.linux.development.system

Contents

  shred or scrub "Bill Cunningham" <nospam@nspam.invalid> - 2014-04-16 18:17 -0400
    Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-17 04:19 -0600
      Re: shred or scrub "Bill Cunningham" <nospam@nspam.invalid> - 2014-04-18 22:30 -0400
        Re: shred or scrub Jasen Betts <jasen@xnet.co.nz> - 2014-04-19 07:42 +0000
          Re: shred or scrub Richard Kettlewell <rjk@greenend.org.uk> - 2014-04-19 10:04 +0100
        Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-19 02:15 -0600
        Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-19 23:05 +0100
          Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-20 02:47 -0600
            Re: shred or scrub John Hasler <jhasler@newsguy.com> - 2014-04-20 07:56 -0500
              Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-21 03:51 -0600
                Re: shred or scrub Jasen Betts <jasen@xnet.co.nz> - 2014-04-21 11:50 +0000
                  Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-21 06:14 -0600
              Re: shred or scrub "Bill Cunningham" <nospam@nspam.invalid> - 2014-04-21 18:44 -0400
            Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-21 13:24 +0100
              Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-22 04:10 -0600
                Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-22 14:39 +0100
    Re: shred or scrub Kristof Provost <kristof@codepro.be> - 2014-04-17 13:15 +0000
      Re: shred or scrub John Hasler <jhasler@newsguy.com> - 2014-04-17 09:40 -0500
        Re: shred or scrub Kristof Provost <kristof@codepro.be> - 2014-04-18 14:40 +0000
      Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-18 02:12 -0600
        Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-18 11:49 +0200
          Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-18 09:59 -0600
            Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-21 16:14 +0200
              Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-22 04:22 -0600
                Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-23 00:06 +0200
                  Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-23 05:50 -0600
                    Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-24 22:46 +0200
                      Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-25 03:57 -0600
                        Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-25 19:14 +0100
                          Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-26 04:02 -0600
                            Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-27 21:26 +0100
                              Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-28 03:27 -0600
                                Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-28 12:17 +0100
                                  Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-28 13:01 +0100
                                    Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-29 02:50 -0600
                                      UNIX(*)/ Linux history & system design (was: shred or scrub) Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-05 21:31 +0100
                                        Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-05 16:02 -0600
                                          Re: UNIX(*)/ Linux history & system design David Brown <david.brown@hesbynett.no> - 2014-05-06 01:17 +0200
                                            Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-06 01:46 -0600
                                              Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-06 15:09 +0100
                                                Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-06 23:47 -0600
                                                  Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-07 16:23 +0100
                                                    Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-07 10:51 -0600
                                                      Re: UNIX(*)/ Linux history & system design Jerry Peters <jerry@example.invalid> - 2014-05-07 20:25 +0000
                                                        Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-08 03:50 -0600
                                                          Re: UNIX(*)/ Linux history & system design Jerry Peters <jerry@example.invalid> - 2014-05-08 20:24 +0000
                                                            Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-09 02:23 -0600
                                                              Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-09 18:36 +0100
                                                              Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-09 21:24 +0100
                                                      Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-07 22:01 +0100
                                                        Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-08 03:37 -0600
                                                          Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-08 14:02 +0100
                                                            Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-09 02:56 -0600
                                              Re: UNIX(*)/ Linux history & system design David Brown <david.brown@hesbynett.no> - 2014-05-07 00:15 +0200
                                                Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-07 00:32 -0600
                                                Re: UNIX(*)/ Linux history & system design Jorgen Grahn <grahn+nntp@snipabacken.se> - 2014-05-07 08:47 +0000
                                                  Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-07 10:59 -0600
                                          Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-06 14:35 +0100
                        Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-26 16:30 +0200
                          Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-27 05:59 -0600
                            Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-27 20:15 +0100
                              Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-28 03:29 -0600
                                Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-28 12:06 +0100
                            Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-27 21:41 +0200
                              Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-28 04:03 -0600
                              Re: shred or scrub Richard Kettlewell <rjk@greenend.org.uk> - 2014-04-28 16:44 +0100
                                Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-28 23:39 +0200
        Re: shred or scrub John Hasler <jhasler@newsguy.com> - 2014-04-18 07:37 -0500
          Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-18 10:16 -0600
            Re: shred or scrub John Hasler <jhasler@newsguy.com> - 2014-04-18 12:01 -0500
        Re: shred or scrub Kristof Provost <kristof@codepro.be> - 2014-04-18 14:42 +0000
    Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-17 16:41 +0200

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

#685 — Re: UNIX(*)/ Linux history & system design

On 05/06/2014 08:09 AM, Rainer Weikusat wrote:
> crankypuss <crankypuss@nomail.invalid> writes:
>> On 05/05/2014 05:17 PM, David Brown wrote:
>>> On 06/05/14 00:02, crankypuss wrote:
>>>> On 05/05/2014 02:31 PM, Rainer Weikusat wrote:
>>>>> crankypuss <crankypuss@nomail.invalid> writes:
>>>>
>>>> <lots snipped>
>>>>
>>>>>> Some seem to disagree with that view, but then some people blindly
>>>>>> parse command output with sed and use that to drive system-critical
>>>>>> functionality; go figure.
>>>>>
>>>>> I did a lot of shell-programming when I was working on 'embeddded Linux
>>>>> project' some years ago because that was the most convenient programming
>>>>> language available to me in that environment (ARM9 machine language
>>>>> and C
>>>>> being the other options) and 'sed' was the most sophisticated
>>>>> text-processing tool available to me. What's wrong with using it?
>>>>
>>>> I know nothing about "ARM9 machine language" but if C was available, yet
>>>> you used bash/sed, you must have been in one bigassed hurry or had some
>>>> extremely simple requirements.  I'm sure your performed your job as
>>>> expected and were rewarded accordingly.
>>>
>>> Someone who writes a C program when a simple bash/sed script could do
>>> the task, would be failing to do their job as expected.  One of the
>>> reasons Linux (and other *nix flavours) have multiple languages and
>>> tools is that different tools are better for different jobs, and that
>>> often it is the developer's time that is important rather than the
>>> systems run time.  A developer who thinks C is the answer to every
>>> question is like a carpenter armed only with a hammer.
>>>
>>> In the embedded Linux systems I have worked on, the final system used C,
>>> Python, Perl, C++, and bash.  I don't think sed was used, except as part
>>> of the makefiles.
>>>
>>
>> Indeed, it is the developer's time that is important, because that is
>> what the employer has purchased.  Most employees are not very good at
>> building tools, and are as shortsighted as their employers.  Tools
>> like sed have been magically provided to them, nobody ever had reason
>> to write those because they were just magically there.  Far better to
>> simply use what has been provided, as expected, rather than building a
>> library of C subroutines that puts knocking out a small utility on the
>> same level of effort as debugging a bash/sed script
>
> [...]
>
> This has indeed been done in the past: The guy who couldn't get his
> (Perl) scripts to work reasonably well within a reasonable time was
> called Rasmus Lehrdorf and he named his library of C subroutines the
> 'Personal Homepage Tools', better known as 'PHP'.

I don't know anything about Perl scripts, never having written one.  If 
I remember correctly, I wrote "exactly" one bash script when I first 
began using linux.  Enough to get the flavor of it.  Enough to recognize 
it as the distant relative of some ancient scripting tools that I'd used 
elsewhere; quite enough for my taste, thank you.

As for PHP, it's something I know a small bit about.  It's a terrible 
language, filled with inconsistencies like the family of routines that 
begins with "is", which includes isset() and is_array(); however it came 
into existence, those who developed it couldn't even be bothered to set 
up a consistent convention for the naming of builtin functions.  They 
made a lot of mistakes, and the language is probably stuck with them. 
Learning PHP is nearly as ugly an enterprise as is learning to spell any 
useful vocabulary of words in the English language, where the ratio of 
exceptions per rule seems quite high.

Still, it works, and many things can be done with it, given that its set 
of capabilities is quite comprehensive (though sometimes flawed), and it 
can be quite adequate in terms of performance for many types of 
applications.

I started using it in 2002 when I needed to build a website, and I still 
use it today.  Not because it's a wonderful language, but because its 
syntax permits certain constructions that I find useful, and because 
I've a sizeable codebase inherited from the time when I needed to build 
a website and added to during the years since.

I was originally drawn to it simply because it was available on the 
web-host that I was using (which was btw running on linux) but I found 
the fact that a Windows version was available to be added value.  In 
other words, it was basically the best crappy language available to me 
for building a website.

Eventually I may autoconvert that existing codebase to C, or simply use 
it as a prototype for new C code, or most likely autoconvert it and then 
use the C code until it is time to rewrite it.  Fortunately I'm at 
liberty to do it however I choose, there's no requirement placed upon 
me, and I certainly should be capable of writing the autoconversion 
code, in PHP.

Yesterday I was pondering the crudeness of bash and wondering why it has 
not entirely atrophied, and realized what I think is the reason for its 
continued existence: make.  So it goes, the more deeply entrenched 
something is, the more effort is required to expunge it from the realm 
of the necessary and banish it into the realm of the optional.

[toc] | [prev] | [next] | [standalone]

#690 — Re: UNIX(*)/ Linux history & system design

crankypuss <crankypuss@nomail.invalid> writes:
> On 05/06/2014 08:09 AM, Rainer Weikusat wrote:

[...]

> I don't know anything about Perl scripts, never having written one.
> If I remember correctly, I wrote "exactly" one bash script when I
> first began using linux.  Enough to get the flavor of it.  Enough to
> recognize it as the distant relative of some ancient scripting tools
> that I'd used elsewhere; quite enough for my taste, thank you.

[...]

> Yesterday I was pondering the crudeness of bash and wondering why it
> has not entirely atrophied, and realized what I think is the reason
> for its continued existence: make.  So it goes, the more deeply
> entrenched something is, the more effort is required to expunge it
> from the realm of the necessary and banish it into the realm of the
> optional.

'bash' is an extended implementaton of the Bourne shell language which
is 'in the realm of the necessary' simply because it's an international
standard. In addition to constructs specific to its purpose, ie invoking
programs, possibly, programs cooperating in a pipeline, it supports the
usual syntactic constructs available in 'pre-OO' high-level languages:
Three kinds of loop ('loop while a condition is true', 'loop until a
condition becomes false', 'loop over the elements of a list'), loop
control statements ('break' and 'continue'), conditionals (if .. elif
... elif ... else, a 'case statement' and the possibility to create
named subroutines (but not lexically-scoped variables, although bash has
them as an extension). It doesn't use braces as universal block
delimiter but keywords depending on the 'block construct' being
used. For loops, thats do .... done, for if then ... fi and for case, it
is case ... esac. The latter two pairs are Algol-68 influenced,
presumably, because Steve Bournce (author of the original Bourne shell)
was very fond of that. It doesn't support named parameters for functions
(or scripts), only positional ones. While I wouldn't to use it (in case
suitable alternatives are available) for something larger than at most a
few hundred lines of code[*], it is a perfectly usable programming language
and often, a very easy to use one.

Would you care to elaborate why precisely you consider it to be 'so
crude that it is astonishing it didn't vanish'?

[*] OTOH, a lot of things can be built by using the shell to combine
other programs, some or all of them being shell scripts, too.

[toc] | [prev] | [next] | [standalone]

#691 — Re: UNIX(*)/ Linux history & system design

On 05/07/2014 09:23 AM, Rainer Weikusat wrote:
> crankypuss <crankypuss@nomail.invalid> writes:
>> On 05/06/2014 08:09 AM, Rainer Weikusat wrote:
>
> [...]
>
>> I don't know anything about Perl scripts, never having written one.
>> If I remember correctly, I wrote "exactly" one bash script when I
>> first began using linux.  Enough to get the flavor of it.  Enough to
>> recognize it as the distant relative of some ancient scripting tools
>> that I'd used elsewhere; quite enough for my taste, thank you.
>
> [...]
>
>> Yesterday I was pondering the crudeness of bash and wondering why it
>> has not entirely atrophied, and realized what I think is the reason
>> for its continued existence: make.  So it goes, the more deeply
>> entrenched something is, the more effort is required to expunge it
>> from the realm of the necessary and banish it into the realm of the
>> optional.
>
> 'bash' is an extended implementaton of the Bourne shell language which
> is 'in the realm of the necessary' simply because it's an international
> standard.

Standards are good things during their time, but over time standards 
evolve or are replaced by something altogether different.

> In addition to constructs specific to its purpose, ie invoking
> programs, possibly, programs cooperating in a pipeline, it supports the
> usual syntactic constructs available in 'pre-OO' high-level languages:
> Three kinds of loop ('loop while a condition is true', 'loop until a
> condition becomes false', 'loop over the elements of a list'), loop
> control statements ('break' and 'continue'), conditionals (if .. elif
> ... elif ... else, a 'case statement' and the possibility to create
> named subroutines (but not lexically-scoped variables, although bash has
> them as an extension). It doesn't use braces as universal block
> delimiter but keywords depending on the 'block construct' being
> used. For loops, thats do .... done, for if then ... fi and for case, it
> is case ... esac. The latter two pairs are Algol-68 influenced,
> presumably, because Steve Bournce (author of the original Bourne shell)
> was very fond of that. It doesn't support named parameters for functions
> (or scripts), only positional ones. While I wouldn't to use it (in case
> suitable alternatives are available) for something larger than at most a
> few hundred lines of code[*], it is a perfectly usable programming language
> and often, a very easy to use one.
>
> Would you care to elaborate why precisely you consider it to be 'so
> crude that it is astonishing it didn't vanish'?

Do I really need to elaborate on that?  Only positional parameters for 
functions, no builtin functions to speak of, which drives the user to 
using sed or some similar external program for even trivial 
functionality.  It's a very basic script processor, not a language.  On 
top of that, the bash rules for escaping the characters it has coopted 
for special purposes make it a syntactic nightmare even worse than C. 
The fact that someone kludged control structures onto a simple 
substitution engine does not make it a programming language.

[toc] | [prev] | [next] | [standalone]

#693 — Re: UNIX(*)/ Linux history & system design

crankypuss <crankypuss@nomail.invalid> wrote:
> On 05/07/2014 09:23 AM, Rainer Weikusat wrote:
>>
>> Would you care to elaborate why precisely you consider it to be 'so
>> crude that it is astonishing it didn't vanish'?
> 
> Do I really need to elaborate on that?  Only positional parameters for 
> functions, no builtin functions to speak of, which drives the user to 
> using sed or some similar external program for even trivial 
> functionality.  It's a very basic script processor, not a language.  On 
> top of that, the bash rules for escaping the characters it has coopted 
> for special purposes make it a syntactic nightmare even worse than C. 
> The fact that someone kludged control structures onto a simple 
> substitution engine does not make it a programming language.

But sed, date, awk, and all of the other text processing utilities are
the "library of functions" for shell programming. Why should there be
duplicate functions in bash? 

[toc] | [prev] | [next] | [standalone]

#696 — Re: UNIX(*)/ Linux history & system design

On 05/07/2014 02:25 PM, Jerry Peters wrote:
> crankypuss <crankypuss@nomail.invalid> wrote:
>> On 05/07/2014 09:23 AM, Rainer Weikusat wrote:
>>>
>>> Would you care to elaborate why precisely you consider it to be 'so
>>> crude that it is astonishing it didn't vanish'?
>>
>> Do I really need to elaborate on that?  Only positional parameters for
>> functions, no builtin functions to speak of, which drives the user to
>> using sed or some similar external program for even trivial
>> functionality.  It's a very basic script processor, not a language.  On
>> top of that, the bash rules for escaping the characters it has coopted
>> for special purposes make it a syntactic nightmare even worse than C.
>> The fact that someone kludged control structures onto a simple
>> substitution engine does not make it a programming language.
>
> But sed, date, awk, and all of the other text processing utilities are
> the "library of functions" for shell programming. Why should there be
> duplicate functions in bash?

Our concepts of what "shell programming" means is not even close to the 
same.  You are apparently content with what a history of add-ons has 
provided.  By all means, carry on.

[toc] | [prev] | [next] | [standalone]

#698 — Re: UNIX(*)/ Linux history & system design

crankypuss <crankypuss@nomail.invalid> wrote:
> On 05/07/2014 02:25 PM, Jerry Peters wrote:
>> crankypuss <crankypuss@nomail.invalid> wrote:
>>> On 05/07/2014 09:23 AM, Rainer Weikusat wrote:
>>>>
>>>> Would you care to elaborate why precisely you consider it to be 'so
>>>> crude that it is astonishing it didn't vanish'?
>>>
>>> Do I really need to elaborate on that?  Only positional parameters for
>>> functions, no builtin functions to speak of, which drives the user to
>>> using sed or some similar external program for even trivial
>>> functionality.  It's a very basic script processor, not a language.  On
>>> top of that, the bash rules for escaping the characters it has coopted
>>> for special purposes make it a syntactic nightmare even worse than C.
>>> The fact that someone kludged control structures onto a simple
>>> substitution engine does not make it a programming language.
>>
>> But sed, date, awk, and all of the other text processing utilities are
>> the "library of functions" for shell programming. Why should there be
>> duplicate functions in bash?
> 
> Our concepts of what "shell programming" means is not even close to the 
> same.  You are apparently content with what a history of add-ons has 
> provided.  By all means, carry on.

I could say the same about C, which also has a long history of "add
ons" which make it better and add useful features.
I'm pointing out that *useful* languages tend to have some sort of
"library" of routines available, Python, for example has a rich set of
extension modules, as does PERL. For the shell the common UNIX command
line utilities provide this rich environment. 

[toc] | [prev] | [next] | [standalone]

#699 — Re: UNIX(*)/ Linux history & system design

On 05/08/2014 02:24 PM, Jerry Peters wrote:
> crankypuss <crankypuss@nomail.invalid> wrote:
>> On 05/07/2014 02:25 PM, Jerry Peters wrote:
>>> crankypuss <crankypuss@nomail.invalid> wrote:
>>>> On 05/07/2014 09:23 AM, Rainer Weikusat wrote:
>>>>>
>>>>> Would you care to elaborate why precisely you consider it to be 'so
>>>>> crude that it is astonishing it didn't vanish'?
>>>>
>>>> Do I really need to elaborate on that?  Only positional parameters for
>>>> functions, no builtin functions to speak of, which drives the user to
>>>> using sed or some similar external program for even trivial
>>>> functionality.  It's a very basic script processor, not a language.  On
>>>> top of that, the bash rules for escaping the characters it has coopted
>>>> for special purposes make it a syntactic nightmare even worse than C.
>>>> The fact that someone kludged control structures onto a simple
>>>> substitution engine does not make it a programming language.
>>>
>>> But sed, date, awk, and all of the other text processing utilities are
>>> the "library of functions" for shell programming. Why should there be
>>> duplicate functions in bash?
>>
>> Our concepts of what "shell programming" means is not even close to the
>> same.  You are apparently content with what a history of add-ons has
>> provided.  By all means, carry on.
>
> I could say the same about C, which also has a long history of "add
> ons" which make it better and add useful features.
> I'm pointing out that *useful* languages tend to have some sort of
> "library" of routines available, Python, for example has a rich set of
> extension modules, as does PERL. For the shell the common UNIX command
> line utilities provide this rich environment.

The common UNIX command line utilities provide only *HALF* of this rich 
environment.  The option/argument format commonly used for command 
invocation provides the passing of boolean options, string options, and 
positional string arguments, as defined by the target command.

But the caller (invoker of the command) receives as a result from the 
command only a simple exit-code and whatever command-specific text has 
been output to stdout (or stderr).  It is up to the "caller" to parse 
the "result" text and hopefully make sense of it.

If the C language were suddenly restricted that way, people would 
immediately revolt by finding some other language to use; that "shell" 
languages are forced to operate in this restrictive mode is accepted 
because that's how the first crude shells were implemented and they 
evolved ad-hoc from there.

Parsing the textual output of a command is basically a "bad" thing for a 
number of reasons, primary among which imo is that it generates an 
ever-increasing codebase which depends on command output remaining in 
its initial (or previous) format.  That now-massive codebase pushes very 
hard against the possibility of change, because a change in the format 
of the default message output of one of the core commands would 
invalidate so much existing code that it must be avoided.

When it is necessary ("oh, never thought of that", or "look, something 
new was invented!") to change existing command output format, in marches 
yet-another-option to request the new format instead of the old one, and 
with it additional logic and additional possibilities for errors within 
the modified command.

And the poor "caller" is forced to parse the command output, which in 
bash scripts is a hideous mess requiring even more external commands 
like 'sed' because bash scripts lack sufficient builtins to perform the 
parsing themselves.

The whole thing is a massive kludge, and imo the longer we wait before 
fixing it the more difficult it will be to ever fix it.  I would say "if 
it is not already too late", but I am of the opinion that it has been 
too late for years, decades even, and that truly fixing it while 
allowing existing code to continue functioning may require a level of 
cleverness exceeding that which is available.

I'm not just "whining Dixie" here, one of the projects I'm working on is 
headed in the direction of providing a possible replacement for much of 
what is involved, but it moves at its own pace.  If by commenting in 
this thread I can succeed in getting even a /few/ people to grasp that a 
problem exists, then that is a minor victory.

[toc] | [prev] | [next] | [standalone]

#701 — Re: UNIX(*)/ Linux history & system design

crankypuss <crankypuss@nomail.invalid> writes:

[...]

> But the caller (invoker of the command) receives as a result from the
> command only a simple exit-code and whatever command-specific text has
> been output to stdout (or stderr).  It is up to the "caller" to parse
> the "result" text and hopefully make sense of it.
>
> If the C language were suddenly restricted that way, people would
> immediately revolt by finding some other language to use;

Considering that C neither supports automatic memory management nor easy
parsing of text, that's not exactly a surpising.  

> that "shell" languages are forced to operate in this restrictive mode
> is accepted because that's how the first crude shells were implemented
> and they evolved ad-hoc from there.
>
> Parsing the textual output of a command is basically a "bad" thing for
> a number of reasons, primary among which imo is that it generates an
> ever-increasing codebase which depends on command output remaining in
> its initial (or previous) format.

That's your opinion. A guy named Doug McIlroy, however, was pretty
convinced that programs should be able to process the output of other
programs and that their output might, in turn, be processed by yet other
programs, cf

	To put my strongest concerns into a nutshell:

	1. We should have some ways of connecting programs like garden
	hose--screw in another segment when it becomes when it becomes
	necessary to massage data in another way.
        
        http://cm.bell-labs.com/cm/cs/who/dmr/mdmpipe.html

> That now-massive codebase pushes very hard against the possibility of
> change, because a change in the format of the default message output
> of one of the core commands would invalidate so much existing code
> that it must be avoided.
>
> When it is necessary ("oh, never thought of that", or "look, something
> new was invented!") to change existing command output format, in
> marches yet-another-option to request the new format instead of the
> old one, and with it additional logic and additional possibilities for
> errors within the modified command.

It is absolutely not 'necessary' to keep hacking features into a single
program until the source code collapses into a singularity under its own
weight. Sometimes, writing a new program might be a better
idea. Actually, most of the time.

[toc] | [prev] | [next] | [standalone]

#702 — Re: UNIX(*)/ Linux history & system design

crankypuss <crankypuss@nomail.invalid> writes:

[...]

> But the caller (invoker of the command) receives as a result from the
> command only a simple exit-code and whatever command-specific text has
> been output to stdout (or stderr).  It is up to the "caller" to parse
> the "result" text and hopefully make sense of it.

[...]

> "shell" languages are forced to operate in this restrictive mode is
> accepted because that's how the first crude shells were implemented
> and they evolved ad-hoc from there.

Possibly interesting in the context of this 'interesting' assertion:

http://www.techworld.com.au/article/279011/a-z_programming_languages_bourne_shell_sh/?

(somehat painful to access due to 'modern web technology' aka nginx/
varnish, but it'll eventually show up)

Some remarks to the text:

1. This person is obviously 'older' than the unholy BSD-tradition (later
on copied by everyone) of the code author (any code author) considering
himself a godlike creature and everybody else a mindless cell aggregate
which IMHO makes it a rather refreshing read.

2. Fork used to be a very expensive operation by the time the Bourne
shell was written because there was no virtual memory support back then:
A fork involved copying all of the forking process to the swap and then
swapping it in for actual execution. This is no longer true on 'modern
systems' and hasn't been for a while.

[toc] | [prev] | [next] | [standalone]

#694 — Re: UNIX(*)/ Linux history & system design

crankypuss <crankypuss@nomail.invalid> writes:
> On 05/07/2014 09:23 AM, Rainer Weikusat wrote:

[cross-posted to colda with follow-up to, as this is not really about
system development[*]]

[*] It is not terribly on topic, there, either, but 'shell-related
groups' tend to be inhabited by 'strange people' solely busy with
telling others why the shell mustn't be used (a la 'in 1492, on a
DUCStation Donald running Infrix PI - 12, /bin/sh was really a hard link
to /usr/ucb/bbc/cia/doa, therefore, this code is not only not portable
but will actually cause a major earth quake in California' ...) and I'm
more interested in 'using the shell to develop actual applications' than
'Klingon-quoting in hostile environments' ...

[bourne shell language]

>> In addition to constructs specific to its purpose, ie invoking
>> programs, possibly, programs cooperating in a pipeline, it supports the
>> usual syntactic constructs available in 'pre-OO' high-level languages:
>> Three kinds of loop ('loop while a condition is true', 'loop until a
>> condition becomes false', 'loop over the elements of a list'), loop
>> control statements ('break' and 'continue'), conditionals (if .. elif
>> ... elif ... else, a 'case statement' and the possibility to create
>> named subroutines (but not lexically-scoped variables, although bash has
>> them as an extension). It doesn't use braces as universal block
>> delimiter but keywords depending on the 'block construct' being
>> used. For loops, thats do .... done, for if then ... fi and for case, it
>> is case ... esac. The latter two pairs are Algol-68 influenced,
>> presumably, because Steve Bournce (author of the original Bourne shell)
>> was very fond of that. It doesn't support named parameters for functions
>> (or scripts), only positional ones. While I wouldn't to use it (in case
>> suitable alternatives are available) for something larger than at most a
>> few hundred lines of code[*], it is a perfectly usable programming language
>> and often, a very easy to use one.
>>
>> Would you care to elaborate why precisely you consider it to be 'so
>> crude that it is astonishing it didn't vanish'?

[...]

> Only positional parameters for functions,

There are (to the best of my knowledge) the following models for dealing
with 'parameters passed to something'

	- assign passed parameters to predefined 'parameter names' 'by
          position', possibly with support for optional or 'default
          value' arguments

	- parameters are passed in form of 'key/value'-pairs where the
          'key' designates the name of a possible input parameter and
          the value its value
          
	- access passed parameters by position

The first is the one which is most common (nowadays) and it has the nice
property that 'correctness of calls' can be checked statically. OTOH, it
is also the least flexible one. The third is the most general as it can
be used to emulate the other two, either by assigning 'passed
parameters' to 'input parameters' based on position or by interpreting
them as key/value pairs. But it needs explicitly written 'parameter
parsing code' which also needs to determine if the call was sensible/
correct at all. This model is not only also used by 'C programs' (and -
more generally - by anything executed via an exec*-function) but also by
some other programming languages, presumably, because these were
influenced by the shell, notably, Perl (the largest individual Perl
program I'm presently dealing with has 15,666 lines of Perl code and
a few thousand more in auxiliary modules so this is a
perfectly workable model).

> no builtin functions to speak of, which drives the user to
> using sed or some similar external program for even trivial
> functionality.

The main purpose of the shell is to provide facilities for invoking
external programs within a 'global control context' supplying the
facilities for using these 'library routines' in order to solve more
complex problems than what could be done with them in isolation.

Eg, a Bourne shell program for printing the numbers from 0 to 9 could be

n=0
until test $n -gt 9;
do
	echo $n
        n=`expr $n + 1`
done

this uses three 'external commands', test which performs some kind of
test and signals the result via its exit code, echo which prints its
arguments and expr which evaluated an arithmetic expression and prints
the results. All three may also actually be 'shell builtins' as an
optimization/ performance hack at the expense of some loss in
flexibility eg, at some time in the past, I was using an expr with
support for floating-point arithmetic. This wouldn't work for a shell
builtin.

> It's a very basic script processor, not a language.

The 'shell program' is an interpreter for 'the shell language'. 

> On top of that, the bash rules for escaping the characters it has
> coopted for special purposes make it a syntactic nightmare even worse
> than C.

In theory, yes. In practice, "it can be dealt with", especially
considering that a lot of 'shell code' is supposed to operate in a
rather controlled environment.

> The fact that someone kludged control structures onto a simple
> substitution engine does not make it a programming language.

Whether or not something constitutes 'a general-purpose programming
language' is a technical question which can be answered objectively ("Is
it turing-complete?"). That's not at all related to the question if the
programming language is actually useful in practice or who might like or
dislike it for whatever reasons.

[toc] | [prev] | [next] | [standalone]

#695 — Re: UNIX(*)/ Linux history & system design

On 05/07/2014 03:01 PM, Rainer Weikusat wrote:
> crankypuss <crankypuss@nomail.invalid> writes:
>> On 05/07/2014 09:23 AM, Rainer Weikusat wrote:
>
> [cross-posted to colda with follow-up to, as this is not really about
> system development[*]]

Actually it rather is, but clearly you do not see it that way; 
apparently you are able to look at "bash scripts" that are part of the 
init system, and "bash scripts" that are divorced from the system 
insofar that they do not participate in boot-manager setup (cf grub2) or 
similar functions, without seeing that they are in any way similar.

> [*] It is not terribly on topic, there, either, but 'shell-related
> groups' tend to be inhabited by 'strange people' solely busy with
> telling others why the shell mustn't be used (a la 'in 1492, on a
> DUCStation Donald running Infrix PI - 12, /bin/sh was really a hard link
> to /usr/ucb/bbc/cia/doa, therefore, this code is not only not portable
> but will actually cause a major earth quake in California' ...) and I'm
> more interested in 'using the shell to develop actual applications' than
> 'Klingon-quoting in hostile environments' ...

Oh look, it has called me "strange", what a quaint and helpful 
introduction, thank you ever so much!

> [bourne shell language]

following from RW:
>>> In addition to constructs specific to its purpose, ie invoking
>>> programs, possibly, programs cooperating in a pipeline, it supports the
>>> usual syntactic constructs available in 'pre-OO' high-level languages:
>>> Three kinds of loop ('loop while a condition is true', 'loop until a
>>> condition becomes false', 'loop over the elements of a list'), loop
>>> control statements ('break' and 'continue'), conditionals (if .. elif
>>> ... elif ... else, a 'case statement' and the possibility to create
>>> named subroutines (but not lexically-scoped variables, although bash has
>>> them as an extension). It doesn't use braces as universal block
>>> delimiter but keywords depending on the 'block construct' being
>>> used. For loops, thats do .... done, for if then ... fi and for case, it
>>> is case ... esac. The latter two pairs are Algol-68 influenced,
>>> presumably, because Steve Bournce (author of the original Bourne shell)
>>> was very fond of that. It doesn't support named parameters for functions
>>> (or scripts), only positional ones. While I wouldn't to use it (in case
>>> suitable alternatives are available) for something larger than at most a
>>> few hundred lines of code[*], it is a perfectly usable programming language
>>> and often, a very easy to use one.
>>>
>>> Would you care to elaborate why precisely you consider it to be 'so
>>> crude that it is astonishing it didn't vanish'?
>
> [...]

from C (the first elaboration):
>> Only positional parameters for functions,

from RW:
> There are (to the best of my knowledge) the following models for dealing
> with 'parameters passed to something'
>
> 	- assign passed parameters to predefined 'parameter names' 'by
>            position', possibly with support for optional or 'default
>            value' arguments
>
> 	- parameters are passed in form of 'key/value'-pairs where the
>            'key' designates the name of a possible input parameter and
>            the value its value
>
> 	- access passed parameters by position
>
> The first is the one which is most common (nowadays) and it has the nice
> property that 'correctness of calls' can be checked statically. OTOH, it
> is also the least flexible one. The third is the most general as it can
> be used to emulate the other two, either by assigning 'passed
> parameters' to 'input parameters' based on position or by interpreting
> them as key/value pairs. But it needs explicitly written 'parameter
> parsing code' which also needs to determine if the call was sensible/
> correct at all. This model is not only also used by 'C programs' (and -
> more generally - by anything executed via an exec*-function) but also by
> some other programming languages, presumably, because these were
> influenced by the shell, notably, Perl (the largest individual Perl
> program I'm presently dealing with has 15,666 lines of Perl code and
> a few thousand more in auxiliary modules so this is a
> perfectly workable model).

from C:

It's a shame that at one level or another we are all surrounded by 
things we don't truly understand.

In regard to a shell language, one should (imo, of course, which you 
have already implied is "strange") perhaps consider the actual form of 
the 'nix command string, which provides for the passing of both 
arguments and options, in other words, positional parameters (arguments) 
plus key/value pairs (options of the short or long flavor), as 
appropriate to the function performed.

You might also consider the possibility that a "functional" script, that 
being one which is called to perform a specific function and passed 
whatever parameters are appropriate to that function, *might* benefit 
from being able to return data to its caller beyond a simple exit-code 
and whatever possibly-parseable string data it has written to stdout or 
stderr, a "result" that does not have to be parsed by something as crude 
as 'sed' in order to obtain an approximation of what could simply be 
returned.

>> no builtin functions to speak of, which drives the user to
>> using sed or some similar external program for even trivial
>> functionality.
>
> The main purpose of the shell is to provide facilities for invoking
> external programs within a 'global control context' supplying the
> facilities for using these 'library routines' in order to solve more
> complex problems than what could be done with them in isolation.
>
> Eg, a Bourne shell program for printing the numbers from 0 to 9 could be
>
> n=0
> until test $n -gt 9;
> do
> 	echo $n
>          n=`expr $n + 1`
> done
>
> this uses three 'external commands', test which performs some kind of
> test and signals the result via its exit code, echo which prints its
> arguments and expr which evaluated an arithmetic expression and prints
> the results. All three may also actually be 'shell builtins' as an
> optimization/ performance hack at the expense of some loss in
> flexibility eg, at some time in the past, I was using an expr with
> support for floating-point arithmetic. This wouldn't work for a shell
> builtin.
>
>> It's a very basic script processor, not a language.
>
> The 'shell program' is an interpreter for 'the shell language'.

I am constantly amazed at the way some can, through the tunes of the 
unthought words they parrot, consistently hide what is not there by 
covering it with popular illusion.

>> On top of that, the bash rules for escaping the characters it has
>> coopted for special purposes make it a syntactic nightmare even worse
>> than C.
>
> In theory, yes. In practice, "it can be dealt with", especially
> considering that a lot of 'shell code' is supposed to operate in a
> rather controlled environment.

Yes, it is sometimes possible to use a three-headed hammer to adjust a 
watch.

>> The fact that someone kludged control structures onto a simple
>> substitution engine does not make it a programming language.
>
> Whether or not something constitutes 'a general-purpose programming
> language' is a technical question which can be answered objectively ("Is
> it turing-complete?"). That's not at all related to the question if the
> programming language is actually useful in practice or who might like or
> dislike it for whatever reasons.

Whatever reasons might those be, maybe that in addition to being fugly 
and clumsy and not very capable at all, it is used in so many places 
through sheer accident of history (presuming they weren't simply mad to 
begin with) that it is intertwined with the very roots of the linux world?

[toc] | [prev] | [next] | [standalone]

#697 — Re: UNIX(*)/ Linux history & system design

crankypuss <crankypuss@nomail.invalid> writes:
> On 05/07/2014 03:01 PM, Rainer Weikusat wrote:
>> crankypuss <crankypuss@nomail.invalid> writes:
>>> On 05/07/2014 09:23 AM, Rainer Weikusat wrote:
>>
>> [cross-posted to colda with follow-up to, as this is not really about
>> system development[*]]
>
> Actually it rather is, but clearly you do not see it that way;
> apparently you are able to look at "bash scripts" that are part of the
> init system, and "bash scripts" that are divorced from the system
> insofar that they do not participate in boot-manager setup (cf grub2)
> or similar functions, without seeing that they are in any way similar.

Something like this was my idea when I changed the subject. But since
the topic has shifted more towards 'apply something to some use' and I'm
actually doing 'application development' and not 'system development'
(except when necessary/ unavoidable) and consequently, use the shell
rather for that (and some people staring making annoyed noises I sort-of
understand [or believe to do so] ...) ...

>
>> [*] It is not terribly on topic, there, either, but 'shell-related
>> groups' tend to be inhabited by 'strange people' solely busy with
>> telling others why the shell mustn't be used (a la 'in 1492, on a
>> DUCStation Donald running Infrix PI - 12, /bin/sh was really a hard link
>> to /usr/ucb/bbc/cia/doa, therefore, this code is not only not portable
>> but will actually cause a major earth quake in California' ...) and I'm
>> more interested in 'using the shell to develop actual applications' than
>> 'Klingon-quoting in hostile environments' ...
>
> Oh look, it has called me "strange", what a quaint and helpful
> introduction, thank you ever so much!

I didn't address anyone personally and actually consider 'behaviour of
people belonging to recognizable groups' generally strange, eg, "on
weekends, this pub is populated by strange people who seem to enjoy
making loud, ugly noises aka 'shouting' very much (at least, they keep
doing this for some inexplainable reason), seem to enjoy drinking until
they can't walk straight anymore (I can understand the first part but
not the second: As soon as this starts to happen to me, and usually, I
avoid that, I'm obviously having a technical problem and should go home
before I trip over a kerbstone and break a leg or something like this)
and who don't ever seem to get much beyond being an arbitrary appendix
of their own sex life (I'm totally flabbergasted that they're apparently
never bored by this topic)".

Computers are easier to deal with: They usually don't resort to violence
when they feel mistreated, hence, "how to avoid mistreating them" can be
determined by experiments.

>
>> [bourne shell language]
>
> following from RW:
>>>> In addition to constructs specific to its purpose, ie invoking
>>>> programs, possibly, programs cooperating in a pipeline, it supports the
>>>> usual syntactic constructs available in 'pre-OO' high-level languages:
>>>> Three kinds of loop ('loop while a condition is true', 'loop until a
>>>> condition becomes false', 'loop over the elements of a list'), loop
>>>> control statements ('break' and 'continue'), conditionals (if .. elif
>>>> ... elif ... else, a 'case statement' and the possibility to create
>>>> named subroutines (but not lexically-scoped variables, although bash has
>>>> them as an extension). It doesn't use braces as universal block
>>>> delimiter but keywords depending on the 'block construct' being
>>>> used. For loops, thats do .... done, for if then ... fi and for case, it
>>>> is case ... esac. The latter two pairs are Algol-68 influenced,
>>>> presumably, because Steve Bournce (author of the original Bourne shell)
>>>> was very fond of that. It doesn't support named parameters for functions
>>>> (or scripts), only positional ones. While I wouldn't to use it (in case
>>>> suitable alternatives are available) for something larger than at most a
>>>> few hundred lines of code[*], it is a perfectly usable programming language
>>>> and often, a very easy to use one.
>>>>
>>>> Would you care to elaborate why precisely you consider it to be 'so
>>>> crude that it is astonishing it didn't vanish'?
>>
>> [...]
>
> from C (the first elaboration):
>>> Only positional parameters for functions,
>
> from RW:
>> There are (to the best of my knowledge) the following models for dealing
>> with 'parameters passed to something'
>>
>> 	- assign passed parameters to predefined 'parameter names' 'by
>>            position', possibly with support for optional or 'default
>>            value' arguments
>>
>> 	- parameters are passed in form of 'key/value'-pairs where the
>>            'key' designates the name of a possible input parameter and
>>            the value its value
>>
>> 	- access passed parameters by position

[...]

>> The third is the most general as it can> be used to emulate the
>> other two,

[...]

> from C:
>
> It's a shame that at one level or another we are all surrounded by
> things we don't truly understand.
>
> In regard to a shell language, one should (imo, of course, which you
> have already implied is "strange") perhaps consider the actual form of
> the 'nix command string, which provides for the passing of both
> arguments and options, in other words, positional parameters
> (arguments) plus key/value pairs (options of the short or long
> flavor), as appropriate to the function performed.

I think I already wrote that: An ordered sequence of arguments can be
interpreted as 'positional argument list' or 'set of key/values pairs'
or both. 'getopt' is both a C library routine and available to 'shell
scripts and functions defined by them' (I think the lack of both
lexically scoped parameters and lexically bound dynamic parameters is a
much graver deficiency, at least theoretically).

> You might also consider the possibility that a "functional" script,
> that being one which is called to perform a specific function and
> passed whatever parameters are appropriate to that function, *might*
> benefit from being able to return data to its caller beyond a simple
> exit-code and whatever possibly-parseable string data it has written
> to stdout or stderr, a "result" that does not have to be parsed by

Whatever facilities might be available in a given environment, they
could always be improved ...

> something as crude as 'sed' in order to obtain an approximation of
> what could simply be returned.

... by adding missing functionality. Eg, a C function can 'simply
return' any C type or a pointer to a C type, however, if that is
supposed to be written to a file in order to be processed next week or
on a computer in Tierra del Fuego, the ability to serialize and
deserialize a complex data structure could also be very helpful. In
general (truism ahead), different programming environment provide
different features and usually, lack some other features. And "weeping
oneself into sleep because reality is much more harsh and ugly than one
would like it to be" usually doesn't accomplish anything.

>>> no builtin functions to speak of, which drives the user to
>>> using sed or some similar external program for even trivial
>>> functionality.
>>
>> The main purpose of the shell is to provide facilities for invoking
>> external programs within a 'global control context' supplying the
>> facilities for using these 'library routines' in order to solve more
>> complex problems than what could be done with them in isolation.

[example program with explanation]

>>> It's a very basic script processor, not a language.
>>
>> The 'shell program' is an interpreter for 'the shell language'.
>
> I am constantly amazed at the way some can, through the tunes of the
> unthought words they parrot, consistently hide what is not there by
> covering it with popular illusion.

I'm pretty certain that you never heard anyone speak to me, hence, you
cannot possibly know what I might or might not have heard elsewhere, and
you certainly can't read my mind, hence, you know nothing about 'my
thoughts' (or lack thereof). 

>>> On top of that, the bash rules for escaping the characters it has
>>> coopted for special purposes make it a syntactic nightmare even worse
>>> than C.
>>
>> In theory, yes. In practice, "it can be dealt with", especially
>> considering that a lot of 'shell code' is supposed to operate in a
>> rather controlled environment.
>
> Yes, it is sometimes possible to use a three-headed hammer to adjust a
> watch.

An interpreter is not a hammer and 'adjusting watches' not something
programs usually do, IOW, this doesn't really communicated anything
except that you're strongly convinced that something I consider to be
'fit for purpose' (for some purposes) because of in depth practical
experiences with it isn't really 'fit any purposes' and you rejected it
almost outright because of this (earlier, you wrote that you had written
'exactly one shell script'). That's your prerogative but I disagree with
it.

[toc] | [prev] | [next] | [standalone]

#700 — Re: UNIX(*)/ Linux history & system design

On 05/08/2014 07:02 AM, Rainer Weikusat wrote:
<snip>

> I think I already wrote that: An ordered sequence of arguments can be
> interpreted as 'positional argument list' or 'set of key/values pairs'
> or both. 'getopt' is both a C library routine and available to 'shell
> scripts and functions defined by them' (I think the lack of both
> lexically scoped parameters and lexically bound dynamic parameters is a
> much graver deficiency, at least theoretically).

The simple fact that getopt supports different "scanning modes" and 
"compatability" levels is imo sufficient evidence to support the 
assertion that it is not (or perhaps, "no longer") what is needed.  That 
different core commands support different methods of specifying options 
(some require "=", others not, etc) supplies the gravestone imo.

>> Yes, it is sometimes possible to use a three-headed hammer to adjust a
>> watch.
>
> An interpreter is not a hammer and 'adjusting watches' not something
> programs usually do, IOW, this doesn't really communicated anything
> except that you're strongly convinced that something I consider to be
> 'fit for purpose' (for some purposes) because of in depth practical
> experiences with it isn't really 'fit any purposes' and you rejected it
> almost outright

There are levels of scripting.  Basic scripting provides for the 
repeated issuance of one command using various plug-in arguments, for 
issuing a fixed sequence of commands that is abbreviated by the script, 
or some combination of the two.

The bash "language" is imo suitable for only that most basic level of 
scripting, and barely that.

It was imo a serious error to make the decision that a "shell" 
supporting various control characters that are useful in an interactive 
mode should support those control characters via the "language" rather 
than the interactive portion of the shell.  The result of that decision 
makes the syntax of the "language" common, but at the cost of reducing 
the readability of non-interactive code.

[toc] | [prev] | [next] | [standalone]

#683 — Re: UNIX(*)/ Linux history & system design

On 06/05/14 09:46, crankypuss wrote:
> On 05/05/2014 05:17 PM, David Brown wrote:
>> On 06/05/14 00:02, crankypuss wrote:
>>> On 05/05/2014 02:31 PM, Rainer Weikusat wrote:
>>>> crankypuss <crankypuss@nomail.invalid> writes:
>>>
>>> <lots snipped>
>>>
>>>>> Some seem to disagree with that view, but then some people blindly
>>>>> parse command output with sed and use that to drive system-critical
>>>>> functionality; go figure.
>>>>
>>>> I did a lot of shell-programming when I was working on 'embeddded Linux
>>>> project' some years ago because that was the most convenient
>>>> programming
>>>> language available to me in that environment (ARM9 machine language
>>>> and C
>>>> being the other options) and 'sed' was the most sophisticated
>>>> text-processing tool available to me. What's wrong with using it?
>>>
>>> I know nothing about "ARM9 machine language" but if C was available, yet
>>> you used bash/sed, you must have been in one bigassed hurry or had some
>>> extremely simple requirements.  I'm sure your performed your job as
>>> expected and were rewarded accordingly.
>>
>> Someone who writes a C program when a simple bash/sed script could do
>> the task, would be failing to do their job as expected.  One of the
>> reasons Linux (and other *nix flavours) have multiple languages and
>> tools is that different tools are better for different jobs, and that
>> often it is the developer's time that is important rather than the
>> systems run time.  A developer who thinks C is the answer to every
>> question is like a carpenter armed only with a hammer.
>>
>> In the embedded Linux systems I have worked on, the final system used C,
>> Python, Perl, C++, and bash.  I don't think sed was used, except as part
>> of the makefiles.
>>
>
> Indeed, it is the developer's time that is important, because that is
> what the employer has purchased.  Most employees are not very good at
> building tools, and are as shortsighted as their employers.  Tools like
> sed have been magically provided to them, nobody ever had reason to
> write those because they were just magically there.  Far better to
> simply use what has been provided, as expected, rather than building a
> library of C subroutines that puts knocking out a small utility on the
> same level of effort as debugging a bash/sed script where a single
> misplaced character can cause unexpected errors that are difficult to
> find.  Not that C syntax is much less finicky, but it at least provides
> a few diagnostic clues regarding syntax errors.
>
> So, you mistake my words, I do not claim that C is the answer to every
> problem, or even that there is no value in the bash/sed combination
> (though I personally can't think of a situation where I'd use that
> particular combination).  One uses the best tools at hand, and if one
> finds that the best tools at hand are inadequate to the task, he obtains
> a more appropriate tool, or if none are available, builds what is
> needed.  I've used some gawdawful tools over the years, still use some
> on a daily basis; some of them annoy me and they are on the to-do list.
>
> People are different, some are more compliant than others, some simply
> grind through a task manually without even the use of something like
> bash/sed.  Some use the tools at hand, others realize that their own
> hands are their best tools.
>
> Since all programmers are equivalent economic units, they are easily
> replaced if they fail to perform as expected: the hive mind has no love
> for independent workers.

I don't know what world you live in, but it does not correspond to 
anything I have ever seen - either in terms of how people work, how 
employees do their jobs, how employers view their customers or their 
staff, or how any non-commercial work is done.  I resent the implication 
that I am incompetent because I use appropriate tools for the job 
(ready-made if they exist - but making my own if I have to), I resent 
the implication that my employer is incompetent because he expects me to 
use appropriate tools for the job, and I resent the implication that I 
am just a mindless drone at my job or that my employer treats me as such.

I don't know what you are doing posting in this newsgroup, since you 
apparently have no concept of how development work is done (either 
commercial or open-source development), and you have nothing but 
contempt for Linux as an operating system, the Linux environment, and 
the Linux and open-source development ethos (which is /precisely/ to 
re-use existing tools when possible, making new ones when needed, and 
making those new tools available for others to use rather than making 
their own).

[toc] | [prev] | [next] | [standalone]

#686 — Re: UNIX(*)/ Linux history & system design

On 05/06/2014 04:15 PM, David Brown wrote:
> On 06/05/14 09:46, crankypuss wrote:
>> On 05/05/2014 05:17 PM, David Brown wrote:
>>> On 06/05/14 00:02, crankypuss wrote:
>>>> On 05/05/2014 02:31 PM, Rainer Weikusat wrote:
>>>>> crankypuss <crankypuss@nomail.invalid> writes:
>>>>
>>>> <lots snipped>
>>>>
>>>>>> Some seem to disagree with that view, but then some people blindly
>>>>>> parse command output with sed and use that to drive system-critical
>>>>>> functionality; go figure.
>>>>>
>>>>> I did a lot of shell-programming when I was working on 'embeddded
>>>>> Linux
>>>>> project' some years ago because that was the most convenient
>>>>> programming
>>>>> language available to me in that environment (ARM9 machine language
>>>>> and C
>>>>> being the other options) and 'sed' was the most sophisticated
>>>>> text-processing tool available to me. What's wrong with using it?
>>>>
>>>> I know nothing about "ARM9 machine language" but if C was available,
>>>> yet
>>>> you used bash/sed, you must have been in one bigassed hurry or had some
>>>> extremely simple requirements.  I'm sure your performed your job as
>>>> expected and were rewarded accordingly.
>>>
>>> Someone who writes a C program when a simple bash/sed script could do
>>> the task, would be failing to do their job as expected.  One of the
>>> reasons Linux (and other *nix flavours) have multiple languages and
>>> tools is that different tools are better for different jobs, and that
>>> often it is the developer's time that is important rather than the
>>> systems run time.  A developer who thinks C is the answer to every
>>> question is like a carpenter armed only with a hammer.
>>>
>>> In the embedded Linux systems I have worked on, the final system used C,
>>> Python, Perl, C++, and bash.  I don't think sed was used, except as part
>>> of the makefiles.
>>>
>>
>> Indeed, it is the developer's time that is important, because that is
>> what the employer has purchased.  Most employees are not very good at
>> building tools, and are as shortsighted as their employers.  Tools like
>> sed have been magically provided to them, nobody ever had reason to
>> write those because they were just magically there.  Far better to
>> simply use what has been provided, as expected, rather than building a
>> library of C subroutines that puts knocking out a small utility on the
>> same level of effort as debugging a bash/sed script where a single
>> misplaced character can cause unexpected errors that are difficult to
>> find.  Not that C syntax is much less finicky, but it at least provides
>> a few diagnostic clues regarding syntax errors.
>>
>> So, you mistake my words, I do not claim that C is the answer to every
>> problem, or even that there is no value in the bash/sed combination
>> (though I personally can't think of a situation where I'd use that
>> particular combination).  One uses the best tools at hand, and if one
>> finds that the best tools at hand are inadequate to the task, he obtains
>> a more appropriate tool, or if none are available, builds what is
>> needed.  I've used some gawdawful tools over the years, still use some
>> on a daily basis; some of them annoy me and they are on the to-do list.
>>
>> People are different, some are more compliant than others, some simply
>> grind through a task manually without even the use of something like
>> bash/sed.  Some use the tools at hand, others realize that their own
>> hands are their best tools.
>>
>> Since all programmers are equivalent economic units, they are easily
>> replaced if they fail to perform as expected: the hive mind has no love
>> for independent workers.
>
> I don't know what world you live in,

One where we both seem connected to a common network, at least.

> but it does not correspond to
> anything I have ever seen -

Please accept my sincere condolences.

> either in terms of how people work, how
> employees do their jobs, how employers view their customers or their
> staff, or how any non-commercial work is done.

Wow, that's quite a difference.  I've always felt that our family 
reunions (back when we had them) were a clear indication that somewhere 
in the past there must have been space aliens in the lineage, but never 
gave the theory much credence.

> I resent the implication
> that I am incompetent

If I wish to call you incompetent then I will simply do so without 
obfuscation, and most likely in an obscene manner; to the best of my 
recollection I have not done so, thus your resentment appears to have 
been self-generated.

> because I use appropriate tools for the job
> (ready-made if they exist - but making my own if I have to),

That seems to be "coming out of left field" since I advocate the use of 
the best tools available whether they are readymade or must be 
constructed from fresh-mined ore.  The fact that we seem to differ 
insofar as what we consider "acceptable" may be, is barely relevant.

> I resent
> the implication that my employer is incompetent

If your employer was able to get the job done without paying you to do 
it, you would not be employed; that is simple economics, no employer 
ever pays for something unnecessary and remains in business for long. 
Nor does any employer ever pay what the job is worth because if he did 
he would be losing money instead of making it; employee pay is *always* 
less than employee value.

> because he expects me to
> use appropriate tools for the job,

I hope this doesn't shock you, but your employer really doesn't care 
what tools you use to get the job done: your employer cares that the job 
*is* done, and those not terminally short-sighted also care that the 
result of your work will be maintainable by others in the future.  If 
you do something manually, use a "good" tool to do it, use a "bad" tool 
to do it, or build your own tool to do it, the employer has no reason to 
care, so long as the job is done and the result is "good".

> and I resent the implication that I
> am just a mindless drone at my job or that my employer treats me as such.

Some employers micro-manage and should be shaken from one's boots like 
the remains of a dog's bowel movements.  Others do not.  If one remains 
employed by a micro-manager he has reduced himself to the level of a 
mindless drone.  That is not a personal insult, it is a general truth.

> I don't know what you are doing posting in this newsgroup,

Presumably discussing the topics appropriate to this newsgroup.

> since you
> apparently have no concept of how development work is done (either
> commercial or open-source development),

Oh, my; that is quite an ambiguous statement.  It may very well be that 
I have no idea how some undefined group of others does development work, 
but I definitely know how I myself have done it, and how those few true 
developers I've worked with have done it.

> and you have nothing but
> contempt for Linux as an operating system,

I am somewhat abashed to need to remind you that linux is not an 
operating system, it is a kernel.  A kernel itself is not an operating 
system, it does not become an operating system until an intermediate 
layer exists to support applications.  In the case of most linux distros 
I have heard of, it is the core-commands that provide the interface 
level which qualifies a kernel as having become part of an operating 
system.  The collection of code that we generally consider "linux" 
seems, to me at least, decidedly bash-centric; that need not continue to 
be the case.

> the Linux environment,

I'm not sure what you mean by that phrase, "the Linux environment" but 
of course it is not Linux it is linux, and the kernel plus the 
associated higher-level code usually available is sufficiently 
tailorable that one can make it into what one prefers.

> and the Linux and open-source development ethos (which is /precisely/ to
> re-use existing tools when possible, making new ones when needed, and
> making those new tools available for others to use rather than making
> their own).

Wrong again, but at least you are consistently erroneous.

[toc] | [prev] | [next] | [standalone]

#689 — Re: UNIX(*)/ Linux history & system design

On Tue, 2014-05-06, David Brown wrote:
> On 06/05/14 09:46, crankypuss wrote:
...

> staff, or how any non-commercial work is done.  I resent the implication 
> that I am incompetent because I use appropriate tools for the job [...]

Me too ...

> I don't know what you are doing posting in this newsgroup

Me neither ... but I have a killfile.  With the combination of the
group's topic, "crankypuss"' views and his/her style, I don't see this
going anywhere.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#692 — Re: UNIX(*)/ Linux history & system design

On 05/07/2014 02:47 AM, Jorgen Grahn wrote:
> On Tue, 2014-05-06, David Brown wrote:
>> On 06/05/14 09:46, crankypuss wrote:
> ...
>
>> staff, or how any non-commercial work is done.  I resent the implication
>> that I am incompetent because I use appropriate tools for the job [...]
>
> Me too ...
>
>> I don't know what you are doing posting in this newsgroup
>
> Me neither ... but I have a killfile.  With the combination of the
> group's topic, "crankypuss"' views and his/her style, I don't see this
> going anywhere.
>
> /Jorgen
>

Killfiles are your own personal business, do what you need to do.

Given the name of the group, "comp.os.linux.development.system", I 
presumed that discussions of linux system development would be 
appropriate here.

I have been getting the impression however that those reading this group 
are so hidebound and conformist in outlook that there is no interest in 
the concept of something "new" that uses the linux kernel (possibly 
modified) to build something "better" than what is dictated by the Posix 
standard.

If that's the case there's no point in my bothering to read the group, 
the source code is available if I should have questions or decide to 
move forward unilaterally.

Time will tell, as always.

[toc] | [prev] | [next] | [standalone]

#680 — Re: UNIX(*)/ Linux history & system design

crankypuss <crankypuss@nomail.invalid> writes:
> On 05/05/2014 02:31 PM, Rainer Weikusat wrote:
>> crankypuss <crankypuss@nomail.invalid> writes:
>
> <lots snipped>
>
>>> Some seem to disagree with that view, but then some people blindly
>>> parse command output with sed and use that to drive system-critical
>>> functionality; go figure.
>>
>> I did a lot of shell-programming when I was working on 'embeddded Linux
>> project' some years ago because that was the most convenient programming
>> language available to me in that environment (ARM9 machine language and C
>> being the other options) and 'sed' was the most sophisticated
>> text-processing tool available to me. What's wrong with using it?
>
> I know nothing about "ARM9 machine language" but if C was available,
> yet you used bash/sed, you must have been in one bigassed hurry or had
> some extremely simple requirements.

Simple solutions don't necessarily preclude complex requirements. As an
example, assuming a list of interfaces and associated IPv4 address is
supposed to be generated as preparation for displaying a 'configuration
info' web page. Using ip(route), sh (ash, to be precise) and sed that's
something like:

ip -f inet -o addr show | sed 's/^.*: \([^ ]*\) *inet \([^ ]*\) .*/\1:\2/'

Equivalent C code already becomes quite lengthy if the 'legacy'
ioctl-interface is being used (see netdevice(7) for details) and this
becomes worse for the 'native' netlink-interface which additionally has
the nice properties that documentation basically doesn't exist (although
this has improved somewhat since 1998) and that one is supposed to use
'helpful libraries' which are usually GPLed, ie, "all your sources
belong to us" (but don't worry, unless you're at least as sucessful as
Android, you can put them in some place where the sun doesn't shine for
all we care about them).

> I'm sure your performed your job as expected and were rewarded
> accordingly.

I wasn't forced to go hungry most of the time, if that's what you're
referring to.

[toc] | [prev] | [next] | [standalone]

#654

On 25/04/14 11:57, crankypuss wrote:
> On 04/24/2014 02:46 PM, David Brown wrote:
>> On 23/04/14 13:50, crankypuss wrote:
>>> On 04/22/2014 04:06 PM, David Brown wrote:
>>
>> My point is that it is impossible for a /filesystem/ to guarantee
>> erasure of old data.
>
> If you're talking specific linux filesystems, or linux filesystems in
> general, that may be the case; if you're talking filesystems in the
> wholly general sense, it is not the case.

No, it is the case for /all/ filesystems.  For some combinations of 
particular filesystems and particular media, you can be sure of erasing 
old data (or at least, sure that you can erase old data assuming you 
don't have a hardware error that prevents the change).

I think you are failing to understand that there are multiple layers 
here.  A filesystem sits on top of a block device of some sort.  There 
can be multiple layers of block devices (such as raid, encryption, lvm, 
etc.).  Then there is the media, which will normally have some sort of 
translation or abstraction layer to make it look like an array of 
sectors (for flash devices, this layer handles logical-to-physical 
translation, garbage collection, etc., while for hard disks it handles 
sector relocations, error correction, etc.).

If the media does not support erasing or overwriting, or if it makes it 
hard to do so reliably, then /no/ filesystem can help.

For example, if we take your mythical "erase on demand" filesystem and 
use it on a raid1 disk pair, it might seem to be able to erase deleted 
files completely.  But if we take out one of the disks, the raid1 array 
will still work fine in degraded mode - you can use it to read, write 
and erase files.  As far as the filesystem is concerned, everything is 
working as it should.  If you ask the filesystem to securely erase a 
file, it will appear to do so - and the filesystem thinks the file is 
erased.  But in reality a copy of the file is still there on the other 
disk that was removed.


>
>> In many cases, you could get very close - on a harddisk, if you
>> overwrite an existing sector then old data on the sector is gone for
>> ever (the idea of having to make multiple passes of random data, ones,
>> zeros, etc., is a baseless myth).
>
> Assuming that you verify that the data has actually been overwritten,
> yes; however, read/verify-after-write is costly, so it becomes a
> performance-vs-reliability issue unless there are specific interfaces to
> specify the various operations.

And what does this verify tell you?  It tells you only that the system 
believes the data has been written to the media.  It says nothing about 
what might have been overwritten or what might still exist.

>
>> However, harddisks occasionally
>> relocate sectors - you can't control this, and you cannot access the old
>> data or erase it without opening the disk physically.  It's a rare
>> occurrence - but for the paranoid, less than 100% is not good enough.
>
> If the device lies about what it is doing, all one can do is put it on a
> blacklist and refuse to support the device entirely, or establish it as
> a device to be used for non-critical data only.

The device is not lying - it is just that no one claims that harddrives 
(or other media) work the way you apparently think they do.  Once you 
realise that your idea of harddisks as a simple array of directly 
addressable sectors is oversimplifying, and completely wrong for flash 
devices, then you will understand the issues here.

In reality, of course, you do not need 100% guarantees that deleted data 
is gone - normally, it is sufficient to say that it will /probably/ be 
gone, and it will certainly be hard to recover anything that happens to 
be left over.  Hard disks and SSDs will give you that, as will most 
filesystems - even those that have a policy of not overwriting old data. 
  Even though in theory there may be some or all of the data on the 
disk, it is extremely hard to get at it.

>
> When "device" behavior reaches a certain point of independence, it has
> actually become a server and needs to be given no more trust than a
> server accessed over a trusted connection.  It is (imo, at least)
> necessary to be cognizant of whether the hardware designers have stuck
> their nose into your business, and treat those "devices" with
> significant skepticism.

Nonsense.  You are imagining rules about what /you/ think device 
manufacturers should do - but they are your rules, and neither device 
manufacturers nor other users play by those rules.  There are no hard 
and fast dividing lines here.

>
>> In other cases, particularly SSDs, then overwriting existing logical
>> sectors does not affect the old data directly, and it could be
>> recovered.  Other forms of data storage, such as cloud storage, are
>> getting more common - and you have even less control there.
>
> If there is no device reliability, the device should not be used for
> critical data.  You wouldn't expect a large business to send its daily
> cash intake to the bank by using some homeless person grabbed off the
> street, and using cloud storage is similar; some may be reliable but
> others not.

Reliability has no remote connection with your desire for 100% erasability.

>
>> So no filesystem can actually guarantee that it can erase old data - and
>> filesystems therefore normally provide no functions to erase old data.
>
> If a courier service routinely grabs homeless people off the street to
> deliver large amounts of cash, one might wisely choose not to use them.
>
> We still disagree about what "no" filesystem can do.  If you mean to
> limit the discussion to "existing linux filesystems" then I might have
> no choice but to concede your point; if you want to talk about what can
> be done, what is actually possible, then we will probably continue to
> disagree on this subject.

See earlier in this post for the critical issue of what a filesystem 
actually /is/, and how it is independent of the media.

A combination of filesystem and media may give you close to the 100% 
data erasability that you are looking for, although it still assumes 
there are no failures in hardware and software along the way.  Old hard 
disks did not have relocatable sectors, and thus no risk of storing data 
in places that were not addressable by the filesystem - such disks could 
be used with a filesystem that supports scrubbing.

Other than that, there is an extremely simple way to get 100% 
erasability on whatever media you want, and with whatever filesystem you 
want - use encryption.  At the point of encryption (either on the block 
layer, or in the filesystem, or on particular files), you can get full 
erasure.

There is also physical destruction of the media itself, if that appeals.

>
>> Whether they re-use deleted sectors quickly or slowly is a matter of the
>> structure of the file-system, and what is most efficient, and it may
>> vary depending on the media (placement to minimise fragmentation and
>> head movement is usually best on harddisks, re-using deleted logical
>> sectors is most efficient on SSD's because it becomes an automatic TRIM
>> - even though the physical sectors will not be reused quickly).
>>
>>>
>>> While encryption does provide the advantage of helping to protect data
>>> stored on a device that becomes physically available to the malicious,
>>> the advent of "key managers" leads one to consider the possibility that
>>> encrypted filesystems may amount to much ado about nothing, and that
>>> other avenues of protection may be more effective overall.
>>
>> That is certainly true - your data is only as safe as your key.  But the
>> usual procedure for encrypted partitions is to enter your key (or
>> passphrase) at the keyboard when mounting.
>>
>
> Yes, and once you have done so, the mounted partition is no more secure
> than a non-encrypted partition; data security has, through your
> supplying the key, been degraded from whatever level of security is
> provided by the encryption algorithm used, to the simple "security"
> provided by the linux protection modes.  To quote you, "for the
> paranoid, less than 100% is not good enough".

It does not take a genius to understand that the key is needed to access 
the data.  If you want to minimise the time that the partition or data 
is unlocked, which is understandable for some uses, then simply unmount 
the encrypted partition when it is not in use.  You can also easily 
encrypt individual files rather than the whole partition, if it makes 
things easier for you.

>
> I am of the opinion that the linux protection modes inherited from Unix
> are not very "adequate", and the idea of a "superuser" is inherently
> crippled "security".  The whole user/group/world concept along with the
> ability to open files for non-exclusive access is ridiculously
> antiquated and stems from the pre-1970 era when bisync was
> state-of-the-art and the concept of servers as separate asset owners was
> little more than a foggy idea in the minds of a few.  The architecture
> survives to the present basically because it is politically advantageous
> to maintain posix compatability, but personally I would prefer something
> a bit more modern even at the cost of losing compatability.

This is again a completely different issue, and bears no connection with 
the discussion so far.  But since you have brought it up, I would say 
that I find the user/group/world arrangement extremely useful, and 
flexible enough to cover almost all security situations I have come 
across (I run the company network and servers, for about 80 users) with 
very little maintenance and effort.  In our pre-Linux days when we had a 
Windows NT 4.0 server, using Windows ACL's was a great pain and a huge 
waste of time - and usually problems with access were "fixed" by giving 
access to everyone, or the administrator having to first claim ownership 
of directory trees before being able to grant appropriate access.  In 
theory it offered very fine grain control - in practice, it was a mess.

Having said that, there /are/ occasions when finer control than posix 
permissions can be useful.  This is why Linux has had a number of 
additional security systems, including Access Control Lists, selinux, 
capabilities, etc., for many years.  Most people (including me) don't 
use them because we don't need them, but they certainly exist for those 
that have special requirements.

>
> Sorry for ranting, I'll stop before beginning to spew about whether
> encryption is really useful. <g>

That's fine - I'm ranting too, and it's best to get it out in the open!

[toc] | [prev] | [next] | [standalone]

#655

On 04/26/2014 08:30 AM, David Brown wrote:

> I think you are failing to understand that there are multiple layers
> here.

> If the media does not support erasing or overwriting, or if it makes it
> hard to do so reliably, then /no/ filesystem can help.

> The device is not lying - it is just that no one claims that harddrives
> (or other media) work the way you apparently think they do.  Once you
> realise that your idea of harddisks as a simple array of directly
> addressable sectors is oversimplifying, and completely wrong for flash
> devices, then you will understand the issues here.

> Nonsense.  You are imagining rules about what /you/ think device
> manufacturers should do - but they are your rules, and neither device
> manufacturers nor other users play by those rules.  There are no hard
> and fast dividing lines here.

> Reliability has no remote connection with your desire for 100% erasability.

> See earlier in this post for the critical issue of what a filesystem
> actually /is/, and how it is independent of the media.
>
> A combination of filesystem and media may give you close to the 100%
> data erasability that you are looking for, although it still assumes
> there are no failures in hardware and software along the way.  Old hard
> disks did not have relocatable sectors, and thus no risk of storing data
> in places that were not addressable by the filesystem - such disks could
> be used with a filesystem that supports scrubbing.

It seems to me that we are talking at cross-purposes here; we are 
largely stating the same meanings but expressing them in different ways, 
and the syntax of our expressions is making their semantics seem more 
different than it is.

You seem to think I'm saying that hardware makers should conform to my 
views of how hardware should work, but what I'm saying is something 
different, I'm saying that the software on the client system needs to 
ascertain the nature of the subsystem that serves it files and behave 
accordingly.  Maybe that will be ascertained by looking up the device 
identification presented by the device (they do sometimes fib about what 
they are when new hardware is introduced to support existing interface 
models) then finding its characteristics in some device information 
database, or it might be ascertained by monitoring the behavior of the 
device; eventually, we are likely to reach the point where only 
behavior-monitoring is useful, bestcase.

However.  One thing I am saying is that if an operating system claims to 
support file-scrubbing on a "filesystem" that does not support 
file-scrubbing, then that operating system is broken.  If it allows a 
file that needs to be placed only on a "scrub-able" filesystem to be 
placed on any other type of filesystem, it is broken.  And if it does 
not provide interfaces that allow the programmer to identify the file as 
"scrub-able" versus "plain-old-data" when it is being created, then it 
is not exactly broken, but it definitely lame.

So, disclaimers that shred/scrub are only useful on this or that kind of 
filesystem are inherently faulty; at a minimum the shred/scrub 
operations should be returning a blatant failure indication when issued 
against a file residing on an unsupported filesystem (and never having 
used them, much less read their code, I don't know if that is the case 
or not, but as I recall the OP didn't mention their failing, only the 
warning not to trust them on this or that filesystem).

Now, granted there are not a preponderance of devices available today 
that support the concept of scrubbing, but that does not mean that 
encryption as we know it today is the answer.  An interim answer, yes, I 
suppose, but not a real answer.  Passphrase-oriented protection 
mechanisms are only useful if the passphrase cannot be guessed (through 
cleverness or a brute-force attack) and if the owner of the data cannot 
be dragged away to some cellar where he is thumped-on or waterboarded or 
whatever until he gives up the passphrase.

No, at present I have nothing better to offer than the passphrase 
concept, but if someone calls you on the phone and claims to be your 
wife or a close friend, it won't take you very long to identify the 
person as a fraud; I think it is the mechanisms involved therein that 
will provide a better answer than passphrase protection.

And I think that operating systems should place more emphasis on who is 
allowed to access "files" than whether whoever it actually is has the 
proper passphrase.  I think that instead of continuing with the old 
philosophy of a "filesystem" implemented as an integral part of a 
monolithic operating system, we need to lean more toward the concept of 
device-handlers as servers, so that the increasingly complex world of 
cloud servers and suchlike does not continue to be handled as a "special 
case" but in the same was as local devices are handled.

One thing that still amazes me is the way linux handles passwords, such 
as the omnipotent 'root' password.  Some GUI screen is presented and the 
user enters the password, with nothing whatsoever to indicate that the 
password request is not bogus.  There are better ways to handle that 
kind of thing imo.  Whatever, enjoy your pancakes.

[toc] | [prev] | [next] | [standalone]

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

Back to top | Article view | comp.os.linux.development.system

csiph-web