Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.arch.embedded > #18972

Re: Parsers for extensible grammars?

Show all headers | View raw

Hi George,

On 10/26/2014 11:36 AM, George Neuner wrote:

>>> The relevance (if any) depends on whether you want to allow grammar
>>> extensions direct access to your device API:  i.e. whether the
>>> extensions should be allowed to manipulate your (part of the) device
>>> or whether they have to go through your provided language to do it.
>>
>> The goal is for people to be able to extend the "system" -- in whichever
>> directions THEY choose -- and provide a simple way of configuring those
>> enhancements (software/hardware/systems).  It's a "low value" operation
>> and, as such, doesn't merit lots of resources.  OTOH, it shouldn't
>> require "surgery" to tweek variables deep in the sources (these sorts
>> of "selections" should be very visible without detailed inspection).
>
> I guess the real question is why *you* should be providing that?
> You've already given them a way to control your gizmo and they have
> the gizmo's API with which to construct a different command set if
> they want.

I have to provide some of these features to get ANY system configured, etc.
I can do this as a completely independant "service" and require others
to invent their own solution to THEIR problems (the customizations that
they may implement).

*Or*, I can put facilities in place that encourage them to "do as I have
done" in the hopes of a tighter, more consistent implementation.  I
expect people to be inherently lazy and opt to *modify* something
that is proven as "working" rather than trying to create something
new, from scratch.

But, in doing so, I have to make a decent attempt at anticipating the sorts
of things they *may* want/need to do.

(e.g., expecting everything to be VERB NOUN pairs might rub some folks
the wrong way)

>> I've considered it in the context of "settings" -- almost like twiddling
>> envars -- except it need not be.  I.e., the "commands" could initiate
>> *actions* if so desired.
>>
>> I *hadn't* considered it as a "programming/scripting language" (which
>> is why I called it a "command parser" and presented command line
>> argument parsing and configuration file parsing as the examples to
>> illustrate its intent.
>>
>> There *could* be some value to a real procedural language, there.
>> But, when I conveyed the "Forth" suggestion to colleagues, it didn't
>> fare well (I suspect that had more to do with "Forth" than the
>> "programming language aspect" of the idea -- almost provincial!
>
> Forth - and Lisp, too - has a bad rep in many circles ... which is
> unfortunate because sometimes it may be the simplest answer to the
> problem.

That's true of all tools.  How many folks avoid dynamic memory allocation
in the belief that it is (inherently!) unsafe?  Or, recursive implementations
("what happens if the recursion never stops?"  "Well, same thing if your
iteration never stops!")

> From what I've read I don't think you're really needing a programming
> language.  But then I'm still trying to figure out how sophisticated
> the commands are likely to be ... they don't look like much from your
> examples, but the discussion hints at more complexity.

I don't know as I can't envision the varieties of hardware (and software)
that will be globbed on.  E.g., I had not envisioned the need for a
conditional operation -- yet, in hindsight, it seems like it could be
extremely useful!  ("Do this -- unless you CAN'T!  In which case, do this
OTHER thing!")

But, how far do you take this before you begin to delve into the realm
of "just add it to the executable directly"?

[When I offered the "C interpreter" as an alternative to Forth -- to see
if the resistance was due to the language or the mechanism -- this was
the reply!  "If you're going to write code, then why create another
means of writing code OUTSIDE the system??"]

>>>> It's a tough call -- assume competence in EVERYTHING?  Or, just
>>>> assume competence in their knowledge of their "extensions"??
>>>
>>> It's reasonable to expect that many developers will have little
>>> experience with _formal_ parsing methods and tools ... quite a lot of
>>> applications can get by with RegEx or even just separator tokenization
>>> and have no need of any formal grammar.
>>
>> Exactly.  That was my (upthread) observation of what I've encountered
>> in config file parsers, command line option parsing, etc.  When you
>> consider how "rich" some of those environments are (options, etc.),
>> it's a wonder that so much ad hoc parsing is done!  I can see
>> "evolution"/feeping creaturism explaining some of it (i.e., "it wasn't
>> this complex when we started") but I doubt that's the real cause.
>>
>> I suspect it's more one of familiarity:  you write "generic code"
>> everyday.  So, writing something to extract a numeric value from
>> the "second whitespace-delimited field" in a statement is trivial.
>> OTOH, writing a formal grammar so that a *tool* can do this for
>> you AUTOMATICALLY probably means you spend more time relearning the
>> tool than you would have spent writing the code!
>
> I think it's more panic upon seeing the manual's introduction to the
> method theory.  The vast majority of programmers today have little or
> no formal schooling, and language theory waters (doesn't matter
> syntactic or semantic) get deep very fast.

Or, a lack of patience -- as exemplified by wanting to write code
"on day one" (before they even know where that code is *headed*).

> As with most things, deep understanding isn't necessary to use the
> tools, but a programmer does need at least a passing familiarity with
> the tool's method to understand what it is trying to do and why it is
> failing.  In this regard parser generators are less forgiving than the
> average compiler and it doesn't much matter which method(s) the tool
> uses - you can make as big a mess with LL(n) as with (LA|S)?LR(1) and
> quite easily hang yourself using PEG or GLR.

My experiences with lex/yacc were far less "pleasant" than writing
arbitrarily complex code in a procedural/object language!  Too much
head scratching as there were interactions "across" the grammar
that were only apparent in hindsight.  The way I thought of the various
constructs "as a human being" was very different from the way the tools
NEEDED me to think of them.  This doesn't appear in *most* languages
(aside from unfortunate issues like "foo =- 2;")

> Learning to use a parser generator isn't really any harder than
> learning any other programming language, but the documentation tends
> to be scarier.  This isn't helped by the tendency of tool developers
> to minimally document and to relate their wares to existing tools that
> the user is presumed already to be familiar with.

There tends to be far more abstraction involved.  People seem to have
a problem with abstraction -- always wanting to know "why do you want
to do that" or "what are you trying to do".

    "Timmy has three apples.  Mary has one.  How many apples would Timmy have
    to give to Mary in order for them to each have the same number of apples?"

    "Why should Timmy have to give her ANY?  Why couldn't *Mary* bring the
    right number of apples to school??"

[Of course, the correct answer is that Timmy should quickly EAT two of his
apples and the problem then disappears!  :> ]

>> <shrug>
>
> Nothing to be done about it except try to convince people that it
> really isn't that hard and that, for most purposes, they really should
> be using a generator tool rather than rolling their own.

Check your mail.  I think the solution I proposed will be *so* tempting
to others that they'll just "cut and paste; edit to taste".  I figured
easier to take this offlist than deal with the inevitable complaints about
"lengthy posts"...  :>

> The only really valid exceptions are very simple interface "languages"
> and very tiny systems 8-) that can't afford the overhead of a
> generated parser.  The constant overhead of a generated parser is
> fairly large: with care using Bison you can shoehorn a fairly complex
> language into ~10KB ... but even the simplest VERB NOUN command parser
> is difficult to bring in under ~3KB.  And Bison actually is pretty
> good at making small parsers - there are a few tools that are better,
> but many more are worse.
>
> However, I think even the parser size argument is losing weight as the
> average "small" system keeps getting bigger (32-bit ARM running Linux
> toasting bread, etc.).  Moreover, many small system developers are
> comfortable using (some form of) RegEx to handle interfaces and
> probably most have no clue about its intrinsic overhead.

But even big systems often have "regions" in which severe constraints
are placed on the implementation.  E.g., many of those "big Linux
implementation" compress the kernel to economize on FLASH/ROM -- this
suggests an unwillingness to add "unnecessary resources" for a one-time
event (decompressing the kernel/system on IPL).  The same sorts of
arguments can apply to "setting configuration options" ("Can't we just
set some bitmaps in a hidden corner of FLASH?")

Or, things like delivering options in BOOTP packets, etc.  Sometimes
you just can't "do what you want" (or what would be "ideal")

Back to comp.arch.embedded | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Parsers for extensible grammars? Don Y <this@is.not.me.com> - 2014-10-22 12:14 -0700
  Re: Parsers for extensible grammars? Don Y <this@is.not.me.com> - 2014-10-22 12:17 -0700
  Re: Parsers for extensible grammars? Tim Wescott <seemywebsite@myfooter.really> - 2014-10-22 15:13 -0500
    Re: Parsers for extensible grammars? Don Y <this@is.not.me.com> - 2014-10-22 22:54 -0700
      Re: Parsers for extensible grammars? Stefan Reuther <stefan.news@arcor.de> - 2014-10-23 18:36 +0200
        Re: Parsers for extensible grammars? Don Y <this@is.not.me.com> - 2014-10-23 12:10 -0700
      Re: Parsers for extensible grammars? Tim Wescott <seemywebsite@myfooter.really> - 2014-10-23 13:45 -0500
        Re: Parsers for extensible grammars? Don Y <this@is.not.me.com> - 2014-10-23 12:15 -0700
      Re: Parsers for extensible grammars? jim.brakefield@ieee.org - 2014-10-24 07:05 -0700
        Re: Parsers for extensible grammars? Don Y <this@is.not.me.com> - 2014-10-24 08:04 -0700
  Re: Parsers for extensible grammars? Clifford Heath <no.spam@please.net> - 2014-10-23 08:22 +1100
    Re: Parsers for extensible grammars? Don Y <this@is.not.me.com> - 2014-10-22 23:25 -0700
    Re: Parsers for extensible grammars? George Neuner <gneuner2@comcast.net> - 2014-10-23 04:45 -0400
      Re: Parsers for extensible grammars? Don Y <this@is.not.me.com> - 2014-10-23 04:14 -0700
        Re: Parsers for extensible grammars? George Neuner <gneuner2@comcast.net> - 2014-10-23 16:49 -0400
          Re: Parsers for extensible grammars? Don Y <this@is.not.me.com> - 2014-10-24 07:38 -0700
            Re: Parsers for extensible grammars? George Neuner <gneuner2@comcast.net> - 2014-10-26 03:47 -0400
              Re: Parsers for extensible grammars? Don Y <this@is.not.me.com> - 2014-10-26 08:41 -0700
                Re: Parsers for extensible grammars? George Neuner <gneuner2@comcast.net> - 2014-10-26 14:36 -0400
                Re: Parsers for extensible grammars? Don Y <this@is.not.me.com> - 2014-10-26 12:37 -0700
                Re: Parsers for extensible grammars? George Neuner <gneuner2@comcast.net> - 2014-10-26 18:54 -0400
        Re: Parsers for extensible grammars? Simon Clubley <clubley@remove_me.eisner.decus.org-Earth.UFP> - 2014-10-25 02:02 +0000
          Re: Parsers for extensible grammars? Don Y <this@is.not.me.com> - 2014-10-25 01:56 -0700
            Re: Parsers for extensible grammars? Simon Clubley <clubley@remove_me.eisner.decus.org-Earth.UFP> - 2014-10-25 11:15 +0000
              Re: Parsers for extensible grammars? Don Y <this@is.not.me.com> - 2014-10-25 10:21 -0700
                Re: Parsers for extensible grammars? Simon Clubley <clubley@remove_me.eisner.decus.org-Earth.UFP> - 2014-10-26 01:05 +0000
                Re: Parsers for extensible grammars? Don Y <this@is.not.me.com> - 2014-10-25 19:37 -0700
  Re: Parsers for extensible grammars? Andrew Smallshaw <andrews@sdf.lonestar.org> - 2014-10-23 20:04 +0000
  Re: Parsers for extensible grammars? Les Cargill <lcargill99@comcast.com> - 2014-10-23 22:30 -0500
    Re: Parsers for extensible grammars? rickman <gnuarm@gmail.com> - 2014-10-25 13:36 -0400
      Re: Parsers for extensible grammars? Les Cargill <lcargill99@comcast.com> - 2014-10-25 13:14 -0500
        Re: Parsers for extensible grammars? rickman <gnuarm@gmail.com> - 2014-10-25 14:23 -0400
          Re: Parsers for extensible grammars? Les Cargill <lcargill99@comcast.com> - 2014-10-26 00:50 -0500
          Re: Parsers for extensible grammars? albert@spenarnc.xs4all.nl (Albert van der Horst) - 2014-10-26 19:21 +0000
            Re: Parsers for extensible grammars? rickman <gnuarm@gmail.com> - 2014-10-26 15:29 -0400
  Re: Parsers for extensible grammars? Paul E Bennett <Paul_E.Bennett@topmail.co.uk> - 2014-10-24 09:56 +0100
    Re: Parsers for extensible grammars? Don Y <this@is.not.me.com> - 2014-10-24 07:46 -0700
      Re: Parsers for extensible grammars? Don Y <this@is.not.me.com> - 2014-10-25 19:54 -0700
        Re: Parsers for extensible grammars? Simon Clubley <clubley@remove_me.eisner.decus.org-Earth.UFP> - 2014-10-26 12:15 +0000
          Re: Parsers for extensible grammars? Don Y <this@is.not.me.com> - 2014-10-26 09:02 -0700
            Re: Parsers for extensible grammars? rickman <gnuarm@gmail.com> - 2014-10-26 12:13 -0400
    Re: Parsers for extensible grammars? rickman <gnuarm@gmail.com> - 2014-10-25 13:35 -0400
      Re: Parsers for extensible grammars? Paul E Bennett <Paul_E.Bennett@topmail.co.uk> - 2014-10-26 01:29 +0100
        Re: Parsers for extensible grammars? rickman <gnuarm@gmail.com> - 2014-10-25 21:11 -0400

csiph-web