Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compilers > #350 > unrolled thread

Looking for volunteers for XL

Back to article view | Back to comp.compilers

Contents

  Looking for volunteers for XL Christophe de Dinechin <christophe@taodyne.com> - 2011-11-22 21:03 -0800
    Re: Looking for volunteers for XL Kaz Kylheku <kaz@kylheku.com> - 2011-11-26 05:43 +0000
    Re: Looking for volunteers for XL Christophe de Dinechin <christophe@taodyne.com> - 2011-11-26 12:38 -0800
      Re: Looking for volunteers for XL "BartC" <bc@freeuk.com> - 2011-11-26 23:19 +0000
        Re: Looking for volunteers for XL Christophe de Dinechin <christophe@taodyne.com> - 2011-11-27 12:34 -0800
          Re: Looking for volunteers for XL "BartC" <bc@freeuk.com> - 2011-11-27 22:24 +0000
            Re: Looking for volunteers for XL Christophe de Dinechin <christophe@taodyne.com> - 2011-11-28 14:12 -0800
              Re: Looking for volunteers for XL ardjussi <jussi.santti@ard.fi> - 2011-11-30 13:16 -0800
                Re: Looking for volunteers for XL Kaz Kylheku <kaz@kylheku.com> - 2011-12-01 05:44 +0000
                  Re: overloading, was Looking for volunteers for XL glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2011-12-02 05:36 +0000
            Re: Looking for volunteers for XL tm <thomas.mertes@gmx.at> - 2012-01-03 09:28 -0800
        Re: Looking for volunteers for XL Kaz Kylheku <kaz@kylheku.com> - 2011-11-28 04:45 +0000
          Re: Looking for volunteers for XL Timothy Knox <tdk@thelbane.com> - 2011-11-27 22:50 -0800
            Re: Looking for volunteers for XL Alex McDonald <blog@rivadpm.com> - 2011-12-01 12:11 -0800
          Re: Looking for volunteers for XL "BartC" <bc@freeuk.com> - 2011-11-28 10:23 +0000
            Re: Looking for volunteers for XL glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2011-11-29 07:37 +0000
              Re: macros, Looking for volunteers for XL Gene Wirchenko <genew@ocis.net> - 2011-12-03 17:36 -0800
                Re: macros, Looking for volunteers for XL glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2011-12-05 04:24 +0000
            Re: Looking for volunteers for XL Kaz Kylheku <kaz@kylheku.com> - 2011-12-01 05:35 +0000
              Re: designing language extensions, was Looking for volunteers for XL Marco van de Voort <marcov@toad.stack.nl> - 2011-12-03 13:02 +0000
              Re: Looking for volunteers for XL jgk@panix.com (Joe keane) - 2011-12-13 00:08 +0000
                Re: macros, was Looking for volunteers for XL Kaz Kylheku <kaz@kylheku.com> - 2011-12-13 01:39 +0000
                  Re: macros, was Looking for volunteers for XL Kaz Kylheku <kaz@kylheku.com> - 2011-12-14 19:00 +0000
                  Re: macros, was Looking for volunteers for XL jgk@panix.com (Joe keane) - 2011-12-15 15:40 +0000
                    Re: macros, was Looking for volunteers for XL Kaz Kylheku <kaz@kylheku.com> - 2011-12-16 17:48 +0000
          Re: Looking for volunteers for XL glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2011-11-28 10:26 +0000

Page 1 of 2  [1] 2  Next page →

#350 — Looking for volunteers for XL

Hello,

I'm the designer of the XL programming language (http://xlr.sf.net).
This language is designed to make it as easy to extend the language as
it is today to add classes in traditional languages.

It exists in two flavors. One, XL2, is imperative, statically
compiled. The other, XLR, is functional, dynamically compiled with
LLVM.

This is a big project, and I've been at it for years. It is now in a
state where many things work, where the general structure is clear,
but also where a large number of details need to be fleshed out.

So we need volunteers. Badly. In particular in the following areas:

1. Documentation. The web site was lost a couple of years ago, and I'm
not done rebuilding it.  Starting with
http://xlr.sourceforge.net/sites/default/files/XLRef.pdf or
http://cc3d.free.fr/xl.pdf, it would be nice to have pages for
generics, syntax, etc on the main web site, but also how-to experience
from folks who didn't write the code in the first place.

2. Productizing XL2: XL2 is quite far along, with things like generics
or meta programming working relatively well. But we need library
contents. We need generic specialization. We need UTF8 support. We
need expression templates or similar meta-programming tricks. And
more.

3. XLR type system: XLR is unique in that it has very few operators.
The tree rewrite operator -> means "transform what is on the left as
what is on the right". It can act as a variable, function, macro
declaration depending on context. It turns out that this makes for a
very "interesting" type system, where you don't have things like
"macros" or "functions", but per-parameter behaviors (i.e. this
parameter is evaluated every time it is used like in a macro, but that
parameter is evaluated before call like for a function). I'd like to
discuss it with like-minded people. I think making this more formal
could be a good PhD topic.

4. Type inference: Knowing that the type system is like nothing else,
I still ventured into trying to implement type inference a la Haskell.
It works relatively well on simple cases, but going from here to a
full-fledged type system is still a lot of work. Right now, on the
cases where type inference works, we are within 15% of the performance
of optimized C. When it doesn't work, we can have a 20x slowdown. So
you see why I'd like to finish this.

So if you feel like you have too much time on your hands, or if
compilers are your thing and you want to take part in creating
something really new in that domain, or if you are just a nice guy who
wants to help, please join us.

[There were a bazillion extensible languages in the 1970s, many quite
sophisticated.  They all disappeared without a trace, largely because
the ability to do per-program extensions meant that every program was
written in a different language, making them all unreadable.  It turned
out that semantic extension, a la C++ overloaded operators, is a lot
more usable than extra syntax.  I assume you're familiar with EL/1,
IMP-72, et al, so why is this any better? -John]

[toc] | [next] | [standalone]

#353

On 2011-11-23, Christophe de Dinechin <christophe@taodyne.com> wrote:
> Hello,
>
> I'm the designer of the XL programming language (http://xlr.sf.net).
> This language is designed to make it as easy to extend the language as
> it is today to add classes in traditional languages.

Common Lisp and Clojure blow this out of the water, sorry.

I will wait until X turns to Y, then to Z, A, B and finally we get
to CL. Greenspunning complete!  :)

XL: user-defined optimizations (example from XL website):

translation Simplify
  when
    'X' * 2
  then
    return parse_tree('X'+'X')

It looks slick, but it's nothing I can't get with 1980 vintage
CL compiler macros:

;; the function
(defun mult (left right) (* left right))

;; the compiler macro to optimize it
(define-compiler-macro mult (&whole form left right)
  (cond
    ;; if left is a constant expression evaluating to 2
    ;; then
    ((and (constantp left) (eql 2 (eval left)))
     `(+ ,right ,right))
    ((and (constantp right) (eql 2 (eval right)))
     `(+ ,left ,left))
    ;; fallback on the original form
    (t form)))

This has worked for more than 30 years. But the above  has a flaw:
multiple evaluation. Does the XL transformer deal with it?
I mean,   parse_tree('X'+'X') looks like it it might calls for two evaluations
of X. Are these X's pointers to a common subexpression that is ultimately
evaluated once? (DAG structure in the AST?) In that case, what if you sometimes
WANT multiple evaluation? How do I take a single X into three copies of X that
are independent?

Fix for the issue: generate code to evaluate once to a temporary
variable, then add the temporary to itself:

(define-compiler-macro mult (&whole form left right)
  (let ((temp (gensym)))
    (cond
      ;; if left is a constant expression evaluating to 2
      ;; then
      ((and (constantp left) (eql 2 (eval left)))
       (let ((temp ,right))
         `(+ ,temp ,temp)))
      ((and (constantp right) (eql 2 (eval right)))
       (let ((temp ,left))
         `(+ ,temp ,temp)))
      ;; fallback on the original form
      (t form))))

> [There were a bazillion extensible languages in the 1970s, many quite
> sophisticated.  They all disappeared without a trace, largely because
> the ability to do per-program extensions meant that every program was

John, most languages that were ever designed disappeared without a trace,
extensible or not, without any common reason for that other than that only a
few such things can be popular at one time.

Some successful technologies are accidentally so.

(Speaking of lines dying out, all human beings came from the same mother.  So,
what happened to her sister's children and their descendants?)

> written in a different language, making them all unreadable.

Common Lisp programs are all written in a different macro language, and it all
works just fine. People understand each other's programs; different macro
packages get loaded into the same image and play along, etc.

It's just as easy (or hard) to understand someone's morass of functions,
or macros.  Data flows can be hard to follow, control flows can be
hard to follow, code transformations can be hard to follow. Or easy.

> It turned
> out that semantic extension, a la C++ overloaded operators, is a lot
> more usable than extra syntax.

C++ /is/ extra syntax that was initially a C preprocessor.

C++ is still acquiring more syntax as we speak, like a giant muddy snowball.

Even if you just use functional abstraction, you are still developing
a new language.

Every program has its own language.

Just because you memorized the specification of C doesn't mean you can
sit down and easily understand the FreeBSD kernel.

Furthermore, there is a language which is not written down: in the designer's
head.

If the designer does not have language extension at his disposal, then he will
resort to "macro expanding" the idea he has in his head and writing the
expansion.  Constructs that would be easily recognizeable in the original
language become an undifferentiated mess in the target language.

The expansion may be written in a language you know, but it would be easier and
more productive for you to learn the original language and the original
concept.

[toc] | [prev] | [next] | [standalone]

#355

> [There were a bazillion extensible languages in the 1970s, many quite
> sophisticated.  They all disappeared without a trace, largely because
> the ability to do per-program extensions meant that every program was
> written in a different language, making them all unreadable.  It turned
> out that semantic extension, a la C++ overloaded operators, is a lot
> more usable than extra syntax.  I assume you're familiar with EL/1,
> IMP-72, et al, so why is this any better? -John]

The first truly extensible language was Lisp, and it never
disappeared. It evolved, largely thanks to its extensible nature,
becoming the first language to get objects in a normalized way, for
example. This is the only language where normalization simply recorded
accepted extensions that had already been validated by its user base.
Contrast this with how C++ evolved by committee, adding features that
no one had really tried before (export templates anyone?) just because
it's so darn difficult to extend the compiler in the first place.

You expressed an opinion about why the languages you refer to failed,
which you do not substantiate. My personal opinion is that they
disappeared because they were too big and complex (for the time),
without bringing much value in return (at the time). That was back
when dedicating 50K to a compiler was considered luxury enough to be
mentioned in the manual for IMP-72 (see
http://www.saildart.org/prog/DOC/_11_DOC/.html/000055?54,173450), and
where there wasn't much need for more than one language anyway.

Things have changed. Making the language extensible no longer costs an
arm and a leg. And the vast majority of software projects today need
more than one language, when they do not simply go as far as using
meta-programming or DSLs. Whether you consider GUIs or distributed
programming, you always resort to some kind of meta-tools,
annotations, IDE, code generators, all sorts of things that
demonstrate a need to go beyond the base language.

Do these tools create dialects that are indecipherable to other users?
Well, yes and no.You need to learn about Qt's slots or about Doxygen
annotations, i.e. learn something above and beyond standard C++. But
that's not different from learning about Posix functions, the Java
class libraries or iOS. All these things are rather big "dialects",
you have to learn them before you can be fluent. We just manage.

As for why XL is better than these older languages: this is absolute
value in IMP-72

    <EXP> ::= ABS ( <A> ) ::= "A<0=>-A ELSE A"

Here is the same thing in XL:

    abs A -> if A < 0 then -A else A

This has much less "syntactic noise". In general, the older language
(not just extensible ones) had a rather complicated internal
representation. In XL, it's really simple, on a par with Lisp, but
enabling easy-to-read source code.

[I wrote actual programs in IMP-72.  The compiler was slow, but not
unduly so for the time, and it was in the same ballpark as BLISS-11
which got a lot of use.  (They both cross-compiled on a PDP-10.) We
gave up on it because we didn't want to try to remember which of six
slightly different case statements each program used. -John]

[toc] | [prev] | [next] | [standalone]

#356

"Christophe de Dinechin" <christophe@taodyne.com> wrote in message
>> [There were a bazillion extensible languages in the 1970s, many quite
>> sophisticated.  They all disappeared without a trace, largely because
>> the ability to do per-program extensions meant that every program was
>> written in a different language, making them all unreadable. ...

> [I wrote actual programs in IMP-72.  The compiler was slow, but not
> unduly so for the time, and it was in the same ballpark as BLISS-11
> which got a lot of use.  (They both cross-compiled on a PDP-10.) We
> gave up on it because we didn't want to try to remember which of six
> slightly different case statements each program used. -John]

Extensible languages have to be used with some care I think. Those
features aren't for everyday use.

They should be used to turn a language X into a new language X2. X2
should be properly designed, implemented, and documented. Then
development should be halted.

(And there should either have been one kind of case statement, or all
six case statements should have been distinct.)

For this kind of purpose, an extensible language might work well.

However, if the design of X2 isn't going to change, you might as well
just write a compiler directly for X2; it's not necessary to make
available, to the programmer of X2, all those untidy language-building
features (for an example, see C++).

--
Bartc

[toc] | [prev] | [next] | [standalone]

#360

On Nov 27, 12:19 am, "BartC" <b...@freeuk.com> wrote:
> Extensible languages have to be used with some care I think. Those
> features aren't for everyday use.

Apply your reasoning to libraries, and you'll see its limits. "It's
not necessary to make available to the Unix programmer all those
untidy function-building features". I heard that exact reasoning from
Basic gurus hearing about Pascal: "User-defined procedures? That can't
be for everyday use. Who would need to add their own keywords? Leave
that to Basic designers."

In reality, the majority of notations, vocabulary, idioms are private,
even in real life. The point of a programming language is to provide
easy to use notations for your ideas. You can only benefit from the
notations suiting the ideas better. The objections that making a
language extensible necessarily makes it unreadable or fragile flies
in the face of our experience with other forms of abstractions, and
even of the limited experience we have with those few languages that
are actually extensible, such as most Lisp derivatives.

[Well, you know, given a choice between my personal experience that
languages with extensible syntax led to unreadable programs, and
hand-waving that this time is somehow different or I didn't see what I
saw, I know what I'd choose.  You're welcome to build any language you
want, but you have a rather steep hill to climb to persuade people
that your language doesn't have all the same problems that previous
failures had. -John]

[toc] | [prev] | [next] | [standalone]

#362

"Christophe de Dinechin" <christophe@taodyne.com> wrote in message
> On Nov 27, 12:19 am, "BartC" <b...@freeuk.com> wrote:
>> Extensible languages have to be used with some care I think. Those
>> features aren't for everyday use.
>
> Apply your reasoning to libraries, and you'll see its limits. "It's
> not necessary to make available to the Unix programmer all those
> untidy function-building features".

Actually, heavy use of a library, especially a proprietary one, has
similar issues.

But at least it is usually obvious what is and what isn't a function
call; the name of the function should give a clue as to what it does,
and sometimes the module where it lives is provided also, useful extra
information.

And everyone understands about functions and knows the simple syntax
involved. Looking at sequence of unknown functions for the first time
won't faze anyone; but how about a long sequence of newly-invented
statements!

When extensibility is applied to syntax itself (for example), then
what will it look like? How will someone even know this is a new
syntax? Will there be keywords involved so that it can be looked up
somewhere, or is it just symbols? Can existing keywords be overloaded?
Can the same syntax mean something different depending on where it's
used?

Creating new functions is a bit like using jargon in English; you just
look the words up in a glossary. But new syntax is more like new
grammar.

The other kind of extensibility I know about is operator overloading,
where the problems are well-known; the expression A+B could
conceivably mean anything, instead of being restricted to a small set
of predefined types.

> I heard that exact reasoning from Basic gurus hearing about
> Pascal: "User-defined procedures? That can't be for everyday use.
> Who would need to add their own keywords? Leave that to Basic
> designers."

I never heard that, but the Basic guys were wrong. They would know
about subroutines, and could not really object to giving them proper
identifiers and having named parameters. And would know there could
potentially be thousands of the things in a large program.

But surely there is an upper limit to how many kinds of statements are
viable in a single language?

> notations suiting the ideas better. The objections that making a
> language extensible necessarily makes it unreadable or fragile flies
> in the face of our experience with other forms of abstractions, and
> even of the limited experience we have with those few languages that
> are actually extensible, such as most Lisp derivatives.

Well, I find Lisp unreadable anyway, without even knowing whether it is
over-using extensibility or not.

A self-extensible language sounds like a good idea and might well
work. I admit I've never used one (although I did play with designing
one once, then gave up), and have no idea what is and isn't possible;
could you create a language that has C syntax for example, then add in
a few Cobol-like statements or APL expressions? Or is the syntax it's
capable of rather more limited?

--
Bartc

[toc] | [prev] | [next] | [standalone]

#369

Pre-scriptum: It may be better to move this dicussion to xlr-talk.


On Nov 27, 11:24 pm, "BartC" <b...@freeuk.com> wrote:
> But at least it is usually obvious what is and what isn't a function
> call; the name of the function should give a clue as to what it does,
> and sometimes the module where it lives is provided also, useful extra
> information.

Right. And similarly, the form of a notation in an extensible language
should give a clue at to what it does. Here is an actual use of XL,
I hope that you understand what it does without looking at the
documentation:

    slide "A good talk needs fancy titles!",
        * "This is my first point"
        * "This is my second point"

Here is another language extension from the built-in library:

    if X < 3 then writeln "X is small" else writeln "X is big"

It's actually defined that way in XLR (in a pre-loaded file):

    if true then X else Y -> X
    if false then X else Y -> Y


Now, you are right that function calls are easily identified in
languages such as C. Similarly, in XL, tree rewrites are easily
identified, since everything is a tree rewrite but for a few
exceptions. The only things the compiler treats specially are:

1) Sequence of instructions, i.e. infix semi-colon or new-line, e.g.

    write "Hello"; write "World"
    writeln "."

2) The rewrite and "don't rewrite" operators, -> and data. "Foo->Bar"
means "rewrite Foo as Bar",

    0! -> 1
    N! -> N * (N-1)!

 "data X" means "don't rewrite something that looks like X"

    // This means that evaluation stops at commas,
    /// i.e. 1,3,4+5 will evaluate as 1,3,9
    data X,Y

3) Type declarations, e.g "X:integer". This means "X has the shape of
an
integer". I'm still struggling with a decent notation for "X has the
shape of an if-then-else", ideas are welcome.

4) Terminating, machine-code-generating notations. Currently, there
are three.

  4a) "opcode" for machine-level instructions

    (X:integer+Y:integer):integer -> opcode Add
    (X:real+Y:real):real -> opcode FAdd

  4b) "C" for declarations of C bindings

    allocate_bytes N:integer -> C malloc

  4c) a special "extern" notation to directly import C declarations.
       (actually lowered to a C declaration internally).

    extern double sin(double);

  These three notations only work when running the compiler at -O3 for
  obscure reasons, don't ask ;-)

5) Blocks can be eliminated in many contexts, e.g. "(1+X)" can
evaluate as "1+X" except in a few special corner cases.


Anything else is a tree rewrite. When you write:

   if X < 3 then writeln "Small"

the following happens:

1) The source text is parsed in a standard parse tree format that
consists of 8 node types (integer, real, text, name/symbol, infix,
prefix, postfix, block), where separate lines are really an infix \n
and indentation is really a block with special open/close delimiters.

2) Forms are looked up based on their shape alone. It finds "if true
then X -> X" and tries to match it. Matching requires testing if "X<3"
can
match "true", hence requires evaluation of X<3. So we recursively
use the same principle to evaluate X<3.

3) Assuming X evaluates to an integer, for "X<3", we end up with an
"opcode", so we generate machine code for a less-than operator that
returns the trees "true" or "false" (at least logically, internally it
may be a machine binary flag at -O3).


> And everyone understands about functions and knows the simple syntax
> involved. Looking at sequence of unknown functions for the first time
> won't faze anyone; but how about a long sequence of newly-invented
> statements!

In my opinion, this really depends on the notations you
invent. Looking at a sequence of functions can faze anybody if the
names are poorly chosen or don't match your expertise.


> When extensibility is applied to syntax itself (for example), then
> what will it look like?

XL does not encourage syntax changes. Most notations should
fit within the standard syntax framework (i.e. the 8 node types
described above). You can change the syntax (see "extern" above) but
apart from connecting to a different language, I have yet to find a
case where it's required.

With the base syntax, you can easily add operators and describe their
precedence. But changing precedence is rarely useful, and like other
syntax changes, it's actively discouraged. For example, there's no
sane
way to make A+B*C parse differently than A added to (B times C). But
naturally, nothing prevents you from declaring:

    A+B*C -> A-B/C

just like nothing prevents you from writing stupid C code like the
following (and live with the consequences):

    double cos(int x) { return sin(x); }

It then becomes a quality-of-implementation issue to detect such
redefinitons of key staples of the language. A good C compiler may
warn about the declaration above. A guess a future good XL compiler
could detect specific redefinitions as probably harfmul.


> How will someone even know this is a new syntax?

Well, there is a way to change the syntax tables, which looks like
this:

  syntax
      POSTFIX 300 oranges apples

Here is how you can then used the added potfix "oranges" and "apples":

  N apples -> 100 * N
  N oranges -> 25 * N
  if 25 apples > 30 oranges then
      writeln "Good"
  else
      writeln "Uh oh?"

> Will there be keywords involved so that it can be looked up
> somewhere, or is it just symbols?

For XL, except for the cases listed above, there aren't any keywords.


> Can existing keywords be overloaded?

If you mean: can I write something that changes the meaning of "A+B"
or "if A then B", then the answer is yes. Actually, there's nothing
special about these forms, they go through the same rewrite process as
anything else.


> Can the same syntax mean something different depending on where it's used?

If we understand "syntax" by extension of what is done in hard-coded
languages such as C, then syntax can change all the time in an XL
source
file: you can change the meaning of "if A then B" locally just like
you can create function overloads in C++.

But the XL view is that the these are really semantics changes, not
syntax changes. The actual syntax is fixed, and you are supposed to
fit in the standard 8-nodes, to produce a standard 8-node syntax
tree. Compilation happens solely based on the shape of these standard
parse trees.


> Creating new functions is a bit like using jargon in English; you just
> look the words up in a glossary. But new syntax is more like new
> grammar.

If I google something or verb a noun or noun a verb, I change rulz
all the time, mon ami, and you _often_ s33 a # of valid rsns for
d-o-i-n-g so even in p.o. English :-) ^^ LOL.

The human brain is flexible.


> The other kind of extensibility I know about is operator overloading,
> where the problems are well-known; the expression A+B could
> conceivably mean anything, instead of being restricted to a small set
> of predefined types.

This argument was made a long time ago by Bertrand Meyer in a very
articulate article, see
http://se.ethz.ch/~meyer/publications/joop/overloading.pdf.

While the article is very well written, I totally disagree with Meyer
on this (even though I respect him a lot as a language designer). Why?
Because I can't even think of using a programming language where I
need to write "1 + 2" for integer addition, and something different
like "1.3 +. 4.2" for floating-point addition. There are languages
like this (Ocaml for this specific case). I think this is a real
nuisance.

One guiding principle in concept programming is that the notation
should match the "conceptual" notation. I call "syntactic noise" what
happens when you stray from that ideal. If you are used to saying "I
add two numbers" and "I add a chair to this room", then your compiler
should be able to deal with this as well and accept the word "add" in
both cases. So having to write "+." instead of "+" just because the
arguments are real numbers is syntactic noise.


> I never heard that, but the Basic guys were wrong. They would know
> about subroutines, and could not really object to giving them proper
> identifiers and having named parameters. And would know there could
> potentially be thousands of the things in a large program.

Yes, they were wrong, but that seems obvious only with the experience
gained since then with languages supporting arbitrary named
procedures and functions.

The arguments at the time were not stupid at all, and made by smart
people, just like the arguments made here to suggest that extensible
languages are a bad idea. For example, linkers at the time often
limited symbol sizes to 6 or 8 characters, so it was reasonable to
constrain the use of that precious and limited resource.

As for Basic programs with thousands of things in them, that was
somewhat infrequent ;-)


> But surely there is an upper limit to how many kinds of statements are
> viable in a single language?

I don't think so. On the contrary, I believe there is value in making
DSLs small, localized, surgical.

Here is a recent example for a presentation written in XL, where I
added a notation that says: "between X and Y on that movie, do this"
as follows:

   [Begin:real..End:real] Body ->
       if movie_time CurrentMovie in Begin..End then
          Body

I then use the notation as follows:

   [0.5 .. 2.5]
       text_caption "We see a giraffe here"

   [2.6 .. 5.2]
       show_3D_object "RotatingLogo.obj"

I personally find it way more readable than "expanding the macro" as
someone else on the list described it. And the notation is defined
just above its use, so it's not unreadable to someone not familiar
with the notation either.


> Well, I find Lisp unreadable anyway, without even knowing whether it is
> over-using extensibility or not.

So do I. But it's still a really good language. That's why XL borrowed
many ideas from Lisp, including having an homo-iconic parse tree with
a very simple structure.


> A self-extensible language sounds like a good idea and might well
> work. I admit I've never used one (although I did play with designing
> one once, then gave up), and have no idea what is and isn't possible;
> could you create a language that has C syntax for example, then add in
> a few Cobol-like statements or APL expressions? Or is the syntax it's
> capable of rather more limited?

You could, but the intent is the other way round: can you make it so
that features that are natural in another language fit in a common,
shared notational framework.

For XL, I started with features that are common to all languages such
as integer addition, if-then-else, and so on. If you can't do that,
the approach is essentially flawed. XL now passes that test to a large
extent in both its functional and imperative variant.

But of course the objective is to go beyond the basics, i.e. to be
able to import the best features of other languages in a way that
blends in. For example, Ada has a rendez-vous based tasking. Can I
blend that in XL? I don't really want the precise Ada syntax, just a
way to express the Ada concept with a clear and concise notation. Or
regular expressions. Or APL array processing. Or Erlang-style message
passing. Or fixed-point combinators. Or user-controlled garbage
collection
(ideally in a form that also applies to files or network sockets).

And quite frankly, XL fails most of these tests in its current
incarnations. This is why I'm asking for help.

[toc] | [prev] | [next] | [standalone]

#372

On 29 marras, 00:12, Christophe de Dinechin <christo...@taodyne.com>
wrote:
>
> > The other kind of extensibility I know about is operator overloading,
> > where the problems are well-known; the expression A+B could
> > conceivably mean anything, instead of being restricted to a small set
> > of predefined types.
>
> This argument was made a long time ago byBertrand Meyerin a very
> articulate article, see
> http://se.ethz.ch/~meyer/publications/joop/overloading.pdf.
>
> While the article is very well written, I totally disagree with Meyer
> on this (even though I respect him a lot as a language designer). Why?
> Because I can't even think of using a programming language where I
> need to write "1 + 2" for integer addition, and something different
> like "1.3 +. 4.2" for floating-point addition.

This is not the case with Eiffel. The addition operator for both
integer and floating is "+". This is not a case of operator
overloading in pure object oriented setting of Eiffel, since the
operator appears in two different classes: integer and floating. An
example of overloading would be "+" appearing twice with different
signatures in the integer class.

br Jussi

[toc] | [prev] | [next] | [standalone]

#375

On 2011-11-30, ardjussi <jussi.santti@ard.fi> wrote:
> On 29 marras, 00:12, Christophe de Dinechin <christo...@taodyne.com>
> wrote:
>>
>> > The other kind of extensibility I know about is operator overloading,
>> > where the problems are well-known; the expression A+B could
>> > conceivably mean anything, instead of being restricted to a small set
>> > of predefined types.
>>
>> This argument was made a long time ago by Bertrand Meyer in a very
>> articulate article, see
>> http://se.ethz.ch/~meyer/publications/joop/overloading.pdf.
>>
>> While the article is very well written, I totally disagree with Meyer
>> on this (even though I respect him a lot as a language designer). Why?
>> Because I can't even think of using a programming language where I
>> need to write "1 + 2" for integer addition, and something different
>> like "1.3 +. 4.2" for floating-point addition.
>
> This is not the case with Eiffel. The addition operator for both
> integer and floating is "+". This is not a case of operator
> overloading in pure object oriented setting of Eiffel, since the
> operator appears in two different classes: integer and floating. An
> example of overloading would be "+" appearing twice with different
> signatures in the integer class.

Overloading is any situation in which a symbol has more than one
meaning, resolved by context, such as the class of some argument. It
is "loaded up" with two or more meanings, hence "overloaded".

"Appearing in the class" is a very language-specific notion.

In fact when we invoke some operation "on" an object: x.foo(y), obj is
an argument to the function foo: this is just a syntactic sugar for
foo(x, y).  Except that in many Blub-type languages, x is given a
special treatment: x's dynamic type gets taken into account to
determine the call, but only the static type of y.

Just because the dynamic type of x is used, and the static type of y
does not mean that foo is overloaded only with respect to argument y.

Let us not confuse "overloading" as it is used Eiffel or C++ with, well,
computer science.

Both the static and dynamic types supply context to resolve the meaning
of the call foo.

[toc] | [prev] | [next] | [standalone]

#379 — Re: overloading, was Looking for volunteers for XL

Kaz Kylheku <kaz@kylheku.com> wrote:

(snip)
> Overloading is any situation in which a symbol has more than one
> meaning, resolved by context, such as the class of some argument. It
> is "loaded up" with two or more meanings, hence "overloaded".

While this is true, I think people tend not to think of it
as overloading when the operations are similar, such as fixed
point and floating point addition.

There have been complaints about Java, which claims not to have
operator overloading, and then uses the + operator for String
concatenation.

Lots of fun is you do things like:

int i=3;
System.out.println(i+i+"="+i+i);

-- glen

[toc] | [prev] | [next] | [standalone]

#402

On Nov 27 2011, 11:24 pm, "BartC" <b...@freeuk.com> wrote:
> A self-extensible language sounds like a good idea and might well
> work. I admit I've never used one ...

Really?
AFAIK you tried Seed7 a little bit (Thank you four your
fannkuch-redux benchmark program). So you tied an extensible
language without noticing that it is extensible. :-)

> (although I did play with designing
> one once, then gave up),

The trick is: You need to restrict the possibilities somehow to
"reasonable" features. I know that this depends on the point of
view...

> and have no idea what is and isn't possible;
> could you create a language that has C syntax for example, then add in
> a few Cobol-like statements or APL expressions?

Funny, such things were the original idea, which lead to Seed7.
Later I found that some restrictions really make sense. BTW:
Seed7 could do this, when APL operators are restricted to ASCII. :-)
The braces of C would conflict with existing braces in the
predefined Seed7 language (Seed7 uses them to define set literals).
So some of the existing Seed7 definitions need to be omitted, which
I did for the following example. The example defines a
C if-statement:

$ include "seed7x_05.s7i";

# Define the syntax of a simple C if-statment:
$ syntax expr: .if.().{.().}      is  -> 25;

# Define the semantic of a simple C if-statement:
const proc: if (in boolean: cond) {
               (in proc: statement)
            }                     is func
              begin
                if cond then
                  statement;
                end if;
              end func;

const proc: main is func
  begin
    # Use the C if-statement:
    if (TRUE) {
      writeln("okay");
    }
  end func;

I left out COBOL and APL, but you should see the principle.
BTW: I don't think that such a brutal mix of language features
(C, COBOL, APL) makes sense.

> Or is the syntax it's
> capable of rather more limited?

Seed7 is not overengineered, and other extensible languages should
also avoid this pitfall.


Greetings Thomas Mertes

--
Seed7 Homepage:  http://seed7.sourceforge.net
Seed7 - The extensible programming language: User defined statements
and operators, abstract data types, templates without special
syntax, OO with interfaces and multiple dispatch, statically typed,
interpreted or compiled, portable, runs under linux/unix/windows.

[toc] | [prev] | [next] | [standalone]

#363

On 2011-11-26, BartC <bc@freeuk.com> wrote:
> "Christophe de Dinechin" <christophe@taodyne.com> wrote in message
>>> [There were a bazillion extensible languages in the 1970s, many quite
>>> sophisticated.  They all disappeared without a trace, largely because
>>> the ability to do per-program extensions meant that every program was
>>> written in a different language, making them all unreadable. ...
>
>> [I wrote actual programs in IMP-72.  The compiler was slow, but not
>> unduly so for the time, and it was in the same ballpark as BLISS-11
>> which got a lot of use.  (They both cross-compiled on a PDP-10.) We
>> gave up on it because we didn't want to try to remember which of six
>> slightly different case statements each program used. -John]
>
> Extensible languages have to be used with some care I think. Those
> features aren't for everyday use.

Actually, perhaps surprisingly, language extensibility features are for
everyday use.

> They should be used to turn a language X into a new language X2. X2
> should be properly designed, implemented, and documented. Then
> development should be halted.

This is worth doing for language extensions that are significant, and
of interest to a wider community of people. But it's a time-consuming
process.

Extensibility in the language allows such a thing to be conducted as a
project which regularly releases code (rather than just paper).

It also allows some fraction of any application to consist of some
extensions to give it a little domain-specific language or whatever.

> However, if the design of X2 isn't going to change, you might as well
> just write a compiler directly for X2; it's not necessary to make
> available, to the programmer of X2, all those untidy language-building
> features (for an example, see C++).

The problem with this idea is that X2 is not a completely new
language, but X with some extensions.  Those extensions can be used in
parallel with other extensions to X.

You're trying to fit extensible languages into the traditional model,
in which a lone guru (or small group of such) working atop a mountain
carves a programming language onto stone tablets, which then descend
down to the masses.

Under an extensible language culture, the lone guru working in
isolation produces not a new language, but some new extension. These
can be released as code for people to try. Then when the bug reports
pour in and it's all hammered out, a formal spec can be written.  The
guru deosn't get to ask everyone to ditch their language, only to add
something to it.

[Are you aware of anyone actually doing this?  I agree that you might
expect extensible languages to be handy design testbeds, but somehow
other than in the Lisp community, it didn't work out that way. -John]

[toc] | [prev] | [next] | [standalone]

#365

Somewhere on Shadow Earth, at Mon, Nov 28, 2011 at 04:45:11AM +0000, Kaz Kylheku wrote:
> [Are you aware of anyone actually doing this?  I agree that you might
> expect extensible languages to be handy design testbeds, but somehow
> other than in the Lisp community, it didn't work out that way. -John]

Well, John, I think the Forth community provide another counter-example.
Granted, I am not aware of any projects of hundreds or thousands of
Forth devs, but some rather significant projects have been done in Forth
by small teams. And lest folks think Forth as a philosophy is dead, you
might want to see http://factorcode.org/ where the language is evolving
in some very interesting directions.
--
Timothy Knox <mailto:tdk@thelbane.com>

Never trust a language where its users won't tell you that it sucks.
    -- Peter Corlett, on We Hates Software (about Python)
[Forth is swell, but you can't extend its syntax the way you can in
languages with BNF-based parsers and rewrite rules. -John]

[toc] | [prev] | [next] | [standalone]

#378

On Nov 28, 6:50 am, Timothy Knox <t...@thelbane.com> wrote:
> Somewhere on Shadow Earth, at Mon, Nov 28, 2011 at 04:45:11AM +0000, Kaz
Kylheku wrote:
>
> > [Are you aware of anyone actually doing this?  I agree that you might
> > expect extensible languages to be handy design testbeds, but somehow
> > other than in the Lisp community, it didn't work out that way. -John]
>
> Well, John, I think the Forth community provide another counter-example.
> Granted, I am not aware of any projects of hundreds or thousands of
> Forth devs, but some rather significant projects have been done in Forth
> by small teams. And lest folks think Forth as a philosophy is dead, you
> might want to seehttp://factorcode.org/where the language is evolving
> in some very interesting directions.
> --
> Timothy Knox <mailto:t...@thelbane.com>
>
> Never trust a language where its users won't tell you that it sucks.
>     -- Peter Corlett, on We Hates Software (about Python)
> [Forth is swell, but you can't extend its syntax the way you can in
> languages with BNF-based parsers and rewrite rules. -John]

To John's observation; perhaps its lack of syntax makes it seem that
way, but Forth programmers are dab hands at extending Forth. Want new
control structures beyond the basics? Not a problem; here's a CASE
ENDCASE constructed from Forth primitives, since the programmer has
access to the compiler; (comments are in ( ) or preceded by \ );

: case  ( -- 0 )   0 ; immediate
: of    ( C: #of -- orig #of+1 )
        ( x -- )
        1+ >r
        postpone over postpone =
        postpone if   postpone drop
        r> ; immediate
: endof ( C: orig1 #of -- orig2 #of )
        >r postpone else r> ; immediate
: otherwise ; immediate
: endcase ( c: orig1..orign #of -- )
        postpone drop
        0 ?do postpone then loop ; immediate

Now we can...

: testcase ( n -- )
  case 1 of ." one" endof
       2 of ." two" endof
       otherwise ." a lot"
  endcase ;
[Hey, I said forth is swell.  But there isn't a real parser, and if you get
your syntax slightly wrong, rather than parser errors, you get exciting
stack explosions. -John]

[toc] | [prev] | [next] | [standalone]

#366

"Kaz Kylheku" <kaz@kylheku.com> wrote in message
> On 2011-11-26, BartC <bc@freeuk.com> wrote:

>> However, if the design of X2 isn't going to change, you might as well
>> just write a compiler directly for X2; it's not necessary to make
>> available, to the programmer of X2, all those untidy language-building
>> features (for an example, see C++).

> You're trying to fit extensible languages into the traditional model,
> in which a lone guru (or small group of such) working atop a mountain
> carves a programming language onto stone tablets, which then descend
> down to the masses.

You've put that well, that's exactly what I think!

Take the well-known language C, which has a crude mechanism to extend
it in the form of its pre-processing macro language.

Most enhancements you might want to make to the language, can be
achieved by some clunky, ugly macro. But the real problem is that your
code now consists of a private, ad-hoc collection of macros which
no-one else understands. You can't upload a simple piece of code (for
example a problem-solving algorithm) without dragging this macro
library with it.

And even then, someone else has to integrate this code with his own,
where the same language issue has been solved in a slightly different
way, or in a way that clashes.

Wouldn't it be much better standardising these enhancements so everyone
writes them the same way?

(Of course macros can work well for application-related tasks; I'm talking
about language enhancements.)

> Under an extensible language culture, the lone guru working in
> isolation produces not a new language, but some new extension. These
> can be released as code for people to try. Then when the bug reports
> pour in and it's all hammered out, a formal spec can be written.  The
> guru deosn't get to ask everyone to ditch their language, only to add
> something to it.

That's fine. There's just one guru; only he needs the tools to create the
extensions. The problem is when everyone can potentially be a guru because
every has the tools (and a much bigger, more confusing toolbox with lots of
intriguing-looking tools for the programmer to express himself with!).

--
Bartc
[Anyone ever look at the code for the Bourne shell, written in faux
Algol-68 using C preprocessor macros. Yow. -John]

[toc] | [prev] | [next] | [standalone]

#371

BartC <bc@freeuk.com> wrote:

(snip)
> Take the well-known language C, which has a crude mechanism to extend
> it in the form of its pre-processing macro language.

Now consider the less well known language PL/I, with a much
fancier preprocessor.  The PL/I preprocessor allows preprocessor
procedures, and procedure calls.


(snip)

> [Anyone ever look at the code for the Bourne shell, written in faux
> Algol-68 using C preprocessor macros. Yow. -John]

You mean like:

#define BEGIN {
#define END }

I have seen faux-Pascal before, but not faux-Algol.

-- glen
[The faux Algol-68 was pretty amazing.  For that matter, so was the
PL/I preprocessor, which let you write entire programs that run at
compile time. -John]

[toc] | [prev] | [next] | [standalone]

#382 — Re: macros, Looking for volunteers for XL

On Tue, 29 Nov 2011 07:37:49 +0000 (UTC), His Excellence, The
Moderator wrote:

[snip]

>[The faux Algol-68 was pretty amazing.  For that matter, so was the
>PL/I preprocessor, which let you write entire programs that run at
>compile time. -John]

    ISTR that there were macro assemblers that did the same.  I recall
an example of Towers of Hanoi being solved that way.

Sincerely,

Gene Wirchenko
[The assembler for IBM mainframes still does. -John]

[toc] | [prev] | [next] | [standalone]

#384 — Re: macros, Looking for volunteers for XL

Gene Wirchenko <genew@ocis.net> wrote:
> On Tue, 29 Nov 2011 07:37:49 +0000 (UTC), His Excellence, The
> Moderator wrote:

>>[The faux Algol-68 was pretty amazing.  For that matter, so was the
>>PL/I preprocessor, which let you write entire programs that run at
>>compile time. -John]

>    ISTR that there were macro assemblers that did the same.  I recall
> an example of Towers of Hanoi being solved that way.

> [The assembler for IBM mainframes still does. -John]

I don't know if they still do, but for S/360 and S/370 the
assembler was used to generate programs, JCL and all, for sysgen.

The traditional method was to punch on real cards, and then put those
cards in a real card reader.  Now they get punched on virtual cards
and go into a virtual card reader.  (At least for emulated hosts.)

I don't know if this is still used for z/OS, though.

And still not quite as powerful as the PL/I preprocessor.

-- glen

[toc] | [prev] | [next] | [standalone]

#374

On 2011-11-28, BartC <bc@freeuk.com> wrote:
> "Kaz Kylheku" <kaz@kylheku.com> wrote in message
>> On 2011-11-26, BartC <bc@freeuk.com> wrote:
>
>>> However, if the design of X2 isn't going to change, you might as well
>>> just write a compiler directly for X2; it's not necessary to make
>>> available, to the programmer of X2, all those untidy language-building
>>> features (for an example, see C++).
>
>> You're trying to fit extensible languages into the traditional model,
>> in which a lone guru (or small group of such) working atop a mountain
>> carves a programming language onto stone tablets, which then descend
>> down to the masses.
>
> You've put that well, that's exactly what I think!
>
> Take the well-known language C, which has a crude mechanism to extend
> it in the form of its pre-processing macro language.
>
> Most enhancements you might want to make to the language, can be
> achieved by some clunky, ugly macro. But the real problem is that your
> code now consists of a private, ad-hoc collection of macros which
> no-one else understands.

That isn't the problem, because collections of macros can be
well-designed, implemented and documented, and even in C there have
been some decent C macros developed, like the <sys/queue.h> stuff in
BSD UNIX.  People other than the original authors understand those and
use them.

No, the only problem is that the C preprocessor is weak.

Using the C preprocessor to argue against macros is a textbook example of a
strawman argument: pick the weakest example of a category as a representative
of the category.

I was struggling with C macros just today. In my codebase I have an
a macro or2(x,y) which is supposed to return X and not evaluate Y,
or else if X is the value nil, then return Y. Problem is I wrote it like
this:

  #define or2(x,y) ((x) ? (x) : (y))

I made a mental note not to use this where the double evaluation of x would
matter. But then of course months later I forgot about the mental note and
misused it, creating a bug. So I set about fixing the macro, but quickly
remembered that there isn't a way to do it. You can't do this simple thing in
standard C, period.

So I came up with this kludge:

  {
    use_or2;

    /*...*/

    /* ... */ foo = or2(xyzzy, flop()) /*...*/
  }

In a lexical scope where you want to use or2, you have to have this
magic incantation "use_or2;".  This declares the hidden variable needed
to do the job, without explicitly revealing what it is.

Not all the problems in the C preprocessor are in the preprocessor.  Never mind
that all it can do is substitute, arguments, paste tokens.  produce strings,
and suck in include files.  One big problem is that the target language for
macro expansions (C) sucks.  For one thing, any syntactic unit that contains
declarations cannot be an expression that returns a value. Macros that need to
generate complex code that requires hidden variables automatically have a
difficulty in returning a value in a function-call-like way, because only
expressions do that.   Macros do not work well over languages that don't have a
uniform syntax that corresponds well to the implied abstract syntax trees.
Divisions between statements, expressions and declarations are
anti-extensibility barriers.  Then there are syntactic issues such as that
MAC({a, b}) doesn't recognize {a, b} as unit, but as {a  and b}.  Only
parentheses are processed.  That could have been easily fixed by specifying
that braces, brackets as well as parentheses must properly nest in a macro
argument, and will protect embedded commas.

[toc] | [prev] | [next] | [standalone]

#381 — Re: designing language extensions, was Looking for volunteers for XL

On 2011-12-01, Kaz Kylheku <kaz@kylheku.com> wrote:
>> Take the well-known language C, which has a crude mechanism to extend
>> it in the form of its pre-processing macro language.
>>
>> Most enhancements you might want to make to the language, can be
>> achieved by some clunky, ugly macro. But the real problem is that your
>> code now consists of a private, ad-hoc collection of macros which
>> no-one else understands.
>
> That isn't the problem, because collections of macros can be
> well-designed, implemented and documented,

So can brainf*ck code :-) A faulty language doesn't precluding
investing an huge amount of engineering, testing and
adminstration/documentation to manage it.

But that doesn't meant that all languages should be faulty :-)

> Not all the problems in the C preprocessor are in the preprocessor.  Never mind
> that all it can do is substitute, arguments, paste tokens.  produce strings,
> and suck in include files.

Well, that is what (pure) preprocessors do? :-)

If you start adding more knowledge to the preprocessor it becomes a
part of the compiler and language, and your macros are not true macros
anymore, but inlinable functions.

> One big problem is that the target language for macro expansions (C)
> sucks.  For one thing, any syntactic unit that contains declarations
> cannot be an expression that returns a value.

I would write that up to keeping the preprocessor mostly a text
manipulator rather than a compiler, not a target language defect.

A preprocessor that fully understood what it was parsing could hoist
such declarations out of the block if the language needed it?

> Macros that need to generate complex code that requires hidden variables
> automatically have a difficulty in returning a value in a
> function-call-like way, because only expressions do that.  Macros do not
> work well over languages that don't have a uniform syntax that corresponds
> well to the implied abstract syntax trees.

I think this is a symptom of the above. A simple hierarchical
structure is easier to duplicate between preprocessor and compiler,
but it is the barriers that are are part of the definition (separate
binary, textual substitution) that are the main problems.

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.compilers

csiph-web