Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #55780 > unrolled thread

What's so great about Forth, anyway?

Back to article view | Back to comp.lang.forth

Contents

  What's so great about Forth, anyway? Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-03-14 13:07 -0500
    Re: What's so great about Forth, anyway? Julian Fondren <julian.fondren@gmail.com> - 2017-03-14 11:54 -0700
      Re: What's so great about Forth, anyway? rickman <gnuarm@gmail.com> - 2017-03-14 15:53 -0400
      Re: What's so great about Forth, anyway? Julian Fondren <julian.fondren@gmail.com> - 2017-03-14 19:24 -0700
      Re: What's so great about Forth, anyway? anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-03-15 10:00 +0000
        Re: What's so great about Forth, anyway? rickman <gnuarm@gmail.com> - 2017-03-15 12:57 -0400
          Re: What's so great about Forth, anyway? Julian Fondren <julian.fondren@gmail.com> - 2017-03-15 11:09 -0700
            Re: What's so great about Forth, anyway? Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-03-16 04:05 -0500
          Re: What's so great about Forth, anyway? anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-03-16 09:41 +0000
            Re: What's so great about Forth, anyway? hughaguilar96@gmail.com - 2017-03-16 10:57 -0700
              Re: What's so great about Forth, anyway? rickman <gnuarm@gmail.com> - 2017-03-16 14:41 -0400
        Re: What's so great about Forth, anyway? hughaguilar96@gmail.com - 2017-03-15 11:34 -0700
        Re: What's so great about Forth, anyway? franck.bensusan@gmail.com - 2017-03-22 03:24 -0700
          Re: What's so great about Forth, anyway? "Elizabeth D. Rather" <erather@forth.com> - 2017-03-22 11:24 -1000
            Re: What's so great about Forth, anyway? franck.bensusan@gmail.com - 2017-03-23 14:51 -0700
          Re: What's so great about Forth, anyway? Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-03-23 04:25 -0500
            Re: What's so great about Forth, anyway? franck.bensusan@gmail.com - 2017-03-23 14:58 -0700
              Re: What's so great about Forth, anyway? Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-03-24 03:03 -0500
                Re: What's so great about Forth, anyway? franck.bensusan@gmail.com - 2017-03-24 04:52 -0700
                  Re: What's so great about Forth, anyway? Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-03-24 09:06 -0500
          Re: What's so great about Forth, anyway? Julian Fondren <julian.fondren@gmail.com> - 2017-03-23 07:47 -0700
          Re: What's so great about Forth, anyway? anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-03-25 17:13 +0000
            Re: What's so great about Forth, anyway? franck.bensusan@gmail.com - 2017-03-28 13:48 -0700
          Re: What's so great about Forth, anyway? Brad Eckert <hwfwguy@gmail.com> - 2017-03-28 18:34 -0700
            Re: What's so great about Forth, anyway? Julian Fondren <julian.fondren@gmail.com> - 2017-03-28 18:52 -0700
              Re: What's so great about Forth, anyway? anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-03-29 11:53 +0000
                Re: What's so great about Forth, anyway? Ron Aaron <rambamist@gmail.com> - 2017-03-29 15:09 +0300
                  Re: What's so great about Forth, anyway? anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-03-29 13:58 +0000
                    Re: What's so great about Forth, anyway? Ron Aaron <rambamist@gmail.com> - 2017-03-29 17:24 +0300
                  Re: What's so great about Forth, anyway? The Beez <the.beez.speaks@gmail.com> - 2017-04-04 03:04 -0700
                    Re: What's so great about Forth, anyway? Ron Aaron <rambamist@gmail.com> - 2017-04-04 15:30 +0300
                      Re: What's so great about Forth, anyway? The Beez <the.beez.speaks@gmail.com> - 2017-04-06 05:36 -0700
                    Re: What's so great about Forth, anyway? anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-04 16:35 +0000
                      Re: What's so great about Forth, anyway? hughaguilar96@gmail.com - 2017-04-05 11:46 -0700
            Re: What's so great about Forth, anyway? "Elizabeth D. Rather" <erather@forth.com> - 2017-03-28 16:12 -1000
              Re: What's so great about Forth, anyway? anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-03-29 11:59 +0000
            Re: What's so great about Forth, anyway? anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-03-29 10:24 +0000
            Re: What's so great about Forth, anyway? franck.bensusan@gmail.com - 2017-04-04 04:05 -0700
        Re: What's so great about Forth, anyway? gavino himself <jackvanc3@gmail.com> - 2017-03-22 13:18 -0700
    Re: What's so great about Forth, anyway? NN <november.nihal@gmail.com> - 2017-03-14 12:20 -0700
    Re: What's so great about Forth, anyway? gavino himself <jackvanc3@gmail.com> - 2017-04-02 19:52 -0700
    Nothing's great about Forth "Robert L." <No_spamming@noWhere_7073.org> - 2017-04-04 10:45 +0000
      Re: Nothing's great about Forth Julian Fondren <julian.fondren@gmail.com> - 2017-04-04 08:00 -0700
      Re: Nothing's great about Forth lehs <skydda.barnen@gmail.com> - 2017-04-04 08:16 -0700
      Re: Nothing's great about Forth Mark Wills <markwills1970@gmail.com> - 2017-04-04 09:11 -0700
      Re: Nothing's great about Forth Ilya Tarasov <ilya74.tarasov@gmail.com> - 2017-04-06 06:42 -0700

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

#56184

On Wednesday, March 22, 2017 at 5:24:38 AM UTC-5, franck....@gmail.com wrote:
> A Forth system is an open bar for hackers :
> - Often data mixed with code.
> - Return stack is wide-open, a hacker can jump wherever he wants.
> - No buffer underflow/overflow checks, ...
> - Definitions can be redefined.
> - Defer can't be frozen.

- It's simple and easily understood.

Which translates to "shellcode is very easy to write once you find a
bug that lets you inject it", which is all it means for the return
stack to be so accessible. Once you identify a bug in a popular Forth
program, you don't need as-skilled a person to turn that into an
exploit.

Checking for overflow remains "something" because a buffer overflow is
not just a convenient way to overwrite program memory.  It's also a
bug.

A program that connects to MySQL and constructs SQL queries can also
delete all of the data in your database, if you have a bug
permitting an SQL injection.

Anyway, I wonder why you emphasize the return stack. When first
reading this I thought you were thinking of a program that could be
extended at runtime with malicious Forth extensions. As an attack
vector the dictionary seems to be more obvious and just as accessible
if you're already in a position to run shellcode.

>
> 3) Burden for the Forth writer, not for the Forth user.
> Simplicity too often means simplicity for the Forth writer.
> And, in order to have the most simpler outer interpreter, burden
> is handled by the user.
> - Still a space needed after some words ( " , ...).
> - ' is not used for characters
> - [char], ['], S", S\", <#, #, ">, #S, [COMPILE], ...
> - Multiple name for the same operation : +, F+, D+, ...
>
> Case sensitive interpreter is mandatory, of course.
>

I don't agree with any of this, but it helps to explain why you'd
design another language.

> 4) Memory management.
> Direct and not controlled access to memory is prone to bugs and
> open security holes.
> There should have a higher level of abstraction for memory.
> Not securised direct access to memory is forbidden.
> This should be CORE.

Definitely not talking about Forth any more.

But, this is still a possibility: implement some safe memory access
words and then provide a word that seals away / replaces all unsafe
memory access. Then you can write code in the safe language and put
more scrutiny on any changes to unsafe code.

Likely more scrutiny in practice than people direct towards the unsafe
code that implements their scripting language's safe abstractions. And
probably easier scrutiny to apply since it's Forth on both sides of
the wall.

*shrug*

[toc] | [prev] | [next] | [standalone]

#56286

franck.bensusan@gmail.com writes:
>Le mercredi 15 mars 2017 11:11:01 UTC+1, Anton Ertl a =C3=A9crit=C2=A0:
>
>> Conversely, he may have described what likes and does not like about
>> Common Lisp, and then decided to do something about the "not-likes",
>> and designed Arc.
>>=20
>> And yes, one could do similar things wrt Forth, and its dialects.
>> Typically, people have been more on the "do" side than on the "write"
>> side of this (e.g., Chuck Moore, Hans Bezemer, Slava Pestov, Ron
>> Aaron, Franck Benunsan), with Hugh Aguilar being the exception that
>> proves the rule.
...
>What you are asking (if I understand) is not easy because I don't think one=
>=20
>can be neutral, here.

I did not ask for anything from you, but I am happy that my posting
inspired you to write something:-).  Neutrality would be strange for a
position paper/posting.

>1) Security features.
>Never say "No security to bother on embedded".
>
>A Forth system is an open bar for hackers :=20
>- Often data mixed with code.

Separating them is possible, but does not help security much (see
below).

>- Return stack is wide-open, a hacker can jump wherever he wants.

Actually, unless the attacker has already taken over your system,
that's unlikely to be a problem.  The reason is that, if a Forth
program has a mistake in return-stack handling, it usually already
crashes during development.  So Forth programs tend not to have such
errors.  The return stack is usually separated from overflowable
buffers (exception: addressable local buffers that reside on the
return stack, i.e., VFX applications), so the attacker has a hard time
overwriting it.

But that does not help security much, see below.

>- No buffer underflow/overflow checks, ...

Yes, but at least almost all buffer-writing standard words (exception:
xc!+) supply buffer lengths, so you don't get the strcpy()/strcat()
problems.

>- Definitions can be redefined.

Not a problem unless the attacker already has taken over your system.

>- Defer can't be frozen.

Not a problem unless the attacker already has taken over your system.

The reason why separating code and data, return stack safety and other
things (e.g., W^X) don't help is that it's enough if the attacker can
change an arbitrarily deeply indirect pointer to code, and code is
later executed by following a chain that runs through this pointer:
E.g., through the overflow of a buffer that is followed by, say an
array of object pointers.  Or, say, overwriting the address and data
that is later used in a !; The attacker can use that to change the
current exception frame to point into your buffer, and if he manages
to let a THROW happen, he can perform arbitrary code execution.

In a Forth system there are usually enough code fragments (aka
gadgets) around that form a Turing-complete programming language
(pretty obviously, all the Forth words), and the attacker can use
these to perform arbitray things by executing them in a kind of
threaded-code execution.  That's trivial for actual threaded-code
systems, but it's also possible for native-code systems (the technique
is known as return-oriented programming).

Conversely, if your Forth variant prevents buffer overflows, all of
the above is not an issue; there are still other security issues, but
you certainly don't need to worry about the return stack,
redefinitions, or deferred words.

One of the other security issues is what plagues everything: they call
libraries written in C, and these are often full of holes.

I held a talk at EuroForth 2016 about this:

http://www.complang.tuwien.ac.at/anton/euroforth/ef16/papers/ertl-secure.pdf
https://wiki.forth-ev.de/lib/exe/fetch.php/events:security.mp4

>- ' is not used for characters=20

In Forth-2012

'a' 

gives you the ASCII code for "a".

>8) Make Data stack unique with 1 cell =3D 1 object.=20
>- Strings need 2 cells -> to be changed.
>- Double need 2 cells -> to be removed (with all double words).
>- Characters value should be unicode
>- String should be UTF8 and take 1 cell on the stack.
>- No dedicated stacks (floats, strings, ...).

That's a pretty strong deviation from classical close-to-the-metal
Forth in the direction of higher-level languages like Postscript and
Factor.

The interesting thing about this is that we get just two groups of
dialects: the low-level ones and the high-level ones.  The low-level
ones are fundamentally incompatible with the high-level ones, but
within the low-level ones, incompatibilities are superficial; and
within the high-level ones, incompatibilities may be less superficial
(e.g., different object systems), but not as fundamental as between
the low-level and the high-level ones.

We see this in other languages, too; there are attempts to live on
both sides of the divide, such as C++ and Rust, but in the end they
have to deal with their low-level side in some way; I don't think we
want to go there.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2016: http://www.euroforth.org/ef16/

[toc] | [prev] | [next] | [standalone]

#56430

Le samedi 25 mars 2017 19:17:52 UTC+1, Anton Ertl a écrit :
> franck.bensusan@gmail.com writes:
> >Le mercredi 15 mars 2017 11:11:01 UTC+1, Anton Ertl a =C3=A9crit=C2=A0:
> >
> >> Conversely, he may have described what likes and does not like about
> >> Common Lisp, and then decided to do something about the "not-likes",
> >> and designed Arc.
> >>=20
> >> And yes, one could do similar things wrt Forth, and its dialects.
> >> Typically, people have been more on the "do" side than on the "write"
> >> side of this (e.g., Chuck Moore, Hans Bezemer, Slava Pestov, Ron
> >> Aaron, Franck Benunsan), with Hugh Aguilar being the exception that
> >> proves the rule.
> ...
> >What you are asking (if I understand) is not easy because I don't think one=
> >=20
> >can be neutral, here.
> 
> I did not ask for anything from you, but I am happy that my posting
> inspired you to write something:-).  Neutrality would be strange for a
> position paper/posting.

So I didn't understood... Not a surprise as, as you can see, my
english is far from fluent :)

Ok, let's talk about a Forth system running a standalone Forth
program, but that was not what I was talking about (see below).

 
> >1) Security features.
> >Never say "No security to bother on embedded".
> >
> >A Forth system is an open bar for hackers :=20
> >- Often data mixed with code.
> 
> Separating them is possible, but does not help security much (see
> below).
>

Mixing data and code can lead to create memory pages with
READ+WRITE+EXECUTE protection (on OS that support page protection).

This does not allow a direct attack but it simplifies a lot those 
attacks as this is the ideal place to store and execute code.
(same as the days when the stack memory was executable).

This would break the data execution prevention technics used by OS
and remove need to use technics (like return-oriented programming).


 
> >- Return stack is wide-open, a hacker can jump wherever he wants.
> 
> Actually, unless the attacker has already taken over your system,
> that's unlikely to be a problem.  The reason is that, if a Forth
> program has a mistake in return-stack handling, it usually already
> crashes during development.  So Forth programs tend not to have such
> errors.  The return stack is usually separated from overflowable
> buffers (exception: addressable local buffers that reside on the
> return stack, i.e., VFX applications), so the attacker has a hard time
> overwriting it.
> 
> But that does not help security much, see below.
> 
> >- No buffer underflow/overflow checks, ...
> 
> Yes, but at least almost all buffer-writing standard words (exception:
> xc!+) supply buffer lengths, so you don't get the strcpy()/strcat()
> problems.

Sending lengths is ok but this is not what will prevent attacks.
Many attacks are completed using integers underflow/overflow, 
especially on those lengths and indexes sent as parameter.
This can lead to buffer overflow or memory corruption.

Checking if an integer operation overflows costs and is not
easy to write in a safe/portable way (if not done directly in assembly). 

Also there are some constructions that are inherently prone
to security issues and buffers overflows : 
PAD
TIB, IN>
<#, #, #>, ...
TYPE
...


> >- Definitions can be redefined.
> 
> Not a problem unless the attacker already has taken over your system.
> 
> >- Defer can't be frozen.
> 
> Not a problem unless the attacker already has taken over your system.
> 
> The reason why separating code and data, return stack safety and other
> things (e.g., W^X) don't help is that it's enough if the attacker can
> change an arbitrarily deeply indirect pointer to code, and code is
> later executed by following a chain that runs through this pointer:
> E.g., through the overflow of a buffer that is followed by, say an
> array of object pointers.  Or, say, overwriting the address and data
> that is later used in a !; The attacker can use that to change the
> current exception frame to point into your buffer, and if he manages
> to let a THROW happen, he can perform arbitrary code execution.
> 
> In a Forth system there are usually enough code fragments (aka
> gadgets) around that form a Turing-complete programming language
> (pretty obviously, all the Forth words), and the attacker can use
> these to perform arbitray things by executing them in a kind of
> threaded-code execution.  That's trivial for actual threaded-code
> systems, but it's also possible for native-code systems (the technique
> is known as return-oriented programming).
> 
> Conversely, if your Forth variant prevents buffer overflows, all of
> the above is not an issue; there are still other security issues, but
> you certainly don't need to worry about the return stack,
> redefinitions, or deferred words.


All this was not what I meant: the objective was to be able to
use Forth as a WebAssembly-like format ie being able to : 
1) Load code from an external (potentially unsafe) source.
2) Compile this code on the flow to native code.
3) Execute this code in a secure way.

In this case, you have to worry about direct access to return stack,
and redefinitions and deferred words.


> One of the other security issues is what plagues everything: they call
> libraries written in C, and these are often full of holes.
> 

Yes.

> I held a talk at EuroForth 2016 about this:
> 
> http://www.complang.tuwien.ac.at/anton/euroforth/ef16/papers/ertl-secure.pdf
> https://wiki.forth-ev.de/lib/exe/fetch.php/events:security.mp4
> 
> >- ' is not used for characters=20
> 
> In Forth-2012
> 
> 'a' 
> 
> gives you the ASCII code for "a".
> 
> >8) Make Data stack unique with 1 cell =3D 1 object.=20
> >- Strings need 2 cells -> to be changed.
> >- Double need 2 cells -> to be removed (with all double words).
> >- Characters value should be unicode
> >- String should be UTF8 and take 1 cell on the stack.
> >- No dedicated stacks (floats, strings, ...).
> 
> That's a pretty strong deviation from classical close-to-the-metal
> Forth in the direction of higher-level languages like Postscript and
> Factor.
> 
> The interesting thing about this is that we get just two groups of
> dialects: the low-level ones and the high-level ones.  The low-level
> ones are fundamentally incompatible with the high-level ones, but
> within the low-level ones, incompatibilities are superficial; and
> within the high-level ones, incompatibilities may be less superficial
> (e.g., different object systems), but not as fundamental as between
> the low-level and the high-level ones.
> 
> We see this in other languages, too; there are attempts to live on
> both sides of the divide, such as C++ and Rust, but in the end they
> have to deal with their low-level side in some way; I don't think we
> want to go there.

I'am not very familiar with this, but, for me, "close to the metal" means
"no OS" or "Forth as OS". Are they other criterias ?
Otherwise, I don't see such a strong deviation, at least on
this point. I see nothing more than pointers to structures on the data
stack...

For instance, I have seen an OO extension, FMS (a great OO extension btw)
and, imo, it is possible to implement most of those points on a Forth
system + such an extension and to remain "close to the metal", no ?

In fact, I'm not sure to understand you're classification between low level and
high level languages.

Franck

[toc] | [prev] | [next] | [standalone]

#56436

Hi Franck, I see you've spent some time thinking about Forth and ways a system could be attacked. This could be fun.

On Wednesday, March 22, 2017 at 3:24:38 AM UTC-7, franck....@gmail.com wrote:

> Requirements : 
> 0) Keep what is great.
Okay, keep it simple.

> 1) Security features.
> Never say "No security to bother on embedded".
No, don't let bad code into your playground.

> 
> - Often data mixed with code.
Some do and some don't. Embedded Forths usually don't.

> - Return stack is wide-open, a hacker can jump wherever he wants.
It's possible with some effort to not use addresses as xts, but then you have to remove all dependencies that correlate data location with return address. A translation table in hardware could look up jump addresses, or a VM could do that if running in an off-the-shelf CPU. That's the best way I can think of to limit EXECUTE to specific jumps.

> - No buffer underflow/overflow checks, ...
That should be the MMU's job.

> - Definitions can be redefined.
Not really, they're early-bound. You just end up hiding old definitions which may be desirable. 

> - Defer can't be frozen.
I freeze defered words when they are compiled into CDATA space. The code is a single branch instruction in ROM, which gets patched once by a smart IS. Defers that are changeable are compiled to IDATA space, and IS makes the distinction. 

> 
> Checking for overflow is nothing if the return stack remains open
> (overflow is only a way to update the return stack addresses and take
> control of the program).
> ==> Make the return stack only handled by the compiler
> ==> Remove words like >R, R>, R@, ...
> ==> Values on the return stack are only available through locals.
Removing words like >R, R>, R@ breaks quite a lot of existing code. You can over/underflow either stack pretty easily. Catching that should be hardware's job.

> 
> 2) Packages
> Remove vocabularies and word lists, list order, ...
> Instead, robust packages must be provided.
> This should be CORE.

That seems to be hard to agree upon. MPE Ltd deals with some large code bases. MODULE, END-MODULE and EXPORT seem to work well for them. The search order still needs to be manipulated to make visible the words inside the modules you're interested in. Any ideas?

> 
> 3) Burden for the Forth writer, not for the Forth user.
> Simplicity too often means simplicity for the Forth writer.
> And, in order to have the most simpler outer interpreter, burden
> is handled by the user.
> - Still a space needed after some words ( " , ...).
You're kidding, right? Blank delimited strings...

> - ' is not used for characters 
Some number recognizers accept 'c' as the character c. Would be nice if it were standard.

> - [char], ['], S", S\", <#, #, ">, #S, [COMPILE], ...
Yes, all to get rid of state-smart words. Of course, if your implementation uses separate wordlists for the compile and interpret semantics of a word, I don't think that matters much. BTW, S", S\" are still state smart. Numeric conversion like <#, #, #S are not state smart but they can be stepped on by other instances of numeric conversion.

> - Multiple name for the same operation : +, F+, D+, ...
Polly want a morphism? StrongForth tried that. Now that many Forths are 32-bit, D+ is rarely used. I don't think overloading + with F+ is all that great either. If you wanted to index into an array of floats, FLOATS MYARRAY + F@ F+ would not like that. 

> 
> Case sensitive interpreter is mandatory, of course.

At least optional. Code developed on a case insensitive Forth is going to break all over the place.

> 
> And there is this subject about semantics, immediate, 
> compilation, execution, interpretation, one xt, two xt, ...
> to clarify.
> ==> Define a clear metamodel for Words.
I'm more interested in swimsuit models.

> 
> 4) Memory management.
> Direct and not controlled access to memory is prone to bugs and
> open security holes.
> There should have a higher level of abstraction for memory.
> Not securised direct access to memory is forbidden.
> This should be CORE.
Okay, now you're off the reservation. Maybe memory spaces should be sandboxed. How can there be a wall between implementation and application? Both are written in Forth. Abstraction isn't your friend.

> 
> 5) Blocks (or "quotations")
> This should be CORE.
> NONAME should be removed.
> With use of parent locals (closures or in the scope of the parent).
What's the replacement for ":NONAME blah blah ; IS FOO" ?

> 
> 
> 6) Effective locals
> Locals should be CORE.
> Needed if the return stack is not directly accessible.
> Locals should be fully optimized, stored directly on the
> return stack for efficiency.
> They are the only way to communicate with the return stack.

I'd go a little further and support scoping of locals over a module rather than within a single word. Locals are great in theory. I've implemented them and found that I never used them. Well, how about those apples?

> 
> 
> 7) Effective multi-tasking features.
> Not just mapping OS threads or cooperative multi-threading.
> Implement a modern multi-tasking design : 
> - Isolated tasks.
> - non blocking threads (wait, I/O, TCP, ...).
> - Channels to communicate between tasks.
> - Reenforced immutability, 
> - Tasks with separated data stack and return stack for each.
> - No global mutable values => remove VARIABLE
> - Data stack separated from return stack helps a lot.
> 
> And this should be CORE : interpreter runs into a task and
> waits for keys in a non blocking way.
You have an OS? Multicore is definitely on the horizon. Single core multitasking has been well tested for 40 years. I think a multicore wordset should be considered.

> 
> 8) Make Data stack unique with 1 cell = 1 object. 
> - Strings need 2 cells -> to be changed.
> - Double need 2 cells -> to be removed (with all double words).
> - Characters value should be unicode
> - String should be UTF8 and take 1 cell on the stack.
> - No dedicated stacks (floats, strings, ...).
The migration from single cell counted-string addresses to the (addr length) format was thoroughly researched. But then, that was before UTF-8 became a thing. Now that character count does not equal byte count, maybe it's time to rethink that. But then, the character count is only needed for rendering so probably not.

> 
> Required to have effective multi-tasking features.
> (you can't create/save n stacks by task).
I think it already has that, just not in the standard.

> 
> 
> 9) Same for return stack
> - Only one kind of local.
> - Local value = 1 object (float, string, array, ...). 
MPE's locals syntax I think supports various sizes of local data. It's doable.

> 
> 10) Built-in collections and HOF 
> Minimum is Array (users are not here to implement arrays).
> Dedicated word for array syntax.
> Checks for array access (out of bounds).
> This should be CORE.
> HOF that can take words, quotations, closures as parameters.
Seems like something that could be built on top of today's CORE words. So, why define it? You could have a "Neat Ideas" appendix.

> 
> 11) Efficiency, but not before requirements.
> All this should be done keeping effeciency (and, if possible,
> small memory footprint).
> But first the requirements, then optimize.
> 

[toc] | [prev] | [next] | [standalone]

#56437

On Tuesday, March 28, 2017 at 8:34:29 PM UTC-5, Brad Eckert wrote:
> On Wednesday, March 22, 2017 at 3:24:38 AM UTC-7, franck....@gmail.com wrote:
> > 4) Memory management.
> > Direct and not controlled access to memory is prone to bugs and
> > open security holes.
> > There should have a higher level of abstraction for memory.
> > Not securised direct access to memory is forbidden.
> > This should be CORE.
> Okay, now you're off the reservation. Maybe memory spaces should be sandboxed. How can there be a wall between implementation and application? Both are written in Forth. Abstraction isn't your friend.

Frank didn't emphasize it (except for repeating and defining
'WebAssembly'), and in the context of the thread subject and Oforth
it's easier to miss, but he's talking about a language that can load
and compile arbitrary code without that code behaving maliciously. Or
only worrying as much as people do about code that runs in a browser:
it might record and report on your every exhalation while you're
viewing the webpage its running in, but it won't decrypt and read your
diary.  So pretty far off the reservation to begin with.  At best
such a language could be defined as something easily implemented in
Forth, after one seals off much of the language.

[toc] | [prev] | [next] | [standalone]

#56447

Julian Fondren <julian.fondren@gmail.com> writes:
>he's talking about a language that can load
>and compile arbitrary code without that code behaving maliciously. Or
>only worrying as much as people do about code that runs in a browser:
>it might record and report on your every exhalation while you're
>viewing the webpage its running in, but it won't decrypt and read your
>diary.  So pretty far off the reservation to begin with.  At best
>such a language could be defined as something easily implemented in
>Forth, after one seals off much of the language.

I think that's pretty hard.  Java tried and failed.  JavaScript also
fails, but is so popular that people don't want to admit it.  Running
the application in an appropriately restricted OS-level virtual
machine looks more promising, but also has its pitfalls.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2016: http://www.euroforth.org/ef16/

[toc] | [prev] | [next] | [standalone]

#56449

On 03/29/17 14:53, Anton Ertl wrote:
> Julian Fondren <julian.fondren@gmail.com> writes:
>> he's talking about a language that can load
>> and compile arbitrary code without that code behaving maliciously...

> I think that's pretty hard.  Java tried and failed.  JavaScript also
> fails, but is so popular that people don't want to admit it.  Running
> the application in an appropriately restricted OS-level virtual
> machine looks more promising, but also has its pitfalls.

It depends on what you mean.

8th already restricts (arbitrary) access to memory (there's only one
word which can give access to arbitrary memory, and it's documented as
'dangerous').  You can make that specific word unavailable by just
"forget"ing it.

It's easy to make it impossible for arbitrary code to access any words
outside a designated namespace, using "only".

It's easy to handle arbitrary input if you don't hand it over to "eval"
(for instance).

8th is "strongly typed", and doesn't have "casting".  So you can't make
an end-run around type protections.

And 8th doesn't let you manipulate the return address with >r etc.
Instead, the "r" words simply manipulate an auxiliary stack.

So there are already a lot of mechanisms in place (in 8th, don't know
about OForth) to make it much more likely that your effort to write a
secure application will be successful.

Of course, "it's not Forth" exactly...

[toc] | [prev] | [next] | [standalone]

#56454

Ron Aaron <rambamist@gmail.com> writes:
>
>
>On 03/29/17 14:53, Anton Ertl wrote:
>> Julian Fondren <julian.fondren@gmail.com> writes:
>>> he's talking about a language that can load
>>> and compile arbitrary code without that code behaving maliciously...
>
>> I think that's pretty hard.  Java tried and failed.  JavaScript also
>> fails, but is so popular that people don't want to admit it.  Running
>> the application in an appropriately restricted OS-level virtual
>> machine looks more promising, but also has its pitfalls.
>
>It depends on what you mean.
>
>8th already restricts (arbitrary) access to memory

So does Java.

>It's easy to make it impossible for arbitrary code to access any words
>outside a designated namespace, using "only".

ONLY does not help general Forth (e.g., you can just EXECUTE anything
without needing access through the name), and if you restrict the
words, you have to be very careful to avoid leaving a hole.  And if
you succeed, you do not load arbitrary Forth code, but a heavily
restricted variant.

>It's easy to handle arbitrary input if you don't hand it over to "eval"
>(for instance).

Loading and compiling arbitrary code as Julien Fondren required means
that you have to hand it over to something like EVALIUATE.

Even if you drop that requirement, myriads of vulnerabilities written
in languages that don't have anything like EVALUATE demonstrate that
it is not easy.

>8th is "strongly typed", and doesn't have "casting".  So you can't make
>an end-run around type protections.
>
>And 8th doesn't let you manipulate the return address with >r etc.
>Instead, the "r" words simply manipulate an auxiliary stack.

Likewise, in Java, and yet the Java applet mechanism was so full of
holes that they eventually abandoned it.  Java without the applet
requirement to allow the attacker to provide the code is probably not
worse than anything else, though (e.g., it still suffers from calling
libraries written in C that contain vulnerabilities).

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2016: http://www.euroforth.org/ef16/

[toc] | [prev] | [next] | [standalone]

#56455

On 29/03/2017 16:58, Anton Ertl wrote:
> Ron Aaron <rambamist@gmail.com> writes:

>> 8th already restricts (arbitrary) access to memory
> 
> So does Java.

Yes; I didn't say that 8th was unique in this regard.  If someone can
stomach Java, more power to them.


>> It's easy to make it impossible for arbitrary code to access any words
>> outside a designated namespace, using "only".
> 
> ONLY does not help general Forth (e.g., you can just EXECUTE anything
> without needing access through the name), and if you restrict the
> words, you have to be very careful to avoid leaving a hole.  And if
> you succeed, you do not load arbitrary Forth code, but a heavily
> restricted variant.

OK; but in 8th you can only execute a word, not an arbitrary address.

Arbitrary code therefore needs some way to get a word token to execute,
which is not possible unless some prior code (before 'only') has looked
up the word and made its value available in the only-ed namespace)


> Loading and compiling arbitrary code as Julien Fondren required means
> that you have to hand it over to something like EVALIUATE.

You can do that if you've first invoked 'only'.  The 'eval' word will be
just as restricted as the REPL to whatever namespace was exposed.

> Even if you drop that requirement, myriads of vulnerabilities written
> in languages that don't have anything like EVALUATE demonstrate that
> it is not easy.

There are all kinds of vulnerabilities.  We're not discussing the entire
universe of possibilities, just the ones we've mentioned.

I'll add stack-smashing and buffer-overflow as another two vectors which
are not possible in 8th. (yes, I know; Java also...)

>> 8th is "strongly typed", and doesn't have "casting".  So you can't make
>> an end-run around type protections.
>>
>> And 8th doesn't let you manipulate the return address with >r etc.
>> Instead, the "r" words simply manipulate an auxiliary stack.
> 
> Likewise, in Java, and yet the Java applet mechanism was so full of
> holes that they eventually abandoned it. ...

None of that is relevant to the discussion however. The fact that Java's
implementation of security mechanisms was faulty, does not negate their
utility.

[toc] | [prev] | [next] | [standalone]

#56689

On Wednesday, March 29, 2017 at 2:09:24 PM UTC+2, Ron Aaron wrote:
> 8th already restricts (arbitrary) access to memory (there's only one
> word which can give access to arbitrary memory, and it's documented as
> 'dangerous').  You can make that specific word unavailable by just
> "forget"ing it.
<snip> 
> 8th is "strongly typed", and doesn't have "casting".  So you can't make
> an end-run around type protections.
> 
> And 8th doesn't let you manipulate the return address with >r etc.
> Instead, the "r" words simply manipulate an auxiliary stack.

Always interesting to see how people come up with different solutions. 4tH is a kind of "sandbox" with a Harvard architecture, which effectively completely segments the different datatypes. The bytecodes limit access to these segments, including their ranges. To the program, there is simply no "outside the sandbox" (arguably apart from the files). Even if you can write to a segment, that doesn't mean you can write to all areas in a segment - some areas may be R/O or can only be accessed by certain bytecode instructions (like the stacks).

Go outside any segment and the VM will stop you. Hence I have no problems manipulating the return stack or much need for strong typing. If you do something crazy you WILL be stopped in your track. That was one of the design objectives 20+ years ago and it hasn't changed.

It's always nice to be mentioned (Thank you, Anton :) but I don't consider myself a non-writer. When 4tH was introduced after three years of development some 20 years ago, it was FIERCELY discussed. E.g. https://groups.google.com/forum/#!searchin/comp.lang.forth/4th$20compiler|sort:relevance/comp.lang.forth/XiDmqyZj0QU/d76CHf0vUIQJ

I've also written a LOT on 4tH - apart from its almost 600 pages manual - e.g. https://sourceforge.net/p/forth-4th/wiki/4tH%20philosophy/

Personally, I don't consider 4tH to be "incompatible". Well written 4tH can easily be converted to Forth. I've donated a humble amount of code to c.l.f, FSL and Rosetta code and all that code was originally developed on 4tH. Notorious deviations are:

- +LOOP with a negative subscript;
- DOES> in interpretation mode;
- Lookup tables, CREATE, array declarations;
- DO..REPEAT with multiple WHILE's;
- Counted strings vs. Zero-terminated strings;
- POSTPONE, [, ], LITERAL, etc.

But it is fun to read how Paul Graham talks about macro's (4tH has got an optional preprocessor - https://sourceforge.net/p/forth-4th/wiki/The%204tH%20preprocessor/), optimization (I don't think a good 4tH programmer can write much tighter bytecode by hand than the compiler produces), bytecode (which 4tH) has and a host of libraries (4tH has almost 450 of them, including regular expressions, a tiny database (https://sourceforge.net/p/forth-4th/wiki/A%20database%20of%20sorts/) and associative arrays).

I like this guy already ;-)

Hans Bezemer

[toc] | [prev] | [next] | [standalone]

#56700

On 04/04/2017 13:04, The Beez wrote:
> On Wednesday, March 29, 2017 at 2:09:24 PM UTC+2, Ron Aaron wrote:
>> 8th already restricts (arbitrary) access to memory ...
> 
> Always interesting to see how people come up with different solutions. 4tH is a kind of "sandbox" with a Harvard architecture, which effectively completely segments the different datatypes. The bytecodes limit access to these segments, including their ranges. To the program, there is simply no "outside the sandbox" (arguably apart from the files). Even if you can write to a segment, that doesn't mean you can write to all areas in a segment - some areas may be R/O or can only be accessed by certain bytecode instructions (like the stacks).

That sounds a bit like the Java VM.  What are the similarities/differences?

What was your motivation to go with a VM architecture?

[toc] | [prev] | [next] | [standalone]

#56783

On Tuesday, April 4, 2017 at 2:30:04 PM UTC+2, Ron Aaron wrote:
> > Always interesting to see how people come up with different solutions. 4tH is a kind of "sandbox" with a Harvard architecture, which effectively completely segments the different datatypes. The bytecodes limit access to these segments, including their ranges. To the program, there is simply no "outside the sandbox" (arguably apart from the files). Even if you can write to a segment, that doesn't mean you can write to all areas in a segment - some areas may be R/O or can only be accessed by certain bytecode instructions (like the stacks).
> That sounds a bit like the Java VM.  What are the similarities/differences?
- 4tH does most checking during execution, Java performs *some* checks during loading;
- Both use their own VM addresses for memory and instruction memory;
- 4tH has no "native methods", while Java allows loading external libraries (the reason for that is, 4tH may not crash under ANY circumstance);
- Both allow threading. 4tH closes all I/O and dumps a stackframe before pausing, Java has Program Counter Registers;
- 4tH uses static allocation and hence reserves exactly as much memory as is required by the program. Java uses a (native) heap;
- Both 4tH and Java differentiate between program- and data memory. 4tH has a Harvard architecture with different segments for bytes and words (R/W) vs. cell and string constants (R/O);
- All Java data is initialized. Although created by calloc() all 4tH R/W segments are considered uninitialized;
- 4tH has a different format for bytecode on disk and bytecode in memory. In memory, all 4tH bytecodes have the same length and format (1 operator, 1 operand). On disk, 4tH has a single byte operator and an optional, variable length operand). Java has no fixed bytecode format.
- 4tH is designed to be a scripting language and hence reserves only 106 bytecodes vs. Java's 198 (+54 for future use);
- The specification permits Java Virtual Machine stacks either to be of a fixed size or to dynamically expand and contract as required by the computation. If the Java Virtual Machine stacks are of a fixed size, the size of each Java Virtual Machine stack may be chosen independently when that stack is created. 4tH's stacks are fixed and shared. Data stack grows upwards, return stack growns downward;
- Java has a Run-Time Constant Pool, comparable to a symbol table. 4tH is more comparable to an assembly program: all that information is lost;
- Java uses a "load/store" architecture. 4tH's VM manipulates the stack directly (accessing items by address) or indirectly (push/pop) as it sees fit for a particular purpose.

> What was your motivation to go with a VM architecture?
It's much easier to make a portable compiler that punches out bytecode for a VM than design a zillion backends for a lot of different architectures ;-) (And still retain SOME performance, that is).

Hans Bezemer

[toc] | [prev] | [next] | [standalone]

#56720

The Beez <the.beez.speaks@gmail.com> writes:
>It's always nice to be mentioned (Thank you, Anton :) but I don't consider =
>myself a non-writer.

You are the second one, so I guess I did not express myself clearly.
My statement was actually intended as criticism of those who write up
requirement documents without producing implementations, and as praise
of people who produce implementations.  One might criticize people who
throw out code without documentation, but my statement was not about
that.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2016: http://www.euroforth.org/ef16/

[toc] | [prev] | [next] | [standalone]

#56759

On Tuesday, April 4, 2017 at 9:43:39 AM UTC-7, Anton Ertl wrote:
> My statement was actually intended as criticism of those who write up
> requirement documents without producing implementations, and as praise
> of people who produce implementations.  

ANS-Forth was submitted to ANSI without any compiler being written.

Now Elizabeth Rather has appointed you to the Forth-200x committee, and Forth-200x is mandated to be 100% compatible with ANS-Forth, so you are careful not to criticize her --- instead you criticize me --- hypocrite!

[toc] | [prev] | [next] | [standalone]

#56439

On 3/28/17 3:34 PM, Brad Eckert wrote:
> Hi Franck, I see you've spent some time thinking about Forth and ways a system could be attacked. This could be fun.
>
> On Wednesday, March 22, 2017 at 3:24:38 AM UTC-7, franck....@gmail.com wrote:
>
>> Requirements :
>> 0) Keep what is great.
> Okay, keep it simple.
>
>> 1) Security features.
>> Never say "No security to bother on embedded".
> No, don't let bad code into your playground.
>
>>
>> - Often data mixed with code.
> Some do and some don't. Embedded Forths usually don't.
>
>> - Return stack is wide-open, a hacker can jump wherever he wants.
> It's possible with some effort to not use addresses as xts, but then you have to remove all dependencies that correlate data location with return address. A translation table in hardware could look up jump addresses, or a VM could do that if running in an off-the-shelf CPU. That's the best way I can think of to limit EXECUTE to specific jumps.

Many Forths nowadays don't use the Return Stack for return addresses 
(some do and do not use it for loop parameters, some only for r> >r and r@).

>> - No buffer underflow/overflow checks, ...
> That should be the MMU's job.

Microcontrollers usually don't have one.

>> - Definitions can be redefined.
> Not really, they're early-bound. You just end up hiding old definitions which may be desirable.
>
>> - Defer can't be frozen.
> I freeze defered words when they are compiled into CDATA space. The code is a single branch instruction in ROM, which gets patched once by a smart IS. Defers that are changeable are compiled to IDATA space, and IS makes the distinction.
>
>>
>> Checking for overflow is nothing if the return stack remains open
>> (overflow is only a way to update the return stack addresses and take
>> control of the program).
>> ==> Make the return stack only handled by the compiler
>> ==> Remove words like >R, R>, R@, ...
>> ==> Values on the return stack are only available through locals.
> Removing words like >R, R>, R@ breaks quite a lot of existing code. You can over/underflow either stack pretty easily. Catching that should be hardware's job.

Besides, in some systems that's the only use that's made of it.

>> 2) Packages
>> Remove vocabularies and word lists, list order, ...
>> Instead, robust packages must be provided.
>> This should be CORE.
>
> That seems to be hard to agree upon. MPE Ltd deals with some large code bases. MODULE, END-MODULE and EXPORT seem to work well for them. The search order still needs to be manipulated to make visible the words inside the modules you're interested in. Any ideas?

Several systems have packages, but the rules vary a lot. You're welcome 
to propose something...

>> 3) Burden for the Forth writer, not for the Forth user.
>> Simplicity too often means simplicity for the Forth writer.
>> And, in order to have the most simpler outer interpreter, burden
>> is handled by the user.
>> - Still a space needed after some words ( " , ...).
> You're kidding, right? Blank delimited strings...
>
>> - ' is not used for characters
> Some number recognizers accept 'c' as the character c. Would be nice if it were standard.
>
>> - [char], ['], S", S\", <#, #, ">, #S, [COMPILE], ...
> Yes, all to get rid of state-smart words. Of course, if your implementation uses separate wordlists for the compile and interpret semantics of a word, I don't think that matters much. BTW, S", S\" are still state smart. Numeric conversion like <#, #, #S are not state smart but they can be stepped on by other instances of numeric conversion.
>
>> - Multiple name for the same operation : +, F+, D+, ...
> Polly want a morphism? StrongForth tried that. Now that many Forths are 32-bit, D+ is rarely used. I don't think overloading + with F+ is all that great either. If you wanted to index into an array of floats, FLOATS MYARRAY + F@ F+ would not like that.
>
>>
>> Case sensitive interpreter is mandatory, of course.
>
> At least optional. Code developed on a case insensitive Forth is going to break all over the place.

Massive code breaker!

>> And there is this subject about semantics, immediate,
>> compilation, execution, interpretation, one xt, two xt, ...
>> to clarify.
>> ==> Define a clear metamodel for Words.
> I'm more interested in swimsuit models.

The philosophy since Forth94 has been to avoid mandating implementation.

>> 4) Memory management.
>> Direct and not controlled access to memory is prone to bugs and
>> open security holes.
>> There should have a higher level of abstraction for memory.
>> Not securised direct access to memory is forbidden.
>> This should be CORE.
> Okay, now you're off the reservation. Maybe memory spaces should be sandboxed. How can there be a wall between implementation and application? Both are written in Forth. Abstraction isn't your friend.

Again, embedded systems have specific memory constraints and issues.

>> 5) Blocks (or "quotations")
>> This should be CORE.
>> NONAME should be removed.
>> With use of parent locals (closures or in the scope of the parent).
> What's the replacement for ":NONAME blah blah ; IS FOO" ?
>
>>
>>
>> 6) Effective locals
>> Locals should be CORE.
>> Needed if the return stack is not directly accessible.
>> Locals should be fully optimized, stored directly on the
>> return stack for efficiency.
>> They are the only way to communicate with the return stack.
>
> I'd go a little further and support scoping of locals over a module rather than within a single word. Locals are great in theory. I've implemented them and found that I never used them. Well, how about those apples?
>
>>
>>
>> 7) Effective multi-tasking features.
>> Not just mapping OS threads or cooperative multi-threading.
>> Implement a modern multi-tasking design :
>> - Isolated tasks.
>> - non blocking threads (wait, I/O, TCP, ...).
>> - Channels to communicate between tasks.
>> - Reenforced immutability,
>> - Tasks with separated data stack and return stack for each.
>> - No global mutable values => remove VARIABLE
>> - Data stack separated from return stack helps a lot.
>>
>> And this should be CORE : interpreter runs into a task and
>> waits for keys in a non blocking way.
> You have an OS? Multicore is definitely on the horizon. Single core multitasking has been well tested for 40 years. I think a multicore wordset should be considered.

Standardizing a Forth multitasker model is a multi-year project at best, 
particularly since Forths running on an OS have very different 
requirements and constraints from native Forths.


>> 8) Make Data stack unique with 1 cell = 1 object.
>> - Strings need 2 cells -> to be changed.
>> - Double need 2 cells -> to be removed (with all double words).
>> - Characters value should be unicode
>> - String should be UTF8 and take 1 cell on the stack.
>> - No dedicated stacks (floats, strings, ...).
> The migration from single cell counted-string addresses to the (addr length) format was thoroughly researched. But then, that was before UTF-8 became a thing. Now that character count does not equal byte count, maybe it's time to rethink that. But then, the character count is only needed for rendering so probably not.

Massive broken code. This is really a new language.

>> Required to have effective multi-tasking features.
>> (you can't create/save n stacks by task).
> I think it already has that, just not in the standard.

There are numerous multitasking Forths, but they're all differently 
conceived. Good luck getting consensus.

>> 9) Same for return stack
>> - Only one kind of local.
>> - Local value = 1 object (float, string, array, ...).
> MPE's locals syntax I think supports various sizes of local data. It's doable.
>
>>
>> 10) Built-in collections and HOF
>> Minimum is Array (users are not here to implement arrays).
>> Dedicated word for array syntax.
>> Checks for array access (out of bounds).
>> This should be CORE.
>> HOF that can take words, quotations, closures as parameters.
> Seems like something that could be built on top of today's CORE words. So, why define it? You could have a "Neat Ideas" appendix.
>
>>
>> 11) Efficiency, but not before requirements.
>> All this should be done keeping effeciency (and, if possible,
>> small memory footprint).
>> But first the requirements, then optimize.

Much of what you propose above is costly both in efficiency/performance 
and memory size.

Cheers,
Elizabeth

-- 
Elizabeth R. Conklin
FORTH, Inc.
6080 Center Drive, Suite 600
Los Angeles, CA  90045
USA

[toc] | [prev] | [next] | [standalone]

#56448

"Elizabeth D. Rather" <erather@forth.com> writes:
>Standardizing a Forth multitasker model is a multi-year project at best, 
>particularly since Forths running on an OS have very different 
>requirements and constraints from native Forths.

It turns out that there is a lot of commonality even between the
wordsets used by various systems for multi-tasking, whether based on
OS threads (with potentially multiple cores active at the same time)
or on the cooperative round-robin multi-tasker; that's probably
because we all started with the round-robin multi-tasker, and have
systems that use OS thread support, as well as systems that don't have
that available.

Concerning the multi-year project, the way the committee is working,
standardizing anything is a multi-year project.

>There are numerous multitasking Forths, but they're all differently 
>conceived. Good luck getting consensus.

Consensus looks closer than I expected at first.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2016: http://www.euroforth.org/ef16/

[toc] | [prev] | [next] | [standalone]

#56445

Brad Eckert <hwfwguy@gmail.com> writes:
>Hi Franck, I see you've spent some time thinking about Forth and ways a sys=
>tem could be attacked. This could be fun.
>
>On Wednesday, March 22, 2017 at 3:24:38 AM UTC-7, franck....@gmail.com wrot=
>e:
>> - ' is not used for characters=20
>Some number recognizers accept 'c' as the character c. Would be nice if it =
>were standard.

It is.

>The migration from single cell counted-string addresses to the (addr length=
>) format was thoroughly researched. But then, that was before UTF-8 became =
>a thing. Now that character count does not equal byte count, maybe it's tim=
>e to rethink that. But then, the character count is only needed for renderi=
>ng so probably not.

The "u" in "c-addr u" as well as the count byte in counted strings
specifies the number of bytes, not Unicode code points (xchars) nor
Unicode characters.  And the character count is not really interesting
for rendering.  This is a simplified view, but it's the relevant one
on byte-addressed machines when using UTF-8 encoding.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2016: http://www.euroforth.org/ef16/

[toc] | [prev] | [next] | [standalone]

#56693

Le mercredi 29 mars 2017 03:34:29 UTC+2, Brad Eckert a écrit :
> Hi Franck, I see you've spent some time thinking about Forth and ways a system could be attacked. This could be fun.
> 

Hi Brad,

Sorry, I just see your answer.

My purpose was just to list what I like/don't like in Forth, not to
describe how Forth should evolve (even if I think it should 
adapt, when possible, to be more tight on security concerns).


> On Wednesday, March 22, 2017 at 3:24:38 AM UTC-7, franck....@gmail.com wrote:
> 
> > Requirements : 
> > 0) Keep what is great.
> Okay, keep it simple.
> 
> > 1) Security features.
> > Never say "No security to bother on embedded".
> No, don't let bad code into your playground.
> 
> > 
> > - Often data mixed with code.
> Some do and some don't. Embedded Forths usually don't.
> 
> > - Return stack is wide-open, a hacker can jump wherever he wants.
> It's possible with some effort to not use addresses as xts, but then you have to remove all dependencies that correlate data location with return address. A translation table in hardware could look up jump addresses, or a VM could do that if running in an off-the-shelf CPU. That's the best way I can think of to limit EXECUTE to specific jumps.
>

xt can remain an executable address provided that you can't directly
execute them.
What you can execute is a word (an xt(s) being an internal value of a 
word). For instance : 
"perform" allows to execute a word (by calling its xt)
"compile" allows to compile a word (by generating a call of its xt 
into the current definition).


 
> > - No buffer underflow/overflow checks, ...
> That should be the MMU's job.
> 
> > - Definitions can be redefined.
> Not really, they're early-bound. You just end up hiding old definitions which may be desirable. 
> 
> > - Defer can't be frozen.
> I freeze defered words when they are compiled into CDATA space. The code is a single branch instruction in ROM, which gets patched once by a smart IS. Defers that are changeable are compiled to IDATA space, and IS makes the distinction. 
> 
> > 
> > Checking for overflow is nothing if the return stack remains open
> > (overflow is only a way to update the return stack addresses and take
> > control of the program).
> > ==> Make the return stack only handled by the compiler
> > ==> Remove words like >R, R>, R@, ...
> > ==> Values on the return stack are only available through locals.
> Removing words like >R, R>, R@ breaks quite a lot of existing code. You can over/underflow either stack pretty easily. Catching that should be hardware's job.


Yes, removing those words breaks lot of existing code.



> > 
> > 2) Packages
> > Remove vocabularies and word lists, list order, ...
> > Instead, robust packages must be provided.
> > This should be CORE.
> 
> That seems to be hard to agree upon. MPE Ltd deals with some large code bases. MODULE, END-MODULE and EXPORT seem to work well for them. The search order still needs to be manipulated to make visible the words inside the modules you're interested in. Any ideas?


Here is how I implement packages : 

1) The system maintains a list of loaded packages and what is the
current package (CURRENT).

2) A package : 
- has a hash of defined words (or a list, it does not matter)
- has a list of imported packages.
- is described by a file that lists the files to load when you load the
package.

3) At startup, a "root" package is created, the current package is set and
the list of packages is initialized with this package.

4) When a word is defined into the dictionary, it is added to the
current package wordlist. Two words can have the same name if they are
created into two differents packages.

5) FIND works like this : 
. If the string is a qualified (ie "package:word"), it retrieves the
word into this package wordline and return (even if not found).
. Otherwise, try to find the word into the current package wordlist.
. Otherwise, try to find the word into the "root" package wordlist.
. Otherwise, try to find the word into the imported packages of the
current package.

6) Loading a package :

There are 2 words to load a package : 
use: package
import: package

If the package is not already loaded (ie into package list) : 
    create a new package and add it to the package list.
    Save the current package.
    set the current package to this package
    load the packages description and the package files.
    restore the current package.
If "import:" : also add this package to the imported packages of
current package.


 
> > 
> > 3) Burden for the Forth writer, not for the Forth user.
> > Simplicity too often means simplicity for the Forth writer.
> > And, in order to have the most simpler outer interpreter, burden
> > is handled by the user.
> > - Still a space needed after some words ( " , ...).
> You're kidding, right? Blank delimited strings...


Sorry, I don't understand... :)
" " should be a string with 1 blank character.

 
> > - ' is not used for characters 
> Some number recognizers accept 'c' as the character c. Would be nice if it were standard.
> 
> > - [char], ['], S", S\", <#, #, ">, #S, [COMPILE], ...
> Yes, all to get rid of state-smart words. Of course, if your implementation uses separate wordlists for the compile and interpret semantics of a word, I don't think that matters much. BTW, S", S\" are still state smart. Numeric conversion like <#, #, #S are not state smart but they can be stepped on by other instances of numeric conversion.
> 
> > - Multiple name for the same operation : +, F+, D+, ...
> Polly want a morphism? StrongForth tried that. Now that many Forths are 32-bit, D+ is rarely used. I don't think overloading + with F+ is all that great either. If you wanted to index into an array of floats, FLOATS MYARRAY + F@ F+ would not like that. 
> 
> > 
> > Case sensitive interpreter is mandatory, of course.
> 
> At least optional. Code developed on a case insensitive Forth is going to break all over the place.
> 
> > 
> > And there is this subject about semantics, immediate, 
> > compilation, execution, interpretation, one xt, two xt, ...
> > to clarify.
> > ==> Define a clear metamodel for Words.
> I'm more interested in swimsuit models.


ok :)


> > 
> > 4) Memory management.
> > Direct and not controlled access to memory is prone to bugs and
> > open security holes.
> > There should have a higher level of abstraction for memory.
> > Not securised direct access to memory is forbidden.
> > This should be CORE.
> Okay, now you're off the reservation. Maybe memory spaces should be sandboxed. How can there be a wall between implementation and application? Both are written in Forth. Abstraction isn't your friend.
> 
> > 
> > 5) Blocks (or "quotations")
> > This should be CORE.
> > NONAME should be removed.
> > With use of parent locals (closures or in the scope of the parent).
> What's the replacement for ":NONAME blah blah ; IS FOO" ?


By allowing to define a quotation outside a definition :

#[ blah blah ] IS FOO

 
> > 
> > 
> > 6) Effective locals
> > Locals should be CORE.
> > Needed if the return stack is not directly accessible.
> > Locals should be fully optimized, stored directly on the
> > return stack for efficiency.
> > They are the only way to communicate with the return stack.
> 
> I'd go a little further and support scoping of locals over a module rather than within a single word. Locals are great in theory. I've implemented them and found that I never used them. Well, how about those apples?


In fact, you do use locals into definitions... It is just that you
handle your locals values with >R, R>, R@ ... words. What if
these words are not available ?

A stack is a great tool but I don't think that this is the more
suitable tool to handle locals into a definition (locals should be all
directly available all the time).

Locals should be here to avoid stack shuffling, but not by 
replacing data stack shuffling by return stack shuffling 
(for instance, think about those >R >R sequences).

There words also prevent the compilation of effective named locals
(as the return stack size can change into a definition).

Without those words, locals can be implemented in an effective 
way (probably more effective than >R R> words) and all return stack
shuffles are removed.



> > 
> > 
> > 7) Effective multi-tasking features.
> > Not just mapping OS threads or cooperative multi-threading.
> > Implement a modern multi-tasking design : 
> > - Isolated tasks.
> > - non blocking threads (wait, I/O, TCP, ...).
> > - Channels to communicate between tasks.
> > - Reenforced immutability, 
> > - Tasks with separated data stack and return stack for each.
> > - No global mutable values => remove VARIABLE
> > - Data stack separated from return stack helps a lot.
> > 
> > And this should be CORE : interpreter runs into a task and
> > waits for keys in a non blocking way.
> You have an OS? Multicore is definitely on the horizon. Single core multitasking has been well tested for 40 years. I think a multicore wordset should be considered.


Yes, of course, multicore multi-tasking.


> > 
> > 8) Make Data stack unique with 1 cell = 1 object. 
> > - Strings need 2 cells -> to be changed.
> > - Double need 2 cells -> to be removed (with all double words).
> > - Characters value should be unicode
> > - String should be UTF8 and take 1 cell on the stack.
> > - No dedicated stacks (floats, strings, ...).
> The migration from single cell counted-string addresses to the (addr length) format was thoroughly researched. But then, that was before UTF-8 became a thing. Now that character count does not equal byte count, maybe it's time to rethink that. But then, the character count is only needed for rendering so probably not.
> 
> > 
> > Required to have effective multi-tasking features.
> > (you can't create/save n stacks by task).
> I think it already has that, just not in the standard.
> 
> > 
> > 
> > 9) Same for return stack
> > - Only one kind of local.
> > - Local value = 1 object (float, string, array, ...). 
> MPE's locals syntax I think supports various sizes of local data. It's doable.
> 
> > 
> > 10) Built-in collections and HOF 
> > Minimum is Array (users are not here to implement arrays).
> > Dedicated word for array syntax.
> > Checks for array access (out of bounds).
> > This should be CORE.
> > HOF that can take words, quotations, closures as parameters.
> Seems like something that could be built on top of today's CORE words. So, why define it? You could have a "Neat Ideas" appendix.


I agree that this is attractive to have a language that can implement arrays
(and this a good exercice).
But I don't think that builting arrays on top of CORE words is effective
enough, especially when you want to check for underflow/overflow
( or a way to reallocate the memory if needed). Maybe I'am wrong...

 
> > 
> > 11) Efficiency, but not before requirements.
> > All this should be done keeping effeciency (and, if possible,
> > small memory footprint).
> > But first the requirements, then optimize.
> >

[toc] | [prev] | [next] | [standalone]

#56133

On Wednesday, March 15, 2017 at 6:11:01 AM UTC-4, Anton Ertl wrote:
> Julian Fondren <julian.fondren@gmail.com> writes:
> >Anyway I don't read this as a genuine paean for an ideal language, but
> >just as Paul Graham thinking ahead about Arc. "People will complain
> >about the hackish terse words -- I'll praise that." "People will
> >prefer Common Lisp -- I'll put in a gibe about code that's retained
> >for compatibility." "It's not going to be fast -- I'll say what every
> >scripting language advocate says about that."
> 
> Conversely, he may have described what likes and does not like about
> Common Lisp, and then decided to do something about the "not-likes",
> and designed Arc.
> 
> And yes, one could do similar things wrt Forth, and its dialects.
> Typically, people have been more on the "do" side than on the "write"
> side of this (e.g., Chuck Moore, Hans Bezemer, Slava Pestov, Ron
> Aaron, Franck Benunsan), with Hugh Aguilar being the exception that
> proves the rule.
> 
> - anton
> -- 
> M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
> comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
>      New standard: http://www.forth200x.org/forth200x.html
>    EuroForth 2016: http://www.euroforth.org/ef16/

how can forth scale to big apps? how use 100 machines together with forth?

[toc] | [prev] | [next] | [standalone]

#55783

Had you not mentioned that this snippet was by PG, I would have assumed you were talking about Oberon-2. Funny how context changes everything.

So what's good about forth? You can change everything, it's terse and nothing's retained compability. That about sums it up then I guess.

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | comp.lang.forth

csiph-web