Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #54966 > unrolled thread

User-space networking

Back to article view | Back to comp.lang.forth

Contents

  User-space networking anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-02-18 07:53 +0000
    Re: User-space networking lukego@gmail.com - 2017-02-19 03:42 -0800
      Re: User-space networking foxaudioresearch@gmail.com - 2017-02-19 06:44 -0800
    Re: User-space networking polymorph self <jackvanc3@gmail.com> - 2017-02-21 08:54 -0800
    Re: User-space networking polymorph self <jackvanc3@gmail.com> - 2017-03-01 12:01 -0800

#54966 — User-space networking

I found the following article interesting:

<https://lwn.net/Articles/713918/>

The Snabb networking toolkit described in the article is written using
LuaJIT, but the problem looks like it would be a good fit for Forth,
and there are some other Forth-related aspects here.

The problem is that they have so many packets to process that going
through the kernel for network access is too slow.  So they apparently
memory-map the network cards' control registers to user space, and
access them directly; and of course the network cards put the data
they reveive in buffers in user space (in L3 cache according to the
article), and get the data they receive from user space.  Only basic
functionality of the network cards is used, not advanced,
highly-touted, and often buggy features like TCP offload engines.
There is little processing time available per packet, so the
processing code cannot be very long.

That's an environment much like the original Forth setup: direct
hardware access without intervening kernel, and performance counts.
so this is an area where Forth could be used profitably.  Of course,
they already have a quick-development solution using LuaJIT, and if
that's not good enough for performance, they will probably drop down
into C or assembly language, so Forth probably cannot get into this
area, but anyway, this shows that the place of Forth is not just in
small embedded controllers with little RAM.

The philosophy they use is also pretty Forthy:

|The idea behind Snabb, he said, is "rewritable software" — as in "I
|could rewrite that in a weekend".
[...]
| Snabb was built around a set of three simple design principles:
|
|    Simple > Complex
|    Small > Large
|    Commodity > Proprietary 
[...]
|For small: the original Snabb implementation had a code budget. Snabb
|as a whole was meant to be less than 10,000 lines of code and build in
|less than one minute. These constraints, it was hoped, would lead to
|problems being solved in a creative way.
[...]
|To stay small, Snabb also avoids depending on big projects. Rather
|than use the DPDK drivers, Snabb's developers have written their own.

The original Forth environments had memory and CPU limits that
enforced the kind of creativity that the Snabb people enforce with
their code-size and build-time limit (I wonder why 10,000 lines of
code take one minute to build, though).

The third Forth tangent is net2o.  AFAIK, currently net2o is tunneling
over UDP/IP, but I think the idea is to eventually have the routers
understand net2o and route it directly.  If the processing is moving
from special-purpose hardware into easily malleable software like
Snabb, it seems that getting direct net2o routing could be much
easier, and therefore could be introduced much faster than I expected
before.  Does that mean that the routers will be programmed in Forth?
Not necessarily.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2016: http://www.euroforth.org/ef16/

[toc] | [next] | [standalone]

#55005

Hi Anton,

Cool to see you make a Snabb - Forth connection here! I started the Snabb project and can offer a little bit of context :-)

Forth is indeed a strong influence. Shortly before starting Snabb I worked as an assistant to Mitch Bradley doing Openfirmware hacking for the OLPC. I was very impressed with Openfirmware in particular and Forth in general and wanted to bring this philosophy to my networking hacking. I decided to use LuaJIT instead of Forth because I thought it would make the project accessible to more people e.g. casual programmers with some python/perl/ruby/etc experience.

Old braindump on the topic mentioning Openfirmware here: http://lukego.github.io/blog/2012/09/25/lukes-highly-opinionated-programming-language-roundup-2012/

I will say that LuaJIT's trace-based JIT has a steeper learning curve than I anticipated. Here we are five years later and I am still working on how to formulate a really simple "how to write fast code" guideline.

The 10,000 lines of Snabb code actually takes about one second to build. However, the one minute build budget includes the LuaJIT runtime system (~50 KLOC of C) and that brings us up to around one CPU-minute in practice.

[toc] | [prev] | [next] | [standalone]

#55009

Very interesting. I have a couple of questions.

Would it be worth some effort to create a Forth version using gForth, understanding it would have a smaller audience?

In hindsight would a Forth based system have been easier to implement?  Could the Forth words be crafted to make a more palatable newbie interface?

B

[toc] | [prev] | [next] | [standalone]

#55032

On Saturday, February 18, 2017 at 3:45:46 AM UTC-5, Anton Ertl wrote:
> I found the following article interesting:
> 
> <https://lwn.net/Articles/713918/>
> 
> The Snabb networking toolkit described in the article is written using
> LuaJIT, but the problem looks like it would be a good fit for Forth,
> and there are some other Forth-related aspects here.
> 
> The problem is that they have so many packets to process that going
> through the kernel for network access is too slow.  So they apparently
> memory-map the network cards' control registers to user space, and
> access them directly; and of course the network cards put the data
> they reveive in buffers in user space (in L3 cache according to the
> article), and get the data they receive from user space.  Only basic
> functionality of the network cards is used, not advanced,
> highly-touted, and often buggy features like TCP offload engines.
> There is little processing time available per packet, so the
> processing code cannot be very long.
> 
> That's an environment much like the original Forth setup: direct
> hardware access without intervening kernel, and performance counts.
> so this is an area where Forth could be used profitably.  Of course,
> they already have a quick-development solution using LuaJIT, and if
> that's not good enough for performance, they will probably drop down
> into C or assembly language, so Forth probably cannot get into this
> area, but anyway, this shows that the place of Forth is not just in
> small embedded controllers with little RAM.
> 
> The philosophy they use is also pretty Forthy:
> 
> |The idea behind Snabb, he said, is "rewritable software" — as in "I
> |could rewrite that in a weekend".
> [...]
> | Snabb was built around a set of three simple design principles:
> |
> |    Simple > Complex
> |    Small > Large
> |    Commodity > Proprietary 
> [...]
> |For small: the original Snabb implementation had a code budget. Snabb
> |as a whole was meant to be less than 10,000 lines of code and build in
> |less than one minute. These constraints, it was hoped, would lead to
> |problems being solved in a creative way.
> [...]
> |To stay small, Snabb also avoids depending on big projects. Rather
> |than use the DPDK drivers, Snabb's developers have written their own.
> 
> The original Forth environments had memory and CPU limits that
> enforced the kind of creativity that the Snabb people enforce with
> their code-size and build-time limit (I wonder why 10,000 lines of
> code take one minute to build, though).
> 
> The third Forth tangent is net2o.  AFAIK, currently net2o is tunneling
> over UDP/IP, but I think the idea is to eventually have the routers
> understand net2o and route it directly.  If the processing is moving
> from special-purpose hardware into easily malleable software like
> Snabb, it seems that getting direct net2o routing could be much
> easier, and therefore could be introduced much faster than I expected
> before.  Does that mean that the routers will be programmed in Forth?
> Not necessarily.
> 
> - anton
> -- 
> M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
> comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
>      New standard: http://www.forth200x.org/forth200x.html
>    EuroForth 2016: http://www.euroforth.org/ef16/

can this replace a openbsd rotuer?

[toc] | [prev] | [next] | [standalone]

#55326

On Saturday, February 18, 2017 at 3:45:46 AM UTC-5, Anton Ertl wrote:
> I found the following article interesting:
> 
> <https://lwn.net/Articles/713918/>
> 
> The Snabb networking toolkit described in the article is written using
> LuaJIT, but the problem looks like it would be a good fit for Forth,
> and there are some other Forth-related aspects here.
> 
> The problem is that they have so many packets to process that going
> through the kernel for network access is too slow.  So they apparently
> memory-map the network cards' control registers to user space, and
> access them directly; and of course the network cards put the data
> they reveive in buffers in user space (in L3 cache according to the
> article), and get the data they receive from user space.  Only basic
> functionality of the network cards is used, not advanced,
> highly-touted, and often buggy features like TCP offload engines.
> There is little processing time available per packet, so the
> processing code cannot be very long.
> 
> That's an environment much like the original Forth setup: direct
> hardware access without intervening kernel, and performance counts.
> so this is an area where Forth could be used profitably.  Of course,
> they already have a quick-development solution using LuaJIT, and if
> that's not good enough for performance, they will probably drop down
> into C or assembly language, so Forth probably cannot get into this
> area, but anyway, this shows that the place of Forth is not just in
> small embedded controllers with little RAM.
> 
> The philosophy they use is also pretty Forthy:
> 
> |The idea behind Snabb, he said, is "rewritable software" — as in "I
> |could rewrite that in a weekend".
> [...]
> | Snabb was built around a set of three simple design principles:
> |
> |    Simple > Complex
> |    Small > Large
> |    Commodity > Proprietary 
> [...]
> |For small: the original Snabb implementation had a code budget. Snabb
> |as a whole was meant to be less than 10,000 lines of code and build in
> |less than one minute. These constraints, it was hoped, would lead to
> |problems being solved in a creative way.
> [...]
> |To stay small, Snabb also avoids depending on big projects. Rather
> |than use the DPDK drivers, Snabb's developers have written their own.
> 
> The original Forth environments had memory and CPU limits that
> enforced the kind of creativity that the Snabb people enforce with
> their code-size and build-time limit (I wonder why 10,000 lines of
> code take one minute to build, though).
> 
> The third Forth tangent is net2o.  AFAIK, currently net2o is tunneling
> over UDP/IP, but I think the idea is to eventually have the routers
> understand net2o and route it directly.  If the processing is moving
> from special-purpose hardware into easily malleable software like
> Snabb, it seems that getting direct net2o routing could be much
> easier, and therefore could be introduced much faster than I expected
> before.  Does that mean that the routers will be programmed in Forth?
> Not necessarily.
> 
> - anton
> -- 
> M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
> comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
>      New standard: http://www.forth200x.org/forth200x.html
>    EuroForth 2016: http://www.euroforth.org/ef16/

can you liek nfs mount a browser and then make web development ez with few basic operations?

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.forth

csiph-web