Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #10985

Re: FastCGI-like stuff and Forth

Show all headers | View raw

Andrew Haley <andrew29@littlepinkcloud.invalid> writes:
>> And on big x86 linuxes these days, the page size is 1MB(?).
> Those are huge pages, and they're only used if your memory request is
> a multiple of the page size.  I rather hope that huge pages aren't
> used by default for thread stacks, but I guess it's not impossible.

Actually I'm not sure.  Maybe 4k is the default.  I wonder how to check.

> Hmmm.  I doubt very much that it makes sense for tasks to share pages,
> but maybe.

Is that not how it's usually done in Forth?  I've only looked at the
Eforth multitasker, and it's brilliantly simple--there is almost nothing
there.  It doesn't have any idea about pages, iirc.

> OK, but somehow you're going to have to do load balancing between
> cores.  The kernel can't split a single thread that happens to be
> running a bunch of tasks, so you have to have some way of moving a
> task from one thread to another.

Hmm, I wouldn't have thought such migration was necessary, but maybe it
is (Erlang and GHC can do it, Python can't).  In a typical web server
you have a lot of semi-persistent connections, each of which is idle
most of the time but sends a request now and then, and serving an
individual request is pretty fast.  So you can do ok just launching a
task for each new connection, in whichever process is the least busy
when the connection arrives, and then leaving the task in that process
til it exits.  If each task serves a few dozen requests in its lifetime
and you have 1000's of tasks per process, this should stay pretty
balanced, I'd expect.

Even if you want to migrate connections, you don't have to do it by
moving threads around.  Linux lets you pass open files between
processes, through Unix domain sockets (the SCM_RIGHTS ancillary
message).  I think Solaris and *BSD have something similar.  So you can
just juggle the requests around.

There's a separate issue that came up in that GHC paper I linked,
though: each task typically will have an associated timeout.  The old
GHC implementation started to bog down when finding the next timeout
required a linear scan of the thread list, and the number of threads got
into the 1000's.  The new implementation put in an O(log n) priority
search structure and got a big speedup.  A Forth implementation would
presumably have to do something similar.  If it uses Unix threads, then
the Unix kernel has to do it instead.  In the latter case, the
complexity isn't eliminated, it's just being swept under the rug.

>> 2) Even if there's two schedulers, is that so bad?
> If it really is the best way to do it, no.  Sometimes complexity
> really is justified.  Color me skeptical.

The overhead of Xen/KVM vs. a single kernel has been measured various
ways and it's not zero, but in practice people seem to find it not too
bad.

Back to comp.lang.forth | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

FastCGI-like stuff and Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2012-04-03 13:28 +0000
  Re: FastCGI-like stuff and Forth Paul Rubin <no.email@nospam.invalid> - 2012-04-03 11:44 -0700
    Re: FastCGI-like stuff and Forth Bernd Paysan <bernd.paysan@gmx.de> - 2012-04-03 21:46 +0200
      Re: FastCGI-like stuff and Forth Paul Rubin <no.email@nospam.invalid> - 2012-04-03 13:25 -0700
        Re: FastCGI-like stuff and Forth Bernd Paysan <bernd.paysan@gmx.de> - 2012-04-03 22:35 +0200
          Re: FastCGI-like stuff and Forth Paul Rubin <no.email@nospam.invalid> - 2012-04-04 02:20 -0700
            Re: FastCGI-like stuff and Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2012-04-04 05:52 -0500
              Re: FastCGI-like stuff and Forth Paul Rubin <no.email@nospam.invalid> - 2012-04-05 00:03 -0700
                Re: FastCGI-like stuff and Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2012-04-05 03:03 -0500
                Re: FastCGI-like stuff and Forth Paul Rubin <no.email@nospam.invalid> - 2012-04-05 10:42 -0700
                Re: FastCGI-like stuff and Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2012-04-06 06:30 -0500
                Re: FastCGI-like stuff and Forth Paul Rubin <no.email@nospam.invalid> - 2012-04-07 01:44 -0700
                Re: FastCGI-like stuff and Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2012-04-07 19:04 -0500
                Re: FastCGI-like stuff and Forth Paul Rubin <no.email@nospam.invalid> - 2012-04-07 17:40 -0700
        Re: FastCGI-like stuff and Forth quiet_lad <gavcomedy@gmail.com> - 2012-04-05 13:37 -0700
    Re: FastCGI-like stuff and Forth Albert van der Horst <albert@spenarnc.xs4all.nl> - 2012-04-04 13:20 +0000
      Re: FastCGI-like stuff and Forth Paul Rubin <no.email@nospam.invalid> - 2012-04-04 11:20 -0700
    Re: FastCGI-like stuff and Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2012-04-04 13:42 +0000
  Re: FastCGI-like stuff and Forth Spam@ControlQ.com - 2012-04-04 17:00 -0400
  Re: FastCGI-like stuff and Forth quiet_lad <gavcomedy@gmail.com> - 2012-04-05 13:36 -0700
  Re: FastCGI-like stuff and Forth quiet_lad <gavcomedy@gmail.com> - 2012-04-13 23:10 -0700

csiph-web