Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #59873

Re: illustration of syscalls fork and mmap

Show all headers | View raw

krishna.myneni@ccreweb.org writes:
>1) What causes the discrepancy in two-process vs one-process speedup for th=
>e two different matrix multiplications cases presented here, which is more =
>pronounced in gforth? Is it simply the overhead involved in forking the pro=
>cess compared to the problem computation time?
>
>2) How are we able to get a speedup greater than 2x, e.g. for the case of g=
>forth-fast, our results show a speedup of 2.2x? Is this reproducible on oth=
>er systems?

Resource (e.g., bandwidth) limitations as well as synchronization
overhead can be responsible for sub-linear speedups.  Dividing the
work in a way that fits in the cache can be responsible for
super-linear speedups (that's basically the cache-blocking effect).  I
have no idea what is the case for your benchmarks, though.

>3) How well does forking scale in problem computation time, for this proble=
>m and for other specific problems, to a larger number of processes, n? Does=
> OS overhead provide diminishing returns as n goes to 8, or even to 16?

Matrix multiplication can be divided very well into lots of parallel
computations.  For a 500x500 matrix multiply, each of the 250k
elements of the result matrix can be computed independently of the
others, and that computation consists of 1000 loads, all of which can
be performed in parallel, 500 multiplies, all of which can be
performed in parallel, and 499 adds, which can be organized as an
11-deep tree.  So if you have unlimited hardware, you can do the 250M
loads (with, say, 50 cycles latency), 125M multiplies (4 cycles
latency), and 124.5M adds (4 cycles latency) in 50+4+11*4=98cycles
total.  Well, not quite: with that many functional units, you probably
have to add quite a number of cycles for getting data from one
functional unit to the next.

Matrix multiplication is used as a benchmark for supercomputers,
because it is very parallelization, but OTOH, has quite a bit of
communication requirements (unlike, e.g., computing the Mandelbrot
set), and is something that supercomputers do a lot in real
applications.

>4) How does this method compare in programming complexity and in computatio=
>n speedup with respect to using multiple threads? Clearly, I would think th=
>at forking involves more system resources, primarily memory. Are threads a =
>better way to achieve the same thing, considering use of system resources, =
>speedup, and programming complexity? I would like to see the matrix multipl=
>ication example(s) given here, recoded for execution under a multi-threaded=
> Forth system, for comparison.

Probably not that much difference on all counts.

>In general, the Forth community (or maybe just me) needs to be more cogniza=
>nt of using parallel techniques. It's not uncommon to see systems with 8 co=
>res now, and using single process/thread methods seems archaic, like using =
>an ox cart when one has a sports car, unless the computing problem cannot b=
>e parallelized. I'm not even utilizing my 10-year old home PC system anywhe=
>re near its capability when I run my own code.

Many people are too euphoric about using multi-threading (or other
kinds of shared-memory or distributed-memory multiprocessing
parallelization) for getting speedups.  You get additional kinds of
bugs (race conditions, deadlocks), execution is not perfectly
reproducible, which makes debugging harder, and as result, you get
only a speedup by a small factory, and you consume at least as much
CPU time as before.  It's better to first apply techniques for
speeding up single threads; in the present case you can use
vectorization and cache blocking to get quite a bit more.  Once you
are close to the limit of single-thread performance, one can consider
multiprocessing.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2017: http://www.euroforth.org/ef17/

Back to comp.lang.forth | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

illustration of syscalls fork and mmap krishna.myneni@ccreweb.org - 2017-05-13 12:25 -0700
  Re: illustration of syscalls fork and mmap krishna.myneni@ccreweb.org - 2017-05-22 16:00 -0700
    Re: illustration of syscalls fork and mmap anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-05-23 12:01 +0000
      Re: illustration of syscalls fork and mmap krishna.myneni@ccreweb.org - 2017-05-23 16:30 -0700
        Re: illustration of syscalls fork and mmap anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-05-24 10:13 +0000
          Re: illustration of syscalls fork and mmap krishna.myneni@ccreweb.org - 2017-05-24 13:59 -0700
            Re: illustration of syscalls fork and mmap anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-05-25 08:47 +0000
              Re: illustration of syscalls fork and mmap krishna.myneni@ccreweb.org - 2017-05-25 04:04 -0700
                Re: illustration of syscalls fork and mmap krishna.myneni@ccreweb.org - 2017-05-25 04:29 -0700
                Re: illustration of syscalls fork and mmap anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-05-25 11:27 +0000
                Re: illustration of syscalls fork and mmap krishna.myneni@ccreweb.org - 2017-05-25 05:23 -0700
    Re: illustration of syscalls fork and mmap krishna.myneni@ccreweb.org - 2017-05-25 16:54 -0700
      Re: illustration of syscalls fork and mmap krishna.myneni@ccreweb.org - 2017-05-25 19:19 -0700
        Re: illustration of syscalls fork and mmap krishna.myneni@ccreweb.org - 2017-05-25 19:34 -0700
          Re: illustration of syscalls fork and mmap Roelf Toxopeus <rt4all@nothishetnet.nl> - 2017-05-26 11:03 +0200
            Re: illustration of syscalls fork and mmap krishna.myneni@ccreweb.org - 2017-05-26 05:15 -0700
              Re: illustration of syscalls fork and mmap anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-05-26 15:22 +0000
                Re: illustration of syscalls fork and mmap krishna.myneni@ccreweb.org - 2017-05-26 17:13 -0700
                Re: illustration of syscalls fork and mmap josv@planet.nl - 2017-05-26 23:26 -0700
                Re: illustration of syscalls fork and mmap krishna.myneni@ccreweb.org - 2017-05-27 06:20 -0700
                Re: illustration of syscalls fork and mmap josv@planet.nl - 2017-05-27 07:41 -0700
                Re: illustration of syscalls fork and mmap anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-05-27 06:21 +0000
                Re: illustration of syscalls fork and mmap krishna.myneni@ccreweb.org - 2017-05-27 06:32 -0700
                Re: illustration of syscalls fork and mmap Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-05-27 01:17 -0500
                Re: illustration of syscalls fork and mmap josv@planet.nl - 2017-05-26 23:43 -0700
                Re: illustration of syscalls fork and mmap Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-05-27 01:49 -0500
                Re: illustration of syscalls fork and mmap mhx@iae.nl - 2017-05-28 03:25 -0700
                Re: illustration of syscalls fork and mmap josv@planet.nl - 2017-05-28 04:56 -0700
                Re: illustration of syscalls fork and mmap krishna.myneni@ccreweb.org - 2017-05-27 06:53 -0700
                Re: illustration of syscalls fork and mmap Roelf Toxopeus <rt4all@nothishetnet.nl> - 2017-05-27 10:23 +0200
          Re: illustration of syscalls fork and mmap krishna.myneni@ccreweb.org - 2017-05-27 12:07 -0700

csiph-web