Re: Speaking of Multiprocessing...

From	David Brown <david.brown@hesbynett.no>
Newsgroups	comp.arch.embedded
Subject	Re: Speaking of Multiprocessing...
Date	2017-03-24 11:19 +0100
Organization	A noiseless patient Spider
Message-ID	<ob2rlb$mfi$1@dont-email.me> (permalink)
References	<ob1ij7$jja$1@dont-email.me> <ob2khf$vd$1@dont-email.me> <_E5BA.329271$fZ.159901@fx41.am4>

Show all headers | View raw

On 24/03/17 10:28, Tom Gardner wrote:
> On 24/03/17 08:17, David Brown wrote:
>> But if you can use dedicated hardware, there are many other methods.
>> The XMOS devices have hardware support for pipelines and message
>> passing.  On a dual-core PPC device I used, there is a hardware block of
>> semaphores.  Each semaphore is a pair of 16-bit ID, 16-bit value that
>> you can only access as a 32-bit read or write.  You can write to it if
>> the current ID is 0, or if the ID you are writing matches that of the
>> semaphore.  There is plenty of scope for variation based on that theme.
> 
> I received my first XMOS board from Digi-Key a couple of days
> ago, and I'm looking forward to using it for some simple
> experiments. I /feel/ that many low-level things will be
> much simpler and with fewer potential nasties lurking in
> the undergrowth. (I felt the same with the Transputer, for
> obvious reasons, but never had a suitable problem at that
> time)
> 
> With your experience, did you find any undocumented gotchas
> and any pleasant or unpleasant surprises?
> 

Before saying anything else, I would first note that my work with XMOS
systems was about four years ago, when they first started getting
popular.  I believe many things that bugged me most have been improved
since then, both in the hardware and software, but some may remain.

I think the devices themselves are a really neat idea.  You have very
fast execution, very efficient hardware multi-threading, very
predictable timings, and a variety of inter-thread and inter-process
communication methods.

Their "XC" programming language was also a neat idea, based on C with
additional primitives to support the hardware features and
multi-threading stuff, and an attempt to make some aspects of C safer
(real arrays, control of when you can access variables, etc.).

However, IMHO the whole thing suffered from a number of serious flaws
that limit the possibilities for the chips.  Sure, they would work well
in some circumstances - but I was left with the feeling that "if only
they had done /this/, the devices would be so much better and could be
used for so many more purposes".  It is a little unfair to concentrate
on the shortcomings rather than the innovations and features, but that
is how I felt when using them.  And again, I know that at least some
issues here have been greatly improved since I last used them.

A obvious flaw with the chips is lack of memory.  The basic device with
one cpu and 8 threads had 64K ram that was for program memory and
run-time data.  There was no flash - you had to use an external SPI
flash which used valuable pins (messing up the use of blocks of 8, 16 or
32 pins), and used up a thread if you wanted to be able to access the
flash at run-time.  And while you could implement an Ethernet MAC or a
480 Mbps USB 2.0 interface on the chip, there was nowhere near enough
ram for buffering or to do anything useful with the interface.  Adding
external memory was ridiculously expensive in terms of pins, threads,
and run-time inefficiency.

The hardware threading is great, and provides a really easy model for
all sorts of things.  To make a UART transmitter, you have a thread that
waits for data coming in on a pipe.  To transmit a bit, you set a pin,
wait for a bit time (using hardware timers), then move on to the next
bit.  The code is simple and elegant.  A UART receiver is not much
harder.  There is lots of example code in this style.

Then you realise that to implement a UART, you have used a quarter of
the chip's resources.  Your elegant flashing light is another thread, as
is your PWM output.  Suddenly you find you are using a 500 MIPS chip to
do the work of a $0.50 microcontroller, and you only have a thread or
two left for the actual application.

And you end up trying to run FreeRTOS on one of your threads, or make
your own scheduler to multiplex several PWM channels in one thread.
Much of the elegance quickly disappears for real-world applications.

Then there is the software.  The XC language lets you write code that
starts tasks in parallel, automatically allocates channels for
communication, lets you declare timers and wait on them.  That's all
great in theory - but it quickly gets confusing when you try to figure
out the details of when you can pass these around, when they get
allocated and deallocated, or when you can have a thread create new
threads.  XC carefully tracks threads and data accesses, spotting and
blocking all sorts of possible race conditions.  If a variable is
written by one thread, then it can't be accessed from another.  You can
work with arrays safely, but you can't take addresses.  Data gets passed
between threads using communication channels that are safe from race
conditions and nicely synchronised.

And then you realise that to actually make the thing work, you would
need far more channels than there are on the device, and they would need
to be far faster - all you really wanted was for two threads to share a
circular buffer, and you know in your application code when it is safe
to use it.  But you can't do that in XC - the language and the tools
won't let you.  So you have to write that code in C, with calls back and
forth with the XC code that handles the multi-threading stuff.

And then you realise that from within the C, you need to access some
hardware resources like timers, that can't be expressed properly in C,
and you can't get back to the XC code at the time.  So you end up with
inline assembly.

Then there are the libraries and examples.  These were written in such a
wide variety of styles that it was impossible to figure out what was
going on.  A typical example project would involve a USB interface and,
for example, SPDIF channels for an USB audio interface.  The
Eclipse-based IDE was fine, but the example did not come as a project -
it came as a collection of interdependent projects.  Some bits referred
to files in different projects.  Some bits merely required other
projects to be compiled.  Some bits of the code in one project would use
assembly for hardware resources, others would use XC, others would use C
intrinsic functions, and others would use a sort of XML file that
defines the setup for your chip resources.  If you change values in one
file in one project (say, the USB vendor ID), you have to figure out
which sub-projects need to be manually forced to re-build in order for
it to take effect consistently throughout the project.  Some parts use a
fairly obvious configuration file - a header with defines that let you
control things like IDs, number of channels, pins, etc.  Except they
don't - only /some/ of the sub-projects read and use the configuration
file, other parts are hard-coded or use values from elsewhere.  It was a
complete mess.

Now, I know that newer XMOS devices have more resources, built-in flash,
proper hardware peripherals for the devices that are most demanding or
popular, and so on.  And I can only hope that the language and tools
have been improved to the point where inline assembly is not required,
and that the examples and libraries have matured to the point that the
libraries are usable as-is, and the examples show practical ways to
develop code.

I really hope XMOS does well here - it is so good to see a company that
thinks in a very different way and brings in these new ideas.  So if
your experience with modern XMOS devices and tools is good, I would love
to hear about it.

Thread

Speaking of Multiprocessing... rickman <gnuarm@gmail.com> - 2017-03-23 18:38 -0400
  Re: Speaking of Multiprocessing... Tim Wescott <seemywebsite@myfooter.really> - 2017-03-23 18:19 -0500
    Re: Speaking of Multiprocessing... Don Y <blockedofcourse@foo.invalid> - 2017-03-23 16:26 -0700
      Re: Speaking of Multiprocessing... Tim Wescott <seemywebsite@myfooter.really> - 2017-03-23 18:47 -0500
        Re: Speaking of Multiprocessing... Don Y <blockedofcourse@foo.invalid> - 2017-03-23 17:49 -0700
          Re: Speaking of Multiprocessing... upsidedown@downunder.com - 2017-03-24 09:48 +0200
            Re: Speaking of Multiprocessing... Don Y <blockedofcourse@foo.invalid> - 2017-03-24 01:22 -0700
        Re: Speaking of Multiprocessing... Clifford Heath <no.spam@please.net> - 2017-03-24 21:04 +1100
          Re: Speaking of Multiprocessing... Robert Wessel <robertwessel2@yahoo.com> - 2017-03-24 22:01 -0500
            Re: Speaking of Multiprocessing... Clifford Heath <no.spam@please.net> - 2017-03-25 18:49 +1100
    Re: Speaking of Multiprocessing... Grant Edwards <invalid@invalid.invalid> - 2017-03-24 14:36 +0000
  Re: Speaking of Multiprocessing... Robert Wessel <robertwessel2@yahoo.com> - 2017-03-23 22:43 -0500
    Re: Speaking of Multiprocessing... rickman <gnuarm@gmail.com> - 2017-03-24 01:05 -0400
      Re: Speaking of Multiprocessing... Robert Wessel <robertwessel2@yahoo.com> - 2017-03-24 00:37 -0500
        Re: Speaking of Multiprocessing... rickman <gnuarm@gmail.com> - 2017-03-24 01:58 -0400
          Re: Speaking of Multiprocessing... Robert Wessel <robertwessel2@yahoo.com> - 2017-03-24 02:07 -0500
            Re: Speaking of Multiprocessing... rickman <gnuarm@gmail.com> - 2017-03-24 13:36 -0400
              Re: Speaking of Multiprocessing... David Brown <david.brown@hesbynett.no> - 2017-03-24 19:06 +0100
                Re: Speaking of Multiprocessing... rickman <gnuarm@gmail.com> - 2017-03-24 14:12 -0400
                Re: Speaking of Multiprocessing... David Brown <david.brown@hesbynett.no> - 2017-03-25 01:48 +0100
                Re: Speaking of Multiprocessing... rickman <gnuarm@gmail.com> - 2017-03-24 22:10 -0400
                Re: Speaking of Multiprocessing... David Brown <david.brown@hesbynett.no> - 2017-03-25 15:25 +0100
                Re: Speaking of Multiprocessing... rickman <gnuarm@gmail.com> - 2017-03-25 15:27 -0400
                Re: Speaking of Multiprocessing... upsidedown@downunder.com - 2017-03-25 22:43 +0200
                Re: Speaking of Multiprocessing... rickman <gnuarm@gmail.com> - 2017-03-25 17:06 -0400
                Re: Speaking of Multiprocessing... David Brown <david.brown@hesbynett.no> - 2017-03-26 17:28 +0200
                Re: Speaking of Multiprocessing... rickman <gnuarm@gmail.com> - 2017-03-26 13:27 -0400
              Re: Speaking of Multiprocessing... Dimiter_Popoff <dp@tgi-sci.com> - 2017-03-25 00:42 +0200
                Re: Speaking of Multiprocessing... rickman <gnuarm@gmail.com> - 2017-03-24 19:22 -0400
                Re: Speaking of Multiprocessing... Rob Gaddi <rgaddi@highlandtechnology.invalid> - 2017-03-24 16:33 -0700
                Re: Speaking of Multiprocessing... Robert Wessel <robertwessel2@yahoo.com> - 2017-03-24 22:47 -0500
                Re: Speaking of Multiprocessing... Dimiter_Popoff <dp@tgi-sci.com> - 2017-03-25 12:40 +0200
                Re: Speaking of Multiprocessing... Robert Wessel <robertwessel2@yahoo.com> - 2017-03-26 00:42 -0500
  Re: Speaking of Multiprocessing... David Brown <david.brown@hesbynett.no> - 2017-03-24 09:17 +0100
    Re: Speaking of Multiprocessing... Tom Gardner <spamjunk@blueyonder.co.uk> - 2017-03-24 09:28 +0000
      Re: Speaking of Multiprocessing... David Brown <david.brown@hesbynett.no> - 2017-03-24 11:19 +0100
        Re: Speaking of Multiprocessing... Tom Gardner <spamjunk@blueyonder.co.uk> - 2017-03-24 11:06 +0000
          Re: Speaking of Multiprocessing... David Brown <david.brown@hesbynett.no> - 2017-03-24 12:21 +0100
        Re: Speaking of Multiprocessing... Tom Gardner <spamjunk@blueyonder.co.uk> - 2017-03-24 16:09 +0000
          Re: Speaking of Multiprocessing... David Brown <david.brown@hesbynett.no> - 2017-03-24 17:33 +0100
            Re: Speaking of Multiprocessing... Tom Gardner <spamjunk@blueyonder.co.uk> - 2017-03-24 17:52 +0000

csiph-web