Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
| From | MitchAlsup <user5857@newsgrouper.org.invalid> |
|---|---|
| Newsgroups | comp.arch |
| Subject | SIMD Considered Harmful |
| Date | 2026-09-24 00:00 +0000 |
| Message-ID | <1790208006-5857@newsgrouper.org> (permalink) |
The old paper https://www.sigarch.org/simd-instructions-considered-harmful/ compares SIMD
implementations with Vector implementations--as-if they were the only 2 games in town. The
paper contains the <toy> benchmark::
void daxpy(const size_t n, const double a, const double x[], double y[])
{
for (size_t i = 0; i < n; i++) {
y[i] = a*x[i] + y[i];
}
}
and goes on to extoll the beneficial properties of Vectors over SIMD using RISC-V as
exemplary. Let me pivot the conversation back to My 66000 virtual-Vector-Method vVM
(an alternative to both SIMD and Vectors). The above code compiles to::
daxpy:
MOV R5,#0
VEC #0,{}
LDD R6,[R3,R5] // LOOP transfers control back to here
LDD R7,[R4,R5]
FMAC R8,R2,R5,R6
ST R8,[R4,R5]
LOOP1 LT,R5,#1,R1
RET
8 instructions total
5 instructions in the Loop {LDD to LOOP1}
Smaller than any of the ISAs mentioned in the paper.
#0 in VEC is compiler telling HW that the loop can be executed as wide as HW has Lanes
{} in VEC is compiler telling HW that no registers inside the Loop are live outside of
the Loop
LOOP1 is the ADD-CMP-BC as 1 instruction
LOOP1 can perform as many ADD-CMP-BC as the width of the Loop in a single cycle
LOOP1 is a single cycle calculation
While code remains "in" the LOOP FETCH-DECODE remains idle and Reservation Stations
perform loop iterations.
Now let us consider a Great-Big-Out-of-Order implementation of vVM in the execution
pipeline::
Assuming a 4-cycle LD cache hit and 4 cycle FMAC delay the execution of one iteration is::
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
| LD R6 | cache | hit | align |
| LD R7 | cache | hit | align |
| ST AG | cache | hit | | ST R8 |
| F M A C |
or 9-cycles of latency. To get here the machine needs 3 AGEN units and at least {2 LD + 1 ST}
or {3 LD-ST} units, 1 FMAC unit, and 1 LOOP unit.
We now write this iteration as a single line::
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
| Iteration |
vVM can perform multiple iterations, starting several per cycle up to the number of
calculation lanes in HW (limited by the small constant in VEC #small,{})::
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
+8 | Iteration 1 |
+8+1 | Iteration 2 |
+8+2 | Iteration 3 |
+8+3 | LD R6 | cache | hit | align Iteration 4 |
+8+4 | LD R7 | cache | hit | align Iteration 5 |
+8+5 | ST R8 | cache | Iteration 6 | ST R8 |
+8+6 | Iteration 7 |
+8+7 | Iteration 8 |
+8 | Iteration 9 |
+8+1 | Iteration 10 |
+8+2 | Iteration 11 |
+8+3 | LD R6 | cache | hit | align Iteration 12 |
+8+4 | LD R7 | cache | hit | align Iteration 13 |
+8+5 | ST R8 | cache | Iteration 14 | ST R8 |
+8+6 | Iteration 15 |
+8+7 | Iteration 16 |
| ad-infinitum
Where each LD and ST accesses a whole 64-Byte cache line, feeding 8 FMAC units with LD data,
and consuming all 8 FMAC results as data to be stored. Due to non-aligned to cache line
boundary issues, in general the AGEN part must run 1 access in front of where the buffers
feed the FAMC units so cache line boundary crossings are penalized once. {One should observe
that when LD R6 hits, ST R8 also hits (TLB too) because it is the same address, performed at
the same time.} {vVM arranges that if the ST line must be replaced in cache, that the miss-
buffers will remain capable of absorbing the results prior to writing to memory hierarchy.}
vVM creates its own masking--so that when the last several iterations are not needed, they
are not performed. So, there is no setup/maintenance of vector length <register>.
In addition, but of more power-importance, is that the LOOP runs out of the reservation
stations by adding a small index to the reservation station entries keyed to the loop
iteration, keeping everything in data flow order on a per iteration basis. So, while the
above Loop is running, FETCH-DECODE-INSERT is IDLE. In this sense, My 66000 GBOoO core
only DECODEs 8 instructions compared to 163 for RISC-V and thousands for other ISAs. So,
while calculation power is essentially identical, front-end power is greatly reduced.
You could say LOOP1 is predicted to be taken, but it uses no predictor, and the LOOP
Function Unit performs a comparison (per cycle) so the LOOP early outs at exactly the
right time--so, FETCHed and DECODEd instructions are not thrown away--nor is a branch
predictor state modified by the LOOP prediction {flow control within iterations will
update branch predictor state as needed}
And finally, vVM does this without any SW visible register file, making context switches
fast, and Thread state in memory small. Nor does SW need to worry about the iteration count
not being a multiple of the SIMD or Vector register length.
vVM is fundamentally different than SIMD or Cray-like vectors because it vectorizes loops
not instructions, But also that if an exception is raised, the person debugging the
application sees a scalar calculation instead of a vector calculation--that is exceptions
are "taken" with the IP pointing at the instruction which raised the exception, and the
register file filled with data of "that iteration".
One could postulate that there are 16 or even 32-lanes of calculations::
a) x86 has shown power problems at 8-wide leading to core frequency reductions
b) the memory units would have to access 2 or 4 cache lines per iteration
c) the cache buffering would grow by 2× to 4×
d) at some point one has to draw the line.
Now as to Vectors versus SIMD versus vVM:
a) adding vectors to a scalar ISA adds on the order of 300 instructions,
b) adding SIMD to a scalar ISA adds on the order of 1000 instructions,
c) adding vVM to a scalar ISA adds exactly 2 instructions.
Can vVM do everything vectors or SIMD can--frankly no. On the other hand, vVM can perform
mixed width calculations such as doubleword = word * byte + halfword--for which no SIMD
ISA has found the OpCode space consumption viable. vVM can vectorize every leaf-function
of str* and mem* C library calls. Given byte copy loop:
char *strcpy(char *restrict dest, const char *restrict src)
{
char *ret = dest;
while (*dest++ = *src++)
;
return ret;
}
strcpy:
ADD R2,R2,-R1 // make R1 index to R2
VEC #0,{R1}
LDUB R4,[R2,R1]
STB R4,[R1]
LOOP3 NE,R1,R4,#0 // LOOP type 3
RET
Given a 64-byte wide data-path (as in the above example 8 Double FP = 64-Bytes) each
iteration of the loop will Load 64-bytes, Store 64-bytes. Considering this is a 4 (or 5)
instruction loop in most scalar ISAs, the loop will be performed as if 256 instructions
per cycle. Thus, one can write byte-by-byte algorithms and vVM performs them at fill data
path width, generating its own masking, without a programmer revisiting the critical data
movement functions each new chip iteration.
The LOOPn subgroup can perform {counted loops, data terminated loops, and counted and data
terminated loops}--strncmp is an example:
int strncmp(const char* s1, const char* s2, size_t n)
{
while(n--)
if(*s1!=*s2)
return *s1 - *s2;
else
s1++, s2++;
return 0;
}
strncmp:
BNE0 R3,exit
MOV R4,#0 // change n-- to j=0; j<n; j++
VEC #0,{R4}
LDUB R5,[R1,R4]
LDUB R6,[R2,R4]
LOOP3 NE,R4,R5,R6 // LOOP type 3
ADD R1,R5,-R6
RET
exit: MOV R1,#0
RET
This 3 instruction loop can also run 64×2 bytes per cycle--but since most strings being
compared are rather short, this tends to run in 1 or at most 2 iterations in applications
like symbol-table lookups.
Summary:
a) fewer total instructions
b) fewer instructions in the loop
c) mixed width calculations
d) mixed width memory accesses
e) no SIMD register File
f) no Vector register File
g) SW is unconcerned about Register width or depth
h) only 2 instructions
i) scalar debug model
j) byte loops as fast as wide register loops
k) ease of software maintenance
Mitch
Back to comp.arch | Previous | Next — Next in thread | Find similar | Unroll thread
SIMD Considered Harmful MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-09-24 00:00 +0000
Re: SIMD Considered Harmful Lawrence D’Oliveiro <ldo@nz.invalid> - 2026-09-24 02:05 +0000
Re: SIMD Considered Harmful MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-09-24 17:09 +0000
Re: SIMD Considered Harmful Lawrence D’Oliveiro <ldo@nz.invalid> - 2026-09-25 00:08 +0000
Re: SIMD Considered Harmful MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-09-25 00:41 +0000
Re: SIMD Considered Harmful Lawrence D’Oliveiro <ldo@nz.invalid> - 2026-09-25 22:13 +0000
Re: SIMD Considered Harmful MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-09-26 00:29 +0000
Re: SIMD Considered Harmful anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2026-09-26 06:41 +0000
Re: SIMD Considered Harmful MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-09-26 18:23 +0000
Re: SIMD Considered Harmful BGB <cr88192@gmail.com> - 2026-09-26 14:47 -0500
Re: SIMD Considered Harmful MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-09-26 21:31 +0000
Re: SIMD Considered Harmful BGB <cr88192@gmail.com> - 2026-09-26 20:56 -0500
Re: SIMD Considered Harmful scott@slp53.sl.home (Scott Lurndal) - 2026-09-28 15:29 +0000
Re: SIMD Considered Harmful BGB <cr88192@gmail.com> - 2026-09-28 17:39 -0500
Re: SIMD Considered Harmful scott@slp53.sl.home (Scott Lurndal) - 2026-09-29 00:09 +0000
Re: SIMD Considered Harmful MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-10-13 21:40 +0000
Re: SIMD Considered Harmful Torbjorn Lindgren <tl@none.invalid> - 2026-09-29 17:01 +0000
Re: SIMD Considered Harmful BGB <cr88192@gmail.com> - 2026-09-29 20:49 -0500
Re: SIMD Considered Harmful BGB <cr88192@gmail.com> - 2026-09-24 15:04 -0500
Re: SIMD Considered Harmful Stephen Fuld <sfuld@alumni.cmu.edu.invalid> - 2026-09-24 07:54 -0700
Re: SIMD Considered Harmful MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-09-24 19:48 +0000
Re: SIMD Considered Harmful Stephen Fuld <sfuld@alumni.cmu.edu.invalid> - 2026-09-24 13:58 -0700
Re: SIMD Considered Harmful MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-09-24 22:24 +0000
Re: SIMD Considered Harmful MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-09-25 00:27 +0000
Re: SIMD Considered Harmful EricP <ThatWouldBeTelling@thevillage.com> - 2026-09-25 17:44 -0400
Re: SIMD Considered Harmful MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-09-25 23:03 +0000
Re: SIMD Considered Harmful EricP <ThatWouldBeTelling@thevillage.com> - 2026-09-27 11:33 -0400
Re: SIMD Considered Harmful MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-09-28 00:04 +0000
Re: SIMD Considered Harmful EricP <ThatWouldBeTelling@thevillage.com> - 2026-09-29 12:17 -0400
Re: SIMD Considered Harmful Thomas Koenig <tkoenig@netcologne.de> - 2026-09-26 11:26 +0000
Re: SIMD Considered Harmful Terje Mathisen <terje.mathisen@tmsw.no> - 2026-09-26 17:57 +0200
Re: SIMD Considered Harmful BGB <cr88192@gmail.com> - 2026-09-26 13:52 -0500
Re: SIMD Considered Harmful MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-09-26 21:24 +0000
Re: SIMD Considered Harmful MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-09-26 21:16 +0000
Re: SIMD Considered Harmful aph@littlepinkcloud.invalid - 2026-09-28 09:53 +0000
csiph-web