Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > sci.math > #647013

The invention of vector and matrix registers [NVIDIA Volta] (Was: AI accelerator extend from GPU to CPU [Zero Copying])

From Mild Shock <janburse@fastmail.fm>
Newsgroups sci.math, sci.physics.relativity
Subject The invention of vector and matrix registers [NVIDIA Volta] (Was: AI accelerator extend from GPU to CPU [Zero Copying])
Date 2026-07-27 13:20 +0200
Message-ID <1147etb$glbs$1@solani.org> (permalink)
References (4 earlier) <ZDGdnbf3T4PLTPv3nZ2dnZfqnPudnZ2d@giganews.com> <11470h4$gapi$1@solani.org> <11476g4$ftjl$1@solani.org> <114776f$fu5f$1@solani.org> <11477vd$fup5$1@solani.org>

Cross-posted to 2 groups.

Show all headers | View raw

Hi,

But the example gives also way to vector
and matrix registers. The int[] X and
int[] Y could be also held in vector

registers. Compilers can also optimize
away int[] Y, and use a inline modification,
in case X isn't used later, then playing

the role of Y:

vec_mul_add(X, 2, 3, X)

Vector and matrix registers in modern GPUs
emerged from distinct architectural milestones:
vector-like register files developed with
early programmable 3D vertex/pixel pipelines

in the late 1990s to early 2000s. While
dedicated multi-dimensional matrix registers
(Tensor Cores/Matrix Cores) were invented by
NVIDIA in 2017, starting with the Tesla

V100 (Volta microarchitecture):

 From Volta To Blackwell
https://newsletter.semianalysis.com/p/nvidia-tensor-core-evolution-from-volta-to-blackwell

You see the scheduling of tensure core occupation
scheduling in the above article, including memory
and register flow, following the section:

MMA Instruction Overview

It went through a couple of generations, leading
to Tensor Memory (TMEM) and collective operations,
basically realizing the PIM idea:

Processing-in-Memory Tutorials
https://www.sigarch.org/processing-in-memory-tutorials-experiences-from-past-two-years-and-thoughts-looking-forward/

Have Fun!

Bye

Mild Shock schrieb:
> Hi,
> 
> The nice thing about AI accelerators, pioneered
> maybe by Apple Silicon and their unified memory.
> The AMD APU model can be extended so that
> 
> vector and matrix operations become uniformly
> available for GPU and CPU. With unified memory
> already a vector operation such as:
> 
> vec_mul_add(X, 2, 3, Y)
> 
> Only needs the X and Y address. But I havent
> got my head around yet how this is all organized.
> Maybe a GPU has still its own GEMM cores,
> 
> but you find Apple Silicon C++/C source code,
> that taps into vector and matrix operations
> by Zero Copying. The Copying is left to the DMA
> 
> of the vector or matrix operation. And moderated
> by the various caches. Leading to the slogan, that
> multiple floating point operations become zero cost:
> 
> Some teaching can be found here
> https://www.hpc-ch.org/category/topics/course-workshop/
> 
> Bye
> 
> Mild Shock schrieb:
>> Hi,
>>
>> One could critisize that my π-WAM doesn't
>> utilize GPU to the fullest, since its GPU
>> backend prototype only uses scalar operations
>>
>> and no vector or matrix operations. And
>> modern GPUs thrive on vector and matrix
>> operations. Especially matrix operations giving
>>
>> a boost of a factor 15x or so. There are
>> many papers already showing how Prolog can be
>> mapped to matrix operations. Only this research
>>
>> is completely ignored by Prolog systems such as
>> SICStus, Ciao, SWI, ECLiPSe etc.. But lets
>> illustrate what vector operations could do
>>
>> for π-WAM, take this compilation of the Prolog
>> goal between(0,1023,X), Y is X*2+3:
>>
>> int X;
>> int Y;
>> for (X=0; X < 1024; X++) {
>>      Y=X*2+3;
>>      [...]
>> }
>>
>> With vector operations, and vectors of size
>> 32 one could do:
>>
>> int X1;
>> int[] X = new int[32];
>> int X3;
>> int[] Y = new int[32];
>> for (X1 = 0; X1 < 1024 / 32; X1++) {
>>      for (int X2 = 0; X2 < 32; X2++)
>>         X[X2] = X1*32+X2;
>>      vec_mul_add(X, 2, 3, Y);
>>      [..]
>> }
>>
>> Have Fun!
>>
>> Bye
>>
>> Mild Shock schrieb:
>>> Hi,
>>>
>>> While Huggingfaces hired GG in 2026,
>>> AK was hired by Anthropic in 2026:
>>>
>>> Andrej Karpathy (born 23 October 1986[3])
>>> is a Slovak-Canadian AI researcher, who
>>> co-founded and formerly worked at OpenAI
>>> In 2026 he joined Anthropic as part of
>>> the pretraining team.
>>> https://en.wikipedia.org/wiki/Andrej_Karpathy
>>>
>>> But his nanochat archivement has an
>>> interesting time line:
>>>
>>> 168 hours , Original OpenAI GPT-2 checkpoint, 2019
>>> 3 hours , d24 baseline, slightly overtrained, Jan 29 2026
>>> 1 1/2 hour, autoresearch round 2, Mar 14 2026
>>> The best ChatGPT that $100 can buy.
>>> https://github.com/karpathy/nanochat
>>>
>>> But what hardware was the enabler. What is the
>>> NVIDIA H100 GPU even. Well the thingy is surely not
>>> a Budget Laptop, performance pretty much
>>>
>>> dependence on data elememt size, the H100 NVL
>>> version (*), and when using tensor operations,
>>> and not only scalar operations:
>>>
>>> 8-bit towards 3000 tera flops
>>> 16-bit towards 1500 tera flops
>>> 32-bit towards 900 tera flops
>>>
>>> Cool! I guess this experiment would tap into 60
>>> tera flops, since it only uses scalar operations so far:
>>>
>>> 11.4 Giga Lips with a Budget Laptop
>>> https://github.com/Jean-Luc-Picard-2021/gigabudget
>>>
>>> You could perform it by migration the web application
>>> using WebGPU into a node.js standalone application
>>> using the dawn library for GPU access.
>>>
>>> Bye
>>>
>>> (*) 
>>> https://resources.nvidia.com/en-us-hopper-architecture/nvidia-tensor-core-gpu-datasheet 
>>>
>>>
>>> Mild Shock schrieb:
>>>> Hi,
>>>>
>>>> Whats this "forget" trope of glue sniffing
>>>> Rossy Boy with his herpes blisters?
>>>>
>>>>  > Bulgarians, that's some real Boris and Natasha crap,
>>>>  > forget Hungarians and Bulgarians.
>>>>
>>>> Why should I forget Bulgarians,
>>>> they are never on my mind. Do you
>>>> see me doing ggml stuff?
>>>>
>>>> I only hypothesized that it is
>>>> over for Python as the machine
>>>> learning language or AI inferencing
>>>>
>>>> locally on AI laptops language, and
>>>> made the ggml case, so I already forgot
>>>> about them. Which might give you a glimps,
>>>>
>>>> why WebGPU was used for this here:
>>>>
>>>> 11.4 Giga Lips with a Budget Laptop
>>>> https://github.com/Jean-Luc-Picard-2021/gigabudget
>>>>
>>>> Is an interesting choice. Even
>>>> github has some Languages statistics,
>>>> giving an account what I used:
>>>>
>>>> HTML 67.5% JavaScript 23.1% CSS 9.4%
>>>>
>>>> Have Fun!
>>>>
>>>> Bye
>>>>
>>>> P.S.: The example below is not p-adics,
>>>> you complete imbecil moron. Its just:
>>>>
>>>> 7-11 cubic Solution by Pritchard & Gries
>>>> https://www.cs.cornell.edu/gries/TechReports/83-574.pdf
>>>>
>>>> Ross Finlayson schrieb:
>>>>> On 07/26/2026 10:52 AM, Mild Shock wrote:
>>>>>> Hi,
>>>>>>
>>>>>> You see it all boils down to find your inner peace
>>>>>> by an immaculate inception of some queue datatype.
>>>>>>
>>>>>> KOAN/Fortran-S was an early 1990s research programming
>>>>>> system for distributed-memory multiprocessors . Developed
>>>>>> at ENS Lyon in the early 1990s . Often listed alongside
>>>>>> other historical parallel programming efforts.
>>>>>>
>>>>>> The Message Passing: The research explicitly
>>>>>> compared the SVM approach against message passing
>>>>>> on the same hardware . The finding was that SVM
>>>>>> could achieve good performance without the low-level
>>>>>>
>>>>>> complexity of managing explicit messages, though
>>>>>> the best results often came from a hybrid approach (sic!)
>>>>>> Here is an interesting baseline, from Java,
>>>>>> a class ElevenSingle that only does:
>>>>>>
>>>>>>      public static void run() {
>>>>>>          for (int A = 1; A < 192; A++) {
>>>>>>              int Y = (771-A)/3;
>>>>>>              for (int B = A; B < Y; B++) {
>>>>>>                  int Z = (771-A-B)/2;
>>>>>>                  for (int C = B; C < Z; C++) {
>>>>>>                      int D = 711-A-B-C;
>>>>>>                      if (A*B*C == 711000000/D &&
>>>>>>                            711000000 % D == 0)
>>>>>>      System.out.println("A="+A+", B="+B+", C="+C+", D="+D);
>>>>>>                  }
>>>>>>              }
>>>>>>          }
>>>>>>      }
>>>>>>
>>>>>> And then compare it to ElevenMulti, doing some
>>>>>> Work Balancing Scheduler Tetris Game with 8 cores:
>>>>>>
>>>>>> ElevenSingle
>>>>>> A=120, B=125, C=150, D=316
>>>>>> 6.628 ms
>>>>>>
>>>>>> ElevenMulti
>>>>>> A=120, B=125, C=150, D=316
>>>>>> 1.941 ms
>>>>>>
>>>>>> Not great, not terrible!
>>>>>>
>>>>>> Bye
>>>>>
>>>>> Oh, that's just "tricks of p-adic arithmetic".
>>>>>
>>>>> Like other sock-puppet howler trolls, when confronted
>>>>> with its base incredulity, it will descend to its
>>>>> lower levers of the pathos variety.
>>>>>
>>>>> You might be happier learning about Julia trees and
>>>>> raster ops, instead of shilling yet another Ramanujan
>>>>> series without saying how it's made.
>>>>>
>>>>> Bulgarians, that's some real Boris and Natasha crap,
>>>>> forget Hungarians and Bulgarians.
>>>>>
>>>>>
>>>>
>>>
>>
>

Back to sci.math | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

The Wuhan Virus that destroyed Python [ggml Manifesto] Mild Shock <janburse@fastmail.fm> - 2026-07-22 21:00 +0200
  Deadlock Exorcism: Switch from Push to Pull [A pi-calculus Specification of Prolog] (Re: The Wuhan Virus that destroyed Python [ggml Manifesto]) Mild Shock <janburse@fastmail.fm> - 2026-07-23 00:23 +0200
    Why do you even need a mpmc queue? [Thunder Kittens] (Re: Deadlock Exorcism: Switch from Push to Pull) Mild Shock <janburse@fastmail.fm> - 2026-07-23 08:43 +0200
      Trivial balancing example for (int i=0; i<global_id; i++) (Re: Why do you even need a mpmc queue? [Thunder Kittens]) Mild Shock <janburse@fastmail.fm> - 2026-07-23 08:57 +0200
        Enqueue/dequeue need not be fast and can spinn ["fairness" questions] (Was: Trivial balancing example for (int i=0; i<global_id; i++)) Mild Shock <janburse@fastmail.fm> - 2026-07-23 09:11 +0200
          The Pixel Phone AI Experiment Song (Re: Enqueue/dequeue need not be fast and can spinn ["fairness" questions] ) Mild Shock <janburse@fastmail.fm> - 2026-07-23 09:21 +0200
      Re: Why do you even need a mpmc queue? [Thunder Kittens] (Re: Deadlock Exorcism: Switch from Push to Pull) Ross Finlayson <ross.a.finlayson@gmail.com> - 2026-07-23 08:24 -0700
  Potential Python Recovery: Free Threading [3.13 release] (Was: The Wuhan Virus that destroyed Python [ggml Manifesto]) Mild Shock <janburse@fastmail.fm> - 2026-07-23 10:19 +0200
    Re: Potential Python Recovery: Free Threading [3.13 release] (Was: The Wuhan Virus that destroyed Python [ggml Manifesto]) Ross Valikhanov <kavna@rl.ru> - 2026-07-23 16:01 +0000
  Re: The Wuhan Virus that destroyed Python [ggml Manifesto] Ramon Dubenkov <omd@nnk.ru> - 2026-07-23 13:38 +0000
  The things XILINX braught to the AMD table (Was: The Wuhan Virus that destroyed Python [ggml Manifesto]) Mild Shock <janburse@fastmail.fm> - 2026-07-23 18:47 +0200
    NIVIDIA evacuated its Chinese market [Tau Scaling] (Was: The things XILINX braught to the AMD table) Mild Shock <janburse@fastmail.fm> - 2026-07-23 19:11 +0200
    NVIDIA evacuated its Chinese market [Tau Scaling] (Re: The things XILINX braught to the AMD table) Mild Shock <janburse@fastmail.fm> - 2026-07-23 19:12 +0200
      Re: NVIDIA evacuated its Chinese market [Tau Scaling] (Re: The things XILINX braught to the AMD table) Lane W <cactus_DAC@yahoo.com> - 2026-07-23 11:22 -0600
        Micro penis mother sung arias (Was: NVIDIA evacuated its Chinese market [Tau Scaling]) Mild Shock <janburse@fastmail.fm> - 2026-07-24 14:38 +0200
          Re: Micro penis mother sung arias (Was: NVIDIA evacuated its Chinese market [Tau Scaling]) Lane W <cactus_DAC@yahoo.com> - 2026-07-24 07:15 -0600
            Micro penis brain is in constant hiatus (Was: Micro penis mother sung arias) Mild Shock <janburse@fastmail.fm> - 2026-07-24 15:24 +0200
              Re: Micro penis brain is in constant hiatus (Was: Micro penis mother sung arias) Mild Shock <janburse@fastmail.fm> - 2026-07-24 15:36 +0200
              Ignoramus or Ignorabimus: I don't care (π-WAM) (Re: Micro penis brain is in constant hiatus) Mild Shock <janburse@fastmail.fm> - 2026-07-24 15:38 +0200
                Re: Ignoramus or Ignorabimus: I don't care (π-WAM) (Re: Micro penis brain is in constant hiatus) Lane W <cactus_DAC@yahoo.com> - 2026-07-24 08:31 -0600
                You are a moron, brainless putin payed (Was: Ignoramus or Ignorabimus: I don't care (π-WAM)) Mild Shock <janburse@fastmail.fm> - 2026-07-24 18:01 +0200
                Re: You are a moron, brainless putin payed (Was: Ignoramus or Ignorabimus: I don't care (π-WAM)) Lane W <cactus_DAC@yahoo.com> - 2026-07-24 10:27 -0600
                Yeah keep reading my posts, uninspired fool (Was: You are a moron, brainless putin payed) Mild Shock <janburse@fastmail.fm> - 2026-07-24 19:45 +0200
                Re: Yeah keep reading my posts, uninspired fool (Was: You are a moron, brainless putin payed) Lane W <cactus_DAC@yahoo.com> - 2026-07-24 12:11 -0600
                LoL (Was: Yeah keep reading my posts, uninspired fool ) Mild Shock <janburse@fastmail.fm> - 2026-07-24 20:12 +0200
                Re: LoL (Was: Yeah keep reading my posts, uninspired fool ) Lane W <cactus_DAC@yahoo.com> - 2026-07-24 12:53 -0600
                Out of the blue accusation span 15 days [Empirical USENET study] (Was: Ignoramus or Ignorabimus: I don't care (π-WAM)) Mild Shock <janburse@fastmail.fm> - 2026-07-24 20:26 +0200
  Little Data Center on Your Palm [AI Laptops for 500 USD] (Was: The Wuhan Virus that destroyed Python [ggml Manifesto] Mild Shock <janburse@fastmail.fm> - 2026-07-24 17:58 +0200
    2008: 4 Blades + Tesla S1070 versus 2026: 1 AI Laptop (Re: Little Data Center on Your Palm [AI Laptops for 500 USD]) Mild Shock <janburse@fastmail.fm> - 2026-07-24 18:16 +0200
    Re: Little Data Center on Your Palm [AI Laptops for 500 USD] (Was: The Wuhan Virus that destroyed Python [ggml Manifesto] Bradford Babkoff <ffb@odbb.ru> - 2026-07-24 18:05 +0000
      LoL (Was: Little Data Center on Your Palm [AI Laptops for 500 USD]) Mild Shock <janburse@fastmail.fm> - 2026-07-24 20:11 +0200
  Hurry the blue bus doesnt stop indefinitely (Was: The Wuhan Virus that destroyed Python [ggml Manifesto]) Mild Shock <janburse@fastmail.fm> - 2026-07-24 20:36 +0200
    Not SIMD, a MIMD design for NVIDIA Volta (Re: Hurry the blue bus doesnt stop indefinitely) Mild Shock <janburse@fastmail.fm> - 2026-07-24 20:57 +0200
      Could take 3-4 months find machine / browser (Was Not SIMD, a MIMD design for NVIDIA Volta) Mild Shock <janburse@fastmail.fm> - 2026-07-24 21:15 +0200
      The Koan of pi-WAM queues [FORTRAN-S] (Was: Not SIMD, a MIMD design for NVIDIA Volta) Mild Shock <janburse@fastmail.fm> - 2026-07-26 19:52 +0200
        The turbo capping of AI Laptops (Re: The Koan of pi-WAM queues [FORTRAN-S]) Mild Shock <janburse@fastmail.fm> - 2026-07-26 20:01 +0200
        Re: The Koan of pi-WAM queues [FORTRAN-S] (Was: Not SIMD, a MIMD design for NVIDIA Volta) Ross Finlayson <ross.a.finlayson@gmail.com> - 2026-07-26 20:33 -0700
          Why forget Bulgarians, never on my mind (Re: The Koan of pi-WAM queues [FORTRAN-S] (Was: Not SIMD, a MIMD design for NVIDIA Volta) Mild Shock <janburse@fastmail.fm> - 2026-07-27 09:14 +0200
            miniTriton CUDA is an alternative to torch variants (Was: Why forget Bulgarians, never on my mind) Mild Shock <janburse@fastmail.fm> - 2026-07-27 09:40 +0200
              Andrej Karpathy original gangster of Budget Laptop (Was: miniTriton CUDA is an alternative to torch variants) Mild Shock <janburse@fastmail.fm> - 2026-07-27 09:51 +0200
                Re: Andrej Karpathy original gangster of Budget Laptop (Was: miniTriton CUDA is an alternative to torch variants) Ross Finlayson <ross.a.finlayson@gmail.com> - 2026-07-27 01:41 -0700
            Re: Why forget Bulgarians, never on my mind (Re: The Koan of pi-WAM queues [FORTRAN-S] (Was: Not SIMD, a MIMD design for NVIDIA Volta) Ross Finlayson <ross.a.finlayson@gmail.com> - 2026-07-27 01:38 -0700
            The evolution of hardware and GPT-2 training (Was: Why forget Bulgarians, never on my mind) Mild Shock <janburse@fastmail.fm> - 2026-07-27 10:56 +0200
              How speed up π-WAM with vector operations (Was: The evolution of hardware and GPT-2 training) Mild Shock <janburse@fastmail.fm> - 2026-07-27 11:08 +0200
                AI accelerator extend from GPU to CPU [Zero Copying] (Was: How speed up π-WAM with vector operations) Mild Shock <janburse@fastmail.fm> - 2026-07-27 11:21 +0200
                The invention of vector and matrix registers [NVIDIA Volta] (Was: AI accelerator extend from GPU to CPU [Zero Copying]) Mild Shock <janburse@fastmail.fm> - 2026-07-27 13:20 +0200
                Maybe they should have named it NVIDIA Einstein [Rossy Boy Toe Sucking] (Re: The invention of vector and matrix registers [NVIDIA Volta] (Was: AI accelerator extend from GPU to CPU [Zero Copying]) Mild Shock <janburse@fastmail.fm> - 2026-07-27 17:14 +0200
                π-WAM is not adding decimals, it is removing decimals (Re: The invention of vector and matrix registers [NVIDIA Volta]) Mild Shock <janburse@fastmail.fm> - 2026-07-27 18:36 +0200
                In Budget Laptops the TOPS come with low energy footprint (Was: π-WAM is not adding decimals, it is removing decimals) Mild Shock <janburse@fastmail.fm> - 2026-07-27 18:44 +0200
    Potato Computer owner impressed by Ukraine Tech [Rossy Boys Brother?] (Re: Hurry the blue bus doesnt stop indefinitely) Mild Shock <janburse@fastmail.fm> - 2026-07-27 16:57 +0200
  Got it. Or are you too stupid? [New Usenet Mantra] (Re: The Wuhan Virus that destroyed Python [ggml Manifesto]) Mild Shock <janburse@fastmail.fm> - 2026-07-27 19:00 +0200
    Re: Got it. Or are you too stupid? [New Usenet Mantra] (Re: The Wuhan Virus that destroyed Python [ggml Manifesto]) Kim Baitchorov <bvhkoc@bmc.ru> - 2026-07-27 22:38 +0000

csiph-web