Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > sci.physics > #896463
| From | Mild Shock <janburse@fastmail.fm> |
|---|---|
| Newsgroups | sci.logic, comp.lang.prolog, sci.physics |
| Subject | How speed up π-WAM with vector operations (Re: The evolution of hardware and GPT-2 training) |
| Date | 2026-07-27 11:10 +0200 |
| Message-ID | <11477a4$fu5f$2@solani.org> (permalink) |
| References | (1 earlier) <1140bdm$bdpp$2@solani.org> <1140cld$bue1$2@solani.org> <1145hle$fdqk$2@solani.org> <11470l1$gapi$3@solani.org> <11476i7$ftjl$2@solani.org> |
Cross-posted to 3 groups.
Hi,
One could critisize that my π-WAM doesn't
utilize GPU to the fullest, since its GPU
backend prototype only uses scalar operations
and no vector or matrix operations. And
modern GPUs thrive on vector and matrix
operations. Especially matrix operations giving
a boost of a factor 15x or so. There are
many papers already showing how Prolog can be
mapped to matrix operations. Only this research
is completely ignored by Prolog systems such as
SICStus, Ciao, SWI, ECLiPSe etc.. But lets
illustrate what vector operations could do
for π-WAM, take this compilation of the Prolog
goal between(0,1023,X), Y is X*2+3:
int X;
int Y;
for (X=0; X < 1024; X++) {
Y=X*2+3;
[...]
}
With vector operations, and vectors of size
32 one could do:
int X1;
int[] X = new int[32];
int X3;
int[] Y = new int[32];
for (X1 = 0; X1 < 1024 / 32; X1++) {
for (int X2 = 0; X2 < 32; X2++)
X[X2] = X1*32+X2;
vec_mul_add(X, 2, 3, Y);
[..]
}
Have Fun!
Bye
Mild Shock schrieb:
> Hi,
>
> While Huggingfaces hired GG in 2026,
> AK was hired by Anthropic in 2026:
>
> Andrej Karpathy (born 23 October 1986[3])
> is a Slovak-Canadian AI researcher, who
> co-founded and formerly worked at OpenAI
> In 2026 he joined Anthropic as part of
> the pretraining team.
> https://en.wikipedia.org/wiki/Andrej_Karpathy
>
> But his nanochat archivement has an
> interesting time line:
>
> 168 hours , Original OpenAI GPT-2 checkpoint, 2019
> 3 hours , d24 baseline, slightly overtrained, Jan 29 2026
> 1 1/2 hour, autoresearch round 2, Mar 14 2026
> The best ChatGPT that $100 can buy.
> https://github.com/karpathy/nanochat
>
> But what hardware was the enabler. What is the
> NVIDIA H100 GPU even. Well the thingy is surely not
> a Budget Laptop, performance pretty much
>
> dependence on data elememt size, the H100 NVL
> version (*), and when using tensor operations,
> and not only scalar operations:
>
> 8-bit towards 3000 tera flops
> 16-bit towards 1500 tera flops
> 32-bit towards 900 tera flops
>
> Cool! I guess this experiment would tap into 60
> tera flops, since it only uses scalar operations so far:
>
> 11.4 Giga Lips with a Budget Laptop
> https://github.com/Jean-Luc-Picard-2021/gigabudget
>
> You could perform it by migration the web application
> using WebGPU into a node.js standalone application
> using the dawn library for GPU access.
>
> Bye
>
> (*)
> https://resources.nvidia.com/en-us-hopper-architecture/nvidia-tensor-core-gpu-datasheet
>
>
>
> Mild Shock schrieb:
>> Hi,
>>
>> Whats this "forget" trope of glue sniffing
>> Rossy Boy with his herpes blisters?
>>
>> > Bulgarians, that's some real Boris and Natasha crap,
>> > forget Hungarians and Bulgarians.
>>
>> Why should I forget Bulgarians,
>> they are never on my mind. Do you
>> see me doing ggml stuff?
>>
>> I only hypothesized that it is
>> over for Python as the machine
>> learning language or AI inferencing
>>
>> locally on AI laptops language, and
>> made the ggml case, so I already forgot
>> about them. Which might give you a glimps,
>>
>> why WebGPU was used for this here:
>>
>> 11.4 Giga Lips with a Budget Laptop
>> https://github.com/Jean-Luc-Picard-2021/gigabudget
>>
>> Is an interesting choice. Even
>> github has some Languages statistics,
>> giving an account what I used:
>>
>> HTML 67.5% JavaScript 23.1% CSS 9.4%
>>
>> Have Fun!
>>
>> Bye
>>
>> P.S.: The example below is not p-adics,
>> you complete imbecil moron. Its just:
>>
>> 7-11 cubic Solution by Pritchard & Gries
>> https://www.cs.cornell.edu/gries/TechReports/83-574.pdf
>>
>> Ross Finlayson schrieb:
>> > On 07/26/2026 10:52 AM, Mild Shock wrote:
>> >> Hi,
>> >>
>> >> You see it all boils down to find your inner peace
>> >> by an immaculate inception of some queue datatype.
>> >>
>> >> KOAN/Fortran-S was an early 1990s research programming
>> >> system for distributed-memory multiprocessors . Developed
>> >> at ENS Lyon in the early 1990s . Often listed alongside
>> >> other historical parallel programming efforts.
>> >>
>> >> The Message Passing: The research explicitly
>> >> compared the SVM approach against message passing
>> >> on the same hardware . The finding was that SVM
>> >> could achieve good performance without the low-level
>> >>
>> >> complexity of managing explicit messages, though
>> >> the best results often came from a hybrid approach (sic!)
>> >> Here is an interesting baseline, from Java,
>> >> a class ElevenSingle that only does:
>> >>
>> >> public static void run() {
>> >> for (int A = 1; A < 192; A++) {
>> >> int Y = (771-A)/3;
>> >> for (int B = A; B < Y; B++) {
>> >> int Z = (771-A-B)/2;
>> >> for (int C = B; C < Z; C++) {
>> >> int D = 711-A-B-C;
>> >> if (A*B*C == 711000000/D &&
>> >> 711000000 % D == 0)
>> >> System.out.println("A="+A+", B="+B+", C="+C+", D="+D);
>> >> }
>> >> }
>> >> }
>> >> }
>> >>
>> >> And then compare it to ElevenMulti, doing some
>> >> Work Balancing Scheduler Tetris Game with 8 cores:
>> >>
>> >> ElevenSingle
>> >> A=120, B=125, C=150, D=316
>> >> 6.628 ms
>> >>
>> >> ElevenMulti
>> >> A=120, B=125, C=150, D=316
>> >> 1.941 ms
>> >>
>> >> Not great, not terrible!
>> >>
>> >> Bye
>> >
>> > Oh, that's just "tricks of p-adic arithmetic".
>> >
>> > Like other sock-puppet howler trolls, when confronted
>> > with its base incredulity, it will descend to its
>> > lower levers of the pathos variety.
>> >
>> > You might be happier learning about Julia trees and
>> > raster ops, instead of shilling yet another Ramanujan
>> > series without saying how it's made.
>> >
>> > Bulgarians, that's some real Boris and Natasha crap,
>> > forget Hungarians and Bulgarians.
>> >
>> >
>>
>
Back to sci.physics | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
The Wuhan Virus that destroyed Python [ggml Manifesto] Mild Shock <janburse@fastmail.fm> - 2026-07-22 21:01 +0200
Deadlock Exorcism: Switch from Push to Pull [A pi-calculus Specification of Prolog] (Re: The Wuhan Virus that destroyed Python [ggml Manifesto]) Mild Shock <janburse@fastmail.fm> - 2026-07-23 00:25 +0200
Why do you even need a mpmc queue? [Thunder Kittens] (Re: Deadlock Exorcism: Switch from Push to Pull) Mild Shock <janburse@fastmail.fm> - 2026-07-23 08:45 +0200
Trivial balancing example for (int i=0; i<global_id; i++) (Re: Why do you even need a mpmc queue? [Thunder Kittens]) Mild Shock <janburse@fastmail.fm> - 2026-07-23 08:55 +0200
The Pixel Phone AI Experiment Song (Enqueue/dequeue need not be fast and can spinn ["fairness" questions]) Mild Shock <janburse@fastmail.fm> - 2026-07-23 09:19 +0200
Enqueue/dequeue need not be fast and can spinn ["fairness" questions] (Re: The Pixel Phone AI Experiment Song (Enqueue/dequeue need not be fast and can spinn ["fairness" questions]) Mild Shock <janburse@fastmail.fm> - 2026-07-23 09:23 +0200
And, where did I talk about rockets? [Hint its about xAI's Grok] (Re: Why do you even need a mpmc queue? [Thunder Kittens]) Mild Shock <janburse@fastmail.fm> - 2026-07-25 01:25 +0200
Why forget something, that was never on my mind (Re: And, where did I talk about rockets? [Hint its about xAI's Grok]) Mild Shock <janburse@fastmail.fm> - 2026-07-25 09:49 +0200
Example Mandel Brot rendering [Faster with MIMD] (Was: Why do you even need a mpmc queue? [Thunder Kittens]) Mild Shock <janburse@fastmail.fm> - 2026-07-25 09:56 +0200
Potential Python Recovery: Free Threading [3.13 release] (Re: The Wuhan Virus that destroyed Python [ggml Manifesto]) Mild Shock <janburse@fastmail.fm> - 2026-07-23 10:21 +0200
The things XILINX braught to the AMD table (Re: The Wuhan Virus that destroyed Python [ggml Manifesto]) Mild Shock <janburse@fastmail.fm> - 2026-07-23 18:48 +0200
NVIDIA evacuated its Chinese market [Tau Scaling] (Re: The things XILINX braught to the AMD table) Mild Shock <janburse@fastmail.fm> - 2026-07-23 19:13 +0200
Micro penis mother sung arias (Re: NVIDIA evacuated its Chinese market [Tau Scaling]) Mild Shock <janburse@fastmail.fm> - 2026-07-24 14:40 +0200
Micro penis brain is in constant hiatus (Re: Micro penis mother sung arias) Mild Shock <janburse@fastmail.fm> - 2026-07-24 15:27 +0200
Ignoramus or Ignorabimus: I don't care [(Re: Micro penis brain is in constant hiatus (Re: Micro penis mother sung arias) Mild Shock <janburse@fastmail.fm> - 2026-07-24 15:35 +0200
You are a moron, brainless putin payed (Re: Ignoramus or Ignorabimus: I don't care) Mild Shock <janburse@fastmail.fm> - 2026-07-24 18:01 +0200
Yeah keep reading my posts, uninspired fool (Re: You are a moron, brainless putin payed) Mild Shock <janburse@fastmail.fm> - 2026-07-24 19:47 +0200
Out of the blue accusation span 15 days [Empirical USENET study] (Re: Ignoramus or Ignorabimus: I don't care) Mild Shock <janburse@fastmail.fm> - 2026-07-24 20:27 +0200
Little Data Center on Your Palm [AI Laptops for 500 USD] (Re: The Wuhan Virus that destroyed Python [ggml Manifesto]) Mild Shock <janburse@fastmail.fm> - 2026-07-24 17:59 +0200
2008: 4 Blades + Tesla S1070 versus 2026: 1 AI Laptop (Re: Little Data Center on Your Palm [AI Laptops for 500 USD]) Mild Shock <janburse@fastmail.fm> - 2026-07-24 18:15 +0200
Hurry the blue bus doesnt stop indefinitely (Re: The Wuhan Virus that destroyed Python [ggml Manifesto]) Mild Shock <janburse@fastmail.fm> - 2026-07-24 20:37 +0200
Not SIMD, a MIMD design for NVIDIA Volta (Re: Hurry the blue bus doesnt stop indefinitely) Mild Shock <janburse@fastmail.fm> - 2026-07-24 20:58 +0200
Could take 3-4 months find machine / browser (Re: Not SIMD, a MIMD design for NVIDIA Volta) Mild Shock <janburse@fastmail.fm> - 2026-07-24 21:16 +0200
The Koan of pi-WAM queues [FORTRAN-S] (Re: Not SIMD, a MIMD design for NVIDIA Volta) Mild Shock <janburse@fastmail.fm> - 2026-07-26 19:54 +0200
The turbo capping of AI Laptops (Was: The Koan of pi-WAM queues [FORTRAN-S]) Mild Shock <janburse@fastmail.fm> - 2026-07-26 20:00 +0200
Re: The Koan of pi-WAM queues [FORTRAN-S] (Re: Not SIMD, a MIMD design for NVIDIA Volta) Mild Shock <janburse@fastmail.fm> - 2026-07-27 09:16 +0200
Why forget Bulgarians, never on my mind (Re: The Koan of pi-WAM queues [FORTRAN-S]) Mild Shock <janburse@fastmail.fm> - 2026-07-27 09:16 +0200
miniTriton CUDA is an alternative to torch variants (Re: Why forget Bulgarians, never on my mind) Mild Shock <janburse@fastmail.fm> - 2026-07-27 09:52 +0200
Andrej Karpathy original gangster of Budget Laptop (Re: miniTriton CUDA is an alternative to torch variants) Mild Shock <janburse@fastmail.fm> - 2026-07-27 09:54 +0200
The evolution of hardware and GPT-2 training (Re: Why forget Bulgarians, never on my mind) Mild Shock <janburse@fastmail.fm> - 2026-07-27 10:57 +0200
How speed up π-WAM with vector operations (Re: The evolution of hardware and GPT-2 training) Mild Shock <janburse@fastmail.fm> - 2026-07-27 11:10 +0200
AI accelerator extend from GPU to CPU [Zero Copying] (Re: How speed up π-WAM with vector operations) Mild Shock <janburse@fastmail.fm> - 2026-07-27 13:21 +0200
The invention of vector and matrix registers [NVIDIA Volta] (Re: AI accelerator extend from GPU to CPU [Zero Copying]) Mild Shock <janburse@fastmail.fm> - 2026-07-27 13:22 +0200
Re: The invention of vector and matrix registers [NVIDIA Volta] (Re: AI accelerator extend from GPU to CPU [Zero Copying]) Ross Finlayson <ross.a.finlayson@gmail.com> - 2026-07-27 07:34 -0700
Potato Computer owner impressed by Ukraine Tech [Rossy Boys Brother?] (Was: Hurry the blue bus doesnt stop indefinitely) Mild Shock <janburse@fastmail.fm> - 2026-07-27 16:56 +0200
csiph-web