Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > sci.physics > #896394 > unrolled thread

The Wuhan Virus that destroyed Python [ggml Manifesto]

Back to article view | Back to sci.physics

Contents

  The Wuhan Virus that destroyed Python [ggml Manifesto] Mild Shock <janburse@fastmail.fm> - 2026-07-22 21:01 +0200
    Deadlock Exorcism: Switch from Push to Pull [A pi-calculus Specification of Prolog] (Re: The Wuhan Virus that destroyed Python [ggml Manifesto]) Mild Shock <janburse@fastmail.fm> - 2026-07-23 00:25 +0200
      Why do you even need a mpmc queue? [Thunder Kittens] (Re: Deadlock Exorcism: Switch from Push to Pull) Mild Shock <janburse@fastmail.fm> - 2026-07-23 08:45 +0200
        Trivial balancing example for (int i=0; i<global_id; i++) (Re: Why do you even need a mpmc queue? [Thunder Kittens]) Mild Shock <janburse@fastmail.fm> - 2026-07-23 08:55 +0200
          The Pixel Phone AI Experiment Song (Enqueue/dequeue need not be fast and can spinn ["fairness" questions]) Mild Shock <janburse@fastmail.fm> - 2026-07-23 09:19 +0200
            Enqueue/dequeue need not be fast and can spinn ["fairness" questions] (Re: The Pixel Phone AI Experiment Song (Enqueue/dequeue need not be fast and can spinn ["fairness" questions]) Mild Shock <janburse@fastmail.fm> - 2026-07-23 09:23 +0200
        And, where did I talk about rockets? [Hint its about xAI's Grok] (Re: Why do you even need a mpmc queue? [Thunder Kittens]) Mild Shock <janburse@fastmail.fm> - 2026-07-25 01:25 +0200
          Why forget something, that was never on my mind (Re: And, where did I talk about rockets? [Hint its about xAI's Grok]) Mild Shock <janburse@fastmail.fm> - 2026-07-25 09:49 +0200
        Example Mandel Brot rendering [Faster with MIMD] (Was: Why do you even need a mpmc queue? [Thunder Kittens]) Mild Shock <janburse@fastmail.fm> - 2026-07-25 09:56 +0200
    Potential Python Recovery: Free Threading [3.13 release] (Re: The Wuhan Virus that destroyed Python [ggml Manifesto]) Mild Shock <janburse@fastmail.fm> - 2026-07-23 10:21 +0200
    The things XILINX braught to the AMD table (Re: The Wuhan Virus that destroyed Python [ggml Manifesto]) Mild Shock <janburse@fastmail.fm> - 2026-07-23 18:48 +0200
      NVIDIA evacuated its Chinese market [Tau Scaling] (Re: The things XILINX braught to the AMD table) Mild Shock <janburse@fastmail.fm> - 2026-07-23 19:13 +0200
        Micro penis mother sung arias (Re: NVIDIA evacuated its Chinese market [Tau Scaling]) Mild Shock <janburse@fastmail.fm> - 2026-07-24 14:40 +0200
          Micro penis brain is in constant hiatus (Re: Micro penis mother sung arias) Mild Shock <janburse@fastmail.fm> - 2026-07-24 15:27 +0200
            Ignoramus or Ignorabimus: I don't care [(Re: Micro penis brain is in constant hiatus (Re: Micro penis mother sung arias) Mild Shock <janburse@fastmail.fm> - 2026-07-24 15:35 +0200
              You are a moron, brainless putin payed (Re: Ignoramus or Ignorabimus: I don't care) Mild Shock <janburse@fastmail.fm> - 2026-07-24 18:01 +0200
                Yeah keep reading my posts, uninspired fool (Re: You are a moron, brainless putin payed) Mild Shock <janburse@fastmail.fm> - 2026-07-24 19:47 +0200
              Out of the blue accusation span 15 days [Empirical USENET study] (Re: Ignoramus or Ignorabimus: I don't care) Mild Shock <janburse@fastmail.fm> - 2026-07-24 20:27 +0200
    Little Data Center on Your Palm [AI Laptops for 500 USD] (Re: The Wuhan Virus that destroyed Python [ggml Manifesto]) Mild Shock <janburse@fastmail.fm> - 2026-07-24 17:59 +0200
      2008: 4 Blades + Tesla S1070 versus 2026: 1 AI Laptop (Re: Little Data Center on Your Palm [AI Laptops for 500 USD]) Mild Shock <janburse@fastmail.fm> - 2026-07-24 18:15 +0200
    Hurry the blue bus doesnt stop indefinitely (Re: The Wuhan Virus that destroyed Python [ggml Manifesto]) Mild Shock <janburse@fastmail.fm> - 2026-07-24 20:37 +0200
      Not SIMD, a MIMD design for NVIDIA Volta (Re: Hurry the blue bus doesnt stop indefinitely) Mild Shock <janburse@fastmail.fm> - 2026-07-24 20:58 +0200
        Could take 3-4 months find machine / browser (Re: Not SIMD, a MIMD design for NVIDIA Volta) Mild Shock <janburse@fastmail.fm> - 2026-07-24 21:16 +0200

Page 1 of 2  [1] 2  Next page →

#896394 — The Wuhan Virus that destroyed Python [ggml Manifesto]

Hi,

Remember when first all local AI was Python
and PyTorch APIs. And then suddently people started
using bare metal C/C++ Code. Here is the story:

How it started:

GPT-J or GPT-J-6B is an open-source large
language model (LLM) developed by EleutherAI
in 2021. As the name suggests, it is a
generative pre-trained transformer model
designed to produce human-like text that
continues from a prompt.
https://www.eleuther.ai/

How it was going [Georgi Gerganov]:

So a few days later comes out the LLaMA, I do
some calculations and I figure out “Okay, 65
billion parameters. You probably need about
40 gigs of RAM, with 4-bit quantization. So
this can run on a MacBook. Why not do it?”

Why I was able to do it so quickly - basically,
for all that I saw it’s pretty much GPT-J architecture
with some modifications, like some extra memorization
layers. It’s minor changes. Basically, again, the
existing code for the GPT-J, I just simply
modified it there, it happened pretty quickly.
https://changelog.com/podcast/532

Georgi Gerganov, Bulgarian, now with Hugging
Face, ggml-cann also running on Chinese AI chips.
ggml Manifesto https://github.com/ggml-org/ggml

Bye

[toc] | [next] | [standalone]

#896400 — Deadlock Exorcism: Switch from Push to Pull [A pi-calculus Specification of Prolog] (Re: The Wuhan Virus that destroyed Python [ggml Manifesto])

Hi,

This is quite fun, how some TLA+ guy fears
the full state of queue like the devil in
itself. But I guess if a service rate is

low and the producer has not much to do to
produce its work items, the arrival rate
has nevertheless to adapt, and dealing

with "full states", which are wrongly
called deadlock here, is the normal:

Tutorial-style talk - BlockingQueue
https://github.com/lemmy/BlockingQueue/tree/main

Prolog is in good position. The bird box
model has a redo port. So sometimes switching
from push to pull, can help without doing

Deadlock Exorcism. You can also translate
the bird box ports into pi-calculus:

A pi-calculus Specification of Prolog
Benjamin Z. Li - University of Pennsylvania
11 Apr 1994, European Symposium on Programming,
Prolog, Unification, Backtracking
https://scispace.com/pdf/a-pi-calculus-specification-of-prolog-3qf2pf04ud.pdf

Have Fun!

Bye

Mild Shock schrieb:
> Hi,
> 
> Remember when first all local AI was Python
> and PyTorch APIs. And then suddently people started
> using bare metal C/C++ Code. Here is the story:
> 
> How it started:
> 
> GPT-J or GPT-J-6B is an open-source large
> language model (LLM) developed by EleutherAI
> in 2021. As the name suggests, it is a
> generative pre-trained transformer model
> designed to produce human-like text that
> continues from a prompt.
> https://www.eleuther.ai/
> 
> How it was going [Georgi Gerganov]:
> 
> So a few days later comes out the LLaMA, I do
> some calculations and I figure out “Okay, 65
> billion parameters. You probably need about
> 40 gigs of RAM, with 4-bit quantization. So
> this can run on a MacBook. Why not do it?”
> 
> Why I was able to do it so quickly - basically,
> for all that I saw it’s pretty much GPT-J architecture
> with some modifications, like some extra memorization
> layers. It’s minor changes. Basically, again, the
> existing code for the GPT-J, I just simply
> modified it there, it happened pretty quickly.
> https://changelog.com/podcast/532
> 
> Georgi Gerganov, Bulgarian, now with Hugging
> Face, ggml-cann also running on Chinese AI chips.
> ggml Manifesto https://github.com/ggml-org/ggml
> 
> Bye

[toc] | [prev] | [next] | [standalone]

#896409 — Why do you even need a mpmc queue? [Thunder Kittens] (Re: Deadlock Exorcism: Switch from Push to Pull)

Hi,

Because I use WebGPU and not WebGL. And
because WebGPU can adresss modern GPU
developed with the NVIDIA Volta evolution,

which happened in 2017. Namley that compute
shaders are not any more subject to the
realization restriction of lock step

execution, but have independent thread state.
And because there is independent thread state
there is also independent time spent for a

a work item by each logical thread, if the
submitted logical thread uses a lot of branching
logic or even loops. But the use of branching

and loops is encouraged in independent thread
state programming of compute shaders. The variables
that can drive such logic are the scalar variables:

Tour of WGSL - Control Flow
https://google.github.io/tour-of-wgsl/control-flow/

Then not to waste GPU compute time, by logical
threads doing nothing. You will need to
introduce some load balancing among multiple

logical threads. And MPMC queues are one way to
readize load balancing. Compute shaders with
producer and consumer entry points are proposed

as fundamental architecture by Thunder Kittens:

ThunderKittens: Simple, Fast, and Adorable AI Kernels
https://arxiv.org/abs/2410.20399

They are used by this SpaceX acquisition:

Composer 2 Technical Report
https://arxiv.org/abs/2603.24477

Thunder Kittens uses Hardware support, i.e. tma_expect().

Bye

Chris M. Thomasson schrieb:
 >> never meant to be used in a GPU.
 >> Dmitry CAS version can be used, but
 >>
 >> Why do you even need a mpmc queue
 >> in your compute shader anyway?

Mild Shock schrieb:
> Hi,
> 
> This is quite fun, how some TLA+ guy fears
> the full state of queue like the devil in
> itself. But I guess if a service rate is
> 
> low and the producer has not much to do to
> produce its work items, the arrival rate
> has nevertheless to adapt, and dealing
> 
> with "full states", which are wrongly
> called deadlock here, is the normal:
> 
> Tutorial-style talk - BlockingQueue
> https://github.com/lemmy/BlockingQueue/tree/main
> 
> Prolog is in good position. The bird box
> model has a redo port. So sometimes switching
> from push to pull, can help without doing
> 
> Deadlock Exorcism. You can also translate
> the bird box ports into pi-calculus:
> 
> A pi-calculus Specification of Prolog
> Benjamin Z. Li - University of Pennsylvania
> 11 Apr 1994, European Symposium on Programming,
> Prolog, Unification, Backtracking
> https://scispace.com/pdf/a-pi-calculus-specification-of-prolog-3qf2pf04ud.pdf 
> 
> 
> Have Fun!
> 
> Bye
> 
> Mild Shock schrieb:
>> Hi,
>>
>> Remember when first all local AI was Python
>> and PyTorch APIs. And then suddently people started
>> using bare metal C/C++ Code. Here is the story:
>>
>> How it started:
>>
>> GPT-J or GPT-J-6B is an open-source large
>> language model (LLM) developed by EleutherAI
>> in 2021. As the name suggests, it is a
>> generative pre-trained transformer model
>> designed to produce human-like text that
>> continues from a prompt.
>> https://www.eleuther.ai/
>>
>> How it was going [Georgi Gerganov]:
>>
>> So a few days later comes out the LLaMA, I do
>> some calculations and I figure out “Okay, 65
>> billion parameters. You probably need about
>> 40 gigs of RAM, with 4-bit quantization. So
>> this can run on a MacBook. Why not do it?”
>>
>> Why I was able to do it so quickly - basically,
>> for all that I saw it’s pretty much GPT-J architecture
>> with some modifications, like some extra memorization
>> layers. It’s minor changes. Basically, again, the
>> existing code for the GPT-J, I just simply
>> modified it there, it happened pretty quickly.
>> https://changelog.com/podcast/532
>>
>> Georgi Gerganov, Bulgarian, now with Hugging
>> Face, ggml-cann also running on Chinese AI chips.
>> ggml Manifesto https://github.com/ggml-org/ggml
>>
>> Bye
> 

[toc] | [prev] | [next] | [standalone]

#896410 — Trivial balancing example for (int i=0; i<global_id; i++) (Re: Why do you even need a mpmc queue? [Thunder Kittens])

Hi,

Is a trivial control construct for(),
when used in a compute shader with
NVIDIA Volta evolution, i.e. MIMD,

can lead to different time spend by
individual compute shaders:

fn main(global_id : i32) {
    i : i32 = 0;
    while (i < globa_id) {
       i++;
    }
}

You can visiualize as the time spent
by each logical thread as follows:

global id, logical thread life line
1     [    ]
2     [        ]
3     [            ]
4     [                ]
5     [                    ]
Etc..

With work items and load balancing you
could run the above with a lower number
of logical threads, I am writing the

work item number now inside the sub life
line inside the overall life line of
the logical thread:

worker , worker work items
A     [3           ]
B     [4               ][2       ]
C     [5                   ][1   ]

The overall time slightly increased by 1,
i.e. the case global_id = k combined
with the case global_id = n-k+1 . Also

one worker didn't have two work items,
only one work item. But the number of
logical threads needed was halfed.

Ok, a mpmc queue will be not that
intelligent, concerning the work sheduling.
But one could experiment with mpmc queue

priority queues etc.. etc..

Have Fun!

Bye

Mild Shock schrieb:
> Hi,
> 
> Because I use WebGPU and not WebGL. And
> because WebGPU can adresss modern GPU
> developed with the NVIDIA Volta evolution,
> 
> which happened in 2017. Namley that compute
> shaders are not any more subject to the
> realization restriction of lock step
> 
> execution, but have independent thread state.
> And because there is independent thread state
> there is also independent time spent for a
> 
> a work item by each logical thread, if the
> submitted logical thread uses a lot of branching
> logic or even loops. But the use of branching
> 
> and loops is encouraged in independent thread
> state programming of compute shaders. The variables
> that can drive such logic are the scalar variables:
> 
> Tour of WGSL - Control Flow
> https://google.github.io/tour-of-wgsl/control-flow/
> 
> Then not to waste GPU compute time, by logical
> threads doing nothing. You will need to
> introduce some load balancing among multiple
> 
> logical threads. And MPMC queues are one way to
> readize load balancing. Compute shaders with
> producer and consumer entry points are proposed
> 
> as fundamental architecture by Thunder Kittens:
> 
> ThunderKittens: Simple, Fast, and Adorable AI Kernels
> https://arxiv.org/abs/2410.20399
> 
> They are used by this SpaceX acquisition:
> 
> Composer 2 Technical Report
> https://arxiv.org/abs/2603.24477
> 
> Thunder Kittens uses Hardware support, i.e. tma_expect().
> 
> Bye
> 
> Chris M. Thomasson schrieb:
>  >> never meant to be used in a GPU.
>  >> Dmitry CAS version can be used, but
>  >>
>  >> Why do you even need a mpmc queue
>  >> in your compute shader anyway?
> 
> Mild Shock schrieb:
>> Hi,
>>
>> This is quite fun, how some TLA+ guy fears
>> the full state of queue like the devil in
>> itself. But I guess if a service rate is
>>
>> low and the producer has not much to do to
>> produce its work items, the arrival rate
>> has nevertheless to adapt, and dealing
>>
>> with "full states", which are wrongly
>> called deadlock here, is the normal:
>>
>> Tutorial-style talk - BlockingQueue
>> https://github.com/lemmy/BlockingQueue/tree/main
>>
>> Prolog is in good position. The bird box
>> model has a redo port. So sometimes switching
>> from push to pull, can help without doing
>>
>> Deadlock Exorcism. You can also translate
>> the bird box ports into pi-calculus:
>>
>> A pi-calculus Specification of Prolog
>> Benjamin Z. Li - University of Pennsylvania
>> 11 Apr 1994, European Symposium on Programming,
>> Prolog, Unification, Backtracking
>> https://scispace.com/pdf/a-pi-calculus-specification-of-prolog-3qf2pf04ud.pdf 
>>
>>
>> Have Fun!
>>
>> Bye
>>
>> Mild Shock schrieb:
>>> Hi,
>>>
>>> Remember when first all local AI was Python
>>> and PyTorch APIs. And then suddently people started
>>> using bare metal C/C++ Code. Here is the story:
>>>
>>> How it started:
>>>
>>> GPT-J or GPT-J-6B is an open-source large
>>> language model (LLM) developed by EleutherAI
>>> in 2021. As the name suggests, it is a
>>> generative pre-trained transformer model
>>> designed to produce human-like text that
>>> continues from a prompt.
>>> https://www.eleuther.ai/
>>>
>>> How it was going [Georgi Gerganov]:
>>>
>>> So a few days later comes out the LLaMA, I do
>>> some calculations and I figure out “Okay, 65
>>> billion parameters. You probably need about
>>> 40 gigs of RAM, with 4-bit quantization. So
>>> this can run on a MacBook. Why not do it?”
>>>
>>> Why I was able to do it so quickly - basically,
>>> for all that I saw it’s pretty much GPT-J architecture
>>> with some modifications, like some extra memorization
>>> layers. It’s minor changes. Basically, again, the
>>> existing code for the GPT-J, I just simply
>>> modified it there, it happened pretty quickly.
>>> https://changelog.com/podcast/532
>>>
>>> Georgi Gerganov, Bulgarian, now with Hugging
>>> Face, ggml-cann also running on Chinese AI chips.
>>> ggml Manifesto https://github.com/ggml-org/ggml
>>>
>>> Bye
>>
> 

[toc] | [prev] | [next] | [standalone]

#896411 — The Pixel Phone AI Experiment Song (Enqueue/dequeue need not be fast and can spinn ["fairness" questions])

Hi,

For graphics, rendering, in the worst case your
FPS can go down. Then you have a 2x as performant
graphic card, and sundently the FPS is ok again!

Or you render into a smaller screen, with less
number of pixels, and things turn good again.
So for my pixel phone AI experiment, or what I

will toy around on a AI laptop, I am singing:

   I'm a spinner, I'm a sinner
   I spin on CAS loops for my dinner
   Some call it busy-wait, I call it fate
   When the queue is empty, I just rotate

Bye

Whats better Steve Miller or Muddy Waters?

The Joker
https://www.youtube.com/watch?v=dV3AziKTBUo

Hoochie Coochie Man
https://www.youtube.com/watch?v=e_l6A7krjrQ

Mild Shock schrieb:
> Hi,
> 
> Is a trivial control construct for(),
> when used in a compute shader with
> NVIDIA Volta evolution, i.e. MIMD,
> 
> can lead to different time spend by
> individual compute shaders:
> 
> fn main(global_id : i32) {
>     i : i32 = 0;
>     while (i < globa_id) {
>        i++;
>     }
> }
> 
> You can visiualize as the time spent
> by each logical thread as follows:
> 
> global id, logical thread life line
> 1     [    ]
> 2     [        ]
> 3     [            ]
> 4     [                ]
> 5     [                    ]
> Etc..
> 
> With work items and load balancing you
> could run the above with a lower number
> of logical threads, I am writing the
> 
> work item number now inside the sub life
> line inside the overall life line of
> the logical thread:
> 
> worker , worker work items
> A     [3           ]
> B     [4               ][2       ]
> C     [5                   ][1   ]
> 
> The overall time slightly increased by 1,
> i.e. the case global_id = k combined
> with the case global_id = n-k+1 . Also
> 
> one worker didn't have two work items,
> only one work item. But the number of
> logical threads needed was halfed.
> 
> Ok, a mpmc queue will be not that
> intelligent, concerning the work sheduling.
> But one could experiment with mpmc queue
> 
> priority queues etc.. etc..
> 
> Have Fun!
> 
> Bye
> 
> Mild Shock schrieb:
>> Hi,
>>
>> Because I use WebGPU and not WebGL. And
>> because WebGPU can adresss modern GPU
>> developed with the NVIDIA Volta evolution,
>>
>> which happened in 2017. Namley that compute
>> shaders are not any more subject to the
>> realization restriction of lock step
>>
>> execution, but have independent thread state.
>> And because there is independent thread state
>> there is also independent time spent for a
>>
>> a work item by each logical thread, if the
>> submitted logical thread uses a lot of branching
>> logic or even loops. But the use of branching
>>
>> and loops is encouraged in independent thread
>> state programming of compute shaders. The variables
>> that can drive such logic are the scalar variables:
>>
>> Tour of WGSL - Control Flow
>> https://google.github.io/tour-of-wgsl/control-flow/
>>
>> Then not to waste GPU compute time, by logical
>> threads doing nothing. You will need to
>> introduce some load balancing among multiple
>>
>> logical threads. And MPMC queues are one way to
>> readize load balancing. Compute shaders with
>> producer and consumer entry points are proposed
>>
>> as fundamental architecture by Thunder Kittens:
>>
>> ThunderKittens: Simple, Fast, and Adorable AI Kernels
>> https://arxiv.org/abs/2410.20399
>>
>> They are used by this SpaceX acquisition:
>>
>> Composer 2 Technical Report
>> https://arxiv.org/abs/2603.24477
>>
>> Thunder Kittens uses Hardware support, i.e. tma_expect().
>>
>> Bye
>>
>> Chris M. Thomasson schrieb:
>>  >> never meant to be used in a GPU.
>>  >> Dmitry CAS version can be used, but
>>  >>
>>  >> Why do you even need a mpmc queue
>>  >> in your compute shader anyway?
>>
>> Mild Shock schrieb:
>>> Hi,
>>>
>>> This is quite fun, how some TLA+ guy fears
>>> the full state of queue like the devil in
>>> itself. But I guess if a service rate is
>>>
>>> low and the producer has not much to do to
>>> produce its work items, the arrival rate
>>> has nevertheless to adapt, and dealing
>>>
>>> with "full states", which are wrongly
>>> called deadlock here, is the normal:
>>>
>>> Tutorial-style talk - BlockingQueue
>>> https://github.com/lemmy/BlockingQueue/tree/main
>>>
>>> Prolog is in good position. The bird box
>>> model has a redo port. So sometimes switching
>>> from push to pull, can help without doing
>>>
>>> Deadlock Exorcism. You can also translate
>>> the bird box ports into pi-calculus:
>>>
>>> A pi-calculus Specification of Prolog
>>> Benjamin Z. Li - University of Pennsylvania
>>> 11 Apr 1994, European Symposium on Programming,
>>> Prolog, Unification, Backtracking
>>> https://scispace.com/pdf/a-pi-calculus-specification-of-prolog-3qf2pf04ud.pdf 
>>>
>>>
>>> Have Fun!
>>>
>>> Bye
>>>
>>> Mild Shock schrieb:
>>>> Hi,
>>>>
>>>> Remember when first all local AI was Python
>>>> and PyTorch APIs. And then suddently people started
>>>> using bare metal C/C++ Code. Here is the story:
>>>>
>>>> How it started:
>>>>
>>>> GPT-J or GPT-J-6B is an open-source large
>>>> language model (LLM) developed by EleutherAI
>>>> in 2021. As the name suggests, it is a
>>>> generative pre-trained transformer model
>>>> designed to produce human-like text that
>>>> continues from a prompt.
>>>> https://www.eleuther.ai/
>>>>
>>>> How it was going [Georgi Gerganov]:
>>>>
>>>> So a few days later comes out the LLaMA, I do
>>>> some calculations and I figure out “Okay, 65
>>>> billion parameters. You probably need about
>>>> 40 gigs of RAM, with 4-bit quantization. So
>>>> this can run on a MacBook. Why not do it?”
>>>>
>>>> Why I was able to do it so quickly - basically,
>>>> for all that I saw it’s pretty much GPT-J architecture
>>>> with some modifications, like some extra memorization
>>>> layers. It’s minor changes. Basically, again, the
>>>> existing code for the GPT-J, I just simply
>>>> modified it there, it happened pretty quickly.
>>>> https://changelog.com/podcast/532
>>>>
>>>> Georgi Gerganov, Bulgarian, now with Hugging
>>>> Face, ggml-cann also running on Chinese AI chips.
>>>> ggml Manifesto https://github.com/ggml-org/ggml
>>>>
>>>> Bye
>>>
>>
> 

[toc] | [prev] | [next] | [standalone]

#896412 — Enqueue/dequeue need not be fast and can spinn ["fairness" questions] (Re: The Pixel Phone AI Experiment Song (Enqueue/dequeue need not be fast and can spinn ["fairness" questions])

Hi,

Since the main goal is to change the
geometry of how work items are assigned
and processed, and since work item processing

costs much more time then queue API calls.
It is irrelevant whether the queue API are
ultra fast or not, or whether they spin or

not. Usually one can affort both things,
since balancing outweights these small extra
costs or silly occupation of a platform thread.

It could be also feasible, in the total
balance sheet of execution and energy consumption
of your work items along workers to use

Atomic.pause() equivalents, special instructions,
for spinning. This will be added in 2027 to JavaScript:

For example, in Intel x86, it may be a
pause instruction as per Intel's optimization manual.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Atomics/pause

But I have to see what works and what doesn't
work. The above are only theoretical
considerations about the friction that queues

induce. Usually friction hurts much less
than balant scheduling mistakes, i.e. unfavorable
schedulings. golang is very much bugged by

unfavorable schedulings. While on paper golang
looks like brilliant idea, examples that create
similar patterns of runtimes as in the below

for (int i=0; i<global_id; i++) can create
horrible scheduling and very bad an erratic
behaviour, especially if the synchronization

mechanism and backpressure signals have to travel
long chains of channel waits. So you will get not
into the terrain of "dead lock" questions, but

basically enter the terrain of "fairness" questions.

Bye

Mild Shock schrieb:
> Hi,
> 
> For graphics, rendering, in the worst case your
> FPS can go down. Then you have a 2x as performant
> graphic card, and sundently the FPS is ok again!
> 
> Or you render into a smaller screen, with less
> number of pixels, and things turn good again.
> So for my pixel phone AI experiment, or what I
> 
> will toy around on a AI laptop, I am singing:
> 
>    I'm a spinner, I'm a sinner
>    I spin on CAS loops for my dinner
>    Some call it busy-wait, I call it fate
>    When the queue is empty, I just rotate
> 
> Bye
> 
> Whats better Steve Miller or Muddy Waters?
> 
> The Joker
> https://www.youtube.com/watch?v=dV3AziKTBUo
> 
> Hoochie Coochie Man
> https://www.youtube.com/watch?v=e_l6A7krjrQ
> 
> Mild Shock schrieb:
>> Hi,
>>
>> Is a trivial control construct for(),
>> when used in a compute shader with
>> NVIDIA Volta evolution, i.e. MIMD,
>>
>> can lead to different time spend by
>> individual compute shaders:
>>
>> fn main(global_id : i32) {
>>     i : i32 = 0;
>>     while (i < globa_id) {
>>        i++;
>>     }
>> }
>>
>> You can visiualize as the time spent
>> by each logical thread as follows:
>>
>> global id, logical thread life line
>> 1     [    ]
>> 2     [        ]
>> 3     [            ]
>> 4     [                ]
>> 5     [                    ]
>> Etc..
>>
>> With work items and load balancing you
>> could run the above with a lower number
>> of logical threads, I am writing the
>>
>> work item number now inside the sub life
>> line inside the overall life line of
>> the logical thread:
>>
>> worker , worker work items
>> A     [3           ]
>> B     [4               ][2       ]
>> C     [5                   ][1   ]
>>
>> The overall time slightly increased by 1,
>> i.e. the case global_id = k combined
>> with the case global_id = n-k+1 . Also
>>
>> one worker didn't have two work items,
>> only one work item. But the number of
>> logical threads needed was halfed.
>>
>> Ok, a mpmc queue will be not that
>> intelligent, concerning the work sheduling.
>> But one could experiment with mpmc queue
>>
>> priority queues etc.. etc..
>>
>> Have Fun!
>>
>> Bye
>>
>> Mild Shock schrieb:
>>> Hi,
>>>
>>> Because I use WebGPU and not WebGL. And
>>> because WebGPU can adresss modern GPU
>>> developed with the NVIDIA Volta evolution,
>>>
>>> which happened in 2017. Namley that compute
>>> shaders are not any more subject to the
>>> realization restriction of lock step
>>>
>>> execution, but have independent thread state.
>>> And because there is independent thread state
>>> there is also independent time spent for a
>>>
>>> a work item by each logical thread, if the
>>> submitted logical thread uses a lot of branching
>>> logic or even loops. But the use of branching
>>>
>>> and loops is encouraged in independent thread
>>> state programming of compute shaders. The variables
>>> that can drive such logic are the scalar variables:
>>>
>>> Tour of WGSL - Control Flow
>>> https://google.github.io/tour-of-wgsl/control-flow/
>>>
>>> Then not to waste GPU compute time, by logical
>>> threads doing nothing. You will need to
>>> introduce some load balancing among multiple
>>>
>>> logical threads. And MPMC queues are one way to
>>> readize load balancing. Compute shaders with
>>> producer and consumer entry points are proposed
>>>
>>> as fundamental architecture by Thunder Kittens:
>>>
>>> ThunderKittens: Simple, Fast, and Adorable AI Kernels
>>> https://arxiv.org/abs/2410.20399
>>>
>>> They are used by this SpaceX acquisition:
>>>
>>> Composer 2 Technical Report
>>> https://arxiv.org/abs/2603.24477
>>>
>>> Thunder Kittens uses Hardware support, i.e. tma_expect().
>>>
>>> Bye
>>>
>>> Chris M. Thomasson schrieb:
>>>  >> never meant to be used in a GPU.
>>>  >> Dmitry CAS version can be used, but
>>>  >>
>>>  >> Why do you even need a mpmc queue
>>>  >> in your compute shader anyway?
>>>
>>> Mild Shock schrieb:
>>>> Hi,
>>>>
>>>> This is quite fun, how some TLA+ guy fears
>>>> the full state of queue like the devil in
>>>> itself. But I guess if a service rate is
>>>>
>>>> low and the producer has not much to do to
>>>> produce its work items, the arrival rate
>>>> has nevertheless to adapt, and dealing
>>>>
>>>> with "full states", which are wrongly
>>>> called deadlock here, is the normal:
>>>>
>>>> Tutorial-style talk - BlockingQueue
>>>> https://github.com/lemmy/BlockingQueue/tree/main
>>>>
>>>> Prolog is in good position. The bird box
>>>> model has a redo port. So sometimes switching
>>>> from push to pull, can help without doing
>>>>
>>>> Deadlock Exorcism. You can also translate
>>>> the bird box ports into pi-calculus:
>>>>
>>>> A pi-calculus Specification of Prolog
>>>> Benjamin Z. Li - University of Pennsylvania
>>>> 11 Apr 1994, European Symposium on Programming,
>>>> Prolog, Unification, Backtracking
>>>> https://scispace.com/pdf/a-pi-calculus-specification-of-prolog-3qf2pf04ud.pdf 
>>>>
>>>>
>>>> Have Fun!
>>>>
>>>> Bye
>>>>
>>>> Mild Shock schrieb:
>>>>> Hi,
>>>>>
>>>>> Remember when first all local AI was Python
>>>>> and PyTorch APIs. And then suddently people started
>>>>> using bare metal C/C++ Code. Here is the story:
>>>>>
>>>>> How it started:
>>>>>
>>>>> GPT-J or GPT-J-6B is an open-source large
>>>>> language model (LLM) developed by EleutherAI
>>>>> in 2021. As the name suggests, it is a
>>>>> generative pre-trained transformer model
>>>>> designed to produce human-like text that
>>>>> continues from a prompt.
>>>>> https://www.eleuther.ai/
>>>>>
>>>>> How it was going [Georgi Gerganov]:
>>>>>
>>>>> So a few days later comes out the LLaMA, I do
>>>>> some calculations and I figure out “Okay, 65
>>>>> billion parameters. You probably need about
>>>>> 40 gigs of RAM, with 4-bit quantization. So
>>>>> this can run on a MacBook. Why not do it?”
>>>>>
>>>>> Why I was able to do it so quickly - basically,
>>>>> for all that I saw it’s pretty much GPT-J architecture
>>>>> with some modifications, like some extra memorization
>>>>> layers. It’s minor changes. Basically, again, the
>>>>> existing code for the GPT-J, I just simply
>>>>> modified it there, it happened pretty quickly.
>>>>> https://changelog.com/podcast/532
>>>>>
>>>>> Georgi Gerganov, Bulgarian, now with Hugging
>>>>> Face, ggml-cann also running on Chinese AI chips.
>>>>> ggml Manifesto https://github.com/ggml-org/ggml
>>>>>
>>>>> Bye
>>>>
>>>
>>
> 

[toc] | [prev] | [next] | [standalone]

#896437 — And, where did I talk about rockets? [Hint its about xAI's Grok] (Re: Why do you even need a mpmc queue? [Thunder Kittens])

Hi,

The primary motivation behind SpaceX's acquisition
of Cursor (and its underlying Composer model tech stack)
comes down to vertical integration of the entire AI
stack—compute, models, and application surface.

Bridging the Coding & Reasoning Gap: While xAI’s Grok
had massive raw compute backing it (via massive clusters
like Colossus), its training data heavily leaned on
social media feeds from X.

That made it great for conversational chat, but left
it lagging behind competitors (like OpenAI and Anthropic)
in pure coding, reasoning efficiency, and long-horizon
agentic execution.

Owning the Developer Workflow: Instead of just building
a model and hoping people use it, acquiring Cursor
gives SpaceX the premier developer-facing application
layer that millions of developers already use
for daily coding.

Coupling Cursor's model architecture with SpaceX/xAI's
massive compute infrastructure allows them to aggressively
scale up training for next-gen models.

So its a Win-Win. Got it?

Bye

Mild Shock schrieb:
> Hi,
> 
> Because I use WebGPU and not WebGL. And
> because WebGPU can adresss modern GPU
> developed with the NVIDIA Volta evolution,
> 
> which happened in 2017. Namley that compute
> shaders are not any more subject to the
> realization restriction of lock step
> 
> execution, but have independent thread state.
> And because there is independent thread state
> there is also independent time spent for a
> 
> a work item by each logical thread, if the
> submitted logical thread uses a lot of branching
> logic or even loops. But the use of branching
> 
> and loops is encouraged in independent thread
> state programming of compute shaders. The variables
> that can drive such logic are the scalar variables:
> 
> Tour of WGSL - Control Flow
> https://google.github.io/tour-of-wgsl/control-flow/
> 
> Then not to waste GPU compute time, by logical
> threads doing nothing. You will need to
> introduce some load balancing among multiple
> 
> logical threads. And MPMC queues are one way to
> readize load balancing. Compute shaders with
> producer and consumer entry points are proposed
> 
> as fundamental architecture by Thunder Kittens:
> 
> ThunderKittens: Simple, Fast, and Adorable AI Kernels
> https://arxiv.org/abs/2410.20399
> 
> They are used by this SpaceX acquisition:
> 
> Composer 2 Technical Report
> https://arxiv.org/abs/2603.24477
> 
> Thunder Kittens uses Hardware support, i.e. tma_expect().
> 
> Bye
> 
> Chris M. Thomasson schrieb:
>  >> never meant to be used in a GPU.
>  >> Dmitry CAS version can be used, but
>  >>
>  >> Why do you even need a mpmc queue
>  >> in your compute shader anyway?
> 
> Mild Shock schrieb:
>> Hi,
>>
>> This is quite fun, how some TLA+ guy fears
>> the full state of queue like the devil in
>> itself. But I guess if a service rate is
>>
>> low and the producer has not much to do to
>> produce its work items, the arrival rate
>> has nevertheless to adapt, and dealing
>>
>> with "full states", which are wrongly
>> called deadlock here, is the normal:
>>
>> Tutorial-style talk - BlockingQueue
>> https://github.com/lemmy/BlockingQueue/tree/main
>>
>> Prolog is in good position. The bird box
>> model has a redo port. So sometimes switching
>> from push to pull, can help without doing
>>
>> Deadlock Exorcism. You can also translate
>> the bird box ports into pi-calculus:
>>
>> A pi-calculus Specification of Prolog
>> Benjamin Z. Li - University of Pennsylvania
>> 11 Apr 1994, European Symposium on Programming,
>> Prolog, Unification, Backtracking
>> https://scispace.com/pdf/a-pi-calculus-specification-of-prolog-3qf2pf04ud.pdf 
>>
>>
>> Have Fun!
>>
>> Bye
>>
>> Mild Shock schrieb:
>>> Hi,
>>>
>>> Remember when first all local AI was Python
>>> and PyTorch APIs. And then suddently people started
>>> using bare metal C/C++ Code. Here is the story:
>>>
>>> How it started:
>>>
>>> GPT-J or GPT-J-6B is an open-source large
>>> language model (LLM) developed by EleutherAI
>>> in 2021. As the name suggests, it is a
>>> generative pre-trained transformer model
>>> designed to produce human-like text that
>>> continues from a prompt.
>>> https://www.eleuther.ai/
>>>
>>> How it was going [Georgi Gerganov]:
>>>
>>> So a few days later comes out the LLaMA, I do
>>> some calculations and I figure out “Okay, 65
>>> billion parameters. You probably need about
>>> 40 gigs of RAM, with 4-bit quantization. So
>>> this can run on a MacBook. Why not do it?”
>>>
>>> Why I was able to do it so quickly - basically,
>>> for all that I saw it’s pretty much GPT-J architecture
>>> with some modifications, like some extra memorization
>>> layers. It’s minor changes. Basically, again, the
>>> existing code for the GPT-J, I just simply
>>> modified it there, it happened pretty quickly.
>>> https://changelog.com/podcast/532
>>>
>>> Georgi Gerganov, Bulgarian, now with Hugging
>>> Face, ggml-cann also running on Chinese AI chips.
>>> ggml Manifesto https://github.com/ggml-org/ggml
>>>
>>> Bye
>>
> 

[toc] | [prev] | [next] | [standalone]

#896441 — Why forget something, that was never on my mind (Re: And, where did I talk about rockets? [Hint its about xAI's Grok])

Hi,

Why I forget something, that was never
on my mind. This here is hardly
about rockets:

11.4 Giga Lips with a Budget Laptop
https://github.com/Jean-Luc-Picard-2021/gigabudget

So stop glue sniffing. Got it? Your are
just confused. I don't care about SpaceX,
that Composer was acquired by

SpaceX was just a factual thing.

Bye

Disclaimer: Who knows, maybe somebody picks
up pi-calculus and/or WAM for rocket engineering,
I do not primarily exclude it.

But its not on my mind, so I cannot forget something
which I don't care about.

Ross Finlayson schrieb:
 > Forget Space-X and forget that heil-throwing fat-ass, too,
 > the rockets they bought are wearing out and the ones they
 > built are blowing up, moving fast and breaking things.

Mild Shock schrieb:
> Hi,
> 
> The primary motivation behind SpaceX's acquisition
> of Cursor (and its underlying Composer model tech stack)
> comes down to vertical integration of the entire AI
> stack—compute, models, and application surface.
> 
> Bridging the Coding & Reasoning Gap: While xAI’s Grok
> had massive raw compute backing it (via massive clusters
> like Colossus), its training data heavily leaned on
> social media feeds from X.
> 
> That made it great for conversational chat, but left
> it lagging behind competitors (like OpenAI and Anthropic)
> in pure coding, reasoning efficiency, and long-horizon
> agentic execution.
> 
> Owning the Developer Workflow: Instead of just building
> a model and hoping people use it, acquiring Cursor
> gives SpaceX the premier developer-facing application
> layer that millions of developers already use
> for daily coding.
> 
> Coupling Cursor's model architecture with SpaceX/xAI's
> massive compute infrastructure allows them to aggressively
> scale up training for next-gen models.
> 
> So its a Win-Win. Got it?
> 
> Bye
> 
> Mild Shock schrieb:
>> Hi,
>>
>> Because I use WebGPU and not WebGL. And
>> because WebGPU can adresss modern GPU
>> developed with the NVIDIA Volta evolution,
>>
>> which happened in 2017. Namley that compute
>> shaders are not any more subject to the
>> realization restriction of lock step
>>
>> execution, but have independent thread state.
>> And because there is independent thread state
>> there is also independent time spent for a
>>
>> a work item by each logical thread, if the
>> submitted logical thread uses a lot of branching
>> logic or even loops. But the use of branching
>>
>> and loops is encouraged in independent thread
>> state programming of compute shaders. The variables
>> that can drive such logic are the scalar variables:
>>
>> Tour of WGSL - Control Flow
>> https://google.github.io/tour-of-wgsl/control-flow/
>>
>> Then not to waste GPU compute time, by logical
>> threads doing nothing. You will need to
>> introduce some load balancing among multiple
>>
>> logical threads. And MPMC queues are one way to
>> readize load balancing. Compute shaders with
>> producer and consumer entry points are proposed
>>
>> as fundamental architecture by Thunder Kittens:
>>
>> ThunderKittens: Simple, Fast, and Adorable AI Kernels
>> https://arxiv.org/abs/2410.20399
>>
>> They are used by this SpaceX acquisition:
>>
>> Composer 2 Technical Report
>> https://arxiv.org/abs/2603.24477
>>
>> Thunder Kittens uses Hardware support, i.e. tma_expect().
>>
>> Bye
>>
>> Chris M. Thomasson schrieb:
>>  >> never meant to be used in a GPU.
>>  >> Dmitry CAS version can be used, but
>>  >>
>>  >> Why do you even need a mpmc queue
>>  >> in your compute shader anyway?
>>
>> Mild Shock schrieb:
>>> Hi,
>>>
>>> This is quite fun, how some TLA+ guy fears
>>> the full state of queue like the devil in
>>> itself. But I guess if a service rate is
>>>
>>> low and the producer has not much to do to
>>> produce its work items, the arrival rate
>>> has nevertheless to adapt, and dealing
>>>
>>> with "full states", which are wrongly
>>> called deadlock here, is the normal:
>>>
>>> Tutorial-style talk - BlockingQueue
>>> https://github.com/lemmy/BlockingQueue/tree/main
>>>
>>> Prolog is in good position. The bird box
>>> model has a redo port. So sometimes switching
>>> from push to pull, can help without doing
>>>
>>> Deadlock Exorcism. You can also translate
>>> the bird box ports into pi-calculus:
>>>
>>> A pi-calculus Specification of Prolog
>>> Benjamin Z. Li - University of Pennsylvania
>>> 11 Apr 1994, European Symposium on Programming,
>>> Prolog, Unification, Backtracking
>>> https://scispace.com/pdf/a-pi-calculus-specification-of-prolog-3qf2pf04ud.pdf 
>>>
>>>
>>> Have Fun!
>>>
>>> Bye
>>>
>>> Mild Shock schrieb:
>>>> Hi,
>>>>
>>>> Remember when first all local AI was Python
>>>> and PyTorch APIs. And then suddently people started
>>>> using bare metal C/C++ Code. Here is the story:
>>>>
>>>> How it started:
>>>>
>>>> GPT-J or GPT-J-6B is an open-source large
>>>> language model (LLM) developed by EleutherAI
>>>> in 2021. As the name suggests, it is a
>>>> generative pre-trained transformer model
>>>> designed to produce human-like text that
>>>> continues from a prompt.
>>>> https://www.eleuther.ai/
>>>>
>>>> How it was going [Georgi Gerganov]:
>>>>
>>>> So a few days later comes out the LLaMA, I do
>>>> some calculations and I figure out “Okay, 65
>>>> billion parameters. You probably need about
>>>> 40 gigs of RAM, with 4-bit quantization. So
>>>> this can run on a MacBook. Why not do it?”
>>>>
>>>> Why I was able to do it so quickly - basically,
>>>> for all that I saw it’s pretty much GPT-J architecture
>>>> with some modifications, like some extra memorization
>>>> layers. It’s minor changes. Basically, again, the
>>>> existing code for the GPT-J, I just simply
>>>> modified it there, it happened pretty quickly.
>>>> https://changelog.com/podcast/532
>>>>
>>>> Georgi Gerganov, Bulgarian, now with Hugging
>>>> Face, ggml-cann also running on Chinese AI chips.
>>>> ggml Manifesto https://github.com/ggml-org/ggml
>>>>
>>>> Bye
>>>
>>
> 

[toc] | [prev] | [next] | [standalone]

#896442 — Example Mandel Brot rendering [Faster with MIMD] (Was: Why do you even need a mpmc queue? [Thunder Kittens])

Hi,

Mostlikely this is faster with MIMD,
than with SIMD. Draw a Mandel Brot Figure:

mandelbrot
https://compute.toys/view/213

Why, because each pixel has a different
result i, since this loop has a break:

     for (i = 0; i < count; i++) {
         if (z_re * z_re + z_im * z_im > 4.0) {
             break;
         }
         let new_re = z_re * z_re - z_im * z_im;
         let new_im = 2.0f * z_re * z_im;
         z_re = c_re + new_re;
         z_im = c_im + new_im;
     }

So a sheduler balancer, that uses independent
thread state (MIMD), from a post NVIDIA Volta
type GPU, can squeeze out more computation,

than a lock step (SIMD) scheduler, from a
pre NVIDIA Volta GPU. I guess I will use that
as a balancing example for pi-WAM.

Bye

Mild Shock schrieb:
> Hi,
> 
> Because I use WebGPU and not WebGL. And
> because WebGPU can adresss modern GPU
> developed with the NVIDIA Volta evolution,
> 
> which happened in 2017. Namley that compute
> shaders are not any more subject to the
> realization restriction of lock step
> 
> execution, but have independent thread state.
> And because there is independent thread state
> there is also independent time spent for a
> 
> a work item by each logical thread, if the
> submitted logical thread uses a lot of branching
> logic or even loops. But the use of branching
> 
> and loops is encouraged in independent thread
> state programming of compute shaders. The variables
> that can drive such logic are the scalar variables:
> 
> Tour of WGSL - Control Flow
> https://google.github.io/tour-of-wgsl/control-flow/
> 
> Then not to waste GPU compute time, by logical
> threads doing nothing. You will need to
> introduce some load balancing among multiple
> 
> logical threads. And MPMC queues are one way to
> readize load balancing. Compute shaders with
> producer and consumer entry points are proposed
> 
> as fundamental architecture by Thunder Kittens:
> 
> ThunderKittens: Simple, Fast, and Adorable AI Kernels
> https://arxiv.org/abs/2410.20399
> 
> They are used by this SpaceX acquisition:
> 
> Composer 2 Technical Report
> https://arxiv.org/abs/2603.24477
> 
> Thunder Kittens uses Hardware support, i.e. tma_expect().
> 
> Bye
> 
> Chris M. Thomasson schrieb:
>  >> never meant to be used in a GPU.
>  >> Dmitry CAS version can be used, but
>  >>
>  >> Why do you even need a mpmc queue
>  >> in your compute shader anyway?
> 
> Mild Shock schrieb:
>> Hi,
>>
>> This is quite fun, how some TLA+ guy fears
>> the full state of queue like the devil in
>> itself. But I guess if a service rate is
>>
>> low and the producer has not much to do to
>> produce its work items, the arrival rate
>> has nevertheless to adapt, and dealing
>>
>> with "full states", which are wrongly
>> called deadlock here, is the normal:
>>
>> Tutorial-style talk - BlockingQueue
>> https://github.com/lemmy/BlockingQueue/tree/main
>>
>> Prolog is in good position. The bird box
>> model has a redo port. So sometimes switching
>> from push to pull, can help without doing
>>
>> Deadlock Exorcism. You can also translate
>> the bird box ports into pi-calculus:
>>
>> A pi-calculus Specification of Prolog
>> Benjamin Z. Li - University of Pennsylvania
>> 11 Apr 1994, European Symposium on Programming,
>> Prolog, Unification, Backtracking
>> https://scispace.com/pdf/a-pi-calculus-specification-of-prolog-3qf2pf04ud.pdf 
>>
>>
>> Have Fun!
>>
>> Bye
>>
>> Mild Shock schrieb:
>>> Hi,
>>>
>>> Remember when first all local AI was Python
>>> and PyTorch APIs. And then suddently people started
>>> using bare metal C/C++ Code. Here is the story:
>>>
>>> How it started:
>>>
>>> GPT-J or GPT-J-6B is an open-source large
>>> language model (LLM) developed by EleutherAI
>>> in 2021. As the name suggests, it is a
>>> generative pre-trained transformer model
>>> designed to produce human-like text that
>>> continues from a prompt.
>>> https://www.eleuther.ai/
>>>
>>> How it was going [Georgi Gerganov]:
>>>
>>> So a few days later comes out the LLaMA, I do
>>> some calculations and I figure out “Okay, 65
>>> billion parameters. You probably need about
>>> 40 gigs of RAM, with 4-bit quantization. So
>>> this can run on a MacBook. Why not do it?”
>>>
>>> Why I was able to do it so quickly - basically,
>>> for all that I saw it’s pretty much GPT-J architecture
>>> with some modifications, like some extra memorization
>>> layers. It’s minor changes. Basically, again, the
>>> existing code for the GPT-J, I just simply
>>> modified it there, it happened pretty quickly.
>>> https://changelog.com/podcast/532
>>>
>>> Georgi Gerganov, Bulgarian, now with Hugging
>>> Face, ggml-cann also running on Chinese AI chips.
>>> ggml Manifesto https://github.com/ggml-org/ggml
>>>
>>> Bye
>>
> 

[toc] | [prev] | [next] | [standalone]

#896413 — Potential Python Recovery: Free Threading [3.13 release] (Re: The Wuhan Virus that destroyed Python [ggml Manifesto])

Hi,

For those who didn't get it. Its not about programming
languages Python versus C/C++ per se. Its about the
runtime enviroments they deliver and/or require.

Python is traditionally bugged by the GIL (Global
Interpreter Lock). I tested this here with Python,
using just a porting of the Java class Thread:

Parallel π-WAM: 1.7 Giga Lips on a CPU
https://medium.com/@janburse_2989/parallel-%CF%80-wam-1-9-giga-lips-on-a-cpu-8a984e75af44

It didn't work. Horrible performance of threading.Thread.
But most of AI does utilize both CPU and GPU, and
bare metal access to system threads, with the avoidance

some annoying GIL nonsense, is essential. But there
is a silver lining on the horizon. Namely free
threading, Starting with the 3.13 release,

but I havent tested it yet:

Python support for free threading
https://docs.python.org/3/howto/free-threading-python.html

Bye

Mild Shock schrieb:
> Hi,
> 
> Remember when first all local AI was Python
> and PyTorch APIs. And then suddently people started
> using bare metal C/C++ Code. Here is the story:
> 
> How it started:
> 
> GPT-J or GPT-J-6B is an open-source large
> language model (LLM) developed by EleutherAI
> in 2021. As the name suggests, it is a
> generative pre-trained transformer model
> designed to produce human-like text that
> continues from a prompt.
> https://www.eleuther.ai/
> 
> How it was going [Georgi Gerganov]:
> 
> So a few days later comes out the LLaMA, I do
> some calculations and I figure out “Okay, 65
> billion parameters. You probably need about
> 40 gigs of RAM, with 4-bit quantization. So
> this can run on a MacBook. Why not do it?”
> 
> Why I was able to do it so quickly - basically,
> for all that I saw it’s pretty much GPT-J architecture
> with some modifications, like some extra memorization
> layers. It’s minor changes. Basically, again, the
> existing code for the GPT-J, I just simply
> modified it there, it happened pretty quickly.
> https://changelog.com/podcast/532
> 
> Georgi Gerganov, Bulgarian, now with Hugging
> Face, ggml-cann also running on Chinese AI chips.
> ggml Manifesto https://github.com/ggml-org/ggml
> 
> Bye

[toc] | [prev] | [next] | [standalone]

#896415 — The things XILINX braught to the AMD table (Re: The Wuhan Virus that destroyed Python [ggml Manifesto])

Hi,

While HBM and RDMA happen outside of a the main
silicon chip. Amazing things are now happening
inside a silicon chip as found in AI laptops.

Basically XILINX later acquired by AMD, had
already the Versal architecture. Where FGPA was
used to custom wire chips. The Versal area
had already Network-on-Chip (NoC):
https://www.adiuvoengineering.com/post/microzed-chronicles-versal-part-two-device-architecture

While a Ryzen AI 7 350 /w Radeon 860M does not
really have a versal area anymore. But the
Network-on-Chip (NoC) survived, with twist:

GEMM Performance Generations of Ryzen AI NPUs
4.3 On-The-Fly Tensor Transformations
We extensively exploit the multi-dimensional
addressing feature of DMAs to reorganize data into
tiled layouts, as needed by the NPU cores.
https://arxiv.org/abs/2512.13282v1

Bye

Mild Shock schrieb:
> Hi,
> 
> Remember when first all local AI was Python
> and PyTorch APIs. And then suddently people started
> using bare metal C/C++ Code. Here is the story:
> 
> How it started:
> 
> GPT-J or GPT-J-6B is an open-source large
> language model (LLM) developed by EleutherAI
> in 2021. As the name suggests, it is a
> generative pre-trained transformer model
> designed to produce human-like text that
> continues from a prompt.
> https://www.eleuther.ai/
> 
> How it was going [Georgi Gerganov]:
> 
> So a few days later comes out the LLaMA, I do
> some calculations and I figure out “Okay, 65
> billion parameters. You probably need about
> 40 gigs of RAM, with 4-bit quantization. So
> this can run on a MacBook. Why not do it?”
> 
> Why I was able to do it so quickly - basically,
> for all that I saw it’s pretty much GPT-J architecture
> with some modifications, like some extra memorization
> layers. It’s minor changes. Basically, again, the
> existing code for the GPT-J, I just simply
> modified it there, it happened pretty quickly.
> https://changelog.com/podcast/532
> 
> Georgi Gerganov, Bulgarian, now with Hugging
> Face, ggml-cann also running on Chinese AI chips.
> ggml Manifesto https://github.com/ggml-org/ggml
> 
> Bye

[toc] | [prev] | [next] | [standalone]

#896417 — NVIDIA evacuated its Chinese market [Tau Scaling] (Re: The things XILINX braught to the AMD table)

Hi,

How it started:

Captain: Throw the switch, Scotty!
Enterprise: Cloaking Device makes it invisible
Spock: Military secrets are the most fleeting of all.
Kirk Escapes the Romulans - The Enterprise Incident
https://www.youtube.com/watch?v=AusAGjwlql8

How its going:

CEO Jensen Huang said the company has “largely
conceded” China’s artificial intelligence chip
market to Huawei, as U.S. export restrictions
continue to reshape the global AI semiconductor landscape.
https://www.cnbc.com/2026/05/21/nvidia-jensen-huang-china-ai-chip-market-huawei.html

Bye

P.S.: What does China do?

Huawei’s semiconductor chief He Tingbo at the IEEE
ISCAS 2026 conference, Huawei's Tau Scaling Law is a newly
introduced semiconductor design framework that
shifts the industry’s optimization focus from
geometric scaling (shrinking physical transistor
sizes) to temporal scaling (compressing signal
propagation delay).
Nvidia Gave Up China - 4 Days Later THIS Happened
https://www.youtube.com/watch?v=dLLw-qADKSU

Mild Shock schrieb:
> Hi,
> 
> While HBM and RDMA happen outside of a the main
> silicon chip. Amazing things are now happening
> inside a silicon chip as found in AI laptops.
> 
> Basically XILINX later acquired by AMD, had
> already the Versal architecture. Where FGPA was
> used to custom wire chips. The Versal area
> had already Network-on-Chip (NoC):
> https://www.adiuvoengineering.com/post/microzed-chronicles-versal-part-two-device-architecture 
> 
> 
> While a Ryzen AI 7 350 /w Radeon 860M does not
> really have a versal area anymore. But the
> Network-on-Chip (NoC) survived, with twist:
> 
> GEMM Performance Generations of Ryzen AI NPUs
> 4.3 On-The-Fly Tensor Transformations
> We extensively exploit the multi-dimensional
> addressing feature of DMAs to reorganize data into
> tiled layouts, as needed by the NPU cores.
> https://arxiv.org/abs/2512.13282v1
> 
> Bye
> 
> Mild Shock schrieb:
>> Hi,
>>
>> Remember when first all local AI was Python
>> and PyTorch APIs. And then suddently people started
>> using bare metal C/C++ Code. Here is the story:
>>
>> How it started:
>>
>> GPT-J or GPT-J-6B is an open-source large
>> language model (LLM) developed by EleutherAI
>> in 2021. As the name suggests, it is a
>> generative pre-trained transformer model
>> designed to produce human-like text that
>> continues from a prompt.
>> https://www.eleuther.ai/
>>
>> How it was going [Georgi Gerganov]:
>>
>> So a few days later comes out the LLaMA, I do
>> some calculations and I figure out “Okay, 65
>> billion parameters. You probably need about
>> 40 gigs of RAM, with 4-bit quantization. So
>> this can run on a MacBook. Why not do it?”
>>
>> Why I was able to do it so quickly - basically,
>> for all that I saw it’s pretty much GPT-J architecture
>> with some modifications, like some extra memorization
>> layers. It’s minor changes. Basically, again, the
>> existing code for the GPT-J, I just simply
>> modified it there, it happened pretty quickly.
>> https://changelog.com/podcast/532
>>
>> Georgi Gerganov, Bulgarian, now with Hugging
>> Face, ggml-cann also running on Chinese AI chips.
>> ggml Manifesto https://github.com/ggml-org/ggml
>>
>> Bye
> 

[toc] | [prev] | [next] | [standalone]

#896422 — Micro penis mother sung arias (Re: NVIDIA evacuated its Chinese market [Tau Scaling])

Hi,

My mother is worried that I fucked Lane W.
aka Micro Penis mother 24 hours straight.
She was screaming, basically singing all

the arias from operas that Luciano Pavarotti
usually sings. You Lane W. aka Micro Penis
should have heard it, since you

live in the basement of your mothers house.

Bye

Lane W schrieb:
 > Mild Shock wrote:
 >> Hi,
 >>
 >> How it started:
 >>
 >> Captain: Throw the switch, Scotty!
 >> Enterprise: Cloaking Device makes it invisible
 >
 > You stupid ass. You posted this twice.

 > Mild Shock wrote:
 >
 >> So a few days later comes out the LLaMA, I do some calculations and I
 >> figure out “Okay, 65 billion parameters. You probably need about 40 gigs
 >> of RAM, with 4-bit quantization. So this can run on a MacBook. Why not
 >> do it?”
 >
 > you are a shame to your mother
 >
 > Perplexity Increase: Quantizing to 4-bit typically increases perplexity
 >
 > Reasoning & Coding: Complex reasoning chains and coding tasks suffer

[toc] | [prev] | [next] | [standalone]

#896426 — Micro penis brain is in constant hiatus (Re: Micro penis mother sung arias)

Hi,

Micro penis brain is in constant hiatus.
He can even not detect a trope.

LoL

Bye

Lane W schrieb:
 > Mild Shock wrote:
 >> Hi,
 >>
 >> My mother is worried that I fucked Lane W.
 >> aka Micro Penis mother 24 hours straight.
 >> She was screaming, basically singing all
 >>
 >> the arias from operas that Luciano Pavarotti
 >> usually sings. You Lane W. aka Micro Penis
 >> should have heard it, since you
 >>
 >> live in the basement of your mothers house.
 >
 > No, actually remarkably, I don't. According to google I live 433 
miles away from her.
 >
 > Strike!
 >
 > See, what i said about you was spot on.
 >
 > What you said about me was generic and incorrect.
 >
 > You really suck, man.

[toc] | [prev] | [next] | [standalone]

#896427 — Ignoramus or Ignorabimus: I don't care [(Re: Micro penis brain is in constant hiatus (Re: Micro penis mother sung arias)

Hi,

If any of you guys do not understand what
is meant by or what the implications are:

11.4 Giga Lips with a Budget Laptop
https://github.com/Jean-Luc-Picard-2021/gigabudget

Well I wouldn't care less. There are two
outcomes for numb nuts:

- Ignoramus: They don't understand it, but
   they will understand it before they die.

- Ignorabimus: They don't understand it, and
   will never understand it, and they die.

So who cares, its not my problem, you people
are stupid as fuck, and slow as fuck...

Bye

Mild Shock schrieb:
> 
> Hi,
> 
> Micro penis brain is in constant hiatus.
> He can even not detect a trope.
> 
> LoL
> 
> Bye
> 
> Lane W schrieb:
>  > Mild Shock wrote:
>  >> Hi,
>  >>
>  >> My mother is worried that I fucked Lane W.
>  >> aka Micro Penis mother 24 hours straight.
>  >> She was screaming, basically singing all
>  >>
>  >> the arias from operas that Luciano Pavarotti
>  >> usually sings. You Lane W. aka Micro Penis
>  >> should have heard it, since you
>  >>
>  >> live in the basement of your mothers house.
>  >
>  > No, actually remarkably, I don't. According to google I live 433 
> miles away from her.
>  >
>  > Strike!
>  >
>  > See, what i said about you was spot on.
>  >
>  > What you said about me was generic and incorrect.
>  >
>  > You really suck, man.
> 

[toc] | [prev] | [next] | [standalone]

#896429 — You are a moron, brainless putin payed (Re: Ignoramus or Ignorabimus: I don't care)

Hi,

You are a moron, and you represent putin payed
trolls from the army of brainless troll morons.

Bye

Lane W schrieb:
 > Mild Shock has no idea who I am or what I represent

Mild Shock schrieb:
> Hi,
> 
> If any of you guys do not understand what
> is meant by or what the implications are:
> 
> 11.4 Giga Lips with a Budget Laptop
> https://github.com/Jean-Luc-Picard-2021/gigabudget
> 
> Well I wouldn't care less. There are two
> outcomes for numb nuts:
> 
> - Ignoramus: They don't understand it, but
>    they will understand it before they die.
> 
> - Ignorabimus: They don't understand it, and
>    will never understand it, and they die.
> 
> So who cares, its not my problem, you people
> are stupid as fuck, and slow as fuck...
> 
> Bye
> 
> Mild Shock schrieb:
>>
>> Hi,
>>
>> Micro penis brain is in constant hiatus.
>> He can even not detect a trope.
>>
>> LoL
>>
>> Bye
>>
>> Lane W schrieb:
>>  > Mild Shock wrote:
>>  >> Hi,
>>  >>
>>  >> My mother is worried that I fucked Lane W.
>>  >> aka Micro Penis mother 24 hours straight.
>>  >> She was screaming, basically singing all
>>  >>
>>  >> the arias from operas that Luciano Pavarotti
>>  >> usually sings. You Lane W. aka Micro Penis
>>  >> should have heard it, since you
>>  >>
>>  >> live in the basement of your mothers house.
>>  >
>>  > No, actually remarkably, I don't. According to google I live 433 
>> miles away from her.
>>  >
>>  > Strike!
>>  >
>>  > See, what i said about you was spot on.
>>  >
>>  > What you said about me was generic and incorrect.
>>  >
>>  > You really suck, man.
>>
> 

[toc] | [prev] | [next] | [standalone]

#896431 — Yeah keep reading my posts, uninspired fool (Re: You are a moron, brainless putin payed)

Hi,

Yeah keep reading my posts, uninspired fool.
Seems you got a glimps of imagination from my post:

 > From: Mild Shock <janburse@fastmail.fm>
 > Subject: NVIDIA evacuated its Chinese market [Tau Scaling]
 > Date: Thu, 23 Jul 2026 19:13:51 +0200

 > How it started:
 >
 > Captain: Throw the switch, Scotty!
 > Enterprise: Cloaking Device makes it invisible
 > Spock: Military secrets are the most fleeting of all.
 > Kirk Escapes the Romulans - The Enterprise Incident
 > https://www.youtube.com/watch?v=AusAGjwlql8

But copying others in trope, is not the same
as jolting a trope into a conservation.
It still makes you a lame copist. Maybe you

don't know with whom you are dealing with, right?
I don't know who you are, but I will look for you,
I will find you and I will let you run my pi-WAM

on your sputnik commodore c64 with 8088.

Bye

Mild Shock schrieb:
> Hi,
> 
> You are a moron, and you represent putin payed
> trolls from the army of brainless troll morons.
> 
> Bye
> 
> Lane W schrieb:
>  > Mild Shock has no idea who I am or what I represent
> 
> Mild Shock schrieb:
>> Hi,
>>
>> If any of you guys do not understand what
>> is meant by or what the implications are:
>>
>> 11.4 Giga Lips with a Budget Laptop
>> https://github.com/Jean-Luc-Picard-2021/gigabudget
>>
>> Well I wouldn't care less. There are two
>> outcomes for numb nuts:
>>
>> - Ignoramus: They don't understand it, but
>>    they will understand it before they die.
>>
>> - Ignorabimus: They don't understand it, and
>>    will never understand it, and they die.
>>
>> So who cares, its not my problem, you people
>> are stupid as fuck, and slow as fuck...
>>
>> Bye
>>
>> Mild Shock schrieb:
>>>
>>> Hi,
>>>
>>> Micro penis brain is in constant hiatus.
>>> He can even not detect a trope.
>>>
>>> LoL
>>>
>>> Bye
>>>
>>> Lane W schrieb:
>>>  > Mild Shock wrote:
>>>  >> Hi,
>>>  >>
>>>  >> My mother is worried that I fucked Lane W.
>>>  >> aka Micro Penis mother 24 hours straight.
>>>  >> She was screaming, basically singing all
>>>  >>
>>>  >> the arias from operas that Luciano Pavarotti
>>>  >> usually sings. You Lane W. aka Micro Penis
>>>  >> should have heard it, since you
>>>  >>
>>>  >> live in the basement of your mothers house.
>>>  >
>>>  > No, actually remarkably, I don't. According to google I live 433 
>>> miles away from her.
>>>  >
>>>  > Strike!
>>>  >
>>>  > See, what i said about you was spot on.
>>>  >
>>>  > What you said about me was generic and incorrect.
>>>  >
>>>  > You really suck, man.
>>>
>>
> 

[toc] | [prev] | [next] | [standalone]

#896433 — Out of the blue accusation span 15 days [Empirical USENET study] (Re: Ignoramus or Ignorabimus: I don't care)

Hi,

Again I posted everything here:

 > 11.4 Giga Lips with a Budget Laptop
 > https://github.com/Jean-Luc-Picard-2021/gigabudget

The repo says, same time when I posted
the link first time:

 > This repository was archived by the
 > owner on Jul 9, 2026. It is now read-only.

Now a USENET user, who had already entitled
himself for a couple of irrational accusations

towards my side, is asking this question:

Chris M. Thomasson schrieb, Jul 24, 2026
 > Show an outline of what you
 > need you compute shader to do?

Bravo, thats a delay of a wooping 15 days.

Bye

Mild Shock schrieb:
> Hi,
> 
> If any of you guys do not understand what
> is meant by or what the implications are:
> 
> 11.4 Giga Lips with a Budget Laptop
> https://github.com/Jean-Luc-Picard-2021/gigabudget
> 
> Well I wouldn't care less. There are two
> outcomes for numb nuts:
> 
> - Ignoramus: They don't understand it, but
>    they will understand it before they die.
> 
> - Ignorabimus: They don't understand it, and
>    will never understand it, and they die.
> 
> So who cares, its not my problem, you people
> are stupid as fuck, and slow as fuck...
> 
> Bye
> 
> Mild Shock schrieb:
>>
>> Hi,
>>
>> Micro penis brain is in constant hiatus.
>> He can even not detect a trope.
>>
>> LoL
>>
>> Bye
>>
>> Lane W schrieb:
>>  > Mild Shock wrote:
>>  >> Hi,
>>  >>
>>  >> My mother is worried that I fucked Lane W.
>>  >> aka Micro Penis mother 24 hours straight.
>>  >> She was screaming, basically singing all
>>  >>
>>  >> the arias from operas that Luciano Pavarotti
>>  >> usually sings. You Lane W. aka Micro Penis
>>  >> should have heard it, since you
>>  >>
>>  >> live in the basement of your mothers house.
>>  >
>>  > No, actually remarkably, I don't. According to google I live 433 
>> miles away from her.
>>  >
>>  > Strike!
>>  >
>>  > See, what i said about you was spot on.
>>  >
>>  > What you said about me was generic and incorrect.
>>  >
>>  > You really suck, man.
>>
> 

[toc] | [prev] | [next] | [standalone]

#896428 — Little Data Center on Your Palm [AI Laptops for 500 USD] (Re: The Wuhan Virus that destroyed Python [ggml Manifesto])

Hi,

Now you can compare this here from 2008
with modern AI Laptops for 500-1000 USD:

Google spotlights data center inner workings
https://web.archive.org/web/20131019063218/http://news.cnet.com/8301-10784_3-9955184-7.html

There is a striking similarity, only what
once occupied a rack, has now the size
of your plam, all inside one silicon chip:

- Multiple CPU cores on the same chip
- Multiple GPU units on the same chip
- Network on the same chip communication
- Crossbar caches on the same chip
- Disk controllers on the same chip
- Multi channel RAM access on the same chip

Pretty cool!

Bye

P.S.: Example such devices with iGPU:

Intel(R) Core(TM) Ultra 7 258V
AMD Ryzen AI 7 350 w/ Radeon 860M
Apple A18 Pro, Darwin Kernel Version 25.5.0
Snapdragon(R) X - X126100 - Qualcomm(R) Oryon(TM) CPU


Mild Shock schrieb:
> Hi,
> 
> Remember when first all local AI was Python
> and PyTorch APIs. And then suddently people started
> using bare metal C/C++ Code. Here is the story:
> 
> How it started:
> 
> GPT-J or GPT-J-6B is an open-source large
> language model (LLM) developed by EleutherAI
> in 2021. As the name suggests, it is a
> generative pre-trained transformer model
> designed to produce human-like text that
> continues from a prompt.
> https://www.eleuther.ai/
> 
> How it was going [Georgi Gerganov]:
> 
> So a few days later comes out the LLaMA, I do
> some calculations and I figure out “Okay, 65
> billion parameters. You probably need about
> 40 gigs of RAM, with 4-bit quantization. So
> this can run on a MacBook. Why not do it?”
> 
> Why I was able to do it so quickly - basically,
> for all that I saw it’s pretty much GPT-J architecture
> with some modifications, like some extra memorization
> layers. It’s minor changes. Basically, again, the
> existing code for the GPT-J, I just simply
> modified it there, it happened pretty quickly.
> https://changelog.com/podcast/532
> 
> Georgi Gerganov, Bulgarian, now with Hugging
> Face, ggml-cann also running on Chinese AI chips.
> ggml Manifesto https://github.com/ggml-org/ggml
> 
> Bye

[toc] | [prev] | [next] | [standalone]

#896430 — 2008: 4 Blades + Tesla S1070 versus 2026: 1 AI Laptop (Re: Little Data Center on Your Palm [AI Laptops for 500 USD])

Hi,

Feature    2008: 4 Blades + Tesla S1070
CPU Cores    16
CPU Clock (GHz)    2.5
CPU IPC (est.)    1
CPU Throughput (units)    16 x 2.5 ×1.0=40
GPU Cores    960 (4x T10P)
GPU Clock (GHz)    1.5
GPU IPC (est.)    ~1.0 (FMA)
GPU Throughput (units)    960 x 1.5x1.0=1440
Total Compute (CPU+GPU)    40 + 1440 = 1480
Memory Capacity    16-20 GB (DDR2)
Storage I/O    ~400 MB/s (HDDs)
Power Consumption    ~1500 W
Physical Size    8-12 RU + 1U GPU
Cost (2008 USD)    ~$33,000

Feature    2026: 1 Al Laptop
CPU Cores    Aug 16
CPU Clock (GHz)    4.5
CPU IPC (est.)    2
CPU Throughput (units)    16 x 4.5 ×2.0=144 (or 72 for 8c)
GPU Cores    4096
GPU Clock (GHz)    ~2.0
GPU IPC (est.)    ~1.5 (modern)
GPU Throughput (units)    4096 x 2.0x1.5=12288
Total Compute (CPU+GPU)    144 + 12288 = 12432 (or 72+12288 for 8c)
Memory Capacity    16-32 GB (DDR5)
Storage I/O    ~7000 MB/s (NVMe)
Power Consumption    ~50-100 W
Physical Size    1 laptop bag
Cost (2008 USD)    ~$500-1000

Feature    Winner
CPU Cores    Tie
CPU Clock (GHz)    Laptop (1.8x faster)
CPU IPC (est.)    Laptop (2x better)
CPU Throughput (units)    Laptop: 1.8-3.6x faster
GPU Cores    Laptop: 4.3x more cores
GPU Clock (GHz)    Laptop (1.33x faster)
GPU IPC (est.)    Laptop (1.5x better)
GPU Throughput (units)    Laptop: 8.5x more GPU throughput
Total Compute (CPU+GPU)    Laptop: 8.4x more total compute
Memory Capacity    Laptop (more, faster)
Storage I/O    Laptop: 17x faster
Power Consumption    Laptop: 15-30x more efficient
Physical Size    Laptop
Cost (2008 USD)    Laptop: 33-66x cheaper

Bye

Mild Shock schrieb:
> Hi,
> 
> Now you can compare this here from 2008
> with modern AI Laptops for 500-1000 USD:
> 
> Google spotlights data center inner workings
> https://web.archive.org/web/20131019063218/http://news.cnet.com/8301-10784_3-9955184-7.html 
> 
> 
> There is a striking similarity, only what
> once occupied a rack, has now the size
> of your plam, all inside one silicon chip:
> 
> - Multiple CPU cores on the same chip
> - Multiple GPU units on the same chip
> - Network on the same chip communication
> - Crossbar caches on the same chip
> - Disk controllers on the same chip
> - Multi channel RAM access on the same chip
> 
> Pretty cool!
> 
> Bye
> 
> P.S.: Example such devices with iGPU:
> 
> Intel(R) Core(TM) Ultra 7 258V
> AMD Ryzen AI 7 350 w/ Radeon 860M
> Apple A18 Pro, Darwin Kernel Version 25.5.0
> Snapdragon(R) X - X126100 - Qualcomm(R) Oryon(TM) CPU
> 
> 
> Mild Shock schrieb:
>> Hi,
>>
>> Remember when first all local AI was Python
>> and PyTorch APIs. And then suddently people started
>> using bare metal C/C++ Code. Here is the story:
>>
>> How it started:
>>
>> GPT-J or GPT-J-6B is an open-source large
>> language model (LLM) developed by EleutherAI
>> in 2021. As the name suggests, it is a
>> generative pre-trained transformer model
>> designed to produce human-like text that
>> continues from a prompt.
>> https://www.eleuther.ai/
>>
>> How it was going [Georgi Gerganov]:
>>
>> So a few days later comes out the LLaMA, I do
>> some calculations and I figure out “Okay, 65
>> billion parameters. You probably need about
>> 40 gigs of RAM, with 4-bit quantization. So
>> this can run on a MacBook. Why not do it?”
>>
>> Why I was able to do it so quickly - basically,
>> for all that I saw it’s pretty much GPT-J architecture
>> with some modifications, like some extra memorization
>> layers. It’s minor changes. Basically, again, the
>> existing code for the GPT-J, I just simply
>> modified it there, it happened pretty quickly.
>> https://changelog.com/podcast/532
>>
>> Georgi Gerganov, Bulgarian, now with Hugging
>> Face, ggml-cann also running on Chinese AI chips.
>> ggml Manifesto https://github.com/ggml-org/ggml
>>
>> Bye
> 

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | sci.physics

csiph-web