Ask Phind: AI inflection point right now [End 2025] (Was: NPUs (Neural Processing Units) are the new normal)

From	Mild Shock <janburse@fastmail.fm>
Newsgroups	comp.lang.prolog
Subject	Ask Phind: AI inflection point right now [End 2025] (Was: NPUs (Neural Processing Units) are the new normal)
Date	2025-10-15 16:10 +0200
Message-ID	<10coa13$12ucf$1@solani.org> (permalink)
References	<107t0fi$6bgl$1@solani.org> <10cmqea$s93$1@solani.org> <10cn157$10qq$1@solani.org> <10co9kp$12u6u$1@solani.org>

Show all headers | View raw

Hi,

But not only Mobie AI and Desktop AI are making
a broader imprint now. We might also experience
Workstation AI, with a 3'000.- USD price tag:

 > You Can't Buy This... Yet! The NVIDIA GB10 from Dell
 > The New Superchip that Terrifies the Cloud!
 > https://www.youtube.com/watch?v=x1qViw4xyVo

So whats going on? I was asking Phind, which is
driven by a 70B model tailored towards developers:

Q: Is there an AI inflection point right now ,
    with NPUs in mobile, desktop and workstation

A: Evidence of the Inflection Point

- Mobile Leadership
   NPUs originated in smartphones
   Now becoming ubiquitous across all device types
   Enabling sophisticated AI features at consumer price points

- Desktop Revolution
   Major manufacturers implementing NPUs across product lines
   Apple's Neural Engine integrated into M-series chips
   Qualcomm, Intel, and AMD incorporating AI accelerators

- Workstation Transformation
   Professional-grade NPUs in mobile workstations
   Demonstrated superior performance for AI-specific tasks
   Enabling local processing of previously cloud-dependent workloads

https://www.phind.com/search/cmgs1s6jv00023h67g5z2aaa0

Bye

Mild Shock schrieb:
> Hi,
> 
> It seems I am having problems pacing with
> all the new fancy toys. Wasn't able to really
> benchmark my NPU from a Desktop AI machine,
> 
> picked the wrong driver. Need to try again.
> What worked was benchmarking Mobile AI machines.
> I just grabbed Geekbench AI and some devices:
> 
> USA Fab, M4:
> 
>      sANN    hANN    qANN
> iPad CPU    4848    7947    6353
> iPad GPU    9752    11383    10051
> iPad NPU    4873    36544    *51634*
> 
> China Fab, Snapdragon:
> 
>      sANN    hANN    qANN
> Redmi CPU    1044    950    1723
> Redmi GPU    480    905    737
> Redmi NNAPI    205    205    469
> Redmi QNN    226    226    *10221*
> 
> Speed-Up via NPU is factor 10x. See the column
> qANN which means quantizised artificial neural
> networks, when NPU or QNN is picked.
> 
> The mobile AI NPUs are optimized using
> mimimal amounts of energy, and minimal amounts
> of space squeezing (distilling) everything
> 
> into INT8 and INT4.
> 
> Bye
> 
> Mild Shock schrieb:
>> Hi,
>>
>> The change from 378 ms to 286 ms is around 25-30%
>> is insane. But I did both tests on a novel AI CPU.
>> To be precise on a AMD Ryzen AI 7 350.
>>
>> But somehow I picked up rumors that AI CPUs now
>> might do Neural Network Branch Prediction. The
>> idea seems to exist in hardware at least since (2012):
>>
>> Machine learning and artificial intelligence are
>> the current hype (again). In their new Ryzen
>> processors, AMD advertises the Neural Net
>> Prediction. It turns out this is was already
>> used in their older (2012) Piledriver architecture
>> used for example in the AMD A10-4600M. It is also
>> present in recent Samsung processors such as the
>> one powering the Galaxy S7. What is it really?
>> https://chasethedevil.github.io/post/the_neural_network_in_your_cpu/
>>
>> It can be done with Convoluted Neural Networks (CNN):
>>
>> BranchNet: A Convolutional Neural Network to
>> Predict Hard-To-Predict Branches
>> To this end, Tarsa et al. proposed using convolutional
>> neural networks (CNNs) that are trained at
>> compiletime to accurately predict branches that
>> TAGE cannot. Given enough profiling coverage, CNNs
>> learn input-independent branch correlations.
>> https://microarch.org/micro53/papers/738300a118.pdf
>>
>> Interstingly the above shows cases a PGO based
>> Machine Learning for Branch Predictors. No clue
>> how they construct the CPU, that they can feed
>>
>> it with offline constructed neural neutworks for
>> their own execution. Maybe an optimizer uses it?
>> But I guess a more modern  solutions would not only
>>
>> use CNN, but also an Attention Mechanism.
>>
>> Bye
>>
>> Mild Shock schrieb:
>>> Hi,
>>>
>>> I spent some time thinking about my primes.pl
>>> test. And came to the conclusion that it
>>> mainly tests the Prolog ALU. Things like
>>>
>>> integer successor or integer modulo. Then
>>> I found that Java has Math.floorMod() which
>>> I wasn't using yet. And peng results are better:
>>>
>>> /* Dogelog Player 2.1.2 for Java, today */
>>> ?- time(test).
>>> % Zeit 286 ms, GC 1 ms, Lips 26302430, Uhr 15.10.2025 02:31
>>> true.
>>>
>>> Maybe the Java backend picks a CPU instruction
>>> for Math.floorMod() instead of executing the
>>> longer code sequence that is needed to correct
>>>
>>> rem/2 into mod/2. Who knows. I also reorganized
>>> the code a little bit, and eliminated an extra
>>> method call in all arithmetic functions, by
>>>
>>> inlining the arithmetic function body in the
>>> evaluable predicate definition code. Comparison
>>> to old measurements and some measurements of
>>>
>>> other Prolog systems:
>>>
>>> /* Dogelog Player 2.1.2 for Java, weeks ago */
>>> ?- time(test).
>>> % Zeit 378 ms, GC 1 ms, Lips 19900780, Uhr 28.08.2025 17:44
>>> true.
>>>
>>> /* SWI-Prolog 9.0.4 */
>>> ?- time(test).
>>> % 7,506,639 inferences, 0.363 CPU in 0.362 seconds
>>> (100% CPU, 20693560 Lips)
>>> true.
>>>
>>> /* Scryer Prolog 0.9.4-639 */
>>> ?- time(test).
>>> % CPU time: 0.365s, 7_517_613 inferences
>>> true.
>>>
>>> /* Trealla Prolog 2.82.23-3 */
>>> ?- time(test).
>>> % Time elapsed 0.868s, 11263917 Inferences, 12.983 MLips
>>> true.
>>>
>>> Bye
>>>
>>> P.S.: The code uses the hated mathematical mod/2,
>>> and not the cheaper rem/2 that CPUs usually have:
>>>
>>> test :-
>>>     len(L, 1000),
>>>     primes(L, _).
>>>
>>> primes([], 1).
>>> primes([J|L], J) :-
>>>     primes(L, I),
>>>     K is I+1,
>>>     search(L, K, J).
>>>
>>> search(L, I, J) :-
>>>     mem(X, L),
>>>     I mod X =:= 0, !,
>>>     K is I+1,
>>>     search(L, K, J).
>>> search(_, I, I).
>>>
>>> mem(X, [X|_]).
>>> mem(X, [_|Y]) :-
>>>     mem(X, Y).
>>>
>>> len([], 0) :- !.
>>> len([_|L], N) :-
>>>     N > 0,
>>>     M is N-1,
>>>     len(L, M).
>>>
>>> Mild Shock schrieb:
>>>> Hi,
>>>>
>>>> WebPL is already outdated I guess. It doesn't
>>>> show the versions of the other Prolog systems
>>>> it is using. While I had these results for
>>>>
>>>> the primes example in the WebPL playground:
>>>>
>>>> /* Trealla Prolog WASM */
>>>> (23568.9ms)
>>>>
>>>> When I run the example here:
>>>>
>>>> https://php.energy/trealla.html
>>>>
>>>> I get better results:
>>>>
>>>> /* trealla-js 0.27.1 */
>>>>
>>>> ?- time(test).
>>>> % Time elapsed 9.907s, 11263917 Inferences, 1.137 MLips
>>>>
>>>> Bye
>>>
>>
>

Thread

WebPL is already outdated Mild Shock <janburse@fastmail.fm> - 2025-08-17 18:37 +0200
  Heap/Stack versus WAM resp. ZIP (Was: WebPL is already outdated) Mild Shock <janburse@fastmail.fm> - 2025-08-18 14:52 +0200
    Trealla knows Program Sharing (PS) Tricks ? (Was: Heap/Stack versus WAM resp. ZIP) Mild Shock <janburse@fastmail.fm> - 2025-08-18 15:06 +0200
      Smarter Partial Strings would use Program Sharing ['$append'/3] (Was: Trealla knows Program Sharing (PS) Tricks ?) Mild Shock <janburse@fastmail.fm> - 2025-08-18 15:42 +0200
        Static Shunting is even better [Dogelog Player] (Was: Smarter Partial Strings would use Program Sharing ['$append'/3]) Mild Shock <janburse@fastmail.fm> - 2025-08-18 15:49 +0200
  The Artificial Intelligence Flip: Acer Swift Go! (Was: WebPL is already outdated) Mild Shock <janburse@fastmail.fm> - 2025-08-31 23:56 +0200
    2025 will be last year we hear of Python (Re: The Artificial Intelligence Flip: Acer Swift Go!) Mild Shock <janburse@fastmail.fm> - 2025-09-01 00:45 +0200
      Apertus: With love, from Switzerland [02 Sept 2025] (Re: 2025 will be last year we hear of Python) Mild Shock <janburse@fastmail.fm> - 2025-09-05 00:36 +0100
      Don't try this (Was: Apertus: With love, from Switzerland [02 Sept 2025] ) Mild Shock <janburse@fastmail.fm> - 2025-09-05 01:03 +0100
      AI means ambracing Non-Linearity (Was: 2025 will be last year we hear of Python) Mild Shock <janburse@fastmail.fm> - 2025-09-19 10:01 +0200
        AI soaked PCs: Is there a Copilot+ Prolog? (Was: AI means ambracing Non-Linearity) Mild Shock <janburse@fastmail.fm> - 2025-09-19 10:10 +0200
          The morning coffee incident [Prolog Community] (Was: AI soaked PCs: Is there a Copilot+ Prolog?) Mild Shock <janburse@fastmail.fm> - 2025-09-19 14:38 +0200
            Root Cause Prediction for Your Brain (Was: The morning coffee incident [Prolog Community]) Mild Shock <janburse@fastmail.fm> - 2025-09-19 18:22 +0200
              Please delete my account and all my posts on SWI-Prolog discourse (Re: Root Cause Prediction for Your Brain) Mild Shock <janburse@fastmail.fm> - 2025-09-19 18:38 +0200
                I will consult a Lawyer of mine (Was: Please delete my account and all my posts on SWI-Prolog discourse) Mild Shock <janburse@fastmail.fm> - 2025-09-19 18:42 +0200
  Scryer Prolog unify_with_occurs_check/2 doesn't scale (Was: WebPL is already outdated) Mild Shock <janburse@fastmail.fm> - 2025-09-19 16:08 +0200
    How bad is Rust, can JavaScript beat it? (Was: Scryer Prolog unify_with_occurs_check/2 doesn't scale) Mild Shock <janburse@fastmail.fm> - 2025-09-19 16:18 +0200
    unify_with_occurs_check/2 might have been fixed (Was: Scryer Prolog unify_with_occurs_check/2 doesn't scale) Mild Shock <janburse@fastmail.fm> - 2025-09-25 01:50 +0200
      Had to Rollback my Jaffar Unification (Was: unify_with_occurs_check/2 might have been fixed) Mild Shock <janburse@fastmail.fm> - 2025-09-25 01:59 +0200
        Trealla Prolog might apply "frozeness" to cyclic terms (Was: Had to Rollback my Jaffar Unification) Mild Shock <janburse@fastmail.fm> - 2025-09-25 02:06 +0200
          Non-intrusive through "frozen" subcategories (Was: Trealla Prolog might apply "frozeness" to cyclic terms) Mild Shock <janburse@fastmail.fm> - 2025-09-25 02:21 +0200
    Scryer Prolog occurs check cannot do hydra (Was: Scryer Prolog unify_with_occurs_check/2 doesn't scale) Mild Shock <janburse@fastmail.fm> - 2025-09-26 12:19 +0200
  WebPL and Scryer Prolog are bad examples (Was: WebPL is already outdated) Mild Shock <janburse@fastmail.fm> - 2025-10-13 09:49 +0200
    Who will win Shift-Reduce or Tabled DCG? [AI Boom] (Was: WebPL and Scryer Prolog are bad examples) Mild Shock <janburse@fastmail.fm> - 2025-10-13 15:09 +0200
  primes.pl mainly tests the Prolog ALU [mod/2 vs rem/2] (Was: WebPL is already outdated) Mild Shock <janburse@fastmail.fm> - 2025-10-15 02:38 +0200
    25-30% is insane, Neural Network Branch Prediction? (Was: primes.pl mainly tests the Prolog ALU) Mild Shock <janburse@fastmail.fm> - 2025-10-15 04:33 +0200
      NPUs (Neural Processing Units) are the new normal (Was: 25-30% is insane, Neural Network Branch Prediction?) Mild Shock <janburse@fastmail.fm> - 2025-10-15 16:04 +0200
        Ask Phind: AI inflection point right now [End 2025] (Was: NPUs (Neural Processing Units) are the new normal) Mild Shock <janburse@fastmail.fm> - 2025-10-15 16:10 +0200
          Eat Tteokbokki before SkyNet kills you [$100 ChatGPT] (Was: Ask Phind: AI inflection point right now [End 2025]) Mild Shock <janburse@fastmail.fm> - 2025-10-18 15:57 +0200
            Give Julio Di Egidio the bloody money [3 RMB¥ MiniMind] (Re: Eat Tteokbokki before SkyNet kills you [$100 ChatGPT]) Mild Shock <janburse@fastmail.fm> - 2025-10-18 16:19 +0200
              Vertex AI Training is more expensive? (Was: Give Julio Di Egidio the bloody money [3 RMB¥ MiniMind]) Mild Shock <janburse@fastmail.fm> - 2025-10-21 00:32 +0200
        The Love Affair: OpenAI and AMD (Was: NPUs (Neural Processing Units) are the new normal) Mild Shock <janburse@fastmail.fm> - 2025-10-18 18:59 +0200
        The NPU in your Browser [WebNN by W3C] (Was: NPUs (Neural Processing Units) are the new normal) Mild Shock <janburse@fastmail.fm> - 2025-10-26 08:39 +0100
          Fuzzy Alert: Boris the Loris on the Dancefloor (Was: The NPU in your Browser [WebNN by W3C]) Mild Shock <janburse@fastmail.fm> - 2025-10-26 11:33 +0100

csiph-web