Googles TPU muscle in 2017 [Prolog Community is Sleepy Joe] (Re: Zeus: A Language for Expressing Algorithms in Hardware)

From	Mild Shock <janburse@fastmail.fm>
Newsgroups	comp.lang.prolog
Subject	Googles TPU muscle in 2017 [Prolog Community is Sleepy Joe] (Re: Zeus: A Language for Expressing Algorithms in Hardware)
Date	2025-11-27 15:23 +0100
Message-ID	<10g9mto$oiad$2@solani.org> (permalink)
References	<106p08a$3b6se$1@solani.org> <10g9j5q$ofl1$1@solani.org> <10g9l0l$ogpf$3@solani.org> <10g9ln5$oh94$3@solani.org>

Show all headers | View raw

Hi,

Well I am currently looking in Local AI when I
consider NPUs, which have very small floats.
An example of bigger AI accelerators are TPUs,

which can deal with larger floats. Subsequently
then can also deal with a larger integer range.
I only read this anecdote yesterday:

"In December 2017, Stockfish 8 was used as a
benchmark to test Google division DeepMind's
AlphaZero, with Stockfish running on CPU and
AlphaZero running on Google's proprietary
Tensor Processing Units (TPUs).

AlphaZero was trained through self-play for
a total of nine hours, and reached Stockfish's
level after just four. AlphaZero also played
twelve 100-game matches against Stockfish starting
from twelve popular openings for a final score
of 290 wins, 886 draws and 24 losses, for a
point score of 733:467."
https://en.wikipedia.org/wiki/Stockfish_(chess)#Stockfish_8_versus_AlphaZero

And then:

"AlphaZero's victory over Stockfish sparked a
flurry of activity in the computer chess community,
leading to a new open-source engine aimed at
replicating AlphaZero, known as Leela Chess Zero.
The two engines remained close in strength for a
while, but Stockfish has pulled away since the
introduction of NNUE, winning every TCEC
season since Season 18."

Meanwhile the Prolog community: Sleepy Joe

LoL

Bye

Mild Shock schrieb:
> Hi,
> 
> What mindset is needed to program an NPU. Mostlikely
> a mindset based on fork/join parallelism is nonsense.
> What could be more fruitful is view the AI accellerator
> 
> as a blackbox that runs a neural network, whereby
> a neural network can be effectively viewed as a form
> of hardware, although unter the hood, it is open weights
> 
> and matrix operations. So the mindest needs:
> 
> Zeus: A Language for Expressing Algorithms in Hardware
> K. J. Lieberherr -  01 February 1985
> https://dl.acm.org/doi/10.1109/MC.1985.1662799
> 
> What changed back to then?
> 
> - 80's Field Programmable Gate Array (FPGA)
> 
> - 20's AI Boom: NPUs, Unified Memory and Routing Fabric
> 
> Bye
> 
> Mild Shock schrieb:
>> Hi,
>>
>> I already posted how to do SAT and Clark Completion
>> with ReLU. This was a post from 15.03.2025, 16:13,
>> see also below. But can we do CLP as well? Here
>>
>> is a take on the dif/2 constraint, or more precisely
>> a very primitive (#\=)/2 from CLP(FD), going towards
>> analogical computing. Might work for domains that
>>
>> fit into the quantization size of a NPU:
>>
>> 1) First note that we can model abs() via ReLU:
>>
>> abs(x) = ReLU(x) + ReLU(- x)
>>
>> 2) Then note that for integer values, we can model
>> chi(x>0), the characteristic function of the predicate x > 0:
>>
>> chi(x>0) = 1 - ReLU(1 - x).
>>
>> 3) Now chi(x=\=y) is simply:
>>
>> chi(x=\=y) = chi(abs(x - y) > 0)
>>
>> Now insert the formula for chi(x>0) based on ReLU
>> and the formula for abs() based on ReLU. Eh voila you
>> got an manually created neural network for the
>>
>> (#\=)/2 condition of CLP(FD), constraint logic
>> programming for finite domains.
>>
>> Have Fun!
>>
>> Bye
>>
>> Mild Shock schrieb:
>>  > A storm of symbolic differentiation libraries
>>  > was posted. But what can these Prolog code
>>  > fossils do?
>>  >
>>  > Does one of these libraries support Python symbolic
>>  > Pieceweise ? For example one can define rectified
>>  > linear unit (ReLU) with it:
>>  >
>>  >                  /   x      x  >= 0
>>  >      ReLU(x) := <
>>  >                  \   0      otherwise
>>  >
>>  > With the above one can already translate a
>>  > propositional logic program, that uses negation
>>  > as failure, into a neural network:
>>  >
>>  > NOT     \+ p             1 - x
>>  > AND     p1, ..., pn      ReLU(x1 + ... + xn - (n-1))
>>  > OR      p1; ...; pn      1 - ReLU(-x1 - .. - xn + 1)
>>  >
>>  > For clauses just use Clark Completion, it makes
>>  > the defined predicate a new neuron, dependent on
>>  > other predicate neurons,
>>  >
>>  > through a network of intermediate neurons. Because
>>  > of the constant shift in AND and OR, the neurons
>>  > will have a bias b.
>>  >
>>  > So rule based in zero order logic is a subset
>>  > of neural network.
>>  >
>>  > Python symbolic Pieceweise
>>  > 
>> https://how-to-data.org/how-to-write-a-piecewise-defined-function-in-python-using-sympy/ 
>>
>>  >
>>  >
>>  > rectified linear unit (ReLU)
>>  > https://en.wikipedia.org/wiki/Rectifier_(neural_networks)
>>  >
>>  > Clark Completion
>>  > https://www.cs.utexas.edu/~vl/teaching/lbai/completion.pdf
>>
>> Mild Shock schrieb:
>>> Hi,
>>>
>>> I am spekulating an NPU could give 1000x more LIPS.
>>> For certain combinatorial search problems. It all
>>> boils down to implement this thingy:
>>>
>>> In June 2020, Stockfish introduced the efficiently
>>> updatable neural network (NNUE) approach, based
>>> on earlier work by computer shogi programmers
>>> https://en.wikipedia.org/wiki/Stockfish_%28chess%29
>>>
>>> There are varying degrees what gets updated of
>>> a neural network. But the specs of an NPU tell
>>> me very simply the following:
>>>
>>> - An NPU can make 40 TFLOPS, all my AI Laptops
>>>    from 2025 can do that right now. The brands
>>>    are Intel Ultra, AMD Ryzen and Snapdragon X,
>>>
>>>    but I guess there might be more brands around,
>>>    which can do that with a price tag less
>>>    than 1000.- USD.
>>>
>>> - SWI Prolog can make 30 MLIPS, Dogelog Player
>>>    runs similar, some Prolog systems are faster.
>>>
>>> Now thats is 10^12 versus 10^6. If some of the
>>> LIPS can be delegated to a NPU, and if we assume
>>> for example less locality or more primitive
>>>
>>> operations that require a layering. Would could assume
>>> that from the NPU 10^12 a factor of 1000 goes
>>> away. So we might still see 10'9 LIPS emerge.
>>>
>>> Now make the calculation:
>>>
>>> - Without NPU: MLIPS
>>> - With NPU: GLIPS
>>> - Ratio: 1000x times faster
>>>
>>> Have fun!
>>>
>>> Bye
>>>
>>> Mild Shock schrieb:
>>>> Mercio’s Algorithm (2012) for Rational
>>>> Tree Compare is specified here mathematically.
>>>> It is based on computing truncations A' = (A_0,
>>>> A_1, etc..) of a rational tree A:
>>>>
>>>> A < B ⟺ A′ <_lex B′
>>>>
>>>> https://math.stackexchange.com/a/210730
>>>>
>>>> Here is an implementation in Prolog.
>>>> First the truncation:
>>>>
>>>> trunc(_, T, T) :- var(T), !.
>>>> trunc(0, T, F) :- !, functor(T, F, _).
>>>> trunc(N, T, S) :-
>>>>     M is N-1,
>>>>     T =.. [F|L],
>>>>     maplist(trunc(M), L, R),
>>>>     S =.. [F|R].
>>>>
>>>> And then the iterative deepening:
>>>>
>>>> mercio(N, X, Y, C) :-
>>>>     trunc(N, X, A),
>>>>     trunc(N, Y, B),
>>>>     compare(D, A, B),
>>>>     D \== (=), !, C = D.
>>>> mercio(N, X, Y, C) :-
>>>>     M is N + 1,
>>>>     mercio(M, X, Y, C).
>>>>
>>>> The main entry first uses (==)/2 for a
>>>> terminating equality check and if the
>>>> rational trees are not equal, falls back
>>>> to the iterative deepening:
>>>>
>>>> mercio(C, X, Y) :- X == Y, !, C = (=).
>>>> mercio(C, X, Y) :- mercio(0, X, Y, C).
>>>>
>>>> I couldn’t find yet a triple that violates
>>>> transitivity. But I am also not much happy
>>>> with the code. Looks a little bit expensive
>>>> to create a truncation copy iteratively.
>>>>
>>>> Provided there is really no counter example,
>>>> maybe we can do mit more smart and faster? It
>>>> might also stand the test of conservativity?
>>>
>>
>

Thread

Mercio’s Algorithm for Rational Tree Compare in Prolog Mild Shock <janburse@fastmail.fm> - 2025-08-04 02:52 +0200
  The Original Ganster (OG) of Gameification: IEEE 1044.1-1995 (Was: Mercio’s Algorithm for Rational Tree Compare in Prolog) Mild Shock <janburse@fastmail.fm> - 2025-08-04 13:48 +0200
    The Bitrot called Math Stack Exchange (Re: The Original Ganster (OG) of Gameification: IEEE 1044.1-1995) Mild Shock <janburse@fastmail.fm> - 2025-08-04 14:00 +0200
      I guess its back to Hopcroft and Karp.Re: The Bitrot called Math Stack Exchange) Mild Shock <janburse@fastmail.fm> - 2025-08-04 14:11 +0200
  Szpilrajn Theorem and Suzumura Consistency (Was: Mercio’s Algorithm for Rational Tree Compare in Prolog) Mild Shock <janburse@fastmail.fm> - 2025-08-06 01:50 +0200
    The good thing is we have at least Mercio’s Algorithm (Re: Szpilrajn Theorem and Suzumura Consistency) Mild Shock <janburse@fastmail.fm> - 2025-08-06 08:10 +0200
      Hopcroft and Karp’s is just Contraction (Was: The good thing is we have at least Mercio’s Algorithm) Mild Shock <janburse@fastmail.fm> - 2025-08-06 08:13 +0200
        Re: Hopcroft and Karp’s is just Contraction (Was: The good thing is we have at least Mercio’s Algorithm) Mild Shock <janburse@fastmail.fm> - 2025-08-06 08:23 +0200
      Should we use minimum_coa/3 ? (Was: The good thing is we have at least Mercio’s Algorithm) Mild Shock <janburse@fastmail.fm> - 2025-08-08 23:45 +0200
        Perfectly balanced , as all things should be! (Was: Should we use minimum_coa/3 ?) Mild Shock <janburse@fastmail.fm> - 2025-08-08 23:49 +0200
  Mercios decidability was already attested in 2012 (Was: Mercio’s Algorithm for Rational Tree Compare in Prolog) Mild Shock <janburse@fastmail.fm> - 2025-08-14 20:26 +0200
    Performance of Mercio’s Total Order (Was: Mercios decidability was already attested in 2012) Mild Shock <janburse@fastmail.fm> - 2025-08-15 23:49 +0200
      Fuzzy Testing is your Swiss Knife (Re: Performance of Mercio’s Total Order) Mild Shock <janburse@fastmail.fm> - 2025-08-15 23:55 +0200
        Yeah, we have another name! (Was: Fuzzy Testing is your Swiss Knife) Mild Shock <janburse@fastmail.fm> - 2025-08-16 12:38 +0200
        Monte Carlo sampling the frontier version (Was: Yeah, we have another name!) Mild Shock <janburse@fastmail.fm> - 2025-08-16 12:42 +0200
        AI most hated by formal verification (Was: Fuzzy Testing is your Swiss Knife) Mild Shock <janburse@fastmail.fm> - 2025-11-26 17:02 +0100
  An NPU could give 1000x more LIPS (Was: Mercio’s Algorithm for Rational Tree Compare in Prolog) Mild Shock <janburse@fastmail.fm> - 2025-11-27 14:19 +0100
    Neural Network based dif/2 respectively (#\=)/2 (Re: An NPU could give 1000x more LIPS) Mild Shock <janburse@fastmail.fm> - 2025-11-27 14:51 +0100
      Zeus: A Language for Expressing Algorithms in Hardware (Re: Neural Network based dif/2 respectively (#\=)/2) Mild Shock <janburse@fastmail.fm> - 2025-11-27 15:03 +0100
        Googles TPU muscle in 2017 [Prolog Community is Sleepy Joe] (Re: Zeus: A Language for Expressing Algorithms in Hardware) Mild Shock <janburse@fastmail.fm> - 2025-11-27 15:23 +0100
    100% serious Giga Logical Inferences per Second (GLIPS) (Was: An NPU could give 1000x more LIPS) Mild Shock <janburse@fastmail.fm> - 2025-11-28 14:50 +0100
      The tables have turned: GigaLIP on the Laptop? (Re: 100% serious Giga Logical Inferences per Second (GLIPS) Mild Shock <janburse@fastmail.fm> - 2025-11-28 15:12 +0100

csiph-web