Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.prolog > #14931
| From | Mild Shock <janburse@fastmail.fm> |
|---|---|
| Newsgroups | comp.lang.prolog |
| Subject | Vertex AI Training is more expensive? (Was: Give Julio Di Egidio the bloody money [3 RMB¥ MiniMind]) |
| Date | 2025-10-21 00:32 +0200 |
| Message-ID | <10d6dao$au23$1@solani.org> (permalink) |
| References | (2 earlier) <10cn157$10qq$1@solani.org> <10co9kp$12u6u$1@solani.org> <10coa13$12ucf$1@solani.org> <10d06ch$17p4u$1@solani.org> <10d07m6$17pvl$2@solani.org> |
Hi, Vertex AI Training is more expensive: "Vertex AI provides a managed training service that enables you to operationalize large scale model training. You can use Vertex AI to run training applications based on any machine learning (ML) framework on Google Cloud infrastructure. Vertex AI also has integrated support that simplifies the preparation process for model training and serving for the PyTorch, TensorFlow, scikit-learn, and XGBoost frameworks. https://cloud.google.com/products/calculator For 10 Jobs à 10 hours, it wants 1000 USD from me. The accelerator is TPU V3. Nevertheless might be the cheaper option to use the RTX™ 6000 for 720 hours. ChatGPT gives me this calculation: Scenario I: Single TPU, RTX 6000 wins 4.43×10^19 FLOPs versus ~2.36×10^20 FLOPs Scenario II: 8 chips TPU, RTX 6000 looses ~3.54×10^20 FLOPs versus ~2.36×10^20 FLOPs Bye Mild Shock schrieb: > Hi, > > Give Julio Di Egidio the bloody money. He is > craving for 300 USD so that he can buy the > ISO Prolog core standard. Just imagine he would want > > to build a MiniMind. Just lets put some more > prespective on the current costs: > > This open-source project aims to train a super-small > language model MiniMind with only 3 RMB cost and > 2 hours, starting completely from scratch. The > MiniMind series is extremely lightweight, with the > smallest version being 1/7000 the size of GPT-3, > making it possible to train quickly on even the > most ordinary personal GPUs. > https://github.com/jingyaogong/minimind/blob/master/README_en.md > > ChatGPT tells me that most of the numbers > are correct when you rent a GPU by the hour. > But what about a 100% ownership of a GPU for > > a year. I find this might cost 12'000 USD. > One has to separate platforms for execution from > those platforms for training: > > GEX44: for AI inference > Nvidia RTX™ 4000, 184 EUR / month > > GEX130: for AI training > NVIDIA RTX™ 6000, 813 EUR / month > https://www.hetzner.com/dedicated-rootserver/matrix-gpu/ > > Bye > > Mild Shock schrieb: >> Hi, >> >> Thinks are definitively accelerating. I really would >> like to use an AI that knows about all the News of today. >> This bloody cut date is so annoying. >> >> Further indicative that AI is accelerating: >> >> In August 2025, Sam Altman dropped a bombshell: >> >> *months, not years: Rushing GPT-6* >> In August 2025, Sam Altman dropped a bombshell: >> GPT-6 is already in development and coming sooner >> than you think. Not in two years, but >> potentially in months. >> https://www.youtube.com/watch?v=44mJb5sKji0 >> >> Karpathy, coined vibe coding, released in October 2025: >> >> *nanochat: The best ChatGPT that $100 can buy* >> This repo is a full-stack implementation of an >> LLM like ChatGPT in a single, clean, minimal, >> hackable, dependency-lite codebase. nanochat is >> designed to run on a single 8XH100 node via >> scripts like speedrun.sh, that run the >> entire pipeline start to end. >> https://github.com/karpathy/nanochat >> >> Bye >> >> Mild Shock schrieb: >>> Hi, >>> >>> But not only Mobie AI and Desktop AI are making >>> a broader imprint now. We might also experience >>> Workstation AI, with a 3'000.- USD price tag: >>> >>> > You Can't Buy This... Yet! The NVIDIA GB10 from Dell >>> > The New Superchip that Terrifies the Cloud! >>> > https://www.youtube.com/watch?v=x1qViw4xyVo >>> >>> So whats going on? I was asking Phind, which is >>> driven by a 70B model tailored towards developers: >>> >>> Q: Is there an AI inflection point right now , >>> with NPUs in mobile, desktop and workstation >>> >>> A: Evidence of the Inflection Point >>> >>> - Mobile Leadership >>> NPUs originated in smartphones >>> Now becoming ubiquitous across all device types >>> Enabling sophisticated AI features at consumer price points >>> >>> - Desktop Revolution >>> Major manufacturers implementing NPUs across product lines >>> Apple's Neural Engine integrated into M-series chips >>> Qualcomm, Intel, and AMD incorporating AI accelerators >>> >>> - Workstation Transformation >>> Professional-grade NPUs in mobile workstations >>> Demonstrated superior performance for AI-specific tasks >>> Enabling local processing of previously cloud-dependent workloads >>> >>> https://www.phind.com/search/cmgs1s6jv00023h67g5z2aaa0 >>> >>> Bye >>> >>> Mild Shock schrieb: >>>> Hi, >>>> >>>> It seems I am having problems pacing with >>>> all the new fancy toys. Wasn't able to really >>>> benchmark my NPU from a Desktop AI machine, >>>> >>>> picked the wrong driver. Need to try again. >>>> What worked was benchmarking Mobile AI machines. >>>> I just grabbed Geekbench AI and some devices: >>>> >>>> USA Fab, M4: >>>> >>>> sANN hANN qANN >>>> iPad CPU 4848 7947 6353 >>>> iPad GPU 9752 11383 10051 >>>> iPad NPU 4873 36544 *51634* >>>> >>>> China Fab, Snapdragon: >>>> >>>> sANN hANN qANN >>>> Redmi CPU 1044 950 1723 >>>> Redmi GPU 480 905 737 >>>> Redmi NNAPI 205 205 469 >>>> Redmi QNN 226 226 *10221* >>>> >>>> Speed-Up via NPU is factor 10x. See the column >>>> qANN which means quantizised artificial neural >>>> networks, when NPU or QNN is picked. >>>> >>>> The mobile AI NPUs are optimized using >>>> mimimal amounts of energy, and minimal amounts >>>> of space squeezing (distilling) everything >>>> >>>> into INT8 and INT4. >>>> >>>> Bye >>>> >>>> Mild Shock schrieb: >>>>> Hi, >>>>> >>>>> The change from 378 ms to 286 ms is around 25-30% >>>>> is insane. But I did both tests on a novel AI CPU. >>>>> To be precise on a AMD Ryzen AI 7 350. >>>>> >>>>> But somehow I picked up rumors that AI CPUs now >>>>> might do Neural Network Branch Prediction. The >>>>> idea seems to exist in hardware at least since (2012): >>>>> >>>>> Machine learning and artificial intelligence are >>>>> the current hype (again). In their new Ryzen >>>>> processors, AMD advertises the Neural Net >>>>> Prediction. It turns out this is was already >>>>> used in their older (2012) Piledriver architecture >>>>> used for example in the AMD A10-4600M. It is also >>>>> present in recent Samsung processors such as the >>>>> one powering the Galaxy S7. What is it really? >>>>> https://chasethedevil.github.io/post/the_neural_network_in_your_cpu/ >>>>> >>>>> It can be done with Convoluted Neural Networks (CNN): >>>>> >>>>> BranchNet: A Convolutional Neural Network to >>>>> Predict Hard-To-Predict Branches >>>>> To this end, Tarsa et al. proposed using convolutional >>>>> neural networks (CNNs) that are trained at >>>>> compiletime to accurately predict branches that >>>>> TAGE cannot. Given enough profiling coverage, CNNs >>>>> learn input-independent branch correlations. >>>>> https://microarch.org/micro53/papers/738300a118.pdf >>>>> >>>>> Interstingly the above shows cases a PGO based >>>>> Machine Learning for Branch Predictors. No clue >>>>> how they construct the CPU, that they can feed >>>>> >>>>> it with offline constructed neural neutworks for >>>>> their own execution. Maybe an optimizer uses it? >>>>> But I guess a more modern solutions would not only >>>>> >>>>> use CNN, but also an Attention Mechanism. >>>>> >>>>> Bye >>>>> >>>>> Mild Shock schrieb: >>>>>> Hi, >>>>>> >>>>>> I spent some time thinking about my primes.pl >>>>>> test. And came to the conclusion that it >>>>>> mainly tests the Prolog ALU. Things like >>>>>> >>>>>> integer successor or integer modulo. Then >>>>>> I found that Java has Math.floorMod() which >>>>>> I wasn't using yet. And peng results are better: >>>>>> >>>>>> /* Dogelog Player 2.1.2 for Java, today */ >>>>>> ?- time(test). >>>>>> % Zeit 286 ms, GC 1 ms, Lips 26302430, Uhr 15.10.2025 02:31 >>>>>> true. >>>>>> >>>>>> Maybe the Java backend picks a CPU instruction >>>>>> for Math.floorMod() instead of executing the >>>>>> longer code sequence that is needed to correct >>>>>> >>>>>> rem/2 into mod/2. Who knows. I also reorganized >>>>>> the code a little bit, and eliminated an extra >>>>>> method call in all arithmetic functions, by >>>>>> >>>>>> inlining the arithmetic function body in the >>>>>> evaluable predicate definition code. Comparison >>>>>> to old measurements and some measurements of >>>>>> >>>>>> other Prolog systems: >>>>>> >>>>>> /* Dogelog Player 2.1.2 for Java, weeks ago */ >>>>>> ?- time(test). >>>>>> % Zeit 378 ms, GC 1 ms, Lips 19900780, Uhr 28.08.2025 17:44 >>>>>> true. >>>>>> >>>>>> /* SWI-Prolog 9.0.4 */ >>>>>> ?- time(test). >>>>>> % 7,506,639 inferences, 0.363 CPU in 0.362 seconds >>>>>> (100% CPU, 20693560 Lips) >>>>>> true. >>>>>> >>>>>> /* Scryer Prolog 0.9.4-639 */ >>>>>> ?- time(test). >>>>>> % CPU time: 0.365s, 7_517_613 inferences >>>>>> true. >>>>>> >>>>>> /* Trealla Prolog 2.82.23-3 */ >>>>>> ?- time(test). >>>>>> % Time elapsed 0.868s, 11263917 Inferences, 12.983 MLips >>>>>> true. >>>>>> >>>>>> Bye >>>>>> >>>>>> P.S.: The code uses the hated mathematical mod/2, >>>>>> and not the cheaper rem/2 that CPUs usually have: >>>>>> >>>>>> test :- >>>>>> len(L, 1000), >>>>>> primes(L, _). >>>>>> >>>>>> primes([], 1). >>>>>> primes([J|L], J) :- >>>>>> primes(L, I), >>>>>> K is I+1, >>>>>> search(L, K, J). >>>>>> >>>>>> search(L, I, J) :- >>>>>> mem(X, L), >>>>>> I mod X =:= 0, !, >>>>>> K is I+1, >>>>>> search(L, K, J). >>>>>> search(_, I, I). >>>>>> >>>>>> mem(X, [X|_]). >>>>>> mem(X, [_|Y]) :- >>>>>> mem(X, Y). >>>>>> >>>>>> len([], 0) :- !. >>>>>> len([_|L], N) :- >>>>>> N > 0, >>>>>> M is N-1, >>>>>> len(L, M). >>>>>> >>>>>> Mild Shock schrieb: >>>>>>> Hi, >>>>>>> >>>>>>> WebPL is already outdated I guess. It doesn't >>>>>>> show the versions of the other Prolog systems >>>>>>> it is using. While I had these results for >>>>>>> >>>>>>> the primes example in the WebPL playground: >>>>>>> >>>>>>> /* Trealla Prolog WASM */ >>>>>>> (23568.9ms) >>>>>>> >>>>>>> When I run the example here: >>>>>>> >>>>>>> https://php.energy/trealla.html >>>>>>> >>>>>>> I get better results: >>>>>>> >>>>>>> /* trealla-js 0.27.1 */ >>>>>>> >>>>>>> ?- time(test). >>>>>>> % Time elapsed 9.907s, 11263917 Inferences, 1.137 MLips >>>>>>> >>>>>>> Bye >>>>>> >>>>> >>>> >>> >> >
Back to comp.lang.prolog | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
WebPL is already outdated Mild Shock <janburse@fastmail.fm> - 2025-08-17 18:37 +0200
Heap/Stack versus WAM resp. ZIP (Was: WebPL is already outdated) Mild Shock <janburse@fastmail.fm> - 2025-08-18 14:52 +0200
Trealla knows Program Sharing (PS) Tricks ? (Was: Heap/Stack versus WAM resp. ZIP) Mild Shock <janburse@fastmail.fm> - 2025-08-18 15:06 +0200
Smarter Partial Strings would use Program Sharing ['$append'/3] (Was: Trealla knows Program Sharing (PS) Tricks ?) Mild Shock <janburse@fastmail.fm> - 2025-08-18 15:42 +0200
Static Shunting is even better [Dogelog Player] (Was: Smarter Partial Strings would use Program Sharing ['$append'/3]) Mild Shock <janburse@fastmail.fm> - 2025-08-18 15:49 +0200
The Artificial Intelligence Flip: Acer Swift Go! (Was: WebPL is already outdated) Mild Shock <janburse@fastmail.fm> - 2025-08-31 23:56 +0200
2025 will be last year we hear of Python (Re: The Artificial Intelligence Flip: Acer Swift Go!) Mild Shock <janburse@fastmail.fm> - 2025-09-01 00:45 +0200
Apertus: With love, from Switzerland [02 Sept 2025] (Re: 2025 will be last year we hear of Python) Mild Shock <janburse@fastmail.fm> - 2025-09-05 00:36 +0100
Don't try this (Was: Apertus: With love, from Switzerland [02 Sept 2025] ) Mild Shock <janburse@fastmail.fm> - 2025-09-05 01:03 +0100
AI means ambracing Non-Linearity (Was: 2025 will be last year we hear of Python) Mild Shock <janburse@fastmail.fm> - 2025-09-19 10:01 +0200
AI soaked PCs: Is there a Copilot+ Prolog? (Was: AI means ambracing Non-Linearity) Mild Shock <janburse@fastmail.fm> - 2025-09-19 10:10 +0200
The morning coffee incident [Prolog Community] (Was: AI soaked PCs: Is there a Copilot+ Prolog?) Mild Shock <janburse@fastmail.fm> - 2025-09-19 14:38 +0200
Root Cause Prediction for Your Brain (Was: The morning coffee incident [Prolog Community]) Mild Shock <janburse@fastmail.fm> - 2025-09-19 18:22 +0200
Please delete my account and all my posts on SWI-Prolog discourse (Re: Root Cause Prediction for Your Brain) Mild Shock <janburse@fastmail.fm> - 2025-09-19 18:38 +0200
I will consult a Lawyer of mine (Was: Please delete my account and all my posts on SWI-Prolog discourse) Mild Shock <janburse@fastmail.fm> - 2025-09-19 18:42 +0200
Scryer Prolog unify_with_occurs_check/2 doesn't scale (Was: WebPL is already outdated) Mild Shock <janburse@fastmail.fm> - 2025-09-19 16:08 +0200
How bad is Rust, can JavaScript beat it? (Was: Scryer Prolog unify_with_occurs_check/2 doesn't scale) Mild Shock <janburse@fastmail.fm> - 2025-09-19 16:18 +0200
unify_with_occurs_check/2 might have been fixed (Was: Scryer Prolog unify_with_occurs_check/2 doesn't scale) Mild Shock <janburse@fastmail.fm> - 2025-09-25 01:50 +0200
Had to Rollback my Jaffar Unification (Was: unify_with_occurs_check/2 might have been fixed) Mild Shock <janburse@fastmail.fm> - 2025-09-25 01:59 +0200
Trealla Prolog might apply "frozeness" to cyclic terms (Was: Had to Rollback my Jaffar Unification) Mild Shock <janburse@fastmail.fm> - 2025-09-25 02:06 +0200
Non-intrusive through "frozen" subcategories (Was: Trealla Prolog might apply "frozeness" to cyclic terms) Mild Shock <janburse@fastmail.fm> - 2025-09-25 02:21 +0200
Scryer Prolog occurs check cannot do hydra (Was: Scryer Prolog unify_with_occurs_check/2 doesn't scale) Mild Shock <janburse@fastmail.fm> - 2025-09-26 12:19 +0200
WebPL and Scryer Prolog are bad examples (Was: WebPL is already outdated) Mild Shock <janburse@fastmail.fm> - 2025-10-13 09:49 +0200
Who will win Shift-Reduce or Tabled DCG? [AI Boom] (Was: WebPL and Scryer Prolog are bad examples) Mild Shock <janburse@fastmail.fm> - 2025-10-13 15:09 +0200
primes.pl mainly tests the Prolog ALU [mod/2 vs rem/2] (Was: WebPL is already outdated) Mild Shock <janburse@fastmail.fm> - 2025-10-15 02:38 +0200
25-30% is insane, Neural Network Branch Prediction? (Was: primes.pl mainly tests the Prolog ALU) Mild Shock <janburse@fastmail.fm> - 2025-10-15 04:33 +0200
NPUs (Neural Processing Units) are the new normal (Was: 25-30% is insane, Neural Network Branch Prediction?) Mild Shock <janburse@fastmail.fm> - 2025-10-15 16:04 +0200
Ask Phind: AI inflection point right now [End 2025] (Was: NPUs (Neural Processing Units) are the new normal) Mild Shock <janburse@fastmail.fm> - 2025-10-15 16:10 +0200
Eat Tteokbokki before SkyNet kills you [$100 ChatGPT] (Was: Ask Phind: AI inflection point right now [End 2025]) Mild Shock <janburse@fastmail.fm> - 2025-10-18 15:57 +0200
Give Julio Di Egidio the bloody money [3 RMB¥ MiniMind] (Re: Eat Tteokbokki before SkyNet kills you [$100 ChatGPT]) Mild Shock <janburse@fastmail.fm> - 2025-10-18 16:19 +0200
Vertex AI Training is more expensive? (Was: Give Julio Di Egidio the bloody money [3 RMB¥ MiniMind]) Mild Shock <janburse@fastmail.fm> - 2025-10-21 00:32 +0200
The Love Affair: OpenAI and AMD (Was: NPUs (Neural Processing Units) are the new normal) Mild Shock <janburse@fastmail.fm> - 2025-10-18 18:59 +0200
The NPU in your Browser [WebNN by W3C] (Was: NPUs (Neural Processing Units) are the new normal) Mild Shock <janburse@fastmail.fm> - 2025-10-26 08:39 +0100
Fuzzy Alert: Boris the Loris on the Dancefloor (Was: The NPU in your Browser [WebNN by W3C]) Mild Shock <janburse@fastmail.fm> - 2025-10-26 11:33 +0100
csiph-web