Groups > alt.comp.os.windows-11 > #34065

Re: MS Edge was never useful but at least it had worked for Copilot but what AI/LLM now?

From	Paul <nospam@needed.invalid>
Newsgroups	alt.comp.os.windows-11, alt.comp.os.windows-10, alt.comp.microsoft.windows
Subject	Re: MS Edge was never useful but at least it had worked for Copilot but what AI/LLM now?
Date	2026-08-27 14:31 -0400
Organization	A noiseless patient Spider
Message-ID	<116pvpp$1nesk$1@dont-email.me> (permalink)
References	<116n9nb$2hss$1@nnrp.usenet.blueworldhosting.com> <116ni78$ui00$1@dont-email.me> <116o3de$13gjm$1@dont-email.me> <6245mmxe7e.ln2@Telcontar.valinor> <116pp6j$1kkob$1@dont-email.me>

Cross-posted to 3 groups.

Show all headers | View raw

On Thu, 8/27/2026 12:38 PM, ....winston wrote:
> On 08/27/2026 3:03 AM, Carlos E.R. wrote:
>> On 2026-08-27 03:20, Paul wrote:
>>> You can run them locally, but you have to be patient. For one request
>>> to write a program, and with high reasoning enabled, it took 50 minutes
>>> to half-write the program. The high reasoning took about eight minutes
>>> of that, and the vast majority of the time was spent on token output.
>>>
>>> They tell me it would use a video card for token output,*if* the question
>>> fits entirely within the video card. But with my puny hardware resources, it
>>> always uses the CPU for token output, which can be slow.
>>
>> There is a chip that can accelerate the ai more than gpus. I read about it in the ieee spectrum magazine, but I don't remember the name of the thing. Laptops will come with that hardware so they can handle ai locally with reasonable battery usage.
>>
>> probably asking an ai about this it will tell better details than me :-D
>>
> 
> 
> Iirc,
> NPU - neural processing unit
> ASIC - app specific
>  - i.e. for a specific AI task
> 
> Also another older type of chip/circuitry based on gate array(?) and re-programmable.
> 

There is a diagram of an NPU here.

  https://intel.github.io/intel-npu-acceleration-library/npu.html

An NPU can be 50 TOPS. a GPU can be 1000 TOPS, but the operations per second
depend on the representation (INT8, FP16). The FP16 rate could be much lower.
An NPU might not even have FP16 capability. The CPU can do any math you can
think of, but at quite-low TOPS rates.

Software loading models, does not "dispatch" onto all your disparate
acceleration devices. The CPU manufacturers like to "total" the TOPS numbers
from the three accelerators they've got. Whereas the model running software
laughs at this silliness, and only uses the most-productive acceleration
device and it ignores the others. If you include a slow device and shard
a calculation, the slow device holds up everybody. Dispatching is not
nearly as optimistic as Intel and AMD would have you believe.

This is one of the reasons, that NPU graph in Task Manager on your new laptop,
it might never ever lift off the peg. For the NPU to "light up", someone
has to go out of their way, to use it. They have to think of a specific
application, make a model of the same type as the single format the
NPU supports, hope there is a driver for the NPU, *then* load it up and
run it. That means a company like Intel, writes that program :-)
The NPU then, is a kind of bar bet. "I betcha I can make that line move...
Hold my beer"

     Paul

Thread

csiph-web