AMD marketing is hoping the “AI” branding is a positive. Antidotally, I know many consumers who are not sold on AI. This branding could actually hurt sales.
We are dealing with a hype, but the reality is that AI would change everything we do. Local models will start being helpful in [more] unobtrusive ways. Machines with decent local NPUs would be usable for longer before they feel too slow.
Your comment is almost completely irrelevant to what the parent is saying. "AI would change everything we do" has nothing to do with "This new chip along with bloat from Windows enables new workflows for you". If you have been paying attention, you'd know that NPUs from these new CPUs barely made any difference from a consumer's perspective.
I am bullish on AI being used in all sorts of useful and discreet and non-discreet ways in the present and future. However I am exceedingly skeptical of NPUs being some winning bet.
No one is running LLMs on current gen NPUs so if we will in the future its a long time coming. Unless they can demonstrate some real (and not marketing) wins I remain skeptical that a large NPU for LLMs is the future.
I can totally see NPU accelerating simple tasks, but to be worth the silicon they have a ways to go imo.
99% of people don't need or want a dev workstation. My travel laptop is 7+ years old and I couldn't tell you the difference between it and a current flagship in terms of browsing and everyday tasks.
I will not lie, I find LLMs useful but the desktop experience is pretty polished already. NPUs seem to be an attempt to ride the AI bandwagon with very little to show for it so far.
I see a lot of people put so much stock in a future of local models, but I am not convinced we’re on track for a world where most people are running local models. Especially when you have companies like Microsoft explicitly trying to make a world people don’t even own their software/hardware anymore and do everything on the cloud.
Indeed, I was buying a laptop for my wife, and she was viscerally against "Ryzen AI": I don't want a CPU with builtin AI to spy on my screen all the time!
Meanwhile, the corresponding "non-standard" desktop PC is the Framework Desktop, which with the Ryzen AI Max+ 395 can use 120GB of its 128GB RAM for the GPU: How to Run a One Trillion-Parameter LLM Locally: An AMD Ryzen™ AI Max+ Cluster Guide https://www.amd.com/en/developer/resources/technical-article...
Hoe much dedicated cache do these NPUs have? Because it's easy enough to saturate the memory bandwidth using the CPU for compute, never mind the GPU. Adding dark silicon for some special operations isn't going to make out memory bandwidth faster.
The Ryzen AI line is actually great if deployed to an entire org as the bottom tier, as it garuantees every device has a 50 TOPs NPU. We deploy local software at $STARTUP and this makes deployment to a Windows corp more predictable.
> This makes them AMD’s first desktop chips to qualify for Microsoft’s Copilot+ PC label, which enables a handful of unique Windows 11 features like Recall and Click to Do.
Microsoft: "Friendship ended with Intel, now AMD is my best friend"
Yeah, but all of this is pointless when RAM is as expensive as two CPUs by itself - if it's even in stock. AMD/Intel should focus on that first if they want to save their DIY business at all - which I'm starting to doubt they don't
The RAM shortage is... a shortage. It's temporary by nature. RAM didn't suddenly get 4x more expensive to produce, it's just in high demand right now. Supply will eventually catch up even if it takes a few years.
> This makes them AMD’s first desktop chips to qualify for Microsoft’s Copilot+ PC label, which enables a handful of unique Windows 11 features like Recall and Click to Do.
This is not the selling point they think it is.
The problem I see with the AM5 socket is simply the fact that DD5 RAM to support it is just too expensive. So this will not really make the big impact they were hopin for.
>> This makes them AMD’s first desktop chips to qualify for Microsoft’s Copilot+ PC label, which enables a handful of unique Windows 11 features like Recall and Click to Do.
> This is not the selling point they think it is.
I'd even say it's an anti-point and if I were them I'd just hide that or, if they are obliged for some reason, but it in small print so that people don't notice.
PC desktop chips have 128 bit busses to right ? From what I know theoretical maximum memory bandwidth of chips is less than 100gbps - which is less than like base M5/M4 chips.
So no matter how much compute you stuff in there it's going to be shit for AI ?
PC architecture is not adapting to AI workloads at all, and no signs of that changing in years to come. I would not be surprised if your phone was more capable of running AI models than an average desktop - especially given gpu pricing.
Is this fast GPU like instructions that anyone can use in any operating system to run any open sourced LLMs on CPU with all your RAM rather than on a discrete GPU with its own limited amount of VRAM?
Or is this a proprietary thing that only works in Windows for some specific use cases and irrelevant for Linux users?
40 comments
[ 2.6 ms ] story [ 63.4 ms ] threadNo one is running LLMs on current gen NPUs so if we will in the future its a long time coming. Unless they can demonstrate some real (and not marketing) wins I remain skeptical that a large NPU for LLMs is the future.
I can totally see NPU accelerating simple tasks, but to be worth the silicon they have a ways to go imo.
99% of people don't need or want a dev workstation. My travel laptop is 7+ years old and I couldn't tell you the difference between it and a current flagship in terms of browsing and everyday tasks.
I will not lie, I find LLMs useful but the desktop experience is pretty polished already. NPUs seem to be an attempt to ride the AI bandwagon with very little to show for it so far.
I wanted a better strix halo (which has 128GB unified RAM and 40cu on the 8080s (or something) iGPU).
This looks like normal Ryzen mobile chips + but with fewer cus.
Microsoft: "Friendship ended with Intel, now AMD is my best friend"
This is not the selling point they think it is.
The problem I see with the AM5 socket is simply the fact that DD5 RAM to support it is just too expensive. So this will not really make the big impact they were hopin for.
> This is not the selling point they think it is.
I'd even say it's an anti-point and if I were them I'd just hide that or, if they are obliged for some reason, but it in small print so that people don't notice.
So no matter how much compute you stuff in there it's going to be shit for AI ?
PC architecture is not adapting to AI workloads at all, and no signs of that changing in years to come. I would not be surprised if your phone was more capable of running AI models than an average desktop - especially given gpu pricing.
Is this fast GPU like instructions that anyone can use in any operating system to run any open sourced LLMs on CPU with all your RAM rather than on a discrete GPU with its own limited amount of VRAM?
Or is this a proprietary thing that only works in Windows for some specific use cases and irrelevant for Linux users?