This looks cool a chiplet can fit 30m params so the biggest card can fit qwen 3.8 27b it would be cool to see some benchmarks on things like that publically.
My understanding (perhaps outdated) is that manufacturing variability is a key challenge for analog computing. Digital designs are also fundamentally analogue under the hood, but if you only need to resolve a 0 or 1 you are much more tolerant of any source of noise. I wouldn't mind hearing even a little bit more from Mythic about how they make this work in practice.
A 2026 EE Times article [1] refers to "compensation" and "calibration" techniques.
I'd assume it does some type of calibrate per device (regularly?) or the design is such that it's differential so things cancel out.
Note it's also on 28nm for the analogue bits because yes it's harder.
You reminded me the anecdote about every SID chip sounding different. If you hear a recording of a C64 made song, it’s unique to that chip (and somewhat to the machine as well, timing, crystal, etc.).
Different generations of chips, or various cheap clone chips, sure.
Within one generation, evidence of significant differences seems weaker. Sure, manufacturing had loose tolerances, you could end up with a "faulty" chip, but most SIDs sounded the same. You could also end up with a 6510 that crashed after 30 minutes.
If you go for a full curve over the space, then yes.
But like NAND, there is room for, I don’t know what to call it, “quantization”? You get a few values out instead of just binary.
My guess is, like other issues of precision, errors can get out of hand if you are not careful.
But being an EE in another life, I can tell you the power waste and slowness of ALUs is kinda wild.
I definitely believe that Gaming on CPUs is like ML on GPUs. Slow, and waiting for something more appropriate to come along.
Don’t know if these guys are the ones to do it (and they are, ahem, not alone). But someone will deliver 100x to 1000x boost either in speed, power efficiency, or both.
I wish they would have done what Taalas did with chatjimmy.ai and just directly host a model for us to view, rather than just claiming it’s 50x faster than Nvidia/groq. Their claim is specifically for a 1 trillion param model. So they could have just grabbed GLM 5.2, or similar, and hosted it.
Joke’s on us, all of their pages are LLM pages! LLM generated, that is.
Btw, can guarantee that they are not ready to demonstrate that yet. They’re using 2D FLASH with 30M weights per die [1], so to get to 1T they will need… 33,333 dies. Interesting scaling problem to say the least
But they also declare having a "Mead" technology that stores at least 175b NNs in a single chip through 3D stacking - see https://www.mythic.ai/mead and other posts in this page.
A confusing thing is that the goal is tackled through a number of proposals... Why Vanguard if they have Mead? If Mead, how to get the memory integration that are explicit on Vanguard?
Their numbers look too good to be true, they have no identified customers and the whole site is generated, but I think the principle behind it is good. If they can pull off the error correction needed to make analog reliable we might have a great new option for cheaper more eco friendly AI. Then again it could turn out to be a total scam.
Bottom of page linked from HN (currently https://www.mythic.ai/) indicates they're hoping to demonstrate something that could that in 2028 or later, and both Nvidia and Cerebra are looking at 10x'ing models to 10T+ plus in 2027.
So they may never catch up on LLMs.
They're a good fit for the companies they're working with and have taken investment from, ex. Toyota, that aren't doing LLMs.
Notice that they do not talk about SRAM when they present the M1 chiplets - but you have to store the kv-cache etc. somewhere to run LLMs.
The technology that could run LLMs should be the "Vanguard", but as the homepage says, "the M1 (scope: Edge/Cameras/Drones) is there, the Vanguard should be a reality in 2027".
I'm pretty sure the answer is that their chip is uniquely unsuited for LLMs, probably because the write speed might be horrendously slow. Most likely too slow for storing the context window in multi user workloads.
Why? I have not seen anything outlandish for a NN implementation (vs a NN simulation).
> can pull off the error correction
There lie the issues that have not been explained. Analog computing means: * the costly digital-to-analog at the input and analog-to-digital at the output; * sensitivity to environmental conditions such as temperature; * signal dispersion hence the need to boost it in the path.
Temperature is a relatively trivial issue because it can either be compensated or the chip simply kept at a constant temperature (they’re low power chips anyway, throttling could be skipped to simplify the design). The ADCs and DACs are indeed the main thing though. The whole analog compute game is about making those power efficient and scaled enough that the math still works out in your favor. The demos do work though, this company is far from the only one; see [1] or [2] for example.
Flash NAND can routinely be bought with 4 bits per cell, perhaps even 5 soon (QLC and PLC drives). Since it has been proven to store 4 digital bits at production scale, I’m willing to bet an analog architecture running an LLM should be able to yield the performance analog of a 4-8 bit quantized model. Where in that 4-8 range is pretty crucial, but it depends on the specific design.
My 15 second read of just the front page aligned with you, but when I saw replies pushing back, I went back and loaded News, then cross-verified some of the claims. It's real.
Bulk bitwise operations in a memory array driven by an out-of-spec controller is a thing, but I'm not sure why they're using NAND for this and not DRAM like everyone else, and why it's so small (just 80M weights). And the page is insufferable and doesn't tell anything about how it works. Everything about it seems off.
The vanguard chiplets even steps down to 30M… but they do claim to have a “Mead” design [1] designed to run GPT-3 in a single chip.
3D NAND flash can indeed routinely store hundreds of GB per die, so that’s proven. The question is about all the peripheral circuitry needed. Each attention block would need its own KV cache (i.e. SRAM or DRAM somewhere), plus DAC/ADC inputs and outputs, unless they figure out a way to keep it analog all the way (really cool but unlikely).
I think this field is very interesting, at least from a technology point of view. Whether it works out or not will sadly be a matter of economics more than physics I fear.
This might end up being a strong use case for FRAM, nonvolatile, high density, and sram fast. In an alternative universe where fram infra was as widespread as flash, memory is fast, cheap, and nonvolatile. It’s just that flash built the infrastructure and climbing that ladder is nearly impossible.
I wouldn't really say SRAM-fast. Maybe DRAM-fast? At least in write speed. But the thing is, you don't need high write speed for this application. And read speed should be the same between flash and FeFETs.
Btw, I've seen some roadmaps in which foundries were heading towards integrating ferroelectrics and floating gates to accelerate switching. The two technologies are not mutually exclusive really, so I think we might see something in the relatively near future. The crazy memory market may help!
I’m willing to believe that one could design a little circuit that multiplies a number stored in a floating-gate MOSFET by an analog input and another circuit that adds the result to an accumulator (in fact there seems to be some prior art from 1989!). But I don’t know who would fab this - I doubt this is something doable is TSMC’s standard process.
And maybe one can use NAND or NOR flash with a different sort of controller to do analog computation, and maybe one could convince a flash memory fab to build it for you.
But there is no mention on the site of how they expect to manufacture the thing.
36 comments
[ 0.22 ms ] story [ 4.8 ms ] threadA 2026 EE Times article [1] refers to "compensation" and "calibration" techniques.
[1] https://www.eetimes.com/mythic-rises-from-the-ashes-with-125...
Within one generation, evidence of significant differences seems weaker. Sure, manufacturing had loose tolerances, you could end up with a "faulty" chip, but most SIDs sounded the same. You could also end up with a 6510 that crashed after 30 minutes.
https://www.youtube.com/watch?v=yadWu7hs4MU
https://www.youtube.com/watch?v=n73-ipWMCCY
But like NAND, there is room for, I don’t know what to call it, “quantization”? You get a few values out instead of just binary.
My guess is, like other issues of precision, errors can get out of hand if you are not careful.
But being an EE in another life, I can tell you the power waste and slowness of ALUs is kinda wild.
I definitely believe that Gaming on CPUs is like ML on GPUs. Slow, and waiting for something more appropriate to come along.
Don’t know if these guys are the ones to do it (and they are, ahem, not alone). But someone will deliver 100x to 1000x boost either in speed, power efficiency, or both.
I wish they would have done what Taalas did with chatjimmy.ai and just directly host a model for us to view, rather than just claiming it’s 50x faster than Nvidia/groq. Their claim is specifically for a 1 trillion param model. So they could have just grabbed GLM 5.2, or similar, and hosted it.
Btw, can guarantee that they are not ready to demonstrate that yet. They’re using 2D FLASH with 30M weights per die [1], so to get to 1T they will need… 33,333 dies. Interesting scaling problem to say the least
[1] https://www.mythic.ai/vanguard
A confusing thing is that the goal is tackled through a number of proposals... Why Vanguard if they have Mead? If Mead, how to get the memory integration that are explicit on Vanguard?
Bottom of page linked from HN (currently https://www.mythic.ai/) indicates they're hoping to demonstrate something that could that in 2028 or later, and both Nvidia and Cerebra are looking at 10x'ing models to 10T+ plus in 2027.
So they may never catch up on LLMs.
They're a good fit for the companies they're working with and have taken investment from, ex. Toyota, that aren't doing LLMs.
The technology that could run LLMs should be the "Vanguard", but as the homepage says, "the M1 (scope: Edge/Cameras/Drones) is there, the Vanguard should be a reality in 2027".
Why? I have not seen anything outlandish for a NN implementation (vs a NN simulation).
> can pull off the error correction
There lie the issues that have not been explained. Analog computing means: * the costly digital-to-analog at the input and analog-to-digital at the output; * sensitivity to environmental conditions such as temperature; * signal dispersion hence the need to boost it in the path.
Maybe checking the patents they registered?
Flash NAND can routinely be bought with 4 bits per cell, perhaps even 5 soon (QLC and PLC drives). Since it has been proven to store 4 digital bits at production scale, I’m willing to bet an analog architecture running an LLM should be able to yield the performance analog of a 4-8 bit quantized model. Where in that 4-8 range is pretty crucial, but it depends on the specific design.
[1] https://www.nature.com/articles/s41928-023-01010-1 [2] https://www.nature.com/articles/s41586-022-04992-8
Which means connecting over 30 chiplets to run a Qwen 3.8 27b and over 3000 chiplets to run Qwen3.8-2.4T-A95B. Cost? Space? Feasibility?
3D NAND flash can indeed routinely store hundreds of GB per die, so that’s proven. The question is about all the peripheral circuitry needed. Each attention block would need its own KV cache (i.e. SRAM or DRAM somewhere), plus DAC/ADC inputs and outputs, unless they figure out a way to keep it analog all the way (really cool but unlikely).
I think this field is very interesting, at least from a technology point of view. Whether it works out or not will sadly be a matter of economics more than physics I fear.
[1] https://www.mythic.ai/mead
Very unlikely for the connection to the cache RAM, seemingly impossible if we want to get an articulate output :)
Btw, I've seen some roadmaps in which foundries were heading towards integrating ferroelectrics and floating gates to accelerate switching. The two technologies are not mutually exclusive really, so I think we might see something in the relatively near future. The crazy memory market may help!
And maybe one can use NAND or NOR flash with a different sort of controller to do analog computation, and maybe one could convince a flash memory fab to build it for you.
But there is no mention on the site of how they expect to manufacture the thing.
They are using GlobalFoundries’ 28nm node for the floating gate transistors, afaiu, which they then bond onto a TSCM 5nm digital I/O wafer.
Analog computation’s principles are sound. It’s mostly doing matrix vector multiplications though. The rest is digital.