> In practice it is possible, in a flash memory made on a mature process such as 40nm, to store reliably a range of charges that correspond to a digital resolution of 8 bits.
> When a charge is programmed into a flash memory device, its electric field
has an effect on any signal passing through it. In the Mythic architecture, the
flash transistor acts as a variable resistor that reduces the signal level passing
to the output. That reduction is proportional to the analog value stored in the
memory. This simple effect implements the multiplication stage found in
DNN calculations. The accumulation process, in which the output from each
of those calculations is summed, is handled by aggregating the output of an
entire column of memory cells. Thanks to these two properties, the Mythic
architecture can process an entire input vector in a single step rather than
iterating at high speed as in a digital processor
> Analog computing provides the ultimate compute-in-memory processing element. The term compute-in-memory is used very broadly and can mean many things. Our analog compute takes compute-in-memory to an extreme, where we compute directly inside the memory array itself. This is possible by using the memory elements as tunable resistors, supplying the inputs as voltages, and collecting the outputs as currents. We use analog computing for our core neural network matrix operations, where we are multiplying an input vector by a weight matrix.
> Analog computing provides several key advantages. First, it is amazingly efficient; it eliminates memory movement for the neural network weights since they are used in place as resistors. Second, it is high performance; there are hundreds of thousands of multiply-accumulate operations occurring in parallel when we perform one of these vector operations. Given these two properties, analog computing is the core of our high-performance yet highly-efficient system.
This is a very specific technology that has serious scaling issues, and the neural networks coming out today are huge in comparison to YOLOv5 and ResNet. The company has already failed once. This will probably have its niche in some computer vision stuff for a little while, but models have already outgrown it.
This particular implementation of the idea of analog matrix processor implemented at low level may fail, but some different implementation will succeed.
There are some fundamental limits to analog computing that start to really hurt at the small nodes, which you need to scale up. There's a very good reason they are stuck at 40 nm.
Are you talking bjt or cmos now?
Theyre not stuck at 40nm due to technical constraints.
If they were using memristors sure, but theyre not really available yet.
I have an analog circuits and a bit of an analog computing background and I will tell you this: analog computers have been around for awhile in commercial applications, still to this day theyre used to great success in some niche applications. They're limited by their flexibility as they are not really reprogrammable. Analog circuits need to be designed for each specific application. There are papers and prototypes built to create "reconfigurable" analog versions of FPGAs but they are limited by physical scalability issues, noise, routing, etc.
I agree with you completely on this, and I also have a bit of an analog circuits background. These chips are unfortunately probably better thought of as purpose-built accelerators for specific neural networks trained almost a decade ago than as the next paradigm of computing.
What got me into analog computing was eliminating the step (data conversion) between sensing the signal and doing something with it. But when you add in the inflexibility of analog circuits, the mostly analog solution is almost always the bad one.
It's basically Ohm's law V = RI, or in this case I = CV (conductance times voltage), on a lattice of wires. On many analog devices setting the values (write) can be very time consuming, although the read operation can indeed be fast and super low energy. Unclear if this is the case here.
I don't suppose these will be retail purchasable any time soon? Would be a useful thing to plug into the Pi 5's new PCIe port, not unlike a NCS2. Although with probably very spotty software support...
What I'd care about here is watts per TOPS. This could be the most important factor for running inference in embedded environments, like directly on cameras, sensors, small drones, implantable medical devices, etc.
They say that their chip only consumes about 10% of power compared to a digital GPU of similar computational capabilities.
I would love to see them succeed. It’s going to be an uphill battle at this point.
“Mythic…ran out of cash last year and was nearly forced to halt operations…it managed to bring in a relatively modest $13 million investment several months later in March” (2023)
Investors hate hardware because they've been crowded out by the fabs.
Fabs no longer think of design creators as their customers. They think of design creators as their portfolio companies. The fab invests in them: not cash, but rather production capacity. Unless you are Apple/Nvidia/Broadcom/Qualcomm-sized, the fabs do not consider you to be a customer. They will pressure you, hard, to take on a "design partner" like Broadcom, and to become that company's customer. The fabs are not particularly interested in having you pay them. At best they might be interested in having you pay the bigger fish (GUC, Synopsys, etc) who are the fab's true customers.
Having the fabs behave like equity investors has, unfortunately, crowded out all of the real equity investors.
I wonder how the QC these chips. Digital chips can run vectors and show that every bit is exactly a 1 or 0 as expected. With an analog chip, each weight, each multiplication, each summation can produce a range of outputs depending on temperature, voltage, and process. I'm sure they have done everything they can to minimize these issues, but they will always be non-trivial.
Of course, the organic neural networks in our brains have all these issues and more and still do fine. On the other hand, our brains have billions of years of evolution that results in them being fault tolerant in ways that this chip can replicate at all.
The nice thing about this particular application (deep neural networks) is they're usually fairly insensitive to weight noise because they have a lot of redundancy, just like our brains.
Your question still stands, however. And for an analog chip it's basically the question of how to qualify, validate, and characterize neural networks in general. I'm not convinced we have the faintest idea how to do that in any formal sense.
28 comments
[ 3.5 ms ] story [ 59.6 ms ] thread- https://mythic.ai/wp-content/uploads/2022/02/MythicWhitepape...
> When a charge is programmed into a flash memory device, its electric field has an effect on any signal passing through it. In the Mythic architecture, the flash transistor acts as a variable resistor that reduces the signal level passing to the output. That reduction is proportional to the analog value stored in the memory. This simple effect implements the multiplication stage found in DNN calculations. The accumulation process, in which the output from each of those calculations is summed, is handled by aggregating the output of an entire column of memory cells. Thanks to these two properties, the Mythic architecture can process an entire input vector in a single step rather than iterating at high speed as in a digital processor
> Analog computing provides the ultimate compute-in-memory processing element. The term compute-in-memory is used very broadly and can mean many things. Our analog compute takes compute-in-memory to an extreme, where we compute directly inside the memory array itself. This is possible by using the memory elements as tunable resistors, supplying the inputs as voltages, and collecting the outputs as currents. We use analog computing for our core neural network matrix operations, where we are multiplying an input vector by a weight matrix.
> Analog computing provides several key advantages. First, it is amazingly efficient; it eliminates memory movement for the neural network weights since they are used in place as resistors. Second, it is high performance; there are hundreds of thousands of multiply-accumulate operations occurring in parallel when we perform one of these vector operations. Given these two properties, analog computing is the core of our high-performance yet highly-efficient system.
https://mythic.ai/technology/analog-computing/
Training might be harder to implement.
This chip is said to perform 25 TOPS. A A100 80GB SXM is said to perform 1248 TOPS. Which is an entire card not a single CPU.
You could theoretically achieve the same with 50 M1076 processors.
Would the benefit be any of:
* a lot cheaper * more energy efficient. * small size * easier to mass produce?
They say that their chip only consumes about 10% of power compared to a digital GPU of similar computational capabilities.
“Mythic…ran out of cash last year and was nearly forced to halt operations…it managed to bring in a relatively modest $13 million investment several months later in March” (2023)
https://www.reuters./technology/nvidias-dominance-ai-chips-d...
Hopefully they have some kind of revenue coming in because it really looks like they're doing some interesting work.
Fabs no longer think of design creators as their customers. They think of design creators as their portfolio companies. The fab invests in them: not cash, but rather production capacity. Unless you are Apple/Nvidia/Broadcom/Qualcomm-sized, the fabs do not consider you to be a customer. They will pressure you, hard, to take on a "design partner" like Broadcom, and to become that company's customer. The fabs are not particularly interested in having you pay them. At best they might be interested in having you pay the bigger fish (GUC, Synopsys, etc) who are the fab's true customers.
Having the fabs behave like equity investors has, unfortunately, crowded out all of the real equity investors.
Of course, the organic neural networks in our brains have all these issues and more and still do fine. On the other hand, our brains have billions of years of evolution that results in them being fault tolerant in ways that this chip can replicate at all.
Your question still stands, however. And for an analog chip it's basically the question of how to qualify, validate, and characterize neural networks in general. I'm not convinced we have the faintest idea how to do that in any formal sense.