> Efficient’s goal is to approach the problem by static scheduling and control of the data flow - don’t buffer, but run. No caches, no out-of-order design, but it’s also not a VLIW or DSP design. It’s a general purpose processor.
No, but multiple GPU shader architectures use VLIW instructions. It's totally doable with modern compilers, but it's only adventageous to parallelizable tasks, hence the use in GPUs.
This is a CGRA. It's like an FPGA but with bigger cells. It's not a VLIW core.
I assume that like all past attempts at this, it's about 20x more efficient when code fits in the one array (FPGAs get this ratio), but if your code size grows past something very trivial, the grid config needs to switch and that costs tons of time and power.
Yeah, I have worked with FPGAs a while ago and still casually follow the space.
There have been many attempts of mapping general purpose/GPU programming languages to FPGA and none of them worked out.
The first leading claim they make - that this is a general purpose CPU, capable of executing anything - I suspect is false.
CPUs are hard because they have to interact with memory, basically 95% of CPU design complexity comes from having to interact with memory, and handling other data hazards.
If this was reducible complexity, they'd have done so already.
I recall Mathstar's FPOA (field programmable object arrays) have had similar architecture. it seems they have done a mixture of stack computer, this and some async programming to get this level of optimization. The other one I had seen with pretty good on chip fabric was Tilera who were using something like a packet switch to interconnect tonnes of on-chip cores.
My first reaction watching this video was that they are just shifting the problem to compiler, which is actually worse and also does not works for dynamic code with tonnes of branches. Also, didnt Intel burn a lot of money trying to do this with Itanium?
Overall, interesting idea but I filed it under 'solution looking for a problem' desk.
As a person who is highly vested and interested in CPU space, especially embedded, I am HIGHLY skeptical of such claims. Somebody played TIS-100, remembered GA144 failed and decided to try their own. You know what can be a simple proof of your claims? No, not a press release. No, not a pitch deck or a youtube video. And NO, not even working silicon, you silly. A SIMPLE FUCKING ISA EMULATOR WITH A PROFILER. Instead we got bunch of whitepapers. Yeah, I call it 90% chance for total BS and vaporware
It's a dataflow architecture.
I assume the hardware implementation is very similar to what is described here:
https://csg.csail.mit.edu/pubs/memos/Memo-229/Memo-229.pdf.
The problem is that it becomes difficult to exploit data locality, and there is so much optimization you can perform during compile time.
Also, the motivation for these types of architectures (e.g. lack of ILP in Von-Neumann style architectures) are non-existent in modern OoO cores.
Pardon me but could somebody here explain to me like I am 15? Because I guess Its late night and I can't go into another rabbithole and I guess I would appreciate it. Cheers and good night fellow HN users.
Instead of large cores operating mostly independently in parallel (with some few standardized hardwired pipeline steps per core), …
You have many more very small ALU cores, configurable into longer custom pipelines with each step more or less as wide/parallel or narrow as it needs to be for each step.
Instead of streaming instructions over & over to large cores, you use them to set up those custom pipeline circuits, each running until it’s used up its data.
And you also have some opportunity for multiple such pipelines operating in parallel depending on how many operations (tiles) each pipeline needs.
I wondered if this was using interaction combinators like the vine programming language does.
I haven't read much that explains how they do it.
I have been very slowly trying to build a translation layer between starlark and vine as a proof of concept of massively parallel computing. If someone better qualified finds a better solution the market it sure to have demand for you. A translation layer is bound to be cheaper than teaching devs to write in jax or triton or whatever comes next.
Pretty interesting concept, though as other commenters have pointed out the efficiency gains likely break down once your program doesn’t fit onto the mesh all at once. Also this looks like it requires a “sufficiently smart compiler”, which isn’t a good sign either. The need to do routing etc. reminds me of the problems FPGAs have during place and route (effectively the minimum cut problem on a graph, i.e. NP), hopefully compilation doesn’t take as long as FPGA synthesis takes.
This is essentially a CGRA (Coarse-Grained Reconfigurable Array) architecture, which historically has shown impressive efficiency in academic research but struggled with compilation complexity and commercial adoption precisely because of the NP-hard routing problems you've identified.
> spatial data flow model. Instead of instructions flowing through a centralized pipeline, the E1 pins instructions to specific compute nodes called tiles and then lets the data flow between them. A node, such as a multiply, processes its operands when all the operand registers for that tile are filled. The result then travels to the next tile where it is needed. There's no program counter, no global scheduler. This native data-flow execution model supposedly cuts a huge amount of the energy overhead typical CPUs waste just moving data.
Though I'm sure this is valuable in certain instances, thinking about many embedded designs today, is the CPU/micro really the energy hog in these systems?
We're building an EEG headband with bone-conduction speaker so in order of power, our speaker/sounder and LEDs are orders of magnitude more expensive than our microcontroller.
In anything with a screen, that screen is going to suck all the juice, then your radios, etc. etc.
I'm sure there are very specific use-cases that a more energy efficient CPU will make a difference, but I struggle to think of anything that has a human interface where the CPU is the bottleneck, though I could be completely wrong.
I would not expect that this becomes competitive against a low power controller that is sleeping most of the time, like in a typical wristwatch wearable.
However, the examples indicate that if you have a loop that is executed over and over, the setup cost for configuring the fabric could be worth doing. Like a continuous audio stream in a wakeup-word detection, a hearing aid, or continous signals from an EEG.
Instead of running a general purpose cpu at 1MHz the fabric would be used to unroll the loop, you will use (up to) 100 building blocks for all individual operations. Instead of one instruction after another, you have a pipeline that can execute one operation in each cycle in each building block. The compute thus only needs to run at 1/100 clock, e. g. the 10kHz sampling rate of the incoming data. Each tick of the clock moves data through the pipeline, one step at a time.
I have no insights but can imagine how marketing thinks: "let's build a 10x10 grid of building blocks, if they are all used, the clock can be 1/100... Boom - claim up to 100x more efficient!"
I hope their savings estimate is more elaborate though...
particularly solar/outdoor/remote applications. I run ESP32s from super-cheap USBC solar panels with integrated battery/BMS for various measurements, keeps them running 24/7 regardless of weather due to how little they consume. They pop on every 4 minutes, power the sensors, connect to the WiFi network, beep out their data, then go back to sleep. Seeed's ~$6 ESP32C3/C6 boards have onboard BMS for 3.7V cell even, if I didn't prefer the integrated panel/battery method.
> The interconnect between tiles is also statically routed and bufferless, decided at compile time. As there's no flow control or retry logic, if two data paths would normally collide, the compiler has to resolve it at compile time.
This sounds like the most troublesome part of the design to me. It's very hard to do this static scheduling well. You can end having to hold up everything waiting for some tiny thing to complete so you can proceed forward in lock step. You'll also have situations where 95% of the time the static scheduling can work but 5% of cases where something fiddly happens. Without any ability for dynamic behaviour and data movement small corner cases dominate how the rest of the system behaves.
Interestingly you see this very problem in hardware design! All paths between logic gates need to be some maximum length to reach a target clock frequency. Often you get long fiddly paths relating to corner cases in behaviour that require significant manual effort to resolve and achieve timing closure.
From a position of naive curiosity -- Would this work as a coprocessor, take the most inefficient/most optimisable procedures and compile (#) them for the fabric? It would you lose all your gains in turn being extra processes to ship data between cores/processors?
How 2D is it: compiling to a fabric sounds like it needs lots of difficult routing. 3D would seem like it would make the routing much more compact?
Sidenote/curio: Arm as a main processor was pioneered in the Acorn Archimedes. Its (non-BBC) predecessor in the Acorn product range was the ... Electron.
35 comments
[ 6.2 ms ] story [ 61.4 ms ] threadhttps://www.patentlyapple.com/2021/04/apple-reveals-a-multi-...
Typically these architectures are great for compute. How will it do on scalar tasks with a lot of branching? I doubt well.
Sounds like a mainframe. Is there any similarity?
Is this effectively having a bunch of tiny processors on a single chip each with its own storage and compute?
I assume that like all past attempts at this, it's about 20x more efficient when code fits in the one array (FPGAs get this ratio), but if your code size grows past something very trivial, the grid config needs to switch and that costs tons of time and power.
There have been many attempts of mapping general purpose/GPU programming languages to FPGA and none of them worked out.
The first leading claim they make - that this is a general purpose CPU, capable of executing anything - I suspect is false.
CPUs are hard because they have to interact with memory, basically 95% of CPU design complexity comes from having to interact with memory, and handling other data hazards.
If this was reducible complexity, they'd have done so already.
My first reaction watching this video was that they are just shifting the problem to compiler, which is actually worse and also does not works for dynamic code with tonnes of branches. Also, didnt Intel burn a lot of money trying to do this with Itanium?
Overall, interesting idea but I filed it under 'solution looking for a problem' desk.
You have many more very small ALU cores, configurable into longer custom pipelines with each step more or less as wide/parallel or narrow as it needs to be for each step.
Instead of streaming instructions over & over to large cores, you use them to set up those custom pipeline circuits, each running until it’s used up its data.
And you also have some opportunity for multiple such pipelines operating in parallel depending on how many operations (tiles) each pipeline needs.
I haven't read much that explains how they do it.
I have been very slowly trying to build a translation layer between starlark and vine as a proof of concept of massively parallel computing. If someone better qualified finds a better solution the market it sure to have demand for you. A translation layer is bound to be cheaper than teaching devs to write in jax or triton or whatever comes next.
should work great for NN.
We're building an EEG headband with bone-conduction speaker so in order of power, our speaker/sounder and LEDs are orders of magnitude more expensive than our microcontroller.
In anything with a screen, that screen is going to suck all the juice, then your radios, etc. etc.
I'm sure there are very specific use-cases that a more energy efficient CPU will make a difference, but I struggle to think of anything that has a human interface where the CPU is the bottleneck, though I could be completely wrong.
However, the examples indicate that if you have a loop that is executed over and over, the setup cost for configuring the fabric could be worth doing. Like a continuous audio stream in a wakeup-word detection, a hearing aid, or continous signals from an EEG.
Instead of running a general purpose cpu at 1MHz the fabric would be used to unroll the loop, you will use (up to) 100 building blocks for all individual operations. Instead of one instruction after another, you have a pipeline that can execute one operation in each cycle in each building block. The compute thus only needs to run at 1/100 clock, e. g. the 10kHz sampling rate of the incoming data. Each tick of the clock moves data through the pipeline, one step at a time.
I have no insights but can imagine how marketing thinks: "let's build a 10x10 grid of building blocks, if they are all used, the clock can be 1/100... Boom - claim up to 100x more efficient!" I hope their savings estimate is more elaborate though...
Increasing the processing power at that near-marginal power consumption broadens the range of battery-powered applications that are possible.
Wonder why they do not focus their marketing on this.
This sounds like the most troublesome part of the design to me. It's very hard to do this static scheduling well. You can end having to hold up everything waiting for some tiny thing to complete so you can proceed forward in lock step. You'll also have situations where 95% of the time the static scheduling can work but 5% of cases where something fiddly happens. Without any ability for dynamic behaviour and data movement small corner cases dominate how the rest of the system behaves.
Interestingly you see this very problem in hardware design! All paths between logic gates need to be some maximum length to reach a target clock frequency. Often you get long fiddly paths relating to corner cases in behaviour that require significant manual effort to resolve and achieve timing closure.
How 2D is it: compiling to a fabric sounds like it needs lots of difficult routing. 3D would seem like it would make the routing much more compact?
https://www.cas-well.com/applications/tilera-the-many-core-s...