That wouldn't work. Modern chips need to be built on epitaxial surfaces, and the only plane they can reliably produce an epitaxial surface on is the horizontal one.
Still wondering how they’ve handled interconnecting beyond each reticle limit/exposure. Are they staggering the exposures for each layer? And how do they handle packaging/IO/power delivery?
I guess these are the secret sauces and we are unlikely to get in depth information.
In the PDF[0] that aveni linked above, they mention they "add wires across scribe line in partnership with TSMC, Extend 2D mesh across die". So the wafer isn't exactly one big chip.
How do you connect such an enormous chip to a circuit board? What packaging and wire bonding technologies are available? I don't think it uses a conventional package like a (Flip-Chip) BGA - the process of soldering a 65535-pin BGA, and instantly losing millions of dollars if there are defective joints is simply unimaginable. The huge chip also have a heat dissipation problem that conventional packaging is unable to solve.
So everything is custom-developed, including wire bonding, packaging, connectors, heat spreaders, an impressive achievement. Unfortunately it's only a three-page overview, I'd love to read more on technical details.
For comparison's sake, 15kw is enough power to heat a house in a moderate winter climate. Getting a furnance worth of power into and out of a chip without melting it is, indeed, very impressive.
15 kW is 7x more than anything that can be (legally) powered from a standard European 230 V, 10 A outlet. With a 2.3 kW wall outlet, you can already power a Bitcoin miner unit or a small IBM mainframe computer. It's indeed impressive.
Which brings us to the next topic: What does the power supply look like, and how does power delivery work? Given the huge amount of power, the PSU for the chip alone is an interesting piece of electronics. Assuming its power supply is similar to a PC or server... First, the mains power is converted to a system DC voltage such as 12 V. For a typical modern cheap and boring switched-mode power supply, the standard efficiency requirement is 80%. But in this case, it means the PSU itself will waste 3 kW of energy, this is unacceptable both from an environmental and thermal perspective. A really good supply can achieve an efficiency of 95%-98%, and this is when things get interesting, at least you'll see some expensive controller chips and high-power, high-frequency transistors in such a PSU, terms like SiC, GaN or IGBT come to mind, switching 1,250 amps on and off. And it doesn't end here, the next step is converting the 12 V system voltage to the chip's core voltage, say 1.0 V, by a Voltage Regulator Module near or on the motherboard. Now, don't even mention the technical challenge of designing a VRM. I can't even imagine how they run the huge current on the motherboard, it's 15,000 amps!
Unfortunately, there's no information about power supply and power delivery.
There are probably several power domains in the chip and therefore several separate power supplies, which avoids the need to manage 15kW in a single spot.
Edit: In the slides shared above [1] (page 16), it says "12x 4kW hot-swappable universal PSUs".
Not sure why this was downvoted - a relatively small natural gas forced air heater as is common nearly everywhere I've lived in the US is about 50,000 BTU or very nearly 15kW. Its a lot of heat.
Definitely a challenge, heat dissipation is "easier" if you can glue it to a plumbed 1/2" thick copper heat exchanger.
I had an opportunity see a laser sintered copper heat exchanger that was very very efficient. Pumping in water and pulling out steam efficient. It was pretty cool.
Now I know its just a LinusTechTips level video, but I guess they havent heard of "Wafer Scale Engine", and I hadn't either, but now this proves the video is already obsolete…
Not obsolete yet, the technology is not mature enough, and the previous arguments against WSI are still valid.
People have been trying Wafer-Scale Integration [0] since the 1970s, there was quite some hype of building a "super chip" back at that time [1], but all efforts failed miserably. Cerebras' success is only the beginning, even if this approach is workable (which remains a question), there's still at least a decade to go from a HPC-specific chip to a general-purpose chip. Another possibility is that WSI will forever be a technology used in massively-parallel computers.
I think it is important to separate the several technical failures in wafer scale from the commercial one: Sinclair's Anamartic worked exactly as planned but the prices for conventional hard disks began to drop and that caused investors to pull out.
The problem is that the real project was a massively parallel computer just like the Cerebras (scaled to 1989 technological limits) and the disk replacement was just a way to develop the needed techniques and finance further developments. If the investors had had a little more patience then computing in the 1990s might have been a bit more interesting.
Not at all obsolete, defect rates rise exponentially as you increase die sizes, so wafer scale chips are stupid and wrong, but in financial sense only.
If you’re paying up for an entire batch and you only need one working example out of it, it probably matter less.
With wafer scale the amount of connections that can cross a border between two tiles is limited by the spacing rules for the top metal layers and by the number of layers you can dedicate to this (1 or 2, I would guess).
For TSMC N7 Design Rules[0] the pitch is 720nm for the top two metals (1388 wires per mm) and 76nm for the middle metal layers (13157 wires per mm).
In the case of chiplets on an interposer, if we suppose that the interposer can use a process similar to the top layers of the wafer then the number of wires between chiplets is the same. But you need pads and solder balls from the chipets to the interposer (and back on the other side). These pads can be in a grid pattern (like in a BGA package) so the area is the limit instead of perimeter. Let's be optimistic and suppose a pitch of 30µm for the copper pillar micro-bumps.
With a conservative 1000 wires per mm of perimeter the pads are not the limit with a chiplet larger than 3.6mm per side. With a more aggressive 10000 wires per mm the chiplets would have to be larger than 36mm per side, which is just beyond the reticle size limit of many chip fabs.
You cannot make DRAM on the same process with competitive, high-end logic, and they want to embed a lot of buffers on chip to reduce communication power use, so SRAM is the only game in town.
Crystal Well was only eDRAM in the marketing sense.
There have been other, actual eDRAM technologies that can be manufactured on some logic processes, however none of them are compatible with the state of the art TSMC 7nm logic process.
Yeah, but it's not enough. The whole chip doesn't have enough SRAM to hold a tenth of GPT-3, let alone GPT-4. These are the kinds of models you would hope a chip like this would help with, but as soon as the model gets too big to fit in the SRAM then the advantage goes way down. And I think the models of the future will continue to get much, much bigger.
There are a few companies doing analog compute-in-memory for ML. Usually on flash processes.
Maybe cerebras could fit well for that.
As for the models of the future getting much bigger - is the GPT family representative ? Are we seeing the same growth is in other domains ? And isn't there a big niche for smaller than GPT models ?
by transistor count, by energy consumption and i'd expect by the price it is an equivalent of 3 x 16 V100 DGX/Lambdasystems. Can it beat such a minicluster? Vertical vs. horizontal scaling seems to be still an open question for DL. It looks like the horizontal - ie. V100 approach in this case - would naturally have advantage over the vertical when it comes to sheer scale, i.e. like the maximum possible size of model.
Did anyone put their first gen to use? Saw that huge chip last year (or maybe the year before at this point... time doesn't exist anymore...), but I never heard of actual users or buyers.
I'm sure companies that do a lot with AI - the FAANGs, universities - would have their hands on these to see if they're viable for their businesses, either for their own use or to rent out to customers through their cloud offerings. But they don't really advertise with it, probably corporate secrecy.
I think it has the same customer base as the quantum processors of today.
Watched the talk at Hot Chips today[0]- 400,000(!) cores on a single wafer is absolutely remarkable! There was a slide not posted in which there were virtual segments of the wafer designated for each step, and the example given by the presenter showed a perfectly in-order workflow execution.
Nah, were near the scaling limit of silicon. And huge chips are super expensive and impractical.
One day we'll find an easier way to print chips. Won't end moore's law but may shift algorithm implementation back to hardware after decades of software eating everything.
Medium term, they'll get a new lease on moore's law by switching off silicon to smaller atoms. And/or switching to something that handles heat better so we can go 3D. This is already happening in power electronics with GaN. My bet is on this tech slowly spreading into digital logic
Might it not be more optimal to attach the individual GPU dielets onto a silicon interposer to avoid the yield issues that are inherent when making a large monolithic chip, and save some of the overheads of dealing with dead dielets? I think I read about some researchers/companies going that route but haven't kept up.
55 comments
[ 3.6 ms ] story [ 109 ms ] threadTop500 Supercomputer in a single Rack. Just imagine.
And if we take liberty to dream extreme, "Available on Amazon, On Demand". :D
And someone will be spending a lot of time trying to figure out how to support a wafer cut lengthwise without shattering it during processing.
I guess these are the secret sauces and we are unlikely to get in depth information.
[0] https://www.hotchips.org/hc31/HC31_1.13_Cerebras.SeanLie.v02...
And SC19: https://secureservercdn.net/198.12.145.239/a7b.fcb.myftpuplo...
So everything is custom-developed, including wire bonding, packaging, connectors, heat spreaders, an impressive achievement. Unfortunately it's only a three-page overview, I'd love to read more on technical details.
Which brings us to the next topic: What does the power supply look like, and how does power delivery work? Given the huge amount of power, the PSU for the chip alone is an interesting piece of electronics. Assuming its power supply is similar to a PC or server... First, the mains power is converted to a system DC voltage such as 12 V. For a typical modern cheap and boring switched-mode power supply, the standard efficiency requirement is 80%. But in this case, it means the PSU itself will waste 3 kW of energy, this is unacceptable both from an environmental and thermal perspective. A really good supply can achieve an efficiency of 95%-98%, and this is when things get interesting, at least you'll see some expensive controller chips and high-power, high-frequency transistors in such a PSU, terms like SiC, GaN or IGBT come to mind, switching 1,250 amps on and off. And it doesn't end here, the next step is converting the 12 V system voltage to the chip's core voltage, say 1.0 V, by a Voltage Regulator Module near or on the motherboard. Now, don't even mention the technical challenge of designing a VRM. I can't even imagine how they run the huge current on the motherboard, it's 15,000 amps!
Unfortunately, there's no information about power supply and power delivery.
Edit: In the slides shared above [1] (page 16), it says "12x 4kW hot-swappable universal PSUs".
[1] : https://secureservercdn.net/198.12.145.239/a7b.fcb.myftpuplo...
I had an opportunity see a laser sintered copper heat exchanger that was very very efficient. Pumping in water and pulling out steam efficient. It was pretty cool.
Maybe they're meant to be used with a DL compiler like TVM [0]?
[0] https://tvm.apache.org/
More of the software stack was described at HotChips today, covered by AnandTech: https://www.anandtech.com/show/16006/hot-chips-2020-live-blo...
Now I know its just a LinusTechTips level video, but I guess they havent heard of "Wafer Scale Engine", and I hadn't either, but now this proves the video is already obsolete…
People have been trying Wafer-Scale Integration [0] since the 1970s, there was quite some hype of building a "super chip" back at that time [1], but all efforts failed miserably. Cerebras' success is only the beginning, even if this approach is workable (which remains a question), there's still at least a decade to go from a HPC-specific chip to a general-purpose chip. Another possibility is that WSI will forever be a technology used in massively-parallel computers.
[0] https://en.wikipedia.org/wiki/Wafer-scale_integration
[1] Giant microcircuits for superfast computers, Popular Science, 1984. https://books.google.com/books?id=eAAAAAAAMBAJ&pg=PA66
> even if this approach is workable (which remains a question)
No need to repeat.
http://www.computinghistory.org.uk/det/3043/Anamartic-Wafer-...
The problem is that the real project was a massively parallel computer just like the Cerebras (scaled to 1989 technological limits) and the disk replacement was just a way to develop the needed techniques and finance further developments. If the investors had had a little more patience then computing in the 1990s might have been a bit more interesting.
If you’re paying up for an entire batch and you only need one working example out of it, it probably matter less.
For TSMC N7 Design Rules[0] the pitch is 720nm for the top two metals (1388 wires per mm) and 76nm for the middle metal layers (13157 wires per mm).
In the case of chiplets on an interposer, if we suppose that the interposer can use a process similar to the top layers of the wafer then the number of wires between chiplets is the same. But you need pads and solder balls from the chipets to the interposer (and back on the other side). These pads can be in a grid pattern (like in a BGA package) so the area is the limit instead of perimeter. Let's be optimistic and suppose a pitch of 30µm for the copper pillar micro-bumps.
With a conservative 1000 wires per mm of perimeter the pads are not the limit with a chiplet larger than 3.6mm per side. With a more aggressive 10000 wires per mm the chiplets would have to be larger than 36mm per side, which is just beyond the reticle size limit of many chip fabs.
[0] https://en.wikichip.org/wiki/7_nm_lithography_process
https://www.hotchips.org/hc31/HC31_1.13_Cerebras.SeanLie.v02...
> Redundancy is Your Friend
> Uniform small core architecture enables redundancy to address yield at very low cost
> Design includes redundant cores and redundant fabric links
> Redundant cores replace defective cores
> Extra links reconnect fabric to restore logical 2D mesh
I wouldnt have thought this amount of sram to be practical from a size and cost standpoint compared to DRAM.
There have been other, actual eDRAM technologies that can be manufactured on some logic processes, however none of them are compatible with the state of the art TSMC 7nm logic process.
Maybe cerebras could fit well for that.
As for the models of the future getting much bigger - is the GPT family representative ? Are we seeing the same growth is in other domains ? And isn't there a big niche for smaller than GPT models ?
I think it has the same customer base as the quantum processors of today.
[0] https://www.anandtech.com/show/14758/hot-chips-31-live-blogs...
One day we'll find an easier way to print chips. Won't end moore's law but may shift algorithm implementation back to hardware after decades of software eating everything.
Medium term, they'll get a new lease on moore's law by switching off silicon to smaller atoms. And/or switching to something that handles heat better so we can go 3D. This is already happening in power electronics with GaN. My bet is on this tech slowly spreading into digital logic
I don't actually know, but that seems to be the route the industry is going and economics of scale will likely make it the winning approach.