Insane. I wonder how you would even design a memory subsystem for this. Back of the envelope calculation says you'd need at least a 212500 bit wide DDR5-7200 bus to even approach that throughput.
These things don't go to a single system. The signal is optically split at the
ends and there are big racks of transceivers, each transceiver deals with a single mode. With 55 modes you are looking at dealing with 27 terabits or 125 terabytes per second on any given system.
Oh wow. No guarantees, but they mentioned they achieved this on a single wavelength, and that wavelength multiplexing should be possible. Current Dense Wavelength Division Multiplexing (DWDM) supports something like 160 channels. If they are able to combine both of these technologies - theoretically you are looking at nearly 250 Petabits per second on a single fiber strand.
Thing is, no one is pulling a single fiber cable. Modern fiber cables can contain up to a thousand fiber strands in a single cable although the highest strand-count single-mode fiber cable commonly manufactured is the 864-count, consisting of 36 ribbons each containing 24 strands of fiber. Even oceanic cables frequently have 48 or 96 fibers.
That means, again theoretically, a land based 864-core cable can carry over 200 exabits per second, and an oceanic 96-core can carry 23.5 exabits per second. That's bananas.
Edit:
Very large numbers are hard for humans to grok. Some context on 200 exabits - There are 335,591,597 people in the United states. If you were to divide the bandwidth evenly between all those people, each person would have approximately a 600 gigabit connection. An UHD Blu-ray (4K, HDR) movie is about 50GB. On this connection you could transfer that in under 1 second (hand waving away overhead, write limitations and connection/handshake delays).
> That means, again theoretically, a land based 864-core cable can carry over 200 exabits per second, and an oceanic 96-core can carry 23.5 exabits per second. That's bananas.
The total amount of data generated per day is estimated to be 97 zetabytes [0] which is only 9 exabits per second. You could run the entire internet off a single cable that winds its way around to every single person and they could just siphon off the data they need.
At 200 exabits per second, or 25 Exabytes per second, it would take you a thousand light seconds to send the entire contents of the internet. With the speed of light in the cable about 2/3rds of C, that would require about 600 light seconds of cable, which is about the distance from the sun to the earth.
Quick googling tells me the typical diameter of an undersea cable is 25 mm. If we assume let's say 15 mm of that is actual fiber (wild guess) and approximate them as a single cylinder that works out to only about 0.03 km^3 of glass... that's a lot more feasible than I was expecting. Comparable to the volume of concrete used to build the Three Gorges Dam. I suppose it's only that "reasonable" because we're talking about a hypothetical 200 exabits/s, puts into perspective just how fast that is.
Subsea cables usually have 4 or so fibers, since they need repeaters every so many km, which drives the cost-per-fiber (the actual strand of glass is approximately free for all practical purposes).
This may have been true in the past but new subsea cables are not this limited. As an example the Dunant transatlantic long haul Google put in place in 2018 (went live in 2020) has 12 pairs (24 fibers). Grace Hopper, due online this year has 32 fibers. But your point stands - I should have said "up to" instead of "frequently".
The cable is named for computer science pioneer Grace Brewster Murray Hopper (1906–1992), best known for her work on one of the first linkers (compilers), which was critical in the development of the COBOL programming language. She’s also credited with famously finding an actual “bug” in a program; her team tracked down the source of a short circuit on the early Harvard Mark II computer to a moth trapped in a panel. It is to honor Grace Hopper’s legacy of innovation by investing in the future of transatlantic communications with a state-of-the-art fiber optic cable.
TIL, thanks. I was at Google around the time Curie was being built and coming online (ca. 2018-2019, I think), and it has 4 pairs. I misremembered it being 4 fibers.
I believe scaling is accomplished in this case by having each node handle encryption. The fiber transport itself need not encrypt the payload. There are cases where businesses encrypt their leased dark fiber for compliance reasons so their limit would be whatever their VPN routers/appliances can handle at the time.
Can you explain what multi-mode fibres are? From what I've read, it allows for WDM over a single fibre. But what is physically different about these multimode fibres that allow for this Space-division multiplexing? Wikipedia says:
>Multi-mode fiber has a fairly large core diameter that enables multiple light modes to be propagated and limits the maximum length of a transmission link because of modal dispersion.[0]
But that's clearly wrong because here we are using the standard diameter.
In a similar article we discussed a ring structure to store data. You could store about an L1 cache’s worth of data in a loop that’s roughly a nanosecond long.
Problem we didn’t discuss is this: L1 cache is less than 1/3ns on high end hardware, meaning that L1 caches currently have about 1/3 the latency of this light loop.
Once this doubles or triples again you might have a solution for L2 cache though, since chip clock cycles are relatively stuck.
These scientists create these impressive hardware systems, important to technologies that we use. Yet, there aren’t all that many jobs in US in hardware and manufacturing sectors, and they don’t pay well either (far less than software engineers with a lot less investment in expertise). The system is not fair.
Is the mode structure of an installed cable stable enough for this? or are the 55 subchannels going to get scrambled up every time the wind blows the cable?
31 comments
[ 4.5 ms ] story [ 57.4 ms ] threadThing is, no one is pulling a single fiber cable. Modern fiber cables can contain up to a thousand fiber strands in a single cable although the highest strand-count single-mode fiber cable commonly manufactured is the 864-count, consisting of 36 ribbons each containing 24 strands of fiber. Even oceanic cables frequently have 48 or 96 fibers.
That means, again theoretically, a land based 864-core cable can carry over 200 exabits per second, and an oceanic 96-core can carry 23.5 exabits per second. That's bananas.
Edit:
Very large numbers are hard for humans to grok. Some context on 200 exabits - There are 335,591,597 people in the United states. If you were to divide the bandwidth evenly between all those people, each person would have approximately a 600 gigabit connection. An UHD Blu-ray (4K, HDR) movie is about 50GB. On this connection you could transfer that in under 1 second (hand waving away overhead, write limitations and connection/handshake delays).
The total amount of data generated per day is estimated to be 97 zetabytes [0] which is only 9 exabits per second. You could run the entire internet off a single cable that winds its way around to every single person and they could just siphon off the data they need.
[0] https://www.statista.com/statistics/871513/worldwide-data-cr...
At 200 exabits per second, or 25 Exabytes per second, it would take you a thousand light seconds to send the entire contents of the internet. With the speed of light in the cable about 2/3rds of C, that would require about 600 light seconds of cable, which is about the distance from the sun to the earth.
https://www.wolframalpha.com/input?i=pi+*+%28%2815%2F2%29+mi...
It's like having a highway named after you, only cooler, because it's data traffic, not automobile traffic.
I mean, fiber is cheap and you can put many strands into a not particularly thick cable, and there's DWDM as another comment says.
Is actually running into the limits of the fiber a frequent occurrence anywhere in the industry, like in the submarine cables?
>Multi-mode fiber has a fairly large core diameter that enables multiple light modes to be propagated and limits the maximum length of a transmission link because of modal dispersion.[0]
But that's clearly wrong because here we are using the standard diameter.
[0]https://en.wikipedia.org/wiki/Multi-mode_optical_fiber
Problem we didn’t discuss is this: L1 cache is less than 1/3ns on high end hardware, meaning that L1 caches currently have about 1/3 the latency of this light loop.
Once this doubles or triples again you might have a solution for L2 cache though, since chip clock cycles are relatively stuck.