Are there any theoretical limits to how much data can be encoded via fiber optics? If not, what's the current bottleneck for putting terabyte fiber in every home? (I'm assuming it's a lack of reliable, cheap and precise hardware clocks, but I've been wrong many times before)
We already hit wavelength transfer limits (though in transmitter accuracy/viability) and can pretty easily circumvent that by transmitting different wave lengths on the same fiber. So you need to multiply this by the amount of distinct wavelengths we can use.
> If not, what's the current bottleneck for putting terabyte fiber in every home? (I'm assuming it's a lack of reliable, cheap and precise hardware clocks,
Even very poor countries these days have ftth at at least 1gbps physical interface data rate (1gbase lx/gepon/gpon). What they don't have is upstream capacity.
America is unique in having tons of upstream fibre, but extremely bad last mile situation.
Theoretical limit is `freqency * log(energy budget / effective[0] noise floor)`, give or take a small multiplicative constant depending on how efficient your error correction is.
0: For low frequencies or (relatively) high noise, this is the actual noise floor, but for high frequencies/very low noise, the limit is that you can only send a integer number of photons, each costing some non-negligible fraction of your power budget (so the effective noise floor is (half, I think?) the power needed to send one photon per cycle).
For the photon quantization regime, I think we have a photon energy scaled as `hbar THz`, so a noise floor as `hbar THz^2` (1/2 photon / cycle is about 23uW at 666THz [450nm], or 19uW at 600THz [500nm]). Your Tbps is then `log(power / noise floor) * frequency` aka `log(power/0.5 hbar frequency^2) * frequency`. The effect of noise floor (and consequently quantization-regime frequency) on data rate isn't really cooperative with dimensional analysis on account of the logarithm.
Also, to optimise data rate at a given power budget, you want to choose freq such that `d rate/d freq` is zero, which (assuming I didn't screw up the math) goes like:
So (again assuming I got that math right), the optimal single photon frequency to use (in the quantum noise floor regime) scales with the square root of your power budget. However, at one milliwatt it's 1600THz or 187nm, and only increases from there, so for practical purposes you probably just want to use the bluest light your fiber can carry, and worry more about classical noise.
This. I have a friend with 1Gbps up/down at his house. I've been waiting for fiber for like 10 years but it's never been available at any place I've lived. This is despite the fact that we live in the same (relatively small) county. This situation could only be possible if they just didn't care about making fiber available to everyone.
Nothing prevents it, except practicality: running enough fibers to support a terabit point to point is already hard enough. Add in needing to support millions of people and it becomes impossible. As is, internet routers and fiber links already use tons of power. Asking for 1000xing that just can't happen without a dyson sphere or something.
I don't know if I'd go that far - we can probably get more than 1000x throughput without absorbing the whole power of the sun - but it would require some meaningful amounts of infrastructure improvements and replacements.
Fair enough. I used a dyson sphere as a shorthand for "a power generation technology giving us orders of magnitude more power than we are generating now"
The fact that the server Joe is pulling from won't feed that file to him at that rate going out its own locally attached ethernet? (Due to either throttling, or serving stuff thousands of others?)
There is a seven lane highway three miles from where Joe lives; why can't he drive to his friend across town in 3 minutes?
> What's preventing the average Joe from enjoying 1+Tb/s broadband?
are other responders not seeing the 'T'?
it is the rare home device which can handle 2.5Gbps. 10Gbps switches are common enough. the cheapest switch i'm seeing on fs.com with a 100Gbps uplink port is $3500.
i don't think we currently have the tech to drive 100Gbps through a single port, i think it's done with multiple wavelengths being multiplexed onto a single fiber, and then split back out.
so... at a minimum, joe is kept from 1 Tbps of bandwidth by $35,000, some optical multiplexing hardware, 10+U of rack space, and having absolutely no conceivable way to put the kind of bandwidth to use.
plus his part of the amortized capital costs to build the ISP's end of that.
You wouldn't even be able to transfer the data out of memory at speed. Even if you overclocked your DDR4 ram to crazy levels - you'd tap out at 50GB/s - a far cry from 1Tbs/s.
There is very little practical use to speeds beyond 10 gigabit for home use unless you have some insane PCIE storage or keep a giant ram drive. (Which again asks the practical use category - what are you doing?)
> There is very little practical use to speeds beyond 10 gigabit for home use
Give it a few years and we'll all be wondering how SPA web apps became gigabytes in size and trying to work out how to make 1Tbs/s to the home practical.
Aside from the actual fiber itself: literally your entire computer. No part of a modern consumer computing device has 1Tb/s bandwidth - RAM, SSDs, networking cards, PCI, the front-side bus, and everything else, with the possible exception of the L1 cache[1], have maximum bandwidths well below 1Tb/s (= 125 GiB/s).
That’s insane. I don’t think my computer can transfer 319 MB/s.
Google, Microsoft, Amazon and Facebook stored about 1.2 million terabytes of data in 2019. With this speed you could download all of this data in about 8 hours.
Geez, and I was surprised to see 400Gbps QDR optics hitting the mainstream. Even if we double the data rate every few of years (as we have been, 10Gbps, 25Gbps, 40Gbps, 100Gbps, 200Gbps, 400Gbps, 800Gbps, I expect it will continue), we are at least 10 full cycles off from realizing those kinds of speeds, or 30-40 years?
Edit: since they are multiplexing, it almost seems like this is DWDM, on 4 optics, and combined into a single datastream. That means is probably more attainable sooner. Seems similar to just adding more cores (or lanes like in current QDR optics).
Agreed - it seems to me they’ve built a medium that signals at the described rate, but didn’t actually send data? I could be misreading. I was looking to find who’s TCP/IP stack got pressed into service, but I guess that’s for a future story.
Edit: this isn’t my area of expertise, but https://news.ycombinator.com/item?id=28674318 and the provided link explain why even “Internet” (let alone tcp/ip) might be quite orthogonal to what this technology really is.
[Quite often I'll think twice before commenting on HN due to this type of out of context, knee-jerk retort. It is really not adding to the conversation or helping anyone learn anything new.]
SONET has optical networking in its name. It appears to me that data not being transmitted over copper wires is a pretty crucial aspect of it... but really none of this matters as I was simply pointing out that TCP is not relevant to the speed record, in answer to the top two comments above, and not trying to make any absolute assertions about how the transport protocol works. So yes, you're right, I guess?
Depending on how the fiber was installed, you’ll probably be able to swap the fiber by just pulling or blowing new fiber down the existing conduit.
You can’t just drop bare fiber in the ground, and cover it up. So pretty much all fiber will probably be inside plastic or metal conduit, which also creates an opportunity to replace it much easier than laying new fiber.
It being the same size means they can just push it through the existing holes to replace cables which is cheaper then digging new holes. But certainly far from cheap or easy
Maybe the backend that can deliver such a rate? Typical RAM speed is 25GB/s, L3 cache around 400GB/s. So having 30TB/s needs 1000 standard RAMs running in parallel.
I was thinking "again"? Turns out this was from July. Last discussions [1], [2]. It is basically just pushing the limit of WDM and MIMO.
Much more interesting would be Hollow-core Optical Fiber across long distance where we could reduce latency by 30%. Unfortunately not coming any time soon.
“Just pushing the limit” is quite the understatement. Someone correct me if I’m wrong, but I believe the total combined bandwidth of internet undersea cables today is ~400Tb/s
Yes and we are already looking at Pbps within this decade. My own view is that we aren't that much bandwidth limited. But we are fundamentally limited by latency.
Japan's internet infrastructure, at least in Tokyo where I lived for a bit, is amazing. Most buildings were wired up with fiber (this is ~2011) with 100 Mbps up/down pretty standard and cheap (~$30/month at the time I think). Pretty sure you had 300+ speeds available then too. The fiber is from the national telecom, with dozens of ISPs for your actual service, and email etc. if you wanted it. My apartment had a switch somewhere too I realized, with each room having an ethernet jack. Needless to say, it was rough going back to the US after that.
Seems like it depends on the city and state. I have symmetric gigabit backed by fiber in a suburb, and most of my friends out in the boonies have the same.
I dream of a world where symmetric 100 is $30/mo everywhere though!
I'm sorry but there's no such thing as a rural part of Bellevue. Regardless of however not-dense your particular area is you're still close to a major population center.
Municipal Fiber is the way to make that happen. It allows ISPs to offer very cheap rates, since they do not have to provide the infrastructure. I live in a building connected to a municipal network, and pay $32 Canadian for symmetric gigabit.
I currently (temporarily) live in the boonies (3 hours north of SF in the woods) and Starlink can't be available soon enough.
I'm renting office space in town where the best I can get is a 40 Mb DSL line because I can't get anything other than HughesNet at the house. More than a few days of SSH over satellite will make you want to regret the day you decided programming would be a fun profession.
Forgetting about the latency for a moment, even though HughesNet advertises nice speeds, it's very much the same "speeds up to" crap most other ISP do. They're so oversubscribed that you can barely browse the internet in the evening most of the time.
Starlink works well (got it for my in-laws place in Placerville). It was a little rocky early on with more frequent dropouts, but now the dropouts are rare and the speeds are fast: 50mbps to 200mbps and this is with the dish being slightly obstructed by a tree.
In SF though I'm in a webpass wired building and have 1gbps up and down for ~$700/yr or ~$58/month.
On the Peninsula Comcast had 1gbps down and 35mbps up, but they also had a (very expensive) Comcast Gigabit Pro plan that was 2gbps down and 2gbps up with a second 1gbps down/up line included too and a fancy switch also. Hard to order though and the reps don't know about it. Cool that it's available though.
I'm sort of in a very rural area. Farm fields across the street from me. But I'm also ~5 minutes or so away from downtown.
I have available to me gigabit (down) from Comcast. I had it for awhile and it was glorious but it was also $80/mo on promo and $110 when not. The reason I got it was not for the download speeds, it was the faster upload speeds that came along with it. Still slow, and not even remotely close to speedy. But it was a solid 12mbit up. Compared to the 5mbit up I get on my 100down plan.
I would kill for some reliable upload speeds. It's awful anymore just sending images to people, never mind you want to send a quick minute long video to someone and it takes several minutes to send.
Any network engineers around? What leads comcast to offer such lopsided connectivity? I mean having some sort of asymmetry is normal and somewhat justified, but usually were talking something like 1:10 upstream:downstream. Here parent comment has fairly extreme 1:83 instead!? I imagine there must be some reasoning behind it, but I just can't figure what that would be.
My understanding is that it’s not a technical issue, but rather a business and marketing one.
Comcast doesn’t want the home service to compete with their costlier Business service, so they artificially limit the upload speeds to prevent home/consumer users from serving a high volume of content using their less-expensive service.
Someone can correct me if I'm wrong, but DOCSIS over coax isn't full duplex. It's multiplexing send and receive over the same wire. Since download bandwidth is more important they provision their network design to maximize the amount of downstream bandwidth available. They also have an extreme level of oversubscription.
The last mile DOCSIS bandwidth is shared among all of its user. So you want to maximise the download which is used 90% of the time. The problem is somewhat solved ( or lessen ) in DOCSIS 4.0 Full Duplex Extension. But as with everything DOCSIS the lead time to market is extremely long.
Parts of the US are catching up, but it is distributed very unequally (and often much more expensive).
Anecdotally, I moved into a newish (~5 year old) apartment building in Seattle and had my choice of three ISPs that each provided symmetrical gigabit for around $60/month, delivered via an Ethernet jack in a low-voltage cabinet in my closet.
Getting it wired up took one phone call, wherein my switch port was activated remotely and I could grab an IP address and start using the internet. IPv6 via DHCPv6-PD and everything else worked without a charm.
On the other hand, where I live in (also Seattle) is served by exactly one ISP via DOCSIS, providing 1000/35 but nothing more. It can vary block by block here unfortunately.
In my neck of the woods, it's all about competition (or that lack there of). I went from 100mbps down for $130 to 300mbps down for $60. What changed? A new ISP in town offering 1 gbps for $100.
If we wanted what other countries have for internet, then what we need is to make the lines public and have ISPs rent them from the government. The reason ISPs can create these duopolies is because every new company has to invest a LOT of money to lay down new data lines. If a company already has their cable/phone lines in place then boom, captured market.
Rentable public data lines would make the cost of setting up new ISPs a LOT cheaper.
I had the same issue ("1000/35" with Xfinity but in reality it was barely 10/1) but keep checking because CenturyLink is regularly bringing fiber to new areas. For the same price I can get 1000/1000 symmetric now.
CenturyLink wired my entire neighborhood a number of years ago, but it looks like _before_ my house was built, and they haven't come back since.
All of my neighboring blocks have 1g/1g from them, but the most they offer at my address is a 40/40 "fiber-based coax" internet that I'm probably not even wired for. I would love for that to change - my DOCSIS internet has been pretty solid, but I miss the gigabit upload.
I remember I recently did speed tests on the free wifi that they offer throughout Tokyo and it was always around 100-150Mbps u/d (I came across this clip when looking to verify[1]). It is kind of disheartening that they offer comparable internet speeds for free from old telephone booths that people in the west can pay over 50$/mo for even in large cities.
Vietnam’s internet infrastructure in big cities is quite impressive too. If paid in advance for an year, you could get 100 Mbps up/down for ~$11/month. There are several ISPs competing with each other so the speed goes up, the price goes down every few years. High speed fiber internet is commonplace. Higher speed is available too.
The only downside is that some websites are blocked, or slowed down at some points.
But I wonder how it looked e.g. in Romania at that time, they had pretty nice fiber infrastructure because they didn't a lot of the old cables, and in 2015 I read that they had 500 Mbps links (not sure if symmetric) affordable.
And in Poland I'm currently paying ~20 USD/month for 1Gbps up / 300 Mbps down (I got it 2 years ago).
4 years ago I had 300 Mbps / 30 Mbps for similar amount.
This is in suburbs of a bigger city (not capital).
Right now I'm at my parents in a small 1000-inhabitants village (closest "large" city, Carcassonne, 45k inhabitants, is 30km away) and they're getting 2gb fiber installed next week.
Compared to the US, unfortunately, it is. I have Verizon FiOS in a major metropolitan center and only get about 200MB both ways. And that's not even the base plan, that's an upgraded plan.
As with most things like this in the US, the cause is privatization of things that should be public utilities and unchecked monopolistic behavior. In most places you have exactly one choice in internet service provider, so they give you what they give you at the price they ask and you deal with it or you don't have home internet.
And in 2011? Forget about it. I had crappy copper-wire DSL in 2011 and was glad for it.
So, here's an incredibly annoying thing about the fiber situation in Japan. An apartment building can advertise that they have high speed fiber internet, if the fiber is run to the building. They don't actually have to provide high speed to the apartments within the building.
My building was advertised this way, and my apartment has 30Mb ADSL, because that's how the complex was wired when it was built.
That's the fantasy, anyway. Older people in Japan are still being taken for a ride with crap Internet service that cuts out when a phone call comes in and can barely sustain LINE video calls.
In 2003, I was chatting with a schoolmate and I sent him a file. It arrived instantly.
Him: Oh my God, that was fast
Me: I know. I have super fast internet.
Him: Ha, what's your internet speed?
Me: 3 MBps.
Him: That's literally not even possible.
I was kidding of course, it must have been a fluke or the stars were aligned just right. We were both on Dial up. So yeah, my slow 100 Mb/s internet at home today is plenty fast. I wouldn't even know what to do with a Tb/s.
for what it's worth such speeds plus massive consumer affordable hard-drives say 100tb then the net might swing back to being peer to peer.
remember the bittorent days, how torents were slow ?
What's even the point for 99% of households and people beyond like 100mb down/20m up with sub 10ms ping? That's perfectly good for like 3 4k streams at once, HD video chatting, gaming, learning, etc. while backing up iCloud stuff.
Are the journalists for freethink.com (hah) incapable of distinguishing between internet speed (if you had 319Tbit/s internet, you'd go bankrupt, genuinely), and signalling bandwidth?
112 comments
[ 22.3 ms ] story [ 3295 ms ] threadWavelength frequency/2
Even very poor countries these days have ftth at at least 1gbps physical interface data rate (1gbase lx/gepon/gpon). What they don't have is upstream capacity.
America is unique in having tons of upstream fibre, but extremely bad last mile situation.
0: For low frequencies or (relatively) high noise, this is the actual noise floor, but for high frequencies/very low noise, the limit is that you can only send a integer number of photons, each costing some non-negligible fraction of your power budget (so the effective noise floor is (half, I think?) the power needed to send one photon per cycle).
6.66 * 10^14 Hz * log(...? / ...?) = x Tbps
Ookla Global Internet Speeds Comparison : https://www.speedtest.net/global-index
The global average speed of Fixed Broadband is 110 Mbps(download) and 60 Mbps(upload).
The fact that the server Joe is pulling from won't feed that file to him at that rate going out its own locally attached ethernet? (Due to either throttling, or serving stuff thousands of others?)
There is a seven lane highway three miles from where Joe lives; why can't he drive to his friend across town in 3 minutes?
are other responders not seeing the 'T'?
it is the rare home device which can handle 2.5Gbps. 10Gbps switches are common enough. the cheapest switch i'm seeing on fs.com with a 100Gbps uplink port is $3500.
i don't think we currently have the tech to drive 100Gbps through a single port, i think it's done with multiple wavelengths being multiplexed onto a single fiber, and then split back out.
so... at a minimum, joe is kept from 1 Tbps of bandwidth by $35,000, some optical multiplexing hardware, 10+U of rack space, and having absolutely no conceivable way to put the kind of bandwidth to use.
plus his part of the amortized capital costs to build the ISP's end of that.
There is very little practical use to speeds beyond 10 gigabit for home use unless you have some insane PCIE storage or keep a giant ram drive. (Which again asks the practical use category - what are you doing?)
Give it a few years and we'll all be wondering how SPA web apps became gigabytes in size and trying to work out how to make 1Tbs/s to the home practical.
The confusion is probably due to the fact that you can also run the 100G using multiple lanes at lower speeds.
It all depends on what optics you use.
You also don’t need 10U worth of rack space to switch 1 Tbps. You can do that with a single 1U switch.
fair enough.
> You also don’t need 10U worth of rack space to switch 1 Tbps. You can do that with a single 1U switch.
how much does that cost?
About six grand on eBay.
Edit: for reference, the FS switch with 1.2 Tbps of switching capacity is $4k new.
[1] https://electronics.stackexchange.com/questions/329789/how-c...
Google, Microsoft, Amazon and Facebook stored about 1.2 million terabytes of data in 2019. With this speed you could download all of this data in about 8 hours.
Edit: since they are multiplexing, it almost seems like this is DWDM, on 4 optics, and combined into a single datastream. That means is probably more attainable sooner. Seems similar to just adding more cores (or lanes like in current QDR optics).
The current trend is actually reducing those. 1 lambda 100gbps sfp56-dd would be a goldmine for the first chipmaker to ace it.
Edit: this isn’t my area of expertise, but https://news.ycombinator.com/item?id=28674318 and the provided link explain why even “Internet” (let alone tcp/ip) might be quite orthogonal to what this technology really is.
[1] https://en.m.wikipedia.org/wiki/Synchronous_optical_networki...
SONET has optical networking in its name. It appears to me that data not being transmitted over copper wires is a pretty crucial aspect of it... but really none of this matters as I was simply pointing out that TCP is not relevant to the speed record, in answer to the top two comments above, and not trying to make any absolute assertions about how the transport protocol works. So yes, you're right, I guess?
Just need to dig up all fiber and replace it with new fiber. Does it get any more difficult than that?
You can’t just drop bare fiber in the ground, and cover it up. So pretty much all fiber will probably be inside plastic or metal conduit, which also creates an opportunity to replace it much easier than laying new fiber.
You can do this, but it's a bad idea, so I think you meant "can't" (with a unstated "and expect it to work properly, especially long-term").
Much more interesting would be Hollow-core Optical Fiber across long distance where we could reduce latency by 30%. Unfortunately not coming any time soon.
Edit: ( Not sure why the downvote )
[1] https://news.ycombinator.com/item?id=27858067
[2] https://news.ycombinator.com/item?id=27818357
I am not sure how one could even measured combined bandwidth of the internet. Just measuring Tier 1 network?
For reference The purposed PLCN connection from US to HK has 144Tbps alone. The current HKA already has 100Tbps.
https://www.capacitymedia.com/articles/3829012/a-new-era-of-...
"Please don't comment about the voting on comments. It never does any good, and it makes boring reading."[1]
[1] https://news.ycombinator.com/newsguidelines.html
International undersea cables that carry a continents internet are scary fast, but in the tens of terabytes.
I dream of a world where symmetric 100 is $30/mo everywhere though!
I'm renting office space in town where the best I can get is a 40 Mb DSL line because I can't get anything other than HughesNet at the house. More than a few days of SSH over satellite will make you want to regret the day you decided programming would be a fun profession.
Forgetting about the latency for a moment, even though HughesNet advertises nice speeds, it's very much the same "speeds up to" crap most other ISP do. They're so oversubscribed that you can barely browse the internet in the evening most of the time.
In SF though I'm in a webpass wired building and have 1gbps up and down for ~$700/yr or ~$58/month.
On the Peninsula Comcast had 1gbps down and 35mbps up, but they also had a (very expensive) Comcast Gigabit Pro plan that was 2gbps down and 2gbps up with a second 1gbps down/up line included too and a fancy switch also. Hard to order though and the reps don't know about it. Cool that it's available though.
I never ended up getting it, but I wrote up some details here: https://www.reddit.com/r/HomeNetworking/comments/fs6un2/comc...
I have available to me gigabit (down) from Comcast. I had it for awhile and it was glorious but it was also $80/mo on promo and $110 when not. The reason I got it was not for the download speeds, it was the faster upload speeds that came along with it. Still slow, and not even remotely close to speedy. But it was a solid 12mbit up. Compared to the 5mbit up I get on my 100down plan.
I would kill for some reliable upload speeds. It's awful anymore just sending images to people, never mind you want to send a quick minute long video to someone and it takes several minutes to send.
Anecdotally, I moved into a newish (~5 year old) apartment building in Seattle and had my choice of three ISPs that each provided symmetrical gigabit for around $60/month, delivered via an Ethernet jack in a low-voltage cabinet in my closet.
Getting it wired up took one phone call, wherein my switch port was activated remotely and I could grab an IP address and start using the internet. IPv6 via DHCPv6-PD and everything else worked without a charm.
On the other hand, where I live in (also Seattle) is served by exactly one ISP via DOCSIS, providing 1000/35 but nothing more. It can vary block by block here unfortunately.
If we wanted what other countries have for internet, then what we need is to make the lines public and have ISPs rent them from the government. The reason ISPs can create these duopolies is because every new company has to invest a LOT of money to lay down new data lines. If a company already has their cable/phone lines in place then boom, captured market.
Rentable public data lines would make the cost of setting up new ISPs a LOT cheaper.
All of my neighboring blocks have 1g/1g from them, but the most they offer at my address is a 40/40 "fiber-based coax" internet that I'm probably not even wired for. I would love for that to change - my DOCSIS internet has been pretty solid, but I miss the gigabit upload.
Wave G (NOT Wave's coax-based DOCSIS that they bought years ago) offers/offered IPv6 via PD, lots of documentation for it around the internet: https://gist.github.com/dmtucker/cf3f241cf002367825633c988ff... (this isn't me).
I haven't been a Wave G customer in a number of years, but I don't anticipate it's changed since ~2018
[1]: https://clips.twitch.tv/SourDreamyButterflyOSsloth
The only downside is that some websites are blocked, or slowed down at some points.
But I wonder how it looked e.g. in Romania at that time, they had pretty nice fiber infrastructure because they didn't a lot of the old cables, and in 2015 I read that they had 500 Mbps links (not sure if symmetric) affordable.
And in Poland I'm currently paying ~20 USD/month for 1Gbps up / 300 Mbps down (I got it 2 years ago). 4 years ago I had 300 Mbps / 30 Mbps for similar amount.
This is in suburbs of a bigger city (not capital).
Right now I'm at my parents in a small 1000-inhabitants village (closest "large" city, Carcassonne, 45k inhabitants, is 30km away) and they're getting 2gb fiber installed next week.
As with most things like this in the US, the cause is privatization of things that should be public utilities and unchecked monopolistic behavior. In most places you have exactly one choice in internet service provider, so they give you what they give you at the price they ask and you deal with it or you don't have home internet.
And in 2011? Forget about it. I had crappy copper-wire DSL in 2011 and was glad for it.
My building was advertised this way, and my apartment has 30Mb ADSL, because that's how the complex was wired when it was built.
Him: Oh my God, that was fast
Me: I know. I have super fast internet.
Him: Ha, what's your internet speed?
Me: 3 MBps.
Him: That's literally not even possible.
I was kidding of course, it must have been a fluke or the stars were aligned just right. We were both on Dial up. So yeah, my slow 100 Mb/s internet at home today is plenty fast. I wouldn't even know what to do with a Tb/s.