I find the mass movement in the home market to 2.5gbe quite odd over 5 or 10gbe, given it’s only marginally faster in contrast, especially given 1gbe has been industry norm for at least 15 years
Sad, especially when Ryzen chips have dual 10G support in the silicon, but only enables in on some products in the name of product differentiation. Dual 10G cards are generally still pretty expensive, and motherboards with dual 10G are often several $100 more than the 1G version ... even when the 10G is supported by the CPU, like on the Xeon D series.
So basically people are still willing to pay more for 10G, and 2.5G is now the "free" upgrade. So my Comcast model, random Ryzen motherboard, and random SFF desktop all came with 2.5G. Sadly same on the Apple side, the Mac mini has an optional 10G, and you have to upgrade to the $$$$ Mac studio to get 10G by default.
And there's even more pre-ethernet POTS wiring. But most of stuff built or renovated in the last 30 years has gotten fiber put in. Which is also much more power efficient than the faster ethernet standards over twised pair.
(Also your cat5e should do 10G for the most common short distances, it's not categorically different, just has looser signal integrity specs than Cat6. Just isn't guaranteed over the 100m max specified distance)
You’ve gone from maybe a dual core processor and 8gb of ram, 500gb of mechanical storage and 512kbps internet w/ a gigabit mic to a 24core processor, up to 128gb of ram, SSDs, gigabit internet to see only a 2.5x speed increase in ethernet?
Edit: to be clear I don’t expect he same rate of growth due to the lack of use at speeds upwards of 10gbe for the average user, but over a 20 year span, 10gbe should be commodity hardware.
Ever since 10MbE, it's been the norm for every new generation to go 10x. 100MbE ("Fast Ethernet") was released in 1995. Gigabit was standardized only 3 years later. Gigabit really started to come to consumer tech in the 2000s. You still find 100MbE on cheaper hardware.
But here we are in 2022 and 10GbE is nowhere near standard and may well never be. It makes me think 10x'ing here was a mistake. We have 2.5GbE and 5GbE standards now but we should've probably had those earlier.
Copper over 10GbE in particular runs really hot. Often adapters will just be massive heatsinks and/or fans.
It's interesting reading here that a lot of 25GbE connectors (50-60% was one figure quoted) failed to negotiate 10GbE. I can't say I expected that. That seems like a problem.
It's kind of wild you can get a 10GbE switch for under $200 as you'll often see 10GbE gear run at $100-200+/port.
I never thought I'd actually shop for a 10Gbe switch here in canada, but I went from a 100mbps trash-tier docsis (ugh) connection to an almost too-good-to-be-true symmetric 1.5gbs FTTH with Bell Fibe. For the same price (!), around 60$/month with a permanent discount that Bell seems to hand out to everyone who asks.
The issue is that it seems to be very hard to find affordable switches with 10gbe ethernet copper ports, instead of SFP. The few that exist either cost more than what I pay for my connection in an entire year, or don't interact well with the ISP provided hub/modem (it has no SFP). Bell also seems to have locked down their new hardware so it is impossible to bypass their hub. And while I have a home server rack, even the more affordable second hand enterprise switches are usually ancient powerhogs.
So for now wifi 6e is basically just as fast as my wired setup, which is weird.
I guess I'm not used to being at the edge of home networking tech as a canadian... but I'd be happy to hear about recommendations and what to look for/avoid with that type of gear either (new or used)
Do you need to have 10GbE? That is very high end kit, especially with copper. The power consumption is extreme and the distances are never going to be very good compared to fiber.
2.5GbE ethernet is super accessible now from Netgear, Trendnet, TPLink etc and doesn't require new NICs on any newer desktops. All my shit that 2.5GbE is useful supports 2.5GbE.
I'm not sure actually, I have a local NAS/Server rack where I store work related datasets (think a few hundred/thousands 12-15GB .h5 files). And while I definitely don't need instant or even super fast access to them, I'm sure it wouldn't hurt either. We already have tons of optimizations and usually process datasets in a cloud environment with "good" IOPS/Network[1] speeds, but it is still faster to use locally in some situations. 10Gbe gear would be a nice to have to avoid bottlenecks within the LAN... but I just realized that I'm making a good case for just shelling out more money instead of cheaping out haha.
[1] As good as it gets on Azure, I guess. Which is not much at all, even on Ultra SSDs so I keep a local cache for smaller scale stuff
I guess to me 2.5gbe just seems like a weird in between with less support, but I agree that it's probably going to be my only option otherwise. Though a 10Gbe switch would also be awesome for local lan, since my servers' NVME SSDs are constantly "throttled" by my 1Gbe (and would be at 2.5Gbe too, but to a lesser extent).
For about $50 US, you can get a 10Gbase-T SFP+ module that gives you a copper port at 10gig within a SFP+ form factor. That, coupled with a cheap Mikrotik switch, is plenty to get started.
Second hand Brocade ICX switches are also plentiful and not too power hungry (but they can be loud).
Depending on how many ports you need, you may get away with an ICX 7150-12. It has 14 fixed Gb copper ports and 2 10 Gb SFP ports. It's fanless and has 12 GbE PoE ports.
Woah, the brocade switches might be a very good option. I don't mind it being loud, since my server rack is in a well insulated part of my basement and have had no noise issues even with a couple 2U servers. Thanks for the suggestion!
Just a heads up to anyone who does intend to go buy a bunch of those modules.
They run absurdly hot, and they eat a lot of power.
If you don't have very good ventilation then you may find these modules running at well north of 80C. I ended up having to mount a 120MM PC fan and attach some mini-headsinks to them to keep them under 50C. Just sitting on a desk they were too hot to touch.
You may find that the power budget for your small 8-12 port SFP+ Switch isn't enough to run more than a few of these at once. I have an 8 Port Mikrotik SFP+ switch that can only run about 4 of them. The Quad-Port SFP+ NIC I bought from fs.com (Intel XL710-BM1) can only do two.
Expect to also run into issues if you use old, poor quality or mistreated cables. If you bend them too far (trying to get around a sharp corner) - they will work fine at 1Gbit, but just not work at all at 10Gb.
You may find it's a whole lot cheaper (and more reliable) to just get some optical cable and transceivers if you want to go more than a few meters.
For distances less than a few meters DAC cables are often reasonable value.
From personal experience, I'm now leaving the 10G-BaseT transceivers for runs where I can't run optical (like existing in-wall wiring).
Yeah, optics or DAC cables are definitely preferable. I wouldn’t run more than one or two 10G-BaseT units in most switches - but when you have a modem or device that only supports it, it’s a much cheaper way to get connected than buying a dedicated switch.
Even for really short runs? I'd mostly use it for the link between my ISP modem to the switch, and wouldn't mind slower speeds for the rest of my devices. But that's definitely something I'll have to keep in mind.
I also never thought I'd bother with 10GbE until I stumbled on some used Mellanox ConnectX-3 cards that were just cheap enough to play around with (~$30/card including DAC cables). Seeing file transfer speeds of >700MBps between my server and NAS convinced me it was worth keeping them installed, though initially just as a point-to-point link between those two systems.[1]
A few months later, I was upgrading to a 24-port managed switch, and found one with four SPF+ ports (Aruba 1930) for little more than I could find a purely gigabit switch (~£180). I moved the 10Gbps NAS/server link to the main network, and added 10GbE to my main PC (with another ConnectX-3), and it's been a really nice system to use. A little overkill for most of my usage, but it's nice to have.
I'm not convinced it's worth the cost of moving most of my other devices to 10GbE, even though I have a >1Gbps (down) Internet connection, as switches with more SPF+ ports or any 10G-BaseT ports (which some of my devices would require IIRC) seem rather expensive. Like you, I'm also open to suggestions if there's a cheap(ish) option I've overlooked.
[1] Side note: I actually moved a USB drive from the server to the NAS as it had significantly faster (by about 30%) sustained read/write speeds that way. I'm presuming the NAS is caching the requests in memory, but both are on a UPS and it's only temporary working data anyway so I'm not too worried if the data doesn't actually hit the disc immediately.
I think you can still get the HH3000 if you manage to escalate a request past the first layers of consumer support.
I also heard about buying special optics that you can completely set up to mimic the original bell GPON, down to the SN but those are expensive. It's so weird to "downgrade" such a useful feature (that does not even hurt any other part of their business, and only serve to make 1.5gbps+ speeds more attractive for those who would actually need that much speed), but hey it's still bell after all
This is interesting but I would much prefer to know what the cheapest copper 10GbE switch is ...
We've been buying the d-link DXS-1210-12TC for top-of-rack 10GbE switches and at (roughly) $15xx it's fine ... but I don't have one of those in the lab ...
I've been searching for the same thing and the cheapest is the QNAP QSW-2104-2T for $160 USD which seems to be the same as the Trendnet TEG-S762.
It has 2 10G multigig ports and 4 2.5G.
My eventual goal is to have NAS and desktop on 10G and rest on 2.5G.
Unfortunately my NAS setup can't really take advantage of the 5G+ speeds and 10G PCIE cards are not competitively priced so it will get a $17 2.5G PCIE card for now.
It may be the time to switch to fiber. You may be too young to remember when ethernet switched from coax (10Base2) to twisted pair (10BaseT), but the economics look like it will soon be time to make the physical media jump again.
First the disclaimer, very much work in progress (WIP)! Don't YET expect everything to work! 10/100Mbit is not working, SFP+ is experimental and the code base is a bit of a mess (but far better then the SDK).
38 comments
[ 3.2 ms ] story [ 105 ms ] threadSo basically people are still willing to pay more for 10G, and 2.5G is now the "free" upgrade. So my Comcast model, random Ryzen motherboard, and random SFF desktop all came with 2.5G. Sadly same on the Apple side, the Mac mini has an optional 10G, and you have to upgrade to the $$$$ Mac studio to get 10G by default.
(Also your cat5e should do 10G for the most common short distances, it's not categorically different, just has looser signal integrity specs than Cat6. Just isn't guaranteed over the 100m max specified distance)
The upper limit for 10GbE over CAT5e is about 45m: https://www.universalnetworks.co.uk/10gbaset-can-this-be-run...
I expect that is sufficient for most residential installations.
I'd imagine the answer is cost & cooling. 10gbe is substantially more energy intensive than 1gbe.
Edit: to be clear I don’t expect he same rate of growth due to the lack of use at speeds upwards of 10gbe for the average user, but over a 20 year span, 10gbe should be commodity hardware.
But here we are in 2022 and 10GbE is nowhere near standard and may well never be. It makes me think 10x'ing here was a mistake. We have 2.5GbE and 5GbE standards now but we should've probably had those earlier.
Copper over 10GbE in particular runs really hot. Often adapters will just be massive heatsinks and/or fans.
It's interesting reading here that a lot of 25GbE connectors (50-60% was one figure quoted) failed to negotiate 10GbE. I can't say I expected that. That seems like a problem.
It's kind of wild you can get a 10GbE switch for under $200 as you'll often see 10GbE gear run at $100-200+/port.
10GBASE-T was standardized in 2006 so 2027+ hopefully? https://en.wikipedia.org/wiki/10_Gigabit_Ethernet#Copper
0: https://www.servethehome.com/the-ultimate-cheap-2-5gbe-poe-u...
The issue is that it seems to be very hard to find affordable switches with 10gbe ethernet copper ports, instead of SFP. The few that exist either cost more than what I pay for my connection in an entire year, or don't interact well with the ISP provided hub/modem (it has no SFP). Bell also seems to have locked down their new hardware so it is impossible to bypass their hub. And while I have a home server rack, even the more affordable second hand enterprise switches are usually ancient powerhogs.
So for now wifi 6e is basically just as fast as my wired setup, which is weird.
I guess I'm not used to being at the edge of home networking tech as a canadian... but I'd be happy to hear about recommendations and what to look for/avoid with that type of gear either (new or used)
2.5GbE ethernet is super accessible now from Netgear, Trendnet, TPLink etc and doesn't require new NICs on any newer desktops. All my shit that 2.5GbE is useful supports 2.5GbE.
[1] As good as it gets on Azure, I guess. Which is not much at all, even on Ultra SSDs so I keep a local cache for smaller scale stuff
Second hand Brocade ICX switches are also plentiful and not too power hungry (but they can be loud).
They run absurdly hot, and they eat a lot of power.
If you don't have very good ventilation then you may find these modules running at well north of 80C. I ended up having to mount a 120MM PC fan and attach some mini-headsinks to them to keep them under 50C. Just sitting on a desk they were too hot to touch.
You may find that the power budget for your small 8-12 port SFP+ Switch isn't enough to run more than a few of these at once. I have an 8 Port Mikrotik SFP+ switch that can only run about 4 of them. The Quad-Port SFP+ NIC I bought from fs.com (Intel XL710-BM1) can only do two.
Expect to also run into issues if you use old, poor quality or mistreated cables. If you bend them too far (trying to get around a sharp corner) - they will work fine at 1Gbit, but just not work at all at 10Gb.
You may find it's a whole lot cheaper (and more reliable) to just get some optical cable and transceivers if you want to go more than a few meters.
For distances less than a few meters DAC cables are often reasonable value.
From personal experience, I'm now leaving the 10G-BaseT transceivers for runs where I can't run optical (like existing in-wall wiring).
https://www.arista.com/assets/data/pdf/Copper-Faster-Than-Fi...
A few months later, I was upgrading to a 24-port managed switch, and found one with four SPF+ ports (Aruba 1930) for little more than I could find a purely gigabit switch (~£180). I moved the 10Gbps NAS/server link to the main network, and added 10GbE to my main PC (with another ConnectX-3), and it's been a really nice system to use. A little overkill for most of my usage, but it's nice to have.
I'm not convinced it's worth the cost of moving most of my other devices to 10GbE, even though I have a >1Gbps (down) Internet connection, as switches with more SPF+ ports or any 10G-BaseT ports (which some of my devices would require IIRC) seem rather expensive. Like you, I'm also open to suggestions if there's a cheap(ish) option I've overlooked.
[1] Side note: I actually moved a USB drive from the server to the NAS as it had significantly faster (by about 30%) sustained read/write speeds that way. I'm presuming the NAS is caching the requests in memory, but both are on a UPS and it's only temporary working data anyway so I'm not too worried if the data doesn't actually hit the disc immediately.
One issue I have with Bell Fibe is the new HomeHub 4000 hardware is mandatory, and the GPON optic is not removable (the one in the HH 3000 is).
I'm not exactly enamoured at having to use Bell's hardware (though I'm sure it gets the job done for most people).
I also heard about buying special optics that you can completely set up to mimic the original bell GPON, down to the SN but those are expensive. It's so weird to "downgrade" such a useful feature (that does not even hurt any other part of their business, and only serve to make 1.5gbps+ speeds more attractive for those who would actually need that much speed), but hey it's still bell after all
We've been buying the d-link DXS-1210-12TC for top-of-rack 10GbE switches and at (roughly) $15xx it's fine ... but I don't have one of those in the lab ...
How much is this with fiber SFPs factored in?
For short distances (e.g. in the same room) you'd be smart to use Twinax.
First the disclaimer, very much work in progress (WIP)! Don't YET expect everything to work! 10/100Mbit is not working, SFP+ is experimental and the code base is a bit of a mess (but far better then the SDK).
Anyway, https://forum.openwrt.org/t/support-for-rtl838x-based-manage... is the forum thread talking about these switches, https://svanheule.net/switches/ is the wiki.
Lots of help is still needed, even if it is just doing teardowns with photo's and help in learning stuff. Full GPL source is available though.