Probably the "machine of the day" for google websearch is an EPYC so they are focused on that. Possibly the issue didn't affect Xeon in the first place, leaving nothing to optimize. <strike>You can see in the absolute values it was already faster on Intel than it is on EPYC after the improvement.</strike> actually I am cross-eyed, I thought there was another zero there.
I wonder if this set of patches was the result of visibility gained from recent `perf mem` features that show loads and stores for each struct member.
> You can see in the absolute values it was already faster on Intel than it is on EPYC after the improvement
I don't know because the table shows a 100 Gbit/s NIC on AMD and a 200 Gbit/s (!) NIC on Intel. Which for some benchmarks would now make AMD twice as fast as Intel.
Note sure if the 100 Gbit vs 200 Gbit labels are correct though.
Epyc has multiple L3 caches while Xeon has a unified L3. If the same TCP socket is accessed on multiple CCXes it would cause cache lines to bounce on Epyc which is pretty costly.
On that subject, it's worth knowing that with the default BIOS settings the whole socket shows up as a single NUMA node, which means that most workloads don't benefit from automatic NUMA balancing or manual NUMA-aware placement within a socket.
One needs to set NPS "Nodes per socket" to e.g. 4 to have one NUMA node per quadrant, and potentially also "L3 cache as NUMA" to have one per CCX [0].
It's definitely a very technical challenge. I have learned a lot since I made the switch to 100 GbE. My website regularly needs to handle more than 100k connections per server, the bottlenecks become very apparent at that scale.
Absolutely wild. We don't have that many simultaneous flows. Maybe like 256 through multiple 40 G cards, but it will be good to try this once it's in-repo on a Debian.
I haven't seen any recent head-to-head benchmarks but Netflix is getting really good performance from FreeBSD networking and they publish detailed information about it. It gets discussed here every six months or so. Of course networking encompasses many different workloads (e.g. routing with PFSense is going to be a different code path than serving data over HTTPS) and each OS probably has its own strengths and weaknesses.
If you want to go down the rabbit hole have a look at VPP. [0] It's a very fast, like 100GBit/s+ and 100 Mpps+, user space routing engine for commodity x86 hardware. Pim van Pelt has many interesting Videos [1] and articles [2] about it. I only tested it briefly before abandoning it because I'm still stuck on measly ~300/120 g.fast for the foreseeable future so I don't need it yet but especially the Linux CP [3] makes it reasonably workable if you have some Linux networking knowledge.
Pour one out for Orinoco support. Those old Wavelan cards were the mainstay of community wireless networks in the early 2000s as well as the card Apple rebranded as the first AirPort cards.
Orinoco was my first experience with WiFi back in my college days. I remember it being incredibly bad, with buggy drivers, and an overall horrendous experience. That was with Windows of course.
Initially the Linux drivers and wireless stack were pretty rough too, I think OpenBSD had the first open support that worked well especially for running in AP mode.
Yes but they existed. I recall this being a time when half the wifi cards used wpa supplicant which was a damn scourge - tho a necessary one. Orinoco cards didn’t require that at least.
Sorry, it's been a long day. My eyes kept moving past that. I wonder why they presented it that way instead of the inverse, like connections per CPU second.
It's the way the kernel-development process works: As soon as a release takes place, there's a period (the "merge window") where new things can be merged in for the next release.
45 comments
[ 0.20 ms ] story [ 107 ms ] threadI wonder if this set of patches was the result of visibility gained from recent `perf mem` features that show loads and stores for each struct member.
I don't know because the table shows a 100 Gbit/s NIC on AMD and a 200 Gbit/s (!) NIC on Intel. Which for some benchmarks would now make AMD twice as fast as Intel.
Note sure if the 100 Gbit vs 200 Gbit labels are correct though.
One needs to set NPS "Nodes per socket" to e.g. 4 to have one NUMA node per quadrant, and potentially also "L3 cache as NUMA" to have one per CCX [0].
[0] https://www.amd.com/content/dam/amd/en/documents/epyc-techni...
[0]: https://fd.io/
[1]: https://video.ipng.ch/search?search=vpp&searchTarget=local
[2]: https://ipng.ch/s/articles/
[3]: https://github.com/pimvanpelt/lcpng
End of an era.
Seems like they're paving the way for removal of all 16-bit PC Card support like FreeBSD.
See https://www.kernel.org/doc/html/latest/process/2.Process.htm... for more details.
To make merges go smoother, there's a set of trees that are designed for developers to stage things that are ready to merge during the next merge window. linux-next is a good example: https://www.kernel.org/doc/man-pages/linux-next.html The Linux Networking folks have their own trees that fulfill the same role for their stuff: https://www.kernel.org/doc/html/next/process/maintainer-netd...
How was this initiative organized?