One thing that I couldn't spot is the number of concurrent connections this setup does? That's a really important use case when testing middleboxes. There are relatively easy ways to generate 10Gbps of traffic, not so many ways to generate traffic with 1M concurrent connections.
The performance is not the greatest in absolute terms, but indeed there's only so much general purpose computing you can do on these miserable network accelerators :-( You'd probably do 10x as much traffic per core on a 5 year old low end Xeon.
Handling a lot of connection should not be an issue for the network stack, at there is no shared state between TCP streams. The bottleneck would rather located at the application layer, if resources are shared between connections.
I do not think that you could get a 10 times improvement by using x86 hardware. It should rather be somewhere near a 2 or 3 times improvement.
Hm. So the load generator with an integrated userspace TCP stack I use at work does about 7Gbps with 100k concurrent connections on a single Xeon E5506 core (released in 2009). It's not quite the same thing as your project since it's a simpler application level protocol than HTTP. But that's still a big difference, and your architecture didn't look insane.
There are some reasons why performance doesn't scale nicely when you add in more connections, even if you've got no shared state between the streams. For example the socket table lookups tend to get more and more expensive, and you start hitting issues like frequent cache misses on the socket records themselves especially. Especially if the representation is at all bloated. So getting a number on the scalability in terms of connections would be a good idea if you intend to continue developing it.
Rusty is the name of the TCP/IP stack that was developed to increase the throughput of the traffic simulator. The software is written using the C++ programming language.
Awes, I was hoping (due to the name) that it would be written in Rust.
5 comments
[ 5.5 ms ] story [ 20.1 ms ] threadOne thing that I couldn't spot is the number of concurrent connections this setup does? That's a really important use case when testing middleboxes. There are relatively easy ways to generate 10Gbps of traffic, not so many ways to generate traffic with 1M concurrent connections.
The performance is not the greatest in absolute terms, but indeed there's only so much general purpose computing you can do on these miserable network accelerators :-( You'd probably do 10x as much traffic per core on a 5 year old low end Xeon.
I do not think that you could get a 10 times improvement by using x86 hardware. It should rather be somewhere near a 2 or 3 times improvement.
There are some reasons why performance doesn't scale nicely when you add in more connections, even if you've got no shared state between the streams. For example the socket table lookups tend to get more and more expensive, and you start hitting issues like frequent cache misses on the socket records themselves especially. Especially if the representation is at all bloated. So getting a number on the scalability in terms of connections would be a good idea if you intend to continue developing it.
Awes, I was hoping (due to the name) that it would be written in Rust.