The numbers in the table for C vs Rust don't make sense, and I wasn't able to reproduce them locally. For a benchmark like this I would expect to see nearly identical performance for those two languages.
$ gcc --version
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
$ gcc -O2 -static -o bench-c-gcc benchmark.c
$ clang --version
Ubuntu clang version 14.0.0-1ubuntu1.1
$ clang -O2 -static -o bench-c-clang benchmark.c
$ rustc --version
rustc 1.81.0 (eeb90cda1 2024-09-04)
$ rustc -C opt-level=2 --target x86_64-unknown-linux-musl -o bench-rs benchmark.rs
$ taskset -c 1 hyperfine --warmup 1000 ./bench-c-gcc
Benchmark 1: ./bench-c-gcc
Time (mean ± σ): 3.2 ms ± 0.1 ms [User: 2.7 ms, System: 0.6 ms]
Range (min … max): 3.2 ms … 4.1 ms 770 runs
$ taskset -c 1 hyperfine --warmup 1000 ./bench-c-clang
Benchmark 1: ./bench-c-clang
Time (mean ± σ): 3.5 ms ± 0.1 ms [User: 3.0 ms, System: 0.6 ms]
Range (min … max): 3.4 ms … 4.8 ms 721 runs
$ taskset -c 1 hyperfine --warmup 1000 ./bench-rs
Benchmark 1: ./bench-rs
Time (mean ± σ): 5.1 ms ± 0.1 ms [User: 2.9 ms, System: 2.2 ms]
Range (min … max): 5.0 ms … 7.1 ms 507 runs
Those numbers also don't make sense, but in a different way. Why is the Rust version so much slower, and why does it spend the majority of its time in "system"?
Oh, it's because benchmark.rs is performing a dynamic memory allocation for each key. The C version uses a buffer on the stack, with fixed-width keys. Let's try doing the same in the Rust version:
--- benchmark.rs
+++ benchmark.rs
@@ -38,22 +38,22 @@
}
// Generates a random 8-character string
-fn generate_random_string(rng: &mut Xorshift) -> String {
+fn generate_random_string(rng: &mut Xorshift) -> [u8; 8] {
const CHARSET: &[u8] = b"0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
- let mut result = String::with_capacity(8);
+ let mut result = [0u8; 8];
- for _ in 0..8 {
+ for ii in 0..8 {
let rand_index = (rng.next() % 62) as usize;
- result.push(CHARSET[rand_index] as char);
+ result[ii] = CHARSET[rand_index];
}
result
}
// Generates `count` random strings and tracks their occurrences
-fn generate_random_strings(count: usize) -> HashMap<String, u32> {
+fn generate_random_strings(count: usize) -> HashMap<[u8; 8], u32> {
let mut rng = Xorshift::new();
- let mut string_counts: HashMap<String, u32> = HashMap::new();
+ let mut string_counts: HashMap<[u8; 8], u32> = HashMap::with_capacity(count);
for _ in 0..count {
let random_string = generate_random_string(&mut rng);
Now it's spending all its time in userspace again, which is good:
> my eyes are immediately drawn to this weird bullshit
Gave me a good chuckle there :)
Appreciate this write up; I'd even say your comment deserves its own article, tbh. Reading your thought process and how you addressed the issues was interesting. A lot of people don't know how to identify or investigate weird bullshit like this.
5 comments
[ 3.0 ms ] story [ 24.8 ms ] thread---
The numbers in the table for C vs Rust don't make sense, and I wasn't able to reproduce them locally. For a benchmark like this I would expect to see nearly identical performance for those two languages.
Benchmark sources:
https://github.com/naveed125/rust-vs/blob/6db90fec706c875300...
https://github.com/naveed125/rust-vs/blob/6db90fec706c875300...
Benchmark process and results:
Those numbers also don't make sense, but in a different way. Why is the Rust version so much slower, and why does it spend the majority of its time in "system"?Oh, it's because benchmark.rs is performing a dynamic memory allocation for each key. The C version uses a buffer on the stack, with fixed-width keys. Let's try doing the same in the Rust version:
Now it's spending all its time in userspace again, which is good:Gave me a good chuckle there :)
Appreciate this write up; I'd even say your comment deserves its own article, tbh. Reading your thought process and how you addressed the issues was interesting. A lot of people don't know how to identify or investigate weird bullshit like this.