Very interesting.. will have to do some local benchmarking. At the very least, the linked google C-state utility is a new addition to my admin toolkit.
1. Don't use RDTSC manually. On many CPUs it produces flat-out wrong results. Use clock_gettime with either CLOCK_REALTIME or CLOCK_MONOTONIC depending on what you're trying to do. It's barely slower than a correctly ordered RDTSC. (You remembered to read the SDM/APM carefully and noticed that RDTSC is executed speculatively and doesn't time what you think it does, right?)
2. Don't tune C states in firmware. Use intel_idle's tuning. It's much more reliable.
3. You think that idle=mwait triggers C1E directly and that's a good thing? Hah! The BIOS setting that's usually described as "[gobbledygook] C1E [more gibberish]" is actually C1E Auto-Promotion, which is a truly awful feature that will detect, in hardware (or really power control firmware) when all cores are in C1 or deeper and promote it to a really deep package state that can take tens of milliseconds to wake from. That's right: it detects when the OS said it wanted a light sleep (C1) for low wakeup latency and promotes it to a very deep sleep. This is bad enough that the excellent intel_idle driver automatically turns it off.
4 comments
[ 6.4 ms ] story [ 20.0 ms ] thread1. Don't use RDTSC manually. On many CPUs it produces flat-out wrong results. Use clock_gettime with either CLOCK_REALTIME or CLOCK_MONOTONIC depending on what you're trying to do. It's barely slower than a correctly ordered RDTSC. (You remembered to read the SDM/APM carefully and noticed that RDTSC is executed speculatively and doesn't time what you think it does, right?)
2. Don't tune C states in firmware. Use intel_idle's tuning. It's much more reliable.
3. You think that idle=mwait triggers C1E directly and that's a good thing? Hah! The BIOS setting that's usually described as "[gobbledygook] C1E [more gibberish]" is actually C1E Auto-Promotion, which is a truly awful feature that will detect, in hardware (or really power control firmware) when all cores are in C1 or deeper and promote it to a really deep package state that can take tens of milliseconds to wake from. That's right: it detects when the OS said it wanted a light sleep (C1) for low wakeup latency and promotes it to a very deep sleep. This is bad enough that the excellent intel_idle driver automatically turns it off.
Question for @dang and @nick: for links to SO questions, is it possible to automatically date them if they aren't the current year?