Alternative unit I like is "how many per second" (when done on single hardware unit, one after another, of course). The numbers from there are then easy to grasp, since then they relate to what we actually want to achieve (I haven't checked them for truth):
In billions: L1 cache reference can be done 2 billion times per second.
In millions: Branch mispredict, L2 cache reference,
Mutex lock/unlock and Main memory reference can be done 200, 140, 40 and 10 million times per second.
In thousands: Compress 1K bytes with Zippy, Send 2K bytes over 1 Gbps network, SSD random read, Read 1 MB sequentially from memory, Round trip within same datacenter and Read 1 MB sequentially from SSD* can be done 300, 50, 7, 4, 2 and 1 thousand times per second.
And finally, Disk seek, Read 1 MB sequentially from disk and Send packet CA->Netherlands->CA can be done 100, 50 and 7 times per second.
No graphs needed, or trying to figure out is a microsecond much or not.
The numbers in the chart better reflect an 'add up the costs' accounting methodology which is useful for deadlines or cost/benefit comparisons. It had also better be worth it to spend very costly human time to save comparatively cheep computer time. Things at E.G. FANG scale rather than the cron-job that does something overnight at a small business.
In particular, those of us using tracers need to understand the frequency of what we instrument, as it relates to overhead. I came up with a frequency table for Chapter 18 of BPF Performance Tools (out soon).
The 2k over a network latency really depends on the networking hardware in the computers and switches and how the software stack is set up. I wouldn't be surprised if it's easily 10x that in most real cases.
I think it's a case of "one of these is not like the other" this number seems to refer to a slice of throughput. There's a separate line item for "round trip in the datacenter".
Yeah, it's "time taken to transmit", not round trip time or even one-way end to end latency. It's useful if you want to know eg. the minimum extra latency being added to other network flows because your NIC is busy transmitting this packet for the next n microseconds.
Definitely more expensive than a function call. You need to switch to a different virtual address space, set up the CPU registers appropriately, et cetera. All of this happens in kernel space, so you also have a switch to kernel mode (and back). Latency wise, this should be somewhere in the single-digit microsecond range, I'd think. However, fallouts of a context switch include dealing with cache misses, TLB misses, and the like, which - depending on what you do - could be pretty expensive.
Function call overhead is on the order of several 10s of clock cycles (round to 100, not 10), or a few dozen nanoseconds. The actual impact of function calls is usually not so much the overhead of the call itself, but rather on the fact that function calls tend to be barriers to analyses and optimizations.
Context switches are closer to about 1µs in the best case, with interprocess context switches maybe reaching 5µs. The time here is much more variable because the really painful thing is the effect of TLB flushing and other microarchitectural state going completely haywire.
> The time here is much more variable because the really painful thing is the effect of TLB flushing and other microarchitectural state going completely haywire.
Isn't context switch also where spectre/meltdown mitigations hit the hardest, too?
An execution unit is basically an ALU with a set of muxes for inputs, where the muxes are pulling either from the registers or from later execution stages that have yet to commit their results to registers. The entire path has to settle down by the time the clock signal reaches the latch at the edge of the stage to retain the value for the next clock cycle. This means that the time it takes to read a register is a fraction of the single clock cycles that 1-clock-cycle µops take to run.
Registers are finitely faster, not infinitely faster, than L1. Now if registers were infinitely faster than L1 then that table would really want to raise that hosanna to the highest.
How about "faster than anything interacting with them (so effectively infinite"? (I agree that calling it infinite is a little weird, although it's better if one considers that infinity is usually shorthand)
Any info on the difference of latency of register access vs. L1 cache on x86_64?
When looking at compiler output of gcc most of the time it puts variables within a scope on the stack and rarely populates all registers or even any at all. Wouldn't using the full 12-14 available first be faster?
As for your second question, my guess is you're looking at unoptimized code. Most compilers keep variables on the stack unless you're optimizing the code to at least some degree.
28 comments
[ 3.5 ms ] story [ 72.2 ms ] threadIn billions: L1 cache reference can be done 2 billion times per second.
In millions: Branch mispredict, L2 cache reference, Mutex lock/unlock and Main memory reference can be done 200, 140, 40 and 10 million times per second.
In thousands: Compress 1K bytes with Zippy, Send 2K bytes over 1 Gbps network, SSD random read, Read 1 MB sequentially from memory, Round trip within same datacenter and Read 1 MB sequentially from SSD* can be done 300, 50, 7, 4, 2 and 1 thousand times per second.
And finally, Disk seek, Read 1 MB sequentially from disk and Send packet CA->Netherlands->CA can be done 100, 50 and 7 times per second.
No graphs needed, or trying to figure out is a microsecond much or not.
if its in ram, then like ram.
then size/amount of parameters affects it, and its not continuous because at some threshold compilers may start doing it differently.
Context switches are closer to about 1µs in the best case, with interprocess context switches maybe reaching 5µs. The time here is much more variable because the really painful thing is the effect of TLB flushing and other microarchitectural state going completely haywire.
Isn't context switch also where spectre/meltdown mitigations hit the hardest, too?
An execution unit is basically an ALU with a set of muxes for inputs, where the muxes are pulling either from the registers or from later execution stages that have yet to commit their results to registers. The entire path has to settle down by the time the clock signal reaches the latch at the edge of the stage to retain the value for the next clock cycle. This means that the time it takes to read a register is a fraction of the single clock cycles that 1-clock-cycle µops take to run.
When looking at compiler output of gcc most of the time it puts variables within a scope on the stack and rarely populates all registers or even any at all. Wouldn't using the full 12-14 available first be faster?
As for your second question, my guess is you're looking at unoptimized code. Most compilers keep variables on the stack unless you're optimizing the code to at least some degree.
Instant coffee?
Maybe French press, if the water is already boiling.