I don't think you're wrong. Virtual functions is a two-pointer dereference operation (vptr, vtable[vptr]), and there we can have a d-cache miss but the main cost of using virtual functions is the increased likeliness of the i-cache miss. Cost of 30-60 cycles as per article assumes an icache-hit, and since virtual call is an indirect call (jump), it also heavily depends on the branch-target predictor and its buffer. I can easily imagine that iterating over a heterogeneous collection of objects would incur much larger cost than ~50 cycles/iteration. Branch target misprediction flushes the whole pipeline (15-20 cycles) and icache miss can easily end up being a fetch from main memory (200-300 cycles)
The article in general is interesting since it gives a rough idea of cost of operations relative one to each other but since CPUs are much more complex beasts it also gives us an incomplete picture, and if you're unaware of it the chance is that you will use it derive incomplete conclusions from it - understanding performance implications of a software running on an actual hardware is much more involved than what one article can fit.
I'm somewhat dubious about anything talking about low level performance programming at the instruction level that doesn't distinguish between latency and throughput, never mind mention the incredibly out-of-order nature of modern desktop/server class CPU cores.
That's a very important point. For instance on Intel's CPUs multiplication is pipelined - its latency is 3 cycles, but throughput is 1 cycle. Thus completing N multiplication takes 2 + N cycles (in the best case), not 3 * N.
what if a language would allow to elegantly pack Optional values?
so the physical layout has a bit vector with one bit for each optional. and a popcnt over that bitvector (masked up to the value we're interested in) will give the actual slot to look into?
would also make sense to reorder / bucket fields by (byte) size
if you want to do that in any low level language (rust, c++) you have to deviate from their standard syntax for optionals, and you have to manually keep track of slot order. but for domains with many optional/default values, this amy really reduce cache pressure, no?
In higher level languages you can fake the effect (with flyweight facades), so from python such a packed "dataclass"-like class can look neat and clean. however at the low level there is no abstraction that allows to create your own data layout.
12 comments
[ 4.4 ms ] story [ 40.1 ms ] threadThe article in general is interesting since it gives a rough idea of cost of operations relative one to each other but since CPUs are much more complex beasts it also gives us an incomplete picture, and if you're unaware of it the chance is that you will use it derive incomplete conclusions from it - understanding performance implications of a software running on an actual hardware is much more involved than what one article can fit.
so the physical layout has a bit vector with one bit for each optional. and a popcnt over that bitvector (masked up to the value we're interested in) will give the actual slot to look into?
would also make sense to reorder / bucket fields by (byte) size
if you want to do that in any low level language (rust, c++) you have to deviate from their standard syntax for optionals, and you have to manually keep track of slot order. but for domains with many optional/default values, this amy really reduce cache pressure, no?
In higher level languages you can fake the effect (with flyweight facades), so from python such a packed "dataclass"-like class can look neat and clean. however at the low level there is no abstraction that allows to create your own data layout.
at least I didn't find anything yet.