Memory optimizations to reduce CPU costs (ayende.com)

The given task can be accomplished with not more than a few kilobytes of RAM, a constant independent of the input and output sizes, but unfortunately I suspect the vast majority of programmers now have absolutely no idea how to do so.

Smaller things are faster to copy etc.,. The fun part is that the opposite is true as well, when you have some constant load on a service, if you make the requests faster then you will have less requests in flight at once (Little's law) and the aggregate memory consumed by those requests while they are in flight will hence be less.

The stunning inefficiency of storing the key with every value, even without any GC-specific issues, should give one pause.

Is this similar to data locality?

https://gameprogrammingpatterns.com/data-locality.html

Couldn’t one just lazy load and parse as needed instead of loading potentially 40 million rows into memory? Or better yet, if you have that many…databases exist