If you’ve brave enough to try managing memory yourself, C++ has a whole library of collections that support custom allocators without the readability penalty for FFI. I wish they would have written why they didn’t use it.
Actually, the implementation of arenas is broken in glibc malloc() when running on multicore systems.
I have been debugging that problem for some time. The charts in the article are quite similar to the ones I got when I initially came across the problem.
Wonder how many threads does their process have? and how many cores on the machine?
There are environment variables that can adjust the behavior of malloc. They should try setting MALLOC_ARENA_MAX=4 and see if the memory usage goes down.
> Over the years, we have tried all tricks of the trade within Go. Using sync.Pool, maintaining our own freelists, avoiding allocations on heap wherever possible, using buffer arenas and so on
I'm interested to know why pools/freelists didn't work out for nodes
6 comments
[ 3.7 ms ] story [ 28.2 ms ] threadI have been debugging that problem for some time. The charts in the article are quite similar to the ones I got when I initially came across the problem.
Wonder how many threads does their process have? and how many cores on the machine?
There are environment variables that can adjust the behavior of malloc. They should try setting MALLOC_ARENA_MAX=4 and see if the memory usage goes down.
I'm interested to know why pools/freelists didn't work out for nodes