I would like the see the source code for libmymalloc.so, however, I don't see anything in the blog post. Nor do I see anything in his GitHub profile: https://github.com/jsikstro
Also, I cannot find his email address anywhere (to ask him to share it on GitHub).
Huh. Why is this emergency pool not statically allocated? Is it possible to tune the size of this pool on libc++ startup somehow? Because otherwise it absolutely should've been statically allocated.
> TLDR; The C++ standard library sets up exception handling infrastructure early on, allocating memory for an “emergency pool” to be able to allocate memory for exceptions in case malloc ever runs out of memory.
Reading this was a good reminder not to be intimidated by assumptions about complexity. (Without giving it much thought) I would have assumed that it would be hard to replace malloc for such fundamental applications as ls, but it's surprisingly simple.
This applies to a lot of things unfortunately. There is a cult of just being afraid and scaring other people.
"You can't do it, just use a library.". "Just use this library, everyone uses it.". "Even google uses this library, do you think you are better." etc.
To add another example to this, you will read that memcpy is super mega optimized on libc and you shouldn't do it yourself etc. etc. etc.
But if you just check clickhouse [1] as an example. They implemented it, it is pretty basic and they say it works well in the comments of the code.
Also you can check musl libc code etc. and it is fairly simple.
People still would argue that you used some intrinsic so it isn't portable or you just benchmarked on one case so it won't work well overall.
Well you CAN benchmark as wide as you want inside your project and have a different memcpy code per project. This kind of thing isn't as bad as people make it out to be in my opinion.
Ofc memcpy is just an example here and it applies similarly to memory allocation, io etc.
As a negative note, imo this is one of the major reasons why most software is super crappy now. Everything uses some library -> those libraries change all the time -> more breaking -> more maintenance. Similar chain happens in terms of performance because the person that wrote that library probably doesn't even know how I am using the library.
This is also why people have endless arguments about what library/tool to use while they can be learning more and more things every day.
100% - the number of times you will need to use a super optimized memcpy() in real life versus the benefit you can get from looking at and writing basic versions of it for different CPU's is very slim.
Then you'll have a much better idea of when to _really_ use one that depends on intrinsics, is optimized etc, and how to benchmark them ... those are the real skills.
Unfortunately, a lot of system level knowledge like this is not found in a single place but spread over many articles/manuals/books/etc.
However, the book Advanced C and C++ Compiling: An Engineering Guide to Compiling, Linking and Libraries using C and C++ by Milan Stevanovic brings together a lot of information which you might find interesting.
So basically, before any of the code even runs, this environment begins by gobbling up more than the total RAM that most of my first computers had (SYM-1, IAMSAI-8080, Ferguson Big Board, Kaypro II, and CCS S-100 Z-80). All of these systems were 8-bit, with various RAM sizes from 8KB to 64KB. That was the maximum RAM available, and it was shared by the OS and the applications.
What's the purpose of making such a comparison? The implication is that we're being wasteful, but I'm not certain that's the point you're trying to make.
17 comments
[ 4.2 ms ] story [ 38.9 ms ] threadAlso, I cannot find his email address anywhere (to ask him to share it on GitHub).
Am I missing something?
Reminds me of Perl's $^M: https://perldoc.perl.org/variables/$%5EM
In Perl you can "hand-manage" that. This line would allocate a 64K buffer for use in an emergency:
"You can't do it, just use a library.". "Just use this library, everyone uses it.". "Even google uses this library, do you think you are better." etc.
To add another example to this, you will read that memcpy is super mega optimized on libc and you shouldn't do it yourself etc. etc. etc.
But if you just check clickhouse [1] as an example. They implemented it, it is pretty basic and they say it works well in the comments of the code.
Also you can check musl libc code etc. and it is fairly simple.
People still would argue that you used some intrinsic so it isn't portable or you just benchmarked on one case so it won't work well overall.
Well you CAN benchmark as wide as you want inside your project and have a different memcpy code per project. This kind of thing isn't as bad as people make it out to be in my opinion.
Ofc memcpy is just an example here and it applies similarly to memory allocation, io etc.
As a negative note, imo this is one of the major reasons why most software is super crappy now. Everything uses some library -> those libraries change all the time -> more breaking -> more maintenance. Similar chain happens in terms of performance because the person that wrote that library probably doesn't even know how I am using the library.
This is also why people have endless arguments about what library/tool to use while they can be learning more and more things every day.
[1] https://github.com/ClickHouse/ClickHouse/blob/master/base/gl...
Then you'll have a much better idea of when to _really_ use one that depends on intrinsics, is optimized etc, and how to benchmark them ... those are the real skills.
Unfortunately, a lot of system level knowledge like this is not found in a single place but spread over many articles/manuals/books/etc.
However, the book Advanced C and C++ Compiling: An Engineering Guide to Compiling, Linking and Libraries using C and C++ by Milan Stevanovic brings together a lot of information which you might find interesting.