This solution doesn't do anything to prevent leaking memory in anything but the most pedantic sense, and actually creates leaks and dangling pointers.
The function just indirects malloc with a wrapper so that all of the memory is traversable by the "bigbucket" structure. Memory is still leaked in the sense that any unfreed data will continue to consume heap memory and will still be inaccessible to the code unless the application does something with the "bigbucket" structure--which it can't safely do (see below).
There is no corresponding free() call, so data put into the "bigbucket" structure is never removed, even when the memory allocation is freed by the application. This, by definition, is a leak, which is ironic.
In an application that does a lot of allocations, the "bigbucket" structure could exhaust the heap even though there are zero memory leaks in the code. Consider the program:
int main(int argc, char** argv) {
for (long i = 0; i < 1000000; i++) {
void *foo = malloc(sizeof(char) * 1024);
free(foo);
}
return 0;
}
At the end of the million iterations, there will be zero allocated memory, but the "bigbucket" structure will have a million entries (8MB of wasted heap space on a 64-bit computer). And every pointer to allocated memory in the "bigbucket" structure is pointing to a memory address previously freed so now points to a completely undefined location--possibly in the middle of some memory block allocated later.
While the provided code may seem like an interesting approach, it's important to note that it introduces a number of issues and potential pitfalls. This code is an attempt to intercept the malloc function using the dlsym function from the dlfcn.h library and store every allocated pointer in a linked list called bigbucket. However, there are several problems with this solution:
Portability: This code relies on the dynamic linking functionality provided by the operating system. It may not work on all systems or with all compilers.
Concurrency Issues: This solution is not thread-safe. If the program uses multiple threads, concurrent calls to malloc may result in race conditions and data corruption in the bigbucket linked list.
Incomplete Solution: This code only intercepts calls to malloc. If the program uses other memory allocation functions like calloc, realloc, or custom memory allocators, memory leaks may still occur.
Performance Overhead: The code introduces additional overhead for every memory allocation, potentially affecting the program's performance.
Undefined Behavior: Overriding standard library functions like malloc can lead to undefined behavior. The behavior of the program is no longer guaranteed to be consistent across different platforms or even different runs.
Limited Practicality: While this approach technically prevents memory leaks by keeping track of all allocated pointers, it does not address the root cause of memory leaks, which is the failure to deallocate memory when it is no longer needed. Encouraging developers not to free memory is not a good practice and can lead to inefficient memory usage.
A better approach to avoiding memory leaks is to adopt good programming practices, such as carefully managing memory allocation and deallocation, using automated tools like static analyzers and memory debuggers, and, when applicable, leveraging programming languages with automatic memory management (e.g., garbage collection in languages like Java or Python).
The last part is funny. Even if you are technically not leaking memory you can still have pretty much the same end result in garbage collected languages if you mess up. The language might be tracking the objects but it can't know if you don't need them anymore.
Why isn't it serious? I can imagine a smart pointer that actually accomplished the stated goal. So the joke is that they took a good idea and made a rubbish solution?
> I can imagine a smart pointer that actually accomplished the stated goal.
Can you elaborate on that?
If it requires you to free correctly, then you can just use plain C and the smart pointer isn't accomplishing anything. If it doesn't require that, how does it work?
Traditionally memory is considered "leaked" if it is still allocated but nothing point to it; i.e. there's no way to navigate to the allocation anymore.
He has made a joke "solution" by simply permanently storing a second pointer to all allocations so that by this definition they never technically leak. You can still always navigate to ever allocation so no allocation has leaked.
Of course it's not a real solution because it doesn't actually change the memory characteristics of a leaky program; it just hides the leak. In other words the technical description of the leak above isn't really the thing we care about.
there is plenty of code that is designed not to free at all because the program will not run long enough to generate much garbage. and having the whole process shut down is simpler than unwinding everything perfectly. so, no, it's not obvious. I had no trouble reading what the code did (and didn't) but it's not obvious that the author understood it. There are plenty of people here who need that much help with C
> It is [...] entirely optional to call free. If you don’t call free, memory usage will increase over time, but technically, it’s not a leak. As an optimization, you may choose to call free to reduce memory, but again, strictly optional.
This is beautiful! Unless your program is long-running, there's no point of ever calling free in your C programs. The system will free the memory for you when the program ends. Free-less C programming is an exhilarating experience that I recommend to anybody.
In the rare cases where your program needs to be long-running, leaks may become a real problem. In that case, it's better to write a long-running shell script that calls a pipeline of elegant free-less C programs.
If you're writing a library, you're not writing a program.
If you write a C library, it is a good practice to leave the allocations to the library user, or at least provide a way to override the library's allocator. Allowing your user to write a free-less *program*.
Any large program is composed of libraries. They may not explicitly be described as libraries, or imported from external sources, but there will be abstraction boundaries somewhere.
Which means that if you're writing a program, you probably are also writing one or more libraries.
The difference is that if you're writing a program you know the scope of use of all libraries, whether they be externally loaded or internal abstraction boundaries, and also know the scope of use of the program, and can make a call as to whether cleanup during opetation is required.
Not really. In theory you can, but in practice there are parts written by a different team and you don't know the scope of there parts.
I also know someone who maintains some Clinton era encryption. that code is controlled who can know about it as obsecurity was all you were allowed. There are other pathalogical cases where you can't know everything about your program-
I was specifically thinking of "creating" when I said writing, not maintaining or extending. And yes, while it can still be large enough to have multiple people working on it, then the "you" is actually a group, and of course you will coordinate on what you're doing and can very easily confirm you all understand the boundaries of how the components you are creating interact.
Extending something you did not originally write may be quite different, of course.
2nd approach for when you ignore the first) you use a type that doesn't "own" the pointer, and have the FFI side that allocated it free it
3rd approach for when you find out you can't do the 2nd) hope that the "owned" type is actually generic over the allocator used, and make an allocator that doesn't free (or even better, calls the correct free over the FFI boundary). `allocator-api` is the effort in this direction.
It’s important to use the same allocator to free as you allocate with. You can’t assume that the same allocator is linked to both sides, so if you are allocating and then giving something to something else over FFI, you want to make sure you give them a way to call back into you to free.
Code gets reused unless you take steps to prevent it. Once upon a time, someone submitting a change to the self-driving car project unknowingly broke Google’s web search engine. Someone else decided that was insane, and that’s why Bazel rules now let you limit who can depend on you in a monorepo.
Early in my career I saw the aftermath of someone trying to deal with this problem. The solution they went with was to replace all allocations in the offending code with a custom allocator and then just throw the allocator away every so often
Plenty of cases fit the case, e.g. if it's some sort of server, you assign a separate allocator to each request and then purge it once the request is processed.
Next thing they do is to optimize the hell out of kernel process management things until it catches up with the overhead of calling a function in a GC language.
It’s definitely inelegant. But I wouldn’t say it’s entirely without merit.
If you’re writing a very simple “sed” like utility then you could probably get away without freeing.
That all said, the kind of problems were freeing becomes complicated is generally the kind of software that is more complex than your typical “sed” command. So the advise of not freeing doesn’t really address the domain where freeing is a risk.
That’s assuming each byte and/or line item is a new allocation rather than reusing already allocated memory for lines that have already been parsed.
Plus not every utility needs to consider large file usage. Eg a date/time formatter.
To be clear, I’m not advocating that people shouldn’t free. Just saying things aren’t always black and white.
A great example of this is one of Americas missile guidance systems famously has a memory leak. But it runs out of memory after its maximum range anyway. So the memory leak doesn’t affect the performance of the missile in any scenario.
Best criticism to this tongue-in-cheek solution yet. I’ll add my own more minor criticism: it will explode if the leaksaver struct allocation ever fails because it doesn’t check the result.
I always thought that was one reason the UNIX terminal login process (with `getty` and `exec` to shell, getty restarting when the shell exits) was the way it was.
I did a watch face on pebble once... the programming style was "uncommon" when you're hardcoding the ONE watch face that is currently rendering the screen. It "felt" very leaky and illegal... but it's a watch face with limited functionality, so... shrug?
Reminds me of the HFT shop that built in Java and simply turned the garbage collector off. Then when the market closed they would restart the process for the next day.
There was a certain Space Shuttle program that never returned from interrupt. At the end of its interrupt handler, it would just spin. The next interrupt would just add to the call stack.
What happens when it runs out of stack? It just resets the stack pointer back to the start.
Using warhead explosion as garbage collector might seem like a clever hack, but all it takes is an upgrade by a different team (say, adding a new engine, or longer-range sensors, or repurposing a surface-to-air missile for a surface-to-surface role), and suddenly your guidance software runs out of memory before it explodes, and your missile falls onto an an elementary school or hospital.
Related: the Ariane 5 rocket failed on its maiden flight because it reused code from its predecessor, which made an assumption that didn't apply to the higher-performance Ariane 5.
IIRC it was the higher acceleration value of Ariane5 that the old code wasn't expecting that caused the problem. Some fuzz testing could have worked, I suppose.
1. The module that failed actually was no longer in use for that part of the flight.
2. The acceleration for the part of the flight where it was in use was within the parameters.
3. Instead of just ignoring / clamping the out-of-bounds values, the no-longer-needed module sent a big error dump.
4. That error dump was sent to the next module in-line, which was expecting...I think numbers, but certainly not big textual error dumps. And so that failed as well.
5. I don't know if that second module was needed.
So had the module just (a) processed the incoming data normally or (b) clamped the values silently or (c) dropped the out-of-bounds values silently, the Ariane 5 would not have exploded.
Instead it did the "make it impossible to represent invalid states"-thing and exploded.
While I understand the appeal of that idea, I think it is overrated with any software that has to interact in some way shape or form with the real world, however indirectly. Because programmers tend to have a pretty limited understanding of what states are valid or invalid in the real world.
(See also: the "falsehoods programmers believe about XXX" series)
I had a friend who worked for one of the big Market Makers, and he told me that they would indeed turn the GC off, but what they'd do is just pre-allocate everything into bigass arrays before-hand, and have incrementers to simulate the "new" keyword. They might do this in something more or less like a threadlocal to avoid having to deal with locks or race conditions or anything like that.
Seen some audio engines that do this too. You can come up with an elegant non-blocking lock free allocator and pass updates from the user thread out to system and reserve the memory, or you can just up front allocate everything that's needed.
If it's a fixed function synth, sometimes just allocating everything up front makes more sense.
Kernel drivers, too. Especially the DMA buffers for devices that don’t support scatter gather: if you don’t allocate them at driver startup, you might not be able to find enough free contiguous regions later. Although maybe that’s not as big of a deal these days, since both Windows and Linux can relocate physical pages.
I remember seeing a Java game engine that included a container that you were supposed to dump unused objects into to prevent GC from running in the middle of a level. At the end of the level, you could empty the container, and let the GC run during the loading screen or whatever.
It was a deliberate design choice that was surprisingly close to what the original article describes.
This is like going back to malloc and free but using array indices rather than pointers, and each type has a fixed-size heap. We did it in one Microsoft service because the very first .NET releases had slow GCs.
it is somewhat common in garbage collected langages to fight the garrbage collector like that. Sure manual free probably adds up to more CPU time, but it is more spread out and thus not noticable (normally, real time still cannot allocate in the sensitive areas)
I’ve done something similar in one of our production services. There was a problem with extremely long GC pauses during Gen2 garbage collection (.NET uses a multigenerational GC design). Pauses could be many seconds long or more than a minute in extreme cases.
We found the underlying issue that caused memory pressure in the Gen2 region but the fix was to change some very fundamental aspects of the service and would need to have some significant refactoring. Since this was a legacy service (.net framework) that we were refactoring anyway to run in new .NET (5+), we decided to ignore the issue.
Instead we adjusted the GC to just never do the expensive Gen2 collections (GCLatencyMode) and moved the service to run on higher memory VMs. It would hit OOM every 3 days or so, so we just set instances to auto-restart once a day.
Then 1 year later we deployed the replacement for the legacy service and the problem was solved.
Actually that is not too far from reality. Data that will be allocated only once does not need be freed. You really only need to free memory that may increase iteratively. If the memory is not used frequently it will end up on swap without major implications. If it is used during the whole execution, it will only be freed when the program ends; there's difference if it's by a 'free' or by the OS; there's just no difference.
As an example, constant data that is allocated by GNU Nano is never freed. AFAIK, the same happens when you use GTK or QT; there were even tips on how to suppress valgrind warnings when using such libs.
The go guys knew this all along, because if I understood it correctly this is what happens when you disable the garbage collector. Voila, now you can claim your garbage collector is totally optional and your language fundamentally doesn't require a garbage collector.
[I love golang, and I think it's one of the best languages around. If only it had a truly optional garbage collector. But then again, it wouldn't be go I guess...]
TFA is clearly written in jest. The provided code is not supposed to be run in production, just to illustrate how easy is to trick a "leak detector" in your debugger. You just put all your mallocs in a list, and at the end of your program you can free them all.
Yet, the idea of not freeing some memory in your program is not entirely stupid. Unless your memory is allocated inside a loop of unpredictable length, it's not really necessary to ever free it. Worse: the call to "free" may even fall after your program has run successfully. Thus, avoiding the useless (but typical) freeing spree at the end of your program may make it more robust!
>In the rare cases where your program needs to be long-running, leaks may become a real problem.
Where did this idea come from? I have seen leaks where a program can consume all available memory in just a few seconds because the programmer (definitely not me...) forgot to free something in a function called millions of times.
A professor actually told us that freeing prior to exit was harmful, because you may spend all of that time resurrecting swapped pages for no real benefit.
Counterpoint is that debugging leaks is ~hopeless unless you have the ability to prune “intentional leaks” at exit
> debugging leaks is ~hopeless unless you have the ability to prune “intentional leaks” at exit
Not in general. It depends on your debugger. For example, valgrind distinguishes between harmless "visible leaks", memory blocks allocated from main or on global variables, and "true leaks" that you cannot free anymore. The first ones are given a simple warning by the leak detector, while the true leaks are actual errors.
I had to debug a program that did just that once, long ago, and the fix was to not free on exit. The program's behavior had been that it took ~20m and then one day it ran for hours and we never found out how long it would have taken. Fortunately it was a Tcl program, and the fix was to remove `unset`s of large hash tables before exiting.
... or unless someone decides to convert it into a daemon "because of that ticket" and then QA goes all "oh, ah, the routing is dead, the sshd is dead and the whole box is all but bricked, what could've possibly caused that".
Using an arena with an event loop and where persistent stuff is always copied has seemed really robust to me. When the event exits the handler resets the arena.
I bet this is what some people on my team would come up with if the ticket acceptance criteria said "program must not leak memory when checked with valgrind"
You can easily turn non-leaking program into a faster and leaking program, but the inverse direction is hard, so that criterion is entirely justified. Any optimization of this sort should be guarded against a compile time switch that simply swaps `free` with a placeholder. I think CPython did this for a long time, before the global initialization step was completely removed.
If only I had known this earlier in my career! They should really add a feature to make a hard copy of the big bucket list to minimize memory overhead, though.
I used to really struggle with memory leaks in C++, until smart pointers came along. I think the only leak I've investigated since then wound up being an actual bug in the MSVC standard library.
OS handles and such are still an issue, but you can wrap them too. Thank goodness for RAII, a fantastic reason to use C++ even if you're otherwise writing C-like code.
Obviously this is a joke, but the real message should be: if you can't manage your own memory, you should be using a language implementation with automatic memory management. If your code really needs to run "close to the metal", you really need to figure out how to manage your memory. In 2024, language implementations without automatic memory management should be reserved for applications that absolutely need them.
It obviously is a joke, but at the same time it's actually a viable approach for the right problem. Sometimes leaking is very tolerable and in terms of programmer time, very cheap!
Yes. Hence the word 'joke' I used. For short running programs, it's fine. The longer running programs you can often find a few places where 95% of the leaks happen and easily 'free' those leaving the last 5% to leak slowly. It depends. Just remember it's a valid solution sometimes.
The joke and the sometimes valid solution are different strategies.
The joke only reminds people of the valid solution. It feels like multiple people are giving the article the briefest skim and assuming the two are the same. Perhaps focusing on the words "optional to call free" and extrapolating off that without looking at the code or properly reading the rest of the text.
Funny, I think the lesson is the opposite: just because you have mechanisms that technically prevent memory leaks doesn't mean that you don't need to think about memory and its allocation/freeing. Or rather, memory leaks generally are not problem, unbounded memory consumption is, regardless if the consumption is technically due leak or some reference stashed somewhere.
> if you can't manage your own memory, you should be using a language implementation with automatic memory management.
I agree, and would expand the idea to all kinds of resources (files for example). Sadly not many languages have "automatic resource management". For example Go has automatic memory management, but if you read an HTTP request's body you have to remember to call req.Body.Close(). If you open a file you have to call file.Close(). If you launch a goroutine, you have to think about when it's going to end.
I'd like to know if some languages manage to automatically managed resources, and how they do it.
C# has Using. When the block ends the destructor is called.
VB Classic has reference counting and the terminate event is fired when the count goes to zero. So as long as you don't store the reference in a global the terminate event is guaranteed to run when it goes out of scope (so long as you don't have circular references of course).
C++ has Resource acquisition is initialization (RAII)
From the comments I get the impression that we have all worked with someone that would have considered this an actual solution and we had to talk them out of it.
Technically all pointers are accessible, but the issue remains that we have not logically accounted for unused resources and are wasting capacity. In this sense, memory leaks are possible in all languages. We could store every object into a global structure, and it will never be freed in any language. Thus, my Rust program balloons forever.
We have a counter that goes up by 1 every time you call malloc.
And down by one every time you call free.
And when the program quits, if the counter isn't zero, an email is fired off and a dollar gets sent from the developers bank account to the users bank account...
That's essentially how all leak detection tools work, minus the money part.
And it is not even always appropriate. It is common to allocate some memory for the entire lifetime of the process. For example, if your app is GUI-based and has a main window, there is no need to free the resources tied to the main window, because closing it means quitting the app which will cause all memory to be reclaimed by the OS. You can properly free your memory but it will only make quitting slower. Usually programmers only do that to satisfy leak detection tools, and if the overhead is significant, it may only be done in debug mode.
This has been PHP’s approach to memory management during its best decades. It’s a fantastic idea for short-running processes, which there should be more of anyway.
It's been a while since I used PHP, but this is something controlled by the server used. If you use Apache then it used to create a new process for each request, yes.
It depends on the SAPI. The php-fpm approach is a pool of processes that are reused for multiple requests. Virtually all of the state visible to plain PHP code is reset between requests, however some interpreter/runtime state is kept, e.g. in-memory bytecode cache, initialized extensions, some persistent network connections, etc.
In terms of memory management, PHP has both reference counting and garbage collection. Internally, an arena per request is used[0], so leaking memory over long periods of time is fairly rare and usually limited to native extensions.
Depends on the setup, but even when it doesn't, it mallocs a huge chunk of memory at the start, lets the script run, allocate away and never free anything. Then, when the script is done, it frees the whole chunk in one fell swoop.
So for some definition of “process” it holds regardless - it’s just not always a real OS process.
They added GC at some point to allow scripts to run longer (eg for websocket servers) but even then last I checked you had to manually enable it.
Although this is a joke, from a systems perspective this is a valid strategy - stateless lambda (FAAS) functions can use this strategy as the state is moved externally, and can take the load of say long running monitoring processes.
Since this seems to be some sort of joke i'm going to provide a "real" answer: write modular software where you can verify that each module is leak proof + use valgrind to actually track and verify you don't have links.
I have written medium to large projects using this approach and a leak-free program is not outside the realm of possible
This does help, but the hard part of memory leaks is leaks across module/api boundaries. You can write a memory safe module, I can write a memory safe program, but when my program starts calling your program there's nothing validating I'm maintaining the invariants required to avoid memory leaks.
Hopefully the invariants are documented, but relying on documentation to help programmers avoid memory leaks is far from foolproof.
Reminds me of a situation I encountered with Salesforce code many years ago. Salesforce had a requirement that their test platform had to cover some percentage of the lines of code in the Sandbox before you could deploy something to production.
Our Salesforce implementation consultants had put 500 lines of 'x = 1' into a piece of code to force it to deploy -- and these were people at a top consulting company with a very lucrative hourly rate.
No idea if SFDC still works this way or if this would fly today, this was back in the days when you had to use Flex to integrate anything with the UI.
There is a bug here... Clearly the author intended to cache the value of nextmalloc to avoid calling dlsym() on every malloc. The correct code should be:
174 comments
[ 3.4 ms ] story [ 241 ms ] threadThe function just indirects malloc with a wrapper so that all of the memory is traversable by the "bigbucket" structure. Memory is still leaked in the sense that any unfreed data will continue to consume heap memory and will still be inaccessible to the code unless the application does something with the "bigbucket" structure--which it can't safely do (see below).
There is no corresponding free() call, so data put into the "bigbucket" structure is never removed, even when the memory allocation is freed by the application. This, by definition, is a leak, which is ironic.
In an application that does a lot of allocations, the "bigbucket" structure could exhaust the heap even though there are zero memory leaks in the code. Consider the program:
At the end of the million iterations, there will be zero allocated memory, but the "bigbucket" structure will have a million entries (8MB of wasted heap space on a 64-bit computer). And every pointer to allocated memory in the "bigbucket" structure is pointing to a memory address previously freed so now points to a completely undefined location--possibly in the middle of some memory block allocated later.There are already tools to identify memory leaks, such as LeakSanitiser https://clang.llvm.org/docs/LeakSanitizer.html. Use those instead.
Clearly the author of TFA is aware of such tools, since the idea is to trick them.
You know this isn't serious right?
While the provided code may seem like an interesting approach, it's important to note that it introduces a number of issues and potential pitfalls. This code is an attempt to intercept the malloc function using the dlsym function from the dlfcn.h library and store every allocated pointer in a linked list called bigbucket. However, there are several problems with this solution:
A better approach to avoiding memory leaks is to adopt good programming practices, such as carefully managing memory allocation and deallocation, using automated tools like static analyzers and memory debuggers, and, when applicable, leveraging programming languages with automatic memory management (e.g., garbage collection in languages like Java or Python).Can you elaborate on that?
If it requires you to free correctly, then you can just use plain C and the smart pointer isn't accomplishing anything. If it doesn't require that, how does it work?
Traditionally memory is considered "leaked" if it is still allocated but nothing point to it; i.e. there's no way to navigate to the allocation anymore.
He has made a joke "solution" by simply permanently storing a second pointer to all allocations so that by this definition they never technically leak. You can still always navigate to ever allocation so no allocation has leaked.
Of course it's not a real solution because it doesn't actually change the memory characteristics of a leaky program; it just hides the leak. In other words the technical description of the leak above isn't really the thing we care about.
Seems like almost nobody here got that.
"If you don’t call free, memory usage will increase over time, but technically, it’s not a leak."
> It is [...] entirely optional to call free. If you don’t call free, memory usage will increase over time, but technically, it’s not a leak. As an optimization, you may choose to call free to reduce memory, but again, strictly optional.
This is beautiful! Unless your program is long-running, there's no point of ever calling free in your C programs. The system will free the memory for you when the program ends. Free-less C programming is an exhilarating experience that I recommend to anybody.
In the rare cases where your program needs to be long-running, leaks may become a real problem. In that case, it's better to write a long-running shell script that calls a pipeline of elegant free-less C programs.
If you write a C library, it is a good practice to leave the allocations to the library user, or at least provide a way to override the library's allocator. Allowing your user to write a free-less *program*.
Which means that if you're writing a program, you probably are also writing one or more libraries.
I also know someone who maintains some Clinton era encryption. that code is controlled who can know about it as obsecurity was all you were allowed. There are other pathalogical cases where you can't know everything about your program-
Extending something you did not originally write may be quite different, of course.
2nd approach for when you ignore the first) you use a type that doesn't "own" the pointer, and have the FFI side that allocated it free it
3rd approach for when you find out you can't do the 2nd) hope that the "owned" type is actually generic over the allocator used, and make an allocator that doesn't free (or even better, calls the correct free over the FFI boundary). `allocator-api` is the effort in this direction.
This is true irrespective of language.
If you’re writing a very simple “sed” like utility then you could probably get away without freeing.
That all said, the kind of problems were freeing becomes complicated is generally the kind of software that is more complex than your typical “sed” command. So the advise of not freeing doesn’t really address the domain where freeing is a risk.
Programmer never free’d, and only ran it on files that were like some hundred megabytes big.
Some poor schmuck tries to process a 15 GB file on his 8 GB RAM machine, and after running for hours it crashes because of the lack of free :(
Plus not every utility needs to consider large file usage. Eg a date/time formatter.
To be clear, I’m not advocating that people shouldn’t free. Just saying things aren’t always black and white.
A great example of this is one of Americas missile guidance systems famously has a memory leak. But it runs out of memory after its maximum range anyway. So the memory leak doesn’t affect the performance of the missile in any scenario.
Technology is often nuanced.
That does it. I am not going to use it.
But I remember the first time I saw such a program which never freed anything: jitterbug, the simple bug tracker which ran as a CGI script.
It indeed allows a very simple style!
Meanwhile, use ccan/tal (https://github.com/rustyrussell/ccan/blob/master/ccan/tal/_i...) and be happy :)
https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98...
What happens when it runs out of stack? It just resets the stack pointer back to the start.
https://en.wikipedia.org/wiki/Ariane_flight_V88
1. The module that failed actually was no longer in use for that part of the flight.
2. The acceleration for the part of the flight where it was in use was within the parameters.
3. Instead of just ignoring / clamping the out-of-bounds values, the no-longer-needed module sent a big error dump.
4. That error dump was sent to the next module in-line, which was expecting...I think numbers, but certainly not big textual error dumps. And so that failed as well.
5. I don't know if that second module was needed.
So had the module just (a) processed the incoming data normally or (b) clamped the values silently or (c) dropped the out-of-bounds values silently, the Ariane 5 would not have exploded.
Instead it did the "make it impossible to represent invalid states"-thing and exploded.
While I understand the appeal of that idea, I think it is overrated with any software that has to interact in some way shape or form with the real world, however indirectly. Because programmers tend to have a pretty limited understanding of what states are valid or invalid in the real world.
(See also: the "falsehoods programmers believe about XXX" series)
Then you just say terrorists were hiding inside…
If it's a fixed function synth, sometimes just allocating everything up front makes more sense.
It was a deliberate design choice that was surprisingly close to what the original article describes.
We found the underlying issue that caused memory pressure in the Gen2 region but the fix was to change some very fundamental aspects of the service and would need to have some significant refactoring. Since this was a legacy service (.net framework) that we were refactoring anyway to run in new .NET (5+), we decided to ignore the issue.
Instead we adjusted the GC to just never do the expensive Gen2 collections (GCLatencyMode) and moved the service to run on higher memory VMs. It would hit OOM every 3 days or so, so we just set instances to auto-restart once a day.
Then 1 year later we deployed the replacement for the legacy service and the problem was solved.
As an example, constant data that is allocated by GNU Nano is never freed. AFAIK, the same happens when you use GTK or QT; there were even tips on how to suppress valgrind warnings when using such libs.
[I love golang, and I think it's one of the best languages around. If only it had a truly optional garbage collector. But then again, it wouldn't be go I guess...]
In short: If it works until it crashes, it doesn't work.
Yet, the idea of not freeing some memory in your program is not entirely stupid. Unless your memory is allocated inside a loop of unpredictable length, it's not really necessary to ever free it. Worse: the call to "free" may even fall after your program has run successfully. Thus, avoiding the useless (but typical) freeing spree at the end of your program may make it more robust!
Where did this idea come from? I have seen leaks where a program can consume all available memory in just a few seconds because the programmer (definitely not me...) forgot to free something in a function called millions of times.
Counterpoint is that debugging leaks is ~hopeless unless you have the ability to prune “intentional leaks” at exit
Not in general. It depends on your debugger. For example, valgrind distinguishes between harmless "visible leaks", memory blocks allocated from main or on global variables, and "true leaks" that you cannot free anymore. The first ones are given a simple warning by the leak detector, while the true leaks are actual errors.
... or unless someone decides to convert it into a daemon "because of that ticket" and then QA goes all "oh, ah, the routing is dead, the sshd is dead and the whole box is all but bricked, what could've possibly caused that".
1: https://en.wikipedia.org/wiki/Goodhart's_law
OS handles and such are still an issue, but you can wrap them too. Thank goodness for RAII, a fantastic reason to use C++ even if you're otherwise writing C-like code.
Define "it" here.
Because "just don't free" is pretty different from what's in the post!
The joke and the sometimes valid solution are different strategies.
The joke only reminds people of the valid solution. It feels like multiple people are giving the article the briefest skim and assuming the two are the same. Perhaps focusing on the words "optional to call free" and extrapolating off that without looking at the code or properly reading the rest of the text.
Unless I'm badly missing something?
I agree, and would expand the idea to all kinds of resources (files for example). Sadly not many languages have "automatic resource management". For example Go has automatic memory management, but if you read an HTTP request's body you have to remember to call req.Body.Close(). If you open a file you have to call file.Close(). If you launch a goroutine, you have to think about when it's going to end.
I'd like to know if some languages manage to automatically managed resources, and how they do it.
VB Classic has reference counting and the terminate event is fired when the count goes to zero. So as long as you don't store the reference in a global the terminate event is guaranteed to run when it goes out of scope (so long as you don't have circular references of course).
C++ has Resource acquisition is initialization (RAII)
Technically all pointers are accessible, but the issue remains that we have not logically accounted for unused resources and are wasting capacity. In this sense, memory leaks are possible in all languages. We could store every object into a global structure, and it will never be freed in any language. Thus, my Rust program balloons forever.
I get the feeling that this might have been response to recent "memory leak" thread(s): https://news.ycombinator.com/item?id=39041520
We have a counter that goes up by 1 every time you call malloc.
And down by one every time you call free.
And when the program quits, if the counter isn't zero, an email is fired off and a dollar gets sent from the developers bank account to the users bank account...
And it is not even always appropriate. It is common to allocate some memory for the entire lifetime of the process. For example, if your app is GUI-based and has a main window, there is no need to free the resources tied to the main window, because closing it means quitting the app which will cause all memory to be reclaimed by the OS. You can properly free your memory but it will only make quitting slower. Usually programmers only do that to satisfy leak detection tools, and if the overhead is significant, it may only be done in debug mode.
Isn’t that close to how ARC (automatic reference counting) works?
In terms of memory management, PHP has both reference counting and garbage collection. Internally, an arena per request is used[0], so leaking memory over long periods of time is fairly rare and usually limited to native extensions.
[0]: https://www.phpinternalsbook.com/php7/memory_management/zend...
So for some definition of “process” it holds regardless - it’s just not always a real OS process.
They added GC at some point to allow scripts to run longer (eg for websocket servers) but even then last I checked you had to manually enable it.
It used to be a quite common approach in UNIX server programming.
Memory hoarding. I hate it but I love it.
I have written medium to large projects using this approach and a leak-free program is not outside the realm of possible
Hopefully the invariants are documented, but relying on documentation to help programmers avoid memory leaks is far from foolproof.
Our Salesforce implementation consultants had put 500 lines of 'x = 1' into a piece of code to force it to deploy -- and these were people at a top consulting company with a very lucrative hourly rate.
No idea if SFDC still works this way or if this would fly today, this was back in the days when you had to use Flex to integrate anything with the UI.
the other bug of course is that it's not thread safe
After running your test suite, it kicks into action and deletes all the lines of your code that were never executed.
[1] https://fgiesen.wordpress.com/2012/04/08/metaprogramming-for...