There is no reason for using native extensions in scripting languages like Ruby when program load is pretty low (especially when they are used for performance reason, not binding).
Great insight, though. Author described this experience as it was a great venture... in hindsight I suppose :-)
2. With over 10M downloads, I assume at least some of them spend large percentages of time using it.
3. It's Markdown; replicating the idiosyncrasies of SmartyPants (https://daringfireball.net/projects/smartypants/) in a different language is not something many people are going to want to do.
I'm not saying that people should or should not use Redcarpet; just that it's unlikely that anyone engaged in "premature optimization" here.
I'd say it's appropriate for anything in the critical path - talking to databases being the only major thing that comes to mind. I agree that a native extension for markdown parsing is way overkill.
Stick with pure-language extensions (they're so much easier to work with!) until you can measure and quantify the cost and benefit of switching.
The reason for the binding can be that no-one can be assed to reimplement the functionality. It is not as if anyone expect the scripting languages to implement their own versions of openssl.
Your criticism really applies to the gem authors, which to me, does not make sense. If I, as a library developer, want my library to be used on the critical path in heavy-load situations, and the library is for a language like Ruby, I'm going to figure out which part of my library are the performance critical parts, and implement them in a native language.
You would think that you (in this case, "you" == "average library author in the ruby ecosystem") would, but you won't. I'm fairly confident the problem is similar, but likely worse in the javascript ecosystem.
Necessity is the mother of invention, and when you're the first person to notice a memory leak, you're on your own.
Both ecosystems tout the low barrier to entry as a great benefit, but memory leaks, mostly benign inefficiencies, and poor algorithmic efficiency becomes at least par for the course if not a crippling liability when reality hits your application like a freight train (or say a 2 order-of-magnitude spike).
C is not really my area of expertise, but doesn't xfree have alternatives? like free, freeif, etc? I'd much rather have to change that later in one place than several (if there end up being more calls to xfree in the code) if at some point the implementation needs to change again.
So a from a quick googling it looks like xfree()[1] is a ruby specific thing, and should be used internally for everything allocated by ruby. It also looks like it's got the same signature as the function created for this so that should work there too. [2]
Okay, interesting. I guess I've seen it before, so I didn't think it was Ruby-specific, but I also haven't had to do that kind of programming in a very super long time.
Replacing malloc/calloc/free etc an application-specific xmalloc/xcalloc/xfree is seemingly a fairly common pattern. There's one example in libiberty[0] (-liberty, get it?) where, for instance, the return value of malloc is checked for NULL and the program terminates if so.
It's fairly common to define malloc and free wrappers called xmalloc and xfree. The usual idiomatic thing to do is to have xmalloc check the return value of malloc and terminate the program if it ever fails, though this is traditionally done in application code rather than a library. I doubt libc would ever add an xmalloc function because it'd break all the code that defined its own xmalloc.
The shortest code you can get away with is not always the best. If the parameter is a structure (which I think it is here, behind the void pointer), it's a good idea to give it its own free function in case later you add additional struct members that will need freeing/cleanup in the same place. (Bonus points if it appears in roughly the same place this thing is malloc'd)
I would have been even more explicit and added a cast to the expected structure pointer type. Entirely useless, except to the human reader.
Call it an overreaction to a past project in which a predecessor scattered free calls over several uses, then I had to add fields.
I don't mind adding 1 useless function per complex type if it saves me those headaches even a small minority of the time, or for the next maintainer. Opinions may differ, but that's me.
I had a similar issue that I was tracing last week that did end up being my Ruby code...and it turns out I was modifying a constant like in the example.
What a fun read! I've been learning more about Ruby's GC since 2.1 and this got me looking even deeper -- definitely picked up a couple of new tricks/tools from this. Thank you be9!
Yeah, it's one of those rare Ruby gotchas. If you assign some kind of a data structure to a constant, you'll get a warning if you try to reassign the constant, but the data structure itself will silently mutable. To disallow this behavior, simply freeze the data structure. If you have nested data structures, you must freeze deeply.
I tried to add ASan to the travis config for this project but I couldn't quite figure out how to change CFLAGS and/or CC. Never used ruby but interwebs hinted that the bundle config/install commands might accept "--cc" and "--with-cflags" commands. It's ignored when I tried it though [1].
PHP has garbage collection now, so it's a lot harder to cause a memory leak. Before PHP 5.3.0, cyclical references were one of the biggest causes of memory leaks.
However, you can still create memory leaks just like you could in Ruby - dynamic allocation in a c extension that isn't freed up (like in this story), or creating objects that can't be garbage collected (like a slowing expanding array in a global variable).
PHP extensions in C are pretty rare since it's so hard to distribute them (outside a few exceptions that exist in common package management systems), so in that sense, I guess?
But no, a PHP extension in C can have a memory leak like any other C code. It's not at all uncommon actually, since there are fewer consequences; CGI mode PHP cleans up after every request, FastCGI mode PHP by default restarts workers after a number of requests because of the class of effects (unwanted state) that includes memory leaks.
shameless plug: I've made malloc hooking preload library for hunting such bugs, that automatizes most of the described work - https://github.com/samsk/log-malloc2
This looks really, really cool. You should download the bugged version of this gem and make a screencast showing how your tool makes this process easier.
Interesting post, but I did want to chime in quickly and say that it's pretty absurd to only have 1GB for a web application in 2016, even for a small one. This is why I've been dismissive of using Heroku for my projects, even though I run a lean stack. They have higher RAM options, but they are incredibly expensive. For the cost of a 14GB Heroku Dyno, I can buy a dedicated server off ebay with 32GB ECC every single month. I don't use all of that RAM, but it's nice to have some extra RAM to toss around when you need to in a pinch.
I get that it means you have to "run a server", and insert arguments for expensive cloud providers vs DIY servers here but I don't think it's any less crazy than being forced to chase a GC white whale for two weeks on a tiny memory leak to avoid a huge rate hike on your hosting bill.
On an aside, I've only had one problematic memory leak with ruby ever (the infamously leaky RMagick), I threw this in every time I used the lib and it solved it for me:
GC.start(full_mark: true, immediate_sweep: true)
Bit of a performance hit, but not enough to cause a problem for my use case.
Though you got voted down, I hadn't really thought that my comment would be interpreted the way you did. Thanks for pointing it out. I will try to be a little more clear in the future.
Our main proxies (that do the majority of the heavy lifting over here) run nginx and use, what, 50MB tops? If even? I throw them on $5/mo VPS instances and they just blast out IO without any problems.
But nginx was carefully written in C over the course of many years, with a general goal of hauling IO and static data around from other systems. Which is all you really need for web pages, in the end. Solved one thing and holy crap they did it well.
It's the business logic side of the web development world where things get nasty. The tradeoff is you get a giant chest of libraries for solving any stupid problem you want. I didn't have to for example figure out how to do IDN encoding for domain validation. You throw enough of that crap together in a single process and things start to get nasty. But how long would I have spent carefully writing all of it in C? Let's just say I'd be out looking for another job.
Worth the tradeoff, IMHO, but I think with projects like Crystal (https://crystal-lang.org) in the future we will discover we can get pretty close to both in the end. That combined with Moore's law and I think we'll be good to go.
Assuming, of course, that cloud providers don't artificially restrict available RAM through economic constraint (know the true costs - demand better!).
I realised (thanks to the other poster) that my comment may have seemed dismissive, which was not my intent. Just marvelling at it myself :-) But I do wonder about this kind of stuff. I have been wrestling with some legacy systems chewing up gobs and gobs of memory lately. Then wrestling with my own code that was chewing up gobs and gobs of memory :-)
It's pretty easy to make mistakes because we don't really get punished, as you say. But I wonder if we might be doing ourselves a disservice by not restricting ourselves. Generally speaking, better code is better and while it often feels slower to write, my experience has been that it doesn't always work out that way.
I might try working in a memory restricted container to see if it improves my code...
Another thing to look into is to write things in Rust. It will be as fast as C code, but a little nicer to work with. Right now the problem with that is that you don't have the nice libraries for whatever you're doing.
But if people actually wrote fast Rust libraries for everything, they're as easy to embed into your apps as C.
If you don't want to wait for Crystal to mature there's a long line of high-level languages with similar performance characteristics - check out OCaml or Haskell, or perhaps F# or Scala (if it's throughput you need rather than fast startup).
Interpreted languages can have pretty high memory overhead. Let's say we have a page in python or php or ruby or whatever, and it needs to fetch 1,000 rows from a database (100 things here, 20 things there, whatever). Each row in the database can be turned into an object several times, one from the model, one from the controller, one for the template/view. On top of that, each object is a composite key/value object of let's make up 10 fields, so that's 20 PyObjects or whatever. In Python an empty PyDict uses up 280 bytes. sys.getsizeof({}). so 1,000321*280 = 17.64MB. That's just a back of the envelope theoretical minimum, include bad coding (for another few multiples), pages that require more data (analytics/stats/reporting pages), and concurrency and you can get up there!
Needing to fetch 1k rows from a database is different from needing to keep all that data in memory, however. Several years ago I had to return 1M rows from a database to the client with a PHP script, and it needed less than the 16MB (yes, sixteen megabytes) that the environment of the time was configured with as a memory limit. 1G rows would not have consumed any more memory either, because at any one time the script never needed to keep more than around a dozen cells in memory, each not more than a few KB. Read a cell from the database, process it, output the result and move onto the next one. Essentially O(1) space regardless of how much data there is.
Or are you saying that web applications are now written to retain all that data in memory despite the fact that it's not actually needed, and such stupid algorithms are actually considered acceptable? I think that's pretty ridiculous. The whole idea of database concepts like cursors is that you shouldn't be required to hold the entire resultset in memory, because it might not fit.
This should be even easier these days. Async I/O platforms like Node.js mean you don't have to worry about a process sticking around while it trickles data to a client. And web frameworks make it easier to deliver datasets asynchronously as JSON to the browser. Generally, there's less excuse to buffer everything server-side, e.g. to generate a full page.
I think the reliance on large frameworks and external libraries is the problem here because they don't generally compose in a way that permits optimizing data flow. That and the usual reduction in average skill required to work in this area.[1] People will spend an eternity arguing over which is the faster hash table algorithm, but have no eye or concern for optimizing the large details even though the relative improvements in performance and scale could be orders of magnitude better and conceptually much easier to implement.
This is why higher-level languages need stackful coroutines, not just promises or async/await. Stackful coroutines make it much easier to implement and compose space-efficient iterators, for example, in the context of uncooperative libraries, and to do so performantly. People are so obsessed with async I/O they've forgotten why it's so darned useful and how you want to best leverage it. They keep going down the rabbit hole of callbacks and actors, but the fact of the matter is that those things are too complex to use pervasively in the pipeline of individual requests. They're the modern equivalent of gotos--easy to use and abuse, but not what you want to use as an abstraction for tying together different pieces of code because they have poor locality of context, require leaking too much detail at the interface boundaries, are more difficult to refactor, and generally require a high cognitive burden. We want to be lowering cognitive load and other constraints!
[1] To be fair, I was one of those less-skilled newcomers back in the late 90s, abusing the newer technologies and frustrating the grey beards.
> Or are you saying that web applications are now written to retain all that data in memory despite the fact that it's not actually needed
Even the standard PHP MySQL API (mysqli) defaults to MYSQLI_STORE_RESULT which buffers all the results in memory. You have to specify MYSQLI_USE_RESULT to get streaming results.
It's about return on time invested. As an IoT company we got super excited after we decided to bump our chip from one that has 64kB instead of 32kB. However, I wouldn't blink an eye if we would need a few more GB for our cloud servers. Of course if it's just a static website I run it of Github pages.
For static sites, surely just load all of the content into memory and serve it based on key/index? eg. load into a C++ map of <const string,const string>, where first is URL and second is content. Even lookups would be fast (and would negate the need to hit the disk too).
Out of curiosity, albiet slightly off topic can you share an example server that you'd buy? I'm totally in the aws kool aid soup and would love to see some reasonable examples of how to buy server alternatives
I work on the project discussed in the article. We're running puma in clustered mode, with multiple workers per dyno and multiple threads per worker. 1GB is plenty in this context - each worker is pretty lightweight. At the time this was written we were running ~10 2x dynos, but we've since switched to 3 performance-m dynos with more puma threads/workers.
> I don't think it's any less crazy than being forced to chase a GC white whale for two weeks on a tiny memory leak to avoid a huge rate hike on your hosting bill.
The main issue we were facing wasn't the hosting bill, but high (for us) traffic, that was leaking memory on every request. Under low traffic it was unnoticeable, but at peak load the leak would cause dyno memory to max out pretty quickly, which would cause timeouts and increase the traffic to other dynos, causing cascading failures. Having more memory available would definitely have made things a lot easier, but we would have run out eventually either way.
My favorite Ruby memory leak was in a text processing app that was extracting via regex short strings (< 50 chars) out of 10MB+ text files. The short strings were apparently substring-like objects that held on the original 10MB text and indexed into it at a particular offset. Solution? Return the short_string + "" and then the GC did its thing.
> it's pretty absurd to only have 1GB for a web application in 2016, even for a small one
what is pretty absurd is the amount of memory a ruby web application uses. 1GB should be enough in 2016 to serve a small web app. Ruby might be a pleasant language to write it was never designed to be efficient memory wise. And frameworks like Rails that have 0 concerns for real performances make things even worse. That's why people are ditching Ruby for statically typed alternatives, constantly. It's just not a good language for the task in 2016.
Wouldn't it be useful if the community tested EVERY gem to see if it leaked memory, perhaps each developer stepping up to test one, in an app that just exercised the gem in question?
To test all of the ones we use individually is prohibitive. But thousands of people could do test one.
Amazing stuff. I took time to understand this article by going through each part in detail. Learnt quite some stuff. Thank you for this incredible story!
Is there a simple way to notice your own subtle mistakes, when writing in a foreign language? The missing articles in texts from Slavic people bug me more than they should and I wonder what kind of (stylistic) errors I myself make.
The English grammar is mostly just simplified German, but there are some constructs, which I know of, but do not naturally use (e.g. "your every …", "to name but two") - they somehow seem twisted to me. Another one are compound words, I have given up on distinguishing between spaces, hypens and actual compounds and just use spaces almost everywhere.
I had similar experience of finding memory leak in Python. It's a 64 G server, but we load everything from database to memory. I didn't success.
This experience made me question myself: is it wrong to create a lot of objects in memory and let them reference each other, and hope Garbage Collector to magically work?
If instead of object, we just put data into memory more organized as a (column) database, then we don't need GC anymore.
OOP made programming easier, by modeling real world as object. At the same time, it made memory management harder, since it's not nature to the hardware's memory model: a linear array of memory cells.
I wish in future, there is a place for this paradigm of in memory database model of programming.
It's called "data-oriented design". Look into the gaming industry, where there's a lot of anti-OOP sentiment. They're primarily coming at it from the standpoint of optimizing memory access for cache lines, but there are side benefits that come from it around reducing memory allocation and usage, and avoiding the cost of abstractions that come with hidden inefficiencies.
70 comments
[ 3.4 ms ] story [ 136 ms ] thread"Update on 2015/09/29. Redcarpet fix has been released."
Great insight, though. Author described this experience as it was a great venture... in hindsight I suppose :-)
The leak was in a gem.
2. With over 10M downloads, I assume at least some of them spend large percentages of time using it.
3. It's Markdown; replicating the idiosyncrasies of SmartyPants (https://daringfireball.net/projects/smartypants/) in a different language is not something many people are going to want to do.
I'm not saying that people should or should not use Redcarpet; just that it's unlikely that anyone engaged in "premature optimization" here.
Stick with pure-language extensions (they're so much easier to work with!) until you can measure and quantify the cost and benefit of switching.
Necessity is the mother of invention, and when you're the first person to notice a memory leak, you're on your own.
Both ecosystems tout the low barrier to entry as a great benefit, but memory leaks, mostly benign inefficiencies, and poor algorithmic efficiency becomes at least par for the course if not a crippling liability when reality hits your application like a freight train (or say a 2 order-of-magnitude spike).
[1] http://clalance.blogspot.com/2011/01/writing-ruby-extensions... [2] http://inferior-products.com/docs/userdocs/ruby19/html/d8/d1...
Publib then? I'm finding this when I search: http://man.cx/xfree(3) http://man.cx/publib(3)
Also, are you sure? I see xmalloc in glibc all over the internet...
#define xmalloc ruby_xmalloc #define xfree ruby_xfree
[0]: http://www.delorie.com/gnu/docs/gcc/libiberty_5.html
I would have been even more explicit and added a cast to the expected structure pointer type. Entirely useless, except to the human reader.
I'd say use a separate free function when it needs one, which is not the case yet (and might never be.)
I don't mind adding 1 useless function per complex type if it saves me those headaches even a small minority of the time, or for the next maintainer. Opinions may differ, but that's me.
I had a similar issue that I was tracing last week that did end up being my Ruby code...and it turns out I was modifying a constant like in the example.
What a fun read! I've been learning more about Ruby's GC since 2.1 and this got me looking even deeper -- definitely picked up a couple of new tricks/tools from this. Thank you be9!
I tried to add ASan to the travis config for this project but I couldn't quite figure out how to change CFLAGS and/or CC. Never used ruby but interwebs hinted that the bundle config/install commands might accept "--cc" and "--with-cflags" commands. It's ignored when I tried it though [1].
[1] https://travis-ci.org/androm3da/redcarpet/jobs/159711520
However, you can still create memory leaks just like you could in Ruby - dynamic allocation in a c extension that isn't freed up (like in this story), or creating objects that can't be garbage collected (like a slowing expanding array in a global variable).
But no, a PHP extension in C can have a memory leak like any other C code. It's not at all uncommon actually, since there are fewer consequences; CGI mode PHP cleans up after every request, FastCGI mode PHP by default restarts workers after a number of requests because of the class of effects (unwanted state) that includes memory leaks.
Ideally much less tooling would have been needed.
I get that it means you have to "run a server", and insert arguments for expensive cloud providers vs DIY servers here but I don't think it's any less crazy than being forced to chase a GC white whale for two weeks on a tiny memory leak to avoid a huge rate hike on your hosting bill.
On an aside, I've only had one problematic memory leak with ruby ever (the infamously leaky RMagick), I threw this in every time I used the lib and it solved it for me:
Bit of a performance hit, but not enough to cause a problem for my use case.While I don't for a minute doubt what you say, as an old guy, the absurdity of 1GB being insufficient for serving up web pages hits me pretty hard.
But nginx was carefully written in C over the course of many years, with a general goal of hauling IO and static data around from other systems. Which is all you really need for web pages, in the end. Solved one thing and holy crap they did it well.
It's the business logic side of the web development world where things get nasty. The tradeoff is you get a giant chest of libraries for solving any stupid problem you want. I didn't have to for example figure out how to do IDN encoding for domain validation. You throw enough of that crap together in a single process and things start to get nasty. But how long would I have spent carefully writing all of it in C? Let's just say I'd be out looking for another job.
Worth the tradeoff, IMHO, but I think with projects like Crystal (https://crystal-lang.org) in the future we will discover we can get pretty close to both in the end. That combined with Moore's law and I think we'll be good to go.
Assuming, of course, that cloud providers don't artificially restrict available RAM through economic constraint (know the true costs - demand better!).
It's pretty easy to make mistakes because we don't really get punished, as you say. But I wonder if we might be doing ourselves a disservice by not restricting ourselves. Generally speaking, better code is better and while it often feels slower to write, my experience has been that it doesn't always work out that way.
I might try working in a memory restricted container to see if it improves my code...
But if people actually wrote fast Rust libraries for everything, they're as easy to embed into your apps as C.
So it is mostly an issue of sloppy coding and bad GC implementations than anything else.
Or are you saying that web applications are now written to retain all that data in memory despite the fact that it's not actually needed, and such stupid algorithms are actually considered acceptable? I think that's pretty ridiculous. The whole idea of database concepts like cursors is that you shouldn't be required to hold the entire resultset in memory, because it might not fit.
I think the reliance on large frameworks and external libraries is the problem here because they don't generally compose in a way that permits optimizing data flow. That and the usual reduction in average skill required to work in this area.[1] People will spend an eternity arguing over which is the faster hash table algorithm, but have no eye or concern for optimizing the large details even though the relative improvements in performance and scale could be orders of magnitude better and conceptually much easier to implement.
This is why higher-level languages need stackful coroutines, not just promises or async/await. Stackful coroutines make it much easier to implement and compose space-efficient iterators, for example, in the context of uncooperative libraries, and to do so performantly. People are so obsessed with async I/O they've forgotten why it's so darned useful and how you want to best leverage it. They keep going down the rabbit hole of callbacks and actors, but the fact of the matter is that those things are too complex to use pervasively in the pipeline of individual requests. They're the modern equivalent of gotos--easy to use and abuse, but not what you want to use as an abstraction for tying together different pieces of code because they have poor locality of context, require leaking too much detail at the interface boundaries, are more difficult to refactor, and generally require a high cognitive burden. We want to be lowering cognitive load and other constraints!
[1] To be fair, I was one of those less-skilled newcomers back in the late 90s, abusing the newer technologies and frustrating the grey beards.
Even the standard PHP MySQL API (mysqli) defaults to MYSQLI_STORE_RESULT which buffers all the results in memory. You have to specify MYSQLI_USE_RESULT to get streaming results.
Your only option in those case with PHP is to truncate the response which IMO is the ridiculous choice here...
For dynamic stuff, adjust accordingly.
You can't really do faster than that.
That really depends on a lot of things. For a simple web app with a couple of users, 1GB should be overkill.
> I don't think it's any less crazy than being forced to chase a GC white whale for two weeks on a tiny memory leak to avoid a huge rate hike on your hosting bill.
The main issue we were facing wasn't the hosting bill, but high (for us) traffic, that was leaking memory on every request. Under low traffic it was unnoticeable, but at peak load the leak would cause dyno memory to max out pretty quickly, which would cause timeouts and increase the traffic to other dynos, causing cascading failures. Having more memory available would definitely have made things a lot easier, but we would have run out eventually either way.
But if the 512MB dyno isn't enough for any of the stacks I use, including Node, then I'm doing something very wrong (or weird).
See: http://www.littlelines.com/blog/2014/07/08/elixir-vs-ruby-sh...
tl:dr;
Ruby/Rails: 235.37MB
Elixir/Phoenix: 34.69MB
what is pretty absurd is the amount of memory a ruby web application uses. 1GB should be enough in 2016 to serve a small web app. Ruby might be a pleasant language to write it was never designed to be efficient memory wise. And frameworks like Rails that have 0 concerns for real performances make things even worse. That's why people are ditching Ruby for statically typed alternatives, constantly. It's just not a good language for the task in 2016.
Love it when reading HN pays off immediately like that :)
To test all of the ones we use individually is prohibitive. But thousands of people could do test one.
The English grammar is mostly just simplified German, but there are some constructs, which I know of, but do not naturally use (e.g. "your every …", "to name but two") - they somehow seem twisted to me. Another one are compound words, I have given up on distinguishing between spaces, hypens and actual compounds and just use spaces almost everywhere.
This experience made me question myself: is it wrong to create a lot of objects in memory and let them reference each other, and hope Garbage Collector to magically work?
If instead of object, we just put data into memory more organized as a (column) database, then we don't need GC anymore.
OOP made programming easier, by modeling real world as object. At the same time, it made memory management harder, since it's not nature to the hardware's memory model: a linear array of memory cells.
I wish in future, there is a place for this paradigm of in memory database model of programming.