This article sure is timely. I've been working on a ruby project for a while now and I'm just about to get it live. The key for me was two things: 1. Forking. Sounds silly, but man oh man did this increase performance. 2. Relying on a unix tool instead of going with a ruby gem.
Now I admit that I'm locking myself into a unix environment, but I really don't have the time or skills to make this project OS agnostic. If it is an eventual success, I may have to pay off some of that technical debt, but that is a problem I wouldn't mind having.
Now I admit that I'm locking myself into a unix environment,
Unless you are making an application for users to install, I don't see how this will be problem. Effectively, the only significant OS you are locking yourself out of is Windows and Unix you can get for free and install anywhere.
I developed a Ruby application a few years ago with a similar structure on OS X to be deployed on Ubuntu servers. All the forking and child process signaling worked like a charm in both OS's.
So yeah... you're locking yourself out of Windows. Consider it a smart move :)
Disclaimer: I am a web developer, so these criticisms include myself.
I feel like so many folks (me included) in the web development community are constantly re-learning new (old) techniques that we all really should have realized long before now.
Though I don't think I would change a thing as I've learned a lot over the past decade that I probably never would have if it not for this community.
Sometimes I just get a sense of deja vú, like none of this is new...
This whole "Unicorn" business for example. I read about this last week, and thought to myself, "Ok... hasn't Apache been doing this with 2.0 for years?"
So isn't this just re-implementing the wheel... yet again, but in Ruby?
Someone once gave me an incredibly apt analogy for this phenomenon: technology is like a helix. It repeats itself but at the same time increasingly gets better.
See: cloud computing, AJAX, NoSQL, and all the other things that are purported to be new ideas but are really re-hashes of 30-40 year old ways of thinking with contemporary tools.
Of course it is reinventing the wheel. And people will continue to post ebullient posts about it until they start studying formal cs theory and stop mistaking the implementations details ("java gets closures! c# gets lambda expressions! new web stack for perl!") as the interesting bits.
It's just a given that the majority of the ideas posted to programming blogs are just old ideas in new contexts.
I am not sure why this has gotten so much attention lately? I don't see anything out of extraordinary to drop to low level to accomplish something that is unusually complex or requires high performance...
Also I am not sure what is with this new tendency to call everything unicorns and ponies? It's intellectually degrading, and depressing to read such articles.
I think "[old school unix] has gotten so much attention lately" because of accidents in the timing of computer science culture and its victories.
The Linux / BSD driven revival of Unix happened, un-fortuitously, just when object-orientation, graphics, UML, patterns, etc, were hitting their peak, largely because they were super successful in dealing with problems in graphic interface design. So these oo approaches became dominant in curriculums and paradigms far beyond their applicability, and no one bothered to study the classics of the previous generation like APUE etc.
Smalltalk and C++ begat Java, begat Ruby/ Python, and now we have a whole bunch of server-side programmers who know how to inherit from multiple base classes but don't know what fork() really does; when these young people (ahem) discover how well their grandfather's Unix addresses many of their current server-side problems, well, they are thrilled.
Let's just say it's the same thing as a twenty something discovering Sly and the Family Stone vinyl in the basement... Yeah, it rocks.
You shouldn't have to call fork() yourself. Most languages have either language level or library level solutions that abstract that away for you. You want to abstract it away because very soon you'll find yourself wanting to manage the forked processes in much the same way these other facilities do it for you. Even when I'm working in C, I almost always use Pthreads instead of calling fork() directly. The only time I call fork() is when it's soon followed by an execve().
The same goes for file locking. Using files for synchronization is primitive, error prone and slow. As soon as you find yourself managing the lock files to deal with crashed programs ("stale locks" as the author calls them), you might as well trade it in for better abstractions.
Perhaps the most valuable part of implementing this kind of stuff by hand, using only systems services, is the appreciation you gain from having access to the same functionality in your language or from well designed libraries.
His contrived example of "atomically creating a file" is kind of strange. If you would want to see which of several processes wins at creating a file, just use
if(-1 != (fd=open("filename",0666,O_CREAT|O_EXCL)){ .. }
which is guaranteed to suceed only for one program.
Or you'd open it without O_EXCL and try locking it. Creating several file and atomically renaming it "over" the old file yields you nothing, because whenver you try to access it afterwards, you have no guarantee that it's not being overwritten just afterwards by another one of the processes.
14 comments
[ 4.9 ms ] story [ 31.3 ms ] threadNow I admit that I'm locking myself into a unix environment, but I really don't have the time or skills to make this project OS agnostic. If it is an eventual success, I may have to pay off some of that technical debt, but that is a problem I wouldn't mind having.
Unless you are making an application for users to install, I don't see how this will be problem. Effectively, the only significant OS you are locking yourself out of is Windows and Unix you can get for free and install anywhere.
So yeah... you're locking yourself out of Windows. Consider it a smart move :)
I'm not sure how compatible these two are with, say, Linux or OS X, but they ought to give you a starting point.
I feel like so many folks (me included) in the web development community are constantly re-learning new (old) techniques that we all really should have realized long before now.
Though I don't think I would change a thing as I've learned a lot over the past decade that I probably never would have if it not for this community.
Sometimes I just get a sense of deja vú, like none of this is new...
This whole "Unicorn" business for example. I read about this last week, and thought to myself, "Ok... hasn't Apache been doing this with 2.0 for years?"
So isn't this just re-implementing the wheel... yet again, but in Ruby?
See: cloud computing, AJAX, NoSQL, and all the other things that are purported to be new ideas but are really re-hashes of 30-40 year old ways of thinking with contemporary tools.
It's just a given that the majority of the ideas posted to programming blogs are just old ideas in new contexts.
Also I am not sure what is with this new tendency to call everything unicorns and ponies? It's intellectually degrading, and depressing to read such articles.
The Linux / BSD driven revival of Unix happened, un-fortuitously, just when object-orientation, graphics, UML, patterns, etc, were hitting their peak, largely because they were super successful in dealing with problems in graphic interface design. So these oo approaches became dominant in curriculums and paradigms far beyond their applicability, and no one bothered to study the classics of the previous generation like APUE etc.
Smalltalk and C++ begat Java, begat Ruby/ Python, and now we have a whole bunch of server-side programmers who know how to inherit from multiple base classes but don't know what fork() really does; when these young people (ahem) discover how well their grandfather's Unix addresses many of their current server-side problems, well, they are thrilled.
Let's just say it's the same thing as a twenty something discovering Sly and the Family Stone vinyl in the basement... Yeah, it rocks.
The same goes for file locking. Using files for synchronization is primitive, error prone and slow. As soon as you find yourself managing the lock files to deal with crashed programs ("stale locks" as the author calls them), you might as well trade it in for better abstractions.
Perhaps the most valuable part of implementing this kind of stuff by hand, using only systems services, is the appreciation you gain from having access to the same functionality in your language or from well designed libraries.