1. a straw man. The Format is very generic, only sections and key-value-pairs. A fairer comparison would be against httpd.conf, or other languages actually used for web server config.
2. Easy to parse, because there are tony of implementations already.
3. Static, like a SQL database, although Zed[1] and others[2] argued we need Turing-complete extension and configuration languages like Ruby.
But how would one represent a configuration like "don't require auth for /, but do require auth for /foo/*.jpeg" in a table? It would be great to just use a row for each possible URL, but with overlapping wildcards, folder hierarchies and multiple levels of allow/deny it gets very complicated very quickly. Granted, the allow/deny and order that Apache uses isn't great, but at least a configuration is understandable by reading it because the admin can lay out the configuration file with whitespace and comments in a way that makes sense. The equivalent might be achievable in sqlite, but it would be very challenging. It's basically a step backwards towards a poorly featured, confusing GUI configuration mechanism like the old IIS configuration used to be.
insert into rules (rule) values('don\'t require auth for /, but do require auth for /foo/*.jpg')
The use of a db is more to create a standard networkable interface for other clients to read/modify the config than a statement about the semantics of the config syntax.
I don't like that idea, because of one simple limit. You cannot diff databases (easily). Of course you can dump them, make sure the linebreaks are there between records, make sure they're sorted, etc. etc. But checking the whole config in or out of a VCS is much more useful in general. It also provides an easy to browse record of changes.
It doesn't matter which VCS. If you check in text, you can review the changes. If you checkin databases, all you get is a blob. Git is not magic and will not solve this problem.
Think of it like this. Zed is combining Apache's nonstandard configuration language with it's internal runtime representation of such. He's doing it in a way that's standard and manipulable at runtime. So changes to the runtime configuration are persistent. You can fix that. That's the beauty of it. It's an incredibly simple and flexible idea that builds upon existing, robust, and standard software. If you want to diff config files, then you could slap together an XML, JSON, or any other representation of your config data and write a loader for it.
Wouldn't the server only read it after a config option has been changed? I don't think the server is going to check this database every time a request comes in.
adamtj said "manipulable at runtime", so I guess he meant the changes are available directly after an UPDATE.
How can you tell if the config is changed? You either poll it every time with SELECT (SQL) or see if the timestamp of the database file has changed. (If SQLite works this way. I never had the need to check on the file status.)
This costs time as well.
Other web servers have a config file and reload it after getting signalled.
But: It's a bit early to say what Zed intends with SQLite. Maybe mongrel2 gets a configuration interface like Cherokee and only reads the DB on startup and after getting signalled.
I'm aiming for operational understanding of the data model used over being able to directly work with the config file. If it's a .ini (or other conf) usually you have no idea how the config maps to internal data structures. It's a mystery why some stanzas in nginx work on FastCGI but not Proxies for example.
With a sqlite3 based "config file" you'll be able to use real, turing complete languages to configure it, create control pannels, distribute the config reliably, and you can run .schema to figure out how it's structured.
It's also incredibly reliable and has many great features for this kind of thing.
Isn't the nginx's problem solvable with consistent implementation and complete documentation too?
I guess running some programming script isn't that bad to configure the server - but right now, people do a lot of basic operations on configs via shell scripting which do the deployment. Using bash/sh to interact with sqlite doesn't sound very inviting.
Possibly, it's a littler early to tell, but I'm sure I'll make it easy to use from the shell. Could even be possible to create converters from other config file formats to the Mongrel2 data model as well.
I'm with petercooper. If you act differently online, you're still responsible. His recent attack on Giles Bowkett was an assholey thing to do even before considering whether either one of them had a point.
That said, I'm not too worried considering his history with the Ruby community. I'd still have a better opinion of him if he didn't antagonize them though.
I'm absolutely certain he's a great guy in person. Asshole doesn't preclude you from being nice, personable, and even entertaining. Instead I take it to mean that when the bad side does come out it's nasty.
Giles Bowkett has never shrunk from attacking others (like his blog post "Chad Fowler is a Vampire", which was a long personal attack against Chad Fowler that Bowkett seems to have since deleted) so it doesn't seem that asshole-y to treat Bowkett as he treats others. I once participated in a thread in which Bowkett swore at and belittled a number of participants who criticized a blog post of his (he subsequently deleted his postings so the recipients wouldn't be able to respond).
You get what you give. I tried for a long time being nice and gentle online and I've found that there's a class of hackers who act like bullies anyway. They'll slander you, try to ruin your job prospects, lie about your personal life, everything you can imagine. Seen it first hand done to myself and to others and it's stupid.
It's ridiculous, but it seems the best way to keep this in check is to be kind to the gentle people, and totally destroy the haters.
Also, it's pretty fun tearing into these guys because they're usually pretty bad at their rants. :-)
I find it bizarre that you can come onto a forum like HN and insult me, call me an asshole, and effectively slander me and that's quite alright. You're totally not an asshole at all.
But if I out-rant a dude who compares me to Lady Gaga and tries to kick me when I'm down for no other reason than to drive traffic to his poorly written blog and sell more videos, then I'm an asshole?
You have the same double standard all geeks have. You let the blow hards, pundits, and bullies get away with treating other people poorly, lie, and hype themselves endlessly, but when someone calls them on it you call them an asshole or worse.
You're right. It's easy to talk about people as though they're not people online, even when you're secretly trying to praise them. I'm usually finding myself agreeing with you when reading things like that and even loving the vitriol while wincing.
So I apologize for writing timidly previously. I apologize for standing behind your back and spreading my opinion in an unbalanced fashion. I made the classic mistake of forgetting I'm actually in public.
I'll say it to you directly. Zed, I'm glad you stand up for yourself, both against my clumsy commentary and all the other blowhards who strike at you. I'm even glad you do it so vocally because it is a damn good story: the wronged who doesn't go quietly into the night.
The only negative side is that your voice is corrosive and bitter and if it weren't I'd feel a lot more comfortable about agreeing with you. As it is I feel like I'm nodding along as the bully takes out that annoying kid who really just had it coming. It feels good, but society is built on not just sitting and watching that happen. Worse, I hate that I end up remembering that feeling whenever I read about any of the cool things you do.
So yeah, I still think you're an asshole. It's unfortunately not what's fair or what's owed but instead how it looks. I'm sure you're ready to tell me why I'm wrong, but don't worry because I've already got a truckload of cognitive dissonance going on. It doesn't help that I think Fret War and Shedding Bikes and Lamson and Mongrel are all great projects and am completely sure that you're a great friend to those who earn your respect.
I just see it as the image you've intentionally built for yourself. Starting with ZSFA and now eking into oppugn.us.
Zed's writing has generally struck me as being pure larrikinism (especially under the banner of ZSFA). Unfortunately I'm not sure that there is an analogue for larrikinsim in American culture and it's not a concept I can easily explain; there is a subtlety to it that makes it both difficult to convey and difficult to execute. If you Google it you might be able get an idea of what it's about and perhaps get a different perspective on Zed's writing.
I do sometimes just throw an idea out there to see if there's interest. No point wasting time these days if nobody gets it.
In this case, I felt like reprising Mongrel and finally that it would make sense, so I whipped up some code, got it working, and made a viable chat demo with it:
I'm surprised that he's doing this... after so much vitriol about the Ruby (and Rails) community and being "ripped off" with Mongrel, I would have expected him to never touch it again. Granted, it's written in C, but I'd expect it to be heavily used with Rails.
That said, its good that he is writing it. The more software is written, the higher the likelihood that _something_ will be great, and when Zed Shaw's writing it, it's probably pretty good.
His primary use case is different for mongrel2 compared to mongrel however its based off the same HTTP parser and is a webserver so I guess the continuation seemed appropriate. Being completely written in C its going to be blindingly fast. I have a feeling it'll surge past unicorn, thin, passenger and everything else out there once its refactored and past beta.
Unicorn and thin are both written in ruby with the core being the ragel HTTP parser which was first implemented in mongrel. Thin took that ragel parser and added eventmachine to the mix which made it much faster the mongrel. Unicorn went a step further by leveraging the kernel in such a way that things got even faster. Macournoyer, creator of thin was working on thin-turbo, a rewrite of the thin backend in C but stopped a while back. Passenger I guess because its nginx or apache module has to be C but their focus was never performance, more ease of deployment. Passenger 3 will have performance improvements and they've got benchmarks here http://blog.phusion.nl/2010/06/10/the-road-to-passenger-3-te... but I'm thinking mongrel2 will one up it.
The advantage of Passenger is that it takes care of dynamically allocating instances of your app, adding them and removing them as needs be. I don't think Mongrel ever did this: you had to guess at how many instances you might need and hope for the best. (At least afaik - no one has ever corrected me when I've said that in the past).
I actually wouldn't compete with those servers, because they are very language specific. I'm aiming for Mongrel2 to unify all the languages in the network architecture so that you can do what companies actually do: use what works.
I could easily see Passenger or others providing a 0mq connector that Mongrel2 can talk to. Passenger is bad ass at running Ruby, so no point in trying to do that better.
That just refers to running multiple processes instead of multiple threads. Rails wasn't thread safe until recently and many ruby libraries still aren't (including the mysql driver). So it's common practice to run a "pack of mongrels" (a few processes simultaneously) and then load balance between them in case a long request ties one up for a while.
Mongrel2 will let you effectively design part of your app to benefit from Rails (for example) and part to use 0mq and do a more evented style of design. All within one configuration / approach.
Ruby 1.8 uses green threads, so running multiple processes lets you use all your cores.
As mentioned in another post, Rails used to not be thread safe. Right now I'm running a Rails site in production with multithreading enabled, but I'm sure many libraries in Ruby are not thread safe.
You can't, obviously, any more than you can do much more than monitoring and jumping in with a new server instance if your site suddenly gets 1000 times the traffic it normally does.
It doesn't takes 1000 times the traffic to max out your workers. Imagine if your database (or some other dependency) had abnormally long response times for a minute.
I would think that by spinning up more workers than you can support, you slow down all responses. This increases the rate of backup, which increases the need for those extra workers, which slow things down, etc.
By keeping workers below the ceiling at all times, requests back up, but you don't degrade performance. I think this makes things much easier to recover.
Correct me if I'm wrong about your scenario, since I've never deployed it on a high traffic site.
Oh, well you certainly keep the total number of workers below a maximum.
All I'm saying is that I have N sites that get different amounts of traffic. If more people are hitting one, I want it to get allocated more passenger instances to handle the traffic. With mongrel, I'd have to, a priori, decide how many mongrels each site gets and hope for the best. With passenger, it dynamically allocates them to handle traffic on an ongoing basis.
He exaggerates. That's just his style. We accept it, and we generally ignore what he says.
Mongrel is nothing but hype most of the time. It doesn't offer particularly advanced features that even nginx and lighttpd offer. It suffers from poor performance. It's only marginally useful if you're using Ruby on Rails.
As it stands, Mongrel2 is basically vaporware at this point.
Also: notice the AGPL license. If mongrel2 hits a sweet spot for the targeted types of web apps, Zed has a good chance of monetizing this project. I appreciate Zed's writing and work - so good luck to him.
The use of AGPL has some interesting ramifications. For example, I still don't understand why AGPL projects like MongoDB can be used as part of large networked systems and this not be a violation of the AGPL if the entire system's source is not released under AGPL also. AGPL is "networked viral."
"To say this another way: if you modify the core database source code, the goal is that you have to contribute those modifications back to the community.
Note however that it is NOT required that applications using mongo be published. The copyleft applies only to the mongod and mongos database programs. This is why Mongo DB drivers are all licensed under an Apache license. You application, even though it talks to the database, is a separate program and “work”."
Thanks. I had read that also a while back, so this sounds like the MongoDB developers offer a sort of waiver to the straight-up AGPL, effectively making it the GPL. In any case, kudos to the MongoDB developers; I have used it on almost every project I have done this year.
Its a little different than the GPL. If you modify some GPL server code, and open that server up to "public" connections, you don't have to contribute back your changes, as long as you don't distribute the server itself.
With the AGPL, however, you do. Some people, though, interpret that to also mean that any of your code that connects to an AGPL server must also be open-sourced; eg, you web app, if it connects to mongodb, you must provide the source. As far as I know, there's been no legal opinion if thats the case, and the mongodb devs have stated thats not how they read it, and thats not what they intend.
I don't think anyone could argue that if you had an AGPL HTTP server that the AGPL would be applicable to browsers that connected to the server. So if the Mongo protocol is presented as an open protocol it seems unambiguous. Though if someone created an "open" protocol to someone else's AGPL code with the clear intention of bypassing the AGPL... it might get weird and awkward.
This is a very cool idea. I think it's actually more along the lines of nanite or fuzed (ironically) than mongrel1.
I think Zed's approach is clever... to create a middleware layer that can intelligently (and without adding tons of config complexity) let people build fast, complex apps that fulfill requests using a variety of backends.
From the description ("… designed for web applications over serving files") it seems that you still need a web server as proxy in front of it for mid-sized projects; to serve static files. (Bigger projects use a server/CDN just for static files.)
It'd be able to serve files just like normal, but at large scales smart companies use a big CDN and don't do their own file serving. The economies of scale are just not there to do it yourself.
Mongrel2's advantage is its focus on apps. Combining HTTP, jssocket, and websockets seamlessly on one port means you can get the best of req/response and async operations in your GUIs.
That's because I find programming languages are marketed similar to religions and work more like Ponzi Schemes. My goal is to change the culture so that programmers start to think this "community/religion" attitude is weird.
Mongrel2 will be very language agnostic, in the same way that Mongrel was very "Ruby Framwork Agnostic". My experience there was that making the server fairly implemented for all created a boom in server frameworks. I'm looking for the same result with Mongrel2, but with many different languages.
1) loves small software (do one thing do it right)
2) loves C + glue code (perf in C, everything else script)
3) loves solid code
Some examples: SQLite3, Fossil SCM, Mongrel1/2, Merb (hinted by him), etc.
Wouldn't be surprised if he uses the tools developed by Dr. Hipps. Perhaps he prefers tools written by like-minded people.
What can I say, UNIX philosophy at its best?
Note: I like Dr. Hipp's software too (and OpenBSD). These tools are strong, solid, high quality, and represents everything that's good about software product. Unfortunately they're not the most "popular" thing out there.
83 comments
[ 3.9 ms ] story [ 165 ms ] thread1. a straw man. The Format is very generic, only sections and key-value-pairs. A fairer comparison would be against httpd.conf, or other languages actually used for web server config.
2. Easy to parse, because there are tony of implementations already.
3. Static, like a SQL database, although Zed[1] and others[2] argued we need Turing-complete extension and configuration languages like Ruby.
[1] http://vimeo.com/2723800 [2] http://lambda-the-ultimate.org/node/3606
insert into rules (rule) values('don\'t require auth for /, but do require auth for /foo/*.jpg')
The use of a db is more to create a standard networkable interface for other clients to read/modify the config than a statement about the semantics of the config syntax.
data = config_file.read()
"insert into config (config) values ('%s')" % data
Are you confusing offline representation with a runtime structure?
How can you tell if the config is changed? You either poll it every time with SELECT (SQL) or see if the timestamp of the database file has changed. (If SQLite works this way. I never had the need to check on the file status.) This costs time as well.
Other web servers have a config file and reload it after getting signalled.
But: It's a bit early to say what Zed intends with SQLite. Maybe mongrel2 gets a configuration interface like Cherokee and only reads the DB on startup and after getting signalled.
With a sqlite3 based "config file" you'll be able to use real, turing complete languages to configure it, create control pannels, distribute the config reliably, and you can run .schema to figure out how it's structured.
It's also incredibly reliable and has many great features for this kind of thing.
I guess running some programming script isn't that bad to configure the server - but right now, people do a lot of basic operations on configs via shell scripting which do the deployment. Using bash/sh to interact with sqlite doesn't sound very inviting.
Looks like he renamed NoNoSQL to MulletDB: http://mulletdb.com/index
MulletDB uses ZeroMQ and sqlite, and communicates with JSON, so I thought maybe this was the same project heading in a new direction.
Not with Zed, though.
But he's dead sincere when it comes to making things. He's like _why's evil twin.
That said, I'm not too worried considering his history with the Ruby community. I'd still have a better opinion of him if he didn't antagonize them though.
I'm absolutely certain he's a great guy in person. Asshole doesn't preclude you from being nice, personable, and even entertaining. Instead I take it to mean that when the bad side does come out it's nasty.
It's ridiculous, but it seems the best way to keep this in check is to be kind to the gentle people, and totally destroy the haters.
Also, it's pretty fun tearing into these guys because they're usually pretty bad at their rants. :-)
But if I out-rant a dude who compares me to Lady Gaga and tries to kick me when I'm down for no other reason than to drive traffic to his poorly written blog and sell more videos, then I'm an asshole?
You have the same double standard all geeks have. You let the blow hards, pundits, and bullies get away with treating other people poorly, lie, and hype themselves endlessly, but when someone calls them on it you call them an asshole or worse.
So I apologize for writing timidly previously. I apologize for standing behind your back and spreading my opinion in an unbalanced fashion. I made the classic mistake of forgetting I'm actually in public.
I'll say it to you directly. Zed, I'm glad you stand up for yourself, both against my clumsy commentary and all the other blowhards who strike at you. I'm even glad you do it so vocally because it is a damn good story: the wronged who doesn't go quietly into the night.
The only negative side is that your voice is corrosive and bitter and if it weren't I'd feel a lot more comfortable about agreeing with you. As it is I feel like I'm nodding along as the bully takes out that annoying kid who really just had it coming. It feels good, but society is built on not just sitting and watching that happen. Worse, I hate that I end up remembering that feeling whenever I read about any of the cool things you do.
So yeah, I still think you're an asshole. It's unfortunately not what's fair or what's owed but instead how it looks. I'm sure you're ready to tell me why I'm wrong, but don't worry because I've already got a truckload of cognitive dissonance going on. It doesn't help that I think Fret War and Shedding Bikes and Lamson and Mongrel are all great projects and am completely sure that you're a great friend to those who earn your respect.
I just see it as the image you've intentionally built for yourself. Starting with ZSFA and now eking into oppugn.us.
In this case, I felt like reprising Mongrel and finally that it would make sense, so I whipped up some code, got it working, and made a viable chat demo with it:
http://mongrel2.org/chat/
Based on that, seems folks get it and it'll work so I'll start hacking on it.
That said, its good that he is writing it. The more software is written, the higher the likelihood that _something_ will be great, and when Zed Shaw's writing it, it's probably pretty good.
And mongrel2 is not embedded. You connect to it. No linking. It's a server.
I could easily see Passenger or others providing a 0mq connector that Mongrel2 can talk to. Passenger is bad ass at running Ruby, so no point in trying to do that better.
Mongrel2 will let you effectively design part of your app to benefit from Rails (for example) and part to use 0mq and do a more evented style of design. All within one configuration / approach.
As mentioned in another post, Rails used to not be thread safe. Right now I'm running a Rails site in production with multithreading enabled, but I'm sure many libraries in Ruby are not thread safe.
I would think that by spinning up more workers than you can support, you slow down all responses. This increases the rate of backup, which increases the need for those extra workers, which slow things down, etc.
By keeping workers below the ceiling at all times, requests back up, but you don't degrade performance. I think this makes things much easier to recover.
Correct me if I'm wrong about your scenario, since I've never deployed it on a high traffic site.
All I'm saying is that I have N sites that get different amounts of traffic. If more people are hitting one, I want it to get allocated more passenger instances to handle the traffic. With mongrel, I'd have to, a priori, decide how many mongrels each site gets and hope for the best. With passenger, it dynamically allocates them to handle traffic on an ongoing basis.
Mongrel is nothing but hype most of the time. It doesn't offer particularly advanced features that even nginx and lighttpd offer. It suffers from poor performance. It's only marginally useful if you're using Ruby on Rails.
As it stands, Mongrel2 is basically vaporware at this point.
The use of AGPL has some interesting ramifications. For example, I still don't understand why AGPL projects like MongoDB can be used as part of large networked systems and this not be a violation of the AGPL if the entire system's source is not released under AGPL also. AGPL is "networked viral."
"To say this another way: if you modify the core database source code, the goal is that you have to contribute those modifications back to the community.
Note however that it is NOT required that applications using mongo be published. The copyleft applies only to the mongod and mongos database programs. This is why Mongo DB drivers are all licensed under an Apache license. You application, even though it talks to the database, is a separate program and “work”."
With the AGPL, however, you do. Some people, though, interpret that to also mean that any of your code that connects to an AGPL server must also be open-sourced; eg, you web app, if it connects to mongodb, you must provide the source. As far as I know, there's been no legal opinion if thats the case, and the mongodb devs have stated thats not how they read it, and thats not what they intend.
Its when you modify the AGPL server code, and open that up, you have to release your modification.
So in you AGPL HTTP server example, the clients never are required to be open, but any changes you make to the server must have the source available.
I think Zed's approach is clever... to create a middleware layer that can intelligently (and without adding tons of config complexity) let people build fast, complex apps that fulfill requests using a variety of backends.
Mongrel2's advantage is its focus on apps. Combining HTTP, jssocket, and websockets seamlessly on one port means you can get the best of req/response and async operations in your GUIs.
I remember reading in his book teaching programming to beginners that the important thing is to build stuff, not to nerd out about language features.
Basically, I'm a linguistic secularist. :-)
1) loves small software (do one thing do it right)
2) loves C + glue code (perf in C, everything else script)
3) loves solid code
Some examples: SQLite3, Fossil SCM, Mongrel1/2, Merb (hinted by him), etc.
Wouldn't be surprised if he uses the tools developed by Dr. Hipps. Perhaps he prefers tools written by like-minded people.
What can I say, UNIX philosophy at its best?
Note: I like Dr. Hipp's software too (and OpenBSD). These tools are strong, solid, high quality, and represents everything that's good about software product. Unfortunately they're not the most "popular" thing out there.
There, ftfy
font-family: sans-serif;
Instead of "sans serif".