The old blog software, Typo, is nowhere near as good as Wordpress. I only used it because it was written in Ruby on Rails and I am a Ruby on Rails coder. This is a very poor means of choosing software.
The VisualWorks Smalltalk Server Pages framework already does this.
SSP results in web pages as compiled machine language.
The HTML file is translated into a new Smalltalk method, the tags become Smalltalk message sends, and the HTML between the dynamic tags just become accesses into an Array. Because Array access is optimized and JITed as machine code, the only message sends are due to the dynamic tags. When the webpage/method is JITed, you have a web page compiled into machine code.
I believe the framework discussion is of a compiled C "Rails" type MVC framework, I'm not familiar with Smalltalk Server Pages but from an initial google search it doesn't seem to be on par with what rails provides?
You're right that SSP is more like ASP/JSP, and not like Rails. But the same idea can be used with a more comprehensive web framework and with a language that has a really good JIT, like LuaJIT, or maybe IronPython/IronRuby.
I don't think it's a troll, but just someone who is really in to performance, like I do, and I agree with him on some points.
But what I don't agree is using just C. C is too low level and is a horrific unfit task for web development. Instead C++ would be a good fit (and writing in C mixed in). The STL/Boost provide a wonderful abstraction layer that does pretty much anything you throw at it with ease, plus there are lot of other libraries for anything else you can ever think of. I've written several web applications in C++, and it was actually very much like writing in Python/PHP/Ruby and at time prefered writing it in C++ after writing a poc in a scripting language. They were just a little bit less readable but very clean, but the biggest plus the the multiple magnitude performance increase of the application it self with FastCGI. (all these applications were serious number crunchers and had lot of CPU and Memory Bound problems).
tldr;
C for web dev painful, C++ with STL/Boost good/easy/fast/clean.
I've been hacking C++ long enough to agree C++/STL/Boost is good and fast... but I can't remember the last time I heard anyone accuse C++ of being easy...
I find it easy. The bad thing about c++ is its size. It's large and that tends to make it complex. However, I find that I only use about 40 to 60 percent of what c++ can actually do. Focus on the parts that you use and you'll find a lot of the complexity goes away.
How about a web-centric programming environment that compiles web page code to DLLs, and changes pages "dynamically" by unloading the DLL, recompiling, and reloading? Base the language on a modified Ngnix as an execution environment and generate C as intermediate code. Use TCC to compile web pages very quickly. The modified Ngnix would have a big hashtable of dynamic methods, which would be pointers to functions in the DLLs. Make it concurrency-friendly like Erlang by using isolation and green threads.
Your comment adds less to an intelligent discussion than does the article you're calling a troll.
First, this isn't a troll. The guy posted a blog post on his own blog. He didn't post off-topic content to a thread and didn't try to provoke an argument in an otherwise civil conversation. The only possible way that you can characterize this as a troll is that he uses some crude language, and says some things that are unpopular on HN. But say what you will about the content, he didn't try to make you read it.
Second, the guy has a point. I've had a rant like this building in my craw for a long time, and this guy nailed nearly every point I've wanted to make. From a software engineering perspective, web development is in the dark ages. What's worse, it's a luddite process -- a totally self-made exile from the benefits of 50 years of progress: Type safety is shunned. Language efficiency is ignored. Data structures beyond the hash table are exotic and rare (and worse, I've met more than one "professional" web developer who couldn't tell me the difference between open and closed hashing!)
I understand very well that programmer time is more valuable than computer hardware, and that the speed/development time trade-off is often biased toward development time. But if you ask me, what was once an intelligent guideline is now becoming a cult. It is possible to do rapid development in a compiled language, and there are some really good reasons -- beyond execution speed -- to revisit the idea.
"Your comment adds less to an intelligent discussion than does the article you're calling a troll."
Sometimes the truth is very simple. It can be considered a troll because it is in the tone "if you don't do as I say you are an idiot, like everyone else".
Second: not all scripting languages are interpreted on the fly. In fact, many of the most popular frameworks load objects once and keep their compiled forms in memory. Even for languages that run under VMs, some of them run under JIT that, in the end, results in machine-language-like times.
Third: I have profiled a lot of web applications and most of the time they are either waiting for the pipe to deliver a request or waiting for the database to deliver the data. Very few web applications I know of do heavy processing. For those, I agree compiled code may help.
and, finally, by the time the guy finishes his C code, an equivalent competing application that uses Rails will be approaching the millionth registered user.
Even though the tone of the article was abrasive, it was far more interesting and insightful than the one-line comment to which I responded. Your asserted arguments for dynamic languages are well-trodden ground:
Assertion: "Some scripting languages are as fast as compiled languages."
No, not really. I challenge you to prove that statement with a non-trivial benchmark.
Assertion: "most of the time, web apps are waiting for something external to respond"
This doesn't survive a few seconds of thoughtful analysis. If a web app is spending most of it's time blocking for I/O, it's poorly designed. There's no reason that it can't be handling other requests while waiting for a DB response or a pipe. And once you've allowed for that possibility, you're back to a situation where you can handle more requests with faster software.
Empirically, this is obviously true: languages like Ruby and Python don't hold a candle to Java when it comes to web application speed. We choose them because of developer convenience, nothing more. Which brings us to...
Assertion: "C programming is terrifically slow for development. <insert snarky comment about Rails applications with a million users here>"
Ignoring the fact that there are other compiled languages than C (that are probably better choices for web development), I can tell you from experience that by the time your Rails app reaches a million registered users, 80% of your week is going to be spent either working around the myriad scalability flaws in Rails and the dog-slow nature of Ruby, or installing hardware to accommodate the load. Scaling Rails is a nightmare.
Now, it may be true that you can develop that Rails app a bit faster than you could a webapp in a compiled language, but it's a wild exaggeration to suggest that the difference is more than a few weeks of skilled developer time. It's a fallacy to imply that all compiled-code projects end up in death-march hell. Truth be told, the real reason that web apps can be written so quickly is because they're easy, and they tolerate a lot of development error. So long as there are good string processing facilities, the choice of language is a fairly minor point.
"There's no reason that it can't be handling other requests while waiting for a DB response or a pipe."
Concurrent request processing most likely belongs in the server, not in the application.
"Scaling Rails is a nightmare."
For someone who complains of "well-trodden ground" in others' arguments, you certainly seem to have a deeply-rooted belief in the myth that particular languages and/or frameworks "scale" or "don't scale".
Write me off if you like, but here's a short list (just off the top of my head) that we've encountered while trying to scale Rails:
Rails has terrible support for database replication (we've had to roll our own). Its caching architecture is laughably bad, and the implementation was/is buggy (again, we rolled our own), and there's next to no support for the kind of robust page cache expiration mechanisms that are necessary to run a large, dynamic site (Observers are only a start). Rails is a memory hog, takes too long to start, has poor translation and locale support, and simply passing a request through ActionController adds huge overhead that cannot be escaped.
And even though you don't want to hear it, Ruby is slow -- even after working around all of the above, we still had to implement a fast page caching layer just to work around the general, all-purpose slowness of Ruby.
Seems like you're picking things which are difficult to deal with no matter the language/framework your application code is written with -- DB replication, cache invalidation, etc.
I find this similar to Woody's fear in the following exchange:
Cliff Clavin: Boy oh boy, talk about your lucky days, huh, I just found twenty smackers back there in the pool room. And to celebrate my good fortune, I'm going to buy a round a drinks for all my friends.
Carla Tortelli: What are you going to do with the other nineteen bucks?
Sam Malone: Woody, that could be your twenty dollar bill, the one you lost.
Woody Boyd: Hey yeah, you know I was in the pool room earlier.
Cliff Clavin: Hey, wait a second. Let's be fair about this. Give me the serial number.
Sam Malone: Oh, come on man. Nobody knows serial numbers.
Woody Boyd: L21886119B
Cliff Clavin: [hands the bill to Woody] That's amazing.
Sam Malone: How did you do that?
Woody Boyd: I memorize the serial numbers on all my currency.
Sam Malone: Why?
Woody Boyd: For just such an occasion. Tell you though, I pray every day I don't get rich.
Not quite a perfect analogy, but my point is worrying about scaling is a very good problem to have, and is likely to be accompanied by the resources to fix it. Twitter is the poster child for Rails scaling problems, but they re-wrote the parts they needed to write, and people kept using it because they liked the service even when they did have scaling problems.
Twitter isn't really the poster child for Rails scaling. A huge chunk of their traffic isn't handled by Rails, and their web interface is fairly perfunctory.
In any case, at this point I think it's fairly safe to say that I have some direct experience scaling Rails, and it's not as easy as the hand-wavers suggest. It's certainly worth a bit of work early on to avoid the nightmares that come with trying to scale something like Rails later on.
Not to be too silly, but if someone wrote a web framework in C++ that was 10% more difficult than Rails, I'd still consider it. The distribution of suck for Rails is exponential over time, so the expected value of suck is much higher than it initially seems.
"A huge chunk of their traffic isn't handled by Rails, and their web interface is fairly perfunctory."
I guess I wasn't clear enough. I meant they rewrote the parts that didn't scale in another language. Scala, I believe. Is that incorrect?
So, my point then is that writing it in Rails to quickly get it in front of users and quickly iterating on features turned out to be worth it, even though they had to rewrite large portions in another language once they had a lot of users. Do you disagree? I'm not a Rails developer, but this is my understanding of how Twitter development played out, and it seems to have worked very well for them.
I'm curious, if you would accept 10% more difficult, would you also accept 20%? 50%? I'm wondering just how exponential the suck can get.
"If a web app is spending most of it's time blocking for I/O, it's poorly designed. There's no reason that it can't be handling other requests while waiting for a DB response or a pipe"
If your request-handling logic is responsible for concurrent requests, your application (and the platform it runs on) is poorly designed. This belongs to a thread/process dispatch mechanism in the platform, not inside your app. When one thread waits for the database, I am sure the processors can be busy with other requests.
"Scaling Rails is a nightmare."
No. It's not. There is little reason for it to be. While I am no Rails expert (I use Django and was a Plone guy not that long ago), the scalability issues most people have happen because of RDBMS concurrency much more than any problem with Rails itself. Like I said before, that time spent waiting for the database to respond is time spent in the database working. It is incredibly rare to have a front-end having heavier usage than the database that feeds it.
If you want your applications to scale, design it around the database limitations. Use C, C++ (I would prefer ObjC), Java or Assembly if you want, but faster request processing won't get you past database scalability issues.
Using C* or Java will only push back your release date.
In my experience once you get over a few hundred thousand users and have tweaked your web server/proxy setup and tuned your database, you will still have major slowness and memory issues with your Rails code. I.e. it's usually not the database's fault.
Also you might want to look at the profile of the guy who said that rails scaling is a nightmare. I wouldn't use the word nightmare as I'd prefer to think my professional life story wasn't written by Kafka. But, I don't know anyone who has scaled a big Rails site and thought it was a pleasant or fun experience. Most guys who say that it's no big deal haven't really done it.
I will concede that, according to my very limited Rails experience, it generates a lot of database chatter when compared to hand-coded queries. Still, this is mostly a database scaling problem.
The beauty of the ORM is that you can replace the underlying layer without disrupting the code running on top or add fancy things like local memory caching or a networked memcached store. I insist that if your application was built from the start with this kind of issue in mind, it will be easy to scale it if and when the time comes. Until then, you are solving a problem you don't know you have.
This gets harder the lower-level your language gets.
The top line in a gprof profile of an evented C web server had better be select() (or equivalent) or read() (or equivalent). So I don't follow you here.
Yes, the average server thread is going to spend far more time blocked on I/O than not. But the server should be able to handle more than one request simultaneously.
(Or you can use an event model; the point is, if your server is spending all of its CPU time blocked while under load, then something is wrong.)
Now you're not following me, timr. An evented server can handle 10,000 connections simultaneously. Run it under load, profile it. Select is at the top of the profile. The program spends all its time in the kernel, waiting on I/O.
That evented server is also faster than the threaded version of same.
What's the compute task you think a program is bottlenecking on in an application that does nothing but shovel data out of a database into a socket based on a btree key?
You're overgeneralizing. Where your server spends its time depends entirely on the profile of your app. If your pages render (on average) in less time than they wait for I/O, then your server will spend most of its time blocked on I/O.
I suspect that you're thinking of something like a lightweight proxy server, that waits for incoming requests, then quickly hands them off to an app server that does most of the work. That's trivial. If you just profile just the webserver process, then yes, you would expect that most of its time would be spent in wait. But while the proxy server is blocked on I/O, the app server is doing a ton of other things for any non-trivial page rendering.
This is worth repeating: only small part of request lifetime is spent in application code so you should worry about performance of the programming language only if you profiled your app and saw, that this is indeed a bottleneck.
Ranting aside, this seems like a valid complaint. Scaling and performance seem to be reasonably important in web applications, so why not write them in C? If the only complaint is the lack of good tools, why not build those tools?
Disclaimer: I don't actually work on web apps, and therefore have no experience with these things.
The increased productivity of writing in a scripting language typically outweighs the cost of extra servers. Whereas with a desktop application, you only have X resources to allocate to a task, you can always "just throw more servers at" a web application to some degree.
It is done occasionally though; last I heard, large portions of Amazon.com were written in C.
If your site starts to take a performance hit in rendering pages, you can simply write a module in C to handle the part of the script that's using an excessive amount of CPU time, and keep the rest in script.
But for most web apps, rendering HTML pages from script is not the bottleneck, it's the interaction with the database, and the solution is intelligent caching to reduce load on the database. The tools to mitigate these problems are already written in C (memcached, etc).
It is highly unlikely that happens.
Once you lower your DB issues, if the application is responsive enough you start working on other features. By the time you're done the application got new users and the DB is a problem again ... it's a cycle :)
Fortunately HTTP servers do scale by caching and load-balancing.
The author of the post probably never worked in real-world conditions. You first have to have users and real needs for scaling, and that doesn't happen unless you deliver a useful application first, and you also have to deliver it pretty fast, otherwise you might lose to a competitor with the same idea.
Reread what you wrote :) Once you lower the the db io issues, rendering does become the bottleneck.
Not really. Sending and receiving data to/from the browser becomes the bottleneck.
If it takes 50ms to receive the query from the browser, 0.1ms to retrieve the data from the cache, 10ms to render it, and another 50ms to send it back to the browser - the bottleneck is now network communications with the browser.
There are, unfortunately, limits on how much that can be improved.
Because most scaling problems in web apps are ultimately architectural. Yes, you may need to buy a second server sooner rather than later if you run Ruby instead of finely tuned C or C++, but for most apps you're going to need to worry about a second database server and replication before that, and long before reaching either you will have enough traffic that you'll need to prepare to handle front-end scaling properly soon anyway.
An example - from messaging middleware, not web frontends since it's somewhere I do have proper data:
I did a messaging middleware system in C a couple of years back. It's easy. I've done it before. After a while needs were changing and I decided to try writing the replacement in Ruby. It was a lot quicker to write (1/10th of the code size for roughly the same feature set).
We were handling 3-4 million events a day, which could easily result in 2-10 messages each.
The Ruby replacement spent 10% of a single 2Ghz Xeon core handling those messages. Of that 90% was spent in the kernel handling system calls that would've been roughly the same in a C/C++ version.
Even if the Ruby part of the equation was 10 times slower for the userland part, the most we'd have saved for our load was 0.9% of a single core. The most we'd save overall, regardless of scale, would be around 9-10%.
Doesn't mean there aren't things for which a language like C is better suited - I did my MSc. thesis on reducing error rates for OCR, and some of the image processing I did was prototyped in Ruby, but then translated to C for the final version because processing literally would've taken days for things that took less than an hour in C.
But scaling is very often constrained by other things than the performance of the laguage you write your app in. Especially in the web space where most interesting apps are database backed.
If your DB gets maxed out, having a hyper-optimizing compiler is not going to help you. If your app isn't designed to be scale across multiple servers (or at least cores - these days you can rent managed hosting on a 24 core server for less than $1k/month) you could very likely be screwed in ways that no compiler will save you from.
The extra developer time spent doing web development in low level languages is unlikely to be worth it unless you're at a massive scale.
If you can fit in 24 cores with a language like Ruby, your best case saving from switching to a compiled language these days is around $10k/year on hosting if you're ok with a single server, and about $20k if you want redundancy.
That doesn't pay for a lot of wasted developer time. Most web apps can fit in that, and for the lucky (or unlucky, depending on reason) that need to scale beyond that, the CPU time spend on the actual web app is likely to be a relatively small fraction of the total CPU time spent (on databases etc.) - architectural decisions is likely to be a far more important cost driver than the language of the web frontend.
As for developer availability - good luck to him finding good C programmers, and even more so finding good C programmers who aren't expensive and knows web development.
I'm not sure that's accurate. c++ is a large language. Finding developers who use the parts of c++ that you need to get the task done is easy (boost, qt, stl, wx, etc). Finding developers that know all of c++ is next to impossible. c++ is large because it can solve very large, very complex problems, but not every c++ application attempts to do this. c++ can be used in very simple ways too and many people use it in that fashion.
To me, C makes a lot of sense for software that doesn't change all the time. The point in the article about being an engineer is valid.
However, most website projects are business-driven, which means that you are constantly changing and refactoring stuff as misunderstandings get cleared up, design flaws emerge, business models change, minds change, etc etc.
Business software projects are not just engineering, and if you treat them as such you will endure pain.
The analogy with a house is odd - if a house was purely an engineering problem, then they would probably all be built out of precast concrete or something, not "wood and steel". Aesthetics, tastes, budgets, and tradition all have an impact, and nearly every house is unique, so you use the "best tool for the job" - the one that is the easiest to customize on-site.
True, if house building was business-driven (instead of solely architect-driven) then it would make more sense to use materials more suitable for iterative process. Like cardboard boxes...
When requirements change weekly and by the projects end your original project description is only useful as a piece of history - then using concrete would mean lots of demolition work in the process.
The usual performance bottleneck in a website is the database, which is usually written in either C or C++ anyway. Other parts of a web application are less likely to cause performance issues, and can usually be scaled horizontally by adding more servers.
I wrote the upload and download handlers for droplink.me in C as FastCGIs, anticipating performance problems. I built it; but they never really came. It was a fun exercise, but if I wasn't interested in having fun with C, I would have written it in a dynamic language.
I think the reason there isn't a "C web development community" is that programmers are more than willing to trade time for performance. Safe, readable code can be written in high level languages in a snap. You can see your ideas happen very quickly. If you run into performance problems (which I think is safe to say is a fairly rare occurrence for the average web developer who produces decent code, in any language), you can a) refactor or streamline what you already have or b) start fresh, lessons learned, in something lower level.
An aside: the site http://halfbakery.com is implemented entirely as an Apache module.
In web the model is different: You develop an app, push it live and find out what works and what does not, change and iterate.
This means that doing the iterations with minimum time and effort are important than worrying about scalability and performance.
You can always throw hardware at the perfortmace problem. If you get enough traffic to become worried about performace, that itself is the mark of success. But to become successful, first you must build the app and get it live somehow instead of worrying about performace issues or wasting months coding in some very low level language!
You can always throw hardware at the performance problem.
No, you can't. That's what scalability means. If you don't have scalability, then additional hardware will only take you so far before you begin wasting it.
Well, you are both kindof taking the extreme view.
If you build your app such that the webservers can act independently, then by using various methods of load balancing you can just throw hardware at the problem.
However, this just usually leaves the database as the single point of contention among all webservers. There are various ways to reduce load on the database server by some constant factor (memcache, query caching), but in the end database load will be roughly linear with site load (requests/time). You can't just add SQL-based DB servers and get linear speedup like (properly thought out) webservers.
If you think your site will get so big so fast you won't have time to hire engineers to figure out the scaling problems before they become unmanageable (for example, facebook got huge, but it took a few years, so they had time to raise money and adapt their stuff, although they were pretty slow for a long time), then use appengine datastore or simpledb or some other db that promises to scale really well (this is a really hard problem, which is why there aren't a lot of open source things that do this. Note: couchedb can't scale writes, so I don't count it).
My personal thinking is that one big properly designed database can get you pretty far, and database clustering can get you far enough. If you find it pleasant, putting your app in appengine would make you pay the price of scalability a little at a time starting right away, instead of letting you push it off (but with a huge performance wall that you won't be able to get over without basically redesigning much of your apps internals).
Many dynamic-languages frameworks (notably PHP) are more scalable than writing a custom webapp in C. This is because C encourages you to store state in memory on the server, because it's really easy when everything is a single long-lived process. Good for performance, terrible for scalability.
PHP (and Rails/Django, if you ignore their ORMs) encourage you to store state on some collection of backend servers and communicate via RPC to retrieve that. This is bad for performance, but good for scalability. The front-end is just a dumb intermediary that converts data into HTML, so if you need to scale, just add more of them. Scaling the backend is more challenging, but at least you can do it without dragging all the front-end code around.
Absolutely! Just for the record -- since people seem to assume otherwise -- I'm not agreeing with the article. I'm just disagreeing with the idea of ignoring "scalability and performance" and then expecting to solve performance problems by "throwing hardware" at them.
I think you don´t have to stick with C (or any another language ) for everything. Each language are good for something. Right now, for example, i´m working in a financial intelligent software analyzer. I wrote the web crawler in ruby (not rails) and now i´m planing to write the core of the system, the decision maker, at Lisp. I think a programmer should not be sticked with just one language, there´s a lot of cool things out of the C world. You dont have to write an entire C framework to build an web apps at C ... it´s just too troublesome. You have another challenges. But when it comes to process data very fast in your server, so maybe you should consider another language.
Most sites aren't CPU or memory intensive. If you are lucky enough to have enough users for that to be a concern you can add more servers. Sites normally bottle neck around the database and writing front end code in C is distraction from fixing the real problem which should be scaling the database.
That is why there are things called "application servers". Java is not perfect but it has a fine feature called JIT (Just In Time compiling) that produces machine code whenever it makes sense. Raw compiled language goes against the current trend for virtualization, whose goal is to save energy too. So in my opinion JIT is a good solution.
It's a valid point. I wrote the Mibbit backend from scratch in Java for these reasons. I can only start to imagine the horror I would be enduring if I'd chosen a scripting language.
Having said that, choose the right tool for the job.
edit: Downmod me all you like. A scripted language wouldn't handle 2k req/sec on a couple of VPSs.
Why would anything need to go to db? :/
Going to db should always be a last, final resort.
People often seem to think a db is the only way to write any webapp, and that everything must depend on the db. That's bad architecture.
In the case of Mibbit, db access is only done for writes, lazily, in the background.
The bottlenecks for me, are CPU, network, and ram. And yes, you can throw servers at it, if you're frivolous and don't intend to make any money. I decided to go the other way and be profitable ;)
The vast majority of web applications are straight-up CRUD. Not making that all about a normalized schema is the stupid mistake.
As far as relational DBs being the problem, that's more a matter of the free ones not cutting the mustard. Oracle and DB2 can handle jaw dropping load in the hands of an expert.
Would you not agree that the web is becoming more "real time". People demand webapps to update as they view them - ajax,comet,getNextBuzzword(). Those demands place a higher load on servers, especially when you scale to large number of users.
For sure, but 99% of apps never scale to "large numbers of users". And when they do, they usually have the time to figure out how to do so without downtime. Often, that may involve rewriting part or all of the backend in a lower level language. If that's needed, so be it - but to do it before you have any performance problems is just premature optimisation.
Judging by the number of blog posts, articles and books about scaling this or that Ruby/Python/PHP framework, people seem to be very busy with "premature" optimisation right from the start, exactly _because_ those frameworks and languages are so horribly inefficient.
Why would I want to run and manage and design for a server cluster just because each and every Rails process needs to load the entire language runtime, the framework and all the libraries using 100MB of RAM for no application functionality at all? Not doing all that work is premature optimisation?
I think Mibbit, much like twitter, is one of those apps that is a non-traditional webapp. While its a web based frontend the traffic and distribution of chat messages is a much different beast then what most webapps do. Java is very likely a much better choice for these reasons.
Actually after I wrote that post, that's something I've been tossing around. So many of the current crop of web superstars, like facebook, twitter, friendfeed etc. are built around streams of data as opposed to serving more traditional static(ish) pages.
Perhaps a new type of web framework is needed to address these breeds of apps.
Web site performance problems usually don't have much to do with the raw executing speed of the language they're written in. The problems derive from poor understanding of how to handle concurrent connections. If your app is written in C, but is accessed through a CGI interface - starting a process for each request - then it's going to be slow. Similarly if you write a web server which makes blocking connections to a backend database, your server will be slow. If you use threads to handle concurrent connections, you're going to allocate 2mb stacks for each user - and your website is going to suck. This all has little to do with how fast a language can invert a matrix.
And then there's situations like porting a Rails app to Lift and seeing a huge improvement in performance without touching the database / storage IO: http://lambda-the-ultimate.org/node/2147
Granted, it's not from the most unbiased source, so take it with a grain ;)
That has to do a lot with Ruby's awful thread implementation. You could possibly port the app again to Ruby EventMachine and see even better performance.
The point is - for 99% of people it's not the language, it's the I/O
I have worked with a project for the past several years that, by necessity not choice, required a web application framework written in C (it's an embedded platform) and I can tell you it is pure hell. The most mundane and simple tasks, often trivial one-liners in scripting languages, require many lines of error-prone C code even with a fairly rich set of libraries available. Working in this environment, you'll quickly realize why the rest of the world doesn't do it this way. Thankfully, this project is finally moving away from this architecture and I couldn't be happier about it.
Would Lua have worked there? It's a ~200k library designed to provide an embedded scripting language to C projects. (It's very similar to Python, but a tenth the size, and with a bit more Scheme influence.) While it may be too big for some especially tight embedded platforms, if you could fit it in, it would have likely been substantially less painful.
It's also a pretty good scripting language on its own merits, and has replaced Python as my default language. (Unlike Python, its design expects you to know at least a little C, but in exchange, it avoids cluttering the language with things that C already does well.)
Lua is meant for embedding in C programs; that is different from running on embdedded systems. The latter typically features drastically less RAM and CPU cycles, so most interpreted languages (which are not shy about trading away some performance) don't cut it. There are some exceptions; FORTH, for instance, can fit in about a kilobyte and its performance isn't miserable. But putting e.g. Lua in a confined space would not work out well.
To my understanding "embedded systems" spans from something with 1k of memory to something like a cell phone or a router. Working with Lua is quite comfortable on the latter, hence my qualification. It's been ported to Lego Mindstorms (http://www.hempeldesigngroup.com/lego/pbLua/index.html), for example, which is "a 32 bit ARM7 microcontroller with 256K of FLASH and 64K of RAM."
It's not clear from the post which end of the spectrum he or she works with. (I don't work with embedded systems professionally, I've just tinkered with OpenWRT, cell phones, etc. as an enthusiastic amateur.) It would be usable in the same niches that SQLite is, for example.
The most mundane and simple tasks, often trivial one-liners in scripting languages, require many lines of error-prone C code even with a fairly rich set of libraries available.
What would be an example of this? I'm mostly a C and Perl person, and once a library is available I've never noticed cases where something was much simpler to write in one than the other. That library might take more time to write in C than Perl, but once it's written I find it just about (less than a factor of 2?) as easy to write the application logic in C as in Perl. My instinct is that if you're writing 'many lines of error-prone C' to do something simple, there's probably room for improvement somewhere.
Funny article, I hope the author was not being serious. C is everything when you need performance, but having something done quick and maintainable - in C - no sorry. If somebody would dare to actually create a rails or django level framework in C it would simply have no community - not because it won't be 'hip' - I'd use it - but because implementing the same app would take exponentially more time. And that's what matters in the webworld.
No matter how many rps your app can serve, if there is no app you get no requests to serve...
The reason why people choose to go with languages like Ruby is because the trade-off on speed is worth it.
Web front-ends scale very easily. You can just add more web heads and load balance between them. Databases are much harder to scale since you need to keep data consistency and can't just throw more boxes at the problem.
So, you don't need to be computationally efficient. However, you do need to develop fast. You need to be able to create features before your competition. You need to be able to make your system more reliable and bug-free than your competition. And that's where Ruby (and similar languages) helps you.
Sure, you and Linus have a point that selection of language can keep out the riff-raff (Linus Torvalds having a huge rant against C++ simply because of the programming community), but it also means that you're missing out on the good parts of the community. Ruby programmers tend to put a lot of thought into user experience. While you've ranted against testing, I don't think any IT manager would agree with you there. Testing substantially improves one's ability to create new features because you can easily spot anywhere you've broken something. And that leads to better quality applications too.
There's a difference between speed and scaling and if something scales fine, that's ok. Look at it this way, array indexing in a slower way to go through an array in C than incrementing a pointer. And if you had a lot of code you could go through and change all instances. However, it wouldn't be worth your time. Such an improvement wouldn't net you any real benefit while you'd be neglecting improving your product.
I think your point is exacerbated by the fact that it's hard to get web UIs "right". I was critical of languages like python and ruby until I had to do the frontend for some company code. Box looks like it needs to be shifted a few pixels right? Modify code, compile, check. Logo looks better with an offset? Modify code, compile, check. Writing in a scripting language makes it just Modify, check.
You just can't do that in compiled languages, and the compiled language of choice for the web-- Java --isn't known for blazing compilation speed. You'd probably also have to jar it up and copy the damn thing over, whereas psps could be modified in-place. I think this is something that compiled folks like me simply failed to grasp: all the things about scripting languages that make them orders of magnitude faster for developing, especially for the web.
I've done a lot of webapp development in both Java and Rails. I would agree that the iteration cycle is shorter in Rails, but it's not as bad as your description makes it sound. If you have to redeploy a jar file as part of your development process, then that's a problem with your process, not your language.
When I do Java webapp development in Eclipse, it compiles and hot-swaps my code in the background without requiring any restart or redeployments. Works pretty well.
I do have to restart occasionally because the hot-swapping has limitations, so that does slow things down a bit, but when you consider the time saved by having good compiler integration, then I don't think there's a big difference in productivity.
I had that basic idea when web app frameworks starting becoming popular. So I implemented a web framework which took a 'description' of the app written in perl, and generated C++ which compiled into a single exe file which served the app with a built in http server. Small bits of business logic were written in C++, but basically it was done by the perl. All templates, internationalisation, everything, compiled down to a few database queries, string concatenations and the odd if statement.
I then build a couple of production sites in this, one on my own, and another with a few other developers.
Conclusions from this:
1) It's really really fast and efficient, and the single exe model for deployment is lovely.
2) It's a pain to implement such a framework, and a pain to use it.
I like Perl ... using modern Perl practices and CPAN modules. It's the best web development environment I worked with ... and I've worked on projects with PHP (my own framework :) + Symfony), Python/Django/web.py, ASP.NET, Java/Struts and Ruby/Rails.
You should give it another try sometimes. Take a look at Catalyst (the web framework), DBIx::Class, Moose and POE.
I think I just got a bit fed up of the crufty syntax for object-orientated code.
Everything computer-related sucks, you just have to choose something which sucks the least where it matters for your project. And familiarity counts for a lot too.
There are already a few options for a MVC web framework in compiled code. I'm working on ASP.NET MVC now. It's compiled, but development does not feel like that because of JIT compilation. Also, it will run in Mono on Linux if I want.
On the upside, it means web developers have another way to slack off: "But we can't work, the project is compiling!" As if web devs needed another way to burn our time.
"I predict in the future all the good web engineers will be moving to compiled languages, and all the academic idiots programming social networking applications, ajax widget home pages, and projects to better improve our democracy will be staying behind coding in the latest and greatest slow language, going to circle jerk conferences to hear some scumbag just oozing hubris from every pore tell them to get in shape and learn to fly a kite."
I don't think the guy is troll. I think he simply doesn't know better. This is a classic case of micro-optimization. While he should look at the bigger picture. It is all about global architecture.
176 comments
[ 5.0 ms ] story [ 183 ms ] threadThe old blog software, Typo, is nowhere near as good as Wordpress. I only used it because it was written in Ruby on Rails and I am a Ruby on Rails coder. This is a very poor means of choosing software.
SSP results in web pages as compiled machine language.
The HTML file is translated into a new Smalltalk method, the tags become Smalltalk message sends, and the HTML between the dynamic tags just become accesses into an Array. Because Array access is optimized and JITed as machine code, the only message sends are due to the dynamic tags. When the webpage/method is JITed, you have a web page compiled into machine code.
But what I don't agree is using just C. C is too low level and is a horrific unfit task for web development. Instead C++ would be a good fit (and writing in C mixed in). The STL/Boost provide a wonderful abstraction layer that does pretty much anything you throw at it with ease, plus there are lot of other libraries for anything else you can ever think of. I've written several web applications in C++, and it was actually very much like writing in Python/PHP/Ruby and at time prefered writing it in C++ after writing a poc in a scripting language. They were just a little bit less readable but very clean, but the biggest plus the the multiple magnitude performance increase of the application it self with FastCGI. (all these applications were serious number crunchers and had lot of CPU and Memory Bound problems).
tldr; C for web dev painful, C++ with STL/Boost good/easy/fast/clean.
Also I've heard of wt, a C++ web framework although I've never used it http://www.webtoolkit.eu/wt
First, this isn't a troll. The guy posted a blog post on his own blog. He didn't post off-topic content to a thread and didn't try to provoke an argument in an otherwise civil conversation. The only possible way that you can characterize this as a troll is that he uses some crude language, and says some things that are unpopular on HN. But say what you will about the content, he didn't try to make you read it.
Second, the guy has a point. I've had a rant like this building in my craw for a long time, and this guy nailed nearly every point I've wanted to make. From a software engineering perspective, web development is in the dark ages. What's worse, it's a luddite process -- a totally self-made exile from the benefits of 50 years of progress: Type safety is shunned. Language efficiency is ignored. Data structures beyond the hash table are exotic and rare (and worse, I've met more than one "professional" web developer who couldn't tell me the difference between open and closed hashing!)
I understand very well that programmer time is more valuable than computer hardware, and that the speed/development time trade-off is often biased toward development time. But if you ask me, what was once an intelligent guideline is now becoming a cult. It is possible to do rapid development in a compiled language, and there are some really good reasons -- beyond execution speed -- to revisit the idea.
Sometimes the truth is very simple. It can be considered a troll because it is in the tone "if you don't do as I say you are an idiot, like everyone else".
Second: not all scripting languages are interpreted on the fly. In fact, many of the most popular frameworks load objects once and keep their compiled forms in memory. Even for languages that run under VMs, some of them run under JIT that, in the end, results in machine-language-like times.
Third: I have profiled a lot of web applications and most of the time they are either waiting for the pipe to deliver a request or waiting for the database to deliver the data. Very few web applications I know of do heavy processing. For those, I agree compiled code may help.
and, finally, by the time the guy finishes his C code, an equivalent competing application that uses Rails will be approaching the millionth registered user.
Assertion: "Some scripting languages are as fast as compiled languages."
No, not really. I challenge you to prove that statement with a non-trivial benchmark.
Assertion: "most of the time, web apps are waiting for something external to respond"
This doesn't survive a few seconds of thoughtful analysis. If a web app is spending most of it's time blocking for I/O, it's poorly designed. There's no reason that it can't be handling other requests while waiting for a DB response or a pipe. And once you've allowed for that possibility, you're back to a situation where you can handle more requests with faster software.
Empirically, this is obviously true: languages like Ruby and Python don't hold a candle to Java when it comes to web application speed. We choose them because of developer convenience, nothing more. Which brings us to...
Assertion: "C programming is terrifically slow for development. <insert snarky comment about Rails applications with a million users here>"
Ignoring the fact that there are other compiled languages than C (that are probably better choices for web development), I can tell you from experience that by the time your Rails app reaches a million registered users, 80% of your week is going to be spent either working around the myriad scalability flaws in Rails and the dog-slow nature of Ruby, or installing hardware to accommodate the load. Scaling Rails is a nightmare.
Now, it may be true that you can develop that Rails app a bit faster than you could a webapp in a compiled language, but it's a wild exaggeration to suggest that the difference is more than a few weeks of skilled developer time. It's a fallacy to imply that all compiled-code projects end up in death-march hell. Truth be told, the real reason that web apps can be written so quickly is because they're easy, and they tolerate a lot of development error. So long as there are good string processing facilities, the choice of language is a fairly minor point.
Concurrent request processing most likely belongs in the server, not in the application.
"Scaling Rails is a nightmare."
For someone who complains of "well-trodden ground" in others' arguments, you certainly seem to have a deeply-rooted belief in the myth that particular languages and/or frameworks "scale" or "don't scale".
Rails has terrible support for database replication (we've had to roll our own). Its caching architecture is laughably bad, and the implementation was/is buggy (again, we rolled our own), and there's next to no support for the kind of robust page cache expiration mechanisms that are necessary to run a large, dynamic site (Observers are only a start). Rails is a memory hog, takes too long to start, has poor translation and locale support, and simply passing a request through ActionController adds huge overhead that cannot be escaped.
And even though you don't want to hear it, Ruby is slow -- even after working around all of the above, we still had to implement a fast page caching layer just to work around the general, all-purpose slowness of Ruby.
But like I said, write me off if you want.
I find this similar to Woody's fear in the following exchange:
Cliff Clavin: Boy oh boy, talk about your lucky days, huh, I just found twenty smackers back there in the pool room. And to celebrate my good fortune, I'm going to buy a round a drinks for all my friends.
Carla Tortelli: What are you going to do with the other nineteen bucks?
Sam Malone: Woody, that could be your twenty dollar bill, the one you lost.
Woody Boyd: Hey yeah, you know I was in the pool room earlier.
Cliff Clavin: Hey, wait a second. Let's be fair about this. Give me the serial number.
Sam Malone: Oh, come on man. Nobody knows serial numbers.
Woody Boyd: L21886119B
Cliff Clavin: [hands the bill to Woody] That's amazing.
Sam Malone: How did you do that?
Woody Boyd: I memorize the serial numbers on all my currency.
Sam Malone: Why?
Woody Boyd: For just such an occasion. Tell you though, I pray every day I don't get rich.
http://www.imdb.com/character/ch0003010/quotes
Not quite a perfect analogy, but my point is worrying about scaling is a very good problem to have, and is likely to be accompanied by the resources to fix it. Twitter is the poster child for Rails scaling problems, but they re-wrote the parts they needed to write, and people kept using it because they liked the service even when they did have scaling problems.
In any case, at this point I think it's fairly safe to say that I have some direct experience scaling Rails, and it's not as easy as the hand-wavers suggest. It's certainly worth a bit of work early on to avoid the nightmares that come with trying to scale something like Rails later on.
Not to be too silly, but if someone wrote a web framework in C++ that was 10% more difficult than Rails, I'd still consider it. The distribution of suck for Rails is exponential over time, so the expected value of suck is much higher than it initially seems.
I guess I wasn't clear enough. I meant they rewrote the parts that didn't scale in another language. Scala, I believe. Is that incorrect?
So, my point then is that writing it in Rails to quickly get it in front of users and quickly iterating on features turned out to be worth it, even though they had to rewrite large portions in another language once they had a lot of users. Do you disagree? I'm not a Rails developer, but this is my understanding of how Twitter development played out, and it seems to have worked very well for them.
I'm curious, if you would accept 10% more difficult, would you also accept 20%? 50%? I'm wondering just how exponential the suck can get.
If your request-handling logic is responsible for concurrent requests, your application (and the platform it runs on) is poorly designed. This belongs to a thread/process dispatch mechanism in the platform, not inside your app. When one thread waits for the database, I am sure the processors can be busy with other requests.
"Scaling Rails is a nightmare."
No. It's not. There is little reason for it to be. While I am no Rails expert (I use Django and was a Plone guy not that long ago), the scalability issues most people have happen because of RDBMS concurrency much more than any problem with Rails itself. Like I said before, that time spent waiting for the database to respond is time spent in the database working. It is incredibly rare to have a front-end having heavier usage than the database that feeds it.
If you want your applications to scale, design it around the database limitations. Use C, C++ (I would prefer ObjC), Java or Assembly if you want, but faster request processing won't get you past database scalability issues.
Using C* or Java will only push back your release date.
Also you might want to look at the profile of the guy who said that rails scaling is a nightmare. I wouldn't use the word nightmare as I'd prefer to think my professional life story wasn't written by Kafka. But, I don't know anyone who has scaled a big Rails site and thought it was a pleasant or fun experience. Most guys who say that it's no big deal haven't really done it.
The beauty of the ORM is that you can replace the underlying layer without disrupting the code running on top or add fancy things like local memory caching or a networked memcached store. I insist that if your application was built from the start with this kind of issue in mind, it will be easy to scale it if and when the time comes. Until then, you are solving a problem you don't know you have.
This gets harder the lower-level your language gets.
(Or you can use an event model; the point is, if your server is spending all of its CPU time blocked while under load, then something is wrong.)
That evented server is also faster than the threaded version of same.
What's the compute task you think a program is bottlenecking on in an application that does nothing but shovel data out of a database into a socket based on a btree key?
I suspect that you're thinking of something like a lightweight proxy server, that waits for incoming requests, then quickly hands them off to an app server that does most of the work. That's trivial. If you just profile just the webserver process, then yes, you would expect that most of its time would be spent in wait. But while the proxy server is blocked on I/O, the app server is doing a ton of other things for any non-trivial page rendering.
Disclaimer: I don't actually work on web apps, and therefore have no experience with these things.
It is done occasionally though; last I heard, large portions of Amazon.com were written in C.
But for most web apps, rendering HTML pages from script is not the bottleneck, it's the interaction with the database, and the solution is intelligent caching to reduce load on the database. The tools to mitigate these problems are already written in C (memcached, etc).
Luckily rendering scales just by adding more servers.
Rewriting templating code in C though rarely adds more speed, most dynamic languages tune their string handling as much as possible.
Also, utilizing http caching mechanisms are a (quite underused) great solution for lowering load.
It is highly unlikely that happens. Once you lower your DB issues, if the application is responsive enough you start working on other features. By the time you're done the application got new users and the DB is a problem again ... it's a cycle :)
Fortunately HTTP servers do scale by caching and load-balancing.
The author of the post probably never worked in real-world conditions. You first have to have users and real needs for scaling, and that doesn't happen unless you deliver a useful application first, and you also have to deliver it pretty fast, otherwise you might lose to a competitor with the same idea.
Not really. Sending and receiving data to/from the browser becomes the bottleneck.
If it takes 50ms to receive the query from the browser, 0.1ms to retrieve the data from the cache, 10ms to render it, and another 50ms to send it back to the browser - the bottleneck is now network communications with the browser.
There are, unfortunately, limits on how much that can be improved.
An example - from messaging middleware, not web frontends since it's somewhere I do have proper data:
I did a messaging middleware system in C a couple of years back. It's easy. I've done it before. After a while needs were changing and I decided to try writing the replacement in Ruby. It was a lot quicker to write (1/10th of the code size for roughly the same feature set).
We were handling 3-4 million events a day, which could easily result in 2-10 messages each.
The Ruby replacement spent 10% of a single 2Ghz Xeon core handling those messages. Of that 90% was spent in the kernel handling system calls that would've been roughly the same in a C/C++ version.
Even if the Ruby part of the equation was 10 times slower for the userland part, the most we'd have saved for our load was 0.9% of a single core. The most we'd save overall, regardless of scale, would be around 9-10%.
Doesn't mean there aren't things for which a language like C is better suited - I did my MSc. thesis on reducing error rates for OCR, and some of the image processing I did was prototyped in Ruby, but then translated to C for the final version because processing literally would've taken days for things that took less than an hour in C.
But scaling is very often constrained by other things than the performance of the laguage you write your app in. Especially in the web space where most interesting apps are database backed.
If your DB gets maxed out, having a hyper-optimizing compiler is not going to help you. If your app isn't designed to be scale across multiple servers (or at least cores - these days you can rent managed hosting on a 24 core server for less than $1k/month) you could very likely be screwed in ways that no compiler will save you from.
The extra developer time spent doing web development in low level languages is unlikely to be worth it unless you're at a massive scale.
If you can fit in 24 cores with a language like Ruby, your best case saving from switching to a compiled language these days is around $10k/year on hosting if you're ok with a single server, and about $20k if you want redundancy.
That doesn't pay for a lot of wasted developer time. Most web apps can fit in that, and for the lucky (or unlucky, depending on reason) that need to scale beyond that, the CPU time spend on the actual web app is likely to be a relatively small fraction of the total CPU time spent (on databases etc.) - architectural decisions is likely to be a far more important cost driver than the language of the web frontend.
As for developer availability - good luck to him finding good C programmers, and even more so finding good C programmers who aren't expensive and knows web development.
Are you saying it's difficult in general to find good C devs? Because it would seem that's the case for C++, but not for C.
However, most website projects are business-driven, which means that you are constantly changing and refactoring stuff as misunderstandings get cleared up, design flaws emerge, business models change, minds change, etc etc.
Business software projects are not just engineering, and if you treat them as such you will endure pain.
The analogy with a house is odd - if a house was purely an engineering problem, then they would probably all be built out of precast concrete or something, not "wood and steel". Aesthetics, tastes, budgets, and tradition all have an impact, and nearly every house is unique, so you use the "best tool for the job" - the one that is the easiest to customize on-site.
When requirements change weekly and by the projects end your original project description is only useful as a piece of history - then using concrete would mean lots of demolition work in the process.
I think the reason there isn't a "C web development community" is that programmers are more than willing to trade time for performance. Safe, readable code can be written in high level languages in a snap. You can see your ideas happen very quickly. If you run into performance problems (which I think is safe to say is a fairly rare occurrence for the average web developer who produces decent code, in any language), you can a) refactor or streamline what you already have or b) start fresh, lessons learned, in something lower level.
An aside: the site http://halfbakery.com is implemented entirely as an Apache module.
This means that doing the iterations with minimum time and effort are important than worrying about scalability and performance.
You can always throw hardware at the perfortmace problem. If you get enough traffic to become worried about performace, that itself is the mark of success. But to become successful, first you must build the app and get it live somehow instead of worrying about performace issues or wasting months coding in some very low level language!
No, you can't. That's what scalability means. If you don't have scalability, then additional hardware will only take you so far before you begin wasting it.
If you build your app such that the webservers can act independently, then by using various methods of load balancing you can just throw hardware at the problem.
However, this just usually leaves the database as the single point of contention among all webservers. There are various ways to reduce load on the database server by some constant factor (memcache, query caching), but in the end database load will be roughly linear with site load (requests/time). You can't just add SQL-based DB servers and get linear speedup like (properly thought out) webservers.
If you think your site will get so big so fast you won't have time to hire engineers to figure out the scaling problems before they become unmanageable (for example, facebook got huge, but it took a few years, so they had time to raise money and adapt their stuff, although they were pretty slow for a long time), then use appengine datastore or simpledb or some other db that promises to scale really well (this is a really hard problem, which is why there aren't a lot of open source things that do this. Note: couchedb can't scale writes, so I don't count it).
My personal thinking is that one big properly designed database can get you pretty far, and database clustering can get you far enough. If you find it pleasant, putting your app in appengine would make you pay the price of scalability a little at a time starting right away, instead of letting you push it off (but with a huge performance wall that you won't be able to get over without basically redesigning much of your apps internals).
PHP (and Rails/Django, if you ignore their ORMs) encourage you to store state on some collection of backend servers and communicate via RPC to retrieve that. This is bad for performance, but good for scalability. The front-end is just a dumb intermediary that converts data into HTML, so if you need to scale, just add more of them. Scaling the backend is more challenging, but at least you can do it without dragging all the front-end code around.
Having said that, choose the right tool for the job.
edit: Downmod me all you like. A scripted language wouldn't handle 2k req/sec on a couple of VPSs.
People often seem to think a db is the only way to write any webapp, and that everything must depend on the db. That's bad architecture.
In the case of Mibbit, db access is only done for writes, lazily, in the background.
The bottlenecks for me, are CPU, network, and ram. And yes, you can throw servers at it, if you're frivolous and don't intend to make any money. I decided to go the other way and be profitable ;)
As far as relational DBs being the problem, that's more a matter of the free ones not cutting the mustard. Oracle and DB2 can handle jaw dropping load in the hands of an expert.
http://www-01.ibm.com/software/data/db2/express/about.html
For the average web app, it's a lousy point, really (and, as pointed out by others, it's a troll/satire).
The fact that you take it seriously is a little worrying tbh. Are you feeling ok? :-)
Maybe because they don't get past the scaling issues ;)
Why would I want to run and manage and design for a server cluster just because each and every Rails process needs to load the entire language runtime, the framework and all the libraries using 100MB of RAM for no application functionality at all? Not doing all that work is premature optimisation?
Perhaps a new type of web framework is needed to address these breeds of apps.
Granted, it's not from the most unbiased source, so take it with a grain ;)
The point is - for 99% of people it's not the language, it's the I/O
Good bit of satire. I wouldn't call it a troll because it's funny and just unbelievable enough, rather than inflammatory and merely misguided.
I have worked with a project for the past several years that, by necessity not choice, required a web application framework written in C (it's an embedded platform) and I can tell you it is pure hell. The most mundane and simple tasks, often trivial one-liners in scripting languages, require many lines of error-prone C code even with a fairly rich set of libraries available. Working in this environment, you'll quickly realize why the rest of the world doesn't do it this way. Thankfully, this project is finally moving away from this architecture and I couldn't be happier about it.
It's also a pretty good scripting language on its own merits, and has replaced Python as my default language. (Unlike Python, its design expects you to know at least a little C, but in exchange, it avoids cluttering the language with things that C already does well.)
It's not clear from the post which end of the spectrum he or she works with. (I don't work with embedded systems professionally, I've just tinkered with OpenWRT, cell phones, etc. as an enthusiastic amateur.) It would be usable in the same niches that SQLite is, for example.
What would be an example of this? I'm mostly a C and Perl person, and once a library is available I've never noticed cases where something was much simpler to write in one than the other. That library might take more time to write in C than Perl, but once it's written I find it just about (less than a factor of 2?) as easy to write the application logic in C as in Perl. My instinct is that if you're writing 'many lines of error-prone C' to do something simple, there's probably room for improvement somewhere.
Only recently they decided to change it to a must robust Erlang based implementation - http://www.37signals.com/svn/posts/1728-nuts-bolts-campfire-...
No matter how many rps your app can serve, if there is no app you get no requests to serve...
Web front-ends scale very easily. You can just add more web heads and load balance between them. Databases are much harder to scale since you need to keep data consistency and can't just throw more boxes at the problem.
So, you don't need to be computationally efficient. However, you do need to develop fast. You need to be able to create features before your competition. You need to be able to make your system more reliable and bug-free than your competition. And that's where Ruby (and similar languages) helps you.
Sure, you and Linus have a point that selection of language can keep out the riff-raff (Linus Torvalds having a huge rant against C++ simply because of the programming community), but it also means that you're missing out on the good parts of the community. Ruby programmers tend to put a lot of thought into user experience. While you've ranted against testing, I don't think any IT manager would agree with you there. Testing substantially improves one's ability to create new features because you can easily spot anywhere you've broken something. And that leads to better quality applications too.
There's a difference between speed and scaling and if something scales fine, that's ok. Look at it this way, array indexing in a slower way to go through an array in C than incrementing a pointer. And if you had a lot of code you could go through and change all instances. However, it wouldn't be worth your time. Such an improvement wouldn't net you any real benefit while you'd be neglecting improving your product.
Your product is what matters.
You just can't do that in compiled languages, and the compiled language of choice for the web-- Java --isn't known for blazing compilation speed. You'd probably also have to jar it up and copy the damn thing over, whereas psps could be modified in-place. I think this is something that compiled folks like me simply failed to grasp: all the things about scripting languages that make them orders of magnitude faster for developing, especially for the web.
When I do Java webapp development in Eclipse, it compiles and hot-swaps my code in the background without requiring any restart or redeployments. Works pretty well.
I do have to restart occasionally because the hot-swapping has limitations, so that does slow things down a bit, but when you consider the time saved by having good compiler integration, then I don't think there's a big difference in productivity.
That said, I don't need no stinkin IDE integration for .psps. Being able to log in remotely and edit something with vi is kind of handy.
I had that basic idea when web app frameworks starting becoming popular. So I implemented a web framework which took a 'description' of the app written in perl, and generated C++ which compiled into a single exe file which served the app with a built in http server. Small bits of business logic were written in C++, but basically it was done by the perl. All templates, internationalisation, everything, compiled down to a few database queries, string concatenations and the odd if statement.
I then build a couple of production sites in this, one on my own, and another with a few other developers.
Conclusions from this:
1) It's really really fast and efficient, and the single exe model for deployment is lovely.
2) It's a pain to implement such a framework, and a pain to use it.
3) I'm never using perl again.
4) Life's too short for this kind of game.
The code is here: http://www.boxbackup.org/trac/ticket/6
You should give it another try sometimes. Take a look at Catalyst (the web framework), DBIx::Class, Moose and POE.
Everything computer-related sucks, you just have to choose something which sucks the least where it matters for your project. And familiarity counts for a lot too.
+1 for "academic idiots"
+1 for "improve our democracy"
+1 for "circle jerk conferences"
+2 for "scumbag just oozing hubris"
I just hope his c isn't as poetic as his prose.