208 comments

[ 4.4 ms ] story [ 245 ms ] thread
The guy behind PlentyOfFish (the dating site) has written similar things in the past. Various links and tech info: http://highscalability.com/plentyoffish-architecture

It seems a lot of MS stuff, used properly, scale well and easily, making it particularly suitable for single founders or small teams where they don't want to devote a lot of time to ops (while still keeping it in-house).

Becoming a member of their "BizSpark" program literally provides you with all the software free of charge. Entering BizSpark is easy and costs nothing.

If you can get in, of course, which is far from certain. My company seemed to fit the criteria well a couple of years ago but was rejected without explanation. Of course, now we are profitable enough to afford any software we want but have stuck to open source as, well.. that's what we built our stack on instead ;-)

I got into it with my game company but once you graduate, then you need to start paying up. Maybe it only cost him a couple hundred bucks a month now but once he needs to pay for the licenses then that's when he can judge if it's really worth it. Until then, he's just getting stuff for free and saying how great it is.
What if... the linked blog post is worth 1 extra year of free licenses and was commissioned by Microsoft PR.
Don't forget StackOverflow: http://highscalability.com/blog/2009/8/5/stack-overflow-arch... - apart from the somewhat unbelievable PlentyOffish project, it's the only prestige project based on MS software most people will recall. In their 2011 update, they claimed to be using "a lot more Linux machines", because apparently it's fun to develop using Microsoft technology, but for some infrastructure components (proxying, Nagios...), there's no way to beat Linux even if you're a MS fanboy. ;-)
Plenty Of Fish shouldn't be considered unbelievable. I've been following information about their infrastructure from the first moment they posted about it. There's nothing unbelievable about it at all. Over the years he has openly described changes to the site's setup as they've scaled, and it makes fine sense.

Stack isn't the only prestige product. Nearly all of Microsoft's infrastructure runs on their own dog food. That's a lot bigger of a product than Stack Exchange is.

Azure offering support for Linux shows how much faith Microsoft have in their own crap.
>It seems a lot of MS stuff, used properly, scale well and easily, making it particularly suitable for single founders or small teams where they don't want to devote a lot of time to ops (while still keeping it in-house).

This is the thing, I've regularly scaled MS, almost RAD tool technologies to multi-thousand request per second scenarios. Normally for in house things, so I've had the benefit of warning for demand.

But it gives me the nicety of fast to dev tools, the fact maintaining C# code is almost pleasurable at times, with good performance.

Sure some custom C in Linux kernel land system will be faster, but the price performance is really good.

Comparing MVC 4 to Ruby on Rails and Django for a new project, the performance of MVC4 was an order of magnitude better under load.

But at the last HN meetup I went to in London they were taking an informal survey of languages. Despite having Erland and Lisp mentioned, the hosts didn't ask about C# or F#, despite my knowledge of at least 10 people there heavily in to it, I'd have guessed at least 30 people must have been.

You also get some bizarre comments, a friend of mine who spends most of his time in just one nameless language, half ask/told some people that Asp.Net doesn't scale... Some people have a religious zeal against it I think. Which is a shame, C# is a great language, and if people want to 'stick it to MS' go contribute on mono instead of bitching about MS!

Stack overflow was another great example of hi load project on .net
How much of this is old-school admin practices vs. the new "cloud" philosophy of spinning up lots of boxes that you don't care about? I don't see a Microsoft vs. Open-source argument for alot of this stuff.

Many of the devs I see working with "cool kids" tools don't know anything about operating IT environments, yet they are playing admin. They're spinning up dozens of single-purpose VMs, sometimes to perform trivial tasks.

The motivation is often to reduce the need to have dedicated admins ($$$) and change process (time). But... managing dozens of VMs ain't free either!

>...managing dozens of VMs ain't free either!

It's not free, but configuration management tools make it pretty cheap. There's a reason Puppet/Chef/Ansible/Salt are getting a lot of attention in the DevOps world.

This--and by "this" I mean cultural or philosophical differences--is what I feel is ultimately behind the stark performance gulf described in the OP's article.

I love Microsoft's technologies. I use Windows on my development environment (even though I don't do Windows development; I know, I am crazy). I own a Surface and a Nokia Lumia phone. Nevertheless, I also know that there's nothing absolutely fast about Microsoft's web stack [1].

I suspect what is at play here, instead, is one side being efficient with resources and the other being reckless.

My opinion is that some platforms and frameworks come bundled with a set of philosophical characteristics that most practitioners adopt. Some popular open source web frameworks include a poster of Knuth's famous quote about premature optimization and adopters consider that quote to mean more than it does, thereby expressly avoiding spending time thinking about performance because doing so might lead to time spent on optimizing code and that is evil.

Further, in our efforts to capture production-grade deployment instructions for many open source frameworks, we were left wanting, often having to turn to community opinion rather than specific instructions. Many frameworks' communities were Balkanized, leading to uncertainty about the best performing production-grade configuration.

It's not surprising when systems get built and deployed without any performance benchmarking, let alone tuning. I have no idea what "Charm" was or if it suffered from this sort of thing, but I've seen the described situation before and suspect it's fairly common.

Meanwhile, although I've not been a Windows developer, I suspect their culture comes bundled with a modest helping of being mindful of CPU and disk resources and preparation for production deployments. Microsoft is too friendly with corporate IT to leave production deployments a guessing game.

Finally, perhaps the OP in particular is just especially careful about performance, perhaps without even consciously making it a priority. Certainly none of us think it should be especially difficult for an m1.large server to process hundreds of thousands of dynamic web requests per day. It's more instructive to me, frankly, that we have come to think that is noteworthy.

[1] http://www.techempower.com/benchmarks/

It's not just a cool cloud thing. Managing multiple services on a single host comes at its own price too. Trying to manage security separation, disk space, specific user access, projecting bandwidth / cpu / memory / disk usage in the future, figuring out which services cannot run on the same host for yet another set of reasons. Then you add HA for only one service and end up with host A with services X,Y,Z and host B with X,P,Q.

There's a lot of great things about being able to say - these are the two DNS boxes. They pretty much serve static data and have minimal traffic - stick them on two minimal instances which cost us ~0, but give a very high isolation from other failures.

It depends on the technology stack that you're using. If your apps are .Net/CLR-based, it's very possible to get the separation that you're looking for. On Unix this is done all of the time with JVM-based apps as well.

End of the day scaling horizontally or vertically requires real engineering and operational discipline. I am seeing much more sloppy horizontal scaling than I used to, because the marginal cost of a VM is so low.

The difference is, when scaling up everyone intuitively understood that many services on one OS instance introduced risk. When scaling out, I think people with weak ops backgrounds intuitively feel that the OS instances are giving them a security barrier. That's a dangerous assumption!

The simplicity of spinning a VM is obscuring the complexity of the system as a whole.

> If your apps are .Net/CLR-based, it's very possible to get the separation that you're looking for.

I'm not sure I follow. Unless I'm missing something, the only thing this separates is the memory between applications and optionally user privileges via the standard permissions / security domain. Other resources are completely shared - that is, one app can fill the disk / bandwidth / server thread pool / ... of other apps, right?

One server? Not much redundancy in that setup. It's no wonder your costs are so low.
BizSpark is kind of awesome for startups. Even if you're not "Microsoft-based" you can still take advantage of their free software (Windows, Office etc.) which otherwise will cost you a small fortune.
I'd consider Windows more readily, but you are a bit screwed if you do decide you want to dip in the Open Source pool. You can dip there, but it's harder and less well supported, and some things simply don't work. So you are tying yourself to Windows, and often, Microsoft tools. Fine if you're sure it'll work, but I would feel like I was gambling.
We're building a web-based saas on a Microsoft stack. We use coffeescript, sass, compass in our development environment and it works really well. There are open source libraries for doing REST, unit testing, performance testing and they work pretty well. Node.js and npm run on Windows. Mercurial always has and git now works as well.

What are the components that you miss?

It does sound a bit as if you were using a node.js stack on top of a Windows operating system, which I wouldn't exactly call a Microsoft stack. In which case there is really just the question of what you know better, linux ecosystem or Microsoft...
No, we're are using a Microsoft stack - ASP.NET MVC and Sql Server.

We're not actually using Node.js in that sense. I was just pointing out that it works well (we use Node and npm to run the coffeescript compiler - that's it)

>"But most importantly, because the customer doesn't give a darn. My clients don't care what I'm based on as long as it works. So when choosing a platform for your great idea - just go with whatever platform you're comfortable with and start hacking, stop wasting time."

This is really the best advise. What you actually use is less important than you being comfortable with using it (assuming it is not something totally bonkers). The customers want results, they don't care if you wrote it in Ruby or .net. They won't know the difference anyway.

While Microsoft might be a good solution for many people (Use what get the job done), this specific article has absolutely no real reasons why you _should_ use Microsoft. The truth is if One 8-gig windows server is what he needed for his start up he could get by with one 8-gig Linux server and save a couple of bucks a month. While that might not be a big deal now, if he ever wants to get big, those little costs start to add up.

Also while its great Microsoft is subsidising his starup via the Biz Stark program, we have seen many times what happens when you rely on "free" to run your buisness. Even tually he will need to start paying for those software licenses he is getting for free and by then he might not have a choice in what platform he is using.

(comment deleted)
I agree, he could easily be using a JVM-based stack to get the same or better efficiency out of his 1-server configuration.

Also there's some clearly naivety when bragging about his "low cost" $200/month setup. His SQL Server licenses alone will probably triple or quadruple his costs when his Bizspark status runs out.

Well, AFAIK he can easily switch to MySQL for instance, under the same ORM.
MySQL is possibly the single hardest database to port to/from, because it supports nothing.
In that case porting FROM it should be simple. TO it would be hard if you're using all those fancy features of your current DB. That's why I eschew all the sirens from Oracle DBA's of "oh... but Oracle can do that too!" Keep database activities in the database and application activities in the app.
In my experience, MySQL lets you get into a state where questions like "hang, on - what's the difference between a DATE of '0000-00-00' and NULL?" are commonplace. This makes porting from it... not simple.
(comment deleted)
Although I have no experience with MySQL, this kind of problems have to be resolved by using LINQ to SQL since every db type is automatically mapped to its .NET counterpart. The DateTime .NET type then has only one representation of null.
Or what you're saying is that if you store a date of zero VALUE to MySQL it could be magically replaced by NULL internally? That would be really crazy.
I seem to recall it is actually the other way around, and storing a null results in a nonsense date.

Added bonus: some versions will segfault if you pass an un-formattable date value.

The problem is that MySQL deviates from the standards (all of them) in surprising and uncomfortable ways. porting from MySQL is fun, when, for example, your target database actually ENFORCES the constraints that MySQL accepted but silently ignored. MySQL doesn't have that feature, but it won't bother to tell you.

Postgres, Oracle, MS SQL all support a (very large) common subset of features, so if you avoid stored procedures and db-specific features, you can port among them easily. This is not true for MySQL.

>"I agree, he could easily be using a JVM-based stack"

Maybe but believe it or not, there are people who HATE Java (that would includes me)

I'm not a huge fan of Java either, but sometimes it's important to get past that and either try and work with a language that's extremely effective for some deployments, or consider that the JVM supports quite a bit more platforms than just Java.
There's always Scala, clojure, groovy, etc.
Oh you're not alone. I dislike Java as well, but there's no denying the power of the JVM and the ecosystem around it. If you really dig into it and look at it as a polyglot platform with choices like Groovy, Clojure, JRuby, and Scala, it becomes more palatable.

Regardless this post is not about .NET tooling, more about server efficiency and cost. Two areas where I think the JVM is a little further ahead.

>"this specific article has absolutely no real reasons why you _should_ use Microsoft"

He does give some reasons, which might not apply to everyone is another thing: (from the article)

    Because I love C#.
    Because "MS is expensive" is a myth. 
    Because of Visual Studio


Of all the three I am very agreed with the third one. I wish it would be different but I haven't used a better alternative to VS so far (XCode, Eclipse, Netbeans, etc. just didn't work for me for several reasons )
Wow I hate VS so much, Eclipse is so much better! I guess it's just personal preference.

I like C# a little more than Java, but I cant stand assemblies. Jars are much easier to handle.

edit:

Troll? Not.

Like a very simple thing: quick watch in debug blocks the edit window? Really WTF?! I have to move it the second screen to see the actual code. I can name a lot of these small annoyances.

I can't help but feel like you're trolling. Eclipse is a slow, bloated, feature-laking pile of crap. I honestly can't believe anyone would prefer it to, well, just about anything else. I've used it and I get by on it when necessary, but in terms of preference its at the bottom of my list.
> "Eclipse is a slow, bloated, feature-laking pile of crap."

You should try Kepler. Some amazing performance improvements out of the box.

Rubbish. Use the latest version with an SSD with 8GB ram and it flies - SSD and ram are cheap enough to not be a barrier to entry, either. It's got more features than any other IDE I can think of, as well as a pretty huge library of plugins. Sounds like you're the one that is trolling. If Eclipse really was that bad, nobody would use it, and it wouldn't be probably the most widely used IDE in the world.
>If Eclipse really was that bad, nobody would use it, and it wouldn't be probably the most widely used IDE in the world

I don't think this means much. It has (poor) support for just about every language under the sun and its free so of course its going to be widely used. It has many features for Java, but features for other languages quickly fall off a cliff. But its the slowness that is a showstopper for me unless I absolutely must go with eclipse. Yes, eventually computer technology will advance to the point where its bloated codebase is no longer a liability, and perhaps we're approaching that. I use the Juno release occasionally for python programming and everything I've mentioned still applies.

Personally I'm not a big fan of plugins in my IDEs: they always feel slow and clunky with a "bolted on" feel. This has been true of every IDE + plugin environment I've tried so far.

>Troll? Not.

If you had said IntelliJ I might have believed it.

Excuse me sir or ma'am, do you have a moment to talk about the Good News that is IntelliJ? :)
And the list seems to be missing the most important point.

    The .Net Framework is extremely powerful and extensive.
The number of problems the framework has prevented me from needing to solve is staggering. We get work done here, fast.

Eg. Recently I had to convert times from any random time zone into another random timezone accounting for daylight savings etc. I cannot imagine having to solve that problem in just about any other language suggested in this comment section.

Anything with access to jodatime (or nodatime, for that matter) would find that a pretty trivial problem.

In my experience you get more built-in with .net, but richer third-party ecosystems elsewhere.

To be fair, the post is titled "My start-up is Microsoft-based..." not "Here's why your start-up should be..." I feel like he just wanted to make a fair case that Microsoft isn't impossible to deploy as a viable solution for a start-up. In start-ups versus say...a corporation , there is a write-off of Microsoft pretty much out of the gate. I agree with most companies, there are certainly a stack of reason why not to use it. But, its nice to see someone putting some personal reasons out there why Microsoft tech isn't useless to a new, fledgling company.
All our stuff runs on one (1) server

This has nothing to do with MS, but why would you set up such an architecture? If one of the apps hits a memory leak (their HeatTest project did), than all of your other critical services/apps are interrupted.

That costs us about $200 a month. By the way, that's Amazon

What Amazon configuration gets you 8GB or RAM for $200/month?? Am I missing something?

Database servers for all of the above

MS SQL licensing costs are crazy stupid. You either (a) have to fork over about $2-3k fee or (b) the cloud based cost is an extra ~$150/month on top of the standards windows vm cost.

I'm genuinely curious about all of the above, because I am working with a MS-based startup and would be good to know.

But most importantly, because the customer doesn't give a darn.

100% agree.

Exactly. If he was paying list price for any of this, there would be a sudden, dramatic increase in price.

Which will happen, when BizSpark runs out.

Keep in mind that BizSpark "runs out" when you pass a million dollars per year in revenue. At which point you can presumably afford a thousand dollars for a SQL Server license.
There's also a cap of 3 years, even if you haven't hit $1m revenue yet: http://www.microsoft.com/bizspark/faqs.aspx#16
Fortunately, they reinvent the program every few years. So in reality you can be in EmpowerISV for 3 years, then BizSpark for 3 years, then [thing that replaces bizspark in 2/2014] for 3 years, etc.

I've personally been in Microsoft's "let's give developers everything for basically free" programs since 2006, and am not worried that I'll suddenly face a crippling bill down the line.

Shafting little startups for a lousy grand or two really is not their business model. Think about it for a second and it'll make sense.

EDIT: To be clear, I'm not advocating gambling your business on Microsoft's goodwill here. For the same reason I don't advocate building a business that a single bill for $1,000 can kill.

The worst case result of MS pulling the rug on all their BizSpark-esque programs is an extra couple thousand dollars a year in the expense column. Here's hoping nobody here works for a shop where that would be a big deal.

What? You're basing your business continuity on a corporation creating a special 'free pass' for you every 3 years explicitly? Because they've done it once before? This seems like a terrible idea. Godspeed man, godspeed.
One could just as easily scoff at someone for basing their business off of "open source community supported" tools.

But, instead, the realistic position is to understand Microsoft's tools and programs will be a decent choice for years to come, just as using FOSS would be.

How is that even remotely comparable? I'm not saying anything about the quality of using MS tools - they're great - but they have a fee attached. They specifically say you will need to pay that fee within X years. Pretending the fee doesn't exist because you don't have to pay it until next year is what I'm scoffing at. Not the tools, the idea that MS will continue to make their expensive tools free when they have put an expiry date on that free license already.

Open source community supported tools are completely different. The price and license are very clear - $0 up front, pay for support if you need it. Don't want to pay for support? No problem. Need a new feature? No problem, pay someone regular rates to add it. You can't lose access to anything because you already have everything. You can't be sued into paying because the GPL specially disallows the creator from suing you.

We might be talking past each other here. No one is saying that it's a good business plan to assume that programs like BizSpark will never run out. That is of course a bad idea. But since no one suggested that, I assumed you were saying that using things like BizSpark at all is a bad idea.... which is not the case.

A valid business plan can be created that involves the pricing structures that Microsoft gives. You can have a plan that involves using a program from MS that gives you their software for free now, with the idea that you will eventually be able to afford to pay for them. Any company worth existing should be able to afford it eventually.

And, hey, look, it even happens that those windows of time can be extended. No one is saying that it would be a good business plan to rely on programs like BizSpark never ending... but it's still worth mentioning that MS has been forgiving if things don't go exactly as planned, and provides a way to delay the fees kicking in.

That sounds like relying on the kindness of strangers to me.
Isn't it what you do when you use a free software stack?
The difference is that in free software, anyone can step up and maintain. Relying on the kindness of Microsoft is a much scarier proposition.
In theory, maybe. But, let's say you want to go maintain X11/GCC/Qt (or even xterm), I doubt "anyone" can. Some people are full time developers on these projects, paid by some company we somehow rely on. You cannot just replace an expert working on such a big piece of software.

Of course, using free software might widen your "trusted base" compared to the situation where you go full MS.

No. Free software doesn't suddenly require payment 3 years down the line to remain legal to use
This is true, but you absolutely cannot build your business on 'Something will show up to keep us going in 3 years'.
As a graduate of the BizSpark program, $100 gets you full license to the software you were using. Software licenses are then transferred to your organization. You don't suddenly incur $1000s of expenses.
IIRC, you get to keep the existing licenses and only need to pay for new ones.
> What Amazon configuration gets you 8GB or RAM for $200/month?? Am I missing something?

You can easily get near 8GB for way under $200/mo using reserved pricing. A one year reservation, heavy utilization m1.large gives you 7.5GB of RAM for $40/mo and $676 up front, or about $100/mo total.

Wait, am I an idiot then? I thought $200/mo just gets you reduced per minute rates? So ~720*0.00x + $200 = $y/mo. As far as I know, there is no way on Amazon to pay a flat monthly fee. It's all "pay for what you drink".
I think you actually understand it just fine but might be confusing yourself a bit.

Paying the reservation fee does just get you reduced per minute rates, but the fee is not $200/mo. The reservation fee in this example is $676 for an entire year. Once you pay that, your hourly rate for a single m1.large goes down to $0.056. My final $100/mo total is based on 24/7 uptime on that single server you've reserved.

Oh sweet. Thanks for the explanation!
Yeah, 8gigs of RAM on ec2 works out to about a m1.large? I assume they're running SQL Server Standard Edition (do Saas apps like this one qualify for web edition?)

Shouldn't that be more than $200 a month?

$200 is a m.large instance (7.5.GB) plus the "provisioned i/o" charges (EBS volume).

MS SQL is free since we're in BizSpark.

> "[insert technology here] is free since we're in BizSpark."

For how much longer? And then what?

http://aws.amazon.com/ec2/pricing/ There are tabs for SQL Server Web and Standard Editions.

Depending on what version of SQL Server he needs, it's either a $0.136/hr additional cost to add Web Edition or $0.610/hr additional cost to add Standard Edition. That works out to an approximate $97.92 or $439.20 additional cost per month. So, not a huge jump in cost.

What about the concerns with running your apps on a single server? Or do you Hyper-V within a single instance?
but one instance on Amazon is not high availability.

The rest of your post could be read as apples with oranges, Charm was expensive because they made massive architectural mistakes. End of story.

This is not to say that what you are doing/saying is incorrect. Your stuff works and is able to cope with the load thrown at it on one server.

To be honest I'm quite mystified why startups boast about using n instances to serve a few thousand hits. Node, some python frameworks and ruby are ridiculously wasteful[1]. All it tells me is that you've not though about the best way to do things, you've just used the latest tools and smooshed them together. If you had to have more than one server to handle 2[2] tickets a minute and 1 hit a second on your website I'd be questioning your programming skill

The one thing I do take issue with is your implied claim of high availability. Hourly backups, and no hot spare or active active replication is not high availability. Yes you can rebuild an instance in 5 minutes, but only in best case, and you've still lost up to an hour's worth of user data (around 60 tickets). That is a commendable recovery rate, but its still downtime.

[1] depending on usage, of course. [2] using stats from the article

I'm a little late to the party, but can't believe so few people brought up the lack of availability admitted in this article. The OP is admitting that the company's official stance of availability is that it will restore from 1 hour ago ... and lose client data.

Losing client data is the worst thing you could do as a SaaS. The second worst thing you could do is to not be online - e.g., as someone pointed out; because one app (or even something out of your control) took out all your services.

MS is not really the story here. Cost of HA in typical Linux vs. MS stacks, which wasn't addressed at all in the article, is what is relevant.

>> All our stuff runs on one (1) server > This has nothing to do with MS, but why would you set up such an architecture? If one of the apps hits a memory leak > (their HeatTest project did), than all of your other critical services/apps are interrupted.

It's hard to believe a company would "admit" to running all of these services on a single server. There's no way I would pay for a CRM or Ticketing App (relatively mission critical for many companies) that wasn't highly-available.

> It's hard to believe a company would "admit" to running all of these services on a single server. There's no way I would pay for a CRM or Ticketing App (relatively mission critical for many companies) that wasn't highly-available.

One also has to take into account how many systems that are distributed over n servers have actually n points of failure. My estimate would be "quite a lot".

Very true. It's expensive and complex to have real HA/redundancy, but they can't even bounce a DB server without taking down every one of their apps. I have no idea about administering Windows servers, but based on my experience with Windows Update on the desktop, I would imagine that you need to do that relatively often.

Hopefully they have a staging environment running somewhere internally at least.

(comment deleted)
I'm not sure why anyone, with the technology options available today, would choose a single monolithic server approach and scale that vertically.

I think this is just poor design and probably not an MS stack issue. But could this be an MS community thing, where they generally prefer proven technical solutions from 2001?

Possibly because it's actually pretty cost-effective. You need at least two machines for availability. But you get more bang per buck by making those two machines bigger than by adding a third.

Until you hit the limit of what you can do on commodity-ish x86 servers, and would need to start buying Gucci kit from Sunacle or IBM or whoever, at which point your CFO is going to start crying like a baby.

This may be cheaper up-front, but scales terribly which will increase costs exponentially whenever there is a need for growth.

Additionally, like others have mentioned, a simple memory leak from one app will affect every other app on this box. Also, it doesn't sound like there's any redundancy. What happens when the HD blows up?

If they fired these app up as separate VMs, they could at least scale each to what their needs are and only pay for what they are using.

I manage an app architecture that consists of several Heroku apps along with a couple DBs at mongoHQ for under $100/month. Heroku definitely isn't the cheapest solution, but it allows me to scale horizontally and takes the massive load of sysadmin tasks off my back so I can focus on development.

Because the original poster shouldn't need more than one server to deal with 1 ticket a minute. (1500 tickets a day) you should be able to grow that by 1000% and still not break a sweat. In fact it should be able to cope with 1 a second.

In reality most people are never going to deal with that level of scale.

BizSpark is fantastic, not for the free MS software but for the $150 a month of free Azure use for 3 years. For my bootstrapped project which has a significant server-side component it has removed my only monthly expenditure. There is also no compulsion to use MS products or to develop for Microsoft platforms either as i'm using Azure to power the Linux based backend of my iOS app.
I don't want to bash microsoft or anything. But 2.5 daily visitors and 1.5k new tickets daily can probably be served by nginx, a raspberry pi and a potato.
500px hosted 10x that on a Mac mini with a USB hard drive when they were starting out.
thats a really bold decision for a photo hosting site. has anyone coined the term "minimum viable hardware" yet?
They give 1.6 volts each. So you will need 3 of them for the power source and you may have a problem with amperage.

But yeah - people seem to gravely underestimate the power of the current hardware and how much of it is wasted currently.

I would rather use a lemon battery. It lasts longer and I won't have to think about redundancy for a few... seconds.
Wouldn't you be doing something horribly wrong if you couldn't serve 70k requests a day? It's actually less than 1 a second: 70000 / (60 * 60 * 24) = .81

I think this article is more about distributed systems being costly (yes) and perhaps overly complicated for the types of applications most people build. I agree. However, you could easily do the same with Linux.

you can serve 70k/requests day on a 386 from 1989.
Requests to a typical smallish site will not be uniformly distributed--not by a long shot. Given that the peak is what matters, if we might get a day's worth of traffic in a single hour, that's 19 requests per second instead of 0.8. Those 19 transactions per second will still sound tiny to many people, but a couple months ago we heard that Mt. Gox could only do 37 TPS. So 70k requests per day is not nothing, especially assuming non-uniform distribution, minimal effort to optimize performance, and cheap hardware (or virtual hardware).
On my home dev machine which is nothing to brag about (i5-750 overclocked to 4.2 ) golang was able to process that per second on a single core if you keep the static stuff prezipped in the ram.
80% of my costs come from securing client data and having a HA environment. We were using Oracle DB for many years, switched to PostgreSQL last month due to usability problems, not cost.

I hope you make clear to your users that there will be data lost (to the point of your last backup) in an event of failure (which is certain thing in AWS).

If you think Multi-AZ will save the day: we lost a whole region of data in 2011 and again in 2012. If we didn't have a slave node on another region, we would have to restore a 13 hours old backup, that would pretty much make us go bankrupt.

I'll pluralize this anecdote.

Every bit of my little software empire runs on a single Windows box. That includes my 3 main money-making products, a couple dozen smaller sites being hosted for consulting clients and my partner's businesses, and the Expat blog. Lots of steady-state traffic to lots of sites, several Techcrunch, Reddit and HN poundings, nothing to write home about as far as downtime or drama in the 6 years it's been running.

Say what you like about Microsoft, but their server stuff just plain works.

Just out of interest - what's your plan for when that host dies? (complete disk corruption) How quickly can you recover to the exact same state?

(I don't mean getting the data back, I mean actually having your software empire back online, serving live traffic again with minimal data loss)

Restore from yesterdays backup onto the machine below it in the cage. Then buy a new machine to replace it.

That 2nd machine is already running all the same sites (as a QA/Staging server), so it's pretty much good to go already.

Sorry if I wasn't clear that "running everything from a single server" doesn't equal "only owning a single computer".

I've worked in IT for 12 years. Microsoft server stuff never just works. I'm in the process of replacing a $1 MM video asset management system that sits on top of Windows Server with OpenStack/Debian precisely because of this.

Microsoft server just works. Hah!

Meanwhile millions of businesses are able to get it to work quite well. Ever use StackOverflow? NewEgg? Their server and tools revenue continues to increase by double digits. You think that's all media manipulation? Go search Dice for for the number of openings.

Just because someone was not able use the right tool for the right job in your instance doesn't prove anything. One of my friends used Windows Server to stream popular sports and got so much traffic that he got shutdown by Dish.

Wow, great. Stackoverflow and Newegg.

Amazon? Linux. Google? Linux. Apple? Linux. Large Hadron Collider? All. Linux.

Using Microsoft server in a production environment? Gonna have a bad time. There's a reason most sane people don't use it in important places.

I don't hate on Microsoft server products because its in fashion. Its because I like to sleep at night.

It would be more enlightening and people might agree with you if you told us what was going wrong to cause you such pain?

I have had many windows servers and often you can almost just forget about them entirely once they're setup, which takes all of like 2 hours even if you're using a whole new stack.

Excellent point. I apologize for my brevity in my previous posts.

I agree that you can't color Microsoft's server offerings by their older products (Server 2000, 2003). I do agree that 2008/2012 are much more solid products comparitively speaking, but I still would not use them in a production environment for several reasons:

Debugging: Have you ever attempted to debug edge case issues in a Windows server environment? I don't mean ASP/.Net issues, but I mean issues within Server itself (random reboots, OS hang, etc). Sometimes, your only answer is "Windows does that occasionally". I have Linux servers with uptime measured in thousands of days, almost as stable as the solid state networking gear.

Cost: I worked on the CMS detector for data taking at the LHC. We had over 6000 servers, not only doing data crunching but also providing REST services, database services (for both data and parallel file system metadata). Everything was Linux. The licensing cost alone for just the Windows OS on all of those machines would have been massive.

Patching/Upgrades: The only time you need to reboot a Linux box is when you're upgrading/patching the Kernel. Windows boxes? Patch Tuesday. Rarely have software updates broken a Linux box, and if they do, its much easier to revert back to a previously known good config.

I can go into greater detail if necessary. I agree, 12 years of experience is just a drop in the bucket, and my experiences are anecdotal in nature. I just rarely see (if ever) someone using Microsoft software in missions critical environments.

I'm curious what the specs were on those 6,000 CMS boxes? Do you happen to recall?
I don't, but I could probably check my email. I was responsible for benchmarking boxes from major manufacturers for the selection process (which ended up being a waste of time, because only one of the vendors met our price criteria).
Amazon and Google write their own custom code. Hell, all of the major "linux" users have hand-built infrastructure that they don't share with the open source community.

Stackoverflow and Newegg use a vanilla Microsoft Stack.

> Large Hadron Collider? All. Linux.

True if you are speaking about the custom built machines doing real time data analysis.

However, many users have Mac OS X and Windows as their desktop machines.

When I was there (2003-2004), quite a few researchers were even moving away from LaTeX to Word and FrameMaker.

There are a few ASP.NET servers at CERN.

I was at Tier1 (Fermilab). All the servers were Linux, and everyone's desktop was either a Mac or a Linux box.
Atlas HLT EP group.
Yeah, gotta say if the world can make it work, but you can't, must be Microsoft's problem, right?

I'm guessing Microsoft isn't your problem.

I've worked in IT for more than 15 years. More often than not, Microsoft's enterprise products do "just work." In fact, due do volume pricing (making server 2008/2012 licenses very cheap), around here I recommend Server 2012 w/ standalone LDAP role when people need an LDAP server. Why? Because it just works.

An a related note, I, too, am in the process of building a new video management system based on windows (why anyone would want to sit and stare at solar PV panels all day is beyond me). Install and set up has been a breeze.

Don't get me wrong, I'm more likely to use linux for things. Most of my development is done on linux, our video wall display management, AMI emulation platform, any many other tools and utilities, but MS does have its place, and where it fits, it usually beats everything else in set up configuration and management.

Have you ever had Active Directory go sideways? I mean, really badly? In a large (100+ employee) setting? If not, please don't say "it just works", especially when you're calling to get "enterprise support" from Microsoft and not even the engineer you're on the phone with can explain why your configuration is failing.

Microsoft might have had its place. On the desktop. 5-10 years ago. But for environments where you need uptime? No. No. No.

EDIT: With regards to your video management system, does it have to be broadcast grade? Mine does. The FCC doesn't let you explain away fines for not being on the air "because the Windows box crashed".

100+ Employees? Well, my current environment is about 500,000 users across 3 campuses, and 4 domains. Throw in federated authentication (shibboleth), custom apps, *nix auth, etc.

With that in mind: It just works!

With any large deployment (100 users isn't large) there are bound to be difficulties. That isn't the exclusive realm of Microsoft. As mentioned elsewhere, have you ever done a large deployment of kerberos? I have. I'll take AD on the infrastructure side any day. "Sideways" AD is a whole lot easier to manage than a sideways MIT Kerberos5 KDC.

Microsoft does have its place. In the enterprise, in the server room, today.

Agree'd.

Anyone who doesn't agree, hasn't tried to setup the Kerboros LDAP crap on Linux.

Kerberos and LDAP are two different things. You are aware Active Directory is just LDAP with extensions, correct?
Not really. AD is krb5 mutual auth backed by an LDAP datastore. LDAP is also used for storing various bits of other metadata. You can (with much pain and sadness) implement a AD-like system using MIT Kerberos5 and OpenLDAP (I've done it)

In fact, even saying that the LDAP portion is extended is a bit misleading. In fact, AD implements a fully compliant LDAP interface. It does lack the more common LDAP Schemas (though they can be installed if you wish), and relies on a schema that MS developed, but the LDAP server is not, itself, extended.

It does just work, very well in fact. I've run a 1.3 million uniques / month site on a $150 / month Windows 2008 server. Zero downtime. Zero crashes. Zero maintenance. Zero effort at all really.

MySQL + IIS + Win2008. The server was cheap, fast, and trivial to set up and operate. I'd have no concerns about using Windows Server again.

Our startup diphur.com is 100% Microsoft and also went through the BizSpark program and highly recommend it.

When you exit the program you get to keep all licenses free with all future patches. We exited the program 2 years ago and have yet to buy a MS license for IIS or SQL.

When we upgrade to Windows 2012 which is not free, we will let our business plan we implemented pay for it and choose our MS Server versions according to function.

So compare like with like. If you had a similar architecture using open-source equivalents - not trendy mongo/node/etc, just basic Java/postgres - what have you got? A similar, slightly worse language. A much bigger open source library ecosystem. Similar, slightly worse IDEs. Better admin and deployment options. Free licenses forever, rather than until MS decides they want you to start paying.

You can write distributed large-scale SoA under microsoft, and you can write high-performance monolithic applications under open source. Stop conflating two very different issues.

The reference to Amy Hoy's Charm is interesting, just because that was such an interesting (and somewhat distressing scenario, if you're into RoR and Ubuntu).

But to say that the Windows architecture is significantly better bang for the buck requires more of an apples to apples comparison. The best I can find is that Hoy describes Charm as having 3,800 interested users in Oct. 2012...The OP describes having hundreds of companies (each with a variable number of users) issuing about 1,500 support tickets daily.

(I'm not counting the traffic visits to the blog because that should be trivial, in the scheme of things, to manage...though I guess kudos for putting it on the same server as the software platform?)

I guess knowing the other costs, such as developer time/salaries, would be helpful.

Two of the biggest help desk app on the web are running on rails. Salesforce helpdesk app with this as its url http://www.desk.com/ and ofcourse the helpdesk startup http://www.zendesk.com/.

Based on the facts above, I submit to you that there is nothing interesting about the reference to Amy Hoy's charm (and there is nothing in there to depress a rails developer using ubuntu).

As far as I am concerned, Amy Hoy's app did not get the kind of traction they envisaged simple. Hence they shut it down.

I am sure that both jitbit's app and Amy Hoy's were not as big customer wise as that form Zendesk and Salesforce. I am sure their traffic numbers wasn't anything near what zendesk and desk.com does.

I'm amazed that more people don't use Mono. Best of world to me is using Visual Studio for local development, and then deploying to a Linux server running Mono and Apache/Nginx. It's actually a lot less painful than people think, after you get ramped up a very small learning curve as to what is not supported in Mono.. And if you want to go completely free, MonoDevelop also isn't too bad, though you will find an occasional bug
The vast majority of startups in the Valley use MacBooks for development.
Unless mono has improved dramatically in the last 5 years it has that poor man's C# feeling about it - when I had to develop for it was nightmare - it had problem managing a lot of io buffers and silent improvements like making dictionary thread safe had some very very bad side effects in non blocking io applications.
MSDN subscription(for work) comes with Azure hours. I can run 6 768MB linux VPS for free. sure I will use MS.
Where can you get 6 linux vps for free? And why would you use 6 VPS as opposed to one?
This article could really do with a disclaimer at the top. I got about 25% in before i realised there was more than meets the eye (i didn't previously know who jitbit were, i had no idea of their Microsoft relationship).

Something like "we're a member of Microsoft's BizSpark program and you can find out what that means for us here: http://www.microsoft.com/bizspark/About/Default.aspx"

Anyone can get into BizSpark. It's not a special relationship. It basically just requires that you have a company of some kind formed and ostensibly are building some kind of software targeting the Microsoft ecosystem.
You're rich man, dude :/
Yeah I really don't understand how anyone can justify paying for a constantly-running EC2 instance. Amazon's prices are absurd; even Linode is better value. For $200/month I could rent three dedicated servers with vastly better hardware specs.
i am curious which datacenter you're looking at. mind sharing?
i was a bit coffee deprived when I read this and was starting on a teeth gnashing trollfest - but then realized it is 'JitBit' not 'FitBit' which I seemingly need to find a Windows machine to get to work...

Then was interested in this article because at our company we have 400-500 servers - mostly Ubuntu with some CentoS and FreeBSD. Lately we need WinSvr to run a BI app that generates nice visualizations for datasci to feed business/ sales.

Funny how not one sysadmin wants to touch this project... ;)

That all being said, the developer affection, in certain neighborhoods, for C# and .Net seems genuine, which makes an argument for developing an app on this platform - that seems clear.

I do not see this scaling out however. And historically Microsoft largesse (in education, research or with this startup initiative) has a way of drying up...

I really don't see any point why this post has been upvoted to this position. This someone got some discount because of being in some MS sponsored program and has no clue on how safe or unsafe his architecture is and has absurd rational on pricing. Good for him if he is happy to build with his MS gifts. Only if I could down vote this post and down rank it to absurdity.
You're bragging that your SaaS app, which serves hundreds of paying customers, runs on a single EC2 instance? lol
From your marketing:

> Small enough to be personal. Large enough to be stable.

What happens when your one server goes offline?