HN Explorer

Ask HN: How can I scale up quickly? Thinking about trying EC2.

7 points by makeee ↗ HN
Traffic to my facebook app is growing at an insane rate and I need some suggestions on how to scale. I'm probably not going to even be able to keep it running by the day's end, as it's hosted from my already overloaded dedicated server. I need to either move it to its own box or a cloud service like ec2.

Something like ec2 seems like the way to go, but my server admin skills are terrible. Is there a good front end management UI for ec2 or another hosting service that is comparable in cost that is fully managed? I don't mind going with something a bit more expensive now if that means I can get everything switched over and running within 24 hours.

22 comments

[ 3.2 ms ] story [ 59.6 ms ] thread
cperciva
E2C? Do you mean EC2?

Scaling is a very big field, and it's going to be hard for us to provide any useful advice without more information. What's your facebook application? What does it do? Where are you currently hosted? In what way is your dedicated server overloaded (CPU time, RAM, bandwidth, disk...)?

makeee
My dedicated server is overloaded because I'm running a number of medium/high traffic sites on it, not really much optimization I can do there.

I definitely need to move the app to the cloud or its own server. In a few days the app could easily be getting 50 requests a second. It's fairly light, in that there is no database interaction (there will probably be a DB soon, but it will only be 1 or 2 simple queries per page load). It's really just a simple html page and a JS file which I'll be moving to S3.

cperciva
If your server is currently overloaded for reasons entirely independent of this app, just move it to a new dedicated server. 50 requests per second is not a huge amount of traffic for a dedicated server, so I don't really see where the problem is.
sync
If it's just HTML + JS, you can host them both on S3.
mechanical_fish
Presumably the HTML page needs to be generated by some code that talks to the Facebook API.

Perhaps the code is also making a bunch of Facebook API calls that are slow to return. That might explain why it doesn't perform well even though it isn't talking to a local database.

makeee
Yeah I do need to execute some PHP. I'm giving rackspace cloud a try and then may move it to ec2 or a dedicated server if traffic slows down.
0nly1ife
If you have a rails app, use heroku and rds.

How long has the app been growing? If you grow too fast FB will delete it without warning.

ronnier
What's the justification for deleting apps that grow too fast?
catch23
In the past, lots of scammy/spammy apps grew fast by bribing the user to invite all of their friends etc. But you have to grow pretty fast for FB to see this and delete you, I doubt anyone can replicate that growth now since lots of the viral features were removed. Seems like the only way to mass a huge audience is to buy it with FB ads.
0nly1ife
They are assumed to be spammy.

My strategy for adopting the new api was to move the userbase from an existing app to a new one. Fb deleted the new app in the middle of the migration.

makeee
Launched it 2 days ago. It's a facebook connect website, so I think I'm okay.
flubba
+1 for heroku. What exactly is RDS? I know its a relational database, but I couldn't figure out exactly what it was from its site.
0nly1ife
It is MySQL running on Amazon. I like it because you can scale a hot database up or down.
tropin
Do you know python? What about AppEngine? It has zero administration.

Also, I'd go with http://www.linode.com, seems to be a cheaper, easier to admin, alternative to EC2.

tlack
EC2 is pretty complicated. You really have to restructure the way you work your app to make it scale well with EC2. You can usually get a lot of scalability out of a normal web app without going that far.

Start "top". Is MySQL using a lot of CPU? If so, get a second dedicated server at the same data center and put your MySQL on that. Next, look at the MySQL top queries and figure out what you can cache out of the equation with memcache.

famfam
People can say what they want, but EC2 is not really about scale. It's about on-demand resources. Which can help with scale, but it's not an automatic solution like AppEngine for example. You might be able to provision a huge box there faster than anywhere else right now (e.g. m1 4xl - 68.4G (!!) of memory and 8 cores -- $2.40/hr or $0.80/hr with spot pricing) and move your app over, but that'd just be scaling up, not out.
durana
I'd be willing to offer advice or a hand to a fellow HNer in need. This will require some discussion so e-mail me if you want to take me up on this offer.
makeee
Hey, I appreciate the offer to help. I ended up getting everything running on rackspace cloud, which was surprisingly easy considering I knew nothing about setting up a server.. so we'll see how this goes.
aymeric
I recommend you get more servers, get a dedicated one for db and make sure the HD is super fast (15000 rpm).

Cloud solutions would be too expensive for your scenario.

I used to develop facebook applications with one that reached 1 million users a day. I needed 5 servers to handle the load. It would cost me three times if I had used cloud solutions.

In Europe, you can use giga-international.com for cheap powerful servers with responsive customer service.

Then use memcache if needed.

aymeric
Tell us the technologies you use so that we can better advice you.