Sure, it's a little more involved than I can type up in a comment, but I can point you in the direction of a few resources I used by email? I see it's in your profile, if that's OK.
I suppose it wouldn't hurt to document it, if there's some interest. I mainly DDG'd and man-paged my way through, but I'll share the resources etc. and link to it here when done.
I just emailed you basically asking for the same thing. It'd be greatly appreciated. As someone else has said, perhaps you should make it another blog post.
I optimized the sites on the 512MB droplet for a few weeks (mysqltuner, nginx configs, plugin selection) and then recently stepped up to the 1GB RAM. The admin pages were chugging a little, especially with commerce plugins in the mix.
I did the same with about 5 WP sites and another 5 custom sites running on DO at 512 (it's now on 2gb). One of the custom sites gets about 15,000 pageviews per day, and APC with memcache seem to handle WP on Apache fairly well for now. If you document your setup I would be interested; I tried to get nginx running on another droplet but lost patience. I'm sure I could have done it, but it was lower on the list ;)
* It made sense traffic/memory wise to go to 2gb, but all of the sites are 'behind' cloudflare and that seems to help a lot as the server rarely starts breathing hard...
Yeah, most of these happen during a small window when Latin America is in school.
In any case it is a small load, but we moved from a shared host setup where we had these old sites sitting and they were beginning to bring down the server. They weren't into installing APC or Memcached; we were able to handle the traffic on the 512mb server using Apache, Cloudflare, APC and Memcached and the sites are faster (also using Google's pagespeed module). I've upped the server memory now, mostly because the WP admin is slow and the cost is still much less than we were paying before...
I was thinking about deploying some Wordpress installations using Docker.
Can anyone recommend a good recipe or ideas to use Docker to deploy different wordpress installs on a VPS like DigitalOcean?
I moved from AWS to DO and find the price performance much better for my WP sites. Whatever you do don't say that DO is better that AWS on HN or you will lose karma points from grumpy AWS admins. Of course AWS has better security and many more features but for many uses it's simply too slow and too expensive. Plus when is AWS going to make a GUI for its scaling tools?
FWIW, one of the startups I'm CTO at is on DO now and looking to move off due to cost. DO is great for ultra cheap servers, but the cost of bigger servers is high (especially for non-dedicated hardware). Bandwidth is also expensive at DO, and I'm a bit concerned about the quality of that bandwidth, too.
We're moving to OVH and their "infrastructure" line. We get a true VLAN, and the overall cost is ~1/3 of what DO charges for the same compute, RAM and bandwidth. Plus we get dedicated hardware, and 2x3TB hard drives in addition to the SSDs we're adding ($7/month extra). The plan is to use the hard drives to offload our current usage of S3, which'll save another $100K or so annually.
I also suspect OVH has "better" bandwidth than DO—more POPs, backbone providers, etc. I actually really like how they're doing things over their now. In the past, it seemed much more complicated, but their current lineup is substantially easier than AWS to understand and price out. It doesn't seem "gimmicky" at all, at least at the high end.
cloud hosting is cheaper on smaller capacities/cpu's/ram. but as your demands increase it makes sense from the point of view of costs to go on dedicated hardware.
How complicated of a config are we talking here? Did you spend like 2 weeks hunting around for a correct Nginx + WordPress config or was it more like "apt-get install nginx php5" ?
I think Wordpress is big and bloated, but it sure is easy to use.
Further, the real resource jump comes from 0 to 1 Wordpress sites. 1 to 4 is simply the overhead of handling connections. No new Apache or mysql processes are needed...
A few years ago, I was running a bunch of WordPress sites on a FreeBSD 8 VPS with only a single-core and 768 MB of RAM. I'm not sure who linked to it first but a page on one of the sites ended up getting linked to by Reddit, The Atlantic, Gizmodo, and I forget who else.
Fortunately, I was running nginx and one of the WordPress caching plugins (W3 Super Cache or W3 Total Cache, I forget which) and had spent a fair amount of time trying to tune the whole setup.
The VPS managed to serve ~200k visitors over a period of a few hours without any issues whatsoever. I was pretty amazed since this particular page has a LOT of images (Pingdom's Website test for the page reports 158 requests and 1.4 MB in size).
Nowadays, I run nginx and varnish on a physical server with four 2.2 GHz cores and 16 GB of RAM. It definitely doesn't have any problems keeping up. After the snapchatdb.info site got shut down by their provider, I hosted it for them and served up ~30 TB over a few days.
reminds me of a talk by a system administrator at Envato a couple of years ago. I remember him saying that until then they were running all their tutorial network on a single machine and their secret was: varnish
My issue with the post: it doesn't actually go into how it is serving up dynamic content. Sure, Nginx is used as the front-end webserver, but PHP still needs to be executed. The popular ways I've seen are either Apache (with a very small max child process count), or php-fpm.
Also, it's pretty well known that Wordpress is a dog in its standard configuration. Using some sort of caching mechanism is required if you have any kind of traffic to the site (some like W3 Total Cache).
While there are fun benchmarking services like the blitz.io and LoadImpact.com, you can normally get a pretty good feel for how your site will do under a load using "ab" (apache benchmark) from your local machine. This will at least let you test dynamically generated pages to make sure caching is working properly. Plus using some like Google PageSpeed to verify browser caching is properly being enabled by your webserver.
It's php-fpm, kyrra. Those processes are somewhat more resource-intensive! I've tried to reduce php calls with the caching, plugin minimalism etc. Thanks for the ab tip.
If you want to get even a little more performance out of it, look into PHP APC (alternative PHP cache). You can configure W3 Total Cache (or similar plugins) to store their cached files within APC (which will be in-memory). Though, since the cache files on disk will be already cached in memory, this may not be all that much faster.
APC is a nice alternative to dealing with something like memcache (since you are working on a smaller scale).
Agree 100% with most of what you've said, but I know this bit to be false:
> ...you can normally get a pretty good feel for how your site will do under a load using "ab" (apache benchmark) from your local machine.
A single web server can easily out-serve the vast majority of internet connections you get at a home or office. I have 35 mbps down, 5 mbps up, and I can saturate my link (usually uplink) well before our web/app servers choke up.
If you're testing a VPS at a provider like DO, you can spin up the cheapest Droplet in the same datacenter, then use it to test. Run your benchmarks in one shell session, while keeping another open with iptraf running to make sure you're not running in to network overhead.
I know this sounds like overkill, and you qualified with "pretty good feel", which could very well mean "find out if your web server falls over at 10 concurrency", but it's always amazing to me how quickly bottlenecks creep in and muck up benchmarking results.
I run about 15 Wordpress sites on the cheapest Linode package, using nginx and php-fpm.
Of course, all of these are very low traffic sites.
There was definitely a learning curve, especially with nginx and Wordpress. I'm much more of a web developer than a server admin, but once everything got up and running, there's very, very little in terms of day to day maintenance.
I used to be on a shared host but also ran into issues with memory and general downtime. Not to mention there is something liberating about having your own (virtual) box.
Yes, a strange and perhaps odd liberating feeling is there. Also slight terror about you yourself being nearly fully accountable for any backups, downtime or losses.
Linode have great hardware, too. I can see how 15 sites would be possible. It's pretty incredible, really. As mentioned elsewhere, if only Wordpress could run on sqlite, trending towards less resources rather than more...
Sometime last year I was on HostGator's VPS and WP Engine for several different sites. I was paying around $150 a month. HostGator started to shut me down for using too many resources and I was going way over the limit with WP Engine's 100k limit. I was fed up so I looked into Digital Ocean. I knew a bit server management but never really wanted to deal with it. I just wanted to produce content and make apps.
I purchased the $40/m plan on DO and started to move my sites over. In the process I learned a whole lot about configuring my own server. I got a pretty impressive setup going now and monitor it on New Relic. It really only took me a few weeks to learn everything and get all my sites stable. It's been several months now and have had to hardly ever login to my server to touch anything. Initially I was worried that I'd have to manage my sites and it'd suck up some of my time but really I'm actually saving time because I don't have to deal with HostGator or going over the limit with WP Engine. I'm saving $110 a month, I learned a whole lot in the process, and it really did not take much time to learn nor is it much of a maintenance at all to run.
thats the way to go. i stopped using shared hosting many years ago for the same reasons and you. I've been running a single dedicated server for the past 8 years (which i keep upgrading every 2 years). hosting 30+ sites some of which have a decent amount of traffic. I get better performance, and more control. it all has been smooth ever since - just keeping an eye on the logs (I setup some alerts), following some security hardening guides and don't run anything I am not sure about. it just works!
Seems like a lot of folks in this transition at the moment. It'd be great to take a poll.
I wonder if a low-cost provider will offer the one-click installs we all came to appreciate with decent shared hosts, along with a decent back-up system.
You're right, but I suppose we move over to VPS partly for performance/control, so an Nginx setup with swap memory and tuned MySQL would be nice in a one-click.
Right now I'm loving DO as a cheap place to run random experiments and host a few files and services for a small group of friends. (things like mumble, or a small starboud server, testing steam on linux, and using it as my IRC portal via irssi to provide an additional layer of seperation in combination with a vpn (giganews)
My main personal website is on dreamhost, and I have been using them since about 06, and they are great for simple domain controls, but I really like DO because I like having root, and having snapshots if I jack something up.
Since no actual configs were posted. I wrote up how I setup my server (which serves a wordpress site along with a custom site) to reverse proxy and cache. There's nginx reverse proxying to my apache server.
full config and details are all here. Not sure since this post really doesn't actually share much, but I think my performance was a lot better.
That definitely does perform better. My blitz tests fell off quite quickly after around 130 concurrent users, so I'm sure there's room for further improvement. But the site did just stay up after being on the front page for a good 2 hours, so I'm satisfied that the arrangement is sufficient for my basic purposes.
You didn't mention your server spec, or confirm that your site is dynamic/Wordpress and not static?
Similar experience here. I set up nginx, php-fpm, php-apc, W3 total cache, and put it all behind cloudflare. I have no problem running this on 256MB RAM servers. I just wish wordpress would support sqlite natively, then I wouldn't need all the mysql overhead.
I have 2 wordpress sites on a DO 5$ instance. I had 1-10 visitors a day on the sites and had no problems.
The setup is nginx +php5-fpm + w3 super cache + cloudflare
I starter getting about 300 visitors a day for one site and i have noticed the site sometimes was showing blank pages.
I was also getting timeout from cloudflare, and the site was moving very slow. Tried to disable the cache plugin but it was the same.
I have upgraded to the 10$ instance, and still could barely navigate in the wp admin area..
This took 2-3 days, now the site looks to be ok (traffic has dropped to 150 day), i think it was a memory issue with their server..
Also if i do a : ab -n 500 -c 200 http://site.com from another server my site stops working..
The object-cache (with APC or MEM) seems to take a decent load off of my droplet. One of the sites doesn't seem to work with the OC, and it is notably slower than the others.
If it wasn't the memory issue, then systematically check your plugins for ropey code, and deprecated calls etc. There's plenty of debugging in the logs and plugins to be had. 300 users really shouldn't trouble the server; something has to be misconfigured?
It is shocking how similar this setup is to mine (I use INIZ, run multiples sites, cron job backup to an RPi every hour). Really awesome to read about someone unplugging from managed hosting and surviving.
I personally like to be involved as far down the stack as I can get (to gain some experience) -- great read. Also, I recommend INIZ, they're pretty cheap (~9GBP/month), and offer pretty stable service.
Looks like we have the same mindset - appreciate hearing from the like-minded. We could quite easily drop our sites on a managed host... but where's the fun in that?
This level of control and xp points gained can't be had elsewhere, at least not at an affordable monthly price.
It has been quite easy for a few years now to put together your own setup on a VPS and run with it. IIRC, I started doing this somewhere around 2008, during the heydays of Slicehost. What has changed is that you now get pretty good and reliable VPS providers for cheap and in some ways it has kind of become a cool thing to do for everyone.
I would, though, add a few words of caution here.
1. It is one thing to put together a VPS to run a bunch of sites. It is another to handle major traffic on it. Some of you have done it, some of you will learn with time how to do it, but a large number won't. If it is critical for your site to be up 24/7 and Google is going to be your best friend in trying to understand which part of the stack is creating the bottleneck, you'll be in for some real trouble.
2. Do you know how to check for rootkits? Would you know if your server has been backdoored? These days attack vectors are so complex that even experienced hands (the main reason why I don't do this on my own anymore) have trouble saying for sure they are fully locked down. Wordpress on a public static IP represents one of the juiciest targets on the web for hackers and a big chunk of the phishing sites are hosted on servers with unpatched Wordpress installations or plugins/themes that have backdoors in them. Most of the site owners have no clue they've been owned till the hosting company takes them down. Please don't wind up being one of those site owners.
If you are going to do this own your own, at least get something like Wordfence installed, so that you have some degree of protection in place.
I run most of my sites these days on Webfaction. I believe they run Cloudlinux these days, which means you get pretty consistent performance from the server, even in the shared hosting environment. The sites with greater resource requirements are moved to Wiredtree, who provide beefier, managed VPS services at reasonable rates as long as you are OK with cPanel and don't veer too much from their standard stack.
Disclosure: I have no business links with Webfaction or Wiredtree.
57 comments
[ 3.2 ms ] story [ 52.5 ms ] threadOver here: https://news.ycombinator.com/item?id=7238107
* It made sense traffic/memory wise to go to 2gb, but all of the sites are 'behind' cloudflare and that seems to help a lot as the server rarely starts breathing hard...
In any case it is a small load, but we moved from a shared host setup where we had these old sites sitting and they were beginning to bring down the server. They weren't into installing APC or Memcached; we were able to handle the traffic on the 512mb server using Apache, Cloudflare, APC and Memcached and the sites are faster (also using Google's pagespeed module). I've upped the server memory now, mostly because the WP admin is slow and the cost is still much less than we were paying before...
We're moving to OVH and their "infrastructure" line. We get a true VLAN, and the overall cost is ~1/3 of what DO charges for the same compute, RAM and bandwidth. Plus we get dedicated hardware, and 2x3TB hard drives in addition to the SSDs we're adding ($7/month extra). The plan is to use the hard drives to offload our current usage of S3, which'll save another $100K or so annually.
I also suspect OVH has "better" bandwidth than DO—more POPs, backbone providers, etc. I actually really like how they're doing things over their now. In the past, it seemed much more complicated, but their current lineup is substantially easier than AWS to understand and price out. It doesn't seem "gimmicky" at all, at least at the high end.
I think Wordpress is big and bloated, but it sure is easy to use.
Further, the real resource jump comes from 0 to 1 Wordpress sites. 1 to 4 is simply the overhead of handling connections. No new Apache or mysql processes are needed...
Fortunately, I was running nginx and one of the WordPress caching plugins (W3 Super Cache or W3 Total Cache, I forget which) and had spent a fair amount of time trying to tune the whole setup.
The VPS managed to serve ~200k visitors over a period of a few hours without any issues whatsoever. I was pretty amazed since this particular page has a LOT of images (Pingdom's Website test for the page reports 158 requests and 1.4 MB in size).
Nowadays, I run nginx and varnish on a physical server with four 2.2 GHz cores and 16 GB of RAM. It definitely doesn't have any problems keeping up. After the snapchatdb.info site got shut down by their provider, I hosted it for them and served up ~30 TB over a few days.
Also, it's pretty well known that Wordpress is a dog in its standard configuration. Using some sort of caching mechanism is required if you have any kind of traffic to the site (some like W3 Total Cache).
While there are fun benchmarking services like the blitz.io and LoadImpact.com, you can normally get a pretty good feel for how your site will do under a load using "ab" (apache benchmark) from your local machine. This will at least let you test dynamically generated pages to make sure caching is working properly. Plus using some like Google PageSpeed to verify browser caching is properly being enabled by your webserver.
APC is a nice alternative to dealing with something like memcache (since you are working on a smaller scale).
> ...you can normally get a pretty good feel for how your site will do under a load using "ab" (apache benchmark) from your local machine.
A single web server can easily out-serve the vast majority of internet connections you get at a home or office. I have 35 mbps down, 5 mbps up, and I can saturate my link (usually uplink) well before our web/app servers choke up.
If you're testing a VPS at a provider like DO, you can spin up the cheapest Droplet in the same datacenter, then use it to test. Run your benchmarks in one shell session, while keeping another open with iptraf running to make sure you're not running in to network overhead.
I know this sounds like overkill, and you qualified with "pretty good feel", which could very well mean "find out if your web server falls over at 10 concurrency", but it's always amazing to me how quickly bottlenecks creep in and muck up benchmarking results.
Of course, all of these are very low traffic sites.
There was definitely a learning curve, especially with nginx and Wordpress. I'm much more of a web developer than a server admin, but once everything got up and running, there's very, very little in terms of day to day maintenance.
I used to be on a shared host but also ran into issues with memory and general downtime. Not to mention there is something liberating about having your own (virtual) box.
Linode have great hardware, too. I can see how 15 sites would be possible. It's pretty incredible, really. As mentioned elsewhere, if only Wordpress could run on sqlite, trending towards less resources rather than more...
I purchased the $40/m plan on DO and started to move my sites over. In the process I learned a whole lot about configuring my own server. I got a pretty impressive setup going now and monitor it on New Relic. It really only took me a few weeks to learn everything and get all my sites stable. It's been several months now and have had to hardly ever login to my server to touch anything. Initially I was worried that I'd have to manage my sites and it'd suck up some of my time but really I'm actually saving time because I don't have to deal with HostGator or going over the limit with WP Engine. I'm saving $110 a month, I learned a whole lot in the process, and it really did not take much time to learn nor is it much of a maintenance at all to run.
I wonder if a low-cost provider will offer the one-click installs we all came to appreciate with decent shared hosts, along with a decent back-up system.
My main personal website is on dreamhost, and I have been using them since about 06, and they are great for simple domain controls, but I really like DO because I like having root, and having snapshots if I jack something up.
full config and details are all here. Not sure since this post really doesn't actually share much, but I think my performance was a lot better.
http://reviewsignal.com/blog/2013/08/29/reverse-proxy-and-ca...
You didn't mention your server spec, or confirm that your site is dynamic/Wordpress and not static?
I have upgraded to the 10$ instance, and still could barely navigate in the wp admin area..
This took 2-3 days, now the site looks to be ok (traffic has dropped to 150 day), i think it was a memory issue with their server..
Also if i do a : ab -n 500 -c 200 http://site.com from another server my site stops working..
I personally like to be involved as far down the stack as I can get (to gain some experience) -- great read. Also, I recommend INIZ, they're pretty cheap (~9GBP/month), and offer pretty stable service.
This level of control and xp points gained can't be had elsewhere, at least not at an affordable monthly price.
I would, though, add a few words of caution here.
1. It is one thing to put together a VPS to run a bunch of sites. It is another to handle major traffic on it. Some of you have done it, some of you will learn with time how to do it, but a large number won't. If it is critical for your site to be up 24/7 and Google is going to be your best friend in trying to understand which part of the stack is creating the bottleneck, you'll be in for some real trouble.
2. Do you know how to check for rootkits? Would you know if your server has been backdoored? These days attack vectors are so complex that even experienced hands (the main reason why I don't do this on my own anymore) have trouble saying for sure they are fully locked down. Wordpress on a public static IP represents one of the juiciest targets on the web for hackers and a big chunk of the phishing sites are hosted on servers with unpatched Wordpress installations or plugins/themes that have backdoors in them. Most of the site owners have no clue they've been owned till the hosting company takes them down. Please don't wind up being one of those site owners.
If you are going to do this own your own, at least get something like Wordfence installed, so that you have some degree of protection in place.
I run most of my sites these days on Webfaction. I believe they run Cloudlinux these days, which means you get pretty consistent performance from the server, even in the shared hosting environment. The sites with greater resource requirements are moved to Wiredtree, who provide beefier, managed VPS services at reasonable rates as long as you are OK with cPanel and don't veer too much from their standard stack.
Disclosure: I have no business links with Webfaction or Wiredtree.
At my end I try to keep things patched, check the listening ports and run Tripwire, rkhunter etc.
Cheers for the Wordfence tip, I'll take a look.