63 comments

[ 3.3 ms ] story [ 138 ms ] thread
Consider moving away from a CMS like Wordpress. It's great at what it does but I think you've out grown it. Static content generators might help here. They let you edit off line and compile to a static version of the web content you currently serve. I'd suggest looking at Hugo or one of the many specialised for blogging. https://opensource.com/article/18/3/start-blog-30-minutes-hu...

You can then push the content to something Amazon's s3 and put cloudfront in front to speed things up.

You'll have to then offload the comments to a service but that is relatively easy and cheap. A service like discus is only $9 a month. Moving to s3 you'll save that in hosting. (I run two sites like this and pay $0.07 on average per month). Not to mention those services are pretty good at filtering spam.

There are some other options around too. https://darekkay.com/blog/static-site-comments/

Wordpress is nice but it will never out perform static content.

EDIT: I just read this in discus "Free For small, personal, non-commercial sites who do not run any ads. This plan includes everything in Plus except Direct Email Support."

I developed and maintained the landing page for a top-100 website that was powered by Contentful, ExpressJS and aggressive in-memory caching. The setup was very stable.

I’m guessing your Wordpress install has a bad theme or plugin that is writing errors to the request body and then attempting to send the remaining HTTP headers.

Yeah, you are right. It can be done. But the bulk of the sites I've seen have some cludge that crept in somewhere. Someone installs a broken plugin or writes a custom tweak and things start to fall over. The beauty of static content is this just doesn't happen. You also get some security through simplicity.

I suspect you are very good at what you do. A lot of people are not so good at maintaining an operating site. Especially not a top 100 level.

That said it's a trade off. For every advantage there is a disadvantage. Having to compile to push changes is not without it's headaches. Using external services for comments and other things can run you into all sorts of cors hell.

Can be done doesn't mean it's a good tool for the job. I don't know enough about wordpress to say but I've seen some big sites built off sharepoint and "can be done" involved an extraordinary amount of manpower and working around sharepoint rather than with it. Those details get left out of the case studies.
Instead of hosting on S3, aI would recommend google fire add hosting
> It's great at what it does but I think you've out grown it.

There is absolutely no justification for him to be running into these sort of issues at such an incredibly low load. Let's pretend he gets 50,000 hits per day (he says 50k per month but I don't know reqs/sec). Than's less than one per second. The $5/mo box over at DigitalOcean can handle 100 times that with the most poorly optimized PHP, Wordpress, and MySQL installation.

The problem is entirely due to a shitty host and/or grossly misconfigured server. No CMS should run into load issues at the microscopic traffic levels he's complaining about. He doesn't need Cloudflare, either.

So, yes, a static site would indeed perform better but to imply that he's actually reaching the limits of a simple CMS is absolutely ridiculous.

I'd pretty confidently bet this guy has some ridiculous and/or buggy wordpress plugin that loads a bunch of images or is horribly written that is causing the problem. Even a naive and basic wordpress install on crappy hardware should be handle this load.
>There is absolutely no justification for him to be running into these sort of issues at such an incredibly low load

Yet he is running into these issues. Is it a buggy plugin or a configuration issue?probably, yes.

But that's exactly the problem static sites solve. I don't mean to say he's out grown it as in you can't serve Wordpress at the volume he's currently getting. I mean you have to have everything setup correctly to do so.One option would be to learn the workings of Wordpress and maybe the web server hosting this. If that's what he wants to do with his time great. I suspect he may have other interests, if that's the case static hosting would be a good alternative. It might not be the best, but it's the best I know.

> Consider moving away from a CMS like Wordpress. It's great at what it does but I think you've out grown it. Static content generators might help here.

This is ridiculous. If he had 100 times the traffic he has, he still wouldn't be anywhere near "outgrowing" Wordpress.

> put cloudfront in front to speed things up.

He already has Cloudfront, and it is completely meaningless snake oil in his situation.

$50 seems quite a bit of money for a wordpress blog, when you can have wordpress.com host you for $5/mo.
For 50$ (Canadian, I guess), will not someone like wordpress.com do better? Their business package is USD25 per month.
Their plugin selection isnt great though and you cant install your own.
Which is probably a good thing, given that shitty plugins are exactly what cause problems like this.
Pssh, all you need to do is install docker, convert static files and blog post into lambdas, then route each unique http request to a serverless IPv6 address located on the nearest continent.
Can you elaborate a little? I'm not sure I understand what you're suggesting but it sounds interesting.
Here is the architecture: https://serverless.com/blog/serverless-wordpress-hosting-wit...

https://www.getshifter.io/ seems to be a relatively turnkey solution for existing Wordpress sites to migrate to serverless hosting. It spins up a Wordpress docker image when somebody is in the admin panel, then builds and deploys a new static site to S3 when a change is made. Each build is immutable and easy to switch between.

There are limitations with static site hosting. Their approach to comments is: Use Disqus.

I, at least, thought this was funny.
Hugo. It's completely static and great out of the box. You can add new posts in Markdown.
Hugo is indeed a great tool. I like it because I can write all my content in Markdown and have full control over how the site looks. Editing a Hugo template, or creating my own, is also much easier than it is in WordPress.
If I recollect correctly (haven't used WP in years), the bulk of error 500-related problems on WordPress are due to poorly written plugins -- of which there are many. One of the better ways to track down a problem is to disable plugins one by one until the problem goes away.
Further to that, delete the folders. Plugins are often targeted directly if written poorly, so if you don't need 'em, delete 'em.
Disabling them is good enough, actually. The php code in them won't run at all unless they're enabled.
Which PHP version is being used? If it is still 5.* I would consider upgrading to 7.*. Just estimating now: 45000 users would be like about 1 user/minute, so maybe 10 requests per user per minute, could equal on peak max. 1-2 requests per second.

Some shared hosting shouldnt run into trouble.

That’s just a bad hosting service.. I manage somewhere in the neighborhood the OP gets with a basic Wordpress deployment. It’s extremely stable, and only ~7/month. I use limited plugins and have optimized for speed.

My guess is a plug-in or theme is bugging out here. Easy solution is to remove all and slowly add until the problem exists again.

In 2015 I had “Crayon Syntax Highlighter was Killing my WordPress” for instance:

https://austingwalters.com/crayon-syntax-highlighter-was-kil...

30 to 50k uniques per month really isn't that much. At a rough guesstimate, I'd say I've got at least one hosting customer that does that much with Wordpress, gracefully, on a site that's not optimized even a little. I charge them $10/mo.

Static site generators and everything else are lovely and all, but it's hard to beat Wordpress when it comes to being able to just plug in a new feature. Which, of course, also gives some folks the rope they need to hang themselves. As other folks have pointed out, there could be a plugin issue.

But my guess is it's the hosting provider. They can have professional-sounding support but still be running an architecture that hasn't been updated very much in the last 5+ years. mpm-event and php-fpm alone really changed the performance characteristics for bog-standard LAMP hosting.

The error logs should give a number if memory's exhausted. I bet it's super low, because a lot of shared hosting providers still try to cram as many sites as they can onto a beleaguered colo'd box. At a glance, that does look like the csoft.net architecture (they have "Two Dedicated, Redundant Servers (with Private Gigabit LAN)").

There's certainly a broader point to be made about the gross, inexorable bloat of web-based applications, but there are also plenty of people running even more bloated, busy stuff for less money with less trouble.

Oh, and you can install the traditional editor for Wordpress, they've made it available as a free plugin and it works just fine.

There are plenty of people around who can sort out stuff like this. I'm one of them, but I'm heading to bed. Email's in my profile, or just accept help from anybody else that knows what they're doing.

Seems like Wordpress has outlived its usefulness. It's like the Windows XP of the internet. It was great and simple, until the security problems and feature creep plagued it to death.
Most of the time the problem is not Wordpress but poorly written themes/plugins (especially free ones). Also $5/month shared hosting running on a decade old hardware are not helping either.
I am running multiple Wordpress customers on digitalocean $20/month plans, and some of them get more than 50k uniques per month, never had any issue. Never did much optimization. I am pretty sure the issue is your hosting. Also check your Apache error log, maybe you are using some plugin or theme that is overloading the memory with some crap code like majority of free Wordpress plugins/themes.

p.s make sure to update to php 7+. The speed difference was very noticeable when I upgraded

Oh, good old traditional LAMP shared hosting and WP. Honestly, I doubt that combo will ever work reliably. WP hosting is bit of a dark art in the best of times, and operating under shared hosting does not help. That being said, self-hosting on something like AWS probably would end up much more expensive, and would take also some effort to set up and maintain.

Personally I would either start looking for fully managed WP service (wordpress.com), or switching to a different, easier to host, platform

Anyone else use Ghost? I've got Ghost running in a Docker container on a DO droplet. It's the smallest droplet there is. I don't have a ton of traffic to my blog (at least, I don't think—I haven't found a privacy-respecting analysis system that I like (maybe roll my own... just count page request)) but it does the job.
+1 for Ghost. We run our development blog on it for our SaaS and it runs on the smallest Digital Ocean droplet just fine. Not a lot of traffic, but even so, the CPU and memory footprint on a small VPS is negligible.
We generally serve several of the front page stories on HN every day (some we host, some we don't) - so very used to dealing with high traffic loads

@OP - If you're interested, drop me an email on john@ghost.org and I'll set you up with a 2 month free trial on our Pro service where you can deploy a new instance in a few seconds and kick the tires. No obligation.

Ignore all the comments telling you to switch to static content generation, that's generally a large time endeavour and for a blog that is getting 50k uniques a month should be pretty easy to serve all requests with a little CloudFlare tuning.

First, let CloudFlare do 99% of the work. Don't worry about optimizing the linux box, PHP, MySQL, PHP Opcode Cache, WP-Cache at first.

First, let's check that CloudFlare is caching and serving all static assets such as js, css, images, .xml, .ico. I don't see a cache-control header being returned on js, css, images, etc. Make sure this is setup in CloudFlare with Caching Level => Standard and Browser Cache Control => 1 month. Also, consider enabling Auto Minify => Javascript and CSS. Finally enable Brotli and make sure HTTP/2 is enabled under network.

Now, here is where you can really make things fly. It is possible to have CloudFlare cache the result of generated html pages, so requests don't even hit your shared hosting provider. This is done with Page Rules. The trick is going to be crafting a rule that you can match posts and posts only. With my site, I leave .html extension on my files, so my rule is simply:

https://imgur.com/a/4iNpB4V

Your challenge will be creating a matching rule while excluding admin backend pages. Also, most certainly you probably going to want a much lower cache time than 8 hours. My site content does not change often. Perhaps something like 30 or 60 minutes to start.

Good luck, let me know any questions, my contact info is in my profile.

I'm just going to say one thing: EasyEngine.

If you need help managing/setting it up, I can do it for $15 a month + DO cost - or AWS/Google/Azure. I'll manage your stack, depending on time to migrate, there might be like $50-100 for migration.

I'm looking to start a WP management service, and hosting company -- mainly to get some MRR during my slow months.

I'm currently a Full stack laravel/vue/react freelancer.

For reference, I ran a political blog during 2016 w/ 100k hits an hour for a few weeks (traffic died down a ton), but my site wasn't functional at all on shared hosting.

After easy engine on DO's lowest tier, everything just worked. Nginx + redis cache is amazing.

Just drop me an email : patrick @ zvive.com -- linkedin: https://linkedin.com/in/patrickcurl

I recommended the static approach but if someone is willing to manage it for $15 a month and charge under $100 for migration, I'd go with that. You wont have to learn a new tool or figure out how to migrate.
Yeah static works good... in fact I'm working on a SaaS app w/ blog built in that will be open source (w/ upgrades), the frontend will have theme-able blog that's built statically via nuxt generate, the backend will be vue SPA - not using Nuxt, since SEO really isn't required. The stack is Laravel for the backend...

It's 'SaaS' in that the actual app will have easy subscription plans, analytics, single/teams, etc...

I was working on my own SaaS and realized other than some potential code examples from dead github repo's and Laravel's Spark, there isn't much on the market or github -- even in other frameworks like express or rails that covers this niche.

Definitely nothing open source, I'm hoping to get some sponsors early on to offset some of the costs of development till I get other streams of revenue...

I got off on a tangent lol, but I don't think $15/month + $99 migration is super sustainable...esp if I scale, but it would be nice to have even $1000/month just on autopilot via hosting.

Plus you can upsell stuff. I've got a brother who's a stock boy (he's 28, I'm almost 40), who graduated with a Bachelor's in Digital Design, I could probably kick him in the ass w/ some motivation and teach him to manage everything and maybe get him to work in his actual career field for once...He'd work for min wage right now just to get some beef on his resume.

How does EasyEngine compare to dokku-wordpress?
Can't say, haven't used dokku-wordpress. I know EasyEngine recently moved to docker images, which I'm not totally a fan of -- simply because you can only have X number of images per instance, iirc -- and that limits how many sites you could potentially host on a single server.
Do you mean it uses more memory? or you are hitting the docker limitation per host?
https://easyengine.io/handbook/v3-to-v4-migration "As v4 depends on docker, it is affected by Docker’s limitation of 31 network per machine.

v4 creates one network per site. Then v4 also needs few Docker networks for global services. So roughly we limit the number of sites to 27."

I'm also wondering what all those docker instances do to performance, seems to me there might be a better way.

I come across colleagues in the same boat as the OP, but the truth of the matter is that a $5/mo VPS hosting WordPress is really not going to handle a surge of high traffic. Been there, done that, when I had a $20/mo Digital Ocean droplet with my blog on it. Anytime I posted a link to it on HackerNews or Reddit, it used to die under the 'hug of death'.

Since moving my blog to SquareSpace, I've never had that issue. In years. Note: Not an advert and not an affiliate for SS, just a happy user.

We still have a WP blog on a VPS for our SaaS, but I am keen to move that over to another platform, because I have enough on my plate to develop code for my SaaS itself. For my blog, I just want to post my thoughts without having to wade through a WordPress Admin console and see dozens of flags, warnings and popups about a myriad of plugins that need attention. Seriously - our WP admin console causes a lot of stress just logging in, which sucks all the creativity out of me.

SquareSpace (and even Medium, for all the negativity surrounding it) makes it so easy to start with a blank slate and just write and let the ideas flow.

To help in case of a surge of traffic you can enable some sort of microcaching on the web server. For example with a cache TTL of as little of a second you are sure that Wordpress will never see more than 1 req/sec (per page, but usually surges are specific to one or a few pages), and at the same time you will never be serving stale information.

The downside is that your Wordpress installation will not see the requests served from cache, which means so if you rely on a Wordpress plugin for statistics and/or analytics the data will not be correct.

(comment deleted)
No offense intended at all but I can't help but wonder how a technologist/computer scientist has ended up asking for help resolving 500 errors on his blog rather than hypothesizing what the problem could be and testing his theories.

A quick Google search on the error he's getting says it's an Apache error either from invalid file permissions or running out of memory. His provider refuses to transparently increase memory so the hypothesis seems stronger here. Now just test it by spinning up a simple load balancer, clone your website on a different server, split the traffic 50/50, and see if the errors continue. Repeat this process until you've isolated the issue. Trying to solve the problem by adding a cache layer isn't going to help before the problem is understood.

It’s getting more funny if you look at his GitHub account.

https://github.com/lemire “His research is focused on software performance and indexing.”

I think it’s just a poor advertising. :)

Looking at one of the papers he has published

"Many applications use sequences of n consecutive symbols (n-grams). Hashing these n-grams can be a performance bottleneck. For more speed, recursive hash families compute hash values by updating previous values. We prove that recursive hash families cannot be more than pairwise independent. While hashing by irreducible polynomials is pairwise independent, our implementations either run in time O(n) or use an exponential amount of memory. As a more scalable alternative, we make hashing by cyclic polynomials pairwise independent by ignoring n-1 bits. Experimentally, we show that hashing by cyclic polynomials is is twice as fast as hashing by irreducible polynomials. We also show that randomized Karp-Rabin hash families are not pairwise independent."

I think he is working on the fundamental science of computer science. People like him are the reason we have nice things like web servers. That doesn't mean they know the specifics of Apache or Wordpres.

(comment deleted)
We don’t know that the author didn’t google and try a few things. And it’s perfectly possible to be a computer scientist and not know how to tune a web server. There are lots of specialisms and paths. Personally I admire someone who, despite being an expert, isn’t afraid to ask questions and admit when they don’t know something.
That's a great point. My comment above seems to be coming from a rather limited mindset
WordPress might be slow, but it's probably just the host or not enough PHP memory assigned to it. There is nowhere near enough traffic to bring down a WP site, unless you have a ton of bots pinging your site that never appear in your actual google analytics (because they are boths) .

* look at an actual server access log. cPanel offers one if your host is offering cPanel. * try and increase your WordPress memory limit: https://docs.woocommerce.com/document/increasing-the-wordpre... * check your server's PHP error log and see if there is an error * set WP_DEBUG to true in wp_config.php

50 CAD a month (33€) for such a small site and then it doesn't even work properly. It boggles the mind. I'm sure you can get a working setup for 5€/month without breaking a sweat.

Having a company that deals with your Wordpress installation is only beneficial if they get it right.

(comment deleted)
Reading this, I feel they should hand this job on to someone else. There comes a point where if you're not experienced enough to do this, no amount of Googling and investing more time will help you.

tl;dr I get the feeling that this person has been sold the idea that throwing money at things will help (why is he using argo for a smallish blog, it's not worth the money?)

Although a lot of the recommendations are great, and to me it also looks like there are specific hosting problems at play, possibly serious overcommit by the provider or some plugin problems, I do think there's a deeper issue:

> What am I missing? How can I be in so much trouble in 2019 with such a relatively modest task?

Something is not quite right with our web-platform when we have these sorts of problems, and the solutions also require quite a bit of sophistication.

As an experiment, I recently switched my http://objective.st site to run on Objective-Smalltalk itself on a Digital Ocean $5/month droplet (their smallest). Surprisingly made it to the HN front-page and didn't hiccup, despite an egregious memory leak on my part (fixed now).

This was reported as "blazing fast"[1], despite being a single binary hosted in Frankfurt, having no CDN support and no static files, with everything served from the embedded web-server. Heck, I haven't even hooked up gzip-compression

I don't log much, so I don't really know how much traffic it's getting. I do know that hitting it with wrk remotely I got around 1000 requests/s, locally on the same box it was around 8K. So I think there's plenty of headroom. Oh, and the logs do show continuous attempts to break in POSTing to what I presume are php vulnerabilities.

I am emphatically not saying I did something amazing or even particularly good, everything was somewhere between straightforward and egregiously simple-minded, and there were other (mostly easy) ways of accomplishing the same thing. Previously the same site was served using static files running the same code as a static site generator.

However, it does confirm my suspicion that we've path-depended ourselves into a corner.

[1] https://news.ycombinator.com/item?id=18965054