249 comments

[ 2.0 ms ] story [ 263 ms ] thread
We have a similar experience trying to move a large local newspaper in WP (500k+ posts) from our architecture to WPEngine, after over 1200 U$D spend, sadly we rollback to old setup.
Of all the things I've ever written on the blog (and ok, most of the earlier stuff is as tedious as watching the guy who watches the grass grow), it's sorta sad that only the one with the sensationalist title has ever made the front page at HN.

So, while I'm here, go look at my very nice bloggers. I regularly submit their stuff.

http://ozblogistan.com.au/

Take, for example, this recent extremely intelligent discussion of privacy in the current internet:

http://skepticlawyer.com.au/2012/10/19/manners-cost-nothing/

Or this discussion of the roots of order/chaos good/evil dualities in agricultural society: http://skepticlawyer.com.au/2012/10/16/war-and-peace/

This confronting moral conundrum from the Massacre of Srebrenica: http://clubtroppo.com.au/2012/10/24/srebrenica-should-the-so...

This discussion of the deadening effect of media laws upon Australian free speech: http://catallaxyfiles.com/2012/10/24/the-deadening-of-free-s...

and so on and so forth.

I am enormously proud of all my bloggers.

"my bloggers"?

How are these blogs related to you?

It's a convenient shorthand for "bloggers whose websites I host at my own expense because I like and admire their sites".
@Jacques, the people who host blogs on your network are lucky :-). I wish more businesses emulate the good example you are setting by thinking about your "customers" success...
Curious why people are downvoting. This is a legitimate question. He refers to these bloggers paternalistically and seems to take some credit for their work, but doesn't actually specify his relationship.

Good for him for sponsoring people, but it's a bit odd to imply he's deserving of any credit for their work simply for hosting a Wordpress instance. If they are paid employees, then that might be more appropriate, though still odd.

(comment deleted)
Seems a bit distasteful to publicly attack a popular service, then try to siphon the traffic their name generates to your "bloggers."
I suppose that's one way of looking at it.

But what I said is true. It's a shame that out of all the things that every site on Ozblogistan has ever posted, this is the one that made the HN front page.

There are far better posts than mine in those blogs. Is it wrong for me to be proud of them?

If you are so proud of them, continue to submit them as single links to HN.
You're ignoring the fact that your post on WPEngine is actually relevant to this site, whereas only one of your promoted posts has anything to do with tech.

It's not wrong to be proud, but I have to agree that blatant advertising is distasteful no matter which way you slice it.

I don't agree with you, but I can understand why you feel that way.

In case you are wondering, I don't make any money from the traffic. Each of the blogs decide for themselves whether to host advertising.

I sometimes forget that because I have about 3k advertising servers blocked in my hosts file ...

But now you are posting off-topic comments to that sensationalist post. This does harm to your credibility as a complaining customer, frankly, because now it seems you have an ulterior motive for complaining so loudly.

Personally, I downvoted the comment and suggest you delete it.

For all who are thinking about moving to WPEngine:

Your staging server does not use WPE's caching technology, which your production server will use. WPE will not give you details on how their secret-sauce caching works. This means that things that work on staging will break on production. WPE also refuse to turn off their caching on production, but also refuse to turn ON their caching on your staging server. This means there is no real way to test. Things that work with their staging server, and even work with the standard WP caching plugins, will break in WPE production.

Their solution? Buy another production site to test on.

That website is now moving from WPEngine after having its launch botched by the aforementioned issues, and after not being able to make a basic paywall work with their caching. Once they realised the use case, WPE simply suggested moving elsewhere without making any further effort to accommodate.

I had a similar reaction to the post yesterday by asmartbear as Jaques, but he made the extra effort to actually write the blogpost, so bravo to him. WPE enjoys this sterling reputation that misleads a lot of people into using them, but their comfort zone it seems is limited to vanilla WP installs with little deviation.

> Their solution? Buy another production site to test on.

... wow.

I've had the opposite experience of WP Engine, and appreciate leaving caching and performance issues to somebody else, given that my company has a somewhat limited staff for supporting servers and CMS upgrades. I am building a semi-complex Multisite installation with multiple domains. So far, it's been pretty decent.

Your frustration with their caching "secret sauce" is understandable. However, you can get a rough idea of their caching back-end by poking around in the WP Engine config files and plugins bundled with your installation. They also provide a nice little overview here: http://wpengine.com/our-infrastructure/.

If you are staging out anything more complex than a single blog, you will definitely want to have two production instances to test on. For some, this is a raw deal. I personally don't mind.

I am also not a huge fan of their CDN feature, which works by rewriting URLs in your page source to point files and assets to NetDNA servers. However, I am hopeful that they will improve upon it.

That said, I have enjoyed a platform that keeps my Wordpress installations secure with automatic upgrades.

Comparing WPEngine to other hosting providers like Linode or Dreamhost is really unfair. They offer completely different services, and specialize in Wordpress-optimized hosting only.

I also have to say that it was really nice to not be part of Amazon/Heroku's outage this week. WP Engine has a control panel hosted on Heroku, but all their sites live in the Linode network, evading downtime.

Then compare it to Page.ly.

Same basic proposition: fully managed Wordpress.

They cost less.

And their service roflstomps WPEngine's.

…and they never accepted my credit card.

Just because you have a bad experience it doesn't mean that it sucks. My experience with WPEngine has been fantastic.

I suspect that for 99% of customers, WPEngine is fine. Excellent, even. Certainly better than barrel-bottom shared hosting outfits.

But I wasn't asking for anything exotic. A multisite network, not very large, not much traffic. It says, right there on their site, that they support multisite networks.

So I guess I ... well ... expected that they would be able to support a multisite network.

Basically all services have a group of satisfied users and a group of unsatisfied users.

What blows a story like this up (as much as hitting the front-page of HN is 'blowing up') isn't that a service failed to meet the needs of one blogger, but rather that the people running the service respond to criticism (in public) sounding more like petulant children than professionals.

As someone not into blogging, I'm not that familiar with WPEngine, but based on comments here that link to various twitter comments now I suddenly have a negative-leaning opinion of them. Primarily because they couldn't take a negative review without calling the poster a "hater". (FWIW, a lot of my negative opinion is tied up in this stupid word, which IME comes out as a last resort when the person using it knows there's a lot of truth to what the other person is saying but they refuse to acknowledge it for whatever emotional reasons).

I had the opposite experience as well -- I tried Page.ly first but could not import any of my sites (which all use a variety of custom post types). Their import process seemed more attuned to vanilla installs. Closing out my billing with them was also somewhat arduous.

WPE has been great so far -- very responsive support -- and their caching handles about 8 million pageviews a month with 600-800 simultaneous visitors and very little downtime.

Page.ly was awful. When my sites weren't down due to botched infrastructure upgrades, users would get locked out of the admin interface by "FireHost Website Protection" (FireHost was/is their underlying hosting provider). The same sad cycle repeated multiple times:

1. Email FireHost with the event IDs and a request to unblock.

2. FireHost doesn't reply for 24 hours, or if they do it's with "we can't find that event in our logs".

3. I open a Page.ly ticket

4. "El Vaquero Furioso" (yes, that was his name in Zendesk) replies that the issue is fixed and won't happen again.

5. I ask for the root cause and what steps were taken.

6. Crickets, or a snide response from the founder that "we're all on the same team".

That sucks.

I guess it's like how, in this thread, lots of people are saying they've had great experiences with WPEngine and then there's me, suddenly internet-famous as that unhappy WPEngine customer.

Hey D,

Sorry we handled that poorly. At the time Firehost was having some significant issues with their firewall policies. We do constantly walk that line between enforcing security at the edge and annoying customers that may run into WAF blocks. It's a delicate balance.

The problems you had have since been resolved and pagely is running under a new firewall (and new blades) which are much more reliable.

In addressing your frustration my comment of 'we're on the same team' was to demonstrate that we were all working towards a solution for you. As professionals dealing with other professionals we aim to acknowledge and address the problem while steering the conversation to a positive outcome. If our customer views us as an adversary it makes the process more challenging for all involved. By re-framing the conversation as we are all in this together, all working towards a solution, we feel it is more likely the positive solution will be found and implemented sooner. My comment was a gentle reminder to that effect: acknowledging the issue, and we are working with you, and with our partners to remedy it.

Thank you for your past business, and we hope one day to earn it again. - joshua strebel, founder pagely.

To be fair, any WP hosting company is going to run into issues with non-standard installs, and they're not going to be able to accommodate any setup. If you have a customised WP install, it might not work on this sort of provider, they're optimising for speed, not dealing with, and they probably don't want the headache of trying to support any plugins or modifications that you might bring to WP.

It would make sense for them to offer a dev plan with every production plan though, and let you upload your site, and serve it to a limited no of people as a testing ground. Given their 'Power Tools for Power Users' stuff on their website, they really should consider at least offering staging servers for testing.

I guess at present they're just not a good setup for developers trying to modify WP, and are instead trying to capture the market for hardened WP hosting for mostly vanilla installs.

Multisite has been part of the base Wordpress install since 3.0.
Just curious, why do some pages sort 189k db rows?
That could very likely be the "Recent Comments" feature, which is incredibly inefficient.
This is basically what I found with some preliminary detective work from Page.ly.
"and are instead trying to capture the market for hardened WP hosting for mostly vanilla installs."

I don't know of any large WP installs that would need the benefit of outsourced WP hosting/experts that use 'vanilla installs'. They may exist, but I can't imagine they're a large portion of the market. Perhaps my scope of experience is too narrow in this respect?

Likely, yes. Most WP installs that I've done use 6-8 of the most supported plugins, a slightly customized theme, and mostly standard server configurations.
So we don't apply the caching stuff to the staging server because the purpose of the staging server is to stage content and test new plugins and themes.

We invented this feature in response to bloggers and site owners who wanted to be able to test a new plugin before adding it to the live/production version of their site. We're also the only managed WP platform to offer such a feature as standard with all accounts.

I can see that in a true dev/staging/production test suite environment, it does't stand up as a true mirror of product for the reasons you stated - but then neither does the staging/local version of Google App Engine or Heroku offer the same exact environment of their production platform.

If you need true like-for-like mirroring of production and staging, then yes, you need to create another WordPress instance - that's the case for most providers. Also its wroth nothing all WP Engine accounts, other than the Personal plan, come with multiple WordPress instances, so rarely that means actually buying anything further.

The solution with Heroku is to create a separate app and use that. The app will be identical except for having fewer (and optionally different) resources. Because it's billed hourly, it doesn't cost too much to run it at full power for a short while to answer a question, before scaling it back down. The reason you needed to "invent" this feature is because your environment isn't as self-service as it could be.
This is not true for App Engine. You can create a "staging" version of your app and access it at staging.my-app.appspot.com. It all runs on the same infrastructure that your main version runs on, and it's easy enough to make it admin-only. Once your satisfied that it's safe to deploy, it's two clicks to change which version is serving.

You may be the only managed Wordpress platform to do so, but App Engine has had this functionality for a long time.

I was helping a friend debug something on WPEngine. We learned that the caching doesn't allow you to set cookies it would seem (from PHP). Had to do it all using JS instead. I thought I was going crazy trying to debug the code when it was definitely an issue with the way they cache.

Not saying it was right or wrong, it just wasn't a fun experience thinking I was going crazy. And hopefully if anyone else is trying SETCOOKIE() and it's not working they also realize they will have to use JS.

That was my issue, and yes, it absolutely is one of the many reasons why staging instances should have a caching on/off switch that mirrors production. It is absolutely 100% feasible that WPEngine's caching will break your site, and you need to know this before you push to production.

Don't get me wrong, as a contractor to a company that uses WPEngine, we have had a great experience with it - but e.g. in the case of setcookie() being completely useless for unauthenticated users, some documentation / warnings / big red flags would be nice.

In my case I was trying to build a simple site selector for unauthed users that would forward them to their selected site every time they hit the main domain. Nothing exotic by a long shot.

It sounds like you and ohashi don't understand how caching works. WP caching serves up static copies of your pages, bypassing the code and DB calls that would normally be required. Since your code is bypassed cookies won't be set.

This isn't something specific to WPEngine, this is how all the good WP caching plugins work.

I had an issue where an API call was failing for Googlebot and other crawlers. I was worried about this being seen as "cloaking," so I was trying to figure out what was causing. Couldn't be my code. The API provider said it wasn't them. I finally realized that it was WPEngine when I visited my staging server as Googlebot. The content that was supposed to show up showed up!

Part of their caching setup was causing the API request to fail for a select list of bots. The issue was escalated to a Sys Admin who figured out what the problem was. He tweaked my server and fixed the issue for me within a day. I think that is pretty good.

My point... if something is broken and you think it might be WPEngine's caching. Try it on your staging server.

All of these sites have fairly standard themes. All could be hosted, worry-free, at WordPress.com.
Not really. WordPress.com's free hosting has limited space, no custom themes, no plugins except a small preapproved set, no ability to run your own ads, and you have to run WordPress's ads for their benefit.
What about the paid plan?
You get a little more storage space, a little bit of CSS control, but you still can't bring your own themes and plugins, edit the code, etc. Plus, there's no way to migrate a multi-gigabyte blog network's data over, AFAIK.

WordPress.com is an SaaS for blogging like Tumblr or Posterous. It happens to be built on a custom, locked down install of the WordPress multi-site software. It's not a website hosting company.

My bloggers have several non-standard plugins and non-standard theme tweaks.

Edit: and oh yes, 2.5Gb of media.

The big kicker is that my migrations can only occur by SQL. Using WXR is fine for any blog that can be turned from XML into PHP objects ... inside the execution-time limit.

Otherwise Wordpress thinks it's just hilarious to silently fail to import anything that doesn't make the execution-time cutoff.

And I don't think I am a big enough fish for wordpress.com to bother to help me do an SQL migration.

Any reason you're not hosting media on S3? That makes it a lot easier to move hosts, too.
Put simply: cost. Collectively it's a few hundred Gb per month of traffic which is completely covered by my Linode plan.

These days I've put most of the sites behind Cloudflare, which has the effect that a lot of my media is being mirrored in their Sydney DC, which is nice because most of the readership is Australian -- the same reason I went to the Tokyo centre this time.

The Tokyo DC isn't a universal win for Australia. The results are actually pretty bimodal depending on your ISP.

From Perth, iiNet routes directly to Asia, and it's a big win. But Amnet routes AU -> US -> Asia and it's a big loss.

It's a big enough issue Australia-wide that Blizzard sells a dual-region Starcraft 2 in Australia, because (ping to battle.net Singapore for some) > (ping to battle.net US) > (ping to battle.net Singapore for others) to a significant degree.

Fascinating, I didn't know that.

So, being in Perth, I've picked the DC that serves me best. Hmm. I guess I will ask my bloggers how noticeable it is.

I tossed up hosting in Australia, but the bandwidth is just so utterly ruinous.

I've been hosting a couple WordPress sites on PHPFog (http://phpfog.com) as well as DreamHost. For some reason DreamHost is faster, but the PHPFog UI is much nicer. Both have great support (DreamHosts' is 24/7 if I'm not mistaken) and relatively cheap.

That being said, once you have the sort of requirements that the original poster seems to have, I don't know if any of those companies would be suitable…

Dreamhost is, like every shared hosting provider, hit-and-miss.

I used to host Club Troppo there. When it got slow they basically shrugged and said ... it's shared hosting, what did I expect for $8/month?

I had the exact same experience, but I was paying for their VPS service. I believe it was over $30/month, and it went down all the time with very little traffic. Dreamhost is a tempting host for a new site because it's so easy to set up, but I think you save time and headaches in the long run starting with someone else.
Are you on Dreamhost VPS or plain-old shared hosting?

I had to migrate my WordPress off Dreamhost shared hosting as it was too slow and blowing their CPU limits. That was around 2006 and I'm sure a lot has changed.

VPS. http://sachagreif.com runs on it and got to the top 5 of HN many times without any trouble (I'm using W3 Total Cache).

The only reason I'm thinking about switching away is that their control panel is a little messy, and they don't support Git deployment out of the box (although you can activate it via ssh, which is what I did).

It also feels faster than PHPFog for some reason, especially the WordPress admin. So overall, I'd say they're a pretty good option for budget hosting.

I wanted to use PHPFog for an ExpressionEngine site but deploys seemed to take an unreasonably long time. Their pseudo-git (CLI task `pf`) is also kind of silly.

If you're not going to be doing a lot of dev work and making a lot of commits, I'd recommend.

You can actually use git natively for PHP Fog. The CLI tool is just an option to make things easier for some users.
Right, that's what I went to. I figured I'd give their tool a whirl but it's just an obfuscation of git. I guess it might be easier for some but it just gave me error after error.
+1 for Page.ly. We had a slightly awkward setup (for a multisite blog) but their support was excellent.
Anyone have experience with ZippyKid? How are they?
FWIW, I spoke to ZippyKid. They (perhaps wisely) don't handle multisite installations.
Thanks for mentioning the email Jacque. And you're correct, currently we don't handle multi site installs due to some issues with MU and our clusters.
Only hear good stuff from ZippyKid, but haven't used them yet. I've tried Page.ly before and just had too much downtime over a number of different months. Have reverted to hosting myself.
My experience with WPEngine : Jason Cohen saved my ass.

We went live on a WebbyNode VPS, thought we had configured our system well and suddenly we hit the top spot on HN. The server was on fire and Jason personally showed up and worked on migrating our site over.

Your configuration may have posed problems, you may have had what seems like a shitty support experience, but this public bashing doesn't help anyone in the startup community and reads as if it's motivated by emotions rather than a real problem-focussed attitude.

Give these guys a break and work with them one on one to find a solution to your problems. I'm SURE they'll move mountains to make you happy because that's in the team's DNA.

I worked "one on one" with them for weeks.

Weeks.

They did not move mountains. It was a parade of apathy. As soon as they could flick pass me to WebSiteMovers, they did.

Your positive experience is every bit an emotional one as my negative experience was.

I can totally understand your negative experience, I have email threads between myself and WPEngine as well as between common customers and WPEngine that prove quite the opposite.

It really sucks that you went through this. I'm just suggesting that you refrain from the public flogging and find another solution. I'm not affiliated with these guys other than as a client and I can tell you that I respect what they bring to the market.

I'm sorry, why is it my obligation to be nice to an expensive, specialist business who utterly let me down?

Especially given the content and tenor of their marketing.

>I'm just suggesting that you refrain from the public flogging and find another solution.

If you've spent more than a day in any online community you should have already learned that this, although it may not be preferable for the company at fault, is the fastest way to get something fixed or something changed.

It'd be irresponsible for Jacques to stand by silently and suck up all the time and money he wasted days after there was a great PR piece for WPEngine on the front-page of this site because it would lead to others having his issue which he could help prevent.

This is a community. It's not an advertisement source. People vote on things they find interesting and discuss them, it's not a place to only post positive things about up and coming companies and hide all sources of negative news.

This man speaks with special authority, given that I have sometimes publicly flogged his employer too.
Fastest != Most appropriate

You're advocating revenge. I'd just like discussions to be discussions. Not attacks.

The blog post read (to me at least) like an attack on a company with some obvious growing pains.

There are ways to write a complaint that doesn't involve pointing out Cohen's self congratulation. The way the post is ended wreaks of jealous hatred. That is not the way you constructively criticize.

I feel that I managed to control myself fairly well, all told. Nobody's parentage was questioned, nobody called "unsound", nobody excluded from the gentlemen's club etc etc.
Are we reading the same thing? I really don't think this post was an attack. It was someone describing a disappointing experience with a service that was supposed to make his side-projects easier to manage and having the exact opposite happen. It's a post about disappointment with a service written on his personal blog, I don't see any pitchfork calling or attacks or jealous hatred. Are you projecting something here?
Considering how well Jacques presented his arguments and experience, I'd say that this is one of the better examples of public flogging.

It seems like he's spent an extended amount of time complaining to internal channels, and he just needed to let other people know about his experience of not receiving the promised levels of service.

>You're advocating revenge.

Wait, what? Where? I saw nothing in that entire diatribe that sounded like a call to arms.

This is important information to me (I manage many wordpress sites). The author is just being frank, and that's a rarity in this world of shilling and murketing.

Where else am I going to read this?

    Give these guys a break and work with them one on one to find a solution to your problems.
In the article it sounds like instead of them trying to work one-on-one to fix the problems, they just outsourced it to a digital removal firm. If that's the response he got, in what way would you expect the OP to be able to work with them one-on-one?
I think it's a matter of how one approaches the issue. With WPEngine, we didn't bounce around e-mail after e-mail.

It took a 10 minute phone call. I knew the names of everyone on my case and they kept me updated through the process.

I'm SURE there was a big fuck up in the support process for the OP. I just feel that there are other ways to skin a cat, particularly in our community where we should be working hard to find constructive solutions to things like this instead of creating a cloud of negativity around young businesses.

I didn't mention it in the blog post, but I also rang them. Several times. Mostly I got placed on hold until I gave up.

I am in Perth, Australia. This means I need to stay up until 10pm in order to speak to them.

Like I said, that part is on me for not finding out their hours in advance.

(comment deleted)
I am using http://presslabs.com/ to host two of my blogs, one of them is a tech blog that gets a high amount of traffic every day and there has been no migration or downtime problems so far. I couldn't be happier.
This is a bit of a meta comment but it must really suck to wake up on a random morning and find a blog post like this about your service as the top story on HN.
Pissing off bloggers is probably the worst thing you can do, given that they'll likely blog about it..!
welcome to my morning ;)
A hosting company without 24 hour support is not a hosting company.

Go with Rackspace. They're expensive compared to budget hosts, but they seem to have an army of sysadmins available 24/7/365 that never say "that's your problem" / "not in scope".

PS: that redirect bug is probably a mismatch of your site_url setting in either wp_blogs, the wp_options table of your primary blog, or the wp-config.php. If the domain set in any of those 3 places doesn't match you get that endless redirect loop. Could also be htaccess sending you to www or no-www, and your config is set the other way. Its an annoyingly common issue when moving MU sites, and they should have been able to fix it.

I highly doubt it's something that simple, the OP had been self hosting a blog network on Linode. There's no way he made that simple of a mistake.
I've had a radically different experience with WP Engine and would happily recommend them to others, and have done so.

I went through 3 or 4 different providers, and the only place that i was really happy with setup, infrastructure and support was WPEngine.

As some of the other guys have stated regards the caching and their suitability to vanilla installs, i think that is most likely the case, and yes mine was a pretty vanilla install in the grand scheme of things, but i'd still choose them over anyone else.

Indeed, another blogger of my acquaintance (Joshua Gans, a really excellent economics blogger) heard about my moving to WPEngine and moved several independent blogs across. He's a fan of theirs.
WPEngine uses NFS/NAS which is a show-stopper. Dreamhost has the same issue, and GoDaddy (still?)

You never, ever, use NFS on something with hundreds of files to load for every page (ie. WordPress).

Every file takes 2-2.5ms to stat/load on NFS (from my tests on WPengine) which seems small but adds up fast.

It means you instantly have 700ms or more of overhead on the SERVER side to render a page (not transmit to the browser, just to render).

You can somewhat get around this by using an opcode cache and turning off file stat but there are still some files that have to be checked like static data read from the disk and turning stat off has other issues when editing code like having to flush the cache.

I recently removed some code that stat'd 4000 files over NFS. It took ~5 seconds, so it wasn't as bad as you say. Also, I'm pretty sure NFS can do better if you tune it.
Well obviously every NFS setup is going to perform differently. I personally benchmarked WPengine which is how I got the 2.5ms number.

What you are saying is 1.25ms on your network (5000/4000). That's twice as fast but still significant.

Yes, I agree..

Found this [1], which might be helpful. The linked python program shows really high numbers for me though. Hopefully it's incorrect or I just misunderstand what the numbers mean.

[1] https://communities.netapp.com/thread/13445

Since NFS and NAS both have caches I don't believe they would be a bottleneck.
I don't think you follow that with a local file system, file stat (basically to see if the timestamp has changed or file modified) can be done directly inside the local memory cache.

On NFS/NAS it cannot be done locally, because other devices may have had access to the remote filesystem and changed files. So it has to travel the network to make the query. Even if it doesn't have to touch the disk remotely and can fetch it from a remote cache, it's significantly slower than local. Even if it's only 1ms per file, if there are 300 files, that's 300ms (a third of a second) on the SERVER side, before it can start sending output to the internet.

There's a fair amount of criticism flying around about us on this thread, but that's one I can't make any apologies over.

There's all sorts of interesting things we're doing with NFS, along with caching layers, that ensure this is never a bottleneck -- and even provides benefits in other ways.

Also, to add, most customers actually don't run via NAS - we have /nas/ in the paths for other historical reasons.
I'm not really certain that this is a valid concern to add into the mix. Yes there is the stat issue with NFS, but if you have a fast filer then 2+ms is an eternity (for example: http://www.storageperformance.org/benchmark_results_files/SP...)

If WPengine is getting 2+ms response times then something is very wrong (are they using linux backed vanilla linux style NFS storage? ....bad, bad, bad). If not, then they need to investigate their load averages on their filers :P.

For what it worth - they are hiring. Probably to this exact reason - to bump up quality of their service (although their recruiter was not responsive, I guess I did not fit their requirements :))
I want a two word tl;dr summary of every article in the title:

How fast is... Apple.com? (tl;dr: not very); Wired's Review of the Microsoft Surface (tl;dr: pretty interesting).

Just write a better, non-link-bait headline.

"Apple.com takes 25 minutes to load" "Wired says Microsoft Surface is Quite the Stunner"

I tend to agree, but sometimes 140 characters come across as too emotionally charged, so imagine two words. Hence, "WPEngine sucks", which seems a bit harsh. Several recent articles could be summarized as "Founders dumb", "VC stupid", "SV sucks".
I sincerely doubt many reasonably interesting articles can be accurately summed up in two words.
I was only being semi-serious but there are certainly a good many titles that could be written in an ambiguous manner.

The original author may want to use that title but that's not to say the person who posts it to HN has to.

could be written in a LESS ambiguous manner...

Oops.

My least favourite part of this type of sharing is when all the competitors come along to now try to pitch their wares. It's not clever, it's not classy, it's just lame.

I don't doubt this was a shit experience, it certainly sounds incredibly infuriating.

Jason & his team work super hard, and have thousands of happy customers, so they're doing lots right. Even the greatest of companies can screw things up now and then, especially in non-straight-forward situations. I'm sure he'll put it right.

The important lesson to be learned here is that even though higher price points tend to shield you from the bulk of toxic customers, you'll still find the occasional toxic customer at even the highest price point from time to time.

As nice a guy as this blogger probably is, he's the bane of services like WPEngine. There is no way to service the needs of his complex edge case of a system for anything like $250/month. They are guaranteed to lose money on him, and the only hope they have is to convince him to leave the service as rapidly as possible to avoid being sucked into the time sink that dealing with him is going to become.

This is the reason you have a "refund" button on the customer page of your admin console. A quick email apologizing for not being able to meet the needs of his unique setup and a refund of his payment, as soon as it became apparent just how impossible things were about to become would have solved this completely.

Or at least one hopes so. The tough thing about dealing with toxic customers is that they grow on you slowly and you don't notice at first. Then one day you realize you've spent four hours just digging into issues and writing emails to this one person who you're probably going to have refund eventually anyway.

I've never met the guys on the WPEngine team. But I feel for them after reading this.

> the only hope they have is to convince him to leave the service as rapidly as possible to avoid being sucked into the time sink that dealing with him is going to become

Then just tell him that the service isn't for him at that price, like Page.ly did.

Pagely tried to help him. Tried really hard. WPEngine didn't even try - their cust support people just read from the same script over and over. And then charged him $500 for support.

If they can't support customers like him, they shouldn't charge him $500 for the time it takes to determine that.

Just to be absolutely clear on the facts here:

WPEngine are not billing me the $500. WebSiteMovers are.

I approved that work and I will pay that bill, just as soon as they help me out with some technical details. WSM are not in the wrong here.

However I do intend to present an invoice to WPEngine for the amount that I have paid to WSM.

Multisite networks are not an edge case.

It is part of the mainline code for Wordpress. It is present in the base installation.

WPEngine specifically advertise that they support it.

Also ... since when did we start considering people who expect to get what they were promised by the sales copy to be "toxic customers"?

ZippyKid wins. Hosts of open source software don't have to support every feature, and the right way to omit a feature is to say that it isn't supported in the documentation.
I think that's unfair. So if you sign up for a premium service because they say they can help you solve problem X, and then you follow their exact instructions to no avail and get the runaround from customer support, you're a toxic customer?

I don't know what your experience with WPEngine is, but suffice it to say, this post is not the first complaint I've heard, and I've personally had...issues with them as well. And my setup is even more vanilla than the one described in the linked post (which is pretty vanilla, honestly). I love Jason Cohen's contribution to the startup community and I want to love WPEngine, but I am starting to wonder if something is amiss on the technical / customer service side there.

Jason, what's up with this comment?

Where did you find this 'complex edge case of a system' in his post? This 'unique setup'?

Why do you think 'impossible things were about to become' when he was able to easily move the sites to WP Engine... after paying a 3rd party to do it?

He didn't describe a single special requirement that I can see. Just plain old WordPress with some existing data to move. Not even a large network or a large amount of traffic. This is supposed to be WP Engine's bread-and-butter customer.

All of WP Engine's plans other than "Personal" say "Hey WordPress MultiSite customer, come here! This plan is for you! Hundreds of thousands of visits a month and 20GB+ of storage!" http://wpengine.com/pricing/

Why are you calling him a 'toxic customer'? He's been running this WordPress install himself on an unmanaged VPS. He wasn't wasting their support staff's time on unnecessary questions -- they were wasting his! That's not a 'toxic customer', that's a 'toxic business'.

I'm having a hard time connecting this comment to the HN submission at all.

He also didn't describe much of any of his technical requirements. I'd say it's a fair assumption his setup was enough different to cause an issue. Whether or not that's a fireable offense is up to the company.
It's a bog ordinary Wordpress multisite installation. The most exotic thing about it was WP-supercache and WPEngine have clever inbuilt logic to safely disable that when you migrate across.
I'm more curious what VPS package you had powering your stuff at Linode and how much you were spending on it each month? Reason being that 250 USD buys a LOT of server these days and it is surprising you ended up with underpowered hardware if your VPS wasn't giving you any problems.

If you're looking to upgrade in the future, you might be much better off throwing a dedicated server at the problem and setting a nightly backup to S3. Also, thank you for posting.

I use 2 512Mb Linodes. One for nginx/PHP, one for MySQL. Putting these on different servers is by far the biggest leap in performance I ever got.

Monthly cost is about $40 ... unless you count my time.

He did, however describe the eventual problem: his comments table was larger than what Wordpress's out-of-the-box feature (Recent Comments) was able to sort. He had very overpowered hardware in his custom situation. It strikes me that this was an extroardinary but still within band problem that WPEngine should have eaten the cost for solving OR tell him MUCH more swiftly they couldn't.
The eventual problem was just the straw that broke the camel's back.

The weeks of shitty service beforehand are what really did it.

Do you not read the part about him having over 400k comments? Don't you think that is a bit outside the regular scope?
I don't think that was in the post originally. Either way, it's not outside scope, it's not a big deal, and it's not an excuse for weeks of failing to provide support. They didn't fail to even move the site because of the number of comments. Matt Mullenweg, the founder of WordPress, commented himself that they host many blogs with far more comments than that without issue.
For sites that are large enough/can afford it, I cannot say enough good things about WordPress VIP - we have 20+ sites on VIP and could not be happier with it.

Since they 'are' WordPress they can make pretty much anything that does not modify core, work. It's not for everyone, but for the right sites (high traffic/high availability/generating revenue) it's fantastic.

Interesting, thanks. I suspect we're still too small, but I'm checking the service out anyway.
As a seasoned WordPress developer and entrepreneur, the OP is making wrong assumptions. If I was the WPEngine owner, I won't be bothered by seeing him go.

WPEngine provides WordPress hosting services. They are professional and they do it for $250/month (which plainly means that their target audience is professional bloggers). For $250/month, they don't provide consultancy service as why your particular setup couldn't install in their server. This is your responsibility.

Setting up a WordPress blog can be easy (5 minutes) and complicated (Professionals charge up to $350/hour). What you are asking for is someone to move your blog. Fine. You are responsible for that. WPEngine offers you a "6 hours coupon" from another service. That is, they are not responsible for that.

Page.ly are probably doing this to get more clients. This is the wrong thing to do, because their support service can't scale this way. I have been doing this to bootstrap my services and it had more negative consequences than positive ones.

tl;dr: The OP is expecting WP consultancy services for signing up to WPEngine.

> Page.ly are probably doing this to get more clients.

... huh?

Edit: are you suggesting that the poor service I got from WPEngine is in fact a cunning conspiracy to boost Page.ly sales?

Because ... well at least temporarily, yeah, I guess it worked perfectly.

"Cui bono?"

-dramatic violin music-

So you are the OP. I won't argue with you much. I'm in the WordPress business for many years. What I meant Page.ly is providing a generous support service to keep its' clients. This is a bad strategy because it makes wrong assumptions of what their main service is.
Yes, I guess they should focus more heavily on the terrible customer service market instead.
(comment deleted)
No, what he means is that Page.ly went above and beyond what is reasonable for the price you're paying them. Yes, it was very nice of them, and most likely they did it because they are a young and hungry company. At the end of if all, I don't really get a sense from what you wrote that WPEngine service was terrible. It sounds like you were a difficult customer with difficult architectural issues that were ultimately the root cause of your performance issues.
I don't see how an ordinary multisite installation -- which they specifically namecheck on their website -- is a "difficult architectural issue".
It it really an ordinary multisite install though? Do you have 188590 comments? Wouldn't you have trouble with any hosted providers if you are doing operations on that many records in memory (whether that is the fault of WP, or custom plugin code, it doesn't really matter)? If that's the case, it looks like your site just isn't a good fit for this type of service, but millions of others (including multi-sites) with say a few hundred posts and a few hundred or perhaps thousand comments are.

It's a shame that you had to go through all of this trauma to discover this, and there are obvious support failings here, but it does seem that your site is out of the ordinary in some respects, just because of the number of comments on one WP instance.

HN cracks me up sometimes. Above-and-beyond customer service is now "bad strategy"? Hah. :-)
Yes, it is. Because it creates bad assumptions. Now, you'll have a ton of noobs assuming that you do that and that for free because you already did it.
There's a section in Guy Kawasaki's Rules for Revolutionaries which talks about exactly this. Rules is still my go-to text for general business management advice. I'll quote the relevant parts here.

"Get Over the Paranoia: The story may be apocryphal, but Nordstrom supposedly allowed a customer to return an automobile tire that he insisted he bought at the store. Of course, Nordstrom doesn't sell tires.

"The paranoid would ask, ``But what if everyone who owns a car returned tires to Nordstrom? Nordstrom would go broke!'' But people won't (and don't), and Nordstrom will never be in danger of going bankrupt for accepting returns of goods it doesn't sell.

"Managers are afraid to implement customer-pleasing, revolution-catalyzing policies because they are afraid that too many people will take advantage of these policies, and they'll end up with the equivalent of a store full of tires."

The entire section goes on to talk about employee empowerment, allocentralism (putting yourself into your customer's shoes), examples from other businesses that either do this or don't, and so on; the relevant summary part from the text would probably be, "However, if your competition is asking people to do something suboptimal, then it's creating an opportunity for you."

So, straight up: it's certainly possible to run a business the way that you're describing. In fact, I'd guess that that's how most businesses run: they do as little as possible to make their money and meet -- but not exceed -- their customers' expectations. So you're not wrong as far as that goes.

But you are wrong to assume that they'll have "a ton of noobs" if they treat one particular customer really well. And, you're equally wrong to assume that having "a ton of noobs" is a bad problem to have; from another part of the same section of the book: "For example, a Whirlpool employee taped a news program's interview with Gail, a woman with several children and a full-time job. Whirlpool employees were challenged to provide appliances that would ``take care of Gail.'' In response, they redesigned the stovetop of Whirlpool's CleanTop to be completely flat, without grease traps or dirt-collecting crevices, and they created the Quiet Partner dishwasher, so the noise of the dishwasher wouldn't distract Gail. Viewing chores through Gail's eyes has helped Whirlpool introduce significant product enhancements."

Having "a ton of noobs" would mean a ton of new customers that are paying you money to show you what is wrong with your product or service. They are a great resource to have, if you take advantage of them. If you can identify one problem that 35% of "a ton of noobs" all have in common, and if you can automate the solution to that problem (or make it significantly easier to solve), then you have just achieved differentiation in a crowded market.

Yes, it can be painful in the short term. You might have to hire some more help -- and you might have to streamline some of your other internal process, like your ticket system, which will also pay dividends in the future.

Avoiding pathological customers is certainly good advice for freelancers, consultants, and really small shops. However, it is not necessarily the only good business strategy.

>As a seasoned WordPress developer

You realize that puts your credibility in the negatives right? Wordpress is infamous for being one of the worst pieces of software ever created. The fact that your software is terrible does not mean a company whose sole purpose is to be experts at making that terrible software run is doing good by failing to make it run.

Puh-leeze. Everyone thinks some other software is terrible. Wordpress might have some funky bits, but a) it's gotten a lot better over the years, b) it's pretty damn popular, and c) end users don't care if the guts suck if the UI works and it does the job.

Get over yourself.

Did you have a point to make? Shitty software being popular doesn't make the authors of that shitty software any less biased when discussing that shitty software.
I don't understand the correlation between the two. From your perspective, since the financial market is terrible, all people working on finance are terrible.

WordPress is a bad piece of software. But there is a lot of money there, and you'll find lots of smart people working on it.

I don't understand how you can not understand. You are in part responsible for this terrible software. Which is horribly bloated and inefficient and runs like crap. So it is in your best interests to try to pretend anyone who has a terrible experience with it is "doing something unusual and different". The reality is we're talking about a setup that is about as simple as wordpress can be. The fact that "as simple as wordpress can be" is an insanely complex monstrosity doesn't let people who said "we can handle your complex wordpress installs no problem" off the hook.
Yeah, WordPress powers (by some estimations) 18-20% of the freakin' Internet. It's not flawless, but it's a great piece of software that makes millions of people (and developers) happy.
"It is popular" does not in any way contradict my statement which was "it is a pile of shit". Internet explorer used to "power the internet" too. It was also a colossal pile of shit.
Open mouth insert foot.
I host several sites on WPEngine and we've had some relatively minor issues. But we're getting ready to switch to multisite and now I'm nervous. We're generally happy, but we haven't been blown away.

Jason, if you're not scrambling to deal with this mess, you will be soon. And the main message I have is that you need to back off the marketing side a little and let the technical and customer service side catch up.

Godaddy makes an incredible amount of money, but they're still a terrible company from a technical and customer service perspective. But their marketing is so effective that they can get away with that [1].

I wonder if Jason Cohen's position in the tech community and general marketing prowess has allowed WPEngine to get ahead of themselves? I know when I read Patrick's blog post about WPEngine, I was really excited and it was a service that I wanted to like. And I do like them, but I'm not excited about them at all anymore.

1. Interestingly, Google is the exact opposite, and their customer service makes me want to take a spoon to my own eyes.

Recently I had an experience on another website where one of the founders basically insulted me personally because I made a sarcastic quip.

Another founder contacted me by email and apologised. It was dignified and mature of them to do that.

And so I consider that matter closed, which is why I am not going to name names here.

The contrast is unflattering and it's a shame.

I really, genuinely, wanted to like WPEngine.

It looks as though those tweets were not aimed at this story, but rather at a different story on HN yesterday.
WordPress, like most LAMP apps of its era, makes a series of architectural assumptions that turn out to have horrible impact on non-functional qualities … but that’s another rant for another day.

Until I came to this sentence, I was coming to the conclusion that CMSes fall down fairly quickly once you add in large data, large traffic, or complex installs. For the life of me, I can't see why people continue to use CMSes for large projects anymore, when they could potentially code something up faster in Django/Symfony/Rails and, here's the kicker, know the codebase inside and out.

My client has simple marketing (read: 99% static) websites that are built on Drupal. Sounds good at first, and it almost makes sense... and then you get to the non-functional requirement that it needs to withstand a marketing event that can bring 2k+ concurrent users for over 24 hours. Then you watch Drupal meltdown the server.

When I applied to do my honours year, I wrote four project proposals. One was to basically turn the architecture of such applications inside out by making the virtual machine the target of deployment. This allows you to get out of a lot of architectural holes that shared hosting forces on you.

Email me if you're interested in reading the proposal, I have it lying around somewhere.

What doesn't make sense about it?

I can see a lot of scenarios where you start with a CMS because it does a good job of doing what you need. Why reinvent the wheel?

Scaling issues is going to occur in all code, even code you write and know inside and out. It just means you become entirely responsible for debugging and re-architecting to improve its performance.

@Drupal specific comment, you could easily throw nginx as a reverse proxy with a cache in front if everything is static and be done with it.

Valid point. What I keyed on in the article is that it is a complex install, and that tells me that maybe his CMS is working against him. In those cases I like to remove the impediment and simplify the data model (which should simplify the code).
I can't comment on the specifics but he said it was something default in the comments table and modules I think. For a big project like wordpress, I would try to understand why a certain decision was made. Try to understand the consequences of that decision vis-a-vis how I would want/expect it to work.
Doesn't even have to be static to put nginx in front of it! As long as you can set proper cache headers from your script (in this case Drupal, and I imagine it does), you can cache even dynamic-ish sites quite well!
There are loads of people who "know the (wordpress) codebase inside and out". There are loads of people who don't. There are loads of people who make crap sites in Django/Symfony/Rails as well.