Are you running Rails 3? We were looking at various gems the other day for ab testing for a side project, and they all (including seven_minute_abs) seem woefully outdated and abandoned.
You will also get a much better indication on the type of audience your site is attracting, enabling you to tailor content and better position your offering.
Exactly! We're monitoring which customer persona gets the most clicks (with good old Google Analytics). Down the line we might even try 8-10 different customer types with A/B testing and see which ones attract the most clicks.
Yeah, and if there is one particular segment that you want to click, but aren't clicking; you can experiment with the phrasing, accentuate the button, or write content aimed at that particular audience.
I like the idea, but you have to pretty sure you know exactly who your customers are don't you? I would move on if I wasn't on the list. Features are more universal, which I guess is why they are also less effective.
I agree. We're pretty sure who are customers are, but there is a chance the unknowns could slip through. We might do another A/B test simply adding "brands" as one option since that's a customer persona, but very generic.
I like how Foursquare appeal to their customer personas, "merchants", "brands", "developers", without making it too obvious but still very findable.
you have to pretty sure you know exactly who your customers are don't you?
Yes, and that is itself a huge benefit. If going this route forces you to think about who will use your product (hint: it's not "everybody!") then it's a net win for everyone involved. Understanding your customer is key to building a useful product.
I was almost amazed at the correct math. A "200% improvement" is definitely "three times what it was previously." But I'm not entirely convinced +200% is a "threefold increase" (I'm a math guy, not a 'human language is fuzzy and ambiguous' guy.)
I thought I was pretty clear that I don't know what "threefold increase" should mean. Sounds like "an increase of three times normal," that's normal (x) plus three times more (3x). so x+3x seems like a reasonable definition.
Yeh, this is weird. A "50% increase" is clearly "in addition to", whereas "threefold" reads as "in comparison to". I have no idea how to interpret a "onefold increase".
I see this sort of thing a lot on sites for products trying to sell to large businesses, sometimes to the point that it's hard to find an actual product description or list of features. I have, several times wanted to yell "but what does your product actually do?" at the person responsible.
I am obviously not the target market here, but why do you think this works?
If I listed all the features of my product, it would overwhelm most people, since 80% of them are only going to use 20% of the features. Those 80% aren't very tech savvy, and reading through everything would make the product sound too complicated.
You can imagine that when they show up for the first time, they most important first reaction is: "oh, so this product is for people like me".
Though you and I might be people who want to see every single feature first, we're in the minority (ie. not scared that the software will be too complicated).
Because if you were in one of the target groups, you'd be nodding your head as you read it saying: "These people understand my problem(s)".
At that point the features don't matter. They understand your problem so well, that odds are they now how to fix it - most potential customers with problems figure someone else knows something they don't.
In fact, the target customer is likely to be more receptive to not needing a particular feature if they are convinced that you understand the problem.
> Because if you were in one of the target groups, you'd be nodding your head as you read it saying: "These people understand my problem(s)".
I can't speak for Zak but this is certainly not true for me and many others.
If I can't find out what you actually offer, after selling me a nice story about solving my problems, I walk away. It's such a frustrating experience that I assume doing business with them will be just as frustrating so I don't bother contacting them with my questions.
I had this happen recently, finding out later that they did almost exactly what I was looking for... and it was a technical product. I can respect that they wanted to market primary to non-technical types but if I can't get the info anywhere on your site, I'm not interested.
To be clear I'm not advocating secrecy. The website should say something about the product of course.
People like you, or rather customers like you (and me) are usually the biggest pain in that ass for a company. You not being interested is a much larger problem for you than it is for them. I'm not being insulting, but I am serious.
If you are technical enough to read HackerNews, you are probably technical enough in your chosen domain to write the software yourself, or a least come very close to doing so. That means you're also more than likely to push the envelope, demand excellence and try to make the software do something that 99% of the user don't do. While this can sometimes be beneficial, when you have a targeted market it is often just very loud noise.
People like you and I tend to be early adopters, and while we are critical for determining market fit, we tend to be a giant pain in the ass once that fit has been found, because most often we are not it.
It all comes down to understanding that you are not the target market, even if it's still true that you would have use for the product.
"The premise is that while features are needed for your product to function in a specific way, you are selling the benefits of using this product and not the features. Focusing on the benefits of use, is fundamentally an emotional appeal."
Thoughts:
If you see the emotional appeal as verifying the benefits of the features, it makes sense. If you see it as them trying to sell you something by using emotion and hiding the features, it won't. I think the best way is a balance of the two where you show the features as linked to emotional appeal. It hits both groups well.
That's a fair point. Customer segmentation can be overdone.
There was an interesting article on HBR about research Clayton innovator's-dilemma Christensen was doing into customer segmentation vs. a job-to-be-done approach (http://hbswk.hbs.edu/item/6496.html).
Once someone clicks on a link that calls out to them as a customer, you need to tell them pretty quickly what you can do and how you can do, i.e. what job you can help them do.
On brandregard.com we simply moved the discussion about features down one level, behind a customer segment link. We also discuss features in length below the fold on the landing page.
In face to face selling you already know who you're dealing with and have prepared to address that person's (persona's) needs. With a website you never know what person (persona) will see it, so you need to cover more angles.
Awesome to see an Icelandic startup on the HN frontpage!
This post is a great example of how applying typical established marketing knowledge to a product presentation can improve things a lot.
It's well known that presenting benefits, use cases and personas is a more successful approach than presenting features and screenshots. - For most users - I thought every A/B test in the world had already proven that.
And yet we have all these startups out there presenting nothing but features. I think presenting features might probably work better for techies and more tech-savy people, but for the great majority stuff like this will always work better.
This is kind of what Apple does. Apple sells the vision and value of the product, not the features and functionality. Apple wants people to buy into the vision
CTR is important, but it isn't the whole story.. I'd love to have more details about how it impacted your whole funnel (even if it is just a simple statement like "ratios after clicking through were the same before and after the change, so we also tripled activations" or similar.)
29 comments
[ 3.0 ms ] story [ 76.0 ms ] threadI'd be interested to hear about other people's experiments both with A/B tests and customer segmentation.
You will also get a much better indication on the type of audience your site is attracting, enabling you to tailor content and better position your offering.
I like how Foursquare appeal to their customer personas, "merchants", "brands", "developers", without making it too obvious but still very findable.
Yes, and that is itself a huge benefit. If going this route forces you to think about who will use your product (hint: it's not "everybody!") then it's a net win for everyone involved. Understanding your customer is key to building a useful product.
</offtopicrambling>
Like I implied, human language is ambiguous.
I am obviously not the target market here, but why do you think this works?
You can imagine that when they show up for the first time, they most important first reaction is: "oh, so this product is for people like me".
Though you and I might be people who want to see every single feature first, we're in the minority (ie. not scared that the software will be too complicated).
At that point the features don't matter. They understand your problem so well, that odds are they now how to fix it - most potential customers with problems figure someone else knows something they don't.
In fact, the target customer is likely to be more receptive to not needing a particular feature if they are convinced that you understand the problem.
I can't speak for Zak but this is certainly not true for me and many others.
If I can't find out what you actually offer, after selling me a nice story about solving my problems, I walk away. It's such a frustrating experience that I assume doing business with them will be just as frustrating so I don't bother contacting them with my questions.
I had this happen recently, finding out later that they did almost exactly what I was looking for... and it was a technical product. I can respect that they wanted to market primary to non-technical types but if I can't get the info anywhere on your site, I'm not interested.
People like you, or rather customers like you (and me) are usually the biggest pain in that ass for a company. You not being interested is a much larger problem for you than it is for them. I'm not being insulting, but I am serious.
If you are technical enough to read HackerNews, you are probably technical enough in your chosen domain to write the software yourself, or a least come very close to doing so. That means you're also more than likely to push the envelope, demand excellence and try to make the software do something that 99% of the user don't do. While this can sometimes be beneficial, when you have a targeted market it is often just very loud noise.
People like you and I tend to be early adopters, and while we are critical for determining market fit, we tend to be a giant pain in the ass once that fit has been found, because most often we are not it.
It all comes down to understanding that you are not the target market, even if it's still true that you would have use for the product.
My comment when sharing this link to others:
"The premise is that while features are needed for your product to function in a specific way, you are selling the benefits of using this product and not the features. Focusing on the benefits of use, is fundamentally an emotional appeal."
Thoughts:
If you see the emotional appeal as verifying the benefits of the features, it makes sense. If you see it as them trying to sell you something by using emotion and hiding the features, it won't. I think the best way is a balance of the two where you show the features as linked to emotional appeal. It hits both groups well.
There was an interesting article on HBR about research Clayton innovator's-dilemma Christensen was doing into customer segmentation vs. a job-to-be-done approach (http://hbswk.hbs.edu/item/6496.html).
Once someone clicks on a link that calls out to them as a customer, you need to tell them pretty quickly what you can do and how you can do, i.e. what job you can help them do.
On brandregard.com we simply moved the discussion about features down one level, behind a customer segment link. We also discuss features in length below the fold on the landing page.
The personas work better. Proof is in the numbers.
This post is a great example of how applying typical established marketing knowledge to a product presentation can improve things a lot.
It's well known that presenting benefits, use cases and personas is a more successful approach than presenting features and screenshots. - For most users - I thought every A/B test in the world had already proven that.
And yet we have all these startups out there presenting nothing but features. I think presenting features might probably work better for techies and more tech-savy people, but for the great majority stuff like this will always work better.
Great job. Iceland ftw!
https://web.easydns.com/About/