Relevant to folks here: what do you think a 60% increase in sales in the Netherlands is worth to Hyundai? Divide by the number of hours you think it would take you to redo this page from scratch. Does that strike you as lower or higher than your hourly as a programmer?
1. 60% increase in 'test drive requests' is not 60% increase in sales.
2. There are thousands of other sites which don't have as much money as Hyundai.
3. There are probably other cases where 60% increase cannot be reached.
4. When I am 'programming' it may happen that I bring a huge revenue to my client, but my client will not pay that for me. He will choose an other programmer if I am too expensive. This might easily be the case with marketing agencies I guess.
5. For me the most relevant question is barrier to entry into the market. I am a quite intelligent person and I might be able to learn online marketing deeper than currently I am involved. But how can I convince Hyundai that they should use my marketing agency? I am programming since I was 12 years old, (I am 36 now) but still I cannot convince companies in rich countries like the U.S. to give me interesting hard tasks. (So I mostly deal with local Hungarian clients, which are less 'rich'.)
You are right, but I think you mis-interpreted Patrick's comment. I guess he is saying that you can probably have a great ROI for the time invested in testing and optimization. Of course, this does not mean that you _always_ have to be doing testing and optimization.
Right, companies can have a great ROI for the time invested in testing and optimization. That is right.
I interpreted patio11's comment as it may be a good idea to shift into the online marketing contractor business instead of being a programmer. Which can be a good idea, I don't disagree, I am just not completely sure. (That's why I said it is a bit complicated.)
I keep telling folks to describe themselves in terms of making companies metric truckloads of money as opposed to using the word "programmer", which is "cost center peon with anomalously high wages."
This is one thing in the bag of tricks that would justify charging $BOATLOADS at one's next salary or contract negotiation. (A/B testing is certainly not the only thing with that desirable characteristic.)
On the other hand, the original page is very uninteresting and a standard CMS page. Any change at all would have increased conversions I think. The starting point was pretty bad to start off with.
Hyundai may have done tens of other tests on this page alone which failed to increase conversions. You would not know about it because only positive case studies are published (we do try to publish some negative case studies though)
Only 1 in 8 for AppSumo yielded positive results which, of course, means nothing for Hyundai. The result that Hyundai achieved by adding clear calls to action on a page which had none is not very surprising to me at all. You don't need multivariate tests to discover that.
What I find great about doing multivariate tests is the results which are surprising due to factors which are not intuitive... the kinds of insights that an experienced marketer would never have thought to suggest. In Hyundai's case, it seemed more of a Marketing 101 lesson than a triumph of A/B testing.
It was clear that the updated version would perform better since the old one really had no clear next step but it's surprising that the overall impact was 60% increase in leads, not just proceeding to the next page.
I'm getting increasingly skeptical about these conversion rates. Where's the science behind this? What is the quality of these tests? Where's the follow up to check if this resulted in increase in actual sales?
My question is if the multivariate testing is really necessary (at-least for smaller sites) ?
With a big budget you can try a lot of designs and find the most effective one by a few percent -- but the changes they made were obviously going to increase conversions.
* A big call to action over a little link in side bar?
* Reasonable sized photo on the right?
I would have just thrown those two in without testing and compared the SEO friendly text which isn't as obvious.
In this case A/B testing would also have worked with a big call to action but in general multivariate testing makes sense if you have multiple hypothesis to prove or disprove (instead of just aiming to increase conversion rate by trying different changes)
21 comments
[ 2.9 ms ] story [ 48.6 ms ] threadYeah, that's right.
1. 60% increase in 'test drive requests' is not 60% increase in sales.
2. There are thousands of other sites which don't have as much money as Hyundai.
3. There are probably other cases where 60% increase cannot be reached.
4. When I am 'programming' it may happen that I bring a huge revenue to my client, but my client will not pay that for me. He will choose an other programmer if I am too expensive. This might easily be the case with marketing agencies I guess.
5. For me the most relevant question is barrier to entry into the market. I am a quite intelligent person and I might be able to learn online marketing deeper than currently I am involved. But how can I convince Hyundai that they should use my marketing agency? I am programming since I was 12 years old, (I am 36 now) but still I cannot convince companies in rich countries like the U.S. to give me interesting hard tasks. (So I mostly deal with local Hungarian clients, which are less 'rich'.)
I interpreted patio11's comment as it may be a good idea to shift into the online marketing contractor business instead of being a programmer. Which can be a good idea, I don't disagree, I am just not completely sure. (That's why I said it is a bit complicated.)
This is one thing in the bag of tricks that would justify charging $BOATLOADS at one's next salary or contract negotiation. (A/B testing is certainly not the only thing with that desirable characteristic.)
http://www.youtube.com/watch?v=y-AXTx4PcKI#t=2m35
Hyundai may have done tens of other tests on this page alone which failed to increase conversions. You would not know about it because only positive case studies are published (we do try to publish some negative case studies though)
What I find great about doing multivariate tests is the results which are surprising due to factors which are not intuitive... the kinds of insights that an experienced marketer would never have thought to suggest. In Hyundai's case, it seemed more of a Marketing 101 lesson than a triumph of A/B testing.
With a big budget you can try a lot of designs and find the most effective one by a few percent -- but the changes they made were obviously going to increase conversions.
* A big call to action over a little link in side bar? * Reasonable sized photo on the right?
I would have just thrown those two in without testing and compared the SEO friendly text which isn't as obvious.