Interesting results. Will you be running any tests on the bar you added to the bottom of your site? I'd be interested to see if that increases conversions.
Yes, definitely. Right now watching the statistics of how users interact with the bar. Next step is to split test it with different versions and also presence and absence of versions.
Though I noticed that not as many people interact with the bar as I imagined. CTR of creating a new test using that bar is < 5% now. Any feedback?
If I remember correctly, those bars are usually a different colour from the rest of the page (probably a shade of blue or green). Yours looks just like a footer because it matches the colour of your page header (black).
I didn't even notice the bar the first time I read the article, I actually had to go back to the site to see what you were talking about in this comment.
Don't you think the different button styles are affecting the results? The "Watch the video" button looks more familiar to me (it's like an Amazon button).
The definite answer can be had using a multivariate test with button style and text. But I doubt that style here would be a major influencer because landing page focussed all attention of the visitor on the button. So, visitors would have definitely not missed noticing the button (which is the primary role of button style). Button text would make them decide whether to take next action or not.
Given that you were able to increase CTR, I don't know how you can be so sure that visitors were not missing noticing the button. And even if you had good reason to believe that, why didn't you just change the button text instead of replacing it with a new button?
Sorry, I'm probably coming across as nitpicking, but I'm just trying to understand how people use A/B testing.
No, you are totally right. The way you describe is how an actual A/B test should be done. But in practice companies resort to making changes in A/B test which involve changing more than one variable (in this case button style and text). That's because they can't afford the time and traffic it will take to methodically test one factor at a time. Also, primary motivation for many companies is to increase conversions. Knowing and deducing what increased conversions is a secondary priority.
By the way, I did not do this A/B test. I just interviewed one of my customers who did this test.
We need your support and trust!!! Dear friends, please temporarily stop your footsteps To our website Walk around A look at Maybe you'll find happiness in your sight shopping heaven and earth You'll find our price is more suitable for you.
Welcome to our website ( http://www.findsoso.com / )
Thanks to the support!
You said I will show you how to do it, thats why people expect a video of you showing how to do it and Watch Video worked better.
When you say Instant Access it sounds like you are going to allow the visitor access tons of information (in text format for some reason)
I bet if Google Books was a subscription only place, your Get Instant Access would outperform the Watch Video button by 95%
1) Get instant access means nothing to me. Watch a video does, and if I take the time to actually watch the video I'm more likely to commit if I find your product worth my money. There's a disconnect on what 'Instant Access' really means and what potential customers think of it.
2) The first button is amateurish (design wise) in my opinion. There is something wrong about it, the colors, the composition, the blue line under it, and most of all the text on top of it just feels wrong. The second button is much much better and it has a bit of a resemblance to Amazon's buttons, which might be a good thing depending on your userbase.
I'm inclined to think that the button design has a much larger role on the conversions than the copy though.
I would guess that the audience is familiar with 'Get Instant Access' as something that normally precedes a porn site asking for payment and therefore has some negative associations.
There are so many things they changed for the 2 different test scenarios that I don't think that we can get anything out of the test except B is better than A.
This experiment doesn't really tell me much except that maybe it helps conversions if people learn about my product more rather than just asking them to sign up right off the bat. And that is a BIG maybe.
If they ran an experiment which all they do is to change the text from 'Get instant access' to 'Watch a video' and the results are significantly different, then we probably would be able to draw more from the findings.
22 comments
[ 3.2 ms ] story [ 66.5 ms ] threadThough I noticed that not as many people interact with the bar as I imagined. CTR of creating a new test using that bar is < 5% now. Any feedback?
EDIT: clarified
Sorry, I'm probably coming across as nitpicking, but I'm just trying to understand how people use A/B testing.
By the way, I did not do this A/B test. I just interviewed one of my customers who did this test.
I shouldn't doubt your A/B testing ninja skills.
The button test, even combining style and text, is still quite a small change to test. Don't your customers want to test more drastic changes?
We need your support and trust!!! Dear friends, please temporarily stop your footsteps To our website Walk around A look at Maybe you'll find happiness in your sight shopping heaven and earth You'll find our price is more suitable for you. Welcome to our website ( http://www.findsoso.com / ) Thanks to the support!
( http://www.findsoso.com / ) ( http://www.findsoso.com / ) ( http://www.findsoso.com /
1) Get instant access means nothing to me. Watch a video does, and if I take the time to actually watch the video I'm more likely to commit if I find your product worth my money. There's a disconnect on what 'Instant Access' really means and what potential customers think of it.
2) The first button is amateurish (design wise) in my opinion. There is something wrong about it, the colors, the composition, the blue line under it, and most of all the text on top of it just feels wrong. The second button is much much better and it has a bit of a resemblance to Amazon's buttons, which might be a good thing depending on your userbase.
I'm inclined to think that the button design has a much larger role on the conversions than the copy though.
This experiment doesn't really tell me much except that maybe it helps conversions if people learn about my product more rather than just asking them to sign up right off the bat. And that is a BIG maybe.
If they ran an experiment which all they do is to change the text from 'Get instant access' to 'Watch a video' and the results are significantly different, then we probably would be able to draw more from the findings.