In my experience "image beats text" is a really common result of an AB test. For an encore, I'd suggest testing an image of a (smiling) person in that same space.
Worth testing, but in my experience stock photos of people make websites look cheesy and fake. Untrustworthy, even. Like one of those fake sites to gather extra domain traffic.
Another advantage of the dog is how it's looking to the right. After your eyes dart to the puppy, you naturally follow it's line of sight to see what it's looking at, and you end up at the sign up button.
If they use an image of a person, they would need to be glancing or gesturing in that direction as well.
Actually, I'd like to see them A/B test the puppy against itself, but flip it horizontally so it's looking to the left in one scenario.
At one point in my career I had two job offers on the table - my wife was helping me decide, and when she saw that one of the companies had a dog on their site, she said I had to work for them. In the end, I did accept that job offer for my own reasons - but it's interesting how the emotional response is to the cuteness factor.
Coincidently I now work at a company with currently 4 dogs in the office.
It's interesting you mention that. I remember just a month ago, seeing pictures of dogs in the offices at Palantir (on their website), and wondered if seeing that became the deciding factor for prospective employees.
"Coincidently I now work at a company with currently 4 dogs in the office"
Jesus, and I complain about how hard it is to concentrate in a noisy open plan office. A bunch of non-human animals roaming about might just tip me over the edge.
At least where I worked, the dog just kept us company and wasn't obnoxious at all. It was very calming to just play with her when you're stuck somewhere, most of the time petting the dog yielded beautiful technical solutions. She should have been on the payroll (well, she did get snacks, but was worth way more than that IMHO).
I was concerned about this at first - but overall it's really great. There have obviously been a few altercations between them, but those are rare. One of the dogs is owned by a member of my team so he sits right by me all day - occasionally he'll come by just to be pet, and when I am stuck on something work related we play fetch for a few minutes to clear my head. Our daily standups are pretty awesome too since we generally have a dog or two there with us, for example, yesterdays standup: http://i.imgur.com/3DABSti.jpg
(to elaborate for the downvote-debbies: there is an entire industry designed around getting users to use your product based on ideas such as this, and this link is one place to learn more about that)
I'd like to see tests between that dog, and a person, or a group of people. Having images generally will help with conversion rates, but what kind of images would produce a better result? And for what kind of services?
i wonder if it simply has to do with the fact that there was AN image there ? Maybe verify by putting something not as cute - say an ORC or something like that? and something neutral like a fruit?
But was it really the puppy? There was a lot more text explaining what was involved (okay, not all that much, because there wasn't a lot on either, but significantly more) on the puppy version.
It could just have been that you described what signing up entailed. Plus you included 'cancel at any time', which might help some people decide.
First of all, great work. It looks like you boosted your conversion rate from 0.19% to 0.43%. Which is a 125% improvement, or with confidence intervals, 55% - 179% improvement.
However, before everybody goes out and puts puppies on their homepages, they need to realize that there are a bunch of things being tested.
Image vs. no image: Is it possible that having any image at all improves the conversion. You should test with other pictures: perhaps some animals, people, nature, and see if the puppy is what makes it work.
Call to action: The 'puppy' version also features a more succinct call to action in "Sign up now" rather than "Start your 30 days free trial." Perhaps this also contributes some of the difference.
Button size: The button size in the 'puppy' version is smaller. Perhaps this has some effect as well.
Length of text: The 'puppy' version has more description of what is involved in the free trial. It says "Pick a plan & sign up in 60 seconds. Upgrade, downgrade cancel at any time." vs. the no puppy version that says "Start you 30 days free trial."
Vertical vs. Horizontal Layout: The 'puppy' version has a vertical layout of the text and button, where they are stacked on top of each other rather than left or right.
So there are at least five different changes made between these two designs. Clearly the second design wins on conversions, but it's not entirely clear to me why it wins.
Not sure how you can get much of value from an A/B test with multiple changes, especially if one is claiming that only 1 of those changes is what is responsible for all the improvement.
You can iterate on the other tests over time. Many A/B tests start with a larger change that may include multiple variables but with that baseline increase, now they can go ahead and test Dog vs Cat vs Human as the image. Or can test a variety of different text sizes and lengths. This seems like a fantastic start, with plenty of room for further iteration and improvement.
If nothing else, they have a hypothesis to test in the next experiment.
Even when possible to isolate and remove ancillary changes to improve split test purity, it's often not beneficial. If there's a significant number of changes, achieving statistical significance across the full matrix probably isn't even possible.
But that's ok, because limiting changes to a single test queue restricts your ability to move fast and try lots of stuff, which is beneficial. So when you test, try cheap multivariate methods (there's a bunch!) to quickly understand how interactions between multiple changes affect results.
The article title gives the puppy image credit but I think the additional text had as much if not more of an impact on the conversion lift than the image.
"Start your 30 days free trial" text
vs
"*Sign up now* 30 days free rial. Pick a plan & sign up in 60 seconds. Upgrade, downgrade, cancel at any time." with a puppy image.
The second version is so much better because it tells the viewer what to expect... the 60 second signup, flexibility in changing plans, no long-term commitments. The dog image does help since he is looking at the call-to-action, which also draws the viewer's eyes there too.
It is a great post and just goes to show how testing can really help boost conversion. It's often a lot cheaper to grow your business by improving conversion rates than by increasing traffic.
"But we can still be fairly confident that the image was the main factor since this test had a much higher change in click through than what we typically see when we just test text vs text."
How can you be fairly confident? They broke the cardinal rule of the scientific method.
Well, it's worth noting that while we're interested in the general science of puppies and conversions, our end goal with this test was not to determine the exact value of the puppy picture, but just generally finding the best way to get people to click our call to action button...
Except you have 2 variables - puppy/no puppy; and copy A/copy B, and therefore 4 combinations. You only tested two combinations, so you have no way of knowing if the puppy or the alternate copy was the reason for the increase.
More people clicked through to the sign-up page, but did the actual number of sign-ups increase too? After all, in the end, that's what matters. If people click your link just because they want to see more pictures of puppies, it might not be such a great improvement.
Emotions do sell things. I like to think of myself as a rational person, but it certainly works on me.
For example, when I was living in Japan around 1996, we needed a kerosene heater for our apartment. We went to the big department store, and they had a lot of different ones. Each of them had a sticker affixed describing the various features and advantages, but they were mind-numbingly similar. But one of the stickers had a photo of Mai-chan, the cheerful and engaging child star of a cooking show that we had enjoyed on Japanese TV a couple years earlier.
My wife pointed it out to me, and we ended up getting that heater. Not because it was any better, but because we liked Mai-chan, and her photo was on it. Of course Mai-chan has absolutely nothing to do with heaters, and we knew it. But we didn't care.
We always called it the "Mai-chan" heater, and even though we left it behind when we returned to the states, I guarantee if I mention the Mai-chan heater to my wife today, nearly 20 years later, she'll smile and know exactly what I'm talking about.
The puppy screenshot in the original post has a similar effect on me. A part of me wants to reach out and pat the cute dog on the head. But I can't. But I can click the button, which is what he obviously wants me to do. Sure it's not at all rational. But it motivates people. I suggest keeping the dog and adding it all your advertising. Make it your mascot. You can't go very wrong with a puppy!
I suggest keeping the dog and adding it all your advertising. Make it your mascot.
That's exactly what I imagined when I saw the title; a startup testing/comparing the conversion rates of a dog mascot/logo. It goes hand-in-hand with establishing a memorable brand. I'd be really curious to see how the conversion rates are affected by a complete website/brand overhaul containing a dog mascot.
Not a dog, but my startup (tabuleapp.com) added a koala as our mascot, and our conversion rates more than doubled. We help college students organize homework, so the fun koala face helped differentiate us from most homework planners that just use a book as their logo.
If you ever plan opening in Poland, add a panda instead.
There's a known few years old meme now, featuring a panda and a number "3". It's a rebus that should be read as "panda 3", and understood as "pan da 3", which is a kind of impolite way of saying "sir, please give me a 3 (D grade)".
We all know that puppies can sell CMSs. That's a given. What they should A/B test is the effectiveness between a black lab and golden retriever.
Joking aside, the text version has no picture at all (could be just that screenshot), one thing I haven't forgotten from my marketing 101 is that pictures grab attention 10X better than text.
This reminds me of Windows Bob but I don't think it worked very well for Microsoft putting that dog there. Initially people (women and children) were impressed with how cute it was but five minutes later they were trying to kill that annoying yellow dog.
It says "the number of clicks on the signup button" was measured. That does not seem like the metric they are interested in which would be "number of actual signups".
November 2012. The headline was something along the lines of "If X does not fix Y, our Alpaca leave you a voicemail." It improved conversion by about 20% (image vs no image). Same headline.
Why a few months? You can rack up visits by just doing blog posts. I can take most blogs and turn it into a 20K visitors/week blog in no time. Not a great deal of traffic, but enough to start making good tests.
People used to game Google Adsense by putting their own images next to the adsense blocks in order to get more clicks. MFA sites did this so much, Google banned the practice of using images next to their ads. This is the same result.
66 comments
[ 2.9 ms ] story [ 117 ms ] threadIf they use an image of a person, they would need to be glancing or gesturing in that direction as well.
Actually, I'd like to see them A/B test the puppy against itself, but flip it horizontally so it's looking to the left in one scenario.
http://pugbomb.me/
Coincidently I now work at a company with currently 4 dogs in the office.
Jesus, and I complain about how hard it is to concentrate in a noisy open plan office. A bunch of non-human animals roaming about might just tip me over the edge.
(to elaborate for the downvote-debbies: there is an entire industry designed around getting users to use your product based on ideas such as this, and this link is one place to learn more about that)
What's next someone telling us they increased conversions because they used photo's of smiling people?
/debbiedowner
- Cute
- Mascot
- Original
- In-joke-ish (octopus merges, the internet is for cats)
It could just have been that you described what signing up entailed. Plus you included 'cancel at any time', which might help some people decide.
Just the fact you used a graphic can make it superior to the text version, not necessarily because it's a dog.
However, before everybody goes out and puts puppies on their homepages, they need to realize that there are a bunch of things being tested.
Image vs. no image: Is it possible that having any image at all improves the conversion. You should test with other pictures: perhaps some animals, people, nature, and see if the puppy is what makes it work.
Call to action: The 'puppy' version also features a more succinct call to action in "Sign up now" rather than "Start your 30 days free trial." Perhaps this also contributes some of the difference.
Button size: The button size in the 'puppy' version is smaller. Perhaps this has some effect as well.
Length of text: The 'puppy' version has more description of what is involved in the free trial. It says "Pick a plan & sign up in 60 seconds. Upgrade, downgrade cancel at any time." vs. the no puppy version that says "Start you 30 days free trial."
Vertical vs. Horizontal Layout: The 'puppy' version has a vertical layout of the text and button, where they are stacked on top of each other rather than left or right.
So there are at least five different changes made between these two designs. Clearly the second design wins on conversions, but it's not entirely clear to me why it wins.
Even when possible to isolate and remove ancillary changes to improve split test purity, it's often not beneficial. If there's a significant number of changes, achieving statistical significance across the full matrix probably isn't even possible.
But that's ok, because limiting changes to a single test queue restricts your ability to move fast and try lots of stuff, which is beneficial. So when you test, try cheap multivariate methods (there's a bunch!) to quickly understand how interactions between multiple changes affect results.
It is a great post and just goes to show how testing can really help boost conversion. It's often a lot cheaper to grow your business by improving conversion rates than by increasing traffic.
"But we can still be fairly confident that the image was the main factor since this test had a much higher change in click through than what we typically see when we just test text vs text."
How can you be fairly confident? They broke the cardinal rule of the scientific method.
Come on. It's a puppy. A puppy!
The best way may be with the puppy or without. From this you have no idea.
For example, when I was living in Japan around 1996, we needed a kerosene heater for our apartment. We went to the big department store, and they had a lot of different ones. Each of them had a sticker affixed describing the various features and advantages, but they were mind-numbingly similar. But one of the stickers had a photo of Mai-chan, the cheerful and engaging child star of a cooking show that we had enjoyed on Japanese TV a couple years earlier.
My wife pointed it out to me, and we ended up getting that heater. Not because it was any better, but because we liked Mai-chan, and her photo was on it. Of course Mai-chan has absolutely nothing to do with heaters, and we knew it. But we didn't care.
We always called it the "Mai-chan" heater, and even though we left it behind when we returned to the states, I guarantee if I mention the Mai-chan heater to my wife today, nearly 20 years later, she'll smile and know exactly what I'm talking about.
The puppy screenshot in the original post has a similar effect on me. A part of me wants to reach out and pat the cute dog on the head. But I can't. But I can click the button, which is what he obviously wants me to do. Sure it's not at all rational. But it motivates people. I suggest keeping the dog and adding it all your advertising. Make it your mascot. You can't go very wrong with a puppy!
That's exactly what I imagined when I saw the title; a startup testing/comparing the conversion rates of a dog mascot/logo. It goes hand-in-hand with establishing a memorable brand. I'd be really curious to see how the conversion rates are affected by a complete website/brand overhaul containing a dog mascot.
There's a known few years old meme now, featuring a panda and a number "3". It's a rebus that should be read as "panda 3", and understood as "pan da 3", which is a kind of impolite way of saying "sir, please give me a 3 (D grade)".
Good call. Aren't black dogs the least likely to be adopted from shelters?
You added an image, period. You also changed the CTA button style.
The split-test did not test an independent variable.
Having said that, it's good that your conversion went up.
[0] http://www.extensis.com/
The most popular color? Blue! IBM is blue, now Windows Blue, Mac OS X aqua blue... you name it.
- babies (various poses, ages)
- tools (wrenches, screwdrivers, axes, etc)
- clothing
- auto parts
- aliens (as in fake grey aliens)
- people in weird situations
- women crying (this one works well near saint valentines)
- an animal with a cast (better yet if its a small bird)
- toilet paper (a new roll. not used)
- random charts (pie, bar, etc.)(The charts doesnt even need to make sense)
- a lock (big master lock)
- and one time I ran an alpaca. Weirdly, it converted better than anything else. Don't ask.
I wonder how well real aliens would do.
People used to game Google Adsense by putting their own images next to the adsense blocks in order to get more clicks. MFA sites did this so much, Google banned the practice of using images next to their ads. This is the same result.