77 comments

[ 4.0 ms ] story [ 108 ms ] thread
Using adwords to provide enough traffic to test your product viability is genius...

I think the HN effect has taken their site down, though!

It has! It's at Rackspace and I'm working on getting it on the largest server they have.

Didn't expect to hit the front page.

Would you mind telling how much traffic (e.g. visits per hour from HN) you get when your article hit the front page?
I got 10k overnight when I put www.jobstractor.com on as a show hn a couple of weeks back. That was at the top spot for about 3 hours. NB, overnight to me in UK is obviously not overnight for majority of HN readers.

I'm hosting it on the smallest linode instance going with nginx caching responses and I'm serving almost no images so that worked perfectly for me.

Thanks for your information.
Actually, other people have done this in the past - Timothy Ferriss (the biz guy and author of "The Four Hour Work Week") was famous for using this adwords multivariate testing technique when he was deciding the future title for his book! See a great interview here: http://www.youtube.com/watch?v=3JHzLlvlVmQ
It's back up -- upgraded the server.
I don't know what you guys have been drinking last night but HN is full of absolute gold this morning. Great article.
@dberube

I have used same method to collect beta tester for one of my idea. Google adword has given amazing reults so far. Thats why i am interested to know if cost factor the only reason to choose microsoft adcenter over google adwords ?

Does it give more ad impressions for less price + better click through rates?

The quality of the people coming to the site, in my experience has been the same.

I actually have better CTR on adCenter compared to AdWords for the same ads on the same keywords (marginally better).

So it's really come down to cost for me. In my experiences it's consistently double or more for the top keywords in the bunch.

I've personally never run a massive campaign to drive tens of thousands of visitors to a site through either service. My guess is, if you need high volume of people quickly, you need AdWords.

Doctors don't buy software on the internet. You can't validate the idea that way. The sales cycle is long and their buying process is totally different from the "convert on the net" SaaS model. I did a startup called Maviq in 2008 for the patient reminder market. There is a v. high barrier to entry in this market for startups.
That was just an example, not the basis for the process.
While you may not be able to validate the idea of the service in isolation, you can however validate whether or not there's a market for a service like this sold over the internet.

In many cases, someone may consider it worth excluding an otherwise potentially profitable idea if the sales cycle is long and the buying process is different from the "convert on the net" SaaS model.

That's changing, frankly... and rather quickly. I speak as someone who builds (and sells) SaaS products to physicians.
There are many private practice small business out there.
Can you define what you consider a "high conversion percentage" in this stage of testing?
I actually like to add an intermediary step between the landing page and the e-mail form: a "Create an account/Buy now for $X/month/year" button. When people click on it, they get a "Sorry, we're not done yet, but give us your email and you will be notified once we release" + the email form.

Not only do you know people are potentially willing to pay X dollars for your product, you can also do a multivariate test on different amounts or pricing schemes to see if they convert differently, giving you some rough idea how much your idea is worth to them. I always add a "Make an account for FREE" as well, as a baseline.

Well, the problem with doing that at this stage of the process is you risk getting the initial price completely wrong and nobody clicks it.

Even though, if you had priced it just $20 lower, everyone would have clicked it.

So by putting price in the mix before you even identify a willing market, you risk missing the ball completely.

You could A/B on price to get the demand curve.
But you still leave open the possibility of not learning the basics of whether or not someone wants the problem solved.
You could put up a separate pricing page to see how many people click through to it, and then look at hovers to see where they linger... But that's obviously going a bit deeper than the original idea of slapping up a single page to gauge interest.
It's probably just me, but I can't help feeling it's a bit underhand to offer something for sale that doesn't exist yet (unless you make it clear upfront).
Really interesting and well written. I particularly liked the part when the author said not to show feature as if they don't buy the silver bullet, they won't buy your product.
Fantastic article dberube, couldn't have come at a better time for me. Few queries: a) What would you call a good collection, conversion rate? (Also asked on yr blog) b) Will this work for ideas that might be seasonal? (eg: I am thinking of selling a particular fruit from India online - it's seasonal and the store will operate only abt 4-5 months. If it's successful, I'll see what else I can do with it, but as of now there are no plans). So, have you worked this strategy for any such ideas which are not 24x7? How early before the season do you think I should run this. My season is still 5 months down the line. If I run too early (now-ish) I run the risk of not getting anyone interested as no one is searching for it. c) Can you give a few figures from your 3 experiments about how many visitors you got from adCenter? And their conversion rates? Thanks.
You could always start by collecting e-mail addresses of those people interested to buy it.

So they sign up and get notification once you have the items ready to buy.

Consider your shipping methods carefully else this type of product might be tricky.

Well, that's my fear. That if I run this too early I may not be able to collect any prospect email addresses (as almost no one/very few might be interested in it or searching for it at this time).

I do have some queries/concerns on the shipping front too. Have you worked on this type of farm/agriculture product or know any pointers where I can get more info?

ps: Updated my email in my profile (if you want to take this offline). Don't see your email ID in yr profile.

Any good alternatives to unbounce?
It's the best one I've seen. There is also http://www.instapage.com/ but it's not as good and I've run into problems with pages not loading -- and when you're paying for traffic to hit a page, you want it to load.
I've read about this idea in the 4 hour work week. This is the way Tim Ferris tested his ideas.
Agreed, the 4HWW covers this fairly well, along with some tips I hadn't thought about previously
I'm kind of biased against using Google AdWords since they banned my account for no explainable reason (they lack complete customer care to the point where they can't even display the ban message right: http://www.devside.net/images/adwords-account-suspended.png), but consider spending more money and getting better validation by going directly to websites specific to your market and buying ad space there.

The AdWords traffic is just too broad and "general public" and it's incredibly difficult because to get good CTR (click-through-rate), you actually have to find keywords that produce bad or irrelevant results for the searcher while being on topic, as otherwise the searcher will simply click the organic results and never even see your sponsored result (when was the last time you went to Google, did a search, got good organic results, and decided to skip that and just go to the sponsored link?).

AdCenter is a little better IMO, but niche websites are absolute killer since you're no longer screwing around, trying to filter out irrelevant traffic, trying to get the right keywords and bid prices, and making sure your landing pages have the right quality score. The moment you get a low CTR is the moment Google AdWords start increasing prices, stops showing your ads.

Go right to the source to test your ideas. It will take more money, but you'll get a confidence level of 95-99% for your validation, rather than something much much less that you'll get using AdWords. It will be quicker too; AdWords keywords can be incredibly difficult and time-consuming to get right.

You have no idea what you're talking about with AW.
ADW is overrated and very expensive. Specially if you need an expensive keyword. I had a good experience with Facebook ads. It was cheap and high CTR.
facebook ads don't convert into buyers - adwords ads do - that's why it cost more. if you do the proper research, set up your campaigns and ad groups correctly, add negative keywords, adwords can work really well. you need to play their quality score game, get a good CTR on your ads and always be testing new ads.
That happened to me too. I was testing an idea with AW and was banned for life, without having any chance to talk to a real person.
I may be biased, as I'm part of the Google Engage for Agencies program, but for all of its defects, AdWords is an incredible marketing resource with an 80% reach.

Their two main problems are:

1) Customer care for end users is admittedly lousy.

2) You really, really, really need to know what you are doing or else it's very easy to be taken to the cleaners.

Ignoring AdWords as part of your online advertising efforts is usually a bad idea. However relying solely on it is a dangerous proposition, too (case in point, you were banned).

Diversify by introducing media buys (as you suggested), other ad channels (like AdCenter, LinkedIN ads, Facebook ads, StumbleUpon ads, etc), and then prioritize based on ROI.

The question is, do you need all this to test out an idea? Probably not. I think that at this stage virtually anything targeted will help you gauge interest.

> AdCenter is a little better IMO, but niche websites are absolute killer since you're no longer screwing around and trying to filter out irrelevant traffic.

Don't forget that with AdWords you can reach niche sites, and even target them individually if you want to, so you don't need to commit to paying $xxx/mo for a media buy directly with the publisher (just to run tests).

> Don't forget that with AdWords you can reach niche sites...

That is true, BUT, since AdSense brings in pennies for the webmaster in general, relative to the other options, a lot of good/non-shit niche sites are not using AdSense, but are either using custom sell space or other targeted networks that payout 2-10X of what Google AdSense does.

IMO, if you are serious about running and monetizing a business, you will get as far away from being 100% dependant on the Google index and their ad network as you can.

So many people have been screwed this way. Imagine what would happen if you dropped a couple of pages down in the index tomorrow, or your AdWords account was banned.

Though I'll still admit there are a couple of sites I can no longer target, because they use AdSense ... but I have considered emailing each one and offering 2 or 3-times whatever they get with AdSense to get rid of it and put up my ads/links instead.

> If everyone is busy, those calls are often not made.

I personally have not seen a single doctor's office that does NOT make reminder calls or forgets to make them. Moreover virtually all of them also have a $50 fee penalty for missed appointments (which is never charged, mind you).

In other words be careful if you are validatng the idea just for the sake of proving yourself right. Especially when your landing page is so ambiguous in its message, that it's basically detached from the original idea.

Once again, the example given in the post was not an actual test. I was using the idea as an example to guide the article.

The startup idea in the article isn't the focus.

I've heard this called "pretotyping". The basic idea is not to create a working prototype (which still can take weeks/months/years) but to judge interest.

http://www.pretotyping.org/

If you're looking for a fast landing page creator like the one mentioned in the article, I'm working on one: http://seekevidence.com
Correct me if I'm wrong...

Are you using unbounce to create unbounce alternative? That is pretty badass.

:)

LaunchRock is a great landing page tool for testing a startup idea. I think they are a Y Combinator company too.
But they are not open to the public yet.
(comment deleted)
You might be scrapping an idea off just because the landing page wasn't optimized - could this be a possible problem?
I don't know about having "silver bullet" advertisements be your landing page and text ad. It looks suspiciously like spam or some kind of con (think about the majority of "Work From Home" ads). Every reasonably smart person I know wouldn't trust "silver bullet" ads enough to seriously consider paying for the product being advertised. In the particular example, most doctors are supposedly smarter than the average population and so I think the conversion rate would be even lower.

I don't claim to know a better way to garner attention through advertisement than eye-catching (yet suspicious) claims, but I think the described method helps you determine the interest level in your product from a small subset (more trusting or naive, depending on your point of view) of your target audience, not your total target audience. Maybe there's a known "naive-to-total" extrapolator that I'm not aware of.

There's a balance to be struck because the polar opposite - discussing features rather than benefits - is the norm and it's worse than ineffective for the average user, it's confusing noise.

I just came back from running two days of usability tests on-site at a security software company. I wish everyone could see the test subjects' eyes glaze over the typical marketing text describing exactly how the thing works and conversely, how they'd light up about 20 minutes into it when I'd guide and moderate them towards the hidden nuggets of real tangible value, like "4x faster than Norton" or a chart showing that the memory usage was a tiny fraction of competitors.

I agree that there needs to be a balance and furthermore, that finding that right balance is very difficult. Probably why marketing whizzes get paid the big bucks.

In your example though, there is difference in that you're an actual person representing a reputable company on-site, not a random anyone-could-have-created-this advertisement on the anonymous web. From more trusted sources, I'd be interested in hearing the benefits and not necessarily the features. But in an environment where seemingly every third website activates my anti-virus software and I read about data hacks every week, I definitely don't trust Internet ads filled with hyperbole and exclamation points.

It's true that people in a usability test don't behave exactly the same as at home, but that actually proves my point. If they're obviously trying to be positive and helpful (no matter how much you tell them to relax and be honest) because they're constantly saying nice things about the site or app, then it's even more noteworthy to hear any negative comments or see that they're confused. Especially in the beginning of the session when their energy is higher.
These landing page systems have a lot of room for improvement.

Upon getting a name and email address, it would be quite valuable to present the new subscriber with some form of survey. Be it short and multiple choice, but instead of JUST seeing if there is interest, you may be able to find out what aspects of your product bare the most importance.

ie If you create a landing page for a new shopping cart product, you may want to ask: Will you be selling digital goods? What country are you in? etc.

Agreed, 100%. I've actually bundled a Wordpress install with the necessary plugins and libs (google analytics, basic survey abilities, a/b testing harness). This makes it super easy for me to spin up an ec2 or linode instance. Then I can run a test and make a decision without touching any of my core products/sites/servers.

Not too difficult to do.

That sounds really interesting. Could you share some more details, or point us to an example that is running? Which plugins work well for you?
I wish you had written this a bit earlier :) I invented a playing card game and was really excited about it because I got good feedback during play testing. I tested it in few bars and they all liked the prototype. So after making some edits to the game rules from the play testing feedback, I went ahead and printed 2000 decks and I have a website where I'm selling the decks ( http://acardgamewithattitude.com )

I'm not seeing quite the success in selling online like the feedback I got offline.

No sure how I would have tested this concept before printing because its a niche market i'm targeting.

Now i'm confused what is wrong - Is it the site design - copy on the site - Am I over explaining - Is it not clear if its ia deck of physical cards

Can i take the site down and do a teaser like it is going to launch? and then sell it?

I don't really know what I'm talking about but I'm not sure this type of card game translates well online. In my opinion, the site design is a problem. The problem I'm seeing is that there's nothing making me want to learn how to play a new card game. I have no idea what the answer to that problem is but there's got to be some motivator on the site, I think, to get people to want to learn it. I'm sure once the game is learned people will love it. Maybe you could look at techniques other card games/board games use to get people interested? If you made a digital version of the game, you may also be able to sell the physical card game online?

The site is too red.

thanks mrbgty for taking the time to look at the site and giving feedback. "Motivation" site is "Too red" I have added that to my list of to do. Thanks. Anyone else?
After the redesign, is there any learning on if the buy button should be upfront on home? or it doesn't matter?

Maybe this is a a really stupid question.

I agree. I wouldn't buy from this site if they paid me double the cost.

Edit: If you want people to take you seriously, you should take yourself seriously. I am sure the game is great, given the offline response, but online we judge you by your website.

Clicking through to the authors blog home page and reading between the lines a little it looks like he is the founder and was the CTO of Mofuse.com and was kicked out just a week ago. Mofuse took $1.2M from 33 angels this past January so I guess they had something to say.

What is the story dberube? Inquiring minds want to know.

I demanded $40MM a year in compensation and the use of a private jet. I was told no.
I think this has been common practice for over 2 years now. Where did you learn this technique from?
(comment deleted)
Rob Walling has a book on doing these things systematically that might be worth your time. You can read the first few chapters for free: http://www.startupbook.net/
Great post. I've used LaunchRock (widget, not the whole page setup), would definitely recommend it.
I've visited LaunchRock to check it out. I didn't get any account after set up. Somehow I have a nagging feeling of being trolled, landing on the landing page of a site promised to create a landing page.

The FAQ says I will get an account after getting 3 people to sign up. Bahh. I just want to check out the features and I have to spend effort in marketing your site? Thanks but no thanks.

Warning...shameless self promotion...

Would love to hear your thoughts on my own start-up, Kickofflabs: http://www.kickofflabs.com

No gimmicks, tricks, etc. Just a solid product with fanatical support.