Ask HN: When do you know that you should stop testing a hypothesis?

3 points by damian_r 10y ago ↗ HN

You've build a lean canvas for your project. You have many hypotheses regarding each part of lean canvas.

Let's say you have:

Sample idea: calculator online

Lean canvas: ... 2. Customer segment - hypothesis 1: analytics in finance companies would use that - hypothesis 2: children in school would use that ...

Testing hypothesis 1: - You have written no code for the calculator. - You build landing page with a subscribe form. - You ask 20 influencers in the financial world. - You post your idea to Hacker News, Reddit and some other forums.

As a result: - You have 160 subscribers ready to use and pay a 1$ a month for it - Your post on ASK HN gets 10 upvotes and 20 comments (15 positive, 5 negative) - Your post on reddit gets 5 upvotes and 3 positive comments - 4 influencers say "yeah I like it, would love to try it", 3 say it's useles, and 13 stay silent

Is this result good enough to keep testing the hypothesis? How much more time would you spend to keep testing this one hypothesis? Do you think that this result is great or very bad? What else would you do to increase the numbers?

In short - what would you do?

2 comments

[ 3.0 ms ] story [ 16.2 ms ] thread

PaulHoule 10y ago ↗

I think it would cost about $1 a month to collect $1 a month. Also I think people in finance companies can afford more than $1 a month.

codegeek 10y ago ↗

"In short - what would you do?"

I would try to talk to as many of those initial captured subscribers directly. Either this will be over phone, in-person etc. I will try and assess "what" and "why" they need based on the discussions and not necessary "how" they need it at this point. Rinse and repeat. Do this manually. Don't worry about using automation and cool stuff. Keep it simple.

There is a big difference between saying "hmm looks cool. i may use it" vs "shut up and take my money". You want to work towards the latter of course. The former is just a start but not the end.

One advice on pricing. If not sure, always price high than low. You can always reduce pricing based on your market demand but almost never a good idea to raise pricing. Also clients who will value your product will not worry about price too much and if there are clients who cannot pay you for what is worth, those are not your ideal clients.