I agree this is a “bad end”, but I don’t think we can fault ourselves for flukes. Sometimes you play your poker hand _perfectly_ given all the information you have, and a random stranger just has better cards against all odds. I think you just have to be ready to accept some flukes, and continue to make the best decision with the info you have
> make sure that there’s a well-defined distinction between success and failure. Don't fall in the messy middle.
I agree this is the right thing to do, but it isn’t always possible to do for a given experiment. For instance, with the cold-calling example, how do you ensure that it’s either a smashing success or definite failure a-priori? Is there a principled way to choose the threshold number of conversions? Do you just pick an arbitrary number? What if the result is just below your threshold? With some experiments I think the only way to win is not to play. Happy to be corrected if anyone has a solution
I've built a few sales team for early stage startups, and invariably the question comes up: "Inbound sales is working. Will outbound sales work?"
The successful experiments I've seen look like this: hire 2 SDRs at the time, focus them exclusively on outbound, then measure their results to identify the CAC. Then, you can make a call if that CAC is something you want to scale.
The failed experiments look like this: the founder doesn't want to invest too heavily in something that might not pay off, so we hire 1 SDR and they have to commit 50% of their time to following up on low quality Marketing leads.
It's not so much about the number itself, but more about the experiment design that will give you clear results
it's a good article, but misses the point.
you can't apply the scientific method early in a startup. You never have enough data and all the factors included change too quickly.
what you need is conviction. if it works for 1/100 calls, and you believe in the idea continue iterating. Also, observe what your peers are doing, you probably don't need/want to innovate on sales or business model.
> So when a startup comes to me with an idea for an experiment, the one thing I tell them is: make sure that there’s a well-defined distinction between success and failure. Don't fall in the messy middle.
Mediocrity is often enough to put food on the table. The world is full of companies aiming to survive for a few more quarters with their little mediocre product. And once a company embraces that idea there's usually no going back.
Settling down with a mediocre product that sustains your company should probably count as success. The vast majority of startups fail, after all. You might have failed to produce a unicorn, but at least you've got a decent workhorse.
I know a guy who settled with a mediocre product that put food on the table. He kept it running for the most part of the 2010s. Then, out of nothing someone came and acquired his company for ~ $14mm. He had no investors and just a handful of workers.
I worked in a SaaS company selling an A/B Testing app where I saw a lot of tests and their outcomes from different clients.
Of course there are the low hanging fruits, which scored the clear success. Most of the tests although were not a clear success, but just a slightly better conversion rate.
Multiple tests applied after each other slowly increased the overall value. If taken the advice not to implement the changes of the small success, the overall amount would not have changed.
Indeed. It's almost like the original article is clickbait :) The problem is that we have a term for what he's describing already - diminishing returns. For some of these "mediocre successes" you may find that your business is already down the optimization curve on some choices, but, having done a test and found a signal to improve, why not?
After all, pretty much all of writing, music, product design, prototyping, etc. is iterative. It's a purely investor mindset that would have you think that small optimizations do not pay back. Having that mindset in creating something is a great way to give up after one draft of a good idea.
"Just slightly better", if you can have confidence that it's not a mirage, is a success and should be taken as such. I don't think that's inconsistent with the article, which is more about the danger of allowing a gap between success and failure which, if you land in it, leaves you with no clear direction to go in.
So long as only 10% or less of your incremental successes are mirages, and so long as the downside of shipping a mirage is only a small incremental harm, then shipping 9 success and 1 harm should still get you an overall ~78% win vs just shipping the 9 successes (assuming the bad result is an equivalently negative result to the good ones).
How much are you willing to spend to reduce the downside risk, and how many “good” experiments are you willing to throw away in the process?
Everything is a series of battles and campaigns. If the overall war is a mediocre success, you still have plenty to learn from if you're digging into those.
Competitive sports isn't for everybody, regardless of your ability to work hard. If you aren't built for it, you're going to be a failure or a mediocre success. Entrepreneurship isn't for everyone either.
Silicon Valley normalized the idea that if you follow the scientific process (The Lean Method, finding Product/Market Fit or whatever this garbage is known in the latest terminology), and burn out for 10 years of your life, then you WILL achieve success. Most people are blind to survivorship bias even if they know the phenomenon. On top of it, incompetent, scumbag VCs muddy the waters with their snake oil "wisdom" which bright eyed, impressionable tech bros gulp down with passion.
If you achieve a "mediocre" success, which allows you financial independence, then you HAVE achieved success, regardless of what the VCs are pontificating. They'd scream because their investment didn't 1000x. Fuck them and continue living your life. Don't burn out for these soul-less MBAs.
And say you achieve the outsized "success". What then? If you go with the history of silicon valley then your so-called success is going to be a shit deal for workers, consumers and the environment.
That's a pretty low bar to call something click bait. It's not click bait just because you disagree with the content. Edit: and I think the title matched the content fairly ok?
The title is "The Worst Outcome is a Mediocre Success" and that's what this is about. How is this deceptive? The "substance" part might be debatable. I personally don't think every post has to go in-depth on everything. I enjoyed his nugget of insight.
It's deceptive because the title is being used in a way that you would not guess without reading the article and suggests highly that it is referring to something completely different. "Mediocre success" is not a synonym for "ambiguous result" and in context most would assume it is referring to financial success.
As someone who went through the startup grind, I think the entire message is in the title and its spot on.
Did you read the article? Running experiments means you need to have a null hypothesis - and in a sales conversion experiment, for example, the null should be calibrated to ensure your conversions are scalable wins and not just one-offs.
“Mediocre success”, as described by the author, is how great science starts out. You try an experiment, (eventually) get a slight indication that your hypothesis might be valid, and then keep iterating until it seems clear that the idea is real (or not).
Startup companies generally shouldn’t do science, but that same interactive process should guide you. Because when you start you never know enough.
It is also about trying to get the most of that hypothesis testing, defining success and failure the best you can.
I have encountered this "mediocre success" many times in AI solutions due to lack of problem definition. For instance, now with LLMs is very easy to write a prompt that gives you the output you want in 5 or 6 examples you have in mind. The problem is to build up your testing scenario from there, and gather as much data as possible until you make it representative of your use cases.
That is the only way to actually test your prompts, RAG strategies, and so on, instead of buying the last CoT-like prompt trend.
Mediocre Success + VC Investment = failure worse than just failing to start a business. It means wasting many years of your life for a "small win" for a VC.
> Making it worse is that we’re all heavily socialized to aim for mediocre success. Schools, universities, large organizations — they don’t want big swings and big misses; they want safety and consistency. A steady 7 is better than 10s interspersed with 0s. This might work well in structured, predictable environments, but in startup-land it’s anathema.
Correction: It's anathema for VC-funded startups whose backers need one or two to succeed to unicorn size to recoup their losses from the other 98 attempts. For self-funded or privately/bank financed startups, it's fine to stay low profile and grow slowly, as long as the investors get their money back and everyone gets paid fairly. You may not end up a billionaire, but hell, even a small trades shop can reasonably be worth a million dollars or two after a few decades.
It's after all how the many, many "hidden champions" of Germany's Mittelstand got to where they are: start small, focus on extremely high quality products/services, and dominate entire markets even though no one but experts of their respective fields have even heard of them. And that model working out isn't restricted to Germany either... Japanese YKK zippers are the best example there [1].
A side note: if you don't have VCs or, worse, the vipers of the public stock market breathing down your neck, you have so much more freedom as a founder. You can do what's best for the company long term.
Having a corporate job is usually a 'mediocre success'. If you strike out on your own with something that involves your passion and you earn a comparable income, you are a stellar success. Don't let a nepo baby tell you otherwise.
I agree, but this blogpost is NOT about professional trajectory. It's about sales and GTM success criteria for startups
> That’s the danger of the mediocre success. The point of startup experimentation isn't the success itself; it’s the learning that comes with clear-cut success or failure. You don't really care about the sales revenue generated by your first two reps; you care about whether this is a strategy you can scale to dozens and then hundreds of reps, or whether you need to use a completely different strategy. It’s all about the learning. And mediocre successes don’t give you any learning.
> Having a corporate job is usually a 'mediocre success'.
Eh, that is doing OK to me, which is better than some nebulous "mediocre success". If you are at least saving for retirement, and able to support a family, that's actual success in my book.
The typical startup outcome, where you try for a couple years, it fails, and then you move on to a corporate job is not terrible.
What's really bad is when you make enough to just barely get by, paying yourself a sub-market-rate salary, and keep limping along for a long while. Waiting for that "success", that big breakthrough, "just around the corner". You can be stuck doing this for years, out of a misguided sense of loyalty, determination or whatever. And then you are not saving for retirement, and letting the best family-creation years (if that is a life goal) pass you by.
That, to me is "mediocre success".
Ask me how I know. On second though, no, don't ask me how I know.
Yes if you earn an income comparable to a corporate job, yeah that’s a big win. “Mediocre success” unfortunately isn’t well defined here, but I assumed the author wasn’t suggesting a corporate salary. It seemed like he was talking about a few early sales but not enough to sustain the business. Imagine starting a company with two other people and grossing $12k the first year and $22k the second year, and then running low on funding. Should you keep going? Should you seek investment, assuming you can even get it? What if the growth remains linear, and it will take at least 10 years to reach sustainability?
Wouldn’t “not enough to sustain the business” be failing? I’d think “mediocre success” would be doing well enough that you can’t just throw in the towel, but not well enough that you can escape the grind.
The author defined mediocre success to be the relative outcome of an experiment, such as an A-B test, he wasn’t talking about income levels or overall success. It might be assumed that, since we are talking about startups and experiments, that we are not discussing self-sustaining businesses? Either way, which interpretation makes the rest of the article make the most sense?
What a useless article. Zero insight. Anyone who has run a study knows this. It’s one of the reasons statistics is useful. And also decision making. Before hiring a team of cold callers the person should have had a clear target. Without that kind of leadership of course its confusing.
> The worst outcome, the very worst outcome, is to get a small but non-zero number of sales — say 1 or 2. Because now you're in a bind. Do you double down or pull the plug? Does cold-calling work or not?
That seems like a failure to define success in the first place. That seems like doing an experiment and then deciding how it made you feel. You probably want to declare beforehand what your minimum number of sales would be in order to qualify as a success. I agree that the results can still be ambiguous, but the situation he's describing sounds like a poorly designed experiment anyway.
> You probably want to declare beforehand what your minimum number of sales would be in order to qualify as a success
It is hard to define that and sometimes can be noise or mask important information.
For example, some products have long conversion times due to pricing. This is a critical metric to follow, but would be ignored by the metric you defined.
> That seems like a failure to define success in the first place.
You are in precise agreement with the article's own text: "So when a startup comes to me with an idea for an experiment, the one thing I tell them is: make sure that there’s a well-defined distinction between success and failure."
Yes, there are nuances to success and failure with business. But, the worst outcome for a startup is being sued for millions by a patent troll and losing the business over it. It's the worst outcome due to the inherent pure evil within the system itself.
I think people are getting hung up on "mediocre" and "success" here.
Author argues for trying to be objective about the viability of business ventures here. It's easy to tell yourself lies that doing just one thing a little better on the execution front will magically triple your conversion rate.
As an entrepreneur, my observation is that the vast majority of the experiments I run result in an outcome where the null hypothesis cannot be rejected. That is the case for most social science experiments (which is what marketing experiments really are.) But since I don't have to publish anything, I don't p-hack my results.
If you think that scientific rigour will help you avoid the need for good judgement, you're in for a great deal of distress.
Edited for typos.
I agree and I don't think it is specific to the world of entrepreneurs and/or social science experiments. If your target variable depends on a large number of correlated variables you are very unlikely to formulate the correct hypothesis by accident. This is why you need intuition or good judgement in science, just as much as in every thing else.
> If you think that scientific rigour will help you avoid the need for good judgement, you're in for a great deal of distress.
Broadly, assuming more data and tighter calculations == more certainty is a folly endemic to engineering-focused organizations and industries. In reality, you're increasing precision without increasing accuracy: a dangerously misleading state. It's understandable: organizations obviously need to play to their strengths, but there's real danger in succumbing to the "when you're a hammer, everything looks like a nail" mindset. Firstly, It's much easier for non-engineers to parse the difficulty of engineering problems than the other way around. Outsiders easily see that they don't have the requisite chops to do engineering work. Secondly, cutting through ambiguity and unpredictability to reveal factors you can control is critical to engineering, while non-engineering jobs like management, design, and community outreach are hard because you must confront those things. Engineering types often reason about non-engineering problems either like they have the same level of predictability, or simply disregard the ambiguous and unpredictable factors because they don't fit into an equation.
It's easy to see why that micromanaging non-technical manager screwed up by insisted on using some technology or approach they read a snappy article about; an engineer's authority is cut-and-dried in that situation. It's harder to see why purely formulaic approaches often fail dealing with people, nature, markets, etc. Most things in the world are far more complex, temperamental, and less predictable than cache invalidation.
> After a candidate's defeat in an election, you will be supplied with the "cause" of the voters' disgruntlement. Any conceivable cause can do. The media, however, go to great lengths to make the process "thorough" with their armies of fact-checkers. It is as if they wanted to be wrong with infinite precision (instead of accepting being approximately right, like a fable writer).
It's probably worse in business types because they haven't gotten the exposure to science enough to realize that a lot of things remain tangible despite being un-quantifiable.
"Firstly, It's much easier for non-engineers to parse the difficulty of engineering problems than the other way around. Outsiders easily see that they don't have the requisite chops to do engineering work"
And if you don't reject the null hypothesis in the vast majority of experiments, then a large portion of those where you do were probably also just a statistical anomaly.
> that scientific rigour will help you avoid the need for good judgement
This is what all big company PMs and directors of engineering believe. But their judgement (good or bad), their metrics (scientific or pseudoscientific), their career trajectory (up or out), whether these are known or unknown in the first place, and their actual material success: who knows how correlated it all is. They don’t have the words to express this stuff, so they wouldn’t be able to see the difference between science and pseudoscience for example, and you'd have a hard time communicating this thing about science to these people.
So while you are right, it’s not saying much that the real problem is communications, and that is the mainstream opinion of people who study microeconomics / the structure of firms. Put differently, people have built cathedrals of bullshit in their minds in service of the status quo where they “test” “everything” in lieu of having falsifiable, forward-looking opinions (aka judgement). You can’t just go, Martin Luther all of corporate white collar bureaucracy.
The same is true in science too; which is why being an experimentalist requires lots of skill & judgement to tease apart the signal we want to understand.
Scientific rigor is inseparable from good judgement.
By the title, I thought this would be another clickbaity post about how we should all quit our jobs and become entrepreneurs, but in actuality, the post is better described as 'if you're going to do that, the worst outcome is to be moderately successful'. Which I agree with, and it's something that really gives me pause when thinking about launching on my own.
I mean, my corporate job is realistically pretty cushy and as the sole income earner, my family depends on me. I'm sure I could do well on my own, but it's hard to predict the success of any venture. Failure I actually don't mind, because the answer is easy... just get another job. The worst possible outcome is 'success' where you have a customer base and demand, but it's just never going to be enough to make up for that lost income. That would truly be the worst. You can't just quit without damaging your reputation; on the other hand, selling would likely be difficult without significant recurring income.
The difference between scalable wins and one-offs is knowing why it worked and being able to repeat it. Can you be a human being and connect with other human beings and their needs, and help them move forward?
The author is right; you want clear signals, not noisey ones. If you're looking for the light but keep seeing shadows you need to turn around.
The same is true for people who launch on product hunt, and get 50 signups. Success? If you're selling $1K deals, yes. But if it's just for $4, or free... do you have PMF?
but to be fair, if you can get a small group of people engaged, and you can tease out why/how, it usually stands that others will respond too. But not everyone will need it, and those that may need it may not be in market for it.
A better phrasing could be 'unclear outcomes are the worst outcomes', which when stated like that seems self defining. If its not clear its a success or failure, or its marginally a success or failure, then it can be hard (or dangerous) to draw firm conclusions.
87 comments
[ 3.8 ms ] story [ 127 ms ] thread…in a zero interest rate environment where there is a bunch of money sloshing around looking to roll the dice multiple times.
I agree this is the right thing to do, but it isn’t always possible to do for a given experiment. For instance, with the cold-calling example, how do you ensure that it’s either a smashing success or definite failure a-priori? Is there a principled way to choose the threshold number of conversions? Do you just pick an arbitrary number? What if the result is just below your threshold? With some experiments I think the only way to win is not to play. Happy to be corrected if anyone has a solution
The successful experiments I've seen look like this: hire 2 SDRs at the time, focus them exclusively on outbound, then measure their results to identify the CAC. Then, you can make a call if that CAC is something you want to scale.
The failed experiments look like this: the founder doesn't want to invest too heavily in something that might not pay off, so we hire 1 SDR and they have to commit 50% of their time to following up on low quality Marketing leads.
It's not so much about the number itself, but more about the experiment design that will give you clear results
what you need is conviction. if it works for 1/100 calls, and you believe in the idea continue iterating. Also, observe what your peers are doing, you probably don't need/want to innovate on sales or business model.
Mediocrity is often enough to put food on the table. The world is full of companies aiming to survive for a few more quarters with their little mediocre product. And once a company embraces that idea there's usually no going back.
Of course there are the low hanging fruits, which scored the clear success. Most of the tests although were not a clear success, but just a slightly better conversion rate.
Multiple tests applied after each other slowly increased the overall value. If taken the advice not to implement the changes of the small success, the overall amount would not have changed.
After all, pretty much all of writing, music, product design, prototyping, etc. is iterative. It's a purely investor mindset that would have you think that small optimizations do not pay back. Having that mindset in creating something is a great way to give up after one draft of a good idea.
How much are you willing to spend to reduce the downside risk, and how many “good” experiments are you willing to throw away in the process?
Silicon Valley normalized the idea that if you follow the scientific process (The Lean Method, finding Product/Market Fit or whatever this garbage is known in the latest terminology), and burn out for 10 years of your life, then you WILL achieve success. Most people are blind to survivorship bias even if they know the phenomenon. On top of it, incompetent, scumbag VCs muddy the waters with their snake oil "wisdom" which bright eyed, impressionable tech bros gulp down with passion.
If you achieve a "mediocre" success, which allows you financial independence, then you HAVE achieved success, regardless of what the VCs are pontificating. They'd scream because their investment didn't 1000x. Fuck them and continue living your life. Don't burn out for these soul-less MBAs.
And say you achieve the outsized "success". What then? If you go with the history of silicon valley then your so-called success is going to be a shit deal for workers, consumers and the environment.
What the author means is ‘get real about continuous learning’
Of course it takes money to run experiments you can actually learn from, and the article is bereft of practical advice about doing this on the cheap.
However, I clicked. But you don’t have to.
The updated title is much better.
Are you referring to the slightly edited title someboty has put here at HN? But still, no dissonance with the content
Did you read the article? Running experiments means you need to have a null hypothesis - and in a sales conversion experiment, for example, the null should be calibrated to ensure your conversions are scalable wins and not just one-offs.
Startup companies generally shouldn’t do science, but that same interactive process should guide you. Because when you start you never know enough.
I have encountered this "mediocre success" many times in AI solutions due to lack of problem definition. For instance, now with LLMs is very easy to write a prompt that gives you the output you want in 5 or 6 examples you have in mind. The problem is to build up your testing scenario from there, and gather as much data as possible until you make it representative of your use cases.
That is the only way to actually test your prompts, RAG strategies, and so on, instead of buying the last CoT-like prompt trend.
Correction: It's anathema for VC-funded startups whose backers need one or two to succeed to unicorn size to recoup their losses from the other 98 attempts. For self-funded or privately/bank financed startups, it's fine to stay low profile and grow slowly, as long as the investors get their money back and everyone gets paid fairly. You may not end up a billionaire, but hell, even a small trades shop can reasonably be worth a million dollars or two after a few decades.
It's after all how the many, many "hidden champions" of Germany's Mittelstand got to where they are: start small, focus on extremely high quality products/services, and dominate entire markets even though no one but experts of their respective fields have even heard of them. And that model working out isn't restricted to Germany either... Japanese YKK zippers are the best example there [1].
A side note: if you don't have VCs or, worse, the vipers of the public stock market breathing down your neck, you have so much more freedom as a founder. You can do what's best for the company long term.
[1] https://slate.com/business/2012/04/ykk-zippers-why-so-many-d...
> That’s the danger of the mediocre success. The point of startup experimentation isn't the success itself; it’s the learning that comes with clear-cut success or failure. You don't really care about the sales revenue generated by your first two reps; you care about whether this is a strategy you can scale to dozens and then hundreds of reps, or whether you need to use a completely different strategy. It’s all about the learning. And mediocre successes don’t give you any learning.
> In Comments
> Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.
> Comments should get more thoughtful and substantive, not less, as a topic gets more divisive.
[0] https://news.ycombinator.com/newsguidelines.html
> Don't be snarky. Converse curiously; don't cross-examine.
Sometimes we break guidelines. Sometimes we dont read. We’re all human.
Eh, that is doing OK to me, which is better than some nebulous "mediocre success". If you are at least saving for retirement, and able to support a family, that's actual success in my book.
The typical startup outcome, where you try for a couple years, it fails, and then you move on to a corporate job is not terrible.
What's really bad is when you make enough to just barely get by, paying yourself a sub-market-rate salary, and keep limping along for a long while. Waiting for that "success", that big breakthrough, "just around the corner". You can be stuck doing this for years, out of a misguided sense of loyalty, determination or whatever. And then you are not saving for retirement, and letting the best family-creation years (if that is a life goal) pass you by.
That, to me is "mediocre success".
Ask me how I know. On second though, no, don't ask me how I know.
That seems like a failure to define success in the first place. That seems like doing an experiment and then deciding how it made you feel. You probably want to declare beforehand what your minimum number of sales would be in order to qualify as a success. I agree that the results can still be ambiguous, but the situation he's describing sounds like a poorly designed experiment anyway.
It is hard to define that and sometimes can be noise or mask important information.
For example, some products have long conversion times due to pricing. This is a critical metric to follow, but would be ignored by the metric you defined.
You are in precise agreement with the article's own text: "So when a startup comes to me with an idea for an experiment, the one thing I tell them is: make sure that there’s a well-defined distinction between success and failure."
https://techhq.com/2023/02/patent-trolling-latest-victim/
Author argues for trying to be objective about the viability of business ventures here. It's easy to tell yourself lies that doing just one thing a little better on the execution front will magically triple your conversion rate.
If you think that scientific rigour will help you avoid the need for good judgement, you're in for a great deal of distress. Edited for typos.
Broadly, assuming more data and tighter calculations == more certainty is a folly endemic to engineering-focused organizations and industries. In reality, you're increasing precision without increasing accuracy: a dangerously misleading state. It's understandable: organizations obviously need to play to their strengths, but there's real danger in succumbing to the "when you're a hammer, everything looks like a nail" mindset. Firstly, It's much easier for non-engineers to parse the difficulty of engineering problems than the other way around. Outsiders easily see that they don't have the requisite chops to do engineering work. Secondly, cutting through ambiguity and unpredictability to reveal factors you can control is critical to engineering, while non-engineering jobs like management, design, and community outreach are hard because you must confront those things. Engineering types often reason about non-engineering problems either like they have the same level of predictability, or simply disregard the ambiguous and unpredictable factors because they don't fit into an equation.
It's easy to see why that micromanaging non-technical manager screwed up by insisted on using some technology or approach they read a snappy article about; an engineer's authority is cut-and-dried in that situation. It's harder to see why purely formulaic approaches often fail dealing with people, nature, markets, etc. Most things in the world are far more complex, temperamental, and less predictable than cache invalidation.
-- N.N. Taleb
The thinking so different than the candy store margin mentality of 3 generations ago.
Love it.
This is what all big company PMs and directors of engineering believe. But their judgement (good or bad), their metrics (scientific or pseudoscientific), their career trajectory (up or out), whether these are known or unknown in the first place, and their actual material success: who knows how correlated it all is. They don’t have the words to express this stuff, so they wouldn’t be able to see the difference between science and pseudoscience for example, and you'd have a hard time communicating this thing about science to these people.
So while you are right, it’s not saying much that the real problem is communications, and that is the mainstream opinion of people who study microeconomics / the structure of firms. Put differently, people have built cathedrals of bullshit in their minds in service of the status quo where they “test” “everything” in lieu of having falsifiable, forward-looking opinions (aka judgement). You can’t just go, Martin Luther all of corporate white collar bureaucracy.
Scientific rigor is inseparable from good judgement.
I mean, my corporate job is realistically pretty cushy and as the sole income earner, my family depends on me. I'm sure I could do well on my own, but it's hard to predict the success of any venture. Failure I actually don't mind, because the answer is easy... just get another job. The worst possible outcome is 'success' where you have a customer base and demand, but it's just never going to be enough to make up for that lost income. That would truly be the worst. You can't just quit without damaging your reputation; on the other hand, selling would likely be difficult without significant recurring income.
The author is right; you want clear signals, not noisey ones. If you're looking for the light but keep seeing shadows you need to turn around.
The same is true for people who launch on product hunt, and get 50 signups. Success? If you're selling $1K deals, yes. But if it's just for $4, or free... do you have PMF?
but to be fair, if you can get a small group of people engaged, and you can tease out why/how, it usually stands that others will respond too. But not everyone will need it, and those that may need it may not be in market for it.
There's a book for that https://www.themessymiddle.com