'We find no evidence, by contrast, that experts can effectively assess the commercial potential of venture ideas in non-R&D-intensive sectors such as consumer web and enterprise software. Finally, we find that industry-specific and scientific expertise is not critical to experts’ collective ability to predict ventures’ commercial viability.'
Paraphrasing Felix Dennis: Barely passable ideas with good marketing and timing can work if the execution is solid.
This also holds up to other research concluding timing is the biggest factor.
I've seen a number of solid startup teams cash in on recycling good ideas from the past (m&a shutdown) and execute well to satiate similar, modern needs. Novelty can be risky (Pebble / Apple Watch) because the latest entrant can learn and improve on what came before.
How can that and "we find that ideas that elicit more positive evaluations are significantly more likely to
ultimately reach commercialization" coexist?
They are also working off 652 ventures coming out of MIT. Seems like a small and biased data set.
OP failed to provide the surrounding context. Here it is:
> Using data on 652 ventures in multiple industry sectors, evaluated over an 8-year period, we find that ideas that elicit more positive evaluations are significantly more likely to ultimately reach commercialization. We further show that these results are driven by venture ideas with
documented intellectual capital in research-and-development-intensive sectors, such as life sciences and
medical devices. We find no evidence, by contrast, that experts can effectively assess the commercial
potential of venture ideas in non-R&D-intensive sectors such as consumer web and enterprise software.
More evidence of why a bunch of tech entrepreneurs (YC) are not the right people to spur biotech (and other hard sciences) innovation: https://news.ycombinator.com/item?id=9997722
What's transferable between different tech ventures (other than connections and money) is knowledge about people and how to sell to (and generally deal with) them - issues that regularly come up long after the venture is formed. But the most important factor in R&D-based hard sciences ventures is deep technical expertise. Once that is in place, it's pretty obvious what needs to happen. And you can be sure that YC isn't getting those 'obvious' inventions.
> ultimately reach commercialization, defined as having recurring revenue and expenses associated with the sale of products and/or services in keeping with the company’s business objective.
How viable something is over the long run and its overall success is different from being able to have it out in the market and get inital paying customers.
Reaching commercialization by this definition and receiving better evaluation from VCs means more funding readily avaiable to get a product out.
Looking at ideas alone is maybe no enough. Most of the 'unicorn' startups started as a yet-another X and many others failed with the same idea.
If even experts like y combinator are playing a numbers game akin to a professional sports better or poker player (playing many hands, happy to lose some but win overall) then I don't think we'll ever find a clairvoyant who can predict if a single team or idea will be successful with any degree of accuracy.
I assume YC mainly selects by first looking at the potential market and the team. The idea itself is less important. If 'only' the founders will stick together, work hard, learn fast and are adaptable.
A good team (history, education, friendship, track record, focused, balanced, ...) in a potential huge market is worth the gamble. Ideally the setup (I guess) 'only' needs a lot of money to make the difference. Because money will then be available in large quantities.
We're probably moving in to near tautological territory, here. If your venture doesn't depend in genuinely novel technology with significant R&D investment, then... its success probably depends on fundraising, marketing, and luck. All of which don't require industrial or scientific expertise.
"elicit subjective evaluations from a large set of experienced entrepreneurs and executives"
Sounds like they're including evaluation by VC-types, not just industrial or scientific expertise. Point being that R&D intensive work is more predictable in general, and VC predictive ability may be overstated.
This is not really true. There are plenty of ideas that don't involve a significant R&D phase (i.e. consist of more or less the implementation of the idea rather than the research-based refinement of an idea) which can immensely valuable.
Plenty of examples exist such as Uber, Basecamp, etc. Although the success of these ventures is dependent on fundraising and marketing, these are certainly not the only factors and these factors apply to research-heavy startups as well.
I think this study is trying to make statistical predictions based on a limited dataset. The industry has shown the outliers (or statistical anomalies) are the ones that result in the biggest companies.
Entrepreneurs face high uncertainty, and often make costly investments in new business ideas without
knowing the expected payoff. This paper empirically examines whether ex-ante assessment of earlystage
startup ideas can predict their subsequent commercialization. We leverage an entrepreneurship
program at the Massachusetts Institute of Technology in which early-stage venture ideas, presented in
the form of succinct standardized summaries, elicit subjective evaluations from a large set of experienced
entrepreneurs and executives. Using data on 652 ventures in multiple industry sectors, evaluated over
an 8-year period, we find that ideas that elicit more positive evaluations are significantly more likely to
ultimately reach commercialization. We further show that these results are driven by venture ideas with
documented intellectual capital in research-and-development-intensive sectors, such as life sciences and
medical devices. We find no evidence, by contrast, that experts can effectively assess the commercial
potential of venture ideas in non-R&D-intensive sectors such as consumer web and enterprise software.
Finally, we find that industry-specific and scientific expertise is not critical to experts’ collective ability
to predict ventures’ commercial viability.
Hmm, so in a tech filed where you have the IP or some legal rigmarole to protect a complicated process-invention-proceedure-etc, it is statistically easier to see that it's a winning bet.
I mean, duh. But this is important because it doesn't hold for companies trying to be another FB or Zynga, those are gambles.
If the company has good stuff and the wherewithal to protect it, good people will know it when they see it.
Their results may be significant to VCs, but likely aren't to anyone else.
They get about a 4% increase in good outcomes, from 23% to 27%, for a 1 sigma increase in expert positive response. This is significant at the 5% level.
It's enough to say that "Experts do slightly better than chance in IP-heavy fields", but about 3/4 of those businesses will still fail, and that remains true regardless of expert evaluation.
If you're a big investor this could make a difference to your bottom line. If you're a small investor or entrepreneur, it is pretty much irrelevant.
Rule 1: Find a startup with
traction significantly large and
growing rapidly in a huge market
and where, still, the founders are
very short on cash and will sign
a term sheet with some really onerous
conditions. E.g., the founders suddenly
go from owning 100% to owning 0% and
may get back to 60% on a four year
vesting schedule. The BoD, dominated
by the venture capital partners, can
fire the founders before their stock
is vested.
Rule 2: Claim a lot of items on a long
list of what venture capital is about
other than Rule 1. Claim it's about
technology, innovation, brilliant founders,
great leaders, changing the world,
with the venture firm helping by
using their long experience, "deep
domain knowledge", etc.
Rule 2 is 100% noise, just
misdirection, to keep
people from noticing Rule 1.
Other major parts of our society are
really good at innovation, creating it,
evaluating it, exploiting it, etc.,
but not the venture capital firms.
They won't evaluate innovations.
Instead, again, they just
follow the two rules.
As in the title, "better ideas"? The
venture firms never consider the
quality of the ideas -- instead, they
just follow Rules 1 and 2. Or,
on evaluating the ideas, they just
let the market decide and then
look at traction to see what the
market has decided.
Sometimes they make a lot of money,
but on average their returns are poor.
Are VC returns really poor on average? I thought their hit rate (1 company out of 50 being successful) was poor, but their returns are stellar because of the few unicorns.
If your conclusion is correct, how are the VCs staying in business?
> If your conclusion is correct, how are
the VCs staying in business?
Reasons:
(1) The money comes from the limited
partners (LPs), and they are heavily
investors with big bucks. We're talking
billions, from university endowments,
state pension funds, sovereign wealth
funds (say, a fund run by a rich Mideast
oil country), some family wealth funds,
maybe some insurance companies, maybe some
hedge funds.
Of course, venture might hit a grand slam
with great returns, and no doubt each
venture firm that raises money from LPs
claims that they are a top tier firm
with great advantages and some great
investment themes that with their "deep
domain knowledge" and great business
experience and insight will let them get
much better than average returns.
These limited partners invest billions in
all good looking asset classes. Venture
capital is regarded by the LPs as an
asset class but is relatively small.
So, for the limited partners, venture
capital is small potatoes and slips in
under the wire even if on average the
returns have been poor.
(2) The LPs have FMO -- fear of missing
out. Due to the rapid progress of
technology, the Internet, etc., the
venture firms can keep telling the LPs
that "This time it's different." with some
possibility of being correct. So, some
portfolio manager at an LP doesn't want to
have to answer why they didn't invest in
KPCB or Sequoia and, thus, missed out on
Google.
(3) The LPs are not good at technology
or able to evaluate "ideas" or look for
"better" ideas. Instead, the LPs are
closer to traditional commercial bank
lending and private equity that put a lot
of weight on accounting statements and
don't expect to evaluate ideas or
technology. So, the LPs are not pushing
their venture firms to evaluate "ideas".
Assuming you are correct then it should be possible to fool VCs by creating products that achieve traction using the age old trick of selling a dollar for 90c. I am sure they would never fall for this.
I suspect the real business model for most of these high-growth-without-a-plan-to-make-a-profit businesses is really eyeball exhaustion. The big players like google and facebook can monetize eyeballs, but they can only do this if they have people on their platforms. A loss making business can grab enough eyeballs that they will significantly affect the bottom line of the large players. The large players have to respond by buying up these startups and shutting them down.
As long as the VCs don’t back too many of these types of startups then it should be quite an effective way to shakedown the large players.
> Using data on 652 ventures in multiple industry sectors, evaluated over an 8-year period, we find that ideas that elicit more positive evaluations are significantly more likely to ultimately reach commercialization.
It sounds to me that ideas with positive evaluations are more likely to be funded through to commercialization. Not a very interesting finding?
> We find no evidence, by contrast, that experts can effectively assess the commercial
potential of venture ideas in non-R&D-intensive sectors such as consumer web and enterprise software.
This paper provides evidence for what some might consider an obvious conclusion, but there is always value in documenting evidence, better yet, generating it.
The paper says established sectors are easier to predict than radical ones, just as progress is easier to measure in established sciences, as opposed to where paradigms may shift. And when faced with a paradigm shifting value proposition, no one really knows what to make of it until the rubber hits the road and the model is tested in the market.
But once established, even a paradigm shifter becomes a crowded R&D sector, no? The iterations are faster, but consumer web and enterprise software do seem more crowded. Search, social networks, and web technology plays also come to mind.
What I didn't find the paper mention were how open source and the lack of IP monopolization as an option could be a factor, and also how the entrepreneurs themselves stacked up in comparison. For me personally, it would suck if "the entrepreneurs didn't really matter" (the conclusion asserts better human capital gravitates towards the ventures with greater potential, but doesn't say anything about any influences they actually have).
I found a rather strange index to rate the success-rate of future projects.
(MentionedInConversation x InfectiousnessToOther)EnduranceOfIdea ( 1/TechAffinity)
Meaning, if your aunt talks to your mum about it, and the general idea comes up once or twice a week and migrates on, even without mentioning the company - and your mum&aunt is not enthusiastic for just the technology, then its going to make it.
Case: AirBnB - mentioned once or twice - misspronounced, then mentioned only as concept for three months - the idea stuck to rent out rooms to total strangers without big formalitys. Such buisness_idea_memes are easy to scale, propagate themselves in a way and the technology has basically some catalytic function to a allready existing small scale buisness concept that a person is allready familiar with but did not dare the risk previously.
"I could make a small hostel.."
One could imagine this for allmost anything..
Make a app for people who wish they where at partys and able the enjoy them and are even willing to pay for that, and connect them with party-animals lacking the money, willing to drag somebody shy along. Idea could run without your app, but your app propells it onwards. Voila!
29 comments
[ 3.1 ms ] story [ 69.9 ms ] thread'We find no evidence, by contrast, that experts can effectively assess the commercial potential of venture ideas in non-R&D-intensive sectors such as consumer web and enterprise software. Finally, we find that industry-specific and scientific expertise is not critical to experts’ collective ability to predict ventures’ commercial viability.'
This also holds up to other research concluding timing is the biggest factor.
I've seen a number of solid startup teams cash in on recycling good ideas from the past (m&a shutdown) and execute well to satiate similar, modern needs. Novelty can be risky (Pebble / Apple Watch) because the latest entrant can learn and improve on what came before.
They are also working off 652 ventures coming out of MIT. Seems like a small and biased data set.
> Using data on 652 ventures in multiple industry sectors, evaluated over an 8-year period, we find that ideas that elicit more positive evaluations are significantly more likely to ultimately reach commercialization. We further show that these results are driven by venture ideas with documented intellectual capital in research-and-development-intensive sectors, such as life sciences and medical devices. We find no evidence, by contrast, that experts can effectively assess the commercial potential of venture ideas in non-R&D-intensive sectors such as consumer web and enterprise software.
More evidence of why a bunch of tech entrepreneurs (YC) are not the right people to spur biotech (and other hard sciences) innovation: https://news.ycombinator.com/item?id=9997722
What's transferable between different tech ventures (other than connections and money) is knowledge about people and how to sell to (and generally deal with) them - issues that regularly come up long after the venture is formed. But the most important factor in R&D-based hard sciences ventures is deep technical expertise. Once that is in place, it's pretty obvious what needs to happen. And you can be sure that YC isn't getting those 'obvious' inventions.
> ultimately reach commercialization, defined as having recurring revenue and expenses associated with the sale of products and/or services in keeping with the company’s business objective.
How viable something is over the long run and its overall success is different from being able to have it out in the market and get inital paying customers.
Reaching commercialization by this definition and receiving better evaluation from VCs means more funding readily avaiable to get a product out.
Looking at ideas alone is maybe no enough. Most of the 'unicorn' startups started as a yet-another X and many others failed with the same idea.
If even experts like y combinator are playing a numbers game akin to a professional sports better or poker player (playing many hands, happy to lose some but win overall) then I don't think we'll ever find a clairvoyant who can predict if a single team or idea will be successful with any degree of accuracy.
A good team (history, education, friendship, track record, focused, balanced, ...) in a potential huge market is worth the gamble. Ideally the setup (I guess) 'only' needs a lot of money to make the difference. Because money will then be available in large quantities.
We're probably moving in to near tautological territory, here. If your venture doesn't depend in genuinely novel technology with significant R&D investment, then... its success probably depends on fundraising, marketing, and luck. All of which don't require industrial or scientific expertise.
Sounds like they're including evaluation by VC-types, not just industrial or scientific expertise. Point being that R&D intensive work is more predictable in general, and VC predictive ability may be overstated.
Plenty of examples exist such as Uber, Basecamp, etc. Although the success of these ventures is dependent on fundraising and marketing, these are certainly not the only factors and these factors apply to research-heavy startups as well.
Entrepreneurs face high uncertainty, and often make costly investments in new business ideas without knowing the expected payoff. This paper empirically examines whether ex-ante assessment of earlystage startup ideas can predict their subsequent commercialization. We leverage an entrepreneurship program at the Massachusetts Institute of Technology in which early-stage venture ideas, presented in the form of succinct standardized summaries, elicit subjective evaluations from a large set of experienced entrepreneurs and executives. Using data on 652 ventures in multiple industry sectors, evaluated over an 8-year period, we find that ideas that elicit more positive evaluations are significantly more likely to ultimately reach commercialization. We further show that these results are driven by venture ideas with documented intellectual capital in research-and-development-intensive sectors, such as life sciences and medical devices. We find no evidence, by contrast, that experts can effectively assess the commercial potential of venture ideas in non-R&D-intensive sectors such as consumer web and enterprise software. Finally, we find that industry-specific and scientific expertise is not critical to experts’ collective ability to predict ventures’ commercial viability.
I mean, duh. But this is important because it doesn't hold for companies trying to be another FB or Zynga, those are gambles.
If the company has good stuff and the wherewithal to protect it, good people will know it when they see it.
They get about a 4% increase in good outcomes, from 23% to 27%, for a 1 sigma increase in expert positive response. This is significant at the 5% level.
It's enough to say that "Experts do slightly better than chance in IP-heavy fields", but about 3/4 of those businesses will still fail, and that remains true regardless of expert evaluation.
If you're a big investor this could make a difference to your bottom line. If you're a small investor or entrepreneur, it is pretty much irrelevant.
Rule 1: Find a startup with traction significantly large and growing rapidly in a huge market and where, still, the founders are very short on cash and will sign a term sheet with some really onerous conditions. E.g., the founders suddenly go from owning 100% to owning 0% and may get back to 60% on a four year vesting schedule. The BoD, dominated by the venture capital partners, can fire the founders before their stock is vested.
Rule 2: Claim a lot of items on a long list of what venture capital is about other than Rule 1. Claim it's about technology, innovation, brilliant founders, great leaders, changing the world, with the venture firm helping by using their long experience, "deep domain knowledge", etc.
Rule 2 is 100% noise, just misdirection, to keep people from noticing Rule 1.
Other major parts of our society are really good at innovation, creating it, evaluating it, exploiting it, etc., but not the venture capital firms. They won't evaluate innovations. Instead, again, they just follow the two rules.
As in the title, "better ideas"? The venture firms never consider the quality of the ideas -- instead, they just follow Rules 1 and 2. Or, on evaluating the ideas, they just let the market decide and then look at traction to see what the market has decided.
Sometimes they make a lot of money, but on average their returns are poor.
If your conclusion is correct, how are the VCs staying in business?
http://www.avc.com/a_vc/2013/02/venture-capital-returns.html...
http://www.kauffman.org/newsroom/2012/07/institutional-limit...
Net, on average, the returns are poor.
> If your conclusion is correct, how are the VCs staying in business?
Reasons:
(1) The money comes from the limited partners (LPs), and they are heavily investors with big bucks. We're talking billions, from university endowments, state pension funds, sovereign wealth funds (say, a fund run by a rich Mideast oil country), some family wealth funds, maybe some insurance companies, maybe some hedge funds.
Of course, venture might hit a grand slam with great returns, and no doubt each venture firm that raises money from LPs claims that they are a top tier firm with great advantages and some great investment themes that with their "deep domain knowledge" and great business experience and insight will let them get much better than average returns.
These limited partners invest billions in all good looking asset classes. Venture capital is regarded by the LPs as an asset class but is relatively small. So, for the limited partners, venture capital is small potatoes and slips in under the wire even if on average the returns have been poor.
(2) The LPs have FMO -- fear of missing out. Due to the rapid progress of technology, the Internet, etc., the venture firms can keep telling the LPs that "This time it's different." with some possibility of being correct. So, some portfolio manager at an LP doesn't want to have to answer why they didn't invest in KPCB or Sequoia and, thus, missed out on Google.
(3) The LPs are not good at technology or able to evaluate "ideas" or look for "better" ideas. Instead, the LPs are closer to traditional commercial bank lending and private equity that put a lot of weight on accounting statements and don't expect to evaluate ideas or technology. So, the LPs are not pushing their venture firms to evaluate "ideas".
I hope that is sarcasm: https://en.wikipedia.org/wiki/Groupon#Business_model
I suspect the real business model for most of these high-growth-without-a-plan-to-make-a-profit businesses is really eyeball exhaustion. The big players like google and facebook can monetize eyeballs, but they can only do this if they have people on their platforms. A loss making business can grab enough eyeballs that they will significantly affect the bottom line of the large players. The large players have to respond by buying up these startups and shutting them down.
As long as the VCs don’t back too many of these types of startups then it should be quite an effective way to shakedown the large players.
But, but, but, they plan to make it up on the volume!
It sounds to me that ideas with positive evaluations are more likely to be funded through to commercialization. Not a very interesting finding?
> We find no evidence, by contrast, that experts can effectively assess the commercial potential of venture ideas in non-R&D-intensive sectors such as consumer web and enterprise software.
The paper says established sectors are easier to predict than radical ones, just as progress is easier to measure in established sciences, as opposed to where paradigms may shift. And when faced with a paradigm shifting value proposition, no one really knows what to make of it until the rubber hits the road and the model is tested in the market.
But once established, even a paradigm shifter becomes a crowded R&D sector, no? The iterations are faster, but consumer web and enterprise software do seem more crowded. Search, social networks, and web technology plays also come to mind.
What I didn't find the paper mention were how open source and the lack of IP monopolization as an option could be a factor, and also how the entrepreneurs themselves stacked up in comparison. For me personally, it would suck if "the entrepreneurs didn't really matter" (the conclusion asserts better human capital gravitates towards the ventures with greater potential, but doesn't say anything about any influences they actually have).
(MentionedInConversation x InfectiousnessToOther)EnduranceOfIdea ( 1/TechAffinity)
Meaning, if your aunt talks to your mum about it, and the general idea comes up once or twice a week and migrates on, even without mentioning the company - and your mum&aunt is not enthusiastic for just the technology, then its going to make it.
Case: AirBnB - mentioned once or twice - misspronounced, then mentioned only as concept for three months - the idea stuck to rent out rooms to total strangers without big formalitys. Such buisness_idea_memes are easy to scale, propagate themselves in a way and the technology has basically some catalytic function to a allready existing small scale buisness concept that a person is allready familiar with but did not dare the risk previously.
"I could make a small hostel.."
One could imagine this for allmost anything.. Make a app for people who wish they where at partys and able the enjoy them and are even willing to pay for that, and connect them with party-animals lacking the money, willing to drag somebody shy along. Idea could run without your app, but your app propells it onwards. Voila!