Basic error is that you think you can predict fully random process. You develop a model that seems to work excellent until it doesn't.
It doesn't matter whether it's sophisticated algorithm running on big iron or hunch running on wetware. You will fail because nobody knows the future. Trying to reliably squeeze money out of the market by better predicting its behavior is like trying to build perpetum mobile by trying more and more convoluted designs.
There is money that can be reliably extracted from investing though. If you can trade faster than others, you can drain some money from them. That's why algorithmic trading yields returns. Your strength comes not from predicting future better, but from acting and reacting faster.
"If you can trade faster than others, you can drain some resources from them"
What's embarrassing is how many intellectual resources are focused on this problem. For those not in finance, compare it to focusing talent on finding better advertising techniques.
Certainly you need to protect against over fitting your models, and you need to consider tail events.
But to say that the entire market is fully random is to claim that the efficient market hypothesis holds, which it clearly doesn't - not even in weak form. Anyone who wants to claim that the EMH is true needs to explain the equity premium puzzle, the success of value investing and the persistent outperformance of momentum investing - among many other price anomalies that I could name.
> But to say that the entire market is fully random is to claim that the efficient market hypothesis holds, which it clearly doesn't - not even in weak form.
Of course it does. All evidence supports the Efficient Market Hypothesis (EMH), and note my use of the word "evidence". A contradiction would have to be a series of funds that applied a given, predefined (internally) strategy and (very important) didn't reveal its picks to the public (to avoid the announcement effect). Such an experiment hasn't been done, and those cases in which a specific strategy was applied have been universal failures.
Let's say for the sake of argument that there is a surefire winning strategy as is so often claimed in worthless investment books. If it were to be recognized and then applied, it would become the new market norm, everyone would apply it, and its differential value would evaporate overnight. That hypothetical experiment merely supports the EMH.
> Anyone who wants to claim that the EMH is true needs to explain the equity premium puzzle, the success of value investing and the persistent outperformance of momentum investing - among many other price anomalies that I could name.
Those result from a combination of the announcement effect and insider trading. Berkshire-Hathaway is successful because people expect it to be successful, and who invest in the same stocks when they're announced. It's a self-fulfilling prophecy.
The only fair, scientific test would be a strategy that is applied in secret, not in public like Berkshire-Hathaway, and in a way that doesn't tweak the market itself by way of large purchases that create their own following of fanboys. In other words, a pure computer model that is defined in advance, not tweaked enroute, that doesn't actually invest but only models hypothetical investments, and that shows its value over other systems in perpetuity. There is no such thing, and the EMH is true.
Remember the Null Hypothesis, the basis of modern scientific thinking. The Null Hypothesis assumes something without evidence is false. All evidence supports the EMH.
Only in academic circles does EMH stay true. Time and time again, reality has shown that EMH is not applicable, or exists in a very weak form. One need only look to the LBO bubble and the dotcom bubble to see counter-examples of EMH.
In a conversation like this, rhetoric is cheap, and there is no evidence for your position. All evidence suggests that the EMH is true. Prove this wrong -- identify a surefire winning stock picking strategy that is immune to the announcement effect and insider trading, that reliably puts its practitioners ahead of market indices. There is no such thing.
> One need only look to the LBO bubble and the dotcom bubble to see counter-examples of EMH.
Those are not counterexamples, and they bear no relevance to the present topic. Anyone can misbehave in spectacular ways, but those misbehaviors don't falsify the EMH, any more than an airplane crash falsifies theories of aerodynamics.
First, from Wikipedia: "Empirical analyses have consistently found problems with the efficient-market hypothesis, the most consistent being that stocks with low price to earnings (and similarly, low price to cash-flow or book value) outperform other stocks." At the very least, accept that it's not such a clear case as you present it to be.
But then, what about HFT arbitrage, low P/E ratios, choosing underperforming companies to take advantage of regression to the mean, Piotroski's F-Score, Graham-style value investing, bait and switch dumping, swing trading, the very real problem of pricing goodwill and non-material resources...? If not undermining weak EMH, at least they cast doubts over the strong forms. Or they should. Do also read John Carter's Mastering the Trade for some examples on how market inefficiencies can be exploited reliably.
Supposedly (seen in Wikipedia, so I'm not that confident) market crashes are interpreted to be impossible under strong EMH (and yeah, the bubbles would definitely falsify them), but are allowed in weak EMH to be statistical events of a random process. Now, I've heard an interesting and convincing argument by Steve Keen[1] that the market is not Gaussian noise; it's a fractal process with dimension ~ 1.7 (Gaussian's dimension is 2) whose crashes obey a power law distribution, very different from what a random walk would generate. That falsifies the weak form.
> Prove this wrong -- identify a surefire winning stock picking strategy that is immune to the announcement effect and insider trading, that reliably puts its practitioners ahead of market indices.
Falsifying EMH does not require such a thing, no more than proving relativity requires observing time dilation at all velocities. This is an onerous, diabolical burden of proof.
> Anyone can misbehave in spectacular ways, but those misbehaviors don't falsify the EMH, any more than an airplane crash falsifies theories of aerodynamics.
If you had periodic epidemics of plane crashes that were more frequent than expected, sure, you should definitely suspect something's wrong.
[1] Yeah, some things he says are somewhat crackpotty, but he's definitely not to be completely dismissed beforehand. I refer to his Behavioural Finance Lectures, published in Youtube.
>> Prove this wrong -- identify a surefire winning stock picking strategy that is immune to the announcement effect and insider trading, that reliably puts its practitioners ahead of market indices.
> Falsifying EMH does not require such a thing ...
As a matter of fact, yes, it does. The only acceptable evidence against the EMH is a performance that the EMH cannot accommodate -- a successful strategy for reading and exploiting the market in a way that performs better than market indices. And not just sometimes, but all the time, reliably.
Science requires evidence. Evidence in this case means producing results above the norms set by the market averages, and not just sometimes, but all the time. Anything else is empty philosophizing.
Performances below the norms can be explained in any number of ways, but above the norms is a different story. And by "above the norms" I don't mean 1/2 the time or 1/2 the practitioners -- that's assured by chance factors.
All this talk about the EMH being flawed, not being the whole story, falls down before the fact that no one can invent a strategy to improve on long-term market averages, and that says it all. It is the ultimate evidence in favor of the EMH.
If no one can think of a way to read the market to his advantage, better than average performance, without engaging in insider trading or reliance on the announcement effect, then the default assumption is supported.
> This is an onerous, diabolical burden of proof.
Be that as it may, it is an essential standard of proof. Nothing else matters -- no amount of philosophizing can stand in for actual results.
The expert gets up and says, "The EMH is flawed, meaningless, unscientific, wrong, etc., etc.." Someone from the audience says "All right -- prove it. Produce a strategy that beats the market norms." The expert is naturally taken aback -- "Are you serious? What about the EMH? What about the fact that every single strategy that has ever been tried, has failed in the long term?"
This is like people in a bar talking about sailing. Any number of ideas about sailing are aired over intoxicating beverages, and it's all very interesting, but since no one goes out on the water, it's completely meaningless.
Proving that EMH is false requires revelation of a profitable method or at least a statistically significant edge, which then makes it vulnerable to getting exploited away. It's actually more useful if the public believes in EMH and in traders' best interest to keep their edges secret.
> Proving that EMH is false requires revelation of a profitable method or at least a statistically significant edge, which then makes it vulnerable to getting exploited away.
Yes, true, and such an outcome ironically supports the EMH.
> It's actually more useful if the public believes in EMH and in traders' best interest to keep their edges secret.
Actually, it is much, much more advantageous for traders to pretend that there is a surefire winning strategy that only they know about. The problem with that plan is the fact that there is no such strategy.
How do I know this? If there really was a surefire winning strategy, a trader who controlled it could (and would want to) do without any public involvement, and invest his own funds in private until he became absurdly rich.
That's the obvious way to play it -- why share the proceeds, and why risk exposing the method, by dealing with gullible, ungrateful clients? The answer is that those gullible, ungrateful clients are what makes the present system work, not some secret sauce that falsifies the EMH.
We are talking here at such high level abstractions and with stack of presuppositions that it's difficult for me to figure out just where to even begin explaining my point and i question if i even should.
If one doesn't know of a 'sure fire winning strategy' (== strong form inefficient market hypothesis ?) ( or just statistically useful may be good enough ) one should not participate in the market in most forms ( which market, participate how ). I've heard it compared to a poker game where if you don't know who the sucker is it's probably you.
The general public is misinformed about the markets and gets consistently fleeced yet keeps getting pulled in by dreams of easy money - akin to a lottery.
Markets have defined purposes (theoretical, practical, legal) and serve them accordingly. Enriching 'gullible, ungrateful clients' is not the primary purpose. EMH is a useful explanation to the clients why they got fleeced, and i don't want to disprove it.
It takes a lot of work to figure out how the lottery works and how to consistently game it. It's just a job.
> ... or just statistically useful may be good enough ...
If "statistically useful" means reliably above market averages year after year, then that would constitute a surefire winning strategy. There is no such thing -- if this were not so, we all would know about it by now.
A tiny advantage -- a legal one, not dependent on insider trading or any other questionable premise -- however small, could be exploited to enrich the player who used it.
But there is no such thing.
> The general public is misinformed about the markets and gets consistently fleeced yet keeps getting pulled in by dreams of easy money - akin to a lottery.
Yes, and the key to all these misleading strategies is ignorance of the EMH.
Evidence for the EMH lies in the fact that, in the long term, buy & hold investors do better than players -- in the long term, on average, if all results are tallied objectively.
> It takes a lot of work to figure out how the lottery works and how to consistently game it.
Again, there is no such thing. If there were, businesses would refuse to use equities to raise capital, knowing they would be fleeced by the players. The only reason equities are a reasonable way to capitalize business is because it's a fair approximation of an efficient market.
For the life of me, I can't figure out why people find this so hard to understand.
Reading back, i actually agree with you on most points. It just seems that you're searching for some sort of 'fairness' in a 'system' where it doesn't really exist.
Just would like to add a distinction that EMH seems to hold from a retail clients perspective, but does not hold from a systematic / structural side.
Markets are structurally exploitable. For example it is (generally) illegal for an institution to front run clients' orders, however it is ok if you are just a trader and know probable (not necessarily sure fire) locations of large orders and front run them.
> It just seems that you're searching for some sort of 'fairness' in a 'system' where it doesn't really exist.
But "fairness" is what the EMH means. "Fairness" expresses in human terms what the mathematics tells us. An efficient market ruthlessly adjusts for inefficiencies and oversights, sort of like biological evolution, and evolution is the ultimate in ruthlessness and indifference to the fate of individuals. It's also fair -- each species has an equal chance to prevail.
Insider trading is unfair to ordinary investors who aren't criminals. The announcement effect is unfair to ordinary investors who don't have that kind of influence over the markets. If you take away insider trading and the announcement effect, the result is ... more fair. That result is closer to an optimally efficient market.
> Markets are structurally exploitable.
Only by cheating (and as your example demonstrates). The SEC knows this, and they know how corrosive distrust in the fairness of equities can be for society as a whole, so they aggressively pursue cheaters.
Absent cheating, equities markets cannot be gamed. Absent cheating, a long-term buy & hold position is the optimal strategy, By the way, and for what it's worth, Warren Buffett agrees.
Markets have many inefficiencies of varying persistence that don't depend on insider knowledge, speed advantages, or cheating, but using simple widely available public information, some on just the price itself.
I think it doesn't validate EMH if such inefficiencies disappear if exposed, if they depend on skill and systemic knowledge rather than 'cheating' - i suppose it depends on where one draws the line between cheating and skill.
Discovering and exploiting them is a matter of study and work which the public is unwilling to do, and actually actively discouraged by the status quo of this industry.
They're guided into ignorance and defering their finances to the sucky professionals who get paid for amount of assets under management rather than performance.
Warren Buffet doesn't buy and hold indiscriminately. He buys things on sale during market crashes when the panicky public gladly sell to him. For example by selling put options he gets paid just for waiting for the discounts.
And as a potentially controvertial aside, i think his size and connections within the government grant him hypocritical priviledges not available to the public (other than through owning his stock of course).
> Discovering and exploiting them [i.e. inefficiencies] is a matter of study and work which the public is unwilling to do, and actually actively discouraged by the status quo of this industry.
Yes, but it's well-established that the average investor comes out behind by trying to exploit such inefficiencies, and this outcome supports the idea that the EMH is true, and that a buy & hold position produces the best returns on average, overall.
The problem with Wall Street anecdotes is that successes are given undue weight in the telling, misleading people into the idea that locating and exploiting inefficiencies puts you ahead. On average, this is false -- very, very false.
> Warren Buffet doesn't buy and hold indiscriminately.
Berkshire Hathaway's long-term success is easily explained -- its position is a matter of public knowledge, so others, hoping to see the same growth, buy the same stocks (too late to benefit themselves but just in time to benefit Berkshire Hathaway). It's called the "announcement effect".
I throw 100 dice 10 times and 2 of them come up 6 every time, i pick them up and call them survivors. Then i throw these 2 survivors 10 times and one of them again comes up 6 everytime - to me its worth the effort to figure out why, regardless if it's branded madoff or rentec or buffett, or that the other 99 exhibit traits of EMH.
Excusing it away feels like a lazy way out, settling for mediocrity. I make a living by throwing sixes time and time again. It's not cheating, it's not easy, it takes skill and work and is doable.
It's tough to know where to begin here. To refute the EMH you don't need a 'surefire winning strategy'. You only need a strategy that is profitable in expectation. And it doesn't need to be laid down in advance - all sources of expected return have a half life, and strategies need to be retired when they stop working. That necessitates continuous research.
Also, you seem to defeat your own argument. The strong form EMH says that excess returns are not possible, even with insider trading - which you admit can lead to outperformance. The weak form EMH would prohibit excess returns from the announcement effect, but you admit that leads to return predictability as well. Which is it - does the EMH hold, or is the announcement effect real?
Your arguments for why momentum, value and equity investing work also don't stand up. There's nothing to be "announced" in those strategies - you could apply them in a vacuum where you didn't know anything except prices and fundamentals. Similarly no insider knowledge is required. So why do you say that they are due to the announcement effect and insider trading?
You also misunderstand the scientific method - the failure of an experiment to refute the null hypothesis does not lend credence to the null hypothesis! At most it tells you that the null is consistent with the observations. Moreover, you talk as if the Null Hypothesis is a particular statement, which it is not. There's a great XKCD comic I'd refer you to if I wasn't on my phone!
If you want such an experiment, I humbly invite you to consider the past and future returns of the fund I work for, which is 100% model-driven. Although obviously we trade based on our predictions - because at the end of the day we're in this to make money, not to write papers or win internet arguments.
> To refute the EMH you don't need a 'surefire winning strategy'. You only need a strategy that is profitable in expectation.
Yes, and, in spite of its tremendous theoretical importance, there is no such thing.
> And it doesn't need to be laid down in advance ...
Is this a scientific conversation? Yes? Then yes, it does. Any number of bogus studies can prove their claims if one can change the study's claims as the evidence comes in.
> The strong form EMH says that excess returns are not possible, even with insider trading ...
Ah, I see -- redefine the EMH, then prove the falsity of the redefinition. The EMH says no such thing.
> The weak form EMH would prohibit excess returns from the announcement effect ...
Also false. You aren't discussing the EMH, you're making it up as you go along.
> There's nothing to be "announced" in those strategies - you could apply them in a vacuum where you didn't know anything except prices and fundamentals. Similarly no insider knowledge is required. So why do you say that they are due to the announcement effect and insider trading?
That's easy to answer -- I never said any such thing, anywhere. You're arguing against a position no one has taken.
I did say that an legitimate test needs to be conducted in private, with no actual investments -- a computer model defined in advance, that cannot be changed as the study proceeds, and that by definition cannot influence the real market being studied. There are no such successful models. The EMH appears to be true.
> If you want such an experiment, I humbly invite you to consider the past and future returns of the fund I work for ...
Ah, so you have a dog in this fight. You're not an impartial participant, striving for objectivity. That means I'm wasting my time talking to you -- you're not a scientist, you're a salesman.
Whatever the merits of your arguments, your ad hominem at the end, as well as the general tone of your comment, undermine your position almost completely.
What ad hominem was that? I correctly identified my correspondent as a salesman, not a scientist. It's a simple statement of fact. He thinks one data point counts as evidence, which means he isn't a scientist, and he is objectively in the business of talking people into investing in his fund -- therefore he is a salesman.
Simple straight-up facts. And it really is pointless trying to discuss these issues with someone whose livelihood depends on the myth that the market can be tweaked in any way we please.
Perhaps you're like me a decade ago and you don't even realise when you're insulting someone (I was like that). If so, I urge you to realise that just because you don't mean to be insulting doesn't mean you aren't.
The last line of your comment was obviously insulting and your comment has been downvoted by others as a result. That's an evident observation. Take it on board instead of disputing it, and if you really did not mean any insult, then think about how you could have phrased that line (and perhaps the rest of your comment) in a way that would not come across as insulting.
Learning to not involuntarily insult people is pretty useful.
> Perhaps you're like me a decade ago and you don't even realise when you're insulting someone ...
You have something against salesmen? I described this person as a salesman, after he identified himself as a salesman. Which part of this aren't you getting? He has a vested interest in the position he was putting forth. That fact, the fact that his livelihood depends on the continued belief in the predictability and malleability of equities trading, is germane to the conversation. It prevents objective evaluation of evidence.
> ... in a way that would not come across as insulting.
Not every salesman is Willy Loman, and the insult you saw is solely in your mind.
He said: "I humbly invite you to consider the past and future returns of the fund I work for ..."
By referring to "future returns", he went beyond any rational position, and he directly contradicts the position taken by the SEC on this issue (brokers should never make claims about future performance).
He is a salesman. Talking to him about this issue is a waste of time. Those are the facts.
No, those are not the facts. That is your interpretation, and it is worded in your vocabulary. That vocabulary means something to you but may mean something else to someone else.
The wiser way to behave here is to realise that your interpretation may or may not be true, since you are working from a highly incomplete data set and therefore tone down the use of vocabulary that may have multiple meanings and instead focus on vocabulary that is less likely to be interpreted as an insult.
For example, you might have said:
> I notice from one of your statements that you seem to be working for a fund yourself. Do you think that this impacts your view of this problem?
Instead of:
> Ah, so you have a dog in this fight. You're not an impartial participant, striving for objectivity. That means I'm wasting my time talking to you -- you're not a scientist, you're a salesman.
All the bits in italics (yes, that's most of your comment) are potentially insulting to the person you're replying to. The first one is a mild character attack, the second one is a direct insult, and the third one is easily perceived as a direct character attack, especially in the context of the other two. Overall, this sentence is the comment equivalent of sneering at someone at a well-to-do party, and loudly dismissing them as a damn fool. If you're the host of the party or someone notable, you may get away with this sort of behaviour, but as a guest, you will be the one led to the door.
The restatement above makes the exact same point, casts the exact same aspersions on the views of the person you're replying to, but does so in a way that is not perceived as an insult.
> That is your interpretation, and it is worded in your vocabulary. That vocabulary means something to you but may mean something else to someone else.
This is a scientific discussion, not one about vocabulary or a post-modern view of reality.
> All the bits in italics (yes, that's most of your comment) are potentially insulting to the person you're replying to.
That's the way it is in science. Science is not diplomacy. And the WSJ Dartboard Contest went on insulting the same people in the same way for years, but without lifting the veil of ignorance in which these people live, as the above exchange proves.
> By referring to "future returns", he went beyond any rational position, and he directly contradicts the position taken by the SEC on this issue.
You asked for an example of a strategy which could reliably beat the market. I offered up the example of the fund I work for. Do you really think that this is me being a salesman?
Here are "the facts":
1. I am not, nor have I ever been, employed in sales. I work in research. I have zero client contact.
2. If I did want to raise investment for the fund, I wouldn't come to an obscure discussion thread on Hacker News to do it. Apart from anything else, most people can't afford the minimum investment.
3. It's an SEC requirement to include something like "past performance does not guarantee future results" on advertisements for mutual funds (NB this isn't an advertisement, and I work for a hedge fund, not a mutual fund). Strictly speaking that's true - nothing guarantees future results. But let's be Bayesian about it - would you rather put your money in the hands of a manager who's returned -10% for the last five years, or one who's returned +10%? [0]
Finally, if you think it's a waste of time talking to me, you presumably think it's a waste of time talking to anyone who works for a hedge fund (or any other kind of fund).
What you're saying is "The opinions of those who have the most experience on this issue, who spend 60+ hours a week thinking about this, and who put their own reputation and investments on the line daily -- those opinions are the ones least worth listening to."
I appreciate your point about conflict of interest. It's true that if everyone believed in the EMH, I wouldn't have the job I do now. But to summarily discard the point of view of everyone who works in finance seems a little bizarre. It's as if I said that it's not worth talking to software developers about programming, since if no one believed it was worth buying software, they wouldn't have a job.
[0]It's a trick question. If you believe in the EMH, your prior on the expected excess return on any strategy is 0% (or slightly negative to account for trading costs) and no amount of data will change that. You should be perfectly indifferent to these two money managers. But are you? What if they had 20 years of history? What if they had 100 years? At what point do you start to revise your opinion about the EMH?
> You asked for an example of a strategy which could reliably beat the market.
I never said any such thing. I asked for scientific evidence that the EMH is false. There is no scientific evidence that the EMH is false -- none whatsoever.
A single data point is not science. Your experience is not science. Your claim about future performance is certainly not science. TO have scientific value, it must be weighed against the entire population of stockbrokers.
> What you're saying is "The opinions of those who have the most experience on this issue, who spend 60+ hours a week thinking about this, and who put their own reputation and investments on the line daily -- those opinions are the ones least worth listening to."
That is exactly right -- every word. People who are closest to this issue, who have a vested interest in a given outlook, are the least reliable sources of evidence. And this is Science 101.
Do we ask murderers their opinions on sentencing guidelines? No, we don't -- their views aren't objective. By the same token, we don't ask stockbrokers their opinions on the theoretical underpinnings of the EMH, for the same reason -- their views aren't objective.
Your position is common among people who don't understand science -- "There is a theory that the equities market cannot be played, that the average return, reflected for example in the DJIA, cannot be improved on in the long term and over all investors and funds. But my experience disproves it -- I work for a fund that is at the right-hand side of the standard distribution, therefore the EMH is false." This is laughably absurd.
No one who understand statistics or science can miss the fundamental logical error you're making. I wonder -- do you by chance live in Lake Wobegon, where all the children are above average?
> At what point do you start to revise your opinion about the EMH?
First, it's not an opinion, it's based on scientific evidence. Second, it would be easy to disprove the EMH in principle, and may have tried but failed. Are you not aware of the Wall Street Journal Dartboard Contest? It is an excellent example of a pragmatic "put up or shut up" challenge to stockbrokers such as yourself, and it shows that there is no basis for the claim that managed accounts are worth their costs, or that the EMH is false.
> But to summarily discard the point of view of everyone who works in finance seems a little bizarre.
First, the views of individuals must be discarded, for reasons that should be obvious. Second, the experience of all stockbrokers as a population is the only rational basis for objective scientific evidence. Also, if the point is to find out which outcome is true in a study of human behavior, the least reliable source of evidence are the opinions of individuals who practice what's being studied. One would instead examine the actual outcomes, not the views of those outcomes expressed by the practitioners.
Let me spell this out for you. In science, your personal experience means precisely nothing. In science the average experience of all people in your position might lead to a useful result, depending on how careful the data was collected and processed.
Case in point. J. B. Rhine of Duke University believed he had uncovered evidence of ESP -- not because of a single data point, but because of many experiments over years. His data showed a persistent, small bias in favor of the precognition idea.
After Rhine passed on, others examined his records and discovered that he had been systematically dropping certain experimental outcomes, on the ground that they were so bad that they couldn't possibly be valid. In other words, Rhine didn't understand statistics, in which, given sufficient trial runs, an occasional very bad result is likely.
The excluded results were reinstated, and Rhine's ESP claim evaporated. True story. Now, you are saying that your personal experience disproves the EMH, or deserves to be counted as evidence against the EMH. But you are wrong -- this is not how science works. If all people in y...
I don't think it does. I disagree with him, but in general one's tone and one's arguments are two different things. I think, though, that he is too dismissive of the positions he does not like and his counter-arguments are not necessarily better.
Overfitting your models? If you fail to predict an earthquake or a tornado do you blame it on overfitting your model? I don't think so. You just humbly say that you don't have good enough model, enough computing power, and what's most important not enough data.
Value investing is just buying things that you think are under-priced. It's successful if you are right and you are faster then other people to notice that, or you are actually wrong but lots of people also make same mistake but later than you. Or if you are Warren Buffet then other people think you must be right and buy the same thing making you even more rich.
As for momentum investing it's easy to just read wikipedia entry on that. It's not even established that this effect exists and even if so it's offset by additional risk undertaken by adhering to this rigid strategy.
I'm not sure how Equity premium puzzle even relates to efficient market hypothesis.
People clearly don't believe that efficient market hypothesis holds but same can be said about any recurring lottery. People keep inventing systems for picking lottery numbers, and claim that some of them work for some of them for some time. Which they do, until they don't.
When you are investing you don't play against the environment. In fact environment is favorable to you. You just compete with other players to rip off other players. It's so full of chaotic feedbacks that you can't predict anything. I mean, you can, until you can't. It's just like gambling. You can always win at fair gambling, just bet more if you loose. You'll be winning until you loose so many times in a row that you loose everything.
Blaming people who develop algorithms for purposes of gambling (also called investing) for decade of market crashes is just silly.
All markets have crashes. Faster markets have crashes more often. Markets are getting faster due to computer technology not trading algorithms.
Even if algorithms were really simple it would still lead to more frequent market crashes just because dumbest algorithm can trade faster than humans previously could.
And blaming physicists for 2007-08 worldwide financial collapse is just silly. It's not them that hid the risk in convoluted instruments and then sold them as if they were as solid as cash in the bank.
> Blaming people who develop algorithms for purposes of gambling (also called investing) for decade of market crashes is just silly.
Unfairly blaming analysts isn't the problem. The problem is assuming that anyone can predict the future of equities markets. There's no evidence to support this idea -- all successes are chance successes, and all failures are chance failures. This is why market index funds perform on average better than funds in which a manager tries to choose future winners and charge you for the privilege of being a member, an "insider".
Interesting but flawed....Scotty79: to argue that models cannot be blamed for market crashes is the same type of argument that says guns don't kill people. Both statements outside context are correct, yet...btw: another academic created the option formula already: Vincent Brontin who taught in Triest in 1910...and not the physicists who knew howcto treat formulae appropriately survived the crashes but street smart traders like Taleb did...experienced option traders never used Black-Scholes anyway because they knew it was flawed...an accident waiting to happen...
actually it didn't ...its successor Spitznagel's Universa made enormous profits in 2008 to the tune that he now manages $6 billion....also: Taleb made his money in the '87 crash...and Empirica made a lot in 2000/2001. Taleb closed it because he suffered from trader fatigue....something quite common btw
A quote: "And at least some physicists have been phenomenally successful as fund managers. Indeed, the fund that is widely viewed as the most successful hedge fund of all time, the Medallion Fund, was run for twenty years by mathematical physicist James Simons–the same James Simons who lent his name to Chern-Simons theory, which has been studied extensively by string theorists."
When are financial sector writers going to stop assuming that a chance success in finance is in any way the result of specific training or background on the part of a fund manager? One of two things is true -- either the Efficient Market Hypothesis (EMH) is true, in which case who you are doesn't matter, or the EMH is false, in which case anyone can make bazillions in the markets by simply applying the kinds of "sure-fire winning strategies" that are touted in any number of worthless investment best-sellers.
All the evidence suggests that the EMH is true, that markets are unpredictable, and all successes are chance successes. The Wall Street Journal "Dartboard Contest" supports this idea, to the chagrin of any number of stockbrokers and analysts, who would prefer it if there were any credible evidence that their services have value.
>A quote: "And at least some physicists have been phenomenally successful as fund managers. Indeed, the fund that is widely viewed as the most successful hedge fund of all time, the Medallion Fund, was run for twenty years by mathematical physicist James Simons–the same James Simons who lent his name to Chern-Simons theory, which has been studied extensively by string theorists."
P Quant math is relatively easy to grasp, so much so that an undergraduate with some statistics and calculus background can understand it quite easily.
>All the evidence suggests that the EMH is true, that markets are unpredictable, and all successes are chance successes.
Markets are unpredictable, but people (read: rational, utility-maximizing investors) aren't. Generally there is a level of herd behavior that can be modeled.
> Markets are unpredictable, but people (read: rational, utility-maximizing investors) aren't. Generally there is a level of herd behavior that can be modeled.
You're overlooking the fact that some of those modeled people are analysts who are also trying to model, and act on, the behavior of the mob. In fact, analysts constitute their own mob. :) That restores the EMH to its rightful place.
> A recursive process with chaotic feedback cycles certainly won't be predictable ...
Not so -- chaotic systems are predictable, if one understands the system. Using "chaotic feedback" as an argument isn't particularly instructive, given that chaos theory is a study of predictable systems that happen to be extraordinary sensitive to initial conditions.
> I also don't know under what model it's efficient either.
Of course such a system is efficient, if by "efficient" we mean that it doesn't violate physical laws. For example, there are any number of chaotic systems in nature, but all of them conserve energy and angular momentum (among other things). That tells us that such a system is difficult to read, but entirely obedient.
> Sprinkle in any measurement uncertainty and you no longer know the initial conditions. Hence, real world chaotic systems are unpredictable.
Which part of your answer replies to my statement? Chaotic systems are predictable. All one needs are the initial conditions. Circle the word you didn't understand and raise your hand.
Nothing is more common than an organization which exploits public ignorance of the innate predictability of complex systems.
One of the arguments which the author makes is that if the EMH is true, and returns follow an essentially random walk, somebody still has to act to make them efficient. A price doesn't reflect all known information about an asset if the people who have that information don't bid.
Companies like Simon's Renaissance Technologies employ modelling techniques first developed by physicists studying chaotic systems. They try to squeeze just enough information about the structure of the market in a given moment to place a winning short term bet.
The fact that Simon's Medallion Fund is the most successful hedge fund in history, only employs physicists and mathematicians, and has been successful over a period of 25 years, suggests rather strongly that it's not just chance and probably is related to training in other fields...
> The fact that Simon's Medallion Fund is the most successful hedge fund in history, only employs physicists and mathematicians, and has been successful over a period of 25 years, suggests rather strongly that it's not just chance and probably is related to training in other fields...
The market consists of players that lie on a standard distribution, with the market averages identifying the mean. Now ask yourself -- how many players will be above the average on a given day? And how many of those "winners" will be predisposed to acknowledge the role of chance in that outcome?
Because of chance factors, some investors will become multimillionaires by chance alone. Because of chance factors, some funds will outperform the market averages for decades running. But without a proper scientific analysis, none of this means anything apart from randomness.
I cannot tell you how many times I've heard the same claim -- "(name a fund) consistently outperforms the market norms, therefore ..."
Therefore nothing. Therefore the dice fell in an interesting bur meaningless way.
Also, a point obviously lost on you, if the fund you describe really has a winning strategy, why is it not formalized and published? There are two possible answers: (a) it is not a system, it is a series of lucky guesses, or (b) it is a system, but they won't publish it and lose its advantage. Neither answer raises this above the default assumption suggested by Occam's razor -- it's a chance correlation.
Science requires evidence, not anecdotes, and the plural of anecdote is not evidence.
The excuse you gave for Buffet's long term success was that legions of others copy his investments and they become self fulfilling prophecies. But Renaissance Technologies are very secretive - they publish virtually nothing about what they do. What are the odds of them greatly outperforming the market (and always having positive returns) for all those consecutive years? What will you if they do it again this year?
(and I dont think anyone claims they've been playing by a single strategy all this time)
> But Renaissance Technologies are very secretive - they publish virtually nothing about what they do. What are the odds of them greatly outperforming the market (and always having positive returns) for all those consecutive years?
It is always possible to pick and choose your data to prove any point you care to make. But the fund you describe is one data point in a large distribution of investors and funds defined by the market averages. Please try to evaluate this scientifically, dispassionately.
> ... What are the odds ...
The odds are excellent that, in a field of tens of thousands of funds, a few of them will do spectacularly well because of chance factors. And this is the explanation favored by Occam's razor -- the idea that the simplest explanation is likely to be the correct one.
-- a model is set up with 100,000 investors, each of whom randomly buys and sells stocks, in a market that creeps up in value over time (like the real market) along with random fluctuations (like the real market). "What are the odds" for success because of chance? Very good -- essentially certain. Of the 100,000 investors with managed portfolios, each with an initial $10,000 stake, 224 became millionaires by chance alone. The top-earning portfolio returns $4,638,235.88 by chance alone.
But at the same time, the average investor with a managed portfolio (who wasn't a buy & hold investor) came out behind -- more than 2/3 of the managed portfolios come out behind the market average. And this model didn't even assess trading fees -- a more realistic model that charges transaction fees makes active portfolios look much worse.
Imagine you're a stockbroker and you know these facts, facts that put an active portfolio in such a bad light. What do you tell your clients? That having an actively traded portfolio is a losing proposition? Or that a handful of investors came out ahead, some far ahead?
The bottom line is that a successful fund, especially one that won't reveal its methods, is the result of chance -- absent scientific evidence that something other than chance is involved.
Imagine that someone throws a fair coin and eventually produces a series of 16 heads in a row. Doesn't that prove that the thrower has psychokinetic powers? No, it means he threw the coin 2^16 (65,536) times, a sequence during which the described outcome has a 50% chance to arise by chance alone.
In the above strategy, a con man mails you a series of predictions so remarkable that you're certain he's a genius, but the strategy is so absolutely predictable (and easily explained) that it can be carried out with a computer and as printer with no human intervention.
So in answer to your question "what are the odds?", I have to say that all these outcomes are easily explained as chance occurrences, and that is by far the most likely explanation.
Chance doesn't explain it. Firstly, in the 90s I doubt there were 65,536 hedge funds around, so simple chance would not dictate one to return above average every year. Secondly, we're not talking about just a 50/50 outperform/underperform situation - the Medallion Fund spectacularly outperformed the market in many of those years and never reported losses. In years when the market crashed they were eg. 90% up. So it's not just the unlikelihood of consistently outperforming year after year, it's also the unlikelihood of outperforming at such a magnitude in any given year, that you need to consider.
About a program I wrote, that generates random numbers, and that therefore can only model chance? Yes, chance does explain it -- it is the only possible explanation for a spectacular performance in a program that simply rolls random dice.
> Firstly, in the 90s I doubt there were 65,536 hedge funds around ...
I'm waiting to hear your argument. The number of funds doesn't change the outcome, and the number you chose has no bearing on the problem.
> so simple chance would not dictate one to return above average every year.
Are you aware that we're discussing a computer program that proves that spectacular returns can be gotten using random numbers? Which part of this aren't you understanding?
> the Medallion Fund spectacularly outperformed the market in many of those years and never reported losses.
Quote: "Between January 1993 and April 2005, RenTech’s flagship Medallion fund had only 17 losses out of 148 months."
Check your facts, Mr. Scientist.
Also irrelevant. If your position had merit, science would be a matter of choosing the most extreme, unrepresentative outlier and calling that the average. But that's not science, that's superstition.
Guess how many funds are above average on a given day? 10%? 20%? Would you believe 50%? The third answer is the right one -- as you would know if you understood statistics.
In science, Occam's razor prefers the simple explanation. The simple explanation is chance. The same chance outcome that means ... wait for it ... fully 50% of funds are above average.
Medallion never had a down year, which is what I was referring to, although they have had some monthly losses. The odds on a fund having Medallion's returns in a strong-EMH environment are astronomical - so either it is a cosmic fluke or the EMH is incorrect. Occam's Razor suggests the latter.
(also I was never talking about your computer program)
Articles like that frustrate me. So much smartness and explanation, and yet it all is still only devoted to figuring out how to be smarter than the other masses of investors out there, in order to make a buck off of them.
I think hiring has shifted away from physics to signal processing, and machine learning people. But I am a signal processing guy so that maybe my bias. Also because we've moved away from complex derivatives.
Speaking as a member of APS... what do you expect from an APS magazine :)? What on earth is a "mathematician"? We have to call Edward Thorp an "American" because we don't know about his "theories" (c.f. the article)! (And how are "theories" different from "theorems"?)
Yes, that's a bit of a stretch and I say so as a physicist. But he is referred to as a "mathematical physicist" which is as close as possible to being a physicist if you are really a mathematician and your work has been used in physics.
61 comments
[ 0.28 ms ] story [ 127 ms ] threadModels are approximations. Don't confuse your models with reality.
Know what your assumptions are, and have a plan for what to do when those assumptions no longer hold.
Know why your models work, so that you know when they are likely to stop working.
Get deep domain knowledge.
Know that the market can stay irrational longer than you can stay solvent.
Experiment ruthlessly. Apply the scientific method.
Start simple and iterate.
This probably applies to more than just finance.
It doesn't matter whether it's sophisticated algorithm running on big iron or hunch running on wetware. You will fail because nobody knows the future. Trying to reliably squeeze money out of the market by better predicting its behavior is like trying to build perpetum mobile by trying more and more convoluted designs.
All the smarts of traders ... they do nothing: http://finance.fortune.cnn.com/2013/01/24/buffett-hedge-fund...
There is money that can be reliably extracted from investing though. If you can trade faster than others, you can drain some money from them. That's why algorithmic trading yields returns. Your strength comes not from predicting future better, but from acting and reacting faster.
What's embarrassing is how many intellectual resources are focused on this problem. For those not in finance, compare it to focusing talent on finding better advertising techniques.
But to say that the entire market is fully random is to claim that the efficient market hypothesis holds, which it clearly doesn't - not even in weak form. Anyone who wants to claim that the EMH is true needs to explain the equity premium puzzle, the success of value investing and the persistent outperformance of momentum investing - among many other price anomalies that I could name.
Of course it does. All evidence supports the Efficient Market Hypothesis (EMH), and note my use of the word "evidence". A contradiction would have to be a series of funds that applied a given, predefined (internally) strategy and (very important) didn't reveal its picks to the public (to avoid the announcement effect). Such an experiment hasn't been done, and those cases in which a specific strategy was applied have been universal failures.
Let's say for the sake of argument that there is a surefire winning strategy as is so often claimed in worthless investment books. If it were to be recognized and then applied, it would become the new market norm, everyone would apply it, and its differential value would evaporate overnight. That hypothetical experiment merely supports the EMH.
> Anyone who wants to claim that the EMH is true needs to explain the equity premium puzzle, the success of value investing and the persistent outperformance of momentum investing - among many other price anomalies that I could name.
Those result from a combination of the announcement effect and insider trading. Berkshire-Hathaway is successful because people expect it to be successful, and who invest in the same stocks when they're announced. It's a self-fulfilling prophecy.
The only fair, scientific test would be a strategy that is applied in secret, not in public like Berkshire-Hathaway, and in a way that doesn't tweak the market itself by way of large purchases that create their own following of fanboys. In other words, a pure computer model that is defined in advance, not tweaked enroute, that doesn't actually invest but only models hypothetical investments, and that shows its value over other systems in perpetuity. There is no such thing, and the EMH is true.
Remember the Null Hypothesis, the basis of modern scientific thinking. The Null Hypothesis assumes something without evidence is false. All evidence supports the EMH.
> One need only look to the LBO bubble and the dotcom bubble to see counter-examples of EMH.
Those are not counterexamples, and they bear no relevance to the present topic. Anyone can misbehave in spectacular ways, but those misbehaviors don't falsify the EMH, any more than an airplane crash falsifies theories of aerodynamics.
But then, what about HFT arbitrage, low P/E ratios, choosing underperforming companies to take advantage of regression to the mean, Piotroski's F-Score, Graham-style value investing, bait and switch dumping, swing trading, the very real problem of pricing goodwill and non-material resources...? If not undermining weak EMH, at least they cast doubts over the strong forms. Or they should. Do also read John Carter's Mastering the Trade for some examples on how market inefficiencies can be exploited reliably.
Supposedly (seen in Wikipedia, so I'm not that confident) market crashes are interpreted to be impossible under strong EMH (and yeah, the bubbles would definitely falsify them), but are allowed in weak EMH to be statistical events of a random process. Now, I've heard an interesting and convincing argument by Steve Keen[1] that the market is not Gaussian noise; it's a fractal process with dimension ~ 1.7 (Gaussian's dimension is 2) whose crashes obey a power law distribution, very different from what a random walk would generate. That falsifies the weak form.
> Prove this wrong -- identify a surefire winning stock picking strategy that is immune to the announcement effect and insider trading, that reliably puts its practitioners ahead of market indices.
Falsifying EMH does not require such a thing, no more than proving relativity requires observing time dilation at all velocities. This is an onerous, diabolical burden of proof.
> Anyone can misbehave in spectacular ways, but those misbehaviors don't falsify the EMH, any more than an airplane crash falsifies theories of aerodynamics.
If you had periodic epidemics of plane crashes that were more frequent than expected, sure, you should definitely suspect something's wrong.
[1] Yeah, some things he says are somewhat crackpotty, but he's definitely not to be completely dismissed beforehand. I refer to his Behavioural Finance Lectures, published in Youtube.
> Falsifying EMH does not require such a thing ...
As a matter of fact, yes, it does. The only acceptable evidence against the EMH is a performance that the EMH cannot accommodate -- a successful strategy for reading and exploiting the market in a way that performs better than market indices. And not just sometimes, but all the time, reliably.
Science requires evidence. Evidence in this case means producing results above the norms set by the market averages, and not just sometimes, but all the time. Anything else is empty philosophizing.
Performances below the norms can be explained in any number of ways, but above the norms is a different story. And by "above the norms" I don't mean 1/2 the time or 1/2 the practitioners -- that's assured by chance factors.
All this talk about the EMH being flawed, not being the whole story, falls down before the fact that no one can invent a strategy to improve on long-term market averages, and that says it all. It is the ultimate evidence in favor of the EMH.
If no one can think of a way to read the market to his advantage, better than average performance, without engaging in insider trading or reliance on the announcement effect, then the default assumption is supported.
> This is an onerous, diabolical burden of proof.
Be that as it may, it is an essential standard of proof. Nothing else matters -- no amount of philosophizing can stand in for actual results.
The expert gets up and says, "The EMH is flawed, meaningless, unscientific, wrong, etc., etc.." Someone from the audience says "All right -- prove it. Produce a strategy that beats the market norms." The expert is naturally taken aback -- "Are you serious? What about the EMH? What about the fact that every single strategy that has ever been tried, has failed in the long term?"
This is like people in a bar talking about sailing. Any number of ideas about sailing are aired over intoxicating beverages, and it's all very interesting, but since no one goes out on the water, it's completely meaningless.
Yes, true, and such an outcome ironically supports the EMH.
> It's actually more useful if the public believes in EMH and in traders' best interest to keep their edges secret.
Actually, it is much, much more advantageous for traders to pretend that there is a surefire winning strategy that only they know about. The problem with that plan is the fact that there is no such strategy.
How do I know this? If there really was a surefire winning strategy, a trader who controlled it could (and would want to) do without any public involvement, and invest his own funds in private until he became absurdly rich.
That's the obvious way to play it -- why share the proceeds, and why risk exposing the method, by dealing with gullible, ungrateful clients? The answer is that those gullible, ungrateful clients are what makes the present system work, not some secret sauce that falsifies the EMH.
If one doesn't know of a 'sure fire winning strategy' (== strong form inefficient market hypothesis ?) ( or just statistically useful may be good enough ) one should not participate in the market in most forms ( which market, participate how ). I've heard it compared to a poker game where if you don't know who the sucker is it's probably you.
The general public is misinformed about the markets and gets consistently fleeced yet keeps getting pulled in by dreams of easy money - akin to a lottery.
Markets have defined purposes (theoretical, practical, legal) and serve them accordingly. Enriching 'gullible, ungrateful clients' is not the primary purpose. EMH is a useful explanation to the clients why they got fleeced, and i don't want to disprove it.
It takes a lot of work to figure out how the lottery works and how to consistently game it. It's just a job.
If "statistically useful" means reliably above market averages year after year, then that would constitute a surefire winning strategy. There is no such thing -- if this were not so, we all would know about it by now.
A tiny advantage -- a legal one, not dependent on insider trading or any other questionable premise -- however small, could be exploited to enrich the player who used it.
But there is no such thing.
> The general public is misinformed about the markets and gets consistently fleeced yet keeps getting pulled in by dreams of easy money - akin to a lottery.
Yes, and the key to all these misleading strategies is ignorance of the EMH.
Evidence for the EMH lies in the fact that, in the long term, buy & hold investors do better than players -- in the long term, on average, if all results are tallied objectively.
> It takes a lot of work to figure out how the lottery works and how to consistently game it.
Again, there is no such thing. If there were, businesses would refuse to use equities to raise capital, knowing they would be fleeced by the players. The only reason equities are a reasonable way to capitalize business is because it's a fair approximation of an efficient market.
For the life of me, I can't figure out why people find this so hard to understand.
Just would like to add a distinction that EMH seems to hold from a retail clients perspective, but does not hold from a systematic / structural side.
Markets are structurally exploitable. For example it is (generally) illegal for an institution to front run clients' orders, however it is ok if you are just a trader and know probable (not necessarily sure fire) locations of large orders and front run them.
But "fairness" is what the EMH means. "Fairness" expresses in human terms what the mathematics tells us. An efficient market ruthlessly adjusts for inefficiencies and oversights, sort of like biological evolution, and evolution is the ultimate in ruthlessness and indifference to the fate of individuals. It's also fair -- each species has an equal chance to prevail.
Insider trading is unfair to ordinary investors who aren't criminals. The announcement effect is unfair to ordinary investors who don't have that kind of influence over the markets. If you take away insider trading and the announcement effect, the result is ... more fair. That result is closer to an optimally efficient market.
> Markets are structurally exploitable.
Only by cheating (and as your example demonstrates). The SEC knows this, and they know how corrosive distrust in the fairness of equities can be for society as a whole, so they aggressively pursue cheaters.
Absent cheating, equities markets cannot be gamed. Absent cheating, a long-term buy & hold position is the optimal strategy, By the way, and for what it's worth, Warren Buffett agrees.
I think it doesn't validate EMH if such inefficiencies disappear if exposed, if they depend on skill and systemic knowledge rather than 'cheating' - i suppose it depends on where one draws the line between cheating and skill.
Discovering and exploiting them is a matter of study and work which the public is unwilling to do, and actually actively discouraged by the status quo of this industry.
They're guided into ignorance and defering their finances to the sucky professionals who get paid for amount of assets under management rather than performance.
Warren Buffet doesn't buy and hold indiscriminately. He buys things on sale during market crashes when the panicky public gladly sell to him. For example by selling put options he gets paid just for waiting for the discounts.
And as a potentially controvertial aside, i think his size and connections within the government grant him hypocritical priviledges not available to the public (other than through owning his stock of course).
Yes, but it's well-established that the average investor comes out behind by trying to exploit such inefficiencies, and this outcome supports the idea that the EMH is true, and that a buy & hold position produces the best returns on average, overall.
The problem with Wall Street anecdotes is that successes are given undue weight in the telling, misleading people into the idea that locating and exploiting inefficiencies puts you ahead. On average, this is false -- very, very false.
> Warren Buffet doesn't buy and hold indiscriminately.
Berkshire Hathaway's long-term success is easily explained -- its position is a matter of public knowledge, so others, hoping to see the same growth, buy the same stocks (too late to benefit themselves but just in time to benefit Berkshire Hathaway). It's called the "announcement effect".
Excusing it away feels like a lazy way out, settling for mediocrity. I make a living by throwing sixes time and time again. It's not cheating, it's not easy, it takes skill and work and is doable.
It's just math. There's nothing special about those dice.
Also, you seem to defeat your own argument. The strong form EMH says that excess returns are not possible, even with insider trading - which you admit can lead to outperformance. The weak form EMH would prohibit excess returns from the announcement effect, but you admit that leads to return predictability as well. Which is it - does the EMH hold, or is the announcement effect real?
Your arguments for why momentum, value and equity investing work also don't stand up. There's nothing to be "announced" in those strategies - you could apply them in a vacuum where you didn't know anything except prices and fundamentals. Similarly no insider knowledge is required. So why do you say that they are due to the announcement effect and insider trading?
You also misunderstand the scientific method - the failure of an experiment to refute the null hypothesis does not lend credence to the null hypothesis! At most it tells you that the null is consistent with the observations. Moreover, you talk as if the Null Hypothesis is a particular statement, which it is not. There's a great XKCD comic I'd refer you to if I wasn't on my phone!
If you want such an experiment, I humbly invite you to consider the past and future returns of the fund I work for, which is 100% model-driven. Although obviously we trade based on our predictions - because at the end of the day we're in this to make money, not to write papers or win internet arguments.
Yes, and, in spite of its tremendous theoretical importance, there is no such thing.
> And it doesn't need to be laid down in advance ...
Is this a scientific conversation? Yes? Then yes, it does. Any number of bogus studies can prove their claims if one can change the study's claims as the evidence comes in.
> The strong form EMH says that excess returns are not possible, even with insider trading ...
Ah, I see -- redefine the EMH, then prove the falsity of the redefinition. The EMH says no such thing.
> The weak form EMH would prohibit excess returns from the announcement effect ...
Also false. You aren't discussing the EMH, you're making it up as you go along.
> There's nothing to be "announced" in those strategies - you could apply them in a vacuum where you didn't know anything except prices and fundamentals. Similarly no insider knowledge is required. So why do you say that they are due to the announcement effect and insider trading?
That's easy to answer -- I never said any such thing, anywhere. You're arguing against a position no one has taken.
I did say that an legitimate test needs to be conducted in private, with no actual investments -- a computer model defined in advance, that cannot be changed as the study proceeds, and that by definition cannot influence the real market being studied. There are no such successful models. The EMH appears to be true.
> If you want such an experiment, I humbly invite you to consider the past and future returns of the fund I work for ...
Ah, so you have a dog in this fight. You're not an impartial participant, striving for objectivity. That means I'm wasting my time talking to you -- you're not a scientist, you're a salesman.
What ad hominem was that? I correctly identified my correspondent as a salesman, not a scientist. It's a simple statement of fact. He thinks one data point counts as evidence, which means he isn't a scientist, and he is objectively in the business of talking people into investing in his fund -- therefore he is a salesman.
Simple straight-up facts. And it really is pointless trying to discuss these issues with someone whose livelihood depends on the myth that the market can be tweaked in any way we please.
Perhaps you're like me a decade ago and you don't even realise when you're insulting someone (I was like that). If so, I urge you to realise that just because you don't mean to be insulting doesn't mean you aren't.
The last line of your comment was obviously insulting and your comment has been downvoted by others as a result. That's an evident observation. Take it on board instead of disputing it, and if you really did not mean any insult, then think about how you could have phrased that line (and perhaps the rest of your comment) in a way that would not come across as insulting.
Learning to not involuntarily insult people is pretty useful.
You have something against salesmen? I described this person as a salesman, after he identified himself as a salesman. Which part of this aren't you getting? He has a vested interest in the position he was putting forth. That fact, the fact that his livelihood depends on the continued belief in the predictability and malleability of equities trading, is germane to the conversation. It prevents objective evaluation of evidence.
> ... in a way that would not come across as insulting.
Not every salesman is Willy Loman, and the insult you saw is solely in your mind.
He said: "I humbly invite you to consider the past and future returns of the fund I work for ..."
By referring to "future returns", he went beyond any rational position, and he directly contradicts the position taken by the SEC on this issue (brokers should never make claims about future performance).
He is a salesman. Talking to him about this issue is a waste of time. Those are the facts.
The wiser way to behave here is to realise that your interpretation may or may not be true, since you are working from a highly incomplete data set and therefore tone down the use of vocabulary that may have multiple meanings and instead focus on vocabulary that is less likely to be interpreted as an insult.
For example, you might have said:
> I notice from one of your statements that you seem to be working for a fund yourself. Do you think that this impacts your view of this problem?
Instead of:
> Ah, so you have a dog in this fight. You're not an impartial participant, striving for objectivity. That means I'm wasting my time talking to you -- you're not a scientist, you're a salesman.
All the bits in italics (yes, that's most of your comment) are potentially insulting to the person you're replying to. The first one is a mild character attack, the second one is a direct insult, and the third one is easily perceived as a direct character attack, especially in the context of the other two. Overall, this sentence is the comment equivalent of sneering at someone at a well-to-do party, and loudly dismissing them as a damn fool. If you're the host of the party or someone notable, you may get away with this sort of behaviour, but as a guest, you will be the one led to the door.
The restatement above makes the exact same point, casts the exact same aspersions on the views of the person you're replying to, but does so in a way that is not perceived as an insult.
This is a scientific discussion, not one about vocabulary or a post-modern view of reality.
> All the bits in italics (yes, that's most of your comment) are potentially insulting to the person you're replying to.
That's the way it is in science. Science is not diplomacy. And the WSJ Dartboard Contest went on insulting the same people in the same way for years, but without lifting the veil of ignorance in which these people live, as the above exchange proves.
http://online.wsj.com/article/SB1000087239639044402420457804...
Quote: "The darts [i.e random stock picks] have historically beaten the readers [i.e. stockbrokers], with 28 wins out of 47 contests."
This is a scientific discussion
No it's not, it's a fucking web argument between a couple of strangers.
not one about vocabulary
All thought is about vocabulary to a large extent. Certainly all communication is about vocabulary.
Anyway, I'm done wasting my time with this. Someday you'll be ready to get on with the rest of us humans. We'll be glad to have you then.
You asked for an example of a strategy which could reliably beat the market. I offered up the example of the fund I work for. Do you really think that this is me being a salesman?
Here are "the facts":
1. I am not, nor have I ever been, employed in sales. I work in research. I have zero client contact.
2. If I did want to raise investment for the fund, I wouldn't come to an obscure discussion thread on Hacker News to do it. Apart from anything else, most people can't afford the minimum investment.
3. It's an SEC requirement to include something like "past performance does not guarantee future results" on advertisements for mutual funds (NB this isn't an advertisement, and I work for a hedge fund, not a mutual fund). Strictly speaking that's true - nothing guarantees future results. But let's be Bayesian about it - would you rather put your money in the hands of a manager who's returned -10% for the last five years, or one who's returned +10%? [0]
Finally, if you think it's a waste of time talking to me, you presumably think it's a waste of time talking to anyone who works for a hedge fund (or any other kind of fund).
What you're saying is "The opinions of those who have the most experience on this issue, who spend 60+ hours a week thinking about this, and who put their own reputation and investments on the line daily -- those opinions are the ones least worth listening to."
I appreciate your point about conflict of interest. It's true that if everyone believed in the EMH, I wouldn't have the job I do now. But to summarily discard the point of view of everyone who works in finance seems a little bizarre. It's as if I said that it's not worth talking to software developers about programming, since if no one believed it was worth buying software, they wouldn't have a job.
[0]It's a trick question. If you believe in the EMH, your prior on the expected excess return on any strategy is 0% (or slightly negative to account for trading costs) and no amount of data will change that. You should be perfectly indifferent to these two money managers. But are you? What if they had 20 years of history? What if they had 100 years? At what point do you start to revise your opinion about the EMH?
I never said any such thing. I asked for scientific evidence that the EMH is false. There is no scientific evidence that the EMH is false -- none whatsoever.
A single data point is not science. Your experience is not science. Your claim about future performance is certainly not science. TO have scientific value, it must be weighed against the entire population of stockbrokers.
> What you're saying is "The opinions of those who have the most experience on this issue, who spend 60+ hours a week thinking about this, and who put their own reputation and investments on the line daily -- those opinions are the ones least worth listening to."
That is exactly right -- every word. People who are closest to this issue, who have a vested interest in a given outlook, are the least reliable sources of evidence. And this is Science 101.
Do we ask murderers their opinions on sentencing guidelines? No, we don't -- their views aren't objective. By the same token, we don't ask stockbrokers their opinions on the theoretical underpinnings of the EMH, for the same reason -- their views aren't objective.
Your position is common among people who don't understand science -- "There is a theory that the equities market cannot be played, that the average return, reflected for example in the DJIA, cannot be improved on in the long term and over all investors and funds. But my experience disproves it -- I work for a fund that is at the right-hand side of the standard distribution, therefore the EMH is false." This is laughably absurd.
No one who understand statistics or science can miss the fundamental logical error you're making. I wonder -- do you by chance live in Lake Wobegon, where all the children are above average?
> At what point do you start to revise your opinion about the EMH?
First, it's not an opinion, it's based on scientific evidence. Second, it would be easy to disprove the EMH in principle, and may have tried but failed. Are you not aware of the Wall Street Journal Dartboard Contest? It is an excellent example of a pragmatic "put up or shut up" challenge to stockbrokers such as yourself, and it shows that there is no basis for the claim that managed accounts are worth their costs, or that the EMH is false.
> But to summarily discard the point of view of everyone who works in finance seems a little bizarre.
First, the views of individuals must be discarded, for reasons that should be obvious. Second, the experience of all stockbrokers as a population is the only rational basis for objective scientific evidence. Also, if the point is to find out which outcome is true in a study of human behavior, the least reliable source of evidence are the opinions of individuals who practice what's being studied. One would instead examine the actual outcomes, not the views of those outcomes expressed by the practitioners.
Let me spell this out for you. In science, your personal experience means precisely nothing. In science the average experience of all people in your position might lead to a useful result, depending on how careful the data was collected and processed.
Case in point. J. B. Rhine of Duke University believed he had uncovered evidence of ESP -- not because of a single data point, but because of many experiments over years. His data showed a persistent, small bias in favor of the precognition idea.
After Rhine passed on, others examined his records and discovered that he had been systematically dropping certain experimental outcomes, on the ground that they were so bad that they couldn't possibly be valid. In other words, Rhine didn't understand statistics, in which, given sufficient trial runs, an occasional very bad result is likely.
The excluded results were reinstated, and Rhine's ESP claim evaporated. True story. Now, you are saying that your personal experience disproves the EMH, or deserves to be counted as evidence against the EMH. But you are wrong -- this is not how science works. If all people in y...
Value investing is just buying things that you think are under-priced. It's successful if you are right and you are faster then other people to notice that, or you are actually wrong but lots of people also make same mistake but later than you. Or if you are Warren Buffet then other people think you must be right and buy the same thing making you even more rich.
As for momentum investing it's easy to just read wikipedia entry on that. It's not even established that this effect exists and even if so it's offset by additional risk undertaken by adhering to this rigid strategy.
I'm not sure how Equity premium puzzle even relates to efficient market hypothesis.
People clearly don't believe that efficient market hypothesis holds but same can be said about any recurring lottery. People keep inventing systems for picking lottery numbers, and claim that some of them work for some of them for some time. Which they do, until they don't.
If efficient market hypothesis doesn't hold, how do you explain this: http://finance.fortune.cnn.com/2013/01/24/buffett-hedge-fund... ?
When you are investing you don't play against the environment. In fact environment is favorable to you. You just compete with other players to rip off other players. It's so full of chaotic feedbacks that you can't predict anything. I mean, you can, until you can't. It's just like gambling. You can always win at fair gambling, just bet more if you loose. You'll be winning until you loose so many times in a row that you loose everything.
All markets have crashes. Faster markets have crashes more often. Markets are getting faster due to computer technology not trading algorithms.
Even if algorithms were really simple it would still lead to more frequent market crashes just because dumbest algorithm can trade faster than humans previously could.
And blaming physicists for 2007-08 worldwide financial collapse is just silly. It's not them that hid the risk in convoluted instruments and then sold them as if they were as solid as cash in the bank.
Unfairly blaming analysts isn't the problem. The problem is assuming that anyone can predict the future of equities markets. There's no evidence to support this idea -- all successes are chance successes, and all failures are chance failures. This is why market index funds perform on average better than funds in which a manager tries to choose future winners and charge you for the privilege of being a member, an "insider".
When are financial sector writers going to stop assuming that a chance success in finance is in any way the result of specific training or background on the part of a fund manager? One of two things is true -- either the Efficient Market Hypothesis (EMH) is true, in which case who you are doesn't matter, or the EMH is false, in which case anyone can make bazillions in the markets by simply applying the kinds of "sure-fire winning strategies" that are touted in any number of worthless investment best-sellers.
All the evidence suggests that the EMH is true, that markets are unpredictable, and all successes are chance successes. The Wall Street Journal "Dartboard Contest" supports this idea, to the chagrin of any number of stockbrokers and analysts, who would prefer it if there were any credible evidence that their services have value.
http://whyfiles.org/037wall_st/predict.html
P Quant math is relatively easy to grasp, so much so that an undergraduate with some statistics and calculus background can understand it quite easily.
>All the evidence suggests that the EMH is true, that markets are unpredictable, and all successes are chance successes.
Markets are unpredictable, but people (read: rational, utility-maximizing investors) aren't. Generally there is a level of herd behavior that can be modeled.
You're overlooking the fact that some of those modeled people are analysts who are also trying to model, and act on, the behavior of the mob. In fact, analysts constitute their own mob. :) That restores the EMH to its rightful place.
Not so -- chaotic systems are predictable, if one understands the system. Using "chaotic feedback" as an argument isn't particularly instructive, given that chaos theory is a study of predictable systems that happen to be extraordinary sensitive to initial conditions.
> I also don't know under what model it's efficient either.
Of course such a system is efficient, if by "efficient" we mean that it doesn't violate physical laws. For example, there are any number of chaotic systems in nature, but all of them conserve energy and angular momentum (among other things). That tells us that such a system is difficult to read, but entirely obedient.
Sprinkle in any measurement uncertainty and you no longer know the initial conditions. Hence, real world chaotic systems are unpredictable.
>by "efficient" we mean that it doesn't violate physical laws.
So...an inefficient market is one in which all the participants are time travelers?
Which part of your answer replies to my statement? Chaotic systems are predictable. All one needs are the initial conditions. Circle the word you didn't understand and raise your hand.
Nothing is more common than an organization which exploits public ignorance of the innate predictability of complex systems.
Companies like Simon's Renaissance Technologies employ modelling techniques first developed by physicists studying chaotic systems. They try to squeeze just enough information about the structure of the market in a given moment to place a winning short term bet.
The fact that Simon's Medallion Fund is the most successful hedge fund in history, only employs physicists and mathematicians, and has been successful over a period of 25 years, suggests rather strongly that it's not just chance and probably is related to training in other fields...
The market consists of players that lie on a standard distribution, with the market averages identifying the mean. Now ask yourself -- how many players will be above the average on a given day? And how many of those "winners" will be predisposed to acknowledge the role of chance in that outcome?
Because of chance factors, some investors will become multimillionaires by chance alone. Because of chance factors, some funds will outperform the market averages for decades running. But without a proper scientific analysis, none of this means anything apart from randomness.
I cannot tell you how many times I've heard the same claim -- "(name a fund) consistently outperforms the market norms, therefore ..."
Therefore nothing. Therefore the dice fell in an interesting bur meaningless way.
Also, a point obviously lost on you, if the fund you describe really has a winning strategy, why is it not formalized and published? There are two possible answers: (a) it is not a system, it is a series of lucky guesses, or (b) it is a system, but they won't publish it and lose its advantage. Neither answer raises this above the default assumption suggested by Occam's razor -- it's a chance correlation.
Science requires evidence, not anecdotes, and the plural of anecdote is not evidence.
(and I dont think anyone claims they've been playing by a single strategy all this time)
It is always possible to pick and choose your data to prove any point you care to make. But the fund you describe is one data point in a large distribution of investors and funds defined by the market averages. Please try to evaluate this scientifically, dispassionately.
> ... What are the odds ...
The odds are excellent that, in a field of tens of thousands of funds, a few of them will do spectacularly well because of chance factors. And this is the explanation favored by Occam's razor -- the idea that the simplest explanation is likely to be the correct one.
In a computer analysis described here --
http://www.arachnoid.com/randomness/#Investment_Genius
-- a model is set up with 100,000 investors, each of whom randomly buys and sells stocks, in a market that creeps up in value over time (like the real market) along with random fluctuations (like the real market). "What are the odds" for success because of chance? Very good -- essentially certain. Of the 100,000 investors with managed portfolios, each with an initial $10,000 stake, 224 became millionaires by chance alone. The top-earning portfolio returns $4,638,235.88 by chance alone.
But at the same time, the average investor with a managed portfolio (who wasn't a buy & hold investor) came out behind -- more than 2/3 of the managed portfolios come out behind the market average. And this model didn't even assess trading fees -- a more realistic model that charges transaction fees makes active portfolios look much worse.
Imagine you're a stockbroker and you know these facts, facts that put an active portfolio in such a bad light. What do you tell your clients? That having an actively traded portfolio is a losing proposition? Or that a handful of investors came out ahead, some far ahead?
The bottom line is that a successful fund, especially one that won't reveal its methods, is the result of chance -- absent scientific evidence that something other than chance is involved.
Imagine that someone throws a fair coin and eventually produces a series of 16 heads in a row. Doesn't that prove that the thrower has psychokinetic powers? No, it means he threw the coin 2^16 (65,536) times, a sequence during which the described outcome has a 50% chance to arise by chance alone.
Also read this:
http://www.arachnoid.com/randomness/#Miracle_Man
In the above strategy, a con man mails you a series of predictions so remarkable that you're certain he's a genius, but the strategy is so absolutely predictable (and easily explained) that it can be carried out with a computer and as printer with no human intervention.
So in answer to your question "what are the odds?", I have to say that all these outcomes are easily explained as chance occurrences, and that is by far the most likely explanation.
About a program I wrote, that generates random numbers, and that therefore can only model chance? Yes, chance does explain it -- it is the only possible explanation for a spectacular performance in a program that simply rolls random dice.
> Firstly, in the 90s I doubt there were 65,536 hedge funds around ...
I'm waiting to hear your argument. The number of funds doesn't change the outcome, and the number you chose has no bearing on the problem.
> so simple chance would not dictate one to return above average every year.
Are you aware that we're discussing a computer program that proves that spectacular returns can be gotten using random numbers? Which part of this aren't you understanding?
> the Medallion Fund spectacularly outperformed the market in many of those years and never reported losses.
False. http://www.hedgefundletters.com/category/renaissance-technol...
Quote: "Between January 1993 and April 2005, RenTech’s flagship Medallion fund had only 17 losses out of 148 months."
Check your facts, Mr. Scientist.
Also irrelevant. If your position had merit, science would be a matter of choosing the most extreme, unrepresentative outlier and calling that the average. But that's not science, that's superstition.
Guess how many funds are above average on a given day? 10%? 20%? Would you believe 50%? The third answer is the right one -- as you would know if you understood statistics.
In science, Occam's razor prefers the simple explanation. The simple explanation is chance. The same chance outcome that means ... wait for it ... fully 50% of funds are above average.
(also I was never talking about your computer program)
EDIT: You beat me to the comment.