Is the author suggesting that we should think twice about running complicated machine learning models on data without understanding the data first, and for which financial econometricians already have a vast set of tools to analyze it in an intuitive and parsimonious way? Rubbish!
Pretty banal article. TLDR: A very complicated model will fit simple data, as will a simple model. Modern ML/AI has nothing to do with 1D timeseries forecasting.
There have been thousands of papers and many books written on the subject of using machine learning for time series forecasting and an unknown amount of money traded/invested with it. Most of them cast trading as a time series forecasting problem where performance is measured by mean square error. They usually then assume that would lead to profitable trading.
You don't get to define what "modern machine learning" is any more than I do.
The application of complex models to higher-dimensional domains have problems of their own. Both the successes and failures are worth writing and thinking about.
The post is an important reminder about simplicity and complexity in modeling. That's why I submitted it.
I don't understand why I'm being downvoted for this opinion -- the topic of overfitting gets beaten to death by the 3rd week of every intro AI/statistics/ML class out there. I didn't find it very worthy of the frontpage (or a blog post).
Yes, in a toy space ML replicates simplistic measures, there's only so much information to be had in such a space to start with. The power of these techniques is when we apply them to high-complexity or even unbounded-complexity areas.
And that you apply them to domains where the knowledge of the domain is basic -at best. It enables people who know one subject well -machine learning- to do speech recognition and weather forecasting well.
Plus this article has a number of problems. It compares machine learning to MA(3). That it works pretty well totally ignores the selection of MA(3) vs MA(5). MA(100) is "common" indicator often talked up (why I do not know, except perhaps it protects you from really big errors, but other than that it's not very helpful).
If you actually try to use the prediction of MA(3) you will find it is often very hard to use, because it's always just too late. I wonder if the machine learning version had the same problem, or not.
> If you actually try to use the prediction of MA(3) you will find it is often very hard to use, because it's always just too late. I wonder if the machine learning version had the same problem, or not.
Judging from the graph in the article, it looks like it doesn't. The model didn't reinvent the moving average, it invented it without the inherent delay. Though it does seem to diverge from real price over time.
>If you actually try to use the prediction of MA(3) you will find it is often very hard to use, because it's always just too late. I wonder if the machine learning version had the same problem, or not.
Why does the sensitivity matter when we know that the accuracy was so similar? The point on sensitivity does make me wonder how an exponential moving average would've performed though.
Because there are "chaotic cases" in statistics. Cases where there are so many possible patterns that stating which pattern the data follows doesn't have any less information than the data itself (or only has a little less information than the data itself). Financial market data is quite notorious for being one such case (if not there wouldn't be any poor mathematicians).
So MA(3) is not a general indicator. It's not universally useful, and it isn't the only one this article considers. It was chosen from a great many possible indicators (go to tradingview.com, find a graph, expand the indicators tab, and look at the list. And keep in mind that ALL MA's are just one entry). So there's tens of thousands of indicators that MA(3) was chosen from. So the predictive value of MA(3) as an indicator in the general case is really only 1/10000th of what this person claims it to be (actually infinitely less, but let's assume you use, say, MA(200) as the limit of what you're willing to consider, which makes it a finite number).
Predictive information is this: let's say I tell you I know what the outcome is of a football match. And at the end of the match, I pull the outcome out of a stack of papers. The size of that stack of papers is inversely related to how informative my prediction was. If I just put down one paper before the match, that's great. If I put down 1000. Not so great. For this guy, the stack of papers was pretty thick.
So the article is saying "I can, out of a great many indicators, find one that performs similar to this machine learning algorithm". Unless the article also gives a clear and definitive reasoning for why THAT indicator was chosen for that dataset, it doesn't contain much information at all.
You're presenting this from the perspective of the author's selection of MA(3), but I think what you're really getting at is yes, MA(3) happened to be comparable to the ML model in this case, but the beauty of the ML model was that it knew to nearly replicate MA(3) out of a great many common indicators. Is that essentially what you're saying? If so, I'm curious how things like longer period moving averages or an exponential moving average perform relative to both MA(3) and the ML model in this case.
"Is advanced machine learning a very complicated way of reproducing the results of trivial forecasting methods? In many cases it appears this is the case."
If by "many cases" you mean "the one example I looked at which was carried out by an undergrad" than sure. That sort of phrasing really gives away the author's bias.
Why are so many people reading this as an attack on ML? The author is simply reminding people that ML isn't a universal remedy for uncertainty. "Stop writing ML models solely for the sake of writing ML models" is a very different message than "ML didn't improve upon MA(3) in this case so it's universally worthless".
I understand that the general sentiment for this article seems to be dismissive but I want to share this awesome University of Vermont lecture series/course. It's related to forecasting mathematics and includes an introduction to fractal geometry which I haven't seen much of elsewhere.
18 comments
[ 3.0 ms ] story [ 51.7 ms ] threadWhat the heck is this doing on the front page?
You don't get to define what "modern machine learning" is any more than I do.
The application of complex models to higher-dimensional domains have problems of their own. Both the successes and failures are worth writing and thinking about.
The post is an important reminder about simplicity and complexity in modeling. That's why I submitted it.
Plus this article has a number of problems. It compares machine learning to MA(3). That it works pretty well totally ignores the selection of MA(3) vs MA(5). MA(100) is "common" indicator often talked up (why I do not know, except perhaps it protects you from really big errors, but other than that it's not very helpful).
If you actually try to use the prediction of MA(3) you will find it is often very hard to use, because it's always just too late. I wonder if the machine learning version had the same problem, or not.
Judging from the graph in the article, it looks like it doesn't. The model didn't reinvent the moving average, it invented it without the inherent delay. Though it does seem to diverge from real price over time.
Why does the sensitivity matter when we know that the accuracy was so similar? The point on sensitivity does make me wonder how an exponential moving average would've performed though.
So MA(3) is not a general indicator. It's not universally useful, and it isn't the only one this article considers. It was chosen from a great many possible indicators (go to tradingview.com, find a graph, expand the indicators tab, and look at the list. And keep in mind that ALL MA's are just one entry). So there's tens of thousands of indicators that MA(3) was chosen from. So the predictive value of MA(3) as an indicator in the general case is really only 1/10000th of what this person claims it to be (actually infinitely less, but let's assume you use, say, MA(200) as the limit of what you're willing to consider, which makes it a finite number).
Predictive information is this: let's say I tell you I know what the outcome is of a football match. And at the end of the match, I pull the outcome out of a stack of papers. The size of that stack of papers is inversely related to how informative my prediction was. If I just put down one paper before the match, that's great. If I put down 1000. Not so great. For this guy, the stack of papers was pretty thick.
So the article is saying "I can, out of a great many indicators, find one that performs similar to this machine learning algorithm". Unless the article also gives a clear and definitive reasoning for why THAT indicator was chosen for that dataset, it doesn't contain much information at all.
If by "many cases" you mean "the one example I looked at which was carried out by an undergrad" than sure. That sort of phrasing really gives away the author's bias.
https://youtu.be/Lr3qrB6dPCM