Math is just one of the ways to think about problem. But you need not have a math degree to analyze data. People do it all the time even without math.
Data analysis needs understanding casualities and correlations, and ability to use some handy tools, like chi-squared. The more tools you can use, the better analyst you are.
I know some psycology scientists who have no math background at all, but who can setup an experiment and do statistical analisys on data gathered. They have a pretty good understanding on what they do while its not strictly math understanding. Math is a tool and as with any other tool you need not to understand how a tool works, you need to know how to use that tool.
In recent times you even need not know nothing about calculations behind different statistical tests, because there are computers and software that are happy to calculate anything for you.
Data analysis without knowing math sounds like a great way to make a bunch of analysis mistakes. I know math decently and data analysis is still hard. The whole field of causality is becoming more rigorous - see the work by Judea Pearl, for example.
Yes, of course, math is needed, when you try to do something more complex than just separate two hypothesis with a ready-made method. But to start with data analisis you need no math. It is about different levels of qualifications.
The repication crisis is rooted not in math but in between math and reality. Math do not work with reality, math lives in its own ideal universe. To use math for data analisis we need reason about how much data we need to separate hypothesis, we need choose acceptable p-value, we need to choose math methods. There are guides how to choose method for data, that guides needs no math but understanding what experiment is. There are 'experimental psychology', and a bunch of good books on this topic, covering all concepts that one needs to speak about choosing math methods. As for p-value and sample sizes -- there are agreement that experiment needs ~30 people and p<0.05.
If we add something like ability to use R or SPSS, then we get scientist skilled enough to setup good experiment.
The replication crisis is not error of individuals, it is system error. Psychology is much more complex than, for example, quantum physics, there are much more causal links in psychology and no one know even how to speak about mind, for example: is it possible to differentiate perception from memory or from thinking? Perception cannot work without memory, and there are no way to separate them as phenomena. Psychology is much more complex than psysics, and at the same time for physicist is is normal to have p<0.001 or sample size of 10k data points, while psychology is bound to p<0.05 (it is probability to get false positive) and sample size of 30. This is itself explains while physics have no replication crisis while psychology have one.
In order to judge whether your math matches reality, you need to understand the math in the first place though. If you don't, then you're bound to repeatedly make mistakes.
By doing good statistics. No-one cares whether a model's application is well founded. What matters is if it's predictions hold up statistically. This means lots of validation runs and defense against overfitting. That can all be done while viewing the model as a black-box.
Model knowledge certainly helps, but it isn't neccesary.
The relationship between math and statistics is an interesting one. In my experience, "Statistics is a form of Math" is true when the math folks want it to be, and is a totally separate thing that doesn't belong in the club at all when they don't.
"Doing good statistics" implies doing mathematics. In my opinion the meme that statistics is not math is perpetuated by people who have experience with statistics and not much exposure to other areas of mathematics. It might not "look" or "feel" like a lot of other undergraduate math but that doesn't mean it isn't.
Just because you're not attacking problems formalized with the underlying probability theory doesn't mean you're not practicing mathematics when you practice statistics. It's not a separate discipline.
Sure, that's the way you'd think you have to do it if your resources about this primarily come from HN. But there's whole different bag of tricks from the purer side of math which those who've decided to follow the data analysis path long-term use very fruitfully.
It really depends upon the organization — both the specific role of the data scientist and the way in which machine learning is applied.
In some organizations, "data scientist" is the modern name for a business analyst and the responsibilities largely involve basic data sanitization and analysis, streamlining processes that were manual and tedious in previous eras. In other organizations, people with advanced mathematics skills find ways to apply mathematics and machine learning to gain a marginal advantage over competitors or to deliver something novel to the market. This latter case does indeed require a deep understanding of mathematics.
Using off-the-shelf implementations of ML algos might seem like it obviates the math skills, but problem definition, algorithm selection and feature engineering are difficult for laymen to do well, and can insidiously poison the validity or efficacy of the results if done poorly.
Good ML people I know or have observed through interviews are good at a bunch of things. The ones that come to mind are: Linear Algebra, Statistics, Python, ETL operations with databases, Application Performance Analysis and testing, and the crucially overlooked and slightly more fuzzy what I call 'problem space analysis' (ie, looking at the problem space, whether it's a game or a process, and discovering which human learned activities can be effectively replaced or reimagined by training a model of some kind).
As a mathematician, people routinely tell me that mathematics isn't applicable in a particular situation. Upon deeper investigation, I usually find that they simply have a very limited understanding of what math is. Most people associate it with numbers and equations -- but those are just a few tools that are taught early on. Actual math is about analysis and some of these tools MAY be used when preforming it. Even some of the deeper topics, such as calculus and algebra are simply tools in a greater framework of understanding.
Even if you're using very little of what you think of as "math" to interact with your machine learning framework, to dismiss math as not being necessary is ignoring all of the underlying mathematics that went into creating it. You may not need calculus, or the like, but the math is there -- even if you don't recognize it as such.
"Even if you're using very little of what you think of as "math" to interact with your machine learning framework, to dismiss math as not being necessary is ignoring all of the underlying mathematics that went into creating it."
This seems like a straw man to me. The article never claimed that math is not necessary for the development of machine learning frameworks. In fact, it explicitly states the opposite:
"First of all, math is particularly important if you’re doing machine learning research in an academic setting."
"In particular, there are people at companies like Google and Facebook who are pushing the boundaries of machine learning – people working on bleeding edge tools. These people almost certainly employ calculus, linear algebra, and more advanced math routinely in their work."
Perhaps adding that last part without acknowledging these claims from the article wasn't fair to the author. I meant it more of a rebuttal to the subject line. I probably should have ended with something along the lines of "data analysis is a different type of math" rather than trying to promote the idea that "there's still math in machine learning".
Agreed. Most people don't see how mathematics is applicable because they think of it as a framework for calculation, when mathematics is really much more about proofs. In addition, most people never learn more than calculus (if that), so they have no concept of what real world problems can be elegantly reduced to mathematical restatements.
It's like the difference between calculus and analysis.
The frameworks can abstract away much of the math. You won't have to actually take the derivatives yourself, but understanding how they work actually matters to network design. It's much harder to get meaningful insights out of papers for designing better networks if you can't read the math in them. And Neural Networks is an area where you will read research papers as part of your job in order to do it well, even in industry. When is RELU better than sigmoid? When should you use a bottleneck layer? When do you want fewer larger layers or more smaller layers? When is it appropriate to add some manual features in your problem? Or even when you get an error message back from your framework when is the error in your network code an when is it in your data design? It's very hard to develop good answers to the questions that come up if you don't have the math to read papers, don't have the math to double check expressions in the error code, or have problems going back and forth between expressions with Matrix and vectors and expressions in summation notation. There are a lot of people who can build a neural network when the architecture is done for them and nothing errors (using no math or stats), but training someone to make decisions on the architecture and handle the errors IS the job.
Yeah, this is how statistics is taught in a lot of psych departments. A bunch of tests and rules of thumb about when to use which equations, together with some training on a piece of software.
IMO the "pragmatic rules-of-thumb" approach to teaching/understanding statistics is probably the genesis point of the reproducability crisis. This mindset toward stats has probably done more damage to Psychology than anything else in the history of the field.
>IMO the "pragmatic rules-of-thumb" approach to teaching/understanding statistics is probably the genesis point of the reproducability crisis. This mindset toward stats has probably done more damage to Psychology than anything else in the history of the field.
Sure, but then we have another problem: if you need to know measure theory to run an experiment, nobody will run experiments. And lots of mathematical statistics just goes ahead and uses the measure theory.
The longer I'm around, the more I'm convinced of this line of thought and the fundamental power, beauty and truth of math. We need more early educators of math with your mindset.
The basic point of the article is:
- if you don't know calculus, differential equations, etc. you can still do good work in this field
- if you do know those types of math, but you don't have experience with data analysis, you CAN'T do good work in this field (until you learn these skills).
Having done both machine learning and data analysis more generally, these points are correct. The article is explicitly aimed at the question of what is needed to do an entry level job in this kind of setting.
i think the tone of the article is good, however, because it's definitely possible to start playing with pylearn, make some SVMs, and have a lot of fun without worrying too much about L^p spaces are in the first place. later on, it could be possible to interpolate more mathematical ideas for a smoother understanding of how to measure how much preprocessing is flattening out their data sets.
(in other words, sssh this is another opportunity to trick people into doing math without immediately recognizing it as such! :-)
Doing machine learning without math is like programming without knowing about unit testing or data structures. Sure, you can get by. Good luck to anyone who has to live with what you've made.
But unit testing and data structures naturally follow as you progress as a developer, even if your early work is poor.
Which supports the headline. The math will naturally follow as you gain experience with ML, but not having good data will make even just starting your journey difficult.
In an academic/learning environment I agree. But, for professional purposes I wouldn't hire even an entry level person who didn't have at least enough of a basis to know when those concepts apply. I also wouldn't hire a data scientist who can't at least tell me the high level workings of an algorithm they might choose. I interview for both types of positions and know what it's like to have to compensate for an amateur.
More like implementing a password hashing scheme using a library and not understanding the maths behind bcrypt. Or mocking up a proof-of-concept in Python with no idea of how to speed it up 10x rewriting it in low-level C.
Lots of folks like to slag these articles and talk about how "If it's not real Maths, it's crap" (http://www.dailymotion.com/video/xgzfxs), but if you've worked in larger orgs, you recognize that there is a need for more than just advanced math. There really are myriad needs, and the failure of AI/ML (yes, I'm combining them for simplicity here) in an org is usually the lack of understanding these needs. Here are some I've seen:
1) What is the problem that AI/ML is being applied to? What is it optimizing, deciding, predicting, forecasting, categorizing? How will this decision be used in a process or flow? This requires a business analytic approach, understanding available data, _what it means_, how it's generated, and how the business might evaluate the impact of the ML/AI. These folks need to interact with the business folks as well, so some communication skills are helpful... though that's true for every role these days.
2) How will said decision be implemented both in tech build, test/QA/FUT/UAT/etc, and prod? This technical architecture and approach is data engineering, but some folks in the data science world are amazing at this. BTW, a model that works in dev may not scale in prod. The fact that so many folks keep "re-discovering" this is scary to me. There is a whole class of amazing folks who can re-implement models to scale them, and if you know them, reward them well.
3) How will models/algos/systems be built? This workflow is often pretty sloppy, just a bunch of jupyter notebooks or a tonload scripts (but they're in Git, so it's ok), and so replication and scaling becomes painful... esp. in regulated industries. Again, data engineering approach, but needs a more nuanced understanding of the vagaries of ML/AI. I find that solving business problems may not always fit a traditional software workflow (call it "agile" all you want, it doesn't always fit) but ymmv. So, you may need to create new workflows for your org's needs, and this tooling may not be off the shelf, but like automated testing, this tech debt will need to be paid sooner or later.
4) What ML/AI approach should I use, esp. if I'm using pre-existing approaches? This becomes more of the data science analytic approach, mixing the understanding of how ML/AI works with the data landscape (how was my internal data generated? What does it mean? Is it stable? What's available at score time, and what's it's latency?) and how to build various working predictive models. Note that this is traditionally where data scientists/model builders/analysts spend their time, from data cleansing to preliminary analysis to generating/training various models to crying when none of them predict well to stumbling onto a fascinating and amazing combination of models and approaches at 3am. Yes, math is defn helpful here, but _understanding_ the underlying math is often helpful enough, vs. _mastery_. A good understanding of the levers affecting each model/algo/DL architecture/etc. and how to diagnose them can get you pretty far, though you may violate assumptions or overfit if you aren't careful.
5) Real Algo design: Using all that math that underlie the models to not just diagnosing a pre-designed package but making your own optimizer, or your minimizers, or your own way of computing the Hessian, or a new weighting approaches, or whatever new approach ...
Not all the good data scientists I used to work with/for are mathematicians but all of them have a deep understanding of mathematics. Knowing math upfront is not a must but if you are not willing to learn it along the way than you shouldn't choose the path of ML.
Great. Thanks for the recommendation.
What I mean by mathematical background is at or above undergraduate level (so definitely covers calculus, linear algebra, intermediate statistics). A background that can read Elements of Statistical Learning (ESL) comfortably.
What I found is that many "data science" books cover how to use R or Pandas at a very introductory level. Books like ESL focus on core theories (which is great) but do not focus on how to tackle a tough real-world data.
I suppose much of data insight come from experience, but I was wondering whether there are sources to help me jump start.
The thing I struggle with is recognising what the problems in front of me are. It's usually bloody obvious, no problem, but then the work is quick and straightforward. The things that fill my calendar are the ones where we have problems which cost massive money which no one has found a treatment for.
Sometime it suddenly becomes apparent and we suddenly see how to do it and everyone feels pretty sheepish!
I'm mixed on this article. While I strongly agree that data analysis trumps math for applying machine learning, I think there's a middle ground the article is missing.
Like programming, machine learning has its 10xers, and I've worked with several. There's one thing they all have in common (beyond experience): they can think like the model. They've read a bunch of ML papers, get the intuition behind them, and envision from start to finish how data can be captured by various models. Beyond a small amount of hyperparameter optimization they often build a great model on the first or second take, and can put out production models in days (which would normally take novices weeks or even months). This requires math knowledge, because ML models are built on top of math. Even in higher level ML packages that do a lot of the work for you, novices get stuck in dead ends where their models don't work and they don't know why. The 10xers drive right through it because they know why their model isn't working.
When I interview for my team, I usually have non-junior candidates describe a ML algorithm they would use for a specific problem, then ask why certain things could go wrong - e.g. if they say they would use a logistic regression model to predict an action, I ask why the model may be returning the same score for every test case. A good candidate would understand that this means the coefficients are getting pushed to zero, and would list off reasons why this could be occurring (too much regularization, underpowered data, constant target variable, etc). You learn these things from experience, but you understand them because you know the math behind the models.
I have to largely agree with the sentiment of this article.
A lot of the materials etc I have seen for learning ML approach it from a very academic perspective complete with all the algebra, equations and proofs of how a neural network works.
This is totally 100% fine, but I am of the view that this is simply not necessary for those starting out in ML who just want to get stuff done.
When we introduce people (including kids/young adults!) to programming, we dont first force them to sit through a semester of learning the in-depth details and nitty-gritty of how modern compilers work before we even begin to let them write "hello world". Yet the ML courses I've seen all seem to start with the usual mathematical hazing of algebra, proofs, and general mental-masturbation around how brainy and rather complicated it all is.
There is, I think, a fundamental disconnect between people who want to learn how ML works and people wanting to learn how to use ML. It feels like most of the MOOCs and courses are more academically-focused and concentrate on teaching how ML works and are as such fairly-disconnected from what I feel like is probably a majority of people just wanting to know how to use ML in their day-to-day activities.
Sure, the theory is important, and yes you may not become an expert in ML if you don't study the theory, but I find it hard to accept the view that you need to know the proofs for why ML works before you can be a productive ML developer/user.
I would imagine that most beginners and non-advanced-users/experts these days are using a library like TensorFlow or Keras to do their ML work. Yet if you look at TensorFlow or Keras, then I think you'll find that pretty much all of the maths is hidden and you just need to know the "building blocks" and the higher-level theories of ML - the layers, the architectures, the data prep, the interpretation etc etc etc.
Just like when I code some Java or Go I can use my "building block" knowledge of programming to write well-designed, high-quality, maintainable, testable, high-performance code without having to worry about the theory of what the runtime is doing. Of course there are scenarios where it might help to know the specific maths for how Go's or Java's garbage collector or whatever works and also be able to do a mathematical proof on a whiteboard about it, but I'd wager that situation does not come up very often! :-) It is certainly not a requirement to write decent code, just like I dont think it is a requirement to use TensorFlow.
Again, to be clear, yes you might not become an expert or even an "advanced" user of ML without mastering all of the theory, and yes you might not be able to create truly-innovative new models and beat DeepMind at their own game (pun intended). Of course. But then, not every programmer is destined to become the next Jeff Dean or John Carmack either.
The fast.ai course is good in that it largely skips the whole maths initiation ceremony and just goes into the building-blocks of ML. Kudos to them.
Yeah, I'm going to politely disagree that one can do multidimensional statistics and understand what one is doing without rudimentary understanding of linear algebra and probability and basic concepts of calculus (derivatives, integrals). The branch of science specialized in analyzing data is known as "statistics": it's a mature field that's been around for some time, and the things a working statistician should know are quite well established.
Similar attitudes in the past have lead into misconceptions like if you are only applying statistics in your research, you don't really have to understand what p-value is, just run the magical tests and report if it's "significant" or not.
The importance of math for machine learning didn't fully sink-in for me until I developed a deeper understanding of the back-propogation algorithm that makes training deep neural networks computationally feasible. If it weren't for the introduction of a continuous activation function combined with the nice properties of the multivariate chain rule, we probably wouldn't be having this discussion at all. Another example that comes to mind is kernel methods used for Support Vector Machines.
No matter where the inspiration for a model comes from, the final formulation is always mathematical, and I think that without an appreciation for the mathematics, it's hard to get a true feeling of a model's effectiveness and limitations.
I am curious why there is so much talk about deep learning and math. In my country calculus was part of high school curriculum as well as a hint of diff eqs. DL theory is very undemanding in terms of math (cf quantum algorithms) and if you know basic differentiation its easy to have a very good grasp of the theory. I find it intellectually lazy to say than one should skip the math. I mean, learning programming is an equally difficult, if not more difficult task. Why is discussion so often about the math ?
Same here. Even to understand the theory, the only maths tools you really need are differentiation of composite functions (derivatives of f(g(x)), taught in high school ), what a gradient is (taught in 5 minutes if not taught in high school), basic matrix operations (and I mean really basic ones like transposition, nothing fancy like finding kernels) and then you can undertand stochastic gradient descent.
The hardest may be thinking in terms of higher dimension tensors (matrices that have 3 or more dimensions) but that is not even mathematics, just an intellectual effort.
The most important thing is not mentioned by the author or by the deep maths thinking he critiques. It is understanding when and how to apply a model, how to evaluate your results for robustness, how to interpret what it's saying. These things can be taught/learned by solving problems along with a master, even if that master is represented by a really good book, without a deep understanding of the math.
In short, to _apply_ machine learning tools, you need a good understanding of _applied_ machine learning, which is not a small subject area, but is very approachable for most.
57 comments
[ 2.9 ms ] story [ 34.7 ms ] threadData analysis needs understanding casualities and correlations, and ability to use some handy tools, like chi-squared. The more tools you can use, the better analyst you are.
I know some psycology scientists who have no math background at all, but who can setup an experiment and do statistical analisys on data gathered. They have a pretty good understanding on what they do while its not strictly math understanding. Math is a tool and as with any other tool you need not to understand how a tool works, you need to know how to use that tool.
In recent times you even need not know nothing about calculations behind different statistical tests, because there are computers and software that are happy to calculate anything for you.
If we add something like ability to use R or SPSS, then we get scientist skilled enough to setup good experiment.
The replication crisis is not error of individuals, it is system error. Psychology is much more complex than, for example, quantum physics, there are much more causal links in psychology and no one know even how to speak about mind, for example: is it possible to differentiate perception from memory or from thinking? Perception cannot work without memory, and there are no way to separate them as phenomena. Psychology is much more complex than psysics, and at the same time for physicist is is normal to have p<0.001 or sample size of 10k data points, while psychology is bound to p<0.05 (it is probability to get false positive) and sample size of 30. This is itself explains while physics have no replication crisis while psychology have one.
Model knowledge certainly helps, but it isn't neccesary.
You can certainly practice statistics in a way that mathematicians would take issue with, but it's intrinsically a branch of math.
Just because you're not attacking problems formalized with the underlying probability theory doesn't mean you're not practicing mathematics when you practice statistics. It's not a separate discipline.
Knowing how to manipulate tables is relational algebra, even if you never learn the formalisms.
Representing the parts of a problem as symbols is just algebra.
In some organizations, "data scientist" is the modern name for a business analyst and the responsibilities largely involve basic data sanitization and analysis, streamlining processes that were manual and tedious in previous eras. In other organizations, people with advanced mathematics skills find ways to apply mathematics and machine learning to gain a marginal advantage over competitors or to deliver something novel to the market. This latter case does indeed require a deep understanding of mathematics.
Using off-the-shelf implementations of ML algos might seem like it obviates the math skills, but problem definition, algorithm selection and feature engineering are difficult for laymen to do well, and can insidiously poison the validity or efficacy of the results if done poorly.
Even if you're using very little of what you think of as "math" to interact with your machine learning framework, to dismiss math as not being necessary is ignoring all of the underlying mathematics that went into creating it. You may not need calculus, or the like, but the math is there -- even if you don't recognize it as such.
This seems like a straw man to me. The article never claimed that math is not necessary for the development of machine learning frameworks. In fact, it explicitly states the opposite:
"First of all, math is particularly important if you’re doing machine learning research in an academic setting."
"In particular, there are people at companies like Google and Facebook who are pushing the boundaries of machine learning – people working on bleeding edge tools. These people almost certainly employ calculus, linear algebra, and more advanced math routinely in their work."
It's like the difference between calculus and analysis.
IMO the "pragmatic rules-of-thumb" approach to teaching/understanding statistics is probably the genesis point of the reproducability crisis. This mindset toward stats has probably done more damage to Psychology than anything else in the history of the field.
So, proceed with caution.
Sure, but then we have another problem: if you need to know measure theory to run an experiment, nobody will run experiments. And lots of mathematical statistics just goes ahead and uses the measure theory.
Having done both machine learning and data analysis more generally, these points are correct. The article is explicitly aimed at the question of what is needed to do an entry level job in this kind of setting.
i think the tone of the article is good, however, because it's definitely possible to start playing with pylearn, make some SVMs, and have a lot of fun without worrying too much about L^p spaces are in the first place. later on, it could be possible to interpolate more mathematical ideas for a smoother understanding of how to measure how much preprocessing is flattening out their data sets.
(in other words, sssh this is another opportunity to trick people into doing math without immediately recognizing it as such! :-)
Which supports the headline. The math will naturally follow as you gain experience with ML, but not having good data will make even just starting your journey difficult.
That is where everyone starts though. With time, they will become the professionals that you want to hire.
Lots of folks like to slag these articles and talk about how "If it's not real Maths, it's crap" (http://www.dailymotion.com/video/xgzfxs), but if you've worked in larger orgs, you recognize that there is a need for more than just advanced math. There really are myriad needs, and the failure of AI/ML (yes, I'm combining them for simplicity here) in an org is usually the lack of understanding these needs. Here are some I've seen:
1) What is the problem that AI/ML is being applied to? What is it optimizing, deciding, predicting, forecasting, categorizing? How will this decision be used in a process or flow? This requires a business analytic approach, understanding available data, _what it means_, how it's generated, and how the business might evaluate the impact of the ML/AI. These folks need to interact with the business folks as well, so some communication skills are helpful... though that's true for every role these days.
2) How will said decision be implemented both in tech build, test/QA/FUT/UAT/etc, and prod? This technical architecture and approach is data engineering, but some folks in the data science world are amazing at this. BTW, a model that works in dev may not scale in prod. The fact that so many folks keep "re-discovering" this is scary to me. There is a whole class of amazing folks who can re-implement models to scale them, and if you know them, reward them well.
3) How will models/algos/systems be built? This workflow is often pretty sloppy, just a bunch of jupyter notebooks or a tonload scripts (but they're in Git, so it's ok), and so replication and scaling becomes painful... esp. in regulated industries. Again, data engineering approach, but needs a more nuanced understanding of the vagaries of ML/AI. I find that solving business problems may not always fit a traditional software workflow (call it "agile" all you want, it doesn't always fit) but ymmv. So, you may need to create new workflows for your org's needs, and this tooling may not be off the shelf, but like automated testing, this tech debt will need to be paid sooner or later.
4) What ML/AI approach should I use, esp. if I'm using pre-existing approaches? This becomes more of the data science analytic approach, mixing the understanding of how ML/AI works with the data landscape (how was my internal data generated? What does it mean? Is it stable? What's available at score time, and what's it's latency?) and how to build various working predictive models. Note that this is traditionally where data scientists/model builders/analysts spend their time, from data cleansing to preliminary analysis to generating/training various models to crying when none of them predict well to stumbling onto a fascinating and amazing combination of models and approaches at 3am. Yes, math is defn helpful here, but _understanding_ the underlying math is often helpful enough, vs. _mastery_. A good understanding of the levers affecting each model/algo/DL architecture/etc. and how to diagnose them can get you pretty far, though you may violate assumptions or overfit if you aren't careful.
5) Real Algo design: Using all that math that underlie the models to not just diagnosing a pre-designed package but making your own optimizer, or your minimizers, or your own way of computing the Hessian, or a new weighting approaches, or whatever new approach ...
https://web.stanford.edu/~hastie/pub.htm
http://bayes.cs.ucla.edu/jp_home.html
http://andrewgelman.com/books/
I guess it all depends on what is a mathematical background and what is advanced!
What I found is that many "data science" books cover how to use R or Pandas at a very introductory level. Books like ESL focus on core theories (which is great) but do not focus on how to tackle a tough real-world data.
I suppose much of data insight come from experience, but I was wondering whether there are sources to help me jump start.
Sometime it suddenly becomes apparent and we suddenly see how to do it and everyone feels pretty sheepish!
Like programming, machine learning has its 10xers, and I've worked with several. There's one thing they all have in common (beyond experience): they can think like the model. They've read a bunch of ML papers, get the intuition behind them, and envision from start to finish how data can be captured by various models. Beyond a small amount of hyperparameter optimization they often build a great model on the first or second take, and can put out production models in days (which would normally take novices weeks or even months). This requires math knowledge, because ML models are built on top of math. Even in higher level ML packages that do a lot of the work for you, novices get stuck in dead ends where their models don't work and they don't know why. The 10xers drive right through it because they know why their model isn't working.
When I interview for my team, I usually have non-junior candidates describe a ML algorithm they would use for a specific problem, then ask why certain things could go wrong - e.g. if they say they would use a logistic regression model to predict an action, I ask why the model may be returning the same score for every test case. A good candidate would understand that this means the coefficients are getting pushed to zero, and would list off reasons why this could be occurring (too much regularization, underpowered data, constant target variable, etc). You learn these things from experience, but you understand them because you know the math behind the models.
Otherwise algorithms will learn nonsense and random correlations made from datasets of random noise.
Look at finance to see how it "works".
But also : can work in a team and leverage relationships for insight.
A lot of the materials etc I have seen for learning ML approach it from a very academic perspective complete with all the algebra, equations and proofs of how a neural network works.
This is totally 100% fine, but I am of the view that this is simply not necessary for those starting out in ML who just want to get stuff done.
When we introduce people (including kids/young adults!) to programming, we dont first force them to sit through a semester of learning the in-depth details and nitty-gritty of how modern compilers work before we even begin to let them write "hello world". Yet the ML courses I've seen all seem to start with the usual mathematical hazing of algebra, proofs, and general mental-masturbation around how brainy and rather complicated it all is.
There is, I think, a fundamental disconnect between people who want to learn how ML works and people wanting to learn how to use ML. It feels like most of the MOOCs and courses are more academically-focused and concentrate on teaching how ML works and are as such fairly-disconnected from what I feel like is probably a majority of people just wanting to know how to use ML in their day-to-day activities.
Sure, the theory is important, and yes you may not become an expert in ML if you don't study the theory, but I find it hard to accept the view that you need to know the proofs for why ML works before you can be a productive ML developer/user.
I would imagine that most beginners and non-advanced-users/experts these days are using a library like TensorFlow or Keras to do their ML work. Yet if you look at TensorFlow or Keras, then I think you'll find that pretty much all of the maths is hidden and you just need to know the "building blocks" and the higher-level theories of ML - the layers, the architectures, the data prep, the interpretation etc etc etc.
Just like when I code some Java or Go I can use my "building block" knowledge of programming to write well-designed, high-quality, maintainable, testable, high-performance code without having to worry about the theory of what the runtime is doing. Of course there are scenarios where it might help to know the specific maths for how Go's or Java's garbage collector or whatever works and also be able to do a mathematical proof on a whiteboard about it, but I'd wager that situation does not come up very often! :-) It is certainly not a requirement to write decent code, just like I dont think it is a requirement to use TensorFlow.
Again, to be clear, yes you might not become an expert or even an "advanced" user of ML without mastering all of the theory, and yes you might not be able to create truly-innovative new models and beat DeepMind at their own game (pun intended). Of course. But then, not every programmer is destined to become the next Jeff Dean or John Carmack either.
The fast.ai course is good in that it largely skips the whole maths initiation ceremony and just goes into the building-blocks of ML. Kudos to them.
Similar attitudes in the past have lead into misconceptions like if you are only applying statistics in your research, you don't really have to understand what p-value is, just run the magical tests and report if it's "significant" or not.
No matter where the inspiration for a model comes from, the final formulation is always mathematical, and I think that without an appreciation for the mathematics, it's hard to get a true feeling of a model's effectiveness and limitations.
The hardest may be thinking in terms of higher dimension tensors (matrices that have 3 or more dimensions) but that is not even mathematics, just an intellectual effort.
1. who teaches it
2. what gets taught first
3. what forms and notations get used
In short, to _apply_ machine learning tools, you need a good understanding of _applied_ machine learning, which is not a small subject area, but is very approachable for most.