Ask HN: How to Seriously Start with Machine Learning and AI
Since years I've been seeing tones of news "how machine learing did smth... " and today that's enough with just reading how other people change the world with AI. I want to join into this area and scientificly understand how it everything works - make my own projects...
-I'm a third-year Computer Science student who just has passed most of the needed courses like obj programming,python course, databases, math statistics, algebra etc... I really enjoy playing with data like projecting databases, programming backed etc...
Everything I know until today - I have learned on my own(swift, python, backend). Mostly by practice and solving problems. Now I really want to start serious journey with Machine Learning and AI.
But by making some small research which made me realised that I don't want just to implement already done frameworks for e.q face recognition (maybe I should?) I would like to understand the topic really seriously and be able to explore this area... ---but here's a problem because I don't know how to start it. I've got enthusiasm, some ideas for a projects, but still don't know almost anything about how exactly everything works.
When I was starting with programming, I read some books, watched online lecture and bang. I started doing my own projects. How to start in this more scientifically sophisticated area?
Are there any courses, books, online lectures which you can recommend me for a start to understand how it all works? Unfortunately, my university doesn't lead any more interesting courses in this area... People here are just fascinated with it but nothing more complex...
I'm still young so why not to lose time on something that seems to be really fascinating ;)
76 comments
[ 3.9 ms ] story [ 140 ms ] threadhttp://aima.cs.berkeley.edu
https://www.allbookstores.com/book/compare/9780201533774
might be good for some foundational stuff. I felt they were pretty readable.
I remember having to do backprop in excel as one assignment.
https://www.class-central.com/subject/ai
Some guys self made Master's program: https://hackernoon.com/my-self-created-ai-masters-degree-ddc...
You can find some fun tutorials here: https://www.redblobgames.com
The CS 188 "Intro to AI" class at Berkeley is excellent: http://ai.berkeley.edu/home.html
It used to be on edx.org but I think a lawsuit about accessibility required them to remove it? Perhaps you can find it in the edX archives.
Edit: looks like you can find the lecture videos and other resources on the Berkeley site: http://ai.berkeley.edu/lecture_videos.html
https://www.kaggle.com/learn/overview
EDIT: I also second the fast.ai suggestion(s) as well
I have the same question and I got my degree 12 years ago.
I remember my AI course in college consisted of implementing a neural net in Lisp and memorizing a ton of dense text.
It is really easy to take a Tensor flow model someone already wrote and tweak the parameters to make it work for your use case. I think that code reuse and open source is the largest advancement in AI in the last 10 years.
So a lot of companies can use AI in their products without even really knowing how it work.
Now once you start training your own models the hardest part to me is the vocabulary. So many tutorials and classes say "use this algorithm" or "take this code and modify it"... I want to know why I chose that algorithm, what were the other choices, and how do I write that code if I don't have an instructor to do the boilerplate for me. It is very frustrating.
Hopefully some of the answers here will answer some of those questions.
As another note, a lot of the advanced AI uses Calculus. algebra is not enough. I don't know what college you went to or what accreditation it has but by third year you should at least have taken Calculus II... maybe even Differential Equations. If you haven't, don't worry, 99% of programming jobs won't use them. But parts of AI will.
It was high level enough, and got me to understand enough, to get me to a point where I could start trying to build a side-project, without exposing me to the deeper math behind Neural Nets.
It was a great starting point without being overwhelming. Now I feel that I have the option to either go deeper if I need to, or go wider.
I find Andrew Ng to be an amazing teacher - explains things simply, clearly, and in a way that I find super easy to understand.
It starts out reaaaally basic but give a thorough grounding of the maths and the intuition behind it.
Fantastic course though.
They believe (and have research backing them up), that the way we teach math (base and rote concepts, building until you can understand something complex) is sub-optimal. They dive into the code and get stuff done, then later bubble back up for concepts.
For me, it was bewildering at first, but if you can trust your instructor, you trust they won't leave you stranded. (It does also require the type of student who does a lot of study on their own!)
I agree - it's a great introduction, and I learned a ton from it (and it answered a ton of questions I had until then).
Book: http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%...
Notes: Its very very math heavy but if you really want to grasp the concepts and the idea around each topic, this is one of the way to go.
Online Lectures: https://www.youtube.com/watch?v=mbyG85GZ0PI&list=PLD63A284B7...
I like how he explains stuff and adds some context behind the math of each topic.
Alternative: https://www.youtube.com/channel/UCWN3xxRkmTPmbKwht9FuE5A
A pretty good youtube channel that follows up on modern machine learning and he has all of his video tutorial demos on GitHub.
Hope this helps fellow Machine Learner :)
I'd go with David Barber's book every time.
First, you need a strong mathematical base. Otherwise, you can copy paste an algorithm or use an API but you will not get any idea of what is happening inside Following concepts are very essential
1) Linear Algebra (MIT https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra... ) 2) Probability (Harvard https://www.youtube.com/watch?v=KbB0FjPg0mw )
Get some basic grasp of machine learning. Get a good intuition of basic concepts
1) Andrew Ng coursera course (https://www.coursera.org/learn/machine-learning)
2) Tom Mitchell book (https://www.amazon.com/Machine-Learning-Tom-M-Mitchell/dp/00...)
Both the above course and book are super easy to follow. You will get a good idea of basic concepts but they lack in depth. Now you should move to more intense books and courses
You can get more in-depth knowledge of Machine learning from following sources
1)Nando machine learning course ( https://www.youtube.com/watch?v=w2OtwL5T1ow)
2)Bishops book (https://www.amazon.in/Pattern-Recognition-Learning-Informati...)
Especially Bishops book is really deep and covers almost all basic concepts.
Now for recent advances in Deep learning. I will suggest two brilliant courses from Stanford
1) Vision ( https://www.youtube.com/watch?v=NfnWJUyUJYU )
2) NLP ( https://www.youtube.com/watch?v=OQQ-W_63UgQ)
The Vision course by Karparthy can be a very good introduction to Deep learning. Also, the mother book for deep learning ( http://www.deeplearningbook.org/ )is good
Also the basic idea of chain rule is important for deep learning.
Regarding statistics, I already mentioned the probability course which describes most of the important statistics concept you need. Also, some idea of Hypothesis testing can be helpful
After that I extended it to solve the hand written digits problem. Tweaking it to get past about 80% accuracy taught me a lot of intuition about how the learning rate/other parameters interact.
After that going through Andrew Ng's machine learning course will fill in some of the underlying math.
The best way to learn the details is of course to read the original papers. This is especially true for following along with the latest developments in deep learning.
[1] https://www.ethz.ch/content/vp/en/lectures/d-infk/2017/autum...
http://www-bcf.usc.edu/~gareth/ISL/
I took that class back in 2015. I found that the lectures were sometimes hard to follow, unless you already know the concepts (which creates a bit of a catch-22). For me the most valuable moments were, when Prof. Buhmann got sidetracked by some anecdotes. Would absolutely recommend, but maybe not as a starting point.
They will guide you through machine learning and deep learning basics without being overwhelming with math. Good luck!
1. Watch short tutorials about TensorFlow on Youtube
2. Look for lectures about specific topics that sound interesting.
3. Read NIPS papers that sound interesting
4. Check out the Deep Learning textbook, but maybe don't read the whole thing ( http://www.deeplearningbook.org/ )
5. By this point you should have a very rough idea about the current and past states of machine learning. You didn't have to spend any money or put in any exhaustive mental effort. If it still sounds interesting and you are motivated you can try a full online course or buy some paid books.
I'll also say something here that isn't established, so I may take some heat. I believe there are two main paths in AI that will eventually converge to general AI: symbolic and sub-symbolic reasoning. If you go the path of symbolic reasoning, studying functional programming, theory of computation,type theory, natural language processing/compilers are in your future. If you go the path of sub-symbolic reasoning, you will be closer to optimization methods, neural networks, etc. It really depends on what you want to do. Ex: Computer vision is all about sub-symbolic reasoning, while natural language processing is heavily about symbolic reasoning. Of course advanced applications mix both! If you want to go after general AI, you gotta figure out how to tackle both forms of reasoning.
In the end, if you are serious about being an AI researcher, you will have to be a great computer scientist and mathematician. This is why it seems so difficult to get into. "Do I focus on working on messing with libraries and algorithms or the mathematical theory? And if I do both, how!?"
It's not easy, but just start and keep going. Others have provided great advice. One of my greatest joys in life has been coming to understand the marriage of computer science and mathematics under the banner of AI. It is really something exciting worth living for.
[Never give up!](https://www.youtube.com/watch?v=KxGRhd_iWuE)
Also, I recently gave a talk on getting into AI and my view on the state of things. It also has a resources section that may be of interest. Slides: https://docs.google.com/presentation/d/1pDZLkFTFjuZzM8lIKkuC...
The eigenvectors of the inverse of the covariance matrix...
Squeeze and twist it into a hypersphere!
Mahalabobis distance is generalized z-score! Oh!
For instance, last year I completed Udacity's "Self-Driving Car Engineer Nanodegree" course. Large parts of the course needed us to train neural nets (tensorflow and similar) on data we either generated or downloaded. We were provided the option to use AWS instances for training (free credits), but I opted to use my local box.
It took me about a day or so of playing around before I finally got everything working properly (CUDA, etc) with my NVidia 750ti GPU. This is a very low-end GPU, but it honestly performed quite well for the course. It could only handle a limited amount of data, and sometimes training cycles took a while for turnaround (depending on the task), but it ran the resulting models quite easily (while still handling the 3D rendering tasks of the vehicle simulator).
For learning purposes, it will all depend - if you already have a machine with a decent GPU, CPU and RAM (say something equivalent to the 750 or better, 4 cores or more, and say 8 gig of RAM), it might make sense to try to do things locally - if you think your skills are up to the configuration challenges (I got my system working, but I ended up breaking Ubuntu's update system, because I'm on 14.04 LTS, and I had to hand-install many things to fix dependencies and such for Python, Tensorflow, C/C++, etc - in order to complete the course).
However, if you are planning on processing a huge amount of data for a large model, but aren't planning on doing this constantly - then an AWS instance might be a better option, as a custom rig for this kind of thing I'd imagine would be a bear to spec out and configure - not to mention cost.
I'm certainly not an expert on all of this, though...ymmv.
That's kind of funny, given that one of ML's biggest criticisms is that not even the field's foremost experts truly understand how it works.
Firstly, my background is not in mathematics or computer science what-so-ever; I'm a classically trained botanist who started came at the issue of programming, computer science, and ML from a perspective of "I've got questions I want to ask and techniques I want to apply that I'm currently under prepared to answer."
Working as a technician for the USDA, I learned programming (R and python) primarily because I needed a better way to deal with large data sets than excel (which prior to 5 years ago was all I used). At some point I put my foot down and decided I would go no further until I learned to manage the data I was collecting programmatically. The data I was collecting were UAV imagery, field and spectral reference data, specifically regarding the distribution of invasive plant species in cropping systems. The central thrust of the project was to automatically detect and delineate weed-species in cropping systems from low altitude UAV collects. This eventually folded into doing a masters degree continuing to develop this project. That folded into additional projects applying ML methods to feature discrimination in a wide range of data types. Currently I work for a geo-spatial company, doing vegetative classification in a wide range of environments with some incredibly interesting data (sometimes).
I think you've got the issue a bit cart-horse backwards. In a sense I see you as having a solution, but no problem to apply it too. The methods are ALL there, and there are plenty of other posts in this thread addressing where to learn the principals of ML. What this doesn't offer you, is a why of why you should care about a thing? My recommendation would be to find something of personal interest to you in which ML may play a role.
With out a good reason to apply the techniques that everyone else here is outlining, I think it would be too challenging to keep the level of interest and energy required to realize how to apply these concepts. Watching lectures, reading articles, doing coursework is all very important, but it shouldn't be thought of as a replacement for having personally meaningful work to do. Meaningful work will do more to drive your interests than anything.
It seems to me that so many (online) courses jump to applying tf/pytorch to a predefined dataset, whereas most of the work is in preparing the data. I have a personal project I'd like to try out classifying images, and haven't had much luck finding resources on building my own training dataset.
Can you recommend any resources on assembling and collating your own dataset?
This is gold advice and the only legit way to stick to something for the long term. OP actually needs a niche, an industry, a cause to care... then instruments will come naturally. The other way round is flooded already.
In light of that, I believe the technology to have progressed to where one can learn how to use existing libraries to solve specific problems. Lots of businesses have specific problems, and will pay me to solve them.
The Fast.AI course (part 1 v2) is the best way to get started, IMO. The fastai library wraps up a lot of boilerplate, and gives you a simple recipe w/ conceptual understandings to achieve state of the art results (top 20% on just about any kaggle competition) in just a couple months of intensive study.
Does FAST.AI get into the grit of actually doing or is it more philosophical?
That said, the lectures are pretty rough around the edges compared to something like a coursera class. And his goal is to get you to play around with stuff on your own using his stuff as a jumping off point. It works well for some people but not as much for others.
Multiple times, the prof said things like "the theory says this shouldn't work, but it does, so we use it." (Thinking about his perspective on the practical reality surrounding the "curse of dimensionality" (i.e. that it's not a curse)
1. ConvNet (for images and structured data)
2. RNN (for text and sequence data).
They don't teach you far advanced stuff, like how to creatively misuse the hammers to knock different nails (example: using ConvNet, tweak as causal convolution to handle sequence data)
I don't want to try to dissuade you in particular, but I think more young people should apply this principle to the question of what field to enter. I've seen dozens of "How do I get into AI/ML?" posts in the last couple of years.
The fast.ai courses are a more top-down approach to ML, and there are plenty of good reasons for taking this approach. You'll start getting practice with libraries like Tensorflow right away. However, if you have a fairly strong math background and linear algebra doesn't give you nightmares, I highly recommend the Andrew Ng courses. A deep (ha!) understanding what's going on "under the hood" in ML will help your debugging, inform your strategy, and make your code better in the long run.
If you want a linear algebra text, I enjoy Strang's Introduction to Linear Algebra