HN Explorer

Ask HN: Best way to get started with AI?

709 points by hackathonguy ↗ HN
Hey guys -

I'm a intermediate-level programmer, and would like to dip my toes in AI, starting with the simple stuff (linear regression, etc) and progressing to neural networks and the like. What's the best online way to get started?

Thanks!

92 comments

[ 4.5 ms ] story [ 172 ms ] thread
hackernewsacct
As a follow up: I want to pursue a math degree study. What course titles and textbooks starting at the calculus level do you guys recommend? I want enough math chomps to then go onto a PhD in ML.
ktta
Calculus: Thomas, Weir & Hass - Thomas' Calculus

Linear Algebra (which is what you really need): Gilbert Strang - Linear Algebra and its applications

These two are all you need, with which you'll get a solid base. Then you're good to go on your own. These two combined are about 4 semesters worth of work. But if you really focus, I think you can get them done in a little less than 6 months.

If you want a 'just what I need' approach, Khan Academy.

dominotw
Spivak calculus. There is nothing like it. You will learn how to think in math, not just calculus.
anothertraveler
Whoa! I was just talking about this text book the other day! I think it's out of print these days, but it's a hidden gem!
_raoulcousins
For self-study, you don't necessarily need to follow the usual progression of math classes that start at calculus. It's more important to get comfortable with linear algebra than calculus, especially the way a lot of intro calculus courses focus on calculating integrals and derivatives. Maybe it's not the best message, but the worst grades in my math degree were in the intro calculus classes.

I don't remember what intro linear algebra books I used, but my college uses this: https://www.math.ucdavis.edu/~linear/ (I took the class before this free textbook was developed).

minimaxir
If you’re genuinely a novice programmer/lesser background in linear algebra, AI should be the last thing on your mind. Any attempts at a shortcut will enhance the difficulty in learning AI, and being able to code things besides simple examples. (which is why I am annoyed by many of the ML MOOCs which are targeted toward novice programmers)
binarymax
Why does this annoy you? Some of the MOOCs are amazing and they certainly don't shortcut. I had many hard-learned lessons and realizations when going through the MOOCs I have taken. They also help in "knowing what you don't know"...which is essential in breaking out of ignorance.
hackathonguy
I've learned computer science in college, so I do have _some_ linear algebra background. I'm by no means an expert coder, but maybe 'intermediary' would have been a better description than 'novice'. :-)
dagw
I disagree. There is nothing magic or hard about basic ML. You can do real work with only some basic linear algebra and programming skills. Sure you won't be doing novel deep learning on 100 terabyte datasets, but most problems aren't that anyway.
aalleavitch
Yeah, and to be honest while it's always important to be able to understand the math behind what you're doing, you can easily get started with just understanding what these algorithms do and why, and then work to expand your knowledge of the math over time from there.
sillysaurus3
Elitism isn't too helpful. Determination counts for more than skill.
minimaxir
I say it for the opposite reason of elitism (and I’ve spoken out many time against elitism in ML/AI).

As other posts note, there are many resources available for teaching the concepts. But they don’t teach the limits of AI, and the rise of MOOCs is setting novice programmers up for a shock when they encounter real world data that is not as nice as the Titanic dataset, and requires making smart decisions to handle the data, and handle it in a way that does not invalidate the results.

Many romanticize ML/AI as something that can solve any problem, which is a dangerous approach.

yters
ML is just stats with non-linear systems. No need for programming to do stats, and while linear algebra helps, it is not essential.
binarymax
I highly recommend Andrew Ng's Coursera courses for both Machine Learning and Deep Learning. Good for beginners, Math is taught along with the course, and gets you a solid foundation:

https://www.coursera.org/learn/machine-learning/

https://www.coursera.org/learn/neural-networks-deep-learning...

hackathonguy
Thank you! Should I start with the Machine Learning one?
binarymax
At your level yes, I would recommend starting with the ML course. It is really beneficial to understanding how the mathematics work.

The two most important things to remember, since the courses are challenging: 1) don't be in a hurry, and 2) don't give up! Take the time to learn every detail presented, do the optional exercises, and dig deep.

xeromal
It's definitely challenging. The math and just seeing the complicated formulas really push me, but the reward is good too. I'm tired of pushing pixels and doing some meaty stuffy like ML is a nice change of pace.
icc97
From my experience Andrew Ng wiped the floor with every other lecturer I've had. Both the ML and his new Deep Learning course.

If the lecturers aren't very interesting Coursera can be as hard as any other lectures. I gave up on the Scala functional programming and disappointingly have stalled with Geoffrey Hinton's Neural Networks courses.

But I really can't understate how good Andrew Ng is, he has a very relaxed manner and manages to make some very complex topics seem almost trivial.

The worst of the mathematics is derivatives and matrix multiplication. You can even avoid matrix multiplication mostly in the ML course, but in his Deep Learning course he takes you through the 300x performance benefit you get from using NumPy and matrix multiplication vs loops.

fargo
I would recommend starting with deep learning first since that's what you are interested in and it covers all the ML principles you need to be familiar with. If you want to go deeper and get familiar with other ML techniques too you can easily follow the old course afterwards.
dagw
Andrew Ng's courses are excellent. Another pretty good Coursera course is Machine Learning Foundations from the University of Washington. It is very high level and novice friendly. While it covers non of the math and very little programming it does give a nice quick introduction to the most popular ML techniques out there and when to use them. It all depends on what level you are interested in starting. They also have follow up courses that go deeper into the different techniques.
ravishk
+1 for this recommendation.

I would specifically recommend Machine Learning Foundations: A Case Study Approach - It is fantastic and helped me greatly start my ML journey last year.

Turi is awesome, I hope Apple is doing something great with it.

dmode
Just hijacking this question for my benefit as well. I am a product manager in enterprise focused software. I want to transition to the world of AI. Is Udacity's $600 Deep Learning Nano degree worth it ?
palatalizacija1
I don't know about Deep Learning Nanodegree, but I took Machine Learning and AI Nanodegrees at Udacity and they are definitely worth it. I would not recommend them to total beginners in the field. You need to have at least some experience with data science and Python to be able to follow along. Do some free courses on Udacity, Coursera, EdX and other platforms, try to implement these algorithms with your own data and problems and then take the Nanodegree to fill the gaps.
newbear
Did you find a job after?
kmax12
It somewhat depends on if you are looking to build AI to address business problems or if you are more interested in the type of AI work you see companies like Google discussing.

I can speak to what "AI" means for most businesses outside Top Tech which more frequently work with tabular, relational, or log data rather than image and text. For these companies, this is what you need to learn how to do

   1. Define a prediction problem and extract labels

   2. Organize and clean the data for prediction

   3. Perform feature engineering by applying domain expertise

   4. Apply an off-the-shelf open source machine learning algorithm like a random forest
Assuming you have access to data and programming skills to clean your data, defining prediction problems and performing feature engineering are the most important skills you have to pick up. For machine learning you can you use open source libraries like scikit-learn or tensorflow.

At my company, we've noticed a lot of programmers are intimated by the feature engineering step in particular, so we tried to make it easier by creating an open source library called Featuretools [0].

[0] https://github.com/featuretools/featuretools

mceoin
fast.ai - it's free, and there's a low level of assumed knowledge from the outset.
wiremine
I'd second this. I'm not a "math person" so to speak, and they provide a great education that doesn't dip too deep into the math.
bhollan
This.

I have two engineering degrees and studied neural networks in college. He cuts down about 3 months worth of math to a couple of lines of code in such a way that makes it make sense AND is productive. It may just be that his teaching style matches my learning style really well, but I'm enjoying going through it.

deepnotderp
For deep learning, my two favorite nominees are:

1. Hugo Larochelle's Deep Learning course available on YouTube

2. Depending on how much math you like, Nando de Freitas's Deep Learning course (also on YouTube) is also superb.

hal9000xp
I'm in the same boat. For long time, I was interested in AI but at the same time intimidated by math. I'm relatively comfortable with discrete mathematics and classical algorithms and at the same time calculus and linear algebra is completely foreign to me. Also, I do not accept way to learn ML without good understanding of core principles behind it. So math is a must.

A few months ago, I stumbled upon very amazing YouTube Channel 3Blue1Brown which explains math in very accessible way and at the same time I got feeling that I finally started understanding core ideas behind linear algebra and calculus.

Just recently he published 4 videos about deep neural networks:

https://www.youtube.com/watch?v=aircAruvnKk

https://www.youtube.com/watch?v=IHZwWFHWa-w

https://www.youtube.com/watch?v=Ilg3gGewQ5U

https://www.youtube.com/watch?v=tIeHLnjs5U8

So my fear of ML was gone away and I'm very excited to explore whole new world for neural networks and other things like support vector machines etc

markatkinson
I came here to write a similar comment. Really make sure to watch the playlists in the correct order on the above YouTube channel.
skytreader
Worth noting that 3Blue1Brown also did a series on linear algebra which is eye-opening to say the least. Playlist at:

https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...

Even if you think you grok matrices, have a go at the first few videos of that playlist, if just for the visualization. It really helped me see what matrices (and operations on matrices) represent!

_xhok
3Blue1Brown is a treasure. The production value is excellent, and he's great at taking seemingly uninteresting ideas and painting a beautiful picture to connect them in twenty minutes. I used to go through a video before falling asleep each night.
tmaly
I just watched the first video. Thank for sharing.
sova
Hi! That's wonderful. What' a support vector machine used for?
pletnes
Classification and regression. Given examples, predict labels or values for new data. SVM used to be more «hot» than neural nets and are still very useful.
kregasaurusrex
Having watched the third one out of sequence, seeing the first two and then watching the third again helped me get a good understanding of the fundamentals. 3blue1brown as a narrator does a excellent job of allowing a rather tricky subject be more approachable, and inspired me to buy a course to allow a deeper dive into the math behind ML+NNs.
darethas
I also recommend taking up computer graphics for honing your skills in linear algebra. Graphics are essentially applied linear algebra.
hwu2whag
Wow thanks for this resource!
sn9
Regarding linear algebra, I highly recommend Klein's Coding the Matrix which uses Python to teach linear algebra.

I believe it was developed for Brown's linear algebra course for CS undergrads.

alexmuro
Personally I recommend Stanford CSI 231n http://cs231n.stanford.edu/

Its specifically geared towards visual recognition, but it starts with the basics of machine learning and moves on to feed forward nets and covnets and covers RNNs and attention towards the end.

The assignments are a great set of jupyter notebooks that really get your hands on the material and you can find a number of peoples complete assignments on github just by searching.

The lectures are available online as well https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-z...

I've done hinton's and Ngs courses and as someone who already has a non-ai development background I found this to be the best introduction. Its really an extension of Andrej Karpathy's Neural Nets for Hackers (http://karpathy.github.io/neuralnets/)

Danilka
Did you signup for the course, or just followed what's available on the page? It doesn't seem like there is any starting date there.
yters
I took a grad ML course based on this book: https://www.amazon.com/dp/B0759M2D9H

It teaches you the foundational theory behind ML, and shows how the fancier stuff is built on it. Good to know the foundations, so you can branch outside of predefined ML techniques.

projectramo
AI != ML

For AI, I would take the Udacity AI courses.

For ML, I would take the Udacity ML courses.

I take a lot of different online courses, I have no affiliation with Udacity, but their courses are just too good.

I studied AI (focused on ML) in a decent grad school (and I like to think I had the best teachers there), and I think the quality of the courses is comparable.

aalleavitch
Isn't AI just applied ML?

Or is it an operant/classical conditioning sort of thing, where AI is specifically about training programs to act rather than to perceive/categorize things?

I suppose you can have AI that incorporates no ML (like most video game AI), but I'd imagine that will become vanishingly rare in the future.

randcraw
In brief, AI uses existing knowledge and/or heuristics (to solve problems that lack a closed-form solution), while ML acquires knowledge and heuristics toward the same end, with the added goal of improving performance as it learns and adapting to changing conditions.

Traditionally, AI has been divided into distinct subfields (e.g. search, planning, natural language and speech processing, game playing, computer vision, robotics, knowledge representation, expert systems, logic, and ML). Today, ML is employed in all AI subfields, but until recently, most subject matter in each AI subfield had been unrelated to ML. In the past decade especially, that's changed as deep learning and probabilistic methods have gained mindshare and now are largely unavoidable when tackling AI-related problems.

In general, AI's subfields have focused on identifying fundamental obstacles and important features in their own problem domain and developing appropriate techniques that operate on those features when solving problems (like using object recognition and localization to solve vision problems like autonomous driving). I suspect AI's past emphasis on feature engineering has faded as NN-based ML has risen.

projectramo
It would seem that many people have divergent definitions but I’ve always learned that AI was Artificial intelligence broadly construed (goal seeking, planning, rule based expertise, logic etc) while machine learning is the specific subset that is (roughly speaking) statistical (neural nets, regressions, svm and the like).

Don’t have strong feelings about these definitions.

austenallred
Eh, I think most people use them somewhat synonymously nowadays. ML is a subset of AI, which has become buzzwordy enough to lost most meaning IMO.
skadamat
I'm involved with a startup that's specifically tackling this very problem -- how do you learn the theory & application of machine learning quickly (especially if you already know programming well). We teach using diagrams and interactive coding exercises in the browser: www.dataquest.io

If you already know Python, you could dive straight into machine learning (https://www.dataquest.io/course/machine-learning-fundamental...) and work your way upto calc / lin al, linear regression, decision trees, neural nets, etc.

If you want to get a taste without signing up, you can check out our blog posts that preview the course (like this one: https://www.dataquest.io/blog/machine-learning-tutorial/)

Happy to answer any questions over DM or email (srini@ourdomain).

gncb
I was at the same point as you until I discovered the new Andrew Ng course on deep learning [1]

It's a good structured way to learn the core of ML while learning about Neural Networks and without having to become and linear algebra expert which for most people including like me was a deal breaker with other courses. The timing is great too as ML now is so much different than it was 2-3 years ago.

[1] https://www.coursera.org/specializations/deep-learning

smortaz
Great resources in the replies. If you want an environment to run code in w/o much setup, try our free service:

https://notebooks.azure.com

it has Py2, Py3, R, F#, anaconda, TF, CNTK, etc. pre-installed.

There are some ML tutorials on it already + you can use the "load from github" feature to load, run, edit, ... many of the great tutorials already on github.

Other similar environments include colab by google and cocalc.

#Disclaimer: Microsoft

yexponential
Interesting, didn't know about this, thanks for sharing
bra-ket
Learn about human intelligence
chestervonwinch
I know you say you'd like to learn online, but I highly recommend picking up Duda and Hart's Pattern Classification to have a theoretical complement to the "hands on", programming type introductions. It's a very accessible intro to the topic, but also covers a lot of material in depth -- in particular, the topics you mention.
lottin
In my opinion the best way to get started is first study statistical inference and modelling, in particular linear regression and the method of maximum likelihood. This will give you a critical eye later on for discerning when it's a good idea to actually use ML and when it's not (an important skill that apparently is in very short supply these days ;).
austenallred
Possibly not what you're looking for (certainly not the cheapest option), but we (Lambda School - YC S17) just announced a live, remote class that trains engineers in AI & ML during weekday evenings for six months.

The next one starts in January, and is taught by an MIT grad that taught a similar course at MIT.

https://lambdaschool.com/artificial-intelligence

amigoingtodie
Would you do 20% for a salary of $100k or above?

What percentage would you do for $100k?

austenallred
It's still/always 17%, but it would cap out at $20,000 if above ~117k, so it would effectively be less than 17% on an annualized basis.

Or paying up-front/in monthly payments is $1041/month for 12 months.

_xhok
It's really important not to skip the math. As a friend once said to me, doing deep learning without understanding the math is like gambling. It's fine to initially take a more practical, project-based approach for the sake of staying motivated, and you'll retain things better if you have project goals in mind, but, the math is that important.

The good news is that compared to other technical fields, the math is also relatively shallow. Here are some good resources that you don't need more than calculus/linalg for (I've used all of them and they got me off the ground):

http://cs231n.stanford.edu/

http://neuralnetworksanddeeplearning.com/

http://course.fast.ai/

Once you feel confident, the Deep Learning book is more math-heavy, but it is really very good. The authors are more or less deep learning gods. It'll teach you a tremendous amount about how/why neural nets work and the principles used to discover new architectures, and gain a strong intuition for how to use neural nets as a tool. Read it slowly---unless you're already good at math, it takes a while to get through. Don't skip the first five chapters. Use Google and Wikipedia to pick up concepts you don't understand along the way instead of skipping over them (it will bite you later).