HN Explorer

Ask HN: Machine learning in 5 papers. What should they be?

11 points by snikolov ↗ HN
Hey folks,

I am helping to organize a reading group to introduce people to machine learning in a way that is more informal and interactive than a class. With only 5 meetings slated for this semester, what do you think would be a good selection of topics to cover? Ideally one would come out of this reading group with high level appreciation of ways to think about problems in machine learning (feature selection, training, supervised vs. unsupervised learning, function approximation, etc) as well as familiarity with some high level tools (statistics, optimization, linear algebra). This is not meant to be a class, which makes it all the more difficult to strike a balance between depth and breadth.

Any thoughts, as well as paper or topic suggestions would be greatly appreciated.

5 comments

[ 3.7 ms ] story [ 24.8 ms ] thread
cschmidt
I would cover some fairly basic topics:

1. Naive Bayes (with review of probability needed to understand)

2. Linear Regression

3. Decision Trees (for a nonlinear model)

4. Boosting (and why it doesn't overfit much)

5. k-means (and other clustering)

Be sure to emphasize cross validation or a holdout set to evaluate all these models.

The new edition of the "WEKA" book would be a nice text, rather than the original papers.

Data Mining: Practical Machine Learning Tools and Techniques, Third Edition Ian H. Witten, Eibe Frank, Mark A. Hall

snikolov
Thanks! This is a great list.
mitko
Few things that I know & like (they are rather heavy but deep):

   F. Cucker and S. Smale. On The Mathematical Foundations of Learning. Bulletin of the American Mathematical Society, 2002.

   Sutton & McAllum : Conditional random fields for relational learning

   Pieter Abiel: apprenticeship learning with a helicopter (inverse reinforcement learning)
You can also add the paper of the team that won the Netflix challenge (collaborative filtering), and something about non-parametric learning (dirichlet processes etc.)

Also check out Lasso regularization (L1 norm) for sparse feature selection.

snikolov
Thanks, this is all great stuff. I've been meaning to read some of Smale's papers on learning theory. It might be over a lot of people's heads though, since we advertised this is an introduction to basic concepts that one might encounter in a first course in Machine Learning. We are doing two groups of 10. The topics will have to depend on the kinds of people that applied and their interests, but I like the idea of having a beginner group and an advanced group (possibly based on mathematical level).
rgens
I'd recommend the slides from Pedro Domingos' (UW) ML class. They give a broad overview of the ML landscape. He follows Tom Mitchell's (CMU) book pretty closely. http://www.cs.washington.edu/education/courses/cse546/10wi/

Both the slides and the book are highly readable. There are no good papers that cover everything. I wouldn't necessarily dive in to a paper on regularization or non-parametrics for a beginning class (is this a slug talk series?). The Netflix paper really doesn't convey good understanding of ML (they combine a large number of techniques at the problem) but you might want to cover boosting using a different source.