This reminds me of the thread where Geoffrey Hinton is wrongly described as laboring in the wilderness in promoting his ideas. Neural net's star has fallen and risen over the years but it always was a mainstream approach. In contrast, despite his fame as a mainstream explainer of CS, Hofstadter's approach has never gotten more than a few nibbles from researchers broadly and I think it's effectively only being pursued now by a dozen or less people who were direct student of his. I think that's a shame but what do I know?
> Abstract. In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarises relevant work, much of it from the previous millennium. Shallow and deep learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.
Yes, I think it's very important work that has not received the attention it deserves.
I've long had an urge to reimplement Metacat in JavaScript, so that people could play with it over the Web (Metacat, by Jim Marshall, is an evolution of Mitchell's Copycat). But as much fun as this would be, I'm not sure it will ever make it to the top of my priority stack.
6 comments
[ 0.24 ms ] story [ 28.2 ms ] threadhttp://people.idsia.ch/~juergen/
Cf. "Deep Learning in Neural Networks: An Overview" http://people.idsia.ch/~juergen/deep-learning-overview.html
> Abstract. In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarises relevant work, much of it from the previous millennium. Shallow and deep learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.
I've long had an urge to reimplement Metacat in JavaScript, so that people could play with it over the Web (Metacat, by Jim Marshall, is an evolution of Mitchell's Copycat). But as much fun as this would be, I'm not sure it will ever make it to the top of my priority stack.