Numenta's based upon the hierarchical temporal memory model described in the book On Intelligence. It's an attempt to replicate the structure of the neocortex.
Mathematically it appears to be just another neural network. IMHO, deep learning is probably more likely to give systems approximating general intelligence.
HTMs are biologically accurate -- they do the same thing as the neurons in our neocortex! The most exciting aspect of Numenta's research is that they are building algorithms which think the way we do (albeit in early stages right now).
Re: "HTMs are biologically accurate": It depends on what you mean by accurate.
Hawkins argues that HTMs capture the systems-level effect of a brain, even if software models of neurons are not the same as biological neurons.
I can't personally comment on what others say about the accuracy, even at the systems-level.
Re: "The most exciting aspect of Numenta's research is that they are building algorithms which think the way we do (albeit in early stages right now)." It is a stretch to use the word "think" in this way, since thinking suggests consciousness. One wouldn't say we "think" when we simply observe. You might argue that HTMs pass signals in an analogous way -- that might be a fair statement, from Hawkins' point of view.
Yes, I realize that Hawkins says some interesting things about 'forward' and 'backward' processing even for 'low-level' processing such as edge detection.
Can deep learning algorithms learn from temporal data ? So far I have seen object recognition from a single image but not some intelligence gathering from a video.
Yes, they are called recurrent neural networks. The disadvantage is higher memory and time requirements. You need to store the state of the neural network at every time step, and then at the end iterate through the whole process in reverse.
Whereas best I understand it, HTM is online and updates the weights every time step without needing to keep a running memory of the states it's been in.
That's not really correct actually. A better way to look at it is that each neuron in the HTM is itself effectively a deep neural network. The HTM is a temporally aware hierarchy of these neurons.
Of course I read it. Fascinating concept, but I'm unimpressed by Numenta's execution. Certainly the ideas of a single neocortical algorithm, a temporal feedback loop minimizing error, the relationship between the hippocampus and the rest of the memory-learn-predict-observe-correct cycle are inspired.
'On intelligence' is the book that got me started on my journey in pursuit of AGI. I look forward to seeing the results from this. Video summary :
http://youtu.be/izO2_mCvFaw
It will be very interesting to see their findings. As far as I know Numenta's software (or HTMs in general) haven't been tested yet against known benchmarks like LSVRC, CIFAR, etc. Or results haven't been published.
Interesting timing for me, I just stumbled across Clortex[0] which is an implementation of Hawkins' HTM in Clojure today after watching How To Make a Brain in Python[1] which may be interesting to anyone that likes this stuff.
My impression is that this headline was made to sound like Numenta is a step on the way to the singularity, when it fact it's one of many companies working on machine learning techniques, the key difference being the founders have good resumes.
Biologically inspired engineering is a great thing, but to say something is a model of the neocortex in a meaningful way is a claim no one deserves to make in 2015.
I see a lot of patents for HTM, but have there been any peer reviewed publications from Numenta?
Jeff Hawkins himself states "HTM does not present any new idea or theory, but combines existing ideas to mimic the neocortex with a simple design that provides a large range of capabilities". Ok got it. Nothing wrong with competing on implementation and engineering.
So regarding that implementation, how are they doing against competitors in their space?
Vicarious, headed by the breakaway co-founder of Numenta had made waves recently claiming that they could solve most of the CAPTCHAs using low end machinery in real time. They too seemingly use HTM. Google too has a large team targeted on similar work. There has been a lot of funding and activity, certain one-off developments (like the one mentioned above), and very little verifiable output. However the scary part is that, the moment machine intelligence surpasses the human brain, technological advances would be of exponential scale and we would witness nothing short of an explosion. I do not think we will get a chance to plug-in extra processing power and memory to our brains and strut around quoting Shakespeare. We, our cognitive capabilities, and consciousness will be left out as insignificant anachronisms. What role can humanity play in such a world, I wonder sometimes...
Even after many years of talk about HTM, I've yet to come across one result which shows HTM outperforming other ML algorithms on any of the myriad data sets out there. Isn't it time they showed something? Anything?
20 comments
[ 2.9 ms ] story [ 68.9 ms ] threadMathematically it appears to be just another neural network. IMHO, deep learning is probably more likely to give systems approximating general intelligence.
What's the difference, broadly speaking?
> IMHO, deep learning is probably more likely to give systems approximating general intelligence.
If HTMs are just another neural net, why do you suspect deep learning will excel comparatively?
I don't know the first thing about HTMs - just curious.
Hawkins argues that HTMs capture the systems-level effect of a brain, even if software models of neurons are not the same as biological neurons.
I can't personally comment on what others say about the accuracy, even at the systems-level.
Re: "The most exciting aspect of Numenta's research is that they are building algorithms which think the way we do (albeit in early stages right now)." It is a stretch to use the word "think" in this way, since thinking suggests consciousness. One wouldn't say we "think" when we simply observe. You might argue that HTMs pass signals in an analogous way -- that might be a fair statement, from Hawkins' point of view.
Yes, I realize that Hawkins says some interesting things about 'forward' and 'backward' processing even for 'low-level' processing such as edge detection.
Whereas best I understand it, HTM is online and updates the weights every time step without needing to keep a running memory of the states it's been in.
Said book spends a good deal of time on criticizing neural networks, FYI.
[0] http://fergalbyrne.github.io/
[1] http://motherboard.vice.com/read/how-to-make-a-human-brain-i...
Biologically inspired engineering is a great thing, but to say something is a model of the neocortex in a meaningful way is a claim no one deserves to make in 2015.
I see a lot of patents for HTM, but have there been any peer reviewed publications from Numenta?
Jeff Hawkins himself states "HTM does not present any new idea or theory, but combines existing ideas to mimic the neocortex with a simple design that provides a large range of capabilities". Ok got it. Nothing wrong with competing on implementation and engineering.
So regarding that implementation, how are they doing against competitors in their space?