Uhm.. no. What is special today about deep learning is the hierarchy of features. Yes, we are talking primarily about CNN's and RNN's trained with SGD TODAY. Yes,the entire catalog of ML techniques are still very interesting. Yes, in the future many of these techniques will be viable in a deep setting. But, no, you don't just get state of the art results by using linear regression or HOG and SIFT SVM's on a big data set.
OP: Just curious, how do you know they don't use deep learning?
@opless: Deep learning may be a fad in the press, but there's substance underneath. It's been breaking one accuracy record after another in raw, unstructured media year after year. ImageNet is just one deep convolutional net competing against another. A lot of other ML algorithms have hit a ceiling, and the ceiling is tied to not having enough humans to engineer enough features. DL doesn't have that chokepoint, and it is able to process vast seas of data that other algos have trouble with. Therefore, for many problems, it will continue to win.
I don't know the real answer in Boston Dynamics' case. However, in general, the answer is that Deep Learning is just beginning to be integrated into the pipelines of major robotics/reinforcement learning pipelines. The same thing took time in complicated speech recognition pipelines.
Deep learning is great for recognition/classification but robotics is much more.. Deep learning has to coexist with a number of other critical components.
However, there is major and extremely exciting work being demonstrated in the literature art recent conferences on this front. Expect major developments this year
I don't know anything about Boston Dynamics Robots, but "Deep Learning" is not a full solution to building robots. Deep Learning architectures may be used for sub-problems that robots need to solve, e.g. image classification, object recognition, or other classifications tasks where hand-designing features is hard and enough data is available.
Robots are a complex system with lots of moving pieces. Often they use some form of reinforcement learning (which in turn may use Deep Learning for estimating rewards) to decide which actions to take.
I think the biggest thing Boston Dynamics are dealing with ( aside from power density ) are improving balance and running algorithms, under the stresses of heavy loads, non-homogenous terrain, and loss of friction.
It's possible to use deep learning for controlling the servo motor positions and torques, the hierarchy of features is then varying levels of abstraction over the set of positions. The most granular layer may be features related to all the torques and positions of the servo, a higher layer may be some combinations of subsets of these, and the highest level may be features akin to the general representation of the spatial position and tension of the limb.
It's quite naturally similar to how human kinetic intelligence and perception works, say you are learning how to dance, you can be aware of the position of all of your limbs simultaneously ( the general pose you are adopting ), or you can be aware of the angle and position of your ankle or your right hip independently ( what your foot is doing ).
I think there could be things to learn from the networks in octopus limbs -- because of their extensive innervation.
It can be computationally expensive for a robot to use any sorts of deep learning on board for improvement during work.
It takes hours if not days sometimes to train a good scaled model. Then retraining the model could take a quarter of that time, which is too much delay for a robot. However, with linear models things can be easier.
Also, there are very few people who know how to manage DNN or creating neural nets from scratch in a proper required way. Otherwise Boston Dynamics have to stick to one of the existing libraries which wouldn't be ideal.
DNN is amazing for some work that am sure they must be using them already, for example models that can help them with the feature engineering and such.
8 comments
[ 487 ms ] story [ 349 ms ] threadThrow enough data at your favourite ML algorithm and you'll get better results.
Hierarchy of features is important.
@opless: Deep learning may be a fad in the press, but there's substance underneath. It's been breaking one accuracy record after another in raw, unstructured media year after year. ImageNet is just one deep convolutional net competing against another. A lot of other ML algorithms have hit a ceiling, and the ceiling is tied to not having enough humans to engineer enough features. DL doesn't have that chokepoint, and it is able to process vast seas of data that other algos have trouble with. Therefore, for many problems, it will continue to win.
Deep learning is great for recognition/classification but robotics is much more.. Deep learning has to coexist with a number of other critical components.
However, there is major and extremely exciting work being demonstrated in the literature art recent conferences on this front. Expect major developments this year
Robots are a complex system with lots of moving pieces. Often they use some form of reinforcement learning (which in turn may use Deep Learning for estimating rewards) to decide which actions to take.
It's possible to use deep learning for controlling the servo motor positions and torques, the hierarchy of features is then varying levels of abstraction over the set of positions. The most granular layer may be features related to all the torques and positions of the servo, a higher layer may be some combinations of subsets of these, and the highest level may be features akin to the general representation of the spatial position and tension of the limb.
It's quite naturally similar to how human kinetic intelligence and perception works, say you are learning how to dance, you can be aware of the position of all of your limbs simultaneously ( the general pose you are adopting ), or you can be aware of the angle and position of your ankle or your right hip independently ( what your foot is doing ).
I think there could be things to learn from the networks in octopus limbs -- because of their extensive innervation.
It takes hours if not days sometimes to train a good scaled model. Then retraining the model could take a quarter of that time, which is too much delay for a robot. However, with linear models things can be easier.
Also, there are very few people who know how to manage DNN or creating neural nets from scratch in a proper required way. Otherwise Boston Dynamics have to stick to one of the existing libraries which wouldn't be ideal.
DNN is amazing for some work that am sure they must be using them already, for example models that can help them with the feature engineering and such.