Check out the Flax ResNet50 example: https://github.com/google/flax/tree/master/examples/imagenet It runs about as fast as any of the other popular machine learning frameworks, occasionally faster. Disclaimer: I work…
To clarify, we found the range shrink if one trains for a fixed number of epochs and expand if one trains for a fixed number of steps. So tuning can be easier or harder depending on the budget.
I work as a researcher on the Brain team. An experienced machine learning researcher like Andrew Ng would probably not join the team as a Brain resident. We hire experienced machine learning researchers and engineers…
We don't have a simple separation of concerns like that. Brain and DeepMind share a common vision around advancing the state of the art in machine learning in order to have a positive impact on the world. Because…
The Google Brain (g.co/brain) team has people in SF. We are part of the same company as DeepMind so maybe this doesn't quite answer your question. ^_^ We are mostly in SF and Mountain View, but we also have people in a…
Here is a better article with an interesting interview from Arthur Chu: http://mentalfloss.com/article/54853/our-interview-jeopardy-...
There were a lot of pros a DeepMind. For example: Volodymyr Mnih, Andriy Mnih, Alex Graves, Koray Kavukcuoglu, Karol Gregor, Guillaume Desjardins, David Silver, and a bunch more I am forgetting.
It has content and communicates an approach to machine learning distinct from other approaches. It isn't like "big data" which is truly meaningless. However, deep learning is also not a single method or algorithm. I…
Will high quality textbooks and the printing press make the medieval university go extinct?
Yes, plenty. Releasing source code is quite common in the ML community.
Generally, the most relevant academic community would be the NIPS community and I have not noticed any Numenta papers at NIPS, but please point me towards any I have missed if you are aware of any. I expect a lot of…
Basically none. Numenta has yet to do anything that has impressed any researchers I know. Perhaps someday they will, but I am not counting on it.
Deep learning is not boosting at all. Deep learning is about composing trainable modules. Adding a layer f(x) to a layer g(x) to get h(x) = f(g(x)). Boosting creates a final classifier that is a weighted sum of the base…
I was involved in the speech recognition work mentioned in the article and I led the team that won the Merck contest if anyone has any questions about those things. I also spend some time answering any machine learning…
That is not entirely accurate. The Science paper described how to (pre)train a deep belief net by training a sequence of RBMs. Contrastive divergence for RBM training (and more generally products of experts) was…
The term "Deep Belief Network" has been abused in the literature (not pointing fingers, I've done it too). The DNNs used mean a neural net pre-trained with RBMs. Sometimes, when people say DBN, that is also what they…
For people interested in some (currently) undocumented research code in python implementing DNNs that is also on my website. Although the code is only an initial release. I will improve it later, but if I waited until…
Senones are just tied triphone HMM states. A context dependent HMM recognizer has a 3-5 state HMM for every context dependent phone. Conceptually, each different HMM state in each different phone HMM has its own…
1. Nothing. People ARE implementing similar things. It takes time, effort, and lots of computation. 2. People often prefer to implement their own ideas and compete (especially researchers). 3. Potentially lack of…
No, the HMM is not replaced in that work. The GMM is replaced, as you surmise. There are three problems with standard ASR: HMMs, GMMs, and n-gram language models. The GMM is the easiest to remove. Keeping the HMM allows…
Check out the Flax ResNet50 example: https://github.com/google/flax/tree/master/examples/imagenet It runs about as fast as any of the other popular machine learning frameworks, occasionally faster. Disclaimer: I work…
To clarify, we found the range shrink if one trains for a fixed number of epochs and expand if one trains for a fixed number of steps. So tuning can be easier or harder depending on the budget.
I work as a researcher on the Brain team. An experienced machine learning researcher like Andrew Ng would probably not join the team as a Brain resident. We hire experienced machine learning researchers and engineers…
We don't have a simple separation of concerns like that. Brain and DeepMind share a common vision around advancing the state of the art in machine learning in order to have a positive impact on the world. Because…
The Google Brain (g.co/brain) team has people in SF. We are part of the same company as DeepMind so maybe this doesn't quite answer your question. ^_^ We are mostly in SF and Mountain View, but we also have people in a…
Here is a better article with an interesting interview from Arthur Chu: http://mentalfloss.com/article/54853/our-interview-jeopardy-...
There were a lot of pros a DeepMind. For example: Volodymyr Mnih, Andriy Mnih, Alex Graves, Koray Kavukcuoglu, Karol Gregor, Guillaume Desjardins, David Silver, and a bunch more I am forgetting.
It has content and communicates an approach to machine learning distinct from other approaches. It isn't like "big data" which is truly meaningless. However, deep learning is also not a single method or algorithm. I…
Will high quality textbooks and the printing press make the medieval university go extinct?
Yes, plenty. Releasing source code is quite common in the ML community.
Generally, the most relevant academic community would be the NIPS community and I have not noticed any Numenta papers at NIPS, but please point me towards any I have missed if you are aware of any. I expect a lot of…
Basically none. Numenta has yet to do anything that has impressed any researchers I know. Perhaps someday they will, but I am not counting on it.
Deep learning is not boosting at all. Deep learning is about composing trainable modules. Adding a layer f(x) to a layer g(x) to get h(x) = f(g(x)). Boosting creates a final classifier that is a weighted sum of the base…
I was involved in the speech recognition work mentioned in the article and I led the team that won the Merck contest if anyone has any questions about those things. I also spend some time answering any machine learning…
That is not entirely accurate. The Science paper described how to (pre)train a deep belief net by training a sequence of RBMs. Contrastive divergence for RBM training (and more generally products of experts) was…
The term "Deep Belief Network" has been abused in the literature (not pointing fingers, I've done it too). The DNNs used mean a neural net pre-trained with RBMs. Sometimes, when people say DBN, that is also what they…
For people interested in some (currently) undocumented research code in python implementing DNNs that is also on my website. Although the code is only an initial release. I will improve it later, but if I waited until…
Senones are just tied triphone HMM states. A context dependent HMM recognizer has a 3-5 state HMM for every context dependent phone. Conceptually, each different HMM state in each different phone HMM has its own…
1. Nothing. People ARE implementing similar things. It takes time, effort, and lots of computation. 2. People often prefer to implement their own ideas and compete (especially researchers). 3. Potentially lack of…
No, the HMM is not replaced in that work. The GMM is replaced, as you surmise. There are three problems with standard ASR: HMMs, GMMs, and n-gram language models. The GMM is the easiest to remove. Keeping the HMM allows…