Surprisingly, it's quite doable in practice. See e.g. https://arxiv.org/abs/1911.13299.
Following this reasoning, why didn't the enlightenment occur in asia, which had earlier access to gunpowder? Iirc the ottoman empire had portable guns before the west. And china had fire lances way before that.
The intrinsic dimension paper indeed doesn't really show that big networks also search in small subspaces (and neither does it claim to), but this has already been shown in related papers like…
Counterpoint: there's a whole civilization on the western/USA diet, which has been shown to not be that great for your health (see e.g. obesity rates).
As to 1, it has already been established that there's no biological plausibility of backprob whatsoever. You can only call the current models "neural" networks in the vaguest sense of analogy. There is significant…
Yes they do (arxiv is a place for scientific papers not press releases). I've only skimmed it, but the paper introduce an adaptive way to clip gradients. Meaning that if the ratio of the gradient norm to weight norm…
As a Belgian, I felt inclined to comment. And give a bit of the perspective on it from here. It's true that the Belgian colonial history slips under many people's radar, and I agree that it should be discussed more. Too…
Surprisingly, it's quite doable in practice. See e.g. https://arxiv.org/abs/1911.13299.
Following this reasoning, why didn't the enlightenment occur in asia, which had earlier access to gunpowder? Iirc the ottoman empire had portable guns before the west. And china had fire lances way before that.
The intrinsic dimension paper indeed doesn't really show that big networks also search in small subspaces (and neither does it claim to), but this has already been shown in related papers like…
Counterpoint: there's a whole civilization on the western/USA diet, which has been shown to not be that great for your health (see e.g. obesity rates).
As to 1, it has already been established that there's no biological plausibility of backprob whatsoever. You can only call the current models "neural" networks in the vaguest sense of analogy. There is significant…
Yes they do (arxiv is a place for scientific papers not press releases). I've only skimmed it, but the paper introduce an adaptive way to clip gradients. Meaning that if the ratio of the gradient norm to weight norm…
As a Belgian, I felt inclined to comment. And give a bit of the perspective on it from here. It's true that the Belgian colonial history slips under many people's radar, and I agree that it should be discussed more. Too…