In the article I’ve linked he talks about how modern neural networks are not very biologically accurate. I follow the logic and while don’t agree with everything he says I do agree with the general sentiment. However, I cannot for the life of me figure out why he hates backpropagation. From what he’s written, it seems he takes issue with the optimization techniques for modern neural networks (reasonable), but then if anything he should have issue with gradient descent? Gradient descent is the optimization algorithm, backpropagation is just the calculus used to calculate the gradients. Yet he specifically singles out backpropagation as the problem. It just feels incredibly bizarre
1 comment
[ 3.1 ms ] story [ 11.4 ms ] thread