There's a long history of cycles in this field. https://en.wikipedia.org/wiki/AI_winter The plateau comes from the problems with vanishing gradients. https://en.wikipedia.org/wiki/Vanishing_gradient_problem
There's a long history of cycles in this field. https://en.wikipedia.org/wiki/AI_winter The plateau comes from the problems with vanishing gradients. https://en.wikipedia.org/wiki/Vanishing_gradient_problem