You're right that if the activation function is linear, like the identity function, then it doesn't matter how many layers you have. But with the step function two layers is enough. We can manually derive a network that…
Nice list and history of common activation units used today. Small note though, the heaviside function used in the the perceptron is non-linear (it can tell you which side of a plane the input point lies), and a…
That's not a bad way of putting it. It reminds me of "It is the user who should parameterize procedures, not their creators."
You're right that if the activation function is linear, like the identity function, then it doesn't matter how many layers you have. But with the step function two layers is enough. We can manually derive a network that…
Nice list and history of common activation units used today. Small note though, the heaviside function used in the the perceptron is non-linear (it can tell you which side of a plane the input point lies), and a…
That's not a bad way of putting it. It reminds me of "It is the user who should parameterize procedures, not their creators."