Great explanation, but the last question is quite simple. You determine the weights via brute force. Simply running a large amount of data where you have the input as well as the correct output (handwriting to text in this case).
Lovely visualization. I like the very concrete depiction of middle layers "recognizing features", that make the whole machine feel more plausible. I'm also a fan of visualizing things, but I think its important to appreciate that some things (like 10,000 dimension vector as the input, or even a 100 dimension vector as an output) can't be concretely visualized, and you have to develop intuitions in more roundabout ways.
I hope make more of these, I'd love to see a transformer presented more clearly.
Nice visuals, but misses the mark. Neural networks transform vector spaces, and collect points into bins. This visualization shows the structure of the computation. This is akin to displaying a Matrix vector multiplication in Wx + b notation, except W,x,and b have more exciting displays.
It completely misses the mark on what it means to 'weight' (linearly transform), bias (affine transform) and then non-linearly transform (i.e, 'collect') points into bins
I agree. This visualization gets the basic idea across, but it doesn't actually tell you how they are implemented mathematically.
It doesn't tell you that each neuron calculates a dot product of the input and neuron weights and that the bias is simply added rather than a threshold, nor does it tell you that there is an activation function that acts as a differentiable threshold.
Without this critical information there is no easy way to explain how to train a neural network since you can't use gradient descent anymore. You're forced to use evolutionary algorithms for non-differentiable networks.
Oh wow, this looks like a 3d render of a perceptron when I started reading about neural networks. I guess essentially neural networks are built based on that idea? Inputs > weight function to to adjust the final output to desired values?
I have a question. With the logic of neural networks, and pattern recognition, is it not then possible to "predict" everything in everything? Like predicting the future to an exact "thing"? Is this not a tool to manipulate for instace the stock market?
This is old. Perhaps late 90s or early 00. The top domain still uses Flash. But the same OCR example is used to teach the concept. For some reason, that site made it all click for me.
30 comments
[ 2.4 ms ] story [ 60.1 ms ] threadIf you want to understand neural networks, keep going.
I hope make more of these, I'd love to see a transformer presented more clearly.
Don't think it's moire effect but yeah looking at the pattern
It completely misses the mark on what it means to 'weight' (linearly transform), bias (affine transform) and then non-linearly transform (i.e, 'collect') points into bins
It doesn't tell you that each neuron calculates a dot product of the input and neuron weights and that the bias is simply added rather than a threshold, nor does it tell you that there is an activation function that acts as a differentiable threshold.
Without this critical information there is no easy way to explain how to train a neural network since you can't use gradient descent anymore. You're forced to use evolutionary algorithms for non-differentiable networks.
https://mlu-explain.github.io/neural-networks/
- make a visualization of the article above and it would be the biggest aha moment in tech
http://www.ai-junkie.com/ann/evolved/nnt1.html
This is old. Perhaps late 90s or early 00. The top domain still uses Flash. But the same OCR example is used to teach the concept. For some reason, that site made it all click for me.