HN Explorer

42 comments

[ 3.1 ms ] story [ 53.5 ms ] thread
danso
At the risk of being pedantic...if you're not familiar with data/math work in Python, the `np` word refers to "numpy", which is an extension to Python that includes array and matrix math...so the OP needs 12 lines, the first being:

       import numpy as np
Bostonian
I knew that np stood for numpy but had to Google to realize that the .T suffix meant transpose.
afarrell
That isn't pedantic at all. Python tutorials should always include imports.
ksenzee
No, definitely not pedantic. In fact, I have only just now realized that "numpy" means "NumPy," instead of being a nonsense word that rhymes with "lumpy." Sigh.
Namrog84
I will still always call it nump-pee, and never num-pie

Lumpy Numpy FTW

sciguy77
Me too! It drives professors nuts, but its just so much more fun to say.
blazespin
... Isn't grammatically correct. Pedantic, I know.
avyfain
Thanks for this. It is really well explained. However, it did feel like it went downhill when you explained how the updating happens. At least it was less of a walkthrough than the previous sections.

Also, lines 22 and 23 in the three layer network are not intuitive for a beginner. Before, syn0 was (3,1) and now its (3,4), and the new syn1 is (4,1). Your previous explanation for this initialization was much better.

But again, awesome work, just being nitpicky. Really appreciate it!

williamtrask
Hey avyfain, I'll look out for that in my next post. Thanks for the note.
BrainInAJar
This is a really fantastic article, I've been looking for a code-based "Neural network implementation for idiots" thing like this.
elyase
Just for comparison this is almost the same using a library [1]:

  import numpy as np
  X = np.array([ [0,0,1],[0,1,1],[1,0,1],[1,1,1]])
  y = np.array([[0,0,1,1]]).T

  from keras.models import Sequential
  from keras.layers.core import Dense

  model = Sequential([Dense(3, 1, init='uniform', activation='sigmoid')])
  model.compile(loss='mean_absolute_error', optimizer='sgd')
  model.fit(X, y, nb_epoch=10000)
  model.predict(X)
[1] http://keras.io
avyfain
But this is not nearly as clear for a beginner, or someone who doesn't know what Sequential, compile, fit and predict are doing.
nudpiedo
But this is the tested, maintained solution by experts in the topic. Hopefully.
w23j
"Do Not Reinvent The Wheel Unless You Plan On Learning More About Wheels"
blazespin
This is amazing stuff. Just want more!
compostor42
Can HN recommend any good neural net tutorials? (Preferably in Python)
[dead] rolandnsharp
Could someone please translate this into JavaScript for me. Or link me to Neural Network resources for JavaScript. Thanks.
ratsimihah
The final "Line 36" in your explanation should be "Line 39."
mjcohen
Interesting fact about the sigmoid function: If you scale it so its slope at the origin is the same as the cumulative error function, it differs from that function by at most 0.017.
bakul
Some of you may like seeing the two level neural network code translated to kona (or k3)!

    nonlin:{:[y;x*1-x;1%1+_exp -x]}
    x:(0 0 1;0 1 1;1 0 1;1 1 1)
    y:+,0 0 1 1
    syn0:-0.5+3 1 _draw 0
    step:{L1:nonlin[x _mul syn0;0]
          L1err:y-L1
          L1delta:L1err*nonlin[L1;1]
          syn0+:(+x)_mul L1delta;L1}
    do[10000;L1:step[x;y]]
    L1
  (,0.009668795
   ,0.007862982
   ,0.9935916
   ,0.9921171)
k3 does this in 140ms, Kona in 2030ms on the same machine.
robmcm
I don't get the point of these "x in n lines" posts.

Fewer lines of code doesn't always equate to simple, comprehensible code.

choudanu4
Well, I personally presumed that neural networks would require two or three orders of magnitude more of code. When its revealed that the code is actually much more manageable, individuals (such as myself) take the dive into the subject, knowing that it is something we can at least get through.
toxik
I wonder if it'd work better with a leaky ReLU. Somebody wanna try? It's a _very_ simple mechanism. [1] Basically, instead of a sigmoid, let f(z·w) be the activation due to inputs z and weight w, then a(z) = z·w if z·w > 0 else 0.01 * z·w. Sigmoid for activation have several problems, most notably they suffer from vanishing gradient problems with bigger networks.

[1] https://en.wikipedia.org/wiki/Rectifier_(neural_networks)#Le...

shubhamjain
Great work! I have been reading a practical guide to neural networks [1], and I am amazed that it is so comprehensible. I had assumed ML to be a lot more complicated. Although, I won't classify it as an easy craft but (maybe) it isn't so hard as it looks.

[1]: http://neuralnetworksanddeeplearning.com/chap1.html

arpgy
Shouldn't the derivative of sigmod(x) be sigmoid(x)(1-sigmoid(x)) and not x(1-x). The code seems to implement the second.