HN Explorer

Blog on Optimization for ML

2 points by alexshtf ↗ HN
Hi all,

I am writing a blog (https://alexshtf.github.io) on optimization for ML. I would like it not to be 'yet another explanation on SGD', so I write about other more interesting stuff. The idea is to share advanced optimization topics with the ML community in an accessible manner.

The first series is about the stochastic proximal point method, its strengths, and it's Python implementation.

I hope you enjoy! Please share your feedback.

1 comment

[ 2.7 ms ] story [ 10.9 ms ] thread
Eridrus
I appreciate you trying to make more content beyond the usual intro content more accessible, but if you really are trying to share things in an accessible manner, you probably need to lose the "Get used to it" attitude that you wrote in your first post and actually think about who exactly your audience is and try to tailor your content to the audience you are targeting.

As a practitioner, I would prefer a very different structure to what you have, because there is a whole lot of math, my first question is: why should I care? I had a look over the experiment section, which gives me some sense that this might be less sensitive to the learning rate, but otherwise no real reason to keep reading.

Try to get somebody who you think this is for to read the content because otherwise you're going to write a bunch of things that make sense to you and your peers but are less useful for those with less background.

It's a lot of effort to write things well for beginners, by definition they can't fill in the gaps of what you wrote with knowledge; I don't know what H(x) here is; is it the Hessian? That doesn't seem right, but I have no other real guesses. I also don't know Fermat's principle or the Sherman-Morrison matrix inversion formula.