Differentiable approximations to the min and max operators (github.com) 3 points by aidanrocke 6y ago ↗ HN
[–] aidanrocke 6y ago ↗ tl;dr1. Within the context of optimisation, differentiable approximations of the min and max operators on R^n are very useful.2. However, in order for these approximations to be useful they must also be numerically stable.
[–] heartbeats 6y ago ↗ How does this stack up to the softmax[0] function, log(exp(n_1) + exp(n_2) + ... + exp(n_i))? Are they analytically equivalent?0: https://en.wikipedia.org/wiki/LogSumExp
2 comments
[ 1.9 ms ] story [ 12.9 ms ] thread1. Within the context of optimisation, differentiable approximations of the min and max operators on R^n are very useful.
2. However, in order for these approximations to be useful they must also be numerically stable.
0: https://en.wikipedia.org/wiki/LogSumExp