szcs
No user record in our sample, but szcs has activity below (stories or comments). Likely we have partial data — the full bulk-load will fill profiles in.
No user record in our sample, but szcs has activity below (stories or comments). Likely we have partial data — the full bulk-load will fill profiles in.
That's just the same distribution laid out along a line instead of a circle
Attention is a 3 matrix product, s(QK)V where s is softmax. Each matrix has as many rows (Q and V) or columns (K) as many tokens you have in your context. The plot looks at the processing of a single row of Q…
There is a particularly nice geometric interpretation of attention I just realised recently in a flash of enlightenment, best explained with an interactive Desmos plot (black dot is draggable):…
Yes, LLMs are planned too.
Author here, I just noticed this. If you have any questions I can try answering them.
Here is a quick graph I just generated from data I had. Horizontal axis is average bit depth, vertical is accuracy. PTQ is Post Training Quantisation, QAT is Quantisation Aware Training. It's a simple ResNet trained on…
Author here. I actually came up with the idea a long time ago, I first experimented with variants of this in Caffe (before Tensorflow was a thing).