apstroll
No user record in our sample, but apstroll has activity below (stories or comments). Likely we have partial data — the full bulk-load will fill profiles in.
No user record in our sample, but apstroll has activity below (stories or comments). Likely we have partial data — the full bulk-load will fill profiles in.
Extremely doubtful that it boils down to quadratic scaling of attention. That whole issue is a leftover from the days of small bert models with very few parameters. For large models, compute is very rarely dominated by…
Under a crossentropy loss the output activations do absolutely represent a probability distribution, since that is what we're modeling.
The output distribution is deterministic, the output token is sampled from the output distribution, and is therefore not deterministic. Temperature modulates the output distribution, but sitting it to 0 (i.e. argmax…
This paper is doing exactly that though, handwaving with a couple of floats. The paper is just a collection of observations about what their implementation of shapley value analysis gives for a few variations of a…
Cosine Similarity is very much about similarity, but it's quite fickle and indirect. Given a function f(l, r) that measures, say, the logprobability of observing both l and r, and that the function takes the form f(l,…