I have a simple question. Suppose that to answer a question I can use different phrases, I know the answer but I have several ways to express it. Then a LLM in this case produces tokens with high or low entropy? Edited…
Mathematica has thousand of functions, this seems to be just an algorithm for defining and using rules like that in mathematica.
So it seems that when you have different sources of errors the average of them cancel the noise. I think some property about the sources of error is necessary so in some sense the sources should be independent.
I have a simple question. Suppose that to answer a question I can use different phrases, I know the answer but I have several ways to express it. Then a LLM in this case produces tokens with high or low entropy? Edited…
Mathematica has thousand of functions, this seems to be just an algorithm for defining and using rules like that in mathematica.
So it seems that when you have different sources of errors the average of them cancel the noise. I think some property about the sources of error is necessary so in some sense the sources should be independent.