How do our base reptilian brains reason? We don't know the specifics, but unless it's magic, then it's determined by some kind of logic. I doubt that logic is so unique that it can't eventually be reproduced in…
This is a great idea but it is only possible if the model(s) can actually reason. Currently, even GPT-4 struggles with: - Scope - Abduction (compared to deduction and induction which it appears already capable of) -…
You were safe until you shared that with us, now they can just include your keyboard setup as an input embedding if they target you.
How would this scale for a use case like writing code? I could imagine that some inputs would require a large number of neurons. Would this architecture be able to do that if it were scaled up? I'm also curious if this…
For chess engines such as Stockfish to beat weaker opponents, you would want to turn up the contempt parameter.[1] To be able to do this, you would want to use a UCI-compliant chess GUI such as Cute Chess.[2] It lets…
It appears that RAG actually dominates for 2k context lengths compared to this method, but that this method outperforms it more and more the longer the context gets (see the graph titled "Retrieval Benchmark Results, by…
I disagree. Mistral actually being open source, as well as the new criteria for how open source models are, is pretty worth being aware of. Plus, if they make a larger Mistral model that's the same level of performance…
How do our base reptilian brains reason? We don't know the specifics, but unless it's magic, then it's determined by some kind of logic. I doubt that logic is so unique that it can't eventually be reproduced in…
This is a great idea but it is only possible if the model(s) can actually reason. Currently, even GPT-4 struggles with: - Scope - Abduction (compared to deduction and induction which it appears already capable of) -…
You were safe until you shared that with us, now they can just include your keyboard setup as an input embedding if they target you.
How would this scale for a use case like writing code? I could imagine that some inputs would require a large number of neurons. Would this architecture be able to do that if it were scaled up? I'm also curious if this…
For chess engines such as Stockfish to beat weaker opponents, you would want to turn up the contempt parameter.[1] To be able to do this, you would want to use a UCI-compliant chess GUI such as Cute Chess.[2] It lets…
It appears that RAG actually dominates for 2k context lengths compared to this method, but that this method outperforms it more and more the longer the context gets (see the graph titled "Retrieval Benchmark Results, by…
I disagree. Mistral actually being open source, as well as the new criteria for how open source models are, is pretty worth being aware of. Plus, if they make a larger Mistral model that's the same level of performance…