when there are two commands in a prompt example do A and then do B. the model completely ignores the second task B.
on gpu that is still huge.
super easy to install and use
mistral 7b v0.2 supports 32k
could be true, we can only speculate.
maybe they are using ring attention, on top of their 128k model.
really like the font and great article btw.
the paper does not live up to the quality of model lol
you need 94gb, does not matter which RAM.
https://github.com/sysid/sse-starlette makes token streaming so much easier in python
and also easy to deploy
its not alway about the size, but yeah its really good!
installing it is a nightmare
you could but it will be very slow. oh and make sure the code is cpu compatible.
when there are two commands in a prompt example do A and then do B. the model completely ignores the second task B.
on gpu that is still huge.
super easy to install and use
mistral 7b v0.2 supports 32k
could be true, we can only speculate.
maybe they are using ring attention, on top of their 128k model.
really like the font and great article btw.
the paper does not live up to the quality of model lol
you need 94gb, does not matter which RAM.
https://github.com/sysid/sse-starlette makes token streaming so much easier in python
and also easy to deploy
its not alway about the size, but yeah its really good!
installing it is a nightmare
you could but it will be very slow. oh and make sure the code is cpu compatible.