Yeah but I think this accelerator is more meant for a small scale deployment. This could probably serve a medium moe llm model at reasonable tk/s plus you could in addition to the llm also run a couple of other models…
could not agree more one pet peeve of mine is that using strict json output also does not work on all the endpoints of the models so I have to go through them one by one try it out and then only whitelist them.
It gets even more interesting if you take into account how the satellites know where they are. Around the world there are fundamental stations: https://en.wikipedia.org/wiki/Fundamental_station I had the opportunity to…
Yeah but I think this accelerator is more meant for a small scale deployment. This could probably serve a medium moe llm model at reasonable tk/s plus you could in addition to the llm also run a couple of other models…
could not agree more one pet peeve of mine is that using strict json output also does not work on all the endpoints of the models so I have to go through them one by one try it out and then only whitelist them.
It gets even more interesting if you take into account how the satellites know where they are. Around the world there are fundamental stations: https://en.wikipedia.org/wiki/Fundamental_station I had the opportunity to…