HN Explorer

3 comments

[ 0.23 ms ] story [ 19.0 ms ] thread
mezark
Falcon 7B running real time on CPU
smcleod
The linked video seems to have no context provided? What is a titan ML server? Is 7B actually that useful? How does the model compare to others? Etc…
mezark
Hey there - TitanML is these guys: https://www.titanml.co/ . I think the impressive thing isn't actually whether the model is good (although it is a good model especially when fine-tuned) - but how fast this model runs on CPU with the TitanML server compared with before.