so turns out you can beat entire teams with a few hundred dollars and claude, who knew

thimble is a model i spent the last week of my evenings trying to train in order to beat needle 2 -- a 45M param tool-calling model trained on 153+ billion tokens by a funded team (u/cactuscompute)

my model: 48M params, ~1 billion tokens, $260 total (so 150x less training data)

the model wins 3 of their 5 published benchmarks, including seal-tools, the headline suite of their eval and the hardest one to move (so not a complete victory but still incredibly impressive results if i have to say so myself)

weights, code and results from every failed experiment is public -- and tbh all the fuck ups were the best part of the whole process, learnt a lot about synth data, model arch and training along the way

code: https://github.com/nikshepsvn/thimble weights: https://huggingface.co/flashvenom/thimble