Tool calling at Q8 for this model is kinda impressive. It's doing patch edits like a champ. It gets stuck looping and doing the wrong things though as the context window grows.
Hi! This is a common issue regarding sampling parameters that are off. I imagine though that you are a smart cookie and read the model card, where it spoon feeds you the sampling parameters lol, so if it isn't this, could you please share more? I am one of the creators of this model :)
How much does it actually matter? I use to bother with these settings, but never really seems to fix the problems I commonly see. The issues I commonly see are small models getting stuck in a repeating loop, or they just fail at follow up calls for editing.
3 comments
[ 3.4 ms ] story [ 17.5 ms ] thread