LLM function calling. Fine tune or use context free grammars, what is better?

1 points by Roark66 ↗ HN
I wish there was some research on this. I'm using local models for function calling and often the most capable models (like mixtral 7x8) answer really complex questions correctly and then make a typo or a silly error like trying to escape an underscore in json variable name fields (sometimes twice). Lowering parameters fixes it mostly, but also causes the model to be a lot less capable.

So I became interested in improving the basic function calling by using context free grammars. The question is, is it a waste of time? Is the model going to loose its best capabilities by being forced into this kind of format "artificially"?

0 comments

[ 4.2 ms ] story [ 7.1 ms ] thread

No comments yet.