Lets solidify definitions. A procedure is deterministic iff for all inputs, it always produces the same output on that input.
Now, I am going to be pedantic because words matter here. I agree with the author that LLMs have downsides that can be addressed with _symbolic_ tools. But _determinism_ has very little to do with this.
> LLMs by nature are non-deterministic
This is false. LLMs are functions. All appearances otherwise are an artifact of how we use them.
This fact already suggests that determinism isn’t (entirely) what you want. Because even if you _could_ use LLMs as functions (I admit you can’t always do this with frontier models), that wouldn’t make you happy.
> I want the output to always be predictable based on the behavior of the program and provided configuration.
Here, I will argue that predictability here is divorced from determinism. You want an output that has a certain semantic relationship with the input. E.g., if you give a spec as input, you want a program that satisfies this spec.
Here it should be obvious that getting the same output on the same input is not very important. Who cares if the arguments to the function are renamed? Who cares if the function is implemented differently but essentially does what the spec asks?
I argue the _only_ thing that matters is that the output satisfies the intended relationship with the input. And this is orthogonal to determinism.
Edit:
It looks like the author has the same realization:
> And while even this example shows how differently an LLM responds to the same query, it ends up producing a more reliable output.
1 comment
[ 3.0 ms ] story [ 14.4 ms ] threadNow, I am going to be pedantic because words matter here. I agree with the author that LLMs have downsides that can be addressed with _symbolic_ tools. But _determinism_ has very little to do with this.
> LLMs by nature are non-deterministic
This is false. LLMs are functions. All appearances otherwise are an artifact of how we use them.
This fact already suggests that determinism isn’t (entirely) what you want. Because even if you _could_ use LLMs as functions (I admit you can’t always do this with frontier models), that wouldn’t make you happy.
> I want the output to always be predictable based on the behavior of the program and provided configuration.
Here, I will argue that predictability here is divorced from determinism. You want an output that has a certain semantic relationship with the input. E.g., if you give a spec as input, you want a program that satisfies this spec.
Here it should be obvious that getting the same output on the same input is not very important. Who cares if the arguments to the function are renamed? Who cares if the function is implemented differently but essentially does what the spec asks?
I argue the _only_ thing that matters is that the output satisfies the intended relationship with the input. And this is orthogonal to determinism.
Edit:
It looks like the author has the same realization:
> And while even this example shows how differently an LLM responds to the same query, it ends up producing a more reliable output.