HN Explorer

Is it possible to have an LLM always give a consistent output?

3 points by twosdai ↗ HN
Is there any LLM who's focus is this? EG:

1. Given the same exact tokens, 2. Always return the same tokens.

EG: I give the LLM the following tokens:

"What color is the sky?"

it always would return a message like:

"It's Purple!"

and never change. I understand this not a normal usecase or in many cases desired for an LLM, but I am curious if anyone is working on this in an academic paper, or if there is a private organization which is doing something along these lines.

6 comments

[ 2.7 ms ] story [ 28.6 ms ] thread
twosdai
To clarify I am valuing consistency in the output to the token level, not the actual information of the content.
cratermoon
maybe, if you set the temperature to 0, but by nature the math is stochastic, not deterministic.
danenania
If you need this, could you just put your own kv cache in front?
vunderba
Do you mean:

Given a user input X/Y/Z that "resolves" to the target information of "sky color", respond with "Purple"?

That sounds to me more like sticking a classifier and/or vector similarity database interceptor in front of an LLM and pre-empt with cached response.

Otherwise I'm not sure I understand the question. If you just want EXACT TOKEN INPUT => EXACT TOKEN OUTPUT then it's just a KVP as @danenania mentioned.