How can maintainers of regulated and critical systems put AI agents into their workflows while keeping behavior deterministic? Today we launch the Agentic Determinism Index (ADI) in open source. We are calling for the agentic community to contribute .
We often hear "LLMs are non-deterministic”. That is a misconception.
At temperature 0, the model itself is deterministic: same input, same output.
The variance comes from the stack serving it: hardware type, OS version, CUDA/driver version, model version, batch composition, autotuners picking different kernels, sampler settings, and silent infrastructure updates. Any of those can change your system's behavior without you touching a line of code.
TL;DR: if you run your agents in the cloud, they will most probably give you non-deterministic outputs.
This post covers how maintainers can select deterministic stacks and detect when the stack they chose shifts underneath them with the ADI. The ADI is an open-source harness that asks one narrow question of every major provider: send the identical request N times, concurrently and across days. It fingerprints and scores the outputs so maintainers get what they need to monitor their stack.
1 comment
[ 0.29 ms ] story [ 10.8 ms ] threadWe often hear "LLMs are non-deterministic”. That is a misconception. At temperature 0, the model itself is deterministic: same input, same output.
The variance comes from the stack serving it: hardware type, OS version, CUDA/driver version, model version, batch composition, autotuners picking different kernels, sampler settings, and silent infrastructure updates. Any of those can change your system's behavior without you touching a line of code.
TL;DR: if you run your agents in the cloud, they will most probably give you non-deterministic outputs.
This post covers how maintainers can select deterministic stacks and detect when the stack they chose shifts underneath them with the ADI. The ADI is an open-source harness that asks one narrow question of every major provider: send the identical request N times, concurrently and across days. It fingerprints and scores the outputs so maintainers get what they need to monitor their stack.