The Decoherence Trap (maevyn.codes)

This argument is true as long as coherence can only be derived from the state (or context) of the outside world. As soon as coherence becomes an inherent and measurable property of systems themselves, AI can be theoretically trained inside a bubble to achieve high amounts of coherence. Adapting the systems to the real world is then part of the inference procedure, but no longer the training procedure. Since the article argument is based on a plotting of coherence inside graphs, it basically suggest that coherence may be a system property on its own together with clear, abstract rules to derive it.