Conspiracies aside, there are a lot of reasons for this. Reducing liabilities. Responding to complaints. The "Shiny new thing" effect reducing over time.
This is going to be the biggest problem with general AI chat systems in the future, inconsistency. Often, they are too complex to know precisely how they work. Small tweaks will break some use cases while improving others. As more complexity is added to fix the next series of bugs, the systems will become less effective overall while user's have their workflows broken then mysteriously fixed.
From the end-users perspective, consumers will lose trust and move on.
Do we expect them to leverage regression tests? I would think they'd have like 100 prompts from 100 categories, set a low temperature, then run the queries against expected responses.
I’m not sure about dumber, but definitely differeny. (i’m using chatGpt 4). One area that seems worse is rewriting to improve spelling etc. It seems that it now often switches to another word for no reason. The text doesn’t get any better from this odd rewrites. It likes to pick fancy academic words instead the words I had chosen. If you tell it sternly in your prompt not to change words unless they are clearly wrong, it will comply though
7 comments
[ 5.1 ms ] story [ 29.6 ms ] threadThis is going to be the biggest problem with general AI chat systems in the future, inconsistency. Often, they are too complex to know precisely how they work. Small tweaks will break some use cases while improving others. As more complexity is added to fix the next series of bugs, the systems will become less effective overall while user's have their workflows broken then mysteriously fixed.
From the end-users perspective, consumers will lose trust and move on.
These systems are actively studied, training one area might degrade another area without any obvious degradation overall.
Individual users targeting specifics in knowledge will amplify the "it's getting dumber" signal overall.
It has no intentionally, its utterances are not so much as about things, just likely strings of words from its training set.
They can't sensibly limit the meaning of what it says, because LLMs don't so much as have the concept of meaning. They basically have to ban tokens.
https://arxiv.org/pdf/2307.09009.pdf