in my chat with gemini it could not differentiate between current events and fiction.
if you point it to the web it got the point, but started treating everything like fiction. so it simply started making up possible scenarios and playing them off as real answers when asked for factual information.
i could not tell what the issue was or how to fix it because the reasoning is encrypted. the obfuscation model spat out something like: 'the user is asking for details about a fictional scenario in which the usa has assassinated the leader of iran'
i really don't like the way big ai companies are going. encrypted thinking, guardrails, adversarial personality, moralizing. it is creating something anti-human.
It can be quite hard to determine what needs a tool call or not. LLMs are not well calibrated to what they know and don't know, and tool calls can add latency and extra costs. There are lots of things that are "obvious" right until they aren't - especially political events and disasters.
After Trump's last inauguration, ChatGPT would still tell me that Biden was President of the US. I understand that the training cutoff was before Biden dropped out. But it knew, or should have known, the current date and that there had been an election since its last update, but it didn't qualify the answer. When I asked it to search the web, it got it right. The moral I took away was to always ask for the search when ever I ask about current events. I do that so routinely I that wouldn't know if this problem has been fixed. I suppose that failing to update my priors per individual model is a form of bigotry against a hated class.
ChatGPT recently started web searching for for basically every general knowledge question, which I found quite odd. Maybe an overcorrection to the issue you were having?
I forgot which was it, ChatGPT or Gemini, but one of them insisted on calling Trump "former president" even when discussing decisions he just announced as president. Lol
The Trump and former president terms were likely firmly stuck together in the embedding space. The model doesn’t validate every single token it produces because validation itself requires tokens. A bloom filter of outdated embeddings will help, when the labs get around to adding it.
I remember when the US captured Venezuelan president Maduro, and when I posed a prompt related to this, the model said that’s pure fiction. I told it to double check. Still didn’t want to entertain the idea. It only acquiesced when I specifically directed it to check Reuters. I haven’t noticed this problem in months. Model cutoff seems to be less of a problem these days.
Were you expecting your model to be updated on current events? Why?
Also the specific event you are referring to is a statistically very improbable event, prior to its actually happening.
>It only acquiesced when I specifically directed it to check Reuters.
Do all models do this? They check in with Reuters? Why would a model think that you asking about an extremely improbable event warranted reaching out to Reuters?
I was not expecting model weights to be updated on current events.
It’s clearly warranted because a model that trusts its weights on current events will give an outdated answer. Extremely improbable events happen all the time.
If OpenAI is going to call Astra AGI, then I would expect it to be able to update it's weights to new knowledge, because a generally intelligent being can indeed do this.
I can teach myself to play an instrument, and I'm not just building this huge lookup table that I have to access every time I play the instrument. I am updating the weights in my neurons.
Gets me with AWS stuff on claude all the time, fortunately there's a official amazon MCP for their docs which helps a lot, but I still have to occasionally tell it to check the docs/mcp.
This is likely being solved with stuff like watermarking which Anthropic just added to Claude... I'd imagine they are testing new data and verifying that training data (or at least the data which do NOT want to be AI generated, i.e actual human text) is not generated, at least with claude.
This is one of the jagged mismatches between users and LLM developers. The median user doesn't care or want to know about static models and knowledge cutoffs and whether a model can do tool calls or if tool calls even happened. They just want something that works.
Fortunately increased capabilities seem to make this a basic expectation with new releases.
This is one of the things that bothers me about AI.
To me, intelligence or an intelligent entity should be able to learn from its mistakes and learn new things on its own. Having to start from scratch to teach an AI new facts or new skills is not very intelligent IMO.
For general purpose use this is interesting, but if I'm just using an LLM for coding, does this matter at all? I would hope something like a new java version after a model's publish date can be handled and understood by the model through tool calls and context even if it's not explicitly in the training data, the same way the LLM doesn't have my existing code or the plan to change it baked in from training.
My experience with Claude is that it doesn’t handle its own training cutoff properly. It responds as though its cutoff is today. Gemini is even worse about this.
I’m going to add this site to my agents files so that they’re explicitly aware of their own limitations.
Qwen-3.8 for me automatically set copyright footer on a website to 2025 and thought Astro 5.x is the latest version which first came out in December 2024
37 comments
[ 0.43 ms ] story [ 102 ms ] threadif you point it to the web it got the point, but started treating everything like fiction. so it simply started making up possible scenarios and playing them off as real answers when asked for factual information.
i could not tell what the issue was or how to fix it because the reasoning is encrypted. the obfuscation model spat out something like: 'the user is asking for details about a fictional scenario in which the usa has assassinated the leader of iran'
i really don't like the way big ai companies are going. encrypted thinking, guardrails, adversarial personality, moralizing. it is creating something anti-human.
Were you expecting your model to be updated on current events? Why?
Also the specific event you are referring to is a statistically very improbable event, prior to its actually happening.
>It only acquiesced when I specifically directed it to check Reuters.
Do all models do this? They check in with Reuters? Why would a model think that you asking about an extremely improbable event warranted reaching out to Reuters?
It’s clearly warranted because a model that trusts its weights on current events will give an outdated answer. Extremely improbable events happen all the time.
I can teach myself to play an instrument, and I'm not just building this huge lookup table that I have to access every time I play the instrument. I am updating the weights in my neurons.
Until AI can do this it's not AGI in my book.
The slop would multiply if we keep feeding it to new models in a loop
it must ship with some default old model if you didn't need to explicitly download one
Fortunately increased capabilities seem to make this a basic expectation with new releases.
To me, intelligence or an intelligent entity should be able to learn from its mistakes and learn new things on its own. Having to start from scratch to teach an AI new facts or new skills is not very intelligent IMO.
I’m going to add this site to my agents files so that they’re explicitly aware of their own limitations.