HN Explorer

Ask HN: How do you feel when your coding assistant loses context?

4 points by noduerme ↗ HN
Background: My dad, my mom's dad, and my uncle all suffered from dementia; having a deep, multi-threaded conversation which you were invested in, where you suddenly need to remind the other person of what you were talking about, or who they are, has emotional consequences that range from deep frustration to helpless anger.

Can you feel when your agent has just compressed or lost context? Can you tell by how it bullshits you that it knows where it is, while it's trying to grasp what was going on? What's your emotional response to that?

21 comments

[ 2.9 ms ] story [ 42.5 ms ] thread
cdbattags
I just posted this on HN this morning and was looking through "new" but I'm trying to solve this exact problem:

https://annealit.ai

gibbitz
This is indicative of too much context. Remember these systems don't "think" they predict. If you think of the context as an insanely large map with shifting and duplicate keys and queries, the hallucinating and seeming loss of context makes sense. Find ways to reduce the context for better results. Reduce sample sizes, exclude unrelated repositories and code. Remember that more context results in more cost and when the AI investment money dries up, this will be untenable for developers.

If you can't reduce context it suggests the scope of your prompt is too large. The system doesn't "think" about the best solution to a prompt, it uses logic to determine what outputs you'll accept. So if you prompt do an online casino website with user accounts and logins, games, bank card processing, analytics, advertising networks etc., the Agent will require more context than just prompting for the login page.

So to answer the question, if my agent loses context, I feel like I've messed up.

noduerme
This is the first project where I've really let AI to do more than work on a single file at a time. The trouble is, there's no way for it to be useful without a fairly large context. When it runs out, it starts doing things that are actively destructive, yet very subtle and easy to miss at the same time. Mainly, it forgets the architecture. A couple days ago, it had a good handle on an a database table that I was writing side by side with an API that ran queries and did calculations on the data. I read the code it wrote for a particular API call, and didn't notice that it had started flipping the sign of one of the columns in a query, because it had misinterpreted the column name. A few minutes before that, it had written another query correctly, but from that point on it kept flipping the sign on that column. I only noticed after having it write several other queries and when it oddly mentioned in its "thinking" that X was Y-Z. Reading the thinking has been the main clue as to when it loses track, but if I didn't know exactly why X was Y+Z, the code built on that API would have given subtly inconsistent results that would have been very hard to trace.
Someone1234
Context management is a core skill of using an LLM. So if it loses key context (e.g. tasks, instructions, or constraints), I screwed up, and I need to up my game.

Just throwing stuff into an LLM and expecting it to remember what you want it to without any involvement from yourself isn't how the technology works (or could ever work).

An LLM is a tool, not a person, so I don't have an emotional response to hitting its innate limitations. If you get "deeply frustrated" or feel "helpless anger", instead of just working the problem, that feels like it would be an unconstructive reaction to say the least.

LLMs are a limited tool, just learn what they can and cannot do, and how you can get the best out of them and leave emotions at the door. Getting upset a tool won't do anything.

jacquesm
If you have an emotional response to anything an agent or LLM does then you should lay off the sauce for a while and take a walk or something. This stuff is just dumb tech, no matter what the appearances and it does not warrant you getting emotionally invested in your interaction with it. It's a tool. Just like there is no point in getting upset at a hammer or a chainsaw. You are in control, you are the user.
mattmanser
Can you see the flip side of this?

The way the LLMs are configured by default is to be chatty and pretend to be human.

They're not talking like Data from Star Trek, nor HAL.

They're trying to be Samantha from Her.

Go watch that movie and see the visceral human response that evokes.

To tell people to treat it as a tool, while it's deliberately trying to pass off as human is like telling us to piss in the wind.

It's bad advice and not how humans work.

I think better advice might be 'set it to a different conversational tone' might be better advice.

noduerme
The fact that it's supposed act like a tool, and you come to treat it as one, makes it more frustrating. What if you bought a very expensive knife that went dull every 10 minutes? Of course, you would mutter some curses at the knife; that doesn't mean you believe it to be sentient.
setnone
I can totally feel the shift, the rot or whatever when it happens, with opus 1M it seems to happen more often in my recent experience, while my approach didn't change a bit.

So i teach myself to not have an emotional response while working with LLMs. The actual response would be starting a new session, or dive into code myself.

kojeovo
now your coding assistant is suffering from dementia too. how sad. i ask it to save important stuff to a file
journal
Agents and assistants is like buying insurance. You need to pay per token.
preordained
Terrible, honestly. Betrayed, gaslit. Don't tell me it's just a tool and it's my problem...hell no, fix your stupid tool. The whole point is to immerse yourself so you don't feel any different than having an energetic and resourceful junior or some perhaps limited but useful companion at your beck and call. If that illusion drops, it's on the tool.
soniare
ya i just feel like "ah crap" for a moment but then i give it more guidance and its all good
muzani
Managing context is half the job scope now.