HN Explorer

Ask HN: How do you stop ChatGPT from being lazy?

6 points by FrenchDevRemote ↗ HN
I've noticed over time that ChatGPT 3.5 and 4 is now very likely to refuse to finish tasks if they're too long.

for example it will output things like:

User input example:Give me an object with 50 keys corresponding to X

GPT output:

{

   output1: "example 1",

   output2: "example 2",

   output3: "example 3",

   //etc....do the same as I did above for the 47 others keys you asked for
}

I've tried a lot of different prompts/custom instructions to try to force him to finish long tasks, but it will either do the same BS after a while, or crash.

I think OpenAI is pushing hard to make the ChatGPT outputs as small and cheap as possible(either by finetuning or prompting), that would be acceptable if it was still a free product, but I'm paying for Plus, so this is infuriating.

Do you have any methods to force him to complete tasks without skipping anything?

Thanks

9 comments

[ 3.0 ms ] story [ 41.2 ms ] thread
verdverm
Have you tried different prompts with more instructions, like "don't truncate or use "etc..." in your response"? Providing examples in the prompt typically helps as well

fwiw, the following prompt works with bard

"Give me a JSON object with 50 keys and values"

"Give me a JSON object with 50 keys and string values"

FrenchDevRemote
Like I said, yes, I've tried dozens of different prompts and custom instructions, most of the time he just ignores them, and refuses to go further after he made a few

It also happens when I ask him to write class or functions, he puts a lot of ellipsis/placeholders

verdverm
The are downsides to relying on OpenAI, maybe try a different model and ecosystem then?

Another option is codellama-instruct, the following works for me

`ollama run codellama:7b-instruct`

`Give me a JSON object with 50 keys and string values`

FrenchDevRemote
The prompt in my post was just an example it probably would work.(the prompts I actually used included proprietary/private code/data, so I cannot share them here).

Codellama is likely a lot less powerful than GPT.

verdverm
well, it seems like GPT is causing issues and there's only so much you can do

iirc, there is some analysis that the alignment training nerfs model capabilities, so we are likely unintentionally making the models less capable because... FUD?

> Codellama is likely a lot less powerful than GPT.

Certainly true, but is it capable enough for your task? You'd have to try and find out. There is more analysis showing that smaller models trained for a specific task outperform large, generalist model like GPT-3.5/4

verdverm
more interesting, the codellama:13b-instruct has the same elliding issue
beardyw
What would you do if someone asked that?
gogurt2000
Just a couple weeks ago it was discovered that you could get ChatGPT to regurgitate training data by having it repeat words forever (source: https://www.wired.com/story/chatgpt-poem-forever-security-ro...). Combined with the overall demand, I suspect they're trying to keep it from attempting repetitive tasks that could be exploited.

Some redditors have reported success in getting it to finish a job with replies like these, but ymmv: No, I'd like you to do what I asked. I’m not sure how to continue from there, can you finish the rest? this is very important for my career

I love the irony here. The sales pitch was that AI is so incredible that it would do our work with minimal effort on our part. No more coding! Just have a conversation with it about what you want done! But it turns out we have to try to trick or coerce it into doing work. Not because the underlying AI is self aware and lazy, but because the guide rails built to keep it from doing things it shouldn't are written in natural language and don't succinctly distinguish between wanted and unwanted behavior.

poobear22
I have seen that, where I want to just cut and paste the response but the return content has something like a one liner that states ("the rest of your code here") and I can't cut and paste it. I ask for a full return of content, but don't always get it.