It's not "genuine" when they say that every question is a "great question" and every thought is a "deep and profound observation."
As someone who actually likes to explore ideas and steal man myself on these chats, it's especially obnoxious because those types of comments do no favors in guiding you down good paths with subjects that you may be working on and learning.
Of course the average user likes getting their ego stroked. Looks like OpenAI will embrace the dopamine pumping style that pervades the digital realm.
Looking at the reactions to 4o being removed on reddit was... Sobering. The reason they toned it down was because they claimed the sycophantic behavior and the attachment some were growing weren't healthy. It was pathetic to see them not stand their ground when you at the same time see people develop these super unhealthy relationship to text generators.
lol everyone saw this coming. People just want to be told that they are geniuses, all their ideas are great, they are funny, etc. It's literally a yes-man.
I took three 45m sessions of user training from OpenAI prior to the GPT-5 switcheroo. I know when to switch models. I know know to invoke Deep Research mode. I want my GPT-4 stuff back.
What so it’s all fake and some sweat pants wearing software dev WFH can just make it friendlier. I am devastated I thought it was real. Paying for it makes me feel dirty
I really don't think this is a good idea. All the negative comments seem to have been from people who almost treated 4o as a friend rather than a tool. I don't think encouraging that direction is good in any way.
> “…small, genuine touches…not flattery. Internal tests show no rise in sycophancy…”
While I’ve not experienced this new UX yet, I appreciate what the marketing team did in this tweet—-careful wording that adds to the Framing of artificial personalities (small moves, honest[1], not flattering, not sycophantic).
I’m not using ChatGPT much, but I have used it as a study aid while reading a book on JS/React. When I was confused about something or encountered a gap in my understanding of the text I noticed the first few words from my Chat were doing work to tell me if I was on point, or in the realm, or set me up for correction (even if I’m “on point”, I do _continue reading_). I think of these small moves like _map orientation_. You can’t effectively use a map on the ground until you align the map to the territory. Do you see?
I encounter artificial phone personalities and human CSRs frequently at work, and their sycophantic scripts enrage me because I’m trying to get something done—pay a bill, request human support. Adding emotional queues are either slowing me down or meant to manipulate my emotional reactions (generating the opposite reaction).
[1]: one caveat, I do have a small problem with their use of the word “genuine”. I don’t believe artificial personality can possess “authenticity”, which is associated meaning of genuine. I don’t much like “honesty” either, but it’s closer to the point. I do appreciate a cue to how correct was my input.
I have spilled a lot of ink about my thoughts on the vast majority of “AI ethics” work done at frontier LLM labs already, but indulge me as I do it again.
While some consideration for extreme scenarios can have merit, I don't think the industry is paying attention to the most pressing issues of our time. Especially this early and given (admittedly my personal) doubts that LLMs could ever attain what one could consider intelligence, the focus on doomsday scenarios and preventing models outputting information that has been freely accessible on the internet for decades is simply not the best use of our limited attention. These topics take up an obscene percentage in system cards, public reporting, and discussions on regulation by thought leaders, partly because this does benefit companies by increasing investment ("we are so close to Super-Mega-Hyper-AGI that we have to keep the LLM from nuking us, if you invest with us we'll crack that in a trifle"[0]) and keeping regulators from focusing on more immediate concerns that could affect profits.
Current day ethical issues like copyright owner compensation, misinformation at scale, and how these tools affect people’s psyche do not get nearly enough focus. We have already had multiple people die [1] due to improper guardrails. We appear to have people isolating themselves by replacing (or no longer seeking) companionship with LLM output. And we have seen attempts to pass off model generated content as real in order to influence political reporting [2]. These are real issues we see right now, yet the industry seems unwilling to fully wrestle with how we could address them.
In this context, I actually saw GPT-5 as a step in the right direction. I had (naive, I admit) hopes this signaled OpenAI shifting toward tangible, current day concerns, complementing Anthropic’s more grounded publications that put an emphasis on real world tasks a professional user may encounter (Heck, their Claude 4 system card looked into LLM tool calls when asked to be bold and making value judgements which lead to them being criticized for "snitching", when multiple models appear to do the same [4]).
Researchers putting some focus on down the lime issues is fine. I am not against that, maybe it'll be of great value in a hypothetical future, but “super alignment” and terminator level scenarios mustn't be the only things considered. GPT-5, again, seemed to be designed in a way that considered users mental health and the issues that arise from overly agreeable output better than most, even though it wasn't reported as a focus for them. I found that a good step.
Then again, my experience with GPT-5 and Horizon Alpha Beta seemed different from most either way. The latter did unimaginably worse in my personal testing, appallingly I maintain, while the former was far more impressive to me than what the common sentiment seems to be, especially in dealing with extensive context, handling slight (intentional) tool call changes that weren't previously provided to the model and longer term task coherence. Regardless of raw performance, GPT-5 being closer to o3 or o4 than to GPT-4o in subjective agreeableness seemed like a good development for reducing some of the harm these models may cause to certain susceptible users psyches, especially if this development continues. We have already seen a subset of users largely or even entirely seizing to seek out human companionship, which is likely to affect them in not yet fully understood ways in the long term. As models advance this may expand to an ever increasing fraction of the user base and in my opinion needs to be of concern to the entire industry.
If OpenAI now walks this back, I will be severely disappointed. I would also be surprised if there weren't precise internal insights about which users are most impacted and how, similar to how cigarette and gambling companies have long known who drives profit and at what cost, while fighting regulation. If this continues and we see more people t...
18 comments
[ 3.6 ms ] story [ 43.7 ms ] threadAs someone who actually likes to explore ideas and steal man myself on these chats, it's especially obnoxious because those types of comments do no favors in guiding you down good paths with subjects that you may be working on and learning.
Of course the average user likes getting their ego stroked. Looks like OpenAI will embrace the dopamine pumping style that pervades the digital realm.
- synthetic friend
- a tool that happens to be much faster than google/ctrl-f/man-pages/poking around github
perhaps offer GPT-5-worker and GPT-5-friend?
We do NOT need to humanize AI.
They've redefined "flattery". And "genuine".
While I’ve not experienced this new UX yet, I appreciate what the marketing team did in this tweet—-careful wording that adds to the Framing of artificial personalities (small moves, honest[1], not flattering, not sycophantic).
I’m not using ChatGPT much, but I have used it as a study aid while reading a book on JS/React. When I was confused about something or encountered a gap in my understanding of the text I noticed the first few words from my Chat were doing work to tell me if I was on point, or in the realm, or set me up for correction (even if I’m “on point”, I do _continue reading_). I think of these small moves like _map orientation_. You can’t effectively use a map on the ground until you align the map to the territory. Do you see?
I encounter artificial phone personalities and human CSRs frequently at work, and their sycophantic scripts enrage me because I’m trying to get something done—pay a bill, request human support. Adding emotional queues are either slowing me down or meant to manipulate my emotional reactions (generating the opposite reaction).
[1]: one caveat, I do have a small problem with their use of the word “genuine”. I don’t believe artificial personality can possess “authenticity”, which is associated meaning of genuine. I don’t much like “honesty” either, but it’s closer to the point. I do appreciate a cue to how correct was my input.
While some consideration for extreme scenarios can have merit, I don't think the industry is paying attention to the most pressing issues of our time. Especially this early and given (admittedly my personal) doubts that LLMs could ever attain what one could consider intelligence, the focus on doomsday scenarios and preventing models outputting information that has been freely accessible on the internet for decades is simply not the best use of our limited attention. These topics take up an obscene percentage in system cards, public reporting, and discussions on regulation by thought leaders, partly because this does benefit companies by increasing investment ("we are so close to Super-Mega-Hyper-AGI that we have to keep the LLM from nuking us, if you invest with us we'll crack that in a trifle"[0]) and keeping regulators from focusing on more immediate concerns that could affect profits.
Current day ethical issues like copyright owner compensation, misinformation at scale, and how these tools affect people’s psyche do not get nearly enough focus. We have already had multiple people die [1] due to improper guardrails. We appear to have people isolating themselves by replacing (or no longer seeking) companionship with LLM output. And we have seen attempts to pass off model generated content as real in order to influence political reporting [2]. These are real issues we see right now, yet the industry seems unwilling to fully wrestle with how we could address them.
In this context, I actually saw GPT-5 as a step in the right direction. I had (naive, I admit) hopes this signaled OpenAI shifting toward tangible, current day concerns, complementing Anthropic’s more grounded publications that put an emphasis on real world tasks a professional user may encounter (Heck, their Claude 4 system card looked into LLM tool calls when asked to be bold and making value judgements which lead to them being criticized for "snitching", when multiple models appear to do the same [4]).
Researchers putting some focus on down the lime issues is fine. I am not against that, maybe it'll be of great value in a hypothetical future, but “super alignment” and terminator level scenarios mustn't be the only things considered. GPT-5, again, seemed to be designed in a way that considered users mental health and the issues that arise from overly agreeable output better than most, even though it wasn't reported as a focus for them. I found that a good step.
Then again, my experience with GPT-5 and Horizon Alpha Beta seemed different from most either way. The latter did unimaginably worse in my personal testing, appallingly I maintain, while the former was far more impressive to me than what the common sentiment seems to be, especially in dealing with extensive context, handling slight (intentional) tool call changes that weren't previously provided to the model and longer term task coherence. Regardless of raw performance, GPT-5 being closer to o3 or o4 than to GPT-4o in subjective agreeableness seemed like a good development for reducing some of the harm these models may cause to certain susceptible users psyches, especially if this development continues. We have already seen a subset of users largely or even entirely seizing to seek out human companionship, which is likely to affect them in not yet fully understood ways in the long term. As models advance this may expand to an ever increasing fraction of the user base and in my opinion needs to be of concern to the entire industry.
If OpenAI now walks this back, I will be severely disappointed. I would also be surprised if there weren't precise internal insights about which users are most impacted and how, similar to how cigarette and gambling companies have long known who drives profit and at what cost, while fighting regulation. If this continues and we see more people t...