> the community trusts that users are submitting answers that reflect what they actually know to be accurate and that they and their peers have the knowledge and skill set to verify and validate those answers.
this confidence is not fully achieved in most answers. It's very likely to come across many answers that don't work at all. and these answers, oddly enough, can get so many points. It is debatable how good he is in terms of trust.
"because GPT is good enough to convince users of the site that the answer holds merit, [the] signals the community typically use to determine the legitimacy of their peers’ contributions frequently fail to detect severe issues with GPT-generated answers. As a result, information that is objectively wrong makes its way onto the site."
Basically it is much faster to generate incorrect answers algorithmically than it is for humans to evaluate their accuracy and flag them.
This is what happens next. The web becomes full of AI-generated stuff, be it art or text or whatever, and people scrape that and train models and it starts consuming its own shit and regurgitating the same kind of stuff. And then human-created content becomes valuable again, and people find new sources of inputs, etc and we go on..
And obviously platforms will be built for creators/artists/writers/likenesses/whatever to be used by AI (and they get paid for training data) if they’re not being built already.
They won't. It is easy to detect AI generated answers (link in the parent thread).
Also the algorithms for detecting fake facts (e. g. "politician_name was born at <random date>") also exist; for knowledge graphs like Google Knowledge Graph it is fully automated; if a website consistently outputs answers that contradict other sources, GKG increases penalty coefficients for all facts from this source.
Given that people already know that GPT derivatives are inaccurate and given that it is trivial to detect it (see https://huggingface.co/openai-detector/), I don't think it is true.
How does Stack Overflow know the answer is generated by ChatGPT?
Also wouldn't they be better suited by addressing incorrect answers rather than who or what created the answer? If they are concerned about incorrect answers they have to concede that humans provide incorrect answers as well.
Part of the problem is ChatGPT’s incorrect answers are so well formed they seem more correct than the average incorrect human answer which is usually short, lazy, not even related to the question, contains no explanation, etc.
ChatGPT creates long logical answers often with explanations and the errors are very subtle. If it were let to run wild across stack overflow it would take a lot more effort to weed them out.
It's worse than that, ChatGPT's answer are superficial on a fundamental level.
Earlier today, I asked it to produce a complicated piece of code in a niche language, and it dutifully did so - but the code had a type error, and used a reserved keyword as a function name.
When I asked it to rename the function, it did so successfully, including usages and comments. (So it didn't totally treat the code piece as opaque.)
But when I told it, "there is a type error here in the function X, the type should be (this)" it replied: "You are right, I fixed the mistake. Here is the code but with the function X having type Y", followed by the original code verbatim with no changes whatsoever.
This really helped me understand what these text generation AIs do. They try to produce a text that looks like a correct answer, which may or may not be correct.
In my example, it clearly didn't have nearly enough training on the specific language and programming style to decompose and "tune" the code to the right answer. But it had a lot of training in saying "right, I fixed the bug, here's the correct code", whether or not it was true.
13 comments
[ 3.5 ms ] story [ 45.7 ms ] threadthis confidence is not fully achieved in most answers. It's very likely to come across many answers that don't work at all. and these answers, oddly enough, can get so many points. It is debatable how good he is in terms of trust.
Basically it is much faster to generate incorrect answers algorithmically than it is for humans to evaluate their accuracy and flag them.
Also the algorithms for detecting fake facts (e. g. "politician_name was born at <random date>") also exist; for knowledge graphs like Google Knowledge Graph it is fully automated; if a website consistently outputs answers that contradict other sources, GKG increases penalty coefficients for all facts from this source.
ChatGPT creates long logical answers often with explanations and the errors are very subtle. If it were let to run wild across stack overflow it would take a lot more effort to weed them out.
Earlier today, I asked it to produce a complicated piece of code in a niche language, and it dutifully did so - but the code had a type error, and used a reserved keyword as a function name.
When I asked it to rename the function, it did so successfully, including usages and comments. (So it didn't totally treat the code piece as opaque.)
But when I told it, "there is a type error here in the function X, the type should be (this)" it replied: "You are right, I fixed the mistake. Here is the code but with the function X having type Y", followed by the original code verbatim with no changes whatsoever.
This really helped me understand what these text generation AIs do. They try to produce a text that looks like a correct answer, which may or may not be correct.
In my example, it clearly didn't have nearly enough training on the specific language and programming style to decompose and "tune" the code to the right answer. But it had a lot of training in saying "right, I fixed the bug, here's the correct code", whether or not it was true.