Use of ChatGPT generated text for posts on Stack Overflow is temporarily banned (meta.stackoverflow.com)

well, nothing stops google from baking this into their search. But it feels harsh to claim SO is finished. SO is human curated and has an edge (today). And for all we know, chatGPT learnt from SO and can synthesize results that are almost good.

Google is most likely already working on its own version. They have the resources and ML chops to probably build one far more powerful. I'd bet they'll stay dominant

I don't think so, especially for StackOverflow. I want correct answers, not answers that just look correct.

"Finished" and "less than 15 years" don't quite go together, I think.

To be fair, Google is working on something very similar: https://blog.google/technology/ai/lamda/

Given Google's resources and scale, I'd wager that whatever they have is on par with, if not better than, OpenAI's models.

All this being said, I disagree with your hypothesis. Google does have certain benefits that pretrained models won't easily be able to replicate. For instance, keeping up-to-date and accurate data about restaurant hours, locations, etc. Additionally, it's pretty difficult to know if the LLMs are giving you correct information or not. I think these models still need more innovations before they replace Google. Not to say the current gen isn't useful, but the current Venn diagram of Google use cases and ChatGPT use cases is barely overlapping IMO.

StackOverflow and Google don't ask my phone number for search... so they're not finished for me

I'm not sure about that, it's a temporary ban and the reasoning presented in their post is sound

they are swarmmed with a massive influx of incorrect answers to questions that 'look somewhat correct at first glance'

    sbcl --script example.lsp --output example.c

    (defun num-atoms(list)
      (cond
       ((null list) 0)
       ((atom list) 1)
       (t (+ (num-atoms (car list)) (num-atoms (cdr list))))))

    #include <stdio.h>
    #include <stdlib.h>

    typedef struct list_node {
      int data;
      struct list_node *next;
    } list_node;

    int num_atoms(list_node *list) {
      if (list == NULL) {
        return 0;
      } else if (list->next == NULL) {
        return 1;
      } else {
        return num_atoms(list->next) + num_atoms(list->next->next);
      }
    }

    int main() {
      list_node \*list = NULL;
      printf("%d\n", num_atoms(list));
      return 0;
    }

  (+ (num-atoms (car list)) (num-atoms (cdr list)))

  return num_atoms(list->next) + num_atoms(list->next->next);

  (+ (num-atoms (cdr list)) (num-atoms (cddr list)))

15 years... is a very long time on the internet. I'd say anything lasting more than 15 years is already a major exception to the rule, so something lasting another 15 hardly sounds "finished".

> but banning an advancement in technology (even temporary) is usually the first step to irrelevance of those pushing back.

Answers given by chatgpt have a high chance of being incorrect, how is this a ban on advancement in technology? How is banning chatgpt in it's current state unreasonable for a website that tries to provide correct answers?

> StackOverflow and Google are finished. This is expected and it may not look like it now, but banning an advancement in technology (even temporary) is usually the first step to irrelevance of those pushing back.

Depends on the reason for banning a technology. In this case it isn't because SO's business model is threatened, really; right now it's largely because the ChatGPT answers are usually wrong, and getting spammed to high hell by people who think they're clever by using ChatGPT to answer questions.

It's similar to image boards banning or requiring labels for AI-generated art. It's not that artists feel threatened... it's that the people looking at art don't want to have to wade through a ton of similar-looking, mediocre AI-generated crap.

Or to put it another way; if I set up some bots to start spamming every HN post with dozens of ChatGPT generated comments based on the submission title[0], and I (rightfully) got banned for such, is that because HN feels threatened by ChatGPT?

[0] not using text from the article as a prompt to more accurately simulate the average HN comment

I played with ChatGPT today. It doesn't replace Google or SO today, and based on the types of issues with what it generated, ChatGPT is not going to replace a competent programmer any time soon.

Where I fear you might be right though, is that the web in general (and so, Google) and sites like SO with their internal content, will inevitably lose value as people post ChatGPT-generated content which superficially looks reasonable, but is subtly wrong. Wading through all those subtly-wrong things looking for the actual answer will be a far worse experience than today.

Which is unfortunate. I don't see a good way to avoid it, sadly. SO's attempt to ban ChatGPT-generated content is useless: while people have the incentive to try to get kudos from SO, and ChatGPT requires skilled, detailed analysis to detect its errors, the ban is unenforceable.

This was the thought I had as well when using it for the first time. Much of what I end up searching for (facts) are more directly answered by ChatGPT.

prompt: write a short essay as a scientist which is better, celsius or fahrentheit

partial answer which some faulty reasoning about precision:

First, the Celsius scale is based on a more intuitive and logical reference point. The Celsius scale sets the freezing point of water at 0 degrees and the boiling point of water at 100 degrees, whereas the Fahrenheit scale sets the freezing point of water at 32 degrees and the boiling point at 212 degrees. This means that the Celsius scale has a smaller unit interval, which allows for more precise temperature measurement.

Here's one: https://stackoverflow.com/a/74675088/19877232

Since it is a centralized service provided by OpenAI, they should really provide services (paid or not) that given a text, check if it is generated by ChatGPT recently.

I assume it's just for cases of people saying "ChatGPT produced this output", not guessing.

Even if it is the same why should it be allowed? We want better answers not same shit or something even marginally worse

> That would be a different product.

That product is called "Google Cloud Support".

An intended double-loop might be interesting: Train one GPT on the output of another.

To determine if a post on StackOverflow is generated by ChatGPT, you could look for certain characteristics that are typical of posts generated by the model. For example, ChatGPT posts may contain repetitive or nonsensical phrases, lack coherence or context, or have a distinctive "machine-like" writing style. Additionally, you can check the user profile of the poster to see if it indicates that the user is a language model, such as ChatGPT. However, keep in mind that not all posts generated by ChatGPT will necessarily exhibit these characteristics, and some human-generated posts may also exhibit similar traits. Ultimately, the best way to determine if a post is generated by ChatGPT is to look for signs of artificiality in the content and context of the post.

- obviously generated by ChatGPT

I read this policy as being, on the ground, "if you post an answer that is wrong and it smells like ChatGPT you're gonna eat a ban".

The mechanism is if it reads like a middle school MLA formatted essay. I'd say at least 50% of the answers returned from chat GPT I've seen have ended with the phrase "in conclusion" or "overall".

Compelling argument

ChatGPT misses the point that it mostly trains and hallucinates on info exclusively produced by humans. When sites like SO, Reddit, HN, Wikipedia begin to be dominated by GPT-3, it is then the humanity commences its ill-advised journey into the Matrix. As long as Chicken tastes good, I guess there will be few complaints (worry about the futility of staunch vegans, though).

— Human who watches SciFi movies

I gotta admit, it puts forward it's fallacious points well:

And the fact that ChatGPT answers are easy to produce means that more people can contribute.

Actually, that means that fewer people and one robot will contribute.

But as the SO administrators say, these answer are close enough to true to require a lot of effort on the part administrators - and this stuff may promise a rocky road for answer and discussion sites in the near future. And that would not mean that more people are "participating" either.

Going to be blunt... I'm kinda sick of this already

This pattern...

  <1-3 paragraphs of ChatGPT generated text here>
  <Summary saying I used ChatGPT to write this>

It was cute for like the first couple of times... Clever even...

But now the novelty is wearing thin and all I see are the fallacies inherent in trying to put this forward as some sort of argument/comment in favour of a viewpoint... because I'm still giving the commenter the benefit of the doubt and assuming they are trying to make a substantive contribution to the discussion.

- False Attribution (https://www.logicalfallacies.org/false-cause-and-false-attri...) almost universally at play since before the bait and switch they are attributing AI/ML blather for that of a genuine human's reasoning and opinion, revealing it was from the AI doesn't remove that. By virtue of posting it on your own account, there is the inherent presumption that its coming from the human who registered the account, and thus the false attribution.

> Sure, some of the answers might not be correct, but that's true for any tool.

Can't agree. When writing code I actually pretty sure that 1 + 1 = 2, and 100 + 100 = 200. Of course, of course it could be not true for all languages and all environments, but in one language and in one environment 1 + 1 = 2 is true is always, and not with 99.9999999% chance.

Thats my issue with ChatGPT answers - I can't trust it. ChatGPT shifts complexity of reading, analyzing, verifying correctness of answer to human.

Before the abuse: intelligent experts attempting provision of a good or perfected answer.

After the abuse: intelligent experts using their time to check the vaticinations of a delirious toy.

I think they should have a flag or something so you can self-identify the post as made by GPT so people can compare/contrast what it's capable of. Missed opportunity imho.

Scientific questions are, without being disparaging, mostly trivia. ChatGPT knows all of Wikipedia. If an answer could be found by extracting a passage from a Wikipedia article, ChatGPT will do a good job of answering it.

Ask ChatGPT to do any engineering work, and it falls flat. It's poor mathematical skills and lack of ability to create complex plans mean that it is relying purely on it's coherent-sounding dialog.

This is the scary thing about this - I keep asking it questions, and it gives confident, lengthy responses that look perfect. But when you read it thoroughly, you start noticing small but important mistakes that are not obvious at first.

I think much of the social corrosiveness of the web and news media as it stands is due to the adoption of advertising as the primary means of funding content creation, the resulting expectation that most content should be free, and the failure to produce effective micropayment systems so that paid content has become synonymous with paywalls (high cost barriers around large bodies of content). I would love to see a world in which it is possible to directly pay for content in a very fine grained and frictionless manner, and in which doing so is the norm.

I hadn't considered that the success of large language models might break the current system in a way that can only be fixed by moving it in the direction I'd like to see (but maybe not the micropayments). What an exciting thought :-D

I must read Anathem, maybe it'll change my mind. Neal Stephenson's always great anyway.

That's basically the Western story of the devil, more relatably portrayed as Sauron during the second age, giving gifts and teaching people, all the while sprinkling the teachings with heretical thoughts until, many generations later, the society was corrupt to the core.

I am quite certain it will damage internet anonymity and search engines immensely. Pseudonymous things will be fine for some time, but will probably start requiring some identity verification.

This might not be essential, but you're missing out on playing with some amazing technology.

They send an SMS code to sign up with.

>not sure why

It is to reduce abuse of their API because these requests are very expensive to serve as it requires 100s of gigabytes of VRAM to serve a request, requests take a few seconds at a time, and they are handling a ton of free requests. If you were paying for their API you could be paying up to about $0.10 per text generation.

I wish HN would be showing who downvotes the post, I am more and more experiecing brigading even on completely innocent posts like sharing my experience with account registration.

Interesting thought. But what makes you think that it won't be good enough for the majority of people? To some extent, online-only bootcamps are already good enough for a large crowd of devs, while others went to universities and labs to get a much more in depth education.

What's the financial imperative to having a high score on SO?

Better job offers because you have high rep?

Repeat questions should not be answered. They get marked as duplicate of the original question.

If only ChatGPT could close as a duplicate...

But there is an answer suggesting using it: https://meta.stackoverflow.com/a/421836/7884305.

I actually got emailed personally by Dan last weekend to stop AI-posting after doing this for more than a year.

If it's cheap enough, it'll be just like email spam - basically everywhere worth influencing in any way will be flooded. And it'll be much harder to filter than email. One possible solution would be to change basically any community-driven site to be invite-only, and to wield the ban hammer ruthlessly.