Absolutely. This can be applied to anything and everything in its output, and this is why I take a very dim view of how quickly this has been trotted out, yet how little understanding the average person has of the maths behind it.
You should try asking LLMs about politics, specifically policies of each party, ask it to help you decide who to vote for et cetera. You'll very quickly see it spout absolute bollocks, especially if you're talking politics beyond the USA or Canada. Extra nonsensical if you start using languages other than English.
Even asking it questions about COVID can yield some nonsensical responses in regards to politics.
Having it gaslight you about the NHS and policies of each party is also amazing, especially when you tell it that it's wrong and it tells you that it isn't. With elections looming, these things are the next Brexit bus or Cambridge Analytica waiting to happen.
>especially if you're talking politics beyond the USA or Canada
I think globally we've very quickly hit LLM limits because they were simply not trained on as much information as we think. Or at least, not nearly as much region-specific information in many countries.
I've hit the same issues trying to ask any LLM to give me tax advice for tax law in specific countries, and even beyond the "ask a professional" type restrictions eg OpenAI has, it's clear that it just does not have the information.
Sometimes that's just because of the US-centric view of these companies training the models, and sometimes it's also because information sources are tucked away on botched government Wordpress sites and random PDFs.
This is why you have to do things like RAG to make a "talk to your doc" kind of bot even if you're feeding it public info.
I've had a few less tech-savvy friends tell me dubious things this past year and asked where they heard it—usually "this neat app I found called ChatGPT". eyeroll
Asking a large language model about facts is like asking an image model for photos. That's not what it's for, or what it does.
It's a bit weird that we're building tools that make the large scale pollution of the information space approach a marginal cost of zero. I get that it's a bit useful for artists, and for helping dyslexics write emails or whatever, but it's highly doubtful this will be worth it.
We already have huge issues with humans being stupid, why would we insert this not-even-stupid noise into the information space?
Then again, it has been marketed as such, and does not tell people that it's lying. Image models don't have that problem because they never claimed to answer truthfully.
> why would we insert this not-even-stupid noise into the information space?
Because US tech would "collapse" otherwise.
Big Tech's stock price is entirely propped up by AI hype. Metaverse didn't go anywhere so they need a new bubble.
US tech startups are even worse. The VC culture is completely incapable of building useful companies, AI is the last hype. Just a rat-race to grab the last funding and get bought out before the end of ZIRP kills the startup scene entirely.
There's no real demand for any of this shit. Even the web spam renaissance is mostly hustle-bros losing money. The writing is already on the wall with investment slowing down, and AI product after AI product bombing.
I mean, it's likely that _some_ fraud is going on, and that tends to become more apparent as the money runs out (for instance see SBF, or the various Ponzi schemes that ran out of road during the financial crisis), so when the AI bubble bursts, yeah, probably a few people will go to jail. But it doesn't seem like a significant factor.
It's incredible, this attention economy. Google anything and you'll have word-vomit articles that don't answer your question - something like "Actress XYZ boyfriend" and you get:
> XYZ, XYZ has emerged as a rising star, captivating audiences with her talent and charm. While her professional life is an open book, her personal life remains a subject of intrigue. Join us as we delve into the untold love story of XYZ, exploring her journey from boyfriend to husband.
With even a wrong headline photo of the actress, and zero information about her boyfriend/husband. At least the above paragraph is still coherent.
Why? Because they can embed ads on the webpage, so they can earn money from Google...
Can we admit that this was the state of the Internet content before LLMs and their introduction just accelerated and existing problem? I feel like the conversation around the information pollution is turning into "AI did it" and we need to take a lot more responsibility for letting it happen in the first place.
I agree, it wasn't AI, or well, much stupider AI / bots that put words together (we had that before GPT - Were they AI? Just like a high-end Nokia was a "smartphone" before iPhone came along...).
But this new AI is making it simpler and will probably accelerate the bullshit pollution.
On that note, today I got 2 Instagram video ads of 2 different national politicians saying they're offering something to "just 50 people", and it was to me obviously deepfake videos. But there are vast numbers of people who will fall for this scam. But hey, at least these ads pays for Zuck's yachts (yeah he has two, 1 for him to enjoy and 1 behind it that carries supplies/sleeps guests)...
Who is “we” here again? I’m not sure why even google engineers would want to take credit for management decisions to screw up search. And it’s deliberate because bad search results means more search! The approach I usually recommend for protest is.. use google exclusively to search for ChatGPT, then leave google. Since they haven’t heard users saying for the last decade that we pretty much prefer classic google search circa 2010 to the garbage engagement and growth hacking monster that they have become… perhaps they’ll hear this version of protest better.
Yea, Google/Ads is the problem for this specific case. Just that Google didn't fight it and Google also was pivotal in creating the modern LLM/AI tech. They are the people who are destroying the internet from many fronts.
It's being adopted by organisations who should be guarding data integrity, preserve sources of truth, and pay utmost attention to PII or corporate and state secrets. I have to congratulate operators of genAI on their marketing prowess, but the results are really worrying and it's only going to get worse. We run real risk of serious disruption to the lives of people wrongly banned from access to shops and places, charged with offences they did not commit, their credit scores ruined and no straight path to rectify the situation. The tools that ought to be only used for academic research have been given to people with no background allowing them to understand how those tools work by cynical people looking to make a quick buck from chaos others will unleash on employees, patients, citizens. Meanwhile the AI operators have all the raw data and can do anything with it.
Yes, it's possible the scale of killings was unintended by the leadership, and the focus was on other things. Hard to say without independent investigation.
> Though Sinwar planned and greenlighted the Oct. 7 attacks, early messages to cease-fire negotiators show he seemed surprised by the brutality of Hamas’s armed wing and other Palestinians, and how easily they committed civilian atrocities.
“Things went out of control,” Sinwar said in one of his messages, referring to gangs taking civilian women and children as hostages. “People got caught up in this, and that should not have happened.”
And you will not win an argument about who's threat to who in this particular conflict in favor of Israeli Jews by comapring numbers of victims.
Oooh, the guy, nickkname "the butcher of Khan Younis, who like to bury alive and torture people "suspected" of collaboration and being gay[0], at the head of a deeply antisemitic organisation [1], is surprised that his minions behaved brutally? He might be the only one.
> you will not win an argument about who's threat to who in this particular conflict in favor of Israeli Jews by comparing numbers of victims.
Exactly. How many Jews would die if Hamas were to keep existing? Just look at how many Palestinians Hamas killed with their own hands.
Upvoted because it's absolutely true.
The Holocaust was a human disgrace that should be taught in schools and written in stone to never be forgotten by anyone. Unfortunately Israel government agents, paid spindoctors and shills or simply brainwashed social drones, are abusing the word "antisemitism" (which itself is a disgrace as well) to paint as antisemitic, ie racist, anyone remotely criticizing Israel in order to damage their credibility. This is a well known tactics that however in the end will damage both the image of good Israeli citizens and the Holocaust memory itself.
Allowing a single holocaust to overshadow all the mass state sponsored attrocities perpetrated ever since is a disgrace in itself, since it moves the focus from the actual problem, and prioritizes some particular small group of people. Real problem is that humans can and do form and support organizational structures (even democratic ones) that perpetrate and justify all kinds of attrocities on others, and can immunize themselves from feeling anything even while supporting mass killings of people, in the "right" context.
Exactly. Every country, including mine, has its own share of atrocities committed in the past and no clear sign that others won't be committed in the future.
In the end it all boils down to the wrong people being put in power, very often even democratically, then using all their powers paired to their disregard for other human being to keep that power forever, until the inevitable escalation brings them to being forced to remain in power as a form of self preservation. Pretty much every dictator follows this path, and usually none of them look as one at the beginning of their "career".
Ah, moving the goal post. Caring about the Nazi genocide of the Jews does not exclude caring about other genocides. But you, of course, knew that. You just used an unfortunate phrase, didn't you?
Dito. "who has the power and money" is your answer to your question "why is UNESCO [sic] specifically concerned about LLMs diluting Holocaust facts, and not the significantly more important, current genocides taking place?"
It is not the only answer, and you know it. As other posters have already pointed out, the Holocaust is a pretty special event in history which makes it an excellent example of the potential dangers of revisionism through LLMs.
What could be more important? The holocaust was the systematic and industrialized homicide of about 6 million human beings. It is witness and evidence of the level of atrocities people commit if they believe the populist lies that everything is somebody else's fault and check out their humanity to "solve" it. It is part of Germany's, the Jewish and some others history, of course does the UNESCO care about that.
I don't think it is reasonable to say Generative AI is a bigger threat than basic forgetfulness. Which UNESCO demonstrates.
The problem is the Holocaust was the first time industrial techniques were applied to killing large groups of citizens. The unique evil is the tools, bureaucracy and mundanity of mass murder. The horror was kids in the wrong family tree being shipped off to well engineered gas chambers - not the details of ideologies and personalities. We're already forgetting the more important lessons of not designating internal citizen groups as the out-group (if we ever learned it) and not building up those registers labelling for one bad government to use.
> The Historical Figures App allowed users to chat with prominent Nazis such as Adolf Hitler and Joseph Goebbels, and falsely claimed that individuals such as Goebbels were not intentionally involved in the Holocaust and had tried to prevent violence against Jews.
I mean really. This is a nonissue. Hitler and Goebbels aren't unique evils and there isn't any point remembering them as people; at this point they are more rhetorical symbols then people. Genocidal maniacs aren't at all uncommon in history (the records suggest Ghengis Khan literally erased entire countries if he felt they were inconvenient to him - and his title is "Great" because he won and enough time passed).
> I don't think it is reasonable to say Generative AI is a bigger threat than basic forgetfulness.
Misinformation is a real, serious threat.
> I mean really. This is a nonissue. Hitler and Goebbels aren't unique evils and there isn't any point remembering them as people; at this point they are more rhetorical symbols then people. Genocidal maniacs aren't at all uncommon in history (the records suggest Ghengis Khan literally erased entire countries if he felt they were inconvenient to him - and his title is "Great" because he won and enough time passed).
I was born and raised in a city where one of the largest concentration camps was built and run by Germans. Members of my family were murdered by Germans following orders of Hitler and told that it was necessary to do so by Goebbels. Then Russians came and repurposed the camp to imprison people who fought Germans and ruled the place for close to 50 years by proxies. I have no desire to see genAI or "works of art" showing Hitler or Goebbels as innocent or even likeable characters. Holocaust was a real thing. The fact that other genocides happened before and after does not change the reality or the legacy of it. Hitler puppets saying he was innocent are not just denying the facts, they are prepping people for another round of persecutions, and not just Jews. If you think it's ok to think Hitler or Goebbels are irrelevant, or that what they did doesn't matter then you are paving the way you will be sent down on to a camp yourself.
> If you think it's ok to think Hitler or Goebbels are irrelevant, or that what they did doesn't matter then you are paving the way you will be sent down on to a camp yourself.
Just taking a feel of the political winds but - if I'm sent to a camp in my lifetime it will probably be by people calling me a Nazi. Assuming an internal group does it. It is a favoured insult right now. Although the odds are still comfortably low as far as I can tell.
And this is the issue. Privileging a hate-cult of Hitler and Goebbels is a distraction from the actual problems which are policies, capabilities and incentives. It doesn't seem to be an effective barrier to hate - in fact philosophically it encourages it over more useful focuses.
Calling someone a Nazi is the the favourite thing Nazi copycats do. Reminding of what Hitler and lots of Germans (and Austrians) did is not practicing cultish behaviour, it is a vaccine against those who want a repeat of what had happened.
Yes, but no. People who are unable to morally reason will go "I'm not a Nazi so what i do is justified" and then do the same bad shit all over again. Ignoring the lessons about Hitler and Goebbels because they are convinced they are better. Very focused on fighting the baddies that they never question if they are just another sort of baddies themselves.
> We're already forgetting the more important lessons of not designating internal citizen groups as the out-group (if we ever learned it) and not building up those registers labelling for one bad government to use.
I don't think that's even the lesson. That's too detailed. That's not what happened in Cambodia or China. It's not what happened in Indonesia. It's not what happened in the Ottoman Empire.
The problem is the same everyhere: how_wrong_you_are_about_facts * power * conviction, these are the three variables that multiply together to produce mass killings. Mao didn't have an out group (well.. that wasn't the big death toll at least). Mao didn't have registers. He just made people destroy the farming and millions starved.
We have to care about truth. Then we have to limit power. Then we have to be doubtful about what we know so we make small changes and slowly.
You don’t need ChatGPT to rewrite WWII history. I choke every time I read “pacifist Japan militarize in response to expansionist neighbor” in American media.
If flat-earthers had the numbers and motivation of revisionist neo-nazis, they might very well be able to decrease popular (not academic) understanding of astrophysics.
64 comments
[ 3.3 ms ] story [ 137 ms ] threadYou should try asking LLMs about politics, specifically policies of each party, ask it to help you decide who to vote for et cetera. You'll very quickly see it spout absolute bollocks, especially if you're talking politics beyond the USA or Canada. Extra nonsensical if you start using languages other than English.
Even asking it questions about COVID can yield some nonsensical responses in regards to politics.
Having it gaslight you about the NHS and policies of each party is also amazing, especially when you tell it that it's wrong and it tells you that it isn't. With elections looming, these things are the next Brexit bus or Cambridge Analytica waiting to happen.
I think globally we've very quickly hit LLM limits because they were simply not trained on as much information as we think. Or at least, not nearly as much region-specific information in many countries.
I've hit the same issues trying to ask any LLM to give me tax advice for tax law in specific countries, and even beyond the "ask a professional" type restrictions eg OpenAI has, it's clear that it just does not have the information.
Sometimes that's just because of the US-centric view of these companies training the models, and sometimes it's also because information sources are tucked away on botched government Wordpress sites and random PDFs.
This is why you have to do things like RAG to make a "talk to your doc" kind of bot even if you're feeding it public info.
It's a bit weird that we're building tools that make the large scale pollution of the information space approach a marginal cost of zero. I get that it's a bit useful for artists, and for helping dyslexics write emails or whatever, but it's highly doubtful this will be worth it.
We already have huge issues with humans being stupid, why would we insert this not-even-stupid noise into the information space?
Then again, it has been marketed as such, and does not tell people that it's lying. Image models don't have that problem because they never claimed to answer truthfully.
We already know that generative performance degrades when they learn from generated data (don’t remember the exact findings).
Because US tech would "collapse" otherwise.
Big Tech's stock price is entirely propped up by AI hype. Metaverse didn't go anywhere so they need a new bubble.
US tech startups are even worse. The VC culture is completely incapable of building useful companies, AI is the last hype. Just a rat-race to grab the last funding and get bought out before the end of ZIRP kills the startup scene entirely.
There's no real demand for any of this shit. Even the web spam renaissance is mostly hustle-bros losing money. The writing is already on the wall with investment slowing down, and AI product after AI product bombing.
They will go from trillions to slightly fewer trillions.
It's incredible, this attention economy. Google anything and you'll have word-vomit articles that don't answer your question - something like "Actress XYZ boyfriend" and you get:
> XYZ, XYZ has emerged as a rising star, captivating audiences with her talent and charm. While her professional life is an open book, her personal life remains a subject of intrigue. Join us as we delve into the untold love story of XYZ, exploring her journey from boyfriend to husband.
With even a wrong headline photo of the actress, and zero information about her boyfriend/husband. At least the above paragraph is still coherent.
Why? Because they can embed ads on the webpage, so they can earn money from Google...
But this new AI is making it simpler and will probably accelerate the bullshit pollution.
On that note, today I got 2 Instagram video ads of 2 different national politicians saying they're offering something to "just 50 people", and it was to me obviously deepfake videos. But there are vast numbers of people who will fall for this scam. But hey, at least these ads pays for Zuck's yachts (yeah he has two, 1 for him to enjoy and 1 behind it that carries supplies/sleeps guests)...
https://www.autoevolution.com/news/mark-zuckerberg-buys-russ...
We're already on the verge of losing reliable search engines. Will this bring back 90's-style Yahoo sites?
Because people will make money from it and the political will to resist such a thing does not exist. Besides the cat is already out of the bag.
Who threatens who, already?
Did Hamas kill over 1000 jews by accident? Did they parade their bodies by accident too?
Do you have an actual argument or do you intend to whatabout in more and more stupid ways?
https://www.wsj.com/world/middle-east/gaza-chiefs-brutal-cal...
> Though Sinwar planned and greenlighted the Oct. 7 attacks, early messages to cease-fire negotiators show he seemed surprised by the brutality of Hamas’s armed wing and other Palestinians, and how easily they committed civilian atrocities. “Things went out of control,” Sinwar said in one of his messages, referring to gangs taking civilian women and children as hostages. “People got caught up in this, and that should not have happened.”
And you will not win an argument about who's threat to who in this particular conflict in favor of Israeli Jews by comapring numbers of victims.
> you will not win an argument about who's threat to who in this particular conflict in favor of Israeli Jews by comparing numbers of victims.
Exactly. How many Jews would die if Hamas were to keep existing? Just look at how many Palestinians Hamas killed with their own hands.
[0] https://archive.ph/khYfS [1] https://archive.ph/IgNwg
Or is this supposed to be your politics-only account because you're too scared to use the other one?
It is not the only answer, and you know it. As other posters have already pointed out, the Holocaust is a pretty special event in history which makes it an excellent example of the potential dangers of revisionism through LLMs.
The problem is the Holocaust was the first time industrial techniques were applied to killing large groups of citizens. The unique evil is the tools, bureaucracy and mundanity of mass murder. The horror was kids in the wrong family tree being shipped off to well engineered gas chambers - not the details of ideologies and personalities. We're already forgetting the more important lessons of not designating internal citizen groups as the out-group (if we ever learned it) and not building up those registers labelling for one bad government to use.
> The Historical Figures App allowed users to chat with prominent Nazis such as Adolf Hitler and Joseph Goebbels, and falsely claimed that individuals such as Goebbels were not intentionally involved in the Holocaust and had tried to prevent violence against Jews.
I mean really. This is a nonissue. Hitler and Goebbels aren't unique evils and there isn't any point remembering them as people; at this point they are more rhetorical symbols then people. Genocidal maniacs aren't at all uncommon in history (the records suggest Ghengis Khan literally erased entire countries if he felt they were inconvenient to him - and his title is "Great" because he won and enough time passed).
Misinformation is a real, serious threat.
> I mean really. This is a nonissue. Hitler and Goebbels aren't unique evils and there isn't any point remembering them as people; at this point they are more rhetorical symbols then people. Genocidal maniacs aren't at all uncommon in history (the records suggest Ghengis Khan literally erased entire countries if he felt they were inconvenient to him - and his title is "Great" because he won and enough time passed).
I was born and raised in a city where one of the largest concentration camps was built and run by Germans. Members of my family were murdered by Germans following orders of Hitler and told that it was necessary to do so by Goebbels. Then Russians came and repurposed the camp to imprison people who fought Germans and ruled the place for close to 50 years by proxies. I have no desire to see genAI or "works of art" showing Hitler or Goebbels as innocent or even likeable characters. Holocaust was a real thing. The fact that other genocides happened before and after does not change the reality or the legacy of it. Hitler puppets saying he was innocent are not just denying the facts, they are prepping people for another round of persecutions, and not just Jews. If you think it's ok to think Hitler or Goebbels are irrelevant, or that what they did doesn't matter then you are paving the way you will be sent down on to a camp yourself.
Just taking a feel of the political winds but - if I'm sent to a camp in my lifetime it will probably be by people calling me a Nazi. Assuming an internal group does it. It is a favoured insult right now. Although the odds are still comfortably low as far as I can tell.
And this is the issue. Privileging a hate-cult of Hitler and Goebbels is a distraction from the actual problems which are policies, capabilities and incentives. It doesn't seem to be an effective barrier to hate - in fact philosophically it encourages it over more useful focuses.
I don't think that's even the lesson. That's too detailed. That's not what happened in Cambodia or China. It's not what happened in Indonesia. It's not what happened in the Ottoman Empire.
The problem is the same everyhere: how_wrong_you_are_about_facts * power * conviction, these are the three variables that multiply together to produce mass killings. Mao didn't have an out group (well.. that wasn't the big death toll at least). Mao didn't have registers. He just made people destroy the farming and millions starved.
We have to care about truth. Then we have to limit power. Then we have to be doubtful about what we know so we make small changes and slowly.
People think the earth is flat—does Generative AI threaten our core understanding of astrophysics?