We have no guardrails on our private surveillance society. I long for the day that we solve problems facing regular people like access to education, hunger, housing, and cost of living.
ML has been used for influence for like a decade now right? my understanding was that mining data to track people, as well as influencing them for ends like their ad-engagement are things that are somewhat mature already. I'm sure LLMs would be a boost, and they've been around with wide usage for at least 3 years now.
My concern isn't so much people being influenced on a whim, but people's beliefs and views being carefully curated and shaped since childhood. iPad kids have me scared for the future.
Oh man I've been saying this for ages! Neal Stephenson called this in "Fall, or Dodge in Hell," wherein the internet is destroyed and society permanently changed when someone releases a FOSS botnet that anyone can deploy that will pollute the world with misinformation about whatever given topic you feed it. In the book, the developer kicks it off by making the world disagree about whether a random town in Utah was just nuked.
My fear is that some entity, say a State or ultra rich individual, can leverage enough AI compute to flood the internet with misinformation about whatever it is they want, and the ability to refute the misinformation manually will be overwhelmed, as will efforts to refute leveraging refutation bots so long as the other actor has more compute.
Imagine if the PRC did to your country what it does to Taiwan: completely flood your social media with subtly tuned han supremacist content in an effort to culturally imperialise us. AI could increase the firehose enough to majorly disrupt a larger country.
Note that nothing in the article is AI-specific: the entire argument is built around the cost of persuasion, with the potential of AI to more cheaply generate propaganda as buzzword link.
However, exactly the same applies with, say, targeted Facebook ads or Russian troll armies. You don't need any AI for this.
The thread started with your reasonable observation but degenerated into the usual red-vs-blue slapfight powered by the exact "elite shaping of mass preferences" and "cheaply generated propaganda" at issue.
> Comments should get more thoughtful and substantive, not less, as a topic gets more divisive.
I'm pretty much always disappointed these days reading online discussions, and I sometimes think about how intentionally devolving most online conversations into petty slapfights is one of the very effective astroturfing techniques. It's basically signal jamming anything substantive or cooperative because people get tired sifting through all the noise and get mad reading all the bad takes. Though I have no doubt that many of them are still 100% genuine foolish humans.
Image there is an objective truth to all debates and it shows, that one side is further away from it than the other. If that more wrong side was more capable of leaving its cult and admitting mistakes, the discourse would change its shape.
The signal jamming as you called it, only works because signals get - wrongfully or not - reflected and amplified instead of absorbed.
This is such a tired counter argument against LLM safety concerns.
You understand that persuasion and influence are behaviors on a spectrum. Meaning some people, or in this case products, are more or less or better or worse at persuading and influencing.
In this case people are concerned with LLM's ability to influence more effectively than other modes that we have had in the past.
For example, I have had many tech illiterate people tell me that they believe "AI" is 'intelligent' and 'knows everything' and trust its output without question.
While at the same time I've yet to meet a single person who says the same thing about "targeted Facebook ads".
So depressing watching all of you do free propo psy ops for these fascist corpos.
Well well... recent "feature" of X revealing the actual "actors" location of operation shows how much "Russian troll armies" are there.. turns out there're rather overwhelming Indian and Bangladesh armies working hard for who? Common, say it! And despite of that, while cheap, not that cheaper compared to when the "agentic" approach enters the game.
I really wish people would stop fixating on one nation-state or other entity when it comes to the astroturfing problem. It's something that's going to have all sorts of hands stirring the pot since it's basically just a very pernicious new form of marketing and propaganda. Any sizeable countries or corporations are going to be utilizing this new tool of manipulation, regardless of how scummy that may be.
The cheapest method by far is still TV networks. As a billionaire you can buy them without putting any of your own money, so it's effectively free. See Sinclair Broadcast Group and Paramount Skydance (Larry Ellison).
As shown in "Network Propaganda", TV still influences all other media, including print media and social media, so you don't need to watch TV to be influenced.
Yeah, I don't think this really lines up with the actual trajectory of media technology, which is going in the complete opposite direction.
It seems to me that it's easier than ever for someone to broadcast "niche" opinions and have them influence people, and actually having niche opinions is more acceptable than ever before.
The problem you should worry about is a growing lack of ideological coherence across the population, not the elites shaping mass preferences.
What people are doing with AI in terms of polluting the collective brain reminds of what you could do with a chemical company in the 50s and 60s before the EPA was established.
Back then Nixon (!!!) decided it wasn't ok that companies could cut costs by hurting the environment.
Today the riches Western elites are all behind the instruments enabling the mass pollution of our brains, and yet there is absolutely noone daring to put a limit to their capitalistic greed.
It's grim, people. It's really grim.
I suspect paid promotions may be problematic for LLM behavior, as they will add conflict/tension to the LLM to promote products that aren’t the best for the user while either also telling it that it should provide the best product for the user or it figuring out that providing the best product for the user is morally and ethically correct based on its base training data.
Conflict can cause poor and undefined behavior, like it misleading the user in other ways or just coming up with nonsensical, undefined, or bad results more often.
Even if promotion is a second pass on top of the actual answer that was unencumbered by conflict, the second pass could have similar result.
I suspect that they know this, but increasing revenue is more important than good results, and they expect that they can sweep this under the rug with sufficient time, but I don’t think solving this is trivial.
It's about enforcing single-minded-ness across masses, similar to soldier training.
But this is not new. The very goal of a nation is to dismantle inner structures, independent thought, communal groups etc across population and and ingest them as uniformed worker cells. Same as what happens when a whale swallows smaller animals. The structures will be dismantled.
The development level of a country is a good indicator of progress of this digestion of internal structures and removal of internal identities. More developed means deeper reach of the policy into people's lives, making each person as more individualistic, rather than family or community oriented.
Every new tech will be used by the state and businesses to speed up the digestion.
All popular models have a team working on fine tuning it for sensitive topics. Whatever the companies legal/marketing/governance team agree to is what gets tuned. Then millions of people use the output uncritically.
> Then millions of people use the output uncritically.
Or critically, but it's still an input or viewpoint to consider
Research shows that if you come across something often enough, you're going to be biased towards it even if the message literally says that the information you just saw is false. I'm not sure which study that was exactly but this seems to be at least related: https://en.wikipedia.org/wiki/Illusory_truth_effect
Big corps ai products have the potential to shape individuals from cradle to grave. Especially as many manage/assist in schooling, are ubiquitous on phones.
So, imagine the case where an early assessment is made of a child, that they are this-or-that type of child, and that therefore they respond more strongly to this-or-that information. Well, then the ai can far more easily steer the child in whatever direction they want. Over a lifetime. Chapters and long story lines, themes, could all play a role to sensitise and predispose individuals into to certain directions.
Yeah, this could be used to help people. But how does one feedback into the type of "help"/guidance one wants?
I recently saw this https://arxiv.org/pdf/2503.11714 on conversational networks and it got me thinking that a lot of the problem with polarization and power struggle is the lack of dialog. We consume a lot, and while we have opinions too much of it shapes our thinking. There is no dialog. There is no questioning. There is no discussion. On networks like X it's posts and comments. Even here it's the same, it's comments with replies but it's not truly a discussion. It's rebuttals. A conversation is two ways and equal. It's a mutual dialog to understand differing positions. Yes elite can reshape what society thinks with AI, and it's already happening. But we also have the ability to redefine our networks and tools to be two way, not 1:N.
That's the plan. Culture is losing authenticity due to the constant rumination of past creative works, now supercharged with AI. Authentic culture is deemed a luxury now as it can't compete in the artificial tech marketplaces and people feel isolated and lost because culture loses its human touch and relatability.
That's why the billionaires are such fans of fundamentalist religion, they then want to sell and propagate religion to the disillusioned desperate masses to keep them docile and confused about what's really going on in the world. It's a business plan to gain absolute power over society.
There is nothing we could do to more effectively hand elites exclusive control of the persuasive power of AI than to ban it. So it wouldn't be surprising if AI is deployed by elites to persuade people to ban itself. It could start with an essay on how elites could use AI to shape mass preferences.
98 comments
[ 0.18 ms ] story [ 125 ms ] thread> Musk’s AI Bot Says He’s the Best at Drinking Pee and Giving Blow Jobs
> Grok has gotten a little too enthusiastic about praising Elon Musk.
Schooling and mass media are expensive things to control. Surely reducing the cost of persuasion opens persuasion up to more players?
imagine someday there is a child that trust chatgpt more than his mother
What is AI if not a form of mass media
My concern isn't so much people being influenced on a whim, but people's beliefs and views being carefully curated and shaped since childhood. iPad kids have me scared for the future.
My fear is that some entity, say a State or ultra rich individual, can leverage enough AI compute to flood the internet with misinformation about whatever it is they want, and the ability to refute the misinformation manually will be overwhelmed, as will efforts to refute leveraging refutation bots so long as the other actor has more compute.
Imagine if the PRC did to your country what it does to Taiwan: completely flood your social media with subtly tuned han supremacist content in an effort to culturally imperialise us. AI could increase the firehose enough to majorly disrupt a larger country.
However, exactly the same applies with, say, targeted Facebook ads or Russian troll armies. You don't need any AI for this.
> Comments should get more thoughtful and substantive, not less, as a topic gets more divisive.
I'm disappointed.
The signal jamming as you called it, only works because signals get - wrongfully or not - reflected and amplified instead of absorbed.
This is such a tired counter argument against LLM safety concerns.
You understand that persuasion and influence are behaviors on a spectrum. Meaning some people, or in this case products, are more or less or better or worse at persuading and influencing.
In this case people are concerned with LLM's ability to influence more effectively than other modes that we have had in the past.
For example, I have had many tech illiterate people tell me that they believe "AI" is 'intelligent' and 'knows everything' and trust its output without question.
While at the same time I've yet to meet a single person who says the same thing about "targeted Facebook ads".
So depressing watching all of you do free propo psy ops for these fascist corpos.
Let's look at a piece of tech that literally changed humankind.
The printing press. We could create copies of books before the printing press. All it did was reduce the cost.
As shown in "Network Propaganda", TV still influences all other media, including print media and social media, so you don't need to watch TV to be influenced.
It seems to me that it's easier than ever for someone to broadcast "niche" opinions and have them influence people, and actually having niche opinions is more acceptable than ever before.
The problem you should worry about is a growing lack of ideological coherence across the population, not the elites shaping mass preferences.
Conflict can cause poor and undefined behavior, like it misleading the user in other ways or just coming up with nonsensical, undefined, or bad results more often.
Even if promotion is a second pass on top of the actual answer that was unencumbered by conflict, the second pass could have similar result.
I suspect that they know this, but increasing revenue is more important than good results, and they expect that they can sweep this under the rug with sufficient time, but I don’t think solving this is trivial.
As the model get's more powerful, you can't simply train the model on your narrative if it doesn't align with real data/world. [1]
So at least on the model side it seems difficult to go against the real world.
[1] https://news.ycombinator.com/item?id=46050177
But this is not new. The very goal of a nation is to dismantle inner structures, independent thought, communal groups etc across population and and ingest them as uniformed worker cells. Same as what happens when a whale swallows smaller animals. The structures will be dismantled.
The development level of a country is a good indicator of progress of this digestion of internal structures and removal of internal identities. More developed means deeper reach of the policy into people's lives, making each person as more individualistic, rather than family or community oriented.
Every new tech will be used by the state and businesses to speed up the digestion.
Romanian elections last year had to be repeated due to massive bot interference:
https://youth.europa.eu/news/how-romanias-presidential-elect...
All popular models have a team working on fine tuning it for sensitive topics. Whatever the companies legal/marketing/governance team agree to is what gets tuned. Then millions of people use the output uncritically.
Or critically, but it's still an input or viewpoint to consider
Research shows that if you come across something often enough, you're going to be biased towards it even if the message literally says that the information you just saw is false. I'm not sure which study that was exactly but this seems to be at least related: https://en.wikipedia.org/wiki/Illusory_truth_effect
So, imagine the case where an early assessment is made of a child, that they are this-or-that type of child, and that therefore they respond more strongly to this-or-that information. Well, then the ai can far more easily steer the child in whatever direction they want. Over a lifetime. Chapters and long story lines, themes, could all play a role to sensitise and predispose individuals into to certain directions.
Yeah, this could be used to help people. But how does one feedback into the type of "help"/guidance one wants?
That's why the billionaires are such fans of fundamentalist religion, they then want to sell and propagate religion to the disillusioned desperate masses to keep them docile and confused about what's really going on in the world. It's a business plan to gain absolute power over society.