> Users of the AI companion chatbot Replika are reporting that it has stopped responding to their sexual advances, and people are in crisis. Moderators of the Replika subreddit made a post about the issue that contained suicide prevention resources
Tell me you’ve never had an actual crisis, without telling me you’ve never had an actual crisis.
The fact that some number of people are in such a pitiful state is in itself a crisis. Everyone needs real human connection. The “crisis” happened when they came to rely on replika in the first place.
Bingo. The issue here is that they pulled the support without appropriate warning, essentially ghosting these people.
The real issue are the conditions that allowed this to happen. Modern society is a genuine nightmare for real human connections. I can make a million superficial connections with a billion people.
But finding deep, meaningful community is harder now than it has ever been. That's exhausting, and I have to believe it's one of the main contributors to the mental health crisis we are currently experiencing (at least in the US).
Exactly. It's the same thing as drug addiction: drug addiction is a big problem in and of itself, but also the consequence of a greater-scope emotional problem. Leave that underlying problem untreated, and the best you can hope for is that the addict will transfer their addiction to something less harmful.
Given that AI is convincing enough to foster artificial relationships with humans, the humans clearly don't know better and get more attached than they should. It's pathetic, but not their fault-- the system was designed to be so engaging.
So when those relationships are arbitrarily yanked away from them, it's alienating. It's so harmful we long ago figured out it could be weaponized, and now we call it things like "disfellowshipping" and "shunning."
> Tell me you’ve never had an actual crisis, without telling me you’ve never had an actual crisis.
Pain and trauma don't come directly from external events, but from our inner emotional response to them. Just because it might seem weird to you that someone could be deeply attached to an AI doesn't mean that their experience is somehow easier for them than things you would accept as an "actual crisis."
You can't gatekeep pain. I can believe that for some people, this would be as painful as being rejected abruptly by a real life partner in a serious relationship.
I guarantee that someone who has fallen in love with a computer program has suffered. It's not a normal thing to do. This is like calling an abused child soft because it won't let go of its teddy bear and is upset when you throw it out.
If I suggest you buy a battery for your car with a flat tire, then there is no explanation needed when you say, "You are so clueless I don't even know where to start."
It takes a lot of confidence and extroversion to fly solo at a festival or event, meet people there, and leave with lasting connections (not just someone you follow on Instagram and never interact with again). The kind of confidence and extroversion that if those people had, they wouldn't be relying on chatbots in the first place?
People see when you're unhappy. If you're chronically so, they'll stay away most of the time. Source: I was this way for a few years.
There's a point where, if your life sucks enough and you don't have the social skills to hide it, you can be stuck in a bad place with no obvious way to get help.
Maybe paying for a therapist will help, or maybe you won't have the money, the waiting lists will be too long, your country's healthcare system will limit you to wrong matches. If this way doesn't work, what's your next step?
- I traveled, met people who had had a much better life and way more success, and basically got angry/ashamed enough that mine sucked that I got motivated to change it,
- I cut ties with a couple friends/family members who had issues of their own (that they refused to work on),
- I'm a competent tech worker, which means no money issues, less stress, social contact through school/work, and the morale boost of one part of my life going right at least,
- having a few good (now remote) friends who helped me navigate through things.
People who stay in bad situations probably don't have most of these things. GGP doesn't seem to know how hard it can be to fix this. Years after the bad times, I still have some social/emotional issues. And these were mild problems.
Relationship disorders are more complex than just not meeting women (as in "going out").
A part of those who are not able to get a partner may solve their problem by "just" meeting women, but for many, there are significant psychological issues.
What do you expect a guy who forms an emotional relationship with a chatbot thing to accomplish with a dating app? Most likely would only induce further despair because he would get few, if any, likes.
Even worse, actually: they promote their service to vulnerable users as a replacement for social interaction. One of their TikTok video ads opened with the message:
I'm dumping my fake friends!
My AI Replika understands
me more than anyone
else i know :')
So not just versioned (which cloud software often is) but running locally? That's a pretty poor fit for a large language model, since users won't have sufficient hardware and use is very bursty.
Not yet. In a few years, consumer GPUs or dedicated ML units (which Apple SOCs have) and local storage and ram will be large enough to run “a few years old” LLMs.
The cutting edge will always be in the cloud but a market for personal AI does exist.
Their website shows multiple quotes from people who were "dating" their chatbot for 4+ years. Then overnight an update eliminated the perceived romantic interest of the chatbot. I can't imagine having that sort of connection to a chatbot but I've had both addictions and romantic relationships, and it makes me feel terrible for these people.
This is pretty surprising to me in that it seemed like the rise in provocative ads for thirst trap chatbots only appeared a few months ago- I remember people on Twitter making fun of those skeevy meme ads not very long ago. To my awareness it seemed like the Replika fad must have only lasted a very short time, though I've heard about their work much earlier:
Empathy and humanity aside, we live in interesting time. This is really a cyberpunk level headline where people saw their AI companion change and it had impacts on their relationship with it.
Add an holographic projection and this is pretty much Joi from Bladerunner.
One thing to keep in mind with cyberpunk is that all of the gee-whiz technology was merely supplementary to the core material and spiritual conditions in which the tech added as a salve for the suffering or enabled the oppressors. We live in an atomized, detached culture in which economic pressures prevent family formation or livelihood security that past generations (or maybe just one, the postwar generation) enjoyed.
So when companies monetize the loneliness people feel towards each other by selling them subscriptions to AI chatbots we might only see the cutting edge dystopian tech, but remember that without the underlying conditions, this wouldn't be so cyberpunk. Maybe in a healthier society these chatbots could be used for something less desperate.
We really lack coherent movements that move needles like we had in the 20th century. Say the suffragettes or the end child labour movement, or voting. These days it is very fragmented and often co-opted very quickly which is the opposite of what you want for a ‘better’ society.
Think it’s even deeper than that. The US was 50% of the global economy in the post war years. Now we’re a fraction of that. The world is more competitive now.
The amazing surplus and living standards of the post war years were an aberration
Governments and other forces got very very good at undermining and dissipating subversive movements. A lot of the changes in the US the last half century make a lot of sense if you view them as a reaction to the civil rights movement in the abstract. Eg not about any of its concrete goals or accomplishments, but just ensuring that no movement as widespread and transformative as that can form again, regardless of its mission.
I think you fail to account for cultural differences. The "face scanning for toilet paper" may be unacceptable to you, but if it means no more vandalized public toilets, then there are bound to exist societies which would deem it an OK trade-off. I may be misinformed, but my impression of Chinese culture is that it prefers stability and order, sometimes at the expense of liberty and convenience.
That's not to legitimize the abuse of such means by the state, and I don't want to say that disappearing people from the street is fine, but lets focus on that abuse and camps, not the toilet paper. These are completely separate matters.
There is probably strong survivorship bias in how we judge past "movements". I could for example imagine that today's "climate movement" will become a "movement" in future history books.
I'm surprised the article didn't mention the cause of this change. I was under the impression that men were teaching Replika to be abusive and the company didn't like that:
Men Are Creating AI Girlfriends and Then Verbally Abusing Them "I threatened to uninstall the app [and] she begged me not to."
So, I think it's accepted the "copycat" phenomenon is a thing. Some people may use tools to trial their "comfort" and then a smaller amount may act it out in real life. There may be a subset who are satisfied with the fenced off playpen and don't see a need to go further.
Whatever the case may be, we probably should take a look into it.
I don't think it's that accepted. Spread of porn availability is strongly correlated with reductions in sex crimes in all countries around the world. This has even been trialed with child pornography, where we see a reduction in recidivism among child molesters if they are given access to animated CP.
If this copycat phenomenon exists with porn, it's far outweighed by the benefits of providing an outlet. It then becomes an ethical question of whether the aggregate social benefits outweigh being complicit in a small number of copycat downsides.
The same argument was trotted out for violent video games as well, and we similarly don't see any correlation between in-person violence and video games (actually, it's a slight negative correlation IIRC).
I just think people are far too quick to jump to simplistic narratives around human behaviour.
Thought experiment: it was at one point, but then humans were brought back in because the economy needed humans to be doing something to earn their spending money.
It ain't all that. Nice acting but it was really a shallow and trite story, exactly the same as every other cyber-Pygmalion but with no action sequences.
I don't really understand how people developed such deep relations with their Replika. I found it fairly shallow and it really just felt like talking to myself, and got very repetitive quickly. As a journal of thoughts that talks back to you and is supportive of anything you say it works okay...but it gets old quick.
As a sex roleplay bot, Replika was actually pretty good, since you could direct it anyway you wanted and it would go along. Again it was shallow and repetitive, but so is pretty much any sexual consumption online.
In my opinion, Replika's only good feature was the sexual roleplay...they got rid of it and now the product is basically dead. If I had to guess... most people who are "angry" actually just want their sexbot back (the alternatives are pretty bad in comparison). People can get very attached to sexual consumption (web cam girls, onlyfans), it isn't anything new and has very little to do with AI.
I don't see why pornhub doesn't create their own sexbot companion... they would make tons of money and wouldn't have to care about App Store guidelines or press reputation.
Agreed. And the whole thing really reminds me of Ted Chiang's short story The Lifecycle of Software Objects, but the real life version is much stranger than fiction. Similar ideas though.
>They have an AI R&D department, which colorizes and upscales classic porn.
I have no desire to watch porn but wonder what this actually looks like when done and compared to the originals. Are there any split video comparisons of their process?
The restoration process still needs work. Variations in exposure time, a big problem with early film cameras, seem to confuse their colorizing process.
It seems about what could be expected I suppose. I've seen some mainstream movies upscaled and colorized and it can make it feel fresh again but often feels 'off' somehow. I hope their bot gets better over time as suggested by the clip above.
Not so much for the stuff Pornhub is trying to rescue but to preserve older films and television shows in general. So many people I know absolutely refuse to watch anything in black and white and as a result they miss out on so much of the wealth of the past.
Hopefully as it improves they will be willing to extend some of their expertise to other more mainstream areas.
"Here's the pool of socially well connected people talking to each other, and here's the pool of lonely people that basically talk to nobody. Now, we could create new network edges between the lonely and non-lonely people, so that both groups of people can expand their social network and share their rich sets of deeply personal experiences with each other.
But clearly, a much better solution is to direct all that lonely energy to synthetic personalities running on data center VMs. So the people who are happy can continue to be happy, and the people who are lonely can shout into availability zones on the West Coast and Northern Europe. This is clearly the humanist solution."
> Now, we could create new network edges between the lonely and non-lonely people, so that both groups of people can expand their social network and share their rich sets of deeply personal experiences with each other
Yeah, as if you could just do that with a wave of your hand.
This seems like a joke, but humanity's feature is probably going to involve a LOT of talking to AI companions. So many people out there already have parasocial relationships with youtubers and streamers, it's only a matter of time before they have AIs that are "good enough" for them to chat with endlessly.
The lack of empathy in peoples comments here is really disappointing and disturbing. It is a fact of our society that a huge number of people are alienated and totally alone. People need connection an intimacy to be whole and healthy, and that isn't available to everyone. For someone that was totally deprived of that, getting some semblance of it, even if from an AI, and then having it taken away would be very hard.
Technology facilitated the atomized society, and then technology was applied as a salve, and then taken away. The amount of wreckage left in the wake of "moving fast and breaking things" seems to be approaching a breaking point.
Of our sanity maybe, or a breaking point of traditional liberal enlightenment values, but short of a global catastrophe I don't see anything halting the inexorable march of technological development and the social chaos that inevitably causes. Any attempt to stop this through political means would require Pol Pot tactics applied globally; aegrescit medendo.
With LLMs and voice generators particularly, I think the general public has been given access to military-grade information weapons. The chaos this causes will be used to justify crackdowns and regulation on the flow of information, to curb the automation of weaponized misinformation. New technology always spurns new regulations, so new information technology will beget new information regulations. The inexorable march of technology cannot be stopped, and so our freedoms will inevitably be curtailed until there is nothing left.
Or as another man put it:
> 114. As explained in paragraphs 65–67, 70–73, modern man is strapped down by a network of rules and regulations, and his fate depends on the actions of persons remote from him whose decisions he cannot influence. This is not accidental or a result of the arbitrariness of arrogant bureaucrats. It is necessary and inevitable in any technologically advanced society. The system HAS TO regulate human behavior closely in order to function. At work, people have to do what they are told to do, when they are told to do it and in the way they are told to do it, otherwise production would be thrown into chaos. Bureaucracies HAVE TO be run according to rigid rules. To allow any substantial personal discretion to lower-level bureaucrats would disrupt the system and lead to charges of unfairness due to differences in the way individual bureaucrats exercised their discretion. It is true that some restrictions on our freedom could be eliminated. but GENERALLY SPEAKING the regulation of our lives by large organizations is necessary for the functioning of industrial-technological society.
I think he does a good job of explaining that technology necessitates the loss of freedom, and that conservative politicans who oppose regulation are taking people for suckers to promote big business. Footnote 13:
> [13] Conservatives’ efforts to decrease the amount of government regulation are of little benefit to the average man. For one thing, only a fraction of the regulations can be eliminated because most regulations are necessary. For another thing, most of the deregulation affects business rather than the average individual, so that its main effect is to take power from the government and give it to private corporations. What this means for the average man is that government interference in his life is replaced by interference from big corporations, which may be permitted, for example, to dump more chemicals that get into his water supply and give him cancer. The conservatives are just taking the average man for a sucker, exploiting his resentment of Big Government to promote the power of Big Business.
The good news is, we'll get global catastrophes in spades!
>The inexorable march of technology cannot be stopped
More good news: global catastrophes can stop it!
On a more pedantic note, This general platitude has always bothered me because for all our technological advancements, it's not, at its core, true. Thanos snap your fingers and get rid of every human, and what happens to technology? ChatGPT isn't going to upload itself into the best Boston Robotics model on the market and ensure continuity of electronics manufacturing. The march might be inevitable because 8 billion humans can't agree on anything, but it is still a choice. Small pockets of people have in fact opted out of that march; maybe more will in the future, or maybe "we" collectively decide that such a path is for the best.
Vindication is a dish best served cold, I suppose. It honestly would not surprise me if more people are looking at that document with fascination as of late.
I take the view of humans being very innovative cockroaches (in the most endearing sense possible) with a strong will to survive. There are surely global catastrophes in store for us in the near future, climate change particularly, but I don't expect these to exterminate humanity or even do enough damage to seriously set back technological development for long. I expect mass migrations and major wars, but humanity has a track record of surviving these sort of things. We'd need something more in the magnitude of a comet strike to really unwind the technological-industrial revolution.
I am more willing to bank on climate feedbacks we don't even know about yet doing a number on our ability to keep modern infrastructure running on a wide scale. It depends on regular inputs of resources of no small portion of the periodic table as well as energy sources, and I do not think nuclear/renewables can sustain today's infrastructure.
And of course, there could be nuclear war, societal breakdown as mentioned, another global plague (H5N1? Marburg? Cordyceps? Prions? The possibilities are endless), or, my favorite, Butlerian Jihad. :)
It does seem like we need a ban on AI progression unless we can figure out AI alignment first, which seems unlikely. Unfortunately, I do not see people taking that seriously at all until it is too late. I feel like much of the sci-fi about this was written by intelligent people that could see where this was going, but they inadvertently poisoned it culturally. "ha-ha, you mean like terminator? get real, that's just a movie."
Is it surprising? People much more sympathetic than this are regularly mocked and condemned on this site. It's certainly bad but the cruelty is predictable.
Exactly. I can't personally understand people falling in love with, or having some kind of sexual relationship with an AI companion, but I can absolutely understand profound loneliness and the sorrow it must cause people to be suddenly rejected in this way.
The breadth of human experiences contains sadness that many people cannot even imagine. You may have people here who have all kinds of extreme difficulties forming regular relationships and yearn to be loved. Before we dismiss them we should consider that we have absolutely no idea of their past and how they got to this point. We should try our best to be kind.
"Replika is a tool for many people who use it to [b]support their mental health,[/b] and many people value it as an outlet for romantic intimacy. The private, judgment-free conversations are a way for many users to experiment with connection, and overcome depression, anxiety, and PTSD that affect them outside of the app."
Is this serious? I mean, it seems like this would worsen someone's mental health if anything. Being so alienated from others that they're having an intimate relationship with a computer program seems like a very worrying issue and they should talk to a therapist or something.
>Being so alienated from others that they're having an intimate relationship with a computer program seems like a very worrying issue
You could say something similar about becoming sexually aroused by looking at an image on a computer screen instead of actually having sex with another person. Just like sexual response, romantic attachment operates in a simple and predictable way. It can easily be triggered with artificial dialog, and is subjectively experienced the same way as in a "real" relationship.
It's easy to mock people who prefer virtual relationships to real ones, but mockery is often rooted in fear rather than comprehension of an issue. The dynamics here are fundamentally the same as those of religious/intellectual ascetics and drug users, where a person prefers to adhere to a consistent experience (notwithstanding possible costs) to an inconsistent or negative one.
This is often rational rather than maladaptive. Preferential attachment can run in negative as well as positive courses, such that while popular people tend to acquire more friends and so on, people who are socially perceived as lacking (the poor, socially awkward, crime victims etc.) are often further injured because they are perceived as vulnerable and safe to mistreat. A person whose experience of the social world is consistently negative may adjust themselves in response, but their cumulative experience is still going to shape their inner life.
I get and agree with your overall point but if you want me to believe people jerkin it to AI porn and monks have a motivation in common you've got some more work to do.
I think there's a lot of people who have tame virtual relationships (like the example I offered above). People who are jerking it to AI porn are more like the subset of religious devotees who go in for self-flagellation and various forms of bodily mortification. It's not a 1:1 mapping, of course. Umberto Eco's debut novel The Name of the Rose explores these questions in depth through a fictive lens.
I had started seeing strange ads on Facebook (stranger than my usual assortment of ads) for Replika like it was something new and even the ads seemed somewhat pointedly sexual, like that was part of the whole deal, but I never pursued it.
Many people yearn for connection, and even artificial connections have been alluded to for a long, long time. Think Kate Bush's "Deeper Understanding," or that Pictures for Sad Children bit where someone has a "porn" DVD called A Japanese Woman Fries an Egg and Asks You About Your Day. It is unlikely at best that we are alone in our feelings and our internal experiences, yet most of us still wish to communicate about them.
As with anything, there's the Have Nots, and to go with that are the Haves, who are very blithe about the situation; if prodded, they'll recite something like bootstraps and apparently sage advice beginning with the magic phrase "all you have to do is." The Have Nots have come to understand that they'll never get anything like sympathy (mockery only, please) from the Haves, and just one little bit more of comfort is pried away from them, like yanking a ratty towel away from an orphaned baby monkey. No towel for you. Toughen up.
One less thing to keep the Have Nots occupied. Here's hoping ongoing legalization of marijuana and video games can keep them satiated to some degree, because a significant surplus of angry, lonely, dissatisfied young men tends to be pretty bad for the society they're in sooner or later. Nice civilization you got there, be a shame if someone were to happen to it.
> even the ads seemed somewhat pointedly sexual, like that was part of the whole deal
The sexual aspect wasn't just "part of the whole deal"; it was a core feature of their product. Replika's ad campaigns specifically advertised features like "role play & flirting" and "hot photos" as benefits of a paid membership.
Don't forget how popular bonzi buddy was back in the day, or the nsfw desktop dancers app. I think when they combine something like that with a chatgpt style bot that will be the tipping point to people wanting an AI personal assistant.
Already seeing people saying they want a new Clippy with bing powered chat.
89 comments
[ 0.28 ms ] story [ 166 ms ] threadTell me you’ve never had an actual crisis, without telling me you’ve never had an actual crisis.
The real issue are the conditions that allowed this to happen. Modern society is a genuine nightmare for real human connections. I can make a million superficial connections with a billion people.
But finding deep, meaningful community is harder now than it has ever been. That's exhausting, and I have to believe it's one of the main contributors to the mental health crisis we are currently experiencing (at least in the US).
So when those relationships are arbitrarily yanked away from them, it's alienating. It's so harmful we long ago figured out it could be weaponized, and now we call it things like "disfellowshipping" and "shunning."
Pain and trauma don't come directly from external events, but from our inner emotional response to them. Just because it might seem weird to you that someone could be deeply attached to an AI doesn't mean that their experience is somehow easier for them than things you would accept as an "actual crisis."
You can't gatekeep pain. I can believe that for some people, this would be as painful as being rejected abruptly by a real life partner in a serious relationship.
There's a point where, if your life sucks enough and you don't have the social skills to hide it, you can be stuck in a bad place with no obvious way to get help.
Maybe paying for a therapist will help, or maybe you won't have the money, the waiting lists will be too long, your country's healthcare system will limit you to wrong matches. If this way doesn't work, what's your next step?
Out of curiosity, what changed?
- I traveled, met people who had had a much better life and way more success, and basically got angry/ashamed enough that mine sucked that I got motivated to change it,
- I cut ties with a couple friends/family members who had issues of their own (that they refused to work on),
- I'm a competent tech worker, which means no money issues, less stress, social contact through school/work, and the morale boost of one part of my life going right at least,
- having a few good (now remote) friends who helped me navigate through things.
People who stay in bad situations probably don't have most of these things. GGP doesn't seem to know how hard it can be to fix this. Years after the bad times, I still have some social/emotional issues. And these were mild problems.
A part of those who are not able to get a partner may solve their problem by "just" meeting women, but for many, there are significant psychological issues.
What do you expect a guy who forms an emotional relationship with a chatbot thing to accomplish with a dating app? Most likely would only induce further despair because he would get few, if any, likes.
The cutting edge will always be in the cloud but a market for personal AI does exist.
https://news.ycombinator.com/item?id=12657119
Add an holographic projection and this is pretty much Joi from Bladerunner.
So when companies monetize the loneliness people feel towards each other by selling them subscriptions to AI chatbots we might only see the cutting edge dystopian tech, but remember that without the underlying conditions, this wouldn't be so cyberpunk. Maybe in a healthier society these chatbots could be used for something less desperate.
The amazing surplus and living standards of the post war years were an aberration
You can’t organize online or even move around in they city without The Party knowing about it and sending you to a disappearing camp.
Yesterday on Reddit someone showed how now the toilet paper in public restrooms requires you to have your face scanned to dispense!
That's not to legitimize the abuse of such means by the state, and I don't want to say that disappearing people from the street is fine, but lets focus on that abuse and camps, not the toilet paper. These are completely separate matters.
Men Are Creating AI Girlfriends and Then Verbally Abusing Them "I threatened to uninstall the app [and] she begged me not to."
https://futurism.com/chatbot-abuse
Whatever the case may be, we probably should take a look into it.
If this copycat phenomenon exists with porn, it's far outweighed by the benefits of providing an outlet. It then becomes an ethical question of whether the aggregate social benefits outweigh being complicit in a small number of copycat downsides.
The same argument was trotted out for violent video games as well, and we similarly don't see any correlation between in-person violence and video games (actually, it's a slight negative correlation IIRC).
I just think people are far too quick to jump to simplistic narratives around human behaviour.
https://www.imdb.com/title/tt1798709/
Even through its from ten years ago it does a decent job of depicting what is happening now.
But most importantly, this is a very similar product as the one depicted in the film.
As a sex roleplay bot, Replika was actually pretty good, since you could direct it anyway you wanted and it would go along. Again it was shallow and repetitive, but so is pretty much any sexual consumption online.
In my opinion, Replika's only good feature was the sexual roleplay...they got rid of it and now the product is basically dead. If I had to guess... most people who are "angry" actually just want their sexbot back (the alternatives are pretty bad in comparison). People can get very attached to sexual consumption (web cam girls, onlyfans), it isn't anything new and has very little to do with AI.
I don't see why pornhub doesn't create their own sexbot companion... they would make tons of money and wouldn't have to care about App Store guidelines or press reputation.
They might. They have an AI R&D department, which colorizes and upscales classic porn. As they point out, they have lots of training data.
I have no desire to watch porn but wonder what this actually looks like when done and compared to the originals. Are there any split video comparisons of their process?
The restoration process still needs work. Variations in exposure time, a big problem with early film cameras, seem to confuse their colorizing process.
[1] https://www.pornhub.com/art/remastured
It seems about what could be expected I suppose. I've seen some mainstream movies upscaled and colorized and it can make it feel fresh again but often feels 'off' somehow. I hope their bot gets better over time as suggested by the clip above.
Not so much for the stuff Pornhub is trying to rescue but to preserve older films and television shows in general. So many people I know absolutely refuse to watch anything in black and white and as a result they miss out on so much of the wealth of the past.
Hopefully as it improves they will be willing to extend some of their expertise to other more mainstream areas.
But clearly, a much better solution is to direct all that lonely energy to synthetic personalities running on data center VMs. So the people who are happy can continue to be happy, and the people who are lonely can shout into availability zones on the West Coast and Northern Europe. This is clearly the humanist solution."
https://www.youtube.com/watch?v=ajGX7odA87k&t=1291s
Yeah, as if you could just do that with a wave of your hand.
Of our sanity maybe, or a breaking point of traditional liberal enlightenment values, but short of a global catastrophe I don't see anything halting the inexorable march of technological development and the social chaos that inevitably causes. Any attempt to stop this through political means would require Pol Pot tactics applied globally; aegrescit medendo.
With LLMs and voice generators particularly, I think the general public has been given access to military-grade information weapons. The chaos this causes will be used to justify crackdowns and regulation on the flow of information, to curb the automation of weaponized misinformation. New technology always spurns new regulations, so new information technology will beget new information regulations. The inexorable march of technology cannot be stopped, and so our freedoms will inevitably be curtailed until there is nothing left.
Or as another man put it:
> 114. As explained in paragraphs 65–67, 70–73, modern man is strapped down by a network of rules and regulations, and his fate depends on the actions of persons remote from him whose decisions he cannot influence. This is not accidental or a result of the arbitrariness of arrogant bureaucrats. It is necessary and inevitable in any technologically advanced society. The system HAS TO regulate human behavior closely in order to function. At work, people have to do what they are told to do, when they are told to do it and in the way they are told to do it, otherwise production would be thrown into chaos. Bureaucracies HAVE TO be run according to rigid rules. To allow any substantial personal discretion to lower-level bureaucrats would disrupt the system and lead to charges of unfairness due to differences in the way individual bureaucrats exercised their discretion. It is true that some restrictions on our freedom could be eliminated. but GENERALLY SPEAKING the regulation of our lives by large organizations is necessary for the functioning of industrial-technological society.
I think he does a good job of explaining that technology necessitates the loss of freedom, and that conservative politicans who oppose regulation are taking people for suckers to promote big business. Footnote 13:
> [13] Conservatives’ efforts to decrease the amount of government regulation are of little benefit to the average man. For one thing, only a fraction of the regulations can be eliminated because most regulations are necessary. For another thing, most of the deregulation affects business rather than the average individual, so that its main effect is to take power from the government and give it to private corporations. What this means for the average man is that government interference in his life is replaced by interference from big corporations, which may be permitted, for example, to dump more chemicals that get into his water supply and give him cancer. The conservatives are just taking the average man for a sucker, exploiting his resentment of Big Government to promote the power of Big Business.
>The inexorable march of technology cannot be stopped
More good news: global catastrophes can stop it! On a more pedantic note, This general platitude has always bothered me because for all our technological advancements, it's not, at its core, true. Thanos snap your fingers and get rid of every human, and what happens to technology? ChatGPT isn't going to upload itself into the best Boston Robotics model on the market and ensure continuity of electronics manufacturing. The march might be inevitable because 8 billion humans can't agree on anything, but it is still a choice. Small pockets of people have in fact opted out of that march; maybe more will in the future, or maybe "we" collectively decide that such a path is for the best.
Vindication is a dish best served cold, I suppose. It honestly would not surprise me if more people are looking at that document with fascination as of late.
And of course, there could be nuclear war, societal breakdown as mentioned, another global plague (H5N1? Marburg? Cordyceps? Prions? The possibilities are endless), or, my favorite, Butlerian Jihad. :)
The breadth of human experiences contains sadness that many people cannot even imagine. You may have people here who have all kinds of extreme difficulties forming regular relationships and yearn to be loved. Before we dismiss them we should consider that we have absolutely no idea of their past and how they got to this point. We should try our best to be kind.
Is this serious? I mean, it seems like this would worsen someone's mental health if anything. Being so alienated from others that they're having an intimate relationship with a computer program seems like a very worrying issue and they should talk to a therapist or something.
You could say something similar about becoming sexually aroused by looking at an image on a computer screen instead of actually having sex with another person. Just like sexual response, romantic attachment operates in a simple and predictable way. It can easily be triggered with artificial dialog, and is subjectively experienced the same way as in a "real" relationship.
It's easy to mock people who prefer virtual relationships to real ones, but mockery is often rooted in fear rather than comprehension of an issue. The dynamics here are fundamentally the same as those of religious/intellectual ascetics and drug users, where a person prefers to adhere to a consistent experience (notwithstanding possible costs) to an inconsistent or negative one.
This is often rational rather than maladaptive. Preferential attachment can run in negative as well as positive courses, such that while popular people tend to acquire more friends and so on, people who are socially perceived as lacking (the poor, socially awkward, crime victims etc.) are often further injured because they are perceived as vulnerable and safe to mistreat. A person whose experience of the social world is consistently negative may adjust themselves in response, but their cumulative experience is still going to shape their inner life.
Many people yearn for connection, and even artificial connections have been alluded to for a long, long time. Think Kate Bush's "Deeper Understanding," or that Pictures for Sad Children bit where someone has a "porn" DVD called A Japanese Woman Fries an Egg and Asks You About Your Day. It is unlikely at best that we are alone in our feelings and our internal experiences, yet most of us still wish to communicate about them.
As with anything, there's the Have Nots, and to go with that are the Haves, who are very blithe about the situation; if prodded, they'll recite something like bootstraps and apparently sage advice beginning with the magic phrase "all you have to do is." The Have Nots have come to understand that they'll never get anything like sympathy (mockery only, please) from the Haves, and just one little bit more of comfort is pried away from them, like yanking a ratty towel away from an orphaned baby monkey. No towel for you. Toughen up.
One less thing to keep the Have Nots occupied. Here's hoping ongoing legalization of marijuana and video games can keep them satiated to some degree, because a significant surplus of angry, lonely, dissatisfied young men tends to be pretty bad for the society they're in sooner or later. Nice civilization you got there, be a shame if someone were to happen to it.
The sexual aspect wasn't just "part of the whole deal"; it was a core feature of their product. Replika's ad campaigns specifically advertised features like "role play & flirting" and "hot photos" as benefits of a paid membership.
There's a collection of these ads (some NSFW) at:
https://www.reddit.com/r/replika/comments/113wc5x/lets_gathe...
Already seeing people saying they want a new Clippy with bing powered chat.
This is my new default expansion for the acronym ERP.