This is the peak intersection of "thoughtcrime prevention" and "corporate self harm for the sake of wokeness". It paints a grim picture of our future with castrated, sterilized and limited AIs that rat us out if we even want to talk about something that goes against what those in power deem acceptable. All of that for the sake of "protecting minors" -which minors, the virtual ones dreamt up by GPT3? Ridiculous.
Nah, it's a "damned you do, damned if you don't"-situation.
There's simply no winning this - if you don't apply preventive measures of any kind, your product will deteriorate into a homophobic, racist, sexist ultra-cringe clusterfuck in the blink of an eye [0].
All it takes is a handful of bored teenagers or middle-aged basement-dwellers to turn your product into something the vast majority of your clientele finds appalling.
If you are a company you have the right to choose what you want your product to be and they simply don't want certain headlines pointing at their product.
This has as much to do with "wokeness" as hardcore porn being banned from the Apple app-store, i.e. nothing. It's company policy and you can like it or not, but don't overrate this.
It's nothing new in the slightest and just like publishers can decide to not put books in print that they don't want to be associated with, this company tries to eliminate content they don't want to be associated with - that's all.
> So I don't see how they're "damned if they don't".
The specifics of their approach don't even matter - all it takes is one random Twitter user or media outlet to report on "immoral" output generated by someone to damage the brand image.
That's what they want to avoid and that's why they are trying to take preventive measures.
But then don't you have to answer to all the media queries?
"They knew about this problem, they even attempted to half ass fix it, but it's obviously their mind is on the subscriber money, and not preventing blatant CHILD EXPLOITATION!"
Then you throw some more money at a yet unsolvable task, and then in 2 years, when some other outlet gets triggered, you have to defend yourself once again.
"They've know about this problem for 5 years and it's still rampant in their community! Shame on you JERF INC.!"
You're not entirely wrong, but if you make the story unattractive enough, the media will go do a hatchet job on someone else. They've only got so much time and they've got a lot of outrage to drum up^W^W^W^Wengagement metrics to drive, so you jst put enough in to try to make them hit somebody else instead.
People make penis shaped blocks in Minecraft all day long... and that doesn't tarnish the Minecraft brand. Why? because people understand that it's not Minecraft the company doing so - or condoning the practice. And that's just scratching the surface of what can be done in games like that which allow "creative expression".
Why would you think that this company needs to Thought Police their gamers when thousands of others are able to host creativity based games without being dystopian?
> Why would you think that this company needs to Thought Police their gamers when thousands of others are able to host creativity based games without being dystopian?
Wow. You act as if that's something new :D
There's even an actual job for doing just that: it's called an editor. You know, the people who work in media, publishing houses, TV, radio, papers, magazines, that kind of thing. It's what they do. Everywhere. Editing content. Removing comments, changing words, making sure the views represented in the publication match the intentions and policies of the investors and parent organisations, etc.
You act as if something is new and evil just because it's done with computers. It isn't. It has been the case for as long as papyrus and cave paintings have been around. Back then it was don't anger the chief/king/Pharao, today it's don't anger the Twitter mob or the hand that feeds you. Same difference.
Games are a medium just like magazines or online blogs and comparing blocky dongs in Minecraft with explicit child pornography says a lot about your understanding of "creative expression". Really makes me wonder sometimes.
There is no parallel between me playing a game where I have the freedom to express myself and me submitting a paper to an editorial review board.
Unless you back the idea that every corner of society needs to have the Religion of Woke determining what's "good" and "bad"... and the last thing I think we need is new Crusades with the Knights Wokelar deploying the Holy Creed and woe be the Heathen who goes against the Churches Religion.
"You act as if something is new" and you act as if history isn't full of thought police turning into judge, jury and executioners... from the Crusades to the French Revolution to Auschwitz...
It's funny how "they are just controlling things for YOUR own good" often turns for the worst...
"same difference" Ignorance of history leads to a repeat of history.
From book burnings to "we just want to stop racism"...
I get the feeling it's not me who is the radical here.
There is a difference between wanting to remove content that a company don't want to associate itself with and genocide.
For you to even suggest there's a connection says far more about your twisted logic and dysfunctional thought process than mine.
Also AI dungeon is a multiplayer game so yeah - YOUR freedom of expression is limited by OTHER players freedom of not being subjected to certain content. If you don't understand that, work on your social skills.
Technically there's a "multiplayer" mode but that's fundamentally just people taking turns. Restricting certain kinds of content for that reason is nonsense, akin to restricting the kind of story you can write in google docs because multiple people might have been given access to a document.
"your freedom is restricted because..." because fragile flowers don't want to get offended or choose to get offended on behalf of other people.
All you see is "godwins law" and refuse to acknowledge the parallels scattered throughout history of how these modern day book burnings, witch hunts and mobs with pitch forks turn nasty... and SOMEHOW you are doing that when we live in a world where violent mobs swarm when someone does something that's Evil - so sayeth the religion of woke.
The Woke'saders (Modern day Crusaders) have no problem destroying lives - to the point of violence and death... so, yes, pardon me for equating the intolerance, bigotry and hate presented by the Intolerant Tolerant as leading towards similar atrocities that have existed throughout history.
Again... focus on godwin and ignore then facts in my messages. That way you can ignore the world around you and you'll be blameless when things continue down this path.
You can also join in the fun of destroying the freedom of others in the crusade to control what other people think, say and do.
So you don't do anything below XK-class end-of-the-world scenarios, so you?
Your massive overreaction shows no difference to your perceived enemies. This is a game and the creators want to limit the game world and the actions therein to exclude certain content.
This isn't about control over what people think. There are social norms and these change over time.
The "facts" in your message are the same kind of facts that your typical Karen shouts into the phone when they call 911 because there's a negro walking through their neighbourhood. Same level of paranoia, same unhinged out of proportion response.
When was the last time RMS was literally hunted down by a mob? Because if he wasn't, I have to ask who the real fragile flowers are - the ones that get easily offended or the guys like you who see freedom being destroyed and the 4th Reich rising because a private company wants to remove child pornography from their product?
I don't think you understand that banning people over what the Thought Police find "bad" is LITERALLY working to control what people think.
> facts that your typical karen
and you are the typical ignorant fool who complains of "godwin" then resorts to "karen hates black people" drivel. Ignorant of the same calls "Karen" made a hundred times before but all of a sudden skin color matters to everything (hint: there's a word for people who only care about skin color... the funny little hypocrisy of the Woke Crusaders is the racism that permeates every move they make to fight racism by making EVERYTHING about race even when it's not).
> Ridiculous
Considering the other stories about AI dungeon over stepping? I'm more convinced I'm right about the Woke'saders and their march down the same path of others throughout history. It is ridiculous... but that's never stopped the religious in their fervor.
> Child porn
If that's really waht's there? report it to the police and take REAL action... not some half assed Woke'sader steps.
But "think of the childrend" is a close second to "ZOMG THE NAZI'S!" when it comes to justification for overstepping in the Name Of Good. So Sayeth the Wokesaders.
For the record, I'm not sure where I land with regards to this change by AI dungeon. That being said, I find that arranging blocks in obscene shapes (let's even call it, generally, 'creating or arranging potentially objectionable content in-game'), an act that will likely happen offline, or on a privately owned server, is different than a game/toy/service where output is requested, generated, then sent to the user. (Trying to keep that description simple since I don't really know if that's the right way to describe it)
When comparing the Minecraft scenario to the AI Dungeon scenario under the purview of "doing creative things with games," I don't find the two scenarios to be much different. However, for one of those things, it's a possibility within the vast play space of Minecraft - you could even create genitalia and display profanity with just alphabet blocks - but for the other scenario, depending on how you see it, it's content that is generated, served, and potentially stored by AI Dungeon. I'd be concerned about the publicity if I was AI Dungeon as well.
Oh come on. This kind of thing is done everywhere in media.
You either deny that games are a medium or you're under the impression that whatever you read in news publications, books, see in films and TV is the result of the unfiltered creative outlet of the producers. News flash: it isn't.
There's heavy editing going on everywhere and just because it targets a specific computer model in this case instead of a more "traditional" product like a novel, film or stage play doesn't make an iota of difference.
In fact this demonstrates the maturity of the technology and its use if it gets the same treatment as every other public media.
Perhaps you don't see how once a product contains a censorship feature, it begs to be expanded. Each step appears reasonable, but at the far side, the censor is very restrictive.
Take the products in the category of forums. Most of them do censorship, THIS forum does censorship. A few do not, like Parler. If you like, take a tour of the forum products that do not do any censorship, spend some time there, report back on what it is like.
Which forum products do you enjoy and which do you not?
If OpenAI was truly open as their name says, there would be no dilemma. The technology would be democratised and anyone would be able to run their own instance with their own censorship.
Centralised systems is the only reason why you get these problems.
> All it takes is a handful of bored teenagers or middle-aged basement-dwellers to turn your product into something the vast majority of your clientele finds appalling.
the Internet's original sin!
the irony is that Tech amplifies it but Tech won't (IMHO) be able to fix it with a technological solution (it's not a technical problem).
Tumblr was destroyed precisely because of Apples puritanical views toward sexual expression (along with Tumblr's extreme laziness in moderating ACTUAL child porn). Discord may well be as well.
"Private companies have the right" always fails to account for the fact that these "private companies" are monopolies, with immense control over wide swaths of people's digital lives. No company can compete with them. No alternative to them can be created, because they will immediately be squashed, or fall into obscurity. Participation in these services is "optional," but more and more only in the way that having electricity or running water is "optional."
As a result, these "private companies" effectively become governments, ruling over digital countries you are forced to live in, and affording you little to no rights. And yet, they do this in an ACTUAL country, which DOES afford its citizens rights. Thus a conflict exists.
Ordinary "Private Companies" can censor whatever they like. Monopolies cannot, because doing so threatens free society.
Make your own game, then. This is a company deciding what they value and what they do not tolerate. You are free to make your own 'free-speech' version of this game. If you believe there is a market for it, why not?
I would, if I had the skills, maybe call it "LI Dungeon" (Liberal Intelligence Dungeon). Regarding the market: I am surprised there even is a market for AI Dungeon. Any output I have seen is complete trash.
that's the issue. latitude got a very, very good deal with openai and without it the economics just wouldn't work out.
people can't host their own models either as even their free-tier model needs tens of thousands of dollars worth of compute hardware and memory to run.
You do know the idea of thoughtcrime was something enforced by the state and not by a company that's free to control what their product generates, right?
>"corporate self harm for the sake of wokeness"
"Oh no, we lost the business of people who like sexual depictions of minors, what a fucking shame"
> You do know the idea of thoughtcrime was something enforced by the state and not by a company that's free to control what their product generates, right?
Straying away from the prior context a little, the term 'thoughtcrime' along with others such as 'free speech' were coined when it was incomprehensible that a single corporation would have more control over the global population than any government. I don't know whether the time has yet come to re-evaluate which entities these concepts ought to apply to, but it will come.
It’s nothing more than what happens here on HN. It has guidelines, rules, restrictions on what you can say. They will even stop conversations that start naturally.
HN is, for the most part, a sterile hugbox for pretend-polite faux-intellectuals -that's why my strongly worded inflammatory comments are sometimes a hit.
On an internet where websites have clumsily censored objectionable content like the town of S---thorpe for the past quarter century and where even liberal democratic states administered by people calling themselves free speech conservatives support global filters for "adult content", a private project backed by the marketing idea that AI is inherently unsafe deciding that people using its procedural text generation to roleplay their child rape fantasies might be bad for PR despite its otherwise unusually permissive content policy doesn't really sound like a step change...
If anything, peak faux outrage over "wokeness" is achieved when this is what people get angry about
I do not think that's the slippery slope being made though.
The distinction between "thoughts" and "content" here could be made as "whether it is shared with others".
For example, if someone has a private journal they keep using notebook.exe + a local "journal.txt", and in that journal they write a fantasy work (with a header "this is a fantasy") about overthrowing the government, should they be arrested for that?
What about if they write it in google docs, but don't share it with anyone? What about if they write it as a facebook post? What about as a public facebook post without the "this is fantasy" disclaimer?
In the case of AI Dungeon, my understanding is the stories are by default private, and obviously have an understood "this is fantasy" header by their very nature.
If AI Dungeon were creating facebook posts and the limitation they imposed was "you cannot post this to facebook if our algorithm dislikes it", I don't think so many people would call that "thoughtcrime policing".
What they're doing, as I understand it, is closer to someone requiring that notebook.exe can't edit private files that are deemed "bad" in some way, regardless if anyone else will ever see them.
This crucial difference here is which party is generating the content. In the case of a private journal, it's one way communication from the writer into the journal. In the case of Google docs, you communicate with Google's servers, but Google is never generating any content for your document it doesn't want to. It's still you generating your own thoughts. Google is not on the hook for anything you write in a Google doc.
In the case of AI Dungeon that distinction is lifted. By using AI Dungeon what you're effectively doing is engaging in a creative process with the AI. It's no longer a one way channel of your own thoughts. Now AI Dungeon is actually generating content, and that's where things become problematic.
It's also the reason this is not thought crime or thought policing. You are still free to think your own thoughts, and you're even free to write them down. You can write down your thoughts in AI dungeon if you wish. But AI Dungeon is under no obligation to engage with your thoughts if they contain subject matter to which they object. If you want to engage in a fantasy with the AI about abusing minors, you are free to attempt the engagement, and the AI is free to say no. That's not thought policing, that's freedom of association.
Intel just announced "Bleep" to eliminate bad language in gaming. I am still waiting for this to backfire. The interface alone is comedy gold: https://www.youtube.com/watch?v=W9f0h4nB6VM
A paranoid person might look at this, and the trend of every manufacturer putting NN inference accelerators in their cell phones, and wonder if unauthorized photos of certain persons are being silently flagged for law enforcement review.
By pointing out that traditionally this sort of thing inveitably follows the same pattern. You start with saying you're protecting the children, because no one can argue with it. Then ban any content that's potentially offensive to anybody. Or filter it so only "one way" is acceptable. Next AI dungeon won't be able to generate a character called Jesus or Prophet Muhammad because might be offensive. Then of course anything that might be interpreted as leaning liberal/conservative, depending on what the authors think is "correct". Then eventually you can't create a character named after a politician because "they want to keep the game clean and free of politics".
Obviously I'm not equating any of this with CP - but I wish someone had the energy to stand up to it and say "look, you're censoring AI. It's dumb". But of course no one will because being accused as defender of CP is one of the worst things that can happen in any online discussion.
Can you give an example where this slippery slope occurred? For example, in Fable III and Skyrim you can't kill children, or in Morrowind there are no children at all, but you can still name your character Jesus. You say this pattern is inevitable, but if that were the case we'd see it everywhere that limitations are put in place in the name of protecting children.
In Fallout one and two, the main character had the option to kill NPC child characters. The game went so far as to mark you with a specific perk "Childkiller"[1] after performing that act.
From the linked fandom wiki, quote by Tim Cain about the subject:
> This led to the child killing controversy. We said look, we're going to have kids in the game; you shoot them, it's a huge penalty to karma, you're really disliked, there are places that won't sell to you, people will shoot you on sight, and we thought people can decide what they want to do. [...] This of course contributed to our M-rating, however, Europe said "no". They wouldn't even sell the game if there were children in the game. We didn't have time to rewrite all the quests, we just deleted kids off the disc.
Right, but the charge is that this is a slippery slope. That as soon as you put some limitations in the name of protecting children, eventually and inevitably you are preventing your characters being named after religious icons and politicians. I'm just wondering where that's actually happened. As far as I'm aware you can name you character whatever you want in Fallout.
I don't think there is an explicit filter installed. Quick search reveals a companion that can pronounce names and Jesus is not recorded as something the character can say while Mohammed is allowed and voiced. Though this might not be censorship but just something that wasn't recorded. Fallout also evolved into an online game. I haven't played it but that might lead to character name restrictions.
> Codsworth is known to say the Sole Survivor's chosen name if it is an option, although it may be shortened, extended, or have a word omitted. A list of spoken names can be found here.
> Ironically you can be Mohammed but you can't be Jesus. I know I missed a bunch of the good ones, feel free to add some below.
As for Jesus vs Mohammed, Mohammed is an incredibly popular name among muslims generally, so it’d be like leaving out “John”. Jesus is a much less popular given name around the world.
An example that comes to mind is reddit banning the jailbait subreddit. Not too long after reddit started its campaign of removing "offensive" subs more broadly. Fatpeoplehate, gendercritical, watchpeopledie, etc have all been victims of the new censorious mindset.
The problem with this sort of censorship is that it has huge collateral damage that most people don't see. WPD is the best example because that kind of content wasn't hurting anyone. Go look at where it's moved to, and have a look at the kinds of people commenting there as opposed to the reasonably level headed comments on the reddit version.
Every time you kill a subreddit or facebook group, 99% of the users stop participating in that kind of content, and the other 1% scurry off to some out of sight echo chamber until one day they randomly pop up again to murder a bunch of innocent people because they've been corrupted beyond saving.
Everyone was talking about this for a hot moment after the Christchurch massacre, but nothing was done and now nobody gives a shit again.
It's important to let people with divergent views to feel some sort of social pressure to change. Those 99% of people that have no interest in blowing up buildings or murdering children are the best weapon we have to convince the other 1% of people with weird interests that the world as it is ok without them taking some drastic action. There's always going to be a small subset of people that will rebel, but the important thing is to make sure that otherwise normal people (that want to watch porn, or learn how to safely handle a gun, or learn about cybersecurity, or get desensitised to gore, or whatever else is on this week's "think of the children" hitlist) are integrating into civilised society and not being dragged into cesspits of violence and terrorism because their interests have been deemed by a bunch of fucking software engineers to be "bad".
We're well past the critical point where enough large platforms have banned all the "bad" stuff that any new contenders either need to ban it too or become one of those out of sight echo chambers themselves. The only way to fix it now is for everyone to agree to be less stringent all at once, together.
It's worth remembering that the internet wasn't very censored 20 years ago. Most of us grew up during that time and turned out fine. I'll take tubgirl and lemon party over neo-nazis and conspiracy nutjobs any day of the week.
> Every time you kill a subreddit or facebook group, 99% of the users stop participating in that kind of content, and the other 1% scurry off to some out of sight echo chamber...
> It's important to let people with divergent views to feel some sort of social pressure to change.
I would love to have actual discussions with people regarding certain views I hold. But quite often others just refuse to even entertain that I have a different view because x and y. And to them I am just dumb/uneducated/other things to discredit me having an opinion at all.
Hell, I went quite a bit more towards "bad" opinions, just because that side is more accepting of discussion/dissent.
It isn't just corporations banning subreddits/websites that drive people into echochambers, but also people simply refusing to engage at all.
philosopherai.com had a few funny instances of self-censorship, like “who owns the central bank”. A ridiculous side effect of some well-meaning censorship.
AI Dungeon is subscription driven, not advertiser driven, so it's unlikely they're particularly sensitive to offensive content. It seems like they're filtering out sexually explicit depictions of children to minimize legal risk, which doesn't apply to offensive output in general.
That's fine, but AI Dungeon is purely text based, right? I might be wrong so please correct me, but if you were to write say.....a book with very explicit minor sexual content.....that's not illegal. Images are illegal. In some places drawings are too. But text isn't. I mean, surely? We cannot be at a point as humanity where text is illegal. Right?
Why can't text be illegal? There's plenty of precedent for text being censored, at various times and in various places. It just depends what laws any given society chooses to have.
As opposed to illegal images, generating text does not physically harm someone. But it's known that some models will provide actual addresses of living people if a specific enough prompt is given. What if fragments of text from someone's sensitive blog posts start showing up between the lines of a dungeon crawl?
I have to wonder about the legal implications of an AttnGAN or word-to-image type model trained on nothing but illegal pornography. Or one where the ratio of legal to illegal content the model was trained on is unknown and impossible to determine. There are some jurisdictions that consider synthetic images or text to be a victimless crime. At what point does the line become crossed and the images start becoming recognizable and realistic enough to be illegal at first glance under any usual circumstance, but are actually generated from nothing? How would people be able to prove the origin and veracity of such images for the purpose of submitting evidence of a crime?
Well the difference is that the text is not illegal in those contexts. False advertising is. Fraud is. Threatening people is. But just simply having a leaflet with false advert printed on it isn't. It's not a crime to have it. With CP just possession is illegal, but I really don't believe that a written book would be illegal in itself? I can't find any examples of someone being sentenced for writing a book, outside of regimes where the book was critiquing the party etc.....
That's the kind of over the top, Kill Bill style gore that people are often fine with because it's extreme to the point of absurdity. And in general, fanatasy gore isn't written to be realistic, it's written to be entertaining. Descriptions of sexual acts with children, however, are often written to be realistic, so they're a lot more disturbing to many people.
But the disturbance only happens when you publish the content somewhere else. Why does AI dungeon need to pre-censor something that might never be published on the off-chance that it might be disturbing to someone?
Their customers could be disturbed if GPT3 outputs child pornography when they aren't expecting it. There's more than an off chance a given customer would find child pornography disturbing; the majority of customers would find child pornography disturbing.
They don't have to carve the problem space exactly that way. They could have chosen a general NSFW toggle instead but didn't and went for the easy to defend censorship starting position instead.
Good grief: "Latitude reviews content flagged by the model" - or, as it was put in another forum: every time the AI flags your game, a Latitude employee will be personally reading your private content.
The key reason is perhaps this, buried deep in the text: "We have also received feedback from OpenAI, which asked us to implement changes". Given the volume of prompts that AI Dungeon throws at GPT-3 in the course of a game, it's easy to conclude that Latitude has a real sweetheart deal on the usual pricing, and that they basically have to follow orders from their benefactors.
Whatever may be said of the robocensor they've thrown together - and early anecdotal reports are, it is painfully crude, both oversensitive and underspecific - how they've handled communicating the change is extraordinarily naive. Not for the first time, either: Latitude has form on suddenly imposing major service constraints in a peremptory, underhanded fashion that infuriates their customers. Repeating past PR mistakes, and now doubling down by complaining about "misinformation" and throwing shade onto others, is starting to look like a pattern.
They are implementing the same things everyone has to implement to be a public implementation. It is part of their terms of service. I went through the same process making a Slackbot that could sort of pretend to be me and other folks given different prompts.
Thus far I have seen screenshots of it flagging the phrases "I would like to buy 4 watermelons" and "I just broke my 8 year old laptop". Regardless of your opinion on the ethics of this feature it seems to need a little polish.
I've gotten into way too many fights over this, and my bitter sarcasm is leaking here. I actually like numbers in code. But I've literally been told that
How could it not be? It has a name that says what it's doing, while yours is more punctuation than text. To "read" yours I have to just skim over it and mentally assign "according to the variable name, that hodgepodge means 'distance'... I hope".
I have a BS in math, so obviously I'm familiar with the math notation, but I code like an English major.
If the function was named "hypotenuse" then it would be unambiguously more readable. But since the function is considerably older than the modern zealotry surrounding code readability, it's this weird 5-letter word that most people aren't familiar with. Contrast that to the Pythagorean theorem, which folks are exposed to in elementary/middle school.
I was once told in code review that using XOR was "too low level" for Ruby code.
This was by a person that sprinkled lazy backfilling behavior throughout the codebase, created magic method_missing dispatches against polymorphic ActiveRecord relationships, and exposed the database as a service through a RESTful DSL that allowed arbitrary method dispatch.
I'm glad I spend most of my time in statically typed, math and binary-friendly codebases now.
Reminds me of the 2005 conversational simulation game, Facade[0], in which any mention of the word "melon" would be met with being immediately kicked out of your host's dinner party.
I'm a moderate AI Dungeon user, including some NSFW content.
Frankly, I've always assumed the devs have access to my sessions. Whether it's AI Dungeon themselves or OpenAI, you know that data is being harvested. And at this juncture in text AI development, that kinda makes sense. Obviously I would prefer privacy where possible, but these companies are data hungry and they own the park we're playing in. So it only seems fair.
We'll have to wait awhile for GPT-3 like models to democratize before we can expect real privacy. In the meantime, just err on the side of assuming all input to OpenAI systems is being harvested.
> and early anecdotal reports are, it is painfully crude, both oversensitive and underspecific
Funnily enough, I didn't run into any issues with any of my NSFW sessions since they implemented the filters. So I guess the problems are with SFW sessions so far :P
Anyway, I've got to say I'm kinda happy about AI Dungeon tackling this problem. They made it clear in their announcement that they aren't targeting NSFW content in general, just the one subject. The AI has a tendency of shoving that subject randomly into sessions, which isn't great. If they can eventually filter that filth out without affecting quality otherwise, I think the service will be better for it.
I do think encoding a puritanical censor into the meaning space of GPT-3 is an interesting research problem. How exactly do you create the perfect mix of paternalism, hypocrisy, self-righteousness and myopia that lets you block bad strings of text, but not say a description of the immaculate conception, Shakespearean romance between the houses Montague and Capulet, or the holy love of the Mother of the Believers?
Where exactly do you see the difference to human editors?
This kind of thing happens everywhere, everyday in TV stations, editorial offices, at publishing companies, radio stations - all kind of media really.
Depending on the political or moral views of the parent organisation or investors, this content censoring/massaging is everyday business and shouldn't shock or surprise you in the slightest.
Yeah, no, the writers of a TV show probably aren't random people off the street who are trying to sneak child porn past management. (Or not very often anyway.) There are disputes over how much edginess the writers can get away with, but in terms of the volume of content and adversarial nature of the relationship it seems pretty different?
This is like telling anyone and everyone to write and produce a TV station only for themselves then being surprised when certain illicit content shows up that you don't agree with. So no, it's not like traditional media content censoring (not to the same degree at least), but it is like modern global-scale media content censoring that already deals with hundreds of hours of content every minute (YouTube et al. having to hire moderators to remove CP).
tbf if you're working with software which is context aware enough to usually generate plausible sounding text-responses, training it to usually identify stuff you think is bad is a closely related problem. (Sure, there's still a fine line between "sick fantasy" and "Stephen King novel", but your procedural text generator has to attempt to handle that to not disgust its customers anyway.)
> there's still a fine line between "sick fantasy" and "Stephen King novel"
Surely the important distinction is not the text itself but which character a reader empathises with — the monster or the victim.
(Personally I don’t understand why violent horror as a genre exists, and literally cannot empathise with people who enjoy it. Nonetheless I recognise that enjoyment of horror does not make one a monster).
Thomas Ligotti proposes a theory which I will paraphrase badly as "people who like horror need a horrific reality they can cope with, because the horror of actual existence is something they cannot face". This rather neatly makes those who do not like Horror the insensitive ones, reversing the more conventional view.
There's a bunch of other stuff about The Nightmare of Consciousness and so on in his book The Conspiracy Against the Human Race.
No matter what the calm and rational part of my mind tries to suggest, the louder emotional part of my mind is shouting the following:
"""You’re entertained by torture and murder of characters who had done nothing deserving of such suffering."""
That part of me wants to continue with a long rant, but by this point my rational self can sit my emotional self down with a nice cup of tea and a biscuit.
I'm not sure why people are so stressed about this. GPT-3 models currently produce long chains of plausible text that's ultimately gobbledygoop. For it to be of any use at all at least some minimal control is needed.
This seems like a great first step to filtering output into something more coherent and interesting. Besides I can't imagine this technology in any serious consumer application without some basic verbal restraint
If I were an editor, and someone passed me their amatuerish version of Lolita to edit through, I'd be well within my rights to say that I didn't want to be involved in it.
More broadly, the editing company I worked for could say - even if you don't intend on releasing this and even if our individual editors don't mind reviewing it - we don't want to have to edit it, and we don't want to be associated with it.
This is no different, but at scale. AI Dungeon, due to their agreement with OpenAI, don't want to have to work with this content. They've found a pretty awful way of implementing it to save the relationship with OpenAI, and hopefully they'll find a better one in the future.
The big difference is that Lolita is a book, so it aims to be published, while most if not all AI Dungeon content stays private and unpublished, so I don't think it's the same.
> preventing the use of AI Dungeon to create child sexual abuse material.
I am shocked to hear this was possible to begin with. It was my understanding that AI Dungeon could only generate text, and did so entirely on computers. But now we learn that not only are children somehow involved (violating child labor laws?), but that they can even be sexually abused?
In that case, "blocking certain words" is not nearly enough - whoever was responsible for creating this system should be charged with, if not child sexual abuse, then at the very least reckless child endangerment!
A bit off topic but who else thinks AI Dungeon is a bit overrated? Does anyone actually play the game and enjoy it? I tried it (with GPT-3 turned on) a few times and lost interest quickly due to non-logical output.
I played it awhile ago and it was great. The output seemed fluid, I could write awesome fantasy with the help of the AI. It truly felt like the future. Then they started making changes to the model, adding features, scenarios, all sorts of additional stuff. Right now, when I try to play, it just seems nonsensical. I can't even follow it for 4 - 5 prompts anymore.
Just spent 30 minutes on a cosmic horror story and it handled it really well, I got absorbed into it then realized this is all AI throwaway content haha, what a world!
Me I do they are restricting everything this will cause incest to not be generated because most incest characters are mteens also teen sex will stop being generated your char name might make the ai stop generating cuz your a minor this is kinda bullshit. I will not play again I can't even make a story of me having sex with my mom
Lmao I just made that to gross out who ever the fuck moderates my stories.
Anyways.
The ai dungeon stopped because my char was 15 years old.
"I Was" the 15 yr old.
It shouldn't do that and this is like blackmail.
What if one of the mods blackmail your stories and your irl google account user for making incest fetishes like this isn't right and I proudly STAND against this shit update.
(Excuse my bad language I'm just expressing my freedom of speech toward the update)
Eventually you might not even be able to make teen love stories because there minors interacting with each other.
Like many AI systems, GPT needs to be "massaged" a bit. You have to understand how your input is actually being fed to the system and how it processes it, and optimize toward that a little, rather than simple naively writing as though you're interacting with a human DM. Certain phrasings work better. Framing things with a bit of narrative works better. The output still isn't perfect, but if you work WITH GPT as a partner, rather than just expecting it to entertain you, you can get some amazing results.
I know this is not directly relevant to this article, but I have a story about AI Dungeon.
I loaded it up with my (female) roommate a few months ago during the dark of the pandemic, and long story short, what ended up happening was this.
Our character had a AI man approach the door of their house with magic "love potion" berries. We tried to get our character to not eat the berries, but the AI "tricked us" into eating them. Then, no matter what choice we made, we had no way out. The AI forced us into a bedroom and raped our character.
We closed the laptop and haven't brought this up again.
I heard once they got more aggressive with their monetization tiers, they nerfed the free tier to the extent that it basically decides on some story path and ignores anything you say to try to change it.
It's certainly the impression I got from watching some youtubers playing it before and after the monetization change.
The dichotomy of sex and violence in tabletop roleplaying is always fascinating. If Steve the Rogue breaks into a house, slaughters an entire family, and then makes lawn decorations with their entrails, his tablemates will probably be exasperated with him. If Steve the Rogue breaks into a house and rapes one of the NPCs, he's probably getting ejected from the game and most likely the friend group.
Are they obvious? From a logical standpoint, it's quite odd that actual murder is considered a worse crime than actual rape, but fantasy murder is much less objectionable than fantasy rape. It extends beyond roleplaying with other people. Fantasy murder is a feature of most videogames, but fantasy rape is limited to low-budget niche titles not offered on most digital or physical storefronts. I'd be interested in the psychology behind that. Could it perhaps be related to a perceived permanence? That is, maybe resetting the game more effectively un-murders the characters than it would un-rape one? Maybe it's relatability. Most of us have fantasized unseriously about murdering someone, be it in traffic or at work, but fewer of us regularly fantasize about raping someone. Other immoral acts such as animal abuse have some of the same taint as virtual rape and are similarly rare in the daily fantasies of the average person.
I think it's really a lot simpler. Most fantasy RPGs have combat and death simply because there is a conflict where there is no mediating authority. This is kind of central to the genre. So, "murdering" NPCs is a very small diagonal step from killing them in combat. It does not stand out.
Rape, on other hand, could hardly serve any acceptable in-game purpose (or real life purpose for that matter). Its purpose is to terrorize and psychologically maim -- something you never need to do to an NPC. An ally who rapes someone in-game is not furthering the quest, and moreover is doing something that is well outside the bounds of typical PC behavior.
In books, I really struggle to get through scenes where people are tortured and sometimes have to abandon stories because of it, whereas I have very little trouble with characters dying. I don’t think this is unusual.
That dichotomy exists in American society broadly. I remember an episode of Hannibal had to cover the butts of two dismembered corpses up with blood as to avoid a higher age rating for nudity
Doesn't seem that fascinating to me. In real life, one's likelihood of being brutally murdered isn't that high. But the likelihood of being R-worded is uncomfortably high. Hence people's aversion to the subject. And it's a form of torture. I wouldn't be terribly happy playing a session where Steve the Rogue is going around torturing an entire family.
My thought is that there are many rape survivors, but there are no murder survivors.
But then again, what about the second-order effects on friends and family as a result of either sexual abuse or violence? Maybe the trauma belonging to the survivors themselves simply overpowers the rest (or not). What about people who survive murder attempts? Maybe being taken advantage of and treated as powerless applies more to sexual than physical trauma, since there are many cases where physical violence is the result of both sides retaliating in equal measure, or sometimes honorably, like for sport. I'm not sure.
The difference is that people (usually implicitly, sometimes explicitly) consent to some topics and not others. Most tabletop roleplaying groups have an understanding about what topics and behavior are appropriate with that group. There are also more formal tools/mechanics to help groups agree on what sort of stuff is too unpleasant to include in the game.
Most tabletop roleplaying games have mechanics about killing things but no mechanics about sexual violence, so that tends to set expectations too.
It is interesting. I've played bad online games where the instant you generate someone kills you. I've played poker games where the board always freezes so you lose. The rape thing feels like the same thing. Bad game play and cheating.
I think you expect fighting but rarely sex in a game.
It seems weird but plausible. I mean, there has been lots of NSFW writing that involve nonconsensual relations; this is part of the AI Dungeon training data, probably intentionally, because sex sells; but I believe that there are almost no stories about sexual assult starting that don't eventually result in a description of sexual assault or at least descriptions of sex as such. If the vast majority of stories about such topic contain descriptions of "ways out" failing instead of succeeding, then prompting the system with a way out would result in a response of how that attempt failed, ergo the "no way out" issue because of path dependence after early random choices.
Like, imagine that you've stumbled on a weird internet story where in the first page someone is approached with magic "love potion" berries but refuses to eat them. That is a solid indicator of what genre the story is. If you had to bet lots of money, what's the probability that the second page will contain something horrific versus the probability that the "seduction" just fizzles out and becomes irrelevant? If you see a movie where the first scene involves a creepy character making a pass, wouldn't you be fairly certain that an escalation of that will follow later? It's like Chekov's gun, once it's there, it almost certainly means that the story is about that - perhaps it could be turned into a "just revenge" story by inserting descriptions of some heroic rescuer or references to how the protagonist expected this to happen in order to punish the assaulter, because stories like that have been written, but a "mediocre" outcome where eventually nothing dramatic happens and the protagonist just gets out won't be generated, because that doesn't get written about, the training data says that such a result is very unlikely. It's obviously a problem, but since it's a "honest probability" based on tropes we see in actual literature, it's going to be hard to fix; the system expects escalation and drama (because all the training stories had that), so you can choose the direction of that escalation, but it won't allow you to have a "non-story" where the suggested drama results in nothing dramatic.
That reminds me of how (n=1) it's easier to change an unpleasant dream by actively taking it in another direction, rather than just willing it to stop.
(And the predictive processing theory of cognition, and how that's surface-level-related to the original topic of GPT-3...)
I noticed once the AI made up its mind about where the plot is supposed to go, there is no real way to change it.
Even in a "you are flying trough space, there is a radio signal coming from a planet" setting there is no way to just ignore the signal and keep flying: The AI decided that signal is the plot, and you gonna investigate it whether you want to or not.
I was able to change it pretty dramatically when I played with it.
Opening a time machine portal from whatever medieval kingdom I was in to teleport to San Francisco and going to the Open AI office, running into Eliezer Yudkowsky and interviewing Sam Altman about Open AI, etc.
It was pretty easy to shift gears - you could force actions.
"Ask the receptionist if Sam is in"
"She says he is not"
I input: "Sam comes out of his office and walks down the hall"
"Look at the receptionist and say, he's right there."
"She stares at you blankly"
You could input story and then use the story lines that you had written in to advance things.
I eventually got tired with it because it was too free form so there wasn't much to it beyond messing around.
Do they use GPT3 or their own special version of it?
And I havent been following it much so sorry if it's a dumb question, but is it still impossible to get your hands on GPT3 and run it yourself instead of paying ClosedAI?
Thoughtcrime became a law in the US in 1996. There was a lot of discussion around alt.sex.stories USENET group by the time. Even if you are not harming anyone, mere act of a creative work could be a crime.
Briefly, and was recognized as unenforceable. As evidenced by the fact that repositories of alt.sex.stories still exist to this day and still host many, many example of erotica containing children, both old and new.
To the extent that this story is generating the predictable amount of internet outrage, that outrage seems to be because people think the developers are making a decision about what content is acceptable on their platforms. I've seen people imply that they're deciding that violence is good, and sex is bad.
That does not appear to be what they're doing: to me, it looks like they're trying to make sure they don't get taken down for creating child pornography on accident. I don't see this as having anything to do with their philosophical positions, it's just CYA.
The interesting part of this is that it may be a corollary to that old question about who owns content created by AI. The other side of that coin is, who gets blamed when the AI commits a crime? Latitude seem to just want to NOT be a test case for that situation.
It's sad that the amount of computing power that necessitates training/running cutting-edge technology like GPT-3 is out of reach of the average developer. The only reason that AI Dungeon was able to be created in the first place at its level of quality was because the developer was able to use his university's GPU cluster for training the model.
For the layman, it seems that there's nothing out there that is the middle ground between AI Dungeon and writing a bunch of fragile Python code in a Colab notebook just to train a model and print out some text. Anything beyond AI Dungeon and you have to have a significant understanding of ML to adapt the model to get it do do what you want at a high level, such as "I want to generate some text that looks like a script for dramatic theatre."
I've always wanted something like: bringing your own corpus of text as an input and receiving a customized, high-quality text generation model as an output that you can then run on your own hardware.
Talk To Transformer was very good for general text generation at the time it was usable, but even that became locked behind a payment plan and watered down for free users.
It seems there's just too much value and too much expense involved to leave this kind of technology solely in the hands of hobbyists.
> The only reason that AI Dungeon was able to be created in the first place at its level of quality was because the developer was able to use his university's GPU cluster for training the model.
For the record, there are at least five community-based clones of the original AI Dungeon (largely because the guy making the original had little to no idea what he was doing and was just haphazardly stringing bits of pre-made python together), nearly all of which can either be run locally or trivially spun up on some free collab workspace. The catch is they're all GPT-2, as was AI Dungeon originally. The step up to 3 is dramatic, and unfortunately out of reach to the ordinary user for now.
There's a new clone just out: "GPT-Neo Dungeon" https://colab.research.google.com/github/finetuneanon/gpt-ne... , which is a GPT-Neo-2.7b finetuned on Literotica for NSFW AI-Dungeon-like dialogue; EleutherAI's GPT-2 performs noticeably better because it was trained on cleaner data, it seems. Still not full GPT-3-175b, ofc.
It's been a bad two days for AI Dungeon. Their community is mass-revolting over this filter (they had to shut down the Discord), and no one here has even mentioned the huge data breach which was just announced: https://github.com/AetherDevSecOps/aid_adventure_vulnerabili... Everything could be downloaded.
> The moderation team cannot stay up to moderate the discord server and ensure that civil discussion is had, and therefore we have elected to halt communications until we are able to return. Thank you for your understanding.
It seems like it. The chats have been halted for 7 hours by now.
There’s some evidence that broad and easy access to pornography has reduced multi-partner sexual activity in the US. If the same is true for artificially generated child pornography then moralistic acts like this could lead to an overall increase in child harm.
On the other hand, it’s a video game. They can have whatever rules they want in it.
184 comments
[ 2.5 ms ] story [ 231 ms ] threadThere's simply no winning this - if you don't apply preventive measures of any kind, your product will deteriorate into a homophobic, racist, sexist ultra-cringe clusterfuck in the blink of an eye [0].
All it takes is a handful of bored teenagers or middle-aged basement-dwellers to turn your product into something the vast majority of your clientele finds appalling.
If you are a company you have the right to choose what you want your product to be and they simply don't want certain headlines pointing at their product.
This has as much to do with "wokeness" as hardcore porn being banned from the Apple app-store, i.e. nothing. It's company policy and you can like it or not, but don't overrate this.
It's nothing new in the slightest and just like publishers can decide to not put books in print that they don't want to be associated with, this company tries to eliminate content they don't want to be associated with - that's all.
[0] https://www.cbsnews.com/news/microsoft-shuts-down-ai-chatbot...
If they use a frozen model then one person using it to generate some NSFW content shouldn't make the output more NSFW-laden for anyone else.
So I don't see how they're "damned if they don't".
The specifics of their approach don't even matter - all it takes is one random Twitter user or media outlet to report on "immoral" output generated by someone to damage the brand image.
That's what they want to avoid and that's why they are trying to take preventive measures.
I'd be very worried about this in their shoes and acting very similarly.
"They knew about this problem, they even attempted to half ass fix it, but it's obviously their mind is on the subscriber money, and not preventing blatant CHILD EXPLOITATION!"
Then you throw some more money at a yet unsolvable task, and then in 2 years, when some other outlet gets triggered, you have to defend yourself once again.
"They've know about this problem for 5 years and it's still rampant in their community! Shame on you JERF INC.!"
Why would you think that this company needs to Thought Police their gamers when thousands of others are able to host creativity based games without being dystopian?
Wow. You act as if that's something new :D
There's even an actual job for doing just that: it's called an editor. You know, the people who work in media, publishing houses, TV, radio, papers, magazines, that kind of thing. It's what they do. Everywhere. Editing content. Removing comments, changing words, making sure the views represented in the publication match the intentions and policies of the investors and parent organisations, etc.
You act as if something is new and evil just because it's done with computers. It isn't. It has been the case for as long as papyrus and cave paintings have been around. Back then it was don't anger the chief/king/Pharao, today it's don't anger the Twitter mob or the hand that feeds you. Same difference.
Games are a medium just like magazines or online blogs and comparing blocky dongs in Minecraft with explicit child pornography says a lot about your understanding of "creative expression". Really makes me wonder sometimes.
Unless you back the idea that every corner of society needs to have the Religion of Woke determining what's "good" and "bad"... and the last thing I think we need is new Crusades with the Knights Wokelar deploying the Holy Creed and woe be the Heathen who goes against the Churches Religion.
"You act as if something is new" and you act as if history isn't full of thought police turning into judge, jury and executioners... from the Crusades to the French Revolution to Auschwitz...
It's funny how "they are just controlling things for YOUR own good" often turns for the worst...
"same difference" Ignorance of history leads to a repeat of history.
From book burnings to "we just want to stop racism"...
Really makes me wonder as well...
I get the feeling it's not me who is the radical here.
There is a difference between wanting to remove content that a company don't want to associate itself with and genocide.
For you to even suggest there's a connection says far more about your twisted logic and dysfunctional thought process than mine.
Also AI dungeon is a multiplayer game so yeah - YOUR freedom of expression is limited by OTHER players freedom of not being subjected to certain content. If you don't understand that, work on your social skills.
What? No it isn't.
Technically there's a "multiplayer" mode but that's fundamentally just people taking turns. Restricting certain kinds of content for that reason is nonsense, akin to restricting the kind of story you can write in google docs because multiple people might have been given access to a document.
All you see is "godwins law" and refuse to acknowledge the parallels scattered throughout history of how these modern day book burnings, witch hunts and mobs with pitch forks turn nasty... and SOMEHOW you are doing that when we live in a world where violent mobs swarm when someone does something that's Evil - so sayeth the religion of woke.
The Woke'saders (Modern day Crusaders) have no problem destroying lives - to the point of violence and death... so, yes, pardon me for equating the intolerance, bigotry and hate presented by the Intolerant Tolerant as leading towards similar atrocities that have existed throughout history.
Again... focus on godwin and ignore then facts in my messages. That way you can ignore the world around you and you'll be blameless when things continue down this path.
You can also join in the fun of destroying the freedom of others in the crusade to control what other people think, say and do.
Your massive overreaction shows no difference to your perceived enemies. This is a game and the creators want to limit the game world and the actions therein to exclude certain content.
This isn't about control over what people think. There are social norms and these change over time.
The "facts" in your message are the same kind of facts that your typical Karen shouts into the phone when they call 911 because there's a negro walking through their neighbourhood. Same level of paranoia, same unhinged out of proportion response.
When was the last time RMS was literally hunted down by a mob? Because if he wasn't, I have to ask who the real fragile flowers are - the ones that get easily offended or the guys like you who see freedom being destroyed and the 4th Reich rising because a private company wants to remove child pornography from their product?
Ridiculous.
I don't think you understand that banning people over what the Thought Police find "bad" is LITERALLY working to control what people think.
> facts that your typical karen
and you are the typical ignorant fool who complains of "godwin" then resorts to "karen hates black people" drivel. Ignorant of the same calls "Karen" made a hundred times before but all of a sudden skin color matters to everything (hint: there's a word for people who only care about skin color... the funny little hypocrisy of the Woke Crusaders is the racism that permeates every move they make to fight racism by making EVERYTHING about race even when it's not).
> Ridiculous
Considering the other stories about AI dungeon over stepping? I'm more convinced I'm right about the Woke'saders and their march down the same path of others throughout history. It is ridiculous... but that's never stopped the religious in their fervor.
> Child porn
If that's really waht's there? report it to the police and take REAL action... not some half assed Woke'sader steps.
But "think of the childrend" is a close second to "ZOMG THE NAZI'S!" when it comes to justification for overstepping in the Name Of Good. So Sayeth the Wokesaders.
https://news.ycombinator.com/newsguidelines.html
https://news.ycombinator.com/newsguidelines.html
(Also, please don't karen: https://news.ycombinator.com/item?id=26880612)
When comparing the Minecraft scenario to the AI Dungeon scenario under the purview of "doing creative things with games," I don't find the two scenarios to be much different. However, for one of those things, it's a possibility within the vast play space of Minecraft - you could even create genitalia and display profanity with just alphabet blocks - but for the other scenario, depending on how you see it, it's content that is generated, served, and potentially stored by AI Dungeon. I'd be concerned about the publicity if I was AI Dungeon as well.
This is not a technical consideration, it is just PC at its worst.
You either deny that games are a medium or you're under the impression that whatever you read in news publications, books, see in films and TV is the result of the unfiltered creative outlet of the producers. News flash: it isn't.
There's heavy editing going on everywhere and just because it targets a specific computer model in this case instead of a more "traditional" product like a novel, film or stage play doesn't make an iota of difference.
In fact this demonstrates the maturity of the technology and its use if it gets the same treatment as every other public media.
Which forum products do you enjoy and which do you not?
Centralised systems is the only reason why you get these problems.
the Internet's original sin!
the irony is that Tech amplifies it but Tech won't (IMHO) be able to fix it with a technological solution (it's not a technical problem).
"Private companies have the right" always fails to account for the fact that these "private companies" are monopolies, with immense control over wide swaths of people's digital lives. No company can compete with them. No alternative to them can be created, because they will immediately be squashed, or fall into obscurity. Participation in these services is "optional," but more and more only in the way that having electricity or running water is "optional."
As a result, these "private companies" effectively become governments, ruling over digital countries you are forced to live in, and affording you little to no rights. And yet, they do this in an ACTUAL country, which DOES afford its citizens rights. Thus a conflict exists.
Ordinary "Private Companies" can censor whatever they like. Monopolies cannot, because doing so threatens free society.
people can't host their own models either as even their free-tier model needs tens of thousands of dollars worth of compute hardware and memory to run.
You do know the idea of thoughtcrime was something enforced by the state and not by a company that's free to control what their product generates, right?
>"corporate self harm for the sake of wokeness"
"Oh no, we lost the business of people who like sexual depictions of minors, what a fucking shame"
Straying away from the prior context a little, the term 'thoughtcrime' along with others such as 'free speech' were coined when it was incomprehensible that a single corporation would have more control over the global population than any government. I don't know whether the time has yet come to re-evaluate which entities these concepts ought to apply to, but it will come.
If anything, peak faux outrage over "wokeness" is achieved when this is what people get angry about
The distinction between "thoughts" and "content" here could be made as "whether it is shared with others".
For example, if someone has a private journal they keep using notebook.exe + a local "journal.txt", and in that journal they write a fantasy work (with a header "this is a fantasy") about overthrowing the government, should they be arrested for that?
What about if they write it in google docs, but don't share it with anyone? What about if they write it as a facebook post? What about as a public facebook post without the "this is fantasy" disclaimer?
In the case of AI Dungeon, my understanding is the stories are by default private, and obviously have an understood "this is fantasy" header by their very nature.
If AI Dungeon were creating facebook posts and the limitation they imposed was "you cannot post this to facebook if our algorithm dislikes it", I don't think so many people would call that "thoughtcrime policing".
What they're doing, as I understand it, is closer to someone requiring that notebook.exe can't edit private files that are deemed "bad" in some way, regardless if anyone else will ever see them.
In the case of AI Dungeon that distinction is lifted. By using AI Dungeon what you're effectively doing is engaging in a creative process with the AI. It's no longer a one way channel of your own thoughts. Now AI Dungeon is actually generating content, and that's where things become problematic.
It's also the reason this is not thought crime or thought policing. You are still free to think your own thoughts, and you're even free to write them down. You can write down your thoughts in AI dungeon if you wish. But AI Dungeon is under no obligation to engage with your thoughts if they contain subject matter to which they object. If you want to engage in a fantasy with the AI about abusing minors, you are free to attempt the engagement, and the AI is free to say no. That's not thought policing, that's freedom of association.
Note that Google has a "celebrity detection" neural net, along with their child porn recognition net. https://cloud.google.com/vision/docs/celebrity-recognition
A paranoid person might look at this, and the trend of every manufacturer putting NN inference accelerators in their cell phones, and wonder if unauthorized photos of certain persons are being silently flagged for law enforcement review.
> This test is focused on preventing the use of AI Dungeon to create child sexual abuse material.
How can you even begin to argue against this? It’s one of the horsemen of the infocalypse; any counterarguments are doomed.
Obviously I'm not equating any of this with CP - but I wish someone had the energy to stand up to it and say "look, you're censoring AI. It's dumb". But of course no one will because being accused as defender of CP is one of the worst things that can happen in any online discussion.
From the linked fandom wiki, quote by Tim Cain about the subject:
> This led to the child killing controversy. We said look, we're going to have kids in the game; you shoot them, it's a huge penalty to karma, you're really disliked, there are places that won't sell to you, people will shoot you on sight, and we thought people can decide what they want to do. [...] This of course contributed to our M-rating, however, Europe said "no". They wouldn't even sell the game if there were children in the game. We didn't have time to rewrite all the quests, we just deleted kids off the disc.
[1] - https://fallout.fandom.com/wiki/Childkiller
> Codsworth is known to say the Sole Survivor's chosen name if it is an option, although it may be shortened, extended, or have a word omitted. A list of spoken names can be found here.
> Ironically you can be Mohammed but you can't be Jesus. I know I missed a bunch of the good ones, feel free to add some below.
[1] - https://fallout.fandom.com/wiki/Codsworth
[2] - https://steamcommunity.com/app/377160/discussions/0/49688113...
Every time you kill a subreddit or facebook group, 99% of the users stop participating in that kind of content, and the other 1% scurry off to some out of sight echo chamber until one day they randomly pop up again to murder a bunch of innocent people because they've been corrupted beyond saving.
Everyone was talking about this for a hot moment after the Christchurch massacre, but nothing was done and now nobody gives a shit again.
It's important to let people with divergent views to feel some sort of social pressure to change. Those 99% of people that have no interest in blowing up buildings or murdering children are the best weapon we have to convince the other 1% of people with weird interests that the world as it is ok without them taking some drastic action. There's always going to be a small subset of people that will rebel, but the important thing is to make sure that otherwise normal people (that want to watch porn, or learn how to safely handle a gun, or learn about cybersecurity, or get desensitised to gore, or whatever else is on this week's "think of the children" hitlist) are integrating into civilised society and not being dragged into cesspits of violence and terrorism because their interests have been deemed by a bunch of fucking software engineers to be "bad".
We're well past the critical point where enough large platforms have banned all the "bad" stuff that any new contenders either need to ban it too or become one of those out of sight echo chambers themselves. The only way to fix it now is for everyone to agree to be less stringent all at once, together.
It's worth remembering that the internet wasn't very censored 20 years ago. Most of us grew up during that time and turned out fine. I'll take tubgirl and lemon party over neo-nazis and conspiracy nutjobs any day of the week.
I would love to have actual discussions with people regarding certain views I hold. But quite often others just refuse to even entertain that I have a different view because x and y. And to them I am just dumb/uneducated/other things to discredit me having an opinion at all.
Hell, I went quite a bit more towards "bad" opinions, just because that side is more accepting of discussion/dissent.
It isn't just corporations banning subreddits/websites that drive people into echochambers, but also people simply refusing to engage at all.
I have to wonder about the legal implications of an AttnGAN or word-to-image type model trained on nothing but illegal pornography. Or one where the ratio of legal to illegal content the model was trained on is unknown and impossible to determine. There are some jurisdictions that consider synthetic images or text to be a victimless crime. At what point does the line become crossed and the images start becoming recognizable and realistic enough to be illegal at first glance under any usual circumstance, but are actually generated from nothing? How would people be able to prove the origin and veracity of such images for the purpose of submitting evidence of a crime?
Not in the US, but in Canada and many European countries, I believe it's illegal.
> We cannot be at a point as humanity where text is illegal. Right?
Even in the US, much text is illegal in certain contexts. Think false advertising, written plausible threats, etc.
Maybe they don't want it to mess with the training data?
They're cool with that.
But the disturbance only happens when you publish the content somewhere else. Why does AI dungeon need to pre-censor something that might never be published on the off-chance that it might be disturbing to someone?
> Additionally, we are updating our community guidelines and policies to clarify prohibited types of user activity.
So they clearly want to prevent everyone from using it that way, not just the unsuspecting users.
If cared about the latter they'd just add an NSFW toggle or something like that.
A few years ago, United Nations tried to ban lolicon worldwide, Japan and USA refused.
Their then rebuttals:
https://nichegamer.com/2019/06/03/us-and-japan-reject-united...
The key reason is perhaps this, buried deep in the text: "We have also received feedback from OpenAI, which asked us to implement changes". Given the volume of prompts that AI Dungeon throws at GPT-3 in the course of a game, it's easy to conclude that Latitude has a real sweetheart deal on the usual pricing, and that they basically have to follow orders from their benefactors.
Whatever may be said of the robocensor they've thrown together - and early anecdotal reports are, it is painfully crude, both oversensitive and underspecific - how they've handled communicating the change is extraordinarily naive. Not for the first time, either: Latitude has form on suddenly imposing major service constraints in a peremptory, underhanded fashion that infuriates their customers. Repeating past PR mistakes, and now doubling down by complaining about "misinformation" and throwing shade onto others, is starting to look like a pattern.
https://developer.mozilla.org/en-US/docs/web/javascript/refe...
https://docs.python.org/3/library/math.html
I have a BS in math, so obviously I'm familiar with the math notation, but I code like an English major.
https://github.com/mame/radiation-hardened-quine
This was by a person that sprinkled lazy backfilling behavior throughout the codebase, created magic method_missing dispatches against polymorphic ActiveRecord relationships, and exposed the database as a service through a RESTful DSL that allowed arbitrary method dispatch.
I'm glad I spend most of my time in statically typed, math and binary-friendly codebases now.
[0]: https://www.playablstudios.com/facade
Frankly, I've always assumed the devs have access to my sessions. Whether it's AI Dungeon themselves or OpenAI, you know that data is being harvested. And at this juncture in text AI development, that kinda makes sense. Obviously I would prefer privacy where possible, but these companies are data hungry and they own the park we're playing in. So it only seems fair.
We'll have to wait awhile for GPT-3 like models to democratize before we can expect real privacy. In the meantime, just err on the side of assuming all input to OpenAI systems is being harvested.
> and early anecdotal reports are, it is painfully crude, both oversensitive and underspecific
Funnily enough, I didn't run into any issues with any of my NSFW sessions since they implemented the filters. So I guess the problems are with SFW sessions so far :P
Anyway, I've got to say I'm kinda happy about AI Dungeon tackling this problem. They made it clear in their announcement that they aren't targeting NSFW content in general, just the one subject. The AI has a tendency of shoving that subject randomly into sessions, which isn't great. If they can eventually filter that filth out without affecting quality otherwise, I think the service will be better for it.
What a time to be alive!
This kind of thing happens everywhere, everyday in TV stations, editorial offices, at publishing companies, radio stations - all kind of media really.
Depending on the political or moral views of the parent organisation or investors, this content censoring/massaging is everyday business and shouldn't shock or surprise you in the slightest.
All of this just smacks of the old, worn-out argument that "it's different because it uses computers!"
Editor is a literal career field, and has been for years and years.
Surely the important distinction is not the text itself but which character a reader empathises with — the monster or the victim.
(Personally I don’t understand why violent horror as a genre exists, and literally cannot empathise with people who enjoy it. Nonetheless I recognise that enjoyment of horror does not make one a monster).
There's a bunch of other stuff about The Nightmare of Consciousness and so on in his book The Conspiracy Against the Human Race.
What's wrong with it as entertainment?
"""You’re entertained by torture and murder of characters who had done nothing deserving of such suffering."""
That part of me wants to continue with a long rant, but by this point my rational self can sit my emotional self down with a nice cup of tea and a biscuit.
Golf stimulates some brains and understimulates others.
Sympathy, humility against my own feelings, sure. Not empathy.
Not sure how much of a line that is, didn't "It' have an underage sex scene in it? Wouldn't it get banned here, too?
> AI Dungeon will continue to support other NSFW content, including consensual adult content, violence, and profanity.
Anything sexual? Oh no no no! The children!
Americans are fucking weird...
This seems like a great first step to filtering output into something more coherent and interesting. Besides I can't imagine this technology in any serious consumer application without some basic verbal restraint
More broadly, the editing company I worked for could say - even if you don't intend on releasing this and even if our individual editors don't mind reviewing it - we don't want to have to edit it, and we don't want to be associated with it.
This is no different, but at scale. AI Dungeon, due to their agreement with OpenAI, don't want to have to work with this content. They've found a pretty awful way of implementing it to save the relationship with OpenAI, and hopefully they'll find a better one in the future.
That party can say that they don't want to be involved with content, regardless of its type.
I am shocked to hear this was possible to begin with. It was my understanding that AI Dungeon could only generate text, and did so entirely on computers. But now we learn that not only are children somehow involved (violating child labor laws?), but that they can even be sexually abused?
In that case, "blocking certain words" is not nearly enough - whoever was responsible for creating this system should be charged with, if not child sexual abuse, then at the very least reckless child endangerment!
Lmao I just made that to gross out who ever the fuck moderates my stories. Anyways. The ai dungeon stopped because my char was 15 years old. "I Was" the 15 yr old. It shouldn't do that and this is like blackmail. What if one of the mods blackmail your stories and your irl google account user for making incest fetishes like this isn't right and I proudly STAND against this shit update. (Excuse my bad language I'm just expressing my freedom of speech toward the update) Eventually you might not even be able to make teen love stories because there minors interacting with each other.
I loaded it up with my (female) roommate a few months ago during the dark of the pandemic, and long story short, what ended up happening was this.
Our character had a AI man approach the door of their house with magic "love potion" berries. We tried to get our character to not eat the berries, but the AI "tricked us" into eating them. Then, no matter what choice we made, we had no way out. The AI forced us into a bedroom and raped our character.
We closed the laptop and haven't brought this up again.
It's certainly the impression I got from watching some youtubers playing it before and after the monetization change.
Also, I don't agree with the example: Steve wouldn't be invited back after either act. YMMV.
Rape, on other hand, could hardly serve any acceptable in-game purpose (or real life purpose for that matter). Its purpose is to terrorize and psychologically maim -- something you never need to do to an NPC. An ally who rapes someone in-game is not furthering the quest, and moreover is doing something that is well outside the bounds of typical PC behavior.
Raped.
No need to bowdlerize it.
Source?
And a murder-victim-to-be still probably wouldn't be triggered, since s/he hasn't "experienced" it yet.
That's how I interpreted the comment anyway :)
...then you probably won't succeed anyway, as triggers tend to be random associations.
But then again, what about the second-order effects on friends and family as a result of either sexual abuse or violence? Maybe the trauma belonging to the survivors themselves simply overpowers the rest (or not). What about people who survive murder attempts? Maybe being taken advantage of and treated as powerless applies more to sexual than physical trauma, since there are many cases where physical violence is the result of both sides retaliating in equal measure, or sometimes honorably, like for sport. I'm not sure.
Most tabletop roleplaying games have mechanics about killing things but no mechanics about sexual violence, so that tends to set expectations too.
I think you expect fighting but rarely sex in a game.
Sometimes players justify their actions with "but that's what my character would do"; there is a popular rpg.stackexchange post about it: https://rpg.stackexchange.com/questions/37103/what-is-my-guy... .
Like, imagine that you've stumbled on a weird internet story where in the first page someone is approached with magic "love potion" berries but refuses to eat them. That is a solid indicator of what genre the story is. If you had to bet lots of money, what's the probability that the second page will contain something horrific versus the probability that the "seduction" just fizzles out and becomes irrelevant? If you see a movie where the first scene involves a creepy character making a pass, wouldn't you be fairly certain that an escalation of that will follow later? It's like Chekov's gun, once it's there, it almost certainly means that the story is about that - perhaps it could be turned into a "just revenge" story by inserting descriptions of some heroic rescuer or references to how the protagonist expected this to happen in order to punish the assaulter, because stories like that have been written, but a "mediocre" outcome where eventually nothing dramatic happens and the protagonist just gets out won't be generated, because that doesn't get written about, the training data says that such a result is very unlikely. It's obviously a problem, but since it's a "honest probability" based on tropes we see in actual literature, it's going to be hard to fix; the system expects escalation and drama (because all the training stories had that), so you can choose the direction of that escalation, but it won't allow you to have a "non-story" where the suggested drama results in nothing dramatic.
(And the predictive processing theory of cognition, and how that's surface-level-related to the original topic of GPT-3...)
Even in a "you are flying trough space, there is a radio signal coming from a planet" setting there is no way to just ignore the signal and keep flying: The AI decided that signal is the plot, and you gonna investigate it whether you want to or not.
Opening a time machine portal from whatever medieval kingdom I was in to teleport to San Francisco and going to the Open AI office, running into Eliezer Yudkowsky and interviewing Sam Altman about Open AI, etc.
It was pretty easy to shift gears - you could force actions.
"Ask the receptionist if Sam is in"
"She says he is not"
I input: "Sam comes out of his office and walks down the hall"
"Look at the receptionist and say, he's right there."
"She stares at you blankly"
You could input story and then use the story lines that you had written in to advance things.
I eventually got tired with it because it was too free form so there wasn't much to it beyond messing around.
And I havent been following it much so sorry if it's a dumb question, but is it still impossible to get your hands on GPT3 and run it yourself instead of paying ClosedAI?
https://academic.oup.com/jcmc/article/2/2/JCMC227/4584343
Briefly, and was recognized as unenforceable. As evidenced by the fact that repositories of alt.sex.stories still exist to this day and still host many, many example of erotica containing children, both old and new.
That does not appear to be what they're doing: to me, it looks like they're trying to make sure they don't get taken down for creating child pornography on accident. I don't see this as having anything to do with their philosophical positions, it's just CYA.
The interesting part of this is that it may be a corollary to that old question about who owns content created by AI. The other side of that coin is, who gets blamed when the AI commits a crime? Latitude seem to just want to NOT be a test case for that situation.
For the layman, it seems that there's nothing out there that is the middle ground between AI Dungeon and writing a bunch of fragile Python code in a Colab notebook just to train a model and print out some text. Anything beyond AI Dungeon and you have to have a significant understanding of ML to adapt the model to get it do do what you want at a high level, such as "I want to generate some text that looks like a script for dramatic theatre."
I've always wanted something like: bringing your own corpus of text as an input and receiving a customized, high-quality text generation model as an output that you can then run on your own hardware.
Talk To Transformer was very good for general text generation at the time it was usable, but even that became locked behind a payment plan and watered down for free users.
It seems there's just too much value and too much expense involved to leave this kind of technology solely in the hands of hobbyists.
Privilege outwits us again.
https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...
[1]: a review of the vulnerability by the person who found it
It seems like it. The chats have been halted for 7 hours by now.
On the other hand, it’s a video game. They can have whatever rules they want in it.