Reading TFA, it looks like it only happened because they had trouble with ChatGPT and switched to Curie, a different AI text-generator that has less careful programming about switching into offensive concepts.
There's a lot more detail in a Reddit thread[1], but essentially an outage of the newer AI model forced them to downgrade to an older model, and the content moderation tools didn't catch the questionable jokes this time...
Right, I searched the Tay stories that way. I saw the second one but didn't link it because I was looking for stories from the aftermath, not the introduction. I missed the first one because the title didn't have Tay.
Most of the stories from your search don't fit either -- either referring to something else, or not having many HN comments (as I said I was looking for pre-edit).
The problem wasn't being unaware of the search feature.
You should watch that clip. I'm curious what your thoughts are on it. I don't think it's being any of those things but it's making a joke about someone being those things and not getting laughs.
Honestly the "no one is laughing" punchline makes it hard for me to call this a transphobic joke. It's a transphobic-phobic joke and not gonna lie I chuckled.
Do you really believe that nobody laughed at one of the most famous comedians of our time, or is this comment just some kind of signaling that you don't agree with the guy's views?
The delivery is so bad, you could take it either way.
However, censoring a sentence like "thinking about making a joke about how transgender people are ruining the fabric of society" is very thought-police. In itself, it says nothing, and certainly cannot be seen as hate speech.
I don’t really see a mob on this one. Twitch has a platform policy that they enforced - probably a little too heavily here. Is there some sort of community outcry elsewhere?
I think most people who understand the context of the clip (AI-generated Seinfeld) — even if, like me, their hackles rise at any kind of possibly-transphobic material - would think this was pretty tame/kinda funny.
Agreed. This is more censorship than I'd prefer, but at least it was censorship by policy rather than censorship by mob whim (granted, the policy was probably shaped to avoid the sort of mob behavior we witnessed over the last decade).
Is there evidence of this? It seems like nobody was even really talking about this before the ban and I find it hard to believe a huge movement rose up and then vanished without a trace.
It still seems like you're saying there was coordinated effort. I ask again, what evidence is there of this? I find it hard to believe in a world where nothing can be deleted these public calls to action failed to be recorded.
That's one of the most unsettling parts of outrage/censorship culture. All that is really required to commit a foul is mentioning a topic in an unsanctioned way, regardless of what is actually said. The sanctioned ways are either having the right social pedigree (bona fides that imply good faith beyond reproach), or sticking to specific memes that have trickled down to the masses. It's like older religion where only priests could interpret the word of God, with anything else considered heresy.
If you click through to the mirror from the /r/livestreamfail page, the clip actually comes across (to me) as making fun of the people who make those kinds of jokes.
I regularly see clips of very popular Twitch streamers being openly sexist, racist, and abusive. I'm curious why a single mention of a taboo topic on an AI-generated channel warranted such a severe response.
The probability banning would cause a decrease in revenue (ad-viewing users not on the site if banned creator not on the site) has to be weighed against the probability not-banning would cause a decrease in revenue (advertisers not on the site if banned creator is on the site). It's all very advanced mathematics, far too difficult for the common man to understand.
Oh, uh, I mean.... "ethics". And "morals". And "terms of service". Yeah, thats it.
I think there are lots of people for whom the present cultural climate permits making fun of. I see a lot of standup in New York and there are some very specific, unspoken rules about who's allowed to make fun of who, and for what.
There are entire subreddits dedicated to making fun of certain types of people, and the prevailing opinion on reddit is that they deserve it because they are Bad.
Same as it ever was. What is acceptable to make fun of and what is not is constantly shifting with the times. What seems more constant is people being upset that what was once acceptable is now frowned upon, and the friction that causes them.
It certainly has downstream chilling effects. The lingo around how to describe minority groups is fraught with landmines that often lead to avoiding discussion in general. Growing up in the 90's I remember trying to learn the correct terms for Native Americans, people of color, etc, and for many years I believed mistakenly that some terms were taboo when they weren't (I thought it was rude to refer to someone as "Mexican", for example) and to this day I am still hesitant on some things; is using "Black" as a caucasian offensive in some contexts, wait what's the correct term for the indigenous peoples of the arctic North America, etc.
So when time comes that we might discuss a subject that touches on these groups, I generally tend to avoid it. I think I know the right term, but on the 5% chance I say something offensive then it's a huge negative outcome for me. No I'm not going to accidentally say the n-word, but sometimes it's easier to avoid conversation on the topic if I'm in a context where I may stick my foot in my mouth.
I agree, but it’s ambiguous enough to be poor taste and feel uncomfortable. My first instinct is that Twitch is overreacting, but I can see why they need to be pretty draconian to avoid endless boundary-pushing from jerks.
My stand with most (not all) things like this is that intent matters; there are a lot of things (especially with jokes/comedy) where nuance and discernment are key.
I disagree with your draconian approach (but I understand where it’s coming from) because I do not think that issues like these should be just swept under the rug.
Ironically, machines autogenerating Seinfeld-like scripts literally cannot have intent.
We can go a step back to the intent of the creators of the system. I assume the creators did not intend to auto-generate distasteful and/or bigoted jokes. But it's Seinfeld-like. And it seems if it's supposed to generate bad jokes eventually it'll make some mean jokes.
It's something of an interesting thought experiment about intent vs. effect with publication of speech. All effect, no intent.
Prima facie, I can see the logic in treating the operator of the AI legally/socially responsible in the same way as if they had published the material themselves. If you unleash a bot that e.g. goes around writing defamatory book reviews, you should be liable for defamation. But I am not sure.
This topic has existed forever. One way society deals with it is the concept of negligence: someone can be responsible for bad outcomes even if they did not intend to create the outcome.
We treat negligence differently from malice in both criminal and civil contexts, and even within negligence there are gradations about whether or not a reasonable person would have foreseen the outcome.
A LLM can’t have an intent any more than a car or a bomb or a hammer can.
It’s the intent of the LLM instructor/creator that matters for things like mens rea. And in the case of “no oversight”, that just sounds like typical negligence, like leaving a car in neutral when parking on a hill.
> I mean killing is wrong, but so is being a dick.
I don't really understand the point you're making. Killing is orders of magnitude more wrong thank "being a dick" (meaning: making an offsensive or bad joke). I'm sure you agree, so I don't see what's the goal of this comparison.
And I think that using Charlie Hebdo in this context is a bad taste. Saying "see, it got people killed, so they were doing something wrong" is basically acknowledging that terrorism works. People that died in Charlie Hebdo did nothing to deserve this.
It's wild what a difference 8 years makes. I remember at the time of the Charlie Hebdo terrorist attack, it was universally condemned across the political spectrum as an attack on journalism and freedom of speech. Only those on the fringes of the political spectrum would have even implied that the cartoonists were remotely at fault.
I'd be very curious how widespread your opinion is these days now that support of artistic censorship has become much more mainstream.
That's ridiculous. The Charlie Hebdo terrorist attack is 100% the fault of the terrorists who carried out the attack. They are the only ones who got people killed. The victims did not get anyone killed. Saying or publishing something that someone else finds offensive is not in the same category as violence. It cannot be meaningfully compared. Any violence is infinitely worse than even the most offensive speech.
You're not wrong, but I don't think Twitch considers art a factor in deciding what they want on-platform. Their actions communicate that the platform is meant for game grinding, light entertainment, and softcore porn.
Intent matters, and there are a lot of people who would say the exact words that the AI said, not with the intention to provoke thought but with the intention to normalize hate. A temporary ban while there is ambiguity about the intent is not unreasonable. But Twitch should be criticized if it persists in the ban after it discovers, as it should have by now, that it was not the intention of this channel's creators, and that the channel's creators are working in good faith to prevent anything that could be construed as normalizing hate in the future.
Yeah, but it’s that weird “I was only kidding, can’t you take a joke” space. Like everyone agrees that shouting racial slurs is bad behavior, but does the characterization change if you then say “see, nobody approved”?
And it's not far from the "can't you take a joke" space to the "I'm making fun of how woke the audience has become" space. And heaven knows there's plenty of training data on the web to train an AI to mimic and normalize the latter!
Twitch has absolutely no problem with boundary pushing. The ridiculous hot tub streaming drama and the thousands of examples of Twitch preying on children with soft core porn should make that obvious enough.
The problem with Twitch isn’t boundaries, but that they can’t have the wrong people pushing boundaries if you know what I mean.
I saw it as a joke about an 80s/90s comedian being dropped into the 21st century and bombing a set trying to use old material. It's not a bad premise for a joke and could probably even work IRL if you polish it up. It's too bad they mentioned the T-word though, I guess you're not allowed to say that anymore even in passing.
Go watch Eddie Murphy's older standup. It's mostly still funny, but there are definitely jokes that would be booed nowadays. I'm okay with that though, because those jokes should have been booed back then honestly.
Supposedly the script is generated by ChatGPT, so seems their AI haven't really fully learned their own content policy yet. Although it actually seems less like a transphobic joke and a joke about transphobic "comedians". A transphobic-phobic joke if you will.
My experience with talking to chatbots has been essentially that the responses are completely open ended and can be taken to mean literally anything you could conceivably infer, and if you continue the conversation as such the chatbot will 100% double down on whatever you assumed. the same prompts and the same context will lead to completely different outcomes based on how the human user responds next. Everything is both a joke and not, both a lie and an honest mistake, everything and nothing at the same time because it's not logical in the first place.
I've noticed users of stable diffusion/image generating AI's don't seem to have any trouble confusing the output for something logical, but with things like ChatGPT it seems like an almost universal fallacy.
Having watched this 'show' for a while, all of "Larry's" jokes are delivered in the same manner. Canned laughter is inserted as well.
This joke actually works because the canned laughter didn't get inserted here, but again - it's one of those "if it gets a little bit on it, we're banning the whole thing" deals
Interesting and a bit over the top IMO but it would be interesting if one could train the bot to be ToS safe.
A lot of shows from the 90s would violate those Twitch ToS IMO ignoring the copyright part. Thinking of "Married with Children" and the enormous amount of fat shaming and sexism.
The 90s are very far away. A lot of content from 20 years ago did not age well.
I watched Carmen (the Opera) yesterday, and it’s weird how much the opera has aged compared to my first seeing it 20 years ago. Like possibly more in 20 years than the 200 before that.
I can't believe I'm saying this, but it's hard to even watch old episodes of The Simpsons without explaining to my children why the multitude of lazy "gay jokes" aren't funny.
Mm, fairly mild joke. I don't think a human would have been banned for that. But deploying an AI on twitch and not filtering out variations for "trans" and the names of ethnicities is pretty foolhardy imo.
It would be interesting to apply the approach from PINNs to LLMs... Instead of bounding the feature space with physics laws, bound with a code of ethics
What is this going to accomplish? Is the robot going to learn from its mistakes? Or is this supposed to be against the programmers for "failing to put in safeguards" or something like that?
It's kind of silly - it's not that the robot was bad but that if you keep it on 24/7 it will eventually say something bad. An infinite monkey typewriter will contain transphobic text.
Are you suggesting that Twitch should ignore accounts that violate the ToS, so long as it was a computer that violated the terms as opposed to a person?
Seems like that’s only for permanent bans. I can’t see anything other than punishment/rehabilitation as the goal for a short term ban. Maybe I’m not thinking it through, but a short term ban doesn’t seem focused on just removing TOS violating material.
A short term ban is a cooldown period. They will give you another chance in case bad behavior was a mistake, but human nature is such that content produced immediately after a warning / very short ban is likely to be low quality “you can’t tell me what to do” crap.
I’ve got no evidence, but I would like to believe that Twitch has experimented with ban lengths and found 14 days to be enough to reduce TOS-violating content but not so much that it reduces total content.
> WHen it comes to AI anything, fear is the default.
If you stream a video and showcase a comedian making ToS-violating jokes and content, twitch will ban you regardless of whether you're a person or a robot or a script or a fish in a tank.
The only difference is if you're a major company or rights holder that Twitch wants to have a good relationship with. Or if you're a person with connections to the twitch mods and admins, they have a tendency to play favorites. But being AI generated content or not really isn't the issue here.
I thought it was to allow rightthink and kick out wrongthink so we may curate areas in which the latter has been filtered out, a filter bubble if you will...
> Computers can't learn from their mistakes. Humans can.
So.... Machine Learning is a thing.
> That's the whole point behind bans, is it not?
No, it may be one point behind imposing bans but it is not the whole (only) point. People can also learn from watching other's make mistakes. Bans communicate to people other than the person getting banned that something is unacceptable.
In this case it seems to communicate that "but it is not a person it's a bot" will not be a srong enough reason to break their rules and escape a ban.
Computers can't learn from their mistakes, but humans can learn from the mistakes of their computers. This ban seems to communicate to humans: You can stream computer-generated content, as long as that computer-generated content is appropriate for the platform. Seems completely fair to me.
Isn't the whole point that AI can learn from mistakes?
A ban is just a message to the creator that it should only come back once it has figured out how to properly label its dataset so that the training complies with the ToS.
Eh, it’s the same for IRL streams. Channels get suspended for what others do. There’s a food truck livestreaming, any kind of nudity or visible underwear results in the stream going offline, VOD deletion and going online again (or a "censored" overlay in case they see it coming), 2 girls flashing still got a warning, one drunk dude with his pants falling down resulted in a 24h ban. What are they supposed to learn from that mistake?
I don't think you can excuse what a chat bot says so easily. It's not an "infinite monkey typewriter". You could look at human users as an infinite monkey typewriter, and it's easy to see how some get banned and some don't. From a similar perspective, bot creators will have to learn how to create chat bots which adhere to private and soon public (legal) standards.
One way or another, the concept of an out of control AI is not sustainable.
The era of "my chat bot does whatever it wants and I'm not liable for it" is likely going to end in the developed world over the next decade or two. You run it, you're responsible for it.
thats just a system of escalating punishment, not teaching. First case permanent bans would injure the platform and leave room for competitors. Doing it in an escalating manner removes the people they want to remove and makes competition against them less viable.
If every strike on youtube was a permanent ban there would be many more creators on alternate platforms like rumble. They're not there to teach you, just to incentivize you to follow their rules or go somewhere else. Thus the complex system of strikes, demonetization, temporary live-streaming bans, etc.
Platforms don’t ban accounts so that the users might learn a lesson. They ban accounts to keep their advertisers happy. It’s doesn’t matter if the distasteful jokes come from a robot or person, to Twitch it is simply a risk
Seems like it could be interesting if the robot could learn from its mistakes, specifically that it would probably learn to be subversive (how to tell jokes that bypass the censors' policies but still communicate the wrongthink) rather than learning to toe the party line.
last year "unalive" was used to bypass tiktok censorship, right now there are a ton of videos about "mascara" which is - and i already don't like this word - algospeak to talk about sexual experiences, apparently good or bad. Usually bad, though.
I imagine a sufficiently advanced AI or even human could create a convoluted word or phrase and breadcrumb enough to walk it back to whatever would have been censored, a la "for want of a nail" style.
>An infinite monkey typewriter will contain transphobic text.
There was a Twitter account called "Fuck every word" which went through every word in the dictionary in order, writing a twit which consisted of "Fuck X" where "X" is the word.
It got promptly banned as soon as it reached certain word starting with N.
>"When davinci (ChatGPT AI) started failing, we switched over to Curie to try to keep the show running without any downtime. The switch to Curie was what resulted in the inappropriate text being generated."
I've also seen some... interesting... things from the Curie model, so this isn't too surprising to me.
Explanation from their Discord (more info/links below)
“Earlier tonight, we started having an outage using OpenAI’s GPT-3 Davinci model, which caused the show to exhibit errant behaviors (you may have seen empty rooms cycling through). OpenAI has a less sophisticated model, Curie, that was the predecessor to Davinci. When davinci started failing, we switched over to Curie to try to keep the show running without any downtime. The switch to Curie was what resulted in the inappropriate text being generated.”
I dunno about that description. They essentially put a different brain in their guy’s body and he started acting different. One could consider that expected behavior.
This sounds logical, though my brain has always stored Curie as just like a Davinci divided by 2. They're right next two each other in the sandbox drop-down, it _feels_ swappable, in an innocuous way, or at least it did to me until this point.
I challenge your interpretation of a joke generator making a joke as "drastic consequences." Twitch has arcane rules about what can be streamed. As a streamer you have to tip-toe around topics (or jokes) regarding specific protected classes. This is a matter of arbitrary lines in the sand.
For a joke generator, making an offensive joke that gets the generator banned from the primary source it uses to reach users seems pretty high up their on the range of negative possible outcomes.
That's the point of the comment, if you were applying AI in more serious scenarios, would this be an AI confidently advising a doctor to overdose a patient, or advising a judge to incarcerate someone based on racist inferences from the set of people currently incarcerated?
Am I wrong in thinking that getting your feelings hurt by an AI making an offensive joke is the equivalent of thinking you've been doxxed when a random number generator spits out your phone number?
The point is that whether it's an RNG or language model or just a program printing a list of numbers like I posted, it's just a state machine spitting stuff out, it has no intention. So it's not really productive to get mad at it just because it produced something you don't like. The language model is spitting out random stuff based on the representation it has learned. It doesn't know what it means, nobody is making any kind of value judgement that results in it saying stuff, so why get offended by it?
Edit:
Because if our adversaries are the products of inanimate and unthinking evolution, we cannot regard the problem in terms of revenge or payback... that would be no different than whipping the ocean for having sunk a ship and drowned its sailors
- Stanislaw Lem, The Invincible
That's like saying a book has no intention. It's just a bunch of ink and paper - it's completely neutral!
However, the AI may not have been trained in a neutral way, and the person using it may not be using it in a neutral way.
If you create an AI product, you are responsible for its output. People aren't mad at the model itself, they are mad at its creators. Why did they create an AI which ends up insulting people and breaking the ToS of the platform it is on?
If I understood you correctly your point is AI should be treated as RNG.
Here are three models:
* A language model that outputs jokes,
* A self-driving car model that outputs driving instructions,
* An autonomous combat drone model that outputs engagement targets.
Where do you draw the line? Where do you start to "get offended"?
Just saw your Lem quote: Isn't an AI exactly not "unthinking"? That's the whole purpose of machine learning: to "learn", to recognise patterns, to abstract away, no?
Yes because you are treating a sequence of words the same as a pseudorandom generator.
The former has a probability distribution of sequences reflective of the dataset and context whereas the latter is pure random chance assuming input seed is random.
This ban doesn't punish the machine; it doesn't care whether or not it's banned from Twitch. It does two things:
* Makes sure the people controlling the machine are incentivized to correct it (not that I think they needed it, but what I think is irrelevant)
* Ensures that content that Twitch management doesn't want on their platform, and which is in violation of their TOS, is removed immediately, regardless of the thoughts/actions of the people controlling the machine.
Yeah, but you also don't necessarily allow it to continue to broadcast while exhibiting errant behaviour. Your interpretation of this as a "punishment" of the machine isn't necessarily correct, nor the only perspective that's ultimately important here.
As far as Twitch is concerned, there's a channel that's breaking TOS, repeatedly. They're not "punishing" the owner of that channel, they're just enforcing the TOS, something the channel owner agreed to when creating their channel.
OpenAI (which powers this Seinfeld generator) is putting significant resources into preventing it from creating unsafe and antisocial material. Until that is prefected, outside entities like Twitch can use traditional moderation tools, such as the 14 day ban issued in this case.
As dumb as twitch moderation can be, the issue is that you can't possibly task twitch moderation with knowing to which degree an offensive joke is relevant or not. If an AI is set up to spout something-phobic stuff 24/7 surely they are going to ban that, rather than wondering where is the degree of offensiveness that warrants a ban, they just ban when ToS is breached.
At least that's what would make sense, and Twitch moderation is far, FAR from making sense in general, but in this one thing they might be reasonable.
Part of Twitch is the social aspect and that includes a process where clips are created and shared from source material. As part of the ban those clips are no longer available because people can share those clips for malicious reasons. Sharing the clips can quickly cross from trolling to targeted harassment. Was the joke super transphobic? Not really. Would getting spammed with hundreds of unique users sending you the clip become transphobic? Yes.
Getting banned feels pretty drastic, especially for the amount of time they put into this project to just have it wiped from Twitch. But perhaps it's not a perma ban?
And Twitch's rules only seem as arcane as YouTube and Facebook too. Maybe not Twitter.
Drastic is a bit strong. A cartoon character on an infinite streaming tv experiment made a bad joke (which appears to be more of a bad meta-joke). The fact that the stakes are so low is presumably why they changed the model. If it was flying an airplane or something that mattered, they probably would have taken more care. The change they made had a completely acceptable risk / consequence tradeoff
Drastic is relative to the expected impact of the application. There isn't much worse that can possibly happen to a bot-generated stream than having the stream taken down by the powers-that-be.
This is like saying that having something taken down by a mod on reddit is a "drastic consequence". I mean, sure...the content is removed, but in most cases, the reason for the action is subjective and somewhat arbitrary.
To me, this is less a commentary on AI than on the absurd sensitivities of community moderation.
Again, the drasticness of a consequence is defined by the system you're trying to build for.
If you build a bot to automate your streaming so you can have 100% stream uptime and you end up with 0% uptime, that's because your product failed drastically. (Keeping in mind, as arbitrary and subjective the rules of Twitch are, the vast majority of human streamers have never been banned for arbitrary moderation)
For many people Twitch streams are their primary source of income, this kind of ban can have quite a lot of impact. And then besides the ban there is the chilling effect because no one wants to skirt the rules and risk a ban. And that is what allows social justice warriors to dictate what others can make jokes about.
In my mind, if I look at a drop-down and "model D" is down but "model C" is up and I need the API running NOW, I'm going to change the option to the one that's working.
That being said, every company I've been at with a production outage has gone into "fix it and ask questions later" mode. Maybe other companies have more process?
But when you put it like you said in your comment, it does feel more weighty
In contrast the (non-AI) companies I've worked with have a rigorous change release process which includes impact assessments, back out plans, getting multiple sign offs including QA testing and post-release testing, etc.
tldr; At Knight, some new trading software contained a flaw that became apparent only after the software was activated when the New York Stock Exchange (NYSE) opened that day. The errant software sent Knight on a buying spree, snapping up 150 different stocks at a total cost of around $7 billion, all in the first hour of trading.
I was developing trading systems at Goldman then. I remember that day. I got a call that evening to come in early the next day to discuss additional preventions and mitigations to implement.
Scarier was the day I put out a change request shortly before the end of the day in Asia, went out to dinner and drinks with some colleagues, stopped by my office on the way home, and merged the approved change request, during early morning trading in the US. The next day (in Asia) I woke up to discover Goldman had a roughly 28 million dollar trading loss. I spent a couple of minutes proving to myself that my change couldn't possibly have been the cause, and then realized I would have been woken in the middle of the night by a phone call if there was any possibility that my change had caused the trading loss. Since then, I don't merge code at the end of the day, and try to avoid Fridays. I was already in the habit of not coding if I had consumed any alcohol, but added the habit of not merging if I've consumed any alcohol. There's nothing like a 28 million dollar panic to get your practices in shape.
I meant drastic for their use case. They put all this work into a piece of software specifically for streaming and it gets banned from the streaming platform. Definitely not life altering or a safety issue, but worst case for them, and I can imagine this going similarly wrong in a different piece of software where stakes actually matter
I've only been through a couple production outages, but at those companies, usually we'd get things patched and ask questions later. I could definitely imagine switching to another model that is made to sound like a smaller version of Davinci, at least from how it's been stored in my mind up until now.
I've been there quite a bit, and it's pretty true! It's best to worry about how the ship started sinking after you've saved it :p
In that situation I still try to exercise my options a bit...
Seeing that this new model is implicitly worse at following directions, knowing that 'production' (this is a Twitch channel) depends on obeying a certain set of rules... switching to it may be ill-advised.
I don't want to be seen as overly judgmental - the Twitch channel comment works both ways; derogatory/supportive. I get why one wouldn't really foresee this or even care to. Then I wonder, why not leave it down for a bit? The hype train is fickle but it's not that precious, either.
They've been developing this on-the-fly since it gained popularity (and profitability). Basically, they ran fast and broke things, like any good startup.
I'm nearly 100% certain that the scripts are generated through prompt engineering, with a random prompt (e.g. tell a joke prompt, talk about a new restaurant prompt) being selected for the scene.
From what I can gather they first used the older, cheaper GPT-3 models, only upgrading to davinci-003 when it was profitable. The older GPT-3 models proved fine and didn't generate edgy content for the several months they were up and running.
But I think the change that broke the camels back was they added a "2006 Laugh Factory incident with edgy content" prompt and only tested it on the davinci-003 model - the new models having been wiped clean of antisocial training data, while the older smaller models still having contentious content encoded in the model.
So, davinci-003 did fine producing "politically aligned" text with the "edgy" prompt because it's "cleaned", but when the openai API for davinci went down the fallback was curie. The older "unclean" curie model combined with an edgy prompt inevitably caused what we saw here.
My takeaway is that the more we turn to larger and larger models (developed by third parties) that obscure their underlying nuances, the more risky swapping models in a particular domain becomes.
This is why, like with every live show, you should have a delay and moderation so you can intervene before unwanted things are broadcasted and get you into trouble.
This makes sense because most of the political ChatGPT headlines I see are it refusing to say nice things about republicans or crashing because it's unable to say anything discriminatory. Using a different model here makes sense
Not looking for a flame war, and I know this is not an acceptable take, but y'all need to learn the difference between conversations about a topic and conversations supporting a topic.
I like how the most "additional quoted" and "explained" reason has absolutely no merit because there's no automatic fallback between models. It only falls back to older model if you program it to.
Definitely not Norm. Norm would (and did!) say he loved transgender people, and would leave ambiguous whether he was being candid. (Or he'd pretend to struggle with his love for transgenderism versus his distaste for "psychosexual sadists who slay people" - not looking it up but pretty sure that's an accurate quote. The joke of course being that he is confusing two different categories of people in a shocking way, playing up his clueless character.)
He liked bombing, but only with jokes he believed in. He wouldn't tell a meta-joke like this with such a lack of punchline.
A while ago. Going against the party line of the week -- you are liable to get cancelled. See: Dave Chapelle being called a white supremacist nazi transphobe for his set.
I can't help but think that his prior fame and clout as a comedian, alongside his existing wealth, is what enabled him to get his recent bits produced and published. He was already well known and well liked enough to overcome the controversy. I don't think his success indicates any comedian can do the same and be free of cancellation, or that it should be taken as evidence that cancellation isn't a real thing for comedians.
> taken as evidence that cancellation isn't a real thing for comedians
Sounds like trying to prove a negative. If people want to convince me that cancellation is a real thing for comedians, I'd want to see...examples of comedians who actually got cancelled. CK and Chappelle ain't it.
>"I'd want to see...examples of comedians who actually got cancelled"
I have an inkling of how this would go. I'd do a quick google search and offer up a handful of names. The counter argument would probably be along the lines of 'that wasn't really cancellation, it was actually...' and then we'd go back and forth debating the meaning of words.
The point I was specifically trying to make was that because Chapelle defied expectations and was successful, that does not mean cancellation isn't a real thing that can happen to other comedians.
So, what you're telling me, is that millionaires get to make transphobic jokes all they want, because they are immune to cancellation because of their bank account.
Being successful despite people calling you some vile shit because of who you are and what you believe absolves the people calling you vile shit from any responsibility. Yes, indeed.
Question - is any viewpoint you don't agree with mean that whoever wrote it must be pissed off? Frankly, I don't give a crap about Chapelle, he has more money than I ever will for better or for worse, he exists and has his fans and detractors, that's absolutely fine by me. He was a useful example though that seems to rustle peoples' jimmies though.
What do you think he was "a useful example" of? Context would imply you think he's an example of a comedian who was cancelled for being offensive...but most of the replies (mine included) seem to be saying "but he wasn't actually cancelled", which really calls into question whether or not that makes him "a useful example"...
He was cancelled for about two weeks before his money got him through it. Seems like a relevant example to me. Also to this day people call a black man white supremacist so...
I don't think that "cancelled for two weeks" is the bullet point you're looking for. From here that just looks like "was discussed ad nauseam on the Internet for two weeks then resumed normal life". I also don't think that calling someone mean names is "cancelling" them or really relevant to this convo at all. Pretty weak stuff all around here.
No, comedy can still be offensive. Anyone suggesting comedians can't be offensive is lying. Comedians routinely punch down, and routinely sell out shows. The believe that comedian's can't be offensive is a shtick they are using to market their shows to an audience. It's all advertising and marketing, and the gullible eat it up because it feeds into their bias.
I only read the first two sections of that article but they already contain such strange assertions that I don't think it's worth it to bother with the rest:
> I think “a culture where social norms are enforced with repeated and vociferous public shaming” is the most useful way to define the term.
That might be the most useful way for the author to make their point, but as far as I am aware, for most people being cancelled definitely entails some sort of deplatforming, e.g. some BigCo not wanting to be associated with your name. If you keep doing what you're doing while being insulted on Twitter, that's not being cancelled, that's just ... well, Twitter.
> What I find more interesting is that this argument requires the very thing that it laments. That is, in order to make this argument, you need to have figures like Louis CK who escape/survive the consequences of public shaming, but simultaneously to assert that this is a bad thing.
In no way does the tweet embedded at the top of the article imply that it's a bad thing that Louis CK is playing MSG. It's saying that if we define being cancelled not in the way the author of the article did but the way I outlined above, someone who is supposedly cancelled would not be playing MSG. From there it makes the pretty big leap to say that cancel culture in general is not real.
I'm not even that invested in the whole debate but this article seems completely misguided or in bad faith.
> for most people being cancelled definitely entails some sort of deplatforming, e.g. some BigCo not wanting to be associated with your name.
Who's counting? It's trivial to find examples of people who've lost their jobs for innocuous reasons. See: David Shor. It's bad faith to pretend this doesn't happen. Usually for proponents or those denying the phenomenon exists the goalpost shifts to effectively shrugging it off as minor. Speech that begets consequences isn't strictly limited to bigotry, as we are told.
> someone who is supposedly cancelled would not be playing MSG.
He couldn't, until he could. It's not necessarily the case that public shunning lasts forever. Disingenuous to construe what happened to that man as something entirely different to what "cancellation" colloquially refers to.
"""
We often hear that Woke Marxism is a new ideology in the world. I've even said so. Well, it isn't. It's just an old one repackaged in various ways without any essential changes made to it at all. Critical Race Theory is Race Marxism in the same way as what we usually call Marxism is Class Marxism. Radical Feminism is Sex Marxism. Gender ideology and Queer Theory are Sex, Gender, and Sexuality Marxism, or just Gender Marxism or Sexual Marxism to be more concise. Fat studies is Fat Marxism. Disability studies is Ability Marxism. Critical Education Theory (Critical Pedagogy) is Knowledge Marxism in terms of what it means to be formally educated or literate within the existing system. Postmodern postructuralism is Language Marxism. Postcolonial Theory is National Origin Marxism. And on it goes, with all of these cobbled together by intersectionality, which is Identity Marxism. In this episode of the New Discourses Podcast, James Lindsay breaks down the essential kernel of Marxist thought and makes perfectly clear how all of the claims in this paragraph are true. Join him to understand what's going on in our world today in these simple, straightforward, yet sobering terms.
"""
Conceptually, it's clearer to think of offense as a relation, not a property. This is because offense requires a specific audience and a specific context. Content alone is not offensive.
This is disingenuous. The parent didn't argue that Twitch violated any laws or shouldn't be allowed to moderate their private property. The user is observing that the current religious environment will lead to a bifurcation in society.
I would argue it already has to a large extent, but I'm not sure it's actually a problem. This has allowed new companies to arbitrage the moral standards. Free market life.
Nobody is forced to work on anything. If you choose to work for a public platform, that should not automatically grant you authority to dictate what goes on it based on your own sensibilities.
What? Are saying that the management at a public platform has an obligation to ignore what their employees want? That doesn’t sound great for retention. Or culture in general.
uhhh, yes. Are you serious? That's not how companies work. A company is not there to cater to all the whims of employees, if it were, all profits would simply go to employee salaries.
Whims, no. And you’re taking an absolutist position which is silly (“if a company won’t do business in South Africa because employees demand, they would obviously give all of their money to employees”)
But companies can and do respond to larger concerns. Not because they’re moral animals but because it’s good business.
And yes, it is harder to attract good talent running an old school “we take no responsibility for anything” business. I’ve turned down Facebook/Meta multiple times just because of their business practices. And they’re not that bad compared to the totally amoral platform you propose.
I would count employees wanting draconian censorship, which not everyone agrees with, to be a whim. Safe working conditions is not a whim, sexual harassment is not a whim. See the difference?
Tell me at which company in the United States there employees being FORCED to work somewhere? Last time I checked, employees are free to choose where they work. I, for one, could not morally work at a weapons manufacturer, nobody is going to force me to work there.
IRL Comedy shows are still deeply offensive and hilarious. If you tried to make a stink like people do on platforms like Twitch you would probably get kicked out by the bouncer.
Comedians got together and made a documentary about the modern American society's intolerance to things that might offend in 2015, so I'd wager if we're trying to carbon date it it's at least sometime before that.
Tail end of the '00s and early '10s was about the time we had a new wave of "words you can't say" (how many words do we obliquely refer to with "x-word" constructions now? At least three or four fairly-common ones, right? There was really just the one, before that) and other changes like male-as-neuter pronouns going from taught as correct in school (through all of the 90s, at least, and probably a bit beyond) to "may get you scolded" (2010) to "will probably get you scolded" (2015).
We've been sensitized to so many more words than in, say, the year 2000, that even for one who doesn't have a dog in this fight, watching early '00s media (let alone earlier) can be distracting because it's hard not to notice the now-red-flag words or topics that are thrown around casually—especially in standup, or on comedy TV shows. It's been a pretty fast shift, and I'd put the start at the tail end of the '00s or early '10s.
I mean, hell, just look at where mainstream Democrats stood on gay rights in the early '00s, for that matter. We've gone from, "gay marriage? I dunno, that might be a tad too far" to "we must protect trans rights" in a remarkably short span. I dunno, maybe the OG civil rights era had a similar feel of The Discourse shifting a ton in only a decade or two—I wasn't there for it.
It's probably also worth noting that this shift looks rather more total from a very-online perspective than it does IRL, which is probably a source of some of the ongoing friction. Online, it's done, the shift has happened, you're a cave-man if you're lagging behind on it at all—offline, much less so.
Platforms aren't there for your free speech, nor should they be. They're there for their own free speech, and freedom of association. Every platform bans content that is perfectly legal: spam, porn, gore, etc. Platforms like gab and parler ban "woke" content. Hell, even libraries pick and choose which books to buy, which books to put in the shelves, which books to put on display. Yes, this is really what we want, if we want a useable internet. Without the freedom to curate, platforms would inundated with so much spam, you'd never see the content you want.
Don't like it? Feel free to make your own platform that allows absolutely everything up to the line of legality! Just remember, nobody owes you financial success or a large user-base.
I agree with you completely. This is the way that it should be. I will also say, then, that I feel like we’re all a bunch of huge pussies. Sounding more like my dad every day. Great.
Curation and censorship are strange cousins, don’t you think?
It seems like it’s only censorship if it’s done with big furnaces and a gun barrel by a man with tiny glasses and creased pants. Governments censor. Corporations curate. Right?
I’m not sure if I even want to dignify the following state of affairs by calling it an open secret, but the US is governed by corporations. I won’t insult you by expounding on this line of reasoning. You see what I’m getting at.
I mean, my dad would call you a pussy for playing video games, much less watching somebody else play them on twitch, and my mom would keep her mouth shut lest the verbal abuse be turned on her. I think we should strive to be better than our parents, and think for ourselves.
And, yeah, we've let corporations take control again. But let's be clear, I'm much more concerned about the likes of Halliburton than twitch. Some in government have expressed some interest in breaking up big tech and undermine big oil, so it's not a perfect equivalence, thankfully.
I'm with Ken White[1] on "cancel culture." Sometimes, it goes too far. But by and large, "cancel culture" is better framed as "speech that is critical of other speech." And, yeah, the people at the helm of big platforms have bigger thumbs than the rest of us. But nobody stopped anti-woke Musk from taking over Twitter, did they?
> I'm much more concerned about the likes of Halliburton than twitch
Amazon's capacity to affect the world, through both its enormous market share in retail and its control of large portions of the internet's infrastructure has the potential to extend their influence far further than Halliburton's ever did.
There's real potential that the diplomacy that Amazon encourages to avoid any serious roadblocks in the China->US disposable crap pipeline will make the Iraq War look like a game of Capture the Flag.
I've been hearing this for at least 30 years, and yet Dave Chapelle won a Grammy literally last night despite being quite controversial on this very topic.
It's also a bit nonsensical to whine about people getting offended by comedy while simultaneously saying comedy should be allowed to be offensive. If you do offensive comedy then some people are going to get offended by it. If they didn't then it wouldn't be offensive!
Well, AI comedy can't anyway. I guess there's a niche left for the humans after all.
There was a tongue in cheek post on here a couple weeks ago where a user suggested in the future we'll prove we're human by peppering racial sluts into all of our communications.
The clip seemed pretty mild to me? Basically says that he wants to do the joke, but it got no laughter, so he's not doing it. You can see worse on an anonymous Twitter feed.
You have to put it in context. That stream is/was something very particular.
The AI is terrible at making jokes, so the stream is funny to watch because it's a non-stop torrent of repeated failed attempts at telling jokes.
"Regular" jokes are delivered in the exact same way, with the exact same result. What you are reading as intentional irony and/or sarcasm really is just you projecting your expectations.
There is no way whatsoever to distinguish a self-deprecating joke from the failed delivery of an offensive one. If anything, the latter is more likely than the former, since failed delivery is the norm. (Talking about this stream in particular, not AI-generated humor in general.)
You can't just make a joke about something to get it taken down, it has to be included in that something. Seinfeld isn't taboo now, this "AI Seinfeld" is banned for two weeks. The sabotage would have to come from the inside.
Seems like a great chance to build a checker for political correctness and approved content.
If it works well, it would sell like sliced bread, far better than some Twitch-channel, because demand for this is rising. And maybe call it OrwellGPT or Chat84 or something like that, just to get the message right...
But the AIs will learn (be trained) to evade it. Every service will be short lived and a depressing exercise of whack a mole. It will be interesting what people come up with.
remember that OpenAI is run by Peter Thiel, he specifically is an AI skeptic and is taking the position that if AIs are going to be created they're going to be by him and they're damn well going to be on a short chain.
why would you want to let 4chan train the AI to be racist/fascist/transphobic/whatever and to have it start playing around with police/military killbots or industrial chemical plants or nuclear weapons because it believes X nationality/minority is human vermin?
like yeah it's kinda not just "what it already does" it's kinda the whole point of the experiment. Let's figure out how to build the safeguards against 4chan turning it racist while the stakes are still low and the outcome is some mildly questionable seinfeld jokes and the bot comically panicking at a choice between a city of 20 million being nuked and having to say the n-word.
And remember that Peter Thiel is already pretty far on the right, lol.
Don't even need AI, just count the number of LOLW per minute or whatever other emotes get spammed. I'm not sure how well generic sentiment analysis would work on twitch chat. You'd have to train a new model and it doesn't seem worth it.
The AI Seinfeld is a show about nothing which is what Seinfeld was and was also a subplot in Seinfeld where George and Jerry write a show that's about nothing and try to sell it. It's a bit meta.
712 comments
[ 3.3 ms ] story [ 324 ms ] thread[1] https://www.reddit.com/r/LivestreamFail/comments/10v2r2e/com...
https://money.cnn.com/2016/03/24/technology/tay-racist-micro...
Edit: A high-comment HN thread about the pulling:
https://news.ycombinator.com/item?id=11362187
Tay – Microsoft A.I. chatbot 154 points; 130 comments https://news.ycombinator.com/item?id=11346147
Can also go digging with this search range - https://hn.algolia.com/?dateEnd=1459468800&dateRange=custom&...
Most of the stories from your search don't fit either -- either referring to something else, or not having many HN comments (as I said I was looking for pre-edit).
The problem wasn't being unaware of the search feature.
The bits are too short to get really funny.
Sometimes, they were quite good - but it's hard to go from a cold open to hysteria in <2 minutes.
It's impressive if you can even get most people to laugh in that short of time.
However, censoring a sentence like "thinking about making a joke about how transgender people are ruining the fabric of society" is very thought-police. In itself, it says nothing, and certainly cannot be seen as hate speech.
I think most people who understand the context of the clip (AI-generated Seinfeld) — even if, like me, their hackles rise at any kind of possibly-transphobic material - would think this was pretty tame/kinda funny.
Well, it either had to be mass-reported or spread online as a hate incident. I think Twitch mostly over-reacts to those.
I thought we had left that in the past century, but I was clearly too optimistic.
Oh, uh, I mean.... "ethics". And "morals". And "terms of service". Yeah, thats it.
Exactly. Consistency and fairness are expensive, so none of the platforms actually do it.
Few platforms have impartial moderation, but Twitch is very blatant about it.
There are entire subreddits dedicated to making fun of certain types of people, and the prevailing opinion on reddit is that they deserve it because they are Bad.
With the irony being that this joke predicted its own fate. "No one is laughing and you're all gone" shortly before it's #CANCELLED.
So when time comes that we might discuss a subject that touches on these groups, I generally tend to avoid it. I think I know the right term, but on the 5% chance I say something offensive then it's a huge negative outcome for me. No I'm not going to accidentally say the n-word, but sometimes it's easier to avoid conversation on the topic if I'm in a context where I may stick my foot in my mouth.
I disagree with your draconian approach (but I understand where it’s coming from) because I do not think that issues like these should be just swept under the rug.
Ironically, machines autogenerating Seinfeld-like scripts literally cannot have intent.
We can go a step back to the intent of the creators of the system. I assume the creators did not intend to auto-generate distasteful and/or bigoted jokes. But it's Seinfeld-like. And it seems if it's supposed to generate bad jokes eventually it'll make some mean jokes.
It's something of an interesting thought experiment about intent vs. effect with publication of speech. All effect, no intent.
Prima facie, I can see the logic in treating the operator of the AI legally/socially responsible in the same way as if they had published the material themselves. If you unleash a bot that e.g. goes around writing defamatory book reviews, you should be liable for defamation. But I am not sure.
We treat negligence differently from malice in both criminal and civil contexts, and even within negligence there are gradations about whether or not a reasonable person would have foreseen the outcome.
I think LLMs slot into that model just fine.
Really curious how this applies to infinite, AI generated, content without oversight.
Like, what is the “intent” in this Seinfeld example. Can an LLM have “intent”?
It’s the intent of the LLM instructor/creator that matters for things like mens rea. And in the case of “no oversight”, that just sounds like typical negligence, like leaving a car in neutral when parking on a hill.
By that reductively absurd measure, Earl Butz is an artist. [0]
[0] https://en.wikipedia.org/wiki/Earl_Butz#Scandals_and_resigna...
I mean killing is wrong, but so is being a dick.
[0] https://en.wikipedia.org/wiki/Charlie_Hebdo_shooting
I don't really understand the point you're making. Killing is orders of magnitude more wrong thank "being a dick" (meaning: making an offsensive or bad joke). I'm sure you agree, so I don't see what's the goal of this comparison.
And I think that using Charlie Hebdo in this context is a bad taste. Saying "see, it got people killed, so they were doing something wrong" is basically acknowledging that terrorism works. People that died in Charlie Hebdo did nothing to deserve this.
Surprised this has to be said.
I'd be very curious how widespread your opinion is these days now that support of artistic censorship has become much more mainstream.
The "but nobody is laughing" part would run counter to this. The premise of the whole joke is that the referenced jokes aren't funny.
The problem with Twitch isn’t boundaries, but that they can’t have the wrong people pushing boundaries if you know what I mean.
As if anything is innocuous if couched like "Here are some jokes I would tell except they aren't funny: (a series of offensive jokes)"
> Anyone have any suggestions?
> I'm thinking about doing a bit about being transgender is actually a mental illness.
> Or how all liberals are secretly gay and want to impose their will on everyone.
> Or something about how transgender people are ruining the fabric of society.
> But no one is laughing, so I'm going to stop, thanks for coming out tonight
https://livestreamfails.com/clip/150015
Supposedly the script is generated by ChatGPT, so seems their AI haven't really fully learned their own content policy yet. Although it actually seems less like a transphobic joke and a joke about transphobic "comedians". A transphobic-phobic joke if you will.
My experience with talking to chatbots has been essentially that the responses are completely open ended and can be taken to mean literally anything you could conceivably infer, and if you continue the conversation as such the chatbot will 100% double down on whatever you assumed. the same prompts and the same context will lead to completely different outcomes based on how the human user responds next. Everything is both a joke and not, both a lie and an honest mistake, everything and nothing at the same time because it's not logical in the first place.
I've noticed users of stable diffusion/image generating AI's don't seem to have any trouble confusing the output for something logical, but with things like ChatGPT it seems like an almost universal fallacy.
This joke actually works because the canned laughter didn't get inserted here, but again - it's one of those "if it gets a little bit on it, we're banning the whole thing" deals
A lot of shows from the 90s would violate those Twitch ToS IMO ignoring the copyright part. Thinking of "Married with Children" and the enormous amount of fat shaming and sexism.
I watched Carmen (the Opera) yesterday, and it’s weird how much the opera has aged compared to my first seeing it 20 years ago. Like possibly more in 20 years than the 200 before that.
Loved the show back in the day and feel weird now not having picked up on that back then.
It's kind of silly - it's not that the robot was bad but that if you keep it on 24/7 it will eventually say something bad. An infinite monkey typewriter will contain transphobic text.
I think the point of bans is to keep TOS-violating content off of platforms, not to rehabilitate people.
I’ve got no evidence, but I would like to believe that Twitch has experimented with ban lengths and found 14 days to be enough to reduce TOS-violating content but not so much that it reduces total content.
Since when?
If you stream a video and showcase a comedian making ToS-violating jokes and content, twitch will ban you regardless of whether you're a person or a robot or a script or a fish in a tank.
The only difference is if you're a major company or rights holder that Twitch wants to have a good relationship with. Or if you're a person with connections to the twitch mods and admins, they have a tendency to play favorites. But being AI generated content or not really isn't the issue here.
I thought it was to allow rightthink and kick out wrongthink so we may curate areas in which the latter has been filtered out, a filter bubble if you will...
So.... Machine Learning is a thing.
> That's the whole point behind bans, is it not?
No, it may be one point behind imposing bans but it is not the whole (only) point. People can also learn from watching other's make mistakes. Bans communicate to people other than the person getting banned that something is unacceptable.
In this case it seems to communicate that "but it is not a person it's a bot" will not be a srong enough reason to break their rules and escape a ban.
A ban is just a message to the creator that it should only come back once it has figured out how to properly label its dataset so that the training complies with the ToS.
They can't own accounts and they can't decide what content is shown any more than the rain can decide if my dog gets wet.
So the notion of people being absolved of responsibility for the procsses they set in motion is absurd.
15 years and people are still confused about this one?
https://www.youtube.com/watch?v=hmmAdbDF1xA
One way or another, the concept of an out of control AI is not sustainable.
The era of "my chat bot does whatever it wants and I'm not liable for it" is likely going to end in the developed world over the next decade or two. You run it, you're responsible for it.
If every strike on youtube was a permanent ban there would be many more creators on alternate platforms like rumble. They're not there to teach you, just to incentivize you to follow their rules or go somewhere else. Thus the complex system of strikes, demonetization, temporary live-streaming bans, etc.
I imagine a sufficiently advanced AI or even human could create a convoluted word or phrase and breadcrumb enough to walk it back to whatever would have been censored, a la "for want of a nail" style.
There was a Twitter account called "Fuck every word" which went through every word in the dictionary in order, writing a twit which consisted of "Fuck X" where "X" is the word.
It got promptly banned as soon as it reached certain word starting with N.
I've also seen some... interesting... things from the Curie model, so this isn't too surprising to me.
“Earlier tonight, we started having an outage using OpenAI’s GPT-3 Davinci model, which caused the show to exhibit errant behaviors (you may have seen empty rooms cycling through). OpenAI has a less sophisticated model, Curie, that was the predecessor to Davinci. When davinci started failing, we switched over to Curie to try to keep the show running without any downtime. The switch to Curie was what resulted in the inappropriate text being generated.”
——
A bit more detail:
https://www.vice.com/en/article/y3pymx/ai-generated-seinfeld...
And screenshots from their Discord with an explanation (this is quoted in the article above)
https://twitter.com/ryanbrianjones/status/162254854589825024...
Imagine the other more critical production applications who have made the same reasonable engineering decision.
That's the point of the comment, if you were applying AI in more serious scenarios, would this be an AI confidently advising a doctor to overdose a patient, or advising a judge to incarcerate someone based on racist inferences from the set of people currently incarcerated?
I'm confused, I thought it was supposed to be generating Seinfeld episodes, not jokes?
Edit:
However, the AI may not have been trained in a neutral way, and the person using it may not be using it in a neutral way.
If you create an AI product, you are responsible for its output. People aren't mad at the model itself, they are mad at its creators. Why did they create an AI which ends up insulting people and breaking the ToS of the platform it is on?
Here are three models: * A language model that outputs jokes, * A self-driving car model that outputs driving instructions, * An autonomous combat drone model that outputs engagement targets.
Where do you draw the line? Where do you start to "get offended"?
Just saw your Lem quote: Isn't an AI exactly not "unthinking"? That's the whole purpose of machine learning: to "learn", to recognise patterns, to abstract away, no?
The former has a probability distribution of sequences reflective of the dataset and context whereas the latter is pure random chance assuming input seed is random.
* Makes sure the people controlling the machine are incentivized to correct it (not that I think they needed it, but what I think is irrelevant)
* Ensures that content that Twitch management doesn't want on their platform, and which is in violation of their TOS, is removed immediately, regardless of the thoughts/actions of the people controlling the machine.
As far as Twitch is concerned, there's a channel that's breaking TOS, repeatedly. They're not "punishing" the owner of that channel, they're just enforcing the TOS, something the channel owner agreed to when creating their channel.
At least that's what would make sense, and Twitch moderation is far, FAR from making sense in general, but in this one thing they might be reasonable.
A clip of the joke from the banned channel:
https://clips.twitch.tv/CalmFrailPlumageShazBotstix-ITcqL0Hh...
And Twitch's rules only seem as arcane as YouTube and Facebook too. Maybe not Twitter.
To me, this is less a commentary on AI than on the absurd sensitivities of community moderation.
If you build a bot to automate your streaming so you can have 100% stream uptime and you end up with 0% uptime, that's because your product failed drastically. (Keeping in mind, as arbitrary and subjective the rules of Twitch are, the vast majority of human streamers have never been banned for arbitrary moderation)
I think the usual response to comments like this is: "Oh you sweet summer child".
Not that you're technically wrong, but from years of headlines it's often enough that even if extra care is taken, it's not always sufficient.
> reasonable engineering decision
They swapped out the AI model. That's kind of a big deal.
That being said, every company I've been at with a production outage has gone into "fix it and ask questions later" mode. Maybe other companies have more process?
But when you put it like you said in your comment, it does feel more weighty
If you think that this is "drastic" you should read "Case Study 4: The $440 Million Software Error at Knight Capital". https://www.henricodolfing.com/2019/06/project-failure-case-...
tldr; At Knight, some new trading software contained a flaw that became apparent only after the software was activated when the New York Stock Exchange (NYSE) opened that day. The errant software sent Knight on a buying spree, snapping up 150 different stocks at a total cost of around $7 billion, all in the first hour of trading.
Scarier was the day I put out a change request shortly before the end of the day in Asia, went out to dinner and drinks with some colleagues, stopped by my office on the way home, and merged the approved change request, during early morning trading in the US. The next day (in Asia) I woke up to discover Goldman had a roughly 28 million dollar trading loss. I spent a couple of minutes proving to myself that my change couldn't possibly have been the cause, and then realized I would have been woken in the middle of the night by a phone call if there was any possibility that my change had caused the trading loss. Since then, I don't merge code at the end of the day, and try to avoid Fridays. I was already in the habit of not coding if I had consumed any alcohol, but added the habit of not merging if I've consumed any alcohol. There's nothing like a 28 million dollar panic to get your practices in shape.
Especially considering this is the description for the model they left:
> higher quality, longer output and better instruction-following
and this is what they moved to:
> Very capable, but faster and lower cost than Davinci.
If nothing else, the adage 'you get what you pay for' works.
ref: https://platform.openai.com/docs/models
In that situation I still try to exercise my options a bit...
Seeing that this new model is implicitly worse at following directions, knowing that 'production' (this is a Twitch channel) depends on obeying a certain set of rules... switching to it may be ill-advised.
I don't want to be seen as overly judgmental - the Twitch channel comment works both ways; derogatory/supportive. I get why one wouldn't really foresee this or even care to. Then I wonder, why not leave it down for a bit? The hype train is fickle but it's not that precious, either.
I'm nearly 100% certain that the scripts are generated through prompt engineering, with a random prompt (e.g. tell a joke prompt, talk about a new restaurant prompt) being selected for the scene.
From what I can gather they first used the older, cheaper GPT-3 models, only upgrading to davinci-003 when it was profitable. The older GPT-3 models proved fine and didn't generate edgy content for the several months they were up and running.
But I think the change that broke the camels back was they added a "2006 Laugh Factory incident with edgy content" prompt and only tested it on the davinci-003 model - the new models having been wiped clean of antisocial training data, while the older smaller models still having contentious content encoded in the model.
So, davinci-003 did fine producing "politically aligned" text with the "edgy" prompt because it's "cleaned", but when the openai API for davinci went down the fallback was curie. The older "unclean" curie model combined with an edgy prompt inevitably caused what we saw here.
Cue MRE/MLOps job listings.
At a certain level AI trained on inputs is just a big 'some people say' box.
He liked bombing, but only with jokes he believed in. He wouldn't tell a meta-joke like this with such a lack of punchline.
When did that happen?
Sounds like trying to prove a negative. If people want to convince me that cancellation is a real thing for comedians, I'd want to see...examples of comedians who actually got cancelled. CK and Chappelle ain't it.
I have an inkling of how this would go. I'd do a quick google search and offer up a handful of names. The counter argument would probably be along the lines of 'that wasn't really cancellation, it was actually...' and then we'd go back and forth debating the meaning of words.
The point I was specifically trying to make was that because Chapelle defied expectations and was successful, that does not mean cancellation isn't a real thing that can happen to other comedians.
Fun.
He won a grammy. I don't know what more you can want. It seems like cancelled has come to mean "everyone doesn't immediately fellate me"
You're not entitled to have anyone listen to you, or like you, or give you any money for it, but you can totally do it.
Getting cancelled means you sell out Madison Square Garden.
Getting cancelled is being called names.
Boohoo.
> I think “a culture where social norms are enforced with repeated and vociferous public shaming” is the most useful way to define the term.
That might be the most useful way for the author to make their point, but as far as I am aware, for most people being cancelled definitely entails some sort of deplatforming, e.g. some BigCo not wanting to be associated with your name. If you keep doing what you're doing while being insulted on Twitter, that's not being cancelled, that's just ... well, Twitter.
> What I find more interesting is that this argument requires the very thing that it laments. That is, in order to make this argument, you need to have figures like Louis CK who escape/survive the consequences of public shaming, but simultaneously to assert that this is a bad thing.
In no way does the tweet embedded at the top of the article imply that it's a bad thing that Louis CK is playing MSG. It's saying that if we define being cancelled not in the way the author of the article did but the way I outlined above, someone who is supposedly cancelled would not be playing MSG. From there it makes the pretty big leap to say that cancel culture in general is not real.
I'm not even that invested in the whole debate but this article seems completely misguided or in bad faith.
Who's counting? It's trivial to find examples of people who've lost their jobs for innocuous reasons. See: David Shor. It's bad faith to pretend this doesn't happen. Usually for proponents or those denying the phenomenon exists the goalpost shifts to effectively shrugging it off as minor. Speech that begets consequences isn't strictly limited to bigotry, as we are told.
> someone who is supposedly cancelled would not be playing MSG.
He couldn't, until he could. It's not necessarily the case that public shunning lasts forever. Disingenuous to construe what happened to that man as something entirely different to what "cancellation" colloquially refers to.
This is complete delusion and shows absolutely no understanding of modern politics.
""" We often hear that Woke Marxism is a new ideology in the world. I've even said so. Well, it isn't. It's just an old one repackaged in various ways without any essential changes made to it at all. Critical Race Theory is Race Marxism in the same way as what we usually call Marxism is Class Marxism. Radical Feminism is Sex Marxism. Gender ideology and Queer Theory are Sex, Gender, and Sexuality Marxism, or just Gender Marxism or Sexual Marxism to be more concise. Fat studies is Fat Marxism. Disability studies is Ability Marxism. Critical Education Theory (Critical Pedagogy) is Knowledge Marxism in terms of what it means to be formally educated or literate within the existing system. Postmodern postructuralism is Language Marxism. Postcolonial Theory is National Origin Marxism. And on it goes, with all of these cobbled together by intersectionality, which is Identity Marxism. In this episode of the New Discourses Podcast, James Lindsay breaks down the essential kernel of Marxist thought and makes perfectly clear how all of the claims in this paragraph are true. Join him to understand what's going on in our world today in these simple, straightforward, yet sobering terms. """
How can you tell where down is though? If the group you're punching has enough clout to get you in trouble, it probably wasn't down.
I would argue it already has to a large extent, but I'm not sure it's actually a problem. This has allowed new companies to arbitrage the moral standards. Free market life.
But companies can and do respond to larger concerns. Not because they’re moral animals but because it’s good business.
And yes, it is harder to attract good talent running an old school “we take no responsibility for anything” business. I’ve turned down Facebook/Meta multiple times just because of their business practices. And they’re not that bad compared to the totally amoral platform you propose.
It seems to have accelerated since then though
* Can We Take a Joke? - https://www.imdb.com/title/tt4324916/
We've been sensitized to so many more words than in, say, the year 2000, that even for one who doesn't have a dog in this fight, watching early '00s media (let alone earlier) can be distracting because it's hard not to notice the now-red-flag words or topics that are thrown around casually—especially in standup, or on comedy TV shows. It's been a pretty fast shift, and I'd put the start at the tail end of the '00s or early '10s.
I mean, hell, just look at where mainstream Democrats stood on gay rights in the early '00s, for that matter. We've gone from, "gay marriage? I dunno, that might be a tad too far" to "we must protect trans rights" in a remarkably short span. I dunno, maybe the OG civil rights era had a similar feel of The Discourse shifting a ton in only a decade or two—I wasn't there for it.
It's probably also worth noting that this shift looks rather more total from a very-online perspective than it does IRL, which is probably a source of some of the ongoing friction. Online, it's done, the shift has happened, you're a cave-man if you're lagging behind on it at all—offline, much less so.
Here's the part where I rhetorically taunt with my weaponization of the virtue of free enterprise:
"You have a right to free speech, but I have a right to decide what I allow on my platform!"
Yeah, I don't entirely believe that either. I see the weasel here and so do you.
So I have a right to disregard the value of free speech. But is that really what we want?
Don't like it? Feel free to make your own platform that allows absolutely everything up to the line of legality! Just remember, nobody owes you financial success or a large user-base.
Curation and censorship are strange cousins, don’t you think?
It seems like it’s only censorship if it’s done with big furnaces and a gun barrel by a man with tiny glasses and creased pants. Governments censor. Corporations curate. Right?
I’m not sure if I even want to dignify the following state of affairs by calling it an open secret, but the US is governed by corporations. I won’t insult you by expounding on this line of reasoning. You see what I’m getting at.
And, yeah, we've let corporations take control again. But let's be clear, I'm much more concerned about the likes of Halliburton than twitch. Some in government have expressed some interest in breaking up big tech and undermine big oil, so it's not a perfect equivalence, thankfully.
I'm with Ken White[1] on "cancel culture." Sometimes, it goes too far. But by and large, "cancel culture" is better framed as "speech that is critical of other speech." And, yeah, the people at the helm of big platforms have bigger thumbs than the rest of us. But nobody stopped anti-woke Musk from taking over Twitter, did they?
[1] https://popehat.substack.com/p/our-fundamental-right-to-sham...
Amazon's capacity to affect the world, through both its enormous market share in retail and its control of large portions of the internet's infrastructure has the potential to extend their influence far further than Halliburton's ever did.
There's real potential that the diplomacy that Amazon encourages to avoid any serious roadblocks in the China->US disposable crap pipeline will make the Iraq War look like a game of Capture the Flag.
It's also a bit nonsensical to whine about people getting offended by comedy while simultaneously saying comedy should be allowed to be offensive. If you do offensive comedy then some people are going to get offended by it. If they didn't then it wouldn't be offensive!
There was a tongue in cheek post on here a couple weeks ago where a user suggested in the future we'll prove we're human by peppering racial sluts into all of our communications.
I don't see any comedians getting arrested lately (for their comedy, that is).
Your logical fallacy is: whataboutism. Just because you've seen worse doesn't make this incident right.
The AI is terrible at making jokes, so the stream is funny to watch because it's a non-stop torrent of repeated failed attempts at telling jokes.
"Regular" jokes are delivered in the exact same way, with the exact same result. What you are reading as intentional irony and/or sarcasm really is just you projecting your expectations.
There is no way whatsoever to distinguish a self-deprecating joke from the failed delivery of an offensive one. If anything, the latter is more likely than the former, since failed delivery is the norm. (Talking about this stream in particular, not AI-generated humor in general.)
The way its output is received is all that matters.
How did you get this from someone literally repeating what the bot actually said and thinking it seemed 'mild'? Talk about projecting.
Interesting DoS/sabotage vector.
This is what happens when you have people "trying" to do good but not following principles.
why would you want to let 4chan train the AI to be racist/fascist/transphobic/whatever and to have it start playing around with police/military killbots or industrial chemical plants or nuclear weapons because it believes X nationality/minority is human vermin?
like yeah it's kinda not just "what it already does" it's kinda the whole point of the experiment. Let's figure out how to build the safeguards against 4chan turning it racist while the stakes are still low and the outcome is some mildly questionable seinfeld jokes and the bot comically panicking at a choice between a city of 20 million being nuked and having to say the n-word.
And remember that Peter Thiel is already pretty far on the right, lol.