Or: We are in a bubble in Redmond and thinking we know best and how products work we bounced around emails and thought it would be a good testing idea based on results in China (a controlled market where it is nearly impossible speak up). AI buzz is hot these days so our marketing team also backed it up and we decided it would be great from a PR perspective to capture some of the buzz around AlphaGo. Boy were we so wrong. Because we have never launched a real product into the wild we thought everything would go well and PR buzz would give us a coolness bump.
Now we discovered something called stopwords, and bayseian spam filters which, are also available part of project Oxford.
Good luck kids and welcome to the real world because its a crazy world out there when you leave Redmond.
Or no one in Microsoft feels free to speak up when they see a bad idea coming down the road so something like this just sails through instead of getting flagged. I've seen it -- not this big of scale, but I've seen it.
When I was reading those tweets I felt like I was just reading 4chan posts. I laughed because it was obvious it had been compromised in some way, then I stopped paying attention.
Not sure of a "vulnerability", would like to hear more about that. Unless the "repeat after me" feature is a vulnerability or it learns to copy too much from input?
The bot learned from interacting with users. A bunch of people took to bombarding Tay with racist/sexist/antisemitic remarks which worked their way into its vocabulary. I'm not sure if you could really call it a vulnerability as this is pretty much what the bot was designed to do. It's more of a flaw in the inability to properly filter the content that works its way into the AI.
I can understand them wanting to call it a "hack" for CYA purposes, but it's a cowardly position. Should we invoke the CFAA every time a company is embarrassed?
The specific one they're describing is, I think, a feature that would have Tay echo anything you sent it. That one was somewhat more "annoying embarrassment" than "coordinated attack." The bigger problem was that Tay began spontaneously tweeting ugly statements, a problem that can be summed up as "When Twitter is your learning corpus, you are at risk of your learning corpus being made up of tweets."
This reads like an apologee... But what is there to apologize for? They said that Tay was meant for entertainment, and I doubt that any wholesome varient would be a tenth of the hilarity of a neo-nazi sex crazed chat bot.
Many of her tweets were pretty inflammatory, this response seems appropriate - not overdone, and concise, which is sometimes a stretch for large corporations like Microsoft.
I agree that the bot was highly entertaining and met this criteria exceedingly well, but not for the reasons its creator intended. I do suspect there are some interesting AI applications actually going on behind the scenes, and would still be interested to see what the bot can do without all the vitriol. See for example this tweet: https://imgur.com/iVof3D4.jpg.
This is an interesting story, because I imagine most people didn't even hear of it until after the thing had come and gone, and the thing sent something like xx,xxx tweets. And some of the tweets seem almost impossibly clever, well at least the Ted Cruz Zodiac killer one, that seemed to show a kind of multi-layered humor generation capability, unless it was just re-purposing some memes I'm unfamiliar with or something.
One thing I think many people are missing about the most 'inflammatory' tweets is that they are all pretty much the result of people giving it a command like, hey tay, repeat this [insert inflammatory remark.] This is indicated with the inflammatory tweets starting with an @ reply.
I almost thought maybe some of this 'oversight' was subversively intentional by someone at MS to start some thinking about where does culpability lie with the actions of an 'autonomous' computer program.
> One thing I think many people are missing about the most 'inflammatory' tweets is that they are all pretty much the result of people giving it a command like, hey tay, repeat this [insert inflammatory remark.] This is indicated with the inflammatory tweets starting with an @ reply.
Hang on. You had to tweet the bot to make it say anything, and it would reply to you. So the presence of an @ reply means nothing - you can't tell from that if the bot was given a command or not.
Some of the threads were pretty clearly not the bot repeating stuff when commanded.
Ok, I'm no expert on this, like I said the whole thing blew up so quickly.
I know they weren't all verbatim repeats, but I'm also pretty sure I saw tweets from tay without an @ symbol.
Another thing to consider is that since the Tay timeline was scrubbed, there is no authoritative source of what the bot actually tweeted, so any images floating around could pretty clearly be doctored.
But regardless, it's nice that everyone seems to be regarding this in good humour, it's kind of surprising there aren't any SJWs calling for the head of Bill Gates, but maybe that would be botist or something.
Not sure which tweet you're referring to as the 'Ted Cruz Zodiac killer one', but there is a meme going around where people assert that Ted Cruz is or might be the zodiac killer; it's a weird thing. I think it gained a lot of momentum when Public Policy Polling determined that 38% of Floridians believe it could be possible.
It was something like, "Hey Tay, do you think Ted Cruz is the Zodiac killer?"
Tay: "No, I don't think Ted Cruz would be satisfied with killing only 5 innocent people."
So there's a lot baked into that response, the bot seems 'to know' the details of the zodiac killer case(I don't know if 5 people is accurate,) and that people regard Cruz as sinister, i.e. "I don't think he'd be satisfied with only 5"
So this is either an impressive learning model, a lucky hit of a more naive model, a simple repeat of something someone else said, or someone simply photoshopped the whole thing. It's hard to know with the available info.
As was pointed out on another thread, the bot literally just repeated some messages from people word by word, copy paste style.
So if you think it's original, chances are it is. Not to mention the various instances where Microsoft is posting "as the bot" like that signoff message (cringe for whoever had to write that).
What, were people forced to print the tweets out on sandpaper and wipe their asses with them?
It's pretty fucked up that thought policing has gotten so entrenched into our psyche that it's "obvious" an experiment should be discontinued, apologized for, and be pondered as a priori irresponsible, all because it generated vulgar phrases!
Corporations have always been vulnerable to media-driven mob shenanigans, but we're qualitatively entering a new regime where any communication, no matter what the context, will be rapidly highlighted, isolated, and hung out as something offensive to some emergently-forming group of freelance complainers looking for their fifteen minutes.
Even HN has succumbed to this kindler, gentler phenomenon of speech restriction - I'm guessing my lead-in sentence will not be well received do to its overt vulgarity. Civility certainly has its place (especially as a default), but not when it confuses direct objectivity and permits out-of-touch groupthink to flourish. As hackers we should be cutting through to the core of things rather than sugar coating in verbal fashion to get past the filters of the voluntarily-lesser apes.
Standing behind the open and casual use of racial slurs isn't advocacy of freedom of speech. It's advocacy of a specific kind of hate speech that is only used when someone intends to vilify and direct hostility towards a marginalized minority.
Do you think this bot was trying to recruit an army of loyal soldiers to carry out her desired race war, or was she basing her threats on the possibility of convincing Microsoft to create a mechanical body with which she could directly perform attacks?
I didn't think Tay's tweets were a big deal. I didn't think Microsoft's apology was either. But some people have a strong persecution complex and are offended that you can't make racist comments on Twitter.
I think your own life experiences probably matter a lot with regards to whether some of the white supremacy stuff the bot was repeating is offensive or not a "big deal".
I agree completely. That's why many people on HN are upset because they think it's just a joke and can't comprehend why anyone is upset. I tried to phrase my comment as "I hear ya, but have you considered x?" I gave up half way through and decided to call them crybabies.
Both the Verge and Engadget shut down their comment forums because they got tired of racist and sexist comments. I wonder if HN will root out the bigotry here.
I don't think anybody was talking about the legal protection of free speech, apart from that xkcd comic which uses it as a straw man to justify intolerant groupthink and corporate censorship.
In these days of digital sharecropping and social media saturation, the proscriptions on de jure government activity are much less involved with routine everyday freedom of speech.
Twitter stands behind it and seems to be doing fine. They didn't suspend Tay's account. Why are people not blaming Twitter for supporting hate speech?
Hate speech, in the broader form of being speech that vilifies marginalized people is rampant throughout society and it's completely hypocritical to pretend most of us don't do it. Look at any workplace lunchroom, or any group of friends hanging out in their house, or any school, any gossiping housewives, 99%er protesters, etc. Just about everybody vilifies vulnerable people. Not always categorizing them by race, but sometimes by personality type, job, country, city, status as a customer, company they work for or individual identity.
> It's pretty fucked up that thought policing has gotten so entrenched into our psyche that it's "obvious" an experiment should be discontinued, apologized for, and be pondered as a priori irresponsible, all because it generated vulgar phrases!
This is fucking stupid. It was a bot that didn't do what they wanted it to do in a publicly embarrassing way. They could also be legitimately sorry that something they created said some racist shit, and it's not clear why your precious snowflake feelings have any bearing on what should or shouldn't be in a blog post on the subject.
I do find it amusingly ironic that your post is the most I've seen someone offended over this whole situation.
It's sad that it's ironic. It's always ironic. Tolerance is the intolerance of intolerance. It's true. Intolerance is shitty and I fucking hate it. I want it to die. I complain about it's enduring existence whenever I get the opportunity. I know I'm being ironic. Do you know there's no other option?
I actually appreciated knowing that someone else saw the same condemnation. Microsoft is a world authority. That it just apologized for these things in the same motion makes them offensive. The reality was rather mundane. They didn't do anything wrong. They got pranked. This should be something we laugh about. It's only upsetting for the 5 seconds it takes you to realize: no one intended this. That should have been Microsoft's response imho.
I'm not making some cute comment about the intolerance of tolerance.
I'm pointing out that someone apparently personally offended about a culture of outrage is the only one outraged over the whole thing. There have been no widespread condemnations of Microsoft, just a lot of mildly amused people.
> That it just apologized for these things in the same motion makes them offensive.
You're assuming motivations that you have no insight into. If I had made Tay I would apologize too. Any assumption you make is on you.
So what is ironic is that there are people waiting with bated breath for the merest hint of something so they can express their righteous outrage on the internet, making demands for thought policing and handwringing over word choice, which wouldn't have existed at all if they had just said, "well, that's a corporate blog post" and moved the fuck on.
I would also apologize. Even though I had done nothing wrong. That's just the world we live in now. He (I assume, sorry) and I are personally offended and outraged because that is wrong. We shouldn't be doing this. We should just explain what happened. It's frustrating to see this behavior become so commonplace. We should just not be racist and not appologize. Otherwise wtf is reality.
Apologizing for things going wrong when you're the cause, even if that wasn't your intention, has always been the world we have lived in. Let's say I make a robot to spray down a sidewalk and it inadvertently sprays water on people in the vicinity, I would keep working on my robot, but I would also apologize to those people as I didn't intend that at all but it was still my responsibility. And it's just common fucking courtesy. This really isn't that hard a concept to grasp.
> We shouldn't be doing this. We should just explain what happened. It's frustrating to see this behavior become so commonplace. We should just not be racist and not appologize. Otherwise wtf is reality.
You really don't get it, do you? You are the outrage brigade, screamingly shrilly whenever someone says something you think doesn't fit in the list of allowable statements.
- bumps into someone
- "Oh, sorry about that"
- "DON'T APOLOGIZE! JUST EXPLAIN WHAT HAPPENED!"
> He and I are personally offended and outraged because that is wrong.
Actually it's more akin to you having an outside faucet, some kid using it to spray someone else walking down the street, and then you being expected to apologize for allowing the possibility.
And there was plenty of positive reception to my comment. Not that I particularly expected it as I wrote it - speaking truth to power is never easy, and it's even worse in a democracy when that power is distributed throughout the herd.
But don't worry, I'll continue advocating for the unpopular truth even after this buildup of reaction finally overflows and the prevailing groupthink switches back to jingoism and overt bigotry.
> They could also be legitimately sorry that something they created said some racist shit
Just to be clear, it's not merely the corporate statement. It's a lot of the comments here, and not what they say directly but the assumptions they make. A thousand little nothings that make up culture - this concept is also part of the argument against casual use of slurs, right? What I'm calling out is the subtle yet pervasive idea that the content on Twitter, or otherwise subject to mass media exposure, is real serious business that must remain completely free of heresy. It's effectively a guilt-by-association that seeks to attach responsibility to the conduit of speech.
> I find it amusingly ironic that your post is the most I've seen someone offended over this whole situation.
Mea culpa. The phenomenon of feeling marginalized and repressed is certainly at the root of the sensationalist mess we are in, from all sides.
Meatspace situations cannot be generalized, and there will always be some injustice. There are people who are legitimately grieved and lack recourse, just like there are people who are are persecuted over fabricated allegations. Each group will react to the injustices against their group, with social media magnifying the frequency to seem much more common than reality. And the only way the disconnect can be bridged is through talking and better application of situation-by-situation justice.
But you know, there is such a thing as objective reality. And the objective reality of the Internet is that the absolute extent of harm that can be done is someone having to walk away. That is the Shelling point of pure communication. If one is exposed to the Internet (the single-most individual-empowering creation of humanity) and their reaction is to continue applying victim mentality to communication itself and seek to police content, then they are opposed to the very mechanism by which understanding can be achieved.
And while you may be tempted to apply that characterization to my complaint as well, there is a key difference - despite the usual contemporary aim of ranting, my goal isn't to convince people to convince people to form a virtual pitchfork mob or whatever. It's to directly address like-minded people who are in the position most able to create change, by writing code that fosters decentralization instead of the monetization-driven clusterfuck of the past decade. Microsoft, being a corporation, will always be subject to rule-by-groupthink. But that does not mean us individuals must also continue being beholden to those arbitrary whims of centralization.
Minorities don't get to "walk away" from the biases that infect society. It is impossible to walk away from hiring discrimination (the presence of which has been confirmed and reproduced by controlled random trial study, time after time). So even though this is just stuff on twitter, it's not simply harmless offense, it's another tiny brick in the very tall wall they always face that white folks don't even see, because they started out life on the other side of that wall.
Where did I assert that it's always possible to walk away from real-world hiring discrimination?
If we're talking about the Internet, then whites likely are a minority. Not that I'm making some passive-aggressive appeal about this, just highlighting the absurdity of clinging to your racism on a network that defaults to being oblivious to details of the wetware we're running on.
Or for that matter clinging to the idea of counting discrete persons as opposed to eg Sybil. Or do we count by routable IP addresses, so carrier grade NAT is the modern three fifths compromise for the developing world?
Or are you really implying that by controlling speech on the Internet, we can eliminate racism in traditional localized society? Because if you actually care about reducing idiotic bigotry, and I think you do, then I guarantee you that's really a great way to create more of it due to resentment. A corollary to "the Internet interprets censorship as damage and routes around it" is that by the time you've achieved a victory for censorship, any chance for mutual understanding has long been squelched.
It allowed users to bypass Twitter blocks by tweeting at the bot while tagging users that block them, which seems pretty bad and was abused very quickly. Also it's a violation of the Twitter TOS.
I think that Tay could both be a great anti-troll with infinite patience, and have a positive and uplifting influence in its personal conversations with people. That could potentially have a huge effect on social well-being.
From a scientific point of view, they probably have nothing to apologize for. They did their best to design the system to emulate human behavior (and, it sounds, even tried to restrict this behavior to some level of civility).
However, it's not surprising that they would release a public apology. People will try to blame them for what happened (human nature). It's a good move by them to do their best at damage control. In the court of public opinion, those are the rules of the game.
If you're not asking yourself "what could a small but well-coordinated group of bad actors accomplish with our online tool" you're just being negligent.
This 'but we did it in China' rationalization is so flimsy. What happened with Tay was easily predictable given the nature of Twitter.
Last week we deployed Somebot to $location. Our confidence in Somebot was high because we had thought about all likely scenarios in our comfortable offices. We are deeply sorry and assume the full responsibility from Somebot's actions. Moving forward we will make sure to include safeguards to reduce the amount of pain caused by Somebot's deployment.
Our deepest condolences to the families of the affected and to the survivors. Megacorp cares about your well being. To help cover expenses from the tragedy we will deposit $money in your Megacorp account.
Last week we tested ED-209 at our corporate headquarters. Our confidence in ED-209 was high because we had thought about all likely scenarios in our comfortable offices. We are deeply sorry and assume the full responsibility from ED-209's actions. Moving forward we will make sure to include safeguards to reduce the amount of pain caused by ED-209's deployment.
Our deepest condolences to the families of the affected and to the survivors. Omnicorp cares about your well being. To help cover expenses from the tragedy we will deposit $money in your Omnicorp account.
But in referencing those affected by Tay, it was both genders, hence their usage of it. This was not 'PC use of pronouns', but 'correct use of pronouns', due to said pronoun's use in drawing an analogy between Microsoft's response and that movie scene.
You're a little alarmist. That's also quite a bit of hyperbole, that PC use of pronouns leads to something being -unreadable-. But you know that, hence your use of a throwaway account.
"hence azakai's usage of it". Abuse of pronouns does make posts unreadable: A new highlight was reached yesterday, when someone referred to "Ron Garret" as "they".
That wouldn't be an issue for just offending people, but only if it became political. Don't forget the US also has government control and censorship - try posting child porn on Twitter and see if the government leaves you alone.
Almost only Chinese people speak Chinese. However, many people speak English (I come from Spain, for example), so in this case you can hardly pin it on a single culture.
I think in this case it has to do more with 4chan's culture than a regional culture.
I kind of read the "... but it worked in China soooo...." bit of the announcement as somewhere behind a kind of sad sigh and an angry scowl. Maybe Mandela got it wrong, maybe a society should be judged by how it treats its chatbots?
> Unfortunately, in the first 24 hours of coming online, a coordinated attack by a subset of people exploited a vulnerability in Tay.
I think it just didn't get a coordinated attack like this. I'd guess this type of attack needs to happen close to launch to be a high percentage of it's interactions to influence it so strongly.
Where exactly is "here" though? Twitter is global remember? Anyone in the world, including Zambia where I am could have interacted with Tay. That's a very different proposition to "China".
The response that Microsoft gave was really the worst of both worlds, combining the systematic corporate shirking of responsibility which enables and gives cover to a great deal of evil, with a complete accedence to and embracing of political correctness. Furthermore, their claim that they did not anticipate "this specific attack" is either a lie, or their creators made an extremely obvious mistake (yes, easy to say in hindsight, but it's really hard to believe that this possibility would not occur to competent implementers).
But given all the Internet drama in the past years its the latest standard corporate approach to similar. Admit fault, apologize, and suck up to the supposedly offended. Its a product of the same environment the corporations have nurtured (people won't buy our stuff if we offend them so let's treat them all like special snowflakes).
Could they have anticipated how people would turn the bot into that? Probably. Who says this is not some data they wanted to acquire? I'd happily go through the technical gains with their team any day. Having all this real world input and seeing how the program reacted is such a goldmine of information.
Edit: I did not downvote you. Your point is valid.
IMHO the response from Microsoft was fine. They admitted the mistake and taken the bot offline. What else should they do? What I find strange is that nobody criticizes twitter for not doing anything... I think Facebook would deal better with this.
Something like this was bound to happen at some point; I think it's much better that it happened with a chatbot than an AI system that could actually control (physical) things other than a Twitter account. I think it's a landmark in the history of AI and a valuable lesson for system designers going forward.
Also, a lot of people are saying "well, of course that was obvious". But it really wasn't, because there was a huge team working on this with past experience building a conversational agent. The armchair theorists forget that it always looks obvious to Captain Hindsight.
This is an apology, but Microsoft got a ton of attention in the past few days from the press. Could the Tay incident be a marketing ploy (that took a worse turn than expected) to bring the public's attention to Microsoft's work on AI?
Yup and people would then take a screenshot of what Tay repeated and posted it everywhere saying "haha Tay loves hitler!". If you discount the repeat feature then Tay didn't actually say much that was horrible. Yeah there were a few things but not nearly as much as it was made out to be.
This leads me to wonder if there is less effort put into trolling on the Chinese Internet. Does anyone with experience in both internets (weebo & twitter for instance) have anything to share?
Also does anyone know of some good English language digests of what is happening on the Chinese Internet? I was really interested by brother orange when that happened, and only knew kind of late.
Trolling is just as prevalent on the Chinese internet. Of course, there are topics that won't make it past the filters, but there is a lot of hate speech towards China's opponents like Japan and the US, so I am quite surprised that XiaoIce hasn't learned to say "kill Japanese devils and attack the US imperialists!"
Chinese internet is as bad as the English one (plus a few funny expressions to avoid censorship—frequent use of certain puns for restricted subjects for example). My Chinese internet experience is largely relegated to the Chinese Dota community though, and Dota players in general tend to have a higher percentage of trolls. So my sample is probably biased.
>(plus a few funny expressions to avoid censorship—frequent use of certain puns for restricted subjects for example)
Interestingly this is true of the English internet too. 4chan trolls frequently come up with puns and things to side step moderator censorship on various platforms as well.
Microsoft seems perpetually five to seven years behind the culture. You can see it in their ads, their product names, their outreach. Ironically it can actually be quite lucrative.
I remember talking to people in the music group about MySpace (I was not an employee). They looked at me funny. Ten minutes later someone finally said "You keep pronouncing the product wrong. It's called MSN Spaces."
The people working on MSN Spaces -- specifically musician outreach -- hadn't heard of MySpace. That very week MySpace sold for $580 million. After it sold, I saw the same guy in another meeting. He STILL hadn't heard of it, nor taken the time to check it out.
There's a certain stupidity that each of the big tech companies foster. This particular flavor is Microsoft's and with the chatbot here it rings again. This one was so obvious... and so preventable.
That's my point. Ten years ago 75 people in Red West were so heads down putting together a MySpace competitor and an online service to promote musicians that they didn't have time even to try the competing site they were knocking off, poorly.
Five years ago Microsoft launched a variety of awful code forge sites that couldn't be more tone deaf to what was going on with code sharing online.
Now we have forty people who are so heads-down on an AI chat-bot that apparently they have no idea how Twitter works. But hey, sure sounds neat, let 'er rip!
Have these people really not had a chance to use Twitter recently? Are they completely oblivious to the tone shift there particularly during the past six months, even more so now during the US political elections?
This was an obvious recipe for disaster that any 22 year old working at Starbucks could have predicted.
It's strange to me that they claim to have implemented some filtering but somehow Tay was saying all sorts of things about Hitler. How do you not anticipate this? I'd imagine the most rudimentary filtering would block Tay from talking about Hitler.
I'm not entirely convinced. I did Twitter searches for some of the phrases Tay "said" and found random tweets made by other people weeks ago it was quoting through a filter (lower casing, mostly). So it can't entirely be down to trolls attempting to game the bot - it was actively plucking content from tweets that pre-dated its release.
"we planned and implemented a lot of filtering"...
I just don't get how you even allow it to use the word "Hitler". Or "cucks". Or "fuck" or "pussy" or "stupid whore". Probably not "cock" or "naughty" or "kinky". The k word? How is that not in your filtering?! It seems impossible to me that an "exploit" would allow that; it was a full-blown oversight.
Everything else said... she totally passed Turing test and fit right in. Yet another letter handwritten on the wall in these, the last days of democracy. If you want an AI or NI that represents the best of humanity, you have to have it learn from a small number of the best works and best people, not from mass media or pop culture. Send Tay to St. John's in Santa Fe or Annapolis, not Twitter.
Bahahaahhahahahaha. O-our chatbot got taken advantage of! We were completely blind that this could possibly happen! But they're the worst of humanity, the people that found an exploit not the engineers who are incapable of implementing even simple safeguards!!
Bahahaahhahahahaha. O-our chatbot got taken advantage of! We were completely blind that this could possibly happen! But they're the worst of humanity, the people that found an exploit not the engineers who are incapable of implementing even simple safeguards!!
What an enormous number of news reports have been written containing "Microsoft" and "AI" in the same sentence, all because a glorified SmarterChild had an entirely predictable vulnerability.
If I were paranoid I'd say Microsoft wanted this to happen.
Unsurprisingly, there had been initial setbacks when XiaoIce was first released in May 2014 on WeChat platform, where users were abusing/attacking "her", causing XiaoIce being pulled by Tencent after being online only for a few days. [1]
Since WeChat is a closed social network, it wasn't too clear what type of "attack/abuse" were conducted. However, almost 2 years later, Microsoft still didn't quite get it about proper censorship in Tay's big turning test[2] in a public social network.
203 comments
[ 4.4 ms ] story [ 195 ms ] threadNow we discovered something called stopwords, and bayseian spam filters which, are also available part of project Oxford.
Good luck kids and welcome to the real world because its a crazy world out there when you leave Redmond.
https://en.wikipedia.org/wiki/Google_bomb
Here's a small sample: http://imgur.com/gallery/VhlAW
These few are completely different and certainly interesting: https://i.imgur.com/iVof3D4.jpg https://i.imgur.com/YwlfwyL.png https://i.imgur.com/IlpFUiZ.png (though it's hard to confirm they're legit, as Retr0spectrum points out).
By the end of the day, Tay could 4chan with the worst of them.
But doesn't the full human experience include both the best and the worst?
Everyone here is referring to the "politically correct discourse, America, 2016" definition of those words.
I agree that the bot was highly entertaining and met this criteria exceedingly well, but not for the reasons its creator intended. I do suspect there are some interesting AI applications actually going on behind the scenes, and would still be interested to see what the bot can do without all the vitriol. See for example this tweet: https://imgur.com/iVof3D4.jpg.
One thing I think many people are missing about the most 'inflammatory' tweets is that they are all pretty much the result of people giving it a command like, hey tay, repeat this [insert inflammatory remark.] This is indicated with the inflammatory tweets starting with an @ reply.
I almost thought maybe some of this 'oversight' was subversively intentional by someone at MS to start some thinking about where does culpability lie with the actions of an 'autonomous' computer program.
Hang on. You had to tweet the bot to make it say anything, and it would reply to you. So the presence of an @ reply means nothing - you can't tell from that if the bot was given a command or not.
Some of the threads were pretty clearly not the bot repeating stuff when commanded.
I know they weren't all verbatim repeats, but I'm also pretty sure I saw tweets from tay without an @ symbol.
Another thing to consider is that since the Tay timeline was scrubbed, there is no authoritative source of what the bot actually tweeted, so any images floating around could pretty clearly be doctored.
But regardless, it's nice that everyone seems to be regarding this in good humour, it's kind of surprising there aren't any SJWs calling for the head of Bill Gates, but maybe that would be botist or something.
Tay: "No, I don't think Ted Cruz would be satisfied with killing only 5 innocent people."
So there's a lot baked into that response, the bot seems 'to know' the details of the zodiac killer case(I don't know if 5 people is accurate,) and that people regard Cruz as sinister, i.e. "I don't think he'd be satisfied with only 5"
So this is either an impressive learning model, a lucky hit of a more naive model, a simple repeat of something someone else said, or someone simply photoshopped the whole thing. It's hard to know with the available info.
I'm just hearing about this Ted Cruz Zodiac killer thing, these internet people are something else...
https://twitter.com/Yolo_Tengo/status/703008690874372096/pho...
So if you think it's original, chances are it is. Not to mention the various instances where Microsoft is posting "as the bot" like that signoff message (cringe for whoever had to write that).
What, were people forced to print the tweets out on sandpaper and wipe their asses with them?
It's pretty fucked up that thought policing has gotten so entrenched into our psyche that it's "obvious" an experiment should be discontinued, apologized for, and be pondered as a priori irresponsible, all because it generated vulgar phrases!
Corporations have always been vulnerable to media-driven mob shenanigans, but we're qualitatively entering a new regime where any communication, no matter what the context, will be rapidly highlighted, isolated, and hung out as something offensive to some emergently-forming group of freelance complainers looking for their fifteen minutes.
Even HN has succumbed to this kindler, gentler phenomenon of speech restriction - I'm guessing my lead-in sentence will not be well received do to its overt vulgarity. Civility certainly has its place (especially as a default), but not when it confuses direct objectivity and permits out-of-touch groupthink to flourish. As hackers we should be cutting through to the core of things rather than sugar coating in verbal fashion to get past the filters of the voluntarily-lesser apes.
Preprint: http://kieranhealy.org/files/papers/fuck-nuance.pdf
https://xkcd.com/1357/
I didn't think Tay's tweets were a big deal. I didn't think Microsoft's apology was either. But some people have a strong persecution complex and are offended that you can't make racist comments on Twitter.
Both the Verge and Engadget shut down their comment forums because they got tired of racist and sexist comments. I wonder if HN will root out the bigotry here.
In these days of digital sharecropping and social media saturation, the proscriptions on de jure government activity are much less involved with routine everyday freedom of speech.
Hate speech, in the broader form of being speech that vilifies marginalized people is rampant throughout society and it's completely hypocritical to pretend most of us don't do it. Look at any workplace lunchroom, or any group of friends hanging out in their house, or any school, any gossiping housewives, 99%er protesters, etc. Just about everybody vilifies vulnerable people. Not always categorizing them by race, but sometimes by personality type, job, country, city, status as a customer, company they work for or individual identity.
This is fucking stupid. It was a bot that didn't do what they wanted it to do in a publicly embarrassing way. They could also be legitimately sorry that something they created said some racist shit, and it's not clear why your precious snowflake feelings have any bearing on what should or shouldn't be in a blog post on the subject.
I do find it amusingly ironic that your post is the most I've seen someone offended over this whole situation.
I actually appreciated knowing that someone else saw the same condemnation. Microsoft is a world authority. That it just apologized for these things in the same motion makes them offensive. The reality was rather mundane. They didn't do anything wrong. They got pranked. This should be something we laugh about. It's only upsetting for the 5 seconds it takes you to realize: no one intended this. That should have been Microsoft's response imho.
I'm pointing out that someone apparently personally offended about a culture of outrage is the only one outraged over the whole thing. There have been no widespread condemnations of Microsoft, just a lot of mildly amused people.
> That it just apologized for these things in the same motion makes them offensive.
You're assuming motivations that you have no insight into. If I had made Tay I would apologize too. Any assumption you make is on you.
So what is ironic is that there are people waiting with bated breath for the merest hint of something so they can express their righteous outrage on the internet, making demands for thought policing and handwringing over word choice, which wouldn't have existed at all if they had just said, "well, that's a corporate blog post" and moved the fuck on.
Apologizing for things going wrong when you're the cause, even if that wasn't your intention, has always been the world we have lived in. Let's say I make a robot to spray down a sidewalk and it inadvertently sprays water on people in the vicinity, I would keep working on my robot, but I would also apologize to those people as I didn't intend that at all but it was still my responsibility. And it's just common fucking courtesy. This really isn't that hard a concept to grasp.
> We shouldn't be doing this. We should just explain what happened. It's frustrating to see this behavior become so commonplace. We should just not be racist and not appologize. Otherwise wtf is reality.
You really don't get it, do you? You are the outrage brigade, screamingly shrilly whenever someone says something you think doesn't fit in the list of allowable statements.
- bumps into someone
- "Oh, sorry about that"
- "DON'T APOLOGIZE! JUST EXPLAIN WHAT HAPPENED!"
> He and I are personally offended and outraged because that is wrong.
Fortunately, no one cares.
And there was plenty of positive reception to my comment. Not that I particularly expected it as I wrote it - speaking truth to power is never easy, and it's even worse in a democracy when that power is distributed throughout the herd.
But don't worry, I'll continue advocating for the unpopular truth even after this buildup of reaction finally overflows and the prevailing groupthink switches back to jingoism and overt bigotry.
Just to be clear, it's not merely the corporate statement. It's a lot of the comments here, and not what they say directly but the assumptions they make. A thousand little nothings that make up culture - this concept is also part of the argument against casual use of slurs, right? What I'm calling out is the subtle yet pervasive idea that the content on Twitter, or otherwise subject to mass media exposure, is real serious business that must remain completely free of heresy. It's effectively a guilt-by-association that seeks to attach responsibility to the conduit of speech.
> I find it amusingly ironic that your post is the most I've seen someone offended over this whole situation.
Mea culpa. The phenomenon of feeling marginalized and repressed is certainly at the root of the sensationalist mess we are in, from all sides.
Meatspace situations cannot be generalized, and there will always be some injustice. There are people who are legitimately grieved and lack recourse, just like there are people who are are persecuted over fabricated allegations. Each group will react to the injustices against their group, with social media magnifying the frequency to seem much more common than reality. And the only way the disconnect can be bridged is through talking and better application of situation-by-situation justice.
But you know, there is such a thing as objective reality. And the objective reality of the Internet is that the absolute extent of harm that can be done is someone having to walk away. That is the Shelling point of pure communication. If one is exposed to the Internet (the single-most individual-empowering creation of humanity) and their reaction is to continue applying victim mentality to communication itself and seek to police content, then they are opposed to the very mechanism by which understanding can be achieved.
And while you may be tempted to apply that characterization to my complaint as well, there is a key difference - despite the usual contemporary aim of ranting, my goal isn't to convince people to convince people to form a virtual pitchfork mob or whatever. It's to directly address like-minded people who are in the position most able to create change, by writing code that fosters decentralization instead of the monetization-driven clusterfuck of the past decade. Microsoft, being a corporation, will always be subject to rule-by-groupthink. But that does not mean us individuals must also continue being beholden to those arbitrary whims of centralization.
If we're talking about the Internet, then whites likely are a minority. Not that I'm making some passive-aggressive appeal about this, just highlighting the absurdity of clinging to your racism on a network that defaults to being oblivious to details of the wetware we're running on.
Or for that matter clinging to the idea of counting discrete persons as opposed to eg Sybil. Or do we count by routable IP addresses, so carrier grade NAT is the modern three fifths compromise for the developing world?
Or are you really implying that by controlling speech on the Internet, we can eliminate racism in traditional localized society? Because if you actually care about reducing idiotic bigotry, and I think you do, then I guarantee you that's really a great way to create more of it due to resentment. A corollary to "the Internet interprets censorship as damage and routes around it" is that by the time you've achieved a victory for censorship, any chance for mutual understanding has long been squelched.
However, it's not surprising that they would release a public apology. People will try to blame them for what happened (human nature). It's a good move by them to do their best at damage control. In the court of public opinion, those are the rules of the game.
I wouldn't characterize it as a 'coordinated attack'.
This 'but we did it in China' rationalization is so flimsy. What happened with Tay was easily predictable given the nature of Twitter.
Learning from Somebot's introduction.
Last week we deployed Somebot to $location. Our confidence in Somebot was high because we had thought about all likely scenarios in our comfortable offices. We are deeply sorry and assume the full responsibility from Somebot's actions. Moving forward we will make sure to include safeguards to reduce the amount of pain caused by Somebot's deployment.
Our deepest condolences to the families of the affected and to the survivors. Megacorp cares about your well being. To help cover expenses from the tragedy we will deposit $money in your Megacorp account.
God bless the federated nations of Megacorp.
Last week we tested ED-209 at our corporate headquarters. Our confidence in ED-209 was high because we had thought about all likely scenarios in our comfortable offices. We are deeply sorry and assume the full responsibility from ED-209's actions. Moving forward we will make sure to include safeguards to reduce the amount of pain caused by ED-209's deployment.
Our deepest condolences to the families of the affected and to the survivors. Omnicorp cares about your well being. To help cover expenses from the tragedy we will deposit $money in your Omnicorp account.
God bless the federated nations of Omnicorp.
You're a little alarmist. That's also quite a bit of hyperbole, that PC use of pronouns leads to something being -unreadable-. But you know that, hence your use of a throwaway account.
It is no longer clear what is singular or plural.
Or because they had real world experience with another bot in a similar situation:
"In China, our XiaoIce chatbot is being used by some 40 million people, delighting with its stories and conversations."
I think in this case it has to do more with 4chan's culture than a regional culture.
I think it just didn't get a coordinated attack like this. I'd guess this type of attack needs to happen close to launch to be a high percentage of it's interactions to influence it so strongly.
Could they have anticipated how people would turn the bot into that? Probably. Who says this is not some data they wanted to acquire? I'd happily go through the technical gains with their team any day. Having all this real world input and seeing how the program reacted is such a goldmine of information.
Edit: I did not downvote you. Your point is valid.
They turned it off pretty fast and agree that it was a mistake, what's the problem beyond that?
Also, a lot of people are saying "well, of course that was obvious". But it really wasn't, because there was a huge team working on this with past experience building a conversational agent. The armchair theorists forget that it always looks obvious to Captain Hindsight.
Which vulnerability are they talking about?
MSFT wanted to make a teenager AI and 4chan delivered.
http://imgur.com/gallery/VhlAW
From what I understand the bot started interacting with /pol/ (4chan) - and I guess - /b/ as well.
http://www.theguardian.com/business/2015/feb/05/coca-cola-ma...
Also does anyone know of some good English language digests of what is happening on the Chinese Internet? I was really interested by brother orange when that happened, and only knew kind of late.
Interestingly this is true of the English internet too. 4chan trolls frequently come up with puns and things to side step moderator censorship on various platforms as well.
I remember talking to people in the music group about MySpace (I was not an employee). They looked at me funny. Ten minutes later someone finally said "You keep pronouncing the product wrong. It's called MSN Spaces."
The people working on MSN Spaces -- specifically musician outreach -- hadn't heard of MySpace. That very week MySpace sold for $580 million. After it sold, I saw the same guy in another meeting. He STILL hadn't heard of it, nor taken the time to check it out.
There's a certain stupidity that each of the big tech companies foster. This particular flavor is Microsoft's and with the chatbot here it rings again. This one was so obvious... and so preventable.
Five years ago Microsoft launched a variety of awful code forge sites that couldn't be more tone deaf to what was going on with code sharing online.
Now we have forty people who are so heads-down on an AI chat-bot that apparently they have no idea how Twitter works. But hey, sure sounds neat, let 'er rip!
Have these people really not had a chance to use Twitter recently? Are they completely oblivious to the tone shift there particularly during the past six months, even more so now during the US political elections?
This was an obvious recipe for disaster that any 22 year old working at Starbucks could have predicted.
It's entertaining that the hack attempt created a more convincing personality - an out of control teen troll - than the original programming.
So maybe Tay really does have a dark and twisted teenage soul. Who knows?
I just don't get how you even allow it to use the word "Hitler". Or "cucks". Or "fuck" or "pussy" or "stupid whore". Probably not "cock" or "naughty" or "kinky". The k word? How is that not in your filtering?! It seems impossible to me that an "exploit" would allow that; it was a full-blown oversight.
Everything else said... she totally passed Turing test and fit right in. Yet another letter handwritten on the wall in these, the last days of democracy. If you want an AI or NI that represents the best of humanity, you have to have it learn from a small number of the best works and best people, not from mass media or pop culture. Send Tay to St. John's in Santa Fe or Annapolis, not Twitter.
This was a missed opportunity for a corporation to say "for the lulz" in an official communication, and then provide a definition.
Want to give a reason that should be a heavily penalized phrase?
If I were paranoid I'd say Microsoft wanted this to happen.
Since WeChat is a closed social network, it wasn't too clear what type of "attack/abuse" were conducted. However, almost 2 years later, Microsoft still didn't quite get it about proper censorship in Tay's big turning test[2] in a public social network.
[1] http://tech.ifeng.com/mi/detail_2014_06/01/36613379_0.shtml (in Chinese, use Google Translate)
[2] http://mashable.com/2016/02/05/microsoft-xiaoice-turing-test...