The linked post is itself linked to the API in question: Perspective [0].
The site has a demo where you can slide the "toxicity" from left to right and view the resulting bucket of comments sourced from various topics of discussion (Brexit, Climate Change, US Elections). You can also provide your own snippet and get a rating.
Obviously this kind of tool will sustain a lot of discussion about what "toxicity" is and how a ML system could infringe on speech by applying bias, either from the engineers building and evaluating the system, or from the inherent bias in data that Google themselves have already identified as a major problem. [1] For what it's worth, Perspective's site offers a definition of "toxic": "a rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion." The training dataset was ostensibly built by having humans rate comments on a toxicity scale based on that definition.
It seems as though the idea is to penalize poorly thought-out or inflammatory comments while encouraging in-depth or at least more measured responses. I think we can all agree that that is an attractive system to build: the main question will be if it can be done in a fair way and what latent problems might exist in such a system.
This is an interesting project for sure, and I see some great potential. I tried out a few phrases, including the one below that include commonly used words in "toxic" comments, but are not actually insulting anyone. This still had a high score (0.74).
> Unfortunately, most Americans are ignorant of the science about climate change. This is not because they are stupid but rather because they do not have all the data, or the data they have is unreliable.
I'm in no way saying this is the best way to communicate that message, and this could prompt a writer to re-phrase, but I don't know if I'd claim that this is toxic. I think that this could be a good tool, but it's clearly highly dependent on "common" understandings of words rather than actual definitions. Maybe that's the correct trade off though.
The problem here is that toxicity and depth are tangential at best and probably completely unrelated in reality.
Even in their demo, you don't really see any particularly deep comments that touch at the root of the issue, rather than just graze the surface.
So instead of a bunch of shallow toxic & benign comments you just end up with a bunch of shallow benign comments, with no tangible benefit going towards solving the issue.
We shouldn't particularly care about keeping as many people in the discussion as possible. Instead, we need to incentivize as many people as possible to write deep, thoughtful comments, and then, and only then, perhaps filter the top comments by toxicity.
Great. If the internet was mission one thing, that would be Google's AI moderating every single comment section on all independent websites. In Google's cyberspace, nobody can hear you scream.
4 comments
[ 5.1 ms ] story [ 23.8 ms ] threadThe site has a demo where you can slide the "toxicity" from left to right and view the resulting bucket of comments sourced from various topics of discussion (Brexit, Climate Change, US Elections). You can also provide your own snippet and get a rating.
Obviously this kind of tool will sustain a lot of discussion about what "toxicity" is and how a ML system could infringe on speech by applying bias, either from the engineers building and evaluating the system, or from the inherent bias in data that Google themselves have already identified as a major problem. [1] For what it's worth, Perspective's site offers a definition of "toxic": "a rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion." The training dataset was ostensibly built by having humans rate comments on a toxicity scale based on that definition.
It seems as though the idea is to penalize poorly thought-out or inflammatory comments while encouraging in-depth or at least more measured responses. I think we can all agree that that is an attractive system to build: the main question will be if it can be done in a fair way and what latent problems might exist in such a system.
[0] https://www.perspectiveapi.com/
[1] https://developers.googleblog.com/2018/04/text-embedding-mod...
> Unfortunately, most Americans are ignorant of the science about climate change. This is not because they are stupid but rather because they do not have all the data, or the data they have is unreliable.
I'm in no way saying this is the best way to communicate that message, and this could prompt a writer to re-phrase, but I don't know if I'd claim that this is toxic. I think that this could be a good tool, but it's clearly highly dependent on "common" understandings of words rather than actual definitions. Maybe that's the correct trade off though.
Edit: Forgot the score.
Even in their demo, you don't really see any particularly deep comments that touch at the root of the issue, rather than just graze the surface.
So instead of a bunch of shallow toxic & benign comments you just end up with a bunch of shallow benign comments, with no tangible benefit going towards solving the issue.
We shouldn't particularly care about keeping as many people in the discussion as possible. Instead, we need to incentivize as many people as possible to write deep, thoughtful comments, and then, and only then, perhaps filter the top comments by toxicity.