I'm curious what they expect Facebook to do in this situation. I get that these moderators have to look at disturbing imagery, but that's the job they signed up for. It's not like Facebook can only show them nice images, it's their job to look at those horrible images. I don't think Facebook was sugar coating the job description or anything.
This feels like a class action lawsuit brought by garbage workers because they have to handle trash all day long.
And I'm sure in the past those lawsuits and collective barganing got them provided PPE, equipment, training, showers etc that let them do their jobs better and more safely. As well as health coverage and other things for dealing with the after effects of accidents, sicknss, etc.
What about more mental health days? More breaks? Are they pressured to consume more than that can handle for metrics? What exactly is the mental health care plan that Facebook provides?(eg. If it’s only one psychologist meeting a month, when standard therapy is often weekly) Etc.
<not a mental health professional>
It seems like maybe one way of improving conditions would be to purposely dilute suspect content with "Known OK" content. So, you would only moderate at most 10-15% of flagged content and the rest would be family photos, cat videos, cool stunts, etc. Sure, this would mean that people in these roles would be doing some useless work, but, you'd also be reminding them that most content is good content and that most people are posting cute, funny, happy, cool, or interesting things.
I'd bet facebook could afford it.
Also, it seems like having a 3 strikes policy on accounts that post this stuff would be a good idea, too. Protect workers by ensuring that they're not moderating one angry nazi that gets every other post flagged. (maybe they already do this?)
There's a lot that could potentially be done, yeah. But we don't know exactly what FB has done. For example I'd be totally cool with a kitten room. FB could probably afford to foster a bunch of kittens and puppies, and allow the people dealing with horrific imagery to also take the time to hug a puppy (call it socialization) multiple times a day. FB could make the entire room painted some color that FB sponsored a study to determine was the most soothing. All sorts of things. Some of them over the top. Some of them intuitive. But the courts will decide if FB did enough. Looking forward for the courts to begin setting precedent for this sort of thing...
> FB could probably afford to foster a bunch of kittens and puppies, and allow the people dealing with horrific imagery to also take the time to hug a puppy (call it socialization) multiple times a day.
These are called emotional support animals, and they are totally a legit strategy for helping people cope with emotional trauma.
Knowing someone who used to work on their moderation/graph integrity team, whatever they're doing, it's not enough.
If the firehose of internet garbage can't be safely handled it's time to get rid of it. We've banned dangerous chemicals. Time to restrict Facebook, if it is a health hazard to its workers.
Why not hire more of them, and rotate people off the front line for a period of non-disturbing-imagery? It's not like there's a shortage of moderation work that needs doing.
Of course, that would require Facebook to spend money, and add headcount.
And we know that they only hired these people as a Public Relations exercise.
It might be more similar to signing up unsuspecting people to operate a jackhammer, when _you_ know, because of past experience, that the vibration will give them nerve damage. Then you hire more people…
There is a simple way to deal with shitty users. The problem is that it runs counter to the interests of the organization that hosts them. Require every user on the platform to use an invitation code from another user. They don't have to use their real names, but if they do something fraudulent or awful both the inviter and the invitee get banned.
This is why Dribbble is still nice and most other places turn to garbage.
HN is a rare exception. It's likely because the subject matter requires a certain degree of intelligence, which tends to negatively correlate with trashiness.
While I see where you are coming from, I think we can all unanimously and irrevocably agree that posting child porn and snuff videos to social media is pretty damn trashy.
I think you're sort of missing the point.
Posting gore/cp on, say, your public feed on a public internet social media site is trashy by the common idea of civil/moral behaviour, but it would be par for the course on liveleak social/ a tor hidden services pedophile social network.
It can be taken for granted that most people think CP is bad, and that sending pictures of dismembered kids to thousands of strangers is rude. Anyone who doesn't agree* will hide anyway, or act anonymously. What's the point in a post like this?
> “bombarded” with “thousands of videos, images and livestreamed broadcasts of child sexual abuse, rape, torture, bestiality, beheadings, suicide and murder,”
Those aren't "shitty users" those are vile and criminal people doing extremely horrible things and sharing them. Most or many of those people should be, where possible, referred to local authorities for judicial processing. FB can very well ID these offenders. It's not like they don't collect enough information to know who they are.
> use an invitation code from another user. They don't have to use their real names, but if they do something fraudulent or awful both the inviter and the invitee get banned.
Interesting. Like a regular meatspace party. Gmail used it and I got the feeling of them using it as social network tracking, so I wrote it off. But this is a great application. Will have to check out Dribble if anyone will invite me on ;).
That doesn't solve anything. How far up the invite tree do you go? If you go all the way to the top, you'd have to ban yourself, the founding user. If you don't go all the way, then unsavoury users will just construct a long invite chain to insulate themselves from whoever they invite. There's probably some more clever graph theory stuff you can do to make this process a bit more reliable, but good luck justifying your algorithm to the people you just banned.
Dribbble is only nice because nobody knows about it. Once the spammers show up with their bots they can turn it into a cesspit too.
It's hard to combat someone willing to set up thousands of accounts to get their message out, especially since some portion of those will be "good" accounts designed to make the system look bad if they get banned.
Facebook could ensure these people have access to mental health care and counseling. In the article Facebook claims that this is already the case, but I happen to know a Facebook content moderation contractor who tells me they do not have access to care, despite having to look at extremely distressing content all day, every day. I guess now it’s for the court to decide.
The moderators could perhaps not be looking directly at the video but a blurred version of it, along with a tool to reveal some part of it. That way they can infer what is going on without looking at the raw material.
Another thing they could possibly do is to include fictional content in the bulk, so that the moderators could at some level believe no real person went through that.
It is unrealistic to expect any normal person to have a grasp on what adverse effect will have on their mental health over time. You can walk away from being a garbage worker, but you can't unsee disturbing imagery.
Besides, the people hired to produce fictional child porn would just end up suing facebook for the mental damage incurred to them, same as the current case.
If someone starts throwing radioactive waste in the garbage I'd sure has hell expect the garbage company to step up and help/protect workers. Don't see how it's any different here.
Looks like it’s heading in the direction of verified identities being linked to social media accounts. That’s about the only action that would reduce the amount of transgressive actions performed on FB et al.
> I'm curious what they expect Facebook to do in this situation. [...]
> This feels like a class action lawsuit brought by garbage workers because they have to handle trash all day long.
Yes. Why are people creating jobs in the first place that obviously suck? It would be far better if they worked 10% on these disturbing images and 90% on something different, like software QA for instance.
I don't want to deep dive into political theory but the mere existence of people whose only job is to dispose garbage or clean toilets makes very little sense, is highly unfair and hardly works. It would be much better if people "cleaned their own sh."
Basic economics is why. Would you want to pay $125/hour for a janitor? Because that is what you get have brain surgeons cleaning toliets. While the jobs suck there are more people who can do them. It works perfectly well even if the job is less than desirable. It does tend to pay a bit more than non-sucky jobs of the same capabilities at least due to less people wanting to be sewage divers.
I'm not saying that I would pay $125/hour for a janitor. I'm saying that people who bring in dirt should every once in a while clean it themselves. Like in a flat share but scaled up. Decentralised somehow.
> Because that is what you get have brain surgeons cleaning toliets.
I guess I'm fine with Brain surgeons getting tumor's out of people's brains. Their job is quite exceptional.
> It works perfectly well even if the job is less than desirable.
It does not. People who work these jobs and work in overly dense areas like SV or NY cannot afford to live there - except if they live at their parent's...with 50.
The point is that the tasks are commodities and by pushing it onto others you effectively waste the specialized labor. Split it around all you like it is still waste. Sentiment doesn't change the math.
As for things working I believe we are speaking of works in two entirely different ways. Having someone as a dedicated cleaner is perfectly functional even if it is a crappy way to make a living. Call it SBW - sucks but works. As long as there are people willing to work the job it is SBW at worst. If they have people willing to head in then it works.
SV and NYC are both not the densest by a wide margin compared The issue there is a lack of willingness to upgrade transit and housing density sufficient. SV especially.
If workers win: FB puts freeze on hiring moderation staff. PR benefits begin outweighing the labor costs, and there’s not enough moderation capacity to block the vile content from getting through.
If FB wins: Workers develop mental conditions, moderation staff quits from the job stress, and there’s lots of job churn and not enough moderators to fight the problem.
It’s one of these classic situations that reveal the nature of a corporation as being distinct from an individual. An individual might follow the thinking you’ve outlined, but’s corporation comprised of endless layers of competing management, personal fiefdoms, and a collective case of shortsighted, goal-oriented thinking will not. It doesn’t matter if this is terrible for FB as a whole entity, or moderators. The mods aren’t making the call, and the people who do are possibly more concerned with showing they can minimize a source of overhead that doesn’t immediately add to their bottom line.
From the article: “We take the support of our content moderators incredibly seriously, ... ensuring that every person reviewing Facebook content is offered psychological support and wellness resources,” said Bertie Thomson, director of corporate communications.
After reading the comments and thoughts about the job. This is just new territory. I am not sure that any company can prepare an employee for this kind of mental barrage. ...or counter that constant consumption of evil.
I'm pretty sure a traumatized content moderator was extracting revenge on the people misbehaving on social media on a CSI Cyber episode, was it on Black Mirror too?
> Facebook not protecting content moderators from mental trauma
> Facebook in the past has said all of its content reviewers have access to mental health resources, including trained professionals onsite for both individual and group counseling, and they receive full health care benefits.
But that`s the job. It`s not like they were doing something else and this is a side effect.
If i employ you to clean human feces, you can sure me for not providing adequate protection etc but you cant sure me because you had to deal with human feces. That is the job.
You've hit exactly what this lawsuit is about. They aren't saying "when I got a job as a content moderator I wasn't expecting to see traumatizing things", they're saying "my employer isn't providing adequate protections from the things I have to see as part of my job".
I wonder if maybe tighter integration with law enforcement would help the situation? Part of the problem may be that people feel so helpless when all they can do is ban the image and maybe send a copy up into the black hole of law enforcement.
It has to be incredibly demoralizing to see yet another child sex slave image posted from that Kremlin IP range taken in what is clearly the same room they've been seeing for months. You can forward all of the pictures to the FBI, but they can't do anything about it and you're seeing victim after victim paraded around the Internet. Facebook's jurisdiction stops at the uplink from their datacenter.
Right, but there's still a very real lawsuit in play, yes? If I were the defendant of that lawsuit, I'd be looking to burn someone to the ground and salt the earth for suing on behalf of an AI.
> How is it any different from ER doctor, coroner, social worker, or investigator?
Yes. All those people should have support too. And many more.
What is it with people and saying we need to race to the bottom? Like when people react to other people's high pay and benefits not by saying, "I should be up there with you", but instead, "why aren't you down here with me?"
The implication being, I guess, we need to do more to spread the suffering and not lift people.
What is it with people saying we need to sugar coat, bubble wrap, and insulate our tender egoes from the reality of the world? Psychologists have a term for that. It's called denial.
If you happen to have a psychologist who is trivializing trauma or PTSD type symptoms in these terms I'd say I think you need to find a different psychologist.
You'd figure that four years after this article came out the larger social media companies would have solved this issue.
A section that stood out to me:
Eight years after the fact, Jake Swearingen can still recall the video that made him quit. He was 24 years old and between jobs in the Bay Area when he got a gig as a moderator for a then-new startup called VideoEgg. Three days in, a video of an apparent beheading came across his queue.
"Oh fuck! I've got a beheading!" he blurted out. A slightly older colleague in a black hoodie casually turned around in his chair. "Oh," he said, "which one?" At that moment Swearingen decided he did not want to become a connoisseur of beheading videos. "I didn't want to look back and say I became so blasé to watching people have these really horrible things happen to them that I'm ironic or jokey about it," says Swearingen, now the social media editor at Atlantic Media.
72 comments
[ 4.1 ms ] story [ 160 ms ] threadThis feels like a class action lawsuit brought by garbage workers because they have to handle trash all day long.
I'd bet facebook could afford it.
Also, it seems like having a 3 strikes policy on accounts that post this stuff would be a good idea, too. Protect workers by ensuring that they're not moderating one angry nazi that gets every other post flagged. (maybe they already do this?)
</>
These are called emotional support animals, and they are totally a legit strategy for helping people cope with emotional trauma.
Knowing someone who used to work on their moderation/graph integrity team, whatever they're doing, it's not enough.
If the firehose of internet garbage can't be safely handled it's time to get rid of it. We've banned dangerous chemicals. Time to restrict Facebook, if it is a health hazard to its workers.
https://selenascola.com
https://www.instagram.com/selena_scola/?hl=en
https://selenascola.com
It is an AI
Of course, that would require Facebook to spend money, and add headcount.
And we know that they only hired these people as a Public Relations exercise.
This is why Dribbble is still nice and most other places turn to garbage.
HN is a rare exception. It's likely because the subject matter requires a certain degree of intelligence, which tends to negatively correlate with trashiness.
I think you're confusing "more homogeneous" with "less trashy".
Trashy is just whatever the local minority does that the local majority finds repulsive.
A lot of the opinions conveyed in the commentary on HN that are reasonably well accepted would go over like a lead balloon in some other communities.
Those aren't "shitty users" those are vile and criminal people doing extremely horrible things and sharing them. Most or many of those people should be, where possible, referred to local authorities for judicial processing. FB can very well ID these offenders. It's not like they don't collect enough information to know who they are.
Interesting. Like a regular meatspace party. Gmail used it and I got the feeling of them using it as social network tracking, so I wrote it off. But this is a great application. Will have to check out Dribble if anyone will invite me on ;).
It's hard to combat someone willing to set up thousands of accounts to get their message out, especially since some portion of those will be "good" accounts designed to make the system look bad if they get banned.
It's one thing to look at naked selfies or bad taste political jokes.
It's another thing altogether to look at rape, pedophilia and dismemberment. Hell, sometimes it's mix of those three in the same photo or video.
The responsibility falls on the employer to provide adequate support.
Another thing they could possibly do is to include fictional content in the bulk, so that the moderators could at some level believe no real person went through that.
It is unrealistic to expect any normal person to have a grasp on what adverse effect will have on their mental health over time. You can walk away from being a garbage worker, but you can't unsee disturbing imagery.
The moderators are sadly enough going to become experts on this stuff in a hurry, it's not likely that you'll be able to fool them anyway.
You don’t get to disregard putting your employees in physical or psychological danger just because you pay them money.
> This feels like a class action lawsuit brought by garbage workers because they have to handle trash all day long.
Yes. Why are people creating jobs in the first place that obviously suck? It would be far better if they worked 10% on these disturbing images and 90% on something different, like software QA for instance.
I don't want to deep dive into political theory but the mere existence of people whose only job is to dispose garbage or clean toilets makes very little sense, is highly unfair and hardly works. It would be much better if people "cleaned their own sh."
I'm not saying that I would pay $125/hour for a janitor. I'm saying that people who bring in dirt should every once in a while clean it themselves. Like in a flat share but scaled up. Decentralised somehow.
> Because that is what you get have brain surgeons cleaning toliets.
I guess I'm fine with Brain surgeons getting tumor's out of people's brains. Their job is quite exceptional.
> It works perfectly well even if the job is less than desirable.
It does not. People who work these jobs and work in overly dense areas like SV or NY cannot afford to live there - except if they live at their parent's...with 50.
This needs to be thought through.
As for things working I believe we are speaking of works in two entirely different ways. Having someone as a dedicated cleaner is perfectly functional even if it is a crappy way to make a living. Call it SBW - sucks but works. As long as there are people willing to work the job it is SBW at worst. If they have people willing to head in then it works.
SV and NYC are both not the densest by a wide margin compared The issue there is a lack of willingness to upgrade transit and housing density sufficient. SV especially.
If workers win: FB puts freeze on hiring moderation staff. PR benefits begin outweighing the labor costs, and there’s not enough moderation capacity to block the vile content from getting through.
If FB wins: Workers develop mental conditions, moderation staff quits from the job stress, and there’s lots of job churn and not enough moderators to fight the problem.
Counselling comes to mind .. in the form of a regular debrief or supervision session
It looks like that's exactly what they do.
e.g. Psychological support could be via a webchat platform.
> Facebook in the past has said all of its content reviewers have access to mental health resources, including trained professionals onsite for both individual and group counseling, and they receive full health care benefits.
...which is it?
If i employ you to clean human feces, you can sure me for not providing adequate protection etc but you cant sure me because you had to deal with human feces. That is the job.
It has to be incredibly demoralizing to see yet another child sex slave image posted from that Kremlin IP range taken in what is clearly the same room they've been seeing for months. You can forward all of the pictures to the FBI, but they can't do anything about it and you're seeing victim after victim paraded around the Internet. Facebook's jurisdiction stops at the uplink from their datacenter.
https://www.theguardian.com/technology/2017/jan/11/microsoft...
The ones who are regular users who choose to moderate a community? No. There's no business relationship.
https://selenascola.com
https://www.instagram.com/selena_scola/?hl=en
Yes. All those people should have support too. And many more.
What is it with people and saying we need to race to the bottom? Like when people react to other people's high pay and benefits not by saying, "I should be up there with you", but instead, "why aren't you down here with me?"
The implication being, I guess, we need to do more to spread the suffering and not lift people.
You'd figure that four years after this article came out the larger social media companies would have solved this issue.
A section that stood out to me:
Eight years after the fact, Jake Swearingen can still recall the video that made him quit. He was 24 years old and between jobs in the Bay Area when he got a gig as a moderator for a then-new startup called VideoEgg. Three days in, a video of an apparent beheading came across his queue.
"Oh fuck! I've got a beheading!" he blurted out. A slightly older colleague in a black hoodie casually turned around in his chair. "Oh," he said, "which one?" At that moment Swearingen decided he did not want to become a connoisseur of beheading videos. "I didn't want to look back and say I became so blasé to watching people have these really horrible things happen to them that I'm ironic or jokey about it," says Swearingen, now the social media editor at Atlantic Media.
the employee suing is "Selena scola"
Coincidence...or just clever MSM ad strategy???