HN Explorer

Tell HN: Dis the Sopranos? Machine Learning Put Me on Facebook Suicide Watch

44 points by tbyehl ↗ HN
Three years ago I posted a 'Change my mind' meme on Facebook dissing The Sopranos. Today FB says my post violated community standards and directs me to resources about self-harm and suicide.

Officially I can't advertise or go live for 30 days (don't do either anyways). Unofficially, my posts are no longer showing up in my own feed nor anyone else's. Were I actually at risk of self-harm, I struggle to imagine how socially isolating me is a proper response.

Warning and original post: https://i.imgur.com/2kzCEza.png

Restrictions: https://i.imgur.com/VHCWfbt.png

25 comments

[ 2.6 ms ] story [ 57.2 ms ] thread
h2odragon
At least they're not (yet) legally obligated to report you to "authorities" of some kind for readjustment and help. Right? This still falls under the usual customer service standards of "their site, their rules, shut up you're just the chum"
ipaddr
How close are we to that? One press release away.
[deleted]
(comment deleted)
jaclaz
Assuming that your post is actually a valid indicator of a risk of self-harm and suicide, three years can hardly be described as "in a timely manner", even if it was appropriate to point you to those resources, it makes no sense to do so three years later.

BTW I cannot see how the posted meme can be connected to risk of self-harm and suicide, am I missing some reference?

the_only_law
> am I missing some reference?

Some of these moderation algorithms are pretty stupid/over zealous.

I was watching a follow-up on the YouTube elsagate crisis that happened some years back, and apparently one of their late solutions to the problems ended up deleting anything with tags or titles that could be remotely connected to children's animation, resulting in a ton of being videos being struck down.

Also, when the "fact checking" was released on FB and Instagram, obvious memes and obvious bait (literally explicitly stating it was bait) were being "fact checked".

jaclaz
Yes, what I don't understand is the possible (even if wrong) connection that the algorithms may have established between the specific meme and suicide.

Is the "change my mind" or the "Sopranos" that may trigger the algorithm?

Or are all memes associated to self-harm/suicide?

the_only_law
> I struggle to imagine how socially isolating me is a proper response

Wild guess, but I'd guess it's more of a protection from some sort of liability for them, as opposed to protecting you.

djur
I believe this restriction is meant more for people who are advocating self-harm ("kill yourself" memes and the like), which explains the isolation. I don't know how Facebook handles it, but on Twitter I've received different algorithmic responses for "we think you referred to hurting yourself" vs "we think you told someone to hurt themselves". (Both of these were misinterpretations by the algorithm.) In the former case, my account did not get locked down, I just got offered resources.

There was obviously a pretty egregious mistake in your case. Hopefully you can get a quick reversal.

desindol
Someone reported your post for self-harm/suicidal tendencies. It's a pest on reddit and is spreading to other social networks as a form of retaliation or disagreement and most of them handle it like dmca requests... better safe than sorry.
Shorel
This is the correct answer. It is a group of people reporting the post, not ML, the cause of this problem.
tbyehl
Doubtful. My FB posts are restricted to friends-only, my FB friends are actual people I have more than a cursory meatspace connection with, and the post is over 3 years old. It's improbable that, if one of them wanted to troll me, they would dig that deep — and, tbh, they wouldn't need to.
Nuzzerino
You’d be surprised what some people are capable of doing.
Zircom
Yeah I just got hit by this on one of my reddit accounts. Haven't even posted anything in a few weeks and randomly got this message[1]. Luckily it didn't restrict my account in any way unlike OP and Facebook.

1. https://imgur.com/a/mui2REE

jart
> Were I actually at risk of self-harm, I struggle to imagine how socially isolating me is a proper response.

I'm going to put on my cynical hat here, but I think you have a perfectly reasonable expectation. Human communities evolved in such a way that everyone knew everyone. If one person got sick, then the community was just as concerned for the individual as they were for risk it posed to the community.

Now, I'm not sure how it came to pass that giant tech companies like Facebook got landed with the responsibility of community shepherding and management (rather than being telecommunications carriers) but it's not a role Facebook should be doing, because at that scale, the people who are unhealthy are going to be treated like a defective unit, which needs to be pulled out of service and replaced, so the machine as a whole can keep humming.

Simple fact is that Facebook is a website, and a website can't care about you or anyone else for that matter, because it's just a website. But now that we're all socially subordinated to a website that controls the means by which we communicate with friends, family, and loved ones, I think we're going to see a lot more of this sort of lack of compassion becoming endemic to humanity.

Schroedingersat
> I'm not sure how it came to pass that giant tech companies like Facebook got landed with the responsibility of community shepherding and management

They didn't get landed with it, they forcefully inserted themselves into the social graph and systematically dismantled alternatives.

rasz
Meme was funny.

FB reaction even funnier.

Now to top FB reaction you should hire a lawyer and sue them for emotional damage - FB bogus warning and cutting off from friends and family made you really think you are suicidal!

pridkett
Facebook, in general, has swung too far on the ML for moderation side. Last year when vaccinations for kids first 12-18 became available I posted “It’s time to stab…er jab the kids”. That got my account locked so hard I needed to contact a friend inside Facebook to get it fixed. To this day I still have a strike on my account for inciting violence or something stupid like that.

I’d honestly rather suffer a shadow ban than know just how bad their ML is for this use case.

tbyehl
Around that timeframe I got 30 days for hate speech for commenting on a meme about little boys being evil by saying that 'little girls are also pure concentrated evil.'
Schroedingersat
Do your best to keep contact with as many people as you can outside of garbage like this or it will get worse.

Badger them to create an xmpp or matrix or fediverse account somewhere. The people who don't have a friend inside facebook to help will just be silently isolated and cut off from community events and services that use facebook.

mgbmtl
Meanwhile, a friend was posting suicide notes on Facebook, and Facebook did nothing about it. She was already in a hospital, but for some reason she had access to her phone for a while.

She also posted a fake death announcement by her mom. Then attempted suicide, and was in ICU for a few days. People were engaging a lot, freaking out.

So yeah, Facebook fails miserably at everything.

ev1
Does Facebook really consider this a failure if it increased clicks and engagement? Something to think about.
exikyut
Hmm.

Ostensibly Facebook has a party line and attendant menagerie of moderation and AI tooling as a broad response to self-harm, but at the end of the day all of that has its box to work within, and thinking and breathing this one subject does not define Facebook's identity and core DNA. And that may be the core of the problem.

I wonder if the auto-assessment ML model passed right on by the described post *precisely because* the "clicks and engagement" ranked it as "good", where "good", intentionally or otherwise, has wound up becoming the TensorFlow distillation of "not our problem"... because *that* is where the focal point, the emergent foundation everything else finds itself orbiting around, at the end of the day.

From the angle I'm squinting at this from, this situation seems unsustainable intractable. FB is attempting to "benignly police" N billion people at once in an effort to monetize a private (non-government, publicly-traded), commercially-delivered digital town square experience. IMHO there is no global point of cohesion that can encapsulate this effort, which is trying to unify multiple fundamentally incompatible dimensions at once. It doesn't work.

ev1
A real death notice removed as targeted harassment would also probably result in some form of public relations nightmare. It's not like the ML model will know about the severe and difficult to reverse reputational damage of a fake harassment death notice.