89 comments

[ 2.9 ms ] story [ 129 ms ] thread
> WhatsApp chat groups are being used to spread illegal child pornography, cloaked by the app’s end-to-end encryption. Without the necessary number of human moderators, the disturbing content is slipping by WhatsApp’s automated systems.

Wait what? Moderators for end-to-end encrypted communication?

Something doesn't add up!

I'm guessing since these are publicly accessible groups via apps as the article suggests, moderators could in theory search out and shut down such groups. Not sure how much data they are able to provide to the authorities though, I guess prosecuting the people is the difficult part.
Yeah. It's just bad writing in the opening paragraph.

WhatsApp doesn't provide search but groups can be joined via links. Third-party apps and sites use the links to index groups. Some groups share illegal content. WhatsApp should hire people to crawl those apps/sites, find the illegal groups, join, verify and ban.

Not entirely unreasonable. Won't fix anything* but if the company is big and well funded they should at least make some effort.

* so you ban the users? $10 gets you a new pre-paid sim card

Could it be that "tech companies" are reluctant to pursue cat and mouse solutions (which also cost money) while "non-tech public" seems content enough if attempts are made to push the issue deeper under the surface?

> so you ban the users? $10 gets you a new pre-paid sim card

And then when they recycle the unused number, and you get it and discover you are "lifetime banned", and I'm quite sure there's no appeal process.

I don’t get why you were downvoted. Life time banning new phone numbers is a real concern considering how often they are recycled. I mean WhatsApp isn’t doing that (as far as i know) but some services do that or even do a step further and ban your whole account if you add a banned phone number.

For instance, steams anti-cheating service will lifetime ban you and your phone number if you ever get chaught cheating in any game that uses steam anti cheating services. And apparently you will also get banned if you update your account to a previously banned number. That sucks for normal users who get a new phone number but doesn’t do much against determined cheaters who just purchase a new SIM card if they get caught (and they won’t use their real steam account as well).

Steam is not the only service that heavily relies on phone numbers not changing and I find it very troubling how little is reported about that (which might become a real concern later).

I believe the only reason WhatsApp doesn’t do that is because 90% of all people aged 16 to 69 use it at least a few times a week in many of their markets and a recycled phone number will very likely be given to a new, innocent user (unlike smaller sites such as steam where giving a cheaters number to another person who also uses steam is quite slim).

> WhatsApp should hire people to crawl those apps/sites, find the illegal groups, join, verify and ban.

This is a problem that needs a good solution, but you're literally suggesting that an arm of the one of the world's largest data collectors and propaganda engines, which offers free encrypted public communication to the masses, should fund and operate an internal spy agency whose job is to blend in and infiltrate groups and conversations in order to find illegal content. At what point is Facebook still not a mass-surveillance network?

(comment deleted)
While it doesn't seem entirely unreasonable, I can't help but feel it's the state we're at. A platform having to trawl the web to find links to its own service, infiltrate itself, verify the contents, then shut it down... I suspect a marginally easier way might be to maintain a list of referrer URLs/joining IPs (or start collecting them) and check them against a list of domains/IPs of sites known for sharing CP.

I'm also not sure to what extent the onus should be on WhatsApp to try and remove this stuff, in the same way that I'm not sure to what extent the onus is on Google to trawl through Gmail inboxes it finds through crawling child porn sharing sites - the latter idea seems ridiculous to me, but maybe it's just because of the scale of the platform or the issue, but with a lack of hard data made available as to the extent of WhatsApp being used for CP, I think it's better to remain skeptical.

But I will add this: if (and it's a very big "if") it is determined that it is right (in whatever sense we can use the word) to stop people using the platform in this way, I would never consider "but it costs money to do that" to be a reason against doing it in most cases. Justice doesn't stop because it's expensive.

Could they go the other way and analyze join events to groups or something on their own networks to find groups? I guess the issue then would be that they're monitoring every group?

I can't imagine these people are writing something like "child porn in this chat --> <whatsapplink>".

I haven't the faintest idea how people who're interested in that sort of thing're finding these links.

Definitely a problem which seems potentially intractable to me, although perhaps more familiarity with the platform and the offending chat groups would help (or... more likely... leave me rather disturbed).

From what I was told about tumbler it really is as obvious as saying that
Those would not be moderators. Those would be undercover investigators. DO NOT DO THIS. Any non-cop can get into real trouble. These sorts of investigations require people authorized to seek out, possess, and transport this material (all crimes in various jurisdictions). Even facebook will hesitate on sending employees out to hunt these groups.
(comment deleted)
A moderator could be a member of that chat group, therefore about to receive encrypted communications.

Obviously it wouldn't work with the way WhatsApp groups work now, but it is in a strange place half way between private and public.

Newsflash: e2e encryption is used to send content that stands to gain most from using it.
Careful there. You don't want to cause people to pull an Australia.
Australia has passed controversial laws designed to compel technology companies to grant police and security agencies access to encrypted messages. The government says the laws, a world first, are necessary to help combat terrorism and crime.

Source: https://www.bbc.com/news/world-australia-46463029

Passing a law is easy, but putting it in practice is enough thing. We of course won't hear about that side of it because "secret".

I have no doubts that the big companies which this seems mostly for will be doing everything to decentralize and remove themselves from the problem citing "systemic weakness" and then doing nothing for the government.

> Passing a law is easy, but putting it in practice is enough thing. We of course won't hear about that side of it because "secret".

ASIO has to provide statistics on how many notices they issued in their yearly reports. In addition, recipients of reports are allowed to provide statistics about his many reports they've received. See s317ZF.

> I have no doubts that the big companies which this seems mostly for will be doing everything to decentralize and remove themselves from the problem citing "systemic weakness" and then doing nothing for the government.

Except that Apple (and other such companies) have explicitly designed their e2e systems in a way that they could be backdoored (users don't need to sign new identities with a key, so Apple can add a new one -- which is how the device adding feature works).

In addition, many such companies have the ability to provide a single backdoored binary to a single user. This, according to the legislation, would not be a systemic weakness. There is work you could do to make yourself resilient to company-wide sabotage but no company I'm aware of is doing it.

Ah yes, here come the obligatory downvotes, lol.
Because it doesn't contribute to the discussion.
Sure it does, maybe not as directly as people would like but I don't see how it warrants downvotes. If you feel it doesn't contribute, don't upvote it.

I for one think there's some element of surprise that accompanies these articles that undermines the mechanisms of the technology being discussed and places blame on platforms that administer it.

Personally I think it does. The OP is highlighting an assumption that it appears to be the duty of the carrier to track and block content ("WhatsApp has ... problem"). Except that is exactly what carriers should not be doing, and what encryption protects against. Using carriers to track and block content is the definition of totalitarianism. I think child pornography is a hideous blight but stamping it out by shifting responsibility to the carrier is not the way forward.
A victim complex will get you even more downvotes.
As someone who strongly believes that people should have the ability to communicate truly privately, I upvoted you because you're correct and people need to be honest about it.

The problem is that "content that stands to gain most from using [encryption]" also includes activists protesting authoritarian governments, or even government officials themselves who don't want to be spied on by their rivals, or victims of stalkers and domestic violence.

With WhatsApp, end-to-end encryption is used by everybody, even though most people probably do not realize it.

EDIT: Okay, that came out more snarky than I wanted it to. On the one hand, you are right, of course: Provide a secure, encrypted, easy-to-use means of communication, and people with something to hide will use it. If I wanted to download or spread child pornography, I would not do it via an unencrypted channel, either.

On the other hand, your comment might be read by some to mean that you think encryption is the problem. And I think, by the time somebody posts a video of a child getting raped to a WhatsApp group, the problem has been around for while and is not going to magically disappear if WhatsApp were to remove end-to-end encryption from their service.

And child pornography, understandably, provokes strong emotions, which is why politicians and law enforcement officials eager to push more surveillance or censorship like to use it as a reason to push their agenda.

In fact, I would say that everyone has something to hide and that is perfectly normal and acceptable.
This problem is only going to grow in the future. You can't control or reliably censor information on a free and open internet. The only way to ensure that nothing "bad" happens on the internet is to completely lock it down and whitelist everything that is posted. This isn't going to happen.

In a few years blockchain based messaging apps will be launching and they will not be controlled by Facebook or anyone else. You won't be able to ban anyone. This is something we are going to have to accept and deal with. There will be things you don't like on the internet.

You can, actually, by not having a free and open internet. Which is what we will eventually have. And it won't be because of some super shady and evil council of old men in suits, but because the public will beg for it.
That doesn't make sense any more than thinking they can somehow ban encryption when their economy is dependent upon it. Even China with its efforts so overkill that it does the opposite of its job by making it incredibly obvious something is going on when words start getting randomly censored like censoring 'Shanghai stock market' after it ended on a number that was a very coded reference to June 4th, 1989 aka Tienanmen square.
blockchain based messaging apps

Out of genuine curiosity, where do blockchains enter into the messaging space? Auditable chat histories and identities? As opposed to other cryptography/security techniques, blockchain doesn't seem like an especially good match for the problem space.

Messengers need someone to host and maintain the servers. That requires a monetization model and requires someone to act as a central figure. That figure (like Facebook) then has to either moderate the content or try to make a case of not being able to but still facilitating it.

A messenger using the blockchain as its medium requires no staff to operate it and has nobody responsible for it. There is nobody to blame for whatever it's used for and nobody to dictate what it can and can't be used for. The group chats will simply exist and require no maintenance which is very desirable from an app point of view. Also a big positive is that you don't have to worry about the company running your messenger going out of business and then you lose the ability to use your app.

There's already p2p messaging apps that do that without any blockchain nonsense. The blockchain would leave a permanent trace to which pseudonym posted what forever.
P2P messaging apps have a hard time with group messages because you have to have someone online to relay messages at all times. Group chats really have to have a shared storage medium to work.
There are plenty of non-blockchain approaches to persistence as well which also fall under either the "distributed" or "p2p" labels.

The part I'm having a hard time with is that blockchain's specific strength relative to other distributed approaches is the ledger. That's not really the crux of the decentralized messaging problem.

Unless you need the immutable ledger, other approaches solve identity, privacy, and persistence aspects of messaging just as well or better without the overhead that blockchain approaches introduce.

What would the storage requirements be for a blockchain like that? It seems to me extravagant... and also would there be all the concensus algorithms? So I'd send a message and wait X amount of time for the network to achieve concensus that I had posted the message?

I can see the value in the distributed part certainly but if the storage requirements are very large due to a large volume of messages (I'd imagine any solution which caught on could quickly outpace the number of transactions on something like the ethereum or btc network), wouldn't it necessarily just end up centralized?

It's an interesting idea though.

knock knock "government telling you you're in trouble and need to take down your content.. wait whose building is this?"
But, if it's encrypted, isn't it impossible to quantify?
The join group links are public. So not really encrypted, since the key is public.
So why does encryption have anything to do with this?
Because it prevents automated mass monitoring and filtering of content in general. Someone has to know there is this group, be allowed to join it, monitor the content, etc. It's much more costly.
What if someone joins every group

What if that someone doesn’t show up in the list of group members because WhatsApp has this built in

What if you don’t control the source code so you can’t verify this

What if RMS was right

What then

Well, that's exactly the point I've tried to make when people say law enforcement can't happen in WhatsApp because e2e supposedly prevents that: they can disable it at any time for certain targets and it'd be hard to detect. But I'm usually downvoted when I say that because, apparently, people trust this faceless corporation a lot.
So then the police should do their jobs?
I actually agree, I don't think this should be WhatsApp's job. I really don't like this trend of making private companies do the job of Police, Judge, and executioner.
> In July 2018, the NGOs became aware... The NGO reported their findings to Israeli police but declined to provide Facebook with their research. WhatsApp only received their report and the screenshot of active child pornography groups today from TechCrunch.

Huh? Who are these NGOs looking out for? Why wouldn't you tell FB in the first instance?

Because once you become aware of a crime you let law enforcement handle it? Neither these NGOs nor Facebook are Batman.
The internet has an unencrypted child porn problem as well.

The medium used to communicate is irrelevant to the discussion here. If they can't control the unencrypted platforms, why are they pulling focus to encrypted platforms if not to influence public opinions regarding encrypted mediums?

Or rather, humanity has a lot of problems. Including rape, pedophilia, murders, etc.

I don't think WhatsApp in particular has a problem.

Are houses a problem because they hide crimes with their walls?

Why aren't builders taking more initiative to employ moderators to monitor crimes inside the houses they build?
That's not a perfect analogy -- these aren't people meeting one-by-one in private spaces, WhatsApp is providing them a shared space where they can meet and share images.

This seems more like blaming a hotel for letting one of their conference rooms be used for an illegal porn swap meet.

You can have a shared space in your private space very easily, so it does seem to me like the analogy holds up.
>providing them a shared space where they can meet and share images

Just like houses. Except you can share more than images on those.

Would you expect your phone service provider to monitor its lines for illegal behavior?

If not, why is that different?

They’re already sending the data to the NSA. The least they could do would be to block these unending spam calls.
This is a popular view, and it's one I may go to sometimes, but I wonder if it's only popular because it's what we want to believe. Really, it's reasonable to think that lowering the cost or risk of an activity will lead to more of it.
Well the thing is that many of the cost-lowerings are so unpredictable, all over the map and inevitable that assigning morality to it seems as futile and nonsensical as trying to decide if squares or circles are more moral.

People always say in retrospect about new technology 'they shouldn't have done this because I don't like the outcome' but how could they have known ahead of time and what would not doing it get them? There are very few cases when they really should have known better - leaded gasoline for instance.

Not the same. Houses aren't a mass medium of communication.
They're worse than that. Houses are a mass medium of child abuse. It's a lot easier to rape children indoors than to find a a suitably secluded spot outside any house.
I don't think you have grasped what a mass medium is - you don't get millions of people joining each house. You get lots of houses with a few people in them.
Those are called towns. Like, you know, a collection of houses.
The metaphor is pretty tortured at this point, but almost all towns have police forces.
Which are not expected to go inside every house because there could be child abusing ongoing there.
Which, having reasonable suspicion, may enter any home with a warrant etc.
In other words, "Guns don't kill people, people kill people"?
No, guns were definitely made to kill people.
Why bother trying to prevent crime is what you’re saying?
More like "Who's crime is it anyway?"
It's not WhatsApp's responsibility to prevent these crimes, and the only way for them to do so would be to violate the privacy of all their users.

That's a very bad trade-off, which would no doubt also result in other types of crimes being committed. It would also be completely pointless since these people could just switch to the next app, or setup their own app, or encrypt the files before sharing them on any random file sharing service.

It is almost like letting people act with impunity by putting the state at an informational disadvantage in a hyper-connected and hyper-mediated age is going to result in pretty unpleasant stuff becoming more common and more accessible.

Goodness, who knew?

Doesn't the Dark Web have the same issues?
Yes but it isn't a single company.
If the news is to be believed, people on the Dark Web screw their privacy up so frequently that law enforcement probably wants them to be there so it's all contained and easier to investigate.
(comment deleted)
Other than closing the Internet as someone suggests, wouldn't be possible to integrate the service that Microsoft offers to detect this material? Other apps like telegram, not e2e, could be integrated more easily, but personally I wouldn't mind such system being native on Android or IOS - allowing to make life much more difficult for pedos.
This kind of shit is like the black mold of communication systems.
Is it possible to use metadata to understand these networks without actually decrypting anything?

If you can track down a bunch of these groups through public / advertised urls you can probably figure out a whole bunch of them based on the participants in each group. I bet it forms a beautiful graph.

There might be some other user patterns you can identify which point to suspicious groups / patterns as well

How else could you solve this? Create hashes of known illegal images and check them locally?

Does the postal service have a problem if people uses their service to share illegal pictures? I think they are just a victim of abuse of their service.
Yes, it cares very much, the postal service has its own police department even.
I take issue with this idea that pedophilia has been "growth-hacked" (stupid term BTW).

Like what? Does no one who looked at this article (before publishing) have any idea how the internet works? Do they have this wonderful illusion that just because something is made slightly easier to access that it suddenly boosts how many pedophiles in the world? Little Johnny doesn't just browse a WhatsApp directory listing and say, "You know what? I've never really wanted to see children abused but now that there's a nice 'Adult' section containing this sort of content... I think I'm a pedophile now!" Anyone viewing that crap voluntarily was a pedophile to begin with, and like every single one of them, public castration should be an acceptable form of punishment (legalize it).

CP (a known abbreviation) has been circulated on the world wide web since the first big wave wherein more than a few thousand people were using it (back before it was referred to as the 'internet'). Hell i remember accidentally running across several images in my early teen years (when I was moderating for a forum), surprise surprise, it didn't "growth-hack" that vile garbage into my life, you know what it did? It ruined my day that's what it dig, and I absolutely hated the fact that I even saw that, it made me want to never see it again! Flagged for deletion, offender's account was terminated, and a report was filed (with the authorities).

It was also around this time that me and some of my friends decided to have some fun on IRC, I was bored enough that I would often troll IRC for hours, found several questionable channels, where RP (a known abbreviation) was huge and a few of the members were looking for someone to RP as... well children. Yikes! So... we didn't RP, what we did was act like real 12 yr. olds just to mess with them, and after a while of conversing, we'd sometimes get these creeps to get on a greyhound and make a trip (several times across the country) to send them to non-existent meetings, often humorously sending them to a place right across from a police station or whatever we thought was funniest.

Innocent fun I'd say, but anyway, more to the point of the article. No this does not "growth-hack" pedophilia, it just makes it all the more apparent that people are not inherently good and it's stupid to think you can stop the rot. Though I do support the push to limit this as much as possible, though I'd like to see arrests, I'd like to see these bastards rot in prison till they drop dead, and any solution that just covers up the issue without actual consequences for the offenders does not get my consideration. Children are our future, they must be protected.

No. The world has a child porn problem. And until you start offering mental health care to paedophiles and going after child porn producers, nothing will change. You can ban the entire internet if you like and paedos will just meet in bars and exchange usb sticks. This is ridiculous, of course every single secure information exchange platform will be used for child porn. And drugs. And teens sending nude pictures to each other. And terrorists. Etc.
How is that WhatsApp's problem?

When data ownership and privacy is concerned, these fakers claim these should be inviolable.

When there's something they don't like, you have no rights to privacy and private enterprises should perform the police work for the State and ignore property rights.

SJWs...

You can't have it both ways. End to end encryption or open groups with censorship by moderation, pick one. This is a side effect of private clubs and private online communities. While I don't condone the behavior, we need to get more creative on how to police this than putting in place mechanisms for violating everyone's privacy, which is just lazy and dangerous.

It's hard to read some of these stories without ignoring the current surge of schadenfreude that writers are capitalizing on.

Aaaand the privacy advocate community collectively goes "Here we go again" all at once.
It seems impossible to determine the amount of secular empirical harm being done by child porn consumption (for example, if all of the child porn being produced in a group was fabricated, who is harmed and how?), but we can at least imagine how much harm is done by a breach of privacy by corrupt governments.

Also, this (Frankly dumb) article perhaps unknowingly just marketed the existence of these groups to new fans (I note that the link to the cited Android app for whatsapp-group-discovery now 404's) and THAT will surely be harmful.

I'd comment on the article itself but Techcrunch's website is a dogpile of shit with a terrible onboarding workflow and site design.

Some people have taken to calling this 'the pedo problem' or 'pedophiles ruin everything' since it's such a recurring theme around encryption and anonymisation.
Should be pretty easy to trace the users if it’s an open group. The encryption only keeps things hidden from people not in it and anyone can join.