This shows the downside of network-effects so succinctly. It's almost hopeless to think people will realistically move to a technically superior alternative without this privacy hole.
This will be true for a lot of people, and it's a huge annoyance. The only things that break this kind of network affect is a market changer - Discord is a good example here.
The nice thing about direct-chat apps is that the network effect isn't nearly as strong as for traditional social networks. If you want to use X to talk with someone, you can just download it and then talk with that person. You don't have to get everyone to switch at once. You'll have more little app icons on your phone but that's about the extent of the cost.
Probably 95+% of all my sent or received Whatsapp messages are for groups. If I don't get most of the people I communicate with to switch to another (the same!) messenger, it's pretty inconvenient.
It doesn't help that, last I checked, the good (open) alternatives were all much jankier or missing one or more major features we use. Makes them hard to sell. :-/
I mean, though it's a variation on the "if you don't have anything to hide..." apology, you really shouldn't be having security sensitive group chats anyway.
The network-effect doesn't have much to do with it. As has been demonstrated many times in the history of technology, the masses do not care about "technical superiority". When it comes to messaging apps specifically, even most software engineers I know don't care. If people don't care then there is no substrate on which to build a network.
It's true that WhatsApp is currently the leader, but I'm wondering how it will work out for them when they start adding ads, as it has been reported they will some time next year. When there are free alternatives that don't (and assuming e.g. Signal catches up in terms of features and reliability -- big assumption I admit), it's not hard to imagine that at least a few will install these alternatives, and use them in parallel.
This article cites a Forbes contributor (Kalev Leetaru) whose articles are endlessly self-cited with no outside sourcing. Can anyone confirm that any of this is actually real? I got 3 layers deep and could find nothing of substance.
Does it really matter? We already know that these companies are willing to collaborate with intelligence services. We already know that Facebook is hoovering up your data. We know Jan Koum already left Facebook over privacy issues. Maybe the article is click bait trash, maybe it isn't, but we should assume that whatsapp isn't secure.
I don't trust Facebook and thus I wouldn't trust WhatsApp. That has nothing to do with whether or not it matters if specific allegations against Facebook are true.
Unfortunately, it's likely we'll never get a rigorous proof of malfeasance. Thus do you go with WhatsApp as a homosexual in Russia or Iran, or as protestor in Hong Kong against the Chinese government, given that Facebook has given us plenty of reasons to distrust them?
I'm not saying to trust Facebook. I think it is wise not to trust them. Just because I don't trust Facebook doesn't mean that I should assume by default that every allegation against Facebook is true.
Yeah, that is a very good point. As consumers we need to demand more ethical reporting. Further, we should be demanding some mechanism to remove unethical reporting from our social news sites (like HN). This sort sort of moderation is ripe for abuse, but I don't think there is an alternative. Social media has proven that crowd consensus does not select for truthful or in-depth reporting.
It doesn't even matter whether they're willing, that they're capable is enough reason for prejudice.
Software which people's civil liberties depend upon should be developed in the open, auditable by the public, as a minimum. Even then, developers need to be vigilant about tainting by state actors.
UPDATE: Per @mic47 in a comment below, the video does not actually discuss content moderation on the E2E encrypted systems (Messenger & WhatsApp). It is only discussing the FB website itself, which is already in the open (to Facebook).
Original message follows for continuity:
Thank you for this!
The 4-layers-deep source is an F8 talk on using AI to filter for FB TOS-violating content. The relevant portion of the talk is discussing hyperlink website previews of the kind that show up next to URLs posted in Messenger/WhatsApp that show the title and feature image next to the link. In order for this functionality to work at all, the app must crawl the link to pull the metadata for display. This AI team is attempting to pre-emptively block URL-masking attempts that fool the Preview into showing benign content before redirecting users to bad content.
To do this, FB must "pierce the veil" of E2E encryption in order to see what the URL is and test the content. This opens the possibility that FB can further expose supposedly encrypted content in the chat/message. The actual talk refers to putting this AI on the device itself, preserving user privacy (a difficult technical challenge).
If this leaking of URL data is status quo on FB messaging systems, is it reasonable to assume that the supposed encryption is more or less invalid? Are there any ways that this could be a secure system with this kind of content validation going on?
The presentation actually talks about FB needed to solve challenges of training and evaluating the AI without getting the data to it's server, and I haven't noticed any point where whey would suggest that they need to break encryption, or get the unencrypted data to the servers.
If everything happens on the device and noting is sent to servers, how does it break encryption?
In case I am wrong, and they actually suggest that they will really send data back to its servers, where it is suggested? I really failed to find it.
Upon reviewing the video, I think you are correct. The video is discussing content filtering through AI, and whether it can be done on the device as opposed to the server, but the images accompanying the talk show a normal FB post on the website - not a Messenger window. It is not implied or suggested that they are screening content on their E2E encrypted platforms (Messenger and WhatsApp). What the speaker says:
...So far, we have been keeping this fight [against bad actors and harmful content] on familiar grounds. And that is, we have been training our AI models on the server and making inferences on the server when all the data are flooding into our data centers.
While this works for most scenarios, it is not the ideal setup for some unique integrity challenges. URL masking is one such problem which is very hard to do. We have the traditional way of server-side inference. What is URL masking? Let us imagine that a user sees a link on the app and decides to click on it. When they click on it, Facebook actually logs the URL to crawl it at a later date. But... the publisher can dynamically change the content of the webpage to make it look more legitimate [to Facebook]. But then our users click on the same link, they see something completely different - oftentimes it is disturbing; oftentimes it violates our policy standards. Of course, this creates a bad experience for our community that we would like to avoid. This and similar integrity problems are best solved with AI on the device.
Yeah but that URL thing is entirely FB's own choice and own problem. Definitely way more a thing of "violates our policy standards" than "oftentimes it is disturbing" because people going around tricking others to click on shock videos using advanced cloaking techniques ... just isn't that big of a problem.
Linking to things is a normal thing to do on the Internet, but Facebook wants to control that, because links often also mean the user moves their attention away from FB, onto the linked thing. This is what FB doesn't want. But it is a very useful thing for the user, to be able to smoothly navigate between apps and websites depending on their needs.
I do wonder why they chose to go with the danger of URL masking that links can be "disturbing" instead of "scams". Because I don't believe these techniques are (widely) being used to trick people into watching animal torture or other disturbing things. They'll be tricked to click on scams and phishing sites.
The difference between "disturbing" and "scam" is literally the difference between subjectively not-okay and objectively not-okay.
Facebook just wants the power to prohibit you from accessing what it deems "disturbing".
Also just from a technical perspective, there's some very weird assumptions: FB's server would see something different than the user's browser. But FB's local AI spy process would see the same thing? Are they seriously pretending that you shouldn't be able to securely log onto another website in a browser? Yes the user's browser sees something different, because it is logged in, and no FB's local AI spy process is not invited ...
At most they could warn that the site looks different than what the server side FB scanner saw. Then the user could decide if they feel more secure because this is expected as they are logged in and FB can't watch along, or if this is unexpected and they should be cautious.
> If everything happens on the device and noting is sent to servers, how does it break encryption?
if the process runs on the device, scans information on the device, and then triggers to notify FB when it matches something, that's breaking the encryption.
it's leaking of encrypted information (the fact that it's there, which rules triggered), but more importantly what's the use of encryption if you got a process that's basically acting as a mole, even if it's running locally?
unless I am wrong how this works, but at some point it'll trigger and notify FB or perhaps a government? if it doesn't do anything, then it might as well no be there.
and if this system gets off, there will be false positives, and there will be trigger rules matching much broader things than they strictly need, "just in case". we kind of know how these stories go by now and which bits we can safely give no benefit of the doubt because a state or a corporation wants them so strongly. I'm gonna guess it will be a black box "algorithm" without clear trigger rules, because it's AI/ML it's hard/impossible to explain exactly how it reaches its conclusions and therefore nobody knows really (and much NDA unto those who do). that's gonna be a hard sell though, even to the general public.
And open source allowing for independent verification. We have tons of good open source apps like Signal and Matrix, why would we ever need one controlled by facebook of all companies?
Because Matrix has terrible UX and its apps have terrible GUIs. And because Signal has this weird hangup about federation, requiring a closed server, being super paranoid secure on one side and expecting that amount of "trust" on the other. (I call it a weird hangup because I've only read bad explanations and evasive answers about why this is)
Give me Matrix with Signal's UX, and I'd be all over it :-)
Google is in the middle of rolling RCS out and that fact that E2E encryption isn't part of the basic spec is appalling. I don't know how a developer could work on something like this at Google and feel good about themselves.
Content moderation is also censorship. You typically use E2E to avoid censorship and surveillance. Facebook wants to mold thought and opinion, it is the ultimate control over society.
my (albeit faint) hope is that this leads some ambitious (EU?) organization to spin this as a US tech/corporate/power overreach angle, starts heavily promoting Signal as an alternative, various security conscious orgs/govts buy in, leading to big Signal popularity boost
Forget this one, EU politicians, especially our German Interior Minister Seehofer, hate proper e2e with a passion - they want to weaken the 5g standards so that they can continue IMSI catching and snooping.
Personally I've felt this has been in the works for some time.
If you look closely at how the founders exited facebook, and what was said at the time, it seemed (to me at least) that they left over exactly this.
I mean, that aside, can you really come up with many other operational arguments that would result in people walking a way from hundreds of millions of dollars? (or by some estimates leaving over a billion dollars on the table between the two of them)
Confirmation bias is bad when others have it, but not when HN readers have it.
You haven't bothered to read the source, haven't bothered to understand what the context is, and haven't even bothered to see that there is no mention of WhatsApp in the talk.
Ducking tired of click bait and people who ducking reward it.
You can in fact say fuck if you want to. There are lots of offensive things said in pleasant tones in polite society. Bad language barely moves the needle so give me an F...
Yeah, changing it to ducking actually shifts the reader's focus from "Wow this person feels very strongly about this issue" to trying to decide what the tone is meant to convey.
It took my brain a few tries to parse the word duck in this context.
I never understood government overreach like this. Any developer can clone an open source crypto implementation and spin up a new chat app. It is ridiculously stupid to backdoor things.
Unless the government plans to go back to the 80s, in which cryptography is heavily restricted from export, this doesn't accomplish anything.
I’ve always interpreted the various Crypto Wars as distractions.
It is a bizarre paradox where designing a secure system is a hard problem, where zero-days are commonly cheaper than brute forcing a 64-bit key, while cryptography is chronically lambasted.
I disagree, and history does not show this to be true. We have been more fickle when it comes to instant messengers than almost any other technology on the face of the planet (pun intended). From IRC, ICQ, AIM, Facebook messages on the website (not Messenger), WhatsApp, Telegram. I count 6 major shifts in the last 19 years. This would indicate that the over/under lifespan of a messenger is roughly 3.5 years.
All it takes is a usable interface, general availability, and a reason to change. If WhatsApp starts doing content moderation, of any form on the client, I am of the strong opinion that users will find this to be "creepy" and will immediately start shifting. As it is, all of my technical contacts have already left WhatsApp for Telegram. Since that group/audience has been the bellwether for all of the previous instant messaging client shifts over the last 2 decades, I imagine it's only time before "everyone else" also shifts off WhatsApp. I don't necessarily think it will be Telegram, but they definitely have the most momentum today when compared to, eg, Line, Zalo, Viber, Kik, et al.
> Unless the government plans to go back to the 80s, in which cryptography is heavily restricted from export, this doesn't accomplish anything.
That wouldn't accomplish anything now, anyway. All the current best crypto we have is out in the world, and plenty of crypto research happens outside the US. If the US revived the export restrictions, it wouldn't have much effect on the rest of the world.
Yes I remember that! It was developed in a joint venture between Nokia and Siemens. People got tortured because of that infrastructure. So, Nokia/Siemens said in a press release, "gosh we didn't know Egypt and Iran would do that if we would help them violate human rights now we are sorry" and fuck them both.
I just don't buy that Facebook will only limit itself to what it tells us publicly that it will do.
Facebook has recently talked about merging Messenger, WhatsApp, and Instagram chat into one while "also keeping end-to-end encryption."
Why would Facebook cuts its own access to all the data ming it was getting with the Messenger and Instagram chat? It wouldn't, unless it uses "faux" e2e encryption. Besides, Facebook currently allows advertisers to send ads Messenger users. How would those ads be managed by FB's servers, if the encryption is truly end-to-end between the sender and the receiver?
Also, this "new encryption system" is exactly what the founders of WhatsApp left:
My understanding of FB's plan to E2E encrypt everything is that it would be done to dodge responsibility for content moderation. If it can't see what's being discussed, it can't be held responsible for reigning in hate speech, misinformation, etc.
Who defines what is hate speech and what is misinformation though? We're in post truth where I don't even think the "good actors" are acting truthfully.
Blackberry has been calling their phones/messaging secure forever and that hasn't been true in a decade.
Fortunately the employees who work on security stuff tend to be anti-authoritarian so it will leak out of FB pretty fast unless it was some secret court order.
I didn't think Blackberry even made their own phones anymore. Aren't the current crop of Blackberry-brand Android phones just white-label from a Chinese manufacturer called TCL?
Unpopular opinions ahead. I like Telegram and Wire, with Telegram far ahead in features. Once I get people to use Telegram and describe how it's different and better than WhatsApp (except default E2E), they realize that it is indeed nicer, and want to reduce using WhatsApp.
Wire is fine (not as good as or fast as Telegram), but the company seems to be focused on paying customers to increase revenues. So free customers hardly get any support or responses from support (the auto-replies from Wire support state this priority). I like that Wire is by default E2E and syncs conversations across devices and platforms!
I've been waiting to recommend Signal to others for at least three or four years now. But it's 2019, and Signal is still bad in UX and messaging reliability (not to mention one still can't migrate conversations to another device on iOS).
Huge plus for regular users is that you can be online in the desktop app without having your phone near or on. When my phone broke, Telegram was one of the few ways to quickly inform my friends.
Much depends on your threat model. Telegram is plenty good for keeping your conversations out of the hands of Facebook and Google, and the user experience with synching messages/drafts between devices and editing/deleting of messages blows the doors off Signal, iMessage, and the like. And for all the whinging about Telegram's security model, there's yet to be a documented hack of the server.
Does anyone else think it's weird that it requires access to your camera roll to view images, access to your camera to use the web version, and access to your contacts to start group chats? (on android anyway)
Aside from the webapp QR-code thing, none of that is technically necessary.
JACK: “No. It’s unconstitutional. You can’t make your own currency. That is the federal government’s job.
PRICE: Jack. Look at me. I am not the problem here. The problem is that hard cash is fading. Rapidly. That’s just the way of the world right now. And Bitcoin is spreading — and if Bitcoin takes over we are all in a world of hell. It is unregulated. It has already reached its transaction volume maximum and it is partly controlled by Chinese miners.
With E-Coin we control the ledger and the mining servers — we are the authority. I will make sure that you will have visibility into every single wallet that’s opened: every loan, every transaction. Which means we can start making new assets. Which means we can start rebuilding the banking sector without you having to inject even more politically unpalatable federal funds into it.
JACK: The President will laugh in my face.
PRICE: But he will know that this is the right thing to do. This is going to be controlled by a good old-fashioned American company. You want to regulate it? Be my guest. Regulate the shit out of it. I’ll give you backdoors, side-doors, trace — whatever you want. Just don’t. Shut it. Down.
We have to TRUST WhatsApp, Telegram, Signal, et al because there are no good OPEN SOURCE alternatives. Otherwise we could just run our own.
We have to TRUST the current PKI and DNS because the alternatives are still immature.
But EVEN IF we develop robust and mature alternatives, and wide adoption, we have to TRUST the Device, OS and Browser makers.
The Trusted Computing Base is made by a handful or companies. Open source in hardware is still a nascent field.
It’s interesting if one can ever be truly sure that some part or chip hasn’t been interdicted. Apple tries to scan components as they arrive vs their hardware designs.
But hardware will always be the weak link. Keyloggers. Cameras watching your fingers from the ceiling.
The future of surveillance is in this kind of stuff. We will ALMOST have secure communications, but not really. The only thing you can be sure of is sending quantum entangled particles from airgapped rooms.
We haven't added a backdoor to WhatsApp. The Forbes contributor referred to a technical talk about client side AI in general to conclude that we might do client side scanning of content on WhatsApp for anti-abuse purposes.
To be crystal clear, we have not done this, have zero plans to do so, and if we ever did it would be quite obvious and detectable that we had done it. We understand the serious concerns this type of approach would raise which is why we are opposed to it.
Sadly it boils down to "trust us" (or really, trust wcathcart), which is a position users have been betrayed in countless times over the past decades (and Facebook has a horrible reputation for user privacy). Compare that with Signal—or any other application with an open-source client—where we can inspect the source code and compile our own client.
If this is done client side, it doesn't boils down to that. You can easily decompile and see for yourself what it does. You will gain quite a bit of notoriety if you are the first one to catch them too.
As he said:
> if we ever did it would be quite obvious and detectable that we had done it.
> Assuming your device allows you to get the binary. Apple is already in a position to disallow this if they choose to in the future.
Theses kinds of thing never stopped anyone. Being the first to share a hash of a system file in a console is always an achievement that many hackers tend race to do when a new one is released.
For sure the harder it is, the less person will do it, thus the more theses things will be able to go under the radar, but for now it's not so much an issue.
I'm glad of this intervention. Straight from my serious concerns.. You had a technical talk about client side AI.
Can you tell us a bit more about the circumstances? Is it something you are exploring to better understand the approach of a competitor (WeChat)? Are you receiving pressures to implement this?
I wasn't involved in the talk so can't speak in detail to it, but as I understand it the purpose of it was to explore spaces other than messaging. For example, one of the applications they showed was making abuse detection more robust to URL cloaking.
>Respect for your privacy is coded into our DNA, and we built WhatsApp around the goal of knowing as little about you as possible ... If partnering with Facebook meant that we had to change our values, we wouldn’t have done it.
I'm sure you personally are a nice, honest and well-intentioned person. Unfortunately WhatsApp's corporate messaging has zero trustworthiness and should be looked at with suspicion. Even when the person saying it happens to believe it.
I appreciate your response, and will give you the benefit of doubt -- however this is in conflict with what a superficial reading of Schneiner's article suggests. Reading the reporting it would seems to indicate that this is happening/in process at Facebook.
While I cannot speak for the Forbes' author, Schneiner is widely reputed as a trustworthy source, especially on matters related to information security. This article calls into question his professional reputation as a information security journalist or yours as an executive at WhatsApp.
As such, in order to help the general community decide for themselves, please shed some light on the following:
1. Does Facebook/WhatsApp have any specific plans for moderating content, via any mechanism, on the client? If so, please enumerate the kind/type of client-based content moderation currently in discussion.
2. Has Facebook/WhatsApp previously looked at doing content moderation on the client? If so, please enumerate the kind/type of client-based content moderation that was previously discussed.
3. What will you do, if Facebook/WhatsApp decides to implement content moderation and/or a content "backdoor" on the client sometime in the next 3 years? Will you continue to work for Facebook/WhatsApp?
4. Should Facebook/WhatsApp decides to implement content moderation on the client, what forewarning will Facebook/WhatsApp give us. What will you personally give?
5. You say that this is easy to detect. Can you please provide technical guidance (or pointers to such) on how to go about detecting this, so that the community at large may better learn how to detect this in any instant messaging app, WhatsApp or otherwise?
I ask the above, in all sincerity, as Facebook's previous poor handling of data requires these kinds of inquiries -- especially when in opposition to reporting by Schneier, who's reputation as a information security journalist is bar-none.
> 1. Does Facebook/WhatsApp have any specific plans for moderating content, via any mechanism, on the client? If so, please enumerate the kind/type of client-based content moderation currently in discussion.
I looked at what WhatsApp promised to do against fake news (something where they had reason to promise harsh measures, since they were basically blamed for murders due to their forwarding features). I'm aware of restrictions and warnings on forwarding, but not some sort of 'fake news detector'.
> 5. You say that this is easy to detect. Can you please provide technical guidance (or pointers to such) on how to go about detecting this, so that the community at large may better learn how to detect this in any instant messaging app, WhatsApp or otherwise?
Reverse engineering their app. Doing it yourself is probably beyond the time you want to invest, paying someone to do it just for you is probably beyond the money you want to invest, but I'd really love if there was a group/entity that consistently checks (through reverse engineering and similar analysis) whether privacy promises given by apps are true, and most importantly, remain true over time.
I believe people would extend much more good faith towards WhatsApp if it was possible to meaningfully use it without exposing all (including non-WhatsApp) contacts to Facebook's servers.
Right now, at least on Android, it seems impossible to add a new contact without adding it to your phone's address book, then giving WhatsApp full access to it. If you revoke the access, you can keep talking to existing contacts, but their names disappear. I would expect that this is just a side effect of nobody caring/testing for the case, but it attracts less charitable interpretations (assumptions that it is intentional to force users to give access).
I genuinely believe that both from a software usability and network effect aspect, WhatsApp is the sweet spot among the secure messengers, and the trade-offs they made (e.g. key escrow for backups and encouragement to do cloud backups) were made in good faith considering the average user's needs.
As a workaround (not a proper solution), you can put WhatsApp in a work profile with an app such as Shelter (https://f-droid.org/en/packages/net.typeblog.shelter/) and put only the contacts you want to use in WhatsApp there.
Granted, it's not ideal, and not even feasible if you already use the work profile fully (with contacts you don't want to share with WhatsApp).
"source is based on overdrawn conclusions from a speculative article. The linked Forbes (F1) article goes to another Forbes article (F2), which links to the Developer talk. F2 is a speculative article based on the Facebook talk..."
> The encryption . . . debate is already over – Facebook ended it earlier this year.
[...]
After all, if either user’s device is compromised, unbreakable encryption is of little relevance.
[...]
The problem is that if Facebook’s model succeeds, it will only be a matter of time before device manufacturers and mobile operating system developers embed similar tools directly into devices themselves, making them impossible to escape. . . . effectively ending the era of encrypted communications.
I guess FB is one of the more visibly monstrous instances of surveillance capitalism so it works well as an example, but I don't understand why the author is acting as if what it allegedly aims to do is unprecedented. As if similar phenomena hadn't already been happening surreptitiously throughout various other hardware/software for quite a long time.
INSERT Blake screaming at Alan and MacLeod in Highlander II: The Quickening
"Please don't bother, Alan.
Every time you turn that thing on it
just gets printed out in my office.
And yes, yes, yes,
yes, yes, yes, yes' yes,
that includes your little computer"
109 comments
[ 0.21 ms ] story [ 309 ms ] threadThis will be true for a lot of people, and it's a huge annoyance. The only things that break this kind of network affect is a market changer - Discord is a good example here.
It doesn't help that, last I checked, the good (open) alternatives were all much jankier or missing one or more major features we use. Makes them hard to sell. :-/
It's better to cultivate that flexibility, than to convert people to flock to the next latest one true DM app.
/s
We are moving ever more towards 1984.
Software which people's civil liberties depend upon should be developed in the open, auditable by the public, as a minimum. Even then, developers need to be vigilant about tainting by state actors.
What do you want? An official announcement from Facebook?
Could be FB Messenger or Instagram DMs for all we know.
https://developers.facebook.com/videos/2019/applying-ai-to-k...
The relevant stuff seems to start at about 23:00.
Original message follows for continuity:
Thank you for this!
The 4-layers-deep source is an F8 talk on using AI to filter for FB TOS-violating content. The relevant portion of the talk is discussing hyperlink website previews of the kind that show up next to URLs posted in Messenger/WhatsApp that show the title and feature image next to the link. In order for this functionality to work at all, the app must crawl the link to pull the metadata for display. This AI team is attempting to pre-emptively block URL-masking attempts that fool the Preview into showing benign content before redirecting users to bad content.
To do this, FB must "pierce the veil" of E2E encryption in order to see what the URL is and test the content. This opens the possibility that FB can further expose supposedly encrypted content in the chat/message. The actual talk refers to putting this AI on the device itself, preserving user privacy (a difficult technical challenge).
If this leaking of URL data is status quo on FB messaging systems, is it reasonable to assume that the supposed encryption is more or less invalid? Are there any ways that this could be a secure system with this kind of content validation going on?
If everything happens on the device and noting is sent to servers, how does it break encryption?
In case I am wrong, and they actually suggest that they will really send data back to its servers, where it is suggested? I really failed to find it.
...So far, we have been keeping this fight [against bad actors and harmful content] on familiar grounds. And that is, we have been training our AI models on the server and making inferences on the server when all the data are flooding into our data centers.
While this works for most scenarios, it is not the ideal setup for some unique integrity challenges. URL masking is one such problem which is very hard to do. We have the traditional way of server-side inference. What is URL masking? Let us imagine that a user sees a link on the app and decides to click on it. When they click on it, Facebook actually logs the URL to crawl it at a later date. But... the publisher can dynamically change the content of the webpage to make it look more legitimate [to Facebook]. But then our users click on the same link, they see something completely different - oftentimes it is disturbing; oftentimes it violates our policy standards. Of course, this creates a bad experience for our community that we would like to avoid. This and similar integrity problems are best solved with AI on the device.
Linking to things is a normal thing to do on the Internet, but Facebook wants to control that, because links often also mean the user moves their attention away from FB, onto the linked thing. This is what FB doesn't want. But it is a very useful thing for the user, to be able to smoothly navigate between apps and websites depending on their needs.
I do wonder why they chose to go with the danger of URL masking that links can be "disturbing" instead of "scams". Because I don't believe these techniques are (widely) being used to trick people into watching animal torture or other disturbing things. They'll be tricked to click on scams and phishing sites.
The difference between "disturbing" and "scam" is literally the difference between subjectively not-okay and objectively not-okay.
Facebook just wants the power to prohibit you from accessing what it deems "disturbing".
Also just from a technical perspective, there's some very weird assumptions: FB's server would see something different than the user's browser. But FB's local AI spy process would see the same thing? Are they seriously pretending that you shouldn't be able to securely log onto another website in a browser? Yes the user's browser sees something different, because it is logged in, and no FB's local AI spy process is not invited ...
At most they could warn that the site looks different than what the server side FB scanner saw. Then the user could decide if they feel more secure because this is expected as they are logged in and FB can't watch along, or if this is unexpected and they should be cautious.
if the process runs on the device, scans information on the device, and then triggers to notify FB when it matches something, that's breaking the encryption.
it's leaking of encrypted information (the fact that it's there, which rules triggered), but more importantly what's the use of encryption if you got a process that's basically acting as a mole, even if it's running locally?
unless I am wrong how this works, but at some point it'll trigger and notify FB or perhaps a government? if it doesn't do anything, then it might as well no be there.
and if this system gets off, there will be false positives, and there will be trigger rules matching much broader things than they strictly need, "just in case". we kind of know how these stories go by now and which bits we can safely give no benefit of the doubt because a state or a corporation wants them so strongly. I'm gonna guess it will be a black box "algorithm" without clear trigger rules, because it's AI/ML it's hard/impossible to explain exactly how it reaches its conclusions and therefore nobody knows really (and much NDA unto those who do). that's gonna be a hard sell though, even to the general public.
Watched through the whole presentation. Not a single mention of WhatsApp.
It could be Messenger exclusive for all we know right now.
https://news.ycombinator.com/item?id=20549610
He has hyperlinks to further articles.
I hope people begin to take notice.
The value of WhatsApp is not that its technically better than Signal or Matrix, the value is that there are 1B+ users.
Give me Matrix with Signal's UX, and I'd be all over it :-)
Google is in the middle of rolling RCS out and that fact that E2E encryption isn't part of the basic spec is appalling. I don't know how a developer could work on something like this at Google and feel good about themselves.
https://www.theverge.com/2019/6/17/18681573/google-rcs-chat-...
"Content moderation" is just another kind of surveillance. You have to have been watching -- surveilling -- in order to moderate.
AT&T already has capital and people invested in surveillance. https://en.wikipedia.org/wiki/Room_641A
... hopefully
Fuck them hard.
If you look closely at how the founders exited facebook, and what was said at the time, it seemed (to me at least) that they left over exactly this.
I mean, that aside, can you really come up with many other operational arguments that would result in people walking a way from hundreds of millions of dollars? (or by some estimates leaving over a billion dollars on the table between the two of them)
You haven't bothered to read the source, haven't bothered to understand what the context is, and haven't even bothered to see that there is no mention of WhatsApp in the talk.
Ducking tired of click bait and people who ducking reward it.
:face-palm:
It took my brain a few tries to parse the word duck in this context.
There's plenty of arguments:
- Wanting to do something new without the constraints of Facebook owning all the IP by default
- Not particularly liking the work environment and being independently wealthy enough not to have to put up with it
- Health problems from stress - money doesn't compensate for bad health
People walk away from huge piles of unvested equity every day of the week.
Unless the government plans to go back to the 80s, in which cryptography is heavily restricted from export, this doesn't accomplish anything.
It is a bizarre paradox where designing a secure system is a hard problem, where zero-days are commonly cheaper than brute forcing a 64-bit key, while cryptography is chronically lambasted.
And they can use it to converse only with themselves. Getting everyone else to stop using their favourite panopticon is the hard part.
All it takes is a usable interface, general availability, and a reason to change. If WhatsApp starts doing content moderation, of any form on the client, I am of the strong opinion that users will find this to be "creepy" and will immediately start shifting. As it is, all of my technical contacts have already left WhatsApp for Telegram. Since that group/audience has been the bellwether for all of the previous instant messaging client shifts over the last 2 decades, I imagine it's only time before "everyone else" also shifts off WhatsApp. I don't necessarily think it will be Telegram, but they definitely have the most momentum today when compared to, eg, Line, Zalo, Viber, Kik, et al.
That wouldn't accomplish anything now, anyway. All the current best crypto we have is out in the world, and plenty of crypto research happens outside the US. If the US revived the export restrictions, it wouldn't have much effect on the rest of the world.
https://www.theregister.co.uk/2019/07/31/home_sec_priti_pate...
So I am not surprised.
Facebook has recently talked about merging Messenger, WhatsApp, and Instagram chat into one while "also keeping end-to-end encryption."
Why would Facebook cuts its own access to all the data ming it was getting with the Messenger and Instagram chat? It wouldn't, unless it uses "faux" e2e encryption. Besides, Facebook currently allows advertisers to send ads Messenger users. How would those ads be managed by FB's servers, if the encryption is truly end-to-end between the sender and the receiver?
Also, this "new encryption system" is exactly what the founders of WhatsApp left:
https://www.nytimes.com/2018/04/30/technology/whatsapp-faceb...
Fortunately the employees who work on security stuff tend to be anti-authoritarian so it will leak out of FB pretty fast unless it was some secret court order.
Wire is fine (not as good as or fast as Telegram), but the company seems to be focused on paying customers to increase revenues. So free customers hardly get any support or responses from support (the auto-replies from Wire support state this priority). I like that Wire is by default E2E and syncs conversations across devices and platforms!
I've been waiting to recommend Signal to others for at least three or four years now. But it's 2019, and Signal is still bad in UX and messaging reliability (not to mention one still can't migrate conversations to another device on iOS).
Does anyone else think it's weird that it requires access to your camera roll to view images, access to your camera to use the web version, and access to your contacts to start group chats? (on android anyway)
Aside from the webapp QR-code thing, none of that is technically necessary.
PRICE: Jack. Look at me. I am not the problem here. The problem is that hard cash is fading. Rapidly. That’s just the way of the world right now. And Bitcoin is spreading — and if Bitcoin takes over we are all in a world of hell. It is unregulated. It has already reached its transaction volume maximum and it is partly controlled by Chinese miners.
With E-Coin we control the ledger and the mining servers — we are the authority. I will make sure that you will have visibility into every single wallet that’s opened: every loan, every transaction. Which means we can start making new assets. Which means we can start rebuilding the banking sector without you having to inject even more politically unpalatable federal funds into it.
JACK: The President will laugh in my face.
PRICE: But he will know that this is the right thing to do. This is going to be controlled by a good old-fashioned American company. You want to regulate it? Be my guest. Regulate the shit out of it. I’ll give you backdoors, side-doors, trace — whatever you want. Just don’t. Shut it. Down.
We have to TRUST WhatsApp, Telegram, Signal, et al because there are no good OPEN SOURCE alternatives. Otherwise we could just run our own.
We have to TRUST the current PKI and DNS because the alternatives are still immature.
But EVEN IF we develop robust and mature alternatives, and wide adoption, we have to TRUST the Device, OS and Browser makers.
The Trusted Computing Base is made by a handful or companies. Open source in hardware is still a nascent field.
It’s interesting if one can ever be truly sure that some part or chip hasn’t been interdicted. Apple tries to scan components as they arrive vs their hardware designs.
But hardware will always be the weak link. Keyloggers. Cameras watching your fingers from the ceiling.
The future of surveillance is in this kind of stuff. We will ALMOST have secure communications, but not really. The only thing you can be sure of is sending quantum entangled particles from airgapped rooms.
Yes, there are. There has been for many years. Matrix or XMPP with OMEMO.
To be crystal clear, we have not done this, have zero plans to do so, and if we ever did it would be quite obvious and detectable that we had done it. We understand the serious concerns this type of approach would raise which is why we are opposed to it.
If this is done client side, it doesn't boils down to that. You can easily decompile and see for yourself what it does. You will gain quite a bit of notoriety if you are the first one to catch them too.
As he said:
> if we ever did it would be quite obvious and detectable that we had done it.
Assuming your device allows you to get the binary. Apple is already in a position to disallow this if they choose to in the future.
Theses kinds of thing never stopped anyone. Being the first to share a hash of a system file in a console is always an achievement that many hackers tend race to do when a new one is released.
For sure the harder it is, the less person will do it, thus the more theses things will be able to go under the radar, but for now it's not so much an issue.
Can you tell us a bit more about the circumstances? Is it something you are exploring to better understand the approach of a competitor (WeChat)? Are you receiving pressures to implement this?
See also:
>Respect for your privacy is coded into our DNA, and we built WhatsApp around the goal of knowing as little about you as possible ... If partnering with Facebook meant that we had to change our values, we wouldn’t have done it.
I'm sure you personally are a nice, honest and well-intentioned person. Unfortunately WhatsApp's corporate messaging has zero trustworthiness and should be looked at with suspicion. Even when the person saying it happens to believe it.
While I cannot speak for the Forbes' author, Schneiner is widely reputed as a trustworthy source, especially on matters related to information security. This article calls into question his professional reputation as a information security journalist or yours as an executive at WhatsApp.
As such, in order to help the general community decide for themselves, please shed some light on the following:
1. Does Facebook/WhatsApp have any specific plans for moderating content, via any mechanism, on the client? If so, please enumerate the kind/type of client-based content moderation currently in discussion.
2. Has Facebook/WhatsApp previously looked at doing content moderation on the client? If so, please enumerate the kind/type of client-based content moderation that was previously discussed.
3. What will you do, if Facebook/WhatsApp decides to implement content moderation and/or a content "backdoor" on the client sometime in the next 3 years? Will you continue to work for Facebook/WhatsApp?
4. Should Facebook/WhatsApp decides to implement content moderation on the client, what forewarning will Facebook/WhatsApp give us. What will you personally give?
5. You say that this is easy to detect. Can you please provide technical guidance (or pointers to such) on how to go about detecting this, so that the community at large may better learn how to detect this in any instant messaging app, WhatsApp or otherwise?
I ask the above, in all sincerity, as Facebook's previous poor handling of data requires these kinds of inquiries -- especially when in opposition to reporting by Schneier, who's reputation as a information security journalist is bar-none.
I looked at what WhatsApp promised to do against fake news (something where they had reason to promise harsh measures, since they were basically blamed for murders due to their forwarding features). I'm aware of restrictions and warnings on forwarding, but not some sort of 'fake news detector'.
> 5. You say that this is easy to detect. Can you please provide technical guidance (or pointers to such) on how to go about detecting this, so that the community at large may better learn how to detect this in any instant messaging app, WhatsApp or otherwise?
Reverse engineering their app. Doing it yourself is probably beyond the time you want to invest, paying someone to do it just for you is probably beyond the money you want to invest, but I'd really love if there was a group/entity that consistently checks (through reverse engineering and similar analysis) whether privacy promises given by apps are true, and most importantly, remain true over time.
Right now, at least on Android, it seems impossible to add a new contact without adding it to your phone's address book, then giving WhatsApp full access to it. If you revoke the access, you can keep talking to existing contacts, but their names disappear. I would expect that this is just a side effect of nobody caring/testing for the case, but it attracts less charitable interpretations (assumptions that it is intentional to force users to give access).
I genuinely believe that both from a software usability and network effect aspect, WhatsApp is the sweet spot among the secure messengers, and the trade-offs they made (e.g. key escrow for backups and encouragement to do cloud backups) were made in good faith considering the average user's needs.
Granted, it's not ideal, and not even feasible if you already use the work profile fully (with contacts you don't want to share with WhatsApp).
https://developers.google.com/android/work/requirements?api=...
because yor denial, only covers the moderation sugar on top. the damage is already done. a long time ago.
References:
[1] https://www.schneier.com/blog/archives/2019/08/facebook_plan...
[2] https://www.reddit.com/r/privacy/comments/ckrmjg/facebook_pl...
[...]
After all, if either user’s device is compromised, unbreakable encryption is of little relevance.
[...]
The problem is that if Facebook’s model succeeds, it will only be a matter of time before device manufacturers and mobile operating system developers embed similar tools directly into devices themselves, making them impossible to escape. . . . effectively ending the era of encrypted communications.
I guess FB is one of the more visibly monstrous instances of surveillance capitalism so it works well as an example, but I don't understand why the author is acting as if what it allegedly aims to do is unprecedented. As if similar phenomena hadn't already been happening surreptitiously throughout various other hardware/software for quite a long time.
"Please don't bother, Alan. Every time you turn that thing on it just gets printed out in my office. And yes, yes, yes, yes, yes, yes, yes' yes, that includes your little computer"