What's a good replacement for Google Apps in a NSA-poisoned post-cloud era?
These days, anyone who values their privacy are probably considering migrating away from US-based cloud-services... In an age of NSA masss-surveilance and service-providers getting hit with secret requests for information, it seems the only thing you can actually trust is open-source software you host yourself.<p>Does anyone have a good guide or experience moving from cloud-services like Google apps to a completely self-hosted open-source solution for email, contact-management and calendering? What about productivity tools?<p>Can you make all of this (more or less) seamlessly integrate with mobile-platforms?
81 comments
[ 3.6 ms ] story [ 150 ms ] threadI'd love for a third viable option to pop up, but for the same reasons as iOS, Windows Phone is out. FirefoxOS seems not quite ready at this point, and until proven otherwise I will consider Tizen vaporware.
That pretty much leaves us with Android. While owned by Google is at least open-source. If you go super-paranoid you can always make/get builds not integrating with Google's services in any way.
Does anyone have any experience with making opens-source solutions for things like email, calendering and contact-management integrate nicely with Android?
http://www.androidcentral.com/easy-privacy-may-be-coming-soo...
Then TextSecure for SMS/chat, RedPhone for calling, and some other alternatives you may find for other apps.
At least for now it might even help if you use services from small, unknown companies, as opposed to services from the big ones, which are a given to be the focus of NSA. But it may be a matter of time before they collect the data (even if encrypted) from those, too, if they aren't already. It would also help a bit to user services hosted in other countries.
Then again, from what we know about the FISC it's hard to say whether legal challenges make much difference.
How so? At least in theory the constitution applies to US citizens and US companies. With a foreign service you have no such claim. Historically the NSA has always been concerned with communication into and out of the US, so if you live the US and use foreign hosting you're essentially stepping right into their spotlight.
Also you can guess that most other governments snoop as much if not more so than the NSA does. It's just the nature of the beast.
The only way to have privacy and a cell phone is to leave it at home.
[citation needed] (and wanted, since I'm curious)
If you search for words "mobile baseband fuzzing" you will find that many people have sought to satisfy their curiosity about this part of a phone.
Owncloud is ok for self-hosted file hosting. It's a little buggy still, but is improving very quickly. It also has Android and desktop apps.
Although Owncloud has contacts/calendar built in, Radicale is more polished and has more features. It's also easy to set up. Make sure you get the latest version through pip rather than your distros package manager.
On Android you can then use CardDav Sync / CalDav Sync to sync calendars and contacts.
It is likely that some of the new-generation open mobile OSs could enable you to build and install your phone's OS more conveniently than Android. That might enable you to do encrypted voice more readily. And that might push Google to provide more confidence-building for Android.
2. Buy a notebook. They sell ones that have a calendar in the front and an address book in the back.
3. Use a stamp and an envelope. The USPO does not keep a record of all sent or received packages. Public drop boxes are also widely available, PO boxes which mask your identity from the sender, and third party mailboxes to mask your identity from The Man. I mean, really, if you're going to send messages to your mistress, girlfriend, lover, or wife, at least have the class to send paper letters she can keep in a lockbox for her great-grandchildren to find after she's dead.
I'm not sure that's true.
JUNE 7--A high-tech computer system that captures images of “every mail piece that is processed” by the United State Postal Service was critical in helping federal agents track the Texas woman arrested today for allegedly sending ricin-tainted letters to President Barack Obama and New York City Mayor Michael Bloomberg.
http://www.thesmokinggun.com/documents/woman-arrested-for-ob...
The technology for doing this in a virtually undetectable fashion has been around for a long time, and I'm sure it's far more advanced now. I very much doubt that ordinary letters have to be opened to see their contents anymore. It can probably be easily done using any number of scanning technologies.
The main problem for them is that it takes more time, effort, and money to do this for snail mail than to just ask a company for a hard drive containing all the emails and tweets of a few hundred million people.
On the other hand, a lot fewer people are using snail mail to communicate these days. So total surveillance of the relatively small amount of snail mail might just be done as a matter of course anyway.
Of course, there are some pesky laws presumably protecting the sanctity of physical mail. But if you think spies have ever let the law stand in their way, you live in a much rosier and prettier world than I do.
To sync data with your mobile device? But doesn't the NSA already record all mobile transmissions? What difference to the NSA intercept program does it make if the data comes from Google Apps or your own open-source platform?
Would you still rely on cloud services for collaborative apps? Well, besides the issue of whether or not the NSA is catching all data packets sent through the fiber...you still have the weak point of any one of your collaborators receiving and transmitting information insecurely.
So if the NSA is a main concern for you, why are you storing portable data in anything else but a large flash memory key that you can use for any of your devices on the road?
2. The NSA is not at all alone among national agencies tapping into their own country's tubes.
The NSA are interested in any and all sources of information, and probably collect just as much or more of it from foreign sources than domestic ones.
Yes, the recent media hoopla has been over the NSA collecting information on Americans in America. But that does not mean that the NSA is limited to collecting information in America. Quite the contrary. Historically, they have been more about collecting foreign intelligence abroad (ie. spying on the rest of the world).
This reminds me of a quote: "You have a choice: you can live under US domestic policy or US foreign policy". If you think you're safe or exempt under either of them, you might have some surprises in store for you.
I'd imagine using zero days to crack into foreign companies would only be for the most specific of information, making them theoretically safer from general abuse.
No. They could simply ask for the data, without any compulsion. Most companies are only too happy to help, and will bend right over -- especially after whoever the boogieman of the day is has appeared on the news lately. They wouldn't want to be seen as not doing thier utmost to help fight the good fight.
Compulsion is the second easiest and second most legal way. This way is also politically, legally and psychologically desirable for them, because they would like to push the envelope of what is considered legitimate surveillance.
Apart from asking and compelling companies, they could just tap the fiber, or use other direct and indirect methods of surveillance. This is really what they do best, and what their original and primary mission has always been: mostly to spy on foreign governments and their agents. After all, the NSA isn't going to get very far asking or trying to compel China to give them a tap in to their data centers. So they'll have to find some other way to do it. And if you think they're going to have a hard time doing it in their own back yard as compared to a hostile country, you might want to reconsider.
Americans get, please don't laugh, protection from their own government. The rest of us are at the mercy of their whims.
The trouble is, in the old days they, well, we all spied on people we had good reason to spy on. "Suspects", if you will. Now they spy on every one they can, in the hope of finding suspects.... to spy more on.
If anything, these revelation prove that our default status to the authorities is "suspect". And we remain a suspect until we die.
And don't go away with the notion that this is all evil America. They just have the resources to do what every other government wants to do, which is why most of them, that can, are doing deals with the US.
Here in the UK, our government (right), like our last government(left), are lapping it up and only to willing to get in to bed with this. They use the same scare tactics, and the evil language of the con man. Nothing to hide? And so on. Before we have a go at the Americans, we need to make it perfectly clear that this is unacceptable in our country, stop it here, and end the endorsement and participation in what the US government is up to. Only then do we have any right to point fingers at the US. Wont happen. Most people simply dont care. Even the satire lot seem happy with it. Yeah, even the comedians are on board.
Honestly, I no longer want part of this world society we are sleep walking in to. Its a disgusting disgrace.
Global system....
Encryption
Of course, email in general isn't secure, and there's no way of knowing what the recipient does with the message anyway, so this is a bit of a tricky problem.
Try this wizard for TXT record to assign to your DNS http://www.microsoft.com/mscorp/safety/content/technologies/...
My setup looks like this:
Mail: Postfix/Dovecot/Roundcube
Setting up a Mail server is the hardest part i think. It's essential to implement measures against spam and abuse. In my setup, this includes: - Get a valid certificate for your mail host and enable opportunistic TLS for SMTP and SMTPD. - Implement DKIM and SPF (and DMARC, http://dmarc.org/) - Use Blacklists: policyd-weight, but look out for changes to public RBL services - Reject Spam in the SMTP session using Milters, never bounce SPAM. - Use spamass-milter and clamav-milter to Filter unwanted stuff. - Use http://sanesecurity.com/ ClamAV signatures, in Debian use the clamav-unofficial-sigs package
Instant Messaging:
http://prosody.im/ is a breeze to configure and it just works.
Telephony / Video calls:
Repro is a secure and simple SIP proxy: http://www.resiprocate.org/About_Repro
But: Resist the urge to install a VoIP PBX like Asterisk and connect it to your phone line & the internet. It will most likely be hacked and abused.
Security:
I've set up a self-signed CA using XCA (http://xca.sourceforge.net/). With a client certificate on a smart card which i embedded into a USB token (http://www.gemalto.com/products/usb_shell_token_v2/), i can run around and plug this USB token in random machines and instantly have my client certificate available in Chrome. This is especially nice if you (like me) don't trust PHP software like the Roundcube Webmailer.
The home (http://xca.sourceforge.net/) is quite overwhelming.
Maybe look for CA stuff in general or some TinyCA tutorial. I've used TinyCA, it's exactly the same complexity as XCA, although I like XCA's UI better.
The whole industry is going crazy about automated deployment and devops stuff. So maybe someone could create a repo containing puppet configs that can set up a "best practice" full stack modern mail server. Don't even have to call it a distro.
Encypting your email and having to deal with key managment issues, and hosting your own email server is going to be more of a hassle than just using someone else's "free" cloud-based service.
Setting up and managing your own services is going to be a burden and a time sink, even if you have the skills, knowledge, and copious free time to do it all.
The question becomes: how much is your privacy and security worth to you? Unfortunately, for most people the answer is "not that much". Nevermind that most of them don't have the knowledge or skills to set up and manage their own services even if they valued their security and privacy enough to sacrifice so much time and effort to it.
The problems are magnified when you consider that most applications are not built with security, privacy, or anonymity in mind. And when you have to use these applications to interact with other people (most of which probably won't be nearly as security or privacy conscious as you are), you're going to have an uphill battle to protect your own privacy. For instance, good luck getting your computer illiterate parents, friends, colleagues, and bosses to properly use GPG to encrypt their emails to you.
This doesn't even begin to address the issues of privacy and security in the "real world" as opposed to the digital one, nor of the digital footprints you leave as you consume online information and media, and interact with others on various forums.
The end result is that online privacy and security are largely becoming the mainstay of the super-paranoid technological elite and the ultra-rich, who can afford to hire security teams to take care of their privacy for them, and of the technological outcast, who (willingly or unwillingly) manages to avoid using the net at all.
The NSA had its own easy-access problem lately...Snowden reportedly snuck data out using USB drives, something that was purportedly banned after a 2008 virus incident...and yet, some sysop apparently thought it'd be a good idea to let a few USB ports survive, because the things are so damned useful. And now look at what that wrought the NSA.
It's not just inconvenience that you have to put up when going for security, it's the risk that the inconvenience will cause you to enact blind spots to ease your day...and easing inconvenience was exactly the reason why people use cloud services (and USB keys) in the first place.
I'm not trying to espouse fatalism (i.e. everyone just give up their data to the G-men). Just pointing out that what the OP wants is basically a unicorn.
The original implementation of Skype was convenient and secure.
It appears that neither the developers of Skype nor the government of Estonia viewed that as a bad trade-off.
That said, if and when an easy-to-use, truly secure p2p communication system ever gains real traction then action against it will probably be on the table. Banning won't really work, as you say, but it can probably be made inconvenient/unpalatable enough to discourage most users.
I think most people agree that there do exist legitimate cases for governmental wiretapping, as long as the rules are agreed upon and followed strictly. Genuine untappable comms will be a threat to that legitimate government function and I don't think, barring some radical shift in popular opinion, there will be much opposition to such a ban.
If the encryption algorithm that's used is well respected among cryptographers and has no known, practically exploitable weaknesses; if this algorithm has been implemented correctly in the encryption software; if the software is used properly by every party involved in the communcation; if these parties' systems have not been compromised; if the people or entitites interested in your communication are not (like the NSA is speculated to be) 20 to 30 years ahead of the public cryptography community -- all very big if's -- then you might buy yourself some time until the encryption is finally broken (something that might happen tomorrow, or next month, or next year, or 20 years from now, or never, who knows?).
In the meantime, your encrypted communication could be intercepted, recorded, and stored indefinitely -- and available for a practically limitless number of decryption attempts in the future.
It was secure if you trust a closed-source program made by a company that could be decrypting your communication because they've got your keys, or that could have put any number of backdoors in the program.
And it's secure if you don't consider the possibility of doing traffic analysis, or the fact that most people are identifiable by their IP addresses, etc.
If you lower your standards enough, I guess anything could be considered "secure".
My comment was more directed toward the fact that ephemeral keys and P2P can be secure and convenient for real-time communication.
There are other challenges too. Decentralised services are probably the way forward but that creates a new set of problems around identity, communication and connectivity (things that are easier when everything just 'dials home'). User experience and design are also something that would have to be considered for such products before most people would/could switch. Keychain management will also be important to ensure security. Think of it like vaccines, where you need to have a certain proportion of the population inoculated before you can benefit from herd immunity [1].
If you're interested in a google-alternative that would deal with mail, contacts and calendars then get in touch with me (email in profile). There are a few of us who'll be building something and we'll need feedback.
Despite all of the above, I'm pretty sure that Hotmail is still the biggest hosted email provider, followed by Yahoo and then Gmail. I can't remember where/when I heard this though.
[1] http://en.wikipedia.org/wiki/Herd_immunity
I've actually done what you're proposing for several years, and I wouldn't recommend it to anyone. And I'm a well respected security professional with the right background who knows all the tricks I'm supposed to do. It was still a huge pain and time sink, which meant I often let patches and updates slide more than I should. The real cherry on top is that all the three different colo/VPS providers I've used were hacked at one point or another, causing disclosure of my personal data and significant outages of my ability to communicate.
My point here is that you're not being honest with yourself if you really think you can manage this as a side project better than the full-time teams at Google do. If you want to pick another provider of email, etc, then investigate carefully and find one you're happy with. But I'd strongly discouraging running your own services.
Nobody is going to dispute that a colo box is still vulnerable to targeted surveillance. But targeted surveillance is not really the issue of the day.
From the NSA's perspective, the expected value of obtaining your data is very small, but they'll bother doing it anyway when the cost is even smaller. Big centralized providers are very cost-effective targets. A sea of tiny servers operated by hundreds of different data centers, hosting heterogeneous applications, monitored by hundreds of different individuals and organizations are really not cost effective to surveil in bulk.
(I agree with the rest of your post -- doing it yourself is a major pain, and no side-project is going to do a great job.)
Google have been hacked too, which I'm sure you aware of since you work for Google on security.
The thing is Justin, right now we don't know for sure just what Google hands over to the Feds and why, and we're all grappling with whether the alternatives are worth the convenience trade-off. I know I am.
You still need to actually home your servers somewhere, which means a colo or VPS that is subject to the exact same laws.
I don't live in the US so no, I'm not subject to the exact same laws. Similar yes, but I don't have to worry about the DMCA for instance. Of course if I do want to use AWS or App Engine then yes, it applies. But I've always assumed that foreign hosted servers can and will be compromised or spied on.
Do you expect the providers of those services to have the capability to fight such requests better than Google, given that Google has a full time legal team dedicated to doing so?
And that legal team has been forbidden to communicate with customers about certain information requests made by Federal agencies. Not much of an advantage is it? Do you guys really get on the phone with your customers and tell them when their data has been handed over? Come to think of it, do you ever speak to customers at all? I can get all of these things with a local provider outside US jurisdiction.
What about the basic security of the provider itself? Do you really expect that to be anywhere near the level of what Google provides on its own systems? To top it all off, there's the maintenance and securing of whatever you're actually running, which requires significant expertise and takes quite a bit of time if you don't want your box popped and turned into a zombie host (which is what is most likely to happen).
If you can't do something like this yourself, or don't have time, then you pay a professional to do it, either by hiring or by outsourcing. It's just business: create a relationship with expected levels of service with people you trust 100% and then pay them what it's worth. You seem to think that not using Google only means having to roll your own when in fact there are many alternatives.
My point here is that you're not being honest with yourself if you really think you can manage this as a side project better than the full-time teams at Google do. If you want to pick another provider of email, etc, then investigate carefully and find one you're happy with. But I'd strongly discouraging running your own services.
If you don't want to spied on, then you absolutely must run your own services. When the feudal lords start fighting or not taking their responsibilities seriously, it's us peasants that get trampled. Perhaps more of us need to move outside the feudal system and help others that also want to.
For email I use exim/dovecot/roundcube and am also experimentally using Bitmessage
For social networking I use Friendica, and eventually I'll be on the Red Matrix.
For dropbox type functionality I either just use plain old ssh or Owncloud. Owncloud also includes a lot of other stuff such as photos and calendar.
For blogging I use Flatpress. It just works.
From a security point of view conventional email remains a problem. It's not yet possible to function in the modern world without email, and trying to persuade even quite technical people to use PGP/GPG routinely has been very unsuccessful, since it's just too inconvenient for most people.
I've never used Google apps to any extent since I don't have much need to collaboratively edit documents. But if I did then there probably are open source systems out there which would do the job. Certainly if I was running a business then I wouldn't even contemplate using Google apps, especially after the recent NSA scandal.
I expect that readers here are more than capable of doing all the above, but sadly the average internet user is still going to remain stuck in the surveillance state with few other options to turn to.
At the moment we have companies making all sorts of set-top boxes, games consoles and media boxes which come in all sorts of shapes and sizes. It seems to me that we can rely on corporations to build user friendly devices for us to consume media.
What we (as citizens) rather than as consumers need is a comms box or a social box or whatever it gets called. Some mentioned "personal data box" on here or words to that effect.
We want (I'm going out on a limb to speak for other around here) secure, federated and private communications: chat communication (instant messaging), mail communication (email), audio communication (voice over ip), video communication (video over ip). Also we'd ideally like our digital social interactions kept secure, federated and private. Finally secure and private online storage of documents would be nice if only for convenience. And to top it all off what would be super-delicious would be a workable single sign-on thingy for the internet age. Please, pretty please? We only have to solve this once. We're at the threshold of the digital age, future generations will thank us for solving these problems when the issues cam e to a head (as they are now).
What is needed for all this is _open federated protocols_. And the corporations that flog proprietary offerings have to embrace these protocols. As far as I can see we have those protocols for email (smtp+tls+...?), for voice over ip (sip), chat (jabber?), social (open soical?), online document storage (?), single sign on (openid?)
We need all this in a box that people can just connect to the net via their router using wifi or whatever. We need it to be peer to peer so that large portions of it cannot be bought out and centralized after the fact. We need to shout loudly when companies roll back on support of open protocols.
We need this in a box. My personal social comms box. I dunno. It seems like a huge but vital need. But we only need to get it right once. We can't trust those in power not to abuse this tech, there will always be an erosion of checks and balances so we need to build a technical solution while we struggle for better social and legal norms.
Who's with me?
The good thing is that now we have momentum to change how we communicate. If we don't, we'll lose this train. We have had research in anonymity going on since 2000 but people haven't been motivated enough to adopt them en masse.
I should given an honorable mention Freedom Box in my previous post: http://freedomboxfoundation.org/
That said, Zimbra does all of what you're after, it's "freemium" open source (meaning that it's a commercial product with a free community edition) and it was fairly easy to deal with last time I set it up 4-5 years ago, presumably it's better now.
They just all choose not to.
I'd prefer if it was hosted in Norway (Opera is a Norwegian software company), but Australia feels like a better choice than the US (too me anyway, perhaps others can chime in).
https://www.fastmail.fm/
Frankly I wouldn't trust any commercially hosted mail service to be immune from government monitoring no matter where it's hosted.
You'd need standards. Standard web service APIs, standard authentication, standard privacy controls, standard database layer, standard encryption between nodes. And that would take most of the effort off the individual apps for handling all this. They'd just say "I have this chunk of data, and it's accessible to these people" and if the everything checks out, the app can grab it from whatever node it's on.
There needs to be this sort of operating-system-for-the-web on which people can build distributed apps. Something like Diaspora is trying to be, but without the limited social networking overtones. A platform where security, privacy, distributed social networking, and sharing are the main concerns, and the apps built on top of it more general.
It might be interesting. I've had this idea since about 2001 but never got around to building something. Maybe it's the right time...
There have been various research projects looking at exactly the things you describe. For example you can look over the work at http://perscon.net (It's one I'm involved with, which I'll be commercialising in due course). It doesn't mention the distributed app piece but it is something we're thinking about.
There are many other groups tackling this from different angles but (thankfully) discussions like these occasionally bring folks together. If you decide to build something, please do feel free to get in touch (email in profile). I'd be very happy to alpha test or give feedback.
It needs a bit more structure on top, a protocol that makes it easy to do certain things in a standard way. Otherwise it's just clay and we're making things that don't fit together real well. We need Legos instead.
Replace your workflow with open-source tools that aren't attached to Microsoft, Yahoo, Google, Facebook, PalTalk, AOL, Skype, YouTube, and Apple.