Ask HN: How Do You Maintain Security When Working Remotely?

63 points by heyalexej ↗ HN
How do you keep communications and your devices safe when working in a distributed team? Maybe from a coffee shop or a coworking space.

I never worked for a company that does that so I was wondering what policies companies like Github & Co. have in place. What best practices beside 2 factor authentication, VPNs, keeping an eye on open ports etc. exist and are used in the real world.

I hope the answers will be useful for people who work remotely.

54 comments

[ 2.7 ms ] story [ 111 ms ] thread
Having worked in an enviroment with people traveling all over the world to conferences, this has happened countless times, although it is much more likely something will be forgotten, or lost, rather than stolen when on the road. Nothing magical, and all common sense, just try to limit your exposure. Here is a quick list:

  - VPN (no split tunnel)
  - 2FA
  - loner laptop pool (with base OS and their presentation)
    - do not take sensitive data off-site
    - encrypted hard disk
    - antivirus
    - screen locks after X minutes
  - let office know if something happens (lock out accounts)
Operate from the standpoint that this is going to happen. Work to limit your exposure. You really want to separate your work and personal life as much as possible, by using two computers, as your personal habits, can have big impacts on your work. I know this is not really what you wanted re: working at Github, but I think most of this still applies. I think a lot of opsec applies to this two.
I guess the "forgotten" and "didn't think about it" part can't be underestimated. I work from coworking spaces and coffee shops in Asia. People do dev work and business related tasks from these places. The devs spin up servers and have databases with standard passwords exposed on the network. Sometimes they might have client or company data dumped into the DB's to work with. It's scary.
I agree with the above post--I can't even count all the times I've seen people bring in a corporate laptop (complete with an asset tag and call if lost sticker--that's just begging for theft attention) at a public coffee house left unattended. If you are going to bother using a VPN 2-factor token and privacy screen, please keep your laptop in your custody and view at all times. Ideally, work from a private executive or home office if you're a remote worker, or on-site at the company. If you must work at a coffee shop, please be sure you don't have PII easily accessible or valuable financial data waiting to be pried (that includes being on a VPN that can potentially have access to such data).
re: "asset tag and call if lost sticker".

I think it's also a great idea to not have your user and hostname set to be "johnbigbucks@bestinvestconsulting" but something more generic or randomized. I use something like this to randomize my hostname.

  awk 'length==6' /usr/share/dict/words | grep -v '[^[:alpha:]]' | shuf -n 1 | tr '[:upper:]' '[:lower:]'
because i'm somehow compelled to golf such things (like unnecessary use of cat):

    egrep '^[[:alpha:]]{6}$' /usr/share/dict/words | shuf -n 1 | tr A-Z a-z
Yep, that's elegant. It's so easy to write bad bash scripts that I do it all the time. So much to learn on my end.
Thanks for your post btw, I did not know about this tool `shuf`. My equivalent involves `sort -r | head -1`
At a previous job (security company) we had an extensive ~/.ssh/config and would forward anything back through a gateway to get at any internal data.

We were pretty much alway connected to the gateway and would used the SSH SOCKS5 proxy feature to get back to any webapps in development and staging staging webapps, code review etc.

To remove anyone from the system we would simply remove their keys/lock one account on the gateway.

Sometimes a bit annoying having to make sure your SSH connection is up all the time but pretty simple to maintain.

Disks encrypted with Full Disk Encryption of course so no worries when laptops get stolen.

disk encryption, totally locked down VPN (forcing everything over VPN with 2FA, if you're not in the VPN your packets are going nowhere), some even forbid USB drives.
I found VPN configuration complicated, routers you have no control over sometimes have problems with VPN bypass, paid service doesn't let it through and so on. Instead I am using the SOCKS5 proxy in OpenSSH together with redsocks and iptables. For me this seems to be simpler and works everywhere. If you have a dedicated IP or your server runs SSLH then you can SSH to port 443 which is rarely if ever blocked.

Also, this allows for easy bastion hosts. While some tutorials still miss it, ssh has -W for 5+ years now and it makes creating bastion hosts even easier than when you needed nc in there. Hire a cheap physical server, put OpenBSD on it, run OpenSSH and absolutely nothing else. Firewall off everything from the sensitive servers except this server and there you go. Of course the bastion host does not have any credentials on it, agent forwarding is your friend.

Put your SSH key on a Yubikey NEO, the NEO on your keychain and never ever ever depart with it. Make it a habit not to be plugged into your laptop more than for the few seconds when actually log into a server. Stealing your laptop is pointless now.

If you work remotely then the exposure to social engineering is bigger as all chat facilities become an attack surface. Practice. Create a Skype account with a disturbingly similar screen name to your own Skype account and try to coax passwords out of your colleagues. If there are no static passwords to be had then you automatically win.

To me, full disk encryption was always of dubious value. You are sitting in a cafe someone walks in and suddenly grabs the laptop and runs out the door. What's good your FDE? Instead, have nothing stored locally and work over SSH. Now at worst what the brazen attacker has is the files open in your IDE but can't open any other files since the connection dropped and the yubikey is still with you. Also, when at the border you are not facing any issues since there are no passwords to be revealed (I do not want to get into a legal debate with anyone whether I can be forced to reveal a password. It's easier to not have the issue.). Your laptop boots, runs etc. Just there's nothing interesting on it.

I like the Yubikey as 2FA better than having some sort of 2FA on my phone because the keychain and the Yubikey is a much lower profile target than a phone. You probably have your keychain affixed to your belt/pocket by a carabiner or similar solutions so ordinary pickpockets, casual coffee shop thieves won't easily walk away with it and honestly don't want too much it either -- phones and laptops are too easy a target. (I do not wear it on a neck chain when showering but you can since it's waterproof. Pick your level of paranoia freely.)

1/vpn is actually easier than ssh, its just that you've probably never used it before

2/fde is nice when you leave the laptop powered off and unattended, which happens generally every night and during travel, obviously doesnt save you if you get it stolen from your bare hands...

Cisco AnyConnect, while a commercial implementation, is incredibly hard to filter by firewalls (including those in China) because it works over HTTPS (and very few people/countries outright block HTTPS), while IPSec is fairly detectable.

I agree that while VPNs are painful to setup, the results are worth it and can really simplify access to protected systems, such as not even exposing services like SSH to the public network. Commercial hardware VPN routers have a very good track record--avoid software VPNs if you can (or use dd-wrt if cost is an issue). What's more is AWS also supports connecting to a VPN router. It may be worth it to co-lo a VPN router and access protected assets (including AWS) via a VPN router (you can of course serve public facing traffic directly from AWS).

> full disk encryption was always of dubious value

On my work laptop, I have the following (because it's simply harder to do development remotely than it is locally):

- Populated onboarding documents, containing everything you would need to steal my identity. These represent my copy of these contracts.

- A local checkout of our code

- Network passwords (encrypted in the repo, but decrypted locally since I need them to deploy). Working to remove these, but they exist now.

- Complete topology to the entire corporate network.

- Logged in email access with a jucy number of archived emails.

- Logged in to various corporate assets from bug trackers to CMS systems, to git frontends...

Now then, an attacker could certainly grab my open laptop out of my hands, and outrun me, all while keeping the laptop from going into screensaver mode. Not too hard for your average federal arrest, but a lot harder for your average opportunist.

If they get my laptop while the lid is closed, or screensaver active, they get nothing but the laptop. Without FDE, they get everything.

It's all about limiting your attack surface.

> since the connection dropped

SSH connections don't magically terminate, by the way: it requires the server to be configured with connection timeouts, and for your client to be inactive long enough to trigger the timeout.

Otherwise, you can change connections, your computer can go to sleep, you can run on mobile with a terrible connection... all while keeping the same open session.

This means your setup just as vulnerable to pickup-and-run attacks as one protected by FDE, with the added downside of their ability to capture data out of your application caches even if they go the with the opportunistic route.

Don't forget a privacy filter for the screen. Amazing how much information you can glean just from shoulder surfing.
I'm always amazed to see New York City banks with employees sitting at street-level, and computer screens in-view of the sidewalk. Simply parking a car outside their office with a powerful video camera aimed at their screen must be able to yield something of value.
privacy filter is v useful, but technically doesn't prevent shoulder surfing if someone is looking at your keyboard for your password
The best approach here is multi-layered with each layer providing another level of protection.

The best way to think about it is to start with the protection of the end-user device, then work on protecting the access from that device to your external network, internal network etc and finally protection of 3rd party services you use as a company.

For the end-user device the important things are probably a secure password manager, full disk encryption, local firewall, antivirus, requiring a password to log in and locking the screen when you leave the machine.

We then move on to access to your external network (i.e. the user facing portion of your service which probably includes admin only tools). A good way to protect this is by requiring strong password + two factor authentication + access from a whitelisted range of IPs. The whitelisted range of IPs is either your VPN network or your network of ssh server hosts used as SOCKS proxy hosts.

To protect your internal network I think the best practice is to require all access to be via a limited set of bastion hosts again either over your VPN or over SSH.

Finally for all 3rd party services you should were possible require two factor auth be enabled and also educate your users on best practices around password strength etc.

Once you have all this setup the next layer is automated / human monitoring. This becomes easier if you require all elevated privilege access to happen via ssh socks proxies and bastion hosts because you can just implement monitoring of new /failed logins to those. You can the setup manual and automated rules to block users. For example, N failed logins block the source IP, login for an unexpected country manually disable the users public key until you verify it was them etc.

Overall the key idea is to help your users stay secure while still making it convenient for them to do there work.

I don't particularly have any security measures other than a secure password, full disk encryption and locking the screen after a few minutes. If you are taking a device outside of the office (e.g. to take home to work over the weekend) you should be using those anyway.

Pretty much all the services (even internal systems for staff only) I interact with are available publicly and have HTTPS, so using a VPN isn't really going to change much. Most of the company's (big multinational) chats happen over Skype, which is known to be monitored by governments so... :)

We don't deal with sensitive data like financial records or personal data, so there isn't really much to compromise. It's probably easier just to guess someone's password if you want to get access our systems as they are available publicly.

> using a VPN isn't really going to change much

I disagree, using a VPN on open wifi protects against HTTP downgrade attacks, weak cypher compromise, replay attacks... HTTPS for most sites is just not as secure as it should be. It also masks your traffic so outsiders can't even see what servers you are accessing, or how.

Wrapping the traffic in a VPN and securing access to your internal assets to office and VPN IPs offers another layer of security against even targeted attacks.

(comment deleted)
If your company uses any sort of publicly available web applications you support, please for the love of all the bits use HSTS and submit the domain to the HSTS preload list.
Is anything special even required, as most services have their own encryption? Email, chat, git server... VPN seems only if you use Windows Filesharing? So sharing docs via some cloud service would get around that?
VPNs aren't that hard to set up, and are relatively cheap insurance against a third party screwing up on their end.
Full disk encryption, close the lid of the laptop when you wander away from it.

No need for a VPN if your connection to the email, chat and vcs servers are using TLS. If not, why not? It is the year 2015.

Why not?

Because you're leaking DNS traffic.

Why not?

Because you're disclosing where your company assets are reachable from a public connection.

I don't know, but a publicly reachable vcs server that's supposed to contain confidential data makes me shiver.

Not to mention vulnerabilities to the myriad of various SSL attacks out there.

Unless you have a crack team of sysadmins who responds to every CVE against OpenSSL and maintains every webserver perfectly, you run the risk of compromising corporate data if you rely solely on SSL to protect your corporate assets.

>I don't know, but a publicly reachable vcs server that's supposed to contain confidential data makes me shiver

Isn't that literally the business model of GitHub (apart from the enterprise edition which seems to have mixed reviews), Visual Studio Online, etc.?

(comment deleted)
I run a 2-3 person shop working on SAAS and iOS apps.

We don't use VPN, because most of the communication goes through HTTPS anyway (github, gmail). I just made sure that everyone has 2FA and strong passwords on key services.

Oh, and don't download user data to your machine.

How do you "make sure" they've got strong passwords without compromising password security for those people? Do you just have their word, hear them press many keys when authenticating, ...?
In our case, we run the widely available password cracking tools and rainbow tables against our stored passwords. If we can get them, those users are notified and their passwords reset.

Yes, we leave any salts and hashes in place - it limits the checks we can reasonably make to the usual password lists but it still catches a large number of poor passwords.

Can't speak for other OSes but OS X constantly phones home to Cupertino, sometimes not even using encryption, thus leaking data when you're booked into a public Wifi.

I literally spent weeks last year grepping the entire Mavericks base installation for hardcoded URLs, domain names and IP addresses and setting up entries in /etc/hosts and NAT rules to hardwire that stuff to 127.0.0.1. I also had to disable lots of LaunchServices/Agents to get the OS to shut up. Can put this up on Github if there is interest. It's only for Mavericks though, couldn't be bothered to upgrade to Yosemite as long as there are security updates for Mavericks.

Oh and another thing a lot of people don't know: The OS stores Wifi passwords in EFI boot variables. This is used for Internet Recovery. So if your device is stolen or just lent to someone else, consider your Wifi passwords compromised, regardless if the disk was encrypted.

> Oh and another thing a lot of people don't know: The OS stores Wifi passwords in EFI boot variables. This is used for Internet Recovery. So if your device is stolen or just lent to someone else, consider your Wifi passwords compromised, regardless if the disk was encrypted.

I can't speak for if this is true or not, but I literally yesterday ran Internet Recovery and it didn't remember my wifi password. I had to re-enter it myself.

Little Snitch is a good reverse-firewall which might be preferable to messing around with /etc/hosts for many, especially since it offers active protection.

Retrieving wifi passwords from NVRAM:

    /usr/libexec/airportd readNVRAM
Alternatively:

    nvram 36C28AB5-6566-4C50-9EBD-CBB920F83843:current-network
    nvram 36C28AB5-6566-4C50-9EBD-CBB920F83843:preferred-networks
    nvram 36C28AB5-6566-4C50-9EBD-CBB920F83843:preferred-count
> I literally spent weeks last year grepping the entire Mavericks base installation for hardcoded URLs, domain names and IP addresses and setting up entries in /etc/hosts and NAT rules to hardwire that stuff to 127.0.0.1

I don't know how OSX works, but can't there still be addressed you missed that are hiding in the proprietary code?

Also, if you are this worried about security, wouldn't you be better off just using a free OS?

Apple locks in IOS developers to OSX - I'm not aware of any practical way of building IOS apps and submitting them to the app store without running OSX. That doesn't mean you need to use it as your regular personal use OS, but if you're spending your days writing IOS apps you don't have any other choice.
As an Android dev who was forced to create my app for IOS because a huge chunk of users are on IOS, the whole process of developing for IOS gave me brain cancer.

XCode (like itunes) is a worthless piece of garbage. It's unfucking unbelievable how little this "IDE" accomplishes. And uploading to app store (with certificates, provisioning profiles, launch images, icons etc) sucked all the brain juice out of me.

The error messages are even much more cryptic than Windows 95. Why on earth is this piece of locked up abomination so popular? Why are we crying about freedom from government so much and then rushing to buy this crapware?

I dropped Ubuntu after three sequential bad quality upgrades (blocking bugs, lost configuration for upgraded packages, Unity). I won't use Windows because the command line isn't unix-style. Remains Mac OS X. Never had a single day of administration. What we need is open-source with a budget...
Because for the end user it offers a clean high-quality user experience with minimal friction.

That is more powerful than anything else. It's more powerful than cost or freedom, and the developers can suck it.

Microsoft was always shit for developers too, but it gave users what they want and users pay.

Can you share that blacklist with us?
Would you consider uploading your configs to Github or posting them on a blog. I'm sure the community would be grateful for it, at least I would.
I think using little snitch or similar would be the better way to go. Since I imagine not every url is hard coded as plain text and there are updates all the time, and changes and caches ....

But I definitely agree, OS X phones home (and other places) all the time. Everytime my phone rings my mac tries to connect to some apple server. ...

On OS X 10.10 there are no WiFi credentials in the EFI boot variables.
VPN, client cert per device. SSH key per device. Full-disc encryption. 2fa. Single sign-on.

That's about it.

Oh, and a clear internal/external machine split with an internal firewall to partition your network.
Some others have provided their inputs, my one is just short: take your devices with you, all the time. Yes, that includes the quick toilet break at starbucks.

I can't remember how many times I heard total strangers asking around "can you look after my stuff quickly?". You can do that for your beach towel, but not for your devices.

Provided that you lock your device before you leave, have the disk encrypted and do regular backups, what's the problem here?
There are lots of quick attack vectors through USB devices, firewire, and other bus systems.
Lots of other attack vectors left wide open, remember Thunderstrike?
They're also (arguably inappropriately) broadcasting that you passed some sort of bizarre, arbitrary profiling heuristic.