Ask HN: How do you prevent code leak from current/former employees?

53 points by bkovacev ↗ HN
How do you secure your code assets from leaking onto github/gitlab? Let’s say hypothetically that the company does not have a work issued laptop and that the company has invited developers to the organization with their personal github accounts.

For instance, a developer worked for your company for two years, but is no longer there, and has leaked some production code he wrote to show to a recruiter/developer?

Another example would be a current malicious employee who posts code snippets of your super custom trading algo to github as part of his next job search or while trying to land a contracting deal?

61 comments

[ 1.6 ms ] story [ 139 ms ] thread
I've had some experience with two groups of employers that care: the military and financial trading firms. The basic answer is simple, although it has many side-effects: your work computer is isolated from the Internet.

In both cases, you can end up going to jail for a long time if you leak the code, so there's a fairly strong disincentive. Of course, that's only effective if you believe you'll get caught. And so your computer and the network is fairly heavily monitored, and very restricted in what you can do.

In the military case, there is no Internet access at all. Anything secure is on an entirely separate network. So, no Googling for hints, no Stack Overflow, no GitHub. In the place I worked, phones were not allowed either. If you needed to look something up, you could leave the secure area, use an insecure computer to look something up, and then go back into the secure area. You could bring stuff in (paper), but not take anything out. And remember the "go to jail" penalty if you tried and got caught.

In trading firms, I've worked for firms that pretty carefully control their Internet access. Internet traffic is white-listed, and any site with an ability to upload is either totally blocked, or if it's really, really useful (eg. GitHub), it gets a customized filter in the firewall/proxy that white-lists only the read-only URLs.

Any remote access is done entirely via RDP, locked down to prevent everything except screen + keyboard + mouse.

If the company is issuing laptops that leave the office, or allowing access to GitHub/Lab (and especially using personal GitHub/Lab accounts), then they're not really serious about protecting their code.

I've known companies that physically removed all USB ports from laptops.

It's common to have monitoring software installed as well.

In most companies it's not really workable to prevent employees from taking their laptops out of the office.

> I've known companies that physically removed all USB ports from laptops.

Second order effect of Stuxnet .

I think some organizations did that even before Stuxnet.
Supposedly the US military used epoxy on USB ports on machines in Afghanistan. That was after sending someone through the local market with a wad of cash to buy up all the thumb drives there.
> I've known companies that physically removed all USB ports from laptops.

I've encountered desktop setups where various ports were filled with epoxy, rather than being removed.

A co-worker once bridged an air gap using audio. I forget most of the details, but his POC used staccato clicks to evade notice rather than supersonic frequencies (a prior unsuccessful attempt IIRC tried modulating the vsync on CRTs in lieu of speakers, which gave me a headache and prompted me to track it down since I could hear it even though no-one else could).

I've heard of stuff like that, but I'm wondering why your coworker was even trying to bridge an air gap.
Could be a security researcher.
Mostly to prove it was possible to get the PTB to either tighten or relax the air-gap requirement. As in, he found the requirement annoying but in his opinion it failed to meaningfully increase security.
(comment deleted)
I've worked for the DoD in SCIFs. Complete physical isolation, armed security, gates, electronic locks, physical locks, EM isolation (Faraday cages, the works). It would have been trivial to exfiltrate anything I had access to. These barriers are about not letting the wrong people and data in; they are not about stopping data from going out, at least not physically. If anyone working in these facilities wants to physically remove data, it's trivial. Witness Snowden and similar leaks. That's just as easy today as it was then.

That's not to say there are no protections in place. Systems are isolated as much as possible; there are access controls; there are audit trails. As the value of the data increases, the number of logs required to access it increases. Someone's going to know. Something like what Snowden did was trivially-easy to do, but to do that without getting caught would have been near-impossible. The key to Snowden is that he didn't mind getting caught.

In the end, it boils down to trust. Trust is the only thing that prevents data from leaking. You put extra protections in place to prevent data from flowing out via a software bug, backdoor, or simple mistake; you use physical security to limit who can come inside in the first place. But ultimately the only thing that stops data from leaving is being selective about who you let inside. In the case of the DoD, that involves extensive background checks and security clearances. Once they trust you, though, you can grab whatever you want, if you're so inclined.

I've also worked in finance, for market makers, brokerages, hedge funds, giant investment banks. The protections there are laughable. Most places pay lip service to it but don't actually do anything that really protects their data from leaving. Beyond the basics, it's all security theater. At the end of the day, it's again completely about trust.

To answer OP's question: you prevent code and data from leaking by hiring decent people; not doing anything to piss them off so much that they're willing to risk jail; and compensating them well enough that they aren't as easily bought. You should still put technological barriers in place, but they are almost entirely about keeping bad actors out. Throw in some auditing so that people feel like they're being watched. Add in some more security theater to remind people to care.

The most important factor is hiring good people. Am once off prevention is worth a pound of cure.
Realistically speaking, short of airgapped networks and military-style security, you can't.

It's also much less of a problem than you'd think, since if somebody does walk away with valuable IP and tries to make money with it, they can be sued to oblivion. Employers are also well aware of this, which is why few competitors would want to touch it. See eg. Lewandowski in the Uber v Google case.

It is also pretty rare for some code to be that valuable. It would be about as much work for to figure out hot to leverage stolen code for profit as to just write your own. Data getting stolen on the other hand is much worse.
Yes that's also my thoughts, I have all the code of my current employer on my laptop, what would you do with that code if you got your hands into it? Nothing really, there's not a single use case I can think of which could be useful to somebody.
It may make finding security vulnerabilities a bit easier.

Other than that I can't really think of use cases either for the code bases I've worked on.

> It may make finding security vulnerabilities a bit easier.

Yes that's about it really, there's so much custom code related to our business that even if you are a direct competitor you are not going to get anything out of it.

Agree with this thread, if you see code as data, there's probably plenty of other data in any organisation that needs higher levels of protection

Would you need to notify the ICO (UK) of a data breach if your code was leaked online?

When you're a giant target you need to be careful, but you often don't need to steal either. Meanwhile, in my experience there are amoral and desperate small businesses on every streetcorner stealing anything they can get their hands on. The alternative is often to not make any money becasuse they lack the skills. When you look internationally, the tech industry consists of a finite number of skilled and capable people and organizations, plus a vast ocean of people in over their heads.
I’ve seen that inadequate pay and poor worker treatment is not an effective way to maintain information security from current or former employees.
As a job seeker, even if ethics didn't prevent it, I'd never show private code from previous work. It would demonstrate that I don't respect the company's ownership of the code, and everybody will assume I cannot be trusted.

> Let’s say hypothetically that the company does not have a work issued laptop and that the company has invited developers to the organization with their personal github accounts.

If you care about the privacy of your code, that's not how you do it. You'd have your own private gitlab instance (or github enterprise, if paying LOTS is your thing) with company-specific accounts, likely tied to your Active Directory or so.

And as others have said, treat your employees fairly, pay them well, educate them about code ownership, private data and so on.

I've actually had jobseekers show me their current employers' code that they put on Github temporarily, in a public repository. It was linked as part of their cover letter, I hadn't even spoken to them before that.

I told them they should take that off the internet and we wouldn't be proceeding with their application. It even had things like notes from standups, design documentation and future development plans - I guess their team took the monorepo thing very seriously.

I don’t think this is possible. Even large corporations have code leaked (Microsoft and Google) as well as secret government entities that take extreme measures.

Probably the most effective way is to hire people who are professionals and treat them like so at all stages of them being with you.

You cannot. When I worked at Apple, we had an intern on our team who told me he hated everything about Apple, and that he used Linux for everything. When I asked him why he worked at Apple for the summer he said "To learn the enemy". He had access to a fairly significant amount of source code. It would have been trivial for him to copy a lot of this onto a USB stick. I have no evidence that he ever did this. Our team was great, though, so I hope that by the end of his internship his attitude had changed.
> we had an intern on our team who told me he hated everything about Apple, and that he used Linux for everything

> I hope that by the end of his internship his attitude had changed.

Why?

Lawyer up and make people sign draconian NDAs. There is no tech solution here. A developer can memorize the code and there's no way you can prevent that.
Snowden found a way past perhaps the strongest approach to this sort of problem. You would probably have more luck detecting it than preventing it.
Prevent? Not sure you can in any reasonable fashion.

The only way I've seen places seriously curb exfiltration of code was by forcing everyone to use thin clients to remote into locked-down windows desktops hosted on cloud infrastructure. These remote images could access the source.

Your only other options are NDAs and trusting your people.

And if you can show someone leaked something, for definite, try the court system.

i smell a startup idea for locally encrypted, remotely controlled, git repo :)
I worked at a place with a super custom trading algo and they made roughly no attempt to prevent this. It typically just doesn't happen because employees don't want to get sued out of existence.
The only thing you can do is to be a good employer! Focus on relationships and if they want to go, let them, and ask they keep you in mind if the grass didn't turn out to be greener on the other side of the fence.

If someone is going to leak you're stuff, there's no tech in the world that can prevent a disgruntled person from wrecking you. So don't try, it's laughable, and it wrecks culture.

Also: code is not valuable. The idea already exists in their head so they'd just rewrite it anyway.

Data is valuable though, which is why there's extensive laws governing it.

I came here to make your point about code not being valuable. Everyone can decompose features, your companies edge is you have those features in production. Even without a leak someone could beat you on features and price and you realize the same risk.

Leaking pre-production IP is probably a bigger issue, and there's a bunch of laws around that.

Exactly. I could easily download almost all of my company's code and send it wherever. But, nobody is going to be able to make any use of it, because they don't know how all of the various pieces (services) go together necessarily, and you'd probably still need a bare minimum of 50 engineers to run it (although that's just a guess).

And, even if you did all of that, and replaced the very few things I'm not allowed to access with your own stuff, you still don't have my company's data, which is way, way, more valuable than the code.

The moral here, I guess is that "code is not a company." Even if I handed you the code, you couldn't replicate the company I work for in any reasonable amount of time.

Exactly. The technical controls required to prevent this will: 1) Degrade employer/employee trust 2) Decrease employee productivity

Anything more than an NDA is probably overkill and is going to work against you as an employer.

While I agree with the sentiment, the employee's tolerance of security measures will likely vary according to industry (at least it should). Finance and military applications have been brought up several times for good reason: there is the issue of protecting sensitive information of the client. That may extend to the code, or procedures that can be inferred from the code. In other industries, the sensitive information may only pertain to the employer itself. That's when relationships will be more important.
This is the right attitude. In cases, information leak can lead to even loss of life (ex: healthcare) and "we're nice to our employees" cannot be a plan to protect this info when you're working with clients of that kind.
I don't think a defense industry CEO or HIPAA covered entity is asking on HN how to prevent leaks, but I've been wrong before.
Legit question and I'm sorry if this comes across as condescending.

Do you know the difference between "you're" and "your"? I'm really asking. It's shocking to me that someone on Hacker News can make this kind of error.

Everyone makes mistakes or is not a native English speaker.
I'm often a grammer/spelling nazi and even I do it occasionally. Typing in haste, especially on a mobile device.
My story with mistakes like these: I am not a native English speaker but have lived in the US for about a decade.

I used to feel the same about those mistakes and as someone who has learned english as a second language it was impossible for me to make such and many more similar mistakes in written english (while I was and still struggle with fluency and pronunciation).

As the language became more natural to me, I started doing those mistakes myself.

I think the culprit is in the way written and spoken english differ. In my native language, if I make up a word and speak it out loud to 100 people, there is a 99% chance that they all write it down exactly as I invented it.

This is unthinkable in english and basically force your brain to learn two languages (spoken and written) and in spoken language "you're" and "your" are really not that different. If you write without focusing too much, you just write down spoken english.

I don't know why I wrote this down! Just my experience.

With that setup, your code is probably going to survive on employee-controlled disks past the employee's end date. Even if they delete it, it's probably still on disk, after all. Companies that care about whether or not their code gets copied implement quite a bit of security beyond "use your own computer to clone a git repo". They will issue laptops and workstations that are locked down and remotely controlled. They will segregate codebases based on whether or not someone needs that specific codebase. And, they won't let you clone the full repo if you only need to work on one part.

I worked at Google for many years and I thought they had a very reasonable approach; you couldn't access any code-related services without a special laptop, they used build systems that run on servers they control instead of developer workstations (probably for better caching rather than security), and they didn't give access to the most important stuff to just anyone. Obviously, building that infrastructure probably cost more than your small company has received in funding (or even plans to ever make!), so it's probably impractical to do it yourself. Plenty of people willing to charge you a lot of money for a version that's 1/10th as good, though, and maybe something is better than nothing.

I would personally worry a lot more about what happens when someone's workstation gets infected with malware, and the malware starts checking in code. Someone having a copy of your code doesn't sound like much of a disaster. Maybe annoying. Someone having write access to your repository does sound like a disaster, however. That's how your database gets leaked, that's how your servers start mining bitcoins, and that's how your website starts serving malware. Given finite resources, I'd focus in code going in rather than code going out.

>Let’s say hypothetically that the company does not have a work issued laptop

Why not provide a laptop?

>that the company has invited developers to the organization with their personal github accounts.

Why do that?

>For instance, a developer worked for your company for two years, but is no longer there, and has leaked some production code he wrote to show to a recruiter/developer?

That would be silly. I wouldn't like it if someone applying to our company showed us their current/previous employer's IP. That would be like someone showing nudes of their current/former partners to "prsospective" partners. It just does not compute, unless if the context were precisely about that. A lot of this is based on trust in my opinion.

>Another example would be a current malicious employee who posts code snippets of your super custom trading algo to github as part of his next job search or while trying to land a contracting deal?

Ask them not to. Also, non disclosure agreements and contract clauses. Most people just don't do that.

You're listing some of the worst behavior that is not "normal" or "typical". NDAs and contracts. It helps to hire better, improve work conditions, and be a better organizations [not doing unethical stuff that pushes employees to rationalize doing this, and others to think you deserve it].

>> that the company has invited developers to the organization with their personal github accounts.

> Why do that?

Why not? You can remove someone from a project as easily with the developers existing login as with a new company created one, what benefit does it have?

If you don't need to do this I suggest you don't. It's a PITA, and another small thing that adds to the annoyance of working somewhre and the feelings of lack of trust.

My comment assumed, wrongly I think, that the company invited developers specifically with their personal github accounts. As in, please use your personal account. That assumption was encouraged by the fact their employees use their personal laptops. I naturally asked about the reasons of both.
As far as I can tell there's no difference between a "personal" Github account and any other.
Using personal GitHub accounts really isn’t as big of a deal as people assume. GitHub disables forking of private repos by default, but even if you enable it you can still revoke forks of private repos from people later. Obviously people can manually clone your repo using git, but that’s no different than any other git scenario. The idea that using GitHub will lead to people accidentally making your code public is just not true.

> That would be silly. I wouldn't like it if someone applying to our company showed us their current/previous employer's IP. That would be like someone showing nudes of their current/former partners to "prsospective" partners. It just does not compute, unless if the context were precisely about that. A lot of this is based on trust in my opinion.

It’s silly, but it’s also a shockingly common impulse among some junior developers. When I was mentoring juniors, it was not uncommon for young people to feel like they had some ownership of the code they wrote for a company, or a tool they wrote to help their work. I spent a lot of time explaining the concepts of work-for-hire and IP ownership. Many young devs just don’t even like the idea of IP ownership or the fact that the company literally owns work they did.

You’re exactly right that no sensible company wants to be exposed to IP from another company, but usually the people who do this don’t understand IP ownership in the first place. I know it sounds hard to believe, but they literally think they have the rights to the code because they wrote it.

The only real defense is to take the time to educate employees during onboarding. Don’t assume that everyone understands how IP ownership works. Many junior employees don’t.

>The only real defense is to take the time to educate employees during onboarding. Don’t assume that everyone understands how IP ownership works. Many junior employees don’t.

Examples may be needed. "If you build someone a house, you can't just put it on Airbnb and collect rent on it."

"If you start a company, you just can't empty the bank account and go to vegas." - This one put a lot of people in jail because they don't get the concept of "legal entity" and think it's my company and I can do whatever I want.

I hear your points.

> Let’s say hypothetically that the company does not have a work issued laptop and that the company has invited developers to the organization with their personal github accounts

Cheap out in basic stuff, that's what you get.

(comment deleted)
Law and policy is the stick, while trust and education is the carrot. As others are pointing out, it’s nearly impossible to implement a technical solution without draconian measures that may cost more than they save. The main defense you have is legal, and the main offense you have is the opportunity to earn employee loyalty.

It may be worth asking what scenarios cause you real damage. Someone using un-runnable snippets of code they themselves wrote just to get another job is unlikely to cause any harm to your company. If this is a concern, why not allow it and implement an exit-interview review of assets they’d like to use for job-hunting? You could offer to let employees who leave take some of their code snippets with them, as long as you can review it before they go. This way it’s allowed and you have some control over what’s shared instead of the alternative no control.

Someone posting a custom/secret algorithm is subject to being sued. If posted to Github, you also have the legal right to request removal from Github. If they take it to another company, you can sue the other company. A common proactive defense against this is patenting the algorithm in question so there’s no question where it came or whether other companies can use it.

nothing other than NDA. one with enough motivation can easily leak it.
Neal Stephenson analyzes this in Snow Crash. You need to reprogram their brains via subliminal random looking screen images and divide up all the work so nobody has a complete picture. Problem solved.
If the candidate doesn't have a public github profile with code written recently, which fairly common, the candidate should say and find another solution to showcase his skills. If the recruiter asks for code ignoring the origin and the candidate provides this then... Good! They deserve each other!

Just make sure you don't confused design patterns the candidate learned/uses daly on the "current job" with actual application code.

Your employees' contracts should make clear that you own the copyright in this code, and what the boundaries of the NDA are, and what information you will require them to delete when they leave. Your protection is the threat of legal action.

You might want to post a reminder of https://en.wikipedia.org/wiki/Sergey_Aleynikov as an example.

> the company does not have a work issued laptop and that the company has invited developers to the organization with their personal github accounts

This on the other hand just indicates that you're not very serious about security.

> For instance, a developer worked for your company for two years, but is no longer there, and has leaked some production code he wrote to show to a recruiter/developer?

For starters, I would not hire someone who opened with "look at the cool code I wrote for my last employer".

Ideas are meaningless, execution is everything.

Or should I say that the sum total of a company isn't a few big ideas. Rather it's an accumulation of a lot of little ideas.

Make it harder: Cloud VM accessible only via browser VNC connection. No copy-paste between client and the machine.

Person can take screenshots or type-it them into their machine by looking side-by-side

Very hard to work with tho. More expensive than having a laptop per person.

---

The proper way. Or the other way

IMHO implement the 'policy' use company accounts for company related work and stuff. So create new github, new gmail etc. Use & force SSO for them. Especially all cloud things.

Be nice to people, help them to get where they want to get. Eg, if they're leaving and want to show them to a recruiter help them to get most of it. This way you'll know how much got leaked. Be transparent. Be passionate.