>Because secrets in the Secure Enclave are not exportable, they are not able to be backed up, and you will not be able to transfer them to a new machine. If you get a new Mac, just create a new set of secrets specific to that Mac.
I consider that to be a deal-breaker. Hardware fails/lost/stolen, and I need to know that I have an offline backup somewhere if disaster strikes.
You could always register two keys, at least where that is supported, one for your Mac, another a hardware token that resides in a safe. Requires diligence, though, and is hard to verify that it’s been done correctly everywhere.
Using private keys the way they are supposed to be used is a deal-breaker? You're supposed to have (at least) one keypair per device, so if you lose the device, you can restrict access by revoking the keypair belonging to the lost one.
Are you currently sharing one keypair for all your devices?
If I use ten services registered with my private key, and the single authenticated hardware device goes poof (dies/lost/stolen) -is there an out of band mechanism for me to regain access on a new device? Maybe for some, but possibly not all.
An offsite backup means that if everything goes down Monday, I can be back in business on Tuesday without fear.
You could replicate some of this assurance with redundant hardware devices, but that requires perfect diligence in ensuring you have multiple devices approved to each service. AWS only just recently allowed multiple hardware tokens.
Generate a separate SSH key which is only stored in your offsite backup (and then also encrypted with a password), and not on any of your physical machines.
I must be missing something. If this novel key only exists inside my backups, it is not registered with any external services, and it cannot be used in the event the primary key is lost.
For me to register the backup-only key with other services, it would have to live on my machine, and be simultaneously registered in addition to the primary key. I am not sure what I am gaining vs having a backup of the primary key other than increased operational burden.
However, that very quickly gets into the problem of untested backups. I am then relying completely upon a private key which has never been validated to work.
Note that this comes with a set of pretty important caveats, notably no way to backup keys. Which means you absolutely need to have some way to regain access to whatever you secured with this - either via a backup key, by having another person that can grant access or similar.
Regardless of how good that project might be: No, we don't.
Might be appropriate to store keys in Secure Enclave for some but a dedicated TPM (Yubikey/Nitrokey/Trezor/Ledger/etc) is often preferred. Some might want or need backups.
And even then, having multiple implementations widely used is preferred over having everyone relying on the same small group of volunteers or corporate working on any single one.
I consider this kind of reporting (and underlying detection by companies capitalizing on supply chain security fears) borderline irresponsible: anybody can upload a package to a public index, by design, and the presence of a blatantly maliciously package doesn’t imply any degree of actual download, use of, or compromise via it.
Removing malware from the index is a laudable goal, but the language of threats, attacks, thefts, etc. is essentially unsubstantiated: there’s no evidence that this goes beyond opportunistic, untargeted script kiddy behavior.
Edit: FTA, the “rapid evolution” in question is that the malicious packages switched from using base64 for obfuscation to…double base64.
I am looking forward to PyPI's "answer" to this class of malicious packages with automated third-party reporting (and hopefully long-term, quarantining or even deletion). Hopefully this project will swing the narrative closer to the /actual/ security threat that these packages represent to their respective package ecosystems. The PyPI Safety and Security Engineer Mike Fiedler is working on this project :)
Yes! I think an under-considered aspect here is that this kind of high-volume malicious package activity isn’t particularly dangerous in and of itself, but does represent a operational/sustainability risk w/r/t compromising the index’s abilities to respond to actual urgent issues in a timely manner. More automation (reporting, quarantining) goes a long way towards addressing that.
It’s more the language of compromise I resent: I think treating every low-effort malicious spam package as a fundamental risk unnecessarily “spends” the security mindshare budget that engineers keep in reserve. But I think my opinions there are already well-known :-)
I feel that the differentiation between "lifting keys" and "maliciously using them" is not a useful one to make. Both actions are very bad. If you want to wait until we have evidence that the compromised keys were used to pull e.g. proprietary code or configuration from Github; why not also wait until that compromised data is sold? Or until a company is formed leveraging it? Ultimately, it's a slippery slope of "nothing will satisfy you", and there's clearly a genesis event here that is reprehensible and should be known about, even if it is, as you say, just opportunistic script kiddies.
Speaking of which; I loath that characterization of threat actors. This is not the 90s anymore, and it's time to grow up and recognize that the opponents in this space can be extremely sophisticated and well-funded. This is, genuinely, something that many people in the industry refuse to accept; because it implies that we absolutely need to hold ourselves and our work to a higher standard, treat what we produce with severe gravity, and ultimately this means the pace of innovation will slow down and we'll need to ask the mirror serious questions like "wait, so you just, npm installed critical code that was handling sensitive user data? without even auditing it?"
NPM (and, by extension, Github) are extremely blame-worthy here. Reports like this one absolutely happen so Sonatype can sell their product; no doubt on that coming from me whatsoever. But, they also put pressure on key infrastructure providers.
> FTA, the “rapid evolution” in question is that the malicious packages switched from using base64 for obfuscation to…double base64.
Ok, yeah; describing that as "rapid evolution" is overselling it. But let's play in that playground; it's not a rapid evolution, it's an unsophisticated additional layer of easily-reversible obfuscation, which npm did not catch. Why? Why is sonatype catching this, and not npm?
> I feel that the differentiation between "lifting keys" and "maliciously using them" is not a useful one to make.
You misunderstood: the differentiation was between malicious packages being uploaded and actually being used, not between the keys being stolen and being used. I agree that the latter would not be a useful or productive distinction to make.
To be clear: supply chain security is serious, and there are sophisticated (organized crime, nation state) actors in the space. But that isn’t what this is; the lack of sophistication makes that clear.
It’s true that indices could probably be doing more to stop this kind of stuff, and many are; see the adjacent comment by ‘SethMLarson.
A few years ago, my company found hundreds of internal projects using the malicious "python3-dateutil" library. One instance was ultimately used to get full ATO of an AWS account and launch hundreds of EC2 instances to mine crypto. So, the threat is real, even if it is overstated by "companies capitalizing on supply chain security fears".
I agree that the threat is real. But I think the threat needs to be quantifiable in terms of metrics that aren’t solely the attacker’s ability to spam the index.
Put another way: what matters is that the attacker can get arbitrary code to run on your machine, not that they can unload arbitrary package-shaped things to a publicly accessible package index. The former can be solved; the latter isn’t solvable in the general case.
I liken it to hosting malware on the open internet. Yes the internet can serve malware, duh, that's the point. You can share whatever you want over the internet and we wouldn't want it any other way. What matters is who you can get to download and execute it. And what they can do with that remote code execution.
As always, the most secure system is the system that doesn't exist. Every other security argument has to contend with our desire to do useful stuff, and therefore it has to begin with a credible attack vector. Not just a hunch or some notion that it's "security best practice".
Better yet, we need a new model of running computer programs. I think the good old running a python file through an interpreter that has at least read-only access to ~/ is not cutting it anymore. This is why Apple's additional access (explicitly granting read access to certain user folders to certain applications) is a good step to the right direction.
The thing I most want from computing right now is the ability to safely run untrusted code in a sandbox on my own devices.
It feels like we're constantly getting closer to this. We've been mostly able to do this in the browser for years now.
I continue to have high hopes for non-browser WebAssembly. I really, really want to be able to run Python, JavaScript, Rust code etc on my phone and laptop inside a WebAssembly sandbox that tightly limits the filesystem, network, memory and CPU access of that code.
Docker provides virtually no security guarantees on its own and container rights escalation vulnerabilities are a serious threat vector. The default containers provided by many applications are often insecure and ignore recommended security hardening practices. Seehttps://kubernetes.io/docs/concepts/security/overview/ 4 c's of container security.
Sadly it doesn't - Docker isn't specifically designed as a security tool and there are plenty of warnings against using it in this way.
Firecracker is the best I've seen in terms of container security - it was designed by AWS for Lambda and is trusted by people I trust. It's not really packaged for easily running on macOS etc though.
Docker actually worsens this. The default installation of docker makes no effort to prevent host root access from a container. Root in container is root in host
While it is perhaps a hamfisted way to approach security, I am getting closer and closer to installing Qubes OS onto my personal machine. A VM boundary between profiles seems the most practical way of keeping private data on my machine.
Everything else feels like trying to patch over the shaky security foundations with just one more abstraction.
WebAssembly should be the world's most robust, well-tested sandboxing technology: it's been running in browsers for five years now, so it's been more extensively tested than anything else.
I want a sandbox that's robust, widely used and widely tested. WebAssembly feels like it should be the best possible option.
If you have a better idea for a sandbox I can use to run untrusted code on my laptop (and phone) I'd love to hear what it is!
Normal machine code runs in even more contexts and has been for much longer. I don't know any numbers on how many mainstream projects are WebAssembly but it feels like a stretch to call it the "worlds most robust and well-tested" anything. And the tech itself doesn't sandbox anything so it just feels like a strange choice.
Use KVM if you're on Linux/Android. It's not about the binaries but where and how it's running, there is 0 isolation in WASM alone, and creating a new runtime to run it outside of browsers will not give you what you're looking for.
Android recently announced they're going the KVM route.
The next challenge is finding good, actively maintained WebAssembly binary blobs for all of the languages I care about.
Pyodide provides one that works well for Python.
I'm finding it surprisingly hard to find a good option for JavaScript. QuickJS or one of its forks might work but I've not yet figured out the incantations necessary to do that.
Then there's Lua, Ruby, PHP, etc - ideally I'd like there to be robust, well maintained, well documented WebAssembly versions of any language that I might want to run code from.
Does ssh-keygen use a good/modern algorithm by default? My unfounded suspicion is that owing to its age and backwards compatibility, the password could be brute-forced on a modern GPU.
Would love to hear differently.
Edit: To put it differently, would it be irresponsible to commit my private key to a public repo if it were locked behind a 32-character passphrase?
Just because OpenSSH (which includes ssh-keygen) has been around for a long time, doesn't mean it is static and outdated. It is under active development, with the latest release two months ago, and nearly 20 commits in the last month.
OpenSSH regularly updates its defaults and removes backwards compatibility. Three months ago, they switched from RSA to ed25519 as the default private key type for `ssh-keygen -t`, and in August they disabled RSA/SHA-1.
From the ssh-keygen manual: "It is possible to specify a passphrase when generating the key; that passphrase will be used to encrypt the private part of this file using 128-bit AES."
Rather than spreading FUD, you can just decide for yourself whether AES-128 is secure enough for you. Since we don't know your threat model, only you can decide whether you're okay with publishing AES-128 encrypted private data in a public location.
I recalled this 2018 article[0] titled, "The default OpenSSH key encryption is worse than plaintext" and was wondering if it is still true. Safe practices are a moving target. One on which I am not qualified to answer, hence my queries.
Good to know that OpenSSH is improving, but is the future uniformly distributed? If you are running a Redhat LTS release, are you getting a secure by default private key protection, or do you have to invoke the correct command?
I default to Github's keygen recommendations, but I can believe that others will run ssh-keygen without arguments.
Thanks for your considered response :) I went a bit hyperbolic there, I appreciate that you answered calmly instead of biting.
I can't speak to Redhat LTS, but I believe they care about security and I've never heard of a compromized Redhat server. As for whether you can then take your private keys from such a server and publish them, I wouldn't advise it, but then again, I wouldn't advise it in general.
People don't know it for the same reason people don't know they can long press their phone's homescreen icons. We don't need some self-satisfying conspiracy.
1. It was only uploading RSA private keys, which are pretty antiquated these days. Generate yourself an ed25519 key, which is both more secure, and not a million characters long when passing round the public key.
2. It wasn’t pulling keys from your SSH agent. This seems like yet another reason to use one. 1Password (unaffiliated, I just really like it) includes an agent that will load keys from your password store and ask for authentication the first time a process requests to use it.
3. I am once again struck by just how fragile the software supply chain is.
Why is this so expensive? Like almost the price of an upper middle class GPU or a modern gaming console (PS5 or XBox or latest SteamDeck) so what's so expensive inside it to justify that much of a price?
It's "expensive"[0] because the product being linked to is completely different than a hardware security token used to store a credential like an ssh key. Yubikeys that can store an ssh key are like $50.
0 - the Yubi HSM is 1-2 orders of magnitude less expensive than competing HSM products.
Components, development cost, a small market, and profiteering.
You’re talking about a hardened appliance meant to store encryption keys, sign transactions with those keys, and resist disclosure of the keys even if an attacker physically steals the device and, for instance, tries to retrieve the keys from something like an xray analysis of the chip.
If you're curious to see what's going inside these malicious PyPI and NPM packages, we host a catalog of all removed packages for research use. Here are links to some examples of the cached malicious source code, along with the signals of malicious intent detected by Socket:
If you're curious to see more examples of the kind of malicious stuff that is posted regularly to package registries, we have a live updating list here: https://socket.dev/npm/issue/gptSecurity
I was trying to install wscat the other day and did
> npm install -g websocat
Met with "Permission denied: node"
Realizing what I wanted was actually
> npm install -g wscat
I did that and left me wondering what was going on with websocat's installation routines because websocat is actually a Rust project, so something someone might accidentally try to do what I have done with, a great name to squat.
In this case it is innocent at a quick glance. Just uses websocat in Node, for some reason.
68 comments
[ 3.1 ms ] story [ 148 ms ] threadhttps://blog.sonatype.com/npm-packages-caught-exfiltrating-k...
I consider that to be a deal-breaker. Hardware fails/lost/stolen, and I need to know that I have an offline backup somewhere if disaster strikes.
Are you currently sharing one keypair for all your devices?
If I use ten services registered with my private key, and the single authenticated hardware device goes poof (dies/lost/stolen) -is there an out of band mechanism for me to regain access on a new device? Maybe for some, but possibly not all.
An offsite backup means that if everything goes down Monday, I can be back in business on Tuesday without fear.
You could replicate some of this assurance with redundant hardware devices, but that requires perfect diligence in ensuring you have multiple devices approved to each service. AWS only just recently allowed multiple hardware tokens.
For me to register the backup-only key with other services, it would have to live on my machine, and be simultaneously registered in addition to the primary key. I am not sure what I am gaining vs having a backup of the primary key other than increased operational burden.
But yeah, if the main device is stolen, it can be a pain to go update all the services with the new device’s key.
I personally use the gpg app on my youbikey for this, and the secret key comes from a backup I install on a livecd.
However, that very quickly gets into the problem of untested backups. I am then relying completely upon a private key which has never been validated to work.
Might be appropriate to store keys in Secure Enclave for some but a dedicated TPM (Yubikey/Nitrokey/Trezor/Ledger/etc) is often preferred. Some might want or need backups.
And even then, having multiple implementations widely used is preferred over having everyone relying on the same small group of volunteers or corporate working on any single one.
Removing malware from the index is a laudable goal, but the language of threats, attacks, thefts, etc. is essentially unsubstantiated: there’s no evidence that this goes beyond opportunistic, untargeted script kiddy behavior.
Edit: FTA, the “rapid evolution” in question is that the malicious packages switched from using base64 for obfuscation to…double base64.
It’s more the language of compromise I resent: I think treating every low-effort malicious spam package as a fundamental risk unnecessarily “spends” the security mindshare budget that engineers keep in reserve. But I think my opinions there are already well-known :-)
I feel that the differentiation between "lifting keys" and "maliciously using them" is not a useful one to make. Both actions are very bad. If you want to wait until we have evidence that the compromised keys were used to pull e.g. proprietary code or configuration from Github; why not also wait until that compromised data is sold? Or until a company is formed leveraging it? Ultimately, it's a slippery slope of "nothing will satisfy you", and there's clearly a genesis event here that is reprehensible and should be known about, even if it is, as you say, just opportunistic script kiddies.
Speaking of which; I loath that characterization of threat actors. This is not the 90s anymore, and it's time to grow up and recognize that the opponents in this space can be extremely sophisticated and well-funded. This is, genuinely, something that many people in the industry refuse to accept; because it implies that we absolutely need to hold ourselves and our work to a higher standard, treat what we produce with severe gravity, and ultimately this means the pace of innovation will slow down and we'll need to ask the mirror serious questions like "wait, so you just, npm installed critical code that was handling sensitive user data? without even auditing it?"
NPM (and, by extension, Github) are extremely blame-worthy here. Reports like this one absolutely happen so Sonatype can sell their product; no doubt on that coming from me whatsoever. But, they also put pressure on key infrastructure providers.
> FTA, the “rapid evolution” in question is that the malicious packages switched from using base64 for obfuscation to…double base64.
Ok, yeah; describing that as "rapid evolution" is overselling it. But let's play in that playground; it's not a rapid evolution, it's an unsophisticated additional layer of easily-reversible obfuscation, which npm did not catch. Why? Why is sonatype catching this, and not npm?
You misunderstood: the differentiation was between malicious packages being uploaded and actually being used, not between the keys being stolen and being used. I agree that the latter would not be a useful or productive distinction to make.
To be clear: supply chain security is serious, and there are sophisticated (organized crime, nation state) actors in the space. But that isn’t what this is; the lack of sophistication makes that clear.
It’s true that indices could probably be doing more to stop this kind of stuff, and many are; see the adjacent comment by ‘SethMLarson.
Put another way: what matters is that the attacker can get arbitrary code to run on your machine, not that they can unload arbitrary package-shaped things to a publicly accessible package index. The former can be solved; the latter isn’t solvable in the general case.
As always, the most secure system is the system that doesn't exist. Every other security argument has to contend with our desire to do useful stuff, and therefore it has to begin with a credible attack vector. Not just a hunch or some notion that it's "security best practice".
It feels like we're constantly getting closer to this. We've been mostly able to do this in the browser for years now.
I continue to have high hopes for non-browser WebAssembly. I really, really want to be able to run Python, JavaScript, Rust code etc on my phone and laptop inside a WebAssembly sandbox that tightly limits the filesystem, network, memory and CPU access of that code.
I'm seeing glimpses of this being possible - https://til.simonwillison.net/webassembly/python-in-a-wasm-s... and https://til.simonwillison.net/deno/pyodide-sandbox for example - but it's still not as easy as I want it to be.
Firecracker is the best I've seen in terms of container security - it was designed by AWS for Lambda and is trusted by people I trust. It's not really packaged for easily running on macOS etc though.
Everything else feels like trying to patch over the shaky security foundations with just one more abstraction.
I want a sandbox that's robust, widely used and widely tested. WebAssembly feels like it should be the best possible option.
If you have a better idea for a sandbox I can use to run untrusted code on my laptop (and phone) I'd love to hear what it is!
Use KVM if you're on Linux/Android. It's not about the binaries but where and how it's running, there is 0 isolation in WASM alone, and creating a new runtime to run it outside of browsers will not give you what you're looking for.
Android recently announced they're going the KVM route.
Pyodide provides one that works well for Python.
I'm finding it surprisingly hard to find a good option for JavaScript. QuickJS or one of its forks might work but I've not yet figured out the incantations necessary to do that.
Then there's Lua, Ruby, PHP, etc - ideally I'd like there to be robust, well maintained, well documented WebAssembly versions of any language that I might want to run code from.
Would love to hear differently.
Edit: To put it differently, would it be irresponsible to commit my private key to a public repo if it were locked behind a 32-character passphrase?
cat private_key_here | head -n -1 | tail -n +2 | base64 -d | xxd
One I created in 2016 is using aes256-cbc with bcrypt for the kdf, which isn't awful at all.
OpenSSH regularly updates its defaults and removes backwards compatibility. Three months ago, they switched from RSA to ed25519 as the default private key type for `ssh-keygen -t`, and in August they disabled RSA/SHA-1.
From the ssh-keygen manual: "It is possible to specify a passphrase when generating the key; that passphrase will be used to encrypt the private part of this file using 128-bit AES."
Rather than spreading FUD, you can just decide for yourself whether AES-128 is secure enough for you. Since we don't know your threat model, only you can decide whether you're okay with publishing AES-128 encrypted private data in a public location.
I recalled this 2018 article[0] titled, "The default OpenSSH key encryption is worse than plaintext" and was wondering if it is still true. Safe practices are a moving target. One on which I am not qualified to answer, hence my queries.
Good to know that OpenSSH is improving, but is the future uniformly distributed? If you are running a Redhat LTS release, are you getting a secure by default private key protection, or do you have to invoke the correct command?
I default to Github's keygen recommendations, but I can believe that others will run ssh-keygen without arguments.
[0] https://latacora.github.io/blog/2018/08/03/the-default-opens...
I can't speak to Redhat LTS, but I believe they care about security and I've never heard of a compromized Redhat server. As for whether you can then take your private keys from such a server and publish them, I wouldn't advise it, but then again, I wouldn't advise it in general.
1. It was only uploading RSA private keys, which are pretty antiquated these days. Generate yourself an ed25519 key, which is both more secure, and not a million characters long when passing round the public key.
2. It wasn’t pulling keys from your SSH agent. This seems like yet another reason to use one. 1Password (unaffiliated, I just really like it) includes an agent that will load keys from your password store and ask for authentication the first time a process requests to use it.
3. I am once again struck by just how fragile the software supply chain is.
https://developers.yubico.com/SSH/Securing_SSH_with_FIDO2.ht...
Can't steal keys that are in a detachable hsm.
https://www.yubico.com/product/yubihsm-2-series/yubihsm-2/
https://www.yubico.com/product/security-key-nfc-by-yubico-bl...
Or it's just enormous profit seeking?
0 - the Yubi HSM is 1-2 orders of magnitude less expensive than competing HSM products.
You’re talking about a hardened appliance meant to store encryption keys, sign transactions with those keys, and resist disclosure of the keys even if an attacker physically steals the device and, for instance, tries to retrieve the keys from something like an xray analysis of the chip.
That’s the theory. In practice these things have been…Less than perfect. https://www.schneier.com/blog/archives/2019/06/hacking_hardw...
https://socket.dev/npm/package/shineouts/files/1.12.16-beta....
https://socket.dev/npm/package/@dynamic-form-components/shin...
https://socket.dev/npm/package/eslint-plugin-shein-soc-raw/f...
https://socket.dev/npm/package/@spgy/eslint-plugin-spgy-fe/f...
If you're curious to see more examples of the kind of malicious stuff that is posted regularly to package registries, we have a live updating list here: https://socket.dev/npm/issue/gptSecurity
[Disclosure: I'm founder of Socket]
> npm install -g websocat
Met with "Permission denied: node"
Realizing what I wanted was actually
> npm install -g wscat
I did that and left me wondering what was going on with websocat's installation routines because websocat is actually a Rust project, so something someone might accidentally try to do what I have done with, a great name to squat.
In this case it is innocent at a quick glance. Just uses websocat in Node, for some reason.