Reading “hex” pointing to a clearly base62-ish string was a bit interesting :-)
Also, could we shard based on a short hash of account_id, and store the same hash in the token? This way we can lose the whole api_key → account_id lookup table in the metashard altogether.
I know sometimes people just like to try things out, but for the love of god do not implement encryption related functionality yourself. Use JWT tokens and OpenSSL or another established library to sign them. This problem is solved. Not essentially solved, solved.
Creating your own API key system has a high likelihood of fucking things up for good!
I don't understand the need for this level of engineering. It appears we are going for an opaque bearer token here. The checksum is pointless because an entire 512 bit token still fits in an x86 cache line. Comparing the whole sequence won't show up in any profiler session you will ever care about.
If you want aspects of the token to be inspectable by intermediaries, then you want json web tokens or a similar technology. You do not want to conflate these ideas. JWTs would solve the stated database concern. All you need to store in a JWT scheme are the private/public keys. Explicit tracking of the session is not required.
Hey OP, sorry for the negativity, I think most of these commenters right now are pretty off-base. My company is building a lot of API infrastructure and I thought this was a great write up!
Side note: the slug prefix is not primarily intended for the end-user / developer to figure out which kind of key it is, but for security scanners to detect when they are committed to code / leaked and invalidate them.
While it's true that API keys are basically prefix + base32Encode(ID + secret), you will want a few more things to make secure API keys: at least versioning and hashing metadata to avoid confused deputy attacks.
Interesting read, I do have some questions though and hope you could answer them:
1. Why do you use the API key ID AND the organization ID, and not just one of them, to prevent the confused deputy problem?
2. Why is not necessary to use something like Argon2id for hashing? You say "our secret is already cryptograhically-secure", but what does this mean exactly? Is it due to the fact that the secret is already very high entropy and cracking it, even if we use much faster hash functions like the ones mentioned in your article, it would practically not be possible even PQ with highly parallelized hardware?
Presumably because API keys are n bytes of random data vs. a shitty user-generated password we don’t have to bother using a salt + can use something cheap to compute like SHA256 vs. a multi-round bcrypt-like?
I don't even understand what approach 3 is doing. They ended up hashing the random part of the API key with an hash function that produces a small hash and stored that in the metashard server is that it?
A bit over-engineered, but it was fun to read about observations on industry standard API keys. I agree it would be nice with more discussion around API keys and qualities one would want from them.
i suspect a lot of tools will try to fetch the url without explicit user action (e.g. messengers do that kind of crap). Gotta be hard to keep keys non-revoked, which is a nice side-effect
> I didn't proceed with this approach since I don't want the API keys to have any info regarding the account, but hey it is all just a matter of preference and opinion.
Well I would have done that and saved half the blog post.
I've always been interested in the technical distinction between an API "key" and an API "token". And the terminology of "key" used to confuse me, because I associated that with cryptography, and I thought an API key would be used to sign or encrypt something. But it seems that in many cases it's basically just a long, random password.
30 comments
[ 2.9 ms ] story [ 57.6 ms ] threadReading “hex” pointing to a clearly base62-ish string was a bit interesting :-)
Also, could we shard based on a short hash of account_id, and store the same hash in the token? This way we can lose the whole api_key → account_id lookup table in the metashard altogether.
Even the random hex with checksum component seems overkill to me, either the API key is correct or it isn't.
PS : I too am working on a APIs.Take a look here : https://voiden.md/
If you want aspects of the token to be inspectable by intermediaries, then you want json web tokens or a similar technology. You do not want to conflate these ideas. JWTs would solve the stated database concern. All you need to store in a JWT scheme are the private/public keys. Explicit tracking of the session is not required.
Here is a detailed write-up on how to implement production API keys: https://kerkour.com/api-keys
1. Why do you use the API key ID AND the organization ID, and not just one of them, to prevent the confused deputy problem?
2. Why is not necessary to use something like Argon2id for hashing? You say "our secret is already cryptograhically-secure", but what does this mean exactly? Is it due to the fact that the secret is already very high entropy and cracking it, even if we use much faster hash functions like the ones mentioned in your article, it would practically not be possible even PQ with highly parallelized hardware?
Anyways, very interesting read, thank you!
Well I would have done that and saved half the blog post.