HN Explorer

Show HN: UPID – a unique ID with built-in prefix (github.com)

5 points by carderne ↗ HN
The discussion about what to use for database keys has come up a few times on HN. Integers or UUID, or maybe ULID/UUIDv7 with a time component. One that also comes up is prefixes (as used by Stripe, most famously).

The biggest downside (imo) of the Stripe approach is that you either have to store them as strings, which isn't great, or strip/prepend in your API layer.

So I made a new thing called UPID. It's most similar to ULID, as it includes a time-component (but with lower precision) and also encodes to a base32 alphabet. But it also includes a 20-bit prefix (allowing four characters). So it can be stored as a UUID or anywhere a 128-bit blob goes, and it always knows what its prefix is because it's baked in.

I've made implementations for Python, Rust and TypeScript, as well as a Postgres extension (with a upid datatype). Mentioned it a few weeks ago in a HN discussion and people seemed pretty interested so thought I'd post it as a submission.

In Rust it looks like this:

  use upid::Upid;
  Upid::new("post"); // post_2acqyme2ygscyguveuhj5a
There are more examples in the repositories.

Repo (Python, Rust, Postgres): https://github.com/carderne/upid

TypeScript: https://github.com/carderne/upid-ts

Demo site: https://upid.rdrn.me/

3 comments

[ 3.2 ms ] story [ 14.5 ms ] thread
nivertech
Blog post:

UPID is as UPID does

https://rdrn.me/upid/

---

BTW: Hi/Lo scheme for Primary Keys / IDs was invented long before Stripe existed. Partcularly, I used short up to 3 letters prefixes to make IDs Human-readable (and writable!) more than 20 years ago. 5 years later Amazon started using similar scheme in their AWS product, i.e i-NNNNN for instances, etc.[1].

In theory there are 2 kinds of ID schemes: technical/surrogate and natural/business/semantic (i.e. those derived from the natural properties of the object the ID refers to).

Adding a prefix with the type, makes ID scheme hybrid. "Hi" part (prefix) is natural, and "Lo" part is technical. In case of UPID/ULID/UUIDv7 even the "Lo" part is hybrid, since it includes an information when the entity was created (or at least inserted into the database/system).

---

[1] List of some AWS ID prefixes:

  i-  
  vol-  
  snap-  
  ami-  
  eni-  
  sg-  
  vpc-  
  subnet-  
  acl-  
  rtb-  
  igw-  
  dopt-  
  vpn-  
  vgw-  
  cgw-  
  nat-  
  rds-  
  lb-
carderne
Thanks for this, hadn’t thought of this in terms of Hi/Lo. The similarity is limited though because there isn’t any sequence of Hi values…?

AWS also a great example! But I wonder if they’re storing them as fixed 128bit and getting the juicy performance benefits.

nivertech
Just because the "Hi" part is categorical rather than numeric and has a low cardinality doesn't make it any less of a Hi/Lo scheme. This is my own classification, but I think I've read at least one blog where the author called similar prefixes a Hi/Lo scheme

The prefixes are primarily used to create human-readable IDs, which will help avoid human errors and make debugging easier for developers

The original purpose of the Hi/Lo algorithm[1] was mainly to allow the client side to generate PKs in batches (as a performance optimization, offline operation, or building entire data structures on the client side), all of which can be replaced today by UUIDs/ULIDs and the like

---

[1] https://stackoverflow.com/questions/282099/whats-the-hi-lo-a...