It's weird under "Get Started" they have links to 40 different languages. You can only get started with 15 of the 40 languages listed, the other 25 are skeleton repos asking for people to start the repo to indicate interest.
It’s kinda clever. The people most likely to look at this project are also likely ideal candidates for implementing the library in a new language (Developer, FOSS enthusiast, interested in the project, need the library in a language they’re familiar with that isn’t implemented yet).
Also, the language pills differentiate between those that have been implemented (color logo, dark text, bold) and those that aren’t (grayscale).
Good points. Those pages also contain links to old implementations (Hashids), because a lot of projects still use those and want to be able to find them.
The approach definitely works. Some time ago I saw .NET listed but discovered it wasn't complete. I was eager to replace an existing Hashids implementation so I made some comments, shared a starter-snippet, and then someone was excited enough to complete in just a few days. It was great to see how quick the community stepped in. Maybe there was a bit of Cunningham's Law in effect with my contribution, ha.
[1]: though with UUID v4 so common to generate and well optimized in most languages I wonder if these userland solutions are really better. You can always generate a UUID and re-encode with base32 or base64 with also is well optimized in most languages
In general, I'm a bit weary of solutions that "guarantee no bad words" – this is usually highly language-specific: One language's perfectly acceptable name is another language's swear word.
We use it in a highly internationalized product spanning multiple languages and haven’t yet ran into a complaint or value on audit that would constitute something offense in any language per our intl content teams anyway.
That isn’t to say it’s 100% (and simply enough we don’t audit every single URL) but I suspect we would have gotten at least a user heads up by now
Never the less we are moving our approach to uuids that get base32 encoded for some of our use case for this. They’re easier to work for us in many scenarios
It's particularly funny because their example docs for .NET outputs "B4aajs", which to any Swedish l33t speaking individual, would read "Bajs", which means "shit"
From a quick look, the lists are pretty short, except for the one with English words that at least have some 404 words, but I can imagine there are far more bad words that you want to avoid than just those?
> That doesn't seem possible. How would that work?
agree; b00b, DlCK, cntfcker
But I suppose, if user doesn't get to craft input, the collision space of converted numerical ids and words like above is sufficiently small to be ignorable.
Besides vowels, nanoid excludes 0, 1, 3, 4, 5, I, l, x, X, v, V, and other lookalikes, so the chances of generating something naughty in any language are close to zero.
Grepping out naughty words in randomly generated text definitely strictly weakens the information content if you're using it for a secure application but is often necessary.
In the early dotcom era the company I worked for were about to go live and the final step was demoing the end to end flow to the ceo. I had done the back end stuff and hadn't paid much attention to the front-end. The person who did the account creation process wanted to nudge people to generate memorable yet strongish passwords, so when it created your account it would generate with a random password which he did by choosing 2 four letter words at random from the unix dictionary and putting a two digit number between them. He ran that past me as an idea and I thought "yeah, good idea" and didn't think more of it.
However he forgot to first grep out all the naughty words so when we demoed it to the CEO/non-technical founder both of the words in his randomly generated password were swearwords.
Odd design decision in that if you provide your own blocklist, it overwrites their (extensive) default list instead of adding to it.
And in general the algorithm is surprisingly complicated for something that could be replaced with simply base64 encoding, the given example (1,2,3) base64 encodes to a string with just one more letter than this algorithm.
That said I do appreciate the semicolon-free-style. I don't typically see that in libs besides my own.
The problem is their block list will change over time. If you don't override it, then your IDs won't decode right when you update. This is a huge risk.
> You have to account for scenarios where a new word might be introduced to the default blocklist
Honestly, I think they need to rethink this. Otherwise you've got different library versions for different languages each using different default blocklists, none of which are compatible.
There explicitly is no uniqueness guarantee. It is known that the same values can encode into many different strings, and they recommend always decoding then reencoding with a stable algorithm to achieve a "canonical" id if one is needed.
What’s the use case for passing in an array of numbers? Typically when generating an ID my input is either a single random number, a string that’s being hashed, or nothing at all.
There is no need for randomised output, and they've said themselves that you can recover the input from the output. It's literally just so that it looks like something cool and technological is happening. You can skip IDs that contain base64 profanity just by doing the reverse encoding on your block list and skipping those IDs. Or you could exclude vowels from your alphabet and do a base 54 encoding. Or you could grow up and accept that your 4 millionth ID which no one is going to read might have a naughty word in it (horror).
I think that's only if you don't want to leak user count when your ID is an autoincrement. Elsewhere people mention cryptographicly remapping integers, which could work (by itself, or before passing the ID to sqids).
I have over 20 years of experience, I run a team, and I can code something like this in my sleep, as well as any average developer (but maybe "average" is lower than where it used to be in my day). Don't talk down to strangers on the Internet so arrogantly, you only end up embarrassing yourself.
I went to check what this code does (beyond a base conversion with a custom alphabet) and I laughed my ass off.
1. Literally TWO-THIRDS of the "library" is a clumsily put together ban list of slurs. Hardcoded, in the library. This DOES NOT belong there.
2. It also contains alphabet checks like say repeated letters, which it'll run every single time it produces an ID, instead of statically checking the alphabet ahead of time, and just using it in production... or even simply selecting an URL-safe alphabet and sticking with it.
3. Despite all the checks, block lists, the default alphabet doesn't do the most elementary thing when producing alpha-numeric ids: exclude letters and digits which look similar, and so will look identical in some fonts. Nasty when you have to write an ID from say a screenshot, or a poster, or what have you.
All in all, that's only borderline more utility than the infamous "leftpad" library, and it may actually be harmful in practice, because it contains a bunch of specific and/or superfluous concerns by the author, almost as if they're trying to justify why this is a library in the first place, but which will likely not align with the precise requirements of the users and their project business rules, while it excludes others (like the one I mentioned in point 3 up there). But I'm sure the kids will be impressed.
Skipping profanity seems like a liability in this design. It means in order to preserve the encoding you need to make the banned word list immutable, otherwise old sqids will decode to the wrong thing when you get them back.
I don't think this holds, you can enforce filtering in the encoding step, i.e. be strict about what you output, but always decode, even if the input is profanity. This means you can also be backwards compatible if you update the list etc. So in short, the old maxim of be strict about your outputs and lenient about your inputs.
From their FAQ: "The best way to ensure your IDs stay consistent throughout future updates is to provide a custom blocklist, even if it is identical to the current default blocklist."
In that case it sounds like a shortcoming on their part. There is no fundamental reason to have that limitation. I understand it can make the implementation easier to not have it, but in my opinion being blocklist change agnostic would be a much better value offering.
The *encoding* changes. The decoding stays consistent:
> Decoding IDs will usually produce some kind of numeric output, but that doesn't necessarily mean that the ID is canonical. To check that the ID is valid, you can re-encode decoded numbers and check that the ID matches.
The reason this is not done automatically is that if the default blocklist changes in the future, we don't want to automatically invalidate the ID that has been generated in the past and might now be matching a new blocklist word.
2. Drop all vowels, numbers, most homoglyphs, and the letter 'x'.
3. Map digits 0-9 to one of the remaining letters.
4. Stringify the integer and replace the digit in each decimal place with its corresponding character.
For my use-case, all the numbers were >7 digits long, so the odds of you getting an offensive acronym were reasonably low unless you started combining them.
But there's no perfect solution. As this dataset shows, you can find offense in almost anything if you look hard enough:
Many of those reviewer comments are utterly moronic. And that is my polite opinion.
How does this work? Is there a review board? Is it put to public review? A few of them like "dick out" and "shtlord" are reasonable, but many of them seem so bonkers it looks like the work of trolls.
Anyway, TIL that 1970s Intel was a MS-13 gang outfit and that Octocat really means "eight vaginas".
I could give a fuck about avoiding swear words, but if you want to avoid slurs and eyebleach-inducing ideas and still have any sort of compact representation, I suspect we have to look not at problematic letters but problematic groups of letters. There's nothing intrinsically wrong with the letter E. Not with G, I, N, or R, but you can sure get a lot of attention you don't want by arranging them in the wrong order. K and Y aren't bad either, unless you're hating on Jewish people.
So maybe there's a 5:4 or a 5:3 encoding out there where you avoid making syllables.
It's a base62 encoder that takes multiple integers as input. Probably a bit-length prefixed encoding. I am assuming it just pads an extra junk integer to re-roll the encoded number.
The mention of one-time passcodes seems odd. Those need to be unguessable, but don't need to be unique. If you supply a suitable random source, then I suppose it works, but the "padded with junk" feature makes these look more complex than they really are.
The standard choice of 4 to 8 random digits works well and it's clear what level of security they provide. Digits are easier to understand than case sensitive latin characters, especially when your native language uses a different character set.
I think you completely misunderstood the article, or you have not read it.
The uniqueness from the system comes from the fact that two different numbers will never have the same id.
The pad only works for things like user ids.
You can also change the alphabet it uses so its not case sensitive; using this alphabet: "ABCDEFGHJKLMNPQRSTUVWXYZ0123456789" and a minimum of 8 digits, it will produce 8 digit ids all the way 4294967295 (0xffffffff).
In a Ruby app we just convert to a high base, like
> 1234567890.to_s(36)
=> "kf12oi"
That gets us most of the way there, but Sqid has a Ruby library and lets you set a much higher base, including upper case characters, and I suppose, emoji. We're going to need much bigger numbers before that space savings makes much difference. I like it, but it's hard to know when something like that is worth adding a dependency.
It looks like an easy brute force too, there's no compute-hard operations here. I guess you could scramble your alphabet? Otherwise Uk always comes after bM, etc.
My understanding is that you can re-order the source alphabet, and encode numbers with swapped characters. Unless you know of 36 numbers that they're exactly 1 id apart, you will always have uncertainty to what the ids actually map to.I guess given a large, large number of ids along with the order they're assigned (which might be given through the time at which they're assigned), you could create a pribabilistic statement on the actual order of the 36 characters based on the fact that most numbers increase/decrease faster the bigger they are. (this fact is e.g. used to detect fabricated bank statements - if the first digit of any number in the statement is equally likely to br 1 or 9, the numbers are randomly generated whereas if they're real, 9 is less likely than 1)
Scambling the source alphabet should have an effect similar to a monoalphabetic substitution cypher. This is not strong cryptography. If the attacker has any ability to generate IDs quickly, like by creating user accounts or other resources they can create many IDs with known ordering. Likely effective against non-serious attempts.
I'd prefer to use crockford-encoded entropy with Stripe-style token prefixes to create unique ID namespaces. Run in through a bad words filter, and it's perfect.
user_1hrpt0xpax7ps
file_xpax7psaz0tv6az0tv6
Etc.
In distributed systems you can use the trailing bytes to encode things like author cluster, in case you're active-active and need to route subsequent writes before create event replication.
Easy to copy, debug, run ops/incall against. If you have an API, they're user-friendly.
Of course you still want to instruct people the prefixes are opaque.
Yeah don’t forget the bad words filter. I worked on an IKEA mailing where the list processing house was adding an autogenerated discount code to the address label. The customers received codes with BOOB, DICK, TWAT, and CUNT embedded within. People were not happy.
Did they never make an IKEA purchase after that or did they get over it like a normal adult?
I don't work retail, but something tells me people will make a stink out of just about anything if it meant potentially free products or other compensation.
Plus, are you filtering just English curse words or all curse words for countries that use Latin characters?
The risk is not in offending someone, but in that someone posting the rude string on social media in real or mock indignation, causing the outrage machine to turn on your brand. There’s a steady supply of bottom-feeding journalists waiting to write the article “IKEA’s new system is sending hate messages to customers”.
We caught it about 2 million into a 15 million print run. I had to go through the control files and change the offending value or identify the pallet and bundle of mail it was in so we could remove and readdress the catalog. It was about 10 years ago and anyone who complained got a coupon.
The mailing house that did the work was fired though.
Correct me if I'm wrong, but, It cannot be unpredictable, which makes the library redundant for security concerns, which would be the one business case to seek for anything other than an UUID (which is already built into Ruby).
The `salt` in hashids just shuffled the alphabet. Now they removed the `salt`, but you can have the same level of "obfuscation" as before if you shuffle the alphabet yourself before calling the library.
Ironically its default encoding is base 55. I figured if you're going to use emoji you should at least use the size of the space to maximize the numbers of bits per grapheme beyond what is possible with ASCII, but apparently not.
Which I guess is the part where senior (citizen) programmers like me get triggered as promised in the README.
I haven’t been able to find a case for this because ids either need to be unique or they’re not going to be large. If they’re unique, I’m using uuid or ulid (uuidv7 of tomorrow) as the sortable primary key type to avoid conflicts without using the db to generate and maintain sequences.
Where do you have unique ids that aren’t the primary key? I would be more interested in a retrospectively unique truncated encoding for extant ulid/uuid; ie given that we’ve passed timestamp foo, we know that (where no external data is merged) we only need a bucketed time granularity of x for the random component of the id to remain unique (for when sortability is no longer needed).
Or just more generally a way to convert a ulid/uuidv7 to a shorter sequence if we are using it for external hash table lookups only and can do without the timestamp component.
The idea is that you encode and decode database IDs with this. You wouldn't save them separately unless you were using it for a purpose other than shareable "identifiers" which don't leak significant amounts of database state. Imagine something like a link shortener where you want to provide a short link to users, but don't want it to just be a number.
> Where do you have unique ids that aren’t the primary key?
For things that need public IDs, I always generate and store uuid4s* to avoid leaking database state or anything like that. Even uuid7 leaks creation timestamps. It's cheap and easy to map public IDs to/from PKs at the API boundaries.
* possibly encoded as something nicer like base64 for APIs or URLs
I wrote a Ruby gem to address this problem of hiding sequential primary keys that uses a Feistel network to effectively shuffle int64 IDs: https://github.com/abevoelker/gfc64
Kinda similar idea to this library but you're encoding from an integer to another integer (i.e. it's format-preserving encryption). I like keeping the IDs as integers without having to reach for e.g. UUIDs
I recommend to review my comment where I also use a Feistel cipher [1] but the difference is that it is not limited to int64 but I can even use 8 bits. Obviously loosing security properties but working as an obfuscation method. If you use a random source with relatively few bits you should check if there are duplicates while with the Feistel cipher you are sure there isn't.
The rate of change over time can be used against you; many people consider their businesses’ month-over-month growth (or lack thereof) to be private information.
“$WEBSITE did 50,000 signups a month during the beginning of the pandemic, but now struggles to sign up a thousand a week” is a story.
Database PK really shouldn't be exposed like this. I think that's what they're getting at, though. The thing is, sequential database PK is still shorter and more readable than a Sqid.
If you use a random number then you need to store it somewhere to map back to the original. Sqids is an encoding, you can decode the sqid back to the original without storage overhead.
Features like the profanity filter avoid creating URL routes like /user/cuntFh.
Cross language support allows interop between the encoder and decoder across microservices written in different languages.
Side note: there are some business insights you can get from a company using serial ids.
i.e if you sign up and get user id 32588 and make another account a few days later, you can tell the growth rate of the company.
And this is possible with every resource type in the application.
I do wonder how much the url bar junk thing matters these days. I tend to use uulids (waiting on uuid v7 wide adoption), and they're a bit ugly, but most browsers hide most of the urls now anyway. The fact that there is a builtin time component comes in clutch sometimes (e.g. object merging rules).
You can even do this when you don’t know the exact interval by using probabilities. The Allies used this method to estimate German tank production in World War II by analyzing the serial numbers of captured or destroyed tanks.
I’m a lawyer and using sequential IDs in a fraud case right now, to determine the number of victims.
Unfortunately, so far, I only have the IDs of two victims, and those are from just within about a month, whereas the fraud has likely been going on for several years. Just simply extrapolating that growth rate isn’t going to be very accurate.
Also, I suspect that the perpetrators did not start at ID 1.
It would be significantly above the average unless the company is ridiculously top-heavy or has shockingly little variation in salary. Or if the "salary" for the CEO ignores certain compensation (eg: paid a salary of $1 + stock options).
Sure thing. I could have worded it better, but I was trying to say that it would be much more skewed if the two samples were, say, CEO and the CFO, or two janitors.
Even with n=1 you can get something useful. IIRC "on average" if you have ID x than the best population estimation is 2*x. Of course the error margin is immense, but it's still better than nothing.
While I tend to strip the tracking params and there are extensions that do this, I don't think most people do. These URLs are pretty 'ugly'.
So if the links that are being shared most on the internet (YT, TikTok, Twitter) don't care, you probably shouldn't either. I think the onus is on the UI layers (Chat apps, etc) to show urls how they look best on their respective platforms.
Edit: to this point, it looks like HN truncates these to make them less ugly too.
At an internship long ago, my boss instructed me to always add a few extra to the auto incremented order ID so customers couldn’t guess how business was going if they happen to order stuff quickly in a row.
I see so many organizations add weird slowdowns from debts associated with this. I reflect on some of the most successful tech businesses of the last decade and remember that all their APIs exposed this kind of data early on and many still do.
Does anyone have an example they can reference of a business being harmed by this information being out there?
I don’t know any stories of digital businesses but there was a case where someone went and counted customers at the door only to realize that the company was lying on the yearly reports.
So I guess that kinda harmed that fraudulent business strategy…
Can confirm, when I worked in VC we used this to verify order volume for a number of startups we were evaluating. For one startup, I wrote a bot to place a small order a few times a day, and log the order number.
Did you use this for due diligence to verify that the data reported by the entrepreneur in the pitch was good or you were looking at some companies in a specific space and checking the company that has more orders?
You don't get the cardinality of the data type, just when the object was created.
There are probably some business cases where the “when” information is potentially useful (I cant think of any) but, you cant know, for example, how many users are in the database.
It's usually benign, but why encode any info into your public IDs? I wouldn't go anywhere near that.
It can make sense for some situational internal database use case where you want temporal locality and can't use full sequential since it's distributed, and even then your DBMS might recommend something else, e.g. Spanner explicitly says not to do this. And it doesn't need to be exposed to users.
Assuming the alternative is a fully random key, this can wreak havoc for performance depending on the database engine and index type used (lots written on this topic).
But I do agree, if performance isn't an issue with your db choice and you're not interested in in getting a free "created_at", might as well go fully random.
Primary keys have to be chosen carefully because they impact disk layout, joins, etc, and full random makes bad PKs in certain distributed DBs. But it's simple and cheap to convert a public user ID (full random) to/from internal row keys (sequential-ish) at the API boundaries using a secondary index or even a cache.
(Like, simple/cheap enough that it's probably worth doing instead of exposing your row keys. Maybe in some careful cases exposing uuid7 row keys makes sense, but not nearly enough to recommend that so broadly. It's not a safe default.)
The solution I prefer is to simply just encrypt the data such as IDs.
Instead of giving user an ID in response, user gets hmac(cipher(Data, secret_key), secret_key) + cipher(Data, secret_key) and then some simple pre-request handler just iterates over query params / form data and decrypts them if signature matches.
It also works as a really nice CSRF protection as user ID of currently signed user can be embedded into Data and checked if current user.id == decrypted data.id.
Another nice advantage is that you can deny the request right in the beginning as you know ahead of time that the provided data is not valid (signature doesn't match), saving some DB queries.
The down side is that URL gets pretty long though, but if that's hidden by browser or user doesn't care, it's a non-issue
248 comments
[ 2.7 ms ] story [ 346 ms ] threadAlso, the language pills differentiate between those that have been implemented (color logo, dark text, bold) and those that aren’t (grayscale).
https://github.com/sqids/sqids-dotnet/issues/2#issuecomment-...
But 127.0.0.1 looks more "readable" to me than lusab-babad
They should use a similar dictionary approach IMO because I looked at the implementation and it’s hardcoded to look for “bad” words
Otherwise looks real straightforward! I’d love to see some performance test suites for it
[0]: https://github.com/sqids/sqids-javascript/blob/ebca95e114932...
[1]: though with UUID v4 so common to generate and well optimized in most languages I wonder if these userland solutions are really better. You can always generate a UUID and re-encode with base32 or base64 with also is well optimized in most languages
That doesn't seem possible. How would that work?
> I looked at the implementation and it’s hardcoded to look for “bad” words.
If you mean https://github.com/y-gagar1n/nanoid-good, that seems to be doing the same thing.
In general, I'm a bit weary of solutions that "guarantee no bad words" – this is usually highly language-specific: One language's perfectly acceptable name is another language's swear word.
We use it in a highly internationalized product spanning multiple languages and haven’t yet ran into a complaint or value on audit that would constitute something offense in any language per our intl content teams anyway.
That isn’t to say it’s 100% (and simply enough we don’t audit every single URL) but I suspect we would have gotten at least a user heads up by now
Never the less we are moving our approach to uuids that get base32 encoded for some of our use case for this. They’re easier to work for us in many scenarios
https://registry.npmjs.org/naughty-words/-/naughty-words-1.2...
From a quick look, the lists are pretty short, except for the one with English words that at least have some 404 words, but I can imagine there are far more bad words that you want to avoid than just those?
agree; b00b, DlCK, cntfcker
But I suppose, if user doesn't get to craft input, the collision space of converted numerical ids and words like above is sufficiently small to be ignorable.
(Sorry)
https://github.com/tttp/dxid
In the early dotcom era the company I worked for were about to go live and the final step was demoing the end to end flow to the ceo. I had done the back end stuff and hadn't paid much attention to the front-end. The person who did the account creation process wanted to nudge people to generate memorable yet strongish passwords, so when it created your account it would generate with a random password which he did by choosing 2 four letter words at random from the unix dictionary and putting a two digit number between them. He ran that past me as an idea and I thought "yeah, good idea" and didn't think more of it.
However he forgot to first grep out all the naughty words so when we demoed it to the CEO/non-technical founder both of the words in his randomly generated password were swearwords.
And in general the algorithm is surprisingly complicated for something that could be replaced with simply base64 encoding, the given example (1,2,3) base64 encodes to a string with just one more letter than this algorithm.
That said I do appreciate the semicolon-free-style. I don't typically see that in libs besides my own.
https://github.com/sqids/sqids-javascript/blob/main/src/sqid...
> You have to account for scenarios where a new word might be introduced to the default blocklist
https://sqids.org/faq#future-blocklist
Honestly, I think they need to rethink this. Otherwise you've got different library versions for different languages each using different default blocklists, none of which are compatible.
They'll decode fine. Encoding might change.
https://sqids.org/faq#valid-ids
> How can I make my IDs unique?
> The library accepts a custom alphabet from which it can generate IDs. Simply pre-shuffle the default alphabet that's provided.
I also have no idea how to use the library.. What are the 3 numbers parameters for encode()? The actual code: https://github.com/sqids/sqids-ruby/blob/main/lib/sqids.rb
I'd simply base-58 a GUID instead. At least you'll know what kind of constraints / sorting the IDs will have.
I went to check what this code does (beyond a base conversion with a custom alphabet) and I laughed my ass off.
1. Literally TWO-THIRDS of the "library" is a clumsily put together ban list of slurs. Hardcoded, in the library. This DOES NOT belong there.
2. It also contains alphabet checks like say repeated letters, which it'll run every single time it produces an ID, instead of statically checking the alphabet ahead of time, and just using it in production... or even simply selecting an URL-safe alphabet and sticking with it.
3. Despite all the checks, block lists, the default alphabet doesn't do the most elementary thing when producing alpha-numeric ids: exclude letters and digits which look similar, and so will look identical in some fonts. Nasty when you have to write an ID from say a screenshot, or a poster, or what have you.
All in all, that's only borderline more utility than the infamous "leftpad" library, and it may actually be harmful in practice, because it contains a bunch of specific and/or superfluous concerns by the author, almost as if they're trying to justify why this is a library in the first place, but which will likely not align with the precise requirements of the users and their project business rules, while it excludes others (like the one I mentioned in point 3 up there). But I'm sure the kids will be impressed.
hoisted by your own petard there, gramps
https://news.ycombinator.com/item?id=38421330
Yes, it's not that complicated (about 160 lines of low-density python), but it's portable, and portable, reusable code goes into libraries.
> Decoding IDs will usually produce some kind of numeric output, but that doesn't necessarily mean that the ID is canonical. To check that the ID is valid, you can re-encode decoded numbers and check that the ID matches.
The reason this is not done automatically is that if the default blocklist changes in the future, we don't want to automatically invalidate the ID that has been generated in the past and might now be matching a new blocklist word.
1. Start with a-z.
2. Drop all vowels, numbers, most homoglyphs, and the letter 'x'.
3. Map digits 0-9 to one of the remaining letters.
4. Stringify the integer and replace the digit in each decimal place with its corresponding character.
For my use-case, all the numbers were >7 digits long, so the odds of you getting an offensive acronym were reasonably low unless you started combining them.
But there's no perfect solution. As this dataset shows, you can find offense in almost anything if you look hard enough:
California Personalized License Plate Requests Flagged for Review 2015-2016: https://docs.google.com/spreadsheets/d/18IUVU9Q4uN_lxqNd5AsN...
How does this work? Is there a review board? Is it put to public review? A few of them like "dick out" and "shtlord" are reasonable, but many of them seem so bonkers it looks like the work of trolls.
Anyway, TIL that 1970s Intel was a MS-13 gang outfit and that Octocat really means "eight vaginas".
Reason for review: hostile, insulting, or degrading
/s
Wow this is a funny peek into a weird perdicment where people need to justify that they have a good reason to have a specific license plate.
Some seems obviously ok such as:
INT13H
314 PI
And some are obviously not:
DRY(hand emoji)JOB
DICK OUT
Come to think of it: Can license plates have emojis now?!
I could give a fuck about avoiding swear words, but if you want to avoid slurs and eyebleach-inducing ideas and still have any sort of compact representation, I suspect we have to look not at problematic letters but problematic groups of letters. There's nothing intrinsically wrong with the letter E. Not with G, I, N, or R, but you can sure get a lot of attention you don't want by arranging them in the wrong order. K and Y aren't bad either, unless you're hating on Jewish people.
So maybe there's a 5:4 or a 5:3 encoding out there where you avoid making syllables.
https://sqids.org/faq#future-blocklist
The standard choice of 4 to 8 random digits works well and it's clear what level of security they provide. Digits are easier to understand than case sensitive latin characters, especially when your native language uses a different character set.
The uniqueness from the system comes from the fact that two different numbers will never have the same id.
The pad only works for things like user ids.
You can also change the alphabet it uses so its not case sensitive; using this alphabet: "ABCDEFGHJKLMNPQRSTUVWXYZ0123456789" and a minimum of 8 digits, it will produce 8 digit ids all the way 4294967295 (0xffffffff).
If I figure out you're using (36) then I know the next number 1234567891 is "kf12oj".
Not the case with Sqids.
user_1hrpt0xpax7ps
file_xpax7psaz0tv6az0tv6
Etc.
In distributed systems you can use the trailing bytes to encode things like author cluster, in case you're active-active and need to route subsequent writes before create event replication.
Easy to copy, debug, run ops/incall against. If you have an API, they're user-friendly.
Of course you still want to instruct people the prefixes are opaque.
I don't work retail, but something tells me people will make a stink out of just about anything if it meant potentially free products or other compensation.
Plus, are you filtering just English curse words or all curse words for countries that use Latin characters?
The mailing house that did the work was fired though.
They call it out on their front page.
Obfuscated
Which I guess is the part where senior (citizen) programmers like me get triggered as promised in the README.
Where do you have unique ids that aren’t the primary key? I would be more interested in a retrospectively unique truncated encoding for extant ulid/uuid; ie given that we’ve passed timestamp foo, we know that (where no external data is merged) we only need a bucketed time granularity of x for the random component of the id to remain unique (for when sortability is no longer needed).
Or just more generally a way to convert a ulid/uuidv7 to a shorter sequence if we are using it for external hash table lookups only and can do without the timestamp component.
For things that need public IDs, I always generate and store uuid4s* to avoid leaking database state or anything like that. Even uuid7 leaks creation timestamps. It's cheap and easy to map public IDs to/from PKs at the API boundaries.
* possibly encoded as something nicer like base64 for APIs or URLs
So instead of
You'll get something like Kinda similar idea to this library but you're encoding from an integer to another integer (i.e. it's format-preserving encryption). I like keeping the IDs as integers without having to reach for e.g. UUIDs[1] https://news.ycombinator.com/item?id=38418198
“$WEBSITE did 50,000 signups a month during the beginning of the pandemic, but now struggles to sign up a thousand a week” is a story.
Features like the profanity filter avoid creating URL routes like /user/cuntFh.
Cross language support allows interop between the encoder and decoder across microservices written in different languages.
i.e if you sign up and get user id 32588 and make another account a few days later, you can tell the growth rate of the company.
And this is possible with every resource type in the application.
I do wonder how much the url bar junk thing matters these days. I tend to use uulids (waiting on uuid v7 wide adoption), and they're a bit ugly, but most browsers hide most of the urls now anyway. The fact that there is a builtin time component comes in clutch sometimes (e.g. object merging rules).
You can even do this when you don’t know the exact interval by using probabilities. The Allies used this method to estimate German tank production in World War II by analyzing the serial numbers of captured or destroyed tanks.
This is know as the German Tank Problem [1]
[1] https://en.wikipedia.org/wiki/German_tank_problem
If you have integer IDs it is also trivial to find authorization flaws on your own. Any pentester will go for it right away.
If you make non guessable IDs they might skip it and go look for other stuff.
I’m a lawyer and using sequential IDs in a fraud case right now, to determine the number of victims.
Unfortunately, so far, I only have the IDs of two victims, and those are from just within about a month, whereas the fraud has likely been going on for several years. Just simply extrapolating that growth rate isn’t going to be very accurate.
Also, I suspect that the perpetrators did not start at ID 1.
that'll be like trying to determine the average salary in a company with only two known ones, which could be the janitor's and the CEO's
Ironically that would be somewhat close to the actual average.
Not chrome...
Also, links are a thing in chat, etc
> https://www.youtube.com/watch?v=fFMzQ3tYTFU&pp=ygURY2hQImVzZ...
Or Twitter:
> https://x.com/elonmusk/status/172853302828286055507?s=20
Or TikTok:
> https://www.tiktok.com/@<userId>/video/730292574259232054785...
While I tend to strip the tracking params and there are extensions that do this, I don't think most people do. These URLs are pretty 'ugly'.
So if the links that are being shared most on the internet (YT, TikTok, Twitter) don't care, you probably shouldn't either. I think the onus is on the UI layers (Chat apps, etc) to show urls how they look best on their respective platforms.
Edit: to this point, it looks like HN truncates these to make them less ugly too.
Don’t know how common that is. Wouldn’t be surprised if no -techies don’t know how to copy from url-bar.
Does anyone have an example they can reference of a business being harmed by this information being out there?
So I guess that kinda harmed that fraudulent business strategy…
Get it wrong and we jump from actionable business metadata to actionable business data (like perhaps which of your customer's customers are poachable)
What's the advantage of uuid7 (sequential + random) vs uuid4 (full random) for this?
There are probably some business cases where the “when” information is potentially useful (I cant think of any) but, you cant know, for example, how many users are in the database.
It can make sense for some situational internal database use case where you want temporal locality and can't use full sequential since it's distributed, and even then your DBMS might recommend something else, e.g. Spanner explicitly says not to do this. And it doesn't need to be exposed to users.
But I do agree, if performance isn't an issue with your db choice and you're not interested in in getting a free "created_at", might as well go fully random.
Instead of giving user an ID in response, user gets hmac(cipher(Data, secret_key), secret_key) + cipher(Data, secret_key) and then some simple pre-request handler just iterates over query params / form data and decrypts them if signature matches.
It also works as a really nice CSRF protection as user ID of currently signed user can be embedded into Data and checked if current user.id == decrypted data.id.
Another nice advantage is that you can deny the request right in the beginning as you know ahead of time that the provided data is not valid (signature doesn't match), saving some DB queries.
The down side is that URL gets pretty long though, but if that's hidden by browser or user doesn't care, it's a non-issue