Each shortcode is 6 characters and is case insensitive. That gives us roughly 1.8 billion (1838265625) combinations.
That's only 1.8 billion combinations if you only have computers talking privately to other computers. But in the scenario outlined in the article, people will see the URL, so you must filter millions of those combinations.
For example, if I come across a recipe for shoo-fly pie, I don't want to forward my grandmother a shortened url like example.com/FuckUG. More importantly, you don't want someone posting a screenshot of your expletive shortcode on social media, or worse.
When wetware is involved, even in just one step, things get messy.
Are you under the impression that there aren't many, many services which only serve english-speaking populaces, by design, or that even including the offensive words in every language would come close to significantly impacting the entropy involved here?
Or that people won't try to find something offensive in a random string... I had a reg plate containing FY...BLX. A friend commented that it spells "fuck you blacks" - didn't even think of reading it like that for years.
Every number substitution for a letter doubles the number of words in the language. Every vowel elimination doubles them again. The internet won't tell me the average number of vowels per word, but lets assume it's 2^3, so you're over a million. Not every word has an A->4 substitution, or a t->7 or an e->, so it's not 2^10th but let's say perhaps another factor of 16.
Do you remember those FCUK shirts? Letter transposition is probably another doubling, maybe quadrupling. Bringing the total to at least 171,476,000 'words'.
And while that's only 6% of the space, now you have to deal with 171 million blacklist entries. That's a bit of a problem.
I recall sitting with a coworker once hashing out a shortcode/slug algorithm for this, that you could also transcribe. First we threw out all the letter like characters, so no l, no 1, no 0 or O. Then we threw out the vowels to reduce the accidental swear words and slurs.
26 + 26 + 10 + 2 is how you get base-64. We took out the symbols too and ended up with an odd base-32, which is still pretty useful practically, as it’s 7 vs 6 characters for 32 bits, 13 vs 11 for 64.
64 - symbols (2) vowels (10, 12 if you include y) - 0,l,1 (3) is still 47 characters, so we removed the rest of the numbers to avoid leetspeak, leaving 39 and don’t recall what we did for the last 7. I think we may have just chopped 7 more off the end of alphabet to hit 32.
Today I’d probably try two things to reduce word-alikes. One option, remove the most common characters (either wheel of fortune NSTRE, which is eight due to previous eliminations, or Morse code characters AEINMT which is 6 for the same reason). Second option, remove the letters used in slurs to break them, let people deal with rude words, though KNT (see? That was on accident) breaks up quite a few of both.
6 characters is pretty trivial to brute force. Someone could set up a script to continuously try all combinations and get access to whatever happens to hit a match.
You could block an IP after a certain number of failures, but that doesn't protect against a network of various IPs attacking (which attackers often have access to). Adding an artificial delay also wouldn't protect against parallel attacks.
A simpler/better solution would be to add a few more characters to the shortcode so it's infeasible to force within your timeframe (and the number of requests your servers can handle in that timeframe).
If you have a network, 1 IP could then try 1 time a user + password. Then switch to another device with a different IP for another one.
The first IP could then try a different user.
You need a combination, not one or the other.
And then, you have to deal with a university or other big institution and users forgetting their passwords after the holidays. That would potentially trigger a ban of everyone.
These days few, if any, such URLs will in practice be typed in much less spoken aloud; they will overwhelmingly be clicked or at copy/pasted. So shortness is probably not a problem. In addition, suppressing obvious typos (lowercase L/digit 1) doesn’t really matter either.
I've wanted to build a similar system for supporting preview of articles on a CMS. Problem is, users may share multiple URLs with a user so a single JWT token in a cookie doesn't suit that flow very well. Each new link followed needs to extend an existing value with additional proofs (and remove old ones?)
23 comments
[ 3.7 ms ] story [ 67.8 ms ] threadThat's only 1.8 billion combinations if you only have computers talking privately to other computers. But in the scenario outlined in the article, people will see the URL, so you must filter millions of those combinations.
For example, if I come across a recipe for shoo-fly pie, I don't want to forward my grandmother a shortened url like example.com/FuckUG. More importantly, you don't want someone posting a screenshot of your expletive shortcode on social media, or worse.
When wetware is involved, even in just one step, things get messy.
36^6=2,176,782,336
Approximate number of words in English: 171,476
Do you remember those FCUK shirts? Letter transposition is probably another doubling, maybe quadrupling. Bringing the total to at least 171,476,000 'words'.
And while that's only 6% of the space, now you have to deal with 171 million blacklist entries. That's a bit of a problem.
26 + 26 + 10 + 2 is how you get base-64. We took out the symbols too and ended up with an odd base-32, which is still pretty useful practically, as it’s 7 vs 6 characters for 32 bits, 13 vs 11 for 64.
64 - symbols (2) vowels (10, 12 if you include y) - 0,l,1 (3) is still 47 characters, so we removed the rest of the numbers to avoid leetspeak, leaving 39 and don’t recall what we did for the last 7. I think we may have just chopped 7 more off the end of alphabet to hit 32.
Today I’d probably try two things to reduce word-alikes. One option, remove the most common characters (either wheel of fortune NSTRE, which is eight due to previous eliminations, or Morse code characters AEINMT which is 6 for the same reason). Second option, remove the letters used in slurs to break them, let people deal with rude words, though KNT (see? That was on accident) breaks up quite a few of both.
Maybe the site showing you how to make shoo-fly pie has a shorter domain, like `shoof.ly/12345`, which resolves to the exact thing.
No need to use URL shortening as a service unless you want to give other people your data.
(See auto-shortened sharing urls: yt.be for YouTube, fb.me for Facebook)
They don't allow [c, s, f, h, u, i, t] to be placed next to each other and instead move those to be separators.
You could block an IP after a certain number of failures, but that doesn't protect against a network of various IPs attacking (which attackers often have access to). Adding an artificial delay also wouldn't protect against parallel attacks.
A simpler/better solution would be to add a few more characters to the shortcode so it's infeasible to force within your timeframe (and the number of requests your servers can handle in that timeframe).
If you have a network, 1 IP could then try 1 time a user + password. Then switch to another device with a different IP for another one.
The first IP could then try a different user.
You need a combination, not one or the other.
And then, you have to deal with a university or other big institution and users forgetting their passwords after the holidays. That would potentially trigger a ban of everyone.
My understanding of your use case is stopping indefinite "exponential" forwarding.
I wonder if the ValidUntil timer could start when the shortlink is first accessed.
That way you can shorten the time it lasts in the wild and let it be indefinitely available for the first intended recipient.