"However, for information security, this redirect should only happen if the content is published, because it can lead (like we see here) to information disclosure."
Or even better: For information security, just don't put information that you don't want to publicly disclose on a public web site. (Or if you do, at least make it accessible only to logged-in users with the proper credentials.)
I did mention another common way to prevent this is to have a content staging site, where all your content is edited; this should ideally be _completely_ separate from the public site. When content is published, it's published to the main site from the staging site.
But in this case, the post seems to be unpublished ("accessible only to logged-in users with the proper credentials"), but the SEO redirect plugin the blog is using is likely hitting it's 302 redirect prior to Wordpress checking it's access on the post ID in question, leading to this URL information disclosure.
> I did mention another common way to prevent this is to have a content staging site, where all your content is edited; this should ideally be _completely_ separate from the public site. When content is published, it's published to the main site from the staging site.
Modern non-enterprise CMSes make this very hard. Wordpress and Drupal mix configuration and content all over their databases, and there's no "sync this content from stage to prod" that works reliably (i.e. they fail as soon as there is a single non-standard plugin loaded).
I haven't evaluated Drupal 8 yet, but I'm not counting my hopes too high.
It's not the redirects that shouldn't happen. It's that if the content isn't yet published, it should not be available.
There are other ways unpublished urls may leak in - for example via sitemaps. Trying to lock them all down is lousy engineering. Instead just password protect the unpublished content and you're done.
Content metadata can be revealing, I reckon that's why the urls in the article say [redacted]. The title of the post gives away information. Even financially significant info in the case of announcements, for instance.
Newbie here: for every one of my projects, I go out hunting for a decent way to do record IDs and still am coming up short. I dislike mightily UUIDs for the user-unfriendly lengths. My preference would be for something like airline codes or YouTube IDs: 6-10 alphanumeric IDs. Should I just be generating a hash and actually storing it in the DB after checking for collisions? Or is there a way to encrypt/decrypt to a specific length with a specific set of usable characters?
You can simply use the database integer identity column and convert it into a series of alpha numeric characters. With a little effort, you can generate randomized, unique values. I've written a library that does exactly that, https://github.com/mikecao/hashkit.
A base 30 time stamp will give you about the right number of characters, case-insensitive, and without the possibly-ambiguous 0, 1, O, I, L (lower-case ambiguous) and Q (possibly upper-case ambiguous) using 23456789ABCDEFGHJKMNPRSTUVWXYZ as valid characters. Adding back the 0 and 1 (that is, anybody listening to a user on a support call, or any code, would automatically interpret I or L as 1 and O or Q as 0) gives you base 32 and a shorter ID code. If milliseconds are going to be too collision-prone (that is, you can't recuperate the time you'll occasionally have to advance), a short entity code (user or server name abbreviation, f'rinstance) prepended with a dash should make it memorable and user-friendly.
Oddly, nobody ever seems to worry about that with hashes, encryption output or GUIDs... but if it's a worry, then, sure, hey, why not. (By the way, that system has been in regular use for more than 20 years in commercial software.)
Easiest way is simply XOR the database auto-increment ID with a fixed number, then truncate to (AND to) however many bits seem reasonable for your site.
Not super secure, but good enough for this purpose.
Unless you have a long ID people can always simply try every ID, no matter what format you use. So fancy encryption on the ID isn't all that useful.
If security is important then generate a random 64bit number and store that, rather than let the database auto-generate a number.
> Easiest way is simply XOR the database auto-increment ID with a fixed number, then truncate to (AND to) however many bits seem reasonable for your site.
XOR has a trivial known-plaintext attack, and you don't even need to determine the entire key to get something useful. If you can create a few posts in order and obtain a series of sequential items, you can obtain the low-order bits of the number XORed in, at which point you can easily browse around for adjacent items from any post.
Just generate a random string (make sure to use the proper generator, e.g. https://gist.github.com/dchest/751fd00ee417c947c252). If the result is shorter than, say 100 bits, you'll have to check for collisions (e.g. unique constraint in database).
Alternatively, if you have 64-bit sequences as IDs (e.g. autoincremented ids in SQL database), you can encrypt then using a real 64-bit block cipher, such as Blowfish or XTEA, with a secret key known only to server. This can give you unsigned 64-bit numbers, which are usable in existing systems (except in JavaScript, where maximum integer has 53 bits), for example:
1 -> 3491820675944448233
2 -> 9264994595240464886
If you use base64 encoding, it will give you 11-character strings, e.g. "0AdUgU8OZfo". Make sure to use URL-safe encoding, though (instead of +, / it uses -, _).
>Compared to Base64, the following similar-looking letters are omitted: 0 (zero), O (capital o), I (capital i) and l (lower case L) as well as the non-alphanumeric characters + (plus) and / (slash).
Many people, when faced with "unwanted information disclosure" via ID enumeration, will look to encryption to solve this problem. Please don't; it's the wrong tool for the job.
I played around with my wordpress install. It looks like if a post is ever "published", the ?p=id url will always forward to the permalink url regardless of the post status being changed to "draft" or "pending review". Edge case for sure, but interesting to know.
That's not an edge case. It's a bona fide bug. Back when pretty URL wasn't a thing yet the logic behind the controller should never redirects you to an hidden/draft post.
It's a shame that the author frames this "issue" in such a negative light. Remember when this used to be the most valuable feature of the Web?
Remember "The URL is the new command line?"
That was back in the the early days of Web 2.0, back when people used that term with a straight face and even a bit of optimism, and where you might also use the term "Mashup" in the same sentence. Having consistent URL formats for Posts, Users, Dishwasher Models and everything else made it so that you could pull information out of web pages quickly and easily without the need for a formal API.
It was a pretty cool time to be writing software because you could quickly suck in data from tons of places in ways that they hadn't specifically anticipated. But it died just as fast as it rose because of SEO.
Over the span of a single year, everybody had to change their urls to look like http://site.org/blog/learn-rails-in-20-minutes and suddenly you couldn't do anything useful with them again.
Shame.
But yeah, at least whatever edge case this author talks about isn't there anymore. To mitigate that, I'd recommend Jason's Law of the Internet: "Try to avoid putting things on the Internet that you don't want to be on the Internet. That way there won't be anything you put on the Internet that you didn't want to be there."
When I did Edgeio, we did what you describe. While we had human readable slugs, there was an item id at the start of every url, and if you stripped off or changed the rest, you just got the same page but with a <link ...> back to the canonical version.
We also built the entire thing basically as an API that just happened to by default pass through a processing step to render HTML. If you gave the right argument, you'd get the page back as our own XML markup [1], RSS or ATOM instead. We didn't go full HATEOAS [1] (didn't exist yet), but we there was a decent amount of descriptive links etc. pointing the way in the XML/RSS/ATOM versions.
[1] That was actually what the web app generated; it was converted server side using XSL, but if you "flipped the right switch" it'd serve up the XML with an XSL stylesheet and it mostly worked fine that way in most browsers, though we never invested the resources in making it flawless (but it was a great debugging tool to get the raw XML served up straight to the browser... everyone came out hating XSL though, for good reasons)
I'm fine with IDs in URLs (either redirected to a pretty URL or not); I'm suggesting that you simply make sure that unpublished content doesn't get redirected to the pretty URL so you can avoid this kind of information leakage.
If you're using nginx, I'm fairly sure you can do an 'internal' directive so that `?id=12345` style URLs won't work when you try typing them in a browser, but nginx can still rewrite the URLs to map slugs to IDs.
I haven't tried it with WordPress, but I did do something similar on a custom CMS.
That said, there are better approaches. You can use a plugin to set publish dates in the future, and before that the page will return a 404 (unless you're logged in).
I spotted this in August 2014 and reported it via Automattic's responsible disclosure process (bug number #25376). They acknowledged it quite quickly and patched the hosted Wordpress.com accounts within a few weeks, and said they'd submitted a report for it to be fixed upstream, but then went silent and didn't respond to any more requests for updates. A year and a half later they awarded a $100 bug bounty with no other information. I still don't know if it's been patched in the open source Wordpress release.
One reason I started using UUIDs for database IDs is in case I make an utterly incompetent mistake and leave a URL-hacking security hole somewhere, you still would need to guess a ridiculous ID to get at anything useful.
EDIT: I realize UUIDs are not "secure random" tokens and this isn't sufficient for real security, it's just a nice to have encumbrance if I screw up something I should've made secure in the first place. It's no substitute for doing your job right in the first place. :)
Not if they are RFC 4122 UUIDs (more accurately, they are insufficiently cryptographically secure): even the random ones are only 122 bits, which only provide 61 bits of security.
UUIDs are still really, really useful. I don't recommend not using them unless one understands why and when not to.
Why would that be? You can fork your PRNG as many times as you have threads in initialization, so there's no need for IO. And secure ones do spend many times more CPU than the cheapest PRNG available, both would be a rounding error on the performance on nearly all real world work-loads.
I'd support removing non-secure random functions from every standard library out there.
We used UUIDs for every media stream in our media engine. Had to create about 15 per second during prime-time conference hours. It was 40% of our CPU load! So they aren't free.
Python's uuid4 is cryptographically secure, as far as I know. One UUID needs ~16 random bytes, my laptop's /dev/urandom gives about 14 MB/s (user-space PRNG can be much faster if needed).
To be cryptographically secure, it must be unguessable. That's a higher bar than random, because many random number generators are guessable (i.e. completely predictable).
54 comments
[ 2.5 ms ] story [ 123 ms ] threadOr even better: For information security, just don't put information that you don't want to publicly disclose on a public web site. (Or if you do, at least make it accessible only to logged-in users with the proper credentials.)
But in this case, the post seems to be unpublished ("accessible only to logged-in users with the proper credentials"), but the SEO redirect plugin the blog is using is likely hitting it's 302 redirect prior to Wordpress checking it's access on the post ID in question, leading to this URL information disclosure.
Modern non-enterprise CMSes make this very hard. Wordpress and Drupal mix configuration and content all over their databases, and there's no "sync this content from stage to prod" that works reliably (i.e. they fail as soon as there is a single non-standard plugin loaded).
I haven't evaluated Drupal 8 yet, but I'm not counting my hopes too high.
It's a lot easier in 8 since it has an API-driven design for content entities.
There are other ways unpublished urls may leak in - for example via sitemaps. Trying to lock them all down is lousy engineering. Instead just password protect the unpublished content and you're done.
https://en.wikipedia.org/wiki/Format-preserving_encryption
http://hashids.org/#why-hashids
Is there a general name for this pattern? I'd like to find some of the other libraries for doing the same thing.
Not super secure, but good enough for this purpose.
Unless you have a long ID people can always simply try every ID, no matter what format you use. So fancy encryption on the ID isn't all that useful.
If security is important then generate a random 64bit number and store that, rather than let the database auto-generate a number.
XOR has a trivial known-plaintext attack, and you don't even need to determine the entire key to get something useful. If you can create a few posts in order and obtain a series of sequential items, you can obtain the low-order bits of the number XORed in, at which point you can easily browse around for adjacent items from any post.
Alternatively, if you have 64-bit sequences as IDs (e.g. autoincremented ids in SQL database), you can encrypt then using a real 64-bit block cipher, such as Blowfish or XTEA, with a secret key known only to server. This can give you unsigned 64-bit numbers, which are usable in existing systems (except in JavaScript, where maximum integer has 53 bits), for example:
If you use base64 encoding, it will give you 11-character strings, e.g. "0AdUgU8OZfo". Make sure to use URL-safe encoding, though (instead of +, / it uses -, _).>Compared to Base64, the following similar-looking letters are omitted: 0 (zero), O (capital o), I (capital i) and l (lower case L) as well as the non-alphanumeric characters + (plus) and / (slash).
[0]: https://en.wikipedia.org/wiki/Base58
Many people, when faced with "unwanted information disclosure" via ID enumeration, will look to encryption to solve this problem. Please don't; it's the wrong tool for the job.
https://paragonie.com/blog/2015/09/comprehensive-guide-url-p...
Use a random lookup instead of encryption. (You'll notice that https://paragonie.com/b/oMFJhGJ0aSgCaZq0 takes you to the same URL.)
Use application logic (and access controls) instead of random lookups, where it makes sense to do so.
If you absolutely must use encryption (i.e. you don't have a place to store state), make sure you use an AEAD mode.
https://gist.github.com/tqbf/be58d2d39690c3b366ad
Remember "The URL is the new command line?"
That was back in the the early days of Web 2.0, back when people used that term with a straight face and even a bit of optimism, and where you might also use the term "Mashup" in the same sentence. Having consistent URL formats for Posts, Users, Dishwasher Models and everything else made it so that you could pull information out of web pages quickly and easily without the need for a formal API.
It was a pretty cool time to be writing software because you could quickly suck in data from tons of places in ways that they hadn't specifically anticipated. But it died just as fast as it rose because of SEO.
Over the span of a single year, everybody had to change their urls to look like http://site.org/blog/learn-rails-in-20-minutes and suddenly you couldn't do anything useful with them again.
Shame.
But yeah, at least whatever edge case this author talks about isn't there anymore. To mitigate that, I'd recommend Jason's Law of the Internet: "Try to avoid putting things on the Internet that you don't want to be on the Internet. That way there won't be anything you put on the Internet that you didn't want to be there."
We also built the entire thing basically as an API that just happened to by default pass through a processing step to render HTML. If you gave the right argument, you'd get the page back as our own XML markup [1], RSS or ATOM instead. We didn't go full HATEOAS [1] (didn't exist yet), but we there was a decent amount of descriptive links etc. pointing the way in the XML/RSS/ATOM versions.
[1] That was actually what the web app generated; it was converted server side using XSL, but if you "flipped the right switch" it'd serve up the XML with an XSL stylesheet and it mostly worked fine that way in most browsers, though we never invested the resources in making it flawless (but it was a great debugging tool to get the raw XML served up straight to the browser... everyone came out hating XSL though, for good reasons)
[2] https://en.wikipedia.org/wiki/HATEOAS
I haven't tried it with WordPress, but I did do something similar on a custom CMS.
That said, there are better approaches. You can use a plugin to set publish dates in the future, and before that the page will return a 404 (unless you're logged in).
I don't understand. What is it spoiling?
EDIT: I realize UUIDs are not "secure random" tokens and this isn't sufficient for real security, it's just a nice to have encumbrance if I screw up something I should've made secure in the first place. It's no substitute for doing your job right in the first place. :)
I support UUIDs instead of nearly any other id space. They solve collision problems completely. Its time to stop managing ids.
Not if they are RFC 4122 UUIDs (more accurately, they are insufficiently cryptographically secure): even the random ones are only 122 bits, which only provide 61 bits of security.
UUIDs are still really, really useful. I don't recommend not using them unless one understands why and when not to.
Why would that be? You can fork your PRNG as many times as you have threads in initialization, so there's no need for IO. And secure ones do spend many times more CPU than the cheapest PRNG available, both would be a rounding error on the performance on nearly all real world work-loads.
I'd support removing non-secure random functions from every standard library out there.
Ah, Java...
The library you are using for creating those UUIDs must have some bug. My guess is they share the same generator for all threads on each machine.
It is a cryptographically secure PRNG, but during a small time in system start up, it may not be correctly seeded.
If you need long term keys, it may be better to get some 256 bits from /dev/random before using /dev/urandom.
http://sockpuppet.org/blog/2014/02/25/safely-generate-random...