Tell HN: Every photo in Facebook is somewhat publicly accessible
Facebook generates a static url for each photo you upload. Regardless of what your privacy settings are, anyone can access that photo if he knows url of that photo. I don't think that this should be the ideal behaviour? I tried changing my privacy settings such that photos I am tagged in, should be visible to 'only me', but regardless of this, anyone who knows this url can see this photo. I think this is a big privacy leak issue, or am I missing some point ?
90 comments
[ 3.2 ms ] story [ 140 ms ] threadPeople might leak the URL unintentionally, for instance by sending the link to others, not knowing that the photo owner doesn't want them to see the photo.
One can also use URLs in places where images are not allowed, for example when submitting them to community websites, without first uploading the photo somewhere and leaving a trail.
For every UID discovered, the entropy shrinks and the system becomes easier to attack. Of course, I'm not saying the shrinking entropy would make it feasible to attack the system, just easier. ;)
ASSUMING: The systems are most likely run in virtual machines, and the UID's are most likely the keys for a key-value store. Whether or not the UID are specific to just images, or are also used for other things (wall posts or whatever) is unknown.
The first question is whether or not your key-value store can handle the increase in key length (UID). Since this is fixable, the safe assumption is "yes" albeit there might be undesirable (or even unsurmountable) performance penalties.
A good PRNG is fed by a system entropy pool, and the pool is populated by various system entropy sources. Some believe virtualization can make the system entropy pool more predictable, but some believe virtualization can make the system entropy pool less predictable. It doesn't really matter which is true. The important point is if you can exhaust the system entropy pool, one of two things will happen; (1) the system will get bogged down waiting on the pool, or (2) a poorly written PRNG will start giving you less random numbers.
Of course, "less random" is the most accurate way to phrase it, but in some cases, the actual result is the (faulty) PRNG gives you "predicable" numbers if the system entropy pool is exhausted.
To do it right, you'd have to evaluate the frequency/pressure on the system entropy pool before increasing the key/UID width or you might suffer some highly undesirable effects. There are essentially two places where you might need to add hardware; (1) performance penalties of increased key width in the key-value store, and (2) performance penalties from exhaustion of the system entropy pool (with a well written PRNG). With the former you'd be paying for more servers, and with the latter you'd be paying for more entropy sources to feed the pools. I'm not sure how one would add specialized entropy source hardware in a virtualized environment, but with enough cost and effort, it might be doable (device mapping?). Needless to say, in either case or both, the costs could be prohibitive.
The admittedly "noodley" point here is correctly maintaining a constant threshold level of entropy is occasionally easier said than done. Of course, in other cases, it might be brain dead simple to maintain a constant threshold. The only way to figure it out is proper testing.
The non-power-of-two 136 bit length you mentioned seems interesting. I might be (incorrectly) reading more into it that is actually there, but I believe it hints at a far more cost effective solution, namely, just add more k-v stores. For example, the key/UID could only be 128 bits wide, and the extra eight bits indicate a particular k-v store. With the k-v stores being located "close" to the user, this could also improve bandwidth costs, usability/latency, and load balancing, in addition to the benefits of the CDN.
I don't have a facebook account, so I can't check, but if there's some degree of predictability in a single byte (leading, trailing, middle, ?) of the UID, it would be further support of the above theory of operation. It doesn't need to be perfectly consistent, since at each location there are probably multiple k-v stores.
Even on a collision of a generated key/uid in a particular k-v store, the UID might not be thrown out unless it was already used in _all_ of the local k-v stores. This would further reduce the pressure on the system entropy pool, but due to the need for replication/redundancy/backup, I seriously doubt this is the case.
Though going from 128 bit to 136 bit (or similar) _is_ an increase of the width of the identifier, it probably has nothing to do with maintaining a constant threshold of entropy. It's probably just a smart business decision to reduce hardware/bandwidth co...
Since it can be easily circumvented anyway, disallowing sharing static photo URLs would be the real "pseudo security", in my opinion.
But that's less of an issue with a hidden IMG tag.
- Public
- Anyone with the link
- Private
The "Anyone with the link" option means that Picasa generates a "difficult to guess" URL for the whole album. That's fine and well documented, but the "Private" option might imply that anyone with the link won't be able to access individual photos, which is not the case. I'm afraid users might feel that their photos are hidden as much as emails are in GMail, for example.
Downside: toggling privacy breaks links to the photo.
Upside: toggling privacy breaks links to the photo.
Every photo in Facebook is accessible if you know the its secret photo ID, which is an unguessably large random number.
It's not completely optimal that Facebook embeds secrets in URLs this way (for example, if you browse directly to a photo via its fbcdn URL, you'll have planted the secret in your browser history). But it's a common industry practice, and since it's used on URLs that shouldn't normally end up in your browser history, it's hard to see the major problem with it.
So yes, you're missing a major point, and you haven't found a big privacy leak. Sorry.
* The leaker and the viewers are more traceable, since they're hitting Facebook's servers
* Facebook can cut off access to the photos by reassigning the IDs
* To get the actual link, you have to dig into the Facebook page source; to get the photo, you just have to right-click on it.
This is a stupid, silly threat to worry about. Unless you find a way to predict fbcdn URLs, there's nothing overtly wrong with what Facebook is doing. Plenty of sites rely on the same technique to protect significantly more sensitive information.
The interface says access is now (going forward) changed, but access doesn't change.
What's overtly wrong with what Facebook is doing shows up in practice in the news every time someone becomes an unexpected celebrity. The person promptly and maybe even preemptively changes their privacy settings, but their images remain available.
http://news.bbc.co.uk/2/hi/8060407.stm
Even if it's technically reasonable, it's not user expected behavior.
If you publish images to the public on Facebook, all bets are simply off. It is a bad idea for Facebook to give people the mistaken idea that any settings change on Facebook could ever take back anything posted to "Everyone" on the site.
More to the point, talking about changing privacy settings, and having the change work, where work is not defined as "defeat tptacek" but "defeat casual users".
The logical conclusion to "It is a bad idea for Facebook to give people the mistaken idea that any settings change on Facebook could ever take back anything" is for Facebook to remove any ability to set privacy more restrictive, ever. I doubt that would be popular.
While "you can't take it back" may be technically correct, most photos aren't being downloaded to repost, merely viewed inline online. There's no reason a user changing privacy to be more restrictive shouldn't expect that change to apply going forward.
However, I disagree that "everyone is doing it this way" is a good reason to do it this way.
Flickr, as I noted separately in this thread, doesn't do it this way. Changing privacy settings changes the URL, so someone using the URL (a common use case for images) can't use it any more. That's the "user expected" behavior. Facebook's behavior is a generally unwelcome surprise.
Additionally, since you're considering building image hosting: "token protected" URLs are well understood and often applied as a best effort solution to this problem. A token protected URL typically has an expiry, and in typical use is only valid briefly when issued for the visitor requesting the (authenticated) page containing embedded photos.
See: http://aws.typepad.com/aws/2009/11/new-amazon-cloudfront-fea...
Of course, the CDN edge cache servers have to cache the object yet respect the tokens, which requires a bit more intelligence from the cache, which is why "general industry practice" is to take the easier shortcut of not protecting the URL and image object at all.
Disclaimer: We offer token protected video and image CDN delivery, calling it "deep link protection". Our DLP, and token schemes in general, protect the link from misuse, not the asset.
Point being, S3 solves a different problem than Facebook does.
Customers and clients of CloudFront CDN (which uses S3 storage as its origin) wanted protected URLs, and AWS went to the time and expense to provide them. Content owners large enough to want or need CloudFront edge caching believe there are legitimate business cases for single use, expiring, IP restricted, or other classes of protected URLs for content.
As for Facebook:
Facebook operates web servers generating authenticated and authorized web pages. These pages are dynamic, generated per user, based on current privacy settings. These privacy-managed pages contain links to assets considered, by users, to be just as private as the page.
When the user changes privacy settings for the page, the linked assets privacy could easily be kept in line, as demonstrated by CloudFront CDN being able to support private content links.
Facebook's fault is that the privacy managed page links to public (non-privacy managed) assets, using links that do not respect the containing page's privacy settings.
To say the image shouldn't have privacy settings is to say the page shouldn't have privacy settings "because anyone could save it and repost it". (Which people do, via screenshots.) That's expected and accepted.
But once they change their privacy settings, users believe access permission changes. Access permission does change for the container page, but not the linked assets. That's a broken model.
On their way through the web (unencrypted, mind you), the urls are visible to anyone. Any proxy server can start farming image urls. And what happens if someone reverse-engineers the number generation algorithm?
This is industry practice because it's cheap and the risk for exploitation is low. That doesn't mean it's secure or good. The OP is not missing a point.
As for the rest of it: if you can capture the ID, you can capture the photo. See other comments on this thread for why that is and why it matters.
There are only two unusual factors. The token is contained in the URL instead of a cookie (which actually reduces the obscurity, but has no other effect). Also, the token is per-resource as opposed to per-user (which has both advantages and disadvantages).
On the other hand, who cares... it's also in their browser cache
This no more prohibits revocation than right-click traps do. It was impossible to stop to begin with.
(Of course, I think you understand this, as you mentioned the browser cache. I'm mentioning this mostly for the benefit of later readers)
Those of us having grown up bringing film to the drugstore aren't too concerned about URLs in our ISP's proxy server logs. Nobody cares about your facebook photos enough to risk their job at the ISP to steal them.
I wonder what those privacy settings mean then, if the authorization checks are not happenning when the photo is accessed?
It may be industry wide practice as you have noted but the bottom line is that the privacy settings are not explicitly checked on every photo access. Makes you wonder where else they are using similar logic.
The difference between this and "People of Facebook", the site where you can upload images your friends show you on Facebook, is that Facebook can detect, track, and disable these images, but can't do anything technical about images uploaded to "People of Facebook".
It is just silly to suggest that Facebook should take steps to prevent your friends from copying images. Nothing they do will work, and anything they do will create a false expectation of privacy on behalf of their users.
But in fact you lost control of the content the moment you published it to your friends (or whomever) on Facebook. We're just talking about 2 numbers, one 136 bits long and the other 73,720 bits long. I don't think the difference between these two numbers is worth arguing about.
http://news.ycombinator.com/item?id=490788
Say all urls containing /static/.. don't refer to an actual physical resource but go to a controller which checks the access and then serves the file. As such, the URL if given to someone else will fail as the controller won't authorize the file transfer.
I did it once with nginx at the front and django at the backend using X-Sendfile. I don't have the code around but it was similar to what is proposed in this discussion: http://groups.google.com/group/django-users/browse_thread/th...
Of course, this can't be done for the CDNs.
When you say it's a common industry practice, is the practice there because of CDNs?
It's done this way because running code to authenticate+authorize the user before serving up an image incurs a much higher performance hit than just shoveling the bits directly off a disk. Especially when those bits can come from a content delivery network.
Facebook handles a ridiculously large number of photos - probably an order of magnitude more than any other site - so this trade off, I would guess, is pretty crucial to keeping the site up.
This is definitely not an ideal setup, but it's hard to improve. The only solution I can think of is having an application server serve the pictures after authenticating the request. The picture servers would need to be connected to some sort of database to do that. Serving a single image would incur multiple times the current cost.
Or, make the URL fbcdn.com/p/timestamp/hash of internal salt + timestamp/unique ID.jpg. If the timestamp isn't from within the past 2 minutes, don't serve the image. Almost every static server offers this as a plugin, and it doesn't need to touch a database. AWS offers something similar as a time-limited "access key".
Post to imgur. How would this change be any different? And a better question is, why bother?
But what if your "reward" was not $100 but a photo of an unknown subject. There's a small chance that photo is something you find valuable (newsworthy, embarrassing to the subject, etc.), but there's a far better chance it's a scene from a toddler's birthday party taken on a bad cell phone camera. That's closer to what Facebook is doing. I wouldn't rent a bloodhound for that.
Much faster than trying to figure out URLs.
If the album owner is really locked down you can only see the photo commented on (I think it depends on whether you allow sharing with "friends of friends" or just "friends").
"Tell HN: Every encrypted message is somewhat publicly accessible*
*if you guess the key/password"
It's just not easily discoverable.
I agree, though, that this is mostly a non issue.
Apart from that, I agree with you.
Once it's been transmitted the end user has full control. (Which is why DRM constantly fails)
As one of the other commentors mentioned, the static url was a good way for him to share with his other mates, via chat, pictures of pretty girls.
Taking a screenshot, cropping it, saving it locally, then uploading it to some host, and then finally emailing or IMing the link. That is not a really easy process to share with his mates.
For clarification, my previous comment was directed at statictype's reply to djhworld:
"No amount of privacy code on FB's servers are going to be able to prevent that."
statictype referring to "that" as "They could just copy the photo and email that you."
By displaying the image in Flash, it eliminates the possibility of simply sending a URL, and instead, requires the process mentioned above.
No more sending your mate URLs of pretty girls.
This violates all of my expectations of privacy.
Even if you delete a photo from an album, Facebook doesn't get rid of it immediately. I know this because I was able to retrieve a photo that I'd seen in someone's album, a day after it had been removed from that album, by retrieving the image url in my browser history.
[1] http://arstechnica.com/web/news/2009/07/are-those-photos-rea...
This is pretty well known but I havn't seen it mentioned in this thread yet.