If another service flops in the next few months, then yes I agree, the house of cards is collapsing, but right now that statement is a little premature.
Broken links have been happening since the web began. Any time you've made any web request, there's been a SPOF(1). In fact, think back to 2002-- remember when a good number of links you'd click were dead? When was the last time in recent memory you've had that happen? In fact, I argue that the web has a lower percentage of broken links today than back in the day. In 5 years, we'll likely have even less.
(1) Yeah yeah, I know, a drastic over simplification. There's Foundry LB hardware, round-robin DNS at different sites, BGP failover, etc.
broken links in the past were the result of actions of the creator of the content, however, in the case of url shorteners, the broken links are pointing to content that still exists, in the same place it was when the link was created.
There's a big difference betwen that an the broken links since the web began...
This is a problem google can solve pretty easily, isn't it? They've indexed both sides of the link... so they can somehow 'heal' the broken link - right?
> Any time you've made any web request, there's been a SPOF
And when you make a request through a shortener, there are two independent SPOF's.
Also, if you go to www.example.com/here/there/article and get a 404, you navigate up in the hierachy, and search for the content. If you get a 500 or server is down, you can try later. If either happens at the shortener-level, you're SOL.
The argument against URL shorteners is specious, and taken to its logical conclusion is an argument against anyone ever linking to anything: any URL might turn up gone in the future, and you have no way of knowing or predicting which ones will last. Amazingly, the web's managed to survive this.
But if you're genuinely concerned that this poses some sort of threat to the web, start an archiving service which preserves mappings of short -> long URLs, or talk to the archive.org folks to see if they'd be interested.
That idea in the second paragraph of your comment is good. Someone ought to do that.
As far as the first, though, I think that you've got it wrong. It's not the same as any old site breaking. Say a person, over time, links to eight sites in his forum comments, and one goes down. Seven links are still valid. If he uses a shortener service that collapses, none of his links are valid.
Say a person links to a series of articles, and the site which hosted them goes down. Now all the links are invalid, and this is nothing more than what we deal with every day.
The alarmist rants I've seen against URL shorteners are simply making this argument without realizing that there is no "new" problem in need of solving, only an old one that we already handle fairly well. Though given the level of irrational rhetoric I've been seeing in these discussions (example here: http://news.ycombinator.com/item?id=713998) I'm not sure there's any point trying to argue with it.
Not only that, but if it's an "original" URL that's broken but at least well formatted you at least have a hint of where to look, for example in the Google cache, archive.org, or elsewhere on the original domain.
Shortened URLs give you no hints as to where the original document was located.
Dude, you keep saying that "it's the same problem we've always had", but it simply isn't. You seem like a smart guy otherwise, so I don't know why you persist in making this assertion. The difference is simple: url shorteners introduce a middle man, another point of failure or abuse, and one that can't be fixed by the person who made the link (because they may forget the original target and not know who to contact) or the person who manages the target (because they can't set up a redirect from the old url).
Adding a link to a chain, any chain will make a it weaker:
Either the link is as strong as all the other links then you have a new element that can fail, which is bad, or your link is weaker than the other links in the existing chain and now the whole chain is weakened to the strength of the new link.
(it runs on Google App Engine, so I expect to keep it up as long as GAE exists. I didn't get TLD for it, to keep maintainance/costs at zero. And it memcaches everything, so feel free to query it as much as you like)
So let's promote the URL shorteners that are open and provided as free services, instead of those who try and sell click information.
http://ur1.ca is free software and provides a text format download of all its mappings right on its home page. Not a fan of shorteners, but that's the one I'd use for posting services with char length restrictions.
Straw man fallacy. A is like B and B is bad. Try again.
The argument is that it makes things MORE fragile, by interposing another party into the resolution chain. Given that it is both easy to do (thus there will be lots of competitors) and hard to profit at means that this party is more than usually vulnerable.
Anyway if the content dissapears, then it is gone. A url shortner failing means the failure could occur even if the content is still available.
Therefore, by definition, it is worse than the other situation.
You're presuming that URL shorteners don't provide any benefits that may justify the cost of a more fragile system. E.g., bit.ly gives you nice analytics, and many links are otherwise impossible to pass via Twitter.
Your argument against shorteners (extra indirection -> more fragile -> bad) works just as well against DNS. The reason why we tolerate the extra indirection of DNS is because there's a substantial benefit to be gained.
DNS servers co-operate and the system has lots of built-in redundancy, whereas URL shortners have none of these properties. It's not just the indirection that makes them fragile.
Probably not. The only place they're really warranted is on Twitter, so if Twitter did it themselves that would go a long way. There is the whole URL masking annoyance as well, so ideally Twitter would support long URLs, but that might not happen because of the 140 character SMS restriction.
A federated protocol seems a little much for this one service which can handle the issue themselves, IMO.
Twitter should really, really provide a separate field for URLs. This would solve this whole thing.
What is the point of an SMS with a shortened URL if you have no internet access? To be able to access the web page you need web access. If you have web access then you can get at the longer URL in the web interface or via Twitteriffic etc.
The inability to pass (some) links via twitter is the fault of twitter (or the "features" thereof), a fault that doesn't suddenly disappear because someone else provides what amounts to a workaround.
It's taken to a conclusion but not a logical one. A further conclusion based on that 'logic' would be that nothing at all should be made as anything might turn up gone in the future heat death of the universe.
Summary of the summary -
URL shorteners have poor usability, providing little, often zero value at the cost of adding further centralized points of fragility.
"3x recommended dose of Vitamin D slashes cancer risk by 77%: http://is.gd/28Nbq
is better than this:
"3x recommended dose of Vitamin D slashes cancer risk by 77%: <a href="url">2005 Academic Study</a>"
I can see why you'd URL shorten when pushing to SMS, but an actual anchor/href gives me the ability to mouse-over and see the target URL and the link isn't dependent on a 3rd party to work. No one can frameset the url without the linkers permission or shove ads into the experience.
I wonder how they could increase SMS limits past 160 chars?
That can't be done unless you change the underlying cellular protocol (e.g. the upcoming 3GPP LTE protocols). The length of an SMS is based on the capacity of one GSM frame.
OTOH, they could be used as failsafes. If a normal link dies, tinyurl or whatever could go to a page that says,
"This link seems to be missing. Do you know where it moved to?" The responses could be published as long as the 404 continue. I have to think someone has already thought of this.
hopefully bit.ly's api will expand to cover stuff you could do with tr.im, like aggregate user statistics-- seen in dave winer's 40 twits app (http://dave.40twits.com/)
First, I don't like URL shorteners for this same reason. Every web server should be in charge of managing long and short urls easily with just one line of code.
Second, how expensive it is to keep the site running? $10 a month for a cheap hosting service and they can make more than that with just google ads.
Now, if they want to live largely just by shortening urls then no, it won't pay the bills.
1) Url shorteners aren't going away. Like it or not, there's a need, particularly for things like Twitter.
2) I think it's more than that. They did more than just shorten urls, they also tracked analytics on those links, clickthrough rates and the like. You can view charts and graphs about how often links are clicked on, etc. Multiply that by millions of links, and it's not an inconsequential amount of storage and bandwidth. I think you way overestimate the capabilities of a $10 hosting package.
All the $10 package would have to do is forward the user, though. It wouldn't have to do all that stuff.
That said, it probably isn't quite a $10 package. On the other hand, it probably wouldn't take much, especially if you cared to take the time to write a custom-purpose HTTP-redirection server.
Actually, that would sort of be an interesting programming challenge: How small and efficient can you get such a server?
But if that's all it's doing, where is its differentiation between it and the hundreds of other URL shortening services.
You CAN run a social network cheaply, but good luck competing with Facebook / MySpace / Orkut / Friendfeed / Livejournal / et al.
Offering the service with the least amount of features can sometimes be a winning idea, but in this space, I'm not entirely sure that it's doable, considering the major feature works pretty similarly across all tr.im's competition.
"But if that's all it's doing, where is its differentiation between it and the hundreds of other URL shortening services."
Nothing. The design specification was just to "keep the site running", not to support a given feature set, for the purpose of not breaking the links, and nothing else. An extra buck or two from Google would be a bonus, but personally I wouldn't count on it.
On the other hand, it probably wouldn't take much, especially if you cared to take the time to write a custom-purpose HTTP-redirection server.
You must consider the opportunity cost. How long does it take to code such custom server ? As tr.im is not profitable, its developers' time could be used to produce other product/service that has a better business model and is more profitable.
I occasionally find them useful outside of Twitter. In my experience, some applications (email clients, etc) don't handle/parse really long links (Google Maps links or ugly eBay auction URLs, for example) very well, so giving someone an is.gd link to a hideously long and complicated URL is usually easier for me.
How many $10/mo accounts do you know of that would give you the server resources and bandwidth needed to scale an application like this? Granted the resource utilization overhead is very small compared to many other apps, but with even a smidge of popularity, this is not a $10/mo hosting app.
Since my comment has been modded down to 0, surely someone has more information about tr.im and can post some examples of $10/mo hosting sites that could run it.
If you remove the statistics and the url submission form, so that only the redirection is kept alive (= no broken links) I think a $10/mo account should be sufficient.
If you only want to do redirection, no statistics stuff, then pretty cheap. If they don't want submission of new links, even cheaper, they could even run it on shared hosting.
Why not just put a URL shortener on the SMS interface, where it doesn't matter much if the shortener service dies anyway, and leave the original URL on the web interface.
I think there is good behind short urls but they should all be same domain. Relying on a third party for links embedded everywhere will now be worthless for users of tr.im.
Short urls are good but also readable urls are better. The whole short url thing stems from twitter and is being originally based on SMS. But it is breaking the web when this happens and not knowing what you are clicking on is a security problem.
I could see a case for a corporate service for short urls like mycompany.tr.im/YHu8 or just mycompany.com/YHu8 etc. Then it redirects to a longer url within the same company. But it needs to be guaranteed to be there as long as the linked to endpoint exists.
I wonder how easy it would be to create an archive.org equivalent for URL shorteners. Any shortener could pay a fee (based on the number of links they had) in order to join, in exchange for which they'd get some form of certification that they were members. The archiving service would be a nonprofit with publicly viewable finances, and would make copies of partial or full link databases available for further download and backup.
If it caught on at all, it would do two things well: 1. it would mean that there were few good reasons to use an un-certified link shortener, and 2. it would make the transition to a post-shortener world much simpler and more graceful: twitter could just download the full data set and repeal the character limit for URLs.
A large reason of why URL shorteners are hard to maintain is the incredible amount of spam they receive. At Slicehost we seem to get spamcop emails every day about yet another URL shortener someone has created because the spammers will use their link instead of the real URL.
I get a lot of value from bitly's analytics. Also, with their pushing clients like AIR apps, iphone apps, and email readers to include referrers, your site's analytics are going to get a lot better.
URL shortening done wrong sucks. Don't use anything besides those that are really serious. Right now, bitly is the best option.
What other tools can show you how a link spreads through a social network? That's why it is valuable - it is like analytics that Twitter / facebook etc have but don't expose
If they were trying to sell the whole package, including the domain name, which is required for the service to continue supporting all the existing links, and no one would buy it even "for a token amount", then why would the domain name alone go for more than that?
Because the domain doesn't cost anything to maintain. The service has hosting and bandwidth costs, which must be relatively substantial for a service of their scale.
Startups die for many reasons, but I can't help but feel the ones tr.im gives are a little dishonest.
There is no way for us to monetize URL shortening --
users won't pay for it
Users won't pay for web search either, and that has monetized very well.
We just can't justify further devleopment since Twitter
has all but annointed [sic] bit.ly the market winner.
Umm, when tr.im got into this business, Twitter had "anointed" tinyurl the "market winner". If it was worth it to compete against tinyurl regardless, why isn't it worth it to compete against bit.ly?
When they got into the business, Twitter wasn't such a big deal, though. (Personally I believe Twitter will go back to not being a big deal in a few years, but so would URL shortening in general)
In know that in the past TinyURL has adopted the domain and databases for failed shorteners in the interest of preventing link rot. Maybe Tr.im should toss an email to Gilby?
This could be damaging to Twitter since it will raise a lot questions about the viability of short urls. It may force them to either buy bit.ly or start their own service.
> this required significant development investment and server expansion to accommodate. tr.im has thousands and thousands of users, creating tens of thousands of URLs per day
10K urls per day x 1000 hits for statistics = 10M hits per day, each of which is a read (duh) but also a write to keep a hit count. That's 120 reads + 120 writes per second.
Most of the distributed nosql options can easily handle that on a single node, and the non-distributed ones generally have even higher one-node throughput.
Not to be an armchair architect, but url mapping is pretty simple. Sounds like they were using a bad tool for the job if they were having to throw hardware at it.
An avg of 240 dbIO/s is entirely different from peak time IO/s. When someone like Jeff Atwood posts a link using their service, they could easily see 10x the volume. Volume might be really stagnant for 45 minutes, then bam, 10 percent of their req's for the hour come in within a 2-3 minute span. Judging by their blog-post, I'm assuming they needed to have the infrastructure to handle that kinda load.
Either way, I modded you up for a pretty thorough analysis (and because I largely agree with you (it basically is a mapped 301 Location: http://newsite/)). From my armchair standpoint(1), a 256 mb slice + Redis + nginx should be able to handle 10x the volume easy.
(1) Well not quite armchair, a good 30 percent of my firm's business is directly related to scaling.
Given that they're shutting down, why isn't Tr.im simply releasing a complete copy of their URL mappings?
There's no need for some industry-wide URL shortener consortium to archive links. Just give out your mappings if you happen to go out of business, so someone could decode your URLs manually if they really needed to.
There are sure to be some private (unpublished) URIs in there, some of which may contain sensitive information. (Users shouldn't submit such URIs to third-party shorteners, but of course they do anyway.)
Edited to add: Of course, those would also be picked up by groups like archiveteam.org that are brute-force crawling the shortener URI space.
116 comments
[ 4.4 ms ] story [ 160 ms ] threadLet's hope it crumbles fast and we can rid the web of these shorteners before they cause more damage to the future of the internet.
If another service flops in the next few months, then yes I agree, the house of cards is collapsing, but right now that statement is a little premature.
... I'm sure we'll survive.
(1) Yeah yeah, I know, a drastic over simplification. There's Foundry LB hardware, round-robin DNS at different sites, BGP failover, etc.
There's a big difference betwen that an the broken links since the web began...
And when you make a request through a shortener, there are two independent SPOF's.
Also, if you go to www.example.com/here/there/article and get a 404, you navigate up in the hierachy, and search for the content. If you get a 500 or server is down, you can try later. If either happens at the shortener-level, you're SOL.
But if you're genuinely concerned that this poses some sort of threat to the web, start an archiving service which preserves mappings of short -> long URLs, or talk to the archive.org folks to see if they'd be interested.
As far as the first, though, I think that you've got it wrong. It's not the same as any old site breaking. Say a person, over time, links to eight sites in his forum comments, and one goes down. Seven links are still valid. If he uses a shortener service that collapses, none of his links are valid.
The alarmist rants I've seen against URL shorteners are simply making this argument without realizing that there is no "new" problem in need of solving, only an old one that we already handle fairly well. Though given the level of irrational rhetoric I've been seeing in these discussions (example here: http://news.ycombinator.com/item?id=713998) I'm not sure there's any point trying to argue with it.
Shortened URLs give you no hints as to where the original document was located.
Either the link is as strong as all the other links then you have a new element that can fail, which is bad, or your link is weaker than the other links in the existing chain and now the whole chain is weakened to the strength of the new link.
http://therealurl.appspot.com/
(it runs on Google App Engine, so I expect to keep it up as long as GAE exists. I didn't get TLD for it, to keep maintainance/costs at zero. And it memcaches everything, so feel free to query it as much as you like)
http://ur1.ca is free software and provides a text format download of all its mappings right on its home page. Not a fan of shorteners, but that's the one I'd use for posting services with char length restrictions.
Conclusion, as an independent business it has very little chance of being viable.
The argument is that it makes things MORE fragile, by interposing another party into the resolution chain. Given that it is both easy to do (thus there will be lots of competitors) and hard to profit at means that this party is more than usually vulnerable.
Anyway if the content dissapears, then it is gone. A url shortner failing means the failure could occur even if the content is still available.
Therefore, by definition, it is worse than the other situation.
Your argument against shorteners (extra indirection -> more fragile -> bad) works just as well against DNS. The reason why we tolerate the extra indirection of DNS is because there's a substantial benefit to be gained.
A federated protocol seems a little much for this one service which can handle the issue themselves, IMO.
Another good reason to have them is to be able to spell them for somebody over a phone line.
What is the point of an SMS with a shortened URL if you have no internet access? To be able to access the web page you need web access. If you have web access then you can get at the longer URL in the web interface or via Twitteriffic etc.
Joshua Schachter has a nice summary of the arguments against that you may have seen before: http://joshua.schachter.org/2009/04/on-url-shorteners.html
Summary of the summary - URL shorteners have poor usability, providing little, often zero value at the cost of adding further centralized points of fragility.
Please explain why this (my last tweet):
"3x recommended dose of Vitamin D slashes cancer risk by 77%: http://is.gd/28Nbq
is better than this:
"3x recommended dose of Vitamin D slashes cancer risk by 77%: <a href="url">2005 Academic Study</a>"
I can see why you'd URL shorten when pushing to SMS, but an actual anchor/href gives me the ability to mouse-over and see the target URL and the link isn't dependent on a 3rd party to work. No one can frameset the url without the linkers permission or shove ads into the experience.
I find the shorteners an annoyance. You can't fix the dependency problem.
I wonder how they could increase SMS limits past 160 chars?
That can't be done unless you change the underlying cellular protocol (e.g. the upcoming 3GPP LTE protocols). The length of an SMS is based on the capacity of one GSM frame.
"This link seems to be missing. Do you know where it moved to?" The responses could be published as long as the 404 continue. I have to think someone has already thought of this.
First, I don't like URL shorteners for this same reason. Every web server should be in charge of managing long and short urls easily with just one line of code.
Second, how expensive it is to keep the site running? $10 a month for a cheap hosting service and they can make more than that with just google ads.
Now, if they want to live largely just by shortening urls then no, it won't pay the bills.
1) Url shorteners aren't going away. Like it or not, there's a need, particularly for things like Twitter.
2) I think it's more than that. They did more than just shorten urls, they also tracked analytics on those links, clickthrough rates and the like. You can view charts and graphs about how often links are clicked on, etc. Multiply that by millions of links, and it's not an inconsequential amount of storage and bandwidth. I think you way overestimate the capabilities of a $10 hosting package.
That said, it probably isn't quite a $10 package. On the other hand, it probably wouldn't take much, especially if you cared to take the time to write a custom-purpose HTTP-redirection server.
Actually, that would sort of be an interesting programming challenge: How small and efficient can you get such a server?
You CAN run a social network cheaply, but good luck competing with Facebook / MySpace / Orkut / Friendfeed / Livejournal / et al.
Offering the service with the least amount of features can sometimes be a winning idea, but in this space, I'm not entirely sure that it's doable, considering the major feature works pretty similarly across all tr.im's competition.
Nothing. The design specification was just to "keep the site running", not to support a given feature set, for the purpose of not breaking the links, and nothing else. An extra buck or two from Google would be a bonus, but personally I wouldn't count on it.
You must consider the opportunity cost. How long does it take to code such custom server ? As tr.im is not profitable, its developers' time could be used to produce other product/service that has a better business model and is more profitable.
Since my comment has been modded down to 0, surely someone has more information about tr.im and can post some examples of $10/mo hosting sites that could run it.
With many of these url shortners going out of business pretty soon now, anybody willing to try and make this a business opportunity?
Can you think of reasonable and serious reason to shorten another shortener?
Short urls are good but also readable urls are better. The whole short url thing stems from twitter and is being originally based on SMS. But it is breaking the web when this happens and not knowing what you are clicking on is a security problem.
I could see a case for a corporate service for short urls like mycompany.tr.im/YHu8 or just mycompany.com/YHu8 etc. Then it redirects to a longer url within the same company. But it needs to be guaranteed to be there as long as the linked to endpoint exists.
If it caught on at all, it would do two things well: 1. it would mean that there were few good reasons to use an un-certified link shortener, and 2. it would make the transition to a post-shortener world much simpler and more graceful: twitter could just download the full data set and repeal the character limit for URLs.
That one website (twitter) has a post limit of 140 characters doesn't mean we should spread the stupidity to everywhere else.
URL shortening done wrong sucks. Don't use anything besides those that are really serious. Right now, bitly is the best option.
Not trolling. This is a serious question.
Tell that to excite, altavista, yahoo, etc...
Please don't make up your own titles, especially when they don't match the content.
I'm sure they would get something for it, I myself would be extremely if the price was right.
That said there seems to be the same air of fatalism in both their attempts to sell the service and the reasons for giving it up.
> this required significant development investment and server expansion to accommodate. tr.im has thousands and thousands of users, creating tens of thousands of URLs per day
10K urls per day x 1000 hits for statistics = 10M hits per day, each of which is a read (duh) but also a write to keep a hit count. That's 120 reads + 120 writes per second.
Most of the distributed nosql options can easily handle that on a single node, and the non-distributed ones generally have even higher one-node throughput.
Not to be an armchair architect, but url mapping is pretty simple. Sounds like they were using a bad tool for the job if they were having to throw hardware at it.
Either way, I modded you up for a pretty thorough analysis (and because I largely agree with you (it basically is a mapped 301 Location: http://newsite/)). From my armchair standpoint(1), a 256 mb slice + Redis + nginx should be able to handle 10x the volume easy.
(1) Well not quite armchair, a good 30 percent of my firm's business is directly related to scaling.
Your user name didn't hint at that at all.
There's no need for some industry-wide URL shortener consortium to archive links. Just give out your mappings if you happen to go out of business, so someone could decode your URLs manually if they really needed to.
Edited to add: Of course, those would also be picked up by groups like archiveteam.org that are brute-force crawling the shortener URI space.