This feels like a hostage situation again. The last time they said they were going away they also prevented people from downloading their images. I'm having a hard time understanding that (granted S3 'out' bandwidth costs money but still.)
Name the next plausible explanation and how likely you think it is relative to 'save money, screw your data'. I think you'll have takers on a bet at those odds, given what you just said.
Err, if they're on S3, isn't data transfer to EC2 free? [1] If I'm reading the pricing chart right, why would it cost Twitpic anything for someone else to spin up some EC2 instances and clone their data thataway?
Part of it is that almost every service you interact with on the internet is a "startup". It's not reasonable to expect everybody and their (literal) grandmother to know who's got an investor-friendly P/E.
You are really in a bubble. I can't think of any start-ups's I regularly interact with. Now older companies that might or might not be profitable sure, but 'start up's' almost by definition don't support large chunks of the worlds population.
Either way, I wonder if the twitpic guys fully understand the damage they may be doing to their personal reputations. Do they expect to be able to run a web service in the future? Because if the current situation persists, no one will ever trust them again.
If we're being honest, the vast majority of their users don't know who was behind Twitpic. The HN crowd is probably a lot more likely to know/care than your typical Twitpic user.
They'll probably be able to start new things without most people noticing. And they probably know this.
Well, a few things. First off, a lot of people on here probably understand that when you don't have money to pay for bandwidth, banning people who use a lot of it might be reasonable. It's not like you have a whole lot of other options. Passing around the hat isn't likely to work, and if they could get additional funding, they already would have.
Second, even if people didn't understand that, the HN crowd is a tiny slice of the good hackers out there. Most of the best people I know do not read it, or even actively disdain it. "Cesspit" and the like are a terms I have heard in person or on newsgroups used to describe the comments section here.
Third, I'll bet most HN people don't even know the names of Twitpic's founders. I don't. I wouldn't be able to identify whether another company, down the line, was founded by them.
Lastly, I doubt most HN people are so principled that a year or two from now they would turn down an otherwise compelling job offer because the founders of the company were once associated with banning Archive Team's IPs. It's mildly annoying at worst.
If someone thinks hacker news is a cesspit of comments on the internet, then I can only imagine that someone hasn't really spent any time on the internet. It's not always sunshine and happiness here, but HN is definitely among the top sites in terms of comment quality, especially considering the broad variety of topics covered here. You really only end up with better quality comments by going to sites that only cater to special interest communities.
Now if only there was a global infrastructure where those people could communicate this. Maybe it would only take 140 characters. Then I bet, those people could use it to warn others.
But that's the beauty of hiding behind a company name. Only the people who have their ear to the ground during this fiasco will know not to trust the founders when their next company is named FaceFlick.
Unfortunately, that doesn't seem to be the case where it's most needed. For example with systemd, after having written the awful pulseaudio, you'd think Lennart would not be able throw any weight around even at Red Hat.
There's the chance that their next venture (if there is one) would have a lot of vocal criticism referencing how they handled this situation if there were similar risks with the new one.
This could impact a launch significantly, but of course there's no guarantee the masses would notice.
TwitPic has always done things like this (such as preventing users from mass-exporting their own photos) but everyone forgets once the news cycle moves on.
What's often missed with Hanlon's razor is that people want to believe something is malicious vs just plain stupidity. You need a succinct expression like this to help repudiate people's inclinations.
System overloaded because of poor DevOps? Absolutely possible. But not talking to Jason about exporting a dump to the Internet Archive? That's "take your ball and go home" stupidity.
"Take your ball and go home" isn't always stupid. Perhaps more stupid is to entrust a random website to store all of your pictures indefinitely, especially when that website has no clear revenue model. Don't put yourself in a situation where data will be lost by someone taking his ball back.
We very well could be having the same situation in a few years when Twitter is in the process of turning off the servers.
If you want to keep your data, make backups. Don't trust cloud providers to be there. If you're the Internet Archive, make backups/dumps/archives before people start turning the servers off.
It's possible they've made a last ditch deal to sell the media to someone (perhaps even Twitter). That would make it important to keep the data private.
"you let the visual component of arguably the greatest communication shift/revolution in the 21st century be under the whim of one idiot". 800 million pictures, just gone. For example https://twitter.com/textfiles/status/523160725158903808
Assuming ~120TB [1], that would cost ~$1300 in AWS Export charges, plus whatever the drives cost. Just sent a tweet to Noah offering to cover all AWS export/drive costs to export the archive.
Yes. This was discussed at length with the Twitpic founder. He said something to the effect of: "don't worry, I'm working on a deal, trust me. But stop downloading the pics"
Yeah, it looks like the API to twitter is up, although the API and even API documentation has been taken down everywhere else. If you click on the link in that tweet, it doesn't show the photo.
"t" is a Base64-encoded blob of data containing a few miscellanous bytes, as well as the URL the proxy should download (in this case, twitpic.com/show/large/...). "s" is the signature, used so we can't just abuse the proxy to download more.
Let's not pretend that they didn't make money off the ads they placed on their websites. Yes it was "free" but they weren't doing this out of the goodness of their hearts. Also any goodwill they may have gained in their heyday is all but used up now with this whole shutting down, not shutting down, shutting down, fuck you and your pictures situation.
Paying for hosting? They were making money from ads they put next to our pictures. They were using our content to drive traffic to their site. Without the user's photos, the site is worthless. They should have been paying users a cut of advertising revenue like YouTube does. "Free" means they're pocketing the profit for themselves.
If they had profits, maybe they wouldn't be closing down. Those ads were to pay bills and they are broke. Allowing massive bandwidth usage now is just throwing bad debt after bad.
There is no indication that Twitpic is broke. They announced their shutdown after Twitter objected to their attempt to trademark their name (as any company might do, if another business applied for a trademark using a large chunk of their name in a similar context.)
So, the profits aren't enough to warrant changing names? The profits aren't enough to be worthy of a buyout - or even letting others foot the bill for backups. Not to mention there's someone out there who is dealing with multiple levels of suck right now.
It might be a better path for people to reach out to help the guy WITHOUT offering to offload the value proposition of what he built.
Also, any sort of agreements between people who uploaded content and Twitpic can't just be sold off to the first person to show up with a hard drive or offer to pay bandwidth costs. That would be a set of legal costs, etc. etc.
This is absurd. They were providing a service that users deemed valuable, and users gave them access to the content (and the ability to monetize that content) in exchange for their service. You don't actually believe Facebook should pay you for using their free service do you?
In fairness, the archive team is attempting to download 120TB of data.. which is going to cost Twitpic about $15K in outgoing data charges from Amazon.
Maybe they just don't have $15K to pay for this. (and that's not counting all of the users who are also downloading their data)
I have absolutely no doubt that people would be willing to donate to cover the archival costs, especially if that donation was overseen by the archive team. Options like this can only be explored if there is a conversation.
Yes but you have that conversation before you start archiving. If I started racking up $1000's of dollars in server fees while going bust I'd shut off the archive team too. Ask and offer to pay first.
Maybe he's busy with other pressing matters...like all of the aspects involved with his business shutting down and how that affects him and his employees?
Perhaps if he had chosen to shutter his business in a more orderly fashion, and made arrangements to take care of the significant quantity of visual contemporary history of which that business was the defacto caretaker, it would be easier to forgive being "busy".
We're talking about 800 MILLION pictures here. Noah is apparently too busy to save them, but not so busy that he can't find the time to actively prevent others from saving them. That's pretty hard to support.
Entitlement really doesn't seem to be the best way to think about it. They don't owe you or anyone any type of end result, and like most topics I see discussed here when I've been involved with a couple of them behind the scenes: there is a lot more going on than you think, and issues are not simply black or white.
When you set up a public service, you do have a moral obligation not to abuse your users, as well as not to abuse the public good. There's an implicit contract with your users to do right by them, especially with a service that has, no joke, a ton of legitimate, important historical record to it.
When you abuse the public trust, you get flak for it. That's a feature, not a bug.
Yes, things that explain why they refuse to allow people to pay for the archive in a sensible way, and seem to be intending to just destroy people's data instead in what can only be described as ostensibly spite or apathy to their users.
No wonder they failed at business if this is how they act during hard times.
Yes, maybe he doesn't care about preserving user data. That's kind of the point...
I have serious doubts that here have been any actual Twitpic employees for a while. If there were, that seems unwise of both them and Everett. Twitter announced the doom of third-party photo sites in late 2010. Twitpic basically went silent a couple years ago. Meanwhile, Everett says he's been working on Pingly for over a year[0]. Note the timing of that article -- not even a month before the announcement of Twitpic shutting down.
No matter how much he wants to try to make this about a trademark dispute, it's clearly not some sudden collapse. By all appearances, he's moved on to other things and has finally realized it's time to shut down an old project that's been on a long and probably inevitable downward spiral. This is understandable, failing to preserve data is not.
Remember, any cloud service you use may shut down and delete all your data tomorrow, whether through bankruptcy, acqui-shutdown, accident, or vandalism. If it's a free service you will have no comeback. If it's a paid service you probably won't have a comeback either.
Free service yes, but if you're paying for SaaS and rely on it for your business you should really get a source code escrow agreement in place. That will allow you to get the necessary components from the provider in case they go belly-up or such.
Of course hosting the application is a whole different thing, but hey, at least you got all your data available.
Seeing as they're shutting down the company/service in a week, they're probably out of cash. While it sucks for your customers, that's what happens when you run out of money. It sucks for everyone. I'm guessing they were floating the company on personal funds in hopes the acquisition would go through, but it did not - so now they're doing everything they can to stop this turning into a personal financial bloodbath from the bandwidth costs spiral due to everyone rushing to get their photos before the shutdown.
Dustin Curtis: "Can anyone put me in touch with the founder of Twitpic?(I'll pay to backup all of the photos, and even host them. Ridiculous situation.)"
Why are people worrying about backups? Certainly people had no expectation of recoverability of photos hosted on a free service? Or was TwitPic promising that?
I realize there's some harshness to it, but you make implicit promises when you set up a public service. Twitpic chose to make themselves an important part of the Twitter ecosystem, and they have a moral obligation to not be assholes when they need to close up shop.
Not getting with Jason and the rest of archiveteam very much falls under "assholes" in my book, especially when other people are offering to pay for the data you're happy to destroy.
I may be speaking out of turn but wouldn't this be an object lesson of 'if you like something, pay for it'? Going out of business hurts everyone and frankly I think it's OK for customers to share some of that pain. I honestly am not familiar with their revenue model so perhaps that is not apropos though.
I do see the sadness of all that bitrot though so I understand your point of view.
Because he's not in the data recovery business. And who are you to tell him how to run his life and his career path?
What moron has been using their service for years and for free without a contingency plan? What idiot/geek/nerd in today's world doesn't keep triplicate backups of important data? Hell, I keep encrypted backups at home AND in the cloud/offsite. Just because you failed at LIFE and protecting your assets does not mean the next person is being petty or owes you a favor.
> And who are you to tell him how to run his life and his career path?
A member of the public who, collectively, trusted him with an irreplaceable part of the historical record. You take on that responsibility in the pursuit of profit, you don't get to discharge it when it goes under.
I've never uploaded anything to Twitpic. But parts of our history are there, and that's important. Blowing off fine folks like Archive Team/dcurtis/toomuchtodo when they're not even asking you to pay for the retrieval of those historical archives is simply antisocial.
I didn't have any data with Twitpic, I just think his attitude shows a clear lack of good business sense, and I'm here for the public spectacle.
That being said, I'll publicly call out displays of bad behavior when they happen in public - and throwing away your customer's data out of spite when they offer to pay for it to be backed up is bad business and incredibly petty behavior.
the company is going under and they are most likely out of cash. They don't want to go into debt/get stuck with a huge bill.
Justin.tv did the same thing. They closed up shop in pretty much 24 hours.
A business isn't a charity to make you feel good. Sometimes you need to make tough decisions like this.
They provided a free service for many years. Be happy that someone put the time and effort into giving you that. I also don't know why some people decide to use it as a backup service, because clearly it isn't.
..and the guy trying to archive all of the customer data? Why would any business hand over all of their Dara like that? It's a lawsuit waiting to happen.
This is the problem with demanding everything is free/cheap: the service doesn't last.
Yeah, I think they just don't want or can't pay for the bandwidth. Or, perhaps they are trying to make Twitter feel bad and buy them. Either way it's a bad situation, but isn't necessarily evil on their part to not want to put anything more into a dying venture.
For those downvoting this - why don't you try paying for 800M images to be leached from the server of a failed company of yours with no hope of recouping a single dollar.
Any such cost would be several orders of magnitude smaller than the millions of dollars of advertising revenue those images have been pulling in for twitpic over the years.
Let's not pretend that Noah is down to his last packet of ramen noodles, here. Twitpic is a long way from being a "failed company" by any reasonable definition of the term.
Random people from the Internet offering to pay a bill likely to be in the low five figures. That sounds like it would work out well. "I sent him a copy of the bill and he's not responding. I don't understand.....he said he would pay. He even gave me his Twitter handle!"
It's more about having a moral obligation to history. Twitpic has been a home for hundreds of millions of images for a good chunk of a decade during which the most monumental shifts in communication and expression have occurred. Unique and irreplaceable moments have been captured in this place. To turn around and delete it all just because you didn't get the trademark application you wanted (and apparently can't be bothered to just think of a new name instead) is nothing other than an unprecedented act of wanton, petulant vandalism against history.
It would be like the world's biggest art gallery turning around and saying 'Oh, sorry, they won't let us call our gallery McDonalds, so we're shutting down and burning all your pictures. After all, easy come, easy go.'
Okay, but we keep getting burned by these irreplaceable droves of stuff going offline (see GeoCities), and all our indignation doesn't bring them back.
At some point we have to learn this lesson for good, and stop trusting random entities to be good stewards of things we find meaningful.
As I understand it, GeoCities actually cooperated to some extent with the backup team that was trying to preserve the content, and didn't take steps to keep people from backing up content.
GeoCities was entirely unmaintained; there was apparently only one person at Yahoo who was paid to do anything with it at all, and who couldn't even answer questions such as how many users there actually were.
The primary pain point when spidering it was that each GeoCities user had a tiny hourly bandwidth cap; to do the job right the spiders had to keep track of the error responses that they got so that they could go back later in the hope of getting the real content rather than the error message.
"We" aren't the problem. The problem is that startups need non-technical users to use their stuff. People who aren't qualified to, nor remotely interested in, managing a tech stack simply are not going to care about how random that entity is. You can't make them care. It falls to awesome people like Jason and the rest of Archive Team to keep that sort of historical record for posterity. Because it's important. Trying to thwart them, as Noah Everett is doing, is just straight-up not good.
The Internet Archive should archive stuff before a company enters financial hardship and can longer afford the outgoing bandwidth costs to have their entire dataset replicated online. It makes sense to block those attempts.
It strikes me that a similar fate would most likely have occurred if Everett was hit by a bus and his bank accounts eventually ran dry.
We should identify historically significant single points-of-failure and make archives before the failures occur, insofar as is possible.
Sure, but it's important to note that I'm not talking about blocking the scrapers. I'm talking about "we're willing to pay for an S3 export, Amazon will mail out a bunch of hard drives." Everett has steadfastly avoided contacting Archive Team or anyone else who wants to do this and I can't think of an ethically not-icky reason for that.
The not-icky reason is that he doesn't want to. He's not under any obligation to do it. Maybe we'd like it if he did this, but I don't think it's fair to assume bad or even just "ethically icky" motives.
If all of twitpic's data is exported or copied elsewhere, he can no longer effectively profit off of it. We don't know that his intention is to just nuke the data and be done, he may be trying to salvage what he can monetarily (and the "we're getting acquired, just kidding" thing is probably good evidence that something like this is going on behind the scenes). I don't think there's anything wrong with this either. It's his service, and if his motive was profit, it makes perfect sense to shop the data around instead of just accepting a total loss. There's no reason twitpic should have to be run for altruistic purposes.
Maybe dcurtis et al would have better luck getting a response if instead of saying "We'll cover the bandwidth costs", they said "We'll buy the data from you at a price commensurate with the years of effort and upkeep you've invested into an apparently massively important historical archive". I don't think it's fair to pretend like Noah Everett is obligated to sell his archive for the cost of bandwidth.
In any case, at this point he's made it clear that he wants to wash his hands of the project in one way or another. It's his personal project and he is and should be free to do that. This is the worst possible time to try to do a bulk archive.
I understand that we can't see into the future and can't always predict when something like this is going to go down, but I'd think we can probably improve our heuristics so that something as seemingly significant as twitpic can't just vanish into the ether next time.
Eight hundred million photos and all their comments. That's historical record. That's our historical record. No one jerk gets to destroy all of that. The "hurr, well, don't use it" is completely bonkers--I don't use it, but my history is on there, too. It's a record I didn't put there but matters to me as a citizen and as a member of society. Destroying a major part of the historical record for everybody is indefensibly shitty, full-stop, you don't get to say "well it's mine" when it's not.
It's ours. Twitpic was allowed to profit by holding onto it awhile. Now it's time to do the right thing.
It is his. This is what happens in an ethereal format like the internet, and it's something we'll have to adapt to -- in the history of mass publication, one wasn't able to just destroy all copies of a thing with the push of a button. Books and pamphlets could be hunted down, but almost always some would survive, because you can't remotely change the contents of the paper. In the digital age, you can. If there's a lesson to take from this, it's the dangers of the non-permanence of the web; we should be looking at solutions that can mitigate that danger. But I don't blame Everett at all for not just giving away his company's IP for $15k in bandwidth bills, and I don't believe he should be shamed into doing so.
If I were him, I'd probably be contacting Sotheby's or another auctioneer that specializes in high-value goods and seeing about selling off the dataset to the highest bidder. Then these fellows can put their money where their mouth is and show us just how important the preservation of that data is.
This celebration of plutocracy gone wild is one of the more disgusting things I've read on Hacker News, and that's saying a lot. His corporation only exists through our societal forebearance. He has an obligation to the society that gave him the legal structure and protections thereof. Part of that is not deleting history just because he's in a snit.
b. If it's worth so much, people should be willing to pay a fair price for it. The implication that he should let it all go for a payment to Amazon is, frankly, insulting, and I'm sure you'd be singing a different tune if it was your own startup that you were trying to liquidate.
c. Talk is cheap. It's easy to try to shame someone into letting you take their intellectual property. It's not plutocracy, it's just business.
d. Getting a lot of users doesn't require your company to convert into a charity. I am 100% sure that Noah Everett has a price, and I'm sure if people actually valued the Twitpic corpus he'd be able to get it.
e. All uses of Twitpic were voluntary and done with the implicit acknowledgement that the images and comments may go away one day. If someone was not OK with this, they should've saved their data, and many probably did. Blocking the Internet Archive from devaluing the dataset is not bad, wrong, or ethically icky in any way. The larger ethical crime lies at the feet of the people trying to convince Everett that he doesn't deserve fair compensation for curating this historically significant dataset.
f. If no one is willing to pay the money to preserve the dataset, it's obviously not as culturally significant as you believe it is. There's plenty of money out there.
This whole thing is a fiasco. But maybe if it wasn't Twitpic wouldn't get as much exposure.
My bet is that this is just a giant media whoring play. If you want to save the company data then you have to be transparent and ASK for help. If it is going to cost $15K someone might very well step up. But if you are tightlipped then people will assume the worst.
I saw this and thought "eh, that doesn't bother me, I already used the 'export your photos' thing." So I went to my TwitPic settings page, clicked the link, and downloaded a 14MB zip file.
The only thing is, it's empty. I don't know how you get a 14MB zip file that's empty, but Windows is telling me it's empty. That's not cool.
14:23 <REDACTED> you have to enable the "show hidden folders" in windows, under their view options
14:24 <REDACTED> each version of windows is slightly different but it is usually press alt, select tools, folder options, view, and the radio next to show hidden files and folders and
then ok
14:24 <REDACTED> then you close and reopen the zip file
14:30 <REDACTED> oh, and sometimes the first download is corrupt.
14:30 <REDACTED> I've had to help too many people at work with this problem plus a person or two on this channel
What's the big deal about these pictures? It's not the first time a major service has shut down. I am assuming historically important images have been saved and hosted elsewhere.
i agree. what's so important about foodie pics, selfies, and baby pics you should have backed up already. Trust me, nobody will notice if Twitpic disappears overnight. Stop putting so much importance in such a trivial company.
Major headache for Twicsy! We now have to delete about a billion pictures from our index. Which isn't trivial... such a shame. We reached out to him several times over the past few weeks to see if we could arrange something, but he never responded.
Dustin Curtis: "Can anyone put me in touch with the founder of Twitpic?(I'll pay to backup all of the photos, and even host them. Ridiculous situation.)"
The Archive Team only backs things up when there is a risk of them being gone.
They cannot do it any other way - the Internet Archive does not have infinite resources to host things, and they need to prioritize. 120TB is not an easy thing to host.
That's what the Internet Archive tries to do. But they have finite (or rather, extremely limited) resources. The Internet is vast things fall through the cracks.
The Archive Team is the "oh shit this site is shutting down and it's not all in the Internet Archive already, we gotta save it before the deadline when it's gone forever" squad.
I reckon the feature that is missing is not the ability to pick and choose, but instead to automatically assign people trying to help preserve something to one chunk, with the idea of getting full coverage of the entire data set.
If there are 100 pieces, and 500 people, you want each piece to be held by 5 people for 5x redundancy. And, if nodes leave the swarm, the system automatically rebalances which nodes are maintaining a particular piece of the dataset.
The only solution being worked on out there that does this is IPFS
165 comments
[ 3.0 ms ] story [ 202 ms ] threadI can't imagine they'll want to spin up new machines, so it makes sense to throttle/limit/block.
Not sure I'd immediately jump to malice here.
I'm not saying that it isn't malicious. I just don't think it's wise to jump to that conclusion straight away.
https://twitter.com/textfiles/status/523162680400830464
[1] http://aws.amazon.com/s3/pricing/
Edit: While at the same time twitpic had no announcement on the frontpage and users were still uploading: https://twitter.com/textfiles/status/522835537543974913
http://twitpic.com/account/settings
I wish I could say this will be a good lesson in not trusting startups with personal data, but it won't be.
However, as this is the internet, until news breaks otherwise I will continue to believe that Hanlon's razor applies here...
Never attribute to malice that which is adequately explained by stupidity.
They'll probably be able to start new things without most people noticing. And they probably know this.
Second, even if people didn't understand that, the HN crowd is a tiny slice of the good hackers out there. Most of the best people I know do not read it, or even actively disdain it. "Cesspit" and the like are a terms I have heard in person or on newsgroups used to describe the comments section here.
Third, I'll bet most HN people don't even know the names of Twitpic's founders. I don't. I wouldn't be able to identify whether another company, down the line, was founded by them.
Lastly, I doubt most HN people are so principled that a year or two from now they would turn down an otherwise compelling job offer because the founders of the company were once associated with banning Archive Team's IPs. It's mildly annoying at worst.
This could impact a launch significantly, but of course there's no guarantee the masses would notice.
Anyone boycotting twitch.tv? Anyone????
We very well could be having the same situation in a few years when Twitter is in the process of turning off the servers.
If you want to keep your data, make backups. Don't trust cloud providers to be there. If you're the Internet Archive, make backups/dumps/archives before people start turning the servers off.
[1] https://twitter.com/textfiles/status/523162867789758464
"t" is a Base64-encoded blob of data containing a few miscellanous bytes, as well as the URL the proxy should download (in this case, twitpic.com/show/large/...). "s" is the signature, used so we can't just abuse the proxy to download more.
http://blog.twitpic.com/2014/09/twitpic-is-shutting-down/
It might be a better path for people to reach out to help the guy WITHOUT offering to offload the value proposition of what he built.
Also, any sort of agreements between people who uploaded content and Twitpic can't just be sold off to the first person to show up with a hard drive or offer to pay bandwidth costs. That would be a set of legal costs, etc. etc.
"Archive Team has 900 IPs aimed at Twitpic. We're grabbing 100 pictures a second."
He subsequently corrected this figure to 42,000 photos per hour, which works out to about 11/sec.
Later he adds:
"In some cases, twitpic (before removing all the image access) was banning entire ISPs to stop archive team backing it up."
Maybe they just don't have $15K to pay for this. (and that's not counting all of the users who are also downloading their data)
People have been trying to talk to him since the day the announcement was made.
The cost for an export would be 1/10th that, and would be covered by others, if only Noah Everett were cooperating.
We're talking about 800 MILLION pictures here. Noah is apparently too busy to save them, but not so busy that he can't find the time to actively prevent others from saving them. That's pretty hard to support.
When you abuse the public trust, you get flak for it. That's a feature, not a bug.
No wonder they failed at business if this is how they act during hard times.
I have serious doubts that here have been any actual Twitpic employees for a while. If there were, that seems unwise of both them and Everett. Twitter announced the doom of third-party photo sites in late 2010. Twitpic basically went silent a couple years ago. Meanwhile, Everett says he's been working on Pingly for over a year[0]. Note the timing of that article -- not even a month before the announcement of Twitpic shutting down.
No matter how much he wants to try to make this about a trademark dispute, it's clearly not some sudden collapse. By all appearances, he's moved on to other things and has finally realized it's time to shut down an old project that's been on a long and probably inevitable downward spiral. This is understandable, failing to preserve data is not.
[0] http://www.postandcourier.com/article/20140810/PC0313/140819...
I note, by the way, that the arguments have degenerated from reasoned to "but rights!!!". You're going to have to do better than that.
Of course hosting the application is a whole different thing, but hey, at least you got all your data available.
Just an ugly situation for everyone.
Edit: On second thought, I agree that it's probably related to the bandwidth cost.
I wonder if the Archive Team has considered sending twitpic a hard drive and asking them to just rsync the data over?
Edit: This assumes that they keep backups of said data outside of EC2. Walking a hard drive over to your local Amazon DC.. won't quite work.
Dustin Curtis: "Can anyone put me in touch with the founder of Twitpic?(I'll pay to backup all of the photos, and even host them. Ridiculous situation.)"
Not getting with Jason and the rest of archiveteam very much falls under "assholes" in my book, especially when other people are offering to pay for the data you're happy to destroy.
I do see the sadness of all that bitrot though so I understand your point of view.
That's the epitome of petty spite.
What moron has been using their service for years and for free without a contingency plan? What idiot/geek/nerd in today's world doesn't keep triplicate backups of important data? Hell, I keep encrypted backups at home AND in the cloud/offsite. Just because you failed at LIFE and protecting your assets does not mean the next person is being petty or owes you a favor.
A member of the public who, collectively, trusted him with an irreplaceable part of the historical record. You take on that responsibility in the pursuit of profit, you don't get to discharge it when it goes under.
I've never uploaded anything to Twitpic. But parts of our history are there, and that's important. Blowing off fine folks like Archive Team/dcurtis/toomuchtodo when they're not even asking you to pay for the retrieval of those historical archives is simply antisocial.
You're misusing the word "discharge." You actually want him to discharge his responsibility in this case.
That being said, I'll publicly call out displays of bad behavior when they happen in public - and throwing away your customer's data out of spite when they offer to pay for it to be backed up is bad business and incredibly petty behavior.
the company is going under and they are most likely out of cash. They don't want to go into debt/get stuck with a huge bill.
Justin.tv did the same thing. They closed up shop in pretty much 24 hours.
A business isn't a charity to make you feel good. Sometimes you need to make tough decisions like this.
They provided a free service for many years. Be happy that someone put the time and effort into giving you that. I also don't know why some people decide to use it as a backup service, because clearly it isn't.
..and the guy trying to archive all of the customer data? Why would any business hand over all of their Dara like that? It's a lawsuit waiting to happen.
This is the problem with demanding everything is free/cheap: the service doesn't last.
Two separate people have offered to cover the bandwidth and other costs of transferring the files to some other place.
https://twitter.com/textfiles/status/523509876560257025
I wouldn't just transfer pictures and customer data to the highest bidder either.
Do I need to keep mentioning Justin.tv? It seems nobody listened to me in the first place (or use conveniently ignored the facts).
It's obvious most people here have never run a business and wouldn't make the right decision.
But I put being decent above making money. And, hopefully, I always will.
Let's not pretend that Noah is down to his last packet of ramen noodles, here. Twitpic is a long way from being a "failed company" by any reasonable definition of the term.
It would be like the world's biggest art gallery turning around and saying 'Oh, sorry, they won't let us call our gallery McDonalds, so we're shutting down and burning all your pictures. After all, easy come, easy go.'
At some point we have to learn this lesson for good, and stop trusting random entities to be good stewards of things we find meaningful.
The primary pain point when spidering it was that each GeoCities user had a tiny hourly bandwidth cap; to do the job right the spiders had to keep track of the error responses that they got so that they could go back later in the hope of getting the real content rather than the error message.
It strikes me that a similar fate would most likely have occurred if Everett was hit by a bus and his bank accounts eventually ran dry.
We should identify historically significant single points-of-failure and make archives before the failures occur, insofar as is possible.
If all of twitpic's data is exported or copied elsewhere, he can no longer effectively profit off of it. We don't know that his intention is to just nuke the data and be done, he may be trying to salvage what he can monetarily (and the "we're getting acquired, just kidding" thing is probably good evidence that something like this is going on behind the scenes). I don't think there's anything wrong with this either. It's his service, and if his motive was profit, it makes perfect sense to shop the data around instead of just accepting a total loss. There's no reason twitpic should have to be run for altruistic purposes.
Maybe dcurtis et al would have better luck getting a response if instead of saying "We'll cover the bandwidth costs", they said "We'll buy the data from you at a price commensurate with the years of effort and upkeep you've invested into an apparently massively important historical archive". I don't think it's fair to pretend like Noah Everett is obligated to sell his archive for the cost of bandwidth.
In any case, at this point he's made it clear that he wants to wash his hands of the project in one way or another. It's his personal project and he is and should be free to do that. This is the worst possible time to try to do a bulk archive.
I understand that we can't see into the future and can't always predict when something like this is going to go down, but I'd think we can probably improve our heuristics so that something as seemingly significant as twitpic can't just vanish into the ether next time.
Eight hundred million photos and all their comments. That's historical record. That's our historical record. No one jerk gets to destroy all of that. The "hurr, well, don't use it" is completely bonkers--I don't use it, but my history is on there, too. It's a record I didn't put there but matters to me as a citizen and as a member of society. Destroying a major part of the historical record for everybody is indefensibly shitty, full-stop, you don't get to say "well it's mine" when it's not.
It's ours. Twitpic was allowed to profit by holding onto it awhile. Now it's time to do the right thing.
If I were him, I'd probably be contacting Sotheby's or another auctioneer that specializes in high-value goods and seeing about selling off the dataset to the highest bidder. Then these fellows can put their money where their mouth is and show us just how important the preservation of that data is.
b. If it's worth so much, people should be willing to pay a fair price for it. The implication that he should let it all go for a payment to Amazon is, frankly, insulting, and I'm sure you'd be singing a different tune if it was your own startup that you were trying to liquidate.
c. Talk is cheap. It's easy to try to shame someone into letting you take their intellectual property. It's not plutocracy, it's just business.
d. Getting a lot of users doesn't require your company to convert into a charity. I am 100% sure that Noah Everett has a price, and I'm sure if people actually valued the Twitpic corpus he'd be able to get it.
e. All uses of Twitpic were voluntary and done with the implicit acknowledgement that the images and comments may go away one day. If someone was not OK with this, they should've saved their data, and many probably did. Blocking the Internet Archive from devaluing the dataset is not bad, wrong, or ethically icky in any way. The larger ethical crime lies at the feet of the people trying to convince Everett that he doesn't deserve fair compensation for curating this historically significant dataset.
f. If no one is willing to pay the money to preserve the dataset, it's obviously not as culturally significant as you believe it is. There's plenty of money out there.
My bet is that this is just a giant media whoring play. If you want to save the company data then you have to be transparent and ASK for help. If it is going to cost $15K someone might very well step up. But if you are tightlipped then people will assume the worst.
Help them: http://archiveteam.org/index.php?title=Warrior
The only thing is, it's empty. I don't know how you get a 14MB zip file that's empty, but Windows is telling me it's empty. That's not cool.
14:23 <REDACTED> you have to enable the "show hidden folders" in windows, under their view options
14:24 <REDACTED> each version of windows is slightly different but it is usually press alt, select tools, folder options, view, and the radio next to show hidden files and folders and then ok
14:24 <REDACTED> then you close and reopen the zip file
14:30 <REDACTED> oh, and sometimes the first download is corrupt.
14:30 <REDACTED> I've had to help too many people at work with this problem plus a person or two on this channel
It sucks to see your work crash into the ground like this.
Just so it's clear, there's just Noah (and his parents) that are left running the company.
Dustin Curtis: "Can anyone put me in touch with the founder of Twitpic?(I'll pay to backup all of the photos, and even host them. Ridiculous situation.)"
hi@dustincurtis.com
- The whole shutdown / acquisition / shutdown saga - Tepid response wrt getting compensated to export data
Someone's still holding out hope for an injection of capital. Because once the export happens and the data is gone, the value is entirely lost.
I wouldn't bank on getting any traction on paid exports until the last possible second.
They cannot do it any other way - the Internet Archive does not have infinite resources to host things, and they need to prioritize. 120TB is not an easy thing to host.
The Archive Team is the "oh shit this site is shutting down and it's not all in the Internet Archive already, we gotta save it before the deadline when it's gone forever" squad.
Good luck.
If there are 100 pieces, and 500 people, you want each piece to be held by 5 people for 5x redundancy. And, if nodes leave the swarm, the system automatically rebalances which nodes are maintaining a particular piece of the dataset.
The only solution being worked on out there that does this is IPFS
https://github.com/jbenet/ipfs https://github.com/jbenet/node-ipfs https://github.com/jbenet/go-ipfs
http://mixergy.com/interviews/twitpic-noah-everett/