2-4 weeks feels so absurd that I reread the alert twice thinking maybe they meant something different. Database problem into discovery of failed backups, perhaps?
It'll be interesting to see the postmortem. I don't know much about their infrastructure.
Earlier today, we detected evidence of a security incident on AO3. We are still in the early stages of response to this security incident and determining its scope. AO3 will remain down while we continue investigating and addressing what has been found so far. However, we want to make some things clear:
First, no AO3 data has been deleted. All your works, bookmarks, kudos, etc., are safe.
Second, while we currently don't believe that the user database was stolen, we are also unable to rule it out. The AO3 database contains account information that may include your username, email address, IP address, and a hashed version of your password.
We use industry-standard techniques for storing passwords in a scrambled form that cannot easily be reversed. Even so, no hashing method can fully protect a weak or reused password from being cracked. As a precaution, we recommend that you change your passwords on any other sites where you use the same password as on AO3.
AO3 does not collect real names, so none could be exposed. Additionally, our donor information is held in an entirely separate system and is never linked to AO3, so that data has also not been exposed.
We will update everyone as our investigation continues; we may be unable to respond to individual inquiries from users. Please keep an eye on our official outlets for updates. Our next update will be posted when we have further information available.
Our current expectation is that AO3 will be down for a minimum of two weeks; however, it could take up to four weeks. During that time, we may be able to bring the site back up in a degraded state, meaning that not all features will be available and/or the site may be read-only. If we are able to do that, please be aware that we might have to bring the site back down unexpectedly as part of our recovery efforts. We will keep you in the loop for any developments
Was it mentioned somewhere that they got compromised? And that it's not for example a hardware failure? I imagine lead times for getting brand new hardware would be in the 2–4 week range these days.
non-skittish hosting providers are uniformly also unprincipled
Yeah, this is what I'm saying (although absolute free speech is itself a principle). I wish it weren't so, but you're 99% right that the cloud isn't really usable for anything outside the Overton window, whether it's far left, far right, erotica, pro-suicide, extreme trolling, etc.
However if you're using Cloudflare you can "hide" your backend hosting provider and they usually won't kick you out because they don't take any reputation hit. So I still suspect a mainstream cloud (with good off-cloud backups) could work for AO3.
> Are you suggesting that AO3 is akin to Nazi content? Are you suggesting that non-skittish hosting providers are uniformly also unprincipled? Are you equating erotica and fascism? I’d prefer to hear your explanation in your own words rather than be forced to infer bad faith participation here.
I think the point they're trying to make is that "normal" hosting providers like AWS, Azure or Google won't accept AO3 as a customer in the first place or yeet them the day the first DMCA email arrives or some Evangelicals yap about furries and they realize what their customer is doing. "Nazi" hosters (aka wherever Gab, Truth Social et al are hosted) or your average cybercrime "bulletproof hoster" are being used to deal with a lot of bullshit, or they just pipe any complaints straight to /dev/null - and they are used to dealing with cyberattacks of all kinds as well because they are a very, very juicy target for anyone from antifascist activists over cybercriminals wanting to cash in on Trump's Truth Social privileges to other nations' secret services that might want to sow chaos by manipulating Trump's Truth Social feed.
So... AO3 is nowhere near Nazi crap in terms of content, but a very, very similar threat model (exhaustion of customer support goodwill).
My understanding is that (at least part of it is)the OTW has legal concerns over hosting fan works on somebody else’s computer. So they took the very reasonable step of hosting it on their own computer.
The data's *always* been safe. /r/DataHoarders would never overlook this.
Nvidia will serve you a free, high-speed half-terabyte download right now[0]. It's one with a cutoff date of 2020—but given that GPT-3 was released mid-2020, that's probably the one you actually want, for ML research.
Well that's annoying. I was literally planning later today to go through the list of active stories I'm following at Fanfiction.net, see which ones are also posted on AO3, and switch to reading those on AO3 because FFN is falling apart.
As someone who is just a reader the main annoyance at FFN is that when a story gets a new chapter it is often hard to get it to actually show the chapter. Sometimes you can only get it if you try to go directly to it, sometime you can only get it if you go to it from the previous chapter. Sometimes you can't get it at all.
Also sometimes if you try to reread the next-to-last chapter to remind yourself of context before reading the new chapter it say the next-to-last chapter was not found. As with the last chapter sometimes you can get it if you approach from the right direction.
Sometimes it is even the last several chapters that get temporarily lost this way.
So far this eventually has cleared up on a given story but sometimes it takes weeks.
From what I've seen authors mention there are also several issues with its support for authors making them as frustrated and annoyed as us pure readers.
Recently I've also noticed even the browsing story categories is falling apart. E.g., go there and from the "Browse" menu pick books. On the books page pick "Harry Potter". Then when it lists the HP stories click the "Crossovers" link to see the HP crossovers. That gives an error, and has done so for quite a while.
22 comments
[ 0.92 ms ] story [ 8.0 ms ] threadIt'll be interesting to see the postmortem. I don't know much about their infrastructure.
Apparently they run bare metal that they own [2] so I'd assume their servers got compromised hard and they'll need to buy entirely new servers.
[1] https://www.reddit.com/r/AO3/comments/1xir586/comment/19rubn...
[2] https://www.reddit.com/r/AO3/comments/1xir586/comment/19ruc1...
Yeah, this is what I'm saying (although absolute free speech is itself a principle). I wish it weren't so, but you're 99% right that the cloud isn't really usable for anything outside the Overton window, whether it's far left, far right, erotica, pro-suicide, extreme trolling, etc.
However if you're using Cloudflare you can "hide" your backend hosting provider and they usually won't kick you out because they don't take any reputation hit. So I still suspect a mainstream cloud (with good off-cloud backups) could work for AO3.
I think the point they're trying to make is that "normal" hosting providers like AWS, Azure or Google won't accept AO3 as a customer in the first place or yeet them the day the first DMCA email arrives or some Evangelicals yap about furries and they realize what their customer is doing. "Nazi" hosters (aka wherever Gab, Truth Social et al are hosted) or your average cybercrime "bulletproof hoster" are being used to deal with a lot of bullshit, or they just pipe any complaints straight to /dev/null - and they are used to dealing with cyberattacks of all kinds as well because they are a very, very juicy target for anyone from antifascist activists over cybercriminals wanting to cash in on Trump's Truth Social privileges to other nations' secret services that might want to sow chaos by manipulating Trump's Truth Social feed.
So... AO3 is nowhere near Nazi crap in terms of content, but a very, very similar threat model (exhaustion of customer support goodwill).
They say they have a second site, the chance of hardware failure taking out both sites at the exact same time is near zero.
Other than that 2-4 does sound more like hardware lead time rather than time it takes to resolve a security incident.
Nvidia will serve you a free, high-speed half-terabyte download right now[0]. It's one with a cutoff date of 2020—but given that GPT-3 was released mid-2020, that's probably the one you actually want, for ML research.
[0] https://huggingface.co/datasets/ray0rf1re/AO3-2020
Related:
Archive of Our Own
https://news.ycombinator.com/item?id=50128197
Fans are better than tech at organizing information online (2019)
https://news.ycombinator.com/item?id=43137627
As someone who is just a reader the main annoyance at FFN is that when a story gets a new chapter it is often hard to get it to actually show the chapter. Sometimes you can only get it if you try to go directly to it, sometime you can only get it if you go to it from the previous chapter. Sometimes you can't get it at all.
Also sometimes if you try to reread the next-to-last chapter to remind yourself of context before reading the new chapter it say the next-to-last chapter was not found. As with the last chapter sometimes you can get it if you approach from the right direction.
Sometimes it is even the last several chapters that get temporarily lost this way.
So far this eventually has cleared up on a given story but sometimes it takes weeks.
From what I've seen authors mention there are also several issues with its support for authors making them as frustrated and annoyed as us pure readers.
Recently I've also noticed even the browsing story categories is falling apart. E.g., go there and from the "Browse" menu pick books. On the books page pick "Harry Potter". Then when it lists the HP stories click the "Crossovers" link to see the HP crossovers. That gives an error, and has done so for quite a while.