> However, due to the system’s large-capacity, low-performance storage structure, no external backups were maintained — meaning all data has been permanently lost.
Yikes. You'd think they would at least have one redundant copy of it all.
> erasing work files saved individually by some 750,000 civil servants
> 30 gigabytes of storage per person
That's 22,500 terabytes, about 50 Backblaze storage pods.
I'm sure they had dozens of process heavy cybersecurity committees producing hundreds if not thousands of powerpoints and word documents outlining procedures and best practices over the last decade.
There is this weird divide between the certified class of non-technical consultants and actual overworked and pushed to corner cut techs.
Does G-Drive mean Google Drive, or "the drive you see as G:"?
If this is Google Drive, what they had locally were just pointers (for native Google Drive docs), or synchronized documents.
If this means the letter a network disk storage system was mapped to, this is a weird way of presenting the problem (I am typing on the black keyboard and the wooden table, so that you know)
Saw a few days ago that the application site for the GKS, the most important scholarship for international students in Korea, went offline for multiple days, surprising to hear that they really lost all of the data though. Great opportunity to build a better website now?
But yeah it's a big problem in Korea right now, lots of important information just vanished, many are talking about it.
Mindblowing. Took a walk. All I can say is that if business continues "as usual" and the economy and public services continue largely unaffected then either there were local copies of critical documents, or you can fire a lot of those workers; either one of those ways the "stress test" was a success.
Funny, because the same thing happened in Nepal a few weeks ago. Protestors/rioters burned some government buildings, along with the tech infrastructure within them, so now almost all electronic data is gone.
TL;DR: Estonia operates a Tier 4 (highest security) data center in Luxembourg with diplomatic immunity. Can actively run critical government services in real-time, not just backups.
> The Interior Ministry explained that while most systems at the Daejeon data center are backed up daily to separate equipment within the same center and to a physically remote backup facility, the G-Drive’s structure did not allow for external backups.
This is why I don't really want to run my own cloud :)
Actually testing the backups is boring.
That said, ones the flames are out, they might actually be able to recover some of it.
It's hard to believe this happened. South Korea has tech giants like Samsung, and yet this is how the government runs? Is the US government any better?
I wonder how many IT professionals were begging some incompetent upper management official to do this the right way, but were ignored daily. You'd think there would be concrete policies to prevent these things...
The government official who insisted that commercial AWS/GCP/Azure couldn't possibly be trusted with keeping the information will be keeping their head low for a few days then...
"The Interior Ministry explained that while most systems at the Daejeon data center are backed up daily to separate equipment within the same center and to a physically remote backup facility, the G-Drive’s structure did not allow for external backups."
If you (as the SK government) were going to do a deal with " AWS/GCP/Azure" to run systems for the government, wouldn't you do something like the Jones Act? The datacenters must be within the country and staffed by citizens, etc.
Usually these mandates are made by someone who evaluates “risks.” Third party risks are evaluated under the assumption that everything will be done sensibly in the 1p scenario, to boot, the 1p option will be cheaper as disk drives etc are only a fraction of total cost.
Reality hits later when budget cuts/constrained salaries prevent the maintenance of a competent team. Or the proposed backup system is deemed as excessively risk averse and the money can’t be spared.
>The government official who insisted that commercial AWS/GCP/Azure couldn't possibly be trusted with keeping the information will be keeping their head low for a few days then...
They can't. The trump admin sanctioning the international criminal court and Microsoft blocking them from all services as a result are proof of why.
I was smirking at this until I remembered that I have just one USB stick as my 'backup'. And that was made a long time ago.
Recently I have been thinking about whether we actually need governments, nation states and all of the hubris that goes with it such as new media. Technically this means 'anarchism' with everyone running riot and chaos. But, that is just a big fear, however, the more I think through the 'no government' idea, the less ludicrous it sounds. Much can be devolved to local government, and so much else isn't really needed.
South Korea's government have kind-of deleted themselves and my suspicion is that, although a bad day for some, life will go on and everything will be just fine. In time some might even be relieved that they don't have this vast data store any more. Regardless, it is an interesting story regarding my thoughts regarding the benefits of no government.
122 comments
[ 4.4 ms ] story [ 92.8 ms ] threadYikes. You'd think they would at least have one redundant copy of it all.
> erasing work files saved individually by some 750,000 civil servants
> 30 gigabytes of storage per person
That's 22,500 terabytes, about 50 Backblaze storage pods.
Or even just mirrored locally.
There is this weird divide between the certified class of non-technical consultants and actual overworked and pushed to corner cut techs.
Does G-Drive mean Google Drive, or "the drive you see as G:"?
If this is Google Drive, what they had locally were just pointers (for native Google Drive docs), or synchronized documents.
If this means the letter a network disk storage system was mapped to, this is a weird way of presenting the problem (I am typing on the black keyboard and the wooden table, so that you know)
But yeah it's a big problem in Korea right now, lots of important information just vanished, many are talking about it.
TL;DR: Estonia operates a Tier 4 (highest security) data center in Luxembourg with diplomatic immunity. Can actively run critical government services in real-time, not just backups.
This is why I don't really want to run my own cloud :)
Actually testing the backups is boring.
That said, ones the flames are out, they might actually be able to recover some of it.
"The Interior Ministry explained that while most systems at the Daejeon data center are backed up daily to separate equipment within the same center and to a physically remote backup facility, the G-Drive’s structure did not allow for external backups."
This is absolutely wild.
Reality hits later when budget cuts/constrained salaries prevent the maintenance of a competent team. Or the proposed backup system is deemed as excessively risk averse and the money can’t be spared.
They can't. The trump admin sanctioning the international criminal court and Microsoft blocking them from all services as a result are proof of why.
> A senior government official overseeing recovery efforts for South Korea's national network crisis has reportedly died by suicide in Sejong.
Recently I have been thinking about whether we actually need governments, nation states and all of the hubris that goes with it such as new media. Technically this means 'anarchism' with everyone running riot and chaos. But, that is just a big fear, however, the more I think through the 'no government' idea, the less ludicrous it sounds. Much can be devolved to local government, and so much else isn't really needed.
South Korea's government have kind-of deleted themselves and my suspicion is that, although a bad day for some, life will go on and everything will be just fine. In time some might even be relieved that they don't have this vast data store any more. Regardless, it is an interesting story regarding my thoughts regarding the benefits of no government.
Maybe it was just convenient to have the possibility of losing everything.