Yes, and this is exactly why they should be keeping both digital (online and offline) as well as paper copies of their records.
If they can't afford to have good backups of their data, they can't afford to have digital records in the first place. This is the equivalent of a community saying they can't afford filing cabinets so they just stack paper on the floor in the boiler room. Nobody would find that acceptable, and they shouldn't find this acceptable either.
I think one issue is that they can’t know for certain when they became compromised. So they may be able to get data back but can’t know that that data has integrity, unless they’ve hashed it since the beginning of time.
Honestly, though, if you just filter to what could be backed up on paper you'll find that backups are trivial. A pretty big storage room, stuffed with ten thousand bankers boxes, each holding five thousand double-sided sheets of paper? At a reasonably generous 80KB per side you have 8TB. That's one data tape. Even a "library of congress" is only 10TB.
Their assumption is that they're being targeted and that this "united front" will give attackers less reason to target them, when the harsh reality is that these CrytoMalware emails/IM Spam are being send to every business/government internationally looking for the softest targets.
They should have passed a resolution to implement a 1-2-3 Backup Strategy with mandatory offline & offsite backups and testing protocols. But that would cost money and require competent management/oversight, instead they'd prefer to pass a meaningless fiat that won't do jack.
Honestly until there are consequences for government officials/management nothing will change. This is 95% about poor resource management and 5% about CrytoMalware. Nobody should be paying, because they should ALREADY have multiple tiers of backups, that are audited, tested, and reviewed.
PS - "It also encrypted our backups" is also pure incompetence. They just didn't want to manage rotated backups or pay the storage fee/costs of high density tape.
The problem with 1-2-3 backup is 95% user education, policies, desktop administration, etc., and 5% making the copies of data. In my experience, at least. You need IT staff who are good at finding solutions that work for everyone. It would be nice if we could just stay “store it on the network drive” and then make backups, but users in the org don’t listen, and this is a fact of reality that you have to spend time & money to adapt to.
So backup is a fairly big management & IT challenge that goes way beyond guidelines like 1-2-3. That, and due to legal restrictions, local governments are rarely empowered to hire the right IT staff capable of making it happen.
I'm not sure what your first paragraph is about. Windows hasn't worked like that in a very long time, a user's default storage locations (Desktop, Documents, etc) are all held centrally and it is completely transparent to the user. This is handled by Folder Redirection and a GPO policy.
So no user education, desktop administration, or specific policies are required. You just need not to alter the defaults and you'll get a centrally managed set of user files perfectly suited to regular backups.
The only thing that remains a challenge is offsite users/traveling. But an edge case isn't a justification for why you wouldn't use folder redirection within a monolithic organization like a local government.
> That, and due to legal restrictions, local governments are rarely empowered to hire the right IT staff capable of making it happen.
What "legal restrictions" stop government from hiring IT Staff?
> Windows hasn't worked like that in a very long time, a user's default storage locations (Desktop, Documents, etc) are all held centrally and it is completely transparent to the user.
This has to be set up and configured, it is not default by any means. That and most places I have seen a mix of macOS and Windows, and people use their personal devices all the time, or put something in DropBox / Google Drive etc to share things, etc. This is not a button to press.
> What "legal restrictions" stop government from hiring IT Staff?
Have you seen IT budgets in local government? That, and salary guidelines set by law. It's not a simple question of whether or not it's possible to hire IT staff, but a question of whether you can pay enough to attract the talent you need, whether you can afford the opex+capex for the level of service you think is standard, etc.
When I've worked at larger organizations like universities, things were usually handled very well with everything set up the way you describe, automated, and hands-off. But there's a lot of variation when it comes to things like city governments.
> This has to be set up and configured, it is not default by any means.
If you use the GPO templates it absolutely is the default. You have to go out of your way to turn it off (and some people do for both good and bad reasons).
> whether you can pay enough to attract the talent you need, whether you can afford the opex+capex for the level of service you think is standard, etc.
Luckily a 3-2-1 backup strategy doesn't require top tier talent. It is mostly policy, costs, and time. You could literally have an intern do it if the business took the time to set a policy and buy the prerequisites.
The difficult part has always been getting business/government buy-in because they'd prefer to be apathetic until something goes wrong or under-spend on something that won't pay dividend immediately.
Talk to government IT some time. I've worked in that environment. There's no lack of talent, there's a lack of funds for things like backups and nobody in management (political appointed or otherwise) wants to make actual decisions/do any managing. I had colleagues "borrow" a spare HDD, copy the data off onto it, then take it home with them because that was the only backup an entire County had (because nothing else was funded/authorized).
You can try to bike-shed/excuse this issue away, but at its core it is managerial incompetence/apathy. If they wanted the money for it, they'd find it, if they wanted the time to produce policy they'd find it, and if there was real consequences to their incompetence you can bet they would have already.
Write your congressperson. The SWAT team probably gets funding from the Department of Homeland Security to “combat terrorism”. Clearly the terrorists have moved online, so perhaps some security funding should go there, too.
State replication is well understood - if one's system(s) allow for data to travel outside of a 'cloud' (network resource), then there needs to be a compelling reason. Not saying that 100% of systems can be pushed up to a cloud - just having a hard time thinking of a particular use case at the moment.
Even banks are moving from vendor managed solutions to cloud based in-house supported software of late, and this group is the most difficult to change their ways (lot-o-hoops).
I do agree with you that many biz/org's might pay for a system and fail to understand the costs around maintaining/future upgrading - that's a management issue and should be dealt with via accountability/consequences.
Agreed that people should be held accountable but there is not a great process for this. Usually the fallout from bad policies for backups / continuity are down to a roll of the dice. Too many of the people in charge are willing to roll the dice, and too many will understaff IT and then not listen.
One thing that you can do, is as a matter of policy, wipe all the machines on a fairly regular basis (maybe once a week). This gets people in the habit of expecting their machine to be ephemeral and to store their documents and other work in a central location that is backed up.
At my job at a local college we had Deep Freeze installed on all our systems. Students and everyone was told to bring their own thumb drives or backup files to the cloud. It worked out just fine.
Speaking of backups, what do you all do for yours? I’m embarrassed to admit I only do a weekly backup to a portable drive for my personal data, and mostly use automated snapshots on aws for database contents.
I recently purchased a Synology NAS that does automated nightly backups to wasabi. Everything I care about is either on the backed up drive of the nas, or in git repositories. My rule is that I should be able to wipe any of my computers at any time, and pick up where I left off with a few hours of work.
All my stuff is in Google so my backup is for "what if Google decides to ban me without recourse?" For that I download the hundreds of gigs of Google takeout and upload it to backblaze. I ought to automate it.
I'm mostly saying this out loud for the chance someone will point out an issue or optimization for my strategy.
For my personal data I use Backblaze and an external drive (using windows file history to manage the external drive). For stuff that originated from my phone (mostly pictures) I let iCloud manage that and then replicate it to OneDrive so I have two cloud copies.
Honestly, I can't add much more than what has already been said here... If your data/biz/org/etc is crippled because of these types of attacks, then you really need to have a frank discussion around IT/resource allocations/goals. These are 100% manageable (worst case).
If the malware author manages to infiltrate your network and quietly starts infecting files and devices on your network over 6 months, even if you restore from a 7 month old backup, what about all of the files that were created between then and now -- are they all going to be recoverable even if you can find and remove the malware? Can you reliably detect all of the malware or will the attack sprout again from your printer in a few months?
This type of acausal deal only works for single-target attacks. If a bad actor is searching for a victim and sees two potential targets, one of which has resolved to never cede to their demands and one who wasn't commented, they will attack the ambivalent party.
Malware is not a single target attack. Whether or not it's probably beneficial to attack one of these cities is not considered. Instead of making gestures, these mayors should be investing in better cybersecurity.
Fascinating. The page manages to bypass uBlock Origin and pop up an ad window in the right bottom corner to start paying heavy (which my computer and connection can barely handle) video with sound (!) automatically.
I’m curious where the cities that have paid ransomware attackers acquired bitcoin. Did they literally just open a Coinbase account and send funds through there?
I am on a local city commission. As part of this, they gave me an email address (made me unhappy, but whatever). They sent me an email to my personal email address telling me how to log in to my city email address, explaining that my password was a trivial algorithm based on my name, followed by a number an exclamation point. This was a form email: every single address in the entire city has the same password format (with the same number, to be explicitly clear). They disabled the feature to let people change their password. So... anyone can log in to the email account of any official in this city and do stuff like delete mail before they see it (as even if they have audit trails turned on for administrators, the official can still delete mail from their own perspective, and would never know if someone helpfully deleted it "for them").
It's funny because ransomware is simply DRM coming in from the other direction.
I can't help but laugh because it's such a clear demonstration of everything that's bad about DRM, except for the fact that, well, the message gets lost amid a tide of egotism and indignation.
47 comments
[ 3.0 ms ] story [ 94.2 ms ] threadIf they can't afford to have good backups of their data, they can't afford to have digital records in the first place. This is the equivalent of a community saying they can't afford filing cabinets so they just stack paper on the floor in the boiler room. Nobody would find that acceptable, and they shouldn't find this acceptable either.
https://www.zdnet.com/article/georgia-county-pays-a-whopping...
They should have passed a resolution to implement a 1-2-3 Backup Strategy with mandatory offline & offsite backups and testing protocols. But that would cost money and require competent management/oversight, instead they'd prefer to pass a meaningless fiat that won't do jack.
Honestly until there are consequences for government officials/management nothing will change. This is 95% about poor resource management and 5% about CrytoMalware. Nobody should be paying, because they should ALREADY have multiple tiers of backups, that are audited, tested, and reviewed.
PS - "It also encrypted our backups" is also pure incompetence. They just didn't want to manage rotated backups or pay the storage fee/costs of high density tape.
So backup is a fairly big management & IT challenge that goes way beyond guidelines like 1-2-3. That, and due to legal restrictions, local governments are rarely empowered to hire the right IT staff capable of making it happen.
So no user education, desktop administration, or specific policies are required. You just need not to alter the defaults and you'll get a centrally managed set of user files perfectly suited to regular backups.
The only thing that remains a challenge is offsite users/traveling. But an edge case isn't a justification for why you wouldn't use folder redirection within a monolithic organization like a local government.
> That, and due to legal restrictions, local governments are rarely empowered to hire the right IT staff capable of making it happen.
What "legal restrictions" stop government from hiring IT Staff?
This has to be set up and configured, it is not default by any means. That and most places I have seen a mix of macOS and Windows, and people use their personal devices all the time, or put something in DropBox / Google Drive etc to share things, etc. This is not a button to press.
> What "legal restrictions" stop government from hiring IT Staff?
Have you seen IT budgets in local government? That, and salary guidelines set by law. It's not a simple question of whether or not it's possible to hire IT staff, but a question of whether you can pay enough to attract the talent you need, whether you can afford the opex+capex for the level of service you think is standard, etc.
When I've worked at larger organizations like universities, things were usually handled very well with everything set up the way you describe, automated, and hands-off. But there's a lot of variation when it comes to things like city governments.
If you use the GPO templates it absolutely is the default. You have to go out of your way to turn it off (and some people do for both good and bad reasons).
> whether you can pay enough to attract the talent you need, whether you can afford the opex+capex for the level of service you think is standard, etc.
Luckily a 3-2-1 backup strategy doesn't require top tier talent. It is mostly policy, costs, and time. You could literally have an intern do it if the business took the time to set a policy and buy the prerequisites.
The difficult part has always been getting business/government buy-in because they'd prefer to be apathetic until something goes wrong or under-spend on something that won't pay dividend immediately.
Talk to government IT some time. I've worked in that environment. There's no lack of talent, there's a lack of funds for things like backups and nobody in management (political appointed or otherwise) wants to make actual decisions/do any managing. I had colleagues "borrow" a spare HDD, copy the data off onto it, then take it home with them because that was the only backup an entire County had (because nothing else was funded/authorized).
You can try to bike-shed/excuse this issue away, but at its core it is managerial incompetence/apathy. If they wanted the money for it, they'd find it, if they wanted the time to produce policy they'd find it, and if there was real consequences to their incompetence you can bet they would have already.
Even banks are moving from vendor managed solutions to cloud based in-house supported software of late, and this group is the most difficult to change their ways (lot-o-hoops).
I do agree with you that many biz/org's might pay for a system and fail to understand the costs around maintaining/future upgrading - that's a management issue and should be dealt with via accountability/consequences.
It could probably even work quarterly without too much lost productivity...
Then again, doing it weekly might work out such that you get really good and fast at doing these wipes, so it becomes unnoticed.
Hmmm, maybe we should review these annual plans...
You redirect a user's folders to the network share, and people get used to it fairly quickly.
I'm mostly saying this out loud for the chance someone will point out an issue or optimization for my strategy.
https://rclone.org/
It is cheaper that way. :D
https://dilbert.com/strip/2002-05-11
If the malware author manages to infiltrate your network and quietly starts infecting files and devices on your network over 6 months, even if you restore from a 7 month old backup, what about all of the files that were created between then and now -- are they all going to be recoverable even if you can find and remove the malware? Can you reliably detect all of the malware or will the attack sprout again from your printer in a few months?
Malware is not a single target attack. Whether or not it's probably beneficial to attack one of these cities is not considered. Instead of making gestures, these mayors should be investing in better cybersecurity.
I assume the malware fucks stole a copy to auction off.
just having the software won't do you any good
I can't help but laugh because it's such a clear demonstration of everything that's bad about DRM, except for the fact that, well, the message gets lost amid a tide of egotism and indignation.
Oh, what fools these mortals be!
paying the ransom ware folks should be considered the cost of doing business. it’s cheaper than actually securing the data.
anyway it’s expected. the same mayors that underfund IT would be the same ones to make this ridiculous “red line”.
good opportunity here for a cookie cutter IT consultancy to come in to all of these cities and offer cookie cutter service.