Or more accurately, accidentally deleted. 'losing' these records would actually be a lot worse. I wouldn't underestimate the police's ability suffer a breach and lose the records either.
It's worth remembering many of these records are are kept on the basis of non conviction (though there is a time limit).
It comes after around 40,000 alerts relating to European criminals were removed from the same database, the PNC, following Britain's post-Brexit deal with the EU.
Right so reading between the lines they needed to do some post-Brexit data cleaning and someone messed up the WHERE clause.
But where are the backups? Surely accidentally deleted data should be fairly easy to recover from historical backups which they surely surely must have?
Wouldn't such international data removal agreements also include backups? What's the point in removing PII from one of the databases, if it remains in their backups?
You have a point but I have doubts that any organisation on the planet has the capability/willingness/resources to surgically remove data from historical backups. I think its more likely that no backups exist, or that, say, no backups exist past X days in the past. Or worse, that backups exist, but they have been found to be unrestorable.
Under GDPR right-to-erasure you dont necessarily have to erase data from backups but must be clear about how long it will hang around.
The key issue is to put the backup data ‘beyond use’, even if it cannot be immediately overwritten. You must ensure that you do not use the data within the backup for any other purpose, ie that the backup is simply held on your systems until it is replaced in line with an established schedule.
You could handle such requests by tombstoning the user ID (in addition to deleting the data). By keeping the tombstone records in separate storage (it's usually only kB, just a list of hard-removed user IDs), you can guarantee (re-do) the erasure even after having to do a full database restore.
Backup. Encrypt the backup. Run the deletion. Verify you haven't lost anything. Verify it again. Put the backup encryption key beyond use other than in the case of a suitably agreed emergency (e.g. in the case of an agreement with another party, hand them the key). Then after a suitable grace period, once you're sure everything is ok, you destroy the backup.
> Put the backup encryption key beyond use other than in the case of a suitably agreed emergency (e.g. in the case of an agreement with another party, hand them the key).
Like search warrants? We know those never get abused.
> Then after a suitable grace period, once you're sure everything is ok, you destroy the backup.
But we need a search warrant to access the backup anyway so we don't really have to delete it. Best keep it, just in case.
> Like search warrants? We know those never get abused.
No, like catastrophic loss of data, hence why I outlined a process of a doing this temporarily in the case of agreement with a another party to put the data entirely beyond your own use unless the party you've agreed the deletion with agrees the situation is serious enough.
> But we need a search warrant to access the backup anyway so we don't really have to delete it. Best keep it, just in case.
If you don't trust the party to delete the data, then adding in a temporary period of retention makes no difference, because in that case they could just as well have ignored the demand for the initial deletion entirely.
You can make pretty much any sort of tech fuckup and be forgiven.
The only two unforgivable sins are “no working backups” and being hacked. Even being hacked is kinda forgivable cause it’s so hard to avoid. No working backups is totally unforgivable.
Any tech person who walks into any tech job at any level up from junior to CTO has a duty as one of their first tasks to find out if working backups are being done of their work.
yeah, long time ago I synced the CRM database of the austria press agency to my sony clie https://en.wikipedia.org/wiki/CLI%C3%89 and then my sony clie synced it back. completely deleting all data other then the email and name on the CRM server.
gladly there was a backup, but the backup could not get used to get the database back-up. the discussion with IT were fabolous: "yes, we have it all backed up, so the backup is there, but we cant use it to bring the CRM back-up. but its backed up and we even have backups from the backup. they can also not get used to bringt it back up."
since then it is my standard procedure to make sure that backups can actually be used to bring projects back up. usually the first test "use backup to bring stuff back up" fails.
If you haven't tested your backups, you don't really have backups.
If you aren't allowed to verify the backups yourself, find a safe "mistake" to make to force the responsible parties to run through restoring from backups. Bonus points for using the backup testing exercise as an excuse to convince your manager to let you do a bit of refactoring to remove some particularly ugly module you've been itching to get rid of. Factor out the ugly module, then "accidentally" delete it.
My fist job out of school was writing software models of pre-prototype communications equipment for a defense contractor. There was lots of government red tape, and crazy prices being charged everywhere. We couldn't use CVS or SVN (this predated git) to store our code, because they weren't government-approved tools. We could use PVCS to store source code, but only for the build artefacts and corresponding source code of official production versions delivered to the government. (And only people that had taken a multi-hour course to be certified on PVCS could touch it. Every release involved a couple government employees coming out to the defense contractor and physically watching the one guy on the team check in the release to PVCS and burn a pair of DVDs for the government. So, we were forced to do source control by "last one out on Friday has to zip up the entire shared drive and name it with the date".
But, at least we were required to pay some crazy price to have the sysadmins backup the shared network drive to tape every week (or was it every day?). Once a month, the admins would take the current tape and send it via courier to a bomb-proof, fire-proof bunker out in the mountains in Pennsylvania or some such. They'd then put the spare tape in the tape drive to start making backups. As soon as the bunker got the latest tape, they'd send back the oldest tape via courier to become the spare tape. So, we always had at least one tape physically present in the bunker. It required 3 tapes to always have at least one tape in the bunker and one tape in the drive.
I was less than a year out of school, but based on my interactions with the system admins, I figured there was at most a 50/50 chance they got the backups correct. I asked them once how the backups were going, and got back a very low key, "yea, yea, don't worry about it, we got it" answer. I convinced my boss to let us fake a source code loss: zip up two copies of the network drive, and store the zip files on two different systems. Check the integrity of both zip files, and then "accidentally" delete a directory containing some source we were in the midst of removing anyway.
The sysadmins were unable to get our "accidentally" deleted source code back for us. They had neglected to erase the spare tape when putting it into the drive, so they were always only appending to our backups. They just ignored the errors when the tape got full.
It sounds to me like a "purge for legal reasons" function was applied to more records. Beyond some possible time limit to undo, it should not be possible to retrieve such a purge from backups. If they can retrieve this, then that is a larger problem.
The PNC has been challenged in the past for holding on to records that legally should have been deleted.
If I put my tin foil hat on for a moment, I wonder if this "mistake" was a way of getting rid of some records that should have already been deleted.
Doing it this way, and publicly admitting it, might have been deemed safer than clandestinely deleting the records and then getting caught in a security audit later.
>The only two unforgivable sins are “no working backups” and being hacked.
I always change that sentence to "no working restore" because that's often forgotten, but yeah it's the same as a "working backup" but changes the focus what's the important part with backups.
I have, on two occasions, discovered that my new place of employment had never tested the restore process and had broken daily backups. One of those companies had a whole failover site ready to go in case of emergency and had never tested that the database replication was operating correctly.
>I have, on two occasions, discovered that my new place of employment had never tested the restore process and had broken daily backups.
Same same, it was with backupexec, once the daily backup-report said everything fine, so i made a restore on a vm (just to check) then a dry-run rsync with the prod Machine (Fileserver) and it wanted to transfer about 1TB of files, the last "real" backup was 2 year ago, turned out the backupexec-agent on the Fileserver was never updated and reported that the last backup was successful (the 2 yo that was).
I checked the backups and they were all something like 10Mb, I asked if that was a reasonable size and everybody freaked out. It's near the top of my CTO/VP first week checklist ... "what is our recovery strategy" along with "who are my team"
About 15 years ago was very happy with his source code backups, the code was on an IRIX server and a cronjob tarred up the backups and he kept multiple copies of the tars offsite, and even offline. The tarballs even extracted fine and at first glance the directory structure and file lengths were ok.
Trouble is a bug in the file system meant that somehow the files had been overwritten with nulls.
The program continued to run, as it sat in memory. Until one day the box was rebooted (power test or something). It didn't come back. The files were great, containing multiple files all of the right looking length. Everything was null. "main.c" or whatever it was was 8 kilobytes of null characters.
This had been happening for some months so even the oldest source backups were wiped, and the actual binaries were out too. The system was called ELVIS, which made for a great status report.
As it was on an ancient irix machine and he was rewriting in PHP at the time he just accelerated that and we were back up in a few days (the program was a front end to a database and file stores on other machines, so it was just the web part)
Good ole' Schrödinger's Backups. Backups both exist and don't exist until you try to restore them. :)
I have a few frequent flyers at work for "I accidentally deleted this file, corrupted this database, etc." that I end up appreciating for making me somewhat regularly pull files from backups.
A more accurate headline would be '150k police records accidentally destroyed'. There is a huge difference between accidental deletion and leaving a hard drive full of data on the tube. I'd give the BBC the benefit of the doubt that the unclear wording is accidental and not for click bait purposes, but I feel like I'm doing that more and more for them at the moment.
The UK took a wrong turn when it gave the police their own extra-judicial justice system.
The system of police cautions, warnings and fixed penalties allow the police to administer punishment with none of the due process that a court would provide.
The idea of a "police caution" is a misnomer that implies it's just harmless warning, when in fact it can affect the course of someone's life many years later.
Furthermore, people are blackmailed into accepting these cautions under the threat of going to trial and potentially receiving a more severe punishment after conviction.
Even more outrageous is the retention of records from the PREVENT programme and "non-crime hate incidents". These are people who have not even committed a crime or been arrested.
The whole system stinks, and given that most people manage to avoid interaction with the police it is sadly unlikely to get the scrutiny it deserves.
The outrageous behaviour of (some of) the police during recent lockdowns actually gives me a little hope, as it as drawn widespread attention to the way they routinely abuse their power.
The general view in the population is the police do not deal with normal people other than as victims. If you're arrested, or even questioned, you're seen as guilty. Besides if you've nothing to hide you've nothing to fear.
This view doesn't hold when the police get you for speeding (go find real criminals)
Many police officers are quite reasonable, but then you have Derbyshire, whose actions over covid have likely killed people (by reducing compliance with lockdown orders)
The difference with the police in covid is the public are suddenly seeing that the police are dealing with normal people, not just criminal scum who should be lucky they aren't shot.
As I understood it, the Protection of Freedoms Act 2012 really clamped down on DNA and fingerprint storage. Not enough, but at least it was a move in the right direction. Particularly bad was that if you were charged and found innocent, they would still keep your records for 3 years.
Even after Protection of Freedoms Act 2012 was implemented, PNC only drives the removal records from IDENT1 (fingerprints) and National DNA database (self-explanatory), not the ASN or nominal records themselves - these are "proper to retain for statutory purpose until the person turns 100 years of age".
The general view in the population is the police do not deal with normal people other than as victims.
Unfortunately, there is an element of human nature that says, "It won't happen to me." And then maybe one day, if you're unlucky, it does happen to you. And everyone else who is similarly confident that it could never happen to them won't necessarily be there for you, because it's so far out of their experience and frame of reference that even if they're a generally decent person, they have trouble relating to the situation you're in or understanding how they might feel in your place. And so the people who are least at risk, who are often the ones who could make the most difference, don't always act as strongly or as frequently as they could to mitigate the risk to others.
> Hang on. This sounds like the fingerprint and DNA data for people arrested and never charged.
It sounds like that as that is exactly what is happening - it has been decided in 2009 case (with a riveting name of "Chief Constable of Humberside Police and others v. Information Commisioner and another" [0]) that it is proper for the statutory public purpose of the Police National Computer - now replaced by Police National Database - to retain nominal records until the person in question turns 100 years of age, as introduced into the new retention policy as of 2006 [1]. Even if you have been unlawfully/wrongfully arrested, cautioned etc. your information still goes into the database but you can apply to have it expunged [2], which in practice may require support from your MP - and many simply don't know about it.
"Nominal records" in this case refer to information linked to specifc named individuals reflecting the fact they have been convicted, cautioned or reprimanded or otherwise dealth with by issue of penalty notice for disorder. It's not a "nominal" amount of data - it is quite substantial. DNA and fingerprints are supposed to be expunged from that set after three years due to "Protection of Freedoms Act 2012", rest stays.
Police National Database that replaced Police National Computer retains all data from the PNC and makes it easier to access by multiple agencies.
I disagree with describing these events as 'technical issues' or 'IT errors'. As other commenters have noted it appears that there just wasn't a working backup. If that's the case, it's more of an organizational failing in my view: no-one cared enough to make a working backup.
It's similar to the 'IT error' in our test and trace system recently, where thousands of positive covid test results were uncounted because Excel truncated the file after a certain number of rows. In my view, it's not really the tech at fault, it's that the organization was using Excel because they didn't care enough to do it properly.
Calling it a 'technical issue' puts the blame on a malfunction, making it seem unfortunate but unavoidable, and absolving the organization that actually caused the issue.
Yeah it's like calling culpable driving occasioning a bad collision "An automotive mechanical issue." A drunken fool driving at twice the shed limit is simply not a mechanical issue.
Calling this a technical issue is deeply misleading and so egregiously so that i begin to have a flicker of doubt about motives.
I wonder how feasible, realistic, &c it would be to have included in the deletion/destruction policy to have a step/section in there to say "after verification of deletion, make a new backup to replace the old backup" which would be a backup refresh "outside" of the usual periodic backup.
That would work if you kept only backup version. That would be bad-what do you do in case of accidental record deletion or encryption malware detected after a backup?
Check out e.g. https://www.backblaze.com/blog/the-3-2-1-backup-strategy/
No you have to also delete any backups. I suppose you'd have to keep all data with such expiry dates in a separate database from other data, and backup the data to different media based on expiry date.
This is an issue with European GDPR "right to be forgotten" requirements.
Some interpretations say that if a user requests their data to be deleted, it must also be removed from backups. If your backups are kept in a binary format, then it's a nightmare. You have to restore them, delete the data then backup them up again for every backup you have.
Other interpretations say that it is enough to remove the data if/when you restore the backup data. I.e. keep a record of deletion requests and on restoring a backup check if you need to re-perform the deletion request and then do so if necessary.
For new systems where this sort of privacy requirement can be designed in from the start, I quite like a strategy that was proposed around the time of the big GDPR discussions, which seems useful in some situations: individually encrypt records of personal data with a separate key for each person (or some other granularity that makes sense for your processing and storage requirements). Then your main database and its backups only contain the separately encrypted data, you keep a rolling backup of your current set of encryption keys, and in response to any permanent data deletion requirement you can delete the key for the relevant record(s) to immediately render the data unusable in any large backups or long-term archives without having to regenerate them. When your key backup rolls over, which might only be a matter of days or even hours, any old copies of they key also get destroyed and the personal data is, for practical purposes, gone forever.
Obviously there are many edge cases around deleting partial or related records that would limit the usefulness of a simple scheme like this, and obviously it's no use for data storage and processing that was already set up before the GDPR came into effect and didn't have this sort of mechanism designed in, but I suspect there are quite a lot of data controllers that have relatively simple record-keeping requirements where this kind of approach could work well.
"(1)If fingerprints are required by section 63D to be destroyed, any copies of the fingerprints held by the police must also be destroyed.
(2)If a DNA profile is required by that section to be destroyed, no copy may be retained by the police except in a form which does not include information which identifies the person to whom the DNA profile relates."
This is where one might use a vaulting appliance. Policies for retention and destruction can be set per location or backup job methodology varies by vendor and I acknowledge these appliances are not cheap. This also assumes the data is stored in a hierarchy that permits this segmentation.
Thanks for clarifying. Does this mean they can't have a backup and then delete that backup after the data has been successfully deleted from the main db?
Of course, it's easy for me to say these things in hindsight but my point is that this is an issue with the organization not having the right processes, rather than the technology going wrong. This is alluded to in the article: "Policing minister Kit Malthouse said the problem had been identified and the process corrected".
You can have a backup and if necessary still delete that record in the backup. It's going to be more expensive and complicated, but it does not preclude backups.
I've worked in health systems where personal data would get selectively removed not just from our database, but also from backups.
I don’t consider an issue solved until there are commodity implementations of the solution. Right now there are bespoke backup systems that can do per-record deletion and lots of orgs are relying on maximum retention policies to do ensure deletion on “traditional” backup systems. Until off-the-shelf backup software like Veeam and Borg can handle the use-case it will always be an option only if you have a staff of developers instead of an IT staff.
Across the US, courts require data to be hard-deleted in cases of, for example, the expungement of a juvenile’s criminal record. This is a technical problem faced by all courts. Are you suspecting that courts are not taking backups, or that they are not properly wiping expunged records from their backups, or that courts employ developers that build bespoke backup pruning systems?
I used to do IT in the public sector but not for the court system. I'm saying that they're almost surely using some off-the-shelf backup tool and having "deletions will filter though the backups in $x months" as a good enough solution.
Reaching into the backups or doing per-record encryption isn't something that you can just buy at this point.
Data retention and deletion are opposites. It's practically impossible to get both right. It's a target that you have to hit exactly, and being extra good at one makes you worse at the other.
You can do that, but doing so would give you no protection against the deletion process being applied to records that should not be deleted as happened here. The whole point of this process is that those records need to be irretrievably destroyed.
I wonder if the IT had been outsourced to one of the big consulting firms, whose business model is to claim expertise over an unfeasibly large domain and to hoover up public sector contracts accordingly. Our local council had a very similar issue whereby their IT was likewise outsourced to one of these behemoths, and backups - in this case for the library service - were not being done correctly, resulting in the loss of a whole load of data.
Being vague about the cause of problems can sometimes save people’s jobs though. Sometimes the people who are technically responsible for it were so overworked they had no way of realistically doing everything they are tasked with. When the poo hits the fan, c level people are often looking for someone to nail the shame and blame on so naming a transient issue as the cause can give upper management the scapegoat they need without burning anyone
As I understand, '[f]ingerprint, DNA and arrest history records were deleted', but it's fine because 'the wiped records were those of people arrested and released when no further action was taken'.
If the police cannot be trusted to put basic data protection measures in place (such as backups), perhaps they should not keep biometric data about people that have done nothing wrong[0]?
[0]: more accurately, have not been charged with wrongdoing, meaning either there was insufficient evidence, or prosecution was not deemed to be in the public interest.
Or they have been processed, not charged with any wrongdoing, but they have not applied to have their records deleted under the post-2006 "Exceptional Case" procedure or post-2015 "Record Deletion" procedure. [0]
Or maybe they just did not yet turn 100 years of age, when the expunge is supposed to happen automatically.
The idea that someone who has been wronged by the criminal justice system should then be required to understand the consequences and make a special application just to have the record set straight is invidious, to say the least.
No normal organisation processing personal data gets to keep whatever they feel like, however unjustified, and only remove it under sufferance and on specific request. That wasn't even appropriate before the GDPR came into effect, and it's clearly prohibited by law now if there was any doubt before.
So why should governments get special privileges in this respect? There could hardly be a more serious case of processing personal data than a database used by police and security services that have exceptional legal powers with the potential to destroy someone's life. The argument that it's necessary for any sort of legitimate policing or security purpose when we're talking specifically about individuals who were not charged or who were acquitted at trial is tenuous at best.
Our governments and courts are on the wrong side of this issue. They're so blinded by some combination of fear, paranoia and the power of modern technologies that they seem to have forgotten the moral imperatives that everyone should be treated fairly and equally before the law and that we are supposed to be living in a free country.
It wasn't just people released with no further action. From elsewhere in the article:
The records are linked to police investigations that were terminated before charge (No Further Action or NFA cases) or to those where an individual had been acquitted at court.
Instead of complaining about "huge dangers for public safety", why isn't the Opposition asking why people who have been through potentially traumatic and potentially damaging experiences within our criminal justice system but who were never found to have done anything wrong in the eyes of the law still have deeply personal data like fingerprints and DNA on file at all?
It looks like Labour are completely on the wrong side of this one, and someone has put out a knee-jerk press release to try to score political points before thinking it through. I'm surprised, because Starmer (Labour's leader) usually comes across as quite cautious and reasonable about legal matters. He was a human rights lawyer and then the DPP (the head of our public prosecution service) before moving into politics, and I would have expected a very different position here.
Not even the government uses backups... crazy. Imagine other issues like privacy, I do not expect them to be compliant. They do not look to be that different than an average citizen at least in IT terms.
Infrastructure and operation of PNC has been outsourced to Logica (now CGI, since 2012). It's not operated by the government itself.
I consider it par the course, though in this case I cannot be objective as my $CurrentCorpo had to kick off CGI/Logica early by 2014 from multiple projects - for, appropriately, gross incompetence in managing their own data centre (a redundant data centre in Wales that goes down for a week due to border routers being out of commission and failover problems tends to ruffle some feathers at high levels, when you asked and paid for for 99.95%+ SLA).
The big thing is that government organizations are very complicated, and due to bidding processes, have a lot of different vendors with different solutions involved. Vendors respond to proposals with complete bespoke solutions that may or may not integrate well with existing infrastructure design, and then some of those vendors are actively resistant to changes that integrate them in.
It's really fun explaining to a vendor that 1. A system is going to be domain-joined so it can be managed and monitored. 2. The users will not have administrative rights to the machine. 3. The vendor will not have the ability to remote access the machine whenever they want. 4. Security software and policy settings will be applied to the machine. It continues to amaze me how often these four items absolutely blow the minds of the vendors I've dealt with.
I recently audited how equitable some funds spend on new sidewalks had been in Bloomington, Indiana.
In the process I found that one column in the project ranking spreadsheet had been sorted in the wrong order for the last five years. This effectively scrambled the final rank order.
Excel was not fault, there was a lack of peer review and clear documentation about how the formula was supposed to work.
Also, unsolicited but maybe helpful: if I have a link to a PDF on a site I try and call it out as a download in the description.
"A two page executive summary is also available for download"
I say this because even though browser settings can be set to NOT download PDF files instead of automatically opening them in the browser itself, for some reason that can vary from site to site. I click a link and all of a sudden I'm downloading something? Panic!
For those of you outside of the UK, the home office has a reputation for a horrible beauracracy where nothing works and everything gets screwed up. Imagine that instead of accidentally destroying police records, you destroyed the paperwork showing people had a legal right to live in the UK. And imagine that you then instituted a policy of deporting the people who couldn't prove they had a legal right to live in the UK. And imagine that all happened under the person who would go on to become prime minister, and that it would all come to light whilst she was prime minister....
And that wasn't the thing to cause her to resign.
This is just a minor blip for the home office, and your ideas of back ups are quaint, but that's not he problem here.
69 comments
[ 3.3 ms ] story [ 195 ms ] threadIt's worth remembering many of these records are are kept on the basis of non conviction (though there is a time limit).
Right so reading between the lines they needed to do some post-Brexit data cleaning and someone messed up the WHERE clause.
But where are the backups? Surely accidentally deleted data should be fairly easy to recover from historical backups which they surely surely must have?
The key issue is to put the backup data ‘beyond use’, even if it cannot be immediately overwritten. You must ensure that you do not use the data within the backup for any other purpose, ie that the backup is simply held on your systems until it is replaced in line with an established schedule.
https://ico.org.uk/for-organisations/guide-to-data-protectio...
Like search warrants? We know those never get abused.
> Then after a suitable grace period, once you're sure everything is ok, you destroy the backup.
But we need a search warrant to access the backup anyway so we don't really have to delete it. Best keep it, just in case.
No, like catastrophic loss of data, hence why I outlined a process of a doing this temporarily in the case of agreement with a another party to put the data entirely beyond your own use unless the party you've agreed the deletion with agrees the situation is serious enough.
> But we need a search warrant to access the backup anyway so we don't really have to delete it. Best keep it, just in case.
If you don't trust the party to delete the data, then adding in a temporary period of retention makes no difference, because in that case they could just as well have ignored the demand for the initial deletion entirely.
The only two unforgivable sins are “no working backups” and being hacked. Even being hacked is kinda forgivable cause it’s so hard to avoid. No working backups is totally unforgivable.
Any tech person who walks into any tech job at any level up from junior to CTO has a duty as one of their first tasks to find out if working backups are being done of their work.
gladly there was a backup, but the backup could not get used to get the database back-up. the discussion with IT were fabolous: "yes, we have it all backed up, so the backup is there, but we cant use it to bring the CRM back-up. but its backed up and we even have backups from the backup. they can also not get used to bringt it back up."
since then it is my standard procedure to make sure that backups can actually be used to bring projects back up. usually the first test "use backup to bring stuff back up" fails.
If you aren't allowed to verify the backups yourself, find a safe "mistake" to make to force the responsible parties to run through restoring from backups. Bonus points for using the backup testing exercise as an excuse to convince your manager to let you do a bit of refactoring to remove some particularly ugly module you've been itching to get rid of. Factor out the ugly module, then "accidentally" delete it.
My fist job out of school was writing software models of pre-prototype communications equipment for a defense contractor. There was lots of government red tape, and crazy prices being charged everywhere. We couldn't use CVS or SVN (this predated git) to store our code, because they weren't government-approved tools. We could use PVCS to store source code, but only for the build artefacts and corresponding source code of official production versions delivered to the government. (And only people that had taken a multi-hour course to be certified on PVCS could touch it. Every release involved a couple government employees coming out to the defense contractor and physically watching the one guy on the team check in the release to PVCS and burn a pair of DVDs for the government. So, we were forced to do source control by "last one out on Friday has to zip up the entire shared drive and name it with the date".
But, at least we were required to pay some crazy price to have the sysadmins backup the shared network drive to tape every week (or was it every day?). Once a month, the admins would take the current tape and send it via courier to a bomb-proof, fire-proof bunker out in the mountains in Pennsylvania or some such. They'd then put the spare tape in the tape drive to start making backups. As soon as the bunker got the latest tape, they'd send back the oldest tape via courier to become the spare tape. So, we always had at least one tape physically present in the bunker. It required 3 tapes to always have at least one tape in the bunker and one tape in the drive.
I was less than a year out of school, but based on my interactions with the system admins, I figured there was at most a 50/50 chance they got the backups correct. I asked them once how the backups were going, and got back a very low key, "yea, yea, don't worry about it, we got it" answer. I convinced my boss to let us fake a source code loss: zip up two copies of the network drive, and store the zip files on two different systems. Check the integrity of both zip files, and then "accidentally" delete a directory containing some source we were in the midst of removing anyway.
The sysadmins were unable to get our "accidentally" deleted source code back for us. They had neglected to erase the spare tape when putting it into the drive, so they were always only appending to our backups. They just ignored the errors when the tape got full.
It sounds to me like a "purge for legal reasons" function was applied to more records. Beyond some possible time limit to undo, it should not be possible to retrieve such a purge from backups. If they can retrieve this, then that is a larger problem.
If I put my tin foil hat on for a moment, I wonder if this "mistake" was a way of getting rid of some records that should have already been deleted.
Doing it this way, and publicly admitting it, might have been deemed safer than clandestinely deleting the records and then getting caught in a security audit later.
I always change that sentence to "no working restore" because that's often forgotten, but yeah it's the same as a "working backup" but changes the focus what's the important part with backups.
Same same, it was with backupexec, once the daily backup-report said everything fine, so i made a restore on a vm (just to check) then a dry-run rsync with the prod Machine (Fileserver) and it wanted to transfer about 1TB of files, the last "real" backup was 2 year ago, turned out the backupexec-agent on the Fileserver was never updated and reported that the last backup was successful (the 2 yo that was).
Trouble is a bug in the file system meant that somehow the files had been overwritten with nulls.
The program continued to run, as it sat in memory. Until one day the box was rebooted (power test or something). It didn't come back. The files were great, containing multiple files all of the right looking length. Everything was null. "main.c" or whatever it was was 8 kilobytes of null characters.
This had been happening for some months so even the oldest source backups were wiped, and the actual binaries were out too. The system was called ELVIS, which made for a great status report.
As it was on an ancient irix machine and he was rewriting in PHP at the time he just accelerated that and we were back up in a few days (the program was a front end to a database and file stores on other machines, so it was just the web part)
I have a few frequent flyers at work for "I accidentally deleted this file, corrupted this database, etc." that I end up appreciating for making me somewhat regularly pull files from backups.
That sentence i steal from you next time i have to tell someone what a backup/restore is. Thanks :)
Storing user passwords instead of hashes for the second, maybe. Bonus points if they're in plaintext.
But which the police insisted on hanging on to anyway "just in case".
Maybe it's to public benefit for it having been lost?
The system of police cautions, warnings and fixed penalties allow the police to administer punishment with none of the due process that a court would provide.
The idea of a "police caution" is a misnomer that implies it's just harmless warning, when in fact it can affect the course of someone's life many years later.
Furthermore, people are blackmailed into accepting these cautions under the threat of going to trial and potentially receiving a more severe punishment after conviction.
Even more outrageous is the retention of records from the PREVENT programme and "non-crime hate incidents". These are people who have not even committed a crime or been arrested.
The whole system stinks, and given that most people manage to avoid interaction with the police it is sadly unlikely to get the scrutiny it deserves.
The outrageous behaviour of (some of) the police during recent lockdowns actually gives me a little hope, as it as drawn widespread attention to the way they routinely abuse their power.
This view doesn't hold when the police get you for speeding (go find real criminals)
Many police officers are quite reasonable, but then you have Derbyshire, whose actions over covid have likely killed people (by reducing compliance with lockdown orders)
The difference with the police in covid is the public are suddenly seeing that the police are dealing with normal people, not just criminal scum who should be lucky they aren't shot.
As I understood it, the Protection of Freedoms Act 2012 really clamped down on DNA and fingerprint storage. Not enough, but at least it was a move in the right direction. Particularly bad was that if you were charged and found innocent, they would still keep your records for 3 years.
Unfortunately, there is an element of human nature that says, "It won't happen to me." And then maybe one day, if you're unlucky, it does happen to you. And everyone else who is similarly confident that it could never happen to them won't necessarily be there for you, because it's so far out of their experience and frame of reference that even if they're a generally decent person, they have trouble relating to the situation you're in or understanding how they might feel in your place. And so the people who are least at risk, who are often the ones who could make the most difference, don't always act as strongly or as frequently as they could to mitigate the risk to others.
It sounds like that as that is exactly what is happening - it has been decided in 2009 case (with a riveting name of "Chief Constable of Humberside Police and others v. Information Commisioner and another" [0]) that it is proper for the statutory public purpose of the Police National Computer - now replaced by Police National Database - to retain nominal records until the person in question turns 100 years of age, as introduced into the new retention policy as of 2006 [1]. Even if you have been unlawfully/wrongfully arrested, cautioned etc. your information still goes into the database but you can apply to have it expunged [2], which in practice may require support from your MP - and many simply don't know about it.
"Nominal records" in this case refer to information linked to specifc named individuals reflecting the fact they have been convicted, cautioned or reprimanded or otherwise dealth with by issue of penalty notice for disorder. It's not a "nominal" amount of data - it is quite substantial. DNA and fingerprints are supposed to be expunged from that set after three years due to "Protection of Freedoms Act 2012", rest stays.
Police National Database that replaced Police National Computer retains all data from the PNC and makes it easier to access by multiple agencies.
[0] https://commonslibrary.parliament.uk/research-briefings/sn06...
[1] http://webarchive.nationalarchives.gov.uk/+/http://www.homeo...
[2] https://www.acro.police.uk/Services/Record-deletion
It's similar to the 'IT error' in our test and trace system recently, where thousands of positive covid test results were uncounted because Excel truncated the file after a certain number of rows. In my view, it's not really the tech at fault, it's that the organization was using Excel because they didn't care enough to do it properly.
Calling it a 'technical issue' puts the blame on a malfunction, making it seem unfortunate but unavoidable, and absolving the organization that actually caused the issue.
Calling this a technical issue is deeply misleading and so egregiously so that i begin to have a flicker of doubt about motives.
Some interpretations say that if a user requests their data to be deleted, it must also be removed from backups. If your backups are kept in a binary format, then it's a nightmare. You have to restore them, delete the data then backup them up again for every backup you have.
Other interpretations say that it is enough to remove the data if/when you restore the backup data. I.e. keep a record of deletion requests and on restoring a backup check if you need to re-perform the deletion request and then do so if necessary.
Obviously there are many edge cases around deleting partial or related records that would limit the usefulness of a simple scheme like this, and obviously it's no use for data storage and processing that was already set up before the GDPR came into effect and didn't have this sort of mechanism designed in, but I suspect there are quite a lot of data controllers that have relatively simple record-keeping requirements where this kind of approach could work well.
Where I come from, that's called "negligence" and depending on motive might rise to "malfeasance".
I've had to fire people for less.
"(1)If fingerprints are required by section 63D to be destroyed, any copies of the fingerprints held by the police must also be destroyed.
(2)If a DNA profile is required by that section to be destroyed, no copy may be retained by the police except in a form which does not include information which identifies the person to whom the DNA profile relates."
Of course, it's easy for me to say these things in hindsight but my point is that this is an issue with the organization not having the right processes, rather than the technology going wrong. This is alluded to in the article: "Policing minister Kit Malthouse said the problem had been identified and the process corrected".
I've worked in health systems where personal data would get selectively removed not just from our database, but also from backups.
This is the key element. A copy may exist, but it must also be deleted. Totally a technical problem that has already be solved.
Reaching into the backups or doing per-record encryption isn't something that you can just buy at this point.
The levels of incompetence and disorganisation often displayed by these “expert” companies never ceases to amaze me.
It’s matched only by the similarly disorganised and incompetent procurement processes in public sector organisations.
So I guess that’s how this “business model” (as you put it) continues to work... sadly!
If the police cannot be trusted to put basic data protection measures in place (such as backups), perhaps they should not keep biometric data about people that have done nothing wrong[0]?
[0]: more accurately, have not been charged with wrongdoing, meaning either there was insufficient evidence, or prosecution was not deemed to be in the public interest.
Or maybe they just did not yet turn 100 years of age, when the expunge is supposed to happen automatically.
[0] https://www.acro.police.uk/Services/Record-deletion
No normal organisation processing personal data gets to keep whatever they feel like, however unjustified, and only remove it under sufferance and on specific request. That wasn't even appropriate before the GDPR came into effect, and it's clearly prohibited by law now if there was any doubt before.
So why should governments get special privileges in this respect? There could hardly be a more serious case of processing personal data than a database used by police and security services that have exceptional legal powers with the potential to destroy someone's life. The argument that it's necessary for any sort of legitimate policing or security purpose when we're talking specifically about individuals who were not charged or who were acquitted at trial is tenuous at best.
Our governments and courts are on the wrong side of this issue. They're so blinded by some combination of fear, paranoia and the power of modern technologies that they seem to have forgotten the moral imperatives that everyone should be treated fairly and equally before the law and that we are supposed to be living in a free country.
The records are linked to police investigations that were terminated before charge (No Further Action or NFA cases) or to those where an individual had been acquitted at court.
Instead of complaining about "huge dangers for public safety", why isn't the Opposition asking why people who have been through potentially traumatic and potentially damaging experiences within our criminal justice system but who were never found to have done anything wrong in the eyes of the law still have deeply personal data like fingerprints and DNA on file at all?
It looks like Labour are completely on the wrong side of this one, and someone has put out a knee-jerk press release to try to score political points before thinking it through. I'm surprised, because Starmer (Labour's leader) usually comes across as quite cautious and reasonable about legal matters. He was a human rights lawyer and then the DPP (the head of our public prosecution service) before moving into politics, and I would have expected a very different position here.
I consider it par the course, though in this case I cannot be objective as my $CurrentCorpo had to kick off CGI/Logica early by 2014 from multiple projects - for, appropriately, gross incompetence in managing their own data centre (a redundant data centre in Wales that goes down for a week due to border routers being out of commission and failover problems tends to ruffle some feathers at high levels, when you asked and paid for for 99.95%+ SLA).
It's really fun explaining to a vendor that 1. A system is going to be domain-joined so it can be managed and monitored. 2. The users will not have administrative rights to the machine. 3. The vendor will not have the ability to remote access the machine whenever they want. 4. Security software and policy settings will be applied to the machine. It continues to amaze me how often these four items absolutely blow the minds of the vendors I've dealt with.
In the process I found that one column in the project ranking spreadsheet had been sorted in the wrong order for the last five years. This effectively scrambled the final rank order.
Excel was not fault, there was a lack of peer review and clear documentation about how the formula was supposed to work.
https://mark.stosberg.com/sidewalk-equity-audit
Also, unsolicited but maybe helpful: if I have a link to a PDF on a site I try and call it out as a download in the description.
"A two page executive summary is also available for download"
I say this because even though browser settings can be set to NOT download PDF files instead of automatically opening them in the browser itself, for some reason that can vary from site to site. I click a link and all of a sudden I'm downloading something? Panic!
And that wasn't the thing to cause her to resign.
This is just a minor blip for the home office, and your ideas of back ups are quaint, but that's not he problem here.