This was one of those things that truly blew my mind: That the hospital could hand confidential patient data over to Google without their consent. My assumption was that it was just an example of the U.K.'s weaker privacy laws, but I'm glad to see that isn't the case.
Admins: please fix the typo in the title, the second word should be "regulator" not "regualtor". It's spelled correctly in the actual article title, so probably just a typo in the submission. Thanks.
> We aim to help organisations comply with PECR and promote good practice by offering advice and guidance. We will take enforcement action against organisations that persistently ignore their obligations.
I don't think that the first step they typically seem to take then is punishment, but steps to bring whatever it is the company is doing back in line with the regulations.
The trust is required to:
> establish a proper legal basis under the Data Protection Act for the Google DeepMind project and for any future trials;
> set out how it will comply with its duty of confidence to patients in any future trial involving personal data;
> complete a privacy impact assessment, including specific steps to ensure transparency; and
> commission an audit of the trial, the results of which will be shared with the Information Commissioner, and which the Commissioner will have the right to publish as she sees appropriate.
> In terms of the technical security of the dataset, it is understood that the data is
subject to encryption at rest and whilst in transit. It is also understood that the
Royal Free has received confirmation from the appropriate body that approval had
been obtained for the Logical Connection Architecture for the transfer of data, and
that the hosting location has been confirmed as compliant with two relevant
Information Security Standards2
. On this basis, the Commissioner accepts that
there is no current evidence that the data has or will be at risk of processing by
an unauthorised third party.
> - It is further understood that all access to raw patient-identifiable data by
DeepMind staff as part of the system administration is carefully logged in
an audit trail, and is only carried out under the instruction of the Royal Free
as part of the data processing. The ICO would like to make it clear that for
as long as the data remains in DeepMind’s or indeed any third parties
possession, appropriate audit trails, logs and restrictive access provisions
should be in place;
I think the level of the breaches, security of the data and how far things have been taken beyond what is legal make a big difference in what they've decided.
Both sides in a contract are responsible for them acting according to laws, yes. (one-sided punishment would be unfair though, so I think in this case the requirement for proper follow-up and doing it better next time is good resolution, unless it caused actual damage)
The issue here doesn't seem to be Google but the NHS providing the data in the first place. So the NHS is to blame. Also, even if the blame was laid on both parties you couldn't punish Google without punishing the NHS - and punishing a publicly funded organisation financially is bad for everyone. It's much more useful to ensure the issue can't happen again.
ICO doesn't have that power. They can prosecute people but only if they've deliberately breached the DPA and that doesn't seem to be the case here. The NHS should definitely hold the people responsible to account though.
So, the system for protection of personal data in the NHS is broken then - if managers can choose to flout the law without any personal repercussions then that law is useless.
IIRC you have to have a license to work with certain types of data, that person/those people need to lose grades to the point they have no controlling decision with respect to personal data. It shouldn't be up to their colleagues whether they get held responsible.
I work in a craft studio and face a large fine - so I'm told - if I reveal personal data, or make that possible ... because perish the thought someone could learn you painted a piece of pottery, but reveal personal health information against patient wishes and nothing happens?
My last sentence stated that the NHS should hold the people who screwed up responsible or fix the broken systems that allowed this to take place. As I said a fine is of no benefit to the public and prosecution would only be used in repeat or intentional breaches of the DPA (which I'm guessing this wasn't). Therefore it's up to the NHS to sort out the problem within their organisation.
>> I work in a craft studio and face a large fine - so I'm told - if I reveal personal data, or make that possible
I'm guessing you're subject to the exact same process the NHS has went through here. The only reason you would be more likely to receive the fine is because it makes sense as you're not funded by the taxpayer. Fining the NHS is in effect fining the taxpayer for breaching the rights of the taxpayer. It's the same reason the police and other public bodies don't often get fined.
Punitive punishment does nothing to stop bad practice from happening. It does make sure that people who make mistakes cover them up.
We especially do not want that in NHS organisations[1] - we want people to freely and openly admit to mistakes made.
[1] I gently dislike it when people say "The NHS" - that's not a thing, and it hasn't existed for many years now. There are 4 NHSs in the UK, and each of those is made up of a bunch of smaller organisations that have little connection to the rest.
In this situation "The NHS" did nothing. One hospital trust made this decision. Many other NHS trusts (and the Caldicott Guardians) were alarmed by the decision.
UK regulators take a light touch approach. They ask people to stop doing what they're doing; then if the behaviour continues they take further action.
This has an advantage when action eventually is taken: the offender finds it harder to claim to not know what the law is, or that their behaviour is compliant with law.
The outrage people feel about this sort of thing always feels misplaced since it assumes that their data is currently being safe guarded properly and that DeepMind/Google having it would somehow lead to worse outcomes.
Hospitals routinely get hacked by the most bottom of the barrel mass malware. I doubt small practices are any better, though I guess they may still have paper files.
I had a work experience gig for a week when I was ~14 in a hospital, and in an effort to try to occupy me they gave me an access database on 50k people including things like HIV status to mess around with.
Yeah, but when hospitals get hacked and if they are just following security protocols set out by the government, the only people to really blame are the hackers. Also what does the fact that you got to see patient data have to do with anything? Didn’t you sign forms saying you wouldn’t disclose patient information? (HIPA) Also, as if you had the authorization to conduct any sort of research or studies on them.
What’s so hard to understand about following laws? And why are you implying that the reason people are concerned is due to the safety of their data? It doesn’t matter if Google has their data, or a shitty hackable hospital has it, it shouldn’t be used in any clinical studies without approval.
>Yeah, but when hospitals get hacked and if they are just following security protocols set out by the government, the only people to really blame are the hackers.
Anyone having your medical records is bad, especially Google ymmv.
I'm sure you can imagine ways how they can abuse it from making shadow profiles, target people with med ads, not to mention they are having leverage on the competition not having access to the same data.
> Hospitals routinely get hacked by the most bottom of the barrel mass malware.
If only that. A close acquaintance of mine who I won't name worked in a big hospital in Poland for a while, and told me some horror stories related to data practices; I think it's enough to say that they could take out a lot of patient data, even by accident, and no one would notice.
Given the ease of it happening, I'm pretty sure the data is already circulating somewhere.
I think there is a difference between having your medical data accessed by some Bulgarian cyber mafia and having it accessed by Google.
Maybe I'm crazy, but I think my data in the hands of Google would be more harmful to me. I mean, next think you know my health insurer in some not very developed country that I have to move to for work, comes to find out about my past medical history and raises my premium and deductible. I find it more plausible that they would get the information from Google than from some shady Bulgarian cyber mafia.
> I think there is a difference between having your medical data accessed by some Bulgarian cyber mafia and having it accessed by Google.
Not if the Bulgarian cyber mafia hacks Google to steal that data. Okay, Google has super-ultra security and it's never been hacked (a false statement), but even so, how many companies have Google's security? I think you can barely count them on one hand. I completely expect drug cartels, for instance, to hack into organizations holding user data any time they want. If Google is easily allowed access to this data, then a dozen other companies with much weaker security will also be allowed access to it.
Have you also looked at government / semi government? I've looked at all three and I'd be more than happy to list those two as being far worse at their data security than healthcare software providers. Caveat: Europe.
As someone in the biomedical sciences in the United States, I wish the barrier to access patient information was much lower. The pendulum has swung too far the other direction--toward too much patient protection at the expense of medical progress.
Where I work, we've spent nearly 2 years trying to get approval to collect samples (with patient consent) that are being thrown away. No one on the ethics board wants to sign off on it (maybe because they are afraid of being blamed if something bad happens?). We agreed to pay ridiculous fees to de-identify samples and are crossing our fingers that we will finally get the samples.
I am glad to see that Google got access to this data and is not being punished. There has to be balance so that medical science can move forward at a reasonable rate
I totally understand why you could find it irritating to have difficulty getting information with patient consent (this sounds like the sort of frustrating bureaucracy I also deal with regularly), but how does that translate into happiness Google was able to violate people's privacy without their consent?
>> There has to be balance so that medical science can move forward at a reasonable rate.
No there doesn't. If I don't want my personal information being provided to Google by the NHS it shouldn't happen. In a country where private healthcare isn't an option for most, the only option (the NHS) shouldn't be giving personal data to private, foreign companies, in breach of the law. 'It'll make my job easier' is awful reasoning for violating privacy and the law.
Fair point. If you were in the UK I would say that means you're trying to do a job that requires you to break the law and it's been decided that it's in the best interests of society that you don't do that job.
Obviously as you said you're in the US and it sounds like your situation is very different (you haven't identified a law stopping your work just an ethics board that won't sign off for a reason you don't know).
But there are laws about protecting patient information such as the HIPAA laws, which make it much harder to do research and to start a business that processes patient information. These laws become a major consideration for any committee that is deciding whether or not to approve access to data/samples
> it's been decided that it's in the best interests of society that you don't do that job.
Remember that what has been decided, and what is actually the case, are not always the same. That's why people advocate for changing the law. OP thinks the law is too restrictive; saying "well that's what the law is" is not much of a reply.
Completely agree, I didn't mean to imply 'because it's law it's what people want' but I don't see any movement in the UK that any significant number of people would want there health data being shared with corporations.
It'll make my job of saving lives/advance medical field more easy IS a good reasoning for wanting a more streamlined mechanism to share statistical data.
No it's not. Arming all police officers, internment, and operating a shoot to kill policy would make stopping terrorism and saving lives more easy. We don't do that. Just because a particular right/freedom seems less important than an other doesn't make it so.
I'm not sure "arming police to better commit violence and take lives" is on the same level as "peacefully enabling our doctors to be better doctors".
One directly causes harm to people. The other _potentially_ causes harm, only if that data gets into the wrong hands and/or is misused. I think saving more lives is worth considering the tradeoff and striking a balance somewhere in the middle (e.g. gathering consent, opting in/out, transparently seeing who what data is shared with, etc).
> "peacefully enabling our doctors to be better doctors".
It is not that.
It’s "giving all information that exists about your medical history to a corporation that already tracks 70% of all payment processes and 96% of website accesses, and which aims to fully control your life". That’s a 1984 scenario if I’ve ever seen one.
Honestly, I feel that's quite an unrealistic stretch and not really conducive to a constructive conversation.
I want my medical history to be analyzed, especially in aggregate of everyone else's: to find anomalies, develop new medications, advance science, better understand the genome, and a myriad other "positive" outcomes that come from a system like this. I don't care if the NHS builds their own data crunching system or they contract it out to someone else (although in this case I'd lean towards preferring someone else with a track record in ML already, e.g. a leader like Google) -- what's most important is the medical advances that are all but guaranteed to come from it, not the potential threats that could come from its potential misuse.
It can be argued that Google, specifically, is too biased to handle the data (in which case, it sounds like they just need to be more transparent with how they're using it). It could be argued that it could fall into malicious hands that would blackmail or do "evil" things like raise premiums (to which transparency and security are, obviously, very important). It could be argued that the chance of misuse outweighs the chance of it doing any good, and so on. Those are good arguments, and address vulnerabilities in a system that shares extremely private data, and each concrete criticism opens up the conversation to how to improve the approach. Just saying it'll be like 1984 isn't very helpful to argue. How will it be like 1984? How can we address that?
If technology can be used for good _and_ bad, we put checks in to make sure it's used for good and not used for bad; we (as a species) don't avoid creating it altogether. Transparency, security, accountability, verifiability, etc are all problems we've solved in similar situations throughout history to make sure a system benefits everyone, even when there is a risk of bad players. I assume you have a bank account, even though a bank manager could just bag up all the money and walk out with it any day (or not).
Not to be cliche, but: with great power comes great responsibility. Utopian goals like "solving disease" are always going to come with the risk of someone misusing some new panacea-enabling technology to spread an even worse disease. We should be making sure technology is properly checked and balanced so we can actually use it to do good, not fighting technology (or advancements in health or technology) from existing in the first place because of fears of "what if".
It’s pretty simple. Tracking data and Private data should not ever be handled by any entity with a profit incentive.
Independent research institutes, funded by endowments, taxes or donations should be able to.
But for-profit companies not, because they will find a way to abuse the data.
I’ve been running IRC bouncers for quite a while, and running stat services. Just the data you can correlate from that – when are users active, what’s their ping at what times, what’s their probable timezones, when do they go afk – allows you to correlate the location of users down to a town, without even having access to the IP. With access to that, it gets even better. Merely with IRC logs you can find out who knows whom, or interacts with whom, if you have detailed enough message logs. This is enough that most people get creeped out if you show them what you can correlate merely from that, and consider it a violation of privacy.
Google is orders of magnitudes worse. The data that you can correlate simply from what Google already has about you – I’ve decompiled quite a few Google applications for research purposes, and analyzed their analytics code – is enough to deanonymize any pseudonymous data. Better yet, it allows you to track where a person is at any time, who they know and talk with, what they read, see, what they listen to. What they pay money for, and who they love.
This dataset is already extremely extensive, and it’s in control of a private corporation that frequently violates antitrust laws to gain more and more power in more and more markets, and whose former CEO has said "privacy is dead". This power is far too much for any company to control, and it should be under democratic control, and not available to people who have something to gain from abusing it.
I'm not sure "enabling reckless doctors and corporations to misuse, lose, and sell my personal data" is on the same level as...
My point being that your twisting of my words to the point they are unrecognisable is silly. Some people may look at my policing suggestion as a positive you look at it as a negative. Same thing applies to privacy. You may trust doctors (and corporations) and mistrust the police, others feel the opposite way - or trust neither.
I think I instinctively agree with your argument, although the particulars of this topic have made me wonder a bit about absolute privacy in the human health space. Specifically, we generally consider privacy not to be an absolute right, but to be one moderated by the rights of the public (i.e. if there is reasonable suspicion that you have committed a crime, police are allowed to violate your privacy and conduct searches with a warrant). I wonder at what point this might extend to issues of medical research/public health. We already know that (in the US at least) cases of certain illnesses must be reported to state/federal authorities, which to some degree violates the privacy of the patient. The selection of what sorts of data are shared is somewhat arbitrary; where is the right place to draw the line? If increasing ease of access (and perhaps violating privacy at some level) to millions of health records could bring about the discovery of more-effective predictors for heart disease 10 years faster than without this access, should this trade-off be made? I'm not saying it should, but it makes me wonder where the line should be drawn.
I also recognize that this might be a false binary, that streamlining the process might be orthogonal to maintaining privacy, but for the sake of argument let's assume they are not totally orthogonal.
I agree mostly with you but I will add that I think that sacrificing privacy for the benefit of the majority is shortsighted in this case. People must be reassured they can talk freely to their doctors (as they do with their lawyers). I think this is way more important for public health than any study that may or may not improve public health.
I agree balancing rights is important. I would argue that the link between more data and curing a disease is too weak to swing the scales. Another major issue is that you'd be violating rights on a major scale. It would be very easy to make a similar argument that, for example, recording everyones communications would stop or significantly reduce crime.
I also don't think it's a debate we really need. More and more people are wearing health tracking devices. There have been research studies done through mobile apps. People are tracking health data themselves more and more and a lot of those people are going to be willing to share it. Let it happen naturally instead of opening the floodgates and leaving people without a choice.
What can an App or a wearable tell you? Is it going to detect a tumor in your colon or your brain like a CAT scan? Can it diagnose you with a genetic disorder like being genotyped?
You cannot only infer everything aspect of health from a single ultrasound scan, which is far more detailed. So what makes you think it would be possible with an App and a heart rate sensor?
Re. the link between more data and curing disease being weak, I'd argue that's clearly false at the very least regarding how we evaluate disease treatments.
Imagine being able to have 10x more patients for a drug effectiveness study, or being able to look at the entire patient population for adverse drug reactions/interactions. All the datasets we have for this sort of thing are currently pretty poor. Despite this, the FDA mines datasets like the Adverse Events Reports database [1] to identify drugs that are performing poorly. With better, more complete data, the FDA could far more quickly identify adverse drug interactions and drugs that are harming patients more than they are helping.
Medicine is inherently about rare events. Most of us are mostly healthy most of the time. More data by definition helps you find those rare events when someone is sick or are risk. As we start to ask more difficult questions about co-occurring events, cancer detection or genome associations, our statistical tools become powerless to find the weak signals in the data. The only real solution is vastly more data.
There seems a compromise possible on the "with patient consent" bit. Maybe by default researchers could have some sort of anonymized data and patients can opt out if they want no data sharing or in if they don't mind researchers seeing it all.
Personally I don't mind Google or similar seeing my not very interesting data.
With patient consent I don't see why there would be an issue (although patients would need to be well informed - T&C's like we have for websites on health information would be terrible and abused). I'd say it should be opt-in though.
Imho there is no such thing as anonymized health care data. If the data is meaningful each dataset contains enough bits of information to de-anoymize the data when combined with other data sets. This makes Google especially dangerous as they have already much or all of the data required for de-anonymization.
Patient consent would make the data much less meaningful as it would remove datasets in a non-random way (better educated people might more likely opt-out). So that's no good solution either. I had recently a doctors visit and was asked exactly that. I opted out as I have no control over the data when released.
Sure, if that what you believe you should fight for it just like other people who think that using personal information for advancement of science is ok will fight for it.
You also have the right to not get yourself vaccinated, but is that best for society?
The reality is, the more data we have on people the more we can help people make better decisions across their lives. If aggregate medical data identifies a way to save lives or help curb waste/reduce C02 emissions etc... then we should be doing that.
Why are you glad that Google got access to illegal data? Why do things like patient confidentiality and privacy exist? It’s not like google should be immune to punishment, and it is completely unethical what the NHS and google have done, especially to the tune of a few millions violations. I’m surprised that because you had a hard time collecting patient data and that you think that medical progress is too slow, you believe illegal methods of collection are justified.
There are a large contingent of people who have more faith in Google to be more ethical than most government organizations, the NHS included. As someone who has been party to the internals at Google, and to some lesser extent with government health organizations, I am truly not sure which side I fall on these days. So I can understand why someone would be okay with Google acquiring the data.
DeepMind didn't know what they were doing was illegal. It took the ICO a year to make a decision and they set the punishment to zero, so apparently they didn't really know either. It still isn't obvious even in retrospect - the NHS itself agreed to the data transfer, and they were the owners of the data, so presumably it was the NHS that broke the law rather than Google.
With respect to patient confidentiality, don't blow it out of proportion. The NHS does not have strong controls on patient data. Breaches are common:
The FOI data discussed there is of course a minimum. Real breach levels are surely far higher given that records are mostly still stored on paper (no real way to know how many people see it).
The NHS could protect data much more strongly than it does, but it chooses to invest its time and money elsewhere, which is probably the right decision.
Finally, leaks of patient data don't kill people, but absence of life-saving drugs that could have been developed via better data do. It's easy to argue that the default should be to make large-scale datasets available to drug and tech firms. The cost/benefit ratio seems extremely positive.
DeepMind didn't know what they were doing was illegal.
How is that an excuse? From a company with hordes of lawyers? Dealing with privacy issues all the time? I'm not even sure Occam would point to a honest mistake either.
The agreement — quietly signed in September 2015 and revealed in full by New Scientist in April 2016 — gave the Google-owned artificial intelligence (AI) lab access to 1.6 million NHS patient records across three North London hospitals without patient's prior knowledge.
Assuming you are right, in that case firing their incompetent lawyers after ponying up a couple of billion pounds in fines would sound like the right course of action.
No it isn't. For one thing health is devolved in all four UK nations, so there is a separate NHS for Scotland, NI, Wales, and England.
And for England the Department for Health is the governmental department, and that's not NHS England. And NHS England isn't "The NHS" - the NHS in England is a bunch of clinical commissioning groups, and acute trusts, and mental health trusts, and GPs, and private providers selling service to the NHS (see eg lots of secure units, or lots of eating disorder inpatient provision).
In this story one NHS Hospital Trust (10,000 people employed by that trust, vs 1.7m by the rest of the NHS) made the decision, independently of NHS England or the Department of Health.
Such information should be difficult to access due to the grave consequences of its misuse. And of course greedy insurance companies, advertisers and employers would be more than happy to misuse it, if it means making more money.
Perhaps you'd like to have chat with the people on that ethics board, you might learn a thing or two. The medical science can move forward without disregarding people's rights.
I would be comfortable sharing my data (for the greater good) if I had a ironclad guarantee -- that I could believe -- that it wouldn't be used against me.
The U.S. is nowhere near that. So, no data for collective you -- except when you rip it from my fear-clenched hands, regardless.
Think I'm speaking in the abstract? In my 20's -- a few decades ago -- I had to purchase individual insurance. I had one -- one -- appointment with a psychologist in the prior two years of medical history then requested. A "touchbase" from some childhood counseling.
Fortunately, a friend of a friend was able to "unofficially" run my insurance application by the underwriting team. If it had been "officially" processed, the result would have ended up in a shared database of application results. (Yes, the insurance companies in the U.S. have long shared their underwriting decisions with each other -- surprise!)
The word back on my application: Denied! Why? (Again, I had an inside track on the process not normally available to the applicant.) Because of that one -- one -- counseling visit, in isolation and with no attendant diagnosis.
I learned my lesson. I will never share such data, that can be used against me, without ironclad assurances. And, as the past couple of decades' experience, both direct and learning by reading and observing the news and other cases, have taught me, such assurance is pretty much impossible to achieve -- even commitments made in law are taken back as soon as power shifts.
The one hope for this changing that I see? Universal health care coverage. Because then, at a societal level, you are covered regardless. Your individual history no longer plays a role in whether you are covered.
I hope that the logical, scientific side of medical science starts to see things this way: That the future, expansion, and greater facility and efficacy of their work, depends on it.
Patients (all of us) should be able to control the information and get health data in return. I'd love to just get an email with all my records and a link to a Google health app where I can get to know everything about my health that can ve inferred
This is the kind of limitation that I imagine do not exist in China, and what will allow/is allowing them to leapfrog the western world in terms of healthcare and science.
Of course in a hundred year or so China will itself be paralyzed by regulations and some other part of the world will take over.
On the other hand, China is using that data to develop a "perfect citizen" score system and use AI to censor, spy on, and deny citizens services for "bad behavior" (like say criticizing the president - but you would never do that in the U.S. anyway, right?!)
So I'm not sure we want to go in that direction anyway...
Can't they fine the strawberrynet blatant disregard for privacy?, It's based on the UK, you can access personal information of anyone registered in the site with only their email address and they say It's not a big deal.
Is there any research done on attempting to use crypto for some smart scheme of anonymizing patient data bundles to:
- allow for cryptographically safe usage of the bundles for research (with some limitations?)
- make exact mapping of the health data bundle to a particular person impractical?
Considering the miracles of public-key cryptography and blockchains, it would seem to me that there could be a scheme that would allow for something like this.
I'm curious what would happen if this breach occured in the US. We have severe penalties for these kinds of breaches outlined in HIPAA. I work in the healthcare industry as a software developer and know full well the costs for a single breach for the company. The thing about it is, there doesn't even have to be a breach per say. There just needs to be the potential for a breach for it to be considered. For instance, someone steals a laptop that has remote access to hospital records. That is a breach. Not because data was stolen, but because of the potential for data to be stolen. I don't know what kinds of privacy laws you have in the UK but if they are even remotely similar to here in the US, I will gladly watch google burn for this.
Edit: Also, the fact that people are actually OK with a corporate entity free access to our medical records is appalling. Especially on a website such as this. I can't believe people are actually defending this.
> “We were almost exclusively focused on building tools that nurses and doctors wanted, and thought of our work as technology for clinicians rather than something that needed to be accountable to and shaped by patients, the public and the NHS as a whole. We got that wrong, and we need to do better.”
Oh wow. How can any Google executive still be ignorant about the privacy implications of its data mining and tracking technologies? That seems unbelievable to me.
With such mentality, people should really question just how seriously Google takes the privacy of their email, Drive docs, and so on, if they don't even consider the privacy implications of using your data when building their data mining tools.
The UK ICO is one of the better regulators out there, and it seems the punishment (nothing) is proportionate to the actual harm done here (apparently none). But this is still a sad outcome.
The message that tech workers will take away from this sort of thing is "don't try to work with the NHS or do anything innovative in healthcare". DeepMind didn't appear to have any actual business plan in mind for Streams - all the stuff I read from them on this project implied it was driven simply by a desire to help the UK's national treasure/obsession. Being owned by Google does let important execs do pet projects like this so I don't see any need for conspiracy theories here.
Both the ICO and the NHS are part of the government and yet they reached different decisions about the same laws, I expect DeepMind had their own legal advice which also concluded it was OK. It is unlikely any other company or startup could get a hard-and-fast 100% guaranteed correct deal with the NHS to do things with patient data.
It's doubly unfortunate because the NHS has deeply dysfunctional IT and badly needs the assistance of data specialists. It faces huge challenges to become more efficient and shows little ability to do so. Healthcare records are still passed around on paper, the biggest IT failure in history was an attempt to digitise the NHS, medical data leaks all over the place and is guarded far more loosely than Google would ever consider, many genetic diseases are fairly rare and need a lot of data analysis to make progress on, and so on. Healthcare and the tech industry should be a match made in heaven. Instead the very, very few attempts to dance together inevitably seem to end badly.
I mostly agree with you - healthcare and tech should absolutely be a match made in heaven, but here in the UK it's the ethics and data protection laws which make that so difficult.
Even with explicit consent from patients (and this article states this was not the case), the rules make it very hard to work effectively with the data. I can give a couple of examples.
[The 100'000 genomes project](https://www.genomicsengland.co.uk/the-100000-genomes-project...) is in the process of sequencing 100'000 genomes of patients with cancer and rare disease and will be the largest sequencing project in the world once completed. Because whole genome data can potentially identify a person and give information on conditions they have or might be prone to, their genomic data is seen as confidential. As a result, even with explicit patient permission, the data can only be accessed through a remote desktop onto a machine which does not have internet access, which makes working with some genomic tools (which rely on large databases) very problematic. I understand they're working on this, but it is just an example of the 'inconvenience' of abiding by security rules.
Another example, with my own research, is that even with patient consent, the thumbs up from the ethics committee, and being clinically involved with patients (with a corresponding contract with the hospital), I am unable to store patient information on encrypted devices on university computers. Instead this must be stored on NHS computers, even though the analysis occurs on university computers.
Having said all of that, there is a way to get really great data from many patients using NHS IT systems - [the million women study](http://www.millionwomenstudy.org/methods/) successfully did this and have not run into any ethical/regulatory problems. That's because they did it the right way - i.e. it is formally registered as research, permission was asked and received from the regional ethics committee, patients were given information, both by paper and verbally, to ask for consent for the study.
Of course this study was run by Oxford University, who, like other universities, know clearly the pitfalls and hoops of conducting research on patients. This is a whole world of bureaucracy (which has come about for good reasons), which tech companies will have to navigate and abide by if they wish to carry out human research.
> The message that tech workers will take away from this sort of thing is "don't try to work with the NHS or do anything innovative in healthcare"
They shouldn't, because ICO has been clear that this decision should not deter people from this type of project, and that the modifications needed are relatively minor.
I am in the healthcare industry (public services) and I actually agree with both sides of the argument. Freeing healthcare data from its current ethical/legal shackles would definitely allow for great things in science. It is just as true that it destroys privacy at the population level. There is no doubt that the data will also be used to evil ends and restrains everyone's right to autodetermination (destabilizing politicians, discrimination on hiring, etc.) so the question is simply "do we, as the whole population, want to take that risk?". Some will say yes, others no. This should be subject to a vote at the national level. Progress has often impinged on individual freedom in the past, and I am pretty confident we will solve this the usual way: without asking the public's opinion.
89 comments
[ 0.22 ms ] story [ 137 ms ] threadWeaker than who's?
What the hell? Why do we let Google get away with this.
> We aim to help organisations comply with PECR and promote good practice by offering advice and guidance. We will take enforcement action against organisations that persistently ignore their obligations.
I don't think that the first step they typically seem to take then is punishment, but steps to bring whatever it is the company is doing back in line with the regulations.
The trust is required to:
> establish a proper legal basis under the Data Protection Act for the Google DeepMind project and for any future trials;
> set out how it will comply with its duty of confidence to patients in any future trial involving personal data;
> complete a privacy impact assessment, including specific steps to ensure transparency; and
> commission an audit of the trial, the results of which will be shared with the Information Commissioner, and which the Commissioner will have the right to publish as she sees appropriate.
edit - ICO site https://ico.org.uk/about-the-ico/news-and-events/news-and-bl...
Edit - some more details about the case
> In terms of the technical security of the dataset, it is understood that the data is subject to encryption at rest and whilst in transit. It is also understood that the Royal Free has received confirmation from the appropriate body that approval had been obtained for the Logical Connection Architecture for the transfer of data, and that the hosting location has been confirmed as compliant with two relevant Information Security Standards2 . On this basis, the Commissioner accepts that there is no current evidence that the data has or will be at risk of processing by an unauthorised third party.
> - It is further understood that all access to raw patient-identifiable data by DeepMind staff as part of the system administration is carefully logged in an audit trail, and is only carried out under the instruction of the Royal Free as part of the data processing. The ICO would like to make it clear that for as long as the data remains in DeepMind’s or indeed any third parties possession, appropriate audit trails, logs and restrictive access provisions should be in place;
https://ico.org.uk/media/action-weve-taken/undertakings/2014...
I think the level of the breaches, security of the data and how far things have been taken beyond what is legal make a big difference in what they've decided.
So Google signed a deal with a government agency that was approved by said agency and they should be fined for that? Wow.
There are several layers between the Royal Free and government.
https://en.wikipedia.org/wiki/NHS_foundation_trust
IIRC you have to have a license to work with certain types of data, that person/those people need to lose grades to the point they have no controlling decision with respect to personal data. It shouldn't be up to their colleagues whether they get held responsible.
I work in a craft studio and face a large fine - so I'm told - if I reveal personal data, or make that possible ... because perish the thought someone could learn you painted a piece of pottery, but reveal personal health information against patient wishes and nothing happens?
>> I work in a craft studio and face a large fine - so I'm told - if I reveal personal data, or make that possible
I'm guessing you're subject to the exact same process the NHS has went through here. The only reason you would be more likely to receive the fine is because it makes sense as you're not funded by the taxpayer. Fining the NHS is in effect fining the taxpayer for breaching the rights of the taxpayer. It's the same reason the police and other public bodies don't often get fined.
We especially do not want that in NHS organisations[1] - we want people to freely and openly admit to mistakes made.
[1] I gently dislike it when people say "The NHS" - that's not a thing, and it hasn't existed for many years now. There are 4 NHSs in the UK, and each of those is made up of a bunch of smaller organisations that have little connection to the rest.
In this situation "The NHS" did nothing. One hospital trust made this decision. Many other NHS trusts (and the Caldicott Guardians) were alarmed by the decision.
That is highly questionable.
This has an advantage when action eventually is taken: the offender finds it harder to claim to not know what the law is, or that their behaviour is compliant with law.
Hospitals routinely get hacked by the most bottom of the barrel mass malware. I doubt small practices are any better, though I guess they may still have paper files.
I had a work experience gig for a week when I was ~14 in a hospital, and in an effort to try to occupy me they gave me an access database on 50k people including things like HIV status to mess around with.
What’s so hard to understand about following laws? And why are you implying that the reason people are concerned is due to the safety of their data? It doesn’t matter if Google has their data, or a shitty hackable hospital has it, it shouldn’t be used in any clinical studies without approval.
Given DeepMind didn't do any trials, I'm not sure how that is relevant.
Labelling statistical analysis as a "trial" seems like it's own farce.
I'd argue the regulators are also to blame.
I'm sure you can imagine ways how they can abuse it from making shadow profiles, target people with med ads, not to mention they are having leverage on the competition not having access to the same data.
If only that. A close acquaintance of mine who I won't name worked in a big hospital in Poland for a while, and told me some horror stories related to data practices; I think it's enough to say that they could take out a lot of patient data, even by accident, and no one would notice.
Given the ease of it happening, I'm pretty sure the data is already circulating somewhere.
Maybe I'm crazy, but I think my data in the hands of Google would be more harmful to me. I mean, next think you know my health insurer in some not very developed country that I have to move to for work, comes to find out about my past medical history and raises my premium and deductible. I find it more plausible that they would get the information from Google than from some shady Bulgarian cyber mafia.
Not if the Bulgarian cyber mafia hacks Google to steal that data. Okay, Google has super-ultra security and it's never been hacked (a false statement), but even so, how many companies have Google's security? I think you can barely count them on one hand. I completely expect drug cartels, for instance, to hack into organizations holding user data any time they want. If Google is easily allowed access to this data, then a dozen other companies with much weaker security will also be allowed access to it.
Where I work, we've spent nearly 2 years trying to get approval to collect samples (with patient consent) that are being thrown away. No one on the ethics board wants to sign off on it (maybe because they are afraid of being blamed if something bad happens?). We agreed to pay ridiculous fees to de-identify samples and are crossing our fingers that we will finally get the samples.
I am glad to see that Google got access to this data and is not being punished. There has to be balance so that medical science can move forward at a reasonable rate
No there doesn't. If I don't want my personal information being provided to Google by the NHS it shouldn't happen. In a country where private healthcare isn't an option for most, the only option (the NHS) shouldn't be giving personal data to private, foreign companies, in breach of the law. 'It'll make my job easier' is awful reasoning for violating privacy and the law.
Obviously as you said you're in the US and it sounds like your situation is very different (you haven't identified a law stopping your work just an ethics board that won't sign off for a reason you don't know).
Remember that what has been decided, and what is actually the case, are not always the same. That's why people advocate for changing the law. OP thinks the law is too restrictive; saying "well that's what the law is" is not much of a reply.
One directly causes harm to people. The other _potentially_ causes harm, only if that data gets into the wrong hands and/or is misused. I think saving more lives is worth considering the tradeoff and striking a balance somewhere in the middle (e.g. gathering consent, opting in/out, transparently seeing who what data is shared with, etc).
It is not that.
It’s "giving all information that exists about your medical history to a corporation that already tracks 70% of all payment processes and 96% of website accesses, and which aims to fully control your life". That’s a 1984 scenario if I’ve ever seen one.
I want my medical history to be analyzed, especially in aggregate of everyone else's: to find anomalies, develop new medications, advance science, better understand the genome, and a myriad other "positive" outcomes that come from a system like this. I don't care if the NHS builds their own data crunching system or they contract it out to someone else (although in this case I'd lean towards preferring someone else with a track record in ML already, e.g. a leader like Google) -- what's most important is the medical advances that are all but guaranteed to come from it, not the potential threats that could come from its potential misuse.
It can be argued that Google, specifically, is too biased to handle the data (in which case, it sounds like they just need to be more transparent with how they're using it). It could be argued that it could fall into malicious hands that would blackmail or do "evil" things like raise premiums (to which transparency and security are, obviously, very important). It could be argued that the chance of misuse outweighs the chance of it doing any good, and so on. Those are good arguments, and address vulnerabilities in a system that shares extremely private data, and each concrete criticism opens up the conversation to how to improve the approach. Just saying it'll be like 1984 isn't very helpful to argue. How will it be like 1984? How can we address that?
If technology can be used for good _and_ bad, we put checks in to make sure it's used for good and not used for bad; we (as a species) don't avoid creating it altogether. Transparency, security, accountability, verifiability, etc are all problems we've solved in similar situations throughout history to make sure a system benefits everyone, even when there is a risk of bad players. I assume you have a bank account, even though a bank manager could just bag up all the money and walk out with it any day (or not).
Not to be cliche, but: with great power comes great responsibility. Utopian goals like "solving disease" are always going to come with the risk of someone misusing some new panacea-enabling technology to spread an even worse disease. We should be making sure technology is properly checked and balanced so we can actually use it to do good, not fighting technology (or advancements in health or technology) from existing in the first place because of fears of "what if".
Independent research institutes, funded by endowments, taxes or donations should be able to.
But for-profit companies not, because they will find a way to abuse the data.
I’ve been running IRC bouncers for quite a while, and running stat services. Just the data you can correlate from that – when are users active, what’s their ping at what times, what’s their probable timezones, when do they go afk – allows you to correlate the location of users down to a town, without even having access to the IP. With access to that, it gets even better. Merely with IRC logs you can find out who knows whom, or interacts with whom, if you have detailed enough message logs. This is enough that most people get creeped out if you show them what you can correlate merely from that, and consider it a violation of privacy.
Google is orders of magnitudes worse. The data that you can correlate simply from what Google already has about you – I’ve decompiled quite a few Google applications for research purposes, and analyzed their analytics code – is enough to deanonymize any pseudonymous data. Better yet, it allows you to track where a person is at any time, who they know and talk with, what they read, see, what they listen to. What they pay money for, and who they love.
This dataset is already extremely extensive, and it’s in control of a private corporation that frequently violates antitrust laws to gain more and more power in more and more markets, and whose former CEO has said "privacy is dead". This power is far too much for any company to control, and it should be under democratic control, and not available to people who have something to gain from abusing it.
My point being that your twisting of my words to the point they are unrecognisable is silly. Some people may look at my policing suggestion as a positive you look at it as a negative. Same thing applies to privacy. You may trust doctors (and corporations) and mistrust the police, others feel the opposite way - or trust neither.
I also recognize that this might be a false binary, that streamlining the process might be orthogonal to maintaining privacy, but for the sake of argument let's assume they are not totally orthogonal.
I also don't think it's a debate we really need. More and more people are wearing health tracking devices. There have been research studies done through mobile apps. People are tracking health data themselves more and more and a lot of those people are going to be willing to share it. Let it happen naturally instead of opening the floodgates and leaving people without a choice.
You cannot only infer everything aspect of health from a single ultrasound scan, which is far more detailed. So what makes you think it would be possible with an App and a heart rate sensor?
Imagine being able to have 10x more patients for a drug effectiveness study, or being able to look at the entire patient population for adverse drug reactions/interactions. All the datasets we have for this sort of thing are currently pretty poor. Despite this, the FDA mines datasets like the Adverse Events Reports database [1] to identify drugs that are performing poorly. With better, more complete data, the FDA could far more quickly identify adverse drug interactions and drugs that are harming patients more than they are helping.
Medicine is inherently about rare events. Most of us are mostly healthy most of the time. More data by definition helps you find those rare events when someone is sick or are risk. As we start to ask more difficult questions about co-occurring events, cancer detection or genome associations, our statistical tools become powerless to find the weak signals in the data. The only real solution is vastly more data.
[1] https://www.fda.gov/drugs/guidancecomplianceregulatoryinform...
Personally I don't mind Google or similar seeing my not very interesting data.
Patient consent would make the data much less meaningful as it would remove datasets in a non-random way (better educated people might more likely opt-out). So that's no good solution either. I had recently a doctors visit and was asked exactly that. I opted out as I have no control over the data when released.
The reality is, the more data we have on people the more we can help people make better decisions across their lives. If aggregate medical data identifies a way to save lives or help curb waste/reduce C02 emissions etc... then we should be doing that.
DeepMind didn't know what they were doing was illegal. It took the ICO a year to make a decision and they set the punishment to zero, so apparently they didn't really know either. It still isn't obvious even in retrospect - the NHS itself agreed to the data transfer, and they were the owners of the data, so presumably it was the NHS that broke the law rather than Google.
With respect to patient confidentiality, don't blow it out of proportion. The NHS does not have strong controls on patient data. Breaches are common:
http://www.dailymail.co.uk/news/article-2833960/NHS-data-sno...
The FOI data discussed there is of course a minimum. Real breach levels are surely far higher given that records are mostly still stored on paper (no real way to know how many people see it).
The NHS could protect data much more strongly than it does, but it chooses to invest its time and money elsewhere, which is probably the right decision.
Finally, leaks of patient data don't kill people, but absence of life-saving drugs that could have been developed via better data do. It's easy to argue that the default should be to make large-scale datasets available to drug and tech firms. The cost/benefit ratio seems extremely positive.
How is that an excuse? From a company with hordes of lawyers? Dealing with privacy issues all the time? I'm not even sure Occam would point to a honest mistake either.
The agreement — quietly signed in September 2015 and revealed in full by New Scientist in April 2016 — gave the Google-owned artificial intelligence (AI) lab access to 1.6 million NHS patient records across three North London hospitals without patient's prior knowledge.
No it isn't. For one thing health is devolved in all four UK nations, so there is a separate NHS for Scotland, NI, Wales, and England.
And for England the Department for Health is the governmental department, and that's not NHS England. And NHS England isn't "The NHS" - the NHS in England is a bunch of clinical commissioning groups, and acute trusts, and mental health trusts, and GPs, and private providers selling service to the NHS (see eg lots of secure units, or lots of eating disorder inpatient provision).
In this story one NHS Hospital Trust (10,000 people employed by that trust, vs 1.7m by the rest of the NHS) made the decision, independently of NHS England or the Department of Health.
Perhaps you'd like to have chat with the people on that ethics board, you might learn a thing or two. The medical science can move forward without disregarding people's rights.
The U.S. is nowhere near that. So, no data for collective you -- except when you rip it from my fear-clenched hands, regardless.
Think I'm speaking in the abstract? In my 20's -- a few decades ago -- I had to purchase individual insurance. I had one -- one -- appointment with a psychologist in the prior two years of medical history then requested. A "touchbase" from some childhood counseling.
Fortunately, a friend of a friend was able to "unofficially" run my insurance application by the underwriting team. If it had been "officially" processed, the result would have ended up in a shared database of application results. (Yes, the insurance companies in the U.S. have long shared their underwriting decisions with each other -- surprise!)
The word back on my application: Denied! Why? (Again, I had an inside track on the process not normally available to the applicant.) Because of that one -- one -- counseling visit, in isolation and with no attendant diagnosis.
I learned my lesson. I will never share such data, that can be used against me, without ironclad assurances. And, as the past couple of decades' experience, both direct and learning by reading and observing the news and other cases, have taught me, such assurance is pretty much impossible to achieve -- even commitments made in law are taken back as soon as power shifts.
The one hope for this changing that I see? Universal health care coverage. Because then, at a societal level, you are covered regardless. Your individual history no longer plays a role in whether you are covered.
I hope that the logical, scientific side of medical science starts to see things this way: That the future, expansion, and greater facility and efficacy of their work, depends on it.
Of course in a hundred year or so China will itself be paralyzed by regulations and some other part of the world will take over.
So I'm not sure we want to go in that direction anyway...
http://www.ibtimes.com/china-use-big-data-rate-citizens-new-...
http://boingboing.net/2017/02/16/chinas-citizen-scores-us.ht...
The list of legislation is here: https://ico.org.uk/about-the-ico/what-we-do/
- allow for cryptographically safe usage of the bundles for research (with some limitations?)
- make exact mapping of the health data bundle to a particular person impractical?
Considering the miracles of public-key cryptography and blockchains, it would seem to me that there could be a scheme that would allow for something like this.
Edit: Also, the fact that people are actually OK with a corporate entity free access to our medical records is appalling. Especially on a website such as this. I can't believe people are actually defending this.
HEALTH: https://ico.org.uk/action-weve-taken/enforcement/?facet_type...
LOCAL GOVERNMENT: https://ico.org.uk/action-weve-taken/enforcement/?facet_type...
Oh wow. How can any Google executive still be ignorant about the privacy implications of its data mining and tracking technologies? That seems unbelievable to me.
https://www.theguardian.com/technology/2017/jul/03/google-de...
With such mentality, people should really question just how seriously Google takes the privacy of their email, Drive docs, and so on, if they don't even consider the privacy implications of using your data when building their data mining tools.
The message that tech workers will take away from this sort of thing is "don't try to work with the NHS or do anything innovative in healthcare". DeepMind didn't appear to have any actual business plan in mind for Streams - all the stuff I read from them on this project implied it was driven simply by a desire to help the UK's national treasure/obsession. Being owned by Google does let important execs do pet projects like this so I don't see any need for conspiracy theories here.
Both the ICO and the NHS are part of the government and yet they reached different decisions about the same laws, I expect DeepMind had their own legal advice which also concluded it was OK. It is unlikely any other company or startup could get a hard-and-fast 100% guaranteed correct deal with the NHS to do things with patient data.
It's doubly unfortunate because the NHS has deeply dysfunctional IT and badly needs the assistance of data specialists. It faces huge challenges to become more efficient and shows little ability to do so. Healthcare records are still passed around on paper, the biggest IT failure in history was an attempt to digitise the NHS, medical data leaks all over the place and is guarded far more loosely than Google would ever consider, many genetic diseases are fairly rare and need a lot of data analysis to make progress on, and so on. Healthcare and the tech industry should be a match made in heaven. Instead the very, very few attempts to dance together inevitably seem to end badly.
Even with explicit consent from patients (and this article states this was not the case), the rules make it very hard to work effectively with the data. I can give a couple of examples.
[The 100'000 genomes project](https://www.genomicsengland.co.uk/the-100000-genomes-project...) is in the process of sequencing 100'000 genomes of patients with cancer and rare disease and will be the largest sequencing project in the world once completed. Because whole genome data can potentially identify a person and give information on conditions they have or might be prone to, their genomic data is seen as confidential. As a result, even with explicit patient permission, the data can only be accessed through a remote desktop onto a machine which does not have internet access, which makes working with some genomic tools (which rely on large databases) very problematic. I understand they're working on this, but it is just an example of the 'inconvenience' of abiding by security rules.
Another example, with my own research, is that even with patient consent, the thumbs up from the ethics committee, and being clinically involved with patients (with a corresponding contract with the hospital), I am unable to store patient information on encrypted devices on university computers. Instead this must be stored on NHS computers, even though the analysis occurs on university computers.
Having said all of that, there is a way to get really great data from many patients using NHS IT systems - [the million women study](http://www.millionwomenstudy.org/methods/) successfully did this and have not run into any ethical/regulatory problems. That's because they did it the right way - i.e. it is formally registered as research, permission was asked and received from the regional ethics committee, patients were given information, both by paper and verbally, to ask for consent for the study.
Of course this study was run by Oxford University, who, like other universities, know clearly the pitfalls and hoops of conducting research on patients. This is a whole world of bureaucracy (which has come about for good reasons), which tech companies will have to navigate and abide by if they wish to carry out human research.
They shouldn't, because ICO has been clear that this decision should not deter people from this type of project, and that the modifications needed are relatively minor.
https://iconewsblog.wordpress.com/2017/07/03/four-lessons-nh...
https://ico.org.uk/about-the-ico/news-and-events/news-and-bl...
> and the NHS
It wasn't "The NHS" (eg NHS England) but a single hospital trust. Many other NHS trusts and Caldicott Guardians expressed concern.
No it shouldn't. It should be a personal decision for each individual.