Our health is at risk because the EPA is choosing not to take scientific publications at face value with not even the most basic way to check work?
Have the authors of this article actually confirmed that this would include non-anonymized data unfit for public consumption?
Added: The actual document does not specify "raw data", it is much more specific about what data they expect.
> The proposed regulation provides that, for the science pivotal to its significant regulatory actions, EPA will ensure that the data and models underlying the science is publicly available in a manner sufficient for validation and analysis.
"A manner sufficient for validation and analysis" is a completely different standard than "all the data the study produced".
Am I the only one worried about the path the world is on? Fascism (or all-out nazism in Europe) on the rise, anti-intellectual and anti-press movements stirring, world leaders shunning science. It feels like we’re well on our way into a new dark age.
Consider the following scenario. A chemical X is spilled in river. A sequence of very small towns with similar populations dot the river both up-stream down-stream. This is a dream setting for good science -- a true natural experiment with lots of genetic and environmental similarity between the groups in each small town, but radically different levels of exposure to X. Fast forward 10 years and researchers find that concentrations of X above some threshold N drastically increase the prevalence of birth defects. However, due to the size of the populations, researchers may not be able to publish their data and analysis publicly without de facto publishing almost complete de-anonymizable data.
The scenario is not so hypothetical. The circumstances and particulars change, but this situation describes a lot of medical and environmental research.
Should the EPA take the researcher's word? Maybe not. But there's a lot of workable middle ground in-between "blind trust" and "anyone with access to a library computer and a stats textbook can figure out participant 494 -- who math tells us can only possibly be Johnny D. of Smallsville, VA -- has testicular cancer".
And the situation is even more difficult than this because de-anonymization is an active area of research with some surprisingly powerful results published in recent years; i.e., it is highly non-trivial to determine if a dataset is de-anonymizable. And the answer always depends on what data about a person's identity is publicly available, which especially in the age of annual massive data leaks change from year to year. So even if researchers don't think it's possible to de-anonymize study participants using today's mathematical tools and today's public datasets, they'll often choose to keep the dataset private just in case.
Is science good (or "the best!") if the available data are not sufficient to verify the analysis? This is not a "world leader" doing anything that could be described as "shunning science".
I don't know. Is Newton's work worthless, because we no longer have access to the full set of raw data that he based it off?
There are thousands of old studies for which the data is unavailable, incomplete, was lost, etc. Are you going to pay for all of them to be reproduced? Are you ready to give up penicillin? Surgeons washing their hands between operations? The links between tobacco and cancer? Every single medical advancement (Because patient confidentiality precludes the release of raw data that goes into medical studies)?
I'm all for reproducibility, but this has enormous consequences.
In the future please avoid using link shorteners (especially bit.ly). The links expire and tend to leave older forum threads full of frustrating link graveyards. In this case, the link points to https://www.gatesnotes.com/Development/Max-Roser-three-facts...
The world was approxiately 100 trillion times more bleak in the 1930s and 1940s. We weren't on our way to a new dark age then either. This is an extraordinary golden age by comparison. There are no world powers currently trying to kill each other, the biggest disputes are mostly between richer nations and about trade, and those disputes are trivial compared to the past. War is rare, famine is rare, food is relatively plentiful, as is global aid, vaccines, etc. Progress across nearly all fields of science and technology continues, rapidly so.
Every category you purport to be causes for celebration are presently reversing themselves the world over due to ignorant, nationalist, neo-fascist governments like the one systematically dismantling the EPA discussed in the article.
The groundwork was laid in the US starting in the 1970s.
With the founding of Cato, Searle, Olin fnds. Murdoch bought the NY Post in 1976. Fairness doctrine was repealed in 1987 allowing Rush Limbaugh to go bigtime. Fox started in 1997.
Just the authoritarian “conservative” movement that seeks to concentrate power.
I am actually a conservative on many issues, including UBI, work rules, tariffs, caution when making new law and regulations, and appropriate pricing of risk. However the real conservative movement has been redirected as a way to link wealthy donors’ interests with identity politics, for votes. Part and parcel of that is a huge propaganda machine which spouts lies. That propaganda is corrupting the West and I am afraid for its effect on democracy and average people.
searching the page for the word "rule" hits 13 occurrences. The only real "meat" if you can call it that surrounding what that rule is was this: Next week (July 17), EPA will hold its only public hearing on the proposed rule called “Strengthening Transparency in Regulatory Science.”
As far as I can see that's the only immediate information this article bothered to speak about the rule itself. Anybody see a summary on this rule to share with the crowd? Or did I miss something?
EPA is proposing to eliminate consideration of scientific studies unless the raw study data are made publicly available — possibly including participants’ personal, confidential, and private information — data that may be subject to legal, ethical, and human subject research protections.
This requirement would eliminate human health studies that use integral medical, lifestyle, and geographic data, as well as studies that include confidential business information, such as information from studies conducted by industry to demonstrate safety of a pesticide or other toxic chemical. It could also eliminate consideration of older studies for which data are no longer available or accessible, even if the data have been reanalyzed and the studies have been validated, replicated, reproduced, and undergone rigorous and independent peer review.
Far from using the best available science for EPA decision-making, the proposal would severely limit the studies and scientific evidence the agency would use to fulfill is statutory mission to protect human health and the environment. This is not a new idea. It is the result of a decades-long campaign to undercut reliance on groundbreaking research...
Instead of quoting the article, how about we quote the actual proposed regulation?
The proposed regulation provides that when EPA develops regulations, including regulations for which the public is likely to bear the cost of compliance, with regard to those scientific studies that are pivotal to the action being taken, EPA should ensure that the data underlying those are publicly available in a manner sufficient for independent validation.[1]
Seem eminently reasonable to me. And seems aligned with the "open science" movement to me.
And in terms of privacy...
EPA believes that concerns about access to confidential or private information can, in many cases, be addressed through the application of solutions commonly in use across some parts of the Federal government. Nothing in the proposed rule compels the disclosure of any confidential or private information in a manner that violates applicable legal and ethical protections.
Why would there be zero studies that study lead and have source data? What happened to those that studied lead - did they destroy their data for some reason? Are the data about lead somehow unique so that, unlike any other data, they can't be published? Did some weird catastrophe occur which erased all data regarding to lead studies, and nobody knows about it? What would be the reason that basic scientific norms would not be preserved in this case? I guess if it happened, we'd need to conduct new studies - and find people who screwed up the old ones and make sure they are not responsible for any new studies ever again, to avoid future screwups.
Good luck tracking down data from ~50 year old studies published in 1970. Further, their is little point in replicating old studies so often that's all we got.
1970 is not a Stone Age, and civilization didn't collapse since then. Why would it impossible to have data? Also, does it mean if a study in 1970 made a mistake, we're now doomed to carry on with this mistake forever? And if not, how would we discover it if not by trying to replicate/re-analyze data?
New research is overwhelmingly more likely to hurt industrial interests than help them, so adding any hoops at all is likely going to benefit industry. In fact, all that this does, from a regulatory perspective, is provide a pathway for the EPA to accept and reject studies at its own discretion, without having to ever actually call the result into question.
Yes we do, and it doesn't matter. Data isn't relevant to the reproduction discussion. The reproducibility crisis is a crisis of methodology, not falsified or misinterpreted data. If I p-hack a result, then no amount of transparency is going to demonstrate that that's what I've done. The only standard of evidence for reproducibility is reproduction, and they are only using the "reproducibility crisis" as a diversionary tactic to increase the cost of doing science to an unacceptable level, at the same time as pulling EPA funding of science.
Give me some examples of science where we make decisions and keep the raw data a secret. I can’t think of any except ones involving national security.
Yes, the best way to prove reproducibility is to reproduce it, but that can’t always be done and at any rate is no excuse for keeping raw data a secret.
Yes, they are. Do you have any familiarity with studies conducted under a human study IRB permission? Please cite one that would permit sufficient sharing with the public of the detailed information needed to replicate the study. Even just sharing one personal detail, such as gender, or age, is almost always completely forbidden.
Also, please don't be so credulous. This rule has been promulgated, invented, and proagandized by people who are trying to hamstring the EPA from using scientific studies to regulate industry. It's not new, it's just newly empowered.
You appear to be falling for the plausible cover story, which is disinformation. People are really bad at being able to discern the intent behind "plausibly good" organizations and movements.
Nope, they are saying "disclose information in ways that do not violate privacy, ask us how if you don't know how to do it, we're already doing it. But if you don't bother, only then we won't consider your study".
>With this notice, EPA is soliciting public comment on a proposed regulation designed to provide a mechanism to increase access to dose response data and models underlying pivotal regulatory science in a manner consistent with statutory requirements for protection of privacy and confidentiality of research participants, protection of proprietary data and confidential business information, and other compelling interests.
>The proposal takes comment on how to ensure that, over time, more of the data and models underlying the science that informs regulatory decisions (over and above the dose response data and models underlying “pivotal regulatory science”) is available to the public for validation [13] in a manner that honors legal and ethical obligations to reduce the risks of unauthorized disclosure and re-identification.
>EPA believes that concerns about access to confidential or private information can, in many cases, be addressed through the application of solutions commonly in use across some parts of the Federal government.[16] Nothing in the proposed rule compels Start Printed Page 18771the disclosure of any confidential or private information in a manner that violates applicable legal and ethical protections. Other federal agencies have developed tools and methods to de-identify private information for a variety of disciplines.[17] The National Academies have noted that simple data masking, coding, and de-identification techniques have been developed over the last half century and that “Nothing in the past suggests that increasing access to research data without damage to privacy and confidentiality rights is beyond scientific reach.” [18] More recently, both the National Academies and the Bipartisan Commission on Evidence Based Policy [19] have discussed the challenges and opportunities for facilitating to secure access to confidential data for non-government analysts.
>Considering the breadth of dose response data and models used in the development of significant EPA regulations, the requirements for availability may differ. These mechanisms may range from deposition in public data repositories, consistent with requirements for many scientific journals,[20] to, for certain types of information, controlled access in federal research data centers that facilitate secondary research use by the public.[21] EPA should collaborate with other federal agencies to identify strategies to protect confidential and private information in any circumstance in which it is making information publicly available. These strategies should be cost-effective and may also include: Requiring applications for access; restricting access to data for the purposes of replication, validation, and sensitivity evaluation; establishing physical controls on data storage; online training for researchers; and nondisclosure agreements.[22]
It's fascinating. EPA says "we won't consider your research unless it's done in open and reproducible fashion so that other people can verify your results and reproduce the conclusions" and people hear "we hate science and want to undercut research". Amazing.
The proposed regulation provides that, for the science pivotal to its significant regulatory actions, EPA will ensure that the data and models underlying the science is publicly available
The public availability requirement sounds nice on paper, but in practice, it means ignoring a lot of science during rule-making because the original data can't be published due to genuine privacy concerns.
There are some common-sense workarounds. For example, requiring an EPA or third-party audit of the original dataset and analysis. Or simply carving out exceptions for research whenever it can be demonstrated that an IRB (and therefore federal law) would not have allowed the research without restricting public access to data/analysis.
> The public availability requirement sounds nice on paper, but in practice, it means ignoring a lot of science during rule-making because the original data can't be published due to genuine privacy concerns.
It is more limited if you read the whole sentence, which goes on to qualify which data they are interested in (specifically "publicly available in a manner sufficient for validation and analysis.").
which de facto excludes 90% of medical research because medical patients have privacy rights and researchers have legal obligations to respect those rights.
These rules are a hot political topic that are years in the making. The people pushing them are not scientists and have never expressed genuine interest in reproducibility outside of this one issue. And scientists who have spent years fighting for reproducibility have publicly stated they oppose this rule. And all the major proponents have strong anti-regulatory preferences.
These rules are not about reproducibility. This rule is an intentionally crafted Catch-22 designed to exclude medical research from consideration so it's easier to justify repealing or not implementing environmental regulations. Otherwise, the rules would find a middle ground; e.g., by allowing the EPA access to relevant scientific data and analysis without risking https://en.wikipedia.org/wiki/De-anonymization by posting sensitive medical information about thousands of patients on the public internet.
It's very hard to believe it is not possible to publish medical data in an anonymized fashion that does not infringe privacy rights. If that's true, how people validate any medical research at all? Do they just ignore the validation, and if some research has an error - well, tough luck, we'd base policy on it anyway and if it turns out to be a disaster, we'd just shrug? Giving up on reproducibility and validation means giving up on one of the most important features of modern science, I have very hard time believing that's the only way to do modern medical research known to our civilization.
> Otherwise, the rules would find a middle ground;
They actually did, if you read the actual proposal, which talks about disclosure in ways that enough for validation but does not infringe privacy. But you seem to have arrived to pre-determined conclusion already, and there in fact can be no middle ground that would be acceptable to you, as it seems.
A lot of environmental medical research results in dense and small data instead of sparse and large data, which obviously means it's even easier to de-anonymize.
And the really important take away from the above paper is that you need to know about all the other public data in the universe in order to really ensure anonymity. Even when anonymity seems like it's "obviously" not going to be a problem.
You never know what dataset will be leaked/published tomorrow correlating columns D-F in the spreadsheet you gave EPA to someone's first and last name. So privacy is like computer security: you can never be totally sure. So when the stakes are high you should be extra cautious. And the stakes when sharing medical research are very often high.
> If that's true, how people validate any medical research at all?.. I have very hard time believing that's the only way to do modern medical research known to our civilization.
This isn't exactly a new problem, so scientists have had a little while to figure out how to make peer review work.
You share the data. You share with trusted, relevant third parties who promise to preserve the anonymity of study participants. Just becuase you do not publish possibly de-anonymizable and definitely sensitive medical information on a public website does not mean you don't share the data with interested and relevant third parties.
> But you seem to have arrived to pre-determined conclusion already, and there in fact can be no middle ground that would be acceptable to you, as it seems.
On the contrary, I do literally exactly that in my very first post in this thread: There are some common-sense workarounds. For example, requiring an EPA or third-party audit of the original dataset and analysis. Or simply carving out exceptions for research whenever it can be demonstrated that an IRB (and therefore federal law) would not have allowed the research without restricting public access to data/analysis.
The situation where IRB says "no" to public disclosure should explicitly trigger exemption, with the duty falling back to the agency to prove the IRB was wrong in the first place. The fact that this exemption is not automatic is what makes this a catch-22 that pins researchers in-between their IRB and the EPA administrator in charge of evaluating their exception.
Nothing in that article (with which I of course was familiar) says anything about "not possible to publish medical data in an anonymized fashion that does not infringe privacy rights". What it talks about is that some data sets may be improperly anonymized, and Netflix data set is one of them. It is undoubtedly true that anonymization is harder than it appears to a naive observer. It does not mean it is impossible.
> important take away from the above paper is that you need to know about all the other public data in the universe in order to really ensure anonymity.
I do not see how it's takeway from that paper.
> You never know what dataset will be leaked/published tomorrow correlating columns D-F in the spreadsheet you gave EPA to someone's first and last name.
You also never know whether somebody doesn't just break into your system, downloads all secret data and publishes them. Yes, you have no guarantee against bad actors and mistakes. But if that's the border condition then no research should be performed at all - after all, once the data exists, there's no guarantee it won't somehow get leaked out.
> The situation where IRB says "no" to public disclosure should explicitly trigger exemption
IRB is a part of the institution, right? So you essentially delegating the decision about the exception to (part of) the institution. What the point of having the rule then if every organization can make their own exceptions on demand? And that makes trivially easy to hide the data in sloppy research - just sprinkle some private data on it, show it to IRB, they cry out "no, this can't be released, no way!" - and voila, you are safe from review.
>..."not possible to publish medical data in an anonymized fashion that does not infringe privacy rights".
1. Sometimes it is.
2. I already gave substantive reasons why this will commonly occur in environmental medical research -- dense datasets about small populations that capture sensitive information about individuals.
> I do not see how it's takeway from that paper.
Because that's literally the only assumption in the adversarial model other than "access to the published dataset". And correlating with an auxiliary dataset is literally how all actual instantiations of this attack work.
> just sprinkle some private data on it, show it to IRB, they cry out "no, this can't be released, no way!" - and voila, you are safe from review.
1. My proposal doesn't say "IRB happened to say you can't publicly publish this one particular protocol." My proposal says that IRB explicitly judges that no such protocol exists.
2. IRBs are not that arbitrary and are themselves audited.
3. If you're this paranoid about intent, then you can't trust the research anyways. If the research is committing intentional fraud, they could just do it at the data collection step.
The way I see it, either:
A. Even if we can't trust individual scientists, we can more-or-less trust a broad subset of scientists with different and well-aligned motives (as in, a whole group of: the IRB board, the PI, the people reviewing the IRB board) when they say that data can't be shared for privacy reasons; or,
B. We should expect that literally dozens of scientists, all with different motives, will routinely and actively collude to commit wide-spread fraud.
If (A), my approach makes sense. If we max out paranoia and go with (B), then I don't know why we should trust the data regardless of whether it's published publicly...
It’s alarmist because people will die because of this rule.
If you get cancer because of a chemical spill do you want all of your personal information released to help prove that the chemicals spilled poisoned you? No. You want your personal data protected.
Your confidentiality agreement with a study can preclude them from even releasing the anonymized raw data, invaliding a study in the eyes of the EPA.
It also mandates reproducibility. It isn't exactly easy or even possible to reproduce a 70-year long study or a study of victims of a disaster which is the point of the American College of Obstetricians and Gynecologists and The American Academy of Pediatrics.
Reproducibility in science means enough information is disclosed, such that the data can be reproduced. It does not mean it has to be repeated to be believed.
Reproducibility is a key part of the scientific method. I'm surprised people are so willing to toss it aside.
Reproducibility doesn't require the original data set to be present, only that the method of conducting that particular study reproducible. There are huge differences.
Scott Pruitt denies basic climate science. But most of the outrage is missing the point. ...
It’s not about Pruitt and it’s not about facts. (from 2017)
[Complaints] are all about what Pruitt believes. And in the end, who cares what he believes? He is a functionary, chosen in part to dismantle EPA regulations on greenhouse gases. If it weren’t him, it would be some other functionary.
The goal is to block or reverse any policy that would negatively affect [conservative] donors and supporters, who are drawn disproportionately from carbon-intensive industries and regions. That is the North Star — to protect those constituencies. That means, effectively, blocking any efficacious climate policy (which, almost by definition, will diminish fossil fuels)
It always stuns me that people still internalize the "both parties" arguments. One party in particular seems significantly more determined and certain to drag American living standards down.
This is actually a good thing. Otherwise, you get laws such as "Coffee labels must have warning labels stating it might cause cancer" as California just passed because a "Scientist" mentioned that coffee might contain a carcinogen. There is no proof or verifiable data supporting it. Now the EPA cannot just ban or prohibit based off "might cause". There needs to be proof that anybody can verify.
Have you technical geniuses figured out that lawyers can write restrictions in a way that sounds reasonable (to get them to pass) and then go on to enforce policy in quite a different way, much later ? Is it not obvious that 'environmental protection' costs money and may prevent some company from making more money ? The US EPA is a number-one target for decades, for these reasons. Many US lawyers are paid very, very well to represent the interests of their commercial employers, which likely does not include any consideration of actual long-term harm.
> Is it not obvious that 'environmental protection' costs money and may prevent some company from making more money?
I cannot tell from your tone and the jab at “technical geniuses” if you’re saying this is a bad thing.
Yes, environmental protection costs money. This money should be paid by the companies who are directly or potentially placing the environment at risk. This burden should not be externalized onto the public so private entities can make more money.
I think it sounds great. Reproducability and verifiability are extremely import in scientific research. Unlike what this article and some people here claim, it does make provisions for protecting the privacy of users. I also don't see anything hinting that it will affect past studies retroactively.
My only complaint is that it targets the EPA specifically, which is pretty suspicious with the current political situation. It would be great if this affected all studies which receive public funding.
The burden seems to be flipped here. You can't release a medical product until you have reproducible and verifiable results proving your product is safe and effective. When it comes to the environment you can release something until it's proven that it's not safe and effective. They're increasing the burden under the nomenclature of science, when in reality it's a cost/benefit between a company's profits and the quality of life of everyone.
I don't mean to sound anti-corprate, but environmental regulations only exist because companies externalize costs onto society and this increases the burden for balancing exposing that cost.
61 comments
[ 628 ms ] story [ 389 ms ] threadHave the authors of this article actually confirmed that this would include non-anonymized data unfit for public consumption?
Added: The actual document does not specify "raw data", it is much more specific about what data they expect.
> The proposed regulation provides that, for the science pivotal to its significant regulatory actions, EPA will ensure that the data and models underlying the science is publicly available in a manner sufficient for validation and analysis.
"A manner sufficient for validation and analysis" is a completely different standard than "all the data the study produced".
> what does that have to do with this regulatory proposal?
Article's subtitle is: The agency’s proposed new rule would allow it to ignore the best available science
The scenario is not so hypothetical. The circumstances and particulars change, but this situation describes a lot of medical and environmental research.
Should the EPA take the researcher's word? Maybe not. But there's a lot of workable middle ground in-between "blind trust" and "anyone with access to a library computer and a stats textbook can figure out participant 494 -- who math tells us can only possibly be Johnny D. of Smallsville, VA -- has testicular cancer".
And the situation is even more difficult than this because de-anonymization is an active area of research with some surprisingly powerful results published in recent years; i.e., it is highly non-trivial to determine if a dataset is de-anonymizable. And the answer always depends on what data about a person's identity is publicly available, which especially in the age of annual massive data leaks change from year to year. So even if researchers don't think it's possible to de-anonymize study participants using today's mathematical tools and today's public datasets, they'll often choose to keep the dataset private just in case.
There are thousands of old studies for which the data is unavailable, incomplete, was lost, etc. Are you going to pay for all of them to be reproduced? Are you ready to give up penicillin? Surgeons washing their hands between operations? The links between tobacco and cancer? Every single medical advancement (Because patient confidentiality precludes the release of raw data that goes into medical studies)?
I'm all for reproducibility, but this has enormous consequences.
No.
>It feels like we’re well on our way into a new dark age.
Bill Gates would probably disagree with you on that (http://bit.ly/2mg08wL)
The groundwork was laid in the US starting in the 1970s. With the founding of Cato, Searle, Olin fnds. Murdoch bought the NY Post in 1976. Fairness doctrine was repealed in 1987 allowing Rush Limbaugh to go bigtime. Fox started in 1997.
I am actually a conservative on many issues, including UBI, work rules, tariffs, caution when making new law and regulations, and appropriate pricing of risk. However the real conservative movement has been redirected as a way to link wealthy donors’ interests with identity politics, for votes. Part and parcel of that is a huge propaganda machine which spouts lies. That propaganda is corrupting the West and I am afraid for its effect on democracy and average people.
As far as I can see that's the only immediate information this article bothered to speak about the rule itself. Anybody see a summary on this rule to share with the crowd? Or did I miss something?
Hope that helps.
EPA is proposing to eliminate consideration of scientific studies unless the raw study data are made publicly available — possibly including participants’ personal, confidential, and private information — data that may be subject to legal, ethical, and human subject research protections.
This requirement would eliminate human health studies that use integral medical, lifestyle, and geographic data, as well as studies that include confidential business information, such as information from studies conducted by industry to demonstrate safety of a pesticide or other toxic chemical. It could also eliminate consideration of older studies for which data are no longer available or accessible, even if the data have been reanalyzed and the studies have been validated, replicated, reproduced, and undergone rigorous and independent peer review.
Far from using the best available science for EPA decision-making, the proposal would severely limit the studies and scientific evidence the agency would use to fulfill is statutory mission to protect human health and the environment. This is not a new idea. It is the result of a decades-long campaign to undercut reliance on groundbreaking research...
The proposed regulation provides that when EPA develops regulations, including regulations for which the public is likely to bear the cost of compliance, with regard to those scientific studies that are pivotal to the action being taken, EPA should ensure that the data underlying those are publicly available in a manner sufficient for independent validation.[1]
Seem eminently reasonable to me. And seems aligned with the "open science" movement to me.
And in terms of privacy...
EPA believes that concerns about access to confidential or private information can, in many cases, be addressed through the application of solutions commonly in use across some parts of the Federal government. Nothing in the proposed rule compels the disclosure of any confidential or private information in a manner that violates applicable legal and ethical protections.
[1] https://www.federalregister.gov/documents/2018/04/30/2018-09...
Suppose zero studies that satisfy that criteria study lead. Should the EPA suddenly say lead must be safe in any concentration?
Try and publish a paper in Nature and tell them “I can’t share the data”.
Good luck tracking down data from ~50 year old studies published in 1970. Further, their is little point in replicating old studies so often that's all we got.
In terms of new research are you saying that it’s ok to keep the raw data a secret? Why? We don’t do that for any other type of science.
Yes, the best way to prove reproducibility is to reproduce it, but that can’t always be done and at any rate is no excuse for keeping raw data a secret.
This is blatant disingenuousness. What they are saying is "you need not disclose private information, but we won't use your study if you don't".
Also, please don't be so credulous. This rule has been promulgated, invented, and proagandized by people who are trying to hamstring the EPA from using scientific studies to regulate industry. It's not new, it's just newly empowered.
You appear to be falling for the plausible cover story, which is disinformation. People are really bad at being able to discern the intent behind "plausibly good" organizations and movements.
>With this notice, EPA is soliciting public comment on a proposed regulation designed to provide a mechanism to increase access to dose response data and models underlying pivotal regulatory science in a manner consistent with statutory requirements for protection of privacy and confidentiality of research participants, protection of proprietary data and confidential business information, and other compelling interests.
>The proposal takes comment on how to ensure that, over time, more of the data and models underlying the science that informs regulatory decisions (over and above the dose response data and models underlying “pivotal regulatory science”) is available to the public for validation [13] in a manner that honors legal and ethical obligations to reduce the risks of unauthorized disclosure and re-identification.
>EPA believes that concerns about access to confidential or private information can, in many cases, be addressed through the application of solutions commonly in use across some parts of the Federal government.[16] Nothing in the proposed rule compels Start Printed Page 18771the disclosure of any confidential or private information in a manner that violates applicable legal and ethical protections. Other federal agencies have developed tools and methods to de-identify private information for a variety of disciplines.[17] The National Academies have noted that simple data masking, coding, and de-identification techniques have been developed over the last half century and that “Nothing in the past suggests that increasing access to research data without damage to privacy and confidentiality rights is beyond scientific reach.” [18] More recently, both the National Academies and the Bipartisan Commission on Evidence Based Policy [19] have discussed the challenges and opportunities for facilitating to secure access to confidential data for non-government analysts.
>Considering the breadth of dose response data and models used in the development of significant EPA regulations, the requirements for availability may differ. These mechanisms may range from deposition in public data repositories, consistent with requirements for many scientific journals,[20] to, for certain types of information, controlled access in federal research data centers that facilitate secondary research use by the public.[21] EPA should collaborate with other federal agencies to identify strategies to protect confidential and private information in any circumstance in which it is making information publicly available. These strategies should be cost-effective and may also include: Requiring applications for access; restricting access to data for the purposes of replication, validation, and sensitivity evaluation; establishing physical controls on data storage; online training for researchers; and nondisclosure agreements.[22]
https://www.federalregister.gov/documents/2018/04/30/2018-09...
It's not a bad goal, though making data public is always a bit tricky when data collected from human beings is involved.
The proposed regulation provides that, for the science pivotal to its significant regulatory actions, EPA will ensure that the data and models underlying the science is publicly available
The public availability requirement sounds nice on paper, but in practice, it means ignoring a lot of science during rule-making because the original data can't be published due to genuine privacy concerns.
There are some common-sense workarounds. For example, requiring an EPA or third-party audit of the original dataset and analysis. Or simply carving out exceptions for research whenever it can be demonstrated that an IRB (and therefore federal law) would not have allowed the research without restricting public access to data/analysis.
It is more limited if you read the whole sentence, which goes on to qualify which data they are interested in (specifically "publicly available in a manner sufficient for validation and analysis.").
These rules are a hot political topic that are years in the making. The people pushing them are not scientists and have never expressed genuine interest in reproducibility outside of this one issue. And scientists who have spent years fighting for reproducibility have publicly stated they oppose this rule. And all the major proponents have strong anti-regulatory preferences.
These rules are not about reproducibility. This rule is an intentionally crafted Catch-22 designed to exclude medical research from consideration so it's easier to justify repealing or not implementing environmental regulations. Otherwise, the rules would find a middle ground; e.g., by allowing the EPA access to relevant scientific data and analysis without risking https://en.wikipedia.org/wiki/De-anonymization by posting sensitive medical information about thousands of patients on the public internet.
> Otherwise, the rules would find a middle ground;
They actually did, if you read the actual proposal, which talks about disclosure in ways that enough for validation but does not infringe privacy. But you seem to have arrived to pre-determined conclusion already, and there in fact can be no middle ground that would be acceptable to you, as it seems.
Then you're one of today's lucky 10,000: "Robust De-anonymization of Large Sparse Datasets" http://www.cs.utexas.edu/~shmat/shmat_oak08netflix.pdf
A lot of environmental medical research results in dense and small data instead of sparse and large data, which obviously means it's even easier to de-anonymize.
And the really important take away from the above paper is that you need to know about all the other public data in the universe in order to really ensure anonymity. Even when anonymity seems like it's "obviously" not going to be a problem.
You never know what dataset will be leaked/published tomorrow correlating columns D-F in the spreadsheet you gave EPA to someone's first and last name. So privacy is like computer security: you can never be totally sure. So when the stakes are high you should be extra cautious. And the stakes when sharing medical research are very often high.
> If that's true, how people validate any medical research at all?.. I have very hard time believing that's the only way to do modern medical research known to our civilization.
This isn't exactly a new problem, so scientists have had a little while to figure out how to make peer review work.
You share the data. You share with trusted, relevant third parties who promise to preserve the anonymity of study participants. Just becuase you do not publish possibly de-anonymizable and definitely sensitive medical information on a public website does not mean you don't share the data with interested and relevant third parties.
> But you seem to have arrived to pre-determined conclusion already, and there in fact can be no middle ground that would be acceptable to you, as it seems.
On the contrary, I do literally exactly that in my very first post in this thread: There are some common-sense workarounds. For example, requiring an EPA or third-party audit of the original dataset and analysis. Or simply carving out exceptions for research whenever it can be demonstrated that an IRB (and therefore federal law) would not have allowed the research without restricting public access to data/analysis.
The situation where IRB says "no" to public disclosure should explicitly trigger exemption, with the duty falling back to the agency to prove the IRB was wrong in the first place. The fact that this exemption is not automatic is what makes this a catch-22 that pins researchers in-between their IRB and the EPA administrator in charge of evaluating their exception.
> important take away from the above paper is that you need to know about all the other public data in the universe in order to really ensure anonymity.
I do not see how it's takeway from that paper.
> You never know what dataset will be leaked/published tomorrow correlating columns D-F in the spreadsheet you gave EPA to someone's first and last name.
You also never know whether somebody doesn't just break into your system, downloads all secret data and publishes them. Yes, you have no guarantee against bad actors and mistakes. But if that's the border condition then no research should be performed at all - after all, once the data exists, there's no guarantee it won't somehow get leaked out.
> The situation where IRB says "no" to public disclosure should explicitly trigger exemption
IRB is a part of the institution, right? So you essentially delegating the decision about the exception to (part of) the institution. What the point of having the rule then if every organization can make their own exceptions on demand? And that makes trivially easy to hide the data in sloppy research - just sprinkle some private data on it, show it to IRB, they cry out "no, this can't be released, no way!" - and voila, you are safe from review.
1. Sometimes it is.
2. I already gave substantive reasons why this will commonly occur in environmental medical research -- dense datasets about small populations that capture sensitive information about individuals.
> I do not see how it's takeway from that paper.
Because that's literally the only assumption in the adversarial model other than "access to the published dataset". And correlating with an auxiliary dataset is literally how all actual instantiations of this attack work.
> just sprinkle some private data on it, show it to IRB, they cry out "no, this can't be released, no way!" - and voila, you are safe from review.
1. My proposal doesn't say "IRB happened to say you can't publicly publish this one particular protocol." My proposal says that IRB explicitly judges that no such protocol exists.
2. IRBs are not that arbitrary and are themselves audited.
3. If you're this paranoid about intent, then you can't trust the research anyways. If the research is committing intentional fraud, they could just do it at the data collection step.
The way I see it, either:
A. Even if we can't trust individual scientists, we can more-or-less trust a broad subset of scientists with different and well-aligned motives (as in, a whole group of: the IRB board, the PI, the people reviewing the IRB board) when they say that data can't be shared for privacy reasons; or,
B. We should expect that literally dozens of scientists, all with different motives, will routinely and actively collude to commit wide-spread fraud.
If (A), my approach makes sense. If we max out paranoia and go with (B), then I don't know why we should trust the data regardless of whether it's published publicly...
Considering one of the tenants of the scientific method is reproducibility, how can you reproduce something if the raw data is unavailable?
If you get cancer because of a chemical spill do you want all of your personal information released to help prove that the chemicals spilled poisoned you? No. You want your personal data protected.
Sounds like a red herring to me.
It also mandates reproducibility. It isn't exactly easy or even possible to reproduce a 70-year long study or a study of victims of a disaster which is the point of the American College of Obstetricians and Gynecologists and The American Academy of Pediatrics.
https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-2...
Reproducibility is a key part of the scientific method. I'm surprised people are so willing to toss it aside.
I probably sound like a broken record, but this is standard across scientific disciplines. No raw data and people call bullshit.
https://www.vox.com/science-and-health/2017/3/10/14871696/sc...
Scott Pruitt denies basic climate science. But most of the outrage is missing the point. ... It’s not about Pruitt and it’s not about facts. (from 2017)
[Complaints] are all about what Pruitt believes. And in the end, who cares what he believes? He is a functionary, chosen in part to dismantle EPA regulations on greenhouse gases. If it weren’t him, it would be some other functionary.
The goal is to block or reverse any policy that would negatively affect [conservative] donors and supporters, who are drawn disproportionately from carbon-intensive industries and regions. That is the North Star — to protect those constituencies. That means, effectively, blocking any efficacious climate policy (which, almost by definition, will diminish fossil fuels)
If so I don’t disagree.
I disagree with this article stating that raw data needs to be kept secret. It’s pure bullshit and anti-science.
I cannot tell from your tone and the jab at “technical geniuses” if you’re saying this is a bad thing.
Yes, environmental protection costs money. This money should be paid by the companies who are directly or potentially placing the environment at risk. This burden should not be externalized onto the public so private entities can make more money.
https://www.federalregister.gov/documents/2018/04/30/2018-09...
I think it sounds great. Reproducability and verifiability are extremely import in scientific research. Unlike what this article and some people here claim, it does make provisions for protecting the privacy of users. I also don't see anything hinting that it will affect past studies retroactively.
My only complaint is that it targets the EPA specifically, which is pretty suspicious with the current political situation. It would be great if this affected all studies which receive public funding.
I don't mean to sound anti-corprate, but environmental regulations only exist because companies externalize costs onto society and this increases the burden for balancing exposing that cost.