I imagine Wansink does get it. Now that he's outed, he either trolling for fun, or hoping the blasé approach is the one that quiets all the fuss the fastest.
I wouldn't be certain. A lot of researchers really are clueless about anything to do with statistics.
It takes much more work/time to do real research, so the funding system has actually been selecting for people who can remain ignorant (so it is not fraud) and just produce p-hacked "results". In many areas, this has been going on for multiple generations now and you are trained to do it as a grad student.
> I wouldn't be certain. A lot of researchers really are clueless about anything to do with statistics.
This is definitely true. I once saw a book with a title similar to "Statistics for Dummies" in a professors office. He had plenty of access to staticians at the university too. Unfortunately if a given field involves many people with ignorance of statistics, these problems may not be called out during the peer review process.
>"Unfortunately if a given field involves many people with ignorance of statistics, these problems may not be called out during the peer review process."
As I said, poor understanding of statistics has been a beneficial trait in many areas of research for decades now. Universities and labs are rewarded with funding because they are able to (honestly, but incorrectly) pump out more "results" than if the researchers were properly trained.
So the issue goes well beyond the problems "not being called out". The problems are institutionalized. Editors and reviewers will try to force you to commit the errors in order to publish (and hence have a career). That is what they have been trained to consider science.
> He had plenty of access to staticians at the university too.
Assuming the book is for his own learning, as opposed just curiosity or for students etc, there is still nothing wrong with learning from book. Assuming he really dont know the statistics.
And it definitely beats up people who don't know either, but prefer to stay ignorant just so they don't look like someone who reads beginners book.
It's worse than just not knowing statistics. These errors were in a table in one of the published papers. Every single row is calculated wrong.
> (43.67 - 44.55)/sqrt(18.5^2/43 + 14.3^2/52)
should be 0.2551872, rounds to 0.26, he put 0.25
> (68.65 - 66.51)/sqrt(3.67^2/43 + 9.44^2/52)
should be 1.503114, rounds to 1.5, he put 1.38
> (184.83 - 178.38)/sqrt(63.7^2/43 + 45.71^2/52)
should be 0.556064, rounds to 0.56, he put 0.52
And that's just basic math, nothing fancy. So, he knows it's bad. I assume it's difficult to get a doctorate at Stanford and then become a professor at Cornell, so there's got to be more to this than him just being stupid.
> a doctorate at Stanford and then become a professor at Cornell
As an aside, academia is no meritocracy. A lot of what's required to get posts at schools like these amounts to self-promotion and luck (graduating in the right year & from the right lab).
Of course hard work and technical competence is also required, so your point stands.
Yeah...wasn't saying that path implies genius. It should, though, denote the ability to do basic math, or the common sense to have someone proofread your papers. This is just so odd to me, has me wondering if he's trolling.
There are variants on the t-statistic calculation. I assume these were all checked? Either way this should be mentioned in the methods of the paper, along with the software used... I wonder if they used excel, in that case who knows what may have happened. I've seen excel change numbers/names to dates, and all sorts of wackiness.
Probably he gets it, but you know, “it is difficult to get a man to understand something, when his salary depends on his not understanding it” (Upton Sinclair).
I've found I'm exceptional in being at all concerned about privacy. I think that if you asked, most people would only want minimal effort put towards confidentiality, especially since the trade-off is that other researchers won't be able to double check the analysis.
> I've found I'm exceptional in being at all concerned about privacy. I think that if you asked, most people would only want minimal effort put towards confidentiality, especially since the trade-off is that other researchers won't be able to double check the analysis.
I've worked as a journalist in which my (and many other data journalists') modus operandi was to publish the data. Often because the data was public record anyway. I now work in academia and the mentality is significantly different. Some of it is logistics -- I would say most traditional news organizations do not have the internal incentive or habit to figure out a way to publish data. Whereas with newer organizations, such as 538 [0] and Buzzfeed News [1], the data teams have editors to whom open-source and digital publishing is more the natural way of things.
But in academia, there are also set rules and precautions governing every study. I haven't proposed any research yet but my understanding is that if your study requires collecting data from participants, the Institutional Review Board requires you to be very clear to participants about privacy and confidentiality and that you follow the guidelines to the letter.
Additionally, there are datasets only available to academics that aren't available to non-academics (i.e. journalists), which speaks to the expectation that academics be very mindful about confidentiality promises.
To add a bit more for people who haven't been involved in an IRB:
The strictness varies by university. Ultimately the IRB is there to ensure safety of the participants and minimize the chance of negative consequences. A couple examples I've heard from around my university include...
A study was taking place outside. There was a chance of a bee sting occurring to the participant. The IRB required that the researcher have an epi-pen ready just in case along with any required training.
A paper used in a study had the wrong stamp on it (not the most recent IRB stamp). The document had not changed from the last time, but the rule was simple. Every document presented to the participant had to have the most recent stamp.
On a study in which it was expected that 1/2 of the participants would feel nauseous, they had to provide a place to sit, water, and small snacks.
And perhaps most importantly, you can quit any study you're participating in at any time, for any reason. Compensation is figured out ahead of time, and you're going to get lots of questions from IRB if you want to do anything to reduce the compensation for leaving part way through the experiment.
There is also the question of how honest people will be when they are anonymous versus not. If you were studying cocaine use in America, publishing your identified data (to enable double-checking the analysis) would give the cops a list of people who admitted to illegal drug usage.
The current system with IRB allows for the collection of identified data. It also allows the data to be released, in full, as long as there is no way for an individual to be identified specifically. In some cases you can simply swap names for a participant ID (after randomly sorting the list). In other cases you have no choice but to publish some summary statistics because the data you collected is only applicable to a very small number of people in a small region, or there is enough data people could reverse-engineer likely participants of the study.
Science is about self-correcting when errors are made. There are plenty of reasons to criticize the current incentive structure for academics, such as there being little funding to replicate other studies, but doing away with confidentiality is absolutely not the way to go about it.
Doing so would be publishing non-representative data.
Imagine an identified versus anonymous survey about cocaine usage in America. People will likely answer honestly if it is anonymous.
If it is identified, most people would be fine with it being identified and would mark no usage. People who do use cocaine would be more likely to not participate or lie on the answer.
Even if they don't care, there is a responsibility by the researchers to the participants to protect their identity. Your participation in a study may have no big effect right now, but down the line it could. If your project has federal funding and involves research on people, there are ethical requirements and trainings required before starting the study.
What if you completed a simple survey for a study and identified yourself as a member of a group (Christian, Jewish, Mexican, African American, etc.) and 5 years down the line, there was a big rounding up of your group. Suddenly your participation 5 years ago is leading to terrible life changes.
Please get to know participant protections more. They are very important for the integrity of the data used in academia. The NIH has a strict policies to ensure the safety of participants.
The Institutional Review Board (IRB) reviews studies for risks participants could face as a result of participating. Every person who has access to the de-anonymized data must complete an NIH training course.
Depending on your institution, the process varies in seriousness. Having worked on 2 submissions and helped on ~3 other IRB approved studies, it can be tedious. My institution takes 6+ weeks from submission to approval, which can really delay things. At the same time, it's a vital part of the process to ensure validity.
Most psychological researchers deidentify subject data, since you can only show it to a very small group of people otherwise. (which might prevent you from e.g. getting statistical help from someone outside the project).
It depends on the situation. I work with a professor who came from industry. He once worked on a project where they did massive data collection inside the military. Over $1 million was spent over 2 weeks collecting data. A couple months after, some researcher called and said "Hey, I heard about the data you collected. Would you mind passing it on to me?". Of course the answer was no, not until we publish damn near everything we can out of it.
Another consideration is the amount of data. Sometimes you're talking about dozens or hundreds of gigabytes of data. Start multiplying this by the scale of universities and you come up with a large expense to store massive amounts of data most of which will be downloaded no more than a couple times. On top of that, the data must be cleaned and made anonymous, and if they make a mistake in the process, there could be large liability issues. See AOL https://en.wikipedia.org/wiki/AOL_search_data_leak
Where's the problem? If that $1 million came from public sources, then the data should have been shared from day one. If others use the data they should put the originators on the data as coauthors on the resulting papers.
> Where's the problem? If that $1 million came from public sources, then the data should have been shared from day one.
I'm not saying the data should never be shared. Often times you can contact the author of a particular study and they will provide anonymized data. Most of the time people don't ask, and to take the time to prepare data for public release would offer little to no benefit.
I do believe there should be "first rights" to publish on data collected with funding from public sources. As for an exact policy by which it could be handled, I don't have one in mind. When you spend dozens of hours writing grant proposals only to have most of them turned down, it would be a sucky situation to have to make the data available for everyone before you have gotten a single paper out of it.
> If others use the data they should put the originators on the data as coauthors on the resulting papers.
It may vary by field, but generating the data alone is almost certainly not sufficient for co-authorship. [1]
In some of my research the participants are young children or are disclosing sensitive social information during the study, so we typically cannot share the data in fine-grained detail. It can be difficult to ensure confidentiality will be maintained for the participants.
I usually do my best to find a subset of variables that can be released in an anonymized form and still capture the true essence of the results and make that available. This isn't always easy to do though depending on the study details.
An old concern is that someone comes along, takes your data, and then
a) shows that you have an error somewhere and disproves your paper, making you look stupid, or
b) does something better with your data than your paper, making you look stupid, or
c) runs a follow up study on the same data you are currently running and publishes before you have the chance
In my experience usually none of this happens, UNLESS you're in a really cut-throat/hotly debated/bleeding edge corner of science, which is fairly rare.
c) is actually problematic if you told your funding body that you're going to generate X studies with the data they paid for, since you can only publish X-1 studies. I've seen this happen with biologists not releasing their genome assemblies with the assembly paper so they could run basic follow up studies using that assembly as reference before anyone else can do it.
There are sometimes ethics rules on how long data is stored for. If there's a commitment to destroy the raw data after a period, eg. five years, often it can't be publicly released (even anonymised) or there'd be no way of ensuring it's been destroyed when the time is up.
I hope you're being sarcastic. I'm a bit hesitant to provide examples, but a lot of top results in computer science are sold as being much more far reaching than what they really are.
Speaking from my own areas of experience, I've been struck by how much some celebrated methods and results in machine learning, broadly speaking, basically amount to tinkering without any hard generalizable proofs. In at least one case I'm aware of, there was a statistical proof later that made it clear the original researchers were close, but if they had a different dataset they would have come up with a different result. In fact, as I write this, I think I recall that different researchers did come up with different results, and it was in large part due to tinkering on different datasets.
So yes, even in computer science you run into similar things. Maybe not the same, but similar processes leading to similar problems.
The problem is that universities have a strong incentive to optimize for juicy press releases. A theorem -- unless it's 100 years old or otherwise somehow easy to relate to the general public -- does not have this effect. Winning a competition or "outperforming humans" does.
Aside from bumping up the quality of K-12 science education by at least an order of magnitude or two, I'm not sure there's a good solution to this problem. It's the human condition to be impressed by some things and not others, regardless of their relative actual importance.
It's weird. If conservatives don't believe scientists, it's because the conservatives are labelled "morons". Could it be that conservatives just think there's a lot more of this crappy "science" out there, and that a situation like this is the canary in the coal mine?
When presented with examples like this, it's actually _irrational_ to think that there aren't other egregious papers out there. Possibly just a few, but possibly a large number of them.
Of course there are lots. There's a bucket-load of papers get published every year, and sometimes reviewers are in a hurry too and miss things. A miniscule number of papers ever replicated, and replication studies regularly find reproducability rates of less than 50%.
Fortunately, critical thought does not stop at the point of publication. When researchers read each other's papers they (hopefully) do not blindly assume them to be bug-free.
I think there's a general popular misunderstanding of the process of science as it is generally practiced.
A single paper isn't as meaningful as you might think. It's more like an interesting blog post. You might use it as a starting point for your own investigation.
A single paper does not represent "the truth" as we currently understand it. For that you need to have a broad scientific consensus.
You might be able to get a better understanding of that from review papers citing recent advances in the field. Or from attending conferences/reading a broad range of papers yourself.
Conservatives don't distrust science carte blanc. Many of them will chuck money at asinine bullshit as long as it's billed as a way to kill the other team. Check out some of the more ridiculous stuff to come out of DoD.
They distrust climate science. And they don't distrust the actual science, just the conclusions. And increasingly not even the most basic conclusion, just the most obvious causal theory behind that conclusion.
> Meanwhile, anti-vax is huge in liberal soccer-mom only-shops-at-whole-foods cities.
Is it really "huge"? Some links to polling data might help here otherwise I would also be reticent to proclaim that it's comparable to climate denial-ism. After all, we have a president that has publicly proclaimed climate change to be a hoax and sitting members of congress that eagerly attack climate scientists on their research [1].
That same president has of course also publicly proclaimed vaccines to cause autism. He and his tens of millions of followers are the antithesis of the liberal soccer mum, of course.
The difference between vaccines and climate change is that we have hard evidence on vaccines. You can create a hypothesis and do a randomized double blind experiment. The results are conclusive.
Climate Science is a bunch of computer models are not testable. When they're run and compared to what actually happened they're always off. e.g. Name a single climate science model that predicted "the pause" starting in '98?
Climate Scientists don't release their model's code. They don't release the exact data that went into their model. Shit, isn't that one of the complaints about Wansink? That he isn't sharing his data?
I'm not saying climate change isn't real, I'm saying that the level of certainty that climate scientists proclaim is problematic. I'm saying that labeling dissenters as deniers (reminiscent of holocaust deniers) is beyond the pale. I'm saying that fossil fuels have done more to advance the human condition than anything else in history of mankind.
>"The difference between vaccines and climate change is that we have hard evidence on vaccines. You can create a hypothesis and do a randomized double blind experiment. The results are conclusive."
I tried to find this a few years ago in the case of measles and it turned out to seem no such studies existed. Have you seen one?
Thanks, it looks like this one was done after I was searching. This study doesn't really have the outcomes we need to measure though. There is no placebo group, they didn't check for measles diagnoses (by doctors who didn't know whether they were vaccinated), and they only measured antibody levels for 1.5 months after 1 vaccine dose in children 12-23 months old (ie before going to school when exposure usually occurs).
Unfortunately, I really don't think this study offers us anything about whether "vaccines work". You can see that the main concern is whether doctors are much more likely to diagnose measles when they do not believe a vaccination occurred, and whether introduction of the lab tests made the diagnostic definition more stringent:
“This was not a blind study, since the investigators knew which children had received measles vaccine. It seems probable that the occurrence of so much ‘measles-like’ illness in the vaccinated children was a reflexion of the difficulty in making a firm diagnosis of measles in the African child at one visit.”
“A likely reason for this is that the case may have been misdiagnosed as a non-specific viral illness. Measles has become relatively uncommon in Singapore with two decades of widespread measles vaccination, and especially after the second dose policy was implemented in 1998. Many primary care doctors may not even see a single case of measles in a year. This makes diagnosis more difficult.
[...]
As only approximately 7% of the clinically-diagnosed cases of measles reported locally turned out to be measles by laboratory testing, there is a need for laboratory confirmation of measles to avoid misidentification of cases and improve disease surveillance.(2)”
Here is another one, apparently we can get a drop of up to 99.5% just by switching to lab tests from clinical diagnosis:
"Indeed, an average of only 100 cases of measles are confirmed annually [32], despite the fact that >20,000 tests are conducted [28], directly suggesting the low predictive value of clinical suspicion alone."
http://jid.oxfordjournals.org/content/189/Supplement_1/S185....
I just don't think these alternative explanations have been investigated very well, and the blinded RCT (vs placebo) is missing. Another thing, the various proxies for "vaccine success" they use do not correlate with each other...
"Our data demonstrate that regression analysis shows only limited correlation between NT results and the ELISA values. This is in agreement with other reports [4]. Similar limitations in the correlation were also reported for other viruses like Cytomegalovirus (CMV) [10]. In case of the gamma globulin samples, the low correlation might reflect the wider spectrum and heterogeneity of the involved or measured measles antibodies."
http://www.ncbi.nlm.nih.gov/pubmed/17308917
That's not a sensible interpretation of that measles data.
Since measles is very contagious, there's a lot of effort putting into identifying and isolating cases. This necessarily involves screening many cases where the disease is only weakly suspected. On top of that, the current protocols cast a really wide drag net. Australia, for example, tests every susceptible patient who shared a waiting room with a measles patient, as well as those who entered less than two hours after the index case left. In a busy hospital or practice, this could be a lot of children.
Thus, the high ratio of tested to confirmed cases doesn't mean doctors are bad at diagnosing measles; they may just be cautious.
So how large do you think such a diagnosis effect may be? That other paper reported only 7% of clinical diagnoses were lab positive for measles in Singapore.
And yet another effect to deal with is that people used to spread measles on purpose:
“Before the introduction of measles vaccines, measles virus infected 95%–98% of children by age 18 years [1–4], and measles was considered an inevitable rite of passage. Exposure was often actively sought for children in early school years because of the greater severity of measles in adults.”
http://jid.oxfordjournals.org/content/189/Supplement_1/S4.fu...
I think for these reasons it is clear the effectiveness of measles vaccines has been overstated, it is only a matter of how much.
> Climate Scientists don't release their model's code
False.
Some publish code before/with their papers, some don't publish at all, and some publish code with a lag after publication (to clean up the code, tussle with legal about IP and/or political issues, etc. Fair IMO as long as the code is eventually published).
The major models, including the NASA one, are open source.
Google it.
> Name a single climate science model that predicted "the pause" starting in '98?
Predicting century-long trends and predicting 5 or 10-year variations within that trend are extremely different tasks.
It's certainly possible that we have models that are exceptionally accurate on the scale of 20 or 50 or 100 years but cannot make accurate predictions over the course of 5 or 10 years.
In fact, in a stochastic system, that's exactly what you would expect to be the case -- trends that are globally but not locally discernible.
More-over, opponents to the climate change thesis don't bother to provide their own models and predictions for falsification. And when they do, they're just as wrong on small time scales and much more wrong on long time scales.
> I'm saying that labeling dissenters as deniers (reminiscent of holocaust deniers) is beyond the pale.
This largely depends on what is being denied:
- CO2 effects climate (requires extraordinary new evidence -- we have both historical Earth data and inter-planetary data).
- Temperatures are changing (requires new evidence -- the '98 "hiatus" as largely played out and skeptics failed to make any predictions that made it through the decade).
- CO2 is causing temperatures to change on earth (again, requires new evidence).
- Concrete predictions about near and mid-term effects of climate change. IMO fair game, but I'm not sure what the point is from a climate policy perspective unless we're really that selfish.
There's also an important difference between skepticism and denial.
Being skeptical of Newtonian physics and providing a competing model against which we can compare is of course fair game.
Denying Newtonian physics in favor of "no particular opinion on how the world works, and let's not try to figure out" is of course worse than useless.
In other words, make your own predictions or get off the pot.
FWIW the only two anti-vaxxers I know in person are a retired west-coast life long democrat, and a super neo-conservative soccer mom. It seems to be more of an "anti-establishment" thing than a "left/right" thing.
Climate science is totally inaccessible to lay people. All we can do is trust. I'm a science teacher and when I teach climate change, it's just the bare basics and a few subjective facts. We can make cardboard greenhouses and draw pictures of IR rays getting absorbed in the atmosphere. But that's not what's in doubt. That's the stuff everyone's accepted for decades. The part people don't trust is the "it'll destroy the world". For that, I haven't got the faintest clue how right it is beyond trust. In most of high-school science, you can measure forces yourself, you can mix the chemicals yourself, you can observe stuff doing what the text book says it should. But not climate science.
Unless it can be made accessible to even science graduates to actually reproduce the results from a critical point of view, it's always going to have a lot of sceptics. Not everyone has so much faith in scientists.
My understanding is that there isn't really a scientific consensus that "it'll destroy the world". But there is a consensus that "human activity is causing global warming" - yet this is something that a significant minority of people do not believe.
EPA has a website describing the mid and long term effects of climate change in N America. With plenty of citations a dedicated student or teacher could follow up on. Very accessible, and did not include "will destroy the world".
Incidentally, your critique also applies to literally most science, including very good and obviously successful science.
Can you really prove to your students that Intel doesn't have magic pixie dust it uses to speed up its processors? Probably not without expensive equipment your classroom cannot afford.
And can you reproduce the LHC experiments? Of course not.
The National Climate Assessment in particular is presented in the form of a slick website with hundreds of citations into the literature for curious readers: http://nca2014.globalchange.gov/report
The information is there, in extremely digestible form, and at the top of google search results. Naturally, reproducing the results of literally hundreds of cited papers is going to take you a while, but in most cases, there's nothing actually stopping you...
It's also worth noting that scientists are usually fairly transparent about when they're guessing WRT effects of climate change and when they're pretty certain. The near-term stuff (~hundred years) is largely guess work based on simulations -- we can't do much better. The most certain prediction is that weather will become harder to predict. The longer-term stuff (~hundreds to thousands of years) is much more well-understood because there're historical records relating GHG levels to climate.
The biggest greenhouse gas by far is water vapor, not CO2. So it makes no sense to call a plot of CO2 a plot of "greenhouse gas concentrations". This type of "information" is low quality and will only make people not trust them.
Ah, I see now it is a plot of CO2 equivalents... that ignores water vapor while the others (BC, OC, CH4, Sulfur, NOx, VOC, CO and NH3) are rather negligible (which is why the ppm values are close to those for CO2 alone).
Anyway you can deny what I am telling you, but it is usual for people to go to supposed authoritative site like that, then check wikipedia and see H20 is the biggest greenhouse gas.[1] As a result they just shrug and figure no one knows what they are talking about, since they can't get the story straight.
> Ah, I see now it is a plot of CO2 equivalents... that ignores water vapor while the others (BC, OC, CH4, Sulfur, NOx, VOC, CO and NH3) are rather negligible (which is why the ppm values are close to those for CO2 alone).
Unless you're planning on boiling an ocean or great lake some time soon, humanity isn't directly effecting the amount of H20 in the atmosphere at appreciable levels as a first order effect. It's not a parameter under variation.
C02 equivalents are a parameter under significant variation, and as a first-order effect of human action.
We are interested in the dynamics of the system as we vary a parameter that we can, and are, directly manipulating.
Larger atmospheric, oceanic and whole system models then incorporate H20 and other effects in a simulation as the parameter we actually control is varied.
If this were explained on the page, you'd complain it was too technical.
> Anyway you can deny what I am telling you, but it is usual for people to go to supposed authoritative site like that, then check wikipedia and see H20 is the biggest greenhouse gas.[1] As a result they just shrug and figure no one knows what they are talking about, since they can't get the story straight.
Is that really common? I mean, there are actually a large population of people who will read [1] but don't bother to click on the link right under the graphic and see how the graphed data is used in climate predictions?
I'm highly suspicious that there's actually a significant population of people who are open to being convinced but make such a sophomoric mistake in their reasoning.
Someone can always find reasons to be uncharitable and dismissive if that's their predisposition. People with that predisposition and willingness and disregard evidence for even the most superficial of complaints aren't going to be convinced anyways.
The reasoning is simple. A warming is said to be due to the "greenhouse effect", the major portion of which is due to "water vapor in atmosphere". If you show me a chart of "greenhouse gas concentration" and leave out water vapor, something is not making sense.
I don't know whats going on with that chart, whatever. I am just telling you the end result of showing it to me, and doubt I am alone.
The meaning and purpose of the chart is extremely clear. They're varying C02 and similar gasses, and looking at what happens. Water vapor is held constant as a first order effect and so isn't plotted.
I don't draw straight lines on all my plots relating constants to (in)dependent variables. Even fifth grade science experiments follow this convention in their plots. It's an extremely common practice. Water vapor isn't being varied, so isn't plotted. Again, nearly every fifth grade science fair project follows this convention in their plots. It really does count as basic literacy.
For someone with some basic scientific literacy, the author's intent is obvious and confirmable by reading the literature. OBVIOUSLY the authors are aware of water vapor concentrations, and you can confirm this awareness with a superficial investigation into how this metric is used in models.
Sorry, but I do not believe a non-antagonistic reader would jump to the conclusion that climate scientists are unaware of tyoical atmospheric H20 concentrations. That's so far from the realm of reasonability.
At some point, the readers' good faith must be assumed or the entire expository enterprise is pointless. I think this plot is by far on the correct side of that line.
Yeah, warming increases atmospheric water vapour slightly so a simplified model that assumes it remains constant is going to slightly underestimate the total warming caused by a given amount of non-water-vapour emissions. Models don't have to take into account every detail to be useful, though.
I've had this same conversation with colleagues. I think there's a mixture of things going on.
There's a lot more of this going on in a lot of fields, especially the biomedical and social sciences, but probably everywhere.
I think conservatives are correct to pick up on this general phenomenon. They know there's a lot of systemic problems with research and academics, and approach things from that perspective.
The problem, to me, is that they then overstep and use that to justify an equally pernicious position, which is to denounce or dismiss science in general, rather than to encourage fixes or constructive change. Of course, depending on the conservative, they're more likely denounce scientific research on certain topics and areas of research than others.
It's throwing baby out with bathwater, basically.
The scientific community then reacts poorly, in my opinion, by denying the kernel of truth there is to their concerns, and not addressing them.
You can see this playing out in some conservative blogs. They're aware of the replication crisis, discuss it, and point to things like the email leaks by climate scientists, which in my memory were scientifically sort of moot, but did reveal how much scientists are very much human with human flaws and motivations.
Maybe I'm wrong, though, and the typical response from the scientific community is best, though: maybe it's better to treat the problems with science separate from the conservative attacks, to tackle one problem at a time.
I guess I see two problems, one being problems with academics and science, the other being the involvement of politicians in science. I think the second thing only makes the first problem worse. It's not like the politicians aren't serving their own unholy masters.
Also some of them are older and remember the "2nd iceage" and "uncontrollable acid rain" doom and gloom.
Unfortunately the media has done a lot to poison the well on these topics, coupled with politicians all too happy to never let a crisis go to waste. Republicans yelling "small government" while trying to make a government big enough to police your bedroom. Democrats yell about stopping climate change and pump billions into crony companies that blow up and fail (all the while laundering taxpayer cash as campaign donations, republicans do this with defense groups).
So you get leftists going "we shouldn't like, have new fighter jets man" and right ringers thinking global warming is a laundering scheme.
"2nd iceage" and "uncontrollable
acid rain" doom and gloom.
It would be an interesting object of study in the history of science to look at old papers in these fields, and at the structure of funding, who was benefitting from funding given to these research subjects, how political pressure was used to silence critics of the scientific merit e.g. forest dieback etc. In retrospect we know that none of the hypotheses held up. Why did they gain such a currency at the time?
The "acid rain will destroy our forests" doom/gloom seems to have originated from a single person, Bernhard Ulrich [1] from the University of Göttingen. His dire warning gained immediate currency in a political climate where a new form of left-wing activism (now known as "Green Party" in various countries) was beginning to emerge. Political activists from the "Green" side of the spectrum effectively demonised all criticism of "acid rain will destroy our forests" and lead to generous funding for any research that seemed to corroborate the doom/gloom, triggering a self-reinforcing spiral.
Yeah. It's certainly interesting watching conservatives pointing to the fact that acid rail didn't turn into a catastrophe to justify their opposition to the kinds of emission limit that actually put a stop to acid rain in the first place.
This is indeed an important narrative, but it's not a priori clear
that you can infer from "the forests are alive and healthy in 2017"
to "If we had not instituted limits on sulfur pollution in the
1980s, the forests would be dead now.
I grew up next to some of the world's largest coal-fired power
plants that regularly featured in "worlds biggest polluters"
lists. The forests were fine before flue-gas desulfurisation just
as much as after. Some countries enacted flue-gas desulfurisation earlier than others. The forests were fine before flue-gas desulfurisation just
as much as after.
Certainly with acid rain, a lot of policies where put in place to try to reduce acid rain in response to those predictions. Any analysis about whether the predictions about acid rain where correct or not cannot be done without also studying the effects of those policies.
Some areas of science are mostly quackery. Nutrition for one. Some areas are resistant to quackery like physics and chemistry. And most are a mix like medicine.
In school we're taught that every field is as self correcting as physics. Reality seems to be different.
"Nutrition" isn't an area of science. Dietetics is, and is a very firm and established field. A lot of the false claims and bad dietary advice come from Nutritionists, which (in the UK at least) isn't a protected term. Anyone can claim to be a Nutritionist, and Nutritionists aren't known for rigourous data.
Unfortunately, the academy and profit-driven-science have given anti-intellectuals and science-rejecting conservatives all too much ammunition over the last several decades. (None of which shows that armchair psuedo-science, or just pulling stuff out of your ass is a better way, of course.)
Using a snappy XKCD is not a valid logical argument even though the poster might think it makes one look hip and cool. I wish researchers were not this lazy.
There are at least a handful of valid arguments that one could use rather than an XKCD cartoon.
The argument is that Wansink is responding to criticism (like the XKCD) with an air of indifference. Wansink's response to the XKCD is simply one of the examples; the XKCD itself is not being used to support the author's argument that the act of publishing non-rigorous work is too widespread in quantitative science today.
That XKCD cartoon succinctly shows the problem with multiple testing. If every scientific paper were as clear, accurate and impactful as that we would have solved half the issues with science discussed on this comment thread.
I went to high school with Brian Wansink. He is brilliant, funny, and affable. I could hear his voice speaking his responses to the objections raised, and snickered a little bit.
I'm sure that Wansink cares about his research and its quality. He's that sort of person. I don't know what happened with these papers, but it seems likely that answering bloggers publicly would not serve the greater good.
It seems to me that all the hand-wringing about these four papers misses the bigger picture: What can we do to improve the practice of science across the board? Why is there almost no field of study which isn't tainted by the politics of left vs. right?
My suggestion is that it began when people began to believe the idea that all political and social questions are best served with some sort of applied scientific analysis. I counter that, in fact, scientists are some of the worst people to ask for solutions to political and social problems, since most of them have almost no understanding of the relevant fields of study, which mainly include history and philosophy, and perhaps psychology. Even fields like theology, linguistics, and literature and the arts have more to say to most political problems than, say, physics or chemistry or meteorology, because they speak to human behavior rather than the behavior of inanimate substances.
Please understand that I say all this not to provoke an argument, necessarily. But as someone whose formal studies were mostly in political history, in general, listening to HN talk about politics is a lot like what I must imagine it is for most on HN to listen to people on TV news shows talk about tech in general and software in particular: they sort of get it, but not really, and certainly are not as expert as they imagine themselves to be.
Wansink doesn't appear to be troubled by his prodigious contributions of unreliable results to the literature, nor does he appear inclined to correct them. All of science and discovery is a joke to him.
Left or right, the troubled extremes of politics start where respect for evidence and truth ends. Scientists are, nominally, people who search for truths about our universe with some degree of rigor, some respect for evidence, some appetite for truth (and usually an insanely competitive nature, but I digress). Absent that, what's the point?
This isn't about society, in my opinion. It's about having some respect for evidence and truth. Without these, science isn't science. And what wansink is doing isn't science. Cold fusion had more rigor. Yet he will probably be rewarded with grants of taxpayer money, taken from fellow citizens as taxes, under threat of force. Meanwhile pediatric bone marrow transplanters will shut down their labs for "lack of funding". Or, more accurately, lack of standards in science.
Yes, it affects people when dishonesty is accepted.
If HN commenters are amateurs in this domain, what do you think of Andrew Gelman, who wrote the linked-to article and has harped on Wasnick's errors in multiple posts? ,any of the commenters on his blog seem to be well versed in the field too.
> “Why is there almost no field of study which isn't tainted by the politics of left vs. right?”
You lost me there. What did you mean when you included this statement here? Are you saying this particular incident is tainted by politics? I discerned nothing political in Wansink's papers, the feedback they received, Wansink's responses, or Andrew Gelman's reaction pieces. Did I miss something?
I ask this because the rest of your comment became about politics vs ___, which didn’t feel natural to follow when I didn’t understand why politics is relevant here.
If he is so unconcerned with verifying his test results, perhaps luminaries in his discipline should write to the journals where his papers are being published pointing out the uncorrected errors and suggest that the journal's editorial department be supplemented to fine tooth comb his assertions.
Wansink applied to the phd program I'm in for graduate school, and didn't get it, and then he applied for a professor position at my school and didn't get it. One of the faculty members told me she thought it was "a missed opportunity."
One thing I have trouble figuring out: is this indicative of a long-term trend? Or did his success create an expectation he couldn't keep up with and so felt a need to rush (or cheat)?
Incidentally, there is another professor at my school who has built a career out of exposing similar problems in research (especially p-hacking). It takes a brave person to inhabit that niche. It is a service to the research community, but I'll bet he sits alone at conferences.
104 comments
[ 3.4 ms ] story [ 190 ms ] threadIt takes much more work/time to do real research, so the funding system has actually been selecting for people who can remain ignorant (so it is not fraud) and just produce p-hacked "results". In many areas, this has been going on for multiple generations now and you are trained to do it as a grad student.
This is definitely true. I once saw a book with a title similar to "Statistics for Dummies" in a professors office. He had plenty of access to staticians at the university too. Unfortunately if a given field involves many people with ignorance of statistics, these problems may not be called out during the peer review process.
As I said, poor understanding of statistics has been a beneficial trait in many areas of research for decades now. Universities and labs are rewarded with funding because they are able to (honestly, but incorrectly) pump out more "results" than if the researchers were properly trained.
So the issue goes well beyond the problems "not being called out". The problems are institutionalized. Editors and reviewers will try to force you to commit the errors in order to publish (and hence have a career). That is what they have been trained to consider science.
Could easily have been a handy reference text for lending or showing to erring students/faculty.
Assuming the book is for his own learning, as opposed just curiosity or for students etc, there is still nothing wrong with learning from book. Assuming he really dont know the statistics.
And it definitely beats up people who don't know either, but prefer to stay ignorant just so they don't look like someone who reads beginners book.
And that's just basic math, nothing fancy. So, he knows it's bad. I assume it's difficult to get a doctorate at Stanford and then become a professor at Cornell, so there's got to be more to this than him just being stupid.
As an aside, academia is no meritocracy. A lot of what's required to get posts at schools like these amounts to self-promotion and luck (graduating in the right year & from the right lab).
Of course hard work and technical competence is also required, so your point stands.
Confidentiality? What if correlations give away subjects' identities?
I don't think that's how laws or ethics rules work, for one. And I wasn't talking about a particular experiment, for two.
Like I said, that's not how laws work.
But in academia, there are also set rules and precautions governing every study. I haven't proposed any research yet but my understanding is that if your study requires collecting data from participants, the Institutional Review Board requires you to be very clear to participants about privacy and confidentiality and that you follow the guidelines to the letter.
Additionally, there are datasets only available to academics that aren't available to non-academics (i.e. journalists), which speaks to the expectation that academics be very mindful about confidentiality promises.
[0] https://github.com/fivethirtyeight/data
[1] https://github.com/BuzzFeedNews/everything
The strictness varies by university. Ultimately the IRB is there to ensure safety of the participants and minimize the chance of negative consequences. A couple examples I've heard from around my university include...
A study was taking place outside. There was a chance of a bee sting occurring to the participant. The IRB required that the researcher have an epi-pen ready just in case along with any required training.
A paper used in a study had the wrong stamp on it (not the most recent IRB stamp). The document had not changed from the last time, but the rule was simple. Every document presented to the participant had to have the most recent stamp.
On a study in which it was expected that 1/2 of the participants would feel nauseous, they had to provide a place to sit, water, and small snacks.
And perhaps most importantly, you can quit any study you're participating in at any time, for any reason. Compensation is figured out ahead of time, and you're going to get lots of questions from IRB if you want to do anything to reduce the compensation for leaving part way through the experiment.
The current system with IRB allows for the collection of identified data. It also allows the data to be released, in full, as long as there is no way for an individual to be identified specifically. In some cases you can simply swap names for a participant ID (after randomly sorting the list). In other cases you have no choice but to publish some summary statistics because the data you collected is only applicable to a very small number of people in a small region, or there is enough data people could reverse-engineer likely participants of the study.
Science is about self-correcting when errors are made. There are plenty of reasons to criticize the current incentive structure for academics, such as there being little funding to replicate other studies, but doing away with confidentiality is absolutely not the way to go about it.
The danger for some people is death or worse. Why are you dismissing that risk?
if enough subjects don't care about confidentiality you could use only them and publish the data.
Imagine an identified versus anonymous survey about cocaine usage in America. People will likely answer honestly if it is anonymous.
If it is identified, most people would be fine with it being identified and would mark no usage. People who do use cocaine would be more likely to not participate or lie on the answer.
What if you completed a simple survey for a study and identified yourself as a member of a group (Christian, Jewish, Mexican, African American, etc.) and 5 years down the line, there was a big rounding up of your group. Suddenly your participation 5 years ago is leading to terrible life changes.
Please get to know participant protections more. They are very important for the integrity of the data used in academia. The NIH has a strict policies to ensure the safety of participants.
The Institutional Review Board (IRB) reviews studies for risks participants could face as a result of participating. Every person who has access to the de-anonymized data must complete an NIH training course.
Depending on your institution, the process varies in seriousness. Having worked on 2 submissions and helped on ~3 other IRB approved studies, it can be tedious. My institution takes 6+ weeks from submission to approval, which can really delay things. At the same time, it's a vital part of the process to ensure validity.
Another consideration is the amount of data. Sometimes you're talking about dozens or hundreds of gigabytes of data. Start multiplying this by the scale of universities and you come up with a large expense to store massive amounts of data most of which will be downloaded no more than a couple times. On top of that, the data must be cleaned and made anonymous, and if they make a mistake in the process, there could be large liability issues. See AOL https://en.wikipedia.org/wiki/AOL_search_data_leak
I'm not saying the data should never be shared. Often times you can contact the author of a particular study and they will provide anonymized data. Most of the time people don't ask, and to take the time to prepare data for public release would offer little to no benefit.
I do believe there should be "first rights" to publish on data collected with funding from public sources. As for an exact policy by which it could be handled, I don't have one in mind. When you spend dozens of hours writing grant proposals only to have most of them turned down, it would be a sucky situation to have to make the data available for everyone before you have gotten a single paper out of it.
> If others use the data they should put the originators on the data as coauthors on the resulting papers.
It may vary by field, but generating the data alone is almost certainly not sufficient for co-authorship. [1]
[1] http://www.icmje.org/recommendations/browse/roles-and-respon...
I usually do my best to find a subset of variables that can be released in an anonymized form and still capture the true essence of the results and make that available. This isn't always easy to do though depending on the study details.
a) shows that you have an error somewhere and disproves your paper, making you look stupid, or
b) does something better with your data than your paper, making you look stupid, or
c) runs a follow up study on the same data you are currently running and publishes before you have the chance
In my experience usually none of this happens, UNLESS you're in a really cut-throat/hotly debated/bleeding edge corner of science, which is fairly rare.
c) is actually problematic if you told your funding body that you're going to generate X studies with the data they paid for, since you can only publish X-1 studies. I've seen this happen with biologists not releasing their genome assemblies with the assembly paper so they could run basic follow up studies using that assembly as reference before anyone else can do it.
Thank god it's not SOP among actual sciences.
So yes, even in computer science you run into similar things. Maybe not the same, but similar processes leading to similar problems.
Aside from bumping up the quality of K-12 science education by at least an order of magnitude or two, I'm not sure there's a good solution to this problem. It's the human condition to be impressed by some things and not others, regardless of their relative actual importance.
When presented with examples like this, it's actually _irrational_ to think that there aren't other egregious papers out there. Possibly just a few, but possibly a large number of them.
Thoughts?
Fortunately, critical thought does not stop at the point of publication. When researchers read each other's papers they (hopefully) do not blindly assume them to be bug-free.
A single paper isn't as meaningful as you might think. It's more like an interesting blog post. You might use it as a starting point for your own investigation.
A single paper does not represent "the truth" as we currently understand it. For that you need to have a broad scientific consensus.
You might be able to get a better understanding of that from review papers citing recent advances in the field. Or from attending conferences/reading a broad range of papers yourself.
Conservatives don't distrust science carte blanc. Many of them will chuck money at asinine bullshit as long as it's billed as a way to kill the other team. Check out some of the more ridiculous stuff to come out of DoD.
They distrust climate science. And they don't distrust the actual science, just the conclusions. And increasingly not even the most basic conclusion, just the most obvious causal theory behind that conclusion.
Let's not throw the baby out with the bath water.
The left has its fair share of science-denying morons as well.
Is it really "huge"? Some links to polling data might help here otherwise I would also be reticent to proclaim that it's comparable to climate denial-ism. After all, we have a president that has publicly proclaimed climate change to be a hoax and sitting members of congress that eagerly attack climate scientists on their research [1].
[1] http://www.sciencemag.org/news/2017/03/lamar-smith-unbound-l...
[1] http://uk.businessinsider.com/trump-vaccines-autism-wrong-20...
Climate Science is a bunch of computer models are not testable. When they're run and compared to what actually happened they're always off. e.g. Name a single climate science model that predicted "the pause" starting in '98?
Climate Scientists don't release their model's code. They don't release the exact data that went into their model. Shit, isn't that one of the complaints about Wansink? That he isn't sharing his data?
I'm not saying climate change isn't real, I'm saying that the level of certainty that climate scientists proclaim is problematic. I'm saying that labeling dissenters as deniers (reminiscent of holocaust deniers) is beyond the pale. I'm saying that fossil fuels have done more to advance the human condition than anything else in history of mankind.
I tried to find this a few years ago in the case of measles and it turned out to seem no such studies existed. Have you seen one?
https://clinicaltrials.gov/ct2/show/NCT01536405?term=AMP&ran...
Unfortunately, I really don't think this study offers us anything about whether "vaccines work". You can see that the main concern is whether doctors are much more likely to diagnose measles when they do not believe a vaccination occurred, and whether introduction of the lab tests made the diagnostic definition more stringent:
“This was not a blind study, since the investigators knew which children had received measles vaccine. It seems probable that the occurrence of so much ‘measles-like’ illness in the vaccinated children was a reflexion of the difficulty in making a firm diagnosis of measles in the African child at one visit.”
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2134550/
“A likely reason for this is that the case may have been misdiagnosed as a non-specific viral illness. Measles has become relatively uncommon in Singapore with two decades of widespread measles vaccination, and especially after the second dose policy was implemented in 1998. Many primary care doctors may not even see a single case of measles in a year. This makes diagnosis more difficult.
[...]
As only approximately 7% of the clinically-diagnosed cases of measles reported locally turned out to be measles by laboratory testing, there is a need for laboratory confirmation of measles to avoid misidentification of cases and improve disease surveillance.(2)”
http://www.ncbi.nlm.nih.gov/pubmed/17609829
I just don't think these alternative explanations have been investigated very well, and the blinded RCT (vs placebo) is missing. Another thing, the various proxies for "vaccine success" they use do not correlate with each other...
"Our data demonstrate that regression analysis shows only limited correlation between NT results and the ELISA values. This is in agreement with other reports [4]. Similar limitations in the correlation were also reported for other viruses like Cytomegalovirus (CMV) [10]. In case of the gamma globulin samples, the low correlation might reflect the wider spectrum and heterogeneity of the involved or measured measles antibodies." http://www.ncbi.nlm.nih.gov/pubmed/17308917
Since measles is very contagious, there's a lot of effort putting into identifying and isolating cases. This necessarily involves screening many cases where the disease is only weakly suspected. On top of that, the current protocols cast a really wide drag net. Australia, for example, tests every susceptible patient who shared a waiting room with a measles patient, as well as those who entered less than two hours after the index case left. In a busy hospital or practice, this could be a lot of children.
Thus, the high ratio of tested to confirmed cases doesn't mean doctors are bad at diagnosing measles; they may just be cautious.
And yet another effect to deal with is that people used to spread measles on purpose:
“Before the introduction of measles vaccines, measles virus infected 95%–98% of children by age 18 years [1–4], and measles was considered an inevitable rite of passage. Exposure was often actively sought for children in early school years because of the greater severity of measles in adults.” http://jid.oxfordjournals.org/content/189/Supplement_1/S4.fu...
I think for these reasons it is clear the effectiveness of measles vaccines has been overstated, it is only a matter of how much.
I don't think this point is relevant to your argument.
All of science is eventually a bunch of models.
> models [that] are not testable. When they're run and compared to what actually happened they're always off
You do realize these statements contradict one another, right?
Also, you're wrong. There have been plenty of successful predictions: https://johncarlosbaez.wordpress.com/2013/02/05/successful-p...
> Climate Scientists don't release their model's code
False.
Some publish code before/with their papers, some don't publish at all, and some publish code with a lag after publication (to clean up the code, tussle with legal about IP and/or political issues, etc. Fair IMO as long as the code is eventually published).
The major models, including the NASA one, are open source.
Google it.
> Name a single climate science model that predicted "the pause" starting in '98?
Predicting century-long trends and predicting 5 or 10-year variations within that trend are extremely different tasks.
It's certainly possible that we have models that are exceptionally accurate on the scale of 20 or 50 or 100 years but cannot make accurate predictions over the course of 5 or 10 years.
In fact, in a stochastic system, that's exactly what you would expect to be the case -- trends that are globally but not locally discernible.
More-over, opponents to the climate change thesis don't bother to provide their own models and predictions for falsification. And when they do, they're just as wrong on small time scales and much more wrong on long time scales.
> I'm saying that labeling dissenters as deniers (reminiscent of holocaust deniers) is beyond the pale.
This largely depends on what is being denied:
There's also an important difference between skepticism and denial.Being skeptical of Newtonian physics and providing a competing model against which we can compare is of course fair game.
Denying Newtonian physics in favor of "no particular opinion on how the world works, and let's not try to figure out" is of course worse than useless.
In other words, make your own predictions or get off the pot.
Wouldn't that be a failure of the model?
http://www.bbc.com/news/science-environment-38513740
Sure. Stupid knows no political affiliation, other than strong and unsubstantiated belief.
Meanwhile, the right (in America) also has its fair share of unscientific beliefs about vaccination: http://www.slate.com/articles/health_and_science/medical_exa...
FWIW the only two anti-vaxxers I know in person are a retired west-coast life long democrat, and a super neo-conservative soccer mom. It seems to be more of an "anti-establishment" thing than a "left/right" thing.
On the right, virtually all mainstream politicians deny climate change.
Unless it can be made accessible to even science graduates to actually reproduce the results from a critical point of view, it's always going to have a lot of sceptics. Not everyone has so much faith in scientists.
Incidentally, your critique also applies to literally most science, including very good and obviously successful science.
Can you really prove to your students that Intel doesn't have magic pixie dust it uses to speed up its processors? Probably not without expensive equipment your classroom cannot afford.
And can you reproduce the LHC experiments? Of course not.
edit: Here's the EPA website: https://www.epa.gov/climate-change-science/future-climate-ch... There are 5 references, all larger reports which themselves link into the actual literature.
The National Climate Assessment in particular is presented in the form of a slick website with hundreds of citations into the literature for curious readers: http://nca2014.globalchange.gov/report
The information is there, in extremely digestible form, and at the top of google search results. Naturally, reproducing the results of literally hundreds of cited papers is going to take you a while, but in most cases, there's nothing actually stopping you...
And here's a list of climate models, some of which you can download. The others have links to contact info: http://www.easterbrook.ca/steve/2009/06/getting-the-source-c...
It's also worth noting that scientists are usually fairly transparent about when they're guessing WRT effects of climate change and when they're pretty certain. The near-term stuff (~hundred years) is largely guess work based on simulations -- we can't do much better. The most certain prediction is that weather will become harder to predict. The longer-term stuff (~hundreds to thousands of years) is much more well-understood because there're historical records relating GHG levels to climate.
The biggest greenhouse gas by far is water vapor, not CO2. So it makes no sense to call a plot of CO2 a plot of "greenhouse gas concentrations". This type of "information" is low quality and will only make people not trust them.
A full description of the methodology and what exactly is being plotted is available in papers linked to here: http://tntcat.iiasa.ac.at:8787/RcpDb/dsd?Action=htmlpage&pag...
Anyway you can deny what I am telling you, but it is usual for people to go to supposed authoritative site like that, then check wikipedia and see H20 is the biggest greenhouse gas.[1] As a result they just shrug and figure no one knows what they are talking about, since they can't get the story straight.
[1] https://en.wikipedia.org/wiki/Greenhouse_gas
Unless you're planning on boiling an ocean or great lake some time soon, humanity isn't directly effecting the amount of H20 in the atmosphere at appreciable levels as a first order effect. It's not a parameter under variation.
C02 equivalents are a parameter under significant variation, and as a first-order effect of human action.
We are interested in the dynamics of the system as we vary a parameter that we can, and are, directly manipulating.
Larger atmospheric, oceanic and whole system models then incorporate H20 and other effects in a simulation as the parameter we actually control is varied.
If this were explained on the page, you'd complain it was too technical.
> Anyway you can deny what I am telling you, but it is usual for people to go to supposed authoritative site like that, then check wikipedia and see H20 is the biggest greenhouse gas.[1] As a result they just shrug and figure no one knows what they are talking about, since they can't get the story straight.
Is that really common? I mean, there are actually a large population of people who will read [1] but don't bother to click on the link right under the graphic and see how the graphed data is used in climate predictions?
I'm highly suspicious that there's actually a significant population of people who are open to being convinced but make such a sophomoric mistake in their reasoning.
Someone can always find reasons to be uncharitable and dismissive if that's their predisposition. People with that predisposition and willingness and disregard evidence for even the most superficial of complaints aren't going to be convinced anyways.
I don't know whats going on with that chart, whatever. I am just telling you the end result of showing it to me, and doubt I am alone.
I don't draw straight lines on all my plots relating constants to (in)dependent variables. Even fifth grade science experiments follow this convention in their plots. It's an extremely common practice. Water vapor isn't being varied, so isn't plotted. Again, nearly every fifth grade science fair project follows this convention in their plots. It really does count as basic literacy.
For someone with some basic scientific literacy, the author's intent is obvious and confirmable by reading the literature. OBVIOUSLY the authors are aware of water vapor concentrations, and you can confirm this awareness with a superficial investigation into how this metric is used in models.
Sorry, but I do not believe a non-antagonistic reader would jump to the conclusion that climate scientists are unaware of tyoical atmospheric H20 concentrations. That's so far from the realm of reasonability.
At some point, the readers' good faith must be assumed or the entire expository enterprise is pointless. I think this plot is by far on the correct side of that line.
This is another contradiction to what I've been told: "Satellites have observed an increase in atmospheric water vapour by about 0.41 kg/m² per decade since 1988." https://www.skepticalscience.com/water-vapor-greenhouse-gas-...
We aren't actively and directly melting large quantities of ice. It's a second-order effect of increased atmospheric C02.
One option is to allow it to vary but not as a first-order effect. Another option is to simply not let it vary to keep the model simple.
There's a lot more of this going on in a lot of fields, especially the biomedical and social sciences, but probably everywhere.
I think conservatives are correct to pick up on this general phenomenon. They know there's a lot of systemic problems with research and academics, and approach things from that perspective.
The problem, to me, is that they then overstep and use that to justify an equally pernicious position, which is to denounce or dismiss science in general, rather than to encourage fixes or constructive change. Of course, depending on the conservative, they're more likely denounce scientific research on certain topics and areas of research than others.
It's throwing baby out with bathwater, basically.
The scientific community then reacts poorly, in my opinion, by denying the kernel of truth there is to their concerns, and not addressing them.
You can see this playing out in some conservative blogs. They're aware of the replication crisis, discuss it, and point to things like the email leaks by climate scientists, which in my memory were scientifically sort of moot, but did reveal how much scientists are very much human with human flaws and motivations.
Maybe I'm wrong, though, and the typical response from the scientific community is best, though: maybe it's better to treat the problems with science separate from the conservative attacks, to tackle one problem at a time.
I guess I see two problems, one being problems with academics and science, the other being the involvement of politicians in science. I think the second thing only makes the first problem worse. It's not like the politicians aren't serving their own unholy masters.
Unfortunately the media has done a lot to poison the well on these topics, coupled with politicians all too happy to never let a crisis go to waste. Republicans yelling "small government" while trying to make a government big enough to police your bedroom. Democrats yell about stopping climate change and pump billions into crony companies that blow up and fail (all the while laundering taxpayer cash as campaign donations, republicans do this with defense groups).
So you get leftists going "we shouldn't like, have new fighter jets man" and right ringers thinking global warming is a laundering scheme.
The "acid rain will destroy our forests" doom/gloom seems to have originated from a single person, Bernhard Ulrich [1] from the University of Göttingen. His dire warning gained immediate currency in a political climate where a new form of left-wing activism (now known as "Green Party" in various countries) was beginning to emerge. Political activists from the "Green" side of the spectrum effectively demonised all criticism of "acid rain will destroy our forests" and lead to generous funding for any research that seemed to corroborate the doom/gloom, triggering a self-reinforcing spiral.
[1] https://de.wikipedia.org/wiki/Bernhard_Ulrich_(Forstwissensc...
I grew up next to some of the world's largest coal-fired power plants that regularly featured in "worlds biggest polluters" lists. The forests were fine before flue-gas desulfurisation just as much as after. Some countries enacted flue-gas desulfurisation earlier than others. The forests were fine before flue-gas desulfurisation just as much as after.
Anecdotal? Sure. But one wonders ...
And presumably (although I've never seen it) there is at least one liberal who doesn't.
In school we're taught that every field is as self correcting as physics. Reality seems to be different.
"Nutrition" isn't an area of science. Dietetics is, and is a very firm and established field. A lot of the false claims and bad dietary advice come from Nutritionists, which (in the UK at least) isn't a protected term. Anyone can claim to be a Nutritionist, and Nutritionists aren't known for rigourous data.
[very good overview:] "https://reason.com/archives/2016/01/19/broken-science/print
http://www.theatlantic.com/magazine/archive/2015/09/a-scient... " "The numbers speak for themselves: evidence based medicine is broken and in crisis. Cites: Ramirez JH. Data (i.e., evidence) about evidence based medicine. figshare. http://dx.doi.org/10.6084/m9.figshare.1093997 http://www.bmj.com/content/348/bmj.g3725/rr/759895
Using a snappy XKCD is not a valid logical argument even though the poster might think it makes one look hip and cool. I wish researchers were not this lazy.
There are at least a handful of valid arguments that one could use rather than an XKCD cartoon.
I'm sure that Wansink cares about his research and its quality. He's that sort of person. I don't know what happened with these papers, but it seems likely that answering bloggers publicly would not serve the greater good.
It seems to me that all the hand-wringing about these four papers misses the bigger picture: What can we do to improve the practice of science across the board? Why is there almost no field of study which isn't tainted by the politics of left vs. right?
My suggestion is that it began when people began to believe the idea that all political and social questions are best served with some sort of applied scientific analysis. I counter that, in fact, scientists are some of the worst people to ask for solutions to political and social problems, since most of them have almost no understanding of the relevant fields of study, which mainly include history and philosophy, and perhaps psychology. Even fields like theology, linguistics, and literature and the arts have more to say to most political problems than, say, physics or chemistry or meteorology, because they speak to human behavior rather than the behavior of inanimate substances.
Please understand that I say all this not to provoke an argument, necessarily. But as someone whose formal studies were mostly in political history, in general, listening to HN talk about politics is a lot like what I must imagine it is for most on HN to listen to people on TV news shows talk about tech in general and software in particular: they sort of get it, but not really, and certainly are not as expert as they imagine themselves to be.
Wansink doesn't appear to be troubled by his prodigious contributions of unreliable results to the literature, nor does he appear inclined to correct them. All of science and discovery is a joke to him.
Left or right, the troubled extremes of politics start where respect for evidence and truth ends. Scientists are, nominally, people who search for truths about our universe with some degree of rigor, some respect for evidence, some appetite for truth (and usually an insanely competitive nature, but I digress). Absent that, what's the point?
This isn't about society, in my opinion. It's about having some respect for evidence and truth. Without these, science isn't science. And what wansink is doing isn't science. Cold fusion had more rigor. Yet he will probably be rewarded with grants of taxpayer money, taken from fellow citizens as taxes, under threat of force. Meanwhile pediatric bone marrow transplanters will shut down their labs for "lack of funding". Or, more accurately, lack of standards in science.
Yes, it affects people when dishonesty is accepted.
You lost me there. What did you mean when you included this statement here? Are you saying this particular incident is tainted by politics? I discerned nothing political in Wansink's papers, the feedback they received, Wansink's responses, or Andrew Gelman's reaction pieces. Did I miss something?
I ask this because the rest of your comment became about politics vs ___, which didn’t feel natural to follow when I didn’t understand why politics is relevant here.
One thing I have trouble figuring out: is this indicative of a long-term trend? Or did his success create an expectation he couldn't keep up with and so felt a need to rush (or cheat)?
Incidentally, there is another professor at my school who has built a career out of exposing similar problems in research (especially p-hacking). It takes a brave person to inhabit that niche. It is a service to the research community, but I'll bet he sits alone at conferences.