Just a note that this is by Geoff Anders from Leverage Research, an organization historically plagued with some controversy in the level of psychological experimentation it is willing to perform on its members:
Hi, this is the official Leverage Hacker News account. Some clarification on the research that took place:
During the relevant period, researchers were permitted substantial freedom in determining what experiments to run and hypotheses to explore. Researchers also participated in experiments as they saw fit. This was voluntary, not all members participated in psychological research, and promotions and salaries were not tied to participation.
One should imagine a purposefully unstructured environment with 30+ people trying to figure out how the mind worked and which self-improvement modalities worked best, rather than subjects at a clinic being experimented on. Our researchers explored tons of hypotheses and we think that was great.
There are difficult questions about balancing people’s freedom to experiment with their own minds with safety in experimentation, including in an institutional context, and that’s something we think there should be more public discussion about.
It seems that what's really needed is a formalization of the uncertainty of scientific claims, which I'm not sure is even possible to implement fully. But it's perhaps possible to schematize the relevant information - a database of who did what experiment, what the current hypotheses are, and which ones scientist believe the evidence favors, like a prediction market. Besides being a valuable tool for science, it would serve as a way of assigning appropriate levels authority.
I find it surprising that the author credits William Whewell for coining the term "scientist", without mentioning the polymath Mary Somerville whom he was describing. Her research spanned mathematics, astronomy, geology, chemistry, and physics, such that simply calling her a physicist was insufficient.
This article is about the relationship between the state and scientific authority during the pandemic, but a notable omission is that it's purely about the relationship between the state and scientific authority during the pandemic in the United States. As far as I know, in developed countries, scientific authority functioned as usual, and pretty effectively. That seems like an important contrast to consider.
No, the problems were pretty much the same everywhere. The article is germane to all countries. Scientific authority was relentlessly abused everywhere and the trust issues are now global.
I have not heard working scientists say “Trust the science” as much as “Trust the scientific method”. And that method is precisely the motto of the Royal Society: “Trust no one’s word” (nullius in verba).
This has been drummed into us for the last 100 years with Popper as just an exclamation mark and Feyerabend as a harsh metacritic.
I therefore find the contrast/dialectic in this short essay to be spurious.
The suggestions for solutions at the end are silly both at conceptual and practical levels. Proposing to further fragment science into smaller silos to prevent premature consensus—that made me laugh.
'Trust the scientific method' begs the question: what is the scientific method? Duhem-Quine thesis and Feyerabend's criticisms suggest that Poppers falsificationism isn't actually useful in practice. I had the false idea drummed into me that the demarcation problem was solved, and this attitude leads to a lot of epistemological arrogance in many fields.
Lakatos must be mentioned alongside Feyerabend. His 'Methodology of Scientific Research Programmes' is a solution to many of the problems people identify with scientific methods. It hasn't been applied to many fields because of the difficulty of converting existing research into explicit research programmes, but I think that soon digital tooling will make it possible.
What we currently use to determine 'truth' ('scientific consensus') is closer to Kuhn's 'scientific paradigms' which is almost the complete opposite of nullius in verba.
One example of the 'trust the science' is from Australia's chief scientist! Her proposed method to get people to 'trust in science' is to simply make sure scientists have 'integrity.' As if the only thing preventing scientists from making false claims is fraud. I think this attitude is common and fits with the article's criticism of state pressure damaging science. https://www.chiefscientist.gov.au/news-and-media/trust-scien...
"[Antiscience] targets prominent scientists and attempts to discredit them. The destructive potential of antiscience was fully realized in the U.S.S.R. under Joseph Stalin." - Dr Peter Hotez
And please don't play No True Scotsman games, Fauci was employed due to his scientific credentials and was therefore very much a working scientist. Moreover, one who controlled most of the others, as has been seen in the various virology related leaks. Hotez is a professor of molecular virology, has testified in front of governments and frequently appeared on TV to make false or blatantly illogical/incoherent claims [1] like "this two dose vaccine has always needed three doses".
Hotez isn't some fringe lunatic disowned by other scientists, far from it. Here's the editor of a journal literally called Science claiming that asking Hotez to engage in debate is a "classic anti-science setup":
Last weekend, Twitter and later the mainstream media exploded with a controversy surrounding an invitation to prominent vaccine scientist Peter Hotez to debate anti-vax charlatan and spoiler presidential candidate Robert F. Kennedy Jr on the podcast, The Joe Rogan Experience. There was an immediate rally around Hotez by scientists and celebrities on Twitter and lots of discussion about why this invitation is a classic anti-science setup.
The picture you're projecting of what scientists do is a pleasant daydream, not the nightmarish reality. Scientists constantly attack anyone who doubts their proclamations even when the criticism is scientifically based, and frequently call for outright censorship of anyone who disagrees with them.
It's a good historical retelling, and it has some unique ideas, but they seem rather milquetoast compared to the scale of the problem. Asking journals to split themselves into exploratory and authoritative isn't going to work in a prestige system where everyone wants to be seen as authoritative. And there was nothing about the more concrete issues like widespread fraud.
Your conclusions about what needs to be changed will depend on how deep you go in your analysis. A lot of writing about the problems in science take a surface level view and so try to come up with ideas like different archiving strategies for data, different ways to communicate.
It's also possible to conclude that these are only treatments for the symptoms not causes, and so won't work. You'd just replace one set of symptoms with another.
Evidence for this view can be found in the outcomes of prior attempts to reform science in the modern era. For example a frequent criticism of academic work is that they don't make their data available, thus papers have to be taken at face value as a claim by authority rather than the original scientific ideal of Nullius In Verba that the article mentions. Distributing data is easy but academics complained it is too hard for them, so some journals compromised and said OK, you just have to sign a document saying that you'll send your data to anyone who asks. The result was predictable. Eventually someone tested how often scientists would send data when asked like they'd agreed to in writing, and discovered it was pathetically low, like 5% or lower. There had been the appearance of reform without reform actually taking place.
This happens because the incentives are bad. Scientists earn more money and gain more power when they are accepted as authorities. Being accepted as such means never showing doubt, never admitting to mistakes, never admitting to uncertainty. All the things scientists are meant to do freely, in practice, they are incentivized not to do. And there are no penalties anywhere for manipulating people through pseudo-scientific claims. Lots to win, nothing to lose = lots of fraud and collapsing trust, except of course amongst that subset of the population that wishes to live by blindly following authority (perhaps because they themselves are a part of the prestige credentials game). Science is structurally creating division and polarization.
The fix for this is as brutal as it is simple: eliminate all charitable or taxpayer funding for science. All science should be corporate. This moves science out of the prestige economy into the market economy, which has many mechanisms and evolved cultures designed to tackle bad incentives. For example, it's hard to directly monetize scientific claims in a market economy, you're usually monetizing results (minor exception: self help books, ted talks, etc). You can try to create fake results but then there are many laws, law enforcement bodies and even journalists trying to catch you. There are truth in advertising laws in some places, your employer can be sued for fraud under many statutes and there's a culture of not blindly trusting companies so those mechanisms actually work. See the huge outcome gap between fraud at Theranos and the endless frauds at universities.
I think the problems are enormous. Fixing them requires either many small, incremental changes or some large, sweeping changes like the one you suggest.
My proposal is meant to be incremental, beneficial, and implementable, since it can be adopted by individual actors in the scientific ecosystem. It has the obvious imperfection of not solving all or most of the problems.
In terms of sweeping changes, I'm in favor but we need to get the answer right. Corporate science generally fails to pursue long-term research aims. Taking an example from the history of electricity, it was more than 200 years from William Gilbert's isolation of static electric attraction until electricity was commercialized. If science is to be owned by business, business will need to think on timeframes many times longer than it currently does.
If people were interested, I could produce a proposal for overhauling science in general. One important challenge is the need to think about both (1) scientific progress, and (2) how science fits into society. Approaches to both need to change.
I wonder if it might be possible to get back to "Nullius in verba" in modern day... Imagine if all the aspects of "settled science" came with instructions for simple at-home or basic-lab setup that could reproduce the result. I know it won't be possible for particle physics, but I bet you most of chemistry and biology could be done without too much fancy equipment.
Even if not really reproducing from scratch at home, you could maybe watch a youtube video of someone using the fancy equipment and showing the result predicted by theory. Lol... maybe TikTok to adapt to the information needs of the new generation.
I guess what I'd like to see is more science communication for the public, but not superficial stuff based on analogies and simplified explanations, but the real stuff. Actual science experiments. Make the settled science accessible to a wider audience + show by example how to run scientific experiments (predict something and try it out empirically to verify your prediction). It's not like I don't trust the scientists, but I want to experience the "high" of doing science, not being told that someone is doing science next door and it's good stuff.
I love this vision, I think of this in the context of the future of science museums.
Imagine a museum where you could go from room to room and make each of the discoveries. There could be the easy walk-through version or the hard "escape room" version where you can only go on to the next room when you make the discovery, and you don't know what the discovery is supposed to be. (Of course you could have hints available.)
>Lone investigators created theories, built instruments, conducted experiments, and wrote accounts of different aspects of nature. There could then be decades or even centuries before the next major recorded advance.
Lots of times they were doing things that no one else they came in contact with can begin to even understand very well.
Natural scientists have often each made constant progress around the world as long as Homo Sapiens has been around.
It's just that the records of what and when they did it are so few and far between, and most of them have been lost or never even recodred to begin with. It simply took decades or even centuries before some of this ultra-rare stuff turns up, often unexpectedly.
If only one percent of research today turns out to be outstanding, you would have to expect that over the whole of human history, and especially prehistory, orders of magnitude more of the outstanding stuff was lost than the paltry developments one single generation can come up with.
18 comments
[ 4.8 ms ] story [ 53.9 ms ] threadhttps://medium.com/@zoecurzi/my-experience-with-leverage-res...
During the relevant period, researchers were permitted substantial freedom in determining what experiments to run and hypotheses to explore. Researchers also participated in experiments as they saw fit. This was voluntary, not all members participated in psychological research, and promotions and salaries were not tied to participation.
One should imagine a purposefully unstructured environment with 30+ people trying to figure out how the mind worked and which self-improvement modalities worked best, rather than subjects at a clinic being experimented on. Our researchers explored tons of hypotheses and we think that was great.
There are difficult questions about balancing people’s freedom to experiment with their own minds with safety in experimentation, including in an institutional context, and that’s something we think there should be more public discussion about.
(For people interested in the linked account, we did an inquiry on that topic. The report is available here: https://www.leverageresearch.org/_files/ugd/51c82b_c477a6576...)
Here's something I've started reading about her (https://www.themarginalian.org/2016/12/26/mary-somerville-sc...), would be happy for further references.
[1]: https://news.ycombinator.com/item?id=37095302 "We need scientific dissidents"
[2]: https://archive.li/6vnq0
I therefore find the contrast/dialectic in this short essay to be spurious.
The suggestions for solutions at the end are silly both at conceptual and practical levels. Proposing to further fragment science into smaller silos to prevent premature consensus—that made me laugh.
https://en.m.wikipedia.org/wiki/Karl_Popper
https://en.m.wikipedia.org/wiki/Paul_Feyerabend
Lakatos must be mentioned alongside Feyerabend. His 'Methodology of Scientific Research Programmes' is a solution to many of the problems people identify with scientific methods. It hasn't been applied to many fields because of the difficulty of converting existing research into explicit research programmes, but I think that soon digital tooling will make it possible.
What we currently use to determine 'truth' ('scientific consensus') is closer to Kuhn's 'scientific paradigms' which is almost the complete opposite of nullius in verba.
One example of the 'trust the science' is from Australia's chief scientist! Her proposed method to get people to 'trust in science' is to simply make sure scientists have 'integrity.' As if the only thing preventing scientists from making false claims is fraud. I think this attitude is common and fits with the article's criticism of state pressure damaging science. https://www.chiefscientist.gov.au/news-and-media/trust-scien...
"So it’s easy to criticize, but they’re really criticizing science because I represent science." -- Fauci
or
https://www.scientificamerican.com/article/the-antiscience-m...
"[Antiscience] targets prominent scientists and attempts to discredit them. The destructive potential of antiscience was fully realized in the U.S.S.R. under Joseph Stalin." - Dr Peter Hotez
And please don't play No True Scotsman games, Fauci was employed due to his scientific credentials and was therefore very much a working scientist. Moreover, one who controlled most of the others, as has been seen in the various virology related leaks. Hotez is a professor of molecular virology, has testified in front of governments and frequently appeared on TV to make false or blatantly illogical/incoherent claims [1] like "this two dose vaccine has always needed three doses".
Hotez isn't some fringe lunatic disowned by other scientists, far from it. Here's the editor of a journal literally called Science claiming that asking Hotez to engage in debate is a "classic anti-science setup":
https://www.science.org/content/blog-post/scientists-shouldn...
Last weekend, Twitter and later the mainstream media exploded with a controversy surrounding an invitation to prominent vaccine scientist Peter Hotez to debate anti-vax charlatan and spoiler presidential candidate Robert F. Kennedy Jr on the podcast, The Joe Rogan Experience. There was an immediate rally around Hotez by scientists and celebrities on Twitter and lots of discussion about why this invitation is a classic anti-science setup.
The picture you're projecting of what scientists do is a pleasant daydream, not the nightmarish reality. Scientists constantly attack anyone who doubts their proclamations even when the criticism is scientifically based, and frequently call for outright censorship of anyone who disagrees with them.
[1] https://www.youtube.com/watch?v=Sj6-QDVYbv8
Your conclusions about what needs to be changed will depend on how deep you go in your analysis. A lot of writing about the problems in science take a surface level view and so try to come up with ideas like different archiving strategies for data, different ways to communicate.
It's also possible to conclude that these are only treatments for the symptoms not causes, and so won't work. You'd just replace one set of symptoms with another.
Evidence for this view can be found in the outcomes of prior attempts to reform science in the modern era. For example a frequent criticism of academic work is that they don't make their data available, thus papers have to be taken at face value as a claim by authority rather than the original scientific ideal of Nullius In Verba that the article mentions. Distributing data is easy but academics complained it is too hard for them, so some journals compromised and said OK, you just have to sign a document saying that you'll send your data to anyone who asks. The result was predictable. Eventually someone tested how often scientists would send data when asked like they'd agreed to in writing, and discovered it was pathetically low, like 5% or lower. There had been the appearance of reform without reform actually taking place.
This happens because the incentives are bad. Scientists earn more money and gain more power when they are accepted as authorities. Being accepted as such means never showing doubt, never admitting to mistakes, never admitting to uncertainty. All the things scientists are meant to do freely, in practice, they are incentivized not to do. And there are no penalties anywhere for manipulating people through pseudo-scientific claims. Lots to win, nothing to lose = lots of fraud and collapsing trust, except of course amongst that subset of the population that wishes to live by blindly following authority (perhaps because they themselves are a part of the prestige credentials game). Science is structurally creating division and polarization.
The fix for this is as brutal as it is simple: eliminate all charitable or taxpayer funding for science. All science should be corporate. This moves science out of the prestige economy into the market economy, which has many mechanisms and evolved cultures designed to tackle bad incentives. For example, it's hard to directly monetize scientific claims in a market economy, you're usually monetizing results (minor exception: self help books, ted talks, etc). You can try to create fake results but then there are many laws, law enforcement bodies and even journalists trying to catch you. There are truth in advertising laws in some places, your employer can be sued for fraud under many statutes and there's a culture of not blindly trusting companies so those mechanisms actually work. See the huge outcome gap between fraud at Theranos and the endless frauds at universities.
My proposal is meant to be incremental, beneficial, and implementable, since it can be adopted by individual actors in the scientific ecosystem. It has the obvious imperfection of not solving all or most of the problems.
In terms of sweeping changes, I'm in favor but we need to get the answer right. Corporate science generally fails to pursue long-term research aims. Taking an example from the history of electricity, it was more than 200 years from William Gilbert's isolation of static electric attraction until electricity was commercialized. If science is to be owned by business, business will need to think on timeframes many times longer than it currently does.
If people were interested, I could produce a proposal for overhauling science in general. One important challenge is the need to think about both (1) scientific progress, and (2) how science fits into society. Approaches to both need to change.
Even if not really reproducing from scratch at home, you could maybe watch a youtube video of someone using the fancy equipment and showing the result predicted by theory. Lol... maybe TikTok to adapt to the information needs of the new generation.
I guess what I'd like to see is more science communication for the public, but not superficial stuff based on analogies and simplified explanations, but the real stuff. Actual science experiments. Make the settled science accessible to a wider audience + show by example how to run scientific experiments (predict something and try it out empirically to verify your prediction). It's not like I don't trust the scientists, but I want to experience the "high" of doing science, not being told that someone is doing science next door and it's good stuff.
Imagine a museum where you could go from room to room and make each of the discoveries. There could be the easy walk-through version or the hard "escape room" version where you can only go on to the next room when you make the discovery, and you don't know what the discovery is supposed to be. (Of course you could have hints available.)
The DIY at-home version is great also.
Lots of times they were doing things that no one else they came in contact with can begin to even understand very well.
Natural scientists have often each made constant progress around the world as long as Homo Sapiens has been around.
It's just that the records of what and when they did it are so few and far between, and most of them have been lost or never even recodred to begin with. It simply took decades or even centuries before some of this ultra-rare stuff turns up, often unexpectedly.
If only one percent of research today turns out to be outstanding, you would have to expect that over the whole of human history, and especially prehistory, orders of magnitude more of the outstanding stuff was lost than the paltry developments one single generation can come up with.