Describing this entirely as "the declining authority of statistics" is like saying the WTC towers suffered "declining structural integrity" before they collapsed - there are people behind both...
WTC7, a solid steel 47-story building, spontaneously disintegrated and collapsed symmetrically into its footprint at near free-fall speed. This was due to small fires on the ground level, not the government.
EDIT: small fires at ground level have caused perfect symmetric demolitions of many skyscrapers since WTC7. This is why small fires are now the preferred method of demolition.
Even if there were explosives planted in the WTC to coincide with the plane crashes and guarantee the buildings' collapse, why would that suggest it is the US government? Couldn't that also be due to Al Queda or perhaps at a higher level the Saudi government? It's not the first time that Islamic terrorists tried blowing up the building.
I think the downvotes are because you jump to conclusions and the conclusion is "the government did it and covered it up". Sounds super kooky.
The science in the link is interesting, but note how it does not talk about why it happened that way. So it is disingenuous to say that the science says the government did it. Furthermore while these scientists believe that it was a simultaneous failure of all supporting columns there are others that think differently.
I find it fascinating that there is so much disagreement on the subject, and that how difficult is to reconstruct an event of such magnitude.
> At one point does policy targeting a statistical measurement render that statistic useless?
When the statistic is a proxy for a quantity of interest, and when policy exploits ways of manipulating that proxy without commensurate change in the underlying quantity of interest. My driving policy of targeting my speedometer does a fine job of controlling my vehicle's speed even though the former is just a potentially-inaccurate measurement.
In the case of the CPI (actually the PCE for the US Fed [1]), the quantity of interest is the "[nominal] cost of living." To the extent the CPI doesn't in fact measure the cost of living, the easy solution is to change to a price index that does measure the cost of living. Since living tends to involve buying and consuming things, a price index, suitably defined, can be a fine measure of this.
To the extent you are concerned about the exact ways in which the CPI is computed, independent measurements [2] generally reproduce the broad trends in US data, which in turn suggests there is no significant manipulation by the BLS.
As usual, the title's description is wrong, and the real news is buried and overlooked in the story:
"People assume that the numbers are manipulated and dislike the elitism of resorting to quantitative evidence."
If people don't believe that something is useful, it doesn't matter what that "something" is, it will start to have a diminishing affect our society. It goes for statistics and liberal democracies just as much as ancient gods and ancient languages of yore.
That's like saying if people don't like military drones, then military drones will start to have less relevance; but some technologies are sufficiently powerful in their own right that it needs only a few people to recognize their power. Then the stress of competition can do the rest.
I doubt statistics or mathematics gained its influence because various cultures were fond of symbolic play. We study these things because we expect concrete payoff, not because of cultural affection.
So is social science lame? Or is Google insight scary? At some point a deliberate posture of mis-devaluation will come to terms with reality.
> That's like saying if people don't like military drones, then military drones will start to have less relevance;
Sure I'll back that up. Thought experiment: if everyone, including all the drone pilots, suddenly magically stand up all at once, walk away, and never talk about it again, then yeah it won't impact humanity.
Is this particular scenario realistic? No. Has mathematic or chemical knowledge been collectively lost before in human history, and been eliminated from society? Yes.
> but some technologies are sufficiently powerful in their own right that it needs only a few people to recognize their power. Then the stress of competition can do the rest.
Sure, those few power can recognize its power, but if it's anything short of immortality, never underestimate the power of every other human being being a Luddite wielding a pitchfork ready to kill over progress, because it violates a cultural norm. Doesn't matter if your statistics can find a cure, societies and collective humans' irrationalities are expert suppressors of technology as both a mean and an end. Look no further than the US stance on nuclear waste reprocessing. The science and power are there. The will of people, is not. Or, look at South Africa and Ukraine giving up nuclear weapons. Power? Yes. Popular will to hold onto this power? No.
> We study these things because we expect concrete payoff, not because of cultural affection.
Exactly, because society values concrete payoff, things like "derivations from first principles" and "following the scientific method". But if society as a whole pressures people to never pursue ideologies along these lines, or turn actively hostile against it (book burning, HDD erasing).
I take this article as an alarming warning. Scientific progress is itself not an end. And it is not an inevitability. I like scientific progress and statistical tools, this article just goes to show how careful -- ethically responsible -- those of us wielding them must be when living in our society. And call out bad actors.
This is why engineering professions have ethics boards. If engineers build bridges and bad actors build bad bridges, people won't trust bridges anymore. And if people don't trust bridges, they don't get used, so more don't get built.
The greatest failure of the "modern" education has precisely been the removal of religious ethics without an equivalent amount of time working to help students develop their sense of personal ethics.
I see this as a good thing. Many 'scientific' studies are manipulated and paid for by special interests. Our existing peer review & journal system hides the actual experimental setup behind paywalls, there is often no repeatability.
The scientific system needs to be opened up or replaced. Distributing information and experimental results no longer requires expensive printing.
People are right not to trust it, scientific journals are now the gatekeepers rather than the gateways they once were.
I can't help but laugh. Make the slightest mention of statistics broken down by race, and the same publication that is despairing of how people no longer trust statistics, would be finding innumerable reasons why those statistics are invalid.
Hey now, in the Guardian's defense, it's not as if any world government has ever been caught lying and misrepresenting facts to its population in the past.
Why does that make you laugh? Your take away should not be "well clearly everything someone calls stats should be supported". The article is about how stats are becoming dismissed before examination, not that they shouldn't be scrutinized.
And that all the examples given of uneducated masses distrusting statistics just happen to align with the Guardian's pet causes is of course pure coincidence.
Really, that's the perfect example of why statistics are distrusted - only those statistics that align with someone's agenda get presented. And sometimes, they're not even gathered in the first place: https://www.thelocal.se/20180508/why-sweden-doesnt-keep-stat...
You're describing a separate phenomenon from what the article is about. Poking at statistics with biased/politically motivated effort is not the same thing as dismissing the field of statistics entirely.
The article is not suggesting that statistics should not be scrutinized.
I guess the point is that, when the statistics support their political bent, they decry people questioning the validity of them, and consider not blindly accepting them as being proof of the idiocy of the reader.
But, give them statistics that go against their pre-ordained narrative, and suddenly it is the reader who doesn't immediately refute the statistics who is the uneducated rube.
Journalism has been one of the worst offenders with regard to the misuse of statistics, not only in "soft" topics, but also in science reporting. Perhaps statistical literacy ought to be normative in that profession the same way that checking sources and using correct grammar (supposedly) is.
Even if statistical literacy was normative wouldn't people's selfishness still cause them to use/abuse statistics to influence those who are less literate in them?
Knowledge can be abused, but lack of knowledge will almost certainly lead to, unintentional, abuse. Especially in the case of statistics.
In many cases, reporters want to be dramatic, but not liars. However, the lack of deeper statistical knowledge often lead to them becoming liars, even though they only wanted to be dramatic.
It is more about declining trust in the source of that information and the institutions that spread it around. People have been given plenty of reasons to distrust the various sources of information over the past decades.
This is an objectively good thing, most statistics don't actually tell most people anything interesting about the world. They don't aid the average untrained guy in the street in gaining a better understanding of the world around them. They are bombarded with opinions that are based on contradictory studies and statistics all day, this is bad in politics and sociology but medical statistics are probably the worst offenders.
Wine is good for you, chocolate is bad for you, cholesterol is good for you, wine is bad for you, red meat good for you, Fluoride bad for you! I could find you a statistically significant study supporting any conclusion I want right this minute. Since that is the case, why should the average Joe believe any of them? This is made even worse by news articles and popsci blogs not even linking to the source papers, so avg Jane can't even investigate further if she wants to.
I am not a trained medical professional, and I don't have time to be scrutinizing and reading background literature to gain a full understanding of the context each specific study has within its sub-field. I don't have access to whether these conclusions are generally accepted or are commonly disregarded for some technicality in the methodology that I didn't understand. Me personally, I can't be an expert in everything all the time, and I can't waste my life gaining a full breadth of understanding of every topic. I rely on extrapolations to do this for me, if those extrapolations are commonly faulty, I won't pay any mind to them.
Unfortunately, this is the best option for modeling the world around us that humanity has come up with. Throwing out the only tool we have to evaluate and re-evaluate what we consider to be knowledge is not the best course of action.
Solution is to support proper scientific procedures, support civic organizations carrying out the important tasks of vetting claims, and being an example of someone who values the correct protocols when evaluating data and making conclusions (or not making them).
It's a very slow process, and we're lucky we're in a timeline where the work of various people throughout history has gotten us here. We'll have to work to keep it going further, and literacy in math, statistics, and the scientific process is crucial.
I disagree here strongly. It is not the best option. Statistics has become the tool of obfuscation and making exaggerated claims.
Compare the two statements: "10 out of 80 people that took the drug lived a month longer but were extremely miserable."
Versus: "A new cancer drug doubles the average survival rates scientists say".
Which one do you understand better? Here I just made up some examples, but as a scientist I'll tell you just about all the papers I read are in the second category, it takes an unreasonable amount of effort to get the first type of infomration out of the publications.
I don't understand the chain of reasoning in your claim regarding the 1st and 2nd statements, and statistics in general.
When I use the word statistics in this conversation, I mean the use of math to analyze and compare data. This can be done inappropriately, of course, but the problem doesn't lie with the principles of math (including statistics), but the agent using them inappropriately.
And medical (or biology related) statistics are used most inappropriately, due to various cognitive biases and financial incentives. But the problem is not statistics, and it's still the best tool we have unless we can find a Magic School Bus to shrink us down and go observe the chemical reactions in real time. But I know that stats are abused, so anytime I see a fantastical claim, especially in medicine, I approach it much skepticism.
I happen to agree with the sentiment expressed above that the entire concept of hypothesis testing is fundamentally flawed. This is what you call the "best tool". It is not, it is the tool that is used because it allows for the most fudging of results.
the problem is that in many (most cases?) even trained scientist would have a hard time deciding which statements are statistically valid.
There are endless ways to be unintentionally obscure, to semi-intentionally obfuscate or to just flat out lie. And it very hard to tell these apart - one would need to dedicate more effort than usually worth.
That's why we have organizations like the CDC, FDA, NHTSA, etc. We have the internet, cheap computing and storage of data, so it should all be transparent. Countries verifying claims independently goes a long way too.
And obviously (sweeping) results from health related experiments should obviously be met with skepticism considering the complex nature of the problem.
Not in physics or chemistry. It's a bigger problem in biology because we want to be able to make claims but don't have the means to properly design and carry out experiments that aren't subject to many errors.
The fact that the statement “most statistics don't actually tell most people anything interesting about the world” relies on two statistical conditions is humorous to me.
That a current generation is under-equipped to use quantitative discourse is sad, but it also suggests that what communities need are experts and not evidence. They want to listen to a trustworthy leader and not conflicting data -- because data will conflict.
no, unfortunately, it is not about lack of education
walk into any university, go from one faculty office to another, ask each professor what a p-value is and the overwhelming majority won't give you even a half-decent definition (I did this and can attest to it)
you know why? because p-values are far more complex than people care to admit. Most statisticians cannot properly use it! There are ongoing "ideological wars" over what it actually means. Even the founders Fisher etc argued all the time about them. If you used p-value correctly, you would almost never be able to find anything, not even valid results.
So people have to "bend" the truth a little bit. Nowadays more often than before, the truth gets bent out of shape.
If people were better educated regarding math and statistics, then it would become much less effective to "bend" the truth, especially with verification of experiments.
But yes, society doesn't function well when people are corrupted.
you have to bend the truth because it would not work otherwise at all.
Your distribution is not normal - it almost never is, now you have to argue that the method is still robust - there is no proof that is indeed robust as you claim. Then your measurement is a bit biased, see on Tuesdays the measurements do turn out a little bit differently, but most often does not matter, so you skimp over that as well.
Statistics is the wrong approach because if you did everything correctly and stuck to the letter of it you only end up hurting yourself. You'll make fewer of the valid discoveries that you make when you bend it a bit.
This is the fundamentally wrong thing about it! Strictly sticking to the rules will make you weaker. Bending statistics a little leads to more valid results.
I just gave you an example, there are many statistical measures that have that particular requirement.
If it is not normality then something else. Every statistical measure makes assumptions, most of which never quite hold true. That was my point. When someone uses statistics they always fudge things a bit. Many times does not matter, if you were stickler for "precision" it would not work.
I think this is missing the point though, I've taken an upper level Statistics class, most people here with CS degrees have, but that doesn't help me understand whether a sociological study has a faulty component in its methodology. This is compounded by the culture of not reporting full data alongside papers, and the culture of writing papers in terse language.
Bet you dollars to donuts none of the underlying studies you complain about pass judgment in the form of x good x now bad. Those are always added by the pop media or implied by the reading.
Which can as well be seen an argument why statistics are necessary, as even with them we are stuck with a lot of bias. Imagine where we would be if we didn't have statistics at all!
A statement like "most statistics don't actually tell most people anything" demonstrates both a profound lack of understanding of statistics. So why comment?
As the old saying goes: "Better to remain silent and be thought a fool than to speak and to remove all doubt."
To understand statistics, you have to either know some statistics, or be willing to learn.
It is it's own thing, the art and science of making data and observations into information. Yes newspapers and the regular pundits abuse it all the time, but don't pin that on statistics, pin that on those that abuse it with, or without (usually) knowing better.
Statistics are unavoidable, unavoidable enough that much of our perception seem to make use of the same statistical calculations sciententists use, only without us having the ability to scrutinize its workings.
Too many science projects reporting statistical results that do not hold up in repeat experiments; contradictory media coverage of the same topic repeatedly as a result.
Reduces trust in science, statistics, and the media all at once.
I think the bigger problem is that the initial experiments are usually funded by interests with a desired outcome in the study - instead of being funded by grant money or educational funds.
I think the solution to this problem is more statistics. In particular, we need a mandatory statistics course at the high school level. The problem is that people are bombarded with "studies" right now claiming to prove stuff and many of them do no such thing. If people had a better sense of what is provable and what isn't, what makes for a good study, how much you should trust the results of a single study etc, I'd hope that they'd be far more informed and effective citizens.
Ultimately, I feel this headline ("Statistics have lost their power," in case the Guardian changes it later) buries the lede. The dispute is not really about statistics as a way of understanding the world, it is about the epistemology of it, and at its core the idea that a national aggregate is a valid proxy for well-being.
It's easy to decry:
> Presented with official estimates of how many immigrants are in the country illegally, a common response is to scoff. Far from increasing support for immigration, British Future found, pointing to its positive effect on GDP can actually make people more hostile to it.
... but the underlying belief appears to be closer to:
> If you live in one of the towns in the Welsh valleys that was once dependent on steel manufacturing or mining for jobs, politicians talking of how “the economy” is “doing well” are likely to breed additional resentment. From that standpoint, the term “GDP” fails to capture anything meaningful or credible.
The first statement makes no sense in and of itself, and it would seemingly suggest a knee-jerk reaction against the very idea of data. However, in the context provided by the second statement we see a different view: a core distrust that "average well-being" (represented by the statistic of choice) in fact represents the common good, suitably defined[1].
As a result, I feel that the article's later musings -- that we're merely collecting the wrong kind of statistics -- misses the core point. If we start reporting on underemployment, then the right circumstances might come along to discredit that indicator, and so on. The article begins to grapple with this issue in its digression into big data (and how there are no national agencies for it in the manner of the ONS), but even there the author deals with a practical problem of said data being more useful in the short term than long term than the core problem:
Ultimately, if society as a whole does not have a shared view of the 'common good', then _no quantity at all_ can measure it. The question of "is this good?" becomes fundamentally unanswerable in a democracy with an identity crisis.
[1] -- That could mean "median well-being," "well-being of people I know / in my region / like me," or "personal, individual well-being of the speaker."
I feel like all of this is overcomplicating the issue. Now, at the risk of oversimplifying, I offer this:
If I am not doing well and some journalist or some economist tells me the GDP has grown a lot and everything is great, I am going to feel resentment toward that person. Doubly so if they see my less-than-enthusiasm and start talking about the “common good.” Why? Because the implication is either that there’s something wrong with me (for not keeping up with the Joneses) or that I am an acceptable sacrifice for the benefit of the common good.
Okay, it’s fine if you (the journalist or economist) think I’m an acceptable sacrifice, but don’t expect me to be happy about it, let alone agree with you. Don’t expect me to support the status quo, come Election Day. DO expect me to try to throw a spanner into the gears. DO expect me to support a strongman who says he’ll champion my cause over the cause of those who think I’m the acceptable sacrifice.
Even if the underlying statistics accurately represent the situation correctly, bad analysis on top of it creates the same results. From last week:
"Paul Krugman and other mainstream trade experts are now admitting that they were wrong about globalization: It hurt American workers far more than they thought it would."
That Foreign Policy article illustrates exactly why the influence of statistics is on the decline. If you read Krugman's original article (https://www.bloomberg.com/opinion/articles/2019-10-10/inequa...), you can see Krugman make several points, all using statistics. One point is that the effect of globalization in the 90s was small, but it had an unexpectedly large impact on manufacturing in the 2000s. The second point is that the overall effect disguised the fact that certain industries were particularly hard hit. He then cites the papers where you can find the details. These papers only exist because of the statistics they do. Krugman is a clear writer, and he is taking the unusual step of admitting he was wrong, so his article should be interesting.
But nobody cares about that, because nobody cares about statistics anymore. They want that dopamine hit of outrage, so they read the FP article, that deliberately misrepresents the argument (notice how "manufacturing" suddenly becomes "workers", despite the fact that most workers are not in manufacturing), and doesn't give any evidence. It just quotes a bunch of people settling scores with Krugman. Even there, it mischaracterizes, for example, Krugman's review of Greider's book to heighten the outrage.
Statistics is boring, and rage is interesting, so the Internet era optimizes for rage.
The quote, "using statistics like a drunk uses a lamppost, for support and not for illumination" comes to mind.
Data is great for illuminating things, but for arguments, it's usually cherry picked. I do think that writers for the guardian vastly underestimate the intelligence of people who don't agree with them. I generally mistrust statistical arguments because someone forearmed with statistical talking points has usually already signalled their intransigence. Data is nice, but alignment of interests and incentives is better.
It reduces to, "according to criteria and data I have selected, your argument does not bear out." It is a passive appeal to the authority of whoever provided the number, and not meaningful or logical information in itself.
Logical arguments tend not to persuade anyway, but to say someones argument is illogical because it does not resist a coded illogical premise (appeal to authority fallacy), is certainly a way to make them give up on arguing with you.
This single sentence explains the problem very well:
> The thinktank British Future has studied how best to win arguments in favour of immigration and multiculturalism.
BF didn't set out to figure out whether immigration and multiculturalism is favorable. They only tried to figure out how to use statistics (or anything else) in support of their foregone conclusion.
Maybe the headline should be "Statistics have lost their usefulness for propaganda."
69 comments
[ 4.8 ms ] story [ 141 ms ] threadEdit: naturally, the hackernews crowd ignores actual science which clearly demonstrates fire could not have been the cause: http://ine.uaf.edu/wtc7
Downvote away, losers.
EDIT: small fires at ground level have caused perfect symmetric demolitions of many skyscrapers since WTC7. This is why small fires are now the preferred method of demolition.
The science in the link is interesting, but note how it does not talk about why it happened that way. So it is disingenuous to say that the science says the government did it. Furthermore while these scientists believe that it was a simultaneous failure of all supporting columns there are others that think differently.
I find it fascinating that there is so much disagreement on the subject, and that how difficult is to reconstruct an event of such magnitude.
When the statistic is a proxy for a quantity of interest, and when policy exploits ways of manipulating that proxy without commensurate change in the underlying quantity of interest. My driving policy of targeting my speedometer does a fine job of controlling my vehicle's speed even though the former is just a potentially-inaccurate measurement.
In the case of the CPI (actually the PCE for the US Fed [1]), the quantity of interest is the "[nominal] cost of living." To the extent the CPI doesn't in fact measure the cost of living, the easy solution is to change to a price index that does measure the cost of living. Since living tends to involve buying and consuming things, a price index, suitably defined, can be a fine measure of this.
To the extent you are concerned about the exact ways in which the CPI is computed, independent measurements [2] generally reproduce the broad trends in US data, which in turn suggests there is no significant manipulation by the BLS.
[1] -- https://en.wikipedia.org/wiki/Personal_consumption_expenditu...
[2] -- http://www.thebillionpricesproject.com/
"People assume that the numbers are manipulated and dislike the elitism of resorting to quantitative evidence."
If people don't believe that something is useful, it doesn't matter what that "something" is, it will start to have a diminishing affect our society. It goes for statistics and liberal democracies just as much as ancient gods and ancient languages of yore.
I doubt statistics or mathematics gained its influence because various cultures were fond of symbolic play. We study these things because we expect concrete payoff, not because of cultural affection.
So is social science lame? Or is Google insight scary? At some point a deliberate posture of mis-devaluation will come to terms with reality.
Sure I'll back that up. Thought experiment: if everyone, including all the drone pilots, suddenly magically stand up all at once, walk away, and never talk about it again, then yeah it won't impact humanity.
Is this particular scenario realistic? No. Has mathematic or chemical knowledge been collectively lost before in human history, and been eliminated from society? Yes.
> but some technologies are sufficiently powerful in their own right that it needs only a few people to recognize their power. Then the stress of competition can do the rest.
Sure, those few power can recognize its power, but if it's anything short of immortality, never underestimate the power of every other human being being a Luddite wielding a pitchfork ready to kill over progress, because it violates a cultural norm. Doesn't matter if your statistics can find a cure, societies and collective humans' irrationalities are expert suppressors of technology as both a mean and an end. Look no further than the US stance on nuclear waste reprocessing. The science and power are there. The will of people, is not. Or, look at South Africa and Ukraine giving up nuclear weapons. Power? Yes. Popular will to hold onto this power? No.
> We study these things because we expect concrete payoff, not because of cultural affection.
Exactly, because society values concrete payoff, things like "derivations from first principles" and "following the scientific method". But if society as a whole pressures people to never pursue ideologies along these lines, or turn actively hostile against it (book burning, HDD erasing).
I take this article as an alarming warning. Scientific progress is itself not an end. And it is not an inevitability. I like scientific progress and statistical tools, this article just goes to show how careful -- ethically responsible -- those of us wielding them must be when living in our society. And call out bad actors.
This is why engineering professions have ethics boards. If engineers build bridges and bad actors build bad bridges, people won't trust bridges anymore. And if people don't trust bridges, they don't get used, so more don't get built.
The greatest failure of the "modern" education has precisely been the removal of religious ethics without an equivalent amount of time working to help students develop their sense of personal ethics.
The scientific system needs to be opened up or replaced. Distributing information and experimental results no longer requires expensive printing.
People are right not to trust it, scientific journals are now the gatekeepers rather than the gateways they once were.
Really, that's the perfect example of why statistics are distrusted - only those statistics that align with someone's agenda get presented. And sometimes, they're not even gathered in the first place: https://www.thelocal.se/20180508/why-sweden-doesnt-keep-stat...
The article is not suggesting that statistics should not be scrutinized.
But, give them statistics that go against their pre-ordained narrative, and suddenly it is the reader who doesn't immediately refute the statistics who is the uneducated rube.
In many cases, reporters want to be dramatic, but not liars. However, the lack of deeper statistical knowledge often lead to them becoming liars, even though they only wanted to be dramatic.
Plus it gets misused all the time - even by statisticians that should know better.
Wine is good for you, chocolate is bad for you, cholesterol is good for you, wine is bad for you, red meat good for you, Fluoride bad for you! I could find you a statistically significant study supporting any conclusion I want right this minute. Since that is the case, why should the average Joe believe any of them? This is made even worse by news articles and popsci blogs not even linking to the source papers, so avg Jane can't even investigate further if she wants to.
Solution is to support proper scientific procedures, support civic organizations carrying out the important tasks of vetting claims, and being an example of someone who values the correct protocols when evaluating data and making conclusions (or not making them).
It's a very slow process, and we're lucky we're in a timeline where the work of various people throughout history has gotten us here. We'll have to work to keep it going further, and literacy in math, statistics, and the scientific process is crucial.
Compare the two statements: "10 out of 80 people that took the drug lived a month longer but were extremely miserable."
Versus: "A new cancer drug doubles the average survival rates scientists say".
Which one do you understand better? Here I just made up some examples, but as a scientist I'll tell you just about all the papers I read are in the second category, it takes an unreasonable amount of effort to get the first type of infomration out of the publications.
When I use the word statistics in this conversation, I mean the use of math to analyze and compare data. This can be done inappropriately, of course, but the problem doesn't lie with the principles of math (including statistics), but the agent using them inappropriately.
And medical (or biology related) statistics are used most inappropriately, due to various cognitive biases and financial incentives. But the problem is not statistics, and it's still the best tool we have unless we can find a Magic School Bus to shrink us down and go observe the chemical reactions in real time. But I know that stats are abused, so anytime I see a fantastical claim, especially in medicine, I approach it much skepticism.
Psychology journal bans P values
https://www.nature.com/news/psychology-journal-bans-p-values...
I happen to agree with the sentiment expressed above that the entire concept of hypothesis testing is fundamentally flawed. This is what you call the "best tool". It is not, it is the tool that is used because it allows for the most fudging of results.
There are endless ways to be unintentionally obscure, to semi-intentionally obfuscate or to just flat out lie. And it very hard to tell these apart - one would need to dedicate more effort than usually worth.
And obviously (sweeping) results from health related experiments should obviously be met with skepticism considering the complex nature of the problem.
But that's practically all they're used for!
IMHO it actually suggests we need to improve the average person's educational level
walk into any university, go from one faculty office to another, ask each professor what a p-value is and the overwhelming majority won't give you even a half-decent definition (I did this and can attest to it)
you know why? because p-values are far more complex than people care to admit. Most statisticians cannot properly use it! There are ongoing "ideological wars" over what it actually means. Even the founders Fisher etc argued all the time about them. If you used p-value correctly, you would almost never be able to find anything, not even valid results.
So people have to "bend" the truth a little bit. Nowadays more often than before, the truth gets bent out of shape.
But yes, society doesn't function well when people are corrupted.
Your distribution is not normal - it almost never is, now you have to argue that the method is still robust - there is no proof that is indeed robust as you claim. Then your measurement is a bit biased, see on Tuesdays the measurements do turn out a little bit differently, but most often does not matter, so you skimp over that as well.
Statistics is the wrong approach because if you did everything correctly and stuck to the letter of it you only end up hurting yourself. You'll make fewer of the valid discoveries that you make when you bend it a bit.
This is the fundamentally wrong thing about it! Strictly sticking to the rules will make you weaker. Bending statistics a little leads to more valid results.
If it is not normality then something else. Every statistical measure makes assumptions, most of which never quite hold true. That was my point. When someone uses statistics they always fudge things a bit. Many times does not matter, if you were stickler for "precision" it would not work.
https://slatestarcodex.com/2014/04/28/the-control-group-is-o...
Basically it boils down to the fact that studies & statistics used are still very, very, prone to human bias.
As the old saying goes: "Better to remain silent and be thought a fool than to speak and to remove all doubt."
It is it's own thing, the art and science of making data and observations into information. Yes newspapers and the regular pundits abuse it all the time, but don't pin that on statistics, pin that on those that abuse it with, or without (usually) knowing better.
Statistics are unavoidable, unavoidable enough that much of our perception seem to make use of the same statistical calculations sciententists use, only without us having the ability to scrutinize its workings.
Reduces trust in science, statistics, and the media all at once.
"doing the same thing over and over and expecting a different result."
probably the most common mistake students everywhere make. They keep taking more courses - it never helps just makes you more confused.
It's easy to decry:
> Presented with official estimates of how many immigrants are in the country illegally, a common response is to scoff. Far from increasing support for immigration, British Future found, pointing to its positive effect on GDP can actually make people more hostile to it.
... but the underlying belief appears to be closer to:
> If you live in one of the towns in the Welsh valleys that was once dependent on steel manufacturing or mining for jobs, politicians talking of how “the economy” is “doing well” are likely to breed additional resentment. From that standpoint, the term “GDP” fails to capture anything meaningful or credible.
The first statement makes no sense in and of itself, and it would seemingly suggest a knee-jerk reaction against the very idea of data. However, in the context provided by the second statement we see a different view: a core distrust that "average well-being" (represented by the statistic of choice) in fact represents the common good, suitably defined[1].
As a result, I feel that the article's later musings -- that we're merely collecting the wrong kind of statistics -- misses the core point. If we start reporting on underemployment, then the right circumstances might come along to discredit that indicator, and so on. The article begins to grapple with this issue in its digression into big data (and how there are no national agencies for it in the manner of the ONS), but even there the author deals with a practical problem of said data being more useful in the short term than long term than the core problem:
Ultimately, if society as a whole does not have a shared view of the 'common good', then _no quantity at all_ can measure it. The question of "is this good?" becomes fundamentally unanswerable in a democracy with an identity crisis.
[1] -- That could mean "median well-being," "well-being of people I know / in my region / like me," or "personal, individual well-being of the speaker."
If I am not doing well and some journalist or some economist tells me the GDP has grown a lot and everything is great, I am going to feel resentment toward that person. Doubly so if they see my less-than-enthusiasm and start talking about the “common good.” Why? Because the implication is either that there’s something wrong with me (for not keeping up with the Joneses) or that I am an acceptable sacrifice for the benefit of the common good.
Okay, it’s fine if you (the journalist or economist) think I’m an acceptable sacrifice, but don’t expect me to be happy about it, let alone agree with you. Don’t expect me to support the status quo, come Election Day. DO expect me to try to throw a spanner into the gears. DO expect me to support a strongman who says he’ll champion my cause over the cause of those who think I’m the acceptable sacrifice.
"Paul Krugman and other mainstream trade experts are now admitting that they were wrong about globalization: It hurt American workers far more than they thought it would."
Ref: https://news.ycombinator.com/item?id=21357096
But nobody cares about that, because nobody cares about statistics anymore. They want that dopamine hit of outrage, so they read the FP article, that deliberately misrepresents the argument (notice how "manufacturing" suddenly becomes "workers", despite the fact that most workers are not in manufacturing), and doesn't give any evidence. It just quotes a bunch of people settling scores with Krugman. Even there, it mischaracterizes, for example, Krugman's review of Greider's book to heighten the outrage.
Statistics is boring, and rage is interesting, so the Internet era optimizes for rage.
Data is great for illuminating things, but for arguments, it's usually cherry picked. I do think that writers for the guardian vastly underestimate the intelligence of people who don't agree with them. I generally mistrust statistical arguments because someone forearmed with statistical talking points has usually already signalled their intransigence. Data is nice, but alignment of interests and incentives is better.
It reduces to, "according to criteria and data I have selected, your argument does not bear out." It is a passive appeal to the authority of whoever provided the number, and not meaningful or logical information in itself.
Logical arguments tend not to persuade anyway, but to say someones argument is illogical because it does not resist a coded illogical premise (appeal to authority fallacy), is certainly a way to make them give up on arguing with you.
> The thinktank British Future has studied how best to win arguments in favour of immigration and multiculturalism.
BF didn't set out to figure out whether immigration and multiculturalism is favorable. They only tried to figure out how to use statistics (or anything else) in support of their foregone conclusion.
Maybe the headline should be "Statistics have lost their usefulness for propaganda."
The other 90% challenge the sources of this article.
(With apologies you Yogi Berra)