Can the average person discover reliable facts?

5 points by ergothus 8y ago ↗ HN

TL;DR: As a lay person (non-scientist/academic), how do I find if a topic has been studied? Given knowledge of the existence of a study, how do I find that study? Given a study, how can I easily evaluate the general reliability of it?

In this age of easy-to-distribute (mis|dis|)information I search for lots of info on the web. I have to filter through the fluff pieces and hopefully find out a study exists. Fluff news pieces appear to be the only real gateway to actual studies outside of academia, which can't be good incentives for study authors. Then I must get access to the study itself, which seems to usually involve paying a surprising amount of money to various paygates that themselves have varying veneers of trustworthiness.

If I actually manage to get a study, I don't know how to establish if it has been: peer-reviewed, published in a respected journal, replicated, and/or if it has good RELEVANT p-values. I can look for obvious logical errors/omissions, but this gives more strength to the communication skill of the author(s) than in the validity of the research.

An example topic I struggled to find studies for was "people find in-person communication to be more effective than text, but (1) how much of that is comfort/familiarity and (2) how much is a false FEELING vs FACTUALLY effective?"

An example study that I don't know how to evaluate: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3092984/

Other topics/studies are welcome, these are just to pre-answer any "can you give an example" responses.

I don't mind proxying reliability evaluation to a third party if they are considered reliable themselves, but I've not found any general answers for that (and knowing which evaluators in a niche are reliable becomes a chicken/egg problem).

Is it realistically possible for us to have confidence in "facts" outside our areas of expertise without an intense level of effort?

1 comment

[ 4.6 ms ] story [ 7.9 ms ] thread

gus_massa 8y ago ↗

It's difficult. The internet if full of crackpots and overhyped press release from the universities, so read a lot, but be aware that a lot of that is misleading or false.

Some pessimistic tips:

* Ignore any result that has less than 5 years. In 5 years it should have been confirmed or debunked. Enjoy the news and try to keep track of them for 5 years ...

* Look at the Wikipedia page. It's not 100% sure but at least there is a good chance that someone review the content.

* Photos or it didn't happen. (Preferably videos.) What evidence are they providing? Has anyone reproduced the experiment?

* Some field like nutrition have an history of extreme claims that are proven wrong a few year later. (How many eggs should you eat per week?) Assume that all field have the same level of rigor. After 10 or 15 years you can start to assign more reliability to some fields. (Why your phone battery is not 1000000 better than now?)

* Peer review is good, but not infallible. In particular be aware of p-hacking and publication bias. Also there are good and bad journals.