Free Project Idea: Website that evaluates quality of scientific research

Why now: Advances in NLP may make this possible as of now.

Why care: Scientific research is highly influential in society. You could influence the influential with a project like this.

Challenges: Getting full text of papers to parse them. Is NLP good enough to parse the important bits?

What might it look like:

Upload full-text (users would probably use sci-hub.ac) or input a link to free online full text, then it creates a permanent page on the site for that paper, with the quality ratings. This is also how it spreads - via SEO.

Other cool things:

Methods for grading evidence exist, see GRADE by cochrane. http://training.cochrane.org/path/grade-approach-evaluating-quality-evidence-pathway

It could indicate whether the study was pre-registered, whether it shows correlation or causation, whether the study shared all of its data, whether it has been reproduced successfully/unsuccessfully. (Eventually, you could cluster results around topics/hypotheses, e.g. stereotype threat, and show total and rolling avg quality and results of research on that topic, etc)

You could prevent people from falling victim to thinking small sample size means low quality research. Small sample sizes with large effect sizes can be high quality.

If you do this and need UI/UX help, build it, and comment below. I'll check back in a month or so.