Meta-Research: Getting the Most Out of Research

Posted on January 6, 2016 by

0


PLOS Blog, Jan 4, 2016 by Roli Roberts — Scientific research, often portrayed as the acme of human rigorous thought, is actually an activity that’s grown rather organically over the centuries from Aristotle, via Bacon, to an endeavour that now involves millions of people across the globe. Recently, however, it’s been recognised that there are grave problems inherent in this organic structure, and that these problems may be obstructing the progress of science, wasting precious and limited resources, and producing misleading conclusions. This realisation has led to the emergence of the field of meta-research, which aims to study how research is done, and ultimately to inform measures that might mitigate these problems and realise the full potential of scientific research.

Ioannidis-thumb3

Time to scrutinise Science itself? (“Microscopio de tres cuerpos para las observaciones simultáneas” from “El mundo físico : gravedad, gravitación, luz, calor, electricidad, magnetismo, etc.” A. Guillemin. – Barcelona Montaner y Simón, 1882, via flickr user fdctsevilla)

A collateral thrust of meta-research is the recognition of the key role that scientific journals have played in perpetuating research practices, and the role that they could and should play in improving the standards of consistency in performance, analysis, reporting and interpretation of research. This is part of the reasoning behind the new broadening of PLOS Biology’s scope to include meta-research papers, and ongoing initiatives to make reporting of research more uniform and rigorous across scientific journals.

An example of how inconsistent reporting can get in the way of science is the non-uniform reporting of clinical rating scales, which can interfere with the subsequent inclusion of studies in meta-analyses; a recent PLOS ONE paper assesses the scale of the problem and proposes a practical solution that could improve the usefulness of individual studies and maximise the overall benefit to society.

Meta-analysis, often thought of as a tool for combining clinical and preclinical trial data (as in the example just mentioned), can be just as powerful in basic science, as seen in this PLOS Genetics meta-analysis of behavioural studies in fruit fly mutants, wresting statistically significant results from studies that are themselves individually marginal.

 

Crucial to all of these endeavours is the availability of the data that underlie studies, which a study in PLOS ONE shows to vary massively between scientific disciplines, being best in biology, but still falling well short of an ideal situation. And within biology itself, field-to-field variation is colossal, with some woeful behaviour reported among phylogeneticists, where some unique data may be lost for ever. Even in genetics, which has a 33-year culture of data-sharing inaugurated by GenBank/ENA/DDBJ, there’s room for significant improvement, as explored by this PLOS Genetics editorial.

Between them these articles – and those included with the launch of the PLOS Meta-Research Collection – exemplify the level of scrutiny, rigour, and practical recommendation and enforcement that will be needed to get the most out of scientific research. This research is after all conducted by a few million scientists on behalf of all humans, and largely funded by their billions of dollars of taxes and charitable donations. Making the most of that investment is a duty, not a nicety.