Got just a single observation? New journal will publish it

Posted on December 7, 2015 by

0


ScienceInsider, 2-12-2015 by Esther Landhuis — About 8 years ago, when he was a postdoctoral researcher, cell biologist Lawrence Rajendran was applying for a job when a colleague predicted that, if he could get two papers he had co-authored published by Science, “you’re sure to get the position.”

As it turned out, the colleague was right. The journal accepted both papers and Rajendran, 40, who grew up poor in rural India, got the job. He now heads a laboratory at the University of Zürich in Switzerland and has won awards for his research on Alzheimer’s disease.

si-scientistandpetridish

But the colleague’s comment long troubled him, Rajendran says, because he felt it overemphasized “the label on my papers” from a prestigious journal. His concerns about scientific publishing grew as he studied several flawed papers that had been yanked from high-profile journals. Certain figures from those retracted manuscripts were subsequently published elsewhere, he discovered.

Those experiences made Rajendran wonder: Was the pressure to publish tempting authors to improperly tweak their findings in order to create more cohesive stories? If researchers could report just the one finding they felt comfortable with, Rajendran mused, perhaps “there would be no need to be dishonest.”

Those ponderings eventually spurred the creation of Matters. Launched on 5 November, the open-access online journal aims to boost integrity and speed the communication of science by allowing researchers to publish discrete observations rather than complete stories.

“Observations, not stories, are the pillars of good science,” the journal’s editors write on Matters’ website. “Today’s journals however, favor story-telling over observations, and congruency over complexity … Moreover, incentives associated with publishing in high-impact journals lead to loss of scientifically and ethically sound observations that do not fit the storyline, and in some unfortunate cases also to fraudulence.”

“We want to reward scientists for honesty, curiosity and the quality of their work—not just good storytelling,” Rajendran says.

Some scientists, however, worry Matters could exacerbate another problem: the long-standing practice of slicing a large body of findings into many manuscripts, in order to boost authorship numbers. “There is already a great abundance of fragmented stories being published, driven [in] large part by the misconception that the number of papers is a metric of success,” says cell biologist Peter Walter of the University of California, San Francisco (UCSF). Matters could amplify this trend, he says, “while vastly increasing the burden on the community by asking us to review individual experiments.”

Misaligned incentives?

The debate over Matters reflects a larger, ongoing discussion over incentive systems in science, and the role of storytelling in scientific communication.

John Ioannidis, a Stanford University epidemiologist in Palo Alto, California, sits on Matters’ scientific advisory board. Trained in statistics and medicine, Ioannidis is widely known for pioneering the field of “meta-research,” which examines how science is conducted and reported. One issue meta-researchers are exploring is misaligned incentives.

To advance their careers, researchers need to publish results, and those that support a compelling narrative have a greater chance of success. This is because scientists and journal reviewers, like most human beings, enjoy game-changing stories with “plot, structure, logic, drama and where hopefully the good prevails,” Ioannidis wrote in an email to ScienceInsider.

“Good,” particularly for high-impact journals, generally means novel, positive results from several lines of investigation that converge on a single hypothesis. Yet things rarely unfold this way. Research is hard, and biology is complex. Clean narratives tend to mislead by oversimplifying complicated phenomena, Ioannidis says. “Researchers are forced to build stories that are often speculative fairy tales.”

Because good stories are more satisfying than ones with false starts and loose ends, “we are incentivized to make our stories more beautiful than reality,” notes Brian Nosek, a social psychologist at the University of Virginia, Charlottesville, who also directs the Center for Open Science.

The push for new, compelling stories carries additional dangers. It discourages publication of work that merely replicates prior experiments. And it prods researchers to discount discrepant findings. Let’s say a student or postdoc tries to replicate a reported result in order to decide whether to pursue a new line of investigation. If the results fail to reproduce the original finding, the researcher faces a dilemma.

There are few, if any, places to publish one-off experiments that aren’t part of a bigger story but might still be informative. So unless the researcher “invests in a series of additional experiments to package the failed reproduction, that result will languish in laboratory notebooks,” says biomedical ethicist Jonathan Kimmelman of McGill University in Montreal, Canada. “This makes it impossible for outside scientists to assess the totality of evidence when judging the reliability of a research claim.”

Paid peer reviewers

Matters aims to address such problems by making it easier to publish bits and pieces of science. It aims to create a freely accessible venue for single findings, even confirmatory data and contradictory data.  The journal series is currently accepting submissions in cell biology, biochemistry, biophysics, neuroscience, and genetics.

The first 500 submissions will be free. After that, Matters will charge $150 per submission from universities and other nonprofit groups, and $300 per submission from for-profit entities. Half of these funds will go to editors and reviewers or, if they wish, to a charitable organization. (Collabra, an open-access journal launched in January, also pays its peer reviewers.)

Researchers can submit quickly—typically in under an hour—using Matters’ online template that combines Word and Photoshop features. Submissions are evaluated in a triple-blind process of peer review.

Observations judged to be technically sound (i.e. scoring above 4 on a 1 to 10 scale) will be published in Matters within two weeks of submission. Outstanding findings that score 8 or above in technical quality, novelty, and impact will be published in a sister journal, Mattersselect.

Will stories emerge?

Despite the emphasis on discrete observations, Matters’ founders believe good stories could still emerge from the fragments. The journal will enable reports on related observations—by the originating authors or others—to be linked, so researchers could build narratives from collections of published observations. And tools that allow scientists to visualize such networks could provide new metrics for success. Rather than being judged by their number of publications or by the impact factor of the journals in which they publish, researchers could be assessed by “how good a ‘seeder’ or ‘extender’ they are,” says Rajendran, who put in $160,000 of his own savings to help launch Matters.

Rajendran hopes Matters can also help level the playing field for researchers in poorer areas, who may lack resources such as antibodies and mouse models, “but can still make good observations.” For example, he says, consider a researcher who is able to show, with proper controls and statistics, that an extract from eucalyptus bark relieves pain under certain conditions. “In today’s world, you can’t publish that in a good journal,” Rajendran says. “You would need to know which molecule it is and whether it cured the population of Nigeria.”

However, if the “A leads to B” eucalyptus finding were published in Matters, someone else could isolate the molecule, then another lab could test it in mice and eventually human patients. The developing story never eclipses the seeding observation, and “everyone gets a piece of the pie,” Rajendran says.

The new journal could also improve the reliability of the published literature by “lowering the activation barrier” for making orphan data accessible to the broader community, Kimmelman says.

Doubts and optimism

Not everyone buys Matters’ atomistic approach. UCSF’s Walter, for example, would prefer to see researchers place a greater emphasis on “only publishing once it is clear one actually has something profound to say.” He won the Lasker and Shaw prizes in 2014 for his research on the intracellular quality control system known as the “unfolded protein response.”

Ioannidis agrees that there is currently “too much salami-slicing of publications into least publishable units on which scientists can claim authorship on seemingly more and more papers.” But he’s cautiously optimistic that outlets like Matters could still help, by allowing researchers to assemble many little bits into a greater, connected entity. The journal may help the scientific community move “in the opposite direction, one of coalescence rather than fragmentation,” Ioannidis says.

Nosek believes the reporting of fragmented findings is inevitable because “science is too complicated to have integrated, complete stories from the outset.” Embracing this reality, he and colleagues at the Center for Open Science are creating tools such as SHARE, to help connect findings, studies, protocols, data, and other parts of the research life cycle. SHARE has so far collected more than 3 million research “events” from 76 content providers, and the system’s next phase of development will focus on building services to help researchers search, filter, and link these events.

“Scientific communication is undergoing a renaissance,” Nosek says. “There is lots of innovation with new approaches to doing it better. I don’t know which will work, but we can’t know for sure without trying them out.”


Science | DOI: 10.1126/science.aad7512