Unintended consequences of modern grant funding

"When a measure becomes a target, it ceases to be a good measure" - Goodhart's law

The pervasive use of bibliometrics in academia results in an incentive structure that encourages scientists to be “productive” and produce “impactful” research. The more a scientist publishes in high-impact factor journals, the more successful they are considered, and the more likely they are to receive grant funding. The phrase “publish-or-perish” was coined to describe this ecosystem. There are several unintended consequences that result from grant-givers’ dependence on inaccurate quality metrics to make funding decisions:

• Competition: The heavily competitive landscape for grant funding pits researchers in the same field against one another. Instead of incentivizing the world’s leaders in a certain topic to collaborate on findings, scientists are compelled to protect their ideas and experimental data until publication for fear of being “scooped”. This lack of communication can also lead to duplication of research studies. Multiple groups work on the same problem in parallel, with only the first to publish receiving credit. Competition among researchers is incredibly wasteful when you consider the source of funding is typically a non-profit or public organization.

• Reporting Bias: Academic journals want to publish studies that will result in a large number of citations in order to bolster their impact-factor rankings. This causes scientists to preferentially address research questions they believe will result in highly-cited studies.

• Replication Crisis: There have been estimates that up to 50% of biomedical research studies published in high-impact factor journals are unable to be replicated. Because replication studies are rarely cited, researchers have very little incentive to pursue them. For the same reason, disappointing or negative results are much less likely to reach publication.

• Artificial Demand for Publication in For-Profit Academic Journals: A large portion of the journals that are considered “high impact-factor” operate under a for-profit business model. These journals played an important role in the scientific ecosystem before the internet, but modern technology has erased their value proposition. Despite this, they maintain a tremendous demand for publication and are able to thrive due to grant-giver’s dependence on using “impact-factor” as a quality metric.

• Scientific Misconduct: There is evidence that results are less trustworthy within fields of science where funding is scarce. Competition for funding increases publication pressure which results in an increased rate of false-positive publications. In addition, the rate at which scientific papers are retracted due to misconduct is increasing faster than the overall rate of retractions.