Featured · Working Paper

Hyper-competition and the Narrowing of Scientific Ideas: Evidence from U.S. Biomedical Research, 1985–2018

by Mattia Pasqualini · PhD Candidate, Eindhoven University of Technology · supervised by Prof. Emilio Raiteri

StatusWorking Paper, 2025
FieldScience of Science
MethodContinuous DiD
SettingU.S. Biomedical / NIH

Abstract

A growing body of evidence documents a long-run decline in the disruptiveness and novelty of scientific output, despite rising research investment and a fast-growing scientific workforce. The dominant interpretation attributes this slowdown to an exogenous "burden of knowledge", breakthrough ideas are becoming harder to find as low-hanging fruit is exhausted.

This paper proposes and tests an alternative, endogenous explanation: the rise in competition for funding, positions, and publications within academia has progressively shifted scientists' idea selection toward safer, more conventional projects, narrowing the set of trajectories actively explored.

Using the American Recovery and Reinvestment Act (ARRA) of 2009 as an exogenous, one-time positive shock to NIH funding availability, which varied substantially across NIH Institutes and sub-fields, I apply a continuous difference-in-differences design to test whether the reduction in field-level competition induced by ARRA translates into a measurable post-shock increase in the novelty of publications.

Research Question

Has the long-run intensification of competition in U.S. biomedical science contributed causally to the decline in the novelty and disruptiveness of publicly funded research?

Key Contributions

01
First systematic, multi-measure assessment of competition and novelty in U.S. biomedical science over more than three decades.
02
Provides evidence for an endogenous, institutional explanation of the innovation slowdown, complementing the exogenous "ideas getting harder" narrative.
03
Leverages text-based novelty measurement that overcomes citation-window biases identified in prior literature.
04
Informs the policy debate on funding design: stability, longer grant horizons, and career security may shape novelty composition, not just research volume.

Methodology

1

Multi-measure novelty

Scientific novelty measured along three independent dimensions: MeSH-based novelty (new term introductions), text-based novelty (novel noun-phrase combinations in titles/abstracts), and disruptiveness via the CD-index.

2

Field-level competition indicators

NIH application success rates, scientists-per-funding-dollar, share of papers in top-decile journals relative to field size, and share of non-tenure-track positions.

3

Continuous difference-in-differences

The 2009 ARRA shock to NIH funding is exploited as a quasi-experiment, with treatment intensity varying continuously across NIH Institutes and sub-fields. Effects are expected to emerge at the typical 2–5 year publication lag.

Interactive data

Explore the underlying grant landscape directly. The interactive explorer covers more than one million NIH grants across ten institutes, filterable by activity type, application type, institute, study section, and ARRA status.

Open the NIH Grants Explorer

Keywords

Innovation economics Science of science Causal inference NIH funding Difference-in-differences CD-index Bibliometrics
Request full paper View all research

Request more information

Interested in the full working paper, data, or methodology? Send a message and I'll get back to you directly.

This opens your email client addressed to m.pasqualini@tue.nl with your message pre-filled.

Opening your email client...