I have been summarizing the news affecting science and education for the last 18+ months. In this series, I try to get my head around it all by delving into some historical context. I am not a historian and this is not intended to be a comprehensive history. I’m trying my best, but please let me know if I’m missing an essential event or perspective.
The previous entry can be found here.
As the American research enterprise expanded in second half of the twentieth century, broader social and political events substantially shaped public trust in scientific institutions[1].
These events differed substantially in their causes and character. Some reflected failures within the American research enterprise, such as the ethical failures of the Tuskegee Syphilis Study and the high-profile cases of research misconduct that motivated a series of congressional hearings in 1981 entitled Fraud in Biomedical Research. Others involved organized efforts to undermine scientific evidence for commercial purposes, including campaigns to cast doubt on the links between cigarette smoking and lung cancer[2] and between fossil fuels and climate change. Vaccine hesitancy, which persisted at very low levels for much of the twentieth century, would become increasingly organized and visible throughout the 1980’s and 1990’s.
In the 2000’s, ongoing conversations within the scientific community about research methods, publication practices, researcher training, professional incentives, and other issues began to coalesce under the broad umbrella of rigor and reproducibility (R&R). In parallel, technological advances lowered barriers to the adoption of “open science” practices which made scientific processes and outputs more accessible for scrutiny and reuse.
In the 2010’s, following several high-profile instances where research findings could not be reliably replicated, the field of psychology was said, by some observers, to be in “crisis”[3]. This narrative was subsequently extended to biomedicine and other fields, transforming longstanding methodological debates into a highly visible account of systemic issues in scientific practice and culture.
As these concerns gained prominence, federal agencies launched initiatives intended to foster reproducibility and transparency. For example, in 2015 the NIH revised its grant application requirements to emphasize R&R. A parallel push by federal agencies towards public access to the products of federally funded research culminated in a 2022 memo from the Office of Science and Technology Policy directing federal agencies to develop policies to ensure that publications and datasets arising from federal research are freely available.
As in earlier periods, these discussions intersected with broader social and political events. Throughout the 2010’s, several members of congress published “wastebooks”, purporting to show examples of wasteful government spending, including spending on basic science research projects. Appeals to rigor and reproducibility would also become explicitly intertwined with broader political disputes. For example, in 2018 the Environmental Protection Agency invoked reproducibility when proposing new evidentiary standards for environmental policy making. The proposal, which required the EPA to only make use of research for which the underlying data were publicly available, was broadly criticized as an attempt to limit the evidence that the agency could use in its decision making[4].
By the late 2010s, two distinct conversations about the practice and culture of science had begun to employ overlapping terminology. Internal to the scientific community were debates over research methods, researcher training, professional incentives, publication practices, and related issues. Within these, the term “reproducibility” often served as shorthand to encompass efforts to improve the reliability of scientific findings while “transparency” generally referred to efforts to make scientific processes and outputs more accessible to scrutiny and reuse (i.e. through open science practices).
A second conversation, involving policymakers and other actors both within and outside the scientific community, concerned the governance, accountability, and use of federally supported research. Here reproducibility and transparency acquired significance beyond the practice of research, instead referring to criteria through which the legitimacy of scientific findings and institutions could be evaluated. Debates about who defined these terms, which forms of evidence satisfied them, and who possessed the authority to make such determinations became increasingly polarized.
[1] This is, obviously, an understatement. My academic writing reflexes prevent me from just saying “science is not separate from the rest of the world and wow did a lot of stuff happen.” I wish I were more familiar with Science and Technology Studies (STS) to cite the relevant work I’m sure has been done on this. But alas.
[2] There is a whole tangent I could go on here about Ronald Fisher. On one hand, his contributions to research methods and statistics were absolutely foundational. On the other hand, the popularization of “correlation does not equal causation” came from his denial of a link between cigarette smoking and cancer. He was, of course, also into eugenics. There’s a lot to unpack, but its all a little out of scope for this particular blog entry.
[3] I, personally, do not like the crisis framing at all. I was finishing up grad school in a psych department when this was all kicking off (though not in social psych). It did feel a little like the sky was falling, but the whole field and the skills and experience of the people working within it were not suddenly rendered invalid. Unfortunately, when statements like “science is broken” are made, people might believe them.
There is a segment on the January 27th, 2011 episode Colbert Report featuring Daryl Bem that I think did more advance the “science is broken” narrative than any methodological discourse within the pages of the scientific literature, but a video/transcript does not seem to be available anywhere.
[4] This proposal became regulation in 2021, but only briefly. It was vacated by the EPA in February 2021, several weeks after the inauguration of Joe Biden as President. This debate has not ended, however. Following intense politicized scrutiny, the National Academy of Sciences recently announced that it has withdrawn a chapter on climate change from its Reference Manual on Scientific Evidence, which provides background on scientific issues for judges and others in the legal profession.