Subtractive Scholarship

BY RICHARD P. PHELPS

With each public remark a scholar may add to society’s collective working memory or subtract from it. Their addition is the new research they present in a journal article or conference presentation. The subtraction, when it occurs, is typically found in the scholar’s portrayal of previous research on the topic.

Editors typically grant scholars, and especially celebrity scholars, quite a bit of latitude in how they reference the universe of other relevant research. The single new study presented in a manuscript sent to a scholarly journal for review may be rigorously critiqued even while the literature review presented at the beginning is not reviewed at all. This dynamic allows scholars to write pretty much anything they please about the universe of research—declaring themselves to be the first in the world to study the topic (and, thus, to be the world’s foremost expert), ignoring or demeaning the work of professional rivals, or referencing only the work of friends, who may return the favor in their own writings.

Preposterous, you say?

Indeed, the practice of “dismissive reviews” pervades contemporary scholarship. Look for yourself. Access a standard internet search engine, such as Google, Bing, or DuckDuckGo, and enter phrases such as: “there is no research,” “this is the first study,” “little research exists,” “there are few studies,” “paucity of research,” and the like.

Here are some search engine counts I got on Google (September 12, 2022) for certain phrases:

  • “absence of research” ~152,000,000
  • “absence of studies” ~102,000,000
  • “this is the first study” ~64,900,000
  • “little research” ~39,200,000
  • “paucity of research” ~11,300,000

Granted, the count totals ascribed by the search engines are only algorithmic guesstimates. Nonetheless, these are huge numbers. Wade through the results and see for yourself how common these claims of originality are, and for commonplace topics that, in fact, have been studied for decades or centuries.

Still, you ask, how can one get away with such deception? Easily.

Consider who is going to discipline the braggadocious or lazy scholar? Scholarly journals operate constantly in crisis mode. A genuinely thorough peer review requires hours or even days of a volunteer scholar’s time. Editors are generally happy if a peer reviewer checks the analysis section of the manuscript for egregious errors. There is typically no expectation that a peer reviewer will attempt to verify a manuscript’s literature review claims, a task that could occupy weeks or months. In subfields I know well, I have often encountered reviews dismissing research literatures hundreds, even thousands, of studies deep. Reviewers are rarely deeply familiar with the relevant research literature outside their subfield.

Granted, occasionally a peer reviewer will be both familiar with the allegedly “nonexistent” research literature and forthright about it. If that reviewer rejects the manuscript for that journal, however, many other journals exist that will be less picky. If one belongs to a group with its own publication platforms, such as a think tank or government-subsidized research center, peer reviews can be extra tolerant and friendly.

Being “first” to study a topic (i.e., making a “firstness claim”) can help attract funding. Some foundations and government agencies prioritize research grants for apparently understudied topics—“research voids.”

The world continues to produce a gargantuan quantity of research, much of it publicly funded, yet some of the most celebrated scholars repeatedly assert a shortfall across the length of their careers. Moreover, they often repeat their dismissive reviews in more popular venues, such as magazines and newspapers, and on radio, television, and podcasts. And they can get away with it because there exist:

  • no negative consequences for doing it;
  • no incentives (and profound disincentives) for anyone to report them; and
  • way too many research documents for anyone to sift through quickly and thoroughly.

Probably, most scholars still try to be additive, by accurately representing the work of others when they present their own. In behaving ethically, say by conducting honest, thorough, and enormously time-consuming literature reviews, however, they cripple their careers. In the race for attention, tenure, promotion, prestige, awards, and higher citation totals, subtractive scholars retain the advantage.

There’s a wicked beauty in dismissing or unfairly denigrating rival research, it:

  • saves a ton of time that a thorough literature review requires;
  • saves time having to explain why other evidence contradicts one’s own;
  • avoids messy and time-consuming scholarly debates;
  • allows one to cover more topics over time, which aids in the (false) appearance that one is broadly informed;
  • signals to journalists and policymakers that they need not bother searching for rival evidence, because, allegedly, it does not exist;
  • does not help readers to find rival scholars and research, as they are not even referenced;
  • puts rival scholars on the defensive, the implication being that they are not mentioned because they are inferior, unimportant;
  • puts rivals in the position of appearing petulant if they protest the exclusion or misrepresentation, of having “sour grapes”;
  • does not seem personal, as rival scholars are not even identified—to protest the misrepresentation, by contrast, an offended rival must identify those who have misrepresented them, which can be interpreted as a personal attack; and
  • seems to be fine with many journalists.

As advantageous as subtractive scholarship is to an individual, the advantages multiply when scholars form cooperative groups—citation cartelsciting and referencing only each other’s work and ignoring or denigrating that of others outside the group.

The considerable advantages of dismissive reviews and citation cartels, however, accrue to individuals and small groups. Meanwhile, society—the public and policy makers—is left misinformed, sometimes dramatically so.

Richard P. Phelps is the founder of the Nonpartisan Education Review and the editor and coauthor of Correcting Fallacies about Educational and Psychological Testing (APA Books).