Sep 19, 2002

Document Tools

Print This Article

E-mail This Page

Font Size
S      M      L      XL

Related Stories

A Monumental Failure of Peer Review

How could allegedly fraudulent research, not just get published, but win prestigious academic prizes?
By Gary Mauser

The Bellesiles scandal is attracting media attention in the U.S., but it
has barely shown up on Canadian radar. Michael Bellesiles, an historian at Emory University in Atlanta, won the prestigious Bancroft prize in 2001 (awarded by Columbia University for outstanding research in history), but he is now scrambling to defend himself against charges of fraud and intentional deception. He is still under suspicion as his department continues to investigate the charges.

Despite the fact that this scandal concerns U.S. history, it raises an important challenge for Canadian academics. How could allegedly fraudulent research, not just get published, but win prestigious academic prizes? The rise and fall of Bellesiles opens a window on a more general problem - the fragility of peer review. According to professor emeritus David Bordua, at the University of Illinois, the real scandal concerns the willing gullibility of ideological reviewers and academic historians. The Bellesiles case represents a monumental failure of peer review. When his book, Arming America: the Origins of a National Gun Culture, was published in 2000, reviewers praised it apparently without bothering to evaluate the research. The book even won the Bancroft prize without anyone on this prestigious panel bothering to check his footnotes thoroughly. Isn't this something academics are supposed to do with run-of-the-mill undergraduate papers?

In 2000, Bellesiles was being fawned upon by academic and popular critics alike. Peter Onuf, the author of Jefferson's Empire,” opined that “Michael Bellesiles moves to the front rank of American historians with this deeply researched, brilliantly argued, energetically written, and timely book.” Robert J. Spitzer, author of The Politics of Gun Control, claimed the book was, “Meticulously, even extravagantly, researched.” Michael Zuckerman, the author of Peaceable Kingdoms even said, “This is stunning history, brilliantly argued. It throws into a cocked hat our most cherished assumptions about guns and gun culture in early America.” Today, these critics are silently eating crow.

What was the fuss about? In his book, Bellesiles claimed he examined more than 11,000 probate records between 1765 and 1850 and found a surprisingly small number of firearms. He concluded that firearm ownership was less widespread than previously believed, so that, before the 1860s, a widespread gun culture didn't exist. More provocatively, he claimed that onset of the U.S. Civil War, and the rapid growth of the arms industry, gave birth to the distinctive U.S. gun culture. If true, this would undercut the myth of the American minuteman who plays an important role in U.S political theory.

This is heady stuff. To many historians and social critics, this study supported their prejudices against firearms and their distrust of industry. Without checking too closely, many rushed to applaud Bellesiles as their new hero.

Unfortunately for Bellesiles, cracks soon began to appear in his argument. First, legal researchers challenged his sources. When these scholars checked the sources they couldn't confirm his findings. For example, when James Lindgren, professor of law at Northwestern University, tried to check the records, he found glaring discrepancies. Bellesiles' records either weren't where he said they would be or, if found, they differed substantially from what Bellesiles said. Other scholars also checked. All found it impossible to get Bellesiles to share his sources.

Worse was in store. When the records were checked further, many of the records Bellesiles claimed existed couldn't be found. When asked, he had various excuses, each more implausible than the ones he'd used earlier. All however were variations of “the dog ate my homework.” Bellesiles told professor Lindgren that he'd kept his notes on yellow foolscap and had no records on disk. This is surprising, but then he said his notes had been destroyed when there'd been a flood in his office. However, professor emeritus Jerome Sternstein at Brooklyn College wasn't impressed. Particularly troubling, archives where Bellesiles claimed to have taken extensive notes had no record he'd ever been there. His goose was cooked when he was confronted with claiming to have studied records that were lost in the great San Francisco earthquake and fire of 1906.

His academic critics have now painstakingly shown that Bellesiles made up or grossly misrepresented much of his data. The academics who nominated him for the Bancroft prize are no longer supporting him. The American Society of Criminology scheduled a panel to examine the scandal, but no one could be found to defend him. The National Endowment of the Humanities has acted to remove its name from Bellesiles' fellowship. After a six-month investigation, Emory University has now placed him on leave pending their decision about the charges of academic dishonesty.

Academics such as professor Bordua argue that the ease with which his book was accepted by academics is even more shocking than Bellesiles' deficient scholarship. How could anyone be awarded the Bancroft prize without any serious effort being made to corroborate his research? The real scandal is not that a professor would conduct fraudulent work, but that his peers accepted it. The most plausible explanation is that ideologues believe the end justifies the means.

In Bellesiles' case, historians eagerly allowed themselves to be hoodwinked. The charitable interpretation is that academics, like anyone else, are not as critical of arguments that support their prejudices as they are with those that are less comforting. This is just human nature. But, academics are supposed to be professionals. Every discipline has set up methodological standards to guard against such human failings. In order to be published, academic books and articles must survive rigorous criticism in the peer-review process. If we do not honourably adhere to the highest disciplinary standards, our research loses credibility.

A less charitable interpretation is that academics are more cynical. Historians may have willingly supported Bellesiles' outrageous claims, even knowing they were implausible, because doing so furthered their political objectives. Historians tend to consider themselves progressive, and so they share a dislike of firearms. Thus, they willingly flocked to support Bellesiles in order to promote their ideological agenda.

Some academics defend such cynicism, arguing that there is no such thing as truth. Everything is subjective. Unfortunately, this standard too easily promotes propaganda. Academic prizes are seen as rewards for your friends or fellow travellers. As one European intellectual told me, “The law only exists to be applied to your enemies.” I believe such a cynical approach undermines faith in all academic research.

This problem isn't limited to history. Nor is it limited to liberals. The Bellesiles case illustrates how crucial peer review is to all academics - Canadian or American, historians or scientists. If peer review can fail in history, it can fail anywhere. Recently, the Journal of the American Medical Association admitted that there are serious problems with peer review in medical publications. All academics must work to keep the peer review process healthy and act vigorously to root out unprofessional conduct.

There is no happy ending to this story yet. Peer review did eventually uncover the gross distortions, but this took years longer than it should. Bellesiles' critics continue to grow as the evidence against him mounts relentlessly. Unfortunately, the American History Association does not appear to be listening. It is too soon to know if the flaws in the peer-review process that failed with Bellesiles have been recognized, let alone fixed.

Gary Mauser (above) is a professor in the faculty of business administration and the Institute for Canadian Urban Research Studies. He was the 2001 winner of the Sterling prize for controversy.

Search SFU News Online