Friday Fun: A Healthy Skepticism About Science?

"There are a substantial number of scientists who have manipulated data, so that they will have dollars rolling into their projects." -- Rick Perry, Governor of Texas

The Daily Show with Jon Stewart
Get More: Daily Show Full Episodes,Political Humor & Satire Blog,The Daily Show on Facebook

Last week a colleague of mine pointed out this wonderful little video from the Daily Show. In it, the Daily Show's "chief science correspondent" Aasif Mandvi asks some pretty hilarious and revealing questions about the scientific method. This piece is amazing and I definitely recommend taking the 6 minutes or so to watch it. At one point, Mandvi speaks to a Republican strategist, Noelle Nikpour, who seems to be skeptical of science (that is a bit of an understatement). In the interview, Ms. Nikpour says "Scientists are scamming the American people for their own financial gain." Hilarity aside, that one person, let alone hundreds or thousands, would believe this about science is a disturbing thought. Particularly since Ms. Nikpour's opinions are likely to influence more people than any single scientist's research ever would. Admittedly, recent events in social psychology have not been a good defense against this opinion (see here and here). Nonetheless, today I'd like to discuss why this opinion is inaccurate.


In the Daily Show piece Mandvi also interviews Nobel Laureate and biologist Dr. Martin Chalfie of Columbia University. In defense of the scientific method, Chalfie points out that "no other occupation has so many people questioning what you do all the time." The process Dr. Chalfie is referring to is of course, the peer review process-- a process wherein scientific research is extensively vetted by other scientists, to determine the intellectual merits, appropriateness of design, and novelty of the research.

For me-- a social/personality psychologist by training-- the peer review process goes like this:

(1) I submit a research paper to a scientific journal.
(2) An editor of the journal assigns the paper to three or four experts in my field of research.
(3) The experts anonymously read and critique the research.
(4) The editor independently reads the paper and the reviews.
(5) The editor sends the results from this process back to me, including the reviewers' comments and a decision on whether the paper is, or will ever be, publishable in the journal.
(6) I either respond to the reviewers' comments and resubmit the paper for a second round of review, or upon rejection by the editor, I send to another journal to undergo the same process.

I imagine that this process sounds like a nightmare, and for many psychologists it can be. The good news is that after these six steps are run through a few times, sometimes at several different journals, most papers are accepted for publication. The bad news is that research papers--even really good ones with novel ideas, great methods, and promising avenues of future research-- face mostly negative criticism and rejection throughout this process.

I don't know how other researchers' experiences in the peer review process differ from my own, but I can give some brief statistics on my experience. Of the 14 empirical papers I've been fortunate to have accepted for publication, exactly 0/14 were accepted on their first submission to a journal. Of those 14/14 rejected on their first submission, I was informed on 8/14 occasions that the paper would never be acceptable for publication at that particular journal.

My record for the most rejections by different journals for a single paper that is now published? Three (a tie between two papers).

My record for longest time revising a paper that is now published? My undergraduate honors thesis was published in 2009 after being originally submitted for publication in 2003.

My record for most rounds of consecutive revision at the same journal? Five rounds.

I hope this look under the hood of the peer review process helps to counter the notion that science is one big scary financial scam (For another account of peer review, go here). If anything, I think peer review is a little too critical of new research, researchers get a little too territorial about their pet ideas, editors are sometimes too risk averse when judging new ideas, and experts vary widely in their judgments of research merits (Peters & Ceci, 2010). Peer review isn't perfect, but in the end, I think the process actually benefits science as a whole by both ensuring that scientific research is held to high standards and providing a clear argument against political strategists who claim that the scientific record should not be trusted.

I'd love to hear about your opinions regarding the peer review process, or about the way science is portrayed in the media, in the comments!

Peters, D., & Ceci, S. (2010). Peer-review practices of psychological journals: The fate of published articles, submitted again Behavioral and Brain Sciences, 5 (02) DOI: 10.1017/S0140525X00011183