Wednesday, August 14, 2013

The Psychology of the "Psychology Isn't a Science" Argument

Tom knows a pseudoscience when he sees one! (wikipedia.org)
Every so often the internet is set ablaze with opinion pieces on a familiar question: Are "soft" sciences, like psychology, actually science? Most of the time the argument against psychology as a science comes from people from the so-called harder sciences (you know, people who don't know ish about psychology). Of course, every once in a while we throw ourselves under the bus by declaring that for our softer sciences to be taken seriously, we must be more like the real sciences. You're still reading this so most likely you are interested in my opinion on this topic. With a quick nod to others who have covered this topic herehere, here, and here, let's review some of the arguments for and against psychology as a science in what follows.

I. Psychologists do unscientific things
Whenever I read a story about how psychology isn't a real science it is usually accompanied by mentions of some psychologists (and I use the term loosely here) engaging in unscientific things. This includes the stacks and stacks of pseudo scientific self-help books that claim to reveal the science of X, the mere existence of celebrity psychologists like Dr. Drew or Dr. Phil, and the fraudulent research of now disgraced psychologists like Dirk Smeesters or Marc Hauser.

It is true that psychology has its fair share of pseudo science, I mean the entire diagnostic manual of mental disorders has continued to resist integration with research findings (see here). However, there is real science happening here in psychology--that is, our scientific journals are packed with research summaries where psychological scientists have used the scientific method to test a specific hypothesis.

The argument that a field is not a science just because some of its members aren't scientists doesn't really hold water. Take the case of fraud as an example: Have you been to retraction watch recently? If you go there you will find that scientific fraud is not the domain of just the soft sciences. Fraud effects the sciences, from hard to soft.

II. Psychology doesn't define its terminology well enough to be considered a science
Some people (usually who know little about psychology) argue that psychologists don't define their terms clearly enough to be considered a science. In one example of this, a physicist named Alex Berezow (using a bunch of sciency terms that my poor psychologist brain struggled to understand) argued that happiness research is a perfect example of a failure to define terms. He states that "the meaning of the word differs from person to person and especially between cultures."

Setting aside the point of cultural variability for a second, I actually think that happiness research is a very bad example of poor definitions in psychology. People who study subjective well-being have spent decades arriving at a definition of the construct that is comprised of three parts--subjective cognitive assessments of one's life as meaningful, positive affect, and negative affect. Importantly, they didn't just come to a definition by writing a random opinion piece in the LA times about a field they know nothing about. Instead, researchers arrived at this definition based on decades of evidence gathered based on thousands (possibly millions by now) of people reflecting on their happiness (go here for the source of this effort). This hardly seems like a deficit in terminology.

III. Psychology relies too heavily on subjective experience
Much of what bothers people about psychology is the sheer subjectivity of it all. That is, how we perceive any number of social phenomena is likely to vary a great deal from person to person. This problem is at the very core of psychology research, and many people feel that, to be science, psychologists must uncover universal human psychological processes.

The problem with this logic is that the search for human universals, with very few exceptions, is likely to be a fool's errand. Studying the human experience means asking people how they feel, and those feelings are likely to vary from person to person, situation to situation, and culture to culture. The inherent messiness is the challenge that each psychologist faces in his/her research (and the fun). That psychological phenomena are often culturally or situationally bound is not evidence of lack of scientific rigor, but rather, acknowledgement of the power of cultures and situations to influence how we perceive and respond to our social environments.

Of course, some psychologists think the subjective isn't sciency enough, and so they operationalize their variables in ways that are far less bound to the meaning messiness of questionnaire measures. In the realm of happiness, if a researcher is unsatisfied with subjective ratings, he or she might measure the length of telomeres (a marker of cell aging) or the levels of glucocorticoid hormones in the blood stream. Psychologists do this too, and this sort of work is much closer to what even a physicist or chemist might consider science.

Of course, whether a psychologists uses subjective self-reports or biological measures (or mathematical models that "precisely" quantify the golden ratio of positive affect) does not make them any more of a scientist. All biological measures really do is make psychologists look more like "hard" scientists to other "hard" scientists.  

IV. Psychology isn't falsifiable
This criticism comes from within our own field as well as from the outside: Psychologists too often publish positive findings--that is findings that support rather than contradict hypotheses. Publication of primarily positive findings suggests that psychologists are more interested in supporting their own beliefs about human experience than in finding truth about that experience. It is because of this trend that one of my colleagues suggested that our field contains more lawyers than scientists (here).  

This criticism is actually a fair one in my book--psychologists are often likely to bury data that does not support their theories about the world (even famous psychologists like Stanley Milgram are guilty of this). This practice strikes me as unscientific because it renders hypotheses more difficult to falsify. However, the good news is that efforts are underway to pay more attention to negative findings (see here).

What is the psychology behind the "Psychology isn't a Science" argument?
I think there are several basic psychological principles that help explain why this argument strikes a nerve with so many people. I tend to think of it in terms of social comparison. Psychologists like to weigh in on the psychology is a science perspective because we are engaging in upward social comparison--We want a seat at the table with the hard sciences, we want to be published in the most prestigious science journals, and we want a larger share of the grant funding from our government. In contrast, the harder sciences engage in downward social comparison with psychology--Hard sciences seek to maintain their elevated position in the science hierarchy, and sometimes they accomplish this by disparaging the softer sciences.

To close, I would just like to point out that psychology is a very young science, and so to expect it to have the same prestige and admiration as other sciences that have been around for centuries is a little far-fetched. Just like it takes a person time to build respect and prestige amongst his or her co-workers, psychology is going to have to be around for a bit longer before it starts earning the respect and admiration of other sciences. I'm okay with this and if you want to talk with me more about this issue I'll be here: Doing science.


Fanelli, D. (2010). Positive Results increase down the hierarchy of the sciences PLOS ONE DOI: 10.1371/journal.pone.0010068

Fredrickson BL, & Losada MF (2005). Positive affect and the complex dynamics of human flourishing. The American psychologist, 60 (7), 678-86 PMID: 16221001

Taylor SE, & Lobel M (1989). Social comparison activity under threat: downward evaluation and upward contacts. Psychological review, 96 (4), 569-75 PMID: 2678204

19 comments:

  1. 'This problem is at the very core of psychology research, and many people feel that, to be science, psychologists must uncover universal human psychological processes.

    The problem with this logic is that the search for human universals, with very few exceptions, is likely to be a fool's errand. Studying the human experience means asking people how they feel, and those feelings are likely to vary from person to person, situation to situation, and culture to culture. The inherent messiness is the challenge that each psychologist faces in his/her research (and the fun). That psychological phenomena are often culturally or situationally bound is not evidence of lack of scientific rigor, but rather, acknowledgement of the power of cultures and situations to influence how we perceive and respond to our social environments.'

    Okay, so if things hinge on multiple other things then I wonder if it might be advisable for psychologists to:

    1) refrain from over-stating conclusions.

    (Looking at one of the abstracts of two of the papers you refer to, the following is stated:

    'Together with other evidence, these findings suggest that a set of general mathematical principles may describe the relations between positive affect and human flourishing.'

    Sounds like a pretty general statement about humans in general to me)

    2) actually investigate different populations.

    (to deal with the "psychology is the study of psychology students by psychology students", i.c. use other participants than psych-students).

    3) use a more diverse set of measurement-instruments/methods to study a specific topic.

    4) incorporate 1), 2) and 3) in a more systematic examination, and subsequent publication, of a certain phenomenon.

    (I have always wondered if there are psychologists who more systematically investigate phenomena, making all subsequent data available [i.c. no file-drawer], and not hop from [in some vague way related-] topic to topic, and from collaboration to collaboration, and from publication to publication. Why not take a certain finding [probably investigated using psych-students] and first try and replicate it with a different set of students. Then do that again. When you have converging evidence from those 3 studies, try using a student population in a different country. Then try it using a non-student population. Then investigate how a different method/ measurement [which supposedly would be measuring the same thing] relates to these findings. Make sure all this information is stored in 1 place, and all non-significant findings are also present there, and make sure you have adequate power, etc.)

    ReplyDelete
    Replies
    1. That you care about these issues places you in good company with other psychologists. Thanks for reading!

      Delete
  2. Where it says "Fraud effects the sciences", it should say "Fraud affects the sciences"

    ReplyDelete
    Replies
    1. Whew... thanks for catching that. I bet it was bothering everyone.

      Delete
    2. This comment has been removed by the author.

      Delete
  3. like a nail scraping a blackboard!

    ReplyDelete
  4. Fraud effects the sciences, from hard to soft.

    Granted that there is some fraud in science, but to say that science is brought about by fraud seems a rather overreaching statement. How typical of a psychologist.

    (Ok, I couldn't think of anything wittier to say about the affect/effect typo.)

    But in general, the main complaint against the current state of psychology as a science is that psychologists overstate their conclusions based on rather poor data (small, unrepresentative samples ("W.E.I.R.D." students)) and little-replicated experiments. Until they stop being so sure of everything without sufficient basis for being sure, it is wise for the world at large to avoid taking psychology's conclusions seriously yet.

    ReplyDelete
    Replies
    1. I think everyone cares a lot about fraud.

      Overstating conclusions, reporting data based on unrepresentative samples, and failures of replication are all concerns for psychologists--and many are currently taking in their own research to avoid these pitfalls.

      Delete
  5. The reason why I started reading this post was because I was interested in the "The Psychology of the "Psychology Isn't a Science" Argument", which, to my disappointment, is only briefly explored in the end of the text.

    I think another reason why hard science scientists like to dismiss Psychology as a science is not only related to the methodology or because Psychology wouldn't fit the necessary requirements to be a science. I think most scientists (and a lot of non-scientists) don't like the idea of considering Psychology a science because of the very nature of Psychology, particularly Social Psychology. Those people are not comfortable with other people telling them and analyzing how they are, who they are and how they behave. But they cannot say this in scientific words, so they use the scientific parameters to criticize Psychology.

    ReplyDelete
    Replies
    1. I'm not sure if this is true or not but it is an interesting point you raise. I might go a bit further and say that we are all, at least in part, experts in psychology because we have human experience. This makes us more confident we know a lot about psychology (even non psychologists) and so, how can something that everyone knows a bit about be a science?

      Thanks for reading!

      Delete
  6. Effective and helpful rejoinder. Thanks for contributing to the discussion.

    ReplyDelete
  7. Dr. Drew is not a psychologist... he's a medical doctor.

    ReplyDelete
  8. I was disappointed with your description of the fourth point: psychology isn't falsifiable. Non-falsifiability doesn't refer to publication bias. It suggests that many psychological theories don't even rise to the level of theories because they don't come with experiments whose results could prove them to be wrong (a la K Popper http://keck.ucsf.edu/~craig/Karl_Popper_Science_Conjectures_and_Refutations.pdf). This is the much stronger critique and the one we should be most concerned with in psychology.

    ReplyDelete
  9. Hypotheses in psychology are falsifiable--variable A is hypothesized to relate to variable B, and it either does or it doesn't.

    I'm not sure what you're talking about.

    ReplyDelete
  10. [sorry if this double-posts … i think i may have lost the 1st upload]

    Well firstly, I'm talking about the fact that non-falsifiability doesn't equal publication bias. This is simply the case. Therefore if you title a section 'IV. Psychology isn't falsifiable', that section should address areas in psychology where theories don't make falsifiable predictions, and not the orthogonal problem of publication bias.

    Secondly, I agree that many hypotheses in psychology are falsifiable. For example a theory working memory may predict a priming effect, which can be tested via measuring reaction times and accuracies on a memory task. However, these are never the type of hypotheses brought up when someone writes an article about how hard or soft the science of psychology is. Rather, theories of things like 'happiness' are discussed. No matter how much you may argue that smart people have thought about happiness for a long time and hit upon a way to operationalize its definition that transcends problems of cultural relativism, it will always be difficult to convince lay people (and scientists) that it is as 'hard' a theory as one which relates mass to force in a straightforward way. The operational definition of things like mass and force are _widely_ agreed upon. The definition of things like happiness are widely debated. Therefore you end up with hypotheses that relate variable A to variable B where the definitions and measurements of the variables of even the existence of the measurable phenomenon are 'fuzzier' and interpreting the results just seems fuzzy to lay people and other scientists alike.

    It's easy to see how F = ma is falsifiable. It's not easy to see how a 'positivity-ratio' is falsifiable or even what it really means.

    ReplyDelete
    Replies
    1. That's a correct statement-- falsifiability is not publication bias. We agree.

      The second point is about defining terminology and not falsifiability though. And on that point I mentioned that smart people didn't just think up a way to measure happiness, they MEASURED it for decades to try to figure out how they could reliably quantify something that seemed so unquantifiable. What they came to, after decades of research and hundreds of thousands of participants is evidence about what happiness is related to (and not related to). It's rigorous, it's falsifiable, it's replicated, it's science.

      On the F = ma example being falsifiable--I don't have any doubts about that, but I am a lay person when it comes to physics so I don't have the slightest idea how one tests the hypothesis that F = ma. I expect that many others don't either. But just because I, the physics idiot, or you, the anonymous blog comment-maker, don't understand something doesn't mean it isn't science.

      Delete
  11. I may have conflated these issues a bit but I think that's because they're intimately related in psychology. When you say that smart people didn't just think up a way to measure happiness, but rather measured it for decades ... this is a little backwards. The whole notion that happiness is a measurable phenomenon is the crux of the issue. And this issue exists with many psychological phenomena.

    It may very well be rigorous, but if you take something like Ed Deiner's Satisfaction with Life Scale (just as a for instance, there are plenty of other examples (don't mean to pick on happiness research but we were already on it)) I might believe it's a good measure of this ephemeral thing called 'happiness' and you might not believe that. It's certainly not the case that all psychologists agree on this or that it's not a matter of debate. This is in stark contrast to something like 'mass', or 'force', or 'heat'. No contemporary physicists will disagree on how to measure these things.

    Every second a GPS device works, it amounts to thousands of rigorous tests of (very falsifiable) general relativity. There is no analogue anywhere of any such psychological technology tested to such an extant. It's not a matter of not understanding it ... definitions and theories are just much fuzzier in psychology than they are in 'harder' sciences. Pretending that it's just as rigorous is just disingenuous.

    I should say, as a kind of 'systems-level' cognitive scientist, I'm on the side of psychology being a science. I just get frustrated when it doesn't admit it's own limitations and short comings. Since I use fMRI, I'm acutely aware of the limitations of the Blood Oxygen Level Dependent signal we measure. Many electrophysiologists won't consider anything an imager ever produces to be of any interest at all because they simply don't believe BOLD is a good measurement for neural activity. For me, it's not the best measurement in the world, but if I keep in mind it's strengths and limitations, I can make conservative inferences within the scope of it's measuring ability. But I have to be extra careful not to overstep, because smart, reasonable psychologists, electro physiologists, philosophers of science ... etc all have legitimate points to make about the short comings of BOLD. I try to keep those short comings in mind all the time and I feel like whenever this debate about psychology being a science comes up, psychologists act so defensive that it comes off as though they think every survey and subjective test they give is just as good as any other measurement in harder sciences and that drives me nuts.

    ReplyDelete
    Replies
    1. I'm glad we agree that psychology is a science!

      About your comment on consensus in physics: Surely physicists have consensus about measuring things like force, but I bet there is some really theoretical stuff in physics that scholars disagree about. I wager that doesn't make physics less SCIENTIFIC. Why the double standard?

      I do think people agree on how to measure happiness, but let's take emotion for example. Researchers disagree all the time about what an emotion is. But we can still define emotion, measure it in any number of ways (psychophysiological changes, brain imaging, self-reports, facial expressions), test falsifiable predictions, and then replicate those findings. Again, that's science.

      And emotion researchers don't sweep the limitations under the rug about the way they measure emotion. Instead, they have open and sometimes hostile debates about what an emotion is and is not. So, your other point about how psychologists should be forthright about measurement limitations is a good one--and places you in good company with other psychologists.

      Delete
    2. I do indeed agree that Psychology is a science. However, I don't think my original standard is a double standard. It's not just that physicists may disagree about really theoretical stuff. There is _nothing_ in psychology that comes anywhere close to as reproducible as almost any part of physics. Saying otherwise is really just silly. And to me that's why psychology really can be considered a 'softer' science. I can't think of a single metric with anywhere near the predictive power, as almost any in physics. If anyone has an example of one I'd love to hear it. I think the closest thing I can think of is Shepard's similarity spaces, which is why I think psych science was so pumped at the time ... it looked 'lawlike' in the way that principles in physics are lawlike. (http://cseweb.ucsd.edu/~gary/PAPER-SUGGESTIONS/shepard-science-87.pdf) but ultimately proved dependent on a lot of contextual stuff.

      When Feynman talk about quantum making predictions that were analogous to 'measuring the width of the united states from coast to coast and getting it right to within a grain of sand' he wasn't being hyperbolic. There is nothing in psychology which is like that. And to me, that's why psychological inferences must be taken with a bigger 'grain' of salt, so to speak. Which isn't to say that it's not zeroing in on some 'truths' about psychological phenomena. Hopefully it is. It's just much slower and it may be the case that there really aren't that many psychological phenomena that are really _as_ law-like as physical principles. (for me, the upshot of this deficit is that there's a lot of fun stuff to pontificate about in psych … in a slightly looser way maybe)

      Delete