Three Guys Talking About Scales

What follows below is the result of an online discussion I had with psychologists Brent Roberts (BR) and Michael Frank (MF). We discussed scale construction, and particularly, whether items with two response options (i.e., Yes v. No) are good or bad for the reliability and validity of the scale. The answers we came to surprised me--and they might surprise you too!

MK: Twitter recently rolled out a polling feature that allows its users to ask and answer questions of each other. The poll feature allows polling with two possible response options (e.g., Is it Fall? Yes/No). Armed with snark and some basic training in psychometrics and scale construction, I thought it would be fun to pose the following as my first poll:

Said training suggests that, all things being equal, some people are more “Yes” or more “No” than others, so having response options that include more variety will capture more of the real variance in participant responses. To put that into an example, if I ask you if you agree with the statement: “I have high self-esteem.” A yes/no two-item response won’t capture all the true variance in people’s responses that might be otherwise captured by six items ranging from strongly disagree to strongly agree. MF/BR, is that how you would characterize your own understanding of psychometrics?

Thought Fragments Concerning Ideology in Social Science

I took a course in sociology my first year as an undergraduate at UC Berkeley. The course was an introduction to sociology taught by professor and social activist, Harry Edwards. The course blew me away because it felt so viscerally real. Professor Edwards would talk about social class, race, and gender in America and students would chime in about their own experiences that brought these big social constructs to life. What I learned in Professor Edwards’ class resembled nothing we had discussed in my high school history classes—I grew up in a politically conservative suburb in San Diego, and we didn’t have much ideological diversity in our discussions of law and society. Sociology, and social sciences more broadly, really spoke to me.

Teaching Undergrads vs. MBAs: Four Observations

Hello and sorry I've been away from blogging for so long! I ended up switching departments and jobs--now I work at Yale University at the School of Management. As you might imagine, a lot of things have changed as a result of the move. What I'd like to do today is to briefly summarize what stuck out to me as the main differences between teaching undergraduate psychology majors and first year MBAs.

A note of caution before we dive in: I've only spent about 27 hours teaching MBAs and three years teaching psychology undergraduates, so it's possible that I know little to nothing about teaching BOTH groups. Also, the undergraduates and MBAs experienced different courses and come from different universities, so the differences I observed might not reflect MBA/undergrad distinctions. What is reported here is simply one person's observations from a relatively short time period.

Gender Imbalance in Discussions of Best Research Practices

Over the last couple of weeks there have been some really excellent blog posts about gender representation in discussions of best research practices. The first was a shared Email correspondence between Simine Vazire and Lee Jussim. The second was a report of gender imbalance in discussions of best research practices by Alison Ledgerwood, Elizabeth Haines, and Kate Ratliff. Before then (May 2014), Sanjay Srivastava wrote about a probable diversity problem in the best practices debate. Go read these posts! I'll be here when you return.

SPSP 2015: Actually Predicting the Future

In regression (a common statistical practice used in social science research) we often attempt to predict the outcome of a given dependent measure (the DV) based on what we know about other measured variables theoretically related to the DV (the IVs). This common regression method has one problem though: We are predicting values for data that we have already collected. What if we were to engage in actual prediction? That is, what if we attempted to predict the values of a DV that is unknown? How might we do this and what would be the benefit?

This was a fascinating talk presented by Liz Page-Gould of the University of Toronto at the Future of Social Psychology Symposium!

SPSP 2015: Status Shapes Preferences for Redistribution

A lot of people think about political ideology as a powerful causal force that influences the structure of our society and our respective positions within it. In the politics and inequality symposium Jazmin Brown-Iannuzzi of the University of North Carolina, Chapel Hill and her colleagues examined political ideology from a different perspective: Instead of shaping the structure of society, does political ideology arise from our position within that structure? That is, do we create our political beliefs strategically in order to justify our own lives and the lives of those around us?

SPSP 2015: The world thinks that atheists are immoral

At the self and identity pre-conference this morning Will Gervais presented a series of studies (available open access right here) suggesting that people seem to automatically associate atheism with a lack of moral character.

SPSP 2015: The Contagious Spread of Affect

I was fortunate to attend the first session of the emotion pre-conference this morning and had a chance to hear about some amazing research conducted by Wendy Mendes (my post-doc advisor), Sara Waters, and Tessa West. The research examined the extent that affective states are transmitted between individuals.

The researchers reasoned that social living organisms are widely characterized by synchronous actions and states--with humans groups being particularly likely to synchronize their emotions and behaviors (here is a profound example of our capacity to synchronize during communal events).

SPSP 2015: We're here in Long Beach!

It's been a few months since I've posted on this blog, and I'm planning to change that over the next several days: It is time for the annual convention of the Society for Personality and Social Psychology and we've got three full days packed with social-personality content to share with the blogosphere. I'll be writing a few short posts over the course of the conference--but before I do I want to point out where you can find some of the current and former bloggers on this website, right here at the convention:

Thursday
Katherine Thorson: Poster Session A
Cardiac Vagal Tone predicts Responsivity to Self-Regulatory Thought Inductions: Poster A273

Friday
Amie Gordon: Symposium Session A
Bringing Sleep to Social Psychology: Considering the Effect of Sleep on our Emotions, Relationships and Intergroup Relations
Friday, February 27, 2015, 9:45 AM - 11:00 AM, Promenade Ballroom 104C

Michael Kraus: Symposium Session A
The Politics of Inequality and the Inequality of Politics
Friday, February 27, 2015, 9:45 AM - 11:00 AM, Room 203ABC

Saturday
Jennifer Stellar: Symposium Session I
A Big Helping of Humble Pie: Novel Benefits and New Methods for Cultivating Humility
Saturday, February 28, 2015, 5:00 PM - 6:15 PM, Room 103A

Maya Kuehn: Poster Session E
Rejection Resiliency and Social Influence: Poster E004 

Everyone Gets a Job!

A terrifying graph for any PhD student! (source)

It's late October and that means we are squarely in the middle of job season for psychology PhDs (and PhD candidates). I was hired during the 2011-2012 job cycle, and so I recently switched to the evaluation side of the job process. Sitting on this side of the fence I feel incredibly fortunate to have a job: There are a ton of accomplished graduate students and postdocs with strong records, interesting research ideas, and stellar (!!!) letters of recommendation. If the system were running optimally, most of these applicants would land jobs. If the system were running optimally...

Crossing Class Boundaries

Yesterday the New York Times published an opinion piece written by University of Toronto, Rotman School of Management Professor, Stephane Cote and I on the challenges of crossing social class boundaries. You can find the article here. This blog post accompanies that article with a few notes about the research.

Big Theory

"Essentially, all models are wrong, but some are useful." --George Box*

I need to be honest with you, I'm not all that good at generating novel ideas: Some of my most well-cited papers involve theories that sociologists came up with decades ago; Reviewers frequently accuse me of running post-hoc analyses (asking the data for ideas, rather than generating apriori predictions); When media cover my research, the most common initial comment is something like: "This is so obvious....blah, blah....you suck." You get the idea. 

I don't view this particular characteristic of my research as a flaw. Rather, I'm acknowledging that not all scientists can be ground-breaking theorists/game changers: Some people come up with great ideas and some people test them. For the most part, I test theories and I do it in (what I hope are) convincing ways. Given this characteristic of my research, you might be surprised then, to learn that I love theory!** 

You read that right.

Psychological Barriers in Economic Inequality Reduction

Today I wrote a blog for New Left Project on psychology research examining perceptions of, and responses to, economic inequality. The post features cutting-edge research by prominent social psychologists Mike Norton and Dan Ariely, as well as research from my own laboratory at the University of Illinois. An excerpt:

"The United States is one of the most unequal and rigidly stratified societies in the industrialised world.  In the wake of the Great Recession, it has become increasingly clear that success in America flows to the wealthy and the well-connected.  Why do these inequities persist in the face of steady unemployment, abject poverty and rising homelessness? 

Psychology at the (Home) Movies: HBO’s The Wire

Source

Although I’m more than a decade late to the party, a recent fortunate Amazon.com prime membership has gifted me with access to HBO’s acclaimed series the Wire. For the last two months I’ve been watching the show weekly, digesting its contents in small consistent doses. My background as a middle class ivory tower academic makes the Wire foreign territory to me—I don’t have much personal experience with drug culture, or poverty, or oppression, or Baltimore (the primary city in the story) for that matter. Nevertheless, there were many themes in the TV series that align well with contemporary research in the social sciences. With an eye towards these themes, I bring you a look into HBO’s the Wire by linking it to our current understanding of basic psychology. As with any piece of film-making, the nuance and detail will not be completely captured in this post, and I welcome comments here or on twitter (@mwkraus). Also, SPOILERS!

Notes on Replication from an Un-Tenured Social Psychologist

Last week the special issue on replication at the Journal of Social Psychology arrived to an explosion of debate (read the entire issue here and read original author Simone Schnall's commentary on her experience with the project and Chris Fraley's subsequent examination of ceiling effects). The debate has been happening everywhere--on blogs, on twitter, on Facebook, and in the halls of your psychology department (hopefully).

(Sample) Size Matters

Sample Size Matters

On this blog and others, on twitter (@mwkraus), at conferences, and in the halls of the psychology building at the University of Illinois, I have engaged in a wealth of important discussions about improving research methods in social-personality psychology. Many prominent psychologists have offered several helpful suggestions in this regard (here, here, here, and here).

Among the many suggestions for building a better psychological science, perhaps the simplest and most parsimonious way to improve research methods is to increase sample sizes for all study designs: By increasing sample size researchers can detect smaller real effects and can more accurately measure large effects. There are many trade-offs in choosing appropriate research methods, but sample size, at least for a researcher like me who deals in relatively inexpensive data collection tools, is in many ways the most cost effective way to improve one's science. In essence, I can continue to design the studies I have been designing and ask the same research questions I have been asking (i.e., business-as-usual) with the one exception that each study I run has a larger N than it would have if I were not thinking (more) intelligently about statistical power.

How has my lab been fairing with respect to this goal of collecting large samples? See for yourself:

The Psychology of Economic Inequality: A Collection

Today I wrote an op-ed piece for livescience about economic inequality. Read the piece here. In it, I argue that though economic inequality is a complex social and political issue, it may be explained, at least in part, in terms of the basic psychological motivations of individuals.

Anyway, I hope you'll check out the piece and send me comments via twitter (@mwkraus). If you'd like to read more about this area of research, below I have collected four past PYM pieces on the topic.

I'm Using the New Statistics

Do you remember your elementary school science project? Mine was about ant poison. I mixed borax with sugar and put that mixture outside our house during the summer in a carefully crafted/aesthetically pleasing "ant motel." My prediction, I think, was that we would kill ants just like in the conventional ant killing brands, but we'd do so in an aesthetically pleasing way. In retrospect, not sure I was cut out for science back then.

Anyway, from what I remember about that process, there was a clear study design and articulation of a hypothesis--a prediction about what I expected to happen in the experiment. Years later, I would learn more about hypothesis testing in undergraduate and graduate statistical courses on my way to a social psychology PhD. For that degree, Null Hypothesis Significance Testing (NHST) would be my go-to method of inferential statistics.

In NHST, I have come to an unhealthy worship of p-values--the statistic expressing the probability of the data showing the observed relationship between variables X and Y, if the null hypothesis (of no relationship) were true. If p < .05 rejoice! If p < .10 claim emerging trends/marginal significance and be cautiously optimistic. If p > .10 find another profession. By NHST standards, an experiment fails or succeeds based solely on this one statistic.

When the Association of Psychological Science proposed using an alternative statistical approach--something called the New Statistics (actually not new, been around for decades)--I was intrigued about the possibility of living an academic life beyond the tyranny of p < .05.

PYM's Graduate Student Guide-Blog

In thinking about the past three years writing for PYM, I just realized that I write a lot of posts about issues that graduate students care about. I've made a list with links to each of these posts below. Now all the "wisdom" I have to offer about graduate school is in one place. I hope this will help you--current and future graduate students in psychology--to navigate the challenges and opportunities that many of us face on our way to a PhD! Good luck in your journey and don't be afraid to leave comments or questions on the post or on twitter (@mwkraus or @psychyourmind).

Pro Tip: Treat Graduate School Like a Job

source

I really enjoyed graduate school. I attended UC Berkeley's social-personality psychology PhD program from 2004-2010 and the whole time I felt like a kid in a candy store. Sometimes my typical day included reading a book while lounging outside in the sun on Berkeley's memorial glade. Other days I would spend several hours in a local coffee shop (the ORIGINAL PEET'S!!!!) writing a manuscript describing a study providing knowledge that no-one before me had produced. On still other days I would play pick-up basketball with a hilarious combination of former college basketball players and current Berkeley psychology professors. It's a time in my life that I will always look back on fondly.

Of course I did also get a little bit of work done while I was in graduate school. I collected data obsessively, I wrote for 20-30 hours each week, I coded nonverbal behavior four hours a day for two