Why I'll Keep Teaching the Stanford Prison Experiment (Even Though It's A LIE!!1!1!!1!)

So there's a new article making the rounds of academic social media (and maybe normal people social media, who knows?) about how the infamous Stanford Prison Experiment (SPE) was bunk. Here's what I believe is the original piece that sums it up as "a sham" while this one calls it "a fraud."  But I would submit it's much less the case that this very famous experiment is a fraud and a lie that academics have been uncritically accepting for decades, and more the case that, as always, science reporting sucks pretty bad.

As far as I can tell, most of the reports suddenly appearing about this decades-old experiment are drawing from the piece linked above to the article written by Ben Blum, who notably has a book he's trying to sell. This is really important, because you don't drum up book sales with nuance, you drum up book sales with splashy headlines about how a famous old study is a FRAUD full of LIES.

The interesting thing about this is that it's almost the inverse of how science reporting typically sucks. How it usually goes is that a study (like all studies!) finds an interesting, but limited, effect of X on Y in a very limited setting, and then reporters blow it out of proportion with headlines like "New study proves X causes Y!" without noting any of the many, many limitations of the study. This one just does the mirror inverse, where it acts as if the SPE was canonical scientific gospel which has been DESTROYED and SHREDDED by new evidence. Like all shitty science reporting, if you actually pay attention to it, there's nothing much here (famous old study has plenty of issues, all of which are well known to pretty much everyone), but it's trumpeted as EXPOSING A LIE!

Really, there's so much wrong with Blum's piece that it truly deserves an FJM-style line-by-line takedown, but ain't nobody got time for that. And yet a lot of otherwise intelligent people are falling for this shoddy writing, so I do feel compelled to hit on some of the major issues with Blum's style of...reporting? Yeah, let's call it reporting.

One big problem is that Blum's critique reeks of anti-intellectual posturing. The whole thing has the distinct tone of "Oh, these nerds think they're so smart, but look at how they were all duped by this fraudulent study!" But...that's just not the case at all. Sure, maybe in the immediate period after study was released academics might have accepted it rather uncritically, but no one currently thinks of it as solid research. I mean, when I first learned of it in undergrad...checks calendar, lets out long defeated sigh...nearly 20 years ago, it was already the go-to example of unethical and shoddy science. Hell, even in Blum's article he has to note "methodological criticism of it was swift and widespread in the years after it was conducted." Swift and widespread. SWIFT. AND WIDESPREAD. That does not sound like uncritical acceptance! Of course, he brushes past that point really quickly, probably because it undermines the entirety of his argument. Even more annoyingly, when he (or any of the other articles I've seen written about this) actually talks to an academic about the SPE, they all pretty much uniformly say something along the lines of "Oh yeah, that study had tons of problems, but it's useful for illustrating some certain points." So again, there's literally no evidence academics are simply uncritically accepting this study, and yet that's the hook of every one of these articles.

This is, of course, to say nothing about the fact that it's one single study. Something people outside of the sciences and/or academia have a hard time grasping is that it's incredibly rare that a single study really amounts to anything. Because no study is perfect, there are always more variables to include, more ideas to consider, etc. Science works by collecting a large number of studies. So even if people were uncritically accepting this as true (which they aren't), it's being disproven (which it isn't) wouldn't really matter, because there's been tons and tons of other research conducted on prisons, the prison environment, the power of authority, etc., many of which come to the generally the same conclusions. Again, one study in isolation is typically pretty useless. That's why we have lots of people studying things lots of different ways. I mean...that's just what science is.

Another major problem is that neither Blum nor any of the other people writing about this at all address the concept of what we call in our fancy-pants social science language "desirability bias." Which is pretty much what it sounds like -- all of us want to present ourselves as good people, consciously or unconsciously. That's why we have all sorts of checks and measures built into survey and interview research, because it's rare that you can just straight-up believe what people say. A classic example of this from political science is asking people who they voted for in prior presidential elections. When you do this, even if your sample is very carefully calibrated to be representative of the US population, you will get significantly more people saying they voted for the winner than actually happened. For some this is a conscious manipulation of the truth (no one likes to be lumped in with the loser, or to have voted "wrong") while for others its subconscious (they don't really remember who they voted for, so their mind fills in the blank with the more memorable winner). But the point is that it doesn't matter at what level it's happening, just that its empirical reality that most people will try to present themselves in what they think is the best light.

How this applies here is that most of Blum's damning expose is based on the fact that several of the principal research participants now say they knew what was happening all along and that they were just playing the roles they thought they were supposed to play. And maybe this is indeed true! But there's zero reason to just accept these guys' word on that. Because you could just as easily argue that if you're someone who's famous for, say, being a sadistic asshole during this world-famous experiment, or freaking out and having a massive panic attack, you have pretty good reason to later say "Oh no, no. None of that was real. I was totally just acting the entire time! I definitely knew what was going on and wasn't tricked in the slightest!" I mean, I sure as hell would. Really, there's no way to actually know, and my point is not that these guys are lying. Rather, the point is that it's just as likely they're inventing a new story to excuse their behavior as it is they had this all figured out from the get-go.

But yet, even if most of the people involved were consciously acting out a part (even though it's obvious not all of them were, such as the fellow who staged a hunger strike), this still stands as a damning condemnation of our prison system. After all, is it not telling that a group of college students told to be prison guards assumed that meant they needed to be abusive assholes?

Furthermore, the training of the guards in this study was not terribly different than the training actual prison guards receive. If you're interested in a great detailed account of prison guard training, I highly recommend Ted Connover's Newjack: Guarding Sing Sing.

For instance, check out this supposedly damning quote from Zimbardo:

"“We cannot physically abuse or torture them,” Zimbardo told them, in recordings first released a decade and a half after the experiment. “We can create boredom. We can create a sense of frustration. We can create fear in them, to some degree… We have total power in the situation. They have none.”"

Yup, that's pretty much what actual prison guards are taught -- what they legally can and cannot do, and then from there are basically told to make it work however they can. Indeed, here's another passage that is somehow supposed to prove the SPE was a fraud:

"In 2005, Carlo Prescott, the San Quentin parolee who consulted on the experiment’s design, published an Op-Ed in The Stanford Daily entitled “The Lie of the Stanford Prison Experiment,” revealing that many of the guards’ techniques for tormenting prisoners had been taken from his own experience at San Quentin rather than having been invented by the participants."

So sure, this proves Zimbardo oversold his experiment (which, again, is a point so thoroughly established and accepted that it's beyond banal). But it really undermines the whole "SPE was nothing like actual prisons!!!!" argument when you note that the guards' techniques were taken directly from actual prison experience.

Here's another passage that, again, while being marshaled as evidence the SPE was bunk, actually makes the very point the study was trying to make:

"Once the simulation got underway, Jaffe explicitly corrected guards who weren’t acting tough enough, fostering exactly the pathological behavior that Zimbardo would later claim had arisen organically."

Again, read the Connover, book -- this is what happens in actual prisons. Any guard who is being too friendly with the inmates or not enforcing tough rules will have superiors and/or coworkers set them straight right quick. So while again Zimbardo comes off as an ass, it actually reinforces the claims of the study.

But what's probably most frustrating about this poorly-formulated take down is how this dude clearly knows little else about prisons and has obviously read little-to-no social science, in general or about prisons specifically. For instance, take a look at these two quotes:

"According to a 2017 survey conducted by Cullen and his colleagues Teresa Kulig and Travis Pratt, 95% of the many criminology papers that have cited the Stanford prison experiment over the years have accepted its basic message that prisons are inherently inhumane."
"The SPE is often used to teach the lesson that our behavior is profoundly affected by the social roles and situations in which we find ourselves."

These things are both true! Seriously, spend 10 minutes perusing the literature on American prisons and I'll be pretty shocked if you don't come away with the notion that they're inhumane. And the idea that "our behavior is profoundly affected by the social roles and situations in which we find ourselves"? That's more-or-less the basic starting point of all social sciences. Even if the SPE was completely made up and completely bullshit, these things would still be true. In this sense, it's a lot like finding problems with one study of climate change and using that to declare that climate scientists are all dopes who have been duped into believing bullshit.

The reason the Stanford Prison Experiment sticks around in textbooks and lectures is because it's an interesting example with a lot of media produced around it, making it accessible in both the figurative sense of being easy to grasp and the literal sense of having all sorts of videos and interviews and whatnot available. It's in many ways the same as how we teach the scientific method to kids in elementary school. I very distinctly remember learning that 6-stage process of science in the 4th grade. And now, as an actual research scientist, I can tell you it's complete bullshit. No scientific study in the history of scientific studies has ever followed that 6-stage process. But that doesn't mean we're teaching our children lies, it's just a simplified version of a much more complex and nuanced process. The SPE is roughly the same -- I have a hard time believing anyone is teaching it as an example of great science, but rather it's handy for discussing all sorts of methodological and ethical issues, as well as serving as general entry point to studies of the prison as well.

Again, Blum's piece undercuts itself quite directly with this quote, which I think more-or-less reflects how the majority of academics feel about the SPE:

“Even if the science was quirky,” said Kenneth Carter, professor of psychology at Emory University and co-author of the textbook Learn Psychology, “or there was something that was wrong about the way that it was put together, I think at the end of the day, I still want students to be mindful that they may find themselves in powerful situations that could override how they might behave as an individual. That’s the story that’s bigger than the science.”

And that's really the entirely of my problem with Blum's "expose." He writes as if he's blowing the lid off of a conspiracy by pointing out the problems with the SPE, when in reality, academics have been discussing these issues with the study for literal decades.

So really, what we learn from all of this is that A) the SPE has all sorts of problems that have been widely recognized from basically the day it was published, B) Philip Zimbardo is a bit of a publicity hound, C) most of the conclusions/arguments of the SPE have been confirmed by subsequent, much better-designed research, and D) none of this is news to anyone who pays any attention to this stuff.

But those conclusions are not nearly as catchy as calling something a sham, and they sure as hell won't help you sell your book.