Click each image to enlarge.

0:00:00
Ben Radford: Welcome to Squaring the Strange, the podcast that examines all manner of the mysterious and the mundane through a critical lens. So let’s square the strange.

(MUSIC)

BR: Welcome back to Squaring the Strange. I’m your co-host, Ben Radford. Along with me is Pascual Romero and Celestia Ward.

Celestia Ward: Howdy.

Pascual Romero: Hey!

BR: We are on episode 60, the big six zero, and we’re talking this week about pseudo experiments, the topic that the three of us have talked about for a while. I’ve always been fascinated by it because it’s in science and skepticism, experiments are of course one of the core aspects. That’s how we find out about the world. That’s how we determine whether our perceptions and our interpretations are valid as we compare them to what does science say, what does the experiment say, what does the test say? And of course, experiments are in many ways the bedrock of scientific progress, but oftentimes we have pseudo experiments, and that is experiments that seem plausible but actually aren’t and in some cases can do some damage. That’s this week’s topic.

CW: I’d imagine you run into quite a bit of that in the paranormal research field, people doing really poorly designed pseudo experiments to prove a particular ghost idea. And so that kind of gives you an added bonus outlook when you find these pseudo experiments in other walks of life.

BR: Yeah, absolutely. I mean certainly you see pseudo experiments in Bigfoot research, in ghost research, oftentimes in parapsychology research all over the place. So it’s an important aspect. But before we do that, why don’t we see what anyone is skeptical of this week.

(MUSIC)

PR: Well, this week I’m skeptical of being a parent of a five-year-old. It’s been a little challenging. He brings home a cold probably once a month and so I’m fighting one today. So you might not hear too much of me today. But Ben, we were briefly chatting about a certain documentary piece that we still have not heard a follow-up on. What is it that you’re skeptical of?

BR: Funny you should mention that, Pascual. As our listeners will recall, there was a documentary about alleged new evidence about Amelia Earhart and her fate. In particular, there was a photograph that was widely shared. Mystery saw, blah blah blah, and at the core of it was a photograph that was allegedly found that depicted Earhart and her co-pilot along with what may or may not have been a blob or actually it was a blob it may not have been her plane and this was a much hyped documentary TV show and within I think it was a couple days if not a couple hours it was debunked there was lots of discussion in among the skeptical groups around the world and in the end was a Japanese researcher who I think used Google for like half hour or something and found that photograph that actually was from the wrong time. Anyway, there’s all sorts of problems with it. The upshot is that after this whole thing blew up in their face, the History Channel issued a statement after being caught egg-faced and red-faced by people saying, Hey, hold on here. You have this big crack team of investigators that somehow missed this obvious thing that apparently don’t have access to the internet which is odd so they issued a statement saying I don’t have a statement in front of me basically saying well you know we were looking into the new information and we have people on it and we’ll be updating you with new information as soon as possible that’s now been I think six months to seven months been it’s been a long time and we’re still waiting.

CW: Yeah I think it was actually longer than that. And I believe they promised complete transparency.

BR: Yes, that was the phrase that you.

CW: Yeah, if anything, what we’re skeptical of this week is that the History Channel will ever come clean or follow up with something that they were embarrassed by.

0:04:39
BR: Yeah, so that’s what I’m skeptical of. And, you know, I think one of the important roles that a skeptic can have is to follow up on original claims. A lot of times some wild weird claim will make a splash and go viral on social media and make headlines and there’s no follow-up because the world changes, things move on, new crises come up. And I think that there are a huge number of wild unsubstantiated claims that when they first appear, they’re given credence and wow, here’s this new development. And the later sober rejoinder that turns out, well, apparently this wasn’t true, is often ignored. And so that’s one of the things that I’m going to try to do with this podcast in general is to remind people of previous false claims and demand accountability where we can.

CW: Yeah, well, we’re fighting an uphill battle there, not just in terms of how television works and how the executives really don’t care to retread all the times their research failed them. But psychologically, we’re all we’re also primed to remember the first time we hear about something. And even if the first time we hear about it is a completely unsubstantiated rumor or a myth, we tend to lock onto that and pull it up later as a truth as a true fact.

0:06:00
BR: Yeah, I think I could be wrong, but I think that’s what in psychology is called the anchoring bias. The first thing that we hear about something tends to sort of anchor our beliefs because that’s new information. So all future information on that topic is linked to that original claim. So if the first thing we hear about Amelia Earhart in a while is her remains being found or this new photograph, then later information will sort of have to go through that prism instead of sort of being evaluated on its own.

0:06:30
CW: Are you sure that was called the anchoring bias or did you just hear about it first as the anchoring bias and it was corrected later?

BR: Oh, good one. I also want to mention one other quick thing that our friend of the show, Kenny Biddle, did a fine recent investigation published on the Skeptical Inquirer magazine website. He did a really thorough investigation and wrote an article titled Demon House Deconstructed. It was about a famous allegedly demon-infested house that made a bunch of news and Zach Baggins was among the people that allegedly researched it. I use researched very, very loosely. But anyway, he did a really good analysis of a basically a pseudo documentary about it. So if you’re interested in how to do good skeptical investigation, science-based investigation into these sorts of claims and basically sort of some media literacy on these things, check out Kenny’s piece, Demon House Deconstructed.

0:07:17
CW: Ooh, sounds interesting. And Zach Baggins has got a whole haunted museum here in Las Vegas. If you’re ever in town, I’ll have to force you to go through it.

BR: I can’t wait. I wonder if they’ll recognize me. I would guess that my pictures on the wall, you know how you go into 7-Eleven and it’s like, these guys passed bad checks and this guy shoplifted a quarter of Jack Daniels, so don’t let him in. It’s probably like, Radford’s here, don’t let him in.

0:07:50
CW: They’re hiring tour guides for minimum wage. I doubt they have the research funding to put anybody’s pictures up on the wall.

BR: That’s probably true. I’m sure if that can’t even investigate a haunted house, he probably doesn’t know who the hell I am.

(MUSIC)

PR: So this week we are talking about pseudo experiments, which is a very interesting sounding word. We see the prefix pseudo a lot in the skeptic world and it generally means that there’s a problem with it. Ben, can you tell everyone what a pseudo experiment is exactly?

0:08:23
BR: Sure. Well, by that I sort of mean an experiment that has a superficial plausibility about them, that appears to be scientific, but in fact isn’t. Of course, that’s, I mean, there’s different definitions and examples of pseudoscience, and we don’t need to, I mean, we could do a whole show on that. But in this particular case, it’s really more sort of experiments that are sort of pop-cultury, and they appear to make a particular point, often a sort of a social point, like, well, see, look at this. We did this, or everybody pay attention to this. We conducted this experiment, and look at the results. They speak for themselves. And oftentimes, that’s not the case at all. So I do want to distinguish it from flawed experiments. So we’re not talking about experiments that failed. We’re not talking about bad research design. So for example, in 2011, there was a physics experiment in Italy in which the physicists claimed to have found faster than light neutrinos, which as many people will probably recognize, would violate Einstein’s laws that nothing can go faster than light. And this made quite a bit of news. In fact, I was one of many people that wrote about it at the time. And well, it later turned out that they reran the experiments and nine months later, they realized that in fact there was faulty wiring in the equipment. And that accounted for the, I think it was like the 60 billionth of a second difference between the expected and observed effects. So for example, the cold fusion fiasco with Pons and Fleishman in 1989 and later on. So you know, there are many examples in science of flawed design where an experiment gave us the wrong answer that was later corrected through later research. That’s not what we’re talking about. We’re more talking about, well, one example we talked about on a previous show was Rocket Man. The guy that shot himself in the air to prove that the earth is flat.

CW: Mad Mike Hughes.

BR: That’s right.

CW: To distinguish from Rocket Man, Kim Jong-un.

BR: Right, right.

0:10:53
CW: Rocket Man Mike Mad Mike Hughes. Mad Mike.

BR: So, again, I mean, I’m sure Mad Mike is an interesting guy. I might or might not want to have a beer with him, but he’s no scientist.

CW: So that’s what we’re talking about. So you’re saying the difference is not – you know, the difference is a pseudo-experiment is almost set up to mislead you with data that doesn’t really mean anything, but is foisted upon the public to mean something. Whereas a flawed experiment could tell you something, but there was a weasel in the works and something went wrong.

0:11:27
BR: Yeah, and another, you know, that’s a good way to put it. Another aspect is that typically actual real experiments are performed by scientists, by researchers, by people with letters after their name. Not always, but oftentimes at the case.

PR: Go figure.

BR: They’re published in peer-reviewed journals, whereas pseudo-experiments are typically performed by advertising executives or activists or flat-earthers.

0:12:04
CW: Well, we don’t want to get into the ivory tower here and say, well, nobody can perform an accurate experiment unless you have letters after your name. But it is important to note, as they say, follow the money, follow the biases of the person doing it. If you’re being said something by a marketing company and they’re calling it an experiment but it’s not coming from a peer-reviewed journal, it’s in a 30 second ad on the Super Bowl, be skeptical.

BR: Exactly, exactly. So, let me just talk briefly about several aspects that I think distinguish pseudo experiments. One of them, as I mentioned before, they have a sort of superficial visceral plausibility and a simple lesson. You know, like, oh, we did this experiment and hey, these are the results. They speak for themselves, you know. You don’t need to look at percentages, you don’t need to wade through references and data and research design and statistics. It’s all laid out there for you. You can see for yourself what’s going on.

CW: It’s like an Aesop’s fable of experiment.

0:13:11
BR: Yes, exactly. And oftentimes they have an obvious agenda. People don’t do them to find X is true, they do it to prove that X is true. And so it’s in that way, it’s sort of, in that way, it’s basically the opposite of a real experiment. I mean, in, you know, the way that science works is you don’t start with the conclusion and work backward. You go, you follow the evidence. And so instead of trying to prove that X is true, you’re trying to find out is X true. There’s also a file drawer effect. And this is where in science the tendency for negative studies to not be published. And this happens a lot in, for example, alternative medicine, where people will say, well, see, look at these published studies, you know, this shows that homeopathy works, or this shows that, you know, some alternative medicine works. And it’s like, okay, well, how many other experiments show that it didn’t work and that it just never got published? You know, there will be some experiments that just by random chance will have a meaningful result, but that’s not really the question. So if you can point to a dozen studies that show that some alternative medicine works, but there are thousands that show that it doesn’t, then that’s a problem.

CW: I think, yeah, I’ve seen that termed publication bias and also just more simply cherry-picking.

BR: Yeah, exactly. So yeah, cherry-picking is a big part and you know keep in mind that again This is this is sort of classic pseudoscience because the people that are conducting these pseudo experiments They want the prestige of science they want… they recognize that the general public actually has an appreciation for science. And they’re like oh, this is cool. This is interesting look at this finding so they it’s like the ghost hunters you know who use their EMF detectors and their gadgets they they want to appear scientific. They want that bid for legitimacy, but they don’t want to put in the effort to actually making it meaningful or valid. They’re just like, hey, let’s play scientist. But typically it fails basic research design. There’s no randomization, no control groups, no blinding, no accounting for confounding variables. It’s set up to look like a valid experiment, but almost always it’s not.

0:15:23
CW: Yeah, it’s the shorter sexier wrong version of science. I remember I was actually a I was a biology major at first in college and I remember the kind just kind of the the drop in my expectations Once they got us into doing serious experiments and you’re sitting there pipetting a little bit of a solution for like 10 hours during the day It’s like wow, this is not exciting. This is not fast. This does not yield results immediately.

BR: It’s not simple. You actually have to be careful with these things.

0:15:52
CW: Yeah, so that’s not sexy enough for a 30-second ad or a two-minute Facebook post that goes viral. So unfortunately, we have pseudo experiments filling that gap.

BR: Now that we’ve talked about what they are, let’s give some concrete examples, hopefully not because of cherry picking in the process. Pascual, I think you have a couple examples. What’s on your plate?

PR: Yeah, I have a couple that really just frustrate me because they get, a big thing is they get shared on social media a lot. There are these man on the street interviews that you see, especially on like late night TV, and usually it’s done for comedic effect, but you will see that the video will still make a claim about what they’re doing. So for instance, Jimmy Kimmel always does these sort of like ambush interviews on the street in front of the theater where he films his show in Hollywood, and it’s always like he has someone asking people a really basic sounding question and they get it spectacularly wrong and it’s funny that they got it so wrong. I will blame, you know, the people who make these for kind of creating this version of these people, but I also have to blame social media for spreading it as a way of demonstrating that this is how people are. There was a video that Jimmy Kimmel did a while back during the 2016 election where he was asking people very basic things, or seemingly very basic things, like, you know, who is your senator? And people were just, you know, being stumped by this person asking these. Or sometimes people would make something up, or, you know, people would kind of bumble with like a few different names or a few different answers and then, you know, settle on the wrong one or something. You know, first off, we have to get out of the way of the fact that a producer and editor are deciding what you see. So you’re not seeing a representation of everybody being asked. Then you have to deal with the fact that this is ambush interviewing and I like to think of it like, I don’t know if this happens to you guys, or maybe this happens as you get older, but like sometimes someone will ask you how old you are and the older you get, the less often you get asked. And like, I forget how old I am, like on the spot when people ask me, like, how old are you? I’m like, it takes me like two seconds to like figure out how old I am. And, and, and it may not be that, but it could be any number of very basic questions that you should know the answer to, but it’s not readily available in your head, especially when you’re walking down the street and someone shoves a camera in your face.

0:18:37
CW: I have female privilege. Nobody ever asks me my age.

PR: That’s true. That’s true. But, yeah, so that’s always an issue. reality TV so I know a little bit about catching people in the moment and there’s an art to it. You find the most distracted person you can and then you play the prank on them. Now I never did these kind of man on the street interviews but we still did things where we kind of involved the public and you’d be able to catch people very off guard by just walking up to them with a camera because they just aren’t expecting to be on camera. And that’s sort of, you know, the biggest issue for me is that it’s just, it’s already off-putting and people aren’t going to give you the clearest answer. And there’s just the fact that these are questions that may or may not sound, may or may not be easy for you to answer when you’re like sitting at your desk with your computer in front of you, but not being able to name your senator, that might mean that you’re maybe not very civic, civically inclined, I guess, but I don’t think that that makes a person dumb. And I think the problem is that they infer that with these kinds of videos.

0:19:47
CW: The Jimmy Kimmel ones you’re mentioning, now, you might not think that asking a bunch of people stuff on the street is presented as an experiment, but a poll is a type of experiment. It’s a measurement of opinions across a population. And most recently, one that really, really aggravated me was, and I had a bunch of skeptical friends pop this up on their on their Facebook feed going, oh, this is why Trump got elected. Proof that America is really dumb. And they’d say this is proof that America is dumb. And then they would show what what happened like last week. It was a Jimmy Kimmel clip where he asked people to name a book. That’s all they had to do. They just had to name the title of a book. And of course he had about 15 people going, uh, Dr. Seuss. So this handful of people that ummed and awed and couldn’t think of a book or thought of a movie name instead, which I guess every movie was once a book, even if it was just a script, it’s kind of a book.

PR: Sure.

0:20:48
He had intro the piece by citing a Pew Research poll. And I went and looked at the Pew Research poll and it had actually showed that in the past couple of years, the percentage of people who admitted that they hadn’t read a book in the past year dropped by like three percent. So it’s like, OK, so the poll actually says that Americans are reading more than we did that year. But you’ve sampled about a dozen people out of possibly 400. Who knows how many he recorded? And you’ve put a couple of dumb people on the air and somehow once it circulates on social media, it has morphed into an experiment that proves America is getting dumber, and that’s why Trump got elected.

0:21:39
BR: Yeah, I think that’s a great example. I mean, you know, it’s not that those people can’t think of a book. I mean, the Bible, you know, everyone knows the title of a book. So it’s not, it’s a pseudo experiment. It’s, you know, again, whether you can come up with a particular book title with a camera in your face at the time, like, why? A book that I’ve read, a book that I loved as a kid, you know, Little House on the Prairie. I mean, I can think of lots of books off the top of my head.

CW: Sometimes that’s a problem. You have 25 book titles in your head, and you don’t know which one to pick.

0:22:13
BR: Right. I mean, for example, you know, if someone put a camera in my face and said, name a book, I would say Monsters of the Jivadon, which is a really good book by Jay Smith on the beast of Jivadon monster that we’re going to do a show on at one point.

CW: No, Ben, if somebody put a camera in your face and said, name a book, you’d say Investigating Ghosts by Ben Radford, available at Amazon.com for under $20.

0:22:41
BR: And tracking the Chupacabra, by the way. But again, I wouldn’t, but I wouldn’t think about that. I’d be like, you know, I’ve got, I’ve got too many books that I could, I could mention that I either like or have read or have written. So I could easily be stammering there like, what kind of do a children’s book, a fiction, novel or what, you know, so it doesn’t mean anything.

PR: Yeah. And when you see these and the reason I cite this, you know, I know that it isn’t technically an experiment when you go out on the street and do this, but the way people interpret these videos as data to prove something ends up making it a pseudo experiment in a sense. And the big thing is that the producers who do this know exactly what they are doing and they know exactly how to get people to trip up. And they know how to word questions, they know how to frame the situation in order to make people mess up on purpose. And that’s exactly what like tabloid reporters do when they chase down celebrities and stuff. They always rapid fire questions at them to get them to sound bad or say the wrong thing. I mean, it’s a game and it should never be taken as evidence of anything. And even when someone sounds really dumb on TV, there’s just no way to quantify it from that clip because these people are, you know, there’s a number of reasons they may say the wrong thing and they may just simply be edited to sound bad for all you know. And so that’s just not good data.

0:24:06
CW: Well, and it’s cherry picking from both sides, because anybody who has like even an intern position at a TV show, looking for people on the street knows that the people who give boring, bland, typical answers like, oh, name a book, Little House on the Prairie or the Bible or something. Those aren’t going to make the cut. You want the people who are stammering or the people who stand out in some way. Whatever your director is looking for. That’s what you call out and then the cherry-picking is on the other side, too I remember my brother and his friend got on the Jay Leno back when Jay Leno had the late show. He did a man on the street thing, too It seems like every late show host does and so my brother and his friend – he was wearing a powder blue leisure suit, looking like a weirdo at a local tech convention, and of course, they zeroed right in on him. And he gave the right answers, he acted like a weirdo, and he got on TV. People know I have to do XYZ to get on TV, and that’s what they give you.

0:25:30
PR: Yeah, that’s absolutely true. Another kind of pseudo experiment that I think a lot of people are familiar with, at least in the United States, is an ABC show called What Would You Do with John Quinones. For those of you who haven’t ever seen this show, it’s basically a show where they’ll set up a place full of actors and set up a fictional moral quandary. So they’ll do something that’s supposed to sort of bring a response out of the unsuspecting person that they target in the video. So like someone might walk into a store and witness someone saying racist remarks to someone else. And then the show is trying to see like, what would this random person off the street do to, you know, they usually do it with more than one person and there’s usually someone who responds poorly and doesn’t do anything or looks to have some kind of moral failing.

CW: I think the subtitle to that show is, Most Human Beings Are Terrible.

PR: Yeah, or you would think so watching that show.

0:26:21
BR: That’s sort of the subtext.

PR: Yeah. But there’s a couple of examples that really, really grabbed me. And the first one was a case of a person was basically setting up an online date with someone. So the mark was the guy looking for a date. He meets a girl online and he sees her online and then they set up, she’s with the crew and what happens is they have a picture of a young, thin, attractive girl on the profile. And then the flip to this is that when he gets there, the woman who comes out is certainly much older looking and heavier set than the pictures that were on the website. And it was basically trying to draw this guy into making him seem like he’s a very superficial person and they’re basically trying to trap him into looking like he won’t go out with her because she’s heavier, she’s older. And this one really frustrates me because the way it’s framed is this guy, he arrives at a restaurant and is sitting, you know, in the middle of the restaurant and the girl comes and sits down and obviously he’s kind of stunned that that’s not the person he saw online. And then they got into a discussion where, you know, she’s like, well, the pictures are old and she’s sort of acts blown away that like he just hadn’t recognized her and then he pulls out his phone and brings up her picture and then she kind of tries to like brush it off and say can we just you know not talk about it and and then the guy’s like not having it you know he’s he’s like no we have to talk about this and so she’s like you know you don’t like me because I’m not as pretty or because I’m older, because I’m heavier. And then, uh, the guy just said he didn’t want to continue the date because she had been dishonest. They made it look like the guy was being judgmental by being upset when his reaction is, is, you know, whether or not he’s attracted to her is kind of immaterial. It’s the fact that she is not the person that he had set up to meet, and the show doesn’t seem to want to, uh, do anything to address that. And, you know, they kind of brush it off and make it seem like this guy’s just, you know, a superficial, you know, asshole. And maybe he is, maybe, I mean, that’s… But that’s besides the point, because they set up the situation to force a bad reaction out of just about anybody. I mean, you know, sure maybe a person who’s really nice might just be like, you know, okay I’m just going to deal with the fact that this is obviously not the person in the picture, but any one of us I think would feel pretty put off by, you know, thinking we’re going to meet a person and a different person shows up.

CW: As a flipside to that. Had they staged that as a woman and a different guy showed up, she would leave immediately and be concerned for her safety.

0:29:49
PR: Yeah.

CW: This guy is obviously a liar. He’s probably a predator. This is not the same photo. This dude is older and creepy and I’m out of here.

BR: I’m really glad you guys brought that up. I think one thing that this example really highlights is the lack of alternative explanations. In a real experiment, the experimenters say, okay, well, what could account for this result? It could be A, it could be variable B, it could be any number of reasons that could explain the result, and let’s talk about this. So in a real experiment, experimenters will go out of their way to sincerely engage with and genuinely explain. Well, you know, this could explain it could be this could be that. Whereas in these pseudo experiments and that example he gave is a perfect one. They they latch right on to well he’s superficial. Like well That’s one interpretation or it could be it could be any number of other interpretations including the isn’t like being lied to, he’s worried that there’s something else going on, but they latch right on to the one that makes him look worse and frankly by extension, people in general look sort of bad. That’s one thing that bothers me is if the producers had said, well, here’s the results, you’re watching what happened. There are a couple of reasons why this could have happened. They zero in on the worst possible one.

0:31:05
CW: It’s shoehorning in what they want to hear rather than providing any sort of differential diagnosis.

PR: Exactly.

BR: Exactly.

PR: And the last one is also kind of a similar situation, another What Would You Do episode. It was an episode that had a restaurant full of people and there was a family, a quote unquote family, actors, and they sat at a table to eat and they start praying. And then another actress walks out playing an atheist and she rudely interrupts them and starts berating them for praying in public. Now, as I’m sure both of you guys are already feeling your blood boil over that one, the situation is basically, I mean, to begin with, which is that there, let’s take all the context out of this. People minding their own business at a table, someone berates them for any reason, yes, that’s wrong, okay, and I think the Christian thing is completely irrelevant and I think that the reactions would likely be the same no matter how it happened. But the problem is that, of course, they do this in Texas where, you know, most likely just about everyone in the restaurant is Christian, and of course it causes a big tizzy and everyone in the restaurant is yelling at this quote-unquote atheist woman, you know, and there’s just a lot of this silly stuff that happens where they try to just kind of cultivate this rage towards this person and the scenario, you know, it’s, I mean, obviously in this situation, the atheist character is in the wrong for walking up to a table and bothering a family. But the way that everyone got so angry at her was frustrating to me because they looked at that, they framed it as if it were a good thing, like, see, look at all these people standing up for this poor family. And it’s just like, this should have nothing to do with whether or not anyone in that room is Christian, because I would argue that just about any time that you see a couple of parents with their children sitting at a table and just some crazy person comes up and starts yelling at them, people are probably going to get upset. And it just is so frustrating that we have to see it framed like this and then just to really put the cherry on top, we drag atheists into this as if this is what we do.

0:33:35
BR: Yeah, that’s a great example. I’m reminded when you were talking about that of the hoax in 2016 where there was a guy named Adam Saleh who recorded a clip while he was on an airline. Of course it went viral and it was all over the place and it was a little short clip where he had claimed that he was being criticized for speaking Arabic on a flight. And it later turned out that he was actually a YouTube prankster and had made several videos of these sort of fake experiment staged pranks about how all Muslims are being targeted for this. Whereas other passengers on the flight said that’s not exactly what happened. They said he was going out of his way to be provocative and outrageous and making comments that could be construed as terroristic. And so it was not that he just happened to speak Arabic to a friend of his. The whole thing was staged and it was staged for outrage.

0:34:39
CW: Yeah, I remember that one. And it wasn’t just a couple of passengers. I remember practically everybody on that plane was like, uh, no, that guy did a whole bunch of rotten sh**. And then he hit record once people were reacting to him and acted like, oh, I was just talking on the phone to my friend in Arabic. That’s all I was doing. Stunts like that are often done with a mind to a noble goal, like the ends justify the means, that you’re for, you know, calling attention to backlash against Muslims, calling attention to religious freedoms, calling attention to some sort of social cause that the quote unquote, experimenter wants to reveal some truth about. And I kind of noticed that pseudo experiments, at least things that I categorize as pseudo experiments fall into one of two categories. One, somebody’s trying to sell you something or two, somebody is trying to make a social justice point. Now, I’m all about social justice points, and that’s actually the reason why I freaking hate pseudo-experiments for that cause. They are easy to knock over with a feather once you start deconstructing how poorly they’re designed. And if you’re throwing out this pseudo experiment in in in the name of a social justice cause, but it can be easily stripped away, then well, the alt right is going to do that for you. It’s just going to make it’s just going to make the point underneath of it look weak in addition to the data you’re presenting. So for as far as social justice things going around, you know, virally online, I feel like we’re just arming the enemy when we put up really bad, badly constructed, you know, pseudo experiments that don’t make a point in service of a point that should be made.

BR: Yeah, I agree. I mean that’s one of the concerns I have often had is people who, in support of causes that I support, will put up bad arguments and misinformation and errors and mistakes. I find that incredibly frustrating because I was like, come on, man, just do it right. It’s not that hard to do it right. And you’ll get good data instead of this sort of this bogus misinformation stuff that just is gonna explode in your face and give ammunition to the detractors. One example of this was, as you may recall, in November, 2014, there was a video that was produced by a company called Holoback. It was an NGO. And you may remember this, it was a two-minute clip that showed a woman, a white actress, walking the streets of New York for 10 hours. They cut it down into a few minutes. Now you’re wondering, hey, hold on here. 10 hours into a few minutes, but sure enough, that’s what happened. And it was interesting to watch the whole thing unfold because of course, most people were duly outraged, right? There was this woman just walking around New York City and she’s getting these cat calls and these, you know, hey mama and this.

0:38:00
CW: The cat calling video, I do remember this. Yeah.

BR: Yeah. And so so there was there was there was legitimate concern, outrage, as there should be, because this is not appropriate. But one thing that sort of sort of quick was quickly noticed was that virtually all the men that were catcalling were black. At the end of the video, it said that there were over 100 instances of verbal street harassment had occurred during 10 hours from people of all backgrounds. And yet almost all of them were blacks that appeared in the video. And so this was problematic for people who were like, well, hold on here. I support the anti-street harassment campaign and this is a worthy thing, but how is it that you seem to have focused on minorities harassing a white woman walking around? Anyway, so I’m not gonna go into the whole thing. There’s actually, when I was researching this, I found a brilliant breakdown of this by a Harvard associate named Zeyep Tufekci on medium.com. It’s titled, Hollaback and Why Everyone Needs Better Research Methods and Why All Data Needs Theory. Again, she’s written a brilliant piece there. You can find it on medium.com and she breaks down exactly many of the methodological problems with it. Now, I’ll just briefly quote one thing she says. She says, in effect, this was a research project and it had implicit research question. That is, conventionally attractive white women get verbally harassed in New York.

0:39:31
CW: That’s the only reason I go to New York. Oh yeah, I feel very attractive when I’m walking down the street in New York.

BR: Hey baby, there you go. Right, so we got that. So she says, however, in doing so without any reflection on its own method, it amply demonstrates the crucial substantive and political importance of research methods. She writes, I’ve taught introduction to research methods to undergraduate students for many years, and they would sometimes ask me why they should care about all this method stuff, besides having a required class for a sociology major.

CW: That makes me sad that they would have to ask why did she care about research methods?

BR: You would… that would be that would be troubling. She says, I would always tell them without understanding research methods, you cannot understand how to judge what you see. So anyway, I just want to highlight that this this piece on medium is a beautiful examination of exactly why this hollaback video is a pseudo experiment and in fact may do more harm than good.

0:40:28
CW: Well, yeah, I think I will check that Medium article out. Thank you, Ben.

BR: We will have a link to it in the show notes.

CW: To get to an example that is kind of, well, it’s one of my favorites. Do either of you guys remember the Pepsi challenge? Of course you remember it, but did you actually participate in it? I remember doing it myself as like a 10-year-old at the mall.

BR: I saw the ads and I wasn’t really feeling that challenged. I’m like, you know what, they’re both pretty good. I like Dr. Pepper, so I did not take the Pepsi challenge, but of course I was there.

0:40:54
CW: See, we weren’t really allowed to have soda at my house growing up. And there was free soda and free crackers. And so of course I did the Pepsi challenge a number of times as a child. And that stands out to me as probably the biggest example of a pseudo experiment done in the name of selling something. Now, as far as the Pepsi challenge goes, it really doesn’t have a giant marker in terms of the experiment design. It wasn’t double blind. It was only single blind. The person tasting the sodas did not know which was which. Interestingly, Pepsi didn’t publish any statistics that I could find indicating what the grand overarching total of people was. But not surprisingly, in every Pepsi challenge that Pepsi conducted, more than 50% of the people surveyed preferred in the blind taste test preferred Pepsi. You can you can argue that that’s actual data. They’re giving the test to people, they’re recording what they prefer, Pepsi or Coke. But this is an example of an experiment measuring something, but what it’s measuring is misleading and incomplete. It led to the people involved wasting a ton of money and a ton of work and leading to what was called one of the biggest business blunders in history, which I don’t know if you recall, New Coke.

BR: Hey, you know what? Let me just, I’m just gonna say this with record. I love New Coke. I, in fact, in fact, I’ll just-

0:42:32
CW: You are dead to me.

BR: So, oh no, here’s the thing. So at one point, I forgot how many years ago, at one point I contacted Coca-Cola and I said, I’m probably the only person in the world, but I like New Coke, where can I get it? And they directed me to a bottling plant, I think it was in Detroit. And so they said, this is the last place that’s bottling New Coke. So I called the bottling plant and the week before they’d stopped making it. I was pissed off.

0:42:55
CW: Wow, that sounds like something you would do. Unlike you, Coke had about 400,000 people write letters of complaint about New Coke. People across the board hated it, it was ridiculed and New Coke came about because of the Pepsi Challenge. Coke looked at the Pepsi Challenge and thought, oh, people are preferring the sweeter tasting cola over our cola, even though Coke had by far the biggest market share and Pepsi was this little upstart trying to chip away a little bit of their market share but certainly they did not take the lead at any point. But it was enough to rattle Coke into changing their whole formula, which was a huge blunder and actually helped Pepsi a lot because once people had this new Coke, they went right to Pepsi. But let’s look back at the experiment. Obviously you take a little sip of one, you eat a cracker, cleanse your palate, and then you take a little sip of the other.

0:44:08
BR: And presumably, they are at the same temperature. They should be. They may not be, but there are little subtle things where if, for example, Coke was a few degrees warmer, for whatever reason, maybe innocently, maybe not, that could affect the taste. So, it’s very important that the measurement of the taste.

CW: Well I don’t think they controlled for that in any of the Pepsi challenges I drank as a kid. But anyway, a good breakdown of this experiment appears in the book Blink by Malcolm Gladwell. And I did a little bit more reading beyond Malcolm Gladwell’s write up, but taste tests do not tell the real story when you’re taking little sips of something. And this is something that food researchers knew. It’s not like Pepsi or Coke was blindsided by this information. There is always a bias for sweetness when you take a small sip of something, which is, I don’t know if you’ve ever seen people drink soda in America, but we do not consume it in small sips and then stop. We sell 85-ounce super duper big gulps and we drink five of them over the course of a day. So in a blind taste test, you’re just gonna gravitate towards the sweeter thing. But if you drink more than a couple ounces, that becomes disgustingly syrupy gross. I asked my husband, I said, hey Rob, do you remember the Pepsi challenge? Can you tell me what the big flaw in that challenge was? And he said, it involves Pepsi. I can’t stand Pepsi. I’m one of those people that just yeah, I prefer the other one, but Anyway, but also in blind taste tests of wine people almost invariably prefer the sweeter varieties This is noted by Felix Asalmon and he quotes Bob Millman of executive wine seminars Which I think is a neat little quote about our taste buds here left to their undirected devices devices the senses will almost always gravitate to the obvious and miss the subtle and It’s just reminded me of of what you know the topic about fooling our senses. You really will miss subtleties if you’re just taking a small sip of something you need to sort of savor it and and have a little bit more and sort of what you’re gonna drink a cup of is very different from what you’re gonna drink a sip of in terms of what you like so based on this huge flaw in the theory underneath the Pepsi challenge, Coke went ahead and redid their whole formula. And the other question is Pepsi never really came out with figures. I could not find them anywhere in terms of how many people actually preferred Coke versus Pepsi. Interestingly enough, just recently for funsies, somebody at Business Insider recreated the challenge with all of the staff members there. So out of 21 staffers they did the the single blind taste test and Coke actually beat out Pepsi 62% to 38% So I don’t think that one little upstart recreation disproves all of the Pepsi challenge, but I say it’s safe to say that that might have happened on a couple of instances of the Pepsi challenge. But Pepsi is not going to put that in their commercial.

BR: I didn’t remember the intricacies of that. I’m wondering, when you’re researching that, did they give the numbers of people that like them equally?

0:47:24
CW: Actually, I don’t think that was an option. I think people were given a yes, no. I did not see any data indicating that anybody was even allowed to say, yes, I like them equally.

BR: Well, that right there is a failure in basic methodology. And that right there is problematic. I mean, it’s entirely possible that if they had given choice, do you like them equally, 90% of them would say, yeah, they’re the same.

0:47:52
CW: I think you have a really good point there. And as I said, I did the test when I was a kid. But I certainly remember feeling, I mean, you’re presented with two things. You’re told to make a choice. I don’t think people would just naturally say, I prefer them equally, put that down as my response. I think they might just randomly pick one. If they didn’t really have a preference, they’d just go with their gut.

BR: Well, right, and that’s, I mean, of course it’s not in Pepsi’s interest to have anyone respond, I like them equally. No. That’s not an answer they wanna get. They want either yes or no, and of course they want it biased to their own products. So again, this is another example of how to distinguish real experiments from fake experiments. Let me shift gears to a more recent one in a different area. It happened in February 2018, earlier this year. And there was a mother in San Antonio, Texas, who was concerned about school shootings, as we all are, and certainly is a big issue in America these days. And it was a couple weeks after one of the shootings, I think it was the one in Florida, but I could be mistaken. And the mother had decided that she was going to film herself on her iPhone walking throug her daughter’s high school. And she wanted to see what would happen. Would she be stopped? And this stunt led to headlines such as, mother walks into South Sand High School to prove district’s need for extra security.

CW: To prove, okay.

BR: To prove! This proves it. In case you’re wondering whether the district needs extra security, here’s your evidence, here’s your proof, watch this video. So I looked at it, and this is being widely shared, of course, on social media, the internet, and like, well, look at this, isn’t this outrageous? This woman’s walking on, in her daughter’s high school, without being stopped, apparently, by armed guards. When you look at it more closely, it turns out that she’s actually a school board member and her daughter goes to that school. And she shared the videos on social media to demonstrate what she claimed was just how endangered our children are. Now think about this for a second. So she’s basically saying that because she, an unarmed middle-aged mom, who’s known to the school staff as a parent there, could walk through the school unconfronted by armed guards or anyone else and was recognized correctly as not a threat meant that there’s lax security and there’s going to be a shooting there any minute. I just, I’m like, what? It proves nothing of the sort.

CW: Every time I hear about something like this I like to sort of just mentally frame it as what would happen if the opposite happened? And I would think any woman who well any middle-aged parent of a child walking into her, you know her child’s school if they were if she was confronted with what are you doing here? And you know come fill out these forms, and you know you chose picture ID and da da da da da It would be thrown up on Facebook as evidence that we are living in a totalitarian society and that look, I’m known to the school and they won’t even let me, they’re harassing me unduly and this is ridiculous and you know, what sort of prisons have our schools become?

0:50:42
BR: Yeah, yeah, it doesn’t, and there’s, in this world view, there’s no right answer. No matter what the school does, if they immediately confront her with armed guards, then they’re overreacting and this is horrible and this is what society has come to. But if they correctly recognize that this middle-aged mom with a cell phone is not a young teenager with an assault rifle, and that she is not in fact a threat to anybody, then well, they’re just, they’re just, they’re, you know, they’re underreacting and they just don’t recognize the need for security. So it just, you know, stunts like this, they don’t demonstrate anything. In fact, they’re counterproductive.

CW: Yeah, that’s that, that, that one kind of did make me roll my eyes when I saw it. As somebody who’s, I’ve painted a bunch of murals at schools and I just, you know, a custodian lets you in, I, you know, I go there early in the morning with my buckets of paint. And you know what? I’ve never been thrown out of a school ever.

BR: Not for that reason anyway.

0:52:12
CW: Let’s not go into those other things back when I was in high school. But okay, well, I remember there was one that we both kind of talked about when it came out, and I had specific reason to talk about it. And it was a Dove ad. And Dove has gotten into trouble a couple of times. I think the most recent thing was last year when they had an ad featuring a couple of models, one of whom was darker skinned, one of whom was lighter skinned, and because it implied that the darker skinned woman was turning into the white woman as a result of using Dove’s cleansing products, it was very poorly framed. They needed somebody to really think about it before they released that ad, but Dove has had a long history of natural beauty, focus on what makes you beautiful and everybody’s beautiful and buy our soap.

0:53:00
BR: As I recall, that was actually a short film produced in 2013. I think it was titled Dove Real Beauty Sketches. I’ll just read from the description on the video. It says, it explores the gap between how others perceive us and how we perceive ourselves. Each each woman is the subject of two portraits drawn by FBI trained forensic artist Gil Zamora, one based on her own description and the other on use on strangers observations. The results.

0:53:42
CW: Now, I don’t know anybody who directly I don’t know Gil, the fellow who is the FBI forensic artist, but I do know people who do similar things. And so my Facebook feed actually kind of lit up with people, with artists kind of debunking and making fun of this ad. But why don’t you tell me your thoughts on it, and then I’ll go into some technical aspects of it.

BR: Sure. Well, one of the first things that jumped out at me was the purpose wasn’t quite clear in terms of what is the research design here? What are we proving? You know, we have, I mean, the goal was to show women that they’re beautiful, that they’re more beautiful than they think by comparing their self-descriptions to those of strangers. So in the video, several women describe themselves to the sketch artist who can’t see his subjects. The same women are then described by strangers whom they met the previous day, and then they compare the sketches. So basically the stranger’s image was invariably more flattering and more accurate than their own, than how they described themselves to the sketch artist. And this went viral. It was all over the place.

0:54:40
CW: It created a big emotional payoff. People would watch this like seven minute video and be like, oh, this made me tear up and oh, I didn’t realize this about myself. I think I do this too, you know.

BR: Yeah, it was all over the place. And again, as with the other examples we’ve talked about, it was widely talked about on social media as being proof that, you know, you need evidence, here you go, right? And so there was one newspaper that called it Dove’s most thought-provoking film yet. Forbes said it was powerful. And all these glowing reviews. And there was a Huffington Post article by Emma Gray that began, “‘Do women see themselves less accurately than strangers do?’ A new Dove campaign says yes, and offers proof in the form of forensic sketches.” And so I’m like, well, if you’d said offers, you know, a suggestion nudged that way, that’s one thing. But when you say that this suit experiment offers proof, I’m like, uh…

CW: It offers an emotionally manipulative marketing campaign.

0:55:54
BR: So one thing that jumped out at me was that, I mean, there’s all sorts of flaws here. The first one that jumped out at me was that the two aren’t really comparable. I mean, there’s so many ways this could go wrong, and the fact that how someone else else described them doesn’t match up with their own self-description doesn’t necessarily mean that just because there’s a gap there doesn’t mean that one is better than the other or is flawed.

CW: I actually went back and looked at some of the Facebook comments from some of my professional artists and animator friends who had a lot to say about this ad. A few people said there is no way that the forensic artists didn’t get some direction from the people hiring him. I mean, it’s not like he walked off from his office and they sat him down and said, just listen to these women and draw. I’m pretty sure he was given some direction. Many of the other artists on my feed suspected that he might have even seen a photo of each subject, as generally speaking, people are terrible at describing themselves. And he wasn’t as off target as he necessarily should have been. Perhaps he’s a brilliant forensic artist. The best part, or rather the most important part of being a forensic artist is eliciting the description from someone, is getting them to answer the right questions about, OK, well, comparing features to somebody else. But yeah, many people who watch this who are in the business are like, yeah, he probably was primed to do it the way he did it. And philosophically, a lot of the caricature artists had a problem with the message, which was, women, your lives will be so much better if you just realize how pretty you are. We’ve hired a man to show you that. The huge thing holding you back is you’re too stupid to realize you’re pretty. And The Ringer actually called the new Amy Schumer movie that’s out right now, I Feel Pretty, which is getting a lot of backlash, The Ringer called that whole movie content to be a feature-length dove ad. And obviously I hang out with caricature artists, we’re a group that slants a bit more toward celebrating what features make a person look like themselves and not a generic bland mannequin type, you know, reduction of maybe looks a little bit like somebody from a distance. So to all of us, we’re looking at the two drawings and we’re like, no, the first one is more accurate. The first one is better. The second one, the second, people do know their features better than a stranger who’s maybe talked to you for 10 minutes. And if you look at it, if I, you know, cause I took some time and looked at these sketches, the second one wasn’t prettier. It was more generic. It was, uh, you know, you could say it looked younger, but that’s because there was less there to go with, you know, people get more distinctive as the age and the first drawing was more distinctive. The second drawing lacked distinction. So it was just like you might as well just look at it through a fuzzy, oh wow, when I put Vaseline on my camera lens, I’m prettier.

BR: Right, right.

0:58:59
CW: It depends on your definition of pretty. If you just would rather be blinded to every little feature, then yeah, I guess you’re prettier.

BR: Yeah, I mean, I would want to know a lot more about the methodology, like, you know, the forensic artist they had, you know, Gil Zamora. What exactly was he told about this? I would be shocked if he knew nothing about it, if he was just brought in cold and wasn’t told, first of all, who hired him? Is this for Dove, the real beauty campaign? Can I guess what answers they might want to get out of this? Are the people paying me maybe wanting one sketch to look better than another one?

CW: Well, and the women themselves, you know, when they were sat down and said, you know, okay, describe yourself, I would bet, I would bet quite a bit of money that the person directing the women said, okay, now describe, describe your features and be honest, you know, just tell them exactly how you look, flaws and all, you know, don’t try to candy coat your appearance just, you know. And just in general, of course, strangers who are describing a person they’ve just met they are gonna say oh, she well – Okay, maybe if it’s a caricature artist because I you know I might be a bit more harsh. But the average person is not gonna say oh, yeah that woman. I just talked to in the other room Well her crow’s feet indicate. She’s you know she’s probably got at least a you know a half a millimeter crow’s feet right here And she’s got you know you can see her nasal labial lines are formed pretty fully so she’s at least in her 40s. People don’t pick up on that. They don’t know how to put that into words.

1:00:34
BR: Right. Well, and there’s another confounding factor here, which is that both men and women, although probably more so women, tend to avoid describing themselves in overly, in superlatives about how beautiful they are.

CW: Yeah.

BR: Because they don’t want to appear vain. And there’s research on this. called Face Value that had some by Autumn Woodfield-Madron, I believe was. And she talks about this. She says, you know, that in society, there is a bias against people saying that they’re stunning or gorgeous or beautiful. When asked to describe themselves, women tend to describe themselves as generally pretty but not over the top gorgeous because, again, there’s people don’t want to appear vain. Oh I’m beautiful.

CW: It’ll make you ugly if you start harping on how beautiful you are. It’s just not it’s not it’s not done. It’s not supposed to be done.

1:01:00
BR: Right and maybe that may be especially true for a male artist. Anyway there’s a piece in Bloomberg by Virginia Postrel and she noted the experiment wasn’t scientifically valid. She says that she knows that the participants were not randomly selected and the sketch artist didn’t show the women the sketches to confirm they were accurate.

1:01:46
CW: Yeah, that deviates from standard procedure for a forensic artist. Yeah, yeah. And again, that’s a big problem here because for all we know, had they shown the sketches to the women and say, oh, no, no, I need changes or change that. So, I mean, they just and and of course there’s also a selection bias, right? I mean, I according to Pastrall, the video editors only chose seven out of twenty women. That’s some cherry picking. That’s like less than 50 percent. Yeah.

1:02:12
BR: Well, it’s about a third, right? So which which third made the made the video cut?

CW: Yeah.

BR: And why?

CW: Obviously, the ones the third that made the point that Dove wanted to make. So it really it tells us nothing it created an interesting talking point I had fun discussing it with other caricature artists friends.

1:02:33
BR: Yeah, and again, I mean I have no particular problems with the the dove real beauty campaign I think it’s problematic in some ways, but overall you know it’s it’s a marketing campaigns advertising, so it’s not well, that’s…

CW: Well another thing people pointed out is that dove is owned by Unilever, which is also the parent company of Axe Body Spray. And so there were some comparisons that came out saying, okay, look at the ads that Axe is running. Again, we’re thinking about the 2013 era where tons of Axe ads on the air with a guy walking down the street and getting mobbed by dozens of women ripping his clothes off. And they’re like, they’re feeding into that male, you know, teenage boy narrative of like I’m gonna be wanted and lusted after by every woman who meets me and then they’ve got this other narrative that they’re feeding to women saying, honey you’re really more more pretty than you think you are and your life will be so much better if you just realize you have good looks.

BR: Just accept it, just understand that everything is fine. Everything is farting roses.

1:03:39
CW: The world has Vaseline over its lens, honey. You’re beautiful. But yeah, the dove ad bothered me. I’ll admit I have a bias there kind of against the whole message underneath of it, but it brought tears to eyes of some people and showed them that there’s absolute psychological scientific proof that they can’t see their own beauty.

1:04:04
BR: Oh, tear. Well, look, I mean, we could go on and on about this. There are many other examples that we could have talked about. We’ll save some for future shows. Really, once you recognize some of the aspects of pseudo-experiments, they sort of pop out at you and often annoy you. It’s sort of a double-edged sword there. There are many more we could touch on. We don’t have time to go into them. I just want to wrap up by pointing out a second type of pseudo experiment. And that is the thought experiment. So we’ve talked about the pseudo experiments where there’s video, it’s online, and there’s sort of these fake experiments and comparisons. But I think there’s another type in which they’re usually framed as a rhetorical question intended to expose hypocrisy or double standard. And there was a post that commented on the recent conviction of Bill Cosby on sexual assault charges and one of the posts said, imagine if Weinstein was black, reference to Harvey Weinstein, who at the time had been accused of somewhat similar conduct, not necessarily drug and raping but but along those lines who at the time had not been arrested despite a variety of accusations and the the suggestion seemed to be that well that that that

1:05:25
was the difference that will cause these black –

CW: And he’s gonna get away with it because he’s white!

BR: Right. There and this is this is one thing that the I was like well there’s there’s lots of different factors here I mean it’s true that one’s black, one’s white. They’re both rich, powerful men in Hollywood. And I’m certain that race plays a role in it. But I don’t know that that is that is the key between the two.

1:05:56
CW: You have to you have to shore up all the other confounding factors. All right. Well, OK, let’s wait a while, because if Harvey Weinstein is going to get out of everything because he’s white, well, let’s give him the same amount of time that Bill Cosby had in the lead up to before he was even arrested. And I think harvey weinstein actually got arrested And let off in handcuffs a lot sooner Than then bill cosby faced charges.

BR: You’re exactly right.

CW: I remember seeing it. It was nice to see.

BR: Um Yeah, and so so the fact that we we don’t know I mean for all we I mean Weinstein could get off cosby could could get could get an appeal. We don’t know But the fact is that at the time Weissen had even been arrested yet so he couldn’t have been convicted because he wasn’t even arrested so again It’s it’s a I would I would argue it It is sort of a pseudo experiment in that we’re being asked to sort of it’s like oh Can you imagine if this happened? You know? It’s not a real experiment. It’s can you imagine X? Can you imagine if Cosby is white? Or if you can imagine if Weinstein is black? Can you imagine if X, Y, Z?

1:06:56
CW: Yeah, I can imagine a lot of things. I can imagine unicorns flying out of my butt. But we don’t have data on it. Well, we do have data in the case of the judicial system and a bias against black defendants. We absolutely have data on that. But if you’re just taking one thing, it kind of just throws your case under the bus. I had somebody point out, again a Facebook thing, somebody was like, they saw somebody get pulled over like in front of their house and they said, well that guy, he was black, I could see he was black from the window but I didn’t want to go out because he might get shot if I intervened or blah blah blah. She didn’t have any, she’s like, they searched the car with no reason whatsoever. And I’m sitting here thinking, okay, I know people who’ve had their cars searched who are white, and it’s because their car smelled like marijuana, which you’re not gonna be able to tell if you’re looking at it from your house window and you’re seeing it across the street. So you can’t say, imagine if he was white, he wouldn’t have been pulled over. No, you can’t say that. Well, we don’t know. I mean, it could have been an expired license, could have been speeding.

1:08:04
BR: We don’t know what happened before the stop. And we don’t know that just because the person was let go that they weren’t given a ticket or a citation or for that matter, even if they had been arrested, we don’t know that that would legitimize the stop.

CW: Yeah, well, see that social media is filled with people commenting on stuff they don’t know a whole lot about. And we’ve all been guilty of that. But hopefully we’ve given the listeners a couple of clues on how because once you break down one or two pseudo experiments and you say, here’s a weakness, here’s a weakness, here’s a weakness, you get a little bit better at spotting it in other pseudo experiments that might have completely convinced you of something in your youth or on another day.

1:09:01
BR: Absolutely. So I think the and in these cases, you know, I agree with the larger point. I mean, there definitely is a disparity between how blacks and whites are treated by the justice system, absolutely. Does that explain why Harvey Weinstein hadn’t been convicted yet? Not necessarily. And again, the problem is that a lot of them, they’re framed as, can you imagine if X, can you imagine that this this had happened well? The answer is always yes, we humans are imaginative people we can imagine any sources that we can imagine any number of circumstances or scenarios In which the outcome is described, but that doesn’t mean that’s likely from probable. It just means that yeah, we can pick –

1:09:30
CW: Yeah and this goes back to the the problem. I mentioned with pulling up thought experiments or pseudo experiments in the service of social justice. The very case you’re mentioning, oh, well, Bill Cosby got a trial. Well, he got two trials. The first one was a mistrial, and it took a long time to get him to justice because that’s the way our justice system works. Somebody doesn’t say, oh, I heard this guy. I heard this guy did something wrong to a woman, well get him in jail tomorrow. That doesn’t happen. You’ve got to build the best case, you’ve got to interview every woman involved so that you put the best ones up, the best scenario cases that are most likely to get a conviction. You’ve got to narrow it down so that you don’t just, you know, hit it with buckshot that’s not going to stick. And so if you use that particular example of imagine if Harvey Weinstein were black. Well, okay It’s been like a week now and that example has just backfired because it took Harvey Weinstein less time to get charged than it did Bill Cosby so you could have the somebody from a white supremacist group saying well see look at the case you know you mentioned there Harvey Weinstein was was let off in handcuffs a lot sooner than Bill Cosby was, if you look at the respective timelines. So that proves that whites are treated unfairly compared to blacks. And that’s not a point you want to make, because that’s ridiculous and nonsensical.

1:11:02
BR: Right. No, that’s a brilliant point. I mean, you know, a lot of these issues are inherently ambiguous. It’s not clear what the circumstances always are. And so you can read it either way. And that’s a problem because if you have people on different sides of a topic or from different political parties, looking at the same evidence and reading them in opposite ways.

1:11:30
CW: Yeah, a data point of one is useless and a data point of one plus a hypothetical is less than useless. Useless-er.

BR: I think the phrase is useless-er.

(MUSIC)

PR: We would like to thank our sponsor level patrons, Amy Freshour-Kelly and Dr. Angela Mattke. If you’re interested in becoming a sponsor or just throwing us a buck or two per episode, check out patreon.com forward slash squaring the strange. Every bit helps us keep the lights on here in Strange Studios and we could not keep bringing you content without the support of our generous friends. If you’re looking to provide moral support, head on over to iTunes and give us a five-star review. Every positive review helps us move up the podcast charts. Thanks again, strangers.

1:12:04
BR: Alright, so that was a look into the topic of pseudo experiments. We hope that you all enjoyed it or got something out of it and maybe a little bit of media literacy going on there. Pascual, as you may have noticed, was still in a coughing fit or maybe in the ER or something. So next week, I wanna talk about something that I’ve researched pretty in depth and that is the Pulse nightclub shooting in Orlando, Florida a couple years back. I think it will be almost two years when the episode comes out. I did a pretty in-depth investigation into some of the causes and myths behind it. I wanted to share some of that with you.

1:13:05
All right. I look forward to that. Pascual, feel better. We’ll see you next week, buddy. Take care. All right. See you next week.

1:13:14
All right. See you next week. and of course our listeners and patrons. The views and opinions expressed in this podcast are those of Swearing Strange and do not necessarily reflect the views and opinions of Cronus Creative, Center for Inquiry, our advertisers, or any affiliated organizations and employers.

Transcribed with Cockatoo