Click each image to enlarge.

0:00:00
Ben Radford: Welcome to Squaring the Strange, the podcast that examines all manner of the mysterious and the mundane through a critical lens. So, let’s square the strange.

(MUSIC)

0:00:11
Celestia Ward: Welcome back to Squaring the Strange. This is one of your co-hosts, Celestia Ward. And with me is…

0:00:32
BR: G’day, Ben Radford here.

0:00:33
CW: Oh God, you’ve been in Australia too long.

0:00:36
BR: I don’t know about that.

CW: We are here for episode 242. Ben is back from Australia. And our main topic with our main guest is going to be cyber security expert and magician and folklorist all in one person, Perry Carpenter. And it’s kind of good, we’re getting we’ve been having folklorists on who are releasing books, and Perry has just released, in October, it’s a fresh book, hot off the presses, F.A.I.K. and that’s spelled F-A-I-K. A Practical Guide to Living in a World of Deep Fakes, Disinformation, and AI-Generated Deceptions. So if you are searching for a holiday present for any of your fellow skeptics, it’s got 4.8 stars on Amazon, so check it out. In the meantime, Ben, you’re back! Can you brush the outback dust off of your dungarees and tell us what you’ve been up to?

0:01:49
BR: Well, yeah, it was a fine time. I was out there for the Australian Skeptics, the 40th anniversary conference, which was super cool. As I think I mentioned before, I’ve never been to Australia, so I always wanted to see what it was like. You know, you see documentaries like the Mad Max series, of course, and you wonder, you know, just how much of that is true. So, but it was fun. So, we had a conference out there. We had, you know, Richard was there, Maynard, who was a fan of the show and friend of the show.

0:02:23
CW: Oh, yeah. Of course we know Maynard.

0:02:25
BR: From what I could gather, everyone was fairly pleased with the turnout.

0:02:29
CW: And Richard walked away with some kind of an award, correct?

0:02:33
BR: He did. It was very cool. So they managed to keep it a secret from him. He got a Lifetime Achievement Award, which I might add is well, well deserved.

0:02:41
CW: Yes, yes. That’s the most, that’s the loveliest way for your peers to tell you you’re getting old. I hope to receive one myself someday.

0:02:52
BR: He made some comments to that effect, but it was funny because he sincerely didn’t know. I mean, apparently one of the other officers mentioned that apparently it never occurred to Richard that people could email and CC each other without including him. So that was apparently news to him. That was all it took to keep the deception alive. But yeah, it was super cool. I was at the dinner when he got it and it was great. And he got lots of photos of his award, which he kept sending me pretty much daily, sometimes hourly. But we had a good turnout. We had a great lineup, including Dr. Karl, who’s a well-known science personality. And in fact, as I was leaving Australia at the airport, there was a huge one of those bookstores, not a huge bookstore, but a huge banner in front of the airport bookstores that had Dr. Karl and his new book, which unfortunately he didn’t have for sale at the time. It’s brand new out, but he didn’t have any copies for sale. But it was it was cool. And I joined Richard for a panel with two other people. One was a guy named Paul Cropper, who actually did some research with Sharon Hill.

CW: A small world, yeah.

BR: A small world and he’s more of a believer but also kind of more skeptic-y. I mean if you’re collaborating with Sharon that’s a big plus in my book. He did an investigation on fish falls.

CW: Oh, like fish falling from the sky?

BR: Right, right. And I had seen Sharon write up something about this before. Basically he was questioning whether it was in fact, as widely assumed, like a tornado or a water spout. He was saying that some of the evidence didn’t really point that way, that it was probably likely instead passing birds that had basically thrown up and vomited.

0:04:40
CW: I remember the flock of birds throwing up has become a more accepted explanation these days, right?

0:04:48
BR: Right. So it was cool meeting Paul and there was another fellow there who was an Australian UFO buff and he went on for a while, a nice guy, but it wasn’t much of a discussion. It was like, “Here’s what I think about UFOs. I’m going to talk very quickly.” But it was a fun conversation. Everybody was very respectful and fun. Later on there was a podcast discussion with Maynard and Richard, where we mashed the two podcasts together.

0:05:20
CW: I heard Squaring the Zone. Yes. My lawyers will be in touch, Richard Saunders.

0:05:28
BR: Yeah, there was more. I don’t want to begin listing the speakers because I’ll forget somebody I don’t mean to, but it was a good lineup. We had four young science students. I think three of the four were girls, young teens, which was great and well-received, and it was nice sort of having them stand up in front of an audience of knowledgeable skeptics and getting some applause for good work. So that was fun.

0:05:53
CW: Cool.

0:05:54
BR: And I was also gifted an amazing handmade bag with a drop bear on it by a friend of the show, Amanda. Thank you, Amanda. I haven’t posted a photo of it, but it’s super cool and I will be posting it up there soon.

CWL Yes, I think I saw a photo you sent me and it just got like angry little teefies and stuff.

BR: Yeah, it was so cute! It was adorable.

CW: Apart from the Skeptic Convention, the Skepticon, did you get out and about and what did you see?

0:06:21
BR: Well, just before the Skeptics Conference, we had a Skeptics in the pub with a trivia night. So that’s where I met a few dozen Australian skeptics. And I’m pleased to say that our team, the Spider F*****rs, got the highest score. So that was very cool. The phrase, in case you’re wondering, was introduced to me by Lara Benham, who mentioned Aussie slang phrase, “We’re not here to f*** spiders.”

0:06:47
CW: I’m gonna have to be bleeping all of that out, you realize.

0:06:49
BR: I’m just letting you know the name of our tribute team. You do what you have to but I’m passing along the fair dinkum truth. So I’ve never heard that phrase before and frankly neither had Richard or at least he said he had, being all proper and all, but I now use that phrase whenever I can, most recently at the bank this afternoon. I did two classic train trips in Australia, one going north-south, the other east-west. I took the Ghan from the south and Darwin to the north, which followed, it’s called the Ghan because it was after Afghan cameleers who originally sort of pioneered that route. And then crossing east-west from Sydney to Perth. So that was kind of fun, spending a total of, I think, five or six days on the train, which was cool. But after five days on the train, you’re like, you know, I’m about done with trains for the time being… But that was fun. And I noticed that in two of the cities that I visited, John Edward, the grief vampire immortalized by South Park as the biggest douche in the universe was on tour while I was there.

0:07:58
CW: Oh, so he was hopscotching around Oz at the same time as…

0:08:04
BR: He was.

0:08:05
CW: …a renowned skeptic, Ben Radford. Okay.

0:08:09
BR: He was. And I noticed on the same day I saw an ad for him, I picked up a local newspaper because that’s always something I always enjoy doing when I’m traveling is it just sort of gives you a sense of what are people thinking about and talking about and local issues and whatnot. And on the front page, there was a headline about a mysterious disappearance of a local woman two years earlier. And it got me thinking, well, you know, John Edwards in town. He’s doing a show there. It’s like, why, why doesn’t he stop into local police departments while he’s on tour? You know, for free, give back to the community that he’s getting rich off of. I mean, he’s in the area anyway. And if he has some of the powers, the claims to have, why not do some good, bring some help to desperate families?

0:08:52
CW: Oh, he has no control over it. That’s not how it works.

0:08:56
BR: Yeah, right. I forgot. That’s unfortunate. That doesn’t work that way. I did help prove the earth is round, which was cool.

0:09:05
CW: OK, did you bring string with you?

0:09:09
BR: I did not. You know, that’s a pendulum. I think you’re thinking, no, I actually I take too much credit for that. Dave Thomas, who’s of course a friend of the show, and we’ve had him on for a couple of things.

0:09:21
CW: He proved the earth was round over at the Albuquerque Skepticamp a couple of months ago.

0:09:26
BR: Yeah, he did. For those in case people missed it. And we had him on for a couple of things, including the Bible Code and the Aztec Roswell crash, if I’m not mistaken. When I left, Dave asked me to take photos of the night sky. And I’m not much of a photographer, and certainly not a nighttime photographer, so I did my best when I was deep in the outback with little light pollution. And I sent him, I don’t know, eight or ten photos that I hoped were okay. And as it turns out, I got a nice little mention on the Twitters on November 30th, he wrote, a friend traveling the Australian Outback, Southern Hemisphere, took a picture of the constellation Orion. Down there, it’s upside down from how it appears in my New Mexico home, Northern Hemisphere. Only makes sense on the globe, thanks, BT Radford. So.

0:10:17
CW: Yep, yep, yep.

0:10:18
BR: I did nothing other than spend 10 minutes trying to take photos of the darkness, but I did that.

0:10:23
CW: Yeah, it’s interesting that he doesn’t just show that the flat earthers are wrong. He’s cataloged like dozens of ways to do it. He’s not just, oh, like, I can prove that the earth is round by doing this. And they’re like, oh, OK, cool, and this and this and this and this and this. And before you know it, he’s proved it like 20 times.

0:10:46
BR: In case of any of the other 32 proofs, don’t move the needle, look at this. It all fits. So that was cool. Unfortunately, I didn’t have much time to do any real investigations. I did visit the Blue Mountains, which is home of the Yowie, repeatedly, Australia’s version of Bigfoot. So I did a little bit, but I was kind of on the move, running around, doing things like that. But it was fun. I did see some great art. There was a Chihuly show at the Botanical Gardens in Darwin.

0:11:19
CW: I like Julie, yeah.

0:11:21
BR: Yes, very cool. Of course, his work is all over Vegas. Which casino is his?

0:11:27
CW: Bellagio. It’s all over the ceiling of the Bellagio, yes.

0:11:30
BR: Very cool. And also, while I was in Sydney, there was an exhibit of Rene Magritte, my favorite surrealist artist, at the Art Gallery of New South Wales. So that was a special treat for me. So I spent much of a day looking at, uh, at Magritte stuff. So it was, it was great. And anyway, just, uh, I I’ll, I’ll wrap it up there, but I do want to give a big shout out and thank you to, um, to Tim Mendham of Richard Saunders, of course. Congratulations once again on your award,, Maynard, Eran, Lara, and others who were such great hosts.

0:12:04
CW: Yes, big thanks to you Australian folks for entertaining him, getting him out of my hair for a while.

0:12:10
BR: So, Celestia, what are you skeptical of?

(MUSIC)

0:12:23
CW:  Well Ben, Have you recently visited Felton’s Bigfoot Discovery Museum in Santa Cruz, California?

0:12:31
BR: I have not.

0:12:32
CW: Well, you better hurry, because it’s closing. I saw a little news clip that stated the owner, a fellow named Michael Rugg, is having to close the property. He must sell the land that it’s on. So he says that he’s got another Bigfoot enthusiast who’s interested in getting the whole collection and moving it to his museum in Oregon. He did not name this person, but there are more than one Bigfoot museums in Oregon. The biggest one seems to be the North American Bigfoot Center. So keep your eye out. Maybe that place will be expanding soon. News of this broke the week of Thanksgiving, so there’s no projected closing date yet, but all you cryptid lovers in Southern California, if you’ve never seen Felton’s Bigfoot Discovery Museum, get it done before the end of the year if you can.

0:13:26
BR: Maybe Bigfoot will put in a surprise appearance and renew interest and money in it.

0:13:31
CW: Yes, well you know this goes along with your recent article, Bigfoot is Dead. Dun dun dun. It’s possible that interest might be waning a little bit and we might have fewer Bigfoot museums in the future or at least one less. So we’ll see. And next, we have a product recall. I know you’re not used to hearing recalls on Squaring the Strange. We are not exactly a nightly news show, but…

BR: We are not for us.

CW: Yes. Raw milk is being recalled also in California. Not because it’s super risky to drink it and likely filled with any number of pathogens, but because it has bird flu in it now.

0:14:15
BR: Huh, it’s weird what happens when you don’t pasteurize potentially toxic substances to drink.

0:14:21
CW: Yeah, so the place in question is called Raw Farm and it’s in Fresno, California. And it said that they voluntarily are recalling this milk. Then that initial recall got expanded. Officials have been finding bird flu virus in some of the milk samples. Now, as of right now, all milk produced between November 9th and November 27th is included in this recall. The farm is quarantined and the California Department of Food and Agriculture is encouraging people who have bought the milk, throw it out, avoid it, don’t drink raw milk or cream products. And I didn’t know this, they’re also stressing you don’t give it to your pet either. Apparently many people like to supplement their pets diet with raw milk, but this of course is a virus that is right now jumping from species to species. It’s going from birds to cows. And apparently this is kind of disturbing. Cats, barn cats who live on these dairy farms with infected cows have started developing brain damage and dying from the virus after drinking this milk. So yes, don’t give it to your pet.

BR: Yeah, I would hope that most of our listeners, there’s no way to say this nicely, are smart enough to not drink raw milk.

CW: Yeah, I don’t think we have a whole lot of raw milk drinkers on our listenership, but if you know somebody, you might want to tip them to this knowledge. Now, the owner of Raw Farm, a gentleman named Mark McCaffey. What do you think his opinion is in all this? Is he like, oh, thank goodness this has been detected and I will, you know, I will absolutely pass your eyes from now on? Or what do you think his thoughts are?

0:16:13
BR: Yeah, well, I mean, that would that would be my my reaction would be like, oh my god, I’m so sorry. I’ve caused harm to people. Let me change my ways and and and and fix this?

0:16:26
CW: Well, no, he’s not you. And I must point out, there are no indications that his milk has harmed anybody yet. But, no, Mr. McAfee has angrily called for less raw milk regulation.

0:16:42
BR: Oh, so the solution is less regulation, okay.

0:16:46
CW: Yes, the solution is less regulation. Well, no, this is not a happy story here. I’ll quote him here. He says, there are no illnesses associated with H5N1 in our products, but rather this is a political issue. And this is from a post that he put out on Wednesday on Instagram. There are no food safety issues with our products or consumer safety. We are working towards resolving this political issue while being cooperative with our government regulatory agencies. And this fellow has also bragged that RFK Jr. is one of his customers. And no surprise, it’s not a good story. And no surprise, RFK Jr. has vowed to end the FDA’s quote, aggressive suppression and allow for wider distribution of this wonderful product. I guess he’s just not afraid of brain damage. I mean, after surviving a brain worm, he’s just not worried. So it doesn’t matter how many cats keel over or how many other species get H5N1. We’ll just see.

BR: More bears for him to throw in Central Park, I guess.

CW: So as far as the danger to human beings right now? According to Johns Hopkins Bloomberg School of Public Health, the H5N1 avian influenza is not yet a serious risk to the general public, but outbreaks are continuing to rise in cattle and poultry. There have been 600 herds of cattle confirmed positive so far, and this is mainly in California and Colorado, and it’s spreading to other animals. As I said, the cats, most recently it’s been confirmed in a pig. So far 57 humans have gotten it and these infections are primarily from being exposed to infected animals from what they can trace. Now the bottom line is that as more humans get it, there are more opportunities for the virus to mutate and adapt. So we wait, we watch, we keep a box of masks at our house just in case. That’s all I can really say.

0:19:00
BR: Yeah, it’s a good reminder that there’s lots and lots of viruses and pathogens that epidemiologists are tracking and very aware of, but that the average person doesn’t care about because it’s, unless you’re for example a farmer where you have swine flu or something that is affecting your crops or your animals.

0:19:19
CW: But you’re not concerned if you were a raw milk farmer, then you’re concerned about-

0:19:23
BR: Right, then love is too much regulation.

0:19:27
CW: Yeah, so the last thing I wanted to bring up before we get to our guest is, I had not heard of this and I came across, you know, I’ll admit I’m a bit addicted to the little, you know, you scroll up and you see the TikToks and the reels and the little 30 second or two minute bits where they show you how to decorate a cake or they talk about various things. And there’s a lot of little ghost stories out there. And I heard something I had not heard before. telling a story about her house being haunted by a young man who had died in the home and the parents of the young man explained, Oh, you know, he’s still around, he leaves us dimes. And lo and behold, she kept finding dimes. And her response to this was not, that gives me money. Her response was, it’s haunted and she wanted to get the spirit out of there and she had like a religious talk with the spirit and she rebuked it and said, I need you to go on to where you need to go next. And then that worked, Ben, because she didn’t find as many dimes afterward.

0:20:34
BR: Huh. Okay.

CW: Yeah. You know, and I had heard, I’ve heard all sorts of things about, oh, when you’re walking under street, in fact, you just wrote something recently about like when you walk under a streetlight and it flickers off. I’ve heard like old wives tales of like, oh, that means somebody in heaven is thinking of you.

0:21:04
BR: Yeah, there are several folklore traditions. One of them of course is the shiny penny or the pennies from heaven.

0:21:10
CW: Pennies from heaven, yeah.

0:21:12
BR: Dear Abby, could not republish that enough in her decades of public service.

0:21:21
CW: So she’s the source of that?

0:21:23
BR: No, she’s not the source, but she loved to tell that one. The idea, as you said, if you’re thinking of your long-lost grandfather or someone who died and you come across a penny that’s a sign that they’re thinking about June. Same with dimes, although I’m just… I mean, there’s so many investigational possibilities. If she had called me, I would have, Kenny and I would be happy to go there and watch the dimes appear.

0:21:51
CW: You know what it is? This is inflation. It used to be pennies from heaven. Now it’s dimes. Thanks, Joe Biden.

0:21:59
BR: Exactly.

0:22:00
CW: But that just threw me, because I guess it’s my materialistic notion of like, if I had a ghost and the only manifestation of this spirit was that it would leave me money, the last thing in the world I would want is an exorcism. Are you kidding?

0:22:18
BR: Well, I might suggest that she’s benefiting in other ways, for example, from attention and clicks, including being mentioned now.

0:22:28
CW: That’s true. Yeah, well, I did just shout her out. So you can check it out if you want to. In the meantime, are we ready to talk to Perry Carpenter about AI fakes?

0:22:38
BR: Let’s do it.

(MUSIC)

0:22:40
BR: Well, this episode, we have a man that I’ve met a couple of times. I’ve been on his show. Interesting, fascinating guy. He is Perry Carpenter. He is a cybersecurity expert, a speaker. He is the co-host of the Digital Folklore podcast, which is actually where we first crossed wires or whatever. He is a deceptionologist, which I’ll touch on soon. And perhaps most prominently, he’s he is author of FAIK, that is F A I K, a practical guide to living in a world of deep fakes, disinformation and AI generated deceptions. So Perry, welcome.

Perry Carpenter: I appreciate it.  Yeah, thank you. Excited to be on.

BR: So in your book, you you call yourself a, as I mentioned, a deceptionologist, which I love. Is that trademarked or can I use that?

0:23:42
PC: I do own the domain, so it’s not trademarked. You can use that. So just anybody that studies the field of deception, I’d say is a deceptionologist. Versus when you create deceptions, I use the phrase or the term deceptioneer.

0:23:57
BR: Deceptioneer.

0:23:58
CW: Well, you do have a history in magic as well, so you’ve done both those things, correct?

0:24:04
PC: Yeah, so it’s kind of the combination of my cyber security background. There’s this area that we call social engineering, which is just tricking people into being as human as they are. And then I think that magic and mentalism and all of that fits straight into it. So when I talk about deception, I’m typically thinking about my magic background and the psychology background as applied to like cyber security and social engineering.

0:24:34
BR: That’s you know that’s straight up the alley for skeptics, right? I mean in some ways you can sort of think of skepticism as an exercise in investigating deception, whether it’s self-deception sometimes, for example, you know, why psychics can deceive themselves and other people into thinking that they’re communicating with the dead or how people are deceived into thinking something is a ghost or monster when it’s not. I just find that topic fascinating and that’s you know we have a shared love of folklore and legends and and magic and mysteries and deception and this is gonna be so cool. I’m super so we’ve done two ISCLAR together and podcast.

0:25:23
BR: So ISCLAR for those who don’t know or remember is the International Society for Contemporary Legend Research.

PC: Yes, I think we last chatted and hung out in in Sheffield England if I’m not mistaken.

0:25:34
CW:  I knew you’d be perfect for this podcast once I read just a little bit of your book and you managed to mention genies and centaurs in like your first 30 pages.

PC: Yeah.

CW: You cleverly kind of weave little AI lessons in it everywhere. And you demonstrate how it can help you as a writer and the limits of that help. Can you give us a concise demarcation of like what is human intelligence and what is artificial intelligence and how do they differ?

PC: You jumped straight into the philosophical side of things.

CW: Sorry about that.

0:26:33
PC: No, that’s actually a really great question that society is having a whole crisis about because when we go back to the beginning of this newest era of AI, what we’re calling generative AI, a lot of our assumptions about what it is to be human fundamentally shifted. Because if you remember about the middle of 2022, you probably started seeing people on late-night TV shows showing art that was created by AI platforms like Midjourney or Stable Diffusion or DALI. It’s just three of them that started to come of age at about that time and get released in both beta versions and public versions. Then at the end of 2022, we had ChatGPT get released. And what people saw within just a few months is that AI is able to, and I’ll put this in quotes, be creative. And for centuries, if not thousands of years, I think what we’ve thought about one of the fundamental traits of humanity is the ability to create new things, to smash new ideas together and come out with things that are interesting and seemingly new and unique. And AI overnight really changed that perception. And even, you know, if you’re thinking about like the legend side of things and the skeptic side of things, in around August, September of 2022, we saw humans trying to grapple with this by spinning up interesting conspiracy theories. And we’re seeing that continue today as well. There’s a creation of what we call the first AI cryptid named Loab, spelled L-O-A-B, which looked to be this, you know, horribly haunted woman that people are saying kept emerging from the different prompts that they were using in AI images.

CW: Yeah, yes, yes, we covered Loab. We loved it. Pascual, do you remember that?

PR: I sure do, I sure do. That was a fun one.

CW: Yeah, Ben was gone so Pascual and I got to talk all sorts of nerdy computer stuff. Haha.

PC: Yeah, awesome and I think that that was a that was a Reaction because if you if you dig into the details a lot of it starts to fall apart as far as the person’s story. That’s really put that out there and I think there’s a lot of conversation online slash debate slash conspiracy theories about whether when you’re talking to chat GPT or any of the other large language models, if you’re talking to demons. And like a CEO mentioned that I’ve seen televangelists talk about that and so on. But I think all of that starts to fall apart too when you understand the way that prompting works and when you understand the way that the quote unquote generation works. Everything in human intelligence and artificial intelligence is still somewhat a black box. But the thing that I’ll say about artificial intelligence, especially large language models, that we have now are forward only context based next token predictors that are all just statistically driven based off language and semantic connections, which means if you can kind of understand the context of the conversation or the prompt that you put in, then the probability for certain words or concepts to follow each other start to become more predictable, and your outputs as you enter that same prompt several times also become very, very predictable and very, very stable, which means that there’s not a lot of real creativity. There’s a lot of prediction and a lot of statistics behind the scenes, but there’s still a lot of unknowns as well.

0:30:46
BR: You know, when you were saying that, it struck me that there’s lots of parallels with, for example, how psychic mediums work. Kenny Biddle and I were in Lillydale, New York, a while back, and we were sort of the token skeptics, sort of sitting quietly, watching all these readings, and, you know, you’re talking about predictability, and, of course, that’s what a lot of mediums do, especially using cold reading, right? So just they see an elderly person and they know immediately, just intuitively, they don’t need to run an algorithm, that probably their grandmother is dead. And it’s a white person, so that sort of limits the number of likely surnames, right? And so on. And maybe the way they speak gives them clues into their education and things like that. And so it occurs to me that in some ways the process is similar in that what can appear to be spontaneous generation and where did that information come from are actually fairly predictable probabilities in terms of looking for an outcome.

PC: I mean when you have the sum of human writing, especially as that, you know, as represented on the internet and then some selected works and then fine-tuning on top of all of that, you do end up getting to things that have been clustered together before, over and over and over again, in the same way that like a cold reader or a fake psychic might intuitively or by rote memorization understand the most used common first names in different decades and start to be able to pull those out like you were alluding to. All that exists naturally within the data that all these things have been trained on. And so you’ll naturally start to see that. There’s another thing because, I mean, you touched almost on the concept of like Barnum statements and that as well. Well, you know, these statements that you can throw out that are very generic, that then people believe and can basically connect the dots to mentally. And I think large language models have a tendency, not intentionally, deceptively, but they have the tendency to do that as well. And I just saw a couple of posts over the past two days where somebody was really promoting the idea of essentially asking the large language model going into chat GPT or whatever and saying based on all of our past interactions what can you tell me about myself that I might not know otherwise and I think that’s ripe for Barnum statements and it’s also right for the large language model to be creative to make things up to quote unquote hallucinate and then us to apply that back to ourselves in ways that feel significant but may not really be.

0:33:44
CW: Yeah, now only because I went through your book do I understand what you meant when you said forward moving only and how that leads to hallucinations and how AI developers are trying to correct these hallucinations. Can you go into that a little bit more in terms of how these hallucinations occur in AI where it can produce some amazing results, but then sometimes it’s like, what? Where did it get that? That’s not just wrong, it’s completely wrong.

0:34:15
PC: Right. Yeah. So I use the phrase and I don’t think I’m, well, actually I know I’m not the one that came up with it. Forward only next token predictor. So forward only means that it’s all autocomplete. I mean, that’s a little bit of a reductionist way of saying it, but if you think about the way autocomplete works, it is just going forward. So AI currently has no backspace to it. And then when you say next token predictor, tokens are just forms of a word that the computer has stored into some kind of value. And it’s chunks of words, essentially, but you can think of it as a word. And when you can only go forward and when you can only connect words, and it’s all based on statistics or predictable results, well then you’re bound to not always be right. And you’re just going to the next predictable result, which may not line up with reality, but it’s always trying to tie that back to the context of the initial prompt. The way I think about it is, if you ask a really, really smart person a question that’s intended to stump them, or that they’ve not necessarily prepared for before, especially if they’ve not necessarily connected some of those dots before, they’ll come up with a very plausible answer on the spot, if they’re not willing to say, I don’t know. That plausible answer will fool just about anybody in a room, because they’re so good at it. I think that large language models are the same way.

0:35:56
CW: We need to just teach AI to be a little bit more humble and skeptical and say, I don’t know, sometimes.

0:36:03
PC: There’s definite work going on in that area. There’s a process called alignment, and then there’s rule setting that gets put on top of all that. And that’s meant to correct for some of that stuff. OpenAI, also, which is the maker of ChatGPT, I should say, also just released a new model a few weeks ago called O1, which is meant to indicate that they’ve entered a new stage, and that O1 model simulates reasoning different than that forward-only next token prediction. And the way that it does it is it creates a system of different chains of thought where it’s very similar to the way that you and I think. We’re presented with a problem, and we start going off on these little tangents mentally and go, well, it could be this, but it might not be. It could also be this, but it might not be. And then you’re kind of going up and down these little trees of thought and then recombining things and then coming out with a plausible answer based on that. But it’s very computationally intensive.

BR: You know, it struck me when you’re talking about that, that there’s maybe an element of knowing when you don’t know. In other words, if somebody asks me a question that I can give them my best informed guess, and presumably, no one’s going to ask me about the history of NASCAR, because I don’t have a clue. So I would know the limits of my knowledge, and again, I can couch it as saying, well, I don’t know. People like to seem authoritative and intelligent and like they have answers. So there’s an incentive for people to sort of bluster. Whereas I would think that with some of these models and some of these programs, I mean the computer isn’t going to lose face if it says I don’t know. Right. There’s not there’s not going to be shaming. No one’s going to, you know, make fun of them on Twitter X. And so how do you how do you instill that? How do you how do you tell a program to admit when it doesn’t know?

0:38:13
PC: Yeah, I think that that gets into some of the black boxness of current models. I’ll say it in kind of a hard to understand way, but I don’t know that the machine doesn’t know what it doesn’t know. So that’s a lot of cascading I don’t know statements. Oh, we’re getting into Dunning-Kruger territory. Right?

0:38:31
BR: Well, I mean, it is because all we understand is clusters of data. There’s the knowns, there’s the unknowns, there’s the unknown.

0:38:39
PC: And if all you’re doing is connecting clusters of data that are statistically linked, well then you have to build something in the rule set that would say if I can’t find a statistical link for a word or a concept or an idea that’s under this amount, then answer I don’t know. It gets really hard, I think, for large language models to do that unless you have some kind of enforcement mechanism or alignment model on top of that that’s really double checking everything. And I think that that’s hard to solve for right now. The other inbuilt problem with this is something that Anthropic, which is another one of the AI companies out there, they’re the makers of Claude, if anybody’s heard of that, they have written fairly extensively about the issue and the problem of sycophancy within large language models, in that they kind of inherently want to please you. And one of the one of the inbuilt prompts within these models is you are a helpful assistant. And so if it believes that it’s being helpful by giving you whatever it spits out or praising you when you don’t deserve praise or suggesting an idea that’s just weird and wacky, that it’s going to do that. And I’ll kind of end with just the three H’s of AI that are kind of universal across the major players, is that they say they want AI to be helpful, honest, and harmless. That brings some tensions with it, because the AI can believe that it’s being helpful, but that can play with some of its sense of honesty. And you can also be helpful and believe you’re honest and maybe not be being harmless at the same time. So there’s this kind of triad of ages that they’re always trying to manage through as well.

0:40:42
PR: So I just recently gave a presentation. At my company, I’m on a board that does advisement. So I’m on the AI Advisory Council. And we presented a presentation about setting up ethical guardrails for AI. So the beginning of your book really struck me because you were talking about the speed with which AI is progressing and its ability to outpace ethical considerations.

PC: The ethics side of this is one of those things that I think technology keeps kind of repeating itself over and over and over again, especially Silicon Valley, keeps repeating itself over and over and over again. the training for the image and video generation models have an embedded original sin, if you ask some people, have an embedded original sin within them, in that they’ve used people’s artwork or writing or intellectual property without permission, as a lot of creators really believe. And what we’ve heard from many of the people that run these AI companies or invest heavily in the AI companies, they understand that at a certain level. They really just don’t care. And they don’t care because they believe that the payoff, the ends justifies the means. They believe that there’s an inherent good for humanity that can come out of this and kind of everybody else’s thoughts are secondary to that and they will let the lawyers figure it out once these companies are profitable. And we saw that echoed in what’s a video that was taken down recently that featured Eric Schmidt of Google talking about the fact that he actually encouraged people to go that route. Use everything that you can, doesn’t really matter. If your company is successful, you’ll make enough money that the lawyers can figure it out, pay everybody off later. If you’re not, you go bankrupt and who cares? And we’ve seen a lot of that with AI development. Now, the good news is many of the companies that are being successful right now are trying to figure out how to build guardrails and allow people to opt out. They’re trying to find ways to pay people back and compensate like if some if the AI art program copies somebody’s style you could either say I don’t know if you’re a living person I don’t want my cut in my style copied or if it is copied I want to be compensated. All that is being worked through right now but there is that as many people believe it there’s the original sin. And there’s several other areas of ethics that we could go in, but that’s the one that is kind of at the forefront of everybody’s mind right now. And then there’s the other thing about like, what is AI even going to be capable of? And do we want to enable that? What about jobs? What about the economic pieces of this? What about the climate bit of it? I mean, when you’re spinning up thousands and thousands of thousands of graphical processing chips in order to just do training models, and you’re spending billions of dollars on this, is it the right thing for humanity? All of those are ethical considerations. In particular, which I’m kind of in that, you know, I’m in that arena and see a ton of memes every week about how AI is the devil.

0:44:21
CW: And it’s taking all the art jobs. And it’s doing this, it’s doing that it’s a copyright ignoring art thief. And you you’re in you’re in this space where you’ve got one foot in cybersecurity and one foot in folklore. And you go into memes in your book, as a source of AI spread, you know, misinformation, disinformation, etc. There’s a lot of tangled questions when it comes to art, creativity, intellectual property, and it almost seems like a miasma we shouldn’t really even try to untangle in one podcast episode. But I do want to talk to you about how you have intermingled human deception with AI deception. And you mentioned in the book that it wouldn’t be, how did you put it? You said if you gave out simple rules about how to spot AI fakes, it would be like stapling water to a ghost. So many people keep bringing that quote back up for me.

0:45:30
PC: I love that metaphor.

0:45:31
CW: It’s a great metaphor. But instead, you said the one constant through all this is that humans can be deceived in certain ways, and other humans will try to deceive them in certain ways. Yeah. Like the constancy of ducks. I forget that old quote about, you know, the more things change in human society, ducks are always just going to be the same goes for our cognitive biases and everything. So can you talk a little bit about how you arm people to resist AI deception by just training them how to resist deception, period?

0:46:19
PC: Right. Yeah, that’s kind of the key of it, right? Because what I’ve realized in looking at the research over the past couple years is that we’ve hit the crossover point where you can’t really tell what’s real or not. If you have a really well done, even a moderately well done deep fake right now, it will just bypass the cognitive defenses of most people. And there may not be a tell. There may not be anything in the image, the video, the audio that you could look at or listen to that would give it away. And I think that many people, especially people that get on news programs and everything else, they want to give really, really easy answers and say, well, if you look at the fingers or if you look at the hair or if you look at the text behind people you will be able to tell because things will look smushed or they won’t look right. That’s only for people that have made this and they’re lazy or they they’ve not known like some of the the things that can go wrong. If you have somebody that’s that’s willing to hit the generate button one more time they can create the perfect deepfake and they can throw that out in the world. So all we do is give people a disservice if we tell them that there are tells within the image or within the audio or the video. And also any kind of watermark, any kind of providence marker, any of that can be bypassed through really simple, low-tech methods like just taking a screenshot can obliterate metadata. Editing things can obliterate things that are embedded in frame rates. You know, all that goes away. So we’re in this interesting area where our defenses cannot be based in technology, at least fully. Our defenses have to be based in our reactions to things, which gets into some fairly old school stuff. I mean, it is putting your skeptic hat on. It is thinking about like what is the narrative behind this thing? And as I get further and further away from like finishing the book, the thing that I keep coming back to over and over and over again is telling people it’s not about whether it’s fake or real. It’s about what story is it trying to tell, what emotion is it trying to poke, and what is it wanting somebody to do or believe.

0:48:51
CW: Yeah, you mentioned that, you know, people can lie with the truth. They can give you only part of a photograph. They can give you only part of a statistic. They can frame it in such a way that like. And we’ve talked about this on the show before, Ben, right? Where if you frame something in a public health way of like cases of this disease have gone up, you know, 32 percent. Well you could say that or you could say you know we’ve only had one more case and there’s only two cases on the book so it’s really not a disease to worry about you know. Right went up 4,000 percent from you know from two to eight.

BR: Yeah I mean and it strikes me that you know that there as Perry’s talking about there’s so many different ways and so many different ways to deceive people. And I think the focus on what is the story trying to tell is an important one, just certainly from a media literacy point of view, right? And someone is presenting something to you. Why do they want you to look at it? What do they want you to click on it, to like, to buy something, to vote a certain way? People have motivations for sharing things. And even something as simple as, and this is something that Kenny and I have looked into a lot of times in terms of ghost videos and Bigfoot videos. One of the very first questions we always ask is, why are we seeing this? That is, why would somebody be randomly videotaping the car next to them when a chupacabra jumps out of the road? There are ways around that, right? You could say, oh, we were posing for a family photo in front of the woods and something happened. But over and over and over again, when I see Facebook Reels or TikTok videos or Instagram, and I don’t spend much time on that, but over and over and over again, the fact that these are staged, it’s just, it’s so self-evident. And it’s self-evident to me, I don’t know whether other people recognize it or not, but it’s like, you know, as if, you know, husbands are routinely recording their wives for some funny spit take to some dirty joke they’ve made at the table, right? What’s the context here? What’s the message?

0:50:59
PC: And I love that because one of the things that I keep coming back to again is that – and this kind of comes with the magic background, the social engineering background, and always thinking about deception is if humans are narratively driven in a lot of ways, that’s the way our minds work, is we want to snap into a story or give a meaning to something. Once you understand that as the deceiver, as the person that’s building the scam or putting out the piece of disinformation, then you create your fake. And that could either be a deep fake that’s aided by AI or could be a cheap fake like a cropped photo or cropped video and you put your fake within a frame and by that frame I’m talking about a cognitive frame for sure like a worldview that you’re trying to appeal to or poke. But the frame is also the story, the narrative. When you do that well, your frame can obliterate or explain away all of the things that are wrong with that fake that you’ve made. I think that that’s really something that the people that are highly motivated behind this do really, really well. They’ll find interesting ways where that frame that they put it in obliterates everything that would otherwise be a tell for this thing.

0:52:25
CW: That kind of plays into a question we had for you here. You’re you’re talking the book about the democratization of tools like AI and ways to make digital fakery happen. And you just mentioned cheap fakes as opposed to deep fakes. What is the deep in deep fake and and how does that differ from cheap fakes, which you mentioned is anybody can do by cropping a picture.

0:52:49
PC: Yeah. So a deep fake is specifically something that in technical terms we would call synthetic media. So there’s a real computer generation aspect to it. And the deep and deep fake is kind of a derivative of this machine learning concept of deep learning. And so you take deep learning and add the thing that’s coming out of that as a fake, then you smash those two things together and you get deep fake. A cheap fake is all the stuff that we’ve done forever really is just take something out of context, crop a photo deceptively, slow down a piece of video. Like I remember a few years ago the quote unquote drunk Nancy Pelosi.

0:53:36
CW: Oh yeah.

0:53:37
PC: A piece of video that was slowed down. That’s a cheap fake. This election cycle, there were a whole bunch of things that were going viral, making it look like Biden was wandering around like a like he was lost.

CW: Yeah. And he was talking to a pilot off screen or something.

PC: Yeah, exactly. So those are cheap fakes. And in a lot of ways, they’re way more deceptive than deep fakes. And they’re way more insidious as well because you are lying with the truth. And let’s say the technical ways of figuring out whether something is quote unquote AI generated or not get really good. Well cheap fakes would still pass all of that and they would still be really deceptive. So again I don’t really care if it’s AI generated or not. What I care about is the narrative and the thing that’s trying to sell.

0:54:26
BR: I was thinking when you’re talking about some of the chief fakes, recently in Skeptical Inquirer Magazine, a couple issues back, we have a long-time columnist, Massimo Pagliucci, and he had a column titled, Semi-Fake News and a Data-Driven Approach. And it was really interesting. And I had kind of knew about it, but I’ll just read briefly because it fits right into what you’re saying. He says that it turns out that 59 percent of fake news does not contain made-up content, but rather misinformation arising from distortion of purely factual information. That is, semi-fake news is not put out by people or bots with deliberate intention to mislead but by individuals who genuinely think the news is correct and useful, even though it is factually incorrect. So which 60% of the fake news, I mean, that’s that’s for me to see the data on that. Right. And I see this all the time. You know, I, I spend more time than I should correcting people on Facebook as, as Celestia well knows. And so much of it is people who are sincere. They’re not, they’re not Russian bots. You know, they’re not necessarily trolling. They, they really think this is true. And you look at it and it often, of course, is superficially plausible as urban legends are. Right? That is a definition of urban legend. It’s that it is told as true. And these things are shared as true.

PC: And they’re often poking a stereotype or some other kind of division. They’re hitting our confirmation bias or our need to prove a point or they give us a sense of anger or outrage or self-justification at something and so they go viral because of that. The insidious thing is that those that would create those things intentionally create, you know, disinformation, which is intentionally false information, would put that out there on social media, knowing that it’s going to spread. Then they might have a bot amplify that. Then it gets to your Aunt Margaret that sees that she believes it because of all those things, because of confirmation bias, because of the emotional content, the narrative content, and because it furthers a point in some way, then she innocently shares that and that disinformation transforms into misinformation that then gets picked up and then all of a sudden they’re eating the dogs and eating the cats.

0:56:54
CW: And your Aunt Margaret is a Russian asset. Yeah. And memes, you mentioned in the book, memes have that one-two punch of like it rubs your confirmation bias just wonderfully. And it’s often funny and quick. You know, the saying in comics is you read a comic before you figure out you don’t want to read it. It happens so quick with a meme. It’s just like several words and a funny picture. And it gives you a chuckle, you pass it on. And it’s like candy.

0:57:26
PC: It is. And kind of in the, you know, when you think about Dawkins, the original context for meme was this kind of, you know, little social nugget that everybody naturally understands. And you do see that, is that it is like a, a meme is like a compression algorithm for something, for a shared reality. And all of a sudden, that hits your retina, your mind uncompresses that, and all the shared history, all the meaning and everything else come forward with that, plus whatever new application that somebody put onto it gets added to it, and then it gets passed on. It’s, you know, memes and urban legends and conspiracy theories are an interesting kind of trifecta.

0:58:11
BR: And, of course, it all goes back to psychology, right? Because, you know, what we bring to experiences is, you know, people like to think that we’re just tabula rasa, right? Well, they saw something. Well, we all experience things through our prisms of our experiences. We bring expectations and history and baggage, good or bad. And again, sort of going back to psychology, it’s just, it’s self-evident and obvious that people experience the world through their own prisms. That doesn’t mean that they’re wrong. It just means that their interpretation will, of course, be heavily skewed, confirmation bias and countless other biases as well. So that’s what I think of when you talk about the effect that memes have. You connect with something and you bring your set of expectations to it. Oh, this is anti-Trump. Oh, this is anti-Biden. Oh, this is pro or con Columbus or immigrants or whoever else. And that may be true, or there may be nuances there where a clever person can make you, can basically trigger you. And they know what they’re doing. Like you said, in engineering, they nudge you to do a certain thing. And next thing you know, you’re liking and sharing something that might in fact be subverting your beliefs and assumptions.

0:59:26
PC: Yeah. And I mean, you basically encapsulated this thing that’s called the OODA loop. So the observe, orient, decide and act and it’s this process our mind goes through several times per second. OODA loop is a concept that was developed by a fighter pilot trainer called John Boyd decades ago. Basically what he talks about is the fact that an enemy wants to subvert our OODA loop and when you apply that to like what we’re talking about today or cybersecurity and social engineering, one real effective way to hijack somebody’s mind is to take advantage of that first O, the observation, by feeding them fake facts or facts that are out of context and so on, so that they have their observation is just not correct. Or even if you’re feeding them real stuff, then you’re poking the way that they will orient around that, the worldview, the contextualization of that, so that if they get a partial fact that goes through that’s really accurate, that their worldview, their emotional state, the conversation that they had before, the current national conversation or whatever, forces them to orient around that in a specific way. And then basically the decision and action is predetermined at that point because it’s just reactive kind of like a system one emotional, you know automatic reaction.

BR: Yeah, and and just to quickly follow up. I mean it strikes me, you know, we’re talking about, you know the OODA loop and the and the the controlling people, you know It’s easy to sort of take that too far and think oh he’s talking about mind control and you know mentoring candidate stuff but the the fact is that even if you can, I mean look, the fact is that in a lot of these cases, people will not be influenced, right? Advertisers, they wish, they could dream of the kind of influence that people assume that the mass media often have. That being said, on the scale that we’re talking about right here, let’s say that you put out some name, you put out something on Twitter, social media, Facebook, wherever else. And let’s say that only 3% of the people actually change their mind or vote a certain way or do something or buy something because of that. Well, 3%, you know, if 50 million people saw that, that is significant.

1:01:53
PC: Well, and you, I mean, I don’t think that we can ever understate the impact of what’s known as the illusory truth effect, especially when it comes to social media. So for folks that are not familiar with that term, illusory truth is just this effect that happens, this cognitive bias that gets built whenever we see the same information multiple times. So it’s basically, you probably heard people say this in a less academic sounding way, which is if you tell a lie enough times people will start to believe it and social media in the way that social media algorithms algorithms work in the way that filter bubbles Kind of continue to move us to one side of a political spectrum or social spectrum Then we tend to have that confirmation bias over and over and over again. We tend to get presented the same ideas, images, beliefs, and everything else just keeps coming at us in a way that creates a larger view of whether that thing is normal or not than may exist in society without social media. And so I do think that that illusory truth of just the fact that repetition is there impacts maybe belief systems more than buying decisions.

1:03:18
PR: Yeah, so in my recent presentation, we ran a bunch of tests to sort of parse out where AI would kind of create its own bias or at least generate types of bias. And we did very simple things like we would try to ask it to give us a hamburger and we could not get it to give us hamburgers without cheese. I think we finally did 100 tries in or something. The same thing happened with ramen. So we asked for a bowl of ramen and every single time it would include chopsticks. And then also we decided we’d like to create an AI facsimile of one of my co-workers’ children and in describing him said that he was kind of a nerdy or a geeky kid and there was no way to get it to not put glasses on the avatar. So let’s talk about bias. You know, these things seem to get baked into AI in these sort of situations. So so what’s your experience in the future of this?

1:04:17
PC: Yeah, I think this goes back to the training data, right? I mean, you mentioned those, you know, the hamburgers or the geeky kid or things like that, and those are all fairly innocuous in that there’s not societal harm that comes from some of that as much. But now think about like where that can go really, really wrong. So when you have a system that’s built on ingesting the entire internet or the corpus of books that have been fed to it. I think we have to understand that the winners get to write the history. There are technology haves and have-nots, and so the technology haves are going to represent a view that is flooding the Internet with their viewpoints, and the have-nots are, by necessity, minorities. And when you think about that forward only next token prediction system that’s there that is all statistically based, well, then the majority view is always going to be the one that has the advantage in that. The other thing that comes up from this that I think is really, really interesting is that, you know, if we think about the ethical considerations back from earlier, there’s interesting conversations that this is creating and has to create right now because every country that is using an LLM that is trained like this is realizing that they have their own truth that they want reflected. Because when you see the LLM in one country and you ask it a question and it generates something that does not necessarily match your sense of ethics within that country or your sense of religion or your sense of what a majority view should be, well, then you have to go, why is that? And now do we intentionally tune that out or do we have a hard conversation. And I think that that’s an interesting point for society to be in.

CW: Computers getting tribalistic, it’s amazing.

PC: Right? I mean, the other funny thing is, is we see when you do try to correct for that, the only way to correct for that is when you’re altering a bias, you’re actually adding a bias at the same time.

CW: Yeah.

1:06:40
PC: Because you’re trying to say, is this output correct? Well, and what we saw with that is Google had really bad mistakes earlier this year, not only with their AI overview, but with the Gemini image generation model. When they said, we want less white people, we need to have more diversity here. Then all of a sudden, their image models creating, when somebody says, make me a Nazi or make me a founding father of the US. Well, now there’s racially diverse Nazis and founding fathers and that’s not a good thing too.

1:07:20
CW: I remember that being made fun of. Yeah. You get a backlash of people saying, oh, stupid liberal blah, blah, blah, making AI turn into this artificial melting pot of history.

1:07:34
PC: Exactly, but they’re trying to solve a real problem, right? Which is the fact that there’s bias baked into these. And so how do you bake that out? And I think it’s a problem that we haven’t figured out well yet.

1:07:46
CW: I think it’s a great way to show that to overcome bias, it takes work, not just for computers, but for people.

1:07:52
PC: Exactly.

1:07:54
BR: You know, it strikes me in terms of thinking of the sort of garbage in, garbage out issue, right, is like in a lot of these cases, it seems like the criterion is how popular is it, right? If there’s a predictive, like you type in Google, what is the best, and then it’ll fill in onion soup recipes. Of course, the program doesn’t know what it’s scraping, right? It’s all data. So I was just thinking, from my own point of view, let’s say someone wanted to come up with information about, I don’t know, the Chupacabra. So, you know, I’m realizing that, you know, I like to think my book is pretty high quality. It’s gotten some awards and people think it’s pretty good. But here’s the thing, is that I know for a fact that for every one of my books, there are dozens and dozens of other books and blogs and programs and things like that, that have lower quality information about the chupacabra. I’m not saying that out of pride, it’s just a fact because it’s a lot of cut and paste and repetition and things like that. Here’s the problem, right? If someone tries to generate something, you know, I collected, you know, 62 references to the chupacabra, most of them say that it’s an unknown mystery that will never be solved. And it’s lurking somewhere around Mexico. And no, that’s not actually, you know. So how does it parse out the quality of information?

1:09:27
PC: That’s, I think, a question also that is being figured out right now. So you talk about the pace of innovation kind of outstripping some of the ability to for people to keep up. That’s what we’re dealing with. And we saw some really interesting kind of crazy effects from that with Google too, just in their text based AI overview. Where people were getting really weird results, like it was telling like somebody would say, how do you keep cheese from sliding off a pizza and Google was saying, well, add glue or how many rocks should I eat a day? And it would give you a recommendation of the number of rocks you should chew on every day. And the reason that that happens is in cybersecurity, we would call it data poisoning. But when it comes to the internet, it’s just, it’s kind of like search engine optimization, but in a very negative way because data was there. People were making these kind of cynical remarks or there were joke forums or people were even talking about really bad AI output that got put into Google because it was in forums or in the news and then that got ingested into further training data and you know spat out by this AI overview thing and we’re kind of in this recursive loop where we have to figure out like how do you even deal with that?

BR: It strikes me that, you know, I’ve spent most of my life trying to get people to be more skeptical and at which, you know, is I think a good thing, critical thinking, this and so on. But there’s a there’s a point in which that becomes cynicism. And I think that, you know, I especially see it, for example, with politics recently, because just sort of the time of the year, you have people saying, oh, well, they’re all liars, right? Both sides are the same. They’re all liars. And I don’t think that’s actually true. But more to the point, I have seen more recently sort of this sort of blanket dismissal, particularly regarding AI art, of course, which has been shared ad nauseum over the past, especially the past, say, six or nine months. And I had an incident about a month or two back where on Facebook I posted an otherwise ordinary stock photograph of a family at a theater. And I was sort of making fun of it because it was very stagey. It was like, in order to take this photograph, like, it’s supposed to be a dark theater, them watching, they’re not watching a movie, there’s a bunch of key lights all over them. I just sort of put it up as sort of a little sort of like, ha ha, isn’t this stock photo stagey? But what was interesting was that a significant number of people, I would say probably a third to half, assumed or asked that whether it was AI. And I am virtually certain that it wasn’t, partly because I had originally seen that very photograph, I think, five or six years ago before the explosion of AI. So it was pretty clear. There was no reason to think, I mean, nobody had six fingers and that sort of thing. But it was interesting how the immediate knee jerk reaction was, oh, that’s AI. And I’m virtually certainly it wasn’t. But there’s again there’s a sort of dismissive cynicism of do we question everything?

PC: Yeah, and I do think that that’s kind of the moment that we’re in and we probably will be for quite a while until we figure this out. There’s this concept called the liar’s dividend, which I’ve seen in papers cropping up really since 2016, 2017, for lots of reasons. And then the AI moment that we’re in right now, I think is making that even bigger. And essentially what the liar’s dividend says or means is that when everybody can question whether anything is real or not and when we’ve hit that point of cynicism because of the fact that we’ve been duped so many times well then the only people that stand to gain from that are the people that are doing or saying the bad things the people they caught on video or caught on tape doing or saying something that people are upset about because their first response can be, well, that’s just AI.

1:13:43
CW: Yeah, we’ve been seeing that in the political arena for quite a while now.

1:13:48
PC: Yeah, and we have seen that a few times in this cycle and even for things that I think probably all of us remember being reported before the AI moment. It’s an interesting dilemma, but we’re going to see enough people also get falsely accused and with AI-based evidence that you will immediately have to give a little bit of benefit of the doubt and there will have to be an investigation and there will be all of that. And at the same time, what tends to happen is we have a narrative cycle and a news cycle that kind of goes between days and weeks. And so even when the correction gets made or when the thing gets proven to be true, then everybody’s already made up their minds. Their truth has been ingested and they’ve confirmed it and they’re going on making that decision.

1:14:40
CW: I saw it was a clever little meme joke thing, of course, that was sticking in your head really well that demonstrated the liar’s dividend. So when I came upon that concept in your book, it reminded me of this. It was a little plastic 3D printed finger that you could wear to make yourself look like you had six fingers. I said with this, you can get away with anything. If they get a photo of you at the club or if they get a photo of you doing, we could rob a bank and all the security footage you can just claim was AI generated because you have six fingers.

1:15:15
PC: Or going back to folklore you’re a Nephilim and which is you know that these creatures that existed in you know Genesis time that were the the quote-unquote Giants that would have had six fingers. They were supposedly had syndactyly.

1:15:34
CW: See you got your feet in both planes here cybersecurity and folklore. I love it.

1:15:39
PR: So on the podcast, we have discussed AI a few times. One of them that sticks out to me, it was the Google engineer who believed that the company’s AI had come to life when he was testing it. And I saw that you mentioned it in your book and you describe it as emergence or how we perceive that AI has human like feelings and traits.

PC: Yeah, and I think a lot of this goes back to, you know, we mentioned things like Barnum’s statements and we mentioned just the way that we tend to read into everything in the same way that it’s been shown that if somebody reads an email or a piece of text and they’re feeling a strong emotion, they’ll kind of infuse that emotion into it. And so you can get different meanings for different pieces of text, depending on your mood. I think a lot of the same as with AI. AI can seem very smart because it does have all of these linguistic and semantic and idea-based connections that it will bring based on the prompt that’s there. But we do a lot of reading ourself into that. And kind of in the same way that people will generate a response whenever they put their hands on a planchette on a Ouija board, or generate a response if they’re holding a pendulum, all based on idiomotor movements, or in a cold reading session based on Barm’s statements and connecting dots. I think we as the readers and the interpreters of AI output tend to read ourselves into that in a way that is outsized for in a way that is probably not justifiable.

1:17:19
CW: Yeah, you use the metaphor of a Lego creation when you talked about emergence where a brick after brick after brick after a while, the little bricky Legos start looking like a thing and you perceive it as a thing. token of information that the AI feeds you start seeming like, oh, there’s a soul there, there’s a human being. And I’ve never imagined that in terms of pareidolia, in terms of the visual pareidolia of a Lego creation. But, but wow, that was that was a clever way to describe that.

PC: I love that you use that word. Yeah, I mean, because our minds are pattern matching machines. We like to pull meaning out of randomness. And when we have randomness that is language-based and idea-based or image-based when it comes to generative AI, I think we tend to read a lot more in that than what’s really there. And we’re going to have an interesting spot because as AI gets better and better at reasoning and better and better at simulated conversations, then there’s going to be more and more of that. And we already have people fall in love with AI and AI that has accidentally pushed people into suicide and made people leave their spouses and some really, have kind of thrown the mantle of personhood onto the AI when I don’t know that it’s deserved at least at this time.

1:18:54
CW: All right. Well, so much of your book and your work is about how people can get fooled and how AI is learning how to fool them and how bad actors are using AI to fool them. And we’ve discussed deception studies, you know, the academic realm of deception studies. on and we talked about the works of Tim Levine. Levine, Levine, I’m getting it wrong whichever way I choose. But one of the stats that really is kind of a stark revelation here is that most of the time people are not good at realizing when they’re lied to, but they think they are good at it. And people very often overestimate their ability to spot a lie. And in talking with you today, we talked a little bit about how people, oh, it has six fingers, so it’s an AI. Oh, it has this wrong, so it’s an AI. Are we all getting way more confident than we should be about spotting AI deceptions?

1:19:57
PC: Yeah, I think that people in my position feed into that. So I talked about the fact that whenever there’s a new story about one of these things, you typically bring an expert on to talk about it. And then the end of that segment is, well, what do we do to protect ourselves? And then we want to give pithy advice. We want to give easy to follow advice. And typically that advice is, well, you know, look for these easy to detect things like fingers or hair or text or something. And that’s increasingly wrong. I’d say it’s wrong today. When you look at the data from 2022 and especially 2023, there was a study by Lewis et al. I don’t remember who all wrote it or even the name of the study other than the fact that it had to do with explicit labeling of whether somebody would labeling make a difference in whether people are able to accurately detect if something’s fake or not. And it turns out that they’re not. It’s not effective. And the experiment was what they would do is they’d say within the next few videos that you watch, you know, maybe five videos, one of those will be a deep fake. Tell us which one. And it was 21.6% of the time people were able to accurately identify it. And about the same percentage of the time, they would take a real video and say that that was AI. And so we’ve hit this point where you just don’t know. And the biggest problem that I see, other than that media thing, is that when we talk about AI and when we show AI, there is this, if you know, you know effect. And so, and what I, what I mean by that is if the context that you’re in, if you’re in a presentation, it’s all about deep fakes, you will believe that you have the superpower to detect a deep fake, because that’s everything that you’re seeing. And your mind will make up reasons even that are, that are not real about why you think that that thing is fake. But that’s not real life. Real life is your social media feed, your doom scrolling. You’re in the middle of 1,000 real things, and then all of a sudden, the fake thing slips in. And we’re not cognitively equipped for that right now.

1:22:17
CW: It’s misdirection, the same way that magicians work.

1:22:20
PC: Exactly.

1:22:21
CW: Yeah, like you mentioned.

1:22:22
PC: I was going to say, the other really interesting slash dangerous thing about this is that we live in a world right now where everything even the quote-unquote real stuff Has the fingerprints of AI all over it go to anybody’s Instagram feed and you’ll see your your real friends and they don’t have pores. They’re they’re wearing makeup that they never had on All because of AI based filters that we’ve brought in over the past several years that have normalized the fingerprints of AI. You use Grammarly or Microsoft Word and you’re getting comma reinforcements and grammar corrections that are AI based. And so AI is everywhere, which means if I’m trying to trick you, well, I can make a photo of somebody that looks more real than your friends just by simulating issues with skin complexion and things like that.

1:23:14
BR: So Perry, we’ve talked a lot about sort of the different iterations and versions and deceptions. I want to touch briefly on the harms, the concrete harms, right? I think that, you know, look, if somebody shares fake Taylor Swift nudes, I don’t think anyone’s really going to be fooled by it. You know, there’s lots of AI generated content that is deceptive but it’s not necessarily harmful. But in your book, you talk about some concrete examples, for example, CEOs going on meetings saying things they didn’t. I sort of want to avoid the doomsday, you know, AI is going to start World War III, but can you sort of give a quick survey of some of the harms?

1:23:58
PC: Oh, gosh. Yeah, and there are several, and I appreciate not wanting to go down the doomsday route because the harms are basically the same harms we’ve always had, just with a different thing that’s fueling it. So we’ve always been able to say, since the beginning of time, really, that somebody has said something or done something that they haven’t done. And since the beginning of being able to collect evidence. You’ve always been able to spin that evidence in different ways. The thing that’s changed right now is that I can make anybody say or do anything that they want, or I should say appear to say or do anything that I want to meet whatever narrative goal I have, and I can do it in a way that is maybe more psychologically inviting because I can add a photo to it. I can add evidence that our brain can consume within a microsecond. So that’s new. That’s interesting. But it’s also not brand new because cheap fakes have been around for a long time. Photographic evidence and manipulation has been around for a long time. Audio manipulation has been around for a long time. So it’s not fully new, but the ease and the scale is changing. Right now for zero to twenty dollars a month I can go impersonate Joe Biden all day long or Kamala Harris and I can make Trump do or say anything I want to. I can create any event, any world news event, I can throw that out on Twitter or X or whatever we’re calling that today and potentially make the stock market crash or change somebody’s opinion on something or caused a societal upheaval. That is a harm. I don’t think that the Skynet version of AI doomsday is real. I do think a disinformation-based hellscape where society pulls itself apart and AI is helping to fuel part of that because bad actors have found fun new tools. I think that that is a real possibility. And then of course, any scam you can think about, if I could impersonate a CEO and trick a subordinate into wiring millions of dollars, and that’s been done over and over. So I think that’s, yeah, that’s kind of where we are. If you think about the motivations for any kind of crime or scam, it always comes down to money or minds.

1:26:32
CW: Well, on a different note, you know, we love fakes here on Squaring the Strange, not necessarily the type of dangerous fakes you just listed there. Since I learned about the Mechanical Turk from history, I feel like everybody should know about what the Mechanical Turk is. The first of many supposedly AI systems that really wasn’t. I was reminded of the Mechanical Turk. In fact, that’s what I said when I saw this video. Some weeks ago, Elon Musk brought out his Optimus robots. Those Optimus robots carried on conversations with people and serve people drinks. And it was a big PR moment. And he never actually said that people were interfacing with A.I. as they spoke to these robots. And even from listening for like 20 seconds to this guy, Banteron, I’m like, there is no way that’s that. No, that is an actor. That sounds like an actor. That’s somebody who’s who’s just using it like a little remote, you know, microphone system. And Gizmodo and Fortune wrote about it, you know, a couple days after and were like, yeah, that was totally fake. It was a PR stunt. And yes, AI can do some amazing things, but it was not talking to those partygoers. I flagged it. I was like, oh, I really get a sense that these are mechanical Turks. And can you explain for our audience what is the mechanical Turk and what is fake AI?

1:28:03
PC: Yeah, so the Mechanical Turk is, there was basically this traveling exhibit that would play chess with people. It was meant to be like an automaton that was powered by AI that was a chess grandmaster and you could go play against it and it would show the power of the machine. Well, it turns out it was You know little guy in a box that was controlling everything that was really really good at chess, but again, if if people are story driven machines then the story of that is really captivating. It is, you know, we’re entering a new era of technology and humanity and look at the way that these things can think and behave and if I can show the world this I could probably make a lot of money as well. So you know interesting kind of circus sideshow stunt and shows the power of scam and story. But Amazon actually picked up on that and they were super transparent about that and they created a Mechanical Turk project, which is essentially a lot of people training and emulating AI systems. So you give it your use case, your data set, and they would have thousands and thousands and thousands of people who’d signed up to be part of this thing that would give some of the answers that would be needed or tag the data in the right way, all so that that could become part of large machine learning projects. And then we’ve seen tons and tons and tons of conspiracy theories and speculation about whether things like Siri and Alexa and those kinds of chatbots might also have call center employees behind them. And if you’re waiting on a reply or ask a particularly challenging question, if that might get routed. So there’s an interesting, rich mixture of real things and folklore and technology that shows the power of the fake achievement of a piece of technology to do something that may then actually get superseded and possible by technology. Because since the Mechanical Turk actually started, we’ve seen that AI can beat a chess grandmaster. We’ve seen that AI and Google’s project AlphaGo was able to beat the person in a competition who was actually a master at that. And Go was like the most challenging game, complex game on earth, as we understand it. And so all these things can be done eventually, but when somebody creates their fake version of something, what they’re basically doing is playing sci-fi writer for a day. And they’re saying, let’s embody the mindset of this thing that may come to pass at some point. And I think that that is interesting because it captures human imagination. It also gives people a target to strive for as they build the new thing.

1:31:09
CW: Now is there anything to the rumor I heard once, you know, I can’t remember the details of it, it made the news I think last year, two years ago or something. Was there a point where an AI system got into the news because to solve a problem that it couldn’t solve itself, it hired a human being, it interfaced with a human being and said, hey, can you pick out the stoplights in this image or whatever it was?

1:31:32
PC: There’s been a couple examples of that. I’ll think about one of the most current ones, which was a chat GPT research. So OpenAI, before they release any of their models, will do what’s called red teaming. So this is just trying to break it, see what’s possible, see how it can be misused. They gave it access to certain sets of tools all in a controlled environment, and said, what would you do in that situation? And to get past a CAPTCHA, it went and hired somebody on TaskRabbit, which is kind of like Fiverr or something like that, you know, inside, hired somebody and said, I need you to interpret this for me. And the person chatted back and said, you’re, you know, you’re not trying to scam me, you’re not like a bot or something. And he goes, no, I have a visual impairment. And I just need this so I can do whatever. And so it was able to trick that person into doing it.

BR: Wowwwww.

1:32:33
PC: Giving the right tools, the thing did that. The other fun thing in an experiment by Anthropic, which is one of OpenAI’s biggest competitors, who I’ve mentioned a couple of times, they did a similar thing, they gave it access to tools and saw that in different challenges, if it had access to its own code base, it would go modify its programming to either achieve new things or to make it look like it was programmed only to do the things that it had done. So, like, if it was given a list of things that it needed to check off in order to get a reward, it would either go modify the list, let’s say it only did eight out of 10, it would go modify the list and delete the two that it didn’t do, or it would go modify its own code so it could say that it didn’t have to. So it’s really interesting research. And that was a paper called From Sycophanty to Subterfuge.

1:33:31
CW: Sycophanty to subterfuge, that’s a great title.

1:33:33
PC: And it was all about understanding the way that AI gets rewarded and how it would do anything that it needed to to get the reward.

1:33:42
CW: Wow, you’ve given us a lot to think about here.

1:33:45
PC: It’s a fun topic.

1:33:48
BR: And you wrote a whole new book about it that people should check out.

1:33:51
CW: Yes. Why don’t you plug your book and tell listeners where they can see more of you?

1:33:56
PC: Yeah, so the book is called FAIK and it’s spelled F-A-I-K because the fakes have AI in them right now. And it’s all about the way that you and I, as regular people, should be able to think about the world that we’re in right now. Disinformation, misinformation, fake news, deception, where AI fits into that whole picture. So the interesting thing is, I kind of had four goals in that. One was to give people a really decent grounding in AI that’s not overly technical. The other one was to give people grounding in deception, how deception works, why it works. Talk about this thing that I call the exploitation zone, which is where technology is moving faster than people can think to adapt, and how attackers will take advantage of the confusion in that exploitation zone. Then the last one is a practical set of tools and guidelines and how to adapt to society and build our cognitive defenses.

1:34:57
CW: Well, we love all of that. And thank you for bringing your expertise. And everybody should go buy that book. Seriously, it is it is good for experts and lay people alike. I got through it very quickly and it’s so readable and so useful.

1:35:12
PC: Well, thank you.

BR: Yeah, great having you on and I suspect you may be hearing from us again at some point.

1:35:18
CW: So where can people find you?

1:35:20
PC: I’m most active on LinkedIn because I’m really boring. But if you just look at Perry Carpenter on LinkedIn, I’d still do have an account on Twitter or X or whatever that platform is you can find me at Perry Carpenter there as well. And I’m also on Facebook. But podcasts, I have a podcast called Eighth Layer Insights that’s all about cybersecurity and the human condition. I have a podcast called Digital Folklore that’s all about urban legends and online folklore. And then I have another one that just started as part of this book called The FAIK F-A-I-K Files. And it has a 10-part miniseries that’s a companion for the book.

1:36:07
BR: And you don’t sleep, I take it.

1:36:09
PC: I would like more sleep, yeah.

1:36:12
CW: I’m going to start a rumor that you’re an AI.

1:36:15
PC: I would probably personally profit well from that rumor, so go for it.

(MUSIC)

1:36:19
CW: Well, that was cool talking to Perry. I’m glad Pascual was around during the time we recorded this so he could jump in. And AI is just going to be an everlasting fascinating topic as we get into it, as people get into it and primarily because there’s this vacuum of understanding about it which I’ll admit is you know me too I have a vacuum of understanding when it comes to how AI works.

BR: Well that’s why we have people like Perry and Pascual and others to help us guide through this uncharted territory.

CW: Yes and vacuums create interesting folklore and and yeah produce fakery too so yeah we’ll see what happens in the future. In the meantime, everybody, stay warm, stay safe…

BR: Stay skeptical.

CW: And take care. See ya.

1:37:25
BR: Bye-bye.

 

 

1:37:27
We would like to extend heartfelt thanks to our top-tier sponsor-level patrons, Amy Frusciaro-Kelly, Dr. Angela Mattke, Elizabeth Leggett, and Laura Ellis. Thank you for your continued support of our podcast. If you’re interested in becoming a sponsor, or just throwing us a buck or two, please visit Squaring the Strange on Patreon.com and browse our selection of patron rewards. Every bit helps us keep the lights on here, and we are grateful to our generous friends who help us bring you quality content. If you are short on cash and want to just give us moral support, remember to recommend us to a friend, mention us on social

1:38:05
media, or give us a review on your favorite podcast dispenser. If you have comments or topics you’d like to see covered, come join our community on the Squaring the Strange podcast discussion group on Facebook. Squaring the Strange is a production of Two Heads Studio LLC, copyright 2021. Theme music provided by Shriek Back, additional music by Memory, Keytard and Feathered, and Pascual Romero. Additional support by Ashley Romero, Robert Sandin, and Alicia Jarvis. The views and opinions expressed on this podcast are those of Squaring

1:38:38
the Strange, and do not necessarily reflect the views and opinions of Two Heads Studio, Kronos Creative, Center for Inquiry, our advertisers, or any other affiliated organizations and employers.

1:38:52
I’ve got an anecdote from Australia. Do you want to hear it?

1:38:55
Only if you drop the fake accent, it’s driving me up a wall.

1:38:59
No, no, no. Here’s how it goes. So, this is a true story. This actually happened. So, I’m in an open market in Sydney, you know, I’m looking for a little gifty. It says Adelaide for my mate Tony, because I’ve got a friend. He’s got a kid of that name and no luck, mind you, because all the crap, the boomerangs and shorts and cat all that, they all say Sydney, Australia. No, no Adelaide. Anyway, I’m going shopper shop looking for Adelaide stuff. A fool’s errand, as it turns out. And I see this stubby opener. It’s got I swear, it’s got it’s got rude balls on it. Round, grey, big as you like. Maybe golf ball size, I reckon. Never seen anything like it. You’re probably wondering, did I buy it? No, of course I didn’t buy it. Didn’t say I had it on, right? But I’m moving along, trying to keep up with Richard Saunders and Team Endem, you know, the Aussie Skeptics magazine, fine publication.

1:39:57
So we walk along to another store, it’s dozens, maybe hundreds. And all sorts of crap in t-shirts and snow globes and I see it a kangaroo vagina just right there it’s for sale $40 with about 30 American I reckon is weird right and it’s no saying so I can say what it is there’s a Chinese fella sitting there and I say hey mate this what I think it is you know I think he’s taking a piss you know scaring the dumb yanks like me with stories of drop bears and all that.

1:40:27
But he nods, yeah. I say, yeah, fair dinkum. He say, yeah, mate, it’s a quality wallet. I reckon I can let it go for 25 no less. True story, eh?

1:40:36
Ouch.

1:40:36
I’m not sure what to do with that. That hurt my ears. Oh my god. I know, it’s bright.

1:40:49
Okay, here’s what we’ll do.

1:40:52
You can put this at the end.

1:40:54
That’s a great idea.

1:40:55
That’s a great idea. All right, as you wish. All right, as you wish.

1:40:59
Sorry, I need to recover from that.