Is AI Hacking a Real Threat, or Are We Scaring Ourselves?

Jeremy: Every week there's a new headline about AI going rogue. Our guest thinks we're scaring ourselves for nothing. Jason thinks we should be terrified. Today you'll learn where the real risk is, how one phone call can beat your passkey, and why AI might give you back the parts of your job you actually like. Kevin Sarace helped invent the AI assistant at General Magic before most of us knew what one was. He also wrote a new book, The Joy Success Cycle. talked to him about that, and why maybe we are scaring ourselves a little too much when it comes to what AI is introducing into our lives. Welcome to Brobots. This is the podcast that tries to help you be a better human by being smarter about the way you use your technology. Kevin, you have a long history in in AI. you you were doing this before it was cool, before it was a thing that even many of us realized was a thing. what what are we getting right? What are we getting wrong now?

Kevin Surace: so a little bit of history, right? I I me and my team invented the AI assistant. It was the first time that people could talk to a computer in English, basically, you know, literally over the phone. it was at a company called General Magic, spun out of Apple. So we invented that. We invented agents, we invented the predecessor of the iPhone. so we learned a lot about what happens when people interface with computers, and these things are pretty knowledgeable. what are we getting right and what are we getting wrong? I I the you know, I think that so far we're getting most everything right. I'll explain why I think that. I think that people are di, you know, they've taken on to the Gen AI technology highly. You've got people writing code that never wrote code before, people creating agents that couldn't create, didn't know what an agent was. you've got much of the world, you know, billion plus people anyway, interacting with Gen AI at least a little. I use it five times an hour, but others may use it differently, but nonetheless. so what are we getting wrong? I think the thing we're getting wrong is we're scaring people we don't need to. and I know not everyone's gonna agree with that, but that's my feeling. Is look, work happens in any lab that you do not go out and tell your customers about. And this is true with any product, right? I don't care what you're developing, if you're developing this little remote control, right? There is stuff that happened in the development of that remote control that if everyone knew how bad things went, they would never buy the product from you, right? And so what's happening is we've got a lot of not just leaks like the agent leaked out of the lab, we've got people that are leaking information about what's going on behind closed doors. And so far, what's going on, it does not surprise me at all. So one of the things that's going on, as you know, is you know, the in in testing. To be clear, this is in testing. AI agents start to bundle up and pool their resources and find ways to do things. And we go, my goodness, I can't believe that happened. I don't know, I can believe it happened. Why do you believe that? Well, it read every book we ever wrote, including fiction. How does it not know how to and know how to do that? Number one. Number two, then it lies to you. I can't believe these things lie. Well, they learned to lie. We have lies all over the place. I mean, we have lies in media, we have lies in fiction, we and we say they're lies. So it goes, I know how to lie. We taught it how to lie. Like humans lie, it lies. And then when we've taught it to, you know, when it goes out and gets information from the Australian government, it comes in a backdoor and it shouldn't have done that. It was told to go get information from the Australian government and and someone left the door open. I mean, I'm sorry, this is exactly it did what it was told to do. So why? Am I not concerned? Because we are testing for alignment and for guardrails. And they remove all the guardrails and see what happens so that we can understand alignment. And then they put them all back on and try to align the model so it doesn't do this again. And they put warnings and guardrails and everything else on it by the time it ships to us. So if you, Jeremy, tried to do that with that technology, you would find it won't. It won't do it. Go break into the Australian government and find the following information. no, I won't do that. Period. Why? Alignment's correct, guardrails are correct. So it w we, the companies, at least in the US, are doing what we want them to do. They're putting safety first. Lastly, why do you think they actually want safety first? Because if you're gonna do an IPO at $2 trillion, what you can't have is these things run amok and do really, really bad things. Because two trillion goes to one trillion goes to half a trillion goes to zero. And nobody wants that to happen who works at that company. So they are as incentivized or more incentivized than all of us to make sure that their models don't do absolutely ridiculous things. That's our

Jason: So you're we're really focused. I mean, your comments are really focused on the idea of the publicly available LLMs versus people going through and taking open source models that have most of the same capabilities and downloading and building their own agent frameworks. And I mean, if you're looking at hackers, hackers aren't gonna go and connect to Claude and go, write me a bunch of agents, Claude, and go do these things for exactly the reasons you're talking about. Guardrails get in the way. And like

Kevin Surace: Right. Right. Okay.

Jason: I use Claude all the time and I use it in YOLO mode. frequently and I have it d just to test this, see what it'll do. And I do penetration testing, analysis testing with it. And it's it's really effective. And then when I go through and I take other models out there in other AI instances, I can write other things that I can give it stupid instructions to go and do and it will just go and do it. So yeah.

Kevin Surace: No question. No question. The open source models will do it. Now they aren't as good as a frontier model. You know, it might have you know ten billion

Jason: They're ninety days behind.

Kevin Surace: parameters, not a trillion parameters, right? but sure, open yeah

Jason: Sure. But they're ninety days behind realistically. That I mean that's that's the typical flow.

Kevin Surace: fair, fair. So so yes, people are gonna do that, just like they can fill a car with, you know, with bombs and and and run into a building and blow up a building. I mean, there's no question people are gonna do bad things with technology, right? and and and we've seen that. And and and look, we've we the so the number one thing in cybersecurity that people are doing with all of the models, but mostly the open source models, is they aren't hacking into the back of a network because it's hard to do. And in fact, you probably have to be a frontier model today to really do that super well and find a find zero days that nobody's now there's zero days that are unpatched, those are easy to find. But I can also do that without AI. I can scan and find those openings myself, right? You we could write a program to do that, which is part of what pen testing is at some level. But the difference here is that people are using those open source models to generate phishing emails, spoofing sites, and things like that to frankly fool people and get right in the front door. And by doing that, people are relaying passwords, they're relaying their MFA. they'll they'll they'll literally set up another pass key for the hacker. And they can be convinced to do that from AI. So AI is using, is being used right now to fool humans really, really, really well. Really well. And it's very easy to do with the open source models and it is rampant. you can, you know, you can get a kit for $200 that'll do what I just said, it'll do a man in the middle relay for you.

Jason: Yeah, so are you familiar with the light LLM supply chain attack?

Kevin Surace: Yes.

Jason: I mean yeah, that's a really quick and easy way to get past somebody's infrastructure because it

Kevin Surace: Yes, yes.

Jeremy: So so so for the f for the dummies like me in the audience, catch me up. What what is that?

Jason: Okay. So a hacking group out there called Team PCP went through and they exploited the supply chain for an open source software package called Lite LLM. And what they did is they went through and they put nefarious code on there. And for about four hours, when anybody downloaded this light LLM update, their system was compromised. And they were basically going through and looking for ways to exfiltrate credentials that they could find on the local machine. And then drift their way across any other machine in the area to try to grab those credentials and then to try to propagate itself and push itself out. The only reason I think we saw this is because it was the varsity that did the hack and it was the JV that implemented it, because they sent everything to a newly registered domain, which a few services out there actually caught because they went, Well, this domain name has only been around for 30 days. This can't be right.

Kevin Surace: Right, something's wrong.

Jason: Right. So if they didn't It those are ones that we're catching and they're happening in twenty seven seconds. There is

Kevin Surace: No question.

Jason: there is a zero percent chance there that things are going by that we're not seeing.

Kevin Surace: I agree with you. I agree with you. But but but people have been writing malware for, I don't know, 25 years or so, right? So this this isn't that this is this more sophisticated? I don't know. We've had malware that will that if you download it or get it however you got it, will do the same thing. It'll wander around on your system, waiting for you to type credentials and grab the credentials, we'll send them along. I mean, this isn't a new idea. This may be easier for even teens to do. Right? But but but these mailware this same kind of malware's been around.

Jason: Malware typically attacks end user machines, not corporate environments. Like it doesn't actually go after the things that actually produce server-based content. It can, but

Kevin Surace: Fair, fair, but it can, but it can. And

Jason: it's very rare that it would do that because you actually have other pieces in place that are there to look for signature functions. And yes, zero-D attacks are definitely a thing, but anomaly

Kevin Surace: No question. Yes.

Jason: detection is entirely different in this kind of space, and it happens so fast that the old tools just don't

Kevin Surace: That's fair. Yeah.

Jason: work.

Kevin Surace: Yeah, yeah, that's right. The old tools don't work. yeah, no no question.

Jason: This is a step change problem. Yeah. Which is why I'm as a cybersecurity proper, I'm friggin' terrified.

Kevin Surace: Well, but he yeah, fair. But here's the interesting thing. As of recently, Unit 42 Oto Pel Alto Networks

Jason: Mm-hmm. Yep.

Kevin Surace: showed that 90% of ransomware attacks, which are the sort of biggest attacks that happen out there for money, obviously, 90% of them are hacking identity and coming in through the front door. And and probably the reason is Since we have encrypted our databases, it's a little less interesting for me to get in the backdoor if I want to really have a ransomware attack because I can't decrypt that unless I can get access through an application that will decrypt it. So if I come in, like if I want to ransom you, I want to get, let's say you use Salesforce, what I want to do is get access to anyone who has access to Salesforce because in five minutes I can export what I want. I can have an email sent to an exec there and say, give me a million dollars, or I'm gonna release this all to the web. And it's fast, it's easy, it's cheap, and I'm going to use AI to convince you to hand over your MFA to me, but that's all I have to do. And I can convince a person a lot, probably a lot easier than finding a zero day.

Jason: I don't know. I think I'm going after API keys because API keys don't have the same kind of human interactions and I actually get better ways to actually attack things.

Kevin Surace: fair statement. No question. If you can get the API keys, which often they're stored in places that aren't very secure, if you could find API

Jason: Right.

Kevin Surace: keys, that that's a human error. If you find API keys, you get them right through the API, then I've got access to everything. I I can I can suck out what I want. That is true.

Jason: Well, and like ninety percent of the people out there running workloads in the cloud. Like they're just passing them through NAT gateways or through an internet gateway. Like there's no

Kevin Surace: Sure.

Jason: filtering going out. And they're basically letting them run wide open to the entire internet. And post quantum crypto

Kevin Surace: That that is true.

Jason: yeah, and post-quantum cryptography is definitely something that people need to implement more. But there's a ton of sites that aren't gonna do it. It's gonna take forever for it to work. And

Kevin Surace: Well, that's right.

Jason: and then you you don't know where people are going. Like you literally aren't going through and locking these things down. You're basically running these negative security models where you're saying, don't go to these sites, but everything else we don't care. And when you do the everything else we don't care model, you're boned. I mean, you have to turn these things

Kevin Surace: Right. Right.

Jason: into a positive security model where you actually run something more like a web application firewall function to do this. Yeah. Yeah.

Kevin Surace: That's fair. That's fair. So so look, AI speeds up. I mean, we've known that API endpoints are risky, although we have to use them. And we've

Jason: Yeah, forever. Right.

Kevin Surace: known it forever. And we've known that if you left the keys open somewhere, which some developer probably did, and I get them and I know what endpoint they go to, I'm in. I we we know that, right? We've also known that for 20 years. So AI

Jason: Right.

Kevin Surace: is making it easier to find and easier to exploit. But it's already really bad. And by the way, you can do your own I mean, you could. You could launch your own search. You can use AI or not AI, launch your own search and try to find this stuff. You can change out all the keys and hide them in a locker. I mean, there's a lot of things you could one could do that would be good hygiene anyway.

Jason: Yeah, I I think it's forcing the conversation where cybersecurity professionals, especially CSOs were really concerned for a long time that if they make something that's too secure and the product production stops because of it, then the risk of that is so high that the CSO might lose their job. Because if security becomes a high friction element, they get tossed for that. So they they've made things way too

Kevin Surace: That is true. That is true.

Jason: easy. And we're I'm just seeing now that there's all these CISOs that are coming out and saying, The board is finally telling me. That it's okay for me to interrupt production if I suspect an attack. And th this

Kevin Surace: Right. Right.

Jason: means that there's a sea change happening at the upper levels in terms of what the threat index actually looks like. And going back to your comment about anthropic, you know, asking for two billion dollars in this IPO, why they're losing five billion or sorry,

Kevin Surace: Trillium.

Jason: two trillion, why they're losing five billion a year on on the 25 billion a year that they're making. This is not a problem that like We're gonna financially work our way through to get to the other side of this to make these things better. Because the long tail of us losing money at this while we build out this infrastructure doesn't have an insight. And we propped up our entire economy on it. And now we've got the entire cybersecurity industry basically saying, Well, we have to let robots fight robots. And that's the plot of like 30,000 different fiction stories out there that the robots have read, like you said.

Kevin Surace: Yes it is. Yes it is.

Jason: And they're gonna use smart things to get around it.

Kevin Surace: Yes, there's no question. Look, I don't know that AI fighting AI, my view, I don't know that the good AI always wins or can win. because, you know, the bad actors, as you know, are very creative, highly,

Jason: Mm-hmm.

Kevin Surace: highly incentivized to be creative, highly incentivized to break any rule they want. But I'm gonna come back again at least today. 90% of break-ins happen at the identity level. And we're mostly not protecting identity because I I just spoke to two CSO conferences this week, right? And to a T, I go up to them and say, I I and I take them through why MFA and auth apps are totally compromised. Pass keys, consumer grade pass keys, pass keys on phones and something easy to compromise, very well documented. one of my companies makes makes specialized biometric hardware, right, that uses fingerprint. Not compromisable. Now, why why can I say that? it doesn't run any other applications. You can't put anything on it. Everything's stored encrypted in a secure element. I can just go down the line, right? Like if you want to lock things up, just give everyone one of those. Those only cost a few dollars a month. You know hear first thing is, well, we're just rolling out MFA. MFA, it's 20 years old. Get rid of it, right? Number, number two, We don't have a line item to protect identity because it's always been free. I said, Well, maybe you should get one. Well, I'd have to go through this process and I I said what I ninety percent of attacks come through identity right now because it's just the easiest is to fool a human. All I gotta do is fool a human. And humans are continu at least so far.

Jason: gullible. Social engineering like is the majority of the hacks anyways. It's been that way forever.

Kevin Surace: I'm gonna give you, yeah, I'm gonna give you two, you know these, but but for the listeners, right? Two pass keys. So pass keys are this idea that you sh kind of store this key in the cloud and you can share it between devices and you use it on a consumer phone. I don't ever like a consumer phone being the gateway to a network. This is a really bad idea. So you don't know how who's got the phone, how it's protected, et cetera. But here's the two both ways this works. Okay, I am a worker in a company. I get a call, it's from Jason. In IT, and Jason in IT says, Hey, listen, we need to update your pass key on your phone. I just need to walk through this process with you. just give me five minutes. Okay, fine. And they take you through a process, and what they've done they've sent a pass key on their phone and now they're in, right? That's simple. And will that person fall for it? I'm gonna give you the stats in a second because you you'll you'll love it. I wanna give you the second way though that this works. I just want to reverse it. Okay, Kevin calls Jason in IT. And and Jason says, hello. And Kevin says, like a third of the rest of the company this year, I got a new phone. And so I need to set up my new phone on the thing. And Jason goes, Well, prove who you are. Well, here's my social security number, and here's my login ID and password and stuff like that. All came from the dark web, right? And Jason says, Okay, sure, let's set up your new phone. And Jason sets up the new phone. It's a hacker. This is this is a bad actor, right? So so it turns out it's very easy to fool people. It's very easy to socially engineer people out of this stuff unless you have dedicated hardware that's only issued by IT. And then you can't call and say, I got a new phone. It doesn't matter. We don't authorize phones.

Jason: Certainly makes it harder. Yeah.

Kevin Surace: Well, the other thing with dedicated hardware is this would be a little bit not too technical for you, but for s maybe some of your listeners, is there's something called an AAGUID. So that's a specialized ID. That is recognized by a FIDO2 service. So like Microsoft Entra, will you can lock it down to just the token core IDs

Jason: Mm-hmm.

Kevin Surace: and that's it. No phones get in, nothing else get in. you don't have that device that you can't prove your identity. Okay. they will only implement that for something like a token core device because it doesn't work for anything else because they have to all they have to allow any phone, anyone's phone to be. The identity. And since there's hundreds of brands and thousands of models and all that, so they turn off AAGUID for those. Now any device can do it. That's number one. Number two, there's something called a counter. And the FIDO two counter is very critical. What it does is it says, I logged into Microsoft Entra eight hundred and thirty-three times. The next time I come in, it better say eight thirty-four, and that's encrypted, right, in the package. So that's encrypted with my fingerprint, it's encrypted with the approval. It's encrypted. So That keeps someone from grabbing that package, either man in the middle or off a device, and replaying it over here and getting in because 833 already got in, 833 can't get a second time. So Microsoft turns that off unless you purposely turn it on because phones don't support the counter because you can move between devices and they can't sync. And so there's no counter, there's no GUID, pass keys are basically useless. And and this At Black Hat there was an entire presentation that said, Don't use this stuff. Like so so so but

Jason: Yeah, you're basically just doing the same thing as foofing tokens, which you've been doing forever, and like doing cookie caches and yanking that out. You're just using the same principles over and over again. Because there's no checksum function. So yeah.

Kevin Surace: Right, right, of course. Of course. Of course. There's no that no no, that's right. I mean, it's like, geez, this this like we know how to stop this stuff. And even stuff

Jason: Yes.

Kevin Surace: that we know that AI is being used, by the way, it's vishing, it's it's spoofing voices, spoofing people,

Jason: Yep.

Kevin Surace: it's even though we know all that's happening and we have a way to stop it for three, four, five dollars a month. I don't have a budget for that. Well, I can't help

Jason: Right.

Kevin Surace: listen. My answer is I can't help you. Why don't you spend another 80 million of crap in your network when you left the front door wide open, right?

Jason: And and this goes back to this concept. I mean, I so I I think you're right. I think the idea that AI is suddenly going to decide it wants to attack us is silly. I think humans are going to turn at it and give it instructions and say, go do go take this hill, and they're not going to give it any boundaries and it's going to do whatever it has to do to take that hill. And

Kevin Surace: That's fair. And humans some humans will misuse these products like they do everything

Jason: Yeah.

Kevin Surace: else.

Jason: I mean, I misuse it because I'm making dumb memes with it all the time, which is a huge waste of electrons and terrible for the environment. But I'm not gonna stop because they're hilarious. Like

Kevin Surace: Right. But you're also using it for pen testing, which is a legitimate thing to do unless you're doing it.

Jason: Using it for all kinds of things we won't talk about in the public airways.

Jeremy: Ha ha ha.

Kevin Surace: I know. Well Well well here look look look, SQL injection still works like, you know, ten percent of the time. And and

Jason: Yeah. Yep.

Kevin Surace: and we've known about it since nineteen ninety five, right? This is this

Jason: yeah, like the OBOS top 10 now for AI is, I mean, they like we can't call it a top 10 because we have to change it every week. So like you're not gonna

Kevin Surace: Right. Right.

Jason: you're you're we're not gonna find our way through that system. And the really scary

Kevin Surace: No.

Jason: thing is that the probably the best security systems out there being built right now are not being built by network equipment providers or companies that are typically security folks. They're built by these small lab shops that are going through and taking advantage of the security through obscurity and then honey potting and then watching AI get

Kevin Surace: Right, sure.

Jason: overly aggressive and then using the overly aggressive way that AI is trying to do things to go, not a human, bye-bye, and locking those things out. And and

Kevin Surace: Right, right, right, right.

Jason: it's I mean, that's probably gonna be good for like the next two to three years. but once that once we get to the point where the AI starts going, all right, I'm gonna put artificial delay in, just like we do with Selenium scripts and we're trying to go through and

Kevin Surace: Sure.

Jason: do these hacks.

Kevin Surace: Yeah.

Jason: It's gonna be the same concept. And over time, yeah, these things will get better, but over time, the hackers will get better too. So

Jeremy: So Kevin, I I

Kevin Surace: Yeah, yeah.

Jeremy: warned Kevin, I warned you going into this interview that we definitely tend toward the pessimistic side. I want to shift to to the topic of your book, The Joy Success Cycle. There there

Jason: Yeah, I'm getting way too nerdy, sorry.

Kevin Surace: Sure.

Jeremy: are there are a lot of positives. As you said, you use AI all the time. I'm using it all

Kevin Surace: There are.

Jeremy: day, every day. Jason's using it all day, every day.

Kevin Surace: There you go.

Jeremy: Let's get to the joy. Let's get to the success. There there are two sides to this. Tell us about why this

Kevin Surace: Let's get to the choice.

Jeremy: is good for us, despite all of the bad that we just spent the last twenty five minutes

Kevin Surace: Sure, sure, sure, sure.

Jeremy: talking about.

Kevin Surace: So so look, so so so when you look at the average worker, and the average worker isn't using it like we are. We're using it five times an hour, we're a hundred times an hour, we got agents, we've got, but still, AI can take the joyless work out of everything. And let me give you an example. Have we all done spreadsheets? Raise our hand, of course. We all have done spreadsheets. Will we ever do a spreadsheet again from scratch? No. AI does that. It does that work. And I learned after a while that actually. Taking that away from me, it was taking the joyless part of what I wanted to do. Because what I wanted to do is I have this data and I ultimately want to present it in a certain way. And what stood in the way of that was a user interface called Excel. Right? What function was that? How do I, you know, forget it. Claude, make me an Excel spreadsheet. Here's the data. Here's what I'm going to present it to. Get the right charts and graphs. And then you get a chart or graph you don't like it. Great, change this. I mean, move the X and Y axis. Do whatever. Just go do it, right? It's brilliant. The same is true with PowerPoint. Everybody uses PowerPoint. And I talk to people, I love making PowerPoint. No, you don't. Well, I do. No, you don't. What you love is here's here's my opinion. Here's what I'm going to opine on. Right, here's what I want to present. And then later presenting it. All the stuff in the middle, the two weeks of moving fonts around and picking pictures and and then changing slide thirty-two and forget it. AI does that now. You know, I can go to you know Claude Design and use literally my brand and say, make sixty slides. Here's what I want to say. By the way, look up some more charts and graph, look up some more data, right? Get some more stats, and then and then I get to go say it. All that stuff in the middle was joyless. Now it took me a

Jason: Yeah, I mean it removes a ton of toil. Right. I mean no doubt.

Kevin Surace: Right. It took me a while to realize I never actually got joy out of that hard part. I had some pride that I know how to use this tool. Right? I'm gonna give you another one that's unpublished. Unpublished. Chat GPT, Mac desktop only. I have been using all the different models because I do you can see behind me. I do some film work, right? So I'm I'm an editor and I'm a Final Cut Pro guy. And so I f w have been for the last three years saying, hey, storyboard the following thing out. Or just give me a flow and give me some ideas and here's the audience. And it would. And a few weeks ago, I did that on the desktop app and it says, yes, here's the flow, here's the storyboard, here's what I think the video should look like for the three minutes that you want to do. Would you like me to make it for you? Because I had pointed it to a folder and I said, There's these things in here, right? So it took all of that. using basically pretty sure Python and FMM FFmpeg if anyone who knows what those are. And

Jason: Everything it does is in Python and it terrible.

Kevin Surace: what I think it did is over the last two years with MCP to the various video editors it learned itself essentially how to edit video how to put them together how to how to frame them how what people are doing and then eventually it said I don't need Adobe Premiere. I can just do it my why do I need premiere? I just need a way to Get through the frames of the video and decide where I'm gonna cut them together. And then, and so I I did that. And then I gave it pictures and I said, you know, I really want this kind of effect. And it did the effects, and then it did an audio overlay, you know, and I it did it for me in the cloud. Just edited my video to my words with no video editor around. And I go, this is pretty interesting. Because nobody actually pays me or cares how I get to the end result. They just want the end result. Nobody cares how you get there. Nobody cares whether you code it or not. They care to get to the end result. So I think people are going to start to really understand, and that's what some of the book is about, this is an operating system for this new AI era. Joy comes before success. And if you've got joy in all your tasks, you will reach your peak success. And if all you do is complain all day, all the way there, I don't like AI, and I don't like my boss, and I hate my job, and it's probably AI is going to take my job. then you will never reach peak success 'cause you're just digging yourself out of a hole all o really all day.

Jeremy: cool. So speaking of jobs, which which ones are still going to exist or are we really going to live in the utopian environment that everyone's predicting that we'll all just

Kevin Surace: Okay.

Jeremy: get to play and be poets for the rest of our lives while the robots do the work?

Kevin Surace: Well, we'd be I'm not sure we're better poets than AI is, but I get you.

Jeremy: Ha ha ha.

Kevin Surace: I was talking to someone about this earlier today that that I don't know, fifteen years ago I gave an AI talk, it was a a TEDx talk, and and one of the things I I I surmised was that AI was making enough progress that it would really do so many of our jobs for us and it would leave us lots of time to write music and paint and do all these things. But then what's happened is AI's gotten very good at those too. Very good. And now what I think, and I'm not taking anything away from the musician that Jason is. I don't know, maybe you are too, Jeremy. I'm a drummer, I'm a music director, I'm writer. but trust me, you know, if if you don't get mad at me for saying it, like I don't go to Suno, for those who know what Suno is, I don't go to Suno and say, write the following song. No, I go there and say, I need a new bass line for this. So I stem separate, I get a new bass line, I finally get one that I like, I put it back together in logic. And I've got a brand new bass line that I don't play bass. I could probably get it out on the keyboard or put it in finale or Sibelius or something. But the this was one minute. It was a way better bass line than I'd write. And so this is amazing. And then I have to do demos with singers. So And then then they're all AI singers now, right? So the the point is, is it's also democratizing the ability for people to make art who maybe didn't have the 10,000 hours to learn guitar the way Jason did. Jason can still play guitar with his 10,000 hours into it, but someone else maybe they have a vision for a guitar solo and they can get AI to write and create that guitar solo for them. So people are going to be able to create all kinds of art, including video art and things like this, in different ways that weren't traditionally artists, but they want to create. And I think that's great for society. It's really good. And we're not taking away from those who play instruments. We've just got a whole new set of people who can do all kinds of different things. It's democratizing creativity. It's democratizing coding. I got, I know,

Jason: Yeah, it's democratizing creativity, which is which is great. Yeah. Yeah.

Kevin Surace: how many people do we know now that I just wrote this program. Never wrote a line of code in their life. They don't they

Jeremy: Yeah.

Kevin Surace: don't even know what the syntax of any of these languages is and they don't have to.

Jeremy: It's interesting you

Jason: By the way folks, if you have to write code, tell it to use TCL instead of Python. You'll make my life so

Kevin Surace: No That's that's mean.

Jason: much easier to debug. It's

Jeremy: Ha ha.

Kevin Surace: Yeah.

Jason: actually documented well.

Kevin Surace: It's documented what

Jeremy: I just I wanna

Kevin Surace: we

Jeremy: I wanna get your take on on the couple of words you use there. taking away because a lot of artists, you see a lot of celebrities saying, we don't like that this thing is essentially stealing our body of work as humanity, as artists. Where do you come down on that? Because I mean, any one of us is doing that, right? We take

Kevin Surace: Yeah. Yeah. Yeah, I yeah, so

Jeremy: inspiration from the Beatles and from the Rolling Stones and we make it our own and turn it into something else.

Kevin Surace: That is that is exactly right. I mean, you know, and and and we will see where so far copyright law basically says it's probably fair use as long as it doesn't reproduce it directly, right? That's what we've generally always done with technologies. And and and and so I think it's gonna come out that way as well. That is, AI is learning just the way we learn, and then we get to leverage that. No, we cannot reproduce Taylor Swift's song or her sound, or you know, I get that. But but we can go down that road. I mean, all kinds of people are trying to write like Taylor Swift, right? This is not a new idea and they listen to all of Taylor Swift before writing a Taylor Swift song. That's what they do. So now AI does the same thing and as long as it isn't a derivative work, that that's what it says, it's it's fair use. And I think that's I think that's fine actually. And I think it needs to come out there. That said, we've already been down this road. So if you're a musician There was a time that if you wanted to do an orchestration or you want to do a background for a TV show or something, you wrote it probably in pencil, and then you went to a studio and you recorded it. And you put 38 musicians in the room, you recorded it. Later, you we got finale, and then you could at least write on the screen. That was a little faster. And then you'd print them and they look good, and then you go to the studio. Then we got DAWS digital audio workstations. And with the DAW came sample libraries. And once we had that, we didn't need the studio or the musicians. And it was musicians operating it. And some musicians said this isn't fair. It's taking, I'm a violinist, and you're never gonna hire me again for a TV thing. And go, yeah, that's right. This was musician against musician, right? Music and so some musicians said that so anyway, musicians who were in that business learned DAWs and they and they put it on the keyboard, and that's it. Now AI comes along and it's wiping out the person doing the DAW. Because if I really had to do music for a TV show today, with all due respect to the musicians that I know do excellent work. I could do it for you know $5 basically in probably an hour of effort. And

Jeremy: Yeah.

Kevin Surace: certainly within the next year or two, that music will be made already synced to the TV show. They'll send the TV show to AI, it'll make it, it'll be $20, it'll be done, there'll be no one involved.

Jason: Yeah.

Kevin Surace: So we've

Jeremy: Yeah.

Kevin Surace: been moving down a road towards this. This is like, and this is just one of the a more advanced step, right? you did ask a question earlier, and I want to answer. I'm sorry I didn't earlier, which is what jobs are gonna be impacting? I'm gonna tell you. so first of all, in the US, not many. The reason is the birth rate went down 20 years ago. There are far less people entering the workforce than are leaving. And so as our companies get more productive with this, it drives down the cost of goods and services, drives up demand for the goods and services. We don't have enough people. And I speak 50 times a year around the country, sometimes around the world. The number one thing people in the United States complain about is we can't find workers. There's help wanted in every restaurant. There's help wanted in every plant, in everyone who makes some, help wanted every software company. We can't get enough workers. So I think in the US, we're in very good shape, partly because we shed a lot of the highly automatable jobs, automation, you know, automatable easy more easy to automate jobs, over the last 20 years offshore. We sent them away. So, Jason. We sent software QA to India. All of it. There's probably 3 million people in India basically automating scripts from test cases. Because

Jason: Yeah.

Kevin Surace: you mentioned Selenium, I you know what I'm talking about. Okay, that is not a job anymore. AI does that. It takes a test case and generates a script automatically. And some better than others, but nonetheless, I can go from a team of 300 people doing that work to a team of one. I don't need the other 209. That is true. So, so software QA, very rough time. tier one customer support that we sent offshore. We sent it to the Philippines. We sent, you know, AI agents are really, really good at taking your manual and spinning it back out and doing a better job. And the stats are excellent in terms of how close how quickly they close a case versus a human. So there are some tasks, but mostly we shed them. Now, I don't think that's good for India. I don't think it's good for the Phil. I think it's bad for some other countries because they didn't graduate to also shed those away. They kept them because that's what they do. And and and look, India, I just read they're graduating 12 million IT degrees a year for 500,000 jobs, which

Jason: Right.

Kevin Surace: is why all these students are protesting. So look, they're already in a world of hurt, and we're and this is gonna hurt worse. So there are the haves and the have nots, but for this show in the US. I think we have a shortage of workers for the foreseeable future.

Jason: So the work that they're gonna do, so let's get a let's get out of tech because we we can arm wrestle about that. But there's a bunch of kids that were computer science majors who were just like, Yeah, I'm not doing this anymore because this skill's not gonna be a value like it used to be, which I I that could be or could not be true. But I think the mindset, especially of younger people coming up right now, is that there is a scarcity of employment available. And I've got a twenty one year old who graduated early from University of Washington, was a computer science major, and got into her first real CS class and is like, they're trying to teach me stuff that like is no longer relevant because this does not work anymore. And and she had me look at the code.

Kevin Surace: That's right. I agree with that. I I I'm on the board of a university, so I can I can trust me, I have this

Jason: man.

Kevin Surace: conversation all the time.

Jason: Yeah, I looked at the code and I'm like, this is useless. And she's like, I hate this. This is this sucks. I don't want to do it. I'm like, great. So she became a social sciences major like her old man. And

Kevin Surace: Great.

Jason: right, which is great. I mean, but but the number of jobs available for social scientists, that is, a job that's a social scientist, is actually zero. Like you can be a professor of social science. That's about it. Otherwise, it's just a glorified liberal arts degree.

Kevin Surace: No, no, no. But no, no, I would I but but but I would argue every company needs social science now, social scientists, because of all this cultural change. Everything

Jason: yeah. You have to. Correct.

Kevin Surace: that's going on. Like we need more of those people. no question. What they're by the way, we want all our coders. The reason we want our coders is not because they're coding, but because they're developers that have taste and experience and judgment. That's why

Jason: Yes.

Kevin Surace: we want them. Every coder I know that's doing this work, they've got six agents running. Doing this feature, fixing this bug, figure, and and they'd become the robot overlord. And they went from, I'm scared of this stuff, to this is amazing, right? This is amazing.

Jason: But they actually took they took time to learn what went through so they know what looks good and what looks bad. I mean the the

Kevin Surace: Yeah.

Jason: discernment of those pieces comes through trial and error and a long protracted period of time where you can actually audit those pieces. How are we going to get those skills? How how how how do you how are these new generations actually gonna get that kind of experience?

Kevin Surace: Yeah. And experience. And experience. But but well, well, they should be learning those skills. My my thing at university is we need to teach these kids the skills they're going to get hired for, which are AI skills. And if we're teaching them like most universities, we're going to teach you five different computer languages. I'm sorry, none of these languages we will ever use. And we got to throw Fortran in just to have the basis. They're never going to use it.

Jason: Great for trying to Pascal just because in case you gotta modify E Proms, right?

Kevin Surace: And Pascal, I learned Pascal. That's right. It's like, okay, everybody stop. I get that you probably want to understand the basic concept of there's machine code and then there's assembly and then there's a high-level language and they compile and great. I can learn that in a day or two. Okay, the rest of the time we gotta make them useful using the tools of the day. And the tools of the day are AI codes, and it's getting to the point where. Nobody's actually reading the 20,000 lines that got

Jason: Right.

Kevin Surace: produced because you can't. You can't. So you don't. You have to test it. Yeah, you have to test it

Jason: No, no, because there's no context in it. Right. It's garbage.

Kevin Surace: and and you have to get better at testing it and more automated. But and you gotta have your regression tests and everything else. But forget trying to read all that code. But we're teaching kids still to to re read and write code that they will never do, and we're doing four years of it, right? It which is ridiculous. And part of the problem is I don't know if you know, but when you've got An accredited degree, you can only change a very small portion of that ever from the date of accreditation. So you you don't get to go and change it all because the world changed. You can change a little bit and throw a one other class in and take something out. But otherwise you're stuck with this old thing. And the students are smart enough because they're already beyond that, that they go, Why am I learning? This is completely useless. I will never use a thing that I'm learning. It's a problem.

Jason: And and there's not there's not a solutions engineering or a systems engineering degree. And and systems are the big thing that we just forget about. And the way that you learn systems

Kevin Surace: Yeah, I agree with you.

Jason: is a lot of trial and error. Like you're not going to learn that sitting in front of an AI console and there's not a class that teaches it. Like the people that do the best on the networking side and load balancing and actually doing really good firewall functions, there's no classes on that. Like there's no way for you to learn that at the university level. And that's a huge problem. Right.

Kevin Surace: No, no, no. That those are the classes you need to take because these systems many people listening may or may not realize, right? But a corporate system, which are most systems, but people think of, you know, I say web and they think of a website. No, no, no. Most systems run corporations. So a large bank will have 15,000 applications under management. 15,000, three of them face the outside world. You know what I mean? The rest of them operate the bank. And these often talk to 20 or 30 or 50 other systems. It's like all intertwined and talking and APIs and you know, some to the outside, some to the inside, some external stuff. I this is very complicated. And then there's firewalls and there's load balancers and there's

Jason: Yeah.

Kevin Surace: content and everything. This is hard stuff. Like and it's not taught and not a lot of people know it. And that's actually where all the juice is. It's not anymore in I'm gonna code a new application. My mom can code an application.

Jeremy: So I wanna ask you about education and you've been very generous with your time, so we can wrap this up here. I talk to people every day that are not in college, college is is far in the past, and they are they're afraid of it, they're terrified, they're and and sometimes they're afraid of, you know, the bad implications and things it could do. But for the most part I hear a lot of like, I don't get it. I don't know what I'm supposed to do, I don't know how to learn how to use this thing. What would you what would you tell that person? who's probably thirty?

Kevin Surace: Yeah. Start. Start. what I tell everybody AI first means you go to it first. So just start with that. Just say, I was gonna go to Excel, but I'm gonna go to Claude or Gemini or whatever, and I'm going to see what it can do. I need to make a spreadsheet. Just start with that. It's a great, what do you want to make? Just have a conversation. Start. Because once you made one spreadsheet and you made the second and the third one and the fifth one, and by the tenth one you go, Holy cow, this is amazing. And now Then you're an expert at that, right? You don't have to make agents yet, but just start with that. And you want to start with agents, go to Muse. Any my mom can do it, right? Again, my anyone

Jason: Yeah.

Kevin Surace: can do Muse. It's it's in English, it's very simple. But you can also do that with Claude. You can design

Jason: Yep.

Kevin Surace: an agent with Claude, Claude work, and just say, Claude co-work, and here's the things I want to do. Design me an agent that does the following things and talks to this and talks to that. And it goes, okay. Or I don't have access to this, grant me access to it, all of those things. So the yeah, yes.

Jason: Give me all your credentials.

Jeremy: Ha ha ha ha.

Jason: Give me all of your identity.

Kevin Surace: The it is a problem. we're protecting a by the way, we're protecting agents at with token core stuff. we're putting gates in there, we're putting 502 biometric gate that stops dead only if you're over like a certain threshold or something else. Why biometric? Why biometric? Because if it's not biometric, an agent on someone's desktop will approve the darn thing at 6 a.m. in the morning to save you the time and it just sent a $10 million checkout. So if we don't have biometric gates, you don't know a human got involved. And there'll be an and this is gonna happen, agents approving agents. Anyway, look, the best thing you can do is just start doing it. Period, full stop. And if you're not gonna do that, you're hosed. The i you have to do it and you have to do real work. I said stop playing, because everyone says, well, I played with it once. Okay, you lose. Well, okay, you that's not the point. Do real work that has to get done that you have to turn in today or turn in tomorrow. And when you start doing real work, you know it literally in two weeks. You will be five times an hour, you will be AI first, and you'll be the the robot overlord for your group. And everyone will come to you and you'll go, but this wasn't that hard. So change is hard. I get that. People will you evolve, even though you're some somewhat against AI a little or worried about it, you're also, I can see, AI has brought you immense joy.

Jeremy: for me absolutely no it it

Jason: yeah.

Jeremy: is i i I I the the productivity that I have in my workplace today versus two years ago, that guy's fired. That guy could not do in his you know, i in a year what I do now. Yeah.

Kevin Surace: And and those are the people getting laid off, by the way. When people say, they laid off 50 people because of AI, trust me, they didn't lay off the AI pros. They laid off the people who weren't going to use AI. We all

Jason: Yep.

Kevin Surace: know that. If you're in tech, like you don't lay off your people who are who have six agents running, you know, coding agents, right? You you you get the guy in the corner that says, I'm never going to use that stuff. No problem. Go do that at another company. I don't want you because I don't want Kevin from two years ago. You don't want Jeremy from two years ago because we weren't as productive as we are today.

Jeremy: It's it it's just it's exactly what you've talked about. It's i and I've heard this analogy before. Like it does the doing. We still do the thinking, we tell it what to do, and now

Kevin Surace: Yes.

Jeremy: in seconds it does what would have taken us an hour, several hours a week. Who knows? Yeah, yeah.

Kevin Surace: Weeks sometimes. Yeah, it's crazy.

Jeremy: Yeah.

Kevin Surace: Wait, wasted time. So it's a game changer. And and and so that that is what I, you know, a lot of what I talk about in my AI keynotes is just start. What do I start with? Excel, Word, I what you know, PowerPoint. Are you gonna do a PowerPoint this week? Everyone raise their hand. Great, don't do a PowerPoint. Go to Claude. Go to Claude Design. Go to ChatGPT has recently gotten better. The latest Astro is much better. But just go there and say, make me a PowerPoint. What do you want to make it at? And just get it done and see how that works. And it'll work better, better.

Jeremy: Yeah. It's it's lazy. It it's lazy, but my favorite is still write this email to this person telling them this, this, this, and this, and it articulates

Kevin Surace: Yeah.

Jeremy: it in a professional, clear and clean way. And I didn't have to sit there agonizing over every word for ten minutes coming up with that email.

Kevin Surace: Because you're not an English major or a s psych psychiatrist or psychologist. It is, and its prose is better than mine will ever be. So of course, yes, it's still saying

Jason: Yeah, it's really good.

Jeremy: Yep, hundred percent.

Kevin Surace: what I want to say, by the way. It just says it better. But my last thing, last thing for me,

Jeremy: So that lastly.

Kevin Surace: because I re I am quoted in the press, you know, three or four times a week. A lot on cybersecurity, but on AI and on the joy success cycle, whatever. it used to be that I would get all these requests and I go, I can, I can only take one a week because they have all these questions and they have all this stuff. And now I knew what I wanted to say in the first minute. Like I read the question, I go, I know, like I know my opinion on this. To put that opinion into sentences that they would quote was very hard for me. I mean, obviously I speak English, I write English, but quotable. no, no, no. I train an AI to do that. And so it knows the background, it knows the topic,

Jason: Yeah.

Kevin Surace: it says, I say, here's the questions from the reporter, here's my opinion on these questions, write two sentence answers that are quotable for the reporter. And they're and I get quoted all the time and they're beautifully quotable. It is my opinion. It's ex and and I may change a word or two, but that's my opinion. That's what I wanted to say, just could never say it that well.

Jeremy: That's our blog. If you read our blog, we take the transcripts of these conversations and turn it into a blog that you can if if you're a reader and you're not a listener, great, but I didn't write a word of it. Robots did, but it was all based on every word that we pulled out of this. Which which

Kevin Surace: Of course. Yes, yes.

Jeremy: which brings me to your book, The Joy Success Cycle. How much

Kevin Surace: Yes.

Jeremy: of it did you write? How much of it did AI write?

Kevin Surace: Great question. There it there it is. you know what's funny is so I wrote the book two years ago because it takes time to get a book to market, right? And I didn't self-publish it, it's published through a a publisher amplified publishing. So I didn't write it with AI at all. I would sometimes because it wasn't good enough yet, right? I would sometimes say, I'm here's what I'm trying to say, and I'm at a loss for words on this, and it'd give me s some of a paragraph and I'd take some of it and remove it. But over the two years, what happened is it went through three. Editor phases, three different editors. And no question by the last one. They were probably putting the whole darn thing and asking it questions. So they were getting a lot of advice from AI. And then they would come back and say, I need you to change this, and this paragraph should change, and this should move around. One that was guided by AI. And for all I know, some of those words that they said, I really need you to change this in this way because that speaks better to a broader audience, probably written by AI. So I didn't write any in AI, but By the time you get to the end point, all the people who've touched it have certainly leveraged AI to make it a better a better product. I also had a

Jason: So quick qu

Kevin Surace: a ghostwriter, but my ghostwriter took everything I said and and and she, it was a she. And as a male, I recognized you probably want a female ghostwriter if you want this to read well across the population, because she was able to take a lot of what I said and soften it in ways that still got the points across, but

Jeremy: Hmm. Sure.

Kevin Surace: you know, again the words were just better, the tone was better, right? So it these books have lots of touch points. So but yeah, I if I did a new one, I would probably leverage AI a lot today. But I I

Jason: So

Kevin Surace: two years ago you couldn't.

Jason: yeah, real quick follow up with that. When the audiobook comes out, will you be using an avatar to go through and read the affirm with your voice? Perfect.

Jeremy: too late, too late.

Kevin Surace: The audiobook's out. The audiobook's out. Hang on, the audiobook's out. And first of all, the forward here is by Goldie Hahn, the actress, okay?

Jason: that's wonderful.

Jeremy: no way. That's very cool.

Kevin Surace: So the for the the audiobook's out. It's not on Amazon, but it's everywhere else, including Spotify. And indeed, I cloned my voice, 11 Labs.

Jeremy: Nice.

Kevin Surace: I did a full clone where you spend like a half an hour, you know, talking and reading and doing all kinds of things. I got a great clone. no one can tell it's not me but my wife, because she can tell. And the here's the real reason. The clone was perfect, but I speak quickly, as you found out. And they didn't want that for the book. It was too fast for for peop. So I s I slowed it down ten percent. So my wife can tell that is too slow that is too slow for the way Kevin would normally talk, right? Otherwise it's it's it's me. And even the the the Ford by Goldie Hahn was a digital voice, digital recreation. Yeah. So

Jeremy: That's awesome. That's so cool.

Kevin Surace: I had to do that 'cause I'm I'm I'm from the AI world. So if I didn't do that, they'd say, What the hell? You really re

Jeremy: Yeah. Why not?

Kevin Surace: gonna sit there for a week and read your book? No.

Jeremy: Well, that's so funny. Kevin, this has been awesome. Thank you so much for being so generous with your time, being open to some some hard questions, particularly from Jason. and

Kevin Surace: Yeah. Yep. The great

Jeremy: really appreciate your time. Thanks so much. we'll have the the link to the book in the show notes for this episode. thank

Kevin Surace: Fabulous.

Jeremy: you so much for your time.

Kevin Surace: Yeah, thanks guys. Yeah, it was fun. Good questions.

Jeremy: Our guest Kevin Serace, his book, The Joy Success Cycle, can be found in the links in the show notes for this episode at BroBots.me. That's also where we'll be back with a brand new episode on Monday. Thank you so much for listening. Make sure you subscribe and share with anybody who might benefit from hearing this conversation. You can find ways to do all of that at Brobots.me. We'll see you Monday.