Ahead of the Threat Podcast: Episode Six - Charles Carmakal
FBI Assistant Director Bryan Vorndran: Hello, everyone, and welcome back to “Ahead of the Threat” in 2025. This is our first episode in January 2025, and we're looking forward to more good dialogue with all of you about cybersecurity and the broader cyber ecosystem and what adversaries are doing. As always, Bryan Vorndran here with the FBI. And joining me is Jamil Farshchi. Before we get into today's episode, we're going to hit our Top Three, as we always do.
And those top three today are the BeyondTrust supply chain compromise. Secondly, PowerSchool and the data breach that occurred there. And lastly, some additional and continuing conversation on Salt Typhoon.
So BeyondTrust, right, this came out in the past couple weeks. Something important for all of us to monitor because it, again, continues to speak to the supply chain and third-party application risks, which we've talked about in settings here before.
But just to level set with everybody, BeyondTrust does a bunch of things in their security platform. Zero trust, real time detection, remote access technology, password management, privilege access management -- these core things to cyber security. But unfortunately, they suffered a breach. Their products are used by broad array of industries, whether that's government agencies, tech firms, retail, e-commerce, energy, almost the list goes on and on.
But when you look at the reporting out there in the media, according to Bleeping Computer, it was discovered that the hackers actually gained access to the remote support SaaS API, and that's how they actually conducted the breach. And during the investigation, according to Bleeping Computer, Remote Trust did find a critical vulnerability in mid-December that they reported as a CVE [common vulnerabilities and exposures], which is now on the CVE list.
You know, when you look at the media reporting about the BeyondTrust compromise, it does point to China being behind this. And there’s some open source and media reporting about the impact on Treasury as one of about 20 victims. But again, more globally and more importantly probably here, it's the supply chain risks, the SaaS platforms the third party applications through trusted connections and trusted relationships and infrastructure that give an adversary – whether nation, state or criminal – access to other parts of an infrastructure.
So, Jamil, over to you for any thoughts.
FBI Strategic Engagement Advisor Jamil Farshchi: It's just a black eye. I mean another incident. This one's a little bit more embarrassing given the fact that it's the source of it is a security vendor itself. And so you're, you're kind of left thinking, “Man, I procure this stuff to be able to help lock down my environment and make me more secure. Yet it ended up being a vector to cause it.” The fact that it was Treasury makes it even worse. I think, was CYsyphus affected in this one too? I feel like it may have been. I'm not entirely positive, but just another example. I mean, our guests have talked about this over the course of the past year and it's this, it's same old, same old.
But, are the vendors out there especially on this on the security side, really have got to step it up in 2025, I think. I mean, it's we can't have another year like we did last year where it's just one after another. And, the source of it is in fact security itself. It's just a huge black eye.
Vorndran: You know, two additional thoughts before we moved to PowerSchool. So first one is it'll be really interesting to follow the attribution on BeyondTrust in the compromise. Initial reports are pointing to China. It'll be really interesting to understand which APT is behind it and really understand the strategic goals behind why the APT did what they did. And secondly, is this thought that, you know, within these supply chain attacks, right, the question always is: What should organizations do to prevent them? The unfortunate answer is there isn't much. You know, we continue to point to really understanding who you're doing business with, understanding, to the best of your ability, what trusted relationships exist.
The National Cyber Security Strategy that was published two years ago, points to shifting the liability to the software provider for security, right? And it will be interesting to see in the upcoming year or two how that actually takes shape.
Farshchi: Well, I do think that there's going to be I mean, you probably know this better than I do, but there will be some changes coming with the new administration stepping in. And so I think there will be potentially a more forceful position on some of these things. So I guess we'll see how that plays out. But I think your first point, you're right, it's really on the defender side. It's really tough. I mean you rely on the vendors to make sure that their products are secure. And if they're not, there's not a whole lot you can do. I mean, aside from some behavioral detections and maybe some access controls internally to be able to help mitigate the lateral movement and potential exfil of information and stuff, that's it's just extremely difficult at that point in time.
So, it's a tough position for us all to be in because you just sort of have to sit there and take it to some degree, which is just the worst position you could possibly ever find yourself in.
Vorndran: Yeah. And I’ll just point… I've mentioned this in this forum before, but I think it's important, you know? One of the…well, the first Cyber Safety Review Board report, which really addressed Log4j. One of the most important findings, I think, in that report is that there are no secure coding standards within the university system in the United States of America. When you equate that to other trades, right? Whether that's civil engineering and building bridges or dams or buildings, there's always these baselines of accreditation to ensure safety and security. Unfortunately, that doesn't exist in our education system right now. Probably something that would be of really, really import to get under control in the coming decade.
Farshchi: It would help. But we also need the prioritization because I think, I mean, I don't know any dev worth their salt is going to know what some of these basic measures are to put in place to ensure that your code is secure. But I don't know whether it's the speed that these folks are being forced to deliver at or it's just the lack of emphasis around security itself, in favor of new features and functionality. It's just it's leading us down the wrong path right now.
Vorndran: Yep. Okay onto PowerSchool and the PowerSchool data breach, which has been reported over the past week or two. We raise this here because we think it's important, not so much because of what happened, but because of who it impacts. And who it impacts is 60 million students and teachers in our K-12 education system here in the United States. PowerSchool services about one-third of the K through 12 school districts here in the United States. And there was a massive data breach that occurred, that impacted most likely PII [personally identifiable information] of students and teachers, as well as potentially medical records. And so the unauthorized access, the activity, the data breach occurred through essentially stolen credentials and access to essentially the platform that accesses the database.
But again, you know, the ‘how’ in this case, to me, doesn't really matter. Right? But the impact about students and the theft of PII, when they may be as early as five, six, seven years old, and the ability of adversaries to have access to that PII for the upcoming decades and the rest of those children’s lives is really, really significant.
You know, we get asked a lot, what can you do in a situation like this? As a downstream victim of somebody who has stolen PII? There is no magic bullet, right? It's these things that you can try to do to help, right? Use identity theft services, monitor your accounts regularly, watch your credit. But for a student or a child under the age of 18, you know, some of those things aren't going to apply. But it just needs to become part of our DNA as individuals, as parents, as colleagues to get in the habit of really monitoring our credit, monitor our financial accounts because all of our PII is going to be out there, it's going to be available to criminals, and it's going to be available to nation-state adversaries. And we've seen both sets of actors, nation-state adversaries and cyber criminals take advantage of the access to the PII.
So just a very, very unfortunate story that impacts, a lot of individuals and children here in the United States. So we want to put on your awareness– put on your radar for awareness in the event that you have an equity.
Farshchi: Oh, man, this is just another one of those where you just have to sit there to some degree and accept it. I mean, we got to hold these folks accountable, I think, to some degree, because at the end of the day, they're the ones with the keys to the kingdom that are…that we're relying on to ensure that this data remains protected. And I know this resonates with you and I both, because we've both got little kids that, you know, are in this roughly the same age range of the impacted folks here. But it's unacceptable, in my opinion.
You did list off a few things that are useful to a degree, but given their ages, yeah, I mean, even on the credit monitoring side stuff, I would recommend, taking advantage of the family accounts and stuff that you have on there. But they're so young that at this point in time, it's tough. The value out there isn't, you know, substantial for that age. And you're right, it's a great way to get a foothold for the adversaries, for these, individuals, you know, lives. Because a lot of this data is– it's not going to change. It is what it is. So it's just another tough situation, unfortunately.
Vorndran: You know, when we look back and we just compare and contrast BeyondTrust and PowerSchool here today, right? The BeyondTrust compromise was fairly sophisticated, right? Or at least it's shaping up to look that way. You know, we'll know the full story here in the coming weeks and months. However, the PowerSchool compromise is not sophisticated. It's stolen credentials. Stolen legitimate credentials.
One of the most interesting cases we've done in the FBI over the past couple of years is what we referred to as Genesis Marketplace, where essentially actors harvested legitimate credentials from bots. Right? Essentially malware installed on user-end equipment throughout the world and throughout the country. The use case we always use there is, a bunch of children, kids, you know, teenage kids essentially bought access to an EA Sports forum that had legitimate user credentials for EA Sports and stole a bunch of intellectual property simply by harvesting those credentials off of computers. A real, real risk for us. Actually, easier to do than many would expect, and a significant risk to us moving forward.
Farshchi: Man, it is simple. 86% of the breaches today are due to compromised credentials -- 86%. I mean, it's like, we always get into these debates around prioritization and this control or that control or whatever, but 86% fall in this particular space; like this is the vector. And so, I don't know, if you're anyone, you probably…. you may have seen I put a LinkedIn post earlier this year where we went– at Equifax, we went password-less. And the reason we emphasize that is for this reason I'm like, ‘man, you know, we have a lot of tough decisions in security. But this one isn't tough.’ Like, the propensity of incidents are all because of this one thing? Let's get rid of it. Let's eliminate that attack vector. It's got to be a focus area for everyone in ’25. I mean, these things aren't going away, and it's just so dang easy for the bad actors to be able to execute these types of attacks.
Vorndran: Okay, moving on to Salt Typhoon and Jamil and I wanted to touch on this to really keep this conversation alive because it is a generational compromise. It is probably the most significant cyber intrusion that we've seen, you know, as we do our work over the past decade to two decades. You know, I would just say this about Salt Typhoon, right?
And just to level set here, we know in Salt Typhoon that the Chinese infiltrated some of the major U.S. telcos – telephone companies and ISPs – here in the United States. And they did a bunch of things while they were in the environment. But one of the things they did was they targeted specific telephone numbers to collect against those individuals, right? Whether that's clear text exchanges or clear voice calls, exchange exchanges.
And I get asked a lot like, "Hey, Bryan, why should I, as an average everyday American, care about the Chinese stealing my call detail records? Or should I care about the Chinese potentially trying to intercept, you know, clear text, clear audio calls?" And I answer it this way: The fact that we have to consider the PII of a sixth grader, a seventh grader, an eighth grader, being in the hands of the People's Republic of China or the Chinese Communist Party and what they may do with it is kind of crazy, right? We know from looking at APT 41 activity over the years that they did use stolen PII to open a host of unemployment accounts, right? So, the Chinese are letting their nation-state-backed, trained individuals moonlight for personal gain over here. We've already talked about the risk for generations in decades to come, about our children's PII being out there.
On the intercept side, if the U.S. government ever attempted to intercept communications of individuals unilaterally without court order, there would be an uprising. And rightfully so. But this is an attack on our sovereignty. The Chinese essentially collecting on Americans’ phone calls, and American text messages here in the United States is an attack on our sovereignty, right? It's a really, really big deal. We want to keep this conversation alive, because it is a generational impact on us.
Farshchi: And it's China, China, China all over the place. I mean, it feels like every day now there's another incident that involves an attack from those folks. We’ve got to do more. You know, one of the things I was disappointed by, I was reading about the, some of the response that we've had on this one and giving CISOs guidance on configuration management and patching -- not useful. Anyone in those roles is going to know that stuff anyway. I don't know how help– how that's really useful at all. I think that we need to do a better job. I mean, we've got to help in a more tangible, practical, effective kind of way. Our response cannot be so laissez faire and you know, cookie cutter, I don’t think. These attacks aren’t going away. They keep getting- it feels like the volume and the severity and the sophistication just continues to improve. And we're just sitting here taking it, not good. Not good at all.
Vorndran: You know, one thing I heard said, and I can attest to this based on my own education or real training or knowledge, but the telcos here are built for speed, right? They're not built for security, and they're built for interconnectivity, with other providers. And that inherently brings risk. And so, Jamil, what you're saying about better guidance, right? More resilience, I agree with that. I think it's really, really complicated when you get into the X's and O's of how to do that. But I also know that there's people in industry right now that are undertaking that exact line of effort. Right? Because it's so, so important.
Farshchi: Well, it is. But I would say that, you know, I know personally, I know the security leaders at these organizations; the highly skilled, like, top tier folks in many respects. And I just don't see why providing someone with some basic stuff is going… it mean it might make a headline, might make you optically look like you're being useful, but it's not. So we need to up our game in this front because we've got a ton of talent within those organizations. We want to do the right thing. We need to help arm them to be able to do their jobs more effectively.
Vorndran: Well, that completes today's top three. So BeyondTrust and the compromise of Treasury, the PowerSchool compromise impacting up to 60 million student and teacher records, and then continued dialogue on Salt Typhoon. We'll now go to a previously recorded episode with Charles Carmakal, the chief technology officer at Mandiant and a name familiar to many of you.
***
Vorndran: Joining us today is Charles Carmakal. Charles, welcome to “Ahead of the Threat.” Just to get us started, can you give us some brief background on your role? Title, what your day-to-day functions look like, to set the scene for our audience.
Charles Carmakal of Mandiant: Yeah. Of course. Absolutely. First of all, thanks so much for having me here. My name is Charles Carmakal, and [I am] the chief technology officer at Mandiant. And I lead a team of security consultants and incident responders that help organizations both respond to security events, as well as try to prepare for and mitigate the risk and the impact of cybersecurity events.
And when I think about what I do on a day-to-day basis, about a third of what I do is I help my clients, mostly the executive teams and their boards, either prepare for a security event or actually respond to it. The other third, what I do is, I do a lot of public speaking, so I do conferences, webinars like this, speak to media and press. And then a third of what I do is I help run the business.
Farshchi: People use the term “webinar” anymore?
Carmakal: You know, I just did.
Vorndran: Well, Charles, we’re going to talk all things Scattered Spider over the next 30 to 40 minutes. And it’s an area that we know that you have tremendous depth in. And I know that I’m personally looking forward to your thoughts on it. But let me just frame what Scattered Spider is. We’ve known in the industry, specifically Google and Mandiant, have known within the industry about Scattered Spider for multiple years at this point.
And I think that really speaks to the value of the work that Mandiant does within the incident response space, because they’re seeing these new threats, these new trends in real time because of their engagement with victims. But with Scattered Spider, we saw a shift in TTPs— tactics, techniques and procedures—that really, really, changed some of the things that we trusted as true and normal. And we’ll get into all that here today.
But, Charles, can you begin by just giving us an overview of what Scattered Spider actually is?
Carmakal: Sure. And look, let me actually take a step back, a little bit further. Beyond, just Scattered Spider, because I think it’s important to do a little bit of level-setting for the audience, just kind of understand who the actors are, that maybe kind of look and feel a little bit like Scattered Spider, but who we treat as distinctly different.
So, I first became aware of groups like Scattered Spider around 2021, and we responded to fairly interesting and different intrusions at organizations that largely relied on telephone-base social engineering to convince people to do things. We got a number of calls from victim organizations in 2021 where somebody within the organization, either an executive of the company or perhaps somebody working at a help desk, got a call from a Western-speaking individual asking them to do something.
That something was usually installing some kind of remote management tool, within their environment. So, I think TeamViewer or AnyDesk or, or something like that. And once they did that, the adversary was able to get access to the computers of the individuals that installed the remote access solution on their computers. And we very quickly saw them download data from the computer that they had access to.
And then within a few days, they started extorting those organizations. And right now, I’m talking about a group that we call “UNC (uncategorized threat group) 3786,” which is a group that we’ve been tracking for quite some time and it was … from our … from the Mandiant perspective, it was one of the first Scattered Spider-like groups that that we attract. And what was pretty interesting about those intrusions was that, you know, we didn’t see, you know, the deployment of ransomware.
We didn’t see a lot of malware being used within an environment. We just saw, you know, Western-speaking individuals tricking employees of organizations to doing things that they wanted. And that led to what ended up being very aggressive extortion. And we could talk a bit about that. The next category of threats that we saw were threats coming out of a group that kind of call themselves LAPSUS$ and this, you know, really changed how we thought about cybersecurity in 2022.
You know, gone were the days where, you know, we saw, you know, a lot of the common tradecraft to break into organizations. We started seeing a lot of different tradecraft. And, by the way, it feels normal today. But back in 2022, we didn’t see a whole lot of intrusions that were started because a Western-speaking individual called up a help desk and convinced them to, you know, reset somebody’s credentials or log in to a website and then type in their username and password.
And LAPSUS$, you know, they had a lot of, you know, clever ways for breaking into organizations. In addition to the telephone calls, we saw them download credentials that were stolen through info-stealing malware—like RedLine Stealer, like Raccoon Stealer—and credentials that were possibly stored on people’s personal computers as opposed to their corporate computers.
So, if you think about what changed in 2020 with Covid, a lot of people worked from home and a lot of people were using their personal computers to do work things. Or they were using their work computers; they were logging into their web browsers on their work computers to check their personal emails. And in the process, the browser said, “Do you want to synchronize your credentials and your bookmarks with your Gmail account?” And so, when people did that, unfortunately, it synchronized passwords from corporate computers to a lot of personal computers. And so, the LAPSUS$ folks were able to benefit from the credentials stolen from a lot of people’s personal computers to get access to organizations. And the LAPSUS$ folks, they broke into organizations not just because they wanted money. And in some situations they got money.
They extorted companies. But for the most part, they were looking for, you know, for street cred. They wanted to brag to their friends that they broke into big companies. And what’s interesting is, you know, one of the members of LAPSUS$ was really obsessed with acquiring the largest collection of stolen source code that he could obtain. So, he broke into a lot of companies and immediately started cloning GitHub repos just so he could say he had petabytes of stolen source code.
Vorndran: And, Charles, just for our audience, how would you describe GitHub?
Carmakal: Yeah, GitHub is a source code repository that allows people, a number of developers at an organization to view source code, to make edits to source code, and essentially push out content to source code repositories, which are ultimately used to build products and software.
Vorndran: Okay, great.
Carmakal: So, fast forward to Scattered Spider, which we started really noticing in mid-2022. You know, in the early days when we started tracking Scattered Spider, for the most part, what we saw they were interested in was breaking into business process outsourcers to ultimately get access to telecommunications organizations with the ultimate objective in mind to SIM-swap individuals.
And so what SIM swapping is, is essentially, you know, telling a telecommunications company that a mobile phone number is now ported over to a different mobile device, and what that allowed a lot of these Scattered Spider folks to do is they could hijack people’s telephone numbers or their mobile device numbers, and they could then get access to people’s accounts, whether it was their bank accounts, their cryptocurrency accounts, or what we later saw them do, they started getting access to people’s corporate accounts.
And the way they would do that is they would essentially hijack people’s telephone numbers and they’d go to a self-service password reset page, and they’d essentially say they needed to reset their password and they would ask the service to send a one-time text to their mobile device so that they could prove they are, you know, who they say they are, and then essentially reset corporate credentials, which were sometimes how the Scattered Spider or UNG 3786 folks got access to organizations.
So, kind of at a very high level, those are kind of the three big groups that Mandiant tracks. There are obviously many others, but those are kind of the three big ones that kind of look and feel like Scattered Spider. And sometimes, the community will use the word “Scattered Spider” to describe a lot of different categories of threat actors. And those are kind of the three big ones that we track.
Farshchi: What’s the, two things here ... Number one, what’s the significance of the Western-speaking aspect and why? What do you think precipitated that shift? That’s one. And then two, have you seen these TTPs from these threat actors evolve to any meaningful degree over the years?
Carmakal: Yeah, absolutely. So, the, the use of Western-speaking individuals kind of changed how a lot of organizations think about protecting their environment. And so, let’s just go back, you know, 10 years ago. You know, a lot of times corporations, when they engaged a, a right team exercise, or, you know, a penetration testing team, some of those pent testers were picking up the phone.
They were calling organizations. They were trying to convince them to do things, install software, click on links, provide credentials or whatnot. But then at some point in time, you know, the community of, you know, organization started to say, “Well, you know, most of these criminals that are hacking into Western organizations are Eastern European criminals.” And they got very thick, very distinct accents and it’s not realistic for these folks of very heavy Russian or Ukrainian or you know, Eastern European accents to pick up the phone, call people to do something because, you know, in the United States, we’d obviously, you know, it looks ... it sounds weird, it sounds different.
So, it seems kind of fake. So, a lot of organizations stop testing for that. And they stop building defenses to address the telephone-based social engineering problem. Now, fast forward by a few years. You know, a number of younger individuals in the United States, sometimes teenagers, sometimes folks in their 20s, they started realizing that they could convince people to do things if you have a Western-accent and you’re trying to emulate being a 24-year-old person working in the IT team of an organization. You know, they pick up the phone, they pretend to be part of the, you know, the security team of a company, and they convince people to go to a website to, you know, to reset their, you know, their cache for their Citrix sessions or something. They come up with a lot of different excuses, and it sounded pretty convincing because they actually were 24 years old. And they sounded like they were like legitimate folks working in an IT department.
And so, a lot of companies had to start to build controls to mitigate the risk and the impact of these Western folks picking up the phone and making telephone–you know–and calling up folks and convincing them to do things. In terms of, you know, how companies are really addressing that now … look, I think because of the awareness of Scattered Spider and of their techniques, a lot of companies have started to build out better processes to verify the identities of individuals. And this is, you know, challenging for a lot of reasons. But, you know, we’ve seen, as one example, we’ve seen a number of companies verify identities based on the last four digits of their Social Security number.
One of the challenges is that the last four digits of a lot of our Social Security numbers have become relatively easily available to a lot of criminals. So, it’s a relatively trivial process to find that. Other times we see help desk will ask for the names of the supervisors of a particular individual to verify their identity. And that’s also can be trivial to obtain.
Yeah, not it’s not always the most easy thing, but you could do that. We've had some situations in certain parts of the world– for example, in Canada, you know, people in the help desk, they tend to want to be as helpful as possible. And we’ve had situations with Scattered Spider victims where somebody called up the help desk, pretended to be somebody else, asked to reset their password.
The help desk, in return, asked a bunch of challenge response questions. The individual calling pretty much got every question wrong, but the person, the help desk wanted to be helpful. They were Canadian. So, they essentially coach them to get the right answer, which ultimately led to the reset of an account. And so, the thing that’s kind of changed, you know, since the Scattered Spider problems become, you know, more broadly known, is a lot of companies have had to really build out some additional processes for identity verification.
And some of these new processes aren’t that scalable and aren’t easy to do in the long term. But, you know, some of the things we’ve seen them do are they’ve requested that, the individual calling asking for a password reset, join a video call, turn their camera on and prove that they are who they claim to be. But by the way, you don’t exactly know what people look like.
And so, in a number of these situations, they’ve had to get the supervisors of the individuals on the line, to verify the video and the audio of the individual or some other way. Again, that can work over the course of weeks or months, but it’s hard to do as a permanent thing. I mean, look, honestly, the Scattered Spider folks, they've really challenged a lot of organizations to think about building new controls to better protect their environments.
Farshchi: I completely agree, and I know a bunch of peers who have some very convoluted processes like you just described, which is fine in the short term. It helps to mitigate that risk, but they’re just not sustainable, certainly not scalable when you have to pull out your ID and scan it. Or they’re not effective in some respects, like you just described, where it’s a video conference or whatever.
But hey, I mean, maybe that’s being deepfaked or, you know, or it’s voice authentication and then you’re like, well, maybe that’s going to get cloned, or whatever. I mean, we put in a technical authentication control that I feel really good about a little over a year ago, but, there’s just a lot of room to grow, I think, in this particular space. So, do you…but what worries me is what’s the next shoe to drop? I mean, maybe this attack, this vector is going to be so successful that they not … they don’t have a need to migrate to something else. But do you see … Like, what’s the, what’s the evolution of this attack or what are you seeing them do today? Or if anything different than what it has been?
Carmakal: I’ll talk about some of the things that they’ve done in the past in just a moment. But maybe to directly answer your question: look, it’s hard to predict what they might do in the future, and I’m also cautious about giving them ideas. But with that said, I will talk about things that we’re seeing right now that aren’t necessarily being done by Scattered Spider, but are being done by other adversaries that are facilitating crimes or scams.
So, we’re seeing a lot of the creation of synthetic voices, or deepfakes, to convince people to move money. So, you think about the kind of the traditional business email compromise scenario where, you essentially tricked somebody in accounts payable team to pay an invoice that maybe might look like a real invoice and maybe from a real vendor. But the bank remittance details are different, such that, you know, the money goes to an account that the attacker or the scammer controls.
Well, we are starting to get a number of calls from organizations that are dealing with scenarios where there is some synthetic voices or deep fakes that are being used to convince people to pay invoices. And just to kind of give us some examples, we see a lot of executives, or people on finance teams being reached out by executives over WhatsApp, basically asking them, to, you know, kind of start the process to wire money for one reason or another. Usually, you know, because of an acquisition or something.
And, you know, there are times where, there’s also a video conference call that’s going on, a Teams call, a Zoom call, where the person ends up talking to somebody who they think is actually, you know, maybe an executive of a company, a controller, a CFO, somebody like that, where there’s, you know, some deepfake that’s being used to essentially mask the identity of the person and make it look like it’s actually an executive.
You know, I’d say the number of cases that we’re dealing with related to this are relatively small now. But I continue to hear about these types of situations more and more. The fact of the matter is, the process for creating synthetic voices and deepfakes today, with the technology available, it’s easy to do a basic style, you know, cloning of somebody whose voice … It’s a little bit harder to make it look smooth and seamless, like it’s a real human being having a real conversation.
So just for fun, you know, somebody on my team, you know, called me up, about six months ago, and he said, ‘Hey, just for demonstration purposes, I’d like to create a synthetic voice of you and integrate it in with ChatGPT to see if we could have essentially, you know, a Charles bot in which somebody could have a real conversation with and have ChatGPT respond as if it was you.” And I said, “Sure, let's do it. That sounds kind of fun.” And so for about $10 in spend, using open-source, you know, tools, and then other commercially available tools and about 10 minutes of time, by basically downloading a video of me that I, you know, of a talk that I gave that was on YouTube, he was able to basically create a voice that sounded pretty much like me.
Now, you could tell there are certain things, like I enunciate certain things differently than how the bot would do it, but it was close enough and it sounded pretty real. And it was essentially $10 of money or 10 minutes of effort. And so, I was thinking, you know ...
Farshchi: It wasn’t as handsome as you, I doubt ... it wasn’t as handsome as you, I doubt.
Carmakal: I’m sorry?
Farshchi: It wasn’t as handsome as you, I doubt.
Carmakal: I don’t think it was, yeah.
Vorndran: Charles, I get asked a lot about violence. Right? What are we seeing in terms of threats? Threats versus actual manifestation of violence against individuals that are being extorted, right? I get this question all the time. From your perspective, maybe inclusive of Scattered Spider, but also other groups, what would you share?
Carmakal: Yeah. So, you know, unfortunately groups like Scattered Spider, groups like UNG 3786, you know, there’s … they … you know, when you are, you know, a loose collective group of folks, if you’re in your teens or you’re in your early 20s, you tend to operate without rules of engagement. So, if I were to look at maybe how China-nexus espionage groups operate or Russian espionage groups operate, there are rules of engagement. You know, either because they are an intelligence officer or they are a contracted entity working a job, a 9-to-5 job. And, you know, they’ve got bosses and they’ve got bureaucracy. And so, there are rules of engagement.
Whereas with some of these groups, like Scattered Spider, they don’t have those rules of engagement. And they have a lot of different personalities, a lot of different, perceptions of what is the line that you can’t cross.
And there’s a lot of members of these groups that don’t quite understand that there are really lines that you shouldn’t across. So in terms of, you know, things that we’ve see; again, don’t want to give folks ideas, but what we’ve observed thus far is, you know, we’ve seen, threat actors directly message employees saying, “I know you live at this home address,” which kind of creates some kind of psychological intimidation to a victim organization, because now you’re not just thinking about the protection of your company and your customers, you’re thinking about the protection of your family.
We also have seen certain threat actors threaten to Swat victim organizations or executives of victim organizations, which creates a certain sense of physical intimidation.
Farshchi: We describe what “Swatting” is?
Carmakal: Yeah, so, Swatting is essentially where you as a criminal, you call the police and you pretend to be your victim, and you say that you’ve committed a heinous crime, and you create a scenario in which the police SWAT team needs to go to the location to, stop the crime from happening, or you know, from taking it any further.
So, to a victim who’s dealing with a swatting situation where you see the police department show up at your home with guns, with, you know, SWAT gear, you know, it can be really intimidating. This is a common thing that we see a lot of teenagers do against other teenagers, you know, that are that are playing video games or that are streaming.
And so, it’s become, you know, a pretty big problem. And so that is something that we’ve seen manifest outside of the gaming community, but into kind of the enterprise extortion community. We’ve also seen certain threat actors send packages to people’s homes. You know, for the most part, it’s been benign, you know, flowers or something like that.
But again, it's like psychological intimidation, you know. Not only are you dealing with, you know, an attack against your company, you’re thinking about, how do I protect my family? We’ve seen certain threat actors, like, UNG 3786, SIM-swap the telephones of children of executives. And then call those executives, where it looks like, you know, just in the back of your mind, you’re thinking, “Oh my gosh, am I … is my child safe?”
Because I received a phone call for my child’s phone number, but it’s not my child that’s speaking on the other end. And so, there’s just a lot of that disruption that, you know, we’ve seen groups like this employee, which, you know, just a tough situation for victims.
Vorndran: You know, our intelligence on our side from working with victims is identical to what you just shared, Charles, to include what I would say is, direct threats against family members, specifically children, delivered to CEOs or, you know, chief officers, or officer-level individuals at different companies or organizations. And, you know, I don’t have any examples in kind of my world where that’s manifested into true physical harm.
But the mental and emotional toll of that is very significant and obviously done to apply direct pressure from an extortion perspective. It’s just from a human level, the fact that we have people operating this way to make money is really, you know, really disappointing in terms of humanity.
Carmakal: Yeah.
Vorndran: But something we have to all contend with.
Carmakal: And Bryan, you know, just maybe share the Mandiant caseload data, we don’t see these physical threats manifesting into actual violence. The closest thing that we’ve seen manifest into something that feels, you know, physically intimidating is the delivery of packages. But again, everything’s been benign. You know, flowers or whatnot. We have seen some victims being Swatted, which is, again, you know, very intimidating to see that happening.
Farshchi: And that does carry risk too, though. I mean, accidents happen when those, you know, those high-pressure situations. So, yeah.
Carmakal: It absolutely does.
Vorndran: You know from a Swatting perspective, personal experience with this on multiple occasions, I would just encourage our audience to talk about this, especially if you have kids in your home, right? Like if this happens … this is what we would expect you to do, right? And that answer is, “Just comply,” right? Just comply. Wait until calmer times. You know, calm and you have an opportunity to work through it because from the police department’s perspective, they’re going into what Charles expected is a heinous crime. Murders, you know, mass murder type of environments where that’s what they’ve been told has happened. And so, they’re going in, really prepped for that. So, just comply.
You know, I have many friends who have had their families removed from their homes, sat out on their front lawn. Just comply and get through it and cooler heads will prevail. Very, very impactful though, especially with smaller children in your home.
Charles, just looking at my list here, just some of the names, right? Earth to Star. The Messiah. Sosa. r1z. Wombat. My question is not who’s behind these people. Because we’re all figuring that out. But how are they thought of within the cybercriminal underground? Are they respected? Are their skillsets legitimate? I mean, how are they these people actually thought about?
Carmakal: Yeah. It depends on who you ask.
Vorndran: Okay.
Carmakal: And it also, I think, it kind of depends on when you ask, you know, people. So, I’ll give you a few different perspectives. A few years ago, when we first started dealing with Scattered Spider-types of intrusions, there is a perception that they weren’t capable, that they weren’t very skilled or sophisticated because they didn’t develop custom malware.
And so, sometimes in the security community, there’s this perception that if you don’t have the ability to develop exploits or if you can’t develop malware that’s really clever, then you’re not necessarily a capable, advanced adversary. You know, my argument against that is that the Scattered Spider folks have been very effective at break into organizations. Irrespective of the tooling that they use, they’re very effective and
Vorndran: And very profitable, right?
Carmakal: Yeah, they’ve made money. We’ll talk about that in a moment. But they’ve been effective and they’ve been incredibly disruptive to the victims that need to deal with them. If you ask members within, you know, this whole community of folks that are, you know, part of these intrusions, what I find is, you know, particularly for, you know, some of these younger individuals that, you know, if you think about, like, how we might have been in high school, you know, one day you might be friends with somebody, but, you know, three days from now, that friend might say something that you might feel insulted by. So, you hate them for the next several weeks. That’s a lot of what we deal with members of Scattered Spider, you know, that’s just the broader community folks. They might be friends one day and hate each other, you know, the next day and maybe for several months. And so, the amount of, you know, discourse and tension amongst them is incredibly high. Which as defenders, we could take advantage of that.
But sometimes that actually hurts us. And so, you know, to give you an example of when am I hurt us is if you … if you’re a victim and you end up paying an extortion demand and you’ve paid this extortion demand for a promise by the threat actor to not publish the data that was stolen, or to essentially leave you alone. Some members of the group might feel like they got paid, and they got paid their fair share, and they might stop, but others may not feel like they got paid their fair share. Or maybe they didn’t get paid anything because they’re no longer friends with the rest of the members. And so, then they might continue to harass a victim organization.
And so, that is sometimes the downside of, you know, the short-term friendships that some of these folks end up having. But look, these folks have done a number of interesting things and different things over the past few years, and I’m going to walk through some of the different things that they’ve done. And as I go through them right now, nothing that I say will probably stand out as being totally novel or super sophisticated. But they were very different when the adversary started doing some of these things, and it forced a lot of organizations to think differently about how they defend themselves.
And so, if you think about, you know, the actor sending text messages to people to convince them to click on links and then provide their username and passwords. Well, just think about, you know, how do corporations protect SMS messages? Most corporations don’t. I mean, they protect inbound emails. They look for malicious content or things that look like phishing content. But for the most part, most corporations aren’t monitoring inbound SMS messages. As an employee, we don’t want them to monitor SMS messages because we sometimes send, you know, very personal things, and we don’t want our company seeing that.
And so if somebody receives a phishing SMS and they tap on the link on their phone, that traffic going to that phishing site isn’t routing through the company’s enterprise, you know, network, or it’s not, you know, being inspected through their corporate controls. It’s getting egress out to the internet, through the cellular provider for the person’s phone. And so, a person could easily type in their username and password and even one-time password to this, you know, fake website, that’s controlled by the attacker, but looks real. And so, a lot of us have had to think about how do we protect, you know, organizations and individuals that are phished through SMSs?
Another thing we saw is, we saw the threat actor essentially compromise virtual or, you know, virtualization environments, whether it’s on-prem or in the cloud.
So, for example, if they were to log in to a company’s Azure environment, we’d see them create and deploy brand new virtual machines that are standard, vanilla Windows deployments that don’t have any of the company’s, you know, EDR tools or their security stack. Same thing for on-prem environments. So, what does that allow the actor to do, but allows them to run a whole bunch of attacker tools within the premises of an organization without having to worry about EDR blocking the execution of the tools.
Now, of course, if they compromised a computer that has an EDR running on it, then yeah, you have some controls that could block you. But in terms of, you know, running, you know, massive scans across the enterprise or, you know, trying to run a bunch of tools in the environment, running down on a vanilla Windows computer on the company’s network, it’s pretty effective for them to do that.
We saw this group deploy encryptors on hypervisors. Now, look, we see that all the time today. But, you know, as EDR software gets better and better on Windows computers, we find that, you know, there is no EDR software on VM or hypervisors. So, we see a lot of threat actors, not just Scattered Spider, but a lot of other groups deploying encryptor on hypervisors.
We saw Scattered Spider essentially create rogue identity providers and essentially attach them to companies, you know, Okta environments or Azure AD environments, such that they could essentially create global enterprise admin accounts and essentially do a Golden SAML-type of attack to access any resource within the target environment network. Golden SAML is essentially a technique that … just think of it as a way in which an adversary can access any single account or any single resource within a company environment.
And the other thing we saw this actor doing is they use commercially available, you know, remote access solutions like AnyDesk and TeamViewer and, you know, remote assistant and a variety of other tools. Again, not necessarily super clever, or like, you know, complicated to do, but they were the first to do a lot of these things. And the challenge that I think a lot of companies had at the time is that they just simply didn’t know what Scattered Spider was doing. And so, they didn’t have great defenses against it. But as soon as they learned what Scattered Spider was doing, it was very easy to create triggers to alert on certain things that are very high fidelity indicators of compromise.
So, for example, seeing a rogue identity provider getting attached to your Okta, it’s a very trivial thing to be able to detect. But if you don’t know that’s what an adversary is doing, you may not necessarily jump on that right away when you see it happening. And so, we were trying to educate companies as much as we could on this is the tradecraft that they’re using, and it’s easy to detect them once you know what to look for. And, by the way, these guys, they didn’t try to be quiet; they wanted to be as loud as possible. And a lot of times they did things that were disruptive. I mean, we saw, you know, members of LAPSUS$ log in to you know, Slack instances saying, “Hey, I hacked into the company network, you know, I’m the hacker.”
And we had other, you know, we saw other employees of those victim organizations kind of laughing at it, thinking that it was a joke by employees. And maybe one, you know, kind of comical situation, which is kind of indicative of, maybe, yeah, some uniqueness about some of these operators, as you know, the guy, Arion, from LAPSUS$, the, you know, at the time, he was probably 15- or 16-years old. He had dialed into a war room for one of our incident response engagements. He didn’t connect over audio or video, but he dialed into the, you know, to the bridge on his computer. And he screenshared and he showed his desktop and he started logging into a whole bunch of the victim’s websites and resources. And the speed in which he was typing and the speed in which he was switching his screens, it actually felt like it was automated. It was faster than an average human being, you know, typically operates at the computer. And it’s just really fascinating for us to see the speed that he was operating at. That call was being recorded. So, we actually had to watch the playback and slow it down to see what websites did he log into, what usernames did he type in. And it was kind of a, you know, it’s not very often where you do instant response, where you see the actor actually logging in and you kind of have to type everything out and, you know, document everything that he did.
Vorndran: Jamil, go ahead, because we’ve run out of time here pretty quickly.
Farshchi: Yeah. So, you did a great job of highlighting what the threats are, how these folks operate. What advice, what recommendation would you have for organizations, given the threat landscape as you’re seeing it today?
Carmakal: Yeah. You know, the probably the first thing I’d recommend is, do a red team exercise where you’re emulating a lot of the techniques that Scattered Spider and other groups will emulate. Again, it sometimes feels like a different kind of red team exercise than a lot of it … you know, the red team exercises before we saw Scattered Spider a lot. But allow your testers to call up employees and try to convince them to do something. Allow them to send SMS messages to employees, allow them to do a lot of the techniques that an adversary like Scattered Spider might do. A lot of times these pen testers, or these red teamers, are really focused on can they get, you know, malware payload on an endpoint?
And with EDR getting better and better, it gets harder and harder to get, to get malware and endpoints. So don’t get too hung up on getting malware on endpoints. Think about what could you do if you had credentials to an environment? Could you log in to SaaS applications? Could you access Salesforce? Could you access SharePoint? What all could you access with the credentials that you have? Can you phish other people and get access to their usernames and passwords and then log into resources with them? So, I think it’s really important to test, you know, from a real-world adversarial perspective, you know, defenses for an organization.
The second thing that we love to be able to, you know, to ask organization to do is do a tabletop exercise, you know, do it with a number of different audiences. Do it with a technical group of folks to see how they respond to a security event. And you could do that by way of a red team exercise or a purple team exercise. Do a management-level tabletop exercise. Have members and management test assumptions. A lot of times, I think, a lot of executives that haven’t lived through a tabletop exercise or a real event. They just assume the CIO or the CISO’s going to kind of own all things cybersecurity during an event. But there’s a lot of responsibility by a lot of different people, and it’s good to test and challenge those assumptions.
It’s also helpful to have the board involved in a board level or maybe a hybrid, you know, board and management, executive/board tabletop exercise. It’s good to understand what decisioning does the board want to be involved with? You know, nowadays they probably want less decisioning, but it’s good to test out the hypothesis and the assumptions. A lot of times boards want to be made aware of security events much earlier than what I think management might want to notify them with. And so, it’s good just to test things out. So, the tabletop exercises are super important.
I’d say those are probably the top two. And there’s obviously other things, but those are probably the top two that will, you know, get people, you know, meaningful outputs.
Vorndran: Charles, let me build on those, just for a second. You know, when we talk to boards about this, right? We really do focus on that, the exercising piece, but really refining decision making and having streamlined communications in all directions and testing that, and understanding and having people understand where they actually fit into that, right? Whether than the board, whether in their operational management execute … executive level. I think that’s important.
The only other thing I would add to the offer, the audiences is: plan, if you do, suffered intrusion, just plan to move to out-of-band communications immediately. I mean, Charles gave an example about the why behind that. We have dozens of similar examples, but just plan for that. You know, assume executive cell phones aren’t going to be usable, because they often aren’t, or they’re not trusted.
Jamil, I’m sorry to interrupt. I didn’t know if you had anything else.
Farshchi: Yeah. Two things. Two things I thought were interesting. One: you said it twice today. EDR has gotten so good that we should, you know, it’s very difficult to be able to get malware on the endpoints. And so, the adversaries are changing their tactics. I completely agree with this. I mean, you mentioned credentials. I mean, I think identity is the new perimeter.
And organizations that they’re not focused on that are going to be, you know, in for a bad day. It’s why we went password-less recently at Equifax. The other one is you were talking about the challenge that sometimes executive team—management teams—are a little leery to escalate things to the board, you know, because they feel like it might be a little premature.
A piece of advice I have there that I found works really well? Establish those rules of the road with the board in advance. But what we’ve done is we’ve set up the specific criteria for, “Hey, here’s the thresholds that we will report on and we’ll get it to you as soon as those things …” It just takes any of the uncertainty out of the equation. Obviously, you’re always free to escalate stuff outside of those things if something is to pass. But at least it gives everyone a really strong sense of like, of the criteria for which is going to result in that escalation. And so, there’s no confusion there.
Carmakal: Yeah, maybe just some follow-up thoughts to things that you said. So, I spend a lot of time talking to boards, whether it’s through a tabletop exercise or a briefing post-incident or, you know, previous to an incident. And what I find is that just conceptually, a lot of members of the board, you know, particularly the audit committee chair or the chairman of the board, they … when we do these tabletop exercises, they always want to know about the incident right away. And in their mind, they’re probably thinking that there is, you know, maybe one major incident every year. And this is the only time that, you know, they’d be notified and there’s just ... maybe this lack of awareness that there are incidents that happen all the time. And it’s not always appropriate to notify the chairman of the board of, you know, any type of incident.
And so, there’s got to be some kind of threshold in which you would definitively notify the chair of the board or an audit committee or whatnot. And, and to your point, Jamil, it’s, it’s good to try to define that as best as you can. Bryan, you made a point about going out of band for communications. So, a lot of times, our clients will either spin up an out-of-band Google Workspace tenant or will move to Signal to communicate online.
And just to maybe share, you know, one story on that helps illustrate, you know, some of the controls that you need to put in place. And I share this story not to, you know, embarrass anybody, but to share a learning. You know, we worked a few cases where we used Signal out of … for out-of-band communication. And you could install Signal on your phone, but you could install it on a desktop. And one of the employees that was responding to the incident installed the Signal desktop client on a VDI, a virtual desktop Windows computer. And the threat actor ended up logging into that VDI system, saw that Signal, the desktop, you know, client was installed, and loaded up the desktop client and introduced themselves to us through the Signal chat, saying that, you know, using a lot of colorful language, explaining how bad we are at security and, reminding us that, we should not install Signal on potentially compromised environments.
Vorndran: But it’s a good point, Charles. Like, I mean, I think, you know, terrible, right? For the victim, right? But the adversary is capable, right? And, I mean, this is one of the things that I get … I get asked all the time: “Why can’t we collectively, the government, you know, industry, deter more?” We’re facing a very, very capable adversary. Very, creative. Very entrepreneurial.
So, Charles, unfortunately, we’re out of time. But before I let you go, I’m going to ask you, as we enter 2025: cyber/non-cyber; one book recommendation for our audience. And then I’m going to ask Jamil. And Jamil thinks he knows what I’m going to answer, but it’s not the truth. So go ahead, Charles.
Carmakal: You know, I’m going to struggle with this one.
Vorndran: Okay.
Carmakal: Okay. You know, I’m going to say, I think it’s called, The Perfect Weapon or A Perfect Weapon by David Sanger.
Vorndran: Okay, okay. I’m surprised you didn’t say Sandworm. Jamil, how about you?
Farshchi: I was completely unprepared for this question. You know, this is not a security book, but I find myself recommending it more and more as of late. And it’s not even a new book, but, there’s a, there’s a book by, I think it’s Chip and Dan Heath, called, Made to Stick. And I think every security person should read this book because it dramatically helps frame out how you can improve your communications to be able to make it more tangible and more thoughtful and resonate more with your audience. And I think a lot of us are really good, and we’ve come up through the ranks in the technology side, so we’re super solid on that front. But the communication piece is often the problem that, trips us up most frequently. So, books like that will demonstrably help you in your career.
Vorndran: Good. So Jamil thinks I'm going to say A Gentleman in Moscow by Amor Towles, because he knows I love that book and I do love that book. But my recommendation is going to be Think Again by Adam Grant, which, Adam Grant writes about why people are so susceptible to not believing the truth. Right? And the psychology behind that that makes us all so susceptible to believe things that aren’t the truth. And it’s a fascinating, fascinating book, all based on legitimate academic research.
Well, Charles, thanks.
Carmakal: If you don’t mind, just real quick. I know you said last question, but I do want to make one quick point before we leave. I do want to thank the Bureau. I want to thank international law enforcement partners for all the actions that they’ve taken against threat actors over the past several years. We have seen a notable decline in Scattered Spider intrusions this year. And that’s largely to thank the FBI and a number of global partners for making that happening ... for making that happen, for arresting a number of members. Obviously, there’s plenty more work that needs to be done, but there has been a notable shift in the velocity of intrusions throughout 2024. So yeah, Bryan, thanks for your leadership. Thanks for all you’ve done. And obviously there’s a lot of … a lot of people, smart folks that are working behind the scenes to make that happen. And I’d be remiss if we ended this call without mentioning that.
Vorndran: No, I appreciate it, Charles, but I would just, say without the industry partnerships with Google, with Mandiant, with Microsoft, we wouldn’t be where we are, right? And, you know, that’s quite frankly, the whole soul of this podcast, right? Is that, you know, bringing all of us together. But again, Charles, we really appreciate it. And thank you for helping us get Ahead of the Threat today.
Carmakal: Excellent. Thanks so much, folks.
***
Vorndran: Charles Carmakal, you know, simply put, is one of the most knowledgeable people in the security industry, you know, globally, right? Certainly in the United States. And I think that came through in droves during the last 35-to-40-minute conversation about the depth of knowledge he has on Scattered Spider. But more broadly, just in this case, of cyber criminal threat. And we know that if we wanted to talk to Charles about the Chinese, the Russians, the Iranians, the North Koreans, it would be a similar conversation: which is 100% a trusted voice in the space.
Just a couple things for me that really stood out, right? We continue to hear, the more and more of these interviews and conversations that we have—the threat of voice-generated AI, right? And we heard that again, and we know that the technology is going to continue to scale to make it more authentic. The other thing that I think came through is really these common applications, right? Charles mentioned many of them. One of them is AnyDesk, right? But these common applications that are used by many different companies and organizations, you know, throughout the country that pose risk and really understanding what third-party risks you have, not just in terms of the hardware, and the firmware, and the software, but understanding, like, how that can be leveraged by the adversary to cause problems.
Jamil?
Farshchi: Yeah. It’s come up. It’s crazy how frequently this comes up, but caller-authentication your help desk hands down. Like, there’s no excuse at this point in time. You’ve got to put some effort behind that to make sure you’ve locked that. Two, another one. And this comes up a fair amount, as well. But the preparation aspect that he was leaning in on, making sure that you’re doing these exercises on a regular basis, so that everyone has the muscle memory to be able to respond. I like to always say that you play how you practice. And I think this is a good example of that. And then I think the final one is some of these other vectors that maybe we don’t typically inspect and monitor or protect, such as SMS. And we don’t do it for good reason, because people would, you know, feel like it’s a violation of their privacy and so forth to be able to monitor that stuff. But you’ve got to consider that from a training perspective, so that your workforce and the people, and yourself are aware of what those threats are, and you can avoid becoming a victim therein. So, great advice. Love, Charles. And I hope he keeps doing what he’s doing because he, because he makes us all a little bit safer. Yeah.
Vorndran: Well, thanks, Jamil. And to our audience, thank you for helping us get ahead of the threat. And we’ll see you next time.