Episode Four - Wendi Whitmore
FBI Assistant Director Bryan Vorndran: Hello, everyone, and welcome to episode four of Ahead of the Threat. Bryan Vorndran here, assistant director of the FBI Cyber Division. And joining me is Jamil Farshchi, the chief technology officer at Equifax. Today on our top three, before we get into our episode, the North Korean IT [information technology] worker threat, Black Basta’s social-engineering techniques, and number three is Blue Yonder and the supply-chain incident that we're currently tracking. So Jamil, welcome. How are you today?
FBI Strategic Engagement Advisor Jamil Farshchi: I'm doing fantastic. Looking forward to hitting this top three.
Vorndran: Sounds great. Well, let's jump right into the North Korean IT worker threat. So, this is a massive campaign run by the North Koreans. It's been a years-long plot for them that's been designed around two primary things: number one, bringing money back to the regime; and number two, trying to surreptitiously install malware into some corporate environments.
And what the North Koreans are doing is essentially having regime-trained intelligence officers and nation-state hackers essentially moonlight as IT workers, and they're making their services available on any number of websites—to include websites that they stand up by themselves. And then, American companies hire these individuals, and they're advertised as remote workers. And so, then, they give them access to their corporate infrastructure by simply sending them a laptop. And the way this scheme works is that there's Americans—actually, Americans, some witting, some unwitting—conspiring with these North Korean actors to essentially set up laptop farms. So, a company would hire an individual who they actually haven't met in a remote-worker capacity.
They would then ship a laptop to the witting or unwitting co-conspirator in the United States, and the North Koreans would essentially remote into that laptop and gain access to the corporate environment. Sure, they would do some regular work, but, obviously, the goals are to bring back money. And, in certain situations, we've seen malware installed in those corporate environments.
These houses that are run by Americans, we refer to them as laptop farms. Sometimes, there's five laptops; sometimes there's 75 or 80 laptops that are serving as intermediary points in this thread. And how this works, Jamil, it's fascinating. I think the FBI has done a really good job with communicating with the country, with corporate America, about this activity.
Farshchi: This has been a big issue that's grown quite a bit. I mean, the number of folks I talked to—peers I talked to—that say that they have dealt with this same exact situation or they've discovered it within their organizations is pretty surprising to me. It just, I think, speaks to how aggressive North Korea is being on this front or how desperate they are or whatever it may be. I think the thing I would ask you is: You know, the obvious answer here is make sure that, when you're hiring folks, you're vetting them out and—to the degree possible—meeting them face-to-face. And so, you have some interaction there, which would help to stem this. But do you have any other recommendations around what companies should do to be able to mitigate this pervasive risk?
Vorndran: Yeah. So, the only way—and the primary way—to avoid it or to mitigate it is to force an in-person interview, right? That is the number-one way. When we've seen companies or organizations that have had challenges identifying individuals who may be remote-worker status—whether they are or are not working with the regime—simply what they do is request in-person interviews. And, very quickly, very easy to determine the person who doesn't show up, who's working for the regime.
You know, but this impacts Americans, as well. We're talking about corporations, organizations here. And that is 100% true. But there's also these individuals that steal PII [personally identifiable information]—the North Koreans, that is, that steal PII to make this possible so that they have a legitimate employee record for what would be an American citizen. And those individuals—those Americans—are receiving tax bills for work they never did. And so, it just turns into a really, really difficult situation—not just for corporations and organizations, but also for everyday Americans.
Farshchi: It's definitely something that I think everyone, if you haven't looked into it before, definitely do some digging and vet out some of those remote workers to make sure that they are who they, in fact, say they are.
Vorndran: Yep. All right. On to Black Basta and their newest social-engineering scheme. So, Jamil, it's, like, what's old is new, right? There's just iteration upon iteration of old-school, tried-and-true practices by the adversary that take on a new shape, a new form. And here we are talking about Black Basta.
Let me just give a little history lesson.
Black Basta came into existence after Conti essentially was fractured back in mid-2022 after an embarrassing set of leaks.
Let's be fair: Conti is a very, very capable set of adversaries. And the group that makes up that group, or the individuals that make up that group. But, in mid-2022, there were what we call "Conti leaks" after Conti actors came out and said, "Anybody that opposes Russia in the invasion of Ukraine, we're going to essentially use all of our resources to attack critical infrastructure in those countries."
And so, Black Basta formed, is one of the elements have formed out of the original Conti group. And here we are now, a few years later, talking about their new tactics. And that tactic looks like this: Essentially, they're sending overwhelming amounts of emails to an individual's inbox, and then, they are calling that individual via their MS [Microsoft] Teams account and essentially claiming that they're the corporate help desk or the organization help desk and offering to stop this massive flow of emails to their inboxes. And it turns into a traditional social-engineering call, where a user who is facing difficult times gets themselves into a position where they're offering information that can provide access to Black Basta. And it's just a spin on the traditional social engineering.
Farshchi: It's a spin on traditional social engineering and also sort of takes a page from the Scattered Spider book of help desk-ish types of attacks. It feels like, more and more, help desks are the, are an attractive target, I guess, for whatever reason?
The thing about this one, to me, is that it feels like if you're able to have good front-end spam controls for email, that you'd effectively mitigate this one—coupled with the fact that if you, you know, in addition to that, if you have good endpoint controls around restricting privilege access and the ability to be able to install software, those two things, alone, would probably completely stuff this one out, don't you think?
Vorndran: Yeah, I think so. But I think it's also important to put back in controls in place. And I'll have a question on that front for you, as well. But let me go back to your comment about Scattered Spider.
Listen: I think when we talk about social engineering, the general consensus is, "Well, that's somebody essentially not thinking before they speak or before they offer information." And while, at a foundational level, that is true, it is really important to recognize that these individuals behind these social-engineering attacks are very, very skilled at their jobs, right? They do this for a living. It is their primary means of income. They're very, very skilled. And so, education within the workforce is really, really important.
Jamil, within your organization—or just based on your experience—what are some things that individuals can do, or organizations can do, to fortify the MFA [multi-factor authentication] process when help-desk requests come through?
Farshchi: What we do is we have authentication for all of our help desk calls. And so, to, you know, you call in and you get the IVR [interactive voice-response] recording that reads off your one-time code that you input into our solution that then authenticates you. So then, when it does the handoff to the help-desk agent, they already know that it's a verified authentic caller. And so, very, very difficult—if not impossible, I guess—to be able to bypass.
So, I think that works out well. I mean, the challenges, look: Help desk, they're the help desk. So they're trying to do their best to be able to do whatever is possible to solve your problem as a user. And so absent, like, really strong authentication mechanisms in the front end, it's difficult to stop. And I've heard from a lot of peers of mine, like, there's a variety of solutions that folks have put in place. Everything from what I just discussed to, you know, requiring folks to be able to, requiring folks to scan their driver's license or their passports. And then, which then goes off in the backend to be able to validate that or getting on, mandating that you get on a video call before you do this.
To me, those things seem like a bridge too far. Not quite as effective, I think, once you put it into practice. But, I think there's generally strong recognition that this is a problem within the space that, for all of the millions of dollars that we typically spend securing our infrastructures from cyberattacks, that this is a weak spot: the help desk itself. And so, while the Black Basta group isn't exactly targeting help desk, they're sort of hitting it in a roundabout way, thematically similar to what we saw from or have seen from Scattered Spider.
Vorndran: Yep. And last thing I'll just add on this is that, certainly, the upfront spam filters on the inbound side will help. And then, I think for anybody out there listening to this, two notes. Number one: One way to neutralize this threat is to not take an inbound call from a help desk and call the help desk yourself, based on the numbers you have for your organization. And number two: For those of you looking for more information, the U.S. government—primarily FBI, CISA [Cybersecurity and Infrastructure Security Agency], and HHS [Department of Health and Human Services] just published a Stop Ransomware guide on Black Basta with this tactic in it on November 8. So, just a quick Google search on “#StopRansomware; Black Basta” will take you to more information.
Farshchi: And one thing: You made a good point there on your former point there with the "Don't take calls from folks from, that claim to be from the help desk." The other thing would be: Make sure, if you're operating help desk within your organization, that you just have it as your standard operating procedure to not actively call folks, because if your workforce is expecting and it's commonplace to receive calls from the help desk, then they're going to be far more susceptible to clicking on those things or to answering the phone call from some of these fraudsters, as well.
Vorndran: Yeah. All right. Blue Yonder. Jamil, what is Blue Yonder?
Farshchi: Blue Yonder? This is why—I wouldn't consider myself an expert by any stretch of the imagination, but, scheduling and time management is what it seems like these folks do. And it seems like, based on the incident, that quite a few organizations leverage them for this functionality, as well.
Vorndran: Yeah. So this is a traditional, you know, we intermix these terms, but "third-party attack," "supply-chain attack." We've talked about the significance of these types of attacks, whether it's by a nation-state or a cybercriminal group in previous episodes. In my words: This is how the adversary has an outsized impact on an industry, on a sector, on a group of companies.
And here we are, again, talking about it. When you have these services—software-as-a-service [SaaS] platforms—that service multiple companies, if there is an upstream compromise in that SaaS platform, it has a very, very real possibility of impacting all downstream customers or, certainly, a majority of downstream customers. And that's what we're talking about here again. It's very, very difficult to run any type of organization today without these types of applications, third-party software platforms. But it remains a very, very significant risk for everybody out there in the country.
Farshchi: I think the key here is that this is, as you describe—this is a very hard problem to solve. I mean, when you go through your vendor selection and things like that, it's tough to, you know, weigh this as the most important thing. But look, I—when you have a third-party program and you're going through the procurement process, I think giving a meaningful nod to the security practices that these vendors have in place, put in place the certifications and things like that, it's really, really, critically important—I mean, now moreso than ever, ever before.
And, look: I have no idea what the security practices are over at this company, Blue Yonder, or any other of these vendors that have been affected by these things. But it does speak to the fact that this is important and not taking a look at them and not at least weighing it fairly heavily when you evaluate vendors is a miss in this day and age because the collateral damage that you can face from these kinds of third-party incidents—we just, as you mentioned before, we see it at time and time and time again, and I don't really see it abating. Like, this is going to continue because it causes a lot of pain. And that pain? These bad actors translate and are able to monetize it for pretty big paydays in a lot of cases.
Vorndran: Yeah, the only thing that I can add, in addition, is when you're looking at your third-party application or your third-party platforms or providers or other vendors, for that matter, what is in the contractual language about what they owe you if they suffer a cyber intrusion incident? Right. That's a really important conversation before those contracts get signed, because your goal should be to get immediate, real-time knowledge with transparent sharing from that company or that provider about what's going on.
Farshchi: Well, that, and also—to the degree possible—push for unlimited liability, as well. Like, you want to be indemnified against those things and make sure that they're going to be able to, they're going to have to pay up and give you whatever remedies are necessary to make you whole after you feel the pain of an incident that they themselves faced.
Vorndran: Great. Well, that completes today's top three for episode four. Again: the North Korean IT worker threat, the Black Basta social-engineering campaign, and then, the Blue Yonder compromise. Now, we're going to go to a previously recorded episode with Wendi Whitmore of Palo Alto Networks' Unit 42.
***
Vorndran: Hello, everyone. Bryan Vorndran here with the FBI—assistant director of the FBI's Cyber Division. And joining me, as always, is Jamil Farshchi, the CTO [chief technology officer] at Equifax. Our guest today is Wendi Whitmore, senior vice president at Palo Alto Networks, and has responsibility for Unit 42, which covers incident response, cyber threat intelligence, and all consulting for Palo Alto Networks. Wendy, thank you so much for joining us today. Do you mind giving our audience a brief background of yourself?
Wendi Whitmore: Yeah. Hey, Bryan, Jamil. First, it's great to be here today.
And yeah, quick background. So, I lead Unit 42. As you mentioned, we're responsible for consulting, threat intel, as well as our MDR [managed-detection-and-response] services, so we kind of say we're like the special forces unit of Palo Alto Networks. We’re the eyes and ears on the ground.
I started in law enforcement, so I started my career as an OSI [U.S. Air Force Office of Special Investigations] special agent focused on computer-crime investigation, and have worked for some names that many of your viewers will be very familiar with. I worked at Mandiant for quite a long time. Then, I left to join CrowdStrike and lead their services team. And, prior to joining Unit 42, I actually led the IBM X-Force team globally. So, I've had a lot of responsibility in incident response and threat intelligence, and really looking at what attackers are doing on a daily basis and how can we help our clients most effectively through these attacks.
Vorndran: Wendi, I just want to say thank—
Farshchi: —Man, you went from Mandiant to CrowdStrike to Palo. You're like—That's the trifecta there. That's incredible. What... so, what—
Whitmore: It’s been a good run.
Farshchi: So, what in the special forces unit—Unit 42, that you guys call it—can you tell us a little bit about what some of the kinds of things are that you tackle? Like, what are the kind of services and what are the kind of threats and things like that you face over there, that you tackle over there?
Whitmore: Yeah, Jamil. So, I think we're most well-known, still, for our incident-response investigations, outside of the threat intel.
And, I guess, let me give a little bit of background here: The Unit 42 was actually founded 10 years ago, in 2014, as a threat-intelligence team. So we're certainly—the brand is known for our threat research and what we release there.
When I joined in 2021, the mandate was we wanted to bring together an acquisition. So, it was the Crypsis [Group] acquisition. Again, that team was really, in Northern Virginia-based, known for a lot of our ransomware investigations, in particular. And I joined to bring those capabilities together, on the threat-intel side. And now, we've also added our MDR services—so, managed detection and response.
And so that team is a global operation at this point, operating all over the world. And, on the investigation side, what we're most well-known for, I think, in particular, ransomware investigations and dealing with cybercriminals and having a deep knowledge there. By the time I joined, we were actually doing, averaging about 1,400 investigations per year.
Now, our investigations tend to be larger in nature and scope. So, we're still around that 1,000 plus-per year mark. And what we've done, though, is really aligned with a lot of Palo Alto Networks’ business, as you could imagine, right? So some bigger G2K [Global 2000] clients throughout the world. And we've seen a tremendous shift in terms of the types of cybercriminal investigations, right? We're also focused on nation-state.
But I think—for the purpose of your question—long answer to, you know, getting into your question, which is, "What are we seeing on the ground?"
Really, it’s just a massive cybercriminal threat and a shift in that space for breaches that I think are more, like, specific in nature when you get to, "Hey, we want to, you know, conduct a ransomware case and command some sort of ransomware payment," to those that are now becoming more widespread in the disruptive nature of those attacks. So, attacks you see actually impacting a customer at that level. And so, that you're seeing the attackers say, "Well, hey: If we can disrupt this business' ability to serve their customer, we're now then getting into a different dialogue where customers can then start to put pressure in the news media, and other sources can put pressure on these organizations," that cybercriminals, I think, hope is helpful for them in terms of commanding payment on the back end.
Vorndran: So, Wendi, it's interesting to me to hear you say that—obviously broad, broad experience and capability and expertise at Palo Alto within 42, but with a real, real keen interest and real good knowledge on the cybercriminal threat. So, how has that evolved during your time at Palo Alto Networks? Can you take us through a little history lesson? Where were we three years ago, maybe 18 months ago? Where are we today in terms of the sophistication of that ecosystem?
Whitmore: Yeah. So I would say to your question, Bryan, let's focus specifically on cybercriminal element, right? Because we could also talk a lot on the nation-state side and, certainly, the insights you both have there. On the cybercriminal element, I think over the, you know, shift of the last three to three-and-a-half years, you've really seen them increase the maturity of their business operations, so really, focus on the sophistication side. And I'll dive into some specifics there. But I think the scale and speed of their attacks are also significantly different than what we've historically seen.
If we look at sophistication, I would break that down into a few areas. First and foremost is the way that these attack groups are running their operations much more like a business, where they've got different divisions and work streams and attackers and team members with different capabilities, right? So, some are very well-versed at the communications; some are of versed at hosting the infrastructure; some are great at writing the malware; some are great at the communications that occur during an actual attack. But then, if you look, you kind of, I would say, zoom out a bit. What we're really seeing is that they're not only interested in making sure that they have great technical capabilities and skillsets once they're inside an organization, but they're actually a lot moreso interested in the business processes of their victims than we've ever seen before. And that's directly contributing to their ability to be effective at disrupting business operations. And so, I think those are very closely coupled and aligned, and their actions are quite intentional.
So, you know, we worked on the CSRB [Cyber Safety Review Board] together, and one of the investigations we did was into Lapsus$. And it was clear that with Lapsus$ in particular, they were targeting individuals to be team members who understood business processes. Right? Who had worked at business process outsourcers. Understood how contracts worked in terms of how does an organization award a contract to a vendor? What does that look like once it's actually in place? Right? There's kind of more vulnerable times, meaning when a contract is first issued, maybe when that vendor is offboarding. And we're seeing that because these ransomware actors and cyber criminals have focused so much in understanding how businesses operate, they're really using that to their advantage to then go in and do what I was talking about, of being disruptive to operations and then using that to put more pressure on their victims and ideally to get higher payments.
Vorndran: Okay. And I mean a lot of that, Jamil, I'll just give me a minute here. A lot of that parallels our picture, as well, here at the FBI. Right? The ecosystem, the business structure is just maturing. It's the bottom line. We see people that are dedicated contractors essentially to, initial access, just by way of example. So, Wendi, it's really helpful feedback. I know Jamil probably has a question. After he's done, though, I want to talk about how well-prepared do you think these victim organizations are? But, Jamil, go ahead.
Farshchi: Yeah. So this is super intriguing and also really, quite frankly, scary at the same time. I mean, I know that back in the day, you hear stories about the bad actors going to try to figure out what the insurance coverage was for future victims, that they knew sort of what to ask for in terms of ransoms. But what you're saying here is they're getting into the nitty gritty of how the businesses actually operate to try to find points of weakness there, or that might maximize the impact therein and it’s super worrisome.
Like, do they just are they just doing a ton of research on these organizations and their supply chains to be able to get this kind of insight, or are they, you know, they identifying individuals internally? I mean, what is their mechanism to be able to identify these business processes that they then ultimately target?
Whitmore: Yeah, that's a great question. I think it's a little bit of all of the above that you mentioned. So first, more effective reconnaissance. As we're all aware, the just volume of mega-breaches that have occurred in terms of data loss, right? So credentials that are out there and available is more than it's ever been before. And so there's already kind of a treasure trove of data that these attackers have access to, via buying and selling it on the dark web.
Unfortunately, we're also all aware and having run businesses that administrators tend to reuse credentials and employees tend to reuse credentials. And so something that may have been captured in a social media breach or some other type of online dialogue is now out there and it's a credential that these attackers are leveraging to see if they can get initial access into an environment.
They're also then becoming more sophisticated at the social engineering pieces. And the reconnaissance. So LinkedIn is such a treasure trove of great information, right? Understanding not only who works where, but oftentimes will have employees list the types of technology they're proficient in, the types of software, the types of hardware, their certifications and all that’s great for a business environment, but it's also great for these attackers to do reconnaissance in an environment and know exactly what types of technologies and vendors your organizations are working with. So they're certainly being able to leverage that.
And then when we talk about these modern cyber-criminal groups and this is an area Bryan can probably add a lot more detail in. But many of them have native language, English language speakers at this point. And that's vastly different than it's been in years past. And so they're able not only to leverage those skillsets in terms of those language-speaking capabilities, but then you couple it with the advent of generative AI. And now we've got where, you know, the communications piece is much less of a barrier than it used to be in previous years.
Vorndran: So, Jamil, I'm going to jump in here on this. One of the things that we've talked about, right, Wendi mentioned Lapsus$. You know, we talk a lot about really for the last two years now, Scattered Spider. English-speaking individuals, that doesn't mean they're U.S.-based, but English-speaking individuals and how, their social engineering capabilities are very, very sophisticated.
But how a lot of these help desks are measuring things that are driving bad behavior. So I would guess that most help desks are measuring time to clear a call, time to clear an inquiry. That is directly in contrast about what value those individuals should be driving for an organization and allows very sophisticated actors to take advantage of this, you know?
And Wendi mentioned the generative AI piece. You know, we don't consider it straight cyber, but it is an initial access, which is phishing. Which is we've seen about a 5,000% increase in phishing attempts over the last year. And the reason for that is because the ability to build what appears to be native English language phishing emails is through the roof now, with generative AI, right? We can tell ChatGPT or these other systems or services to build a letter, build an email that says ‘XYZ’; it sounds legitimate because it is legitimate, and the ability to generate that in plain English language is very high. So I would agree with everything Wendi just said.
Farshchi: I just, yes… Do we measure those things, you know, call handle times and stuff like that for our help desks or call centers in general? Sure. But just look at the name: ‘help desk;’ ‘help.’ I mean, this is what their job is. And so, yeah, I mean, it's a thoughtful attack vector, I think for these guys to target. It's always discouraged me when people say that, ‘Hey, this is some new... they're super advanced and sophisticated because they're native English speakers.’
I mean, that doesn't seem super advanced to me. Some of the recon tactics that you mentioned, Wendi, I think those are those are meaningful. And getting that kind of insight certainly gives you a leg up and with social engineering. But, the worrying thing for me on this front is that this is just we've heard this, I think in almost every single interview that we've done for this podcast: is that credentials number one, attack vector, everyone uses them.
The proliferation of all the breaches out there have disseminated so much PII, so much personal information, that and credentials themselves that it makes it very easy for these folks to do this stuff. It's just over and over and over again. And to hear you say this yet again, and this being a predominant factor in terms of all of these incidents that you're seeing the, what, 1,000-plus that you're seeing a year? Just really scary, despite the fact that it's still something that we, you know, a challenge that we've all kind of faced forever. And then compound it all -- and here's my monologue -- compounded all with you know, you know, AI, voice cloning, and potentially deepfakes, on top of it, as well. Just sets up for a really scary backdrop, I think, for all of us.
Vorndran: So, Wendi, we talk a lot about, you know, the current threat landscape. And you've just done just a great job outlining that, especially for the cyber-criminal threat and knowing that you could do the same for the APT (advanced persistent threat) space, as well. But let's shift gears here a little bit and talk about, based on your observations and what Palo Alto is seeing in the IR (incident response) space and cyber-threat intelligence space and the consulting space, what are your core recommendations about how organizations should prepare? It's really something that Jamil and I are trying to drive through this forum, which is, ‘Okay, here are the key takeaways to understand how to best prepare.’ What would your thoughts be on that question?
Whitmore: Well, so, a few things. Let me start, kind of dive into a little bit more because we talked about, you know, what are attackers doing when they're outside the perimeter? But we haven't talked as much about what they're doing inside the perimeter. And I think that's the key to, part of the key, to your question and something that you guys covered with Kevin (Mandia). You know, I heard him say one of the hardest things to test is resilience, right? You actually oftentimes unfortunately have to kind of go into battle to really understand, like where are the gaps in your system and how are those going to impact you in terms of actually during an attack.
So, one of the areas that we see once attackers get inside, that just enables them to move incredibly quickly is the lack of network segmentation that exists in so many environments. And we're not talking about just, you know, having in it most organizations don't have an entirely open flat network, but what they don't necessarily have is the amount of access controls and authentication, at the identity device and multiple layers within their environment. And so that makes it really easy for an attacker to get in and start moving through different divisions. And then pretty quickly identifying, ‘Hey, what's the data that we either want to steal or that we want to encrypt, or that we want to steal to potentially extort you for later?’ And the reality then is when we're working with these security teams and other types of individuals within the victim environment, if we don't have access to that data to get those answers quickly, then it makes it really tough to answer some of those questions.
And so, in terms of how organizations can prepare more effectively, I agree with everything. You know, I've watched the previous episodes: really interesting perspective from the CEO (Aron Ain). Certainly very interesting perspective from Kevin (Mandia). I agree with the testing. I agree with the understanding, you know what kind of exploitation, like, kind of, approved exploitation, right? In terms of pen (penetration) testing and assessments that you're doing in advance.
But I also think that we have to build environments with visibility built in so we can quickly answer these questions. So that's everything from having visibility at the endpoint, having it within the cloud, having it at the network layer and then within an environment, having whatever your zero-trust strategy is. Right?
Like, Bryan, you and I, I think I've talked about this a long time, because in investigating these attacks for over 20 years, we were recommending zero-trust. Before zero-trust was a common phrase any of us use. Right? It was that concept that, ‘Hey, you, when you're operating in an environment like this, you have to treat every communication and every connection as something that's potentially, bad.’ Right? So something where a bad actor has compromised an email communication, maybe they've compromised an edge device. It doesn't have as much visibility onto it. And so you have to be able to assume, ‘Okay, I'm going to treat these as these are bad. And then I'm going to layer in defenses based on that so that I can more effectively get an attacker out once they're in an environment and secure the communications that I do have.’
So that really, comes into play from the very beginning in terms of how we're looking at devising networks, and certainly then as we're in these environments where we're responding to attacks, we're looking to do that after the fact, often. So, if it's a ransomware actor who's looking to take an entire system offline, we need to set up new, clean infrastructure and then move as much of the data over.
So, whether that's applications, services, and do that in a way that allows us to contain, you know, the actual communications and make sure that what we're bringing over to a clean environment is not also still compromised. So, I guess high level, up-leveling on that if we want to, you know, synthesize this into something more, you know, more concise, is just looking at how we have effective visibility at all across the network, and the host and the infrastructure within the environment, so we can get answers quickly.
And one of the things that we're seeing more organizations do is actually having pre-built an environment that they would potentially move towards in the event of a disruptive or destructive attack. And I think the more that organizations can be thinking about that as they're going through and game planning, working with these tabletop scenarios, bringing in their executives, the more effective they're going to be at instantiating that with a moment's notice in the event of a breach.
Vorndran: Wendi... go ahead, Jamil, go ahead...
Farshchi: When you say a pre-built environment, that's more than just having, you know, HA (high availability) pairs for your devices and things like that. Can you explain more about what your what you mean by that, by a pre-built environment and what exactly, what assets and what services would, would be within that environment?
Whitmore: Yeah. So I, you know, I would separate the infrastructure piece from all of the applications and service layers, right? Because those are typically the more challenging parts we get into when it comes to, and I guess, let me take a step back. So, when we're talking about the concept of rebuilding or creating a new clean environment, the idea is that when these organizations are breached, they typically will take a part of their network offline and services. And oftentimes those are the services that are connecting to the vendors on the back end that allow them to actually do business and deliver business to their customers. And so then as we move forward, those vendors and those partners need to actually receive some sort of certification. Again, depends on the industry and the regulation that's impacting it. But oftentimes, they need to receive some sort of certification that, ‘Hey, now I can reconnect to this clean environment because it is clean. It is safe for me to connect to.’ So, what we're often talking about is having infrastructure set up, that will enable organizations to do that quickly.
So that's having a separate network, having access to network controls. So firewalls at that device layer, being able to have that pre-built in an ideal situation, or at least be able to pull it out pretty quickly so that then that serves as a baseline infrastructure. So you have maybe your minimum cloud connections through that, your minimum hardware that's set up for that organization. And then over time, connecting or reconnecting those critical applications that allow the business to do its work. And as that goes on, being able to continually certify it, that it's clean so that all of those vendors and the partners can continue to transact. And then re-instantiate the business services. So, if you have some event infrastructure kind of pre-built, maybe it's that you've, procured some of the hardware that's needed in advance. Maybe you've got that set up and it's completely disconnected, but you have the ability to then instantiate it and it's within a matter of hours or days. That's something we're starting to see a lot more organizations take very seriously and do in a manner that's they're saying, ‘Okay, we know these systems are down, but we can operate these baseline, you know, minimum operations in a matter of hours if we need to.’ And in an environment that you don’t have to play whac-a-mole in because, you know, it's clean and operating and effectively. Yeah. Right? Right?
And Jamil and Bryan, I don't mean to interrupt you. Jamil, you're really familiar with this in the sense of like, you don't necessarily know how much access the attacker has immediately. That will come out over the course of hours, days, and in some cases, you know, they may be sitting on credential access and not using any of it for some time because depending on who it is, if it's a certain nation state, right, they may be a lot more patient. If it's a cyber criminal, we've generally seen them be less disciplined with that.
But that's changing too, right? Where we're seeing them really apply some of the same discipline that we've traditionally seen nation-state actors have. And so, you know, you don't want to move too quickly and say, ‘Hey, we think we're good to go.’ And then they're still compromised devices or compromised accounts and attackers, especially on the cyber-criminal side, oftentimes, you know, if they're going to wait on that, then they're going to come in and actually just start wiping and blowing out environments afterwards and being pretty destructive. So that's where this, you know, having a totally new, network we can set up is really critical.
Farshchi: When I was at Los Alamos, there was a breach at one of the sister labs, and they tried to contain that thing for many, many weeks. And finally they gave up. They had to pull the plug from the internet entirely for that exact same reason as you're describing there, because they just could not figure out where the bad actor had credentials and access in areas they did not discover, and every time they tried to bring it back, they realized, ‘Oh my gosh, this environment's still infected.’ So yeah, it's really good advice.
Vorndran: So Jamil, I want to touch on that. And then I have another question for Wendi. You know, Jamil what you're talking about, I get this question all the time, ‘Hey, Bryan, has such and such victim guaranteed that they've evicted the adversary?’ And my answer to that question is the smart ones generally never guarantee that. Right? Because the sophistication, especially of certain APT actors, is so, so high, to the point that you're talking about a loss at Los Alamos, it's almost impossible to guarantee that. Right? So, Wendi, your commentary about the different and setting up this environment is really, really interesting to me. It would seem, right? I don't do this type of work on a day-to-day basis or a week-to-week basis, but it would seem that keeping the environment updated and current would be a very significant amount of work. Do you see IT staff organic to organizations, being tasked with doing that, or is this a contract element of most organizations to stay in this type of environment up and make it, current?
Whitmore: That's a great question. So we're not seeing a huge amount of maintenance that's required in this new environment before they're set up. Right? It's more, ‘Hey, we've procured the hardware in advance, and we've procured whatever the software licenses and other types of technology that's going to, serve as kind of the foundation for that environment. And we've got that on standby, right?’ And so maybe it's not even connected to ideally, right, it's certainly not connected to the primary network, but it's set up and it's something that, ‘Okay, as soon as we need to go we can stand this up.’ It's kind of like having a go-kit on steroids, if you will. Right? And so there's not a lot of maintenance required. I've primarily seen organizations where that's in-house. Right? They're more mature organizations. They've got the resources to think about that. They're probably highly regulated. And so, they've got their in-house staff kind of having that prepared and ready to go versus it contracted out. But they haven't done a lot of work to kind of continuously update it.
Vorndran: Okay. That's really helpful. Wendi, what else in terms of preparation? I mean, we talk a lot about, you know, and Jamil and I have been very specific, doubling down on the basics, but the basics are actually really hard to do at scale. Right? So we've moved away from these terms, ‘basics and fundamentals’ and said ‘do the tried-and-true cybersecurity practices,’ but what else? Is that your, is that how you coach your clients? Is that what you're seeing? What are some other things that people should be doing?
Whitmore: Well, you know, we mentioned tabletops and Kevin (Mandia) covered those right in great detail. And I think the concept there is just, practicing as comprehensively as possible before an actual breach occurs. So what I would say, is related to that and then related to testing some of the bigger gaps is oftentimes there's key gaps that are identified during those scenarios.
During the penetration testing and vulnerability assessments that occur regularly, but they're not always remediated. And, Aron (Ain), the CEO you spoke with, I thought, covered this so well when he talked about their breach and talked about, hey, you know, the subset of clients that were really impacted were those that had not moved away from a particular subset of the software or their technology, not because the organization hadn't recommended it or because something bad or nefarious had occurred. It was because those clients loved that technology. It worked really well in their environment, and that's so many times, often the case. Right? And so, having those conversations and testing out where those gaps are and then really pushing to close them and remediate them prior to them becoming an issue is key. And oftentimes this is just about the push-pull of the business. Right?
And so we as technologists can come in and, you know, give you a 45-page report on all of these gaps. And we do this really, often with our own CISO. And we work very closely with our team at Palo Alto. And there's a real conversation that occurs of like, ‘Okay, hey, you guys came in and you said none of these systems can have admin rights.
But here's the reality of what that means to the business, right?’ This means that this sales team or this engineering team is it's going to require and force their job to take, one specific task might take three days instead of five minutes. And that's not real. That's not reality for the business. And so the more that we can start having those conversations, which I think means technologists are speaking business language and business leaders are also more educated about technology and really, building a culture and a company that security is everybody's job. It's not just the CISO’s job or the CTO’s job. It's truly every business leader. I think the more effective we're going to become, because that's where we still see a lot of the gaps occurring.
Farshchi: It's such a good point. I mean, that we oftentimes in security get so stuck in our ways or so, scared of every ghost behind every single door, that we oftentimes forget that, hey, there's a business here that we're trying to run, that we have to be able to work with to find manageable solutions to this stuff.
And, you know, compromise is the name of the game. You just got to find, you know, you can think outside the box and things like that, but compromising and working with folks on the business side to find the best possible way to manage your risk is critical. Otherwise, you run into exactly what you just said. You identify the risk, put it up there on some I don't know, risk register and it just sits there. Nothing ever gets remediated. Nothing ever happens. And then the bad thing happens and you point back to and go, ‘Oh, well, we knew about this. We just never did anything about it because the solutions weren't viable at the time.’
Whitmore: Right.
Vorndran: So we're running out of time with Wendi here, Jamil. So I want to shift a little bit to the future. Wendi, these terms are out there, right? Gen AI, Quantum, even Post-Quantum at this point. From your business perspective and the work you do with clients, both in a reactive and a proactive posture, talk to us about those new emerging technology spaces. And what does it mean to your business? What does it mean to your clients and the victims?
Whitmore: Well, I think the biggest, you know, words on everyone's tongue right now is certainly related to AI and the impact that has on the business. And a lot of people want to look at that in a really negative sense. Right? And we can look at it of in terms of, hey, attackers are leveraging AI technology to get better at their jobs, right? You highlighted that earlier in terms of just the barrier to entry from a communication standpoint, whether that's written, whether it's verbal, whether it's video and deepfakes, we're certainly seeing that be leveraged by attackers.
But on the flip side, I actually look at it as quite positive in terms of the impacts that we're starting to be able to bring to businesses. I briefed a board of directors two weeks ago where we were talking about the work that we're doing with our team and the threat landscape, and I was able to tell them, hey, based on the investments that you guys have made related to automation within your SoC (Security Operations Center), you've actually now taken, a savings of 223 days, meaning 223 days, times eight hours a day, times a human working those hours and actually automated those tasks now and, for them, that was in a way that many of their peers had not, based on the industry that they’re in. They were highly advanced out of that. So we saw true savings. And then what that meant is they were able to do a much better job of defending their environment with the same amount of people.
And it also solved a problem that we all have, which is a skill shortage. And keeping our employees challenged, right? It's one thing to hire and attract great employees. The other challenge for me is retaining them, right? How do we keep people engaged? You need to give really smart people consistent challenges. They want to solve puzzle pieces and hard problems, and feel like the work that they're doing is making an impact. And so the more we can automate so many of those tasks that we've all done previously to allow our smartest people to focus on challenging or solving the most challenging problems, I think the better off we're getting.
And so we're actually seeing a lot of benefits with AI, not only in the investments we're making of our technology that we provide for our clients, but also our internal workflows and how we conduct investigations and how we can learn from previous investigations and accelerate that workflow so much more efficiently for our clients.
Vorndran: So I'm going to, just double down on what Wendi said about the positives of AI. One: business related and one: entertainment. Right? So the business related one is just we have very specific experience in the recent past where AI has been instrumental to understand how a PowerShell was built, how it was coded, and what it was actually being used for. That would in 2 or 3 years ago, would have taken us multiple days to get through. And the ability to understand that and scale that, you know, is spot on. And I would agree.
On the more entertaining side, a very good friend of mine who I was with this weekend, he, apparently is using GenAI to build birthday songs for each of his kids. There's five kids, so he gives it 40 inputs about their personalities and a genre, and they each have their annual birthday song, which apparently is fairly entertaining for him and his family. So, Jamil, any thoughts from you?
Farshchi: Next level, man. AI is going to change the world. All right, one last question for you, Wendi. This has been fantastic. You mentioned, the discussion you had with the board of directors recently. What are their, what are their insights are you getting; what are the questions are executives and directors asking you as you go through and have dialogues with them about this stuff?
Whitmore: I think the number one question and, and I think actually Kevin (Mandia) might have said the same thing, but was, they always want to know, well, ‘How do we compare to our peers?’ ‘What are we doing that's working well, what are we missing?’ Right? ‘What should we be doing?’ I think also when you look at that, the discussion with Aron (Ain), the CEO, his commentary on, you know, related to boards asking the right questions and having the right level of deeper dive insight into technology issues versus just a financial and audit, right, is really key. So I'm starting to see boards ask more of those questions and really become more engaged and understanding, ‘How can we be more resilient?’ ‘What are other organizations doing when they're going through a breach and it is disrupting their business?’ ‘How are they making decisions relative to attacks and what we should or should not be doing?’
So I think we're starting to see a lot more of those kind of specific, deeper, insightful questions. And the more that dialogue we have across those different audiences, the better off are going to be.
Farshchi: Totally agree.
Vorndran: So overall theory being trust but verify, verify, verify. Right? Which is what we've heard consistently. I remember Aron talking about that about well, ‘How do we actually know this to be true?’ Right. And really not taking that on face value. Well, Wendi, as always, just thank you, for helping us and being part of this.
For our audience, I've been fortunate to know Wendi for about three years now, and there's probably not a better American out there and just appreciate her service to our country and what she's done for the broader cyber ecosystem for over 20 years now. And, Wendi, thank you for your time. We really appreciate it.
Farshchi: Thank you, Wendi.
Whitmore: I'm honored to be here today. So, thank you both.
Vorndran: Okay. Well, that concludes this episode Ahead of the Threat. Wendi. Thank you. Jamil. Thank you as always and everybody out there. Have a great day.
Whitmore: Thank you.
***
Vorndran: Well, what a great episode with Wendi Whitmore of Palo Alto Networks’ Unit 42. I usually kick this part of the close-out of every interview to Jamil, but I'm going to lead here, Jamil, for just a second, if that's okay? I mean, I think Wendi's conversation and commentary about network segmentation and identity management in different areas–and her terms, not mine– “visibility built-in” -- was a really, really fascinating conversation.
You know, broadly, we refer to it as “zero trust.” Very difficult to do at scale. But her “visibility built-in” piece, I thought, was just fantastic. And then her other commentary about prebuilding an environment to move to when there is a problem. I think that's the first time I've heard of that becoming a best practice within industry. So just two really, really key takeaways from her comments.
Farshchi: Yeah, it was it was fantastic insight there. Man, I have a ton of notes that I took through this thing. You know, make sure you're testing your resilience, eliminate credentials because those seem to be a massive threat. And we've heard that, I think, from basically every one of our guests so far. The lack of network segmentation that you highlighted. Authenticate your help desk calls. That was a major topic there. Visibility and the environment, pre-built environments.
And then the thing that also keeps coming up is boards and their interest in using comparators for, you know, where are they relative to the other players within the space? Tons of great insight and a lot of stuff that's quite frankly, pretty actionable right off the bat. So I thought it was a really great discussion.
Vorndran: Yep. And you know, the last thing I'll close out with here is just, you know, this conversation always about is it net positive, net negative with Gen AI?’ And I really appreciated Wendi's comments about all of the positive outcomes she's seeing in the cybersecurity space for from GenAI. Well while everybody that completes what will be episode four of ahead of the thread. We thank you as always for joining myself Jamil Farshchi from Equifax. And in this episode, Wendi Whitmore from Palo Alto Networks’ Unit 42.
Farshchi: Thank you everybody.