Episode 22
Cybersecurity News Round Up with Willie Hicks
Willie Hicks, CTO of Public Sector at Dynatrace, joins Carolyn and Mark to discuss the top Cybersecurity news stories so far in 2022. Willie offers his expert opinion on the White House Executive Order on Improving the Digital Government Experience, the recent Log4j vulnerability, and the Pentagon's new Zero Trust office.
Episode Table of Contents
- [00:43] Unpacking the Biggest Headlines in Cybersecurity News
- [08:21] Major Catastrophe
- [16:03] Cybersecurity News Highlights the Highest Level of Vulnerability
- [23:59] A Quantum Shift in Cybersecurity News
Episode Links and Resources
Unpacking the Biggest Headlines in Cybersecurity News
Carolyn: Today we talked to Willie Hicks, Dynatrace public sector CTO. He’ll unpack some of the biggest headlines of late from the Executive Order on Transforming Federal Customer Experience and Service Delivery to Log4j. I know Willie, you're so sick of this topic, but we're going to cover it anyway, and then Zero Trust Thunderdome Awards.
I want to go first to the Executive Order requiring improving the digital government experience. Willie, will you give us the big takeaways from this Executive Order? What does it mean for our agencies?
Willie: First of all, I think that the Executive Order on Transforming is transforming the federal customer experience. It is going to impact the agencies, but I also think it's going to impact the digital citizens of the day, the real customers of the federal government. I think President Biden reiterated this, it’s supposed to be a government for the people, by the people.
We're trying to put people back into the equation. I think the big takeaway for me is that the federal government is coming back into or getting to a point where they're really understanding that customer experience, well, they already understood it. But they’re really starting to internalize and figure out how to make customer experience like the customer experience most citizens expect to see with anybody who shops on Amazon, anyone who does a Google search.
A Fundamental Shift in Customer Experience
Willie: They expect, with the push of a button, that they got all the groceries shipped to them the next day or the same day. That kind of experience you do not get with the federal government today. I think that we're seeing a fundamental shift now, not just that kind of digital experience, but I think across the board. Like when you even walk into a brick or a mortar building, when you interface on the phone with a government employee, I think we're going to start, hopefully, seeing more customer-focused, customer-centric type attitudes.
This is really long overdue. I've been in this business for many years. I remember one of my early visits to a federal agency that will remain nameless, but I was speaking to this agency about what we call our digital user experience. How we need to focus on the real metric who's the end user. Right now, you are focused on the back end. You're focused on, is the server up or down? Is this process running? Do I have availability for this device? No one's actually really looking at the end user. So how do you know they are getting a good experience? Not only are the systems running, but are they running efficiently? Are they getting transactions back in a timely manner, or are they frustrated?
I remember one engineer saying, "Well, why does that matter?" I'm like, "It does matter because they're your number one responsibility. That is who pays your salary." This person, an engineer, actually said to me, "Well, there's not another X agency. It's not like they're going to go somewhere else. This isn't Amazon or another commercial entity. If it doesn't work, they'll come back later." That was the response. And I was like, wow!
Smooth Government Transactions
Carolyn: It makes me think of when my dad died a few years ago. We wanted to give him a full military burial, but we couldn't find the papers that we needed. He had shown me where everything was except these particular papers that we had to have to get him this burial. We spent hours and hours online trying to track them down. But, we never did, we never were able to find them. Ultimately, it came down to us, calling Camp Williams and saying, "Colonel Ford is gone and we need some help."
They stepped up and did it, and it was awesome. Fast forward to today, I'm in Utah and we've got billboards all over saying something about finding my cash.gov. I was like, all right, I'll bite, I want to see how easy this is. You guys, it was so slick. In 10 minutes I put in my name, I think I put in my address. There was a quick database search and it said, "Oh yes, you've got money here from these closed old accounts. We'll send you a check." A week later I got a check. I'm still blown away that I did a government transaction and it happened that smoothly.
Willie: Unfortunately, that's been the exception not the rule except for certain agencies that I work with. I work closely with organizations like the VA. To your point, the VA has been making a lot of great strides to improve their customer service, their image and so forth. A lot of that is around, I think, customer experience. Or I should say the veteran experience and making sure that they're putting veterans first.
The Cybersecurity News Features How Backend Systems Work
Willie: They're putting a lot of investment there to understand not just how their backend systems are working, but how the actual end-users are performing, how quickly that transaction took.
Carolyn: But to your engineer's point, why now? He's right. We have to have these services. We're just going to come back later or we're going to give up.
Willie: Well, I would say right now, why now? It’s because to be quite frank, that's the wrong attitude. At the end of the day, that person was right. It's not like I can fire this agency and say I'm going to take my business elsewhere. But there are ways we can speak at the voting booth, calling our senators and our representatives. Getting those types of attitudes changed by having them hold in front of Congress and ask, "Why are you treating my constituents that way? We pay your salaries, so we expect the same kind of response that they would get from any other service."
I think that's what we're seeing with the administration today. They focus on the need to bring the people back into the equation, and that the citizenry understands that they are our most important priority. We treat you that way in everything you do. If at the state level, it’s going to the DMV, which I think everyone dreads, hopefully this will translate down to the states.
But going to the Social Security Office to get a new Social Security Card or going to, in worst-case scenarios, I think this is even going to translate into disasters. How quickly do I get disaster relief? How quickly do I get relief because I just lost everything from a flood?
Major Catastrophe
Willie: When there’s a major catastrophe, how do I get to the right organization to help me just making sure I have the right avenues? I've seen reports lately where agencies, due to COVID, were slow to respond. They were slow to get PPE out and things like that. Those things, they are customer service. But there are consequences also to bad customer service. People don't get the services they need and they get sicker because they don't have that. They don't do that.
Mark: This is a confidence in government issues, Willie.
Willie: 100%.
Mark: I hope that this Executive Order has the staying power that generates money to put behind it. I mean, really it's a nonpartisan political issue that impacts all citizens. However, I think it's being leveraged a little bit politically. I do feel whatever administration is going to implement services and things like that, that they're putting out there for citizens to take advantage of, you have to have confidence. The citizen has to have confidence that the government can actually deliver. If you can't even access the application and the information online then you lose confidence that the government knows what they're doing.
Carolyn: Before we leave this topic, are there any teeth to this Executive Order? Are there deadlines? I mean what is it other than saying, "Yes, we need to do better."
Willie: Yes. That is always the issue that we have, how's this going to be implemented? How is this going to be upheld? Because unfortunately, it is an EO, it's an Executive Order. It is not codified in law. It's not like there was a bill passed on customer service.
Hard Deadlines on Cybersecurity News and Updates
Willie: Although there are ancillary bills that do cover some of these topics, I think of nothing that's more all encompassing as what we're seeing in the EO. So, does it have teeth? We'll see. Right now, I don't know of any hard deadlines that are imposed. I think they're really putting a framework in place to do all of these things. I'm sure the administration wants to be able to report by the end of the term to make sure that they are seeing progress.
My hope in all of this is that, in this period of time, even without the teeth, agencies will be forced to think about these things. They will be forced to internalize some of these things. Moving forward, regardless of administration, these ideals are self propagated throughout the agencies. These continue on regardless of the law and regardless of what's in place. At the end of the day, there are certain things that should be done just because they're the right thing and not because it's a law. It’s not because someone has to tell you to do it.
Carolyn: We've started the conversation, which is a really good thing. So, all right, let's go to Log4j. Just school me, what is it?
Willie: I won't bore everyone with the details, but I think because everyone's really probably heard a lot about this. You've heard of Log4j, Log4Shell. In a nutshell, it's an extremely critical, extremely severe vulnerability in a component. It’s a module of what's called Apache, a kind of server technology that is utilized.
Carolyn: That everybody has.
Full Court Press
Willie: That a lot of people have. Probably, I've seen millions, maybe billions of instances of this module that is distributed across multiple platforms. It could be in embedded devices, it might be in servers. Honestly, the scope and scale of it is unprecedented. That's why there was this full-court press.
Carolyn: Unprecedented beyond SolarWinds?
Willie: I think, I won't mix the two per se because with SolarWinds, we've got documented attacks. We know that our adversaries have taken advantage of this. They were lying in wait in some of these systems for months. They’re slowly making moves, lateral movements and so forth when they get into the system. With Log4j, I think it's still early to find out the true impact of it.
At the end of the day, I think from a scale standpoint, yes, because SolarWinds is a commercial product that agencies trusted. They brought it in-house and they left it behind the gates. Unfortunately, supply chain issues were there, which allowed for some malicious code to be in that product that was almost like a Trojan Horse. It was brought behind the gates, and then they were able to take advantage of that.
Log4j on the other hand, was a vulnerability in the code, but that was propagated over not just a few 100 customers or 1000 customers. I don't know SolarWinds' customer base, but whatever that customer base was, versus something that was just distributed across millions of devices. So really different scales.
Carolyn: Who did the Log4j?
Willie: Well, it's a vulnerability. It wasn't like a supply chain issue. As far as we know, it wasn't like somebody planted this.
Security Vulnerability
Willie: It’s just something that has been there for a while, just a security vulnerability that wasn't accounted for. Once it was discovered, it was figured out you could use this. You could exploit this to take over a machine. So basically to get remote code execution capability, so you could run remotely.
Mark: That was open source wasn't it?
Willie: Yes. So there's a whole other conversation we can have. I don't want to get into that about open source and so forth. Although open source has its merits and its benefits because a lot of eyes are on it, sometimes these things still happen. It also depends on the open source. Some open source projects are very well maintained, and very well scrutinized.
People are always looking at it, tinkering with it, understanding when they find a vulnerability like Log4j they quickly bubble that up. There are some open source projects that aren't so well maintained, but people still use them. But the vulnerabilities don't come out as quickly.
Carolyn: We're a month into this. Have we got it under control? Do we have our arms around it? What did agencies do to manage this?
Willie: I would say, well, that's a loaded question. Do we have our arms around it? I would say we are, well, let's just talk about what happened. Because I can even speak from a company, from my perspective on what our company has done around this. Immediately after at least it became public, I think it was December 9th that this was released or became known, this was escalated to the highest levels.
Cybersecurity News Highlights the Highest Level of Vulnerability
Willie: CISA made this a highest-level vulnerability. They instructed agencies to start immediate searches of their systems. I personally know of agencies we worked with where they might have had a team of 100 people plus over weekends. They were going through servers, looking for unfortunately, sometimes looking manually for this vulnerable code to remediate it. We quickly got our hands around it from that standpoint.
Is there still Log4j out there that we haven't caught? Of course. I know we haven't gotten to eradicate every bit of vulnerable code. But, there was a very concerted effort, especially at the federal government level. I know at the commercial level, the corporate level, and the private sector, there was an equally frantic push to get this taken care of because it is a major vulnerability. You don't want someone with the ability to remotely execute code on your servers. They could do anything they want at that point then once they get into the system.
I would say that, time will tell how well we got our hands wrapped around it. From our standpoint, we were able to quickly analyze our code to quickly find out where our vulnerabilities were. We’re able to quickly notify our customers, our government partners, and agencies of what our vulnerabilities were, and how we were remediating them quickly. We had patches out, I think on that day, to make sure our systems were patched. I think our SaaS environment was doing that. We quickly accelerated our testing cycles to make sure that we didn't break or blow up anything when we applied the patches.
Mark: You're also helping our customers with their application security module. Identify those vulnerabilities, yes.
Core Capability
Willie: That was the internal, but from an external standpoint as Mark accurately pointed out, I was saying earlier that some of our customers were manually looking for this. Luckily, for our customers that have agents deployed and had observability by Dynatrace, we actually had a capability that is core to our platform called AppSec. It’s designed just for this, to look for vulnerability.
Within 10 minutes of the announcement, our databases were updated with this vulnerability signature and pushed out to all of our customers that were connected online. Then for those customers that received that, they immediately, who were using the AppSec module, they immediately started getting flags all over the place of where this module was.
Those people who had AppSec and had our agents fully deployed didn't have hundreds of people. They were going to their Dynatrace consoles and were seeing all of the vulnerable systems. Then we actually had remediation steps built into the platform where they could see what they needed to do to remediate that. We were able to take that from a multitude of people down to the team that was monitoring the system.
Then we distribute that information to the admin teams or to the automation teams so they could use Ansible or whatever they were using to automate the remediation of that. It’s really powerful when you have that visibility, that observability into the system. There was no better example of how important it is for any type of DevSecOps organization. We can talk about it from a zero trust standpoint. I can even talk about it from just understanding the build of your systems.
The Last Big Boulder Cybersecurity News
Willie: There's all this talk about bringing legislation around companies. Having a software bill of materials, SBOMs, built in as part of their products so you can see exactly what components are in software. It’s something we don't normally do in the industry. Having that observability so you might not have that SBOM, but we can light up and say, this is how that application is built and all the components. Really invaluable.
Carolyn: Let's go to our last topic, the last Big Boulder news item, which is the Zero Trust Thunderdome contract. I'm going to be honest when I read this seven million contract developed, to develop zero trust architecture, I thought this was already happening. So what is the significance of Thunderdome?
Willie: I will preface this by saying I'm not a security expert, but I can talk to it from my industry perspective. But the Thunderdome award, which I like the name by the way, is seven million dollars. It's just a prototype to prove out the schemes and the technologies that they're going to be using. They’re technologies that DESO wants to use to build out their zero trust architectures or to validate the zero trust architectures that they've been developing.
To your point, there are pockets of the DoD. There are pockets of the service branches that have been already investing in zero trust. So, you look at programs like Platform One where this is already being built into the platform. It was part of what was really, I think, revolutionary about the Platform One environment and what they were trying to do at Platform One. This was already being built-in in DESO.
What Happened With Log4J
Willie: The DoD had already been investigating this for, I'm sure, several years. But I think what happened with Log4j, SolarWinds, all the ransomware attacks, the administration basically has put a stake in the...
