A PLC Might Say “Oh No”: Unlocking Comprehensive Asset Visibility with Enrique Martinez

Transcript

John Vecchi:

Well, hello everybody. You’re listening to the IoT Security Podcast, live on Phosphorus Radio. I’m your host, John Vecchi, and we’ve got a fantastic show today. In fact, I’m worried about running out of time to get to all the things I want to talk to our guests today about. And as you know, we’ve spent a lot of time this season as we approach the end of this season of this podcast talking about the OT industrial control system side of this whole XIoT landscape. It’s just been a top of mind topic, what can I say, the entire year. We’re going to continue that today.

And so with that, I’d love to introduce our very special guest today. He’s a titan in this industry, everybody. Those of you who listen to this podcast will probably know him. He’s the OT IoT Security Practice Lead at World Wide Technology. Please welcome our guest, Enrique Martinez. Enrique, welcome to the podcast, my friend.

Enrique Martinez:

Thank you, John. How are you?

John Vecchi:

Very, very, very good. It’s been a bit of time to get you and get the two of us on here together. You’re a busy man, and it’s so great. I think it’s lining up perfectly, as I said, because we’re kind of at the end of our season as we look to a whole new season coming up ahead, and I think it’s a perfect time to have you with us to just talk about, man, the year you’ve had, the things you’re seeing. I want to cover some of the trends. I definitely want to talk about something that I know you’re focused on, which is the whole AI side. We’ve just got a lot to cover today. But look, before we get going, Enrique, for those who might not be completely familiar with World Wide Technology, WWT as we like to call it, can you tell our listeners a little bit about the organization?

Enrique Martinez:

The organization, WWT, World Wide Technology, the company started, well in 1990, so now it’s 34 years ago. It’s still led by the original founder.

John Vecchi:

Wow.

Enrique Martinez:

Originally it was VAR, right? So it was just reselling boxes, if you will. But over time, it’s morphed into this monster, if you will. Right?

John Vecchi:

It’s a big company, man.

Enrique Martinez:

Yep. Yeah. It’s funny because World Wide in the name, but originally it was just a small outfit here in St. Louis, St. Louis, Missouri, but now we’re truly worldwide. Right? We have pretty much every major continent, we have offices and representation. Now we are a systems integrator, a services provider, really just leading the charge in many of these efforts, especially you mentioned it, AI. AI is huge. And recently, our founders made a big investment and commitment to AI, so that’s one of our big efforts right now along with security.

John Vecchi:

Yeah. And you’ve been through the journey, and obviously we’re going to talk IoT, OTICS today. That is your focus area there at WWT, but I don’t think that’s where you started. Right? I think for a lot of the guests we have, we talk about this industry and you know it better than anyone. There’s a handful of people. When you say you’re the IoT OT Practice Lead, you’re probably the IoT OT practice lead within WWT. That’s a huge organization, but you can see there just aren’t a lot of you out there that are those experts. So tell us about that journey. How did you move into this from where you started?

Enrique Martinez:

So I’m a biologist by training.

John Vecchi:

Oh, wow.

Enrique Martinez:

Which is funny because I didn’t really work in biology a day of my life. While working on my degree, I started a small outfit, an interior designer firm, just doing their warehouse. They took an outage in their server and they didn’t know what to do, so I went and fixed it. All of a sudden, I was their systems administrator. Right?

After that, I liked the IT stuff. I self-learned. I got a lot of computers at home. This is back when the dual boot of computers was the thing, and I had multiple. Anyways, from there, I got a job at a financial outfit here in St. Louis, a big one, started doing implementation services and their Knock call center, so on and so forth, eventually found my way to their security department. Once I got there, at the same time, I started my master’s and I took an information security class, and that was it for me. Right? After that, it was like, “Okay, I’m hooked. I love this.” So while working in the security department at the financial firm, I started doing compliance. PCI compliance was brand new.

John Vecchi:

Yep.

Enrique Martinez:

Went from PCI compliance to managing vulnerability management, managing web filters at the time, doing forensics. And one day I had a conversation with the big utility here and they said, “Hey, you do compliance. Do you want to come do compliance for us?” I’m like, “Sure.” So I ended up doing compliance of the nuclear plant.

John Vecchi:

Wow.

Enrique Martinez:

It was really the development and implementation of the compliance program for cybersecurity for the nuclear reactor.

John Vecchi:

Geez.

Enrique Martinez:

And then, from there, it’s all history. Right?

John Vecchi:

Mm-hmm.

Enrique Martinez:

That was where I got my first real taste of ICS and OT and the challenges, right?

John Vecchi:

Yeah.

Enrique Martinez:

How do you secure these assets that were developed for a single purpose? They can’t handle anything else. From there, spent about 10 years in the utility serving different functions, specifically with compliance as well as security architecture. And one day, I said, “You know what? I’ve learned a lot. Because of compliance requirements, the utility industry is so far ahead in security compared to everyone else, let me find a place where I can go and use what I’ve learned to help others.” And then, that’s how I landed at World Wide.

John Vecchi:

Yeah.

Enrique Martinez:

So I’ve been there for five years now. I was pretty much the first person dedicated for OT security. We built a lab. We brought in a very, I call it the cream of the crop of partners in the OT space. And the way we do it, we put them in the lab and we do a vetting process to make sure that the solutions do what they say they do, they perform the way they can. And then, we bring customers in and let them experience it. And to me, that’s a differentiator. Right?

John Vecchi:

Yep.

Enrique Martinez:

We’re not leading with marketing. We’re not leading with just a demo bias. It’s more of, “Hey, here, experience it.” And it’s been very successful. From that perspective, I support every vertical, my mainly utilities and energy, manufacturing, life sciences, but OT is everywhere. Right?

John Vecchi:

It is.

Enrique Martinez:

So retail, global finance, obviously healthcare.

John Vecchi:

Healthcare, yeah.

Enrique Martinez:

So we get to support that and we support it worldwide.

John Vecchi:

Yeah, you just said it, right? And obviously, when most hear ICS, industrial control systems, OT, industrial IoT, they think they tend to just exist within only these kind of energy, utilities, manufacturing environments. But the reality is, the OT stuff is everywhere. And we have a lot of guests talking about the healthcare side, and I always say, “Geez, there’s not only OT in healthcare environments, there’s robotics, there’s industrial stuff.” That’s kind of why I like to say this extended internet of things. To me, I call it extended because these devices extend not only in the types of devices, but they extend throughout about any environment vertical you’ll find. You’ll find all types of these, which makes it particularly difficult.

And as you said, these are purpose-built devices. They’re designed to do a very specific thing. But they’re computers. Right? They’re network connected. And like I always remind people, you can’t put… In your practice on the IT side, you could put many agents on a typical traditional IT. You can’t put any agents on these. Right?

Enrique Martinez:

I believe so.

John Vecchi:

That right there is kind of foundationally one of the challenges when you move over to these types of devices in these environments. Right?

Enrique Martinez:

Yeah.

John Vecchi:

Is that safe to say?

Enrique Martinez:

Yeah. Yeah. The challenges are, again, your typical IT device, your computer, it has the resources available to support, because built with the purpose of, “Hey, it’s going to be a multi-use machine. You can use it for data processing, you can use it for imaging processing, so on and so forth.” So you have a surplus of resources. Right? When we have a PLC, there’s only certain resources that it needs because they know exactly how it’s going to be used, and that’s all they put in there.

The problem with that is that it might have a communication stack, it might have the processing stack, and it doesn’t have anything else. So I don’t have additional resources to run an agent, like you said. I don’t have the ability to, like a computer, you talk to it in any port, it will tell you, “Yeah, it’s open. No, it’s not.” A PLC might not do that. A PLC might actually say, “Oh, no. What do I do now?” And then, stuck. Right?

John Vecchi:

Right. Right.

Enrique Martinez:

Now I don’t communicate back to you because I’m thinking, if you will.

John Vecchi:

Right. Right.

Enrique Martinez:

So those are differences. Right? Traditionally and historically, we did not need to worry much about it because those were isolated. They were not connected. There was no need for it. However, over time, now we want to do a little bit more with what we have. How can we improve our operation, whether it is power generation, whether it is output at a plant. We want to get all the data we can for logistics, for billing, for anything. Right? And in order to do that, now we need to communicate with what I call the lower side of the network, because it’s a lower trust from my perspective, which is the IT side. The IT side, the problem is now you have users that might be savvy, might be not savvy. They might have email. “Oh, they;ve got an email. It’s a link. Then you click on it.” And now you run across issues. Right?

John Vecchi:

Yeah. Yeah.

Enrique Martinez:

So now, if you don’t have the proper security controls to prevent unwanted communication between the IT side and the OT side, this is where we start getting into trouble. Right?

John Vecchi:

Mm-hmm.

Enrique Martinez:

With that said, when I start thinking of security controls across IT and OT, to me the control is the same. Right? You have access control. You might have your network security. You might have, I don’t know, any control. Just think of it, any control. Right? They’re the same. And if you look for example, NIST 800-53 and NIST 800-52, they’re very similar because the controls are the same. However, their application changes. How can you apply an identity and access management on a device that does not have the capability to do it? Right?

John Vecchi:

Yeah. MFA or something like that too, how do you even do it? And most operators don’t even know what that is, let alone, know how to do it.

Enrique Martinez:

No. So you have to move those controls to the network level.

John Vecchi:

Yeah.

Enrique Martinez:

And there’s more complexity there as well. Right? Because what works for IT might not work for OT based on the application of that specific control.

John Vecchi:

Yep. Yeah, yeah. Well there are, as you say, I can only imagine the multitude of challenges you face when you’re helping your clients with this issue. Let’s take a step back a little bit. So about five years you’ve been with WWT.

Enrique Martinez:

Yeah.

John Vecchi:

If you were to just kind of glance back at those five years, from where you started, the kinds of problems and challenges you’re addressing, have they changed over those five years in a specific way? Or is it kind of Groundhog Day in moving from organization to organization, customer to customer? Right?

Enrique Martinez:

I’ll give you the standard security answer. It depends. Right?

John Vecchi:

Yeah.

Enrique Martinez:

No, there has been a change in the scope, I would say.

John Vecchi:

Okay.

Enrique Martinez:

Yeah, you’re going to get the same recurring theme at times. “Hey, we want segmentation.”

John Vecchi:

Yep.

Enrique Martinez:

Usually that’s going to be a universal one because you have institutions that are a lot more mature in their security program than others, but there’s always the ones that haven’t done it. Right?

John Vecchi:

Mm-hmm. Mm-hmm.

Enrique Martinez:

The way I see this is, “Hey, we’ve got critical infrastructure. Well, critical infrastructure is usually more mature because there’s requirements there.” However, you go to some areas of manufacturing where there’s no such requirements. And in those cases, they might have grown organically, they started connecting stuff. Now all of a sudden one of their peers got compromised, and they’re like, “Oh, we don’t want to be the next one.” Right? “So how do we do this?” So this is where we start that conversation. And the first thing they want to talk about is isolation, segmentation, maybe access control. But then, I take a step back and I’m like, “Okay, no problem. But what are your assets that you’re trying to protect? Where are they? How are they communicating?” And in many cases, they say, “I don’t know how many assets I have.”

John Vecchi:

Or where they are or what they are.

Enrique Martinez:

Yeah. So in that case, we have to take a step back. And to me, this is something that I keep harping on. It’s like it all starts with asset awareness.

John Vecchi:

And visibility, yeah.

Enrique Martinez:

You need to be aware of your assets. What are they doing? Where are they? What are they?How are they expected to be behaving? Right? So now there’s an asset behavior element here. So now once I have all that information, now I’m able to come up with the appropriate strategy, not only for segmentation, but for secure access as well as remediation. Right? Why? Because if I have an asset that’s vulnerable to some sort of network vulnerability, but I have the appropriate segmentation controls, maybe even virtual patching based on a next gen firewall, I can actually protect that asset. And that counts as it’s a compensating control. It’s a remediation of sorts. Right? So that risk goes lower on that specific asset. But if I have no segmentation, maybe I need to go ahead and patch that asset.

John Vecchi:

Right, right, right. So again, I imagine that if you go back to the start of your time with WWT and today, that visibility gap, that issue is probably still as prevalent today as it was before.

Enrique Martinez:

Yes.

John Vecchi:

Is that right?

Enrique Martinez:

Yes.

John Vecchi:

Always, it’s what I have, where are they, what they are, what is their risk profile? Right? It’s so tough. How has been your approach to deal with that visibility? Obviously, ultimately you want to secure these systems in these environments, but again, and we say it every day, “You can’t secure what you don’t know you have.”

Enrique Martinez:

Yeah.

John Vecchi:

But what is your approach to dealing with that before you even can then go and do segmentation, do remediation, do secure access, MFA, all those things?

Enrique Martinez:

So historically, it’s been passive visibility solutions. Right? You start deploying them whichever way you can, which is usually, “Hey, we’re spanning off a switch. We’re processing this traffic.” It gives me continuous visibility. However, there’s some element of guessing in those, if you will. The other piece is the coverage. Right? In order for me able to gain full coverage, I have to go all the way from the core to the access level to get that full visibility. Right? From that perspective, it’ll give me all the communications on those assets, which is good.

However, if I have unmanaged switches, I won’t have access to view the communications that are happening within that switch. Right? So those are assets that I might not see unless they talk out. On the other hand, if we’re doing asset disability and we’re starting to deploy, and it’s a large network and they have a lot of sites, how long will it take me to have a full deployment? Five plus years, and then at best I can get maybe 85% of assets?

John Vecchi:

Right.

Enrique Martinez:

I don’t know. Right? It depends on the site. It depends on their architecture. I always say that the way you deploy your security controls doesn’t only depend on what those security controls are as identified let’s say in the Purdue Model. Your operation, your architecture will determine where you put those.

John Vecchi:

Right.

Enrique Martinez:

The Purdue Model is perfect to start with if you have nothing. But if you have a defined architecture, there’s going to be variation. That’s just a guide. That’s your controls. Right?

John Vecchi:

Yeah.

Enrique Martinez:

So that’s one piece. Now, from that, I can get on my integration layer between IT and OT, I’ll have thread detection.

John Vecchi:

Yep.

Enrique Martinez:

Which is perfect. Right? Because if I have something bad going in or coming out trying to communicate with command and control, I’ll be able to catch it.

John Vecchi:

Yeah.

Enrique Martinez:

So what I try to do now is, “Okay, this is taking too long. How do we accelerate this?” Right?

John Vecchi:

Yeah. Yeah.

Enrique Martinez:

Well, for years upon years upon years, we said, “Thou shall not scan OT devices.” Right?

John Vecchi:

Yep.

Enrique Martinez:

Why? Because scans are too intrusive. We didn’t know how to talk to the devices. So what we were doing is doing what we do with a machine, a laptop. We send them a packet on pretty much every port or every couple of ports.

John Vecchi:

Thousands of ports.

Enrique Martinez:

And expect that response. Right?

John Vecchi:

Yeah.

Enrique Martinez:

Again, based on what we were talking about earlier, a PLC will not give me a response on every port, and it might actually hang because it doesn’t know how to process that request. Right?

John Vecchi:

Yeah. I would say asking a PLC if it’s running SMB, it’s probably going to die.

Enrique Martinez:

Right. So now it’s a matter of learning how to talk to those devices so that then we can talk to them, get the information we need, not cause any issues, focus on safety and availability, but then get what we need so that then we can do what we need to do, which is secure those assets. Right?

John Vecchi:

Yeah. And with that, are you seeing, you just said it. Right? When I think of operators and what we call traditionally active scanning with typically the vulnerability management, vulnerability assessment kind of tools. Right? Fantastic for IT assets, they’re allergic to it. It’s usually they don’t even want to, the minute you say the word, it’s just shut down. Is that changing? Because there are technologies now that can safely do this because you can more talk specifically to the device in its native language in the way it’s meant to be spoken to. Is that changing? What are you seeing there?

Enrique Martinez:

It is changing, and so is, I would say the awareness and the acceptability of having such a solution, if you will. And you still have skeptics. Right? When I was in the utility, we had the issue that we had a vulnerability scan that was run every so often. And we had to put whitelists as to where we were going, or blacklists, “Hey, these devices are not to be touched.”

John Vecchi:

Right.

Enrique Martinez:

But once in a while, someone would forget to add the blacklist and they would scan a door access controller that could not handle it, and the result was, “Well, now you lost a door access controller. You have to go replace it because there’s no bringing it back.”

John Vecchi:

Right. It’s dead at that point, yeah.

Enrique Martinez:

Yeah. So from that perspective, now, yes, there’s solutions that we have that will give you the ability to talk to those devices in its native protocol or language. Right?

John Vecchi:

Mm-hmm.

Enrique Martinez:

So we understand each other. We are not being intrusive with those devices. We’re able to really just get information about the device itself without having to guess. Right?

John Vecchi:

Right.

Enrique Martinez:

And then, now we have actionable intelligence on the devices that we can go and say, “Hey, this is what we have and these are the issues that we find in them”

John Vecchi:

Right.

Enrique Martinez:

And I’m seeing more and more, initially it was that segmentation. Well, for segmentation, you need asset visibility, which I think it’s going to be a pervasive issue from now until eternity. You will have to have asset awareness, so you’re going to need it. Right? The method that you do it is changing. Every passive visibility solution that’s out there that initially said, “We are passive. We don’t touch devices because that’s dangerous. We will never do it that way.” Every single one of them has some sort of active discovery or active query capability now. Why? Because now that we know, now that we learned, now that we have the right techniques, we can do it, and it is a lot more effective.

John Vecchi:

Yeah.

Enrique Martinez:

Right? Now, you’re speeding that discovery process. If you have a TSA requirement that you have to come up with a list of assets within a certain amount of time, it’ll take me five years to deploy this. What do I do? Right?

John Vecchi:

Right.

Enrique Martinez:

Well, it used to be that I had a computer or a tablet with a spreadsheet and I would go to every site and catalog each device. The problem with that is I’m a person, I will make errors and I will miss stuff. Right?

John Vecchi:

Yeah. Yeah.

Enrique Martinez:

So now if I have something that’s connected to the network, it is routable. I have that ERC, which is usually the requirement for security, for compliance, electronic routable communication. If I have that and I can route to it, now let me go talk to it. Right?

John Vecchi:

Yeah.

Enrique Martinez:

If it’s on the network, I’ll be able to talk to it, I’ll be able to identify it, identify its issues, maybe do a risk analysis on it, and then now I’m set for remediation as well.

John Vecchi:

Yeah. Yeah. It’s a great point. It’s great. It’s a very positive trend that we’re going back to we call it intelligent active discovery, that type of discovery that’s more of a probe. Right? It’s gentle, it’s safe, it’s only sending very minimal traffic. But now are you seeing, so on the passive network monitoring IDS type solutions that now are adding active on top of it, do a lot of those still require the underlying passive platform?

Enrique Martinez:

Many do.

John Vecchi:

So in other words, it’s on top of it. So does that mean there’s still kind of maybe that issue with the complexity that a lot of these guys, they’ve had a plan to roll this out, but if they’ve got a hundred locations, that’s hard. Is that still there? There’s still some balancing of that, or is that still a challenge?

Enrique Martinez:

So that’s where I’m trying to get to with many of my customers. Right? I’ve had customers that say, “Hey, I want to buy this passive solution.” “Okay, well, there’s multiple methods of architecting it.” You can do an out-of-band network where you can aggregate traffic. It’s going to be more expensive upfront, but you’re not having to maintain all the sensors throughout your networks. They’re like, “Yeah, that’s too expensive. We’re going to do distributed sensors.” Well, you start doing distributed sensors and a year down the line you’re like, “Holy crap, this is too expensive.” Then they start either pulling back or looking for alternatives. Right?

John Vecchi:

Mm-hmm. Mm-hmm.

Enrique Martinez:

So my goal really is, as I mentioned earlier, you put passive detection, which gives you that continuous monitoring piece for threat detection at your ingress and egress. Maybe put it in front of your most critical system so that you can do that threat analysis continuously. Right?

John Vecchi:

Yeah.

Enrique Martinez:

But then, utilize an alternative, another solution. It’s a one, two punch to actually go through your network, discover all these assets, give you that actionable intelligence on those assets, give you remediation capabilities. And even, let’s just say you use that and you actually use that solution through the plane of monitoring of a passive solution, you’re going to pull those assets into that solution as well. Right?

John Vecchi:

Yeah.

Enrique Martinez:

So now you have a co-witness of sorts between two solutions.

John Vecchi:

Yeah.

Enrique Martinez:

However, one of the most common occurrences of conversations that I have nowadays is, “We need to populate our CMDD. How do we do this?” Right?

John Vecchi:

Yes.

Enrique Martinez:

And if we have the right solution that actually can go and identify those specific assets that are otherwise unidentifiable, unmanageable, whatever you’re going to call them, now that system can become my source of truth for populating my CMDD.

John Vecchi:

Yeah. ServiceNow, things like that. Right?

Enrique Martinez:

Mm-hmm. And that’s where I’m trying to change this conversation into, “Okay, I know that you need to do threat detection. Let’s do it with this system that does the passive, if it’s continuous. Let’s accelerate the identification of assets through an intelligent, active discovery techniques where you will also have remediation capabilities because otherwise we don’t have them. And then, take your time in expanding that passive system if you need it or if you want it.” Right?

John Vecchi:

Yeah.

Enrique Martinez:

We do have certain industries, energy, utilities will have a requirement to do internal network security monitoring, so they will have to have sensors there. Again, that will not give you a hundred percent of your assets, and this is where active querying comes in place.

John Vecchi:

Comes in place. So it’s definitely, it’s clearly, it’s always still on the IT side, it’s a layered approach. You’re going to have multiple things. But look, I think and far to me, far more amazing and even a bit novel to hear you say this word multiple times so far in this podcast, remediation. Seriously, to hear you say that and what you do as seriously a titan in this industry, to hear that word come out of your mouth multiple times, I think that’s even more amazing to me and our listeners probably than just the discussion of active discovery, which has been going on for years.

So I want to just double click on that a little bit. Talk to me about that. Similar to active discovery, touching these devices, we know the state of these devices. Right? They’re 1994. They’re default passwords, insecure default configurations till the cows come home, they’re riddled with critical CVEs, CVSS. It’s battery firmwares, decade-old unpatched, so we know that. But historically, again, just any mention of touching those devices and actually fixing them, not only have we not had really had technology like Tanium and all these other type of things on the IT side for these, but that was again, “You’re crazy.” Talk a little bit about your comfort with saying that word. Talk about the evolution of that.

Enrique Martinez:

I think, and it’s one of those pieces that it depends on the situation. Right? I’m not going to go into a nuclear reactor that’s active and try to do remediation at that point. Right? However, there’s a lot of systems, physical access control systems, that will provide a layer of security for those vulnerable systems that are operating within the reactor. You’re going to have your six walls requirement. Right? So now your physical access control system becomes a critical system, and it is something that we can patch. Right?

John Vecchi:

Right.

Enrique Martinez:

Your security cameras for monitoring, to me, those are one of the biggest headaches. Why? Because I will deploy 10,000 cameras. I will change the password maybe once when I put it in, and I don’t touch them again.

John Vecchi:

Exactly.

Enrique Martinez:

All of a sudden, you have a zero day that affects all your cameras, or let’s say 50% of your cameras. How many people do you have dedicated to those cameras? One, maybe two? Right?

John Vecchi:

And who is it? Who is it, by the way?

Enrique Martinez:

It’s usually a third-party contractor too.

John Vecchi:

Yeah.

Enrique Martinez:

So now you’re able to, “Hey, there’s a zero day. It covers 50% of my cameras, I have 10,000. That’s 5,000 cameras. How long will it take one person to go log into each camera and do this, versus how can I automate that and accelerate that to ensure that our risks remain slow?” Same thing with security cameras. Right? Many places will have a requirement that, “Hey, if Enrique leaves this company for a cause, we need to rotate passwords to everything he had access to.” Well, I know the password to every camera because it’s the same password. And cameras are everywhere in my network, whether it’s segmented or not. That means that you’re going to have cameras that might share space with other type of equipment. A camera in many cases, is a Linux box.

John Vecchi:

It’s a Linux box. Right.

Enrique Martinez:

A stripped down Linux box. Right?

John Vecchi:

Yeah.

Enrique Martinez:

And it’s not hardened. I might have, I don’t know, SSH on it. I might have, who knows what I have in it. Telnet and a bunch of insecure ports that are open by default, they were not hardened. Maybe I know the password. Who knows. Right? Maybe there’s a zero day, now we have a problem. How do we do this? Right?

John Vecchi:

Yeah.

Enrique Martinez:

And that’s where that remediation piece comes in. Right? When we’re talking about PLCs, HMIs, I take that with a little more caution.

John Vecchi:

Exactly.

Enrique Martinez:

Because the consequences can be big.

John Vecchi:

Incredibly high.

Enrique Martinez:

Can be big.

John Vecchi:

Can be very high. Yeah.

Enrique Martinez:

This is where we talk about the difference between going home at the end of the day or not.

John Vecchi:

Yeah. These are cyber-physical systems, mission-critical. Completely get that. I think it’s so encouraging to hear you talk about, I call these OT adjacent IoT in many cases. Does that seem to make sense, these systems that coexist?

Enrique Martinez:

Yep. Yeah. And I classify them, and it depends. Right?

John Vecchi:

Yeah.

Enrique Martinez:

Because you will have some IoT devices that are data acquisition only, and if you deploy them on a parallel network, you have no risk. Right? They’re just getting data and sending it out for processing. However, in some cases, they’re right next to your control system.

John Vecchi:

Yeah.

Enrique Martinez:

They’re IoT devices, which historically don’t have the best track record of being secure, and now they’re adjacent. Right?

John Vecchi:

Right.

Enrique Martinez:

So how do we handle that piece? The other piece would be when we’re talking IoT. And this is where I’m going to go left a little bit and go to a hospital. A hospital is one of the most dynamic environments ever.

John Vecchi:

Yeah.

Enrique Martinez:

You have power generation, you have air handling, you have all the medical devices, you have all the patient entertainment devices. You have video games in each room nowadays.

John Vecchi:

Yeah.

Enrique Martinez:

And then, you have a wireless that in many cases allows customers to connect their phones and their devices. So it’s one of the most dynamic environments. And then, you have people coming in and out and it’s very difficult to track. It’s very difficult to come up with a good strategy to secure everything. And this is again, where asset awareness is paramount to ensure that ability to manage your risk and maintain it at an acceptable level.

John Vecchi:

Yeah. Right? Because you could see once you get a sense of your assets, inventory, what those devices are, and your attack surface, then you could then step back and say, “Okay, what’s the low-hanging fruit?” As you said, some of these other devices we might secure in a hospital, as I say, an HVAC controller and a camera can be life critical devices. Right? You have PDUs and USB, there’s UPSs and door controllers and printers. In a hospital, a printer, those printers that print the wristbands could completely take the hospital operations offline. Any of these things, if you pop them or ransomware, I think hospitals are concerned as well about if all the passwords are default, I could just simply change all the passwords and lock you out, and it could be the easiest ransomware attack you’ve ever had. Right? So that visibility probably gives someone like you the opportunity to have a strategy. And even in a hospital, there will be devices, you do not touch those devices.

Enrique Martinez:

Right.

John Vecchi:

Right? It’s life critical, certain life critical medical devices maybe you don’t touch, but that doesn’t mean you don’t go do a lot of other things. Is that generally-

Enrique Martinez:

No, I think it’s more about being able to create a safety envelope around those devices. Right?

John Vecchi:

Yeah.

Enrique Martinez:

So again, compensating controls. Right? You said something that made me chuckle here a minute ago, and you said UPSs. Right?

John Vecchi:

Mm-hmm.

Enrique Martinez:

UPSs, PDUs, you will connect them to your network. You will not change the password.

John Vecchi:

Never.

Enrique Martinez:

You will never patch them.

John Vecchi:

Yep.

Enrique Martinez:

And in, I would say, I don’t know a good number, but I would say in most cases, if you go to a data center or any place that has a UPS, you will find at least one that has default credentials and it hasn’t been patched. And it is connected to your network in your data center. Right?

John Vecchi:

So true.

Enrique Martinez:

So why are we doing it?

John Vecchi:

Yeah.

Enrique Martinez:

Is it segmented or is it just connected to the network there? Because it’s just a UPS. It’s the system to keep my server running if the power goes out, so it’s fine, it’s fine next to the server. However, it’s a vulnerable system.

John Vecchi:

Yeah. Right. And I do want to kind of touch a little bit on AI, but someone says, “How would you ever shut down AI if it gets out of control?” Shut the power off. Maybe if it’s a big data center, how about if we just go shut the power off? Right?

Enrique Martinez:

Exactly. Yeah.

John Vecchi:

You could interrupt AI pretty easily by just going to that UPS or that PDU and shutting that thing off, which sometimes can be very easy because many of them could be connected to the open internet with default passwords. Right?

Enrique Martinez:

Yep. Yep.

John Vecchi:

That could be quite a scenario.

Enrique Martinez:

If it gets out of control, the AMP would probably be the best way. Right?

John Vecchi:

Exactly.

Enrique Martinez:

AI, it’s one of those things. We’ve been talking about AI for years already. Right?

John Vecchi:

Yeah.

Enrique Martinez:

I have solutions that I sell that I work with that have been claiming or saying AI for 10 years already, AI and machine learning.

John Vecchi:

Machine learning, yeah.

Enrique Martinez:

But to me, the new AI is really more of the evolution of it. Right?

John Vecchi:

Yeah. Yes.

Enrique Martinez:

New ways of doing things. And I think for the OT side, the power of AI really is going to be more on the data analytics side. What do you do with all this data you’re getting out? Whether it’s security data, whether it is process data, it’s really that modeling that you do with the output of what you have. In some cases, I think going back a little bit to that conversation of the control network, you have devices that are data acquisition, you have devices that are control. However, you have devices that are data acquisition that are utilized as an input to make a decision about your devices and take action. When you do that, that data acquisition device to me becomes a controlled device.

John Vecchi:

Yeah. It’s a mixture of both.

Enrique Martinez:

That’s right.

John Vecchi:

Yeah.

Enrique Martinez:

So you can actually now take AI, overlay it on this, and with all this data that you’re acquiring, now you can have a system that makes decisions about the process and take automated action. Right?

John Vecchi:

Got it. Yep.

Enrique Martinez:

Now, we have certain systems that might actually have, if it’s a closed system, I don’t have too much concern about it. But if we have a system that has third-party inputs like, “Oh, I need weather data from somewhere in the internet” or “I need input from utilization of my distributed energy resources that are sitting at customer sites that’s outside of my network, now I have to bring that information in.” That’s a perfect place to bring bad stuff in. Right?

John Vecchi:

Absolutely. A hundred percent.

Enrique Martinez:

So now this is where ensuring that you have the right validation of your data and a proper break in the communication, maybe stage it, validate it, and then bring it in. It’s going to help you really maintain that envelope around your critical assets.

John Vecchi:

Yeah, yeah, yeah. No question, no question. And the other thing, I talk in terms of the number of these devices. Some people don’t sometimes fathom the billions, I predict over 60 billion of these extended internet of things, devices in the world. There’s an order of magnitude more of them than your traditional endpoint device.

Enrique Martinez:

Right.

John Vecchi:

So humanly securing and managing these devices will be almost virtually impossible. You could even say that to traditional ITM once you move to this. And so, is there an AI side that can help with more autonomous kind of security and management of these devices on the OT, IoT side,>

Enrique Martinez:

Yeah. We’re not there yet. I think eventually. There’s a lot of great work being done with developing capabilities within AI. I think eventually we’ll get there. Right now, I don’t think we’re there yet.

John Vecchi:

Right.

Enrique Martinez:

You’re talking about cyber physical systems. Before deploying something like this, we need to be 150% sure. Right?

John Vecchi:

Yeah.

Enrique Martinez:

Because otherwise what’s going to happen? We lose control of a system at a boiler. A boiler explodes, right? Pipeline, you have an issue with the pipeline. Right?? So those are the main, I would call it main concerns there.

John Vecchi:

Yeah, yeah. It’s amazing. Look, Enrique, it’s just fascinating. We’re going to have to have you back to cover all these things, there’s no question. But as we kind of wrap this up, like I said, we’re ending our season of the IoT Security Podcast, and now in production of our next season. We’re kind of entering into a new year. It’s been quite a year on the IoT, OT security side. Almost every day there’s some breaking news of some other type of attack and malware. As you look out ahead, anything you see that probably is in store for us in the coming year that might be interesting in your mind as you look out ahead based on not only this last year, but before? Anything interesting you see that might surprise us a little bit in the coming year or just something we might expect?

Enrique Martinez:

I think it’s going to be interesting. I think it’s going to be more of an acceleration of capabilities, both on the adversary side as well as on the defender side. It just keeps happening. Right?

John Vecchi:

Yeah.

Enrique Martinez:

So it’s just you keep accelerating. Right?

John Vecchi:

Yeah.

Enrique Martinez:

But now with this whole AI piece, it can be used by both sides. Right?

John Vecchi:

Yeah.

Enrique Martinez:

So this is where it’s going to get interesting. But to me, again, going back to what I said earlier, we need to be aware of what our devices are, what they’re doing, what we expect them to do. Right?

John Vecchi:

Yeah.

Enrique Martinez:

So we need to continue and expand that capability, because ultimately that’s what’s going to give us the assurance and the ability to protect our assets. Right?

John Vecchi:

Yeah. Yeah.

Enrique Martinez:

So to me, I think it’s going to be interesting. Right? You see in the news, just recently I saw I think October 31st there was an article about, “Hey, because of IT and OT convergence, now there’s a lot of more people are aware of the need to protect OT.” I’ve been living this for 15, 16 years now, but some people are just discovering it now. Right?

John Vecchi:

Yeah.

Enrique Martinez:

So I think from my perspective is the duty we have is to ensure that there’s the proper education, the proper mentoring, and really just helping all these institutions to get to a point where they can protect and save their operations. Right?

John Vecchi:

Yeah.

Enrique Martinez:

Because this is what we’re running on.

John Vecchi:

Yeah, exactly. Enrique, that’s awesome. And listen, again, you’re out there. You talk about this. People know who you are. But for those of you who might want to get in touch with you, not only you maybe for some advice or WWT, where can people find you?

Enrique Martinez:

So I think the best place is wwt.com.

John Vecchi:

Okay.

Enrique Martinez:

We do have an OT security topic page. I always say, “Go search for OT security. You’ll land in that topic page and follow me.” Right?

John Vecchi:

Yeah.

Enrique Martinez:

There’s a follow button for OT security, and then there’s a link to contact us or connect with us. It’s not just me, it’s just the whole team, very, very good capable team. And if not, an email, it’s [email protected]. I’d really like to have these conversations. I’m on the engineering side, I’m on the background. But I love to provide all this support to pretty much everyone, every customer.

John Vecchi:

It’s been just a pleasure to have you, and we need more of you in this world. Your experience and knowledge in this area is just so beneficial and so needed, and it’s just the perfect way for us to almost wrap up our season here. So Enrique, man, thanks so much for joining us, my friend. And we’d love to have you back again.

Enrique Martinez:

No, totally. John, I really appreciate it, happy to come back whenever you guys want. And it’s fun to have these conversations. I think it’s a good way for us to share what we know and help others. So I’ll be back.

John Vecchi:

A hundred percent, absolutely. We love it. We love it. And remember, everybody, the IoT Security Podcast is brought to you by Phosphorus, the leading provider of proactive cyber-physical system security and remediation for the extended internet of things. Thanks so much to our special guest, Enrique Martinez. And then everybody, until we meet again, I’m John Vecchi. We’ll see you next time on Phosphorus Radio.