Unlocking Resilient Cybersecurity Solutions in Healthcare and Beyond with Jason Taule

Transcript

John Vecchi:

Well, hello everybody. You’re listening to the IoT Security podcast live on Phosphorus Radio. I’m John Vecchi.

Brian Contos:

And I’m Brian Contos. And joining us today is Jason Taule. Welcome to the show, Jason.

John Vecchi:

Welcome, Jason.

Jason Taule:

Great to be here. How are you guys?

Brian Contos:

Well, we’re great and we’re real excited to have you on, man, so thanks for giving us some of your time.

Jason Taule:

My pleasure.

Brian Contos:

Jason, as we kick things off, could you give our listeners a little background about you and how you came up in cyber, and what exactly it is you do today?

Jason Taule:

I’ll give the short answer and we can drill down where you want some additional details.

Brian Contos:

Sure.

Jason Taule:

I don’t think I’m the guy, but I’d humbly suggest I’m one of the guys, and that’s only because of the accident of my birth, what year I was born in where computers were that day. My first computer had no hard drive, it had no keyboard and no monitor. It was a CPU and a processor and I learned how things worked, much like the old term hackers, i.e., people that take things apart to figure out how they work. So I was really there seeing how everything was built. I was one of the first people on the ARPANET before Al Gore’s legislation made it available for commercial purposes. And I can remember thinking, “ow, I’m going to get left behind. This internet thing is crazy and I better get smart about it.”

It’s the only thing I’ve ever done. My first job out of school was with Booz, Allen & Hamilton. 1986, the Computer Security Act got passed and a bunch of federal agencies didn’t have programs for their EDP, or we didn’t even call it EDP then. We called it AIS, Automated Information Systems. So this is the late ’80s. My folks at the team at Booz gave me a copy of the Rainbow Series and said, “Stay a chapter ahead of the customer.” And now I’m a security expert because I’ve done a couple of these projects.

After a while, I did get pretty good at it and I had the very good fortune of working with and for some of the smartest, best people, I helped build the VA’s program. I helped build NASA’s program and I learned really important lessons at each one of these. So for example, if you’re NASA and you’ve sent something in orbit that’s going out to the edges of the solar system and it has a vulnerability, guess what you’re not doing? You’re not pushing a patch to that thing and restarting it because you might take all of the investment and spoil it if the thing doesn’t restart.

Brian Contos:

It’s [inaudible 00:02:47] time to take the satellite down.

Jason Taule:

Exactly. So I followed one of the partners that I had worked with. One of the good mentoring things to tell young kids today that are coming up is align yourself with good people. And so when he left and went to another opportunity to start a consulting firm, he said, “Hey, would you like to be the cyber guy? You can come here, you’re not competing with anybody else.” And I went and I ran and built a multi-million dollar practice for one of the international consulting firms. I expected that at some point, one of my customers and I would come to a marriage where they liked me and I them. That happened, and this is again the late ’90s now, I was one of the first people to become a CISO in any company anywhere. FISMA had come out and I worked a company that built systems for the Department of Health, Medicare in particular.

So now I had this whole federal-mandated program where you’ve got to get a system accredited, earn its right to operate, get its authority to operate and then remain free from compromise. And I was able to leverage all those experience along the way. I followed that for about 25 years because the government then started spending more on healthcare than we do on defense. And if you think about it, it’s kind of crazy that the two biggest buckets is sensitive healthcare information in the country or sensitive identity information right here in Maryland, by the way. We’ve got one group called Social Security and the other one’s called Medicare. One’s got data on every person in the country, every citizen, and then the other one’s got certainly the two-thirds of the country that’s on Medicare or Medicaid.

Brian Contos:

How is it that those programs haven’t been breached yet?

Jason Taule:

I have an answer. It’s not that we’re that good. It’s that most hackers don’t know what a mainframe is, and I know it’s supposed to be cloud-first, but thank God those major systems are still on mainframe systems that are very well secured. I got recruited away by one of the standards bodies. That was probably one of my most exciting career opportunities because now I could take all the pain points that I and other CISOs that I had worked with had because the need to audit yourself or the need to report your posture to somebody else is pretty universal. We’re all realizing this with the solar winds and the supply chain issues that have all come up. Now we’re talking about SBOMs and a whole different way to exchange information.

Well, a SOC report wasn’t exactly the best way of doing it. So I could take these pain points and say, well, this control requirement, I keep getting a finding on it because it’s not clear what I’m supposed to do or the auditor didn’t understand what I was supposed to do or the person who needed to accept the report didn’t understand because we didn’t have consistent scoping and terminology, so we were going to try to achieve that kind of clarity. I think I made some meaningful progress there.

And then now I’m in the third third of my career where I’m giving back. I’m a virtual CISO. I support a number of different organizations. I have a customer in the agricultural sector, one in heavy manufacturing, one in healthcare. And then lastly, I’m also the CISO in residence at one of the incubators in the state. And that’s really interesting because it’s a very different set of drivers for a small to medium-sized business. And I’m able to look at, well, what are the things that are different and unique to each of those sectors, but it’s surprising how much is the same, especially when it comes to things like IT and OT, which we’re going to talk about here in a little bit.

But the agriculture sector has it, the hospital, biomedical engineering devices, and healthcare sector, it’s all over the place. And every manufacturing, all of the manufacturing stuff, a lot of it has firmware, a lot of it’s on network and it has these issues, and frankly, if we’re not addressing them, we’re leaving the bad guys far too many opportunities to get in.

Brian Contos:

So, Jason, that’s a really fascinating journey. I’m wondering, being a virtual CISO, what’s your day-to-day like and are you interacting as much with the executive leadership team and the board like you would as a traditional CISO or are you working with the technical teams and the IT group, or is it a combination of everything?

Jason Taule:

Wow, that’s a six-hour essay to answer that question. So let me start by saying a couple of things that I’ve learned, which is that the CISO role is consistent in its objectives, company to company, but it’s very different one company to the next. It has to be attuned to the company and the culture, the balance that you have to achieve in terms of the risk appetite that the leaders and the boards have. Certainly, it’s also driven by the nature of the information that they have, whether it’s regulated and increasingly what your customers, I’ll use that term generically, what their expectations are. Because look, this is 2024. I can’t imagine many business interactions where I’m not sharing my data with you, and I want you to show respect for me. And increasingly, the privacy question, which is the other side of the security coin, is factoring into it.

So whether I’m wearing that hat or not, I still have to support those objectives. So the role varies with each organization, what they want. For some, I am the CISO, not a V CISO. So I am the CISO. I evaluate contracts, I negotiate terms with vendors. I oversee the program. I have budget authority, I have signing authority, etc. Others, it’s more of a CISO advisory role. And in some respects, the goal is over a series of years, perhaps to put myself out of a job, help them figure out where they are, where they need to be, and then develop a plan to get there.

I think the big advantage of the virtual CISO model over an internal CISO is you get the benefit of the lessons learned, the battle-hardened experience, but you’re not paying for that person on a full-time basis if you don’t need it. Typically, once we figure out where we need to be and we have those programs, I’m overseeing the implementation, but I’ve helped the organization identify or bring in technically competent people to do that implementation.

The challenge, of course, is helping the organization who’s newer to this that hasn’t spent a lot of time thinking about it. In my opinion, the CISO’s role is not to say yes or no. The CISO’s role is to inform the conversation, the leaders of the company, the board, they’re the ones that should be. And do the men and women say yes or no to things every single day? It’s okay to accept a risk, a cybersecurity risk if there is awareness of its existence and you find it to fall within acceptable limits. It’s not okay to accept one by default because they’re not aware it exists in the first place. So that’s where my job, whether I’m an internal CISO or a VCISO is first to make sure that they have that awareness. Then they often will seek my opinion because they don’t know how likely it is.

I think the journey’s also changed a little bit over the 30 years, not just with the migration from traditional CISO to virtual, but the understanding of the people with whom I partner. Initially, I was chicken-little, I was saying something was going to happen that they didn’t believe was going to happen. Does this even happen, was the question. Yes, here’s the data. Now, then it was, it’s not a question of if it’s a question of when and then I don’t know that he was the first person to say it, but the first person I heard it from was Bob West, who was the first CISO at Homeland security when that first got put together. He said, “No, there’s only two kinds of companies out there. Those of them [inaudible 00:10:05] know it. Those of them don’t hack don’t. In which one are you?”

So I actually had a member on one of the boards I served say, “Well, if breach is inevitable, why are we investing anything if it’s an exercise in futility?” Well, the answer that I had in my head, not the one I gave out loud was, well, so-and-so, it’s because there’s a difference between negligence and gross negligence and there’s a difference between a fine and going to jail. Ask Joe Sullivan what he thinks about that right now.

Again, you have to help them understand it is a partnership. Let me give you an example of the evolution. The first time I had a present to a board, I belonged to a CISO round table. Actually, I’m drinking from their cup today, the CISO Executive Network. And I asked my peers, “This is my first time going before a board. What can you tell me?” What are the lessons learned? And they all said the same thing: “Five slides or three slides, five minutes.” I said, “I get that, but what do I say? What do I put on those slides? Because it’s really hard to manage expectations at that level of aggregation.”

I looked around and I realized what everybody was doing, and I think a lot of CISOs still do this, and it’s probably one of my biggest lessons learned. Your job is not to go to the board and beg for money. I mean, that’s an encapsulation of what happens in a lot of cases. No, your job and what I did for the first board presentation, the IT Governance Institute from ISACA, I don’t know if they’re still around, but they had this set of survival kits, they called them, and it was a list of questions that every different level of management in the organization was expected to have answers to. So the one for the board said, “Here are 17 questions the board of directors are expected to be able to answer.” So I laminated, I put it at everybody’s seat at the boardroom, and I said, “What questions do you have for me? I’m not here to present to you. You tell me what level of appetite you have, what level of expectations you have about our posture, how aggressively you want to close any gaps, and most importantly, what level of funding you would like to give me so that I can help you fulfill your responsibilities.”

Well, I took that role as an inside CISO, and that’s very much the definition of the virtual CISO. So in that regard, it hasn’t changed. I think what’s changed, when I was a federal contractor … One of the things we were required to do is certify that our policies and procedures every year had remained adequate and or were updated to address changes in the way we behave, new technology and the evolving threat landscape.

So perfect example, a couple of years ago the world went work from home under COVID. Now we’re dealing with return to the office and what do those questions mean and how do we do that in a way that through virtual technology … There are all those solutions that we had to work in. So that was a paradigm shift. Now everybody in the mother’s worried about AI, and honestly, I don’t care about the technology. I’m always concerned about how we use the technology and is there a false sense of security.

There is an overall flattening of culture right now, I remember my wife and I wanted to buy a dining room table, and my wife has this somewhat misguided belief that everybody in our family should be able to sit at the same table. Okay, big family. I mean, that’s a big table, right? You go online looking for big coffee tables, it’s the same six tables. I don’t care what the vendor is, it’s the same six tables that there’s no choice. Well, that flattening of culture, that is the internet. We all get the same feeds now. If that’s how the AI engine works, the results are based on, frankly, it might be a large language model, but it’s a limited large language model. How do we ensure that it’s got the benefit of the full perspective? To me, that’s a question of how we’re using the technology, not necessarily the technology itself.

So the other thing to answer your question, Brian, is there’s different kinds of CISOs. We all say we’re not supposed to be the doctor. No, don’t say no to things. And I don’t. I am empowered to say no to some egregious things, but for the most part, it’s tell me what you want to do, tell me your business objective, that’s going to get a business strategy. 2024, hard to imagine a business strategy that doesn’t have an IT strategy. And now once we’ve articulated that, now we can say, if you want to do that, that’s going to expose us to these risks. If you’re not okay with those risks, here are things we can do to buy that down to an acceptable level. If you’re not okay with the risk or you’re not okay with the spend, then let’s revisit the strategy or if necessary, go all the way back and revisit the objectives. That’s the conversation.

John Vecchi:

Yeah. And Jason, you mentioned before you were talking about the fact that really depending on the company, your job will differ, right? Was the company do, what’s their objectives? Like you just said, some of those questions you asked the board, I mean, those are different for different companies. So you’ve got kind of that uniqueness of the CISO role based on the company. But on the other hand, you’ve also been at a lot of different types of companies in different industries, right? Whether it’s public sector, you said manufacturing, healthcare, finance. Can you talk a little bit about when you add in different industries, what kinds of challenges does that present to CISOs on top of that, right?

Jason Taule:

Sure. Sure. So the first one is the jargon and the vocabulary of an industry can be very specific. Healthcare is a perfect example. IT existed long before it had made major inroads into the delivery of care. We in IT had terms for things that weren’t in existence within the hospital. Then the hospital started coming up with their own terms for it, right? Informatics, right? Well, what does that mean? Is that IT or does that means something else, right? When we say risk management, that means something to us but that means slips and falls to a hospital. So a lot of times you’re using the same vocabulary, so you got to work through the vocabulary.

Ultimately though, again, going back to that, tell me what you want to do, and my role as a CISO is to find a way to enable us to do that without putting us or our customers and the data at risk. What varies is the ability to go after that. Frankly, the healthcare sector operates on very thin margins. As an industry, it came off of probably the worst financial year in recorded history because the COVID bill came due, and when it happened, we didn’t look at costs. We said, “We got to take care of people, do whatever it takes.” Well, that bill’s come due now. So it’s competing with other things.

The agricultural sector, I really thought I was going to have to talk farmers into buying security. I expected to have a hard time saying, I work in the dirt, I grow things. That’s not a complex problem. I don’t need cyber. Well, you bet they do and they want it. I was really surprised. So if you think about farming, and I don’t mean the small farmer, I mean big farms now that run that John Deere tractor is run by a GPS. It’s not a farmer driving it, let’s put it that way. And frankly, the farmers want it because I bring up John Deere because they were hacked a little over a year ago I think it was, and allegedly it was the Chinese.

I don’t know anything. Why would they do that? I don’t know. Because they have a couple of billion people they need to figure out how to feed. So this goes back to understanding the why does this happen. Absolutely. And if you take the morality out of it, it’s actually a financially prudent strategy. I need to figure out how to solve a solution to this problem. It’s the same problem you’ve already solved. I’m just going to borrow your good idea, your solution. Why would I invest a decade of time and God knows how many billions in R&D if I can just … What are you doing to maximize the yield per acre for a crop? Because I can get that data off of how you irrigate your fields or what chemical mix you use or what your tractors are. So the farmer doesn’t want to have to drive 200 miles to change an irrigation setting. They want to do it from a keyboard. Well, if I can do it from here, they can do it from the other side of the world.

So they’re eager for it. So the problem is, and I don’t want to focus exclusively on OT, but shame on me, shame on you, shame on all of us for allowing IT and OT to evolve in parallel, separately. That’s crazy. I don’t know how we did this. I don’t know what I thought OT was, but certainly, in the energy sector, they’re probably ahead of most because there’ve been long since with SCADA and other purposes. Plus frankly, the energy sector is the most demanding, especially on the nuke side, but they’re using IT to manage the OT. They certainly have interconnected them. Well, in most organizations, firmware. Look, I was a federal contractor. The FISMA rules that were all subject to say, you must … Wherever it says you must scan for or you must patch, you must fix, it says hardware, software, vulnerability.

Okay, well, we all took care of hardware. Then we eventually got around to look at software vulnerabilities. We ignored firmware like that word wasn’t even on the page. And if you say to the vendor, Hey, I found something. If you could even get their attention, what are they going to say? They’re not going to say, here’s a patch. They don’t operate the way Microsoft and all the other vendors do with IT. There’s no patch Tuesday for OT stuff. Their answer is, well, you got to buy a newer version of the thing. Well, but the thing still works fine. It’s a large, I don’t know … I got a manufacturing customer. They got a large steel-cutting laser CNC machine that’s running an SMB 1.0. Can you say, want to cry, right? And so if you talk to the folks, they’re like, well, it’s not on the network, it’s not a problem. Oh, okay. I said skeptically. “It’s not on the network. So the CAD guys, when they come over with a new design, how does that get over here to this machine to do its thing?”

“Oh, I guess it is. Is the guy going to come over with a thumb drive and plug it in?” Oh, of course. And, of course, it’s on the network. So then the next question is, well, what do you think the guy’s going to? Do you think a bad actor’s going to hack into this and cut his girlfriend’s name out and steal? No, it’s an access point that people can use to gain a purchase point and then move east, west horizontally into the data network.

I forget whether it was Target or Home Depot, but they had an exploit because somebody hacked into their HVAC systems. So there is those kinds of demands in terms of the stuff of security frankly has remained largely the same. It’s just a question of the stuff to which we need to apply those controls. Yes, I have to apply it to hardware and I have to apply it to software and I have to apply it to firmware. We had most of everything figured out. I will tell you that the government contracting has changed. They now tend to award things to the low-priced technically acceptable solution. Back in the day, it was best value. And back before, the expectation was that you had your own security program. If a customer wanted cyber and they told you you had to meet it and pointed to some sort of regulation, you could charge. That was part of your cost. So I could align with the business and help earn our award fees.

And I can remember that CMS in particular measured what they cared about. You couldn’t get a hundred percent of your award fees if you had any high findings. Well, that’s a great way for me to demonstrate value. I’m not a tax on operation if I can help the company earn its revenue. So what do we have to do? We’d gotten to a place where I had no high findings. I had 100% vulnerabilities patched. I mean, imagine that today. I don’t know anybody and myself included who can say that anymore. You used to be able to come in, figure out where you were, figure out where you need to be, and put together a plan. Over two or three years, you would close the gap and then you would be in maintenance mode. I don’t know that … I think the end state is moving away from us faster than most organizations can reasonably make progress towards that goal unless you have huge unlimited budgets, and none of us have that. So that’s a change.

Now it’s a much more deliberate decision about how we allocate scarce resources. It didn’t used to be that way. Then we moved everything to the cloud. I get why we did it. But remember, I have the legacy of being there at the beginning. I got shipped off and was part of the team that went to Carnegie Mellon to help build the original commuter emergency response team when we started using the internet because we understood that this could become an important thing upon which we had critical dependencies. And if it didn’t work, it would have catastrophic results. Well, if 100% on-prem, which led to the invention of the internet or the creation of the internet, if that was wrong, then a hundred percent in the cloud is wrong for the same reason.

So let me come back to my agricultural customer. My manufacturing customer, their computer systems can be completely down as long as I’ve got power, the machinery in the plant can continue to make things so they’re able to still contribute to revenue. Hospital, the exact opposite. I don’t know if most people think about this, but the heavy regulator in the hospital space, in the healthcare space is not HIPAA. It’s the joint commission that credits the hospital. And ever since the dawn of technology, they’ve had this basic rule that says you have to, as a hospital, have the ability to continue providing services in the complete absence of tech. Well, what if the cloud’s up, but I can’t get to it because I had a fiber cut? That stuff still happens.

All of us are working from home now. I don’t say all of us, many of us work from home. Many of us rely on that local, whoever your local internet provider is, local internet providers who still know, well, still know, still think they know the right time to patch things. Regularly, my internet provider still thinks 12 noon on a workday is the perfect time because nobody’s supposed to be home in your neighborhood at noon. Well, we’re not in the workforce anymore. We’re here at workforce at home. You can’t take me down in the middle of the day. You disrupt my comms, you disrupt my process. So I think we need to have solutions that build resiliency. To me, that’s much bigger changes and the more resiliency I can build in, the better.

But one last thing is aging infrastructure. It’s everywhere. Some organizations and industries have it more so than others. So a lot of the reason it persists is not that I want to run 2008 server. I want to continue running it. It’s that the thing that’s on top of it still works fine, but it won’t run on a newer version, a later version of the operating system. So what do I do? Do I buy a new version of that thing just because of the underlying tech?

No, but that’s why I want to keep it on-prem. And then, of course, you have Microsoft that’s trying to use nothing against them. I understand why they’re trying to use their dominant position in the office suite space. They want to drive you into the cloud. If you’re not in Azure, at least AD home to Azure or hybrid, you can’t get things like Elpass, that whole thing that happened last March where that thing stopped working. The answer is you got to go hybrid. You got to go to the cloud. Well, not if it’s a 2008 server that I got to continue to run. I need to have that inside with no public access. So those are some of the challenges that we’re dealing with. I do want to talk because it’s you guys. I do want to talk a little more about the OT challenge in particular. Can we go there? Do you have any questions about that?

Brian Contos:

Yeah. Well, Jason, before we jump right, I do want to hit on a couple of things as it relates to healthcare providers. So we’ve worked with a lot of healthcare providers and one of the things that we discovered, and you’re probably well aware of, most of them start with about a 15% deficit because they can’t refuse care. So this was even before COVID. So to your point, they have really, really small margins even compared sometimes to retail, which has very, very thin margins.

Now, the healthcare payers, the insurance side, the sciences, the pharmaceutical, sometimes that’s a little bit different, but speaking specifically to the providers that have a lot of traditional IT but they certainly have a ton of IoT, digital door locks, security cameras, and on top of that, they have medical IT devices as well. These sort of internet of medical things, if you will. Given the explosion of these types of connected devices and how critical many of them are, given the margins that healthcare providers generally have and now to your point about post-COVID, the bills coming due, well, now they have this massive footprint of tech that may or may not be secure, probably not. When you’re having conversations with these teams, with these boards and executives, are they able to understand and prioritize the need to invest in security as it relates to these devices?

Or are they like we’ve got some big bills to pay. We’re going to have to continue to put this on the back burner even though we understand that the risk file is really high, or, and I’m hoping you’re going to say this, they get it and they understand they are prioritizing these because it could have a direct impact on patient care?

Jason Taule:

Wow. So let me start high level and gradually narrow in. So you’re right. There is a substantial difference in the healthcare industry. The parent provider is schizophrenic. The providers have frankly, traditionally a lower posture than the payers. The payers are in the insurance business. Insurance is about risk management. It flows. They tend to have greater revenue. So they’re in pretty good shape. The providers, although if you’ve seen one hospital system, you’ve seen one hospital system, there are similarities and there are certain dominant players that are helping to address a lot of these issues. But I don’t know that you can extrapolate what a board and does and doesn’t know from one health system to the other.

I think there are a couple of challenges that all CISOs face, however, number one is getting that attention. So they’re going to say, okay, I get that there’s this other thing that we’re supposed to be managing. I get that if it’s left unmanaged, it could provide the means for that unwanted data spill or compromise. How big’s the problem? Well, do you have asset management? Does it incorporate all of your IoT stuff? Probably not, right? So right off the bat, if you’re trying to give them an expression of risk because that’s what it’s about, most of them, yes, all of the ones that I work with, certainly, we take a risk triage approach, just like you do in the emergency department. This is our biggest risk. And frankly, for a time when I was the CISO over at CSE’s public health sector, we had a contract with OCR. So OCR, as many people know, is the Office for Civil Rights. That’s the part of the Health Department, Department of Health that enforces HIPAA.

There’s seven or eight people in that office. It’s a very small office. They leveraged other resources. So my team, anytime a complaint was filed against an organization, a covered entity or business associate, we did the initial investigation and we said, okay, first show us your risk assessment. Some people, it was a flimsy little piece of paper. Others, it was a comprehensive document. So the substantial difference is there. I think most organizations now know in healthcare, you got to have that document. It needs to say, here’s what my risks are, here’s where I need to be and here’s my plan. And as long as you’re continuing to make forward progress, gradual risk reduction, I don’t think you’re going to get a lot of trouble. The market could punish you if you still have a breach in between. But generally, OCR is not trying to generate revenue for itself. It’s about helping hospitals improve their posture and showing respect for patients’ data.

If the organization has the appropriate segmentation between their data network and the other networks or network, then you can afford to be more permissive. If there’s no segregation, it’s all on one big messy network, now it’s a much bigger problem. So different organizations operate differently. Some healthcare systems were also comprised of multiple hospitals that may be one system in name or may actually be in one system at the network and the technology level. So again, you’ve got different levels of maturity there that will influence that outcome. I think most boards have an acute understanding. Within the last few years, this OT has been my SOPAC issue probably for 10 or 15 years. It has not been one that I’ve been able to do much about.

In fact, one of the government customers that I supported, many of their contractors were getting findings and they were out to issue a technical direction letter, which was about to give them the freedom not to have to worry about, just patch hardware and software, you can ignore firmware because there were no tools available to do anything with it. Hang on. There were some tools I happened to be part of … Again, because of the fact that I’m in the Mid-Atlantic, there’s a technology transfer program between some of the local cyber folks and Fort Meade and NSA. If NSA has technology that they want us to benefit from, they will help facilitate this transfer. So there’s a company called ReFirm Labs that came out to start scanning this stuff. There’s, of course, as many of your listeners will know, that’s one of the several companies that Microsoft bought to underlie their Defender for IoT program.

So I eagerly awaited that because if I could get that capability under my E5 or my enterprise pricing with Microsoft, why would I separately leverage try to acquire, configure, and deploy another tool? Well, it came out and let’s just say I was underwhelmed. It doesn’t find everything. And as you guys know, it focuses more on helping people find than fix. So again, now you go looking for other tools, which of course, is how I came to you guys. I think there are trade-offs. Again, if you go back to understanding what you have, managing it, and in some cases you are going to choose to accept the risk and be a little more permissive. In other cases, you’re not. And there are multiple ways of solving the problem. Again, if you’ve got defense in depth and you’ve got the segmentation in place, you can afford to be a little more permissive and then watch those interconnection points.

The thing that really concerns me is that hospitals in particular, I don’t want to be disparaging of them, but given that many things competing for their attention, they tend to roll over a little bit on vendors and do what vendors say. Vendor says, for this to work, I need you to punch a hole in the firewall. Okay, so you want persistent access, VPN 443. You want that on a persistent basis and I give you an account. But now that person left and you’re sharing that account with somebody I don’t know. And I’ll give you an example of a vendor that I no longer work with.

Epic … This is not the vendor. Epic is the example that others seek to model. As you may know, they’re the electronic health record in use in a lot of systems. They give their customers a report card on how well the customer has secured and configured their instance of Epic, right? That’s amazing that there’s a company, not only do they care about security, they’re telling you there’s a true partnership here, you need to change these settings. You’re doing things that might be a little permissive, a little too risky.

Brian Contos:

That’s a great idea. That’s a really good.

Jason Taule:

So another company said, Hey, I want to do the same thing. So what do they do? They build a capability to evaluate the configuration of their asset. They deploy it. This isn’t come to market yet. They deployed it into my instance. Well, this thing’s collecting data and it’s sending it back to them. Well, if you don’t understand that, that’s what’s happening. That has all the markings of exfiltration up on C-2. So, of course, our guys get all spun up. It escalates to me. We decide do we have an event? We try to work through it. We do have an investigation that’s going on. First thing we do is we contact the vendor. The vendor says, it’s not us. Oh, that’s worrying. I’m about to engage leadership and invoke an incident response. When I said, do me a favor, just because again, I’ve been around the block a couple of times, and it’s been my experience that much more often than not, things that seem to be incidents are usually somebody trying to do something well-intended, but failed to execute, think it through completely. So it turns out it was this vendor.

The people that we had talked to were the sales folks and the relationship managers. They hadn’t been briefed on this new capability because it hadn’t been fully built yet. The developers, look, you don’t test something in your own production. You certainly don’t test in a customer’s production instance. And they did it because we’d given them access, but they didn’t involve our change management protocols. All the things that they were supposed to do, they didn’t do. Now I’ve got a lot of things to worry about, and I do vet my vendor before I engage them, and I do make sure that I have the appropriate security controls in place, but ensuring that a vendor made a change by working with us through our change management process and was approved to make that change, I’m not there yet.

And I think what many organizations need is a single secure tool to allow third parties access that I can control. I can revoke just as I revoke all of my other accounts in a timely manner. And it’s not indiscriminate and it’s not indiscriminate access. It’s you or the vendor to support this. You go from here to there … and I can control it. That’s also something I think we need because otherwise, this whole third-party supply chain thing is definitely going get us again.

John Vecchi:

Yeah. And that’s such a great point. And let’s talk a little bit. I know the kind of OT ICS side is near and dear to your heart. You’ve been dealing with that for a long time and you touched on a lot of things about it. I mean, if we just step back and look at those devices, they’re in not only a poor state of security, but the state of that security just continues to dwindle. The average age of firmware, you talked about it on these things, is seven to 10 years old. They’re riddled with eight, nine, and 10 critical vulnerabilities. Ports and protocols like Telnet and others are wide open. Most organizations don’t know where they are. Certificates are expired. I mean, they’re a mess for all intent and purposes. And then, of course, you have kind of this issue with the collision, obviously, of IT and OT, and then you have the team. So you have the operators on one side and the network defenders on the other. They don’t really work together too well.

Are CISOs in general kind of, like you, I mean, are they knowledgeable of this? Are they kind of approaching this in a better way today than in the past? Is that improving or is it kind of flat to downward? How do you see that?

Jason Taule:

No, I think they are increasingly aware. Anybody that’s getting any kind of industry briefings and is paying attention knows what’s happening. And anybody that’s been at this a while knows that every time we plug a hole, the bad guys move on to the next hole, and there’s way more holes and opportunities for them than there is for us. So when there weren’t vendors in this space, there were a lot of folks that shrugged their head and shrug … It’s the Serenity prayer. “Give me the wisdom to change the things I can and the wisdom to know the difference.” So if I can’t do anything about it, I have to accept the risk. So with the advent of the tools, I think that there’s been a lot of interest. Then some people have said, well, I don’t want to buy another tool, and they talk maybe their existing vendors, if there’s something that’s already got the real estate.

So if I’ve got a vulnerability engine that can scan OT, and here’s one of the issues, and this was probably the surprising lesson I learned from my agricultural customers. They’re eager for this. The frustration is that most of the security vendors speak IT; they don’t speak OT.

John Vecchi:

Yes. I mean, right, they’re dangerous on those devices, right? Right.

Jason Taule:

Now, I think you guys still use the term scan, although it’s not a scan. You use that term to describe what you do because that’s what the industry has come to understand is I process, I need to evaluate that asset for weakness.

John Vecchi:

Correct.

Jason Taule:

Now you’re doing it by evaluating network traffic because if you’re not doing it that way, if you’re not doing a passive analysis, you could put that thing in jeopardy. It could freeze up, it could lock, it could die, and now we have real consequences. So I think that there is a much greater awareness now that we’ve got to stop with technical-specific policies and procedures, follow the data, and that’s always been the right answer. And I think many people operate this way already. Others out of necessity have to focus on what they can. Sometimes it’s a question of the authority and the sphere and the scope of responsibility that CISO has been given, but that’s okay. Then advocate and call it out and point to it. Look, I would be lying if I said I didn’t feed my external auditors issues every now and then, because if it shows up in an audit report, we got to do something about it.

I’ve been yelling and screaming and couldn’t get people to attend to it. I think the other thing that’s driving this is, well, two other major drivers. Number one, if there isn’t a regulation, and I’ve had the privilege of working in some of the most heavily regulated sectors, and I say privilege because then I’m not the crazy person saying, we have to do this. There’s an external requirement. We chose to go into that sector. We chose to work with those companies and that type of data, and they’re flowing these requirements down to us. It’s an expectation. It’s an understanding. Again, if you don’t want to do that, go into a different sector, pick a different line. But in other cases, certain industries don’t have that. So what comes into play? You’re supposed to do what’s reasonable and appropriate. Reasonable and customary. I’ve never met a reasonable person in my life, certainly not in a courtroom. I don’t know what that means.

The best definition I’ve ever gotten is if 51% of your industry is doing it, you need to be doing it too or have a good explanation for why you’re not doing it. So I see us this issue, we’re at that tipping point. Most people are aware that they need to be doing something about it. If they don’t have an appreciation for how big the problem is, they can work with a vendor, they can do a trial, they can do a proof of value, they can get some of the numbers and go, oh wow, we’ve got a big problem here. No, we’re in reasonably good shape. And then go from there. So at least take that step.

John Vecchi:

Yeah, it’s visibility still kind of a big starting issue. Just what the heck do I have? Where is it? I mean, how can I even do what you just said you need to do if I don’t even know what I have and where it is and what’s the state of it?

Jason Taule:

So here’s the other job of the CISO is when you’re having that conversation and informing them, it’s not just of the requirements, it’s also impact. My goal is to try to figure out a way to do what we need to do without it having an undue impact. But there is going to be impact. And I don’t just mean any change. There’s an impact with any kind of change in any part of our operations. Leadership has to have your back. The CISO has to have people’s back. There is going to be some impact. It shouldn’t be undue. We don’t want to get in the way, but it has to be a yes, but this is the way to do it. You can’t do that thing. As an example, we have DLP engines, and I know for years that when the rules were created, I would try to do research on a new vulnerability or a new virus or a new piece of malware.

Well, the engines are looking at that and they think it’s the thing. So they’re blocking my ability to get to that resource to investigate or to learn how others are addressing it. So I’d have to have an exception rule. So rather than manually process an exception, the tools built so that they could have a rule that would echo back to the user, are you really sure, well, is the organization prepared to entrust individuals at that level? Are people simply going to click through and ignore it or are they going to pay attention to it? That’s kind of where we’re navigating. I think a lot of organizations now, that’s their journey with respect to some of this.

Brian Contos:

Do you think on the OT side that it’s the regulatory mandates that get OT to the level it needs to be? Do you think it’s organizations telling their vendors, Hey, you have to make more secure devices for us, or give us a path to take a device and update it more easily or secure it more easily? Or do you think it has to come from the actual vendors, the Honeywells, the Siemens, the [inaudible 00:41:49] and those folks pushing it down saying, Hey, we understand there’s a risk here, so we’re going to build more secure stuff. And maybe it means you have to pay a separate license for the secure version. I don’t know, does it have to come down from that? But what’s going to push this? Or do we have to have some just massive event to occur before all three of those sides step up and say, all right, we’ve been kind of putting this in the parking lot for a while, but now we actually have to bring it in and we have to address this collectively?

Jason Taule:

Well, jokingly, I would say all of the above, but let me get specific. I think there are really two drivers of change, money and pain, and there’s a relationship between those two things as well as them being individuals. So pain is never let a good disaster go to waste. Hopefully it’s a disaster that happened to somebody else, not you. And then the next question you get from your board or from leadership is could that happen here? Oh, you bet, right? How likely is it? Well, that’s where it could be a little bit different. No, for us, it’s a little less than that peer of ours because we have these things in place. So we’re okay. Oh no, we’re even worse. We better step up and do something immediately.

So pain tends to be the major driver, but money. And money can be bidirectional. I’ve had very good success recently, and maybe it’s because of the state of play at the organizations who are engaging me as a VCISO. So they have nothing. They’re not ready for the full-time CISO, they’re engaging the VC. So that’s kind of an interesting place. One of the things that has happened, the cyber insurance market, most of us are required to have some level of coverage. It’s a condition our customers expect of us. In order to qualify for insurance and get it at rates that are preferable, you have to have some basic hygiene in place. The problem is the insurance questionnaires, this is one of my other soapbox topics, so please indulge me here. There’s a couple of problems.

One, the insurance questionnaires, the applications. First of all, there’s a lot of us that want to say what we plan to do. If we’re going after business, maybe it’s an RFP and the customer says, I need really strong security and you don’t have it, okay, I’ll build it for you. If I win this piece of business, I fully intend to with integrity, I’m going to build you all the things you want. I haven’t invested in it. Now I don’t have the case, but as soon as that project and that business, I can make a sound case. You can’t say that on an application for insurance. You have to describe what is right now. If you fill it out, even if it’s future speak, you fully intend to do it. If it’s not actually in place, it’s called insurance fraud. There’s financial penalties and other consequences, and the insurance company could use your false answer as a basis for denying a claim. So that’s issue one.

Second, the insurance companies don’t know what they’re doing. I don’t mean to insult them. This is a statement of fact. In everything else they do, they had good actuarial data. I’m going to sell you a life insurance policy. You will die. We haven’t solved that problem. You will die How quickly you die. Well, look, you’ve got these indicators, these risk factors. You smoke, you’re a diabetic, whatever, and therefore we’re going to price that policy accordingly. Well, I don’t think they understood. The breach was kind of inevitable. They didn’t understand how prevalent in muskie, but it was going to cost and they took a bath and now they’re trying to make up for their losses on the backs of the industry. 3, 4, 5x in terms of renewals is what we’re seeing. So clearly, I get engaged by my customer’s knees. Help us with this. So I’ll give you a couple of examples.

Some of the questions just are wrongly worded. They’ll say something like, do you use this technology everywhere? Well, just like when we were taking tests, I was worried about the all or none. Oh, everywhere. This is what got Joe in trouble with Uber because he was asked, “Was all this data encrypted?” And he said yes. Even if I had a report that somebody handed me on my way into the courtroom that said 100% of our stuff is encrypted, I would never give a hundred percent guarantee because technology moves. Somebody just stood up a new server, somebody disabled something, who knows?

All or nothing is problematic. Other questions will say, do you use this technology? Well, think about it. If I’m at 99% coverage, I can’t answer the first question. So they’re going to think it’s way more risky than it is. Or if they ask the question, do I use it at all, but I only use it for 1%, now I can take credit for something they don’t think is risky, and I’ve got a huge gap. So they’re not really getting at the question of risk. So what I’ve had to do is provide addendum that go along with the application to put things in context when I know that they’re going to infer a negative conclusion. Perfect example. How many employees do you have? So for one customer, we said, whatever the number was, 4,000 people. Next question, how many of them have MFA? Not how many of them need MFA or how many of those who need have MFA have it? How many? So if you say 4,000 employees, if the next number is in 4,000 risk. Well, hang on a second. This is a manufacturing customer.

They have manual punch clocks for time. These folks don’t have email addresses. They come in steel tip shoes, and they’re working in a factory floor. They don’t need MFA. The office workers have it, 100%, but you didn’t ask that question. So I have to put that into perspective, and I was able to do that the first year for this one customer, their renewal came back at about 3x. I got it almost back to the original renewal by giving them that context the next year, because we had built a plan, here’s our posture, here’s our plan, we’d started making investments and enhancing our posture, lowering our risk. We actually got a reduction in the premium. And the broker even said to us, “I don’t know what you guys are doing. You should do this for other customers, because nobody’s getting a discount in their cyber policy.”

So for us, there’s a huge opportunity now to be able to say, because the return on your security investment has always been a challenge. It’s not I kept the bad thing from happening, and then the bad thing didn’t happen. How do I know that it didn’t happen because of what I did and just simply didn’t happen because we weren’t targeted at all? Here, I was able to say, we made this investment and we got this. In one case, I got a million-dollar reduction in the premiums. I got to change the way I do business. I got to price it so I get a piece of that savings.

John Vecchi:

Yeah, I mean, it’s true that the other complexity, I mentioned this too, Jason, is when you look at the IoT, the OT, all these cyber-physical systems relative to this cyber insurance, and you look at the state of them, I mean, default credentials, 75% of the credentials on these devices are running default. And then change something like low-hanging fruit like that, that can also present challenges. On the cyber insurance side, if you have the huge piece of your estate or your attack surface that kind of isn’t being addressed. I mean, firm or no patching, credentials are default. Is that a challenge as well?

Jason Taule:

Of course. I think there is a PRISMA maturity model, which allows you to get at risk in a much more intelligent and appropriate way. It’s not bullying. There are certain things in life that are bullying. You’re pregnant or you’re not. You are secure or you’re not. I don’t like the term security. I don’t even know what that means. For something to be secure, I’d have to turn this computer off, disconnect it from the network, lock it up in a box, put a box in the closet and put a guard on the closet. Is it secure? I have no idea. I know that it’s perfectly useless in that state. And by definition, we are going to use these things, and by using them, we’re going to put ourselves and our customers and our data at risk. And the question is, how do we achieve that balance? So when I evaluate, a pick a framework, pick a security control, am I doing it? It’s not yes or no. How well am I doing it? Because in a lot of corporate …

So the PRISMA model is basically, I have a policy that says, I will do it. I have a process for doing it. I’m actually doing it. I’m collecting data about doing it, and then I’m using that data to optimize things. Well, there’s plenty of organizations that are not at that level of fifth level of maturity. Some cases, they’re doing it, but it’s not even documented. So that’s good. We’re keeping the bad guy out. Now I have a different risk. What if somebody on my team that I’m relying on wins the lottery and chooses not to come in the next day? I don’t have anything documented. I don’t have anything to hand to somebody to come in, even on a supplemental basis or if I give it to somebody else to say, this is how we do these things. And I think we’re running so lean that that’s more prevalent than people could imagine. I think people are increasingly looking to the cloud to do much of this for them. Strongly encourage that solution. But understand, you can get about 40% of what you need from the cloud provider at best.

You’re buying their infrastructure. You don’t manage it. They care for that. But if there’s an issue, they alert it to you. You need to have a team to follow up with investigation. You need to have incident response. You need to train your people. So, CSA has a shared responsibility model. HITRUST has one. There’s a couple of different ones that basically say you can’t inherit everything. Cloud providers I think, are finally starting to wake up and realize that the more they can do for their customers, the more they can charge for. The reason I went with you guys is because Microsoft solution is not effective. It doesn’t provide me coverage I need, it doesn’t do everything I need, but if there was something that they offered and I could get it by just checking an additional box and it was already included in my enterprise license, how would you not want to at least consider that right now? It may not do everything or it may not do it well, but the labor cost, the maintenance costs, there are other things to be considered.

Brian Contos:

And Jason, as we wrap up here, and this conversation could go on for hours and hours, you definitely have some great feedback. For those of our listeners that are CISOs or other security leaders that have large OT and or IoT, internet of medical things, internet of military things, whatever, that genre, any advice that you can give to them in terms of some approaches they can take to start to get their hands around this and really address some of the risks related to [inaudible 00:51:54]?

Jason Taule:

Sure. Well, number one, admit that you have this problem. If you deny it’s like people in the early stages who denied that they were using the cloud. You have OT. I guarantee you, you have unmanaged OT. Guarantee that as well, and I know that firmware is the stuff that IT never wanted to change. The perfect example probably would’ve been the BIOS. Back in the day, if I mess with the BIOS and stuff goes sideways, I lose my job, so I’m not going to touch it. So you should have some sort of discovery capability if you even need that so that you understand the size and scope of the problem. Then you need to look at your architecture. Is that stuff segmented or not?

Understand that many of the OT vendors … And the numbers on this are, especially in the medical space, are outstanding. A lot of the traffic is not encrypted. That’s a problem that you need to see. We’re not even talking about the devices themselves, but an overwhelming percentage of the devices have never been patched or maintained and we know that … I should look for, see if I can find it, the dawn of the internet, there was something called the San Diego Experiment where these guys, it’s some academic institution in California, put an unpatched server directly on the internet and I think it took like three months for the world to find it and to own it, and then they recreated that same experiment more recently and it took less than a minute for it to be pumped, right?

All of this stuff is vulnerable and if I can find it, if I can enumerate it, all sorts of bad things can happen from it. I don’t know. I bet that you guys, I’m sure have estimates to me, it’s probably about a third of the devices on my network are OT devices. Is a rough count. Well, think about that. We just said the bad guys only have to find one hole. We got to find them all, and I’ve got a great vulnerability management program for my IT. I need to extend that to OT. So that’s the first thing. And understand that there’s a lot more to OT than you think.

What’s this? I don’t care. To me, I’m going to make up. There is no I, there is no O, it’s all just T. It needs to be owned and or managed. Even if you want to allow bring your own thing, it still needs to be managed and your customers have that expectation and the regulators have this expectation. Most of those cyber insurance renewals, I had to fill out a supplement specific to OT, so I would use that as the driver, as the attention getter, especially if there’s a supplement and you look at those questions and you don’t think you have good answers, or if you think that providing those answers are going, you have to pay an expensive premium, I think that’s the attention.

Then you need to start with some policies. Policies at least are a start. Thou shalt not do it this way. I don’t have a technical enforcement control mechanism yet, and a lot of these cases, again, you asked the question, Brian. This has been the case for anything that we got from anybody else ever. If we weren’t asking them or demanding more of them, and I don’t bust Microsoft for not providing more, I don’t bust Amazon for not providing more as a cloud provider. Once they finally realized that there was revenue to be made by providing these services, suddenly they started listening to what we were saying. But shame on us if we’re not more … Nobody’s going to do more if there aren’t consequences for not doing it. Now, I’m not saying that has to be on a punishment side or on a regulatory side. I don’t want any more regulations. We need to just be more demanding. So shame on us if we’re not doing that. I would like certain industry groups to get together.

I have standard security clauses that I encourage our legal teams and our contracts team to get inserted when we have the opportunity to redline a document. Many organizations have finally gotten to the point where they’re addressing the OT threat at time of initial acquisition. This is part of your stack on an ongoing basis. Once you connect it, this is not a one-time thing. For every project, there needs to be the program. I think that we as an industry need to have Yelp reviews because I have to vet that third party’s technology. Well, if I didn’t have to do it myself, if there was an industry that I could look at and say, this has been vetted, it’s how did that app get to the Google Play Store? Epic has an orchard in healthcare. There’s an Epic ecosystem where they vetted that product and you can’t even install it if they didn’t say it was okay. Great. That makes it much easier. We’re solving it as an industry, not as an individual. We’re not each duplicating effort.

And then you also mentioned a very important thing. It’s not just about the vulnerabilities, it’s about the fact that this got configured or didn’t get disfigured. Default credentials. It’s not integrated with your PAM or your PIM solution for those kinds of problems. And one of the other problems that we haven’t even talked about, I haven’t seen any issues of it, but anecdotally, when I was at General Dynamics, one of the programs that we have is the Warfighter network. As you may know, this is how the Pentagon talks to forward deployed in Napoleon’s Day. Armies traveled on their stomachs. Well, that’s when you walk to war. It’s not about food anymore. It’s about intel. What’s over that rise? What’s awaiting me? What direction do I have to aim? Those kinds of things.

Well, that network relied on satellites for communication — satellites that we didn’t put up because at the time we had a space shuttle program, so we relied on our friends in China and France to put those up. What would we do if China said to the United States, Hey, do me a favor. We got a communication satellite. We’d like you to put an orbit for us. We would willingly put that up there, but I guarantee you there’d be a little extra payload on there. I’m not worried about anybody intercepting the data. I’m worried about the fact that bird’s going to point in a different direction and mysteriously go boom. Right?

Brian Contos:

Yeah.

Jason Taule:

So where I’m going to is the embedded firmware threat. I don’t hear anybody talking about this. We hear about SIM jacking, right? We hear about phone cloning, we hear about RF issues, and we all have these special wallets now, so our stuff doesn’t get compromised. If I go into another country and I go to one of those airport vendors that’s selling me a SIM card, I don’t know what’s on that SIM card, and the chip makers have alerts that will alert you to a change in the firmware. What if it was on the firmware when it got made? Nobody’s opening up the chassis of a server and inspecting things at that level. Well, that too. I mean, again, I’m not saying that’s something that everybody has to do. That’s probably beyond the need for many organizations, but depending on what sector you’re in, depending on how critical your information is, you may need to evaluate taking things to that level as well.

John Vecchi:

Yeah. Well, I’m going to assume, and I might be a little presumptuous, but I’m going to assume Jason, that, and I know as a CISO you have a lot of things to worry about. You’re probably worried about that one-third of these estates. I’m guessing it’s actually a real valid threat and you’re worried about it, and that’s usually the question of the day, and it’s been this so insightful. Fantastic discussion.

Jason, thanks so much for joining us. Out there publicly, Jason, is there anywhere our listeners could find you? Is there anywhere you want them to find you? Do you speak publicly? Are there places they should look for you?

Jason Taule:

Yes, to all of the above. I’m on LinkedIn. That is sort of the one indulgence to social media that I will do. I’m on the other platforms, but not in my own name. I have several online persona that I cultivate because if I’m going to be going into deep and dark red circles, I need them to trust me and I don’t want to get knocked out and have the conversation end abruptly before I get to the good stuff. So, LinkedIn is a good place, and then we can start a conversation from there.

John Vecchi:

Fantastic. Well, thanks so much. And remember everybody, the IoT Security Podcast is brought to you by Phosphorus, the leading provider of proactive, full-scope, and unified security management for the extended Internet of Things. Thanks so much again to our guest, Jason Taule, and until we meet again, everybody, I’m John Vecchi.

Brian Contos:

And I’m Brian Contos.

John Vecchi:

And we’ll see you all next time on Phosphorus Radio.