Framing The AI Governance Challenge

Speaker 2 0:00

Welcome to the Van Rien Compliance Podcast with Rob and Dawn. We help growing teams reduce risks, build trust, and stay audit ready without the overwhelm.

Rob 0:09

Well, Dawn, it has been a busy month here on the Van Rien podcast.

Dawn 0:13

It has.

Rob 0:14

Hasn't it?

Dawn 0:15

Yes, it has. AI, AI, and AI.

Rob 0:17

And more AI. And not to say AI is is going to be pushed to the curb because it's only getting going, but I want to do a in this week's podcast, we're doing a quick kind of recap, like what have we really talked about?

Human In The Loop And Legal Liability

Rob 0:28

Yeah. A couple podcasts ago, a couple weeks ago, we talked to Dr. Camille Howard. I really talked about the focus of having that human in the middle or that pause authority, or sometimes called the emergency authority, right? You know, making sure that it's not just the AI bot doing the bot things, making sure the humans are still in the mix.

Dawn 0:47

Right.

Rob 0:47

Because at the end of the day, we have to remember with AI, even though you think it's going to do everything, legalities are still there, meaning all the legal frameworks and the laws are still in place. They haven't changed. And your management, your board, and even your individuals who are who are configuring AI are liable, either from a look from a legal perspective.

Dawn 1:06

Yep. Absolutely. And the other thing too is we AI is here to stay. And you've got to learn how to play in the sandbox with it, whatever that looks like for your organization. But yes, you've got to have the the human piece of it. You've got to review what is AI presenting to you. And let's let's uh make sure that um, you know, it is it is valid. And um you just have to double check it. You have to double check it. So the human in the middle. That is that was a great, that was a great podcast. So if you haven't listened to it, I I highly recommend

Governance Goals And Secure LLM Setup

Dawn 1:36

going back to that one. And then we we did another one with uh with our um um auditor Benny, Benny Cleveland, and he talked more into the AI governance piece of it. So what were some of the highlights there, Rob, that we talked about with him?

Rob 1:49

Yeah, the governance and the and then obviously the tabletop. So the governance pieces, you know, starting with, you know, Benny really talked about, and this is why he he brings a lot of value to the Van Rien team, talks about starting with why. You know, what is the why? Why do you really want AI? What is the problem it's solving? Is it a billing issue? Is it a, you know, is it a phone issue? Is it a data aggregation issue? Is it a coding issue? All of that, right? Making sure that you start with the why and then building the frameworks around how you you build and implement that AI bot because those are the key pieces. Be very specific on where it goes and also be very careful. I know Benny talked about making sure that the security parameters are set, right? So don't allow the large language models to, you know, ingest and absorb the data and use it for all of their models. Make sure it's something trained for your own. So the big thing I'd always recommend everybody is build your own. It's a simple checkbox, it's very low cost right now. Obviously, as it scales, it gets pricier. But you can build your own, you know, Gemini's or Clauds, or build your own uh open uh I was gonna say open GBT.

Dawn 2:56

GBTs. Yeah, you can use your own GBTs. Yeah.

Rob 2:59

Get your own GPT set there. What did you think all it the other day? Wasn't it GBTs?

Dawn 3:04

Um there's so many.

Rob 3:05

I don't know. Our kid had something fun to say about it. Yeah. But it was I don't even remember. Um everybody's got the GBTs, right? Like the heebie jeebies. But make sure you build your own. So Benny was really, really good at that. And then dove telling that piece into as a recap into tabletop.

Tabletop Readiness And HIPAA Changes

Rob 3:21

So we're gonna talk in the next next upcoming podcast, um, the new HIPAA 2.0, we're kind of calling it here at Van Rien. Yeah, and that is gonna be the new regulations and laws that are gonna go into effect, looks like in May. So May, we will see new uh HIPAA regulations and new laws on the books. And part of those, one of those I've seen so far, um, is a tabletop exercise, is actually going through your disaster recovery plan. Or the time of this recording, um, we know there is a state of emergency in the northeast with all the snowstorms. Do you know what to do when your team can't connect, right? Or if you have someone that physically needs to be in the office, what do we do? Or what if your team members doesn't have power? What if they don't have, you know, you know, power, food, or shelter? What do you what do you do? Um, what if you're in the AWS East region and they lose power? Can you roll over to the Midwest or the West region? Or um if you're in you know Azure or GCP or any of that, you've got to test because it's sure we're getting, you know, we'll get the emails or the messages or the Slack messages that all is well and we're getting our we're getting our uh backups and everything, but if we don't test it, we don't know it. You know, it's like studying the playbook, but until unless you're tested on the football field or in this case on the ice rink when you team, you know, USA wins the gold, as they did over the weekend, you have to be tested, right, Dawng?

Dawn 4:43

Absolutely, absolutely. So, how do we how do we test? Um, well, in speaking about AI governance, let's kind of unpack

Testing Disasters And Cloud Failover

Dawn 4:50

that. How you would test it is you would do an AI governance and audit. There are two different ways you can go. Um, you can go to the ISO 402001, which is actually an international standard for AI management. Um, you can you can go that route, or you can go the route of a NIST uh NIST AI uh RMF, which is the risk management framework. Both are, you know, uh mapped, you know, to each other. Um is uh one is you can get a certification for an ISO 42001. You can actually get actual certification. You can just uh adhere to the standards on ISO 402001 and NIST AI and and just know that you've gone through the framework, you've got the governance in place, and you can get it set there. But we want to kind of take a deep dive into the difference between the two. So that's kind of an overall general. You've got one that's a true certification, but you've got another one that's

Two Paths: ISO 42001 And NIST AI RMF

Dawn 5:46

a true framework, like I said, both equally great, uh, you know, uh in in in out, you know, um guiding you through the necessary controls. But it's just a depending on which way you want to go with your organization. So we're just gonna kind of dive into kind of the differences and today and so you can understand.

Rob 6:04

Yeah, because you're you're gonna have to make a, you know, for our listeners and our clients, you're gonna have to make a decision on how you're gonna be audited uh again with AI, right? What does that look like? It's not an if, it's a when. So do you want to go to a certification with ISO or do the internal framework? And yes, you can do both. And we're gonna break those down a little bit. Um, I know Dawn, you do you obviously do the ISO because you're more detail-oriented. So you're the ISO queen, and you have your certifications. You can be, you are the lead auditor in Van Rien here for the ISO. What I like about kind of what I like about it is they talk about it as a management system, right? That's the big thing. So that management system really puts the AI governance together and puts that framework together, the structures, the process, the roles, the controls to manage AI responsibly in the life cycle. That's what I see. What else do you, what do you else do you see in that standard?

Dawn 6:54

Yeah, so so ISO, the ISO standard, it more it doesn't use AI governance, but it more talks about the management system because ISO is based on your information security management system. So you're gonna hear that terminology. So just to be clear, is that both of them, yes, is AI governance, generally speaking, but you're gonna hear ISO if you go through that audit, you're gonna hear a management system. So you really are building governance, a framework, a system when you do data security, when you talk about AI, any

What ISO 42001 Demands

Dawn 7:28

kind of security, you're you that's what you're doing. You're building a system and then you're gonna implement it. So Yep.

Rob 7:34

And you're building, you're actually asking like you're building an organization. You're actually building like a department within your organization. It's not saying you have to hire five people, right? But you would tap like five people, right? You'd have like you're gonna have leadership as a as a stakeholder, you're gonna have IT in there, but IT is not responsible for it. They're just there at the table. Your stakeholders are gonna be leadership, your executives, uh board members, like that. You'll maybe you'll have someone, you probably have someone from a program or coding standpoint, and then also, you know, one or two people like that the testing kind of like that front lines, like the people that are that are working with the day-to-day. Make sure it works right and test it, test it, test it. But that's that's really I like how I kind of like that that ISO piece.

Dawn 8:15

Yeah.

Rob 8:15

Now there's a lot of misconceptions that it's designed for any organization, large or small, but I think it fits in any organization, right? If you're if you're a solopreneur to a thousand, thousand-head uh employee company, don't you think?

Dawn 8:28

Any form of AI. It doesn't matter if you've built one or you're just using it as a chat bot or a, you know, something, something just on the side. You've created your own chat GBT. It it's for any usage of AI. You know, AI is going to have data in it, you know, and and you have to build that framework around what data it's using, where the data is being stored, and is this customer-facing? Is it just an internal? These are all questions that that you need to need to understand the answer to and set the guidelines for for your organization on

Building The Cross‑Functional AI Program

Dawn 9:02

how you want your staff to use it or how you want to, how you want your customers to use it.

Rob 9:07

So those are definitely the key pieces. And it's already there, right? We already have the chat bots, hiring tools, uh, fraud detection. One of the key key warnings is making sure you know the type of data that's ingested, right? However, it's health information, that's fairly regulated. If it's credit card information, that's PCI data, if you call if you want to call it that way, and it's regulated. Personal identifiable information from a state federal law. If you have citizens of the EU, there is the EU data protection laws and GD GDPR, and then also there's the AI EU laws as well. So you need to use you need to have a very fine scope of how you want to use it and where you want to use it.

Dawn 9:43

Yeah. Yep.

Rob 9:46

Now, has any country actually m made this mandatory, Dawn, that you're aware of?

Dawn 9:50

Nope, not yet. Um, but there is the EU AI Act, and that's where you know other countries are pushing into. But so far, um, no. But I I would guess that this is probably over the next year, it's gonna become maybe next year it's gonna become a requirement because there's gonna be so much usage of AI.

Rob 10:08

So put the guardrails on.

Dawn 10:10

Yep. It's better to start start with the program now, build it now, and then as you build out how you're using AI, maybe your company is gonna create their own AI. Maybe you're gonna create a company,

Data Scope, Laws, And EU AI Act

Dawn 10:21

a new company. You know, maybe you're gonna have a, you know, an offshoot of your current company and you're just gonna have an AI company, you know. It's better to have those those guardrails and have have things in place before you start doing that. So absolutely. And and yeah, I mean, it's it's all it's it's the important thing is to know like what data you're putting in it and where is that data being stored.

Speaker 2 10:42

Yeah.

Dawn 10:43

And how are you using this data? Your customers will want to know that too. If they're gonna use a chat bot on your website, where is that, where's that data going? That's you need to advise them that in your privacy policy, your terms of service, all that kind of stuff. So Yeah.

Rob 10:57

The then the data leaking. No, make sure you you know where that data goes. Dawn't let it just leak. It'll it will absorb and ingest anything and everything it can, right? Yeah. Think about it as your um think about any of your social media ads, right? Say you look up a vacation or a heck, it could be anything, a coffee, coffee shop. All of a sudden there's you know 20 different ads for coffee coffee shops in your social media, excuse me, platforms. Um, it's

Policies, Privacy, And Data Leakage Risks

Rob 11:21

the same thing with AIs, just how we're doing with marketing and ingesting your cookies and your tokens, we can do the same thing with the AI piece. You gotta be very careful of that in the structure.

Dawn 11:30

Yep. Yep.

Rob 11:31

Now, 42,001 also obviously follows the 27,001 structures. Correct, it's the high-level structure. Yeah.

Dawn 11:39

Yep. So there's um if y'all know ISO 27,001, um, you know that that's based on clauses. You know, HIPAA, we've got controls, talk to controls. You know, ISO calls them clauses and X's. So um, you know, again, the terminology ISO versus NIST, it's gonna be, it's gonna be um NIST, it's uh, you know, I think it's uh,

Mapping ISO Clauses To Practice

Dawn 12:01

you know, GF1, uh AF1, you know, it it the different acronyms, but the controls, I mean, when you look at it side by side, you look at it's gonna have, you know, you know, the scope, the risk assessment, you're you're gonna have similar, similar questions, and and it's it's really gonna map and and be very similar, just just some different verbiage. So, you know, again, it's it's still it's doing the same thing. It is auditing your AI governance. So just just know that it's gonna be worded just a little bit different. Yeah, because uh the ISO is an international standard, so it is it's gonna have you know more of a European, more robust uh verbiage to it.

Rob 12:41

Yeah, and if you walk through those, you know, you have your clause five. There's different clauses, like you mentioned, like controls, right? Like four is context for the stakeholders, five is leadership. We talked about leadership, has got to be bought into that. Six is planning, you know, how are you planning to to implement? And and that includes risk audits and risk assessments and those standards. And then it also goes down into from that six and seven and eight, and just really diving through the standard management system, the territory, right? The resources, the training, the monitoring, internal audits and corrective actions. The the training piece

Required Docs And Annex Controls

Rob 13:16

is key. You know, Van Rien, we do a lot of training. You have to train, make sure you train your team how to use the AI. And it's on all their devices, it's plugged into every meeting, it's everywhere. They have to know what's an approved AI for the organization. And then you gotta be able to monitor that and have those audits because you're gonna have internal audits from an AI perspective for ISO. But also we do it in our HIPAA audits now. We include a scope for AI, like how are you using it, where are you using it, what are you doing with that, and then also have your corrective actions and risk register. Those are the key pieces. And it sounds like a lot because it it is and it's supposed to be robust, but that's how we just kind of really help out and walk through those steps with everybody. Yeah.

Dawn 13:59

And and so let's let's kind of, you know, basically ISO 42001 requires documentation. There's there's 38 NXA controls, unless you explicitly exclude them. So there is core documents, you've got a you know, a scope document, you've got a policy risk management methodology, risk assessment, you've got a statement of applicability, a risk treatment plan, got all these, these other things. Now, so you've got those. I mean, don't don't get all stressed out about that. So then let's let's go to NIST AI, RMF.

NIST AI RMF: Govern, Map, Measure, Manage

Dawn 14:35

What does that look like? And and what does NIST call it, call it? So let's just kind of fast forward here. The NIST is built around the four core functions govern, map, measure, and manage. So for things like the the uh govern, that's gonna, it's gonna map over to clauses like four and five and parts of six for ISO four 2001. So you're gonna, again, measure is gonna do, you know, the risks through testing, manage is is responding, monitoring. It's all gonna map over. Again, it's just, it's just the way it's laid out, but it's all really gonna map over. And at the end of the day, it's all going to build your your AI governance program. So it's how you want to go about it. What we like to do here at Van Rien is we do a mix, we blend it. So we bring both together, if you will. Now we don't do an actual for 2001 certification. We would use uh an external auditor that we work with, but we we will verify that you have an AI governance program built and we utilize both, like I said, the NIST AI RMF and the ISO 42001. So bless you. Um, yeah, I know that was a lot. That was a lot. So so yeah, so that's I know that's kind of a lot in a nutshell, but I don't want to get into the weeds too much because that it um they're both very important. It's just a matter of which way you want to go. But with us, you're we're gonna do a blend, but you can do a separate for 2001 certification. Yeah. And we can, we can, we can definitely help you with that.

Rob 16:08

Yeah, and it's a certification by an accredited body. Yes. Yes. That is

US vs International Use Cases

Rob 16:12

key. There are people that'll say they're auditors and they can give you certifications, but if they're not accredited, it's not a valid certification by the IAF board, which is really good. And it takes about, I don't know, probably four to five months, I think, Dawn, pretty much to get through readiness. So you got to get through the readiness, prepare the IMS, build it, get the documentation, and then you have a 45-day um evaluation period, stage one and stage two, to verify can uh the controls and clauses are applied and then verify that they're actually activated and they're actually running. So those are the key pieces.

Dawn 16:46

Yeah, and it also depends, you know, your organization size as well. So yes, that's a good, you know, average, average, you know, a three, three to six month type of type of time frame.

unknown 16:56

Yep.

Rob 16:57

That's then they get to maintain it because then you're gonna come back around and do it all over again. And you need meeting minutes, you mean meet meeting notes and items like that.

Dawn 17:04

Yep. Yep, exactly.

Rob 17:07

Very good. Should we shift over to NIST?

Dawn 17:10

Well, we that I yeah, we can. Yes. That I kind of wanted to give that preface. So yes, I did just preface kind of kind of the differences and where it maps over. Um but yeah, why don't you go into if there's an other specific items that you wanted to to outlay, outlay um for for that?

unknown 17:26

Yeah.

Rob 17:27

Yeah, I think we I think we've gotten through the through the NIST piece. Sorry, through the ISO piece. So ISO

Comparing Audit Rigor And Maturity

Rob 17:31

is a true certification um and internal external audit. Now let's kind of talk about the NIST AI RMF, which we all which requires an internal audit. You don't have that third party like you do with ISO, even like with SOC, you have that third party examination. Um, so with the with the NIST piece, obviously the NIST uh RMF, uh AI RMF was published in January of 23. So we're just barely three years, three years old. Just a few months before the ISO standards. So it pretty much everything's kind of built off NIST, and then it kind of goes from there. But we know the NIST is for our US federal agencies, so there's a lot more weight in the US with the American government and regulated industries. So that's kind of the key pieces. If you're doing a lot of work through Europe and Asia, I would definitely go for the ISO and as a focus, right? But if you're doing work here in the States, start with the with the NIST, and then it's like, okay, how can I layer that IMS on top? Um, and what does that look like? So the big difference is is right now is there is no certification. Um, there is internal gathering of information and internal readiness audit. Uh, and then we just put together a we put together a summary report for you. That's what we do at Van Rye. So there's not a certification from an exit from an accredited body that you can't been certified like ISO, but we can do the internal auditing and then we can provide you with documentation that you've gone through an SAOF AI RMF audit and have those controls in place. Those are the big ones.

Dawn 19:00

Yep. Yep.

Rob 19:01

Yeah. Why don't you go through those core functions again, Dawn, with NIST?

Dawn 19:05

Yeah. And and just just so you know, they both they both are very similar. They're both uh risk-based. Um, this is this, they're both

Blending Standards And Next Steps

Dawn 19:12

uh um audits, govern, you know, governance to see where you're at. Um and they do have an output as far as where your gaps are. So that again, and and Rob said it right, the biggest difference is ISO um is is for 2001 is you to get a certification, it's through a certified body. But so for NIST, so it you know, we talked about the clauses in ISO for 2001. For NIST, it there's four core functions. So govern, app, measure, and manage. You know, uh leave it to the government to to kind of look at it. Yeah, it it's very government-y. So govern is the governance, it's the you know, the policies, the structures, the culture, that type of thing. And this is, you know, what I was saying before, it basically maps and mirrors clauses four, five, and part of six um for ISO for 2001. Map is where you identify and categorize the risks. So um, you know, that's like risk impact, you know, uh risk treatment assessments, things like that as far as ISO and then measure. It's analyzing, tracking the risks. Um, it's where the gaps are. So, and then manage. How do you respond? How do you monitor it? How are you going to mitigate it? How are you documenting it? So it's laid out very well. Um, you can see if you were to put them side by side, you it it really maps nicely. Um, so again, ISO, they're they're very similar. But again, it just it just depends on what structure you like. Here at Van Rien, we blend both of them.

Speaker 2 20:45

Yeah.

Dawn 20:46

So you're you're getting kind of the best of both worlds. Um, but again, it's not that true certification on your on your website. But again, you're doing what you need to do. We're helping you build the program, get ready, build the program, and then you and and the tools to implement it in your organization. So yep.

Rob 21:02

Yeah, that's those are the different different key pieces. So if you kind of go head to head on how we can, you know, really compare these. The ISO standard is going to be more auditable in a structured sense where you have this different stage as a two-stage audit. We'll do internally uh with with the NIST RMF AIs, we'll do the we'll do probably about four stage audits to bring everything in and get through it like within a month or so. Yeah.

unknown 21:27

Yeah.

Rob 21:28

You'll have the key pieces of the applicability state and the exits and the policies and all that. Um, you know, what are you really gonna audit against within ISO and then with the NIST AI RMF, and then understand the maturity of the assessment, which is a really key piece. Those are kind of the key, key things.

Speaker 2 21:45

Yeah.

Rob 21:45

Um obviously if you're doing work within the government, even some of the large organizations, they're gonna probably lean more onto that NIST standard because it is a, you know, it was created here, my own our own national institute of of uh security and technology. Yeah. I can talk today. There we go. And it's but it's laid out the it's basically laid out the framework of what we've got to do for globally or even a lot of the European standards. Yep. Now we do get this question sometimes, and we have to do do a lot on the ISO piece, but can you blend them together, Dawn? Can you actually take both standards and blend them together? If so, what was that look like?

Dawn 22:20

Yeah, you um you should. You can, you should, um, because it it's the best programs are using both. And that's why here at Van Rijn, um, we always do more. We don't do just the minimum. And that's why we have blended both. And and so we recommend that. Now, that's not to say if you're just gonna go full on, you know, ISO4 2001, we're gonna go do that. Not to say it's not going to be good. Of course, it's going to be a great certification, a great program to do. We are just, like I said, we we like to blend both. Um, and to that's how we we like to present that to our customers and when we're preparing, preparing them. So yeah, use both, definitely. If you're just gonna go ISO, if you're already one of my our customers for 27,000 one, you're just gonna do the 42,001. I think that's great as well. Um, but um just yeah, you can you can do both, you can do one or the other. Um but they they're they're you just really need what works best for you and your organization, what is the best governance program? And uh and we'll help you with that.

Rob 23:23

Yeah, and that's why we're here to walk through that and see what's best. Uh you may not need an international standard, but you need a US standard, which a lot of our our clients would need. Well, those were good, Dawn. We don't went through, you know, everything from from our podcast three weeks ago with Dr. Howard to Benny. We talked about the human in the middle, the pause authority to Benny talking

Final Takeaways And How To Reach Us

Rob 23:42

about the auditing and and the tabletop exercises now into these two standards. So there are questions, reach out. You know, we find the information there in the notes of the podcast, and we're here to help guide you through that and just love talking about it. So we want to make sure you're successful that you can grow your business in the way you want to with AL.

Dawn 24:00

Yep. Awesome. Yep, reach out if you need. Just a little last note is AI governance is not really optional anymore.

Speaker 2 24:07

Correct.

Dawn 24:07

It's going to become a standard. And so uh we're here to help answer any questions you may have. But hopefully we we unpack that a little bit. I know that was a lot. Uh we uh if you have any questions, just let us know.

Rob 24:19

Yeah, yeah. That's it. All righty. Until next week. Bye bye.

Dawn 24:23

Bye bye.