You Can't Manage AI Risk You Can't See

August 15, 2026, by: ComplyAIQ

There is no shortage of discussion about how healthcare organizations should govern artificial intelligence. Risk assessments, policies, oversight committees, monitoring, human review, and so on.

But before getting to any of those issues, there is a more basic question:

Do you actually know where AI is being used in your organization?

The U.S. Department of Health and Human Services has made that question part of its own approach to AI governance. HHS maintains a public inventory (see here) of current and planned AI use cases as part of its internal AI-governance approach. Led through an annual process by HHS's Chief Artificial Intelligence Officer, the inventory is intended to support transparency, accountability, and secure AI use.

This is an internal U.S. federal-government practice, not a requirement imposed on healthcare providers or other private healthcare organizations. Still, HHS's approach is worth paying attention to. HHS considers an up-to-date understanding of where AI is being used important enough to make it part of its own governance structure.

The logic is fairly simple: you can't effectively manage AI risk you can't see.

For healthcare compliance professionals, that makes the AI inventory much more than a list of technology. It can be the starting point for connecting AI use to things compliance programs already do, such as identify risk, assess it, establish accountability, monitor it, document it, and escalate concerns when necessary.

Finding the AI Is Harder Than It Seems

An AI inventory seems simple enough. Find the AI systems the organization uses and make a list.

However, the task is more complicated in reality.

Some AI tools are easy to identify. An organization may deliberately purchase an AI documentation tool, diagnostic application, or generative AI platform. Those implementations will probably pass through some combination of procurement, IT, information security, privacy, legal, and/or operational review.

However, AI increasingly arrives in less obvious ways.

A vendor may add an AI feature to software the organization has used for years. An existing platform may introduce predictive capabilities or a generative AI assistant through a product update. Employees may begin using publicly available generative AI tools without going through a traditional procurement process. A third-party service provider may use AI in delivering services even though the healthcare organization is not directly operating the AI system.

Even the terminology can get in the way. A product may use machine learning or predictive models without users thinking of it as “AI” at all. And an AI-enabled feature inside an existing product may present a different governance question than a stand-alone AI system or a particular AI use case.

Federal health IT policy provides a useful example. Under the HTI-1 final rule, the Office of the National Coordinator for Health Information Technology (ONC) established algorithm-transparency requirements for certain predictive algorithms, including AI, that are supplied by certified health IT developers as part of certified health IT such as electronic health record systems and modules that meet applicable requirements under ONC's Health IT Certification Program.

The requirements are intended to give clinical users a consistent baseline of information about these algorithms, including their purpose and intended use, development, validation and performance, and relevant risk-management practices. The objective is to help users assess whether an algorithm is fair, appropriate, valid, effective, and safe for its intended use.

Importantly, these requirements operate through ONC's Health IT Certification Program, and they reach only certain algorithms within certified systems—not every AI tool a provider organization may be using, including many tools vendors add on top of certified systems or that staff adopt independently. They do not create a general AI-inventory mandate for healthcare provider organizations. Still, ONC has noted that certified health IT supports care delivered by more than 96 percent of U.S. hospitals, which illustrates how consequential algorithmic capabilities can be embedded in technology healthcare organizations already use, and why an AI-discovery process cannot depend solely on looking for products labeled "AI," or assume that a framework like this one is already doing that discovery work.

So the challenge isn't just identifying products with “AI” in their names. Organizations need some way of recognizing AI-enabled functions and use cases that may matter from a governance and compliance perspective.

An Inventory Should Tell You More Than What You Own

NIST’s voluntary AI Risk Management Framework includes an outcome within its GOVERN function calling for mechanisms to inventory AI systems, with those mechanisms resourced according to organizational risk priorities. Its accompanying Playbook provides suggested actions for putting that outcome into practice.

That's useful because it moves the conversation beyond simply making a list.

A spreadsheet with the name of an AI product, the vendor, and the department using it may be an inventory in the most literal sense. But it doesn't necessarily tell the organization what it needs to know to manage risk.

What does the system actually do? Where is it being used? Who is responsible for it? What information does it process? Who may be affected by its outputs? Is it influencing patient care, billing, employment, compliance, or another significant decision? Is a vendor involved? Has the organization assessed the use for risk? What oversight applies?

Not every organization needs the same inventory structure, and there is no universal federal rule prescribing a particular set of inventory fields that every healthcare organization must maintain.

The more important question is whether the inventory gives the organization enough information to connect an AI use to the appropriate governance and risk-management processes.

For compliance, that connection is important.

If a significant AI use isn't known, it may never make its way into a compliance risk assessment. If a new vendor capability isn't identified, questions about privacy, contractual obligations, or appropriate use may never be asked. If no one knows who owns an AI use, accountability becomes much harder when concerns arise.

The value of the inventory is not the list itself. It is what the organization can do because the list exists.

The Inventory Can't Be a One-Time Project

Even a good inventory can become outdated quickly.

AI systems change. Vendors release new features. Models are updated. Employees find new uses for existing tools. A pilot expands into normal operations. A tool approved for one purpose begins being used for another.

This is one reason HHS’s annual AI-use-case inventory process is notable. It provides an example of a recurring process designed to capture current and planned AI uses, rather than treating inventory as a one-time exercise.

The same lifecycle issue appears elsewhere in federal AI oversight. FDA’s final guidance, Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions, addresses information that device manufacturers may include in a marketing submission for a Predetermined Change Control Plan, or PCCP. A PCCP may describe specified planned modifications to an AI-enabled device software function, the methodology to develop, validate, and implement those modifications, and an assessment of the modifications’ impacts. When FDA authorizes a PCCP, the manufacturer may implement modifications described in that PCCP without submitting a new marketing submission for each individual modification.

That guidance does not apply to every AI tool used by a healthcare organization, and it is not an AI-inventory requirement for hospitals or health systems. But it illustrates the broader lifecycle reality: AI-enabled technology may change materially after implementation.

That is particularly important with vendor technology.

An organization may carefully evaluate a product when it is purchased, only for the vendor to introduce new AI functionality months later. The contract may be the same. The product name may be the same. But what the technology is doing, and the risks that need to be considered, may have changed.

An AI inventory therefore has to connect in some way to change management. The organization needs a mechanism for new AI uses, and meaningful changes to existing ones, to make their way back into the governance process.

An inventory that is accurate only on the day it is created is not much of a governance control.

Someone Has to Own the Process, but No One Can Do It Alone

One of the harder questions is who is responsible for finding all of this AI.

The answer probably isn't one department.

IT may know what technology is deployed but not every way employees are using it. Procurement may know what was purchased but not every feature a vendor later adds. Privacy may have visibility into systems involving protected health information but less visibility into tools that never trigger a privacy review. Clinical leaders may understand AI being used in patient care but know little about AI in revenue cycle, HR, compliance, or other administrative functions.

Compliance won't see everything either.

AI discovery therefore has to be multidisciplinary. Procurement, IT, information security, privacy, clinical informatics, legal, compliance, vendor management, operational leaders, and others may all have pieces of the picture.

The important part is having a process that brings those pieces together.

The Joint Commission’s voluntary Responsible Use of AI in Healthcare certification reflects this broader move toward organization-level governance. Its standards address governance, effective data management, risk and bias reduction, monitoring, evaluation and validation of AI safety and performance, and transparency, education, and training. The certification evaluates organizational practices rather than certifying individual AI products.

The relevance to an AI inventory is practical. Organization-wide governance and ongoing monitoring become difficult if the organization doesn't have a reliable way of knowing which AI systems, features, and uses fall within the governance process in the first place.

Where Compliance Professionals Fit

None of this means compliance needs to own the AI inventory (you’re welcome, compliance professionals!).

However, compliance professionals do have something important to contribute.

Much of compliance work already involves finding risk across a large organization, determining which risks deserve closer attention, documenting controls, monitoring activity, evaluating third parties, following up on concerns, and escalating issues when necessary.

AI changes the subject matter, but many of those underlying disciplines are familiar.

The Department of Justice’s Evaluation of Corporate Compliance Programs makes the connection particularly relevant. Its September 2024 revisions ask prosecutors to consider how a company assesses risks associated with new technologies, including AI; how it addresses negative or unintended consequences; and how it mitigates deliberate or reckless misuse. Where a company uses AI or similar technology in its business or as part of its compliance program, DOJ also asks whether controls exist to monitor and ensure its trustworthiness, reliability, and use in compliance with applicable law and the company’s code of conduct.

DOJ does not say companies must maintain an “AI inventory.” But there is an obvious practical problem: an organization cannot meaningfully assess the compliance risks created by AI uses it does not know exist.

This is where compliance professionals can add value without becoming AI engineers or claiming ownership of every aspect of AI governance.

They can ask whether important AI uses are being identified. Whether those uses are reaching the right risk-assessment processes. Whether ownership is clear. Whether vendor changes are being considered. Whether higher-risk uses receive appropriate review. And whether concerns discovered through monitoring, auditing, investigations, or other compliance activities make their way back into the governance process.

Those aren't entirely new compliance skills. They are familiar skills being applied to a new and rapidly changing area of risk.

Start With Visibility

Healthcare organizations are developing increasingly sophisticated approaches to AI governance. Policies, committees, risk classifications, approval processes, and monitoring programs all have a role.

But all of them have the same limitation: they can only govern the AI that enters their field of view.

HHS's own AI inventory is a useful example. NIST includes inventory mechanisms in its voluntary AI Risk Management Framework. And healthcare-specific governance efforts increasingly emphasize organization-wide oversight and monitoring throughout the AI lifecycle.

None of this creates a universal federal requirement for healthcare organizations to maintain a particular AI inventory.

It does point to something more basic.

Before an organization can decide how an AI system should be assessed, monitored, or governed, it has to know the system exists and understand how it is being used.

For healthcare compliance professionals thinking about where to begin with AI governance, that may be one of the most practical questions to ask:

Do we actually know where we're using AI?

AI and the DOJ’s Signal to Compliance Programs

August 8, 2026, by: ComplyAIQ

Artificial intelligence is increasingly becoming part of the compliance professional’s responsibility.

One of the clearest indications came in September 2024, when the U.S. Department of Justice Criminal Division updated its Evaluation of Corporate Compliance Programs (ECCP) to expressly address artificial intelligence and other emerging technologies.

The ECCP is designed to assist federal prosecutors in evaluating the effectiveness of corporate compliance programs when making charging and resolution decisions. Importantly, the DOJ emphasizes that its questions are neither a checklist nor a rigid formula. Compliance programs are evaluated in the context of an organization’s particular size, industry, regulatory environment, risk profile, and other circumstances.

Within that framework, however, the DOJ now expressly directs attention to how organizations identify and manage risks associated with AI and other emerging technologies.

For healthcare compliance professionals, the questions provide a useful framework for considering how AI oversight fits within an effective compliance program.

What the DOJ Is Asking About AI

The DOJ addresses AI within the risk assessment portion of the ECCP, beginning with a broader question about emerging risks:

“Does the company have a process for identifying and managing emerging internal and external risks” that could affect compliance with applicable law?

From there, the DOJ expressly turns to artificial intelligence:

“How does the company assess the potential impact of new technologies, such as artificial intelligence (AI), on its ability to comply with criminal laws?”

The ECCP then asks a series of questions addressing how AI is governed and managed within an organization. Among them:

• Is management of AI and other emerging-technology risks integrated into the organization’s broader enterprise risk management strategy?

• What governance approach has the organization established for AI used within the business and within the compliance program?

• How does the organization address unintended consequences and the potential for deliberate or reckless misuse of technology?

• Are controls in place to monitor AI for “trustworthiness, reliability, and use in compliance with applicable law and the company’s code of conduct”?

• Are controls in place to ensure that technology is used only for its intended purposes?

• “What baseline of human decision-making is used to assess AI?”

• “How is accountability over use of AI monitored and enforced?”

• “How does the company train its employees on the use of emerging technologies such as AI?”

Taken together, these questions provide a meaningful picture of how the DOJ is approaching AI within the broader evaluation of compliance program effectiveness.

They also map closely to functions that are already familiar to healthcare compliance professionals: risk assessment, governance, policies and controls, education, monitoring, accountability, and corrective action.

AI Within the Compliance Risk Assessment

Risk assessment has long been foundational to the DOJ’s evaluation of compliance program effectiveness. The ECCP describes the starting point as understanding how an organization has identified and assessed its risk profile and whether the compliance program devotes appropriate scrutiny and resources to those risks.

The 2024 update makes clear that AI and emerging technology should be part of that analysis where relevant.

For healthcare organizations, this means understanding where AI is being used and the risks associated with those uses. AI may now appear across clinical decision support, documentation, billing and claims processes, privacy and security functions, research, workforce management, compliance operations, and numerous administrative functions.

The analysis should also be risk-based. An AI application performing a limited administrative function does not necessarily warrant the same level of oversight as an AI system influencing clinical, financial, or other consequential decisions.

The objective is not uniform oversight of every AI application. It is oversight proportionate to the particular technology, use case, and organizational risk.

Governance and Accountability

The DOJ specifically asks about an organization’s governance approach to AI and how accountability over AI use is monitored and enforced.

These questions are particularly significant because effective AI governance extends beyond identifying individual risks. Organizations need to determine who has authority and responsibility for making decisions about those risks.

Depending on the organization and use case, governance may involve compliance, legal, privacy, information security, information technology, clinical leadership, internal audit, risk management, and/or operational stakeholders.

Compliance does not necessarily need to own every aspect of AI governance. It should, however, have a defined role where AI intersects with regulatory and compliance risk.

Organizations should be able to answer practical questions such as:

Who evaluates AI-related compliance risks? Who approves higher-risk uses? What issues require escalation? Who can suspend or restrict an AI system when significant concerns arise? How are important decisions documented? Who is accountable when established requirements are not followed? Are staff sufficiently trained and educated on AI use and risk? Who is responsible for monitoring AI systems? Does the organization have a current inventory of all AI tools in use? Are policies regarding AI use clear and communicated to staff?

These are governance questions that fit naturally within the broader compliance framework.

Controls, Intended Use, and Human Oversight

The DOJ questions move beyond governance structures to the controls surrounding AI itself.

The ECCP asks whether controls exist to monitor the trustworthiness and reliability of AI, whether the technology is being used consistently with applicable law and the organization’s code of conduct, and whether controls ensure that technology is used only for its intended purposes.

The DOJ also specifically asks about the baseline of human decision-making used to assess AI.

For healthcare organizations, these considerations can be incorporated throughout the lifecycle of an AI system.

Before implementation, organizations can define approved uses, identify prohibited or restricted uses, assess relevant compliance and regulatory risks, and establish appropriate human oversight.

After implementation, those expectations can be reinforced through policies, access controls, monitoring, training, escalation mechanisms, and periodic reassessment.

Human oversight becomes particularly important where AI outputs influence consequential decisions, especially patient care. The appropriate form of oversight will vary by use case, but AI should support rather than silently displace professional accountability where meaningful human judgment remains necessary.

Monitoring AI in Practice

The ECCP also asks how organizations address potential negative or unintended consequences associated with technology and mitigate deliberate or reckless misuse.

Those questions point toward an important principle: AI governance does not end when a system is approved.

AI systems and their uses can change over time. Performance may shift. Organizational workflows may evolve. Vendors may modify products or introduce new capabilities. Employees may also begin using approved technologies in ways that were not originally contemplated.

Healthcare compliance programs can address these risks by extending established auditing and monitoring practices to AI.

Depending on the system and its risk profile, this might include monitoring adherence to approved-use requirements, evaluating vendor performance, reviewing incidents and unexpected behavior, assessing bias or performance drift where relevant, and periodically reassessing whether existing controls remain appropriate.

The technology is different, but the compliance discipline is familiar: identify the relevant risk, establish controls, monitor their effectiveness, and respond when weaknesses are identified.

Training, Communication, and Accountability

One of the DOJ’s most direct AI-related questions is:

“How does the company train its employees on the use of emerging technologies such as AI?”

That question has immediate relevance for healthcare organizations.

Employees using AI tools may need guidance regarding acceptable use, privacy and confidentiality, human review, organizational policies, and escalation of concerns.

Managers and operational leaders may require additional education regarding their oversight responsibilities. Compliance, privacy, legal, audit, and other governance professionals may require deeper knowledge to effectively evaluate AI-related risks and controls.

Training should therefore be tailored to role and risk rather than treated as a single, uniform educational requirement.

Education should also connect employees to established mechanisms for seeking guidance and reporting concerns. When inappropriate AI use or other problems are identified, organizations should be prepared to investigate, address the issue, apply accountability where appropriate, and use lessons learned to strengthen the governance process.

Integration, Not Addition

Perhaps the most useful takeaway from the DOJ’s treatment of AI is where these questions appear.

The DOJ did not create a separate evaluation framework for an “AI compliance program.” It incorporated AI and emerging technology into its broader framework for evaluating corporate compliance programs.

Healthcare organizations already have much of the architecture needed to begin addressing AI-related compliance risks.

Existing risk assessments can incorporate AI.

Codes of conduct and policies can establish expectations for appropriate AI use.

Governance structures can define responsibility and accountability for AI-related decisions.

Training programs can incorporate AI-related responsibilities.

Reporting and escalation mechanisms can receive AI-related concerns.

Auditing and monitoring functions can assess AI systems, uses, and controls.

Established investigation and corrective-action processes can respond when AI-related issues occur.

The emergence of AI does not make these established compliance functions obsolete. It expands the environment in which they must operate.

For healthcare compliance professionals, the DOJ’s 2024 ECCP update therefore provides more than a list of questions about a new technology. It illustrates how emerging AI risks can be addressed through the core disciplines of an effective compliance program.

As artificial intelligence becomes increasingly integrated into healthcare operations, the opportunity is to evolve those established compliance practices alongside it—supporting innovation while maintaining appropriate governance, accountability, and oversight.

The Takeaway

The DOJ’s inclusion of AI questions in the 2024 ECCP sends a clear signal to compliance professionals: as AI becomes more integrated into organizational operations, the organization’s compliance program should be utilized to address associated risks.

For healthcare compliance leaders, this does not require building an entirely new compliance framework. It means applying established compliance disciplines—risk assessments, governance, policies, training, monitoring, accountability, and corrective action—to the evolving use of AI.

The opportunity for compliance professionals is to help their organizations adopt AI responsibly and to evolve the compliance program alongside the evolution of technology.