You Can't Manage AI Risk You Can't See
August 15, 2026, by: ComplyAIQ
There is no shortage of discussion about how healthcare organizations should govern artificial intelligence. Risk assessments, policies, oversight committees, monitoring, human review, and so on.
But before getting to any of those issues, there is a more basic question:
Do you actually know where AI is being used in your organization?
The U.S. Department of Health and Human Services has made that question part of its own approach to AI governance. HHS maintains a public inventory (see here) of current and planned AI use cases as part of its internal AI-governance approach. Led through an annual process by HHS's Chief Artificial Intelligence Officer, the inventory is intended to support transparency, accountability, and secure AI use.
This is an internal U.S. federal-government practice, not a requirement imposed on healthcare providers or other private healthcare organizations. Still, HHS's approach is worth paying attention to. HHS considers an up-to-date understanding of where AI is being used important enough to make it part of its own governance structure.
The logic is fairly simple: you can't effectively manage AI risk you can't see.
For healthcare compliance professionals, that makes the AI inventory much more than a list of technology. It can be the starting point for connecting AI use to things compliance programs already do, such as identify risk, assess it, establish accountability, monitor it, document it, and escalate concerns when necessary.
Finding the AI Is Harder Than It Seems
An AI inventory seems simple enough. Find the AI systems the organization uses and make a list.
However, the task is more complicated in reality.
Some AI tools are easy to identify. An organization may deliberately purchase an AI documentation tool, diagnostic application, or generative AI platform. Those implementations will probably pass through some combination of procurement, IT, information security, privacy, legal, and/or operational review.
However, AI increasingly arrives in less obvious ways.
A vendor may add an AI feature to software the organization has used for years. An existing platform may introduce predictive capabilities or a generative AI assistant through a product update. Employees may begin using publicly available generative AI tools without going through a traditional procurement process. A third-party service provider may use AI in delivering services even though the healthcare organization is not directly operating the AI system.
Even the terminology can get in the way. A product may use machine learning or predictive models without users thinking of it as “AI” at all. And an AI-enabled feature inside an existing product may present a different governance question than a stand-alone AI system or a particular AI use case.
Federal health IT policy provides a useful example. Under the HTI-1 final rule, the Office of the National Coordinator for Health Information Technology (ONC) established algorithm-transparency requirements for certain predictive algorithms, including AI, that are supplied by certified health IT developers as part of certified health IT such as electronic health record systems and modules that meet applicable requirements under ONC's Health IT Certification Program.
The requirements are intended to give clinical users a consistent baseline of information about these algorithms, including their purpose and intended use, development, validation and performance, and relevant risk-management practices. The objective is to help users assess whether an algorithm is fair, appropriate, valid, effective, and safe for its intended use.
Importantly, these requirements operate through ONC's Health IT Certification Program, and they reach only certain algorithms within certified systems—not every AI tool a provider organization may be using, including many tools vendors add on top of certified systems or that staff adopt independently. They do not create a general AI-inventory mandate for healthcare provider organizations. Still, ONC has noted that certified health IT supports care delivered by more than 96 percent of U.S. hospitals, which illustrates how consequential algorithmic capabilities can be embedded in technology healthcare organizations already use, and why an AI-discovery process cannot depend solely on looking for products labeled "AI," or assume that a framework like this one is already doing that discovery work.
So the challenge isn't just identifying products with “AI” in their names. Organizations need some way of recognizing AI-enabled functions and use cases that may matter from a governance and compliance perspective.
An Inventory Should Tell You More Than What You Own
NIST’s voluntary AI Risk Management Framework includes an outcome within its GOVERN function calling for mechanisms to inventory AI systems, with those mechanisms resourced according to organizational risk priorities. Its accompanying Playbook provides suggested actions for putting that outcome into practice.
That's useful because it moves the conversation beyond simply making a list.
A spreadsheet with the name of an AI product, the vendor, and the department using it may be an inventory in the most literal sense. But it doesn't necessarily tell the organization what it needs to know to manage risk.
What does the system actually do? Where is it being used? Who is responsible for it? What information does it process? Who may be affected by its outputs? Is it influencing patient care, billing, employment, compliance, or another significant decision? Is a vendor involved? Has the organization assessed the use for risk? What oversight applies?
Not every organization needs the same inventory structure, and there is no universal federal rule prescribing a particular set of inventory fields that every healthcare organization must maintain.
The more important question is whether the inventory gives the organization enough information to connect an AI use to the appropriate governance and risk-management processes.
For compliance, that connection is important.
If a significant AI use isn't known, it may never make its way into a compliance risk assessment. If a new vendor capability isn't identified, questions about privacy, contractual obligations, or appropriate use may never be asked. If no one knows who owns an AI use, accountability becomes much harder when concerns arise.
The value of the inventory is not the list itself. It is what the organization can do because the list exists.
The Inventory Can't Be a One-Time Project
Even a good inventory can become outdated quickly.
AI systems change. Vendors release new features. Models are updated. Employees find new uses for existing tools. A pilot expands into normal operations. A tool approved for one purpose begins being used for another.
This is one reason HHS’s annual AI-use-case inventory process is notable. It provides an example of a recurring process designed to capture current and planned AI uses, rather than treating inventory as a one-time exercise.
The same lifecycle issue appears elsewhere in federal AI oversight. FDA’s final guidance, Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions, addresses information that device manufacturers may include in a marketing submission for a Predetermined Change Control Plan, or PCCP. A PCCP may describe specified planned modifications to an AI-enabled device software function, the methodology to develop, validate, and implement those modifications, and an assessment of the modifications’ impacts. When FDA authorizes a PCCP, the manufacturer may implement modifications described in that PCCP without submitting a new marketing submission for each individual modification.
That guidance does not apply to every AI tool used by a healthcare organization, and it is not an AI-inventory requirement for hospitals or health systems. But it illustrates the broader lifecycle reality: AI-enabled technology may change materially after implementation.
That is particularly important with vendor technology.
An organization may carefully evaluate a product when it is purchased, only for the vendor to introduce new AI functionality months later. The contract may be the same. The product name may be the same. But what the technology is doing, and the risks that need to be considered, may have changed.
An AI inventory therefore has to connect in some way to change management. The organization needs a mechanism for new AI uses, and meaningful changes to existing ones, to make their way back into the governance process.
An inventory that is accurate only on the day it is created is not much of a governance control.
Someone Has to Own the Process, but No One Can Do It Alone
One of the harder questions is who is responsible for finding all of this AI.
The answer probably isn't one department.
IT may know what technology is deployed but not every way employees are using it. Procurement may know what was purchased but not every feature a vendor later adds. Privacy may have visibility into systems involving protected health information but less visibility into tools that never trigger a privacy review. Clinical leaders may understand AI being used in patient care but know little about AI in revenue cycle, HR, compliance, or other administrative functions.
Compliance won't see everything either.
AI discovery therefore has to be multidisciplinary. Procurement, IT, information security, privacy, clinical informatics, legal, compliance, vendor management, operational leaders, and others may all have pieces of the picture.
The important part is having a process that brings those pieces together.
The Joint Commission’s voluntary Responsible Use of AI in Healthcare certification reflects this broader move toward organization-level governance. Its standards address governance, effective data management, risk and bias reduction, monitoring, evaluation and validation of AI safety and performance, and transparency, education, and training. The certification evaluates organizational practices rather than certifying individual AI products.
The relevance to an AI inventory is practical. Organization-wide governance and ongoing monitoring become difficult if the organization doesn't have a reliable way of knowing which AI systems, features, and uses fall within the governance process in the first place.
Where Compliance Professionals Fit
None of this means compliance needs to own the AI inventory (you’re welcome, compliance professionals!).
However, compliance professionals do have something important to contribute.
Much of compliance work already involves finding risk across a large organization, determining which risks deserve closer attention, documenting controls, monitoring activity, evaluating third parties, following up on concerns, and escalating issues when necessary.
AI changes the subject matter, but many of those underlying disciplines are familiar.
The Department of Justice’s Evaluation of Corporate Compliance Programs makes the connection particularly relevant. Its September 2024 revisions ask prosecutors to consider how a company assesses risks associated with new technologies, including AI; how it addresses negative or unintended consequences; and how it mitigates deliberate or reckless misuse. Where a company uses AI or similar technology in its business or as part of its compliance program, DOJ also asks whether controls exist to monitor and ensure its trustworthiness, reliability, and use in compliance with applicable law and the company’s code of conduct.
DOJ does not say companies must maintain an “AI inventory.” But there is an obvious practical problem: an organization cannot meaningfully assess the compliance risks created by AI uses it does not know exist.
This is where compliance professionals can add value without becoming AI engineers or claiming ownership of every aspect of AI governance.
They can ask whether important AI uses are being identified. Whether those uses are reaching the right risk-assessment processes. Whether ownership is clear. Whether vendor changes are being considered. Whether higher-risk uses receive appropriate review. And whether concerns discovered through monitoring, auditing, investigations, or other compliance activities make their way back into the governance process.
Those aren't entirely new compliance skills. They are familiar skills being applied to a new and rapidly changing area of risk.
Start With Visibility
Healthcare organizations are developing increasingly sophisticated approaches to AI governance. Policies, committees, risk classifications, approval processes, and monitoring programs all have a role.
But all of them have the same limitation: they can only govern the AI that enters their field of view.
HHS's own AI inventory is a useful example. NIST includes inventory mechanisms in its voluntary AI Risk Management Framework. And healthcare-specific governance efforts increasingly emphasize organization-wide oversight and monitoring throughout the AI lifecycle.
None of this creates a universal federal requirement for healthcare organizations to maintain a particular AI inventory.
It does point to something more basic.
Before an organization can decide how an AI system should be assessed, monitored, or governed, it has to know the system exists and understand how it is being used.
For healthcare compliance professionals thinking about where to begin with AI governance, that may be one of the most practical questions to ask:
Do we actually know where we're using AI?
