Model risk management for generative AI is now a practical problem, not a future one. Banks, insurers, healthcare companies, and large enterprises are already trying to fit copilots, summarization tools, internal chat assistants, and workflow agents into governance processes designed for credit scoring, pricing, and other conventional models. That mismatch is producing two bad outcomes at once: slow approvals and weak control.
The old playbook is not useless. It is just incomplete.
Traditional model risk management assumes you can define the model, test it against a stable objective, document the training and validation logic, and monitor for drift against known thresholds. That works reasonably well when the model is bounded and the business use case is narrow. Generative AI is different in ways that matter operationally. The model may be external. The weights may change. The behavior may depend heavily on prompts, retrieval sources, system instructions, guardrails, and user workflows. The “model” is often really a stack.
If your approval process still treats generative AI like a slightly more complicated regression model, you are governing the wrong object.
Where model risk management for generative AI breaks down
The first failure mode is scope confusion. Teams say they are reviewing a model, but the real risk sits in the application around it. An enterprise support bot might use a frontier model from one vendor, a retrieval layer pulling from internal knowledge bases, a prompt template written by a product team, and a human escalation process owned by operations. Which part is being validated? Usually, only the easy part.
The second failure mode is false provenance. In conventional model governance, firms expect lineage: data sources, training approach, assumptions, validation evidence, performance results. With generative AI, especially vendor-hosted systems, much of that is partial or unavailable. So firms compensate with long questionnaires and legal language. That may satisfy internal process, but it does not create technical assurance.
The third failure mode is unstable behavior. A generative AI system can produce different outputs for similar inputs, degrade when source content changes, or behave differently after a vendor update that never hits your internal model inventory in a meaningful way. A once-a-year validation cycle is theater here.
The fourth failure mode is ownership drift. Risk signs off one piece, security signs off another, privacy reviews data handling, procurement manages the vendor, and the business owns the use case in theory. In practice, no one owns the end-to-end system after launch. That is where incidents start.
The real implication: paper compliance, live exposure
This is not just a governance aesthetics problem. It affects production risk.
When model risk management for generative AI is misapplied, firms end up approving systems they cannot actually supervise. They also block low-risk tools because the process is too blunt to distinguish between a drafting assistant and an autonomous customer-facing workflow. That pushes teams into workarounds.
And workarounds are where risk gets interesting.
If business teams cannot get an internal tool approved in time, they buy a small vendor under a budget threshold, paste sensitive data into a sanctioned-but-unreviewed feature, or wrap an API into an existing application without updating any inventory. Now governance has excellent records for a handful of officially blessed systems and very poor visibility into what people are actually using.
That is the dirty secret in a lot of AI governance programs right now: the more rigid the front door, the more activity leaks through side entrances.
What a workable control model looks like
The fix is not to throw out model risk management for generative AI. The fix is to narrow what must be controlled, speed up what can be standardized, and stop pretending every use case deserves the same process.
Start with system-level governance, not model-level governance alone. For generative AI, the unit of control should usually be the deployed use case: model, prompts, retrieval sources, tools, data flows, output channel, and human oversight. If you only document the model vendor and high-level purpose, you have not captured the real risk object.
Next, tier by impact, not by novelty. Too many programs still escalate anything with “AI” in the name into a heavyweight review. That is lazy. A low-impact internal drafting assistant should not wait behind a customer eligibility engine or a claims decision support workflow. Build review lanes based on actual harm: customer impact, regulatory exposure, autonomy level, data sensitivity, reversibility, and dependence on generated output.
Then require pre-production evidence that matches generative AI failure modes. That means scenario testing, adversarial prompting, failure injection, and evaluation against real task conditions. Not just generic performance claims from the vendor. If the use case involves retrieval, test the retrieval. If it involves policy guidance, test outdated or conflicting source material. If it is customer-facing, test tone, refusal behavior, escalation triggers, and silent fabrication under pressure.
You also need change monitoring that reflects the stack. For conventional models, teams often monitor performance metrics and drift. For generative AI, monitor prompt changes, retrieval source changes, safety setting changes, feature updates from the vendor, and where humans are overriding or ignoring outputs. A model can remain the same while the operational risk changes materially.
Finally, assign one accountable owner for the live system. Not a committee. Not a shared mailbox. One named owner who can answer basic questions quickly: what this tool does, what data it touches, what can go wrong, what changed last month, and who has authority to disable it.
If no one can answer those questions in ten minutes, the control environment is mostly decorative.
A practical operating model for the second line
Second-line teams do not need to become prompt engineers. They do need to stop reviewing generative AI as if the main challenge is mathematical opacity. In many enterprise settings, the larger issue is operational opacity.
A useful second-line approach has four artifacts.
First, a short use-case record that captures system purpose, business owner, decision context, data sensitivity, user population, output channel, and escalation path.
Second, a technical dependency map that identifies the model provider, retrieval sources, prompt owners, connected tools, and any external APIs or plugins. This is the minimum viable map of the actual system.
Third, a testing pack tied to failure scenarios that matter for the use case. Not 80 pages of generic controls. Evidence of how the system behaves when the context is ambiguous, the source data is incomplete, the user is malicious, or the model is confidently wrong.
Fourth, a change log with thresholds for re-review. Most firms already know how to trigger review for a new vendor or a new data source. They are less disciplined about prompt changes, role expansion, or product updates that alter behavior without changing the architecture diagram.
That is the shift: less reverence for the model as an abstract object, more attention to the live operating system around it.
The executive takeaway
Executives should be suspicious of any program that says generative AI is fully covered by existing model risk management with only minor adjustments. That usually means one of two things: either the process is so superficial it misses the real exposure, or it is so heavy that people are already bypassing it.
Model risk management for generative AI should do three things well: identify which use cases actually matter, force evidence on real failure modes, and keep ownership attached after deployment. If it does not do those things, it is compliance furniture.
The point is not to make AI governance elegant. The point is to keep control attached to the system people are actually using.
That is harder than updating a policy. It is also where the real work starts.