Governing the AI you didn't build

Most of the AI now making decisions inside your business came from someone else. You can't open it, you can't fully test it, and you can't subcontract the accountability. What you can still own is the gate it has to pass through.

The uncomfortable starting point

Walk the floor of any bank, insurer, or health system and ask where the AI is. The honest answer is rarely "the model our data science team built." It's the foundation model behind a vendor's summarization feature. It's the agent embedded in a SaaS platform that now drafts adverse-action language, routes claims, or screens applications. It's an API your team wired in over a quarter because it shipped faster than anything you could build. It's a copilot inside a tool your staff already trusted.

This is the reality most governance programs were not designed for: the majority of enterprise AI is bought, not built. And bought AI carries a property built AI does not. You cannot validate a model you cannot see inside. The weights are not yours. The training data is undisclosed. The update cadence is set by the vendor, not your change-control board. The model you assessed in March may behave differently in June, and no one is obligated to tell you.

For a risk or compliance function, that is not a technical inconvenience. It is an accountability problem.

"The vendor's model did it" is not a defense

Here is the part that does not move. When a third-party model produces a bad decision inside your workflow — a wrongful denial, a discriminatory pattern, a hallucinated disclosure, an unauthorized action — the customer holds you accountable. The regulator holds you accountable. The plaintiff names you. The vendor is, at most, a line item in your indemnity negotiation.

Accountability does not transfer with the procurement contract. You can outsource the model. You cannot outsource the consequence of the decision it made on your behalf, in your name, to your customer. An examiner reviewing a harmed consumer's file does not care whose GPUs produced the output. They care that your institution acted, and they want to see what controlled the action.

So the practical question is not "how good is the vendor's model?" It is "what did we do, at the moment of action, to keep it inside the lines we are answerable for?"

Why TPRM is necessary and not sufficient

Most organizations already have an answer to vendor AI, and it is third-party risk management. Send the questionnaire. Collect the SOC 2. Review the security posture. Negotiate the contract clauses on data use, model changes, and liability. Re-attest annually.

Do all of it. None of it is wasted. But understand precisely what it gives you and what it does not.

Traditional TPRM assesses the vendor, at a point in time, against a set of attestations. It tells you the vendor has a process. It does not tell you what the vendor's AI will actually do inside your environment at 2:14 on a Tuesday when a real customer's file moves through it. A questionnaire is a photograph of a posture. The decision that harms someone is an event — live, specific, and downstream of every assessment you ran.

The gap is structural. Point-in-time assessment of the supplier cannot govern run-time behavior of the model. And run-time is exactly where accountability attaches.

You don't control the model. You control the gate.

This is the shift that makes vendor AI governable.

A black-box vendor model is still, functionally, a thing that proposes. It proposes a classification, a draft, a routing decision, an action. What turns a proposal into an action inside your business is the path between the model's output and the real-world effect — and that path runs through systems you own.

So put a control there. A deterministic gate you own, sitting between the vendor's AI and the consequential step, that checks every proposed action against the rules you are answerable for:

• Authority — is this agent permitted to take this kind of action, at this scope, on this object?
• Policy — does the proposed action conform to your written policy, not the vendor's defaults?
• Evidence — is the basis for the action present and recorded, or is the model asking you to act on something it cannot show?
• Harm — does this cross a line you have defined as unacceptable regardless of how confident the model is?
• Regulation — does it satisfy the specific obligations that apply to this decision type?

The model can be entirely opaque and this still works, because the gate does not need to understand the model. It needs to evaluate the action against rules expressed deterministically — allow, block, or escalate to a human. You did not earn the right to see inside the model. You do not need it. You own the doorway it has to walk through.

Named agents, bounded authority, no shadow AI

The same discipline applies to vendor agents, which are now the sharper edge of this problem. An agent that can take actions — not just emit text — needs to be a named, scoped, accountable entity in your environment, with explicit bounded authority. Not an anonymous capability that arrived bundled in a product and quietly started doing things no one chartered it to do. The first failure mode in vendor agentic AI is not a bad model. It is a shadow agent operating outside anyone's authority map. Name it, bound it, and route it through the gate, or do not let it act.

Where the regulators are heading

You do not need an invented citation to read the direction of travel. The supervisory expectation for model risk and the expectation for third-party risk are converging. The same governance discipline that mature institutions apply to in-house models — inventory, defined authority, documented controls, evidence of what happened and why — is increasingly expected for vendor and third-party models too. The fact that the model came from a supplier is becoming less and less of an excuse, not more. Plan for the world where "we bought it" raises the bar instead of lowering it.

An honest note on what we enforce today

KAiM is direct about its stage. Formal vendor due-diligence artifacts — structured intake, attestation capture, the supplier-facing side of this — are on the roadmap, and we will say so plainly rather than overstate them. The part KAiM Helm enforces today is the part that matters most and is hardest to retrofit: control at the gate. Vendor model or yours, the proposed action passes through a deterministic check you own, with named, bounded-authority agents and a record of every allow, block, and escalation. This is design-partner and controlled-demonstration work, not a roster of customer claims. We would rather show you the gate working than quote you a number we cannot stand behind.

Key takeaways

• Most of your AI is bought, not built — and you cannot validate a model you cannot see inside.
• Accountability does not transfer. When a vendor's model makes a bad call in your workflow, you answer for it. "The vendor did it" is not a defense.
• TPRM is necessary but insufficient. It assesses the supplier at a point in time; it does not govern what the model does at the moment of action.
• You may not own the model, but you own the gate. A deterministic control checks every proposed action against authority, policy, evidence, harm, and regulation — and works even against a black box.
• Treat vendor agents as named, bounded, accountable entities. No shadow agents acting outside your authority map.
• The regulatory direction is convergence — vendor models held to the same governance bar as in-house ones.

You can't open the model. You can still control what it's allowed to do. Our Control Gap Assessment maps where third-party and vendor AI is already taking consequential actions in your environment without a gate you own — and shows you the shortest path to closing it. Start there.