Model Risk

Model risk is the risk that a model’s output is wrong, or correct but misused, and that nobody catches it before a decision gets made on it. The term predates AI; banks have managed model risk in credit and trading models for decades under frameworks that require validation, monitoring, and documented limits.

The two halves of the risk

The first half is a defective model: wrong logic, bad assumptions, or inputs that don’t represent what the model claims to represent. A revenue forecast calibrated on a period with unusual seasonality is defective in this sense even if every formula in it is correct.

The second half is correct output used outside its intended scope. A model built to size a hiring plan gets pointed at a covenant calculation. A sensitivity range built for internal discussion appears in a board deck as a commitment. This half causes more real damage than the first, and it’s the half that validation testing tends to miss, because the model passes every test of its own logic.

Why AI makes it harder to bound

Traditional model risk management assumes a model you can open. You can read the formulas, document the assumptions, define the range of inputs it’s valid over, and rerun it to reproduce yesterday’s answer. A language model offers none of that: there’s no fixed logic to validate, the valid input range isn’t definable, and the same prompt can produce different output on two runs, so even reproducibility, the foundation the older frameworks stand on, isn’t free.

That’s why “an AI in the loop” tends to expand model risk rather than manage it. The controls that governed the spreadsheet don’t transfer, and adding a confidence score doesn’t replace them.

How teams narrow it

The practical move is to shrink the surface the model touches. Keep arithmetic in a deterministic calculation engine so it’s reproducible and testable. Require every stated figure to carry a citation, which converts a validation problem into a lookup. Scope the model to reading, mapping, and explaining rather than deciding. Document what the model is for, and check its actual use against that, since the misapplication half of the risk is a process problem rather than a technical one. And keep a replayable audit trail so an error found in June can be traced to the run that produced it.

Rexfin narrows it by keeping the AI out of the arithmetic entirely: calculations run deterministically, sources reconcile into one governed model, and policy checks and role scoping apply before a figure renders. See what the 2026 model-risk framework requires.