Calibration and Trust Tuning: How Rexfin Decides What Blocks an Export

A verifier that blocks nothing is decoration. A verifier that blocks everything gets disabled by the second week. The interesting design problem isn’t whether to check that assets equal liabilities plus equity, or that cash flows tie to the change in cash: it’s deciding exactly how tight that check should be before it’s allowed to stop someone from exporting a board pack.

Get it too loose, and a genuinely broken balance sheet slides through untouched. Get it too tight, and a correct statement that happens to be rounded to the nearest thousand gets flagged as an error, and your finance team stops trusting the tool by lunchtime. Both failure modes destroy the same thing: confidence that when Rexfin says a number is fine, it’s actually fine.

The line isn’t a developer’s judgment call

We don’t decide where that line sits by feel. We measure it. The process starts with a set of real, audited filings whose every statement line has been hand-keyed by a person independently (not derived from the extraction pipeline itself, so it can’t grade its own homework). Each filing is then paired with deliberately corrupted variants: a figure scaled wrong by a factor of a thousand, a transposed digit, a balance sheet that no longer balances, a cash-flow statement that no longer ties. One variant is a control that should never trip anything at all: a correctly rounded statement whose components sum to a total that differs by a fraction of a rounding unit. If a tolerance setting fails that control, it’s too tight, full stop.

Against that set, we sweep through candidate tolerance settings for each check and measure two things: does it catch the planted defects, and does it ever wrongly flag something correct. Only a setting that lets zero correct statements through as false failures, while catching the large majority of planted errors, is even a candidate.

Why the test can’t grade its own homework

Here’s the part that’s easy to skip and dangerous to skip: with only a handful of real filings to calibrate against, it’s tempting to pick the tolerance that works best on those exact filings and call it done. That number would be flattering and meaningless: it’s the tuning error, not a real measure of how the check performs on a filing it has never seen. So every candidate setting is tested by holding one filing out entirely, tuning on the rest, then scoring only against the filing that was never part of the tuning. That’s the number that actually decides anything.

From advisory to hard block

A check only gets promoted from advisory (it warns, but never stops an export) to a hard block once it clears that bar on filings it never trained on: zero false failures, and it catches the overwhelming majority of planted defects. Checks that don’t clear it stay advisory. That’s not a permanent demotion either: if a later batch of filings pushes a previously-promoted check back below the bar, it gets demoted again. Promotion is never allowed to outlive its evidence.

The result of all that work gets frozen into a single versioned file the verifier reads when it starts up. Every tolerance, every promotion decision, and the evidence behind it live in that one place, never as a hardcoded number buried in application logic. When an exported board pack is later questioned, the exact calibration version that blessed those numbers can be pulled up and inspected. Change the calibration, and it shows up as a reviewable difference, not a silent edit.

None of this happens before a filing has already passed a much blunter gate: every extracted line has to match the hand-keyed original within tolerance, completely, before anything downstream (including answers, audits, or verification) is allowed to trust it at all.

The honest limits

Calibration doesn’t make Rexfin infallible. It makes the boundary between “we’re confident enough to block you” and “we’re not sure, so we’ll just flag it” a measured decision instead of a guess, and it makes that decision visible and reproducible rather than baked silently into code. It also means some checks (the ones with less real-world evidence behind them, like certain cash-flow reconciliations) stay advisory-only for a long time, and that’s the correct, honest outcome, not a shortcut.

If you want to understand what happens once a check does fire, see how verification works. And for how this evidence-based mindset extends to the AI’s answers themselves, see how Rexfin measures answer quality. Both sit under the same principle covered across inside-the-rexfin-platform: nothing gets to claim trust it hasn’t earned with evidence.