AI Got the Numbers Right and the Math Wrong: The Hidden Class of Financial Hallucinations

A finance lead asked an AI assistant for the change in operating margin between two quarters. The model quoted both figures correctly: 18.2% last quarter, 14.9% this quarter. Both numbers tied out to the filing. Then it reported the change as a “3.3% decline.” It wasn’t. Margin fell 3.3 percentage points, which is an 18% relative drop. Two different facts, one of them wrong, and the wrong one is the one that goes in the board deck.

This is the error class almost nobody screens for. When teams worry about AI making things up, they picture invented revenue or a fabricated citation. Those happen, and they’re easy to catch because the number simply doesn’t exist in the source. The quieter failure is different: every input is correct, the answer is wrong, and nothing in the output looks suspicious. The model retrieved fine. It reasoned badly.

Right inputs, wrong arithmetic

Recent benchmarking on financial question-answering keeps landing on the same split. In one analysis of model failures, calculation errors and rounding or precision issues each accounted for roughly a third of all wrong answers, together more than two-thirds of the total (arXiv, FinanceQA-style evaluation). A separate study of multi-modal models on financial tasks attributed about 42% of failures specifically to calculation and reasoning errors rather than retrieval (arXiv, FAITH benchmark). The pattern is consistent: the model finds the right cells, then fumbles what it does with them.

A few error shapes show up again and again in finance work:

• Absolute vs. percentage change. Confusing a move in percentage points with a relative percentage change, exactly the margin example above.
• Ratio inversion. Reporting debt-to-equity when the question asked equity-to-debt, or flipping the numerator and denominator on a coverage ratio. Both numbers are real; the relationship is backwards.
• Wrong sign on cash flows. Treating a use of cash as a source, or netting a financing inflow against an operating outflow so the direction of free cash flow flips.
• Unit and scale drift. Mixing thousands with millions, or carrying a figure stated in basis points into a calculation that expects a decimal.

None of these trip a retrieval check. The source numbers are present and accurate. That’s what makes the class dangerous. Your instinct, and your reviewer’s instinct, is to spot-check whether the AI got the figures right. It did. The defect is one layer deeper.

Why a language model does this

It helps to understand the mechanism instead of treating it as random flakiness. A language model doesn’t compute. It predicts the next token from statistical patterns in text. Numbers get tokenized for language, not for math, and a single figure is often split across multiple tokens, so the model never operates on the mathematical structure of the expression (arXiv, tokenization and arithmetic). It has also learned that “the change from 18.2 to 14.9” is frequently followed by phrasing like “a 3.3% decline,” because that phrasing is common in text, even when it’s imprecise. The model is reproducing how people talk about the calculation, not performing the calculation.

There’s no scratchpad it has to honor, no intermediate state it’s required to preserve between steps, and no internal check that a ratio came out the right way up. Researchers have measured how brittle this is: small changes in how a problem is phrased can swing accuracy by tens of percentage points in either direction, a robustness gap documented repeatedly across arithmetic-reasoning benchmarks. A process that fragile is fine for drafting a sentence. It is not fine for a covenant calculation.

To be fair about the limits of the worry: frontier models have gotten much better, and on some curated financial-hallucination benchmarks the strongest models now clear 90%+ accuracy. But “90% of the time the math is right” is a description of a problem, not a reassurance, when the output feeds decisions and the failures are invisible to a quick read. A 9% error rate that you can’t see is worse than a 20% one you can.

The fix is structural, not a better prompt

You can’t prompt your way out of this reliably. Asking the model to “show its work” or “double-check the math” reduces the rate but doesn’t eliminate it, because the checker is the same flawed mechanism that produced the error. The only way to remove an entire class of arithmetic mistakes is to stop letting the language model do the arithmetic at all.

That’s the design principle behind Rexfin. We split the two jobs that current tools collapse into one. The language model is good at understanding intent, finding the relevant figures, and explaining results in plain language. So we let it do exactly that, and nothing more. The actual computation, every ratio, every period-over-period change, every cash-flow sign, runs through a deterministic calculation engine that executes defined formulas the same way every time. Percentage points and percentage change are different operations to the engine, not two phrasings it might blur together. A ratio is defined once, with a fixed numerator and denominator, so it can’t invert.

It works because the figures don’t come from free text in the first place. Rexfin connects to your accounting and financial-data sources, or to uploaded statements, and builds one reconciled financial model that ties out to the ledger. The AI retrieves from that single source of truth, the engine calculates, and every output traces back to the figure and the formula that produced it. When a number lands in front of you, you can see both where it came from and how it was derived.

What to do with this

Stop validating AI finance output by checking only whether the source numbers are right. They usually are. That check passes while a flipped ratio or a points-versus-percent mistake sails through underneath it. If you want a practical screen, our verification checklist for board decks covers what to look for, and if you’re evaluating a tool, building a golden dataset of known-answer calculations will surface exactly this failure mode before it reaches a client.

The broader point: a tool that hands the math to the model is asking you to trust a process that was never built for arithmetic. The better question to ask any AI finance vendor is simple. Where does the calculation actually happen? If the answer is “inside the language model,” you already know the risk you’re carrying. If you’d like to see what it looks like when the answer is “a deterministic engine that ties to your ledger,” book a demo.