Why Finance Agents Must Call Tools, Not Compute

Ask a large language model to add two nine-digit numbers and it will often return something that looks exactly right and is off by a few thousand. The digits are plausible. The magnitude is correct. The answer is wrong. That single behavior is the reason most “AI for finance” demos fall apart the moment a controller checks the math.

Here is the uncomfortable mechanism. An LLM does not run arithmetic. It predicts the next token. When you ask for 14,820,116 minus 9,335,402, the model is not subtracting; it is generating the sequence of digits most statistically likely to follow your prompt, given everything it absorbed in training. For small, common sums it has effectively memorized the patterns and gets them right. For the long, irregular figures that fill a real general ledger, it is guessing at a shape. Sometimes the guess lands. You have no way to know which times from the output alone, because a confident wrong number reads identically to a confident right one.

The number looks right, which is the problem

This is worse than ordinary software bugs because the failure mode is camouflage. A broken formula in a spreadsheet usually throws #REF! or a wildly impossible total. A language model produces a figure that sits comfortably inside the range a CFO would expect. Revenue that should be 47.3 million comes back as 46.9 million. Gross margin that ties to 62% comes back as 64%. Nothing screams. The error survives review precisely because it was generated to look like the truth.

The benchmarks bear this out. On FinanceBench, a test suite of question-answer-evidence triplets drawn from public company filings, GPT-4-Turbo paired with a retrieval system answered or attempted only a minority of questions correctly, getting roughly four out of five wrong or refused in the original evaluation. Even when handed the exact document containing the answer, accuracy on financial-statement questions has been reported anywhere from the high teens to the high eighties depending on phrasing and context placement. Newer specialized stacks push much higher, but the spread itself is the warning. A system whose accuracy swings forty points based on where you put the question in the prompt is not something you build a board number on.

None of this means the model is useless. It means you have asked it to do the one thing it structurally cannot guarantee.

Reasoning in the model, math in the layer

The fix is not a better prompt or a larger model. It is a division of labor.

Language models are genuinely good at the soft parts of finance work: understanding that “burn” means net cash outflow, deciding that a “runway” question requires cash balance divided by monthly burn, noticing that a user probably wants the figure year-to-date rather than quarterly, and stitching a plain-English explanation around the result. That is reasoning, and it is where the model earns its place.

What the model must not do is the subtraction, the division, the currency conversion, the period roll-up, or the ratio. Those go to deterministic tools: actual code that runs the formula the same way every time. The model’s job is to pick the right tool, supply the right arguments, and read back the result. This is what tool calling, or function calling, was built for: let the linguistic component be flexible and let the computational component be exact, and never confuse the two.

Said plainly: the agent should reason about what to calculate and then call something else to actually calculate it. When an agent computes a margin in its own head, you are trusting a probability distribution with your audited accounts. When it calls a tool, you are trusting arithmetic.

Why a tool call still needs a reconciled model underneath

Tool calling alone does not finish the job, and this is where a lot of 2026 architectures stop one step short. Giving an agent a calculate_ratio() function is necessary but not sufficient, because the function is only as trustworthy as the numbers you feed it. If the agent pulls a revenue figure from one system, a cost figure from a stale export, and a headcount number from a slide, the tool will compute a perfectly deterministic ratio over inconsistent inputs. Determinism on bad data just gives you a repeatable wrong answer.

So the tools have to run against one reconciled model: a single financial model that ties out to the ledger, where every figure has a known source and the inputs already agree with each other. That is the part Rexfin treats as the foundation. We connect your accounting and financial-data platforms (QuickBooks, Xero, NetSuite, Sage, SAP, Oracle, a warehouse) or uploaded statements, build one reconciled model that ties to the ledger, and then expose deterministic calculation tools over it. The agent reasons; the deterministic engine computes; every output traces back to source.

The sequence matters. Connect, reconcile, then let the engine do math: skipping the middle is why most demos produce numbers no controller will sign. We walk through that flow in how it works, and the same principle is why MCP for finance is not enough on its own: governed access to your ERP gets the model live data, but without a reconciled layer and deterministic compute, the LLM is still doing fragile probabilistic arithmetic on raw fields.

What this buys a skeptical CFO

Three things, concretely.

First, replayability. The same question, asked twice, returns the same number, because the math ran in code rather than in a sampler. You can prove a figure the way you prove a spreadsheet formula: by re-running it.

Second, traceability. Because the calculation ran over a reconciled model, every output points back to the source rows that produced it. An auditor asking “where did this come from” gets a lineage, not a shrug.

Third, a defensible boundary for autonomy. Once math leaves the model, you can let agents do more without lying awake about it. This is the prerequisite for the autonomy tiers that decide which workflows an agent can run on its own, and it is what makes governed autonomy safer than putting a human inside every transaction. You cannot govern an agent whose numbers you cannot reproduce.

The takeaway

An LLM that computes is guessing at a number’s shape. An LLM that calls a tool over a reconciled model is asking arithmetic to do arithmetic and keeping language for language. The first feels magical in a demo and fails in an audit. The second is boring, and boring is exactly what you want underneath a board deck.

If you are evaluating agentic finance tools, ask the vendor one question: does the model compute the figure, or does it call something deterministic that does? If they cannot answer cleanly, the math is a guess. Book a demo and we will show you the difference on your own numbers.