Why 'Just Use a Code Interpreter' Doesn't Make AI Finance-Safe

Every time someone points out that large language models are bad at math, a confident engineer says the same thing: “Just give it a code interpreter.” The model writes Python, the Python runs, the arithmetic is exact. Problem solved.

It isn’t.

The code interpreter solves a real but narrow problem. It removes the one failure everyone can see and leaves the four nobody screenshots. Your AI stops claiming that 8.4 times 12 is 99.2. Good. But the number it multiplied by 12 was still pulled from the wrong column, in the wrong period, without a source you can trace. The output is now precisely wrong instead of approximately wrong, and a CFO can’t tell the difference by looking.

What a code interpreter actually fixes

When a model offloads computation to a sandboxed Python (or any function-calling tool), it stops doing arithmetic in its head. That matters. LLMs tokenize numbers as text and pattern-match the result, which is why you see arithmetic accuracy fall off a cliff as digit counts grow. Hand the calculation to numpy and you get a deterministic, repeatable answer. Sum a column of 4,000 transactions and it ties out every time.

So if the question is “can the AI add up a list of numbers correctly,” the answer with a code interpreter is yes. We’re not arguing with that. We use deterministic execution ourselves, because the LLM should never be the calculator.

The trouble is that “add up these numbers” is almost never the actual job. The job is “what was net revenue retention in Q3, and is it the same number we’ll show the board.” That question has at least four ways to go wrong before any multiplication happens.

The four failures the interpreter never touches

1. Wrong inputs

A code interpreter computes flawlessly on whatever you feed it. Feed it gross revenue when you meant net, deferred revenue booked as recognized, or a duplicated invoice line, and it returns an exact answer to the wrong question. Garbage in, perfectly-computed garbage out. The sandbox has no opinion about whether the inputs are the right inputs. It can’t, because it doesn’t know what “right” means in your chart of accounts.

This is the failure that kills you in a board deck, because the math checks out. Nobody questions a number that adds up.

2. Unreconciled data

Most companies don’t have one set of numbers. They have QuickBooks, a billing system, a CRM with ARR fields someone updates by hand, and three spreadsheets. Pull “revenue” from the billing tool and “revenue” from the ledger and you get two different figures, both defensible, neither reconciled to the other.

An AI with a code interpreter will happily calculate on either one. It won’t notice they disagree. It has no concept of a single source of truth that ties out to the general ledger, because reconciliation is an upstream data problem, not a runtime computation problem. That gap is the whole reason autonomous finance agents need a reliable numbers layer first. This is the part teams underestimate most. Surveys keep finding data quality and integration as the top blocker to getting value from AI in finance, not model accuracy. The model was never the bottleneck.

3. Missing provenance

Ask a CFO to put a number in front of an auditor, an investor, or a regulator, and the first question is “where did this come from.” A code interpreter gives you a result and a transcript of the Python that produced it. It does not give you a line back to the source document, the journal entry, the period, the version of the model. If you can’t click a figure and walk it back to its origin, it isn’t audit-ready, no matter how clean the arithmetic is. This is exactly why audit trails matter more than accuracy alone once real money is on the line.

4. Locale and format ambiguity

Here’s the quiet one. Is 12,345 twelve thousand or twelve-point-three-four-five? In the US it’s twelve thousand. In Germany it’s roughly twelve. Mix Eastern Arabic numerals into the same document and the parsing gets worse. A code interpreter executes whatever the model decided the number was. If the model misread the separator, the sandbox faithfully computes on a value that’s off by three orders of magnitude. No error is thrown. The Python ran fine.

Currency, fiscal-year boundaries, percentage-vs-basis-points, thousands-vs-millions scaling in a reported table. Every one of these is an interpretation that happens before the calculation, and the calculator can’t catch a misinterpretation it never sees.

The pattern: computation is the easy 20%

Notice what all four have in common. They happen before or after the arithmetic, not during it. A code interpreter is a fix at the wrong layer. It’s like hiring a brilliant accountant and handing them a shoebox of unsorted, mislabeled, possibly-duplicated receipts. Their arithmetic will be perfect. The financials will still be wrong.

Getting the numbers right in finance is mostly a data problem and a provenance problem. The math is the easy part, and it’s the only part the interpreter addresses.

What actually makes AI finance-safe

The fix isn’t a smarter model or a faster sandbox. It’s a reconciled model layer that sits between your raw data and the AI, and it has to do three things the interpreter doesn’t.

• Reconcile first. Connect the accounting and financial sources, resolve the conflicts, and build one model that ties out to the ledger before the AI ever asks a question. The AI retrieves from verified figures, not from whichever system answered first. We go deeper on this in why reconciled data is the real unlock.
• Calculate deterministically, against the right inputs. Yes, use a deterministic engine. But point it at reconciled, correctly-parsed, period-aligned numbers, with locale and scaling resolved up front. The engine being exact only helps if the inputs are right.
• Trace every figure to source. Each number the AI reports should carry its lineage: the source record, the calculation path, the period, the version. That’s what turns an answer into something you can defend.

This is the neuro-symbolic split done properly. The LLM does what it’s good at: understanding the question, choosing the method, explaining the result in plain language. The deterministic engine does the math. And a reconciled, provenance-tracked model underneath makes sure the math is happening on numbers that are actually yours. You can see how the deterministic engine works and how it connects to your existing systems without you re-keying anything.

A code interpreter is necessary. It is nowhere close to sufficient. The companies that get burned are the ones who saw the arithmetic get fixed and assumed the trust problem was fixed with it.

If you want AI to touch your real numbers, start at the layer the interpreter skips. Book a demo and we’ll show you a figure traced from a board-ready output all the way back to the journal entry it came from.