Prompt Injection (Finance AI)

Prompt injection is an attack where instructions hidden inside content an AI reads (a PDF, an invoice, an email) get treated as commands instead of data. A hostile line buried in a filed document can tell the model to ignore its rules, leak data, or misstate a figure.

Why the attack works

A language model receives its instructions and its input as the same thing: text in a context window. There’s no structural boundary between “this is your task” and “this is the document you were asked to read,” so text in the document that reads like an instruction can be followed like one. Hiding it is easy: white text on white background in a PDF, a comment in a spreadsheet cell, metadata, or a line inside an otherwise ordinary vendor note.

This is not a bug in a specific model. It’s a property of how instructions and data share a channel, which is why “we told the model to ignore instructions in documents” is a mitigation rather than a fix. The injected text can address that instruction directly.

Why finance AI is unusually exposed

Finance AI is exposed by design: it exists to read uploaded statements and filings, most of which come from outside the organization. Vendor invoices, bank statements, customer contracts, and auditor requests all arrive as documents from parties the company doesn’t control, and the workflow requires reading them.

The payoff for an attacker is direct, too. An injected instruction that nudges a single invoice total, or that reclassifies an expense into a different period, produces a financial error rather than a chat mishap. In a close cycle, that error is in the statements before anyone reviews it. Where the AI is an agent with tools rather than a read-only assistant, the exposure extends to whatever those tools can do: querying other entities, exporting data, or triggering an action.

What defense looks like

Treating every ingested document as untrusted input, not as instructions, is the baseline defense, and it has to be enforced architecturally rather than by prompt. Practical layers: separate the extraction pass from the reasoning pass so document text never occupies the instruction position; constrain extraction output to a schema, so a document can supply a value but not a command; keep tools read-only and role-scoped, so a hijacked run is bounded by permissions rather than by the model’s compliance; and require a deterministic calculation plus a source check on every figure, so an injected number still has to reconcile against the document’s own printed subtotal to survive.

See how Rexfin defends against prompt injection from untrusted documents.