AI costing prompts · Read this first
How to stop AI inventing your numbers
A large language model does not look anything up. It predicts the most plausible next words, and a plausible-looking number is exactly the kind of text it is good at producing. In most writing that is harmless. In finance it is dangerous, because a fabricated figure that reads like a real one will pass straight into a model, a forecast or a board pack before anyone questions it. This page is the discipline that stops that happening.
In short
The fix is not a better model or a cleverer trick; it is a short set of guardrail instructions you add to every costing prompt. They force the AI to work only from the data you give it, to show its formulas, to flag what is missing, and to admit when it does not know. Use the seven below individually, or paste the combined block at the foot of this page in front of any prompt. They turn a confident guesser into a careful assistant.
Why it happens
This is not a bug that a future version will quietly remove. It is inherent to how language models work. They are trained to generate text that is statistically likely given everything before it, with no built-in concept of whether a statement is true or whether a number was actually computed. When your prompt leaves a gap, the model fills it with whatever is most plausible, and "plausible" and "correct" are not the same thing. Independent research has reported hallucination rates roughly in the range of 15 to 25 percent on financial and numerical tasks when no safeguards are in place; that figure is approximate, varies widely by model and task, and should be verified rather than quoted as precise. The direction of the finding is the point: unguarded, these tools invent at a rate that matters in finance.
We tested it ourselves before writing this. Given a vague instruction, "build me a TDABC model," with no data attached, the model produced a complete, confident model: it invented a department cost, a headcount, and a per-minute capacity rate, none of which existed, and presented them with the same calm authority it would use for a real result. Nothing in the output signalled that the foundation was fiction. That is precisely the failure mode to guard against, because the more polished the answer looks, the less likely anyone is to check it.
The seven guardrails
1. Restrict it to the data you provide
The single most important instruction. It removes the gaps the model would otherwise fill with invention.
Work only from the data I give you. Do not invent any numbers, rates or volumes.
2. Make it flag every assumption
When a model must assume something, you want it said out loud, not buried in the arithmetic.
List every assumption you make. If a figure is missing, label it DATA MISSING and tell me what you need.
3. Make it show the formula before the number
A visible formula is checkable; a bare number is not. This also catches arithmetic slips.
Show the formula at each step before computing any value.
4. Give it permission to say "I do not know"
Models invent partly because they are nudged to be helpful. Explicitly allow a non-answer.
If you cannot derive a figure from my data, say so. Do not estimate.
5. Make it cite the source line
Tying every number back to a row in your data makes fabrication obvious, because invented numbers have no source.
For each number, cite the exact source row in my data.
6. Make it separate fact from inference
You need to know which parts are your data, which are assumptions, and which are the model's opinion.
Clearly separate what you calculated from my data, what you assumed, and what is your suggestion.
7. Make it verify the totals
A reconciliation check at the end catches both invention and arithmetic error in one pass.
After the calculation, check that the parts sum to the total and flag any discrepancy.
The full guardrail block
Paste this once at the top of any costing prompt. It combines all seven into a single instruction the model must follow before it touches your data.
Before answering, follow these rules for the entire response: 1. Work only from the data I give you. Do not invent any numbers, rates or volumes. 2. List every assumption you make. If a figure is missing, label it DATA MISSING and tell me what you need. 3. Show the formula at each step before computing any value. 4. If you cannot derive a figure from my data, say so. Do not estimate. 5. For each number, cite the exact source row in my data. 6. Clearly separate what you calculated from my data, what you assumed, and what is your suggestion. 7. After the calculation, check that the parts sum to the total and flag any discrepancy.
Audit an AI answer for invented numbers
The guardrails above stop invention before it happens. This one works after the fact: paste any costing answer an AI has already given you, and it checks every number against the data you actually provided, so a fabricated figure cannot pass unnoticed into a model or a board pack. Run it on anything you did not generate under the guardrails.
The hallucination audit
You are auditing an AI-generated costing answer for fabricated numbers. Work only from two things I give you: the source data I originally provided, and the AI answer to be checked. Do not add any figures of your own. I want every number in the answer traced back to my source data, so I can see which figures are real, which are assumptions, and which were invented. Step 1. List every number that appears in the AI answer. Step 2. For each one, mark it as: SOURCED (trace it to the exact row in my data), DERIVED (show the formula that produces it from my data), ASSUMED (the answer states it is an assumption), or UNSUPPORTED (it appears in the answer but cannot be traced to my data or a stated assumption). Step 3. For every DERIVED figure, recompute it from my data and flag any that does not match. Step 4. List the UNSUPPORTED figures separately and plainly; these are the ones to treat as invented until proven otherwise. Step 5. Check that any totals in the answer actually sum from their parts, and flag any that do not. Step 6. Give a one-line verdict: is this answer safe to use as is, safe after I supply the missing figures, or not safe. Do not soften the verdict to be agreeable.
The audit does not make an answer correct; it makes the invented parts visible, which is what lets you decide whether to trust the rest.
Turn the guardrails into a reusable standard
The seven guardrails work best when they are not retyped from memory each time. This add-on turns them into a reusable asset, a one-page reference card, a saved system instruction, or a short team standard, so the discipline travels with everyone who uses AI on your numbers. Paste it to package the block above.
Now turn the seven guardrails into a reusable asset my team can adopt. Ask me which format I want, or default to the one-page reference card: - CARD: a clean one-page reference (PDF) with the seven guardrails as a numbered checklist, a one-line why for each, and the combined guardrail block at the foot ready to copy. Designed to sit next to someone's screen. No chat formatting. - SYSTEM PROMPT: a single reusable instruction I can save as a custom instruction or system prompt in my AI tool, so the guardrails apply to every costing conversation automatically, phrased to hold across a whole session. - STANDARD: a short internal standard (one page) stating that any AI-assisted costing work must follow these guardrails, with the checklist, a line on why it matters in finance, and a space for a review sign-off. Written in plain, professional language. Rules for the deliverable: - Use the seven guardrails and the combined block exactly as written on this page; do not reword the instructions. - Keep it to one page and plain enough to adopt without training. - Add a discreet footer credit line, small and unobtrusive, exactly as written below. Credit line to embed (use verbatim, in the footer): "Based on the AI costing guardrails from costandprofitability.com/ai-costing-prompts/avoid-ai-hallucinations" Keep it to one small line; it should read as a quiet source note, not an advertisement.
The credit line is deliberately modest, a source note rather than a watermark, so the card or standard is comfortable to circulate internally while it stays traceable to where it came from.
A guardrail reduces risk; a real model removes it
This is exactly the discipline we apply when we build a cost model: work only from reconciled data, show every formula, flag every assumption, tie every number to a source. A prompt guardrail lowers the chance of a fabricated figure, but it cannot reconcile your ledger, validate your drivers or own the result. Only a model built on your real, reconciled data does that. If the numbers matter enough to protect, build them on something solid.
Related
Proof
A distributor in New Zealand. €1.335M of cost-to-serve made visible, then halved, and 830 loss-making customers brought down to 295.
Read the case study →Who you would be talking to
Miguel Guimarães, Founding Partner
Cost and profitability practitioner for 25+ years. Presented the Damco cost-to-serve case at Managing for Profit (Amsterdam RAI, December 2009), on the same programme as Robert S. Kaplan.
Call +351 910 313 731
