Charge by the token and pricing looks solved: usage goes up, revenue goes up, in perfect lockstep with cost. In practice the lockstep breaks almost immediately, because the cost per token is not one number. A short factual query and a long document draft can carry the same “per request” label and a completely different cost, and a pricing page built around a single per-token rate quietly loses money on the second one every time.
How should AI products price per-token usage profitably?
A flat per-token rate ignores that input tokens, output tokens, cached context and retrieval calls all carry different costs, and that heavy users concentrate disproportionately in the highest-cost requests. Profitable pricing separates these cost drivers, prices output tokens higher than input tokens where the gap is real, and checks gross margin per plan tier against actual usage data - not against the average query the pricing model assumed.
Why “per token” is not one price
Model providers already price input and output tokens differently, often by a factor of three to five, because generating a token costs more compute than reading one. Products that resell access at a single blended rate inherit that asymmetry without pricing for it: a user who asks short questions and gets long answers costs more per request than the average the rate was built on, and a user who pastes in a long document and asks for a one-word answer costs less. Without tracking the input/output split per customer, the pricing model is guessing at an average that few real users actually match.
Context and retrieval are part of the bill, not overhead
Every token of context sent to the model - conversation history, retrieved documents, system instructions - is billed as an input token even though the user never typed it. A feature that keeps a long conversation history or retrieves several documents per query can spend more tokens on context than on the user’s own words, and that cost scales with product design choices, not with anything the customer controls. Caching repeated context lowers it, but caching has its own infrastructure cost that a per-token price rarely accounts for separately.
Heavy users are not just more usage, they are different usage
Flat per-seat or per-token pricing assumes usage scales evenly across the customer base, and it never does. A small share of accounts typically drives a disproportionate share of tokens, and those accounts often run the most expensive request types: long context windows, multi-step agent chains, high-output-token tasks. A pricing model that was profitable on the average customer can be badly underwater on the top decile, and that decile is usually the one a growing product depends on retaining.
Building a gross margin you can actually defend
Pricing an AI product on real unit economics starts the same way any TDABC exercise does: define the activity - a chat turn, a document summary, an agent task - and trace the cost drivers that a unit of it actually consumes, separately for input tokens, output tokens, retrieval calls and any cached-context discount. Once those are separated, gross margin can be tested per plan tier against the usage mix real customers generate, not the mix the price list assumed. That is what turns “we charge per token” into a number finance can stand behind.
A single per-token rate prices the average request. Your margin is decided by the requests that are not average.
What to check before the next pricing update
Before adjusting a per-token or per-seat price, confirm the input/output token split is measured per customer segment, not assumed. Confirm context and retrieval costs are traced to the feature that causes them rather than folded into a shared infrastructure line. And confirm gross margin is checked against the heaviest-usage decile as well as the average - a price that works on average and fails on your best customers is a price that will need to change again soon.
Book a live walkthrough of CostCtrl and see how input tokens, output tokens, retrieval and review costs come together into one real margin per plan.
Related reading: AI gross margin and what it really costs to serve an AI feature.