Intelai
Reasoning Models Change the Unit of Cost
A reasoning model bills you for thinking and hands back a summary. That changes how you budget, evaluate, and audit an AI workflow.
The first invoice after a reasoning model pilot is a surprise. The token count is higher than the transcript suggests, because the model spent tokens thinking and did not show them to you.
That single fact changes three things at once: how you budget an AI workflow, how you evaluate a model, and what you can honestly call an audit trail.
What changed
OpenAI's o1, released in December 2024, established the pattern that the rest of the industry followed. Before answering, the model generates a long internal sequence of reasoning. More compute at inference time buys better performance on problems that require multiple steps.
That is a real capability change. It is also a billing change. Reasoning tokens are charged, and the raw trace is not returned. The caller receives the answer and, in some interfaces, a summary of the reasoning.
For a chat product, none of that matters. For a workflow that has to be costed, evaluated, and defended, all of it does.
The invoice is for tokens you never see
A standard model's cost is roughly proportional to the size of the prompt and the answer. You can estimate it from the transcript.
A reasoning model's cost is proportional to the prompt, the answer, and an internal budget you do not observe. The same question can consume very different amounts of compute on two runs, because the model stops when it decides it is done.
Three consequences follow.
Budget by distribution, not by average. A per-query estimate built from a sample will be wrong on the tail. Cost the ninety-fifth percentile, not the mean, or the first heavy user will blow the monthly envelope.
Cap the reasoning budget where the interface allows it. Some APIs expose a limit on reasoning effort or tokens. Setting it is a product decision: how much thinking is this task worth.
Meter per workflow, not per seat. A single expensive workflow can dominate a license that looks cheap per user. Attribution matters more than the headline price.
Evaluation becomes a sampling problem
A reasoning model is not uniformly better. It is better on problems where a wrong first step is expensive and a correct chain is available. On tasks that are mostly pattern matching, it is slower and more expensive for the same result.
That makes single-run benchmarks misleading. The right measurement is a distribution over attempts on your own tasks.
- Run each test case multiple times. A model that gets a hard case right one time in five is a different product from one that gets it right four times in five.
- Score the workflow, not the answer. If a human reviews every output, the model's accuracy is only half the cost. The review time is the other half.
- Include the failure cost. An error that requires rework, a customer explanation, or a correction has a price. Multiply it by the failure rate.
- Hold the test set. The cases you evaluate on should be cases the model has not been tuned on, and they should be owned by the program, not the vendor.
The metric that survives all of this is cost per accepted answer. Not cost per token, not benchmark accuracy. Price per attempt, times attempts, plus review, plus the cost of the errors that get through.
A reasoning trace is not an audit log
This is the part that gets oversold, and it matters most in regulated work.
A reasoning summary is generated text about the process. It is produced by the same model that produced the answer, and it can be incomplete, post hoc, or simply wrong about how the answer was reached. It is useful for debugging and for helping a reviewer follow the logic. It is not a record of what happened.
The record of what happened is the inputs, the model version, the parameters, the outputs, and the timestamp. If the workflow has tools, the record also includes which tool ran with which arguments and what came back. That is a system log, and it exists whether or not the model narrates itself.
The distinction has a practical edge. An auditor who asks "how did the system reach this decision" is asking for the record. A vendor who answers with a chain-of-thought transcript is answering a different question.
Where not to use one
Reasoning models are the wrong tool for a large share of enterprise work, and the share is worth naming.
- Extraction and classification. A deterministic task with a fixed output schema does not benefit from ten thousand tokens of deliberation. Use a smaller model, validate the schema, and spend the savings on review.
- High-volume, low-stakes generation. Drafting routine correspondence, summarizing a ticket, or tagging a document. Latency and cost dominate.
- Anything with a hard latency budget. A reasoning model is not an interactive component. If a user is waiting, the wait is part of the product.
- Tasks where the answer must be reproducible. If two runs must produce the same output for the same input, a system that reasons for a variable number of tokens is a poor fit unless you pin and cache.
The useful question is not which model is smartest. It is which model's failure mode you can afford.
A reasoning model fails slowly, expensively, and with a plausible explanation attached. A smaller model fails quickly and cheaply, and is easier to catch. Both are fine, in the right place.
The teams that get this right route work by consequence. Cheap models for the volume, reasoning models for the decisions that would be expensive to get wrong, and a record around all of it that does not depend on the model explaining itself.