What Companies budget AI by the token, but token counts can’t tell them whether the work got done or if it was any good.
How AI is budgeted.
AI is billed by the token, and most companies play a guessing game on how to budget for AI usage. This spring, Mavvrik and Benchmarkit asked 396 companies how they manage AI costs. Almost all of them track their AI spending and give AI a formal budget. Only 11% can forecast that spend within 10% of the eventual bill. Four in ten have had to take a surprise AI cost to their board, and one in four has delayed or canceled an AI project over cost.
The Wall Street Journal picked up that survey this week in “Spending on AI Is Becoming Almost Impossible for Businesses to Budget.” We’d say the budgets miss because they count the wrong thing. Companies measure AI by its output, in tokens, and almost nobody measures the outcome: whether the contract was reviewed correctly or the answer was accurate. The token count says how much text the model produced on the way to an answer. It doesn’t say whether the work got done or what it was worth to the organization. Tokens are a count of guesses, and we’ve never met a finance team that budgets for guessing on purpose.
What the research found.
A study published earlier this year by researchers at Stanford, Carnegie Mellon, UC Berkeley, and Microsoft Research measured what reasoning models, the kind that work through a problem step by step before answering (OpenAI’s GPT-5.4, Google’s Gemini 3.1 Pro, and Anthropic’s Claude Opus 4.7), cost to run.
Across eight models and 6,877 tasks, the lower-priced model cost more to finish the job in 32% of head-to-head comparisons. Gemini 3 Flash is priced about 80% below GPT-5.4 and cost 38% more across the tasks because it used far more tokens to get there.
The same model, given the same prompt again, didn’t cost the same amount either. Between the cheapest and most expensive run of a single question, cost spread as much as 9.7 times. The authors concluded that “for a fixed prompt and a fixed model, actual cost is intrinsically a random variable.”
The Journal took one task from the study, analyzing a frame of a YouTube video, and looked at how two Google models handled it. Gemini 3.1 Pro, the more expensive model, finished in 85 steps for about a dollar. Gemini 3 Flash kept working for 952 steps, trying one approach after another, until its input ran past the limit. It had used about $14 in tokens and never produced an answer. A token budget records those two runs as $1 and $14, with no column for the fact that one produced the work and the other produced an error message. This price range doesn’t seem like much, but imagine it at scale, with employees burning through tokens and producing no outcomes.
How a model decides when it’s finished.
A language model writes an answer one piece at a time, picking the most probable next piece given everything it has written so far. A reasoning model adds a loop: it writes out its thinking and then decides whether it has done enough or needs another pass. The process has no picture of the finished answer ahead of time, so it can’t judge how far away it is, and at each step it guesses again whether to keep going. This is how two runs of the same prompt can cost nearly ten times as much as each other, and how a model can spend 952 steps on a job it was never going to finish without noticing it isn’t getting anywhere. The bill shows the cost of each route, and nothing about the route itself or whether two routes reached the same answer.
Budgeting for the job.
The study and the Journal both recommend the same two things: match the model to the task, and watch what each request costs. Both steps help. Neither changes the unit, and we think the unit is the problem. A budget needs a different number: what a finished, defensible piece of work is worth, and what it took to get one.
The fix is to stop asking a language model to do the whole job. A language model is good at language and a poor tool for finding an answer in your records, because every step of the search is a guess. Use it for writing, and something else for finding and reasoning. Then you can see where an answer came from and judge whether it was any good.
Hybrid is the answer.
Franklin Discovery™ by entigenlogic® is a hybrid. A large language model handles the language, since that’s what those models do well. The finding and reasoning happen on a concept map entigenlogic has built over many years, where each idea is written down and connected to related ideas. Your documents are read into that map. When someone asks a question, Franklin Discovery follows the connections to the passages in your own documents that answer it, and shows you where the answer came from.
Because the finding happens on the map rather than by prediction, there is no open-ended loop deciding whether to keep going. Ask the same question twice, and you get the same answer, found by the same route. If the answer isn’t in your documents, Franklin Discovery tells you that, and it doesn’t spend more tokens trying to come up with one.
What to put in the budget.
As long as AI is measured by tokens, the bill will be hard to forecast, and so will the return. Measure it by the value created instead: the answers your people could file, cite, or act on without re-checking them, and the times the system said the answer wasn’t in the record. Before you buy, ask the vendor which parts of the job the language model is doing, and what the rest does.
Counting the job is the first correction. The second is deciding which jobs AI should do at all, and new research says most companies get that wrong too. That’s the second part of this piece.
See how Franklin Discovery works at franklindiscovery.ai.