Skip to content
← AI articles

AI at work · Costs

The cost of AI: why bills rise and how to keep them under control

8 min read

The price per token falls while the bill rises. What do the bills at Uber, Amazon and Salesforce tell us? Here is why agents use so many tokens and how to set limits before a workflow costs more than it delivers.

The article in 30 seconds

  • Without context management, each step sends the agent everything it has already read. Cost grows faster than the number of steps.
  • Caching, model choice, smaller contexts and batch processing can lower the bill. Stopping a loop takes a limit enforced outside the model, such as a maximum step count, repeated failures or a budget.
  • Instead of counting tokens, measure the cost of work that passes review, including review time.
A scale on a grainy green background. Stacks of coins sit in the raised pan, while a crouching Tesla Optimus robot weighs down the lower pan in high-contrast black and white.
AI-generated illustration.

“The budget I thought we needed is already blown.” In April 2026, Uber’s CTO described what AI coding tools had done to his annual budget. [1] It took four months to use up the full year’s allocation.

Uber had done what many companies wanted: encouraged adoption. An internal leaderboard compared teams by usage. Claude Code use rose from about one-third of engineers in February to 84% in March. Average spending was $150 to $250 per engineer per month. The heaviest users reached $2,000. [2]

A few weeks later, the COO asked whether these figures translated into more useful features. His answer was that the link had yet to be established. [2]

Usage could be measured by the minute. Value was much harder to measure.

What the 2026 bills tell us

Uber was far from alone, but large bills tell different stories. Three cases illustrate a budget overrun, a poor incentive and deliberate spending.

Three companies, three situations
CaseWhat was reportedThe response
Uber: budget overrunAnnual AI coding tool budget exhausted in four months. [1]A cap of $1,500 per employee per tool each month, with a dashboard and exceptions available. [3]
Amazon: the wrong incentiveEmployees reportedly ran the internal tool on pointless tasks to climb the leaderboard. [4]The leaderboard was removed. [5]
Salesforce: deliberate spendingAbout $300 million in Anthropic tokens planned for 2026, mainly for coding. [6]The CEO called for a routing layer that reserves the most capable models for requests that justify them. [6]

Microsoft is also frequently cited. Press reports say it withdrew most direct Claude Code licences, which it had opened to thousands of employees six months earlier, in favour of GitHub Copilot CLI. [7] Some engineers reportedly spent $500 to $2,000 a month. [8] Microsoft did not comment to Fortune. [7] The stated reason reported in the press was to unify tooling. The role of cost remains a press interpretation. [8] Uber’s budget total is also undisclosed.

High spending can be justified. Meta’s CTO defended heavy use by an engineer he considered five to ten times more productive. [4] At Databricks, a manager said in June that engineers’ budgets remained unlimited. [5]

Prices are falling and usage is rising

Industry estimates put GPT-4-equivalent performance at $0.40 per million tokens, down from $20 in late 2022. Jellyfish observed an 18.6-fold increase in usage per developer over nine months. Its heaviest users were about twice as productive as its lightest users while consuming ten times as many tokens. [9] The question is what each additional token delivers.

Amazon’s case also illustrates a familiar rule: reward consumption and you get more consumption. Token counts tell you how much was spent. They do not tell you what was accomplished.

Without context management, the agent pays again for what it has already read

A question to an assistant takes one round trip. An agent reads, calls a tool, receives the result and repeats the cycle. By default, tool results stay in the conversation and count towards every subsequent turn. [10] Caching and context management, discussed below, reduce this effect.

Without caching or trimming, every step resends what has already been read

↶ Context sent back to the model for the next step

Stable prefixInstructions and tool definitions
CacheReread unchanged content at a lower price
HistoryMessages and reasoning
CompactSummarise what was decided
Tool resultsFiles, pages and API responses
Clear or trimRemove content that can be retrieved again
Model responseText and the next tool call
Set a capCheck steps, failures and budget before the call

Lowering cost does not stop a loop. Enforce a limit before each call.

Take a fictional example at public Claude Sonnet 5.5 rates: $2 per million input tokens and $10 per million output tokens. [11] The agent starts with 10,000 tokens. Each step produces 500 output tokens and adds 3,000 tokens to the context, including the response. Nothing is cached or removed.

20 steps$1.64770,000 input tokens
200 steps$124.4061.7 million input tokens

Under these assumptions, ten times as many steps cost about 76 times as much. That factor comes from letting the context accumulate without caching or trimming. It shows why a growing loop should be measured and then limited.

A published Anthropic test illustrates the effect. An agent comparing eight documents reached a context of 335,279 tokens in five turns. Files it read accounted for 96%. [10]

Two costs are easy to miss. Tool definitions are billed : Claude’s browsing toolkit adds about 6,600 tokens to each request in its default configuration. Token counts can also change. Claude 4.7 and later models use a tokenizer that produces about 30% more tokens for the same text, depending on its content. [11] Comparing models on the price per token alone is insufficient.

What actually reduces the bill

The Anthropic prices below were checked on 9 November 2026. They illustrate mechanisms available elsewhere under different terms. The measurements come from one published test by that provider.

Six ways to cut costs and their limits
MethodBenefitWhat to watch
Cache the start of the requestCache reads cost 5% of the input price on Sonnet 5.5 and Opus 5.5, and 10% on most other models. [11]A five-minute cache write costs 1.25 times the input price. If nothing is reused before expiry, caching costs more.
Process in batches50% off input and output, on top of caching discounts. [11]Only suitable for work that can wait.
Choose the model to suit the taskHaiku 5.5 input costs one-fortieth of Opus 5.5 up to 100,000 tokens. [11]Correcting an error can cost much more than the saving.
Clear old tool resultsThe test’s peak dropped from 335,279 to 173,137 tokens. [10]The agent may have to fetch results again. Clearing them invalidates the cache from the changed content onwards.
Compact the conversationA peak of 169,164 tokens with a summary of about 2,800 tokens. [10]The three major facts tested survived. Three appendix details were lost.
Keep notes across sessionsSecond session: four reads instead of eight, with a peak of 172,623 instead of 333,977 tokens. [10]The benefit depends on the quality of the notes.

In the fictional 20-step example, a cache reused on every turn reduces the cost from $1.64 to $0.34, a saving of 79%. This assumes the start of the request stays the same and each turn arrives before the cache expires.

Routing is attracting attention, including from Salesforce’s CEO. [6] Tools for routing are coming onto the market. [5] A cheaper model can still get stuck in a loop : routing lowers the cost per call while leaving the number of unnecessary calls unchanged.

Set limits on the workflow before optimising the model

The cheapest token is one you never send. Before tuning the cache, I would look at how the work is organised.

  • Skip the model when a rule is enough. If the steps are known, a workflow can do the job. Reserve agents for cases in which each document determines what to check next.
  • Give the assignment a clear endpoint. “Analyse the whole file” has no natural limit. “Extract these five fields from these three documents” does.
  • Send the excerpt. An agent reviewing a contract does not need the entire inbox. Every unnecessary tool also adds to each request.
  • Limit retries. When a tool fails twice in the same way, report the failure instead of making an identical third attempt.
  • Plan how to stop. When the limit is reached, have the agent report what it found and what is missing.

An alert and a cap serve different purposes

An alert signals that a threshold has been reached. A cap prevents the next call. The same distinction applies to instructions for an assistant that reads email : enforce the control outside the model.

I would set three limits: steps per assignment, budget per task and a cap per person or department. Uber combines a monthly cap, a dashboard employees can see and exceptions that require approval. [3] Microsoft and Databricks now offer gateways for tracking and limiting this spending. [5]

Two other steps matter. Assign a budget owner and compare the provider’s bill with your own measurements. Priceline reportedly found differences between its provider’s figures and internal data. [9] A cap set too low can also suppress the gains that justified the tool. [5] Set it based on observed use.

The right measure: cost per accepted result

A cheaper model may cost more overall. Take 100 cases and two options. The figures are fictional and show what you would need to measure.

A case is accepted when the reviewer approves it against criteria set before the trial and applied to both options. Each case takes two minutes to review. A correction takes 12 minutes. At €45 an hour, that is €1.50 per review and €9 per correction.

Fictional example · 100 cases accepted after correction
Cost itemLower-priced optionHigher-priced option
AI cost per case€0.40€1.20
Cases needing correction405
Total AI cost€40€120
Review of all 100 cases€150€150
Correction cost€360€45
Total cost for 100 accepted cases€550€315
Cost per accepted case€5.50€3.15

In this example, at equal quality, the option that costs three times as much per call ends up about 43% cheaper per accepted case. Review costs the same on both sides, so it does not change the €235 difference, but it is part of the full cost. The result depends mostly on how many cases need reworking and how long corrections take. The reverse may hold for simple tasks. A trial will tell you.

The FinOps Foundation takes a similar approach. It suggests tracking tokens that actually contributed to a business action, excluding retries, abandoned sessions and inadequate outputs. It also says these metrics are meaningful only alongside a measure of value, and that focusing on cost alone can create false savings. [12]

What I check in my own setup, and the first trial

In my setup, I first ask what triggers a charge. Shared memory calls a billed model when an assistant requests that something be saved, within the limits I authorised. Background consolidation is off. Search runs on my server and incurs no per-call model charge. The server, maintenance and my time still cost something, and a more thorough search takes more computation. The dedicated API key has its own cap.

I have no measured results to publish yet. My personal setup also tells us little about the cost of a service for 50 people.

The first trial: twenty closed cases

At a company, I would start with twenty closed cases before rolling anything out. I would rerun the intended workflow and record:

These measures give you a cost per accepted case and a basis for setting caps. I would ask what heavy use produced before deciding whether to restrict it.

Before looking at the price of a token, I would ask: how much does it cost to get work I can actually accept?

Sources and references

  1. The Information — Uber’s CTO and the Claude Code budget ↩
  2. Techstrong.ai — Uber’s AI spending and uncertain returns ↩
  3. TechCrunch — Uber caps employee AI spending ↩
  4. Fortune — Amazon, Meta and token leaderboards ↩
  5. The Next Web — companies cap AI usage ↩
  6. Business Insider — Salesforce and Anthropic tokens ↩
  7. Fortune — Microsoft and the cost of tokens ↩
  8. The Next Web — Microsoft’s withdrawal of Claude Code ↩
  9. The Next Web — token prices and corporate AI bills ↩
  10. Anthropic — memory, compaction and clearing tool results ↩
  11. Anthropic — API prices, checked 9 November 2026 ↩
  12. FinOps Foundation — Token Economics ↩