AI at work · Costs
The cost of AI: why bills rise and how to keep them under control
Published8 min read
The price per token falls while the bill rises. What do the bills at Uber, Amazon and Salesforce tell us? Here is why agents use so many tokens and how to set limits before a workflow costs more than it delivers.
The article in 30 seconds
- Without context management, each step sends the agent everything it has already read. Cost grows faster than the number of steps.
- Caching, model choice, smaller contexts and batch processing can lower the bill. Stopping a loop takes a limit enforced outside the model, such as a maximum step count, repeated failures or a budget.
- Instead of counting tokens, measure the cost of work that passes review, including review time.

“The budget I thought we needed is already blown.” In April 2026, Uber’s CTO described what AI coding tools had done to his annual budget. [1] It took four months to use up the full year’s allocation.
Uber had done what many companies wanted: encouraged adoption. An internal leaderboard compared teams by usage. Claude Code use rose from about one-third of engineers in February to 84% in March. Average spending was $150 to $250 per engineer per month. The heaviest users reached $2,000. [2]
A few weeks later, the COO asked whether these figures translated into more useful features. His answer was that the link had yet to be established. [2]
Usage could be measured by the minute. Value was much harder to measure.
What the 2026 bills tell us
Uber was far from alone, but large bills tell different stories. Three cases illustrate a budget overrun, a poor incentive and deliberate spending.
| Case | What was reported | The response |
|---|---|---|
| Uber: budget overrun | Annual AI coding tool budget exhausted in four months. [1] | A cap of $1,500 per employee per tool each month, with a dashboard and exceptions available. [3] |
| Amazon: the wrong incentive | Employees reportedly ran the internal tool on pointless tasks to climb the leaderboard. [4] | The leaderboard was removed. [5] |
| Salesforce: deliberate spending | About $300 million in Anthropic tokens planned for 2026, mainly for coding. [6] | The CEO called for a routing layer that reserves the most capable models for requests that justify them. [6] |
Microsoft is also frequently cited. Press reports say it withdrew most direct Claude Code licences, which it had opened to thousands of employees six months earlier, in favour of GitHub Copilot CLI. [7] Some engineers reportedly spent $500 to $2,000 a month. [8] Microsoft did not comment to Fortune. [7] The stated reason reported in the press was to unify tooling. The role of cost remains a press interpretation. [8] Uber’s budget total is also undisclosed.
High spending can be justified. Meta’s CTO defended heavy use by an engineer he considered five to ten times more productive. [4] At Databricks, a manager said in June that engineers’ budgets remained unlimited. [5]
Prices are falling and usage is rising
Industry estimates put GPT-4-equivalent performance at $0.40 per million tokens, down from $20 in late 2022. Jellyfish observed an 18.6-fold increase in usage per developer over nine months. Its heaviest users were about twice as productive as its lightest users while consuming ten times as many tokens. [9] The question is what each additional token delivers.
Amazon’s case also illustrates a familiar rule: reward consumption and you get more consumption. Token counts tell you how much was spent. They do not tell you what was accomplished.
Without context management, the agent pays again for what it has already read
A question to an assistant takes one round trip. An agent reads, calls a tool, receives the result and repeats the cycle. By default, tool results stay in the conversation and count towards every subsequent turn. [10] Caching and context management, discussed below, reduce this effect.
↶ Context sent back to the model for the next step
Lowering cost does not stop a loop. Enforce a limit before each call.
Take a fictional example at public Claude Sonnet 5.5 rates: $2 per million input tokens and $10 per million output tokens. [11] The agent starts with 10,000 tokens. Each step produces 500 output tokens and adds 3,000 tokens to the context, including the response. Nothing is cached or removed.
Under these assumptions, ten times as many steps cost about 76 times as much. That factor comes from letting the context accumulate without caching or trimming. It shows why a growing loop should be measured and then limited.
A published Anthropic test illustrates the effect. An agent comparing eight documents reached a context of 335,279 tokens in five turns. Files it read accounted for 96%. [10]
Two costs are easy to miss. Tool definitions are billed : Claude’s browsing toolkit adds about 6,600 tokens to each request in its default configuration. Token counts can also change. Claude 4.7 and later models use a tokenizer that produces about 30% more tokens for the same text, depending on its content. [11] Comparing models on the price per token alone is insufficient.
What actually reduces the bill
The Anthropic prices below were checked on 9 November 2026. They illustrate mechanisms available elsewhere under different terms. The measurements come from one published test by that provider.
| Method | Benefit | What to watch |
|---|---|---|
| Cache the start of the request | Cache reads cost 5% of the input price on Sonnet 5.5 and Opus 5.5, and 10% on most other models. [11] | A five-minute cache write costs 1.25 times the input price. If nothing is reused before expiry, caching costs more. |
| Process in batches | 50% off input and output, on top of caching discounts. [11] | Only suitable for work that can wait. |
| Choose the model to suit the task | Haiku 5.5 input costs one-fortieth of Opus 5.5 up to 100,000 tokens. [11] | Correcting an error can cost much more than the saving. |
| Clear old tool results | The test’s peak dropped from 335,279 to 173,137 tokens. [10] | The agent may have to fetch results again. Clearing them invalidates the cache from the changed content onwards. |
| Compact the conversation | A peak of 169,164 tokens with a summary of about 2,800 tokens. [10] | The three major facts tested survived. Three appendix details were lost. |
| Keep notes across sessions | Second session: four reads instead of eight, with a peak of 172,623 instead of 333,977 tokens. [10] | The benefit depends on the quality of the notes. |
In the fictional 20-step example, a cache reused on every turn reduces the cost from $1.64 to $0.34, a saving of 79%. This assumes the start of the request stays the same and each turn arrives before the cache expires.
Routing is attracting attention, including from Salesforce’s CEO. [6] Tools for routing are coming onto the market. [5] A cheaper model can still get stuck in a loop : routing lowers the cost per call while leaving the number of unnecessary calls unchanged.
Set limits on the workflow before optimising the model
The cheapest token is one you never send. Before tuning the cache, I would look at how the work is organised.
- Skip the model when a rule is enough. If the steps are known, a workflow can do the job. Reserve agents for cases in which each document determines what to check next.
- Give the assignment a clear endpoint. “Analyse the whole file” has no natural limit. “Extract these five fields from these three documents” does.
- Send the excerpt. An agent reviewing a contract does not need the entire inbox. Every unnecessary tool also adds to each request.
- Limit retries. When a tool fails twice in the same way, report the failure instead of making an identical third attempt.
- Plan how to stop. When the limit is reached, have the agent report what it found and what is missing.
An alert and a cap serve different purposes
An alert signals that a threshold has been reached. A cap prevents the next call. The same distinction applies to instructions for an assistant that reads email : enforce the control outside the model.
I would set three limits: steps per assignment, budget per task and a cap per person or department. Uber combines a monthly cap, a dashboard employees can see and exceptions that require approval. [3] Microsoft and Databricks now offer gateways for tracking and limiting this spending. [5]
Two other steps matter. Assign a budget owner and compare the provider’s bill with your own measurements. Priceline reportedly found differences between its provider’s figures and internal data. [9] A cap set too low can also suppress the gains that justified the tool. [5] Set it based on observed use.
The right measure: cost per accepted result
A cheaper model may cost more overall. Take 100 cases and two options. The figures are fictional and show what you would need to measure.
A case is accepted when the reviewer approves it against criteria set before the trial and applied to both options. Each case takes two minutes to review. A correction takes 12 minutes. At €45 an hour, that is €1.50 per review and €9 per correction.
| Cost item | Lower-priced option | Higher-priced option |
|---|---|---|
| AI cost per case | €0.40 | €1.20 |
| Cases needing correction | 40 | 5 |
| Total AI cost | €40 | €120 |
| Review of all 100 cases | €150 | €150 |
| Correction cost | €360 | €45 |
| Total cost for 100 accepted cases | €550 | €315 |
| Cost per accepted case | €5.50 | €3.15 |
In this example, at equal quality, the option that costs three times as much per call ends up about 43% cheaper per accepted case. Review costs the same on both sides, so it does not change the €235 difference, but it is part of the full cost. The result depends mostly on how many cases need reworking and how long corrections take. The reverse may hold for simple tasks. A trial will tell you.
The FinOps Foundation takes a similar approach. It suggests tracking tokens that actually contributed to a business action, excluding retries, abandoned sessions and inadequate outputs. It also says these metrics are meaningful only alongside a measure of value, and that focusing on cost alone can create false savings. [12]
What I check in my own setup, and the first trial
In my setup, I first ask what triggers a charge. Shared memory calls a billed model when an assistant requests that something be saved, within the limits I authorised. Background consolidation is off. Search runs on my server and incurs no per-call model charge. The server, maintenance and my time still cost something, and a more thorough search takes more computation. The dedicated API key has its own cap.
I have no measured results to publish yet. My personal setup also tells us little about the cost of a service for 50 people.
The first trial: twenty closed cases
At a company, I would start with twenty closed cases before rolling anything out. I would rerun the intended workflow and record:
These measures give you a cost per accepted case and a basis for setting caps. I would ask what heavy use produced before deciding whether to restrict it.
Before looking at the price of a token, I would ask: how much does it cost to get work I can actually accept?
Sources and references
- The Information — Uber’s CTO and the Claude Code budget ↩
- Techstrong.ai — Uber’s AI spending and uncertain returns ↩
- TechCrunch — Uber caps employee AI spending ↩
- Fortune — Amazon, Meta and token leaderboards ↩
- The Next Web — companies cap AI usage ↩
- Business Insider — Salesforce and Anthropic tokens ↩
- Fortune — Microsoft and the cost of tokens ↩
- The Next Web — Microsoft’s withdrawal of Claude Code ↩
- The Next Web — token prices and corporate AI bills ↩
- Anthropic — memory, compaction and clearing tool results ↩
- Anthropic — API prices, checked 9 November 2026 ↩
- FinOps Foundation — Token Economics ↩