FinOps for AI: how to stop runaway model costs

TL;DR: FinOps for AI ties model spend to business outcomes: visibility, allocation, optimisation, governance. Measure cost per outcome, not cost per API call. Stamp every request with a feature or customer ID so a spike traces back in minutes. Model routing and prompt caching are the two biggest levers. Instrument during the build, because retrofitting attribution after six months of unallocated invoices costs far more.

Most teams find their AI cost problem when the invoice lands. By then the data that would tell you where the money went is gone.

FinOps for AI is just the habit of tying model spend to business outcomes. Four parts: see it, allocate it, cut it, govern it. The one move that matters most is request-level tracking, done before your usage scales. Everything else sits on top of that.

Here is how it works in practice.

Why AI spend breaks normal cloud cost tracking

Traditional cloud costs are steady. A server costs what a server costs. You tag it, you watch it, you move on.

AI does not behave that way. One feature can call three models, hit a cache layer, and burn wildly different token counts depending on what the user typed. The bill moves with user behaviour, not with your infrastructure.

The FinOps Foundation treats AI as its own practice for that reason. Spend is granular, it crosses feature boundaries, and it does not map onto the resource tags that worked fine for servers.

Same four pillars, different mechanics:

• Visibility. Spend by model, token type and request. Not just by cloud account.
• Allocation. Which feature, team or customer caused it.
• Optimisation. Right model, cached repeats, right-sized infrastructure.
• Governance. Budgets, alerts and kill switches, set before an agent loop eats a month of budget in an afternoon.

One thing founders get backwards: inference costs more than training for most product teams. Training happens once. Inference runs on every single user action. Budget the one that scales with your users.

Measure cost per outcome, not cost per call

Cost per GPU hour tells you nothing useful. Cost per outcome tells you whether the feature is worth running at all.

Pick the unit first. If your AI summarises support tickets, the unit is cost per ticket summarised. Not cost per API call. The API call is a number. The ticket is the thing the business actually buys.

Then track four things:

• Cost per unit of business outcome
• GPU utilisation rate, because idle capacity is pure waste
• How fast you catch a cost spike
• How long from launch to the unit economics turning positive

Show finance a trend line, not a snapshot. One month's number starts an argument. Six months of a falling curve ends one.

What to instrument

Here is the usual mess. Three teams share one API key. The monthly invoice arrives as one number. Nobody can say which product surface caused it.

Static tags will not save you, because AI usage crosses feature and customer lines inside the same key.

1. Log the right fields on every call. Token counts, model name and version, latency, GPU hours, API key, request ID.
2. Stamp each request. Put a feature or customer ID on every call through a light routing proxy. The FinOps Foundation recommends this exact pattern, because shared keys make static tags useless.
3. Pull every provider bill into one schema. Normalise tokens, requests and dollars so you are comparing like with like.
4. Map the chain. API key to feature to customer. A spike should trace back in minutes, not days.

Do this during the build. Retrofitting attribution after six months of unallocated invoices costs far more, because the history you need to check your own numbers no longer exists.

For multi-tenant SaaS, match per-customer token counts against revenue tiers. Unprofitable segments show up early, while they are still small.

The tactics that cut the bill

Model routing is the big one. Send routine work, classification, short extraction, simple summaries, to a cheaper model. Save the expensive model for output where quality changes the business result.

• Caching. Cache repeated queries and system prompts. Anthropic documents prompt caching cutting costs by up to 90% on repeat requests. It is usually the fastest win available.
• Right-size the infrastructure. Match instance type to real load. Use spot instances for batch work that can wait.
• Should-cost benchmarks. Set an expected cost range per request type. Alert when reality drifts.

One warning. Do not set a single hard cap and call it governance. A blunt cap stops experiments as readily as it stops waste. A benchmark that flags drift is far more useful than a wall. Keep throttles as a safety net for anomalies, not as your main control.

Who owns it, and how often you look

This fails as a one-off audit. It works as a rhythm with named owners.

Four roles cover it. A FinOps lead owns the dashboards. A product owner owns each feature's unit economics. An engineering owner owns instrumentation and model choice. A finance sponsor turns the numbers into budget calls.

Then set the cadence:

• Automated cost alerts daily or weekly. Not a quarterly review that finds the problem three months late.
• A monthly session where product, engineering and finance read the trend together.
• Soft quotas that warn, hard kill switches for genuinely runaway processes.
• A written policy for agents specifically. An autonomous loop burns tokens far faster than a human clicking a button.

For agents, run hourly baselines. Day-level checks are too slow to catch something that spends a budget in minutes.

Crawl, walk, run

Do not try to build full maturity in one sprint.

1. Crawl. Get every provider bill in one place. Basic showback. Token counts per feature. Rough budgets, even if they are guesses.
2. Walk. Automate attribution down to the request. Add routing and caching on your highest-volume features. Move anomaly checks to hourly.
3. Run. Normalise billing across providers. Build cost-per-customer dashboards tied to revenue tiers. Automate the response, like throttling a feature that breaches its benchmark.

Do not skip crawl to chase run. Automated remediation on dirty attribution data just automates the wrong fix, because the dashboard is blaming the wrong team.

Most founders underestimate crawl. Getting multi-provider bills to line up, Azure OpenAI against Anthropic against Google, often eats weeks before one dashboard is worth trusting.

What we see on real builds

We have shipped over 200 apps, including work for NSW Government, Briometrix, Vivid and Huskee. AI runs through most of what we build now.

Two patterns come up again and again.

Billing transparency has to be asked for early. Provider invoices rarely split cleanly by feature. Teams that wait until spend spikes to ask lose months of attributable history they cannot get back.

Instrumentation is not a post-launch job. Put token logging and request IDs in the architecture from the first sprint. Then unit economics is a report you pull, not a project you start in a panic.

Where this connects

Cost control starts with knowing what the models charge. LLM API pricing compared and OpenAI API costs explained cover the raw numbers. What it costs to add AI features to your app covers the build side, and software infrastructure cost covers everything running underneath.

The tracking itself needs somewhere to land. A data warehouse for AI covers that, and AI platform architecture covers where the proxy layer sits.

Across the group, LLM cost optimisation covers the three biggest levers, Claude API pricing explained covers one provider in detail, and usage-based pricing in the AI era covers charging for it on the other side.

For the wider discipline, the FinOps Foundation's AI guidance and Microsoft Learn's cost management module are both worth the read.

Can you answer what your AI costs per customer?

If your AI feature is live and that question stumps you, that is the gap to close first. Everything else is guessing.

We build AI agents and AI-first platforms with tracking, routing and cost dashboards in the architecture, not bolted on later. Have a look at AI app development, or AI programs if you are turning a program into a platform. You can also see a vibe-coded MVP taken to a multi-tenant AI SaaS platform.

Worth a chat?

Frequently asked questions

What is FinOps for AI?

It applies the four FinOps pillars, visibility, allocation, optimisation and governance, to AI spend. AI costs behave differently to normal cloud bills because they are driven by tokens, model choice and request volume rather than steady infrastructure.

How is it different from normal cloud cost management?

AI spend crosses feature and customer boundaries inside a single shared API key. Static resource tags cannot split that, so you need request-level attribution instead.

What should founders measure first?

Cost per unit of business outcome, per ticket, per onboarding, per recommendation. Then GPU utilisation and how fast you catch a spike. Add time to positive unit economics once the baseline is stable.

How much does prompt caching save?

Anthropic documents cost reductions of up to 90% on repeat requests with prompt caching. It is usually the fastest win available, before you touch model routing or infrastructure sizing.

Who should own AI cost governance?

Four roles: a FinOps lead on the dashboards, a product owner on unit economics, an engineering owner on instrumentation and model choice, and a finance sponsor who turns the numbers into budget decisions.