Fine-tuning cost: the training bill is the small one

TL;DR: Fine-tuning has two bills. Training is a one-off: tokens times epochs times the rate. Serving repeats monthly, with an hourly hosting fee that runs even when the model is idle. Run a small pilot, model the serving cost early, and check you need fine-tuning at all.

Most teams price the training run and stop there. That's the small bill. The model then has to be hosted and served every month, and that bill never ends.

Here's the short version. Fine-tuning cost has two parts. Training is a one-off. Serving is monthly, and it's often the bigger line once the model is live.

How training is billed

The maths is simple for supervised fine-tuning. Microsoft Foundry's cost guide gives the formula: training tokens, times epochs, times the training price per token.

So the bill scales in a straight line. Double the tokens and the bill doubles. Double the epochs and it doubles again.

Three things catch people out:

• Token counts. Use the model's own tokeniser to count them. A word-count guess is rough.
• Training tier. Rates differ between regional, global and developer training. Developer training uses spare capacity, so a job can take longer.
• Run time. Bigger models, bigger data sets and slower GPUs all push the bill up.

Run a small pilot first. It gives you real token counts and real run time, and that removes most of the guesswork.

How serving is billed

Once training ends, you pay to run the model. The Foundry guide lists the main options:

The key point is the hourly fee. A fine-tuned model on a standard deployment has a hosting charge that runs whether or not anyone sends a request. So an idle model still costs money.

Foundry's worked example is a fine-tuned chatbot that handles 10,000 conversations in a 30-day month. It uses symbolic rates, which is the honest way to do it, because prices change:

• Hosting: 720 hours at the hourly rate (H)
• Input: 20 million tokens at the input rate (I)
• Output: 40 million tokens at the output rate (O)
• Monthly total: 720H + 20I + 40O

Plug in your vendor's current rates and you've got a monthly figure. Check the hosting term. At low traffic it can be the biggest part of the bill. For wider price comparisons, see our guides to LLM API pricing and OpenAI API costs.

What drives the bill

• Model size. Bigger models need more compute per token.
• Data set size and epochs. Cost grows with both.
• GPU type. A profiling study of LLM fine-tuning models cost from model size, data set size and GPU type, and tests A40, A100 and H100 cards. A faster card costs more per hour but can finish sooner. Compare total run cost, not the hourly rate.
• Output length. Output tokens usually cost more than input tokens, so a chatty assistant costs more to serve than a classifier.
• Traffic pattern. A back-office tool can run on a light deployment. A customer-facing assistant needs capacity at peak.

How to estimate your own job

1. Count the tokens in your cleaned data set.
2. Pick the number of epochs.
3. Look up the current training and serving rates.
4. Multiply to get the training cost.
5. Add monthly hosting hours, plus expected input and output tokens.

Do it twice. Scenario A is a pilot: a small model, a modest data set, a few epochs. Scenario B is production: more data, more epochs, a bigger model and always-on hosting. In B, serving usually becomes the larger line, so traffic matters more than the training run.

Ways to bring it down

• Start smaller. Use the smallest model that hits your accuracy target.
• Use parameter-efficient methods. LoRA and QLoRA train a small set of extra weights, so they need far less GPU memory. One QLoRA tutorial puts a 70B model at roughly 28 to 32 GB of GPU memory on a single card.
• Delete idle deployments. Don't pay hosting for a model nobody calls.
• Reserve capacity only for peaks.
• Cache prompts and batch requests where your setup allows it. Our post on token cost optimisation covers this.

API or your own GPUs?

With a vendor API, you pay for training, hosting and per-token use. Setup is easy and billing is predictable. You keep paying per token for as long as the model runs.

With your own GPUs, you pay by the hour, or buy the hardware. At steady high volume that can cost less per token. But you take on engineering, monitoring and upgrades, and those people costs never show up on a price list.

At low or uneven traffic, an API usually wins, because you're not paying for idle machines. As Harvard Business Review notes, leaders need to budget for integration and ongoing operations as well as the model itself. We cover the build side in custom AI software versus off-the-shelf.

Retraining adds a recurring cost too. Every time your data changes, you pay again, and you need to track each model version. Our AI model versioning guide shows how.

Do you need fine-tuning at all?

Often, no. Better prompts or retrieval can reach the same accuracy for far less. Our post on fine-tuning vs RAG walks through the choice. The group has more on what AI model fine-tuning is and cutting LLM costs.

If fine-tuning is the right call, here's the order I'd work in:

1. Run a small pilot to get real token counts and run time.
2. Model the monthly serving cost, not just the training invoice.
3. Judge it by cost per successful task, not cost per hour.

Treat every price you read, including any in a guide like this, as a starting point. Rates change, and your data isn't anyone else's.

Want a budget you can trust?

We stage the work: discovery, then a pilot, then production. Discovery counts your tokens and reviews your data. The pilot runs one short training job and a small deployment. Production adds monitoring and cost alerts. We've shipped apps for NSW Government, and that experience shapes how we plan spend before a job runs.

Ask for a pilot budget and we'll send back an estimate. Start with our AI platform development or AI app development pages, or get in touch.

Frequently asked questions

How is fine-tuning billed?

For supervised fine-tuning, you pay for training tokens times epochs at the training rate. After training, serving adds an hourly hosting fee on many deployment types, plus input and output token charges.

Do fine-tuned models cost money when nobody uses them?

Often yes. Standard fine-tuned deployments carry an hourly hosting fee, and provisioned capacity bills by the hour. Delete deployments you aren't using.

How can I lower my fine-tuning bill?

Use a smaller model, try LoRA or QLoRA, delete idle deployments and reserve capacity only for peak traffic. First check whether better prompts or retrieval already do the job.

Is the training run the biggest cost?

Usually not for long. Training is a one-off. Serving, testing and retraining repeat every month after launch, so a cheap pilot can hide an expensive product.