Model Fine-Tuning

What fine-tuning is

The mental model that avoids most disappointment: prompting is instruction, retrieval is memory, fine-tuning is habit. If your problem is that the model has not been told something, fix the prompt. If the problem is that it does not know something specific about your world, fix retrieval. If the problem is that it has been told correctly, repeatedly, and still drifts back to a generic behaviour, that is habit, and habit is what training changes.

Terms used on this page

Supervised fine-tuning

Training on pairs of inputs and the outputs you want, so the model learns to produce that style of response. It is the form most teams mean when they say fine-tuning.

Parameter-efficient fine-tuning

Training a small set of added weights, commonly low-rank adapters, instead of updating the whole model. Cheaper, faster, and easy to swap or roll back because the base model is untouched.

Catastrophic forgetting

The loss of general capability caused by training hard on a narrow dataset. A model tuned only on successful cases can lose the ability to refuse, hedge or handle anything off-distribution.

Leakage

The same underlying case appearing in both training and holdout data, usually through near-duplicate rows. It inflates your score and hides the failure until production finds it.

Distillation

Using a large model to generate training data for a smaller one, so the small model learns to imitate the large one on your specific task at a fraction of the running cost.

What fine-tuning cannot fix

This section exists because most fine-tuning enquiries we receive describe a problem that fine-tuning does not solve. Reading the table below saves people a month.

Who it is for, and who it is not for

Fine-tuning suits a team with one narrow, high-volume, repeatable task, where a written evaluation already shows the base model failing in a consistent way, and where enough correct examples exist or can be produced.

• High volume on a single task, so a shorter prompt and a smaller model repay the work quickly.
• A behaviour you can demonstrate on a holdout set, not a feeling that outputs are a bit generic.
• At least a few hundred correct examples, ideally drawn from work your team has already done and reviewed.
• Somebody who can adjudicate labels, because a dataset with two labellers who disagree teaches the model to be inconsistent.

Who should not buy this

• Anyone who has not written an evaluation set. Without a holdout score for the base model, there is no way to prove a tuned model is better, and every judgement collapses into taste.
• Anyone whose task changes monthly. Each change means new data, a training run and requalification, and the treadmill costs more than the prompt you were avoiding.
• Anyone with fewer than roughly a hundred clean examples, unless a larger model can generate them and a human will review every one.
• Anyone hoping to make a small model as capable as a large one in general. Distillation transfers a narrow task, not broad ability.

What we actually build

Four things, and only one of them is a model. The pipeline is the deliverable, because you will train again.

1. A dataset pipeline: extraction from your systems, normalisation, deduplication, labelling workflow, provenance on every row, and a split that cannot leak.
2. An evaluation harness that scores the base model and every candidate on the same holdout, per behaviour class rather than as a single average.
3. The training configuration itself, version controlled, with hyperparameters, seed and data snapshot recorded so any run can be reproduced.
4. A serving path with the tuned model behind a flag, so traffic can be split, compared and reverted without a deployment.

The evaluation harness is the part that outlives the model. It is what tells you next quarter whether a new base model has made your tuned variant redundant, which happens more often than vendors like to mention. See Evaluation and Guardrails.

How it works technically

Training adjusts weights so your examples become more likely. Everything interesting is in what you train, how much you change, and how you prove it worked.

Full fine-tuning against adapters

Full fine-tuning updates every weight and produces a complete new model, which is expensive to train, expensive to store and awkward to roll back. Parameter-efficient methods train a small number of added weights, typically low-rank adapters, and leave the base untouched. Adapters are the default for nearly all business tasks: cheaper, faster to iterate, and swappable, so you can hold three variants and route between them while comparing.

The hyperparameters that actually matter

Learning rate and epochs, in that order. Too high or too many and the model memorises your examples and loses its general ability, which shows up as brittle behaviour on anything phrased unusually. Too low and nothing changes and you conclude fine-tuning does not work. Start conservative, evaluate after each epoch, and keep the checkpoint that scores best on holdout rather than the last one, because the last one is frequently overfitted.

Serving and rollback

The tuned model goes behind the same interface as the base, selected by configuration. Route a small share of live traffic to it, compare outputs against the base on identical inputs, and keep both running until the comparison is boring. Rollback is a flag flip. If rolling back requires a deployment, the design is wrong.

The gate we apply before accepting the work

We turn down more fine-tuning briefs than we accept, and this is the test. All five conditions must hold. If one fails, the work below it is cheaper and usually sufficient.

The dataset is the project

Model quality tracks dataset quality almost exactly, so the artefacts below are the ones we treat as the real deliverable. Here is the training format, with the two conventions that catch teams out: the system prompt must be identical to production, and every row carries provenance.

# One training example per line. Keep the system prompt IDENTICAL to the one
# you will use in production, or you are training for a world that never ships.

{"messages":[
  {"role":"system","content":"You triage support tickets for a payments platform. Reply with one label and one sentence of reason."},
  {"role":"user","content":"Card declined at checkout, code 51, customer says funds are available."},
  {"role":"assistant","content":"issuer_decline. Code 51 is an issuer-side insufficient-funds response, so the merchant integration is not at fault."}
],"meta":{"source_ticket":"T-88213","labelled_by":"rmk","split":"train","behaviour":"declines"}}

{"messages":[
  {"role":"system","content":"You triage support tickets for a payments platform. Reply with one label and one sentence of reason."},
  {"role":"user","content":"Can you tell me what our monthly volume was in March?"},
  {"role":"assistant","content":"out_of_scope. Triage does not have access to account reporting, so this routes to the account team."}
],"meta":{"source_ticket":"T-88907","labelled_by":"rmk","split":"holdout","behaviour":"refusals"}}

Then the rules the dataset is built under. Splitting by row rather than by underlying case is the single commonest cause of a model that scores well and behaves badly, because near-duplicate tickets land on both sides of the split and the holdout is quietly measuring memorisation. See Building a Golden Dataset from Real Work.

{
  "dataset": "ticket_triage_v4",
  "min_examples_per_behaviour": 50,
  "behaviours": ["declines", "chargebacks", "integration_errors",
                 "refusals", "out_of_scope"],

  "hygiene": {
    "dedupe": "normalise whitespace and casing, hash the user turn, drop exact
               and near duplicates above 0.95 similarity",
    "split_by": "source_ticket customer_id, NEVER by row",
    "leakage_check": "no customer_id may appear in both train and holdout;
                      build fails if any does",
    "holdout_fraction": 0.2,
    "label_provenance": "every example records who labelled it and when"
  },

  "balance": {
    "refusal_examples_min_fraction": 0.1,
    "note": "a dataset of only successes teaches the model to never decline,
             which is how a tuned model loses the ability to say no"
  },

  "gates_before_training": [
    "base model scored on the same holdout, result recorded",
    "two labellers agree on a 30-example sample above 0.9",
    "no example longer than the production context budget",
    "system prompt byte-identical to production"
  ]
}

The build process, stage by stage

What you get at handover

The dataset is yours and is not reused for other clients or for any other model. Where a provider hosts the tuned model, we document exactly what that means for your data under their current terms, and you sign it off before any training begins.

Where fine-tuning projects go wrong

What it costs to run once it is live

Fine-tuning is usually justified by running cost rather than by quality, so the arithmetic decides it. The saving comes from a shorter prompt, a smaller model, or both, and it has to repay the training run plus the requalification you now owe forever.

Work it through with stated assumptions you should replace. Suppose 2,000 requests a day. Before tuning, the prompt carries instructions and eight examples, about 2,500 input tokens, on a larger model at a stand-in 3 per million input tokens, which is 0.0075 a request. After tuning, the examples live in the weights, so the prompt drops to about 400 tokens on a smaller tuned model at a stand-in 0.60 per million, which is 0.00024 a request. The saving is roughly 0.0073 a request, about 15 a day and 435 across thirty days. If the training run plus engineering time costs 300, it repays in around three weeks at that volume.

Halve the volume and the payback doubles. Drop to 200 requests a day and it takes most of a year, at which point maintenance costs more than the saving and the correct decision is a longer prompt. This is the whole reason the gate exists.

One cost the estimator cannot hold: every base model release starts a requalification. Budget a day of engineering per release to rerun the harness and decide whether the tuned variant is still ahead of a plain prompt on the current base. Sometimes it is not, and retiring a tuned model is a good outcome rather than a failure.

How to tell whether you need this

Run the five conditions. If all five hold, fine-tuning is probably the right call and the arithmetic will confirm it. If any fail, the cheaper work below is the honest recommendation.

• If there is no evaluation set, build one first. It is a week of work, and it frequently solves the problem by revealing that the failures are three separate issues.
• If the failures are factual, build retrieval instead, and keep the facts where you can edit them.
• If the prompt was never properly tested, try constrained output and a handful of well-chosen examples before touching weights.
• If you hold fewer than a hundred examples, collect for a month with the current system, capturing corrections, then reconsider.
• If the volume is low, accept the longer prompt. It is cheaper than owning a model.

How to start

1. Send twenty failures30 minutes

   Twenty real cases where the current output is wrong, with what you wanted instead. Failures tell us in an afternoon what a specification cannot tell us in a fortnight.

2. Gate call60 minutes

   We walk the five conditions against your case together, and read a sample of the failures to decide whether they are behavioural or factual.

3. Written verdict and priceAbout a week

   Either a fine-tuning plan with the dataset work, the evaluation design, the timeline and the break-even arithmetic, or a written recommendation to do something cheaper and what that is.

4. Baseline before anything elseWeek 0

   The holdout set is built and the base model is scored on it. Every later claim is measured against that number, including ours.