Total Cost of Ownership for an AI System Over Three Years

What TCO for an AI system actually includes

The useful part of a TCO exercise is not the total. It is discovering which cost lines are driven by your own volume and which are driven by other people's release schedules. Those two groups behave completely differently, and a model that mixes them produces a number that looks precise and forecasts nothing.

The four clocks that drive every AI bill

Sort your cost lines into these four before you add anything up. Each clock ticks for a different reason and at a different rate, and the sorting exercise itself finds the lines nobody has budgeted, which is almost always everything on clocks three and four.

Why year two is the expensive one

Year one hides the drift clock inside the build budget. The team that built the system is still around, so a schema change is absorbed as a bug fix and nobody records it as a cost. In year two that team has moved on, there is no build budget, and the drift clock is billing against a support line that was sized against the run clock. That gap is the most predictable failure in the whole exercise.

Model deprecation is a scheduled cost, not an accident

Providers retire model versions. That is normal platform operation rather than a betrayal, and it is the most predictable drift event you face. What it costs you depends almost entirely on one thing: whether you can tell if the replacement is worse. Everything below exists to make that question answerable in days rather than months.

1. Pin the exact model version everywhereday one

   Never call a floating alias from production. Pin the full version string in config and record which version produced which output in your logs. Without this, a silent upgrade shows up as a mysterious change in quality and you have nothing to compare it against.

2. Keep a golden dataset that reflects real trafficongoing

   A few hundred real cases with accepted outputs, refreshed as the input distribution moves. This is the asset that turns a migration from a research project into a measurement. How to assemble one without poisoning it is in building a golden dataset.

3. Shadow run the candidate before you need tothe week the notice lands

   Send a copy of live traffic to the new version, store both outputs, and compare on your own metrics. Nothing is switched and no user is affected. The comparison itself is the deliverable, and it is what turns a deadline into a schedule.

4. Budget prompt and threshold work, not a string changedays per system

   A replacement model responds differently to the same instructions. Confidence thresholds, retry rules, output parsing and any few shot examples may all need re-tuning. Budget days per system rather than days per company, because the tuning does not transfer between systems even on the same model.

5. Migrate behind a flag with a fast path backone to two weeks

   Route a small percentage of traffic, watch the acceptance rate and the retry rate rather than a smoke test, then increase slowly. Keep the old version reachable until the deprecation date arrives, not until you feel confident.

6. Record the delta before you close the ticketan hour

   Write down what changed in cost, latency and acceptance. That record is what makes the next migration an estimate instead of a guess, and there will be a next one within the year.

What the first three years actually look like

Run your own three year number

Fill in your own figures. The point of this model is not the total, it is the two ratios it exposes: how much of the three year cost is the build, and how much of it is driven by things outside your control. Both change the kind of decision you are making.

Two readings matter. A low build share means you are approving an operating commitment rather than a project, and it should be reviewed the way operating commitments are reviewed. Then stress test the drift line, because it is the only input here that your own volume does not predict. Double it. If the case does not survive, the honest conclusion is that the system needs fewer moving parts, not that the estimate needs to be lower. The per unit build figures that feed the first line are worked through in what AI automation costs to build, and the token component in token cost arithmetic.

Own the stack or rent the platform

This choice moves cost between the four clocks rather than removing it. Renting shortens the build clock and hands the drift clock to somebody else, which is genuinely valuable right up until the vendor changes something you had come to depend on. Owning gives you control of drift and hands you the bill for it.

One caution about portability. An abstraction layer that has never been run against a second provider is a design intention rather than portability, and it usually hides at least one assumption that only holds for the provider you built against. Run the second provider once, on a subset of traffic, and you will find out which. The deeper lock-in is rarely the API anyway. It is the accumulated prompt tuning and the evaluation data, so keep both in your own repository whichever route you take.

The exit cost nobody funds

Every system is eventually switched off, replaced or absorbed. The exit bill is small if a handful of decisions went the right way early, and large if they did not. None of these are expensive to get right during the build. All of them are expensive to fix at the end.

• A usable export. Decide at design time what an export looks like, then actually run one during the build. An export nobody has executed is a hypothesis about a script that does not exist yet.
• Retention that outlives the system. If records must be kept for years, the archive is a live cost after the system is dead, and somebody has to be able to read the format in year six.
• In-flight work. There is always a queue mid-process on the day you switch over. Someone has to decide whether it is drained, replayed or abandoned, and that decision is cheaper made in advance.
• Credentials and integrations. Every service account, webhook and API key you created becomes an unowned entry point in someone else's system the moment yours goes away.
• Prompts and evaluation data. If they live only in a vendor console, your accumulated tuning leaves with the vendor and the next build starts from zero.
• The knowledge. If the runbook is in one person's head, the exit cost includes rediscovering how the thing worked, usually from logs, usually under time pressure.

The cheapest way to lower an exit bill is a small export drill run during the build, while somebody still cares and the person who wrote the schema is still on the project. It takes a day. It also doubles as a disaster recovery test, which is usually easier to get funded. Who holds this responsibility after handover is its own question, covered in who owns automation after launch.

A system card that keeps the number honest

A TCO estimate decays the moment it is approved, because every input in it is a fact about a moving system. The fix is a small file that lives beside the code, gets reviewed quarterly, and holds the facts the estimate depends on. Copy this, fill it in for one system, and you will find at least two unknowns you did not know you had.

# system-card.yaml, one per production AI system, reviewed every quarter.
# If a field is unknown, write "unknown". An unknown is a finding, not a gap to hide.

system: invoice-extract
owner:
  named: "one person, not a team"          # a team is not an owner
  backup: "a second named person"
  days_allocated_per_month: 2              # if this is zero, the drift clock is unfunded
unit_of_work: "one supplier invoice posted to the ledger"
criticality: "money moves, so a wrong output is recoverable but escalates"

models:
  - role: extract
    provider: "your provider"
    version_pin: "the exact version string, never a floating alias"
    pinned_on: "2026-04-02"
    deprecation_notice: "none received"    # fill in the date the moment one arrives
    fallback_version: "the version you would migrate to, decided in advance"
prices:
  source: "config/pricing.yaml"
  checked_on: "2026-08-19"                 # a stale date here invalidates every cost figure

prompts:
  location: "repo path, under version control"   # never only inside a vendor console
  change_process: "pull request, evaluation suite must pass before merge"

evaluation:
  golden_dataset: "repo path, and the number of cases in it"
  refreshed_on: "2026-07-15"
  runs: "on merge, and nightly"
  metrics: ["field level exact match", "escalation rate", "cost per accepted outcome"]
  last_run: "your own figures and the date they were produced"

drift_register:                            # everything you depend on and do not control
  - watch: "source system schema"
    owner: "finance systems"
    last_change: "2026-05-11"
  - watch: "provider model deprecations"
    owner: "same as the system owner"
    checked: "monthly"
  - watch: "supplier document layouts"
    owner: "accounts payable lead"
    checked: "quarterly"

exit:
  export_format: "one JSON record per unit, plus the source document"
  export_drill_run_on: "2026-06-20"        # run one during the build, not at the end
  retention_obligation: "seven years, in the archive rather than in the system"
  decommission_days_estimated: 20

Three fields carry most of the value. The allocated days per month is the honest signal about whether the drift clock is funded, and a zero there is a finding rather than an omission. The export drill date is either a date or an admission that nobody has tested the exit. And the pinned date sitting next to the deprecation notice tells you at a glance whether the next migration is a scheduled job or an ambush.

Terms, and the review that keeps them true

Definitions worth agreeing on with finance

Total cost of ownership

The sum of building a system, running it, keeping it working as its dependencies change, and eventually switching it off. A figure covering only build and run is a build estimate wearing a longer name, and it will be wrong in the same direction every time.

Drift

Any change outside your control that makes a working system work less well: a deprecated model version, an altered source schema, a new document layout, a reorganised process, new regulatory guidance. Drift is scheduled by other people, so it must be funded as a standing allocation rather than handled as a series of incidents.

Golden dataset

A curated set of real inputs paired with accepted outputs, used to measure whether a change makes the system better or worse. It is the asset that converts a forced model migration from an open ended research project into a measurement with an end date.

Run rate

The recurring cost of operating the system per unit of work, including model tokens, paid tool calls, infrastructure and human review minutes. Run rate only falls per unit as volume grows if somebody has been given time to tune it, which is a decision rather than a trend.

Decommission cost

The one off cost of switching a system off properly: exporting records in a usable form, honouring retention obligations, clearing in-flight work, and revoking every credential and integration the system created. It is determined by design decisions made years earlier.