Build, Buy or Wait: Choosing Your AI Approach

The question is not build or buy, it is which layer you own

Teams that debate build against buy as a single choice tend to get the split exactly backwards. They build the chat interface, which is a commodity with a hundred adequate implementations, and they rent the evaluation, which means the vendor decides what good looks like for their business. Two years later they own a user interface anyone could have bought and none of the labelled data that would let them move.

There is a simple test for every layer. Ask what it would cost to move off it in twelve months, in weeks of engineering and in data you could not take with you. Where that answer is small, rent without much agonising. Where it is large, either own the layer or negotiate the exit before you sign, because the moment to secure your data export is while a salesperson still wants your signature.

The Ownership Line

Draw one line through the stack. Below it you rent and you accept somebody else's roadmap. Above it you own, you version, and you carry the maintenance. The line sits in the same place for almost every organisation, which is directly under the layer where your business judgement lives.

Layer by layer, with the tell that you crossed the line

Build and buy fail in different directions

Compare them on how each one goes wrong, not on the brochure. Both options work. They fail differently, and the failure you can absorb is the one to choose.

The pattern worth copying is a hybrid that is deliberate rather than accidental. Buy the fastest path to the first working version, and on day one keep your golden cases, your labels and your written rules in your own repository in an open format. If the bought tool holds, you have saved months. If it does not, you keep the expensive part and replace the cheap part.

Wait is a real option, and it has exactly three conditions

Waiting is legitimate when all three of these hold at once. If any one fails, waiting is procrastination with a business case attached to it.

1. The blocker is capability, not integration. Your use case fails today because the technology cannot do it yet, not because your data is unreachable or your approval chain is slow. Integration blockers do not dissolve on their own, so waiting on one wastes the whole period.
2. The workaround cost is bounded and known. You can state what not doing it costs per month. An unbounded or unmeasured cost means you are not choosing to wait, you are avoiding a decision, and the difference shows up in the budget conversation twelve months later.
3. You are not forgoing a data advantage. If doing the work now would produce labelled examples, recorded outcomes or process rules that compound, waiting has a hidden price. The examples you do not collect this year do not appear retroactively next year.

The three-year arithmetic, including the parts left out of the pitch

Run this with your own rates and your own quoted prices. Two lines matter more than the totals: the maintenance share, which is real and which build estimates habitually omit, and the crossover month, which tells you how long the bought option stays cheaper.

Two things this arithmetic cannot express, and you should say both out loud in the meeting. The bought option starts producing value in a fortnight while the built one produces nothing for two months, and that gap has a real value that belongs on the buy side. Against it, the built option accumulates a labelled dataset and a set of written rules that the bought option keeps in its own schema. Put a number on the second by asking what you would pay to already hold your golden set on the morning your vendor changes its pricing. A fuller version of this model, including the people cost of running a bought tool, is in total cost of ownership for an AI system.

Score every vendor on the way out, not on the way in

Vendor evaluations mostly test the demo, which every serious vendor passes. The questions that predict regret are all about leaving. Run this during the trial, while you still have a salesperson motivated to get you answers, and actually perform the export rather than accepting that one exists.

# Score BEFORE signing. Every question here is about leaving, because the
# joining experience is what the vendor optimised and it tells you little.

vendor: <name>
layer: model | orchestration | retrieval | application | connector
reviewed_by: <name>
date: YYYY-MM-DD

data_out:
  export_raw_inputs: yes | no
  export_outputs_with_version_stamps: yes | no
  export_labels_and_human_corrections: yes | no   # the usual no
  format: open | proprietary
  export_actually_performed_during_trial: yes | no

logic_and_prompts:
  prompts_visible_and_exportable: yes | no
  business_rules_live_where: our repository | their console
  can_a_change_be_reviewed_as_a_diff: yes | no

evaluation:
  can_we_run_our_own_golden_set_against_it: yes | no
  changelog_for_model_or_prompt_changes: yes | no
  notice_before_an_underlying_model_changes_days: 0

access_and_audit:
  scoped_service_accounts: yes | no
  audit_log_export: yes | no
  data_residency_options: [list them]
  subprocessors_published: yes | no

commercial:
  price_metric: seat | run | token | outcome
  quote_at_2x_volume: get it in writing now
  price_change_notice_days: 0
  termination_notice_days: 0

switching_cost:
  weeks_to_replace: 0
  data_we_could_not_take: describe it honestly
  integrations_to_rebuild: 0

verdict: rent | rent with a written exit plan | do not rent this layer

A two-week bake-off instead of a six-month debate

Most build against buy arguments are unresolvable in a meeting room and trivially resolvable with two weeks of evidence. Run both options against the same cases and let the numbers arbitrate.

1. Fix the golden set first, before either option is touched1 day

   Take fifty real cases with agreed correct answers, stratified across case types rather than drawn at random. This set belongs to you permanently and is the reason the bake-off produces a decision instead of two demos and a preference.

2. Configure the bought option to its realistic best3 days

   Use the trial, ask their solutions engineer for help, and record every configuration you make. You are measuring the product as a competent buyer would run it, not as an unhelped first-time user would. Note anything that needed their intervention, because that is future dependency.

3. Build the thinnest possible version yourself4 days

   One model call, retrieval if the task needs it, no interface beyond a script, no queue. Four days is enough to learn whether the hard part is the model or your data, which is the single most useful thing the fortnight can teach you.

4. Score both against the same set, with the same scorer1 day

   Report per case type rather than one aggregate. The interesting outcome is not which option scores higher overall, it is that they usually fail on different case types, which tells you where the remaining work sits either way.

5. Write the decision as a layer table, not a verdict1 day

   The output is a row per layer with rent or own beside it, an exit cost for every rented layer, and a review date. That document survives a change of vendor and a change of staff, which a single build or buy verdict does not.

When the decision lands on building the layer above the line, that is the work described under custom LLM applications: software with a model inside it, wired into your systems, with the evaluation harness as a deliverable rather than an afterthought.

Definitions for the procurement conversation

Definitions

Exit cost

What it would take to stop using a component, measured in weeks of engineering to replace it plus the data you could not take with you. It is the only vendor metric that reliably predicts regret, and it should be estimated before signing rather than discovered during a migration.

Lock-in

Not the impossibility of leaving, but the cost of leaving. A vendor with an open export and visible prompts can be left in a fortnight. One that holds your labelled corrections in a schema it will not export has locked in the expensive layer, whatever the contract says about termination.

Commodity layer

A part of the stack where several providers are close substitutes and switching takes days. Model hosting, vector storage and queueing are commodity layers. Building your own version of one is defensible only when a measured constraint such as latency, cost or data residency cannot be met by any of them.

Evaluation harness

The golden cases, the scoring functions and the runner that grades any version of a system against the same bar. It is the asset that makes vendors and models interchangeable, and it is the first thing to build regardless of which side of the build or buy decision you end up on.