What not to automate, and how to tell in advance

The short answer

This is the guide people skip, and skipping it is how automation programmes acquire a reputation for not working. A project that should never have started still consumes a build, a rollout, a period of quiet distrust and eventually a removal, and each of those costs more than the assessment that would have stopped it. The five checks below are cheap, and three of them can be answered from records you already hold.

The five hard disqualifiers

Notice that none of these are about the technology being capable. Model capability is rarely the binding constraint on this kind of work, and treating it as the only question is why so many projects pass a technical evaluation and fail in production. The constraints that bind are stability, verifiability, detectability, arithmetic and legibility of the inputs, and every one of them is a property of your organisation rather than of the model.

Judgement is not the disqualifier, unverifiable judgement is

The received wisdom says automate the routine and leave judgement to people. It is too blunt to be useful, because plenty of judgement-heavy work automates well with a reviewer, and plenty of routine work does not automate at all because nobody can tell when it went wrong. The sharper question is what it costs to verify one output.

• Cheap to verify. A competent person can look at the result and call it right or wrong in seconds, usually because the answer is checkable against something visible. This class automates well even when the work looks like judgement, because the review is fast and the reviewer stays accountable.
• Expensive to verify. Checking the output requires redoing most of the work: reading the whole document, opening the other system, reconstructing the reasoning. Here automation moves the work rather than removing it, and it often adds a step, because now somebody re-derives an answer they did not produce.
• Impossible to verify. There is no fact of the matter within any useful timeframe. Forecasts, prioritisation calls and relationship judgements sit here. These can be assisted with better information, but they should not be handed over, because you will never learn whether the system is any good.

The Silent Error Test

The value of running this before the build is that it changes the design rather than the verdict. A process that fails on latency and blast radius does not become unautomatable; it becomes a system with a confirmation step, a daily cap and an aggregate check, which costs a little more to build and considerably less to live with. Skipping the test does not avoid that cost, it defers it to the incident that forces the retrofit.

The worst quadrant is the one that looks most attractive

Rank candidate processes on three axes: volume, variance and error tolerance. High volume with low variance and forgiving errors is the sweet spot, and everyone knows it. The trap is the combination of high volume, high variance and low error tolerance, because the volume is what gets the project funded and the other two are what defeat it.

1. Volume attracts the funding. A large number of units makes the business case arithmetic look decisive, and it is the number that appears in the slide that gets approval.
2. Variance defeats the build. A wide exception spectrum, which you only see properly once you have mapped the process, means the automated path covers a smaller share than anyone assumed, and each additional variant costs roughly the same to handle as the first few did.
3. Low error tolerance defeats the economics. When every output must be checked, review cost scales with volume, so the same size that made the case attractive now makes the residual work large.

Does the arithmetic even close?

Run this before the enthusiasm has anywhere to go. The figure that surprises people is not the build cost, it is the fixed overhead that every automated system carries regardless of size, and small processes cannot absorb it.

The fixed overhead is the part left out of most business cases, and it is roughly constant whether the system handles fifty units a month or five thousand. That is why tiny automations have poor unit economics even when the build looks trivial.

• Credentials and access that expire, rotate and break, each time needing someone who understands the integration.
• Monitoring and alerting, plus the person who receives the alert and knows what to do with it.
• Dependency drift. Providers change models, deprecate endpoints and adjust behaviour, so a system that is finished still needs periodic attention.
• Documentation and handover, without which the system becomes unmaintainable the moment its author changes team.
• The review queue itself, which is real recurring work and belongs in the running cost rather than being treated as free.

A useful rule of thumb: if a candidate process cannot carry that overhead on its own, it is only viable as one more flow inside a system you already run and already monitor. Judging a small automation on its own build cost is how organisations end up with a dozen unowned scripts, which is a worse position than the manual process they replaced.

Automate the draft, not the decision

Most processes that fail the tests above still contain a large amount of work that automates cleanly. What disqualifies them is the final decision, and the final decision is usually a small share of the time. Splitting the two is the single most useful move in this guide.

The autonomy ladder, and how to climb it

1. Retrieve. The system finds and presents the relevant information. Nearly always safe, and it removes more time than people expect, because gathering is often the bulk of the work.
2. Draft. The system proposes an output that a person edits and sends. Promote to this rung once retrieval has been running without complaints and the edit rate is measured.
3. Act with confirmation. The system takes the action after an explicit approval click, with the reasoning attached. Promote when edits become rare and the ones that remain are cosmetic rather than substantive.
4. Act autonomously, within a bounded scope. Only for a defined class of unit, with a cap on how many it may process before a human check, and a stated rule for demotion. Promote on evidence from the rung below, never on a demonstration.

Write the promotion criteria down at the start, with the numbers that would justify each step and the conditions that would send it back down a rung. Teams that do not write them either stay stuck on the bottom rung forever, because nobody wants to own the decision to move, or they jump to the top after a good demo. The mechanics of making that review queue survive a busy team are covered in human in the loop design.

Some processes are symptoms, not workloads

A whole category of work exists only because something upstream is broken. People copy data between two systems that do not talk to each other, chase a field that should have been mandatory on a form, or reconcile two records that should never have diverged. Automating this class of work is the most expensive mistake in the guide, because it makes the underlying defect permanent and gives it a defender.

• The verbs give it away. Chasing, fixing, reconciling, copying, re-keying, cleaning up. Processes described that way are repairs, and repairs point at something that needs repairing.
• The one-field test. Ask whether the process would disappear if a single upstream field were mandatory, validated or shared. A surprising share of internal work fails this test, and the fix is usually a form change and a conversation.
• The migration scar. Processes created shortly after a system change and never revisited. They exist to bridge a gap that was supposed to be temporary, and everyone involved has forgotten that.
• The political tell. The upstream fix belongs to a different team with different priorities, which is precisely why an automation is being proposed instead. Say this out loud in the meeting. It is often the real reason the project is on the table, and it deserves an explicit decision rather than a quiet one.

None of this means the automation is always wrong. Sometimes the upstream system genuinely cannot be changed this year, and a bridge is the right engineering answer. When that happens, build the bridge with an end date and a written note of the fix it stands in for, and review it when the upstream change lands. What you should refuse to do is treat a repair as a permanent process and optimise it into the architecture, because the next person will assume it was always meant to be there.

The go or no-go rubric, and the definitions behind it

Score every candidate before a build is approved. The vetoes are the five disqualifiers, and the scored gates are the things that vary by degree. Copy this into the repository next to the process map so the reasoning survives the person who did the assessment.

# automation-go-no-go.yaml
# Score every gate before a build is approved, using counted values rather than opinions.
# Any veto set to true ends the discussion, whatever the scored total says.

vetoes:
  process_changed_materially_in_last_six_months: false
  two_experts_disagree_on_more_than_one_case_in_five: false
  a_wrong_output_would_go_unnoticed_for_a_full_cycle: false
  decisive_inputs_exist_only_in_someones_head: false
  action_is_irreversible_and_nobody_reviews_it: false

scored_gates:              # 0 to 3 each, 18 available
  volume:
    question: "completed units per month, counted from a system"
    value: 800
    score: 3
  variance:
    question: "share of units covered by the top three variants"
    value: 0.82
    score: 2
  verifiability:
    question: "seconds a competent reviewer needs to judge one output"
    value: 20
    score: 3
  reversibility:
    question: "cost and elapsed time to undo one wrong action"
    value: "same day, no effect outside the company"
    score: 3
  input_legibility:
    question: "share of decision inputs that are machine readable"
    value: 0.9
    score: 3
  ownership:
    question: "named person who owns the system after launch"
    value: "ops lead, written into their objectives"
    score: 2

thresholds:
  minimum_total: 14
  minimum_any_single_gate: 2      # one gate at 0 or 1 forces the draft pattern instead
  decision: "build, with a review queue for the tail"

if_below_threshold:
  - "narrow the scope to the highest volume variant and score it again"
  - "fix the upstream cause instead, when the process exists to repair another system"
  - "automate retrieval and drafting only, and leave the decision with a person"
  - "do nothing, and write down what would have to change for the answer to differ"

Terms used in the rubric

Silent error

A wrong output that produces no failure signal anywhere in the system and looks identical to a correct one until a human happens to look. Silent errors are what make detection lag, rather than accuracy, the governing risk in automated decisions.

Verification cost

The time a competent person needs to judge whether one automated output is correct. When verification costs about as much as producing the output manually, automation relocates the work instead of removing it.

Blast radius

The number of units affected by a single fault before anyone notices. A per-unit fault is an incident; a fault in a shared rule applies uniformly to everything processed since it changed.

Autonomy ladder

The four rungs an automated system can occupy: retrieve, draft, act with confirmation, and act autonomously within a bounded scope. Promotion between rungs should follow measured evidence from the rung below rather than a demonstration.

Symptom process

Work that exists only to repair a defect somewhere upstream, such as copying between systems that do not integrate. Automating a symptom process makes the underlying defect permanent and gives it an institutional defender.

Most teams find that running this rubric across a list of candidates changes the order more than it changes the count. Two or three candidates fail outright, several move to the draft pattern, and one that nobody was excited about turns out to be the best first project because it is high volume, low variance and instantly checkable. Choosing between what survives is the subject of picking the first process to automate, and a structured version of this whole assessment is what an AI readiness audit produces.