Migrating Off Brittle RPA Bots

The short answer

That reframing changes who has to be in the room. Most RPA bots exist because somebody wanted data out of a system and was told no, or quoted a licence price, or given a six month integration queue. The bot was the workaround. Removing it therefore requires the same conversation with the vendor or the internal platform team that was avoided the first time, and that is a procurement and negotiation problem sitting inside what looks like an engineering project.

None of this argues that RPA was a mistake. It bought real capability at a time when the alternative was a project nobody would fund. The problem is that the maintenance cost compounds while the value stays flat, and the bill arrives as a slow drip of small incidents rather than an outage anybody escalates. RPA replacement work is mostly about turning that drip into a decision.

Why these bots break, mechanically

Bots break because they are coupled to a presentation layer that nobody promised to keep stable. Every item below is a real, recurring cause, and none of them is a defect in the bot.

• Selector drift. A vendor release renames a field, adds a wrapper element, or changes a generated identifier. The bot was targeting a path through the document that was never part of any contract.
• Rendering differences. Image based automation depends on resolution, scaling, theme, font rendering and window size. Move the bot to a different host or let Windows change a display setting and matching fails.
• Unexpected modals. A cookie banner, a survey prompt, a mandatory password change or a release notes dialog appears once and the bot is now clicking on the wrong thing entirely.
• Session and authentication. Idle timeouts, forced re-authentication, certificate changes and any move to multi-factor authentication break unattended runs. This is why so many bots run under a shared account with a static password, which is its own problem.
• Locale and format. Date order, decimal separators and thousands separators change with regional settings. A bot that reads 03/04 correctly for a year can silently reverse day and month after a host rebuild.
• Timing assumptions. Fixed waits are tuned to the speed of the environment on the day they were written. A slower morning turns a working step into an intermittent failure that nobody can reproduce.
• Silent success. The worst class. The bot navigates, finds no rows because a filter defaulted differently, completes its loop over zero items, and reports success. Nothing is logged, nothing alerts, and the data is missing for as long as it takes a human to notice.

The rule that decides whether a migration is real

Every process you touch sits on one of five rungs, and the only migrations worth doing move it downward. Rebuilding a bot in a different tool at the same rung feels like progress, produces a demo, and leaves you with identical fragility on a newer licence.

Applied honestly, the rung rule kills a lot of proposals in their first week, which is the point. If the only available move for a given process is rung 2 to rung 2, then leave the bot alone, fix its logging so that failures are loud, and spend the budget on a process where a real descent is possible.

What a language model actually changes here

A model earns its place at the unstructured edges of a process, which is exactly where classic RPA has always been weakest. It does not make interface automation reliable, and treating it as a smarter clicker is the most expensive mistake available in this whole category.

The practical shape is a deterministic workflow with one or two model shaped holes in it. Fetch the document, extract fields against a schema with a confidence value and an explicit abstain path, validate against business rules in code, then post through an API. Everything the model returns is checked before it is used, as in getting structured output from LLMs, and the document reading half is ordinary document processing work rather than anything experimental.

Inventory and triage before you touch anything

Start by counting what you actually have, from orchestrator history rather than from a list somebody maintains. Estates of any age contain bots that nobody has claimed for years, bots that duplicate each other, and bots that run every night and produce a file nobody opens any more.

# bot-inventory.yaml
# One record per bot. Fill it from orchestrator history, not from memory.
# The score sets the order. Argue with the score, not with the room.

- id: BOT-014
  name: "Invoice posting into the ERP"
  business_owner: "AP team lead"        # a named person, never a department
  technical_owner: null                 # null here is a finding, not a blank field

  runs_per_week: 210
  minutes_saved_per_run: 4
  failures_last_90d: 37                 # straight from the orchestrator log
  mean_minutes_to_fix: 25
  silent_failure_seen: true             # has it ever reported success and posted nothing?

  blast_radius: financial               # none | internal | customer | financial | regulated
  runs_unattended: true
  credentials: "shared service account, password rotated by hand"
  host: "VM-FIN-03, owner unknown"      # if this line is uncomfortable, that is the point

  rung:                                 # see The Rung Rule
    current: 2                          # 1 pixels  2 DOM  3 internal endpoint
    target: 4                           # 4 public API  5 vendor data contract
    blocker: "API is on a paid tier. Procurement problem, not an engineering one."

  unstructured_edge:                    # what the bot handles with regex today
    present: true
    description: "supplier invoice PDFs, 40+ layouts, 3 languages"

# TRIAGE SCORE, computed per bot, highest first:
#
#   value_hours   = runs_per_week * minutes_saved_per_run * 52 / 60
#   pain_hours    = failures_last_90d * mean_minutes_to_fix * 4 / 60
#   score         = value_hours + (pain_hours * 2)
#                 + 10 if (rung.target - rung.current) >= 2 else 0
#                 + 15 if silent_failure_seen else 0
#                 - 20 if rung.blocker is a procurement blocker else 0
#
# Pain counts double because a bot that breaks weekly is already costing you an
# engineer. Silent failure scores highest of all: a bot that fails loudly wastes
# an hour, and a bot that fails quietly corrupts a ledger for six weeks.

One deliberate omission from that scorecard: how much the original bot cost to build. It is spent, it should carry no weight in the decision, and it is the argument that keeps failing automations alive long past their usefulness.

Prove parity before you switch anything off

Run the old bot and the new pipeline against the same inputs for several weeks and compare their outputs field by field. Nobody should be asked whether the replacement looks correct. The parity harness answers that, and its output is what lets you turn the bot off without a meeting.

1. Capture real inputsweek 1

   Record what the bot actually receives, including the malformed cases it currently fails on. Synthetic test data will not contain the supplier who sends a scanned photograph of an invoice, and that supplier is why the exception path exists.

2. Run both, write to different placesweeks 2 to 5

   The new pipeline runs on every input in shadow and writes to its own store. The bot keeps doing the real work. Neither knows about the other, and nothing downstream changes.

3. Compare field by field, not by totalscontinuous

   Totals hide compensating errors. Compare per record, per field, with a per field tolerance where a tolerance is genuinely appropriate. Log every mismatch with both values and the input that produced it.

4. Triage every mismatch into three bucketsthe real work

   The new system is wrong, the bot was wrong, or the rule was never defined. The third bucket is the interesting one and is usually the largest. Each item there needs a decision from the business, written down, before cutover rather than after.

5. Promote by branch, not all at onceweeks 6 onward

   Route the highest confidence case type to the new pipeline while everything else stays with the bot. Widen one branch at a time. Keep the bot runnable, scheduled off but not deleted, until a full cycle has passed with the new path handling everything.

Measure the outcome against the baseline you took before starting, not against how the new system feels. Throughput, exception rate, cost per item and end to end time all belong in the same table for both systems, using the approach in measuring automation impact.

The arithmetic, on your own numbers

The case for migrating is usually maintenance and licence cost rather than new capability, and both are easy to underestimate because they arrive as small pieces spread across a year. Put your own figures in and treat the result as a floor, since it excludes the cost of the errors nobody caught.

Two adjustments make that number honest. Subtract only the licence you will genuinely stop paying, because a partial migration usually leaves the platform in place for the bots you did not touch, and the saving arrives in the year the last one is retired. Then add the running cost of the replacement: hosting, model calls, monitoring and the hours somebody will spend on changes in year one. A migration that only pays back if you count the whole licence and none of the new running cost has not been costed, it has been argued for.

Decommissioning, the part everybody forgets

A bot is not retired when the new pipeline goes live. It is retired when its schedule, its host, its credentials and its licence seat are all gone, and until then it is a live system with write access to production that nobody is watching.

The bot that runs in parallel and posts twice

The most common incident in this whole category. The new pipeline goes live, the bot is left scheduled because turning it off felt risky, and both post the same invoices for a week. Disable the schedule in the same change that promotes the pipeline, then verify from the orchestrator that no run occurred, rather than trusting that the checkbox saved.

The shared service account

The bot logs in as a named human or a shared account whose password has been in a runbook for four years and is known to people who have left. Retirement is your opportunity to fix that properly with a machine identity and scoped permissions. Do it while the migration has attention and budget, because the moment the project closes this becomes nobody's job again.

The orphan host

Estates accumulate virtual machines running scheduled tasks that predate the current team. Before decommissioning, list what else runs on that host. Somebody's quarterly report is on it, undocumented, and it will fail in eleven weeks with no obvious connection to your project.

Definitions

Terms used precisely on this page

Robotic process automation (RPA)

Software that performs a business process by driving the same user interfaces a person would use, through recorded clicks, element selectors or image matching, rather than through a programmatic interface designed for machines.

Selector drift

The gradual failure of interface automation as the target application's markup changes across releases. It occurs because the automation depends on a structure the vendor never committed to keeping stable, so no notice is given and no error is raised until the run fails.

Silent failure

A run that reports success while producing no effect, typically because a filter, a login redirect or an empty result set left the automation with nothing to process. It is the most damaging RPA failure mode because it appears nowhere in failure statistics.

Parity harness

A test setup in which an existing automation and its replacement process the same real inputs in parallel, and their outputs are compared field by field. Its results, rather than a demonstration or an opinion, decide whether cutover may proceed.

Unattended run

An automation that executes on a schedule with no person present, usually under a service account. Unattended execution is what makes authentication changes, unexpected dialogs and silent failures materially more dangerous than in a supervised run.