Making an agent's decisions auditable

The short answer

The trigger is rarely a regulator. It is a customer disputing a refund, a finance lead asking why a batch went out wrong, or an engineer asking whether Tuesday's prompt change caused the complaints. All three ask the same thing: what did the system see, and why did it act.

What auditable actually means

Auditable is not a property of a system. It is a list of questions you can answer without guessing, so write your own list and check that each row has an artifact behind it.

Two rows tend to be missing entirely. Nobody records where each argument came from, so the invented-number question is unanswerable, and nobody records subject ids, so an erasure request means a full text scan or a promise you cannot keep.

The Six Pins

A run is reconstructable when six things are pinned at the moment of the decision. Miss one and the record becomes a story about the past rather than a copy of it.

The decision record

One record per decision that has an effect, not one per model call. Chat turns are telemetry. Decisions are what an auditor, a customer or a court asks about, and they deserve their own durable object with its own retention class.

{
  "record_version": "2",
  "run_id": "run_2b91f7",
  "step_id": "step_04",
  "occurred_at": "2026-08-26T09:14:37.221Z",
  "logical_time": "2026-08-26T09:14:00Z",

  "actor": { "type": "agent", "name": "refund-agent", "deployment": "prod-eu-1" },
  "subject_ids": ["cust_1099"],

  "pins": {
    "prompt":  { "template": "refund/v14", "rendered_sha256": "b41c..." },
    "model":   { "provider": "acme-ai", "id_requested": "model-x",
                 "id_returned": "model-x-2026-07-19", "params": { "temperature": 0, "top_p": 1 } },
    "tools":   [ { "name": "issue_refund", "version": "3.2.0", "code_sha": "9d21f7c",
                   "schema_sha256": "0af3..." } ],
    "policy":  { "id": "refunds", "version": "2026-03" },
    "evidence":[ { "id": "doc_7a1f", "sha256": "9c1e...", "as_of": "2026-03-02T00:00:00Z" },
                 { "id": "orders/88213", "sha256": "77bd...", "as_of": "2026-08-26T08:02:11Z" } ]
  },

  "decision": {
    "action": "issue_refund",
    "args": { "invoice_id": "INV-88213", "amount": 420.00, "currency": "GBP" },
    "args_source": {
      "invoice_id": "evidence:orders/88213",
      "amount":     "policy:refunds#4.2",
      "currency":   "evidence:orders/88213"
    }
  },

  "controls": {
    "gate": "human_approval",
    "human_actor": "u_442",
    "decided_at": "2026-08-26T09:16:02Z",
    "decision": "approve_with_edit",
    "edited_fields": ["amount"]
  },

  "outcome": {
    "status": "applied",
    "external_ref": "pmt_re_88213_01",
    "idempotency_key": "run_2b91f7:issue_refund:5f2a..."
  },

  "model_narrative": {
    "text": "Policy 4.2 applies because delivery exceeded seven days.",
    "label": "generated after the action, treat as a hint, not as evidence"
  },

  "retention": { "class": "financial_7y", "pii_fields": ["subject_ids"], "key_id": "k_cust_1099" }
}

• args_source is the field people leave out and later need most. For every argument, say where it came from: an evidence id, a policy clause, a human edit, or model generation. An amount tagged model_generated in a financial action is either a bug or a policy decision, and either way you want to know before the customer does.
• subject_ids belong on every record. They are how you answer access and erasure requests without scanning free text, and adding them later means backfilling a store you may not be allowed to read.
• Link the outcome to the external reference. The payment id, the message id, the ticket id. Without it you can prove what was decided but not what happened, and those diverge exactly when it matters.
• Keep the model narrative in its own labelled field. It is useful context for a human reader. It is not evidence, and mixing it into the record is how it ends up quoted as though it were.

The model's explanation is not evidence

A model asked why it did something produces a fluent, plausible account that need not correspond to the computation behind the action. It is generated after the fact, by the same process that produced the output. Treat it as a hint for a human reader, never as the record.

The reliable substitute is mechanical. Record the inputs, the retrieved evidence, the exact arguments and where each one came from. Those are checkable facts. If the refund amount is tagged as sourced from policy clause 4.2, anyone can open clause 4.2 and confirm it. If it is tagged model_generated, you have learned something important, and no explanation the model offers changes either conclusion.

Three levels of replay

Replay is what turns a record into an investigation tool. There are three levels, and most teams need the middle one.

1. Reconstruct. Re-render the prompt from the template version and bindings, and display it beside the retrieved evidence. No model call. This answers most questions and costs nothing.
2. Deterministic replay. Re-execute the run against recorded tool responses, so the tools return exactly what they returned then. This isolates the model from a changed world and is how you test whether a fix would have prevented the incident.
3. Live replay. Re-execute against live systems in a sandbox. Useful for checking whether current behaviour is acceptable, useless for reconstructing the past, and dangerous if anything writes.

One caution about level two. Even with sampling parameters fixed, providers do not generally guarantee identical output for identical input across time and infrastructure. Replay is evidence about inputs and control flow, not proof that the model would emit the same token sequence. Say that out loud in your audit documentation, because a claim of exact reproducibility is one someone will eventually test.

Decide early how much of each tool response you keep. Full responses make level two possible and grow fast; hashes alone are cheap and make it impossible. The usual answer is full responses for decisions with an effect, hashes for the rest.

What it costs to keep

Audit storage is usually cheap and occasionally surprising, and the surprise is always evidence rather than text. Run the arithmetic with your own numbers, because the difference between 90 days and seven years should be a deliberate decision.

Two levers matter more than the price. Content addressing collapses the evidence term, because the same policy document referenced by ten thousand runs is stored once and pointed at ten thousand times. And retention classes let a financial decision keep its seven years while routine telemetry expires in ninety days, instead of the whole store inheriting the longest obligation in the building.

Personal data against an immutable record

Append only storage and a legal duty to erase look like a contradiction, and they are not. Three techniques resolve it, and all three have to be in place before the first record is written.

• Per subject encryption keys. Encrypt subject-linked fields with a key held per person, and delete the key on an erasure request. The record and its hash chain survive, the personal data becomes unreadable, and the audit story stays intact. This is the approach that holds up when someone asks how you erased data from an immutable store.
• Structure before you store. You cannot redact what you did not structure. Personal data hides in free text, and the worst offender is the model narrative, so give it its own field inside the subject-keyed envelope rather than sprinkling names through prose.
• Pointers, not copies. Where a system of record already holds the personal data and honours deletion, store an id and a content hash instead of the bytes. Your audit trail proves what was read without becoming a second uncontrolled copy of it.
• Record what left your estate. Which provider processed the content, under which configuration, and what that configuration says about retention on their side. The question arrives in every security review and is unanswerable later if nobody wrote it down. Wider rules are in handling personal data in an AI pipeline.

Building it in the right order

This is cheap when built in sequence and expensive when retrofitted. The first step is the one that makes every later step possible.

1. Emit and propagate one run idDay one

   Generated at the entry point and attached to every log line, tool call, queue message and retry across every service. Nothing else here works if you cannot join the pieces of a run afterwards.

2. Pin at call timeWeek one

   Prompt version and rendered hash, requested and returned model ids, sampling parameters, tool versions and schema hashes. A few lines around the call site, and impossible to add later.

3. Content address the evidenceWeek two

   Hash every retrieved document, store it once, reference it by hash with an as_of timestamp. This is also the change that stops storage growing linearly with traffic.

4. Write decision recordsWeek three

   One durable record per action with an effect, including args_source, the controls block and the external reference from the outcome. Separate store, separate retention, indexed on run id, subject id, action type and prompt version.

5. Build the replay harnessMonth two

   Recorded tool responses, a command that takes a run id and re-executes against them, and a diff against the original outcome. Until this exists, every incident review is an argument between recollections.

6. Add retention classes and subject keysMonth three

   Classify records by obligation, encrypt subject-linked fields per person, then run an erasure drill on a test subject and confirm the record survives while the personal data does not.

7. Rehearse an audit quarterlyOngoing

   Pick a random decision from three months ago and answer the seven questions in the table with a stopwatch running. Anything taking more than a few minutes is the gap, and a rehearsal is a cheaper place to find it than a dispute.

The vocabulary, used precisely

Terms worth pinning down

Audit trail

A durable, queryable record of what a system decided, what it saw when it decided, and which versions of prompt, model, tools and policy were in force. It is distinct from application logging, which records what your code chose to print.

Pin

A version identifier or content hash captured at execution time so a past run can be reconstructed exactly. Pins are cheap to write and impossible to add retrospectively, which is why they belong in the first commit rather than the first audit.

Content addressing

Referring to a document by the hash of its bytes rather than by a location such as a URL or a file path. It makes evidence tamper evident, deduplicates storage automatically, and proves which version an agent actually read.

Deterministic replay

Re-executing a recorded run against the tool responses captured at the time, so the outside world is held constant. It shows whether a change to prompts or code would have altered the decision, without depending on live systems.

Argument provenance

A per argument record of where each value in an action came from: a retrieved document, a policy clause, a human edit, or the model itself. It is the shortest path to answering whether a number was read or invented.

Crypto-shredding

Encrypting personal data under a key held per data subject and destroying the key to satisfy an erasure request. The surrounding record and its integrity chain remain intact while the personal data becomes permanently unreadable.

The tracing, the replay harness and the decision store are what we build in agent observability. The field level detail sits in what to log so future you can debug it, the failures these records exist to explain are in how agents fail, and the approval fields in the controls block come from human in the loop design.