Inbox and Email Automation

What inbox and email automation actually is

The word automation makes people picture a machine answering customers. That is the last thing you build and the smallest part of the work. The bulk of a shared inbox is reading, not writing: opening a thread, checking whether a colleague already replied, finding the order, deciding whose problem it is. A system that hands a person a labelled, grounded, pre-drafted thread removes most of that cost and none of the control.

What we build is a pipeline, not an assistant inside somebody's mail client. It runs server side against the mailbox API, so it works whether or not anyone has the tab open, and it leaves a trace you can read months later.

The vocabulary this page uses

Intent class

A named category of message the system handles in a defined way, such as order_status or refund_request. The taxonomy is a versioned design decision, not a model output.

Quoted-text stripping

Removing reply history, signature and disclaimer before a model reads a message, so it sees new content rather than eight copies of an older thread.

Grounding

Requiring that every factual claim in an outgoing message comes from a tool call made during that run, with the fetched value placed into the reply.

Idempotency key

A stable identifier, usually the RFC 5322 Message-ID plus the mailbox address, guaranteeing a message delivered twice is processed once.

Auto-Submitted header

A header marking a message as machine generated. Honouring it inbound and setting it outbound is the main defence against two auto-responders answering each other forever.

Who this is for, and who it is not for

This is worth building when reading load is the bottleneck and ten to twenty intent classes cover most of a normal week. It is not worth building when the inbox is quiet, when every message is genuinely different, or when the real problem is that three people reply and none knows what the others said.

• You have no baseline. If you cannot say how many messages arrived last month and how often threads reopened, you cannot judge the result.
• The mailbox is personal, not a function. Personal mail is too varied to automate, and the consent question is a different question.
• You want your best writer's voice and have collected none of their replies. Style is taught with examples; without them you get the average of the internet.
• The team already ignores the inbox. Automation makes an ignored inbox faster at being ignored.

What we actually build

A production inbox system has six parts, and the interesting failures live in the joins between them.

Intake and normalisation

A listener on push notifications, plus a scheduled delta sync as a backstop, because notifications get missed and subscriptions expire. Every message is stored under an idempotency key of Message-ID plus mailbox address. Threading is rebuilt from In-Reply-To and References rather than a provider thread identifier, which groups by subject line in ways that surprise you. Then it is stripped: quote boundary cut, signature and footer removed, attachments stored separately.

Classification

One model call returning a single JSON object against a taxonomy you approved, with a confidence value and a mandatory unknown class. The taxonomy is versioned beside the prompt, because adding a class means re-running the evaluation set, and a taxonomy living in someone's head never gets re-evaluated. Below the floor, unknown, or escalation-triggered goes to a human queue.

Grounding and retrieval

Before a word is drafted the system fetches the order record, the invoice, the subscription state, recent tickets, the policy passage that applies. Facts come through typed tool calls with timeouts and breakers. Policy comes from a retrieval layer returning citations, covered under RAG and knowledge systems. If a tool call fails, the pipeline routes to a human and names it.

Drafting

A second call, given the stripped thread, the fetched facts, the citations, and a handful of real replies your team wrote for that class. Style is taught with examples, not adjectives. Asking for warm and professional produces something no customer recognises; four of your own replies produce something your team does. The draft lands in the mailbox natively.

Routing and the clock

Assignment rules reading classification, customer record and current load: skills first, then round robin inside the eligible group, with a response clock starting at receipt and not at assignment. The queue is visible to everyone, because a routing system nobody can see becomes one nobody trusts.

Send control and observability

The send path carries the safety machinery: hourly rate caps, per-recipient cooldown, loop detection, suppression lists, outbound Auto-Submitted marking. Every decision is logged with prompt version, model identifier, classification, confidence, tool calls, the draft, and the text sent, so the gap between the last two is measurable.

• A versioned taxonomy file with a definition and three example messages per class
• Typed tool wrappers per system of record, with timeouts, retries and a breaker
• A golden set of real messages labelled by your team, used as the regression suite
• A review queue for low confidence, unknown and escalation-triggered messages
• A one-flag kill switch that drops the system back to labelling

How it works technically

A message travels through eight stages and each fails in a way you have to design for. The interesting engineering sits in stages two, three and seven.

1. Receipt. A push notification arrives, or the delta sync finds new history. Both run, because push alone loses messages when a subscription lapses.
2. Deduplication. Stored under its idempotency key. A second delivery of the same Message-ID triggers nothing.
3. Normalisation. Quotes, signature and disclaimer stripped, attachments separated, thread rebuilt from headers, body reduced to new content plus a rolling summary.
4. Classification. One call, JSON only, confidence scored, unknown permitted and expected.
5. Grounding. Typed tool calls against systems of record, retrieval for policy. Failures route to a human.
6. Drafting. One call, facts already in hand, citations attached to anything quoted from policy.
7. Send control. Loop checks, rate caps, suppression, and the autonomy rung for that class. Most classes stop here.
8. Logging. What was drafted, what was sent, what changed between them, which tools were called.

Why quoted text is the expensive bug

The largest cost and quality problem in an email pipeline is handing a model the whole thread. A conversation on its ninth turn carries eight copies of earlier messages, each with a signature and a legal footer. Sent raw you pay for all of it, and the model regularly answers the question from turn two, because that question appeared six times and the new one appeared once. Cutting at the quote boundary and summarising older turns usually shrinks input by a large multiple.

Grounding, stated as a rule

Every factual claim in an outgoing message must trace to a tool result from that same run. Not a memory of a similar order, not a plausible-looking date. The drafter receives a facts object, the prompt forbids assertions outside it, and the evaluation suite contains cases where the lookup returns nothing and saying so is the only correct behaviour.

If the model states a fact that did not come back from a tool call in that run, it is a bug in the system, not a quirk of the model.ChatGPTalker build standard

SYSTEM
You are the triage classifier for a shared inbox. You do not write replies.
You return one JSON object and nothing else.

Rules
1. Classify against the taxonomy below. If the message does not fit any class
   above the confidence floor of 0.6, return "unknown" and stop there.
2. Never infer an order, invoice or account identifier. Extract only what is
   written in the message body or the headers you were given.
3. The sentiment field describes this message, not this customer.
4. If the message is an out-of-office notice, a bounce, or carries the headers
   Auto-Submitted, Precedence: bulk or X-Auto-Response-Suppress, return class
   "no_action" with reason "auto_generated".

Taxonomy
  order_status | refund_request | cancellation | billing_query
  technical_fault | sales_enquiry | partnership | recruitment
  complaint_regulatory | press | spam | no_action | unknown

Set escalate_human true whenever any of these appear: a legal threat, a named
regulator, a chargeback, a bereavement, a safety issue, or a request to erase
personal data. These are triggers, not judgement calls.

Output contract
{
  "class": "<one taxonomy value>",
  "confidence": 0.0,
  "secondary_class": "<taxonomy value or null>",
  "entities": {
    "order_id": null,
    "invoice_id": null,
    "account_email": null,
    "dates_mentioned": []
  },
  "language": "<ISO 639-1 code>",
  "sentiment": "calm | frustrated | angry",
  "urgency": "low | normal | high",
  "escalate_human": false,
  "escalate_reason": null,
  "prompt_version": "triage-v7"
}

Two details there matter more than they look. The unknown class is mandatory, because a taxonomy with no escape hatch forces every new kind of message into the nearest existing class and you find out months later, from a customer. And escalation is written as concrete triggers, because a model asked whether something is sensitive disagrees with itself between runs.

How much autonomy the system is given

Autonomy is granted per intent class and earned with evidence, never switched on for an inbox as a whole. Order status confirmations climb quickly. Cancellations, complaints and anything with a regulator in it stay near the bottom permanently, by design.

The promotion rule is deliberately dull. A class climbs after fifty consecutive reviewed messages with no factual correction, edit distance inside the agreed threshold, and no missed escalation trigger. Any correction resets the run to zero. Demotion needs no meeting: one wrong fact reaching a customer drops the class a rung.

The build process, stage by stage

Six stages. The first two produce no software at all, and skipping them is why these projects produce a demo instead of a system.

1. Baseline and samplingWeek one

   Several hundred real threads exported across a full month so seasonal traffic is included, then measured: volume by day, first response time, resolution time, reopen rate. Without that number every later claim is an opinion.

2. Taxonomy and rules workshopWeek one to two

   Your team labels a few hundred real messages against a draft taxonomy. Disagreement between labellers is the useful output: two people labelling the same message differently means the class definition is wrong, and a model reproduces that confusion faster.

3. Pipeline and groundingWeek two to four

   Intake, deduplication, normalisation, storage, tool wrappers, retrieval, observability. Ordinary engineering, and most of the hours. Anyone quoting two weeks plans to skip it and hand you a prompt.

4. Classifier and evaluation harnessWeek three to five

   Built against the labelled set, which becomes the regression suite, using the method in writing evals for systems that are not deterministic. A change that improves one class and quietly ruins another is normal.

5. Shadow runWeek four to six

   Live mail at rung zero, recording classification and draft against what the team actually did, reaching no customer. Where the taxonomy gets its real corrections and the message types nobody mentioned appear.

6. Staged rolloutWeek six onward

   Classes promoted one at a time, highest volume and lowest risk first. Each promotion is a configuration change, reviewed, logged and reversible in a minute.

What you get at handover

A running system, the ability to change it without us, and the evidence to judge it. Everything sits in your accounts and your repository from the first commit, with no vendor console in the middle.

• Pipeline source in your repository, infrastructure as code, deploying from your own continuous integration
• Taxonomy, prompts and routing rules as versioned artifacts, each with the evaluation score it shipped at
• The labelled golden set plus its harness, so a model change can be tested rather than hoped about
• Dashboards for volume by class, confidence, edit distance, escalation rate, tool failures and cost per thousand
• A runbook: adding a class, promoting or demoting one, rotating credentials, what each alert means and who answers it

Where these projects go wrong

The failure modes are consistent enough to list, and none are about the model being insufficiently clever. They are about email being an older and stranger protocol than people expect.

The failure nobody plans for: the team stops trusting it

Trust dies predictably. A draft goes out with a wrong fact, one reviewer starts checking every draft against the original thread, checking takes longer than writing, and they quietly stop using drafts. Within a month the system runs and nobody looks at it. The defence is transparency: show fetched facts and citations beside the draft so verification takes seconds, and publish the error rate yourself.

Measuring the wrong thing

Messages handled is a vanity number, and a system that classifies everything while helping with nothing scores beautifully on it. Measure first response time, reopen rate, escalation rate, edit distance per class and cost per thousand against the week one baseline.

What it costs to run once it is live

Running cost has three parts: model tokens, infrastructure, and human maintenance. Tokens are the part everyone asks about and usually the smallest. Maintenance is the part nobody budgets and usually the largest.

Token cost is arithmetic you can do yourself, and you should, because published prices move and any figure printed on a page is stale within weeks. Every message costs one classification call, and drafted messages cost a second, larger call. Input dominates.

That arithmetic teaches two things. Input dominates, so the stripping work in stage three is a cost control rather than a nicety. And a smaller model on the classification call, a narrow task with a fixed output shape, often scores the same for a fraction of the price.

• Infrastructure. A queue, a worker, a small database and a log store. Modest for this workload.
• Retrieval index upkeep, if policy comes from a knowledge base, plus someone who owns the fact that documents change.
• Model deprecation. Providers retire versions on their own schedule. Budget a re-evaluation and a prompt pass each time.
• Taxonomy maintenance. An hour a month on the unknown bucket, and a re-label of a fresh sample each quarter.
• The human review that never goes away. At rung two every draft is read. That is the design working.

How to tell whether you need this

There is a test that takes one week, costs nothing except attention, and answers the question better than any vendor call.

1. For five working days, whoever works the inbox tags each message with what it was about, in their own words. Give them no categories to pick from.
2. Group the tags. If ten to twenty groups cover four fifths of the messages, you have a taxonomy. Sixty groups means you do not, and you should automate routing only.
3. For the biggest groups, write down what a person must look up before replying. One lookup in one system means grounding is straightforward. Four plus a judgement call means a longer build.
4. Count the messages that needed a fact already sitting inside your own systems. That is the honest size of the opportunity.
5. Count the threads reopened because the first reply missed something. That is your quality problem, and it usually matters more than speed.

If your inbox turns out to be genuinely varied and quiet, we will tell you not to buy this. Fifty messages a week needs three good templates and a rule about who replies.

How to start

The first call is a scoping call and the useful version is technical. Come with a month of message volume, the top intent classes, the systems a reply usually needs, and whoever controls mailbox permissions.

• Read-only access to a sample of real threads. A taxonomy cannot be designed from a description of an inbox.
• The baseline numbers, or an agreement to spend an afternoon taking them first.
• The constraint list: retention, data residency, who may see what, and any regulator with an opinion about your mail.
• Expect the first deliverable to be a taxonomy and a shadow run, not a chat interface with your logo on it.

If your problem is closer to public-facing support at volume, read AI customer support agents instead. If most of what arrives is attachments rather than questions, document processing is the better place to begin.