Automating a Shared Inbox Without Losing Anything

What shared inbox automation actually is

The first mistake almost everyone makes is treating the mailbox as the system of record. It is not. A mailbox is a shared mutable folder that four people, two server side rules, a retention policy and your automation all write to at once, with no transactions and no ordering guarantee. Your system of record is a separate store keyed by the provider's immutable message id, holding case state, the custody log and the classification history. The mailbox is an input device and an output device.

Four parts do the whole job: an ingest path that turns provider events into idempotent work items, a classifier that is mostly deterministic and only partly a model, a custody layer that guarantees ownership, and a write path that produces drafts, labels and state transitions. See inbox and email automation for how these get scoped.

Why a mailbox is the hardest input you will connect

Mail is hard because it is a distributed system with no ordering guarantee, no delete guarantee, and a different sync model at every provider. The protocol is older than most assumptions your code makes about it.

• Threading is a heuristic. RFC 5322 gives you In-Reply-To and References, and real clients populate neither reliably. Gmail groups by its own thread id and also groups on subject. Graph gives you conversationId. One case in Gmail can be three conversations in Outlook.
• Quoted history balloons the payload. The newest reply is often 40 lines inside a 900 line quote chain. Feed the whole body to a model and you pay for the same text every turn, then watch it answer a question resolved four replies ago.
• Aliases hide the real recipient. Route on the To header and you miss everything that arrived by BCC or a forwarding rule, where the mailbox address appears in no visible header.
• The sent folder is state. An agent replying from their own client just changed the case without telling you. Skip ingesting sent mail and your automation chases cases a human already closed.
• Other rules run before you do. Server side rules, retention policies and a colleague's filters move, mark and delete messages under your feet.

Then there is sync. IMAP gives you IDLE, a long lived connection that says something changed and nothing about what. Gmail gives you a historyId plus push over Pub/Sub. Microsoft Graph gives you delta tokens plus change notifications on a subscription that expires and needs a validation handshake. The modern two share a sharp edge: the cursor expires. When a history or delta token goes stale the provider tells you to start again, and your ingest path has to survive a full resync of a mailbox holding years of mail without reprocessing any of it.

Push against pull is not an either or, and the reasoning generalises well past mail. It is worked through in webhooks vs polling.

The custody model that stops messages vanishing

Messages go missing in the gap between systems, not inside them. A bot picks up a message, the worker dies mid classification, and the message sits in neither a human queue nor a bot queue until the customer follows up angrily eleven days later. Classification is the easy half of inbox automation. Custody is the half that loses things.

The custody log has a second use nobody plans for. Filter to messages where a human took custody back from the bot and changed the classification, and you have an evaluation set drawn from real production disagreements rather than from a workshop.

Classify with rules before you classify with a model

Run deterministic header rules first, because the entire class of message that causes catastrophic behaviour is identifiable from headers alone, with no ambiguity and no cost. A model asked whether a bounce notification deserves a friendly reply will occasionally say yes. A header rule never will.

Only what survives that table is worth a model call, which is the message with a human on the other end and free text in the body. Keep the classifier narrow: one queue, one intent, one confidence, one escalation flag. Do not ask a single call to classify, summarise, extract fields and draft a reply, because you then cannot tell which of the four regressed when the output gets worse.

SYSTEM
You are the triage stage of a shared inbox. You classify exactly one message.
You never write a reply and you never decide whether one gets sent.

You receive: sender address, subject, the newest message body with quoted history
stripped, the case state if this thread is already open, and allowed_queues.

Rules
1. Choose exactly one value from allowed_queues, or "unrouted".
2. confidence is your estimate that a trained agent would pick the same queue.
   Below 0.75, set queue to "unrouted" and fill needs_human.
3. Never infer a queue from the signature block or the sender domain alone.
4. If the message contains a legal notice, a threat, a deletion request, a security
   disclosure or a payment dispute, set escalate true whatever the queue.
5. evidence is copied verbatim from the message, 12 words or fewer. If you cannot
   find evidence in the message, you do not have a classification.
6. Return the JSON object only. No prose, no code fence.

OUTPUT CONTRACT
{
  "queue":       "one of allowed_queues, or 'unrouted'",
  "intent":      "verb first, 8 words max: 'requests refund for duplicate charge'",
  "confidence":  0.0,
  "escalate":    false,
  "reopen_of":   "existing case id, or null",
  "sentiment":   "calm | frustrated | angry | unclear",
  "needs_human": "empty unless queue is 'unrouted'",
  "evidence":    "verbatim quote, 12 words max"
}

RETURN 'unrouted' AND SAY WHY WHEN
- the body is empty after quote stripping
- the message is a forwarded chain with no new text from the sender
- two unrelated requests share one message, name both

Drafting replies a human is willing to send

Draft into the thread and let a person press send, for every class, until that class has earned autonomy through weeks of measured agreement. That is not timidity. A draft is reversible, observable and cheap to evaluate, and the review action labels your dataset for free.

Get the mechanics right or the draft is worse than useless. Create it against the existing thread, using the thread id in Gmail or a reply object in Graph, so it appears under the conversation in the agent's own client and carries the correct References chain when it goes. Compose a fresh message with a Re prefix instead and the customer's client starts a second thread, so one case now has two histories. Verify send-as permission too, because a draft sent from a mailbox someone can read but not send as fails at send time, silently in some clients.

Ground the draft in something. Retrieval over resolved threads and current policy documents beats a long static prompt, because policy changes and the prompt will not. Attach retrieved source ids to the draft record rather than the draft body, so a reviewer sees what the model read without deleting a citation block before sending. Fluent and wrong is the specific failure to design against here, and the mechanisms behind it are in why RAG systems give confident wrong answers.

The graduation test for a class is boring and it works. For eight consecutive weeks the class needs a send rate above your threshold, edits below your threshold, zero escalations caused by a draft, and no policy change touching that class in the window. Miss one and the class stays on drafts. Human in the loop design covers placing the review step so it does not become theatre.

The arithmetic, on your own numbers

Inbox automation economics are decided by triage volume, not by reply quality, and model cost is almost never the number that matters. Run the arithmetic before the build, and treat every price as a variable you look up today rather than a fact you inherit from an article.

Two things that calculator hides will decide the project. Saved minutes are not saved money until the rota changes or volume grows into the freed capacity, so a team of six that saves four hours a day is still a team of six. And the cost of wrong decisions belongs on the same page: if one message in fifty is misrouted, and each misroute costs twenty minutes of customer patience plus ten minutes of internal chasing, put that in before deciding the system pays for itself.

Where it goes wrong

Three failure modes cause most of the damage: the mail loop, the duplicate, and the fight between a human and the bot over one thread. None of them is prevented by a better prompt.

The mail loop

Your automation replies to an out of office. The out of office replies back. Neither side counts, so the two of you exchange several thousand messages before a provider starts throttling and your sending domain picks up a reputation problem. The guards stack. Never respond to a message carrying Auto-Submitted with any value other than no. Set Auto-Submitted: auto-generated on everything your system originates. Enforce a per address ceiling, say three automated messages to one address in twenty four hours, counted in your own store rather than the mail provider. Then add a global outbound circuit breaker that trips on messages per minute and pages someone, because the loop that beats the first three guards is the one you did not imagine.

Duplicate processing

The same message arrives twice for ordinary reasons: push notifications are delivered at least once, a delta token expired and you resynced, or two workers grabbed the item because the lease outlived the visibility timeout. Use the provider's immutable message id as the idempotency key, never a hash of subject plus sender plus timestamp, because two customers really do send identical one word replies in the same second. Record the key before the side effect, and where the side effect is external, use an outbox so the record and the intent commit together. The pattern is in idempotency in automation.

The bot and the human fighting over one thread

An agent moves a thread out of billing because it is obviously a shipping question. Four minutes later a reconciliation job moves it back, because reconciliation reads the mailbox and reapplies the model's opinion. This is the default behaviour of any system that treats its own classification as truth. A human write sets a lock on that thread, and after the lock the automation may read, draft and suggest, but never change the queue, the labels or the state. Locks expire on case close, never on a timer.

How to roll it out without a bad week

Move in phases where each exit test is a number, and widen by message class rather than by percentage of traffic. Widening by percentage leaves every class partly automated and no class understood.

1. Shadow2 to 3 weeks

   The system ingests, dedupes, classifies and drafts, and writes nothing anyone can see. Exit test: agreement with the team on the top three classes is high enough that you would have accepted the machine's answer, and the unrouted rate is stable rather than climbing.

2. Suggest3 to 4 weeks

   Classifications appear as suggestions and drafts appear as real drafts on the thread. Nothing sends. Exit test: send rate and edit distance per class, measured. Any class where agents rewrite more than they keep has a policy problem, not a model problem.

3. Act on one narrow classweek 8 onward

   Pick the least dangerous class you have, usually an acknowledgement or a status lookup with no numbers in it, and automate that path only. Watch the follow up rate on that class, which is the honest measure of whether the customer was served.

4. Widen one class at a timeongoing

   Each new class repeats shadow, suggest and act with its own exit tests. Never promote two classes in the same week, because when the complaint rate moves you will not know which one moved it.

5. Put the kill switch within reachbefore go live

   One flag that stops all outbound automation while leaving ingest and classification running, reachable by the operations lead without an engineer. A kill switch that needs a deploy will not be used until far too late.

Definitions

Terms used precisely on this page

Shared inbox automation

A system that ingests messages from a shared mailbox, deduplicates them against an immutable message id, classifies each into a queue, assigns a single custodian, and drives the message through defined states to a coded closure. The mailbox is an interface to it, not the store behind it.

Custody

The property that exactly one party owns a message at any instant. Custody is held by a named human, by a named automation under a time limited lease, or by the queue itself. A message with no custodian is a defect, not a state.

Idempotency key

A stable identifier that lets a system recognise repeated work as the same work. In a mailbox it is the provider message id, or the RFC 5322 Message-ID header when a distribution list fans one message out into several copies.

Mail loop

A repeating exchange between two automated systems, usually started when an automated reply is sent to an automated reply. It is prevented by honouring the Auto-Submitted header, setting it on your own sends, and capping automated messages per address per day.

Reconciliation sweep

A scheduled pass that compares the mailbox against the case store and reports messages present in one and missing in the other. The count it finds is the honest measure of how much your event stream is dropping.