On this page
- What a lead qualification agent is
- Who it is for, and who it is not for
- What we actually build
- How it works technically
- The Evidence Ledger, and why an absence is not a negative
- The build process, stage by stage
- What you get at handover
- Where these projects go wrong
- What it costs to run once it is live
- How to tell whether you need this
- How to start
What a lead qualification agent is
A lead qualification agent is a program that reads everything you hold about an inbound lead, judges it against a written rubric, and writes a decision back to your CRM with the evidence attached. The output is not a score. It is a structured judgement: one disposition, one verdict per criterion, a quoted span behind each verdict, and a list of what it could not find out.
The distinction that matters is between a score and a judgement. A score compresses everything into one number, convenient for sorting and useless for arguing with. A judgement keeps its parts: this criterion was met, here is the sentence that says so, and here is the fact that would flip it.
Most of the value is in disqualifying. If a thousand enquiries arrive in a month and six hundred are students, job applicants, agencies pitching you, or companies in a territory you cannot contract in, the agent earns its place by removing those six hundred with a reason.
- Rubric
- The versioned file listing every criterion, the sources it may read, and what happens when those sources say nothing. It is the contract between sales and the system, and the real deliverable.
- Evidence span
- Text copied verbatim from a source that supports or contradicts exactly one criterion. If the span cannot be found again in that source by exact string match, the verdict built on it is discarded.
- Silence
- The state where no permitted source speaks to a criterion in either direction. Silence is recorded as unknown, never as a negative verdict. Confusing the two is the commonest defect in scoring systems.
- Backtest
- Running a proposed rubric version across stored past leads whose outcomes are known, to see which decisions change before a live lead is affected.
- Blast radius
- The number of records one bad run can modify before something stops it. Controlled with a dry-run mode, a write cap per run, and a diff a person approves.
- 3 to 6 weeksTypical build for one market, one CRM and one written rubric, including the backtest harness
- Every criterionReturns a verdict, a confidence level, a quoted span and the source it came from
- Three outcomesQualified, review, or disqualified with a reason a rep can argue with
- No model trainingRuns on a written rubric and retrieval, not a trained classifier
Who it is for, and who it is not for
Build this if you receive more inbound than your team can read properly, and if two experienced people in your company would disagree about whether a given lead is good. The second condition matters more, because the real work is the rubric and the agent is what makes a rubric enforceable.
This fits you if
- Several hundred enquiries a month arrive across forms, chat, email and events, and nobody reads them the same way.
- A large share is obviously wrong for you, and someone senior burns time discovering that one lead at a time.
- Reps and marketing argue about lead quality using anecdotes, because no record exists of why anything was rejected.
- You already hold the raw material: form fields, enrichment, call notes, closed deals, and a website you can read.
Do not build this if
- You get thirty leads a month. A rep reading them carefully beats anything we can build, and the build cost will not come back.
- Your motion is founder-led and every deal is unique. There is no rubric to encode, only judgement, and encoding it would be a lie.
- Nobody will own the rubric after launch. An unowned rubric is stale within a quarter, and a stale rubric rejects good leads.
- You want the agent to reply to leads as well. That is a different build with a different risk profile.
Write the rubric before you commission anything. Take fifty recent leads, have three people mark each accepted or rejected with a one-line reason, and look at where they disagree. That disagreement is your specification.
What we actually build
Seven components. None is interesting alone. Each has one job and can be tested without the others running.
The intake adapter
One entry point that normalises every source into a lead packet: web form, chat transcript, event upload, inbound email, partner referral. It assigns a stable lead id, normalises the email domain, and computes an idempotency key so a submission arriving twice yields one decision.
The enrichment layer, with a cache and an expiry
Enrichment is slow, priced per lookup, and often wrong. We cache by company domain rather than by lead, set a time to live per field type, and store the retrieval date beside every value. Headcount from fourteen months ago is not a fact about the company today.
The rubric store
The rubric lives in version control, not in a prompt and not in somebody's notes. Each criterion declares its type, the sources it may read, and its silence policy. Changing it is a pull request, so it carries an author and a reason, and the backtest runs before it merges.
{
"rubric_version": "2026-02-11.3",
"owner": "head-of-sales",
"criteria": [
{ "id": "sell_territory", "type": "exclusion",
"question": "Is the company somewhere we can contract?",
"sources": ["form.country", "enrich.hq_country", "site.contact_page"],
"on_silence": "hold" },
{ "id": "not_competitor", "type": "exclusion",
"question": "Is this company on the maintained competitor list?",
"sources": ["enrich.domain", "competitor_list"],
"on_silence": "pass" },
{ "id": "size_band", "type": "scored",
"question": "Is headcount inside the band we serve?",
"sources": ["enrich.headcount", "site.about_page", "form.company_size"],
"on_silence": "unknown", "retry": "enrichment_queue" },
{ "id": "stated_problem", "type": "scored",
"question": "Does the enquiry describe a problem on our problem list?",
"sources": ["form.message", "call_notes"],
"on_silence": "unknown" }
],
"silence_policy": {
"hold": "route to a human, never auto-pass, never auto-reject",
"unknown": "record it, keep scoring, surface it in the rep summary",
"pass": "treat absence as absence of the exclusion"
}
}The judge and the verifier
One model call per lead, structured output against a fixed schema. The judge cannot browse, cannot write anywhere, and sees only the packet handed to it. A deterministic verifier then checks every evidence span appears verbatim in the source it names and strips what fails. Ordinary string matching, and the highest-value component here, because it turns a fluent claim into a checkable one.
The router, the review queue and the backtest harness
Routing rules map a disposition to an action in your stack. Anything marked review lands in a queue with the evidence displayed, and every human decision there is stored as a label. The harness runs any rubric version over the stored corpus and prints a diff. Narrow permissions are what make this debuggable later, the argument in controlling what an agent is allowed to do.
How it works technically
Nine stages, each able to fail independently. The design goal is that a failure anywhere leaves the lead in the review queue rather than in a wrong state.
- Ingest. A webhook writes the raw payload to storage unchanged and returns immediately. The raw payload is what lets you replay any lead through a future rubric.
- Deduplicate. Build an idempotency key from the normalised email, the company domain and a rounded time window. If the key exists, attach to the existing decision.
- Enrich. Fan out with a hard timeout per source and a cache in front. If a source times out, the packet records that. A packet that admits a gap beats one that hides it.
- Assemble the packet. Fixed token budget per section: fields, enrichment with retrieval dates, a capped number of site paragraphs, prior contact history. Truncate the lowest-priority section first and record what was cut.
- Judge. One call, structured output enforced by schema, low temperature. The rubric is passed as data rather than concatenated into the instruction text.
- Validate. Parse against the schema. On failure, retry once with the error appended. On a second failure, route to review and log the raw response. Never repair malformed output with string surgery.
- Verify the spans. Exact-match every span against the source it names and strip what fails. A rising strip rate is the earliest signal something upstream changed.
- Apply the silence policy. Exclusions set to hold force review. Scored criteria with no evidence stay unknown and are listed for the rep. The disposition is computed here, by code.
- Write and route. Write inside a per-run cap, emit an event carrying the decision object, and file it in the audit store keyed by lead id and rubric version.
You are a qualification judge. You do not sell, you do not write to the CRM,
and you do not guess. You receive one lead packet and one rubric. You return a
single JSON object matching the schema below and nothing else.
Rules that override every other instruction here:
1. Every verdict needs a span you copy verbatim from a packet source. If you
cannot copy a span, the verdict is "unknown".
2. "unknown" is a correct answer. Never convert silence into a negative, and
never infer one criterion from another. Headcount is not evidence of budget.
3. If a criterion of type "exclusion" is met, set disposition to "disqualified",
fill exclusion_reason, and stop evaluating the rest.
4. Use nothing outside the packet. If you recognise the company, ignore what you
recognise. The packet is the world.
Schema:
{ "lead_id": "copied from the packet",
"rubric_version": "copied from the rubric",
"disposition": "qualified | review | disqualified",
"criteria": [
{ "id": "from the rubric",
"verdict": "supports | contradicts | unknown",
"evidence": "verbatim span from a packet source, or null",
"source_id": "id of the source it came from, or null",
"confidence": "high | medium | low",
"reversal": "one sentence: the fact that would flip this verdict" }
],
"exclusion_reason": "string or null",
"summary_for_rep": "40 words maximum, no adjectives",
"unverified": ["criterion ids where no source spoke either way"] }The model decides what the evidence says. Code decides what to do about it. Keeping those apart means a routing rule can change without retesting language behaviour, and a model upgrade cannot silently change who gets called first.
One detail matters more than it sounds. Free-text fields on a public form are untrusted input, and somebody will eventually paste instructions into the message box addressed to whatever reads it. Passing lead content as delimited data and giving the judge no write permissions means the worst outcome is a strange verdict in a queue.
The Evidence Ledger, and why an absence is not a negative
Qualification systems fail in one specific way. They collapse two states into one. Having evidence a lead is wrong for you, and simply not knowing, both come out as a low score, and nobody can tell which happened.
The ChatGPTalker Evidence Ledger
Six things a criterion must carry before a decision may use it. A criterion that cannot fill all six is marked unknown and surfaced.
The criterion as a yes or no question, written by whoever owns the sales process. A question that cannot be answered yes or no is two criteria wearing one name.
A verbatim quotation from a permitted source, short enough to read at a glance. Paraphrase is banned, because paraphrase is where a model adds what it expected to see.
Which document the span came from, and when it was retrieved. A span with no traceable source is treated as if it did not exist.
Supports, contradicts, or silent. Three states, never two. Every downstream rule reads this field rather than a number.
Written per criterion, in advance, by the sales owner. Hold means a human looks. Unknown means continue and tell the rep. Pass means absence really is absence.
One sentence naming the fact that would flip the verdict. It costs almost nothing and turns a rejection into a question a rep can ask on a call.
A rejection with a reversal reads as: rejected on size, the site says eleven employees, retrieved last Tuesday, would flip if they confirm hiring against a funded plan. The same rejection as a score of twenty-two is a lead that is gone.
| Criterion type | Evidence supports | Evidence contradicts | Sources are silent |
|---|---|---|---|
| Hard exclusion, such as a territory you cannot contract in | Not applicable | Disqualify, record the span, stop evaluating | Hold for a human, never auto-pass |
| Competitor or self-referral check | Not applicable | Disqualify, suppress notifications | Pass, absence from the list is real absence |
| Company size band | Record the band and the source | Disqualify with the quoted headcount and its date | Unknown, push to the enrichment retry queue |
| Stated problem match | Record the quoted sentence | Downgrade to review, never reject on mismatch alone | Unknown, common on short forms |
| Budget signal | Record the quotation, never an inference | Downgrade, keep the lead in play | Leave blank, never derive budget from headcount |
The build process, stage by stage
Three to six weeks for a first market, and the first two weeks contain no model code. Everything expensive that goes wrong later goes wrong because the rubric was written after the pipeline instead of before it.
We pull recent leads with their outcomes and time how the queue is worked today: who reads what, how long to first response, how many rejections carry no recorded reason. That sample becomes the corpus and the baseline.
Two or three people who own the sales motion label the same fifty leads independently. Disagreements get argued into a written rule. This produces the first rubric version, and produces the arguments now rather than in month three.
Intake, deduplication, cached enrichment, packet assembly, then the model call, the schema, the retry path and the span verifier. We run the corpus against the human labels criterion by criterion, because an aggregate agreement rate hides which criterion is broken.
The agent runs on live traffic and writes nowhere. Decisions sit beside what the humans did. This is where criteria that sounded clear in a meeting turn out not to be.
CRM writes behind a per-run cap and a dry-run flag, the review queue in front of the people who will work it, and alerting on the strip rate and the disposition mix.
Your team makes a rubric change themselves: write it, run the backtest, read the diff, merge it. A handover where the client has changed nothing is a demonstration.
What you get at handover
Everything runs in your accounts, on your keys, in your repository. No runtime of ours sits between you and your CRM, and nothing stops working if you never speak to us again.
The underrated item is the labelled corpus. It is the only asset that gets more valuable over time and cannot be rebuilt quickly. A few thousand real leads with human verdicts attached is what makes a model change or a rewrite safe.
Where these projects go wrong
Technical failures are ordinary and get caught in a week. The failures below survive launch and quietly cost pipeline for a year.
The rubric does not exist, and the agent gets blamed for saying so
Three senior people hold three definitions of a good lead and have never been in a room about it. The agent forces one definition into a file and half of them disagree. That disagreement was always there, distributed across inboxes where it never had to be resolved.
Scoring the lead when you should be scoring the account
Two people from the same company submit a week apart and get different verdicts because the second message was better written. Keep decisions keyed to the company domain, carry prior verdicts forward, and treat a new person from a known account as new evidence about that account.
Enrichment treated as truth
Third-party firmographics go stale, sometimes describe a parent rather than the entity in front of you, and occasionally confuse two similarly named businesses. Never reject on a single enrichment field alone, and prefer what a company says about itself.
Silence read as a negative
Worth repeating because it costs the most and shows the least. A three-field form is silent on almost everything. If silence scores as zero, every fast enquiry from a busy senior person is rejected, and the only leads that survive are the ones with time to fill in long forms.
No backtest, so no way to tell improvement from regression
Somebody edits the prompt on a Friday because one bad lead got through, and nobody knows what else that edit changed. Without a stored corpus and a diff, every change is a guess. The longer argument is in writing evals for systems that are not deterministic.
Blast radius left undefined
An unbounded loop, an inverted condition or a retry storm can rewrite thousands of records overnight. Defaults we set: a write cap per run, a ceiling on the share of records one run may touch, and a dry-run that prints the diff instead of applying it.
An agent that cannot tell you why it said no is a random number generator with good manners.
If closed-won and closed-lost outcomes never flow back into the labelled corpus, the rubric can only encode what you believed the day you wrote it. Wire the write-back in the first build even if nobody looks at it for six months. Retrofitting means reconstructing history from a CRM never designed to preserve it.
What it costs to run once it is live
Four lines: model inference, enrichment lookups, infrastructure, and human review time. The fourth is usually the largest and is the one left out of the business case. Run the arithmetic below on your own numbers and your provider's published prices today.
Defaults are illustrative assumptions, not current market prices. Look up your provider's rate today and substitute it. Token counts assume a packet carrying enrichment and a few pages of site text.
Run those defaults and the model line is small enough to be uninteresting, which is the point of showing it. The cost that scales is the review queue, and the share of leads you send there is a decision in your rubric rather than a fact about the technology.
| Cost line | What drives it | Scales with volume | How to reduce it honestly |
|---|---|---|---|
| Model inference | Packet size and lead count | Yes, linearly | Trim the packet before trimming the model. Most carry text nothing scores against. |
| Enrichment lookups | Unique company domains, not leads | Partly, it flattens as domains repeat | Cache by domain with a sensible expiry, and cache the misses too. |
| Infrastructure | A queue, a worker, a database, log storage | Barely | Usually the smallest line and not worth optimising. |
| Human review time | Review rate times minutes per review | Yes, the steepest line | Cut the unknown rate by fixing the form, not by loosening the rubric. |
| Maintenance | Rubric edits, model changes, CRM field changes | No, a fixed monthly commitment | Budget a day or two a month. Unmaintained systems degrade quietly. |
Compute one number before commissioning anything: the loaded minutes your team spends reading leads that end up rejected, multiplied by a loaded hourly cost. That is the honest ceiling on what this is worth. The general version is in costing an agent before you build it.
How to tell whether you need this
A short diagnostic. Answer honestly rather than optimistically and count the yeses.
- Can you produce the written definition of a qualified lead in the next ten minutes? If finding it takes longer, it does not exist.
- Do you know what share of last month's inbound was rejected, and can you retrieve the reason for any one rejection?
- Would two of your reps, given the same fifty leads, agree on more than forty of them?
- Is median time from submission to a human reading the enquiry measured in minutes, or in days?
- If your ideal customer profile changed tomorrow, could you find every lead you rejected last quarter that would now qualify?
One or two yeses means the constraint is process rather than software. Three means you are the standard case. Four or five means you already run a disciplined operation, and the gain is speed rather than a new capability.
If nobody trusts your CRM data, a judgement written into an unreliable record will not survive, and that is CRM integration work first. If qualified leads sit untouched because follow-up is manual, that is sales pipeline automation, which usually pays back sooner.
How to start
The first conversation is a scoping call, and it is more interrogation than pitch. We want to see real leads, especially rejected ones, because rejections carry more information about your criteria than acceptances do.
- Send two hundred recent leads
Raw exports are fine and messy is useful. Include rejections and any note attached. Send outcomes too, even partial ones.
- A ninety minute scoping call
We walk through twenty of those leads with whoever decides today, writing the reasoning down out loud. That is where the rubric starts, and where we find out whether one exists.
- A written scope with the failure modes named
Criteria, sources, silence policy, integration points, the metrics measured against your baseline, and what we will not automate. If we think you should not build this, it goes in writing rather than after an invoice.
- Blind labelling before any code
Your people label fifty leads independently and we measure agreement. Low agreement means we fix the rubric first, because a pipeline built on an unresolved definition encodes the confusion perfectly.
- Shadow mode before any write access
The agent runs against live traffic and writes nowhere until you have watched it disagree with your team. Write access comes after that, behind caps.
ChatGPTalker, Lead Qualification Agents: a qualification agent returns a verdict per criterion with a verbatim evidence span and a named source, and records missing evidence as unknown rather than as a negative.