Running It: The Corpus, the Escalation Path, and the Team

Every capability in this cursus depends on retrieval finding the right document, which means the deployment's quality ceiling is set by the documentation, not by the model.

This is the finding most operations make late and painfully. A support assistant performs badly, the team investigates the model, tunes the prompts, considers a different vendor, and eventually discovers that the answers it was retrieving were wrong, outdated or absent. The model was working correctly on bad input.

Three specific problems, all covered in the company brain cursus and all acute here.

Staleness. Support documentation is written when a feature ships and rarely revised, so a corpus contains policies superseded twice. Semantic similarity is blind to time, so the 2023 refund policy retrieves as readily as the current one.

Contradiction. The knowledge base says one thing, the help centre another, and the macro a third. All three are in the corpus.

And absence. The most common questions frequently have no documented answer, because agents learned to handle them and nobody wrote it down. The corpus covers what someone thought to document rather than what customers ask.

The practical consequence for planning: budget for documentation work as the substantial part of the project. An operation that treats the corpus as an input already available will build something that answers confidently from the wrong page.

The sequence that produces a corpus worth retrieving from, using material the operation already holds.

Start from demand rather than from supply. Cluster the last quarter's contacts by theme, which gives the ranked list of what customers actually ask. That list, not the existing documentation structure, is the specification.

For each top theme, find the current correct answer. Sometimes it is documented. Sometimes it exists only in resolved tickets, where an agent worked it out. Sometimes it does not exist and has to be written, which is the gap the clustering exposed.

Curate the ticket-derived answers rather than ingesting them. Historical resolutions include wrong answers, superseded policies and improvisation, and an unfiltered archive reproduces all of it confidently.

Mark authority and date on every item, so recency and authoritative-location ranking have something to work with.

And assign an owner per theme, with a review trigger bound to product changes rather than a calendar, which is the change-triggered review from the company brain cursus.

The loop closes: new contact clusters reveal new gaps, and the corpus follows demand rather than following whoever last had time to write documentation.

The escalation decision is where a support deployment is won or lost, and the default configuration triggers it too late.

The reason is the incentive built into deflection: every escalation counts against the metric, so systems are tuned to keep trying. That produces the experience where a customer goes several rounds with a system that cannot help before reaching someone who can.

Signals that should escalate, most of which are available and unused.

Retrieval confidence is low, meaning nothing relevant came back. This should escalate before generating, since the alternative is improvisation. It is the cheapest and most reliable signal available.

The customer has rephrased the same question. One rephrase is normal; two means the system is not understanding.

Explicit frustration or a request for a human. This should be honoured immediately and without friction, and systems that make it difficult produce a specific and lasting anger.

The topic is in a defined sensitive category: complaints, cancellations, anything financial, anything about a vulnerable circumstance.

And conversation length beyond a threshold, since a long automated exchange rarely ends well.

The design principle: escalate early and cheaply. An escalation that happens quickly costs one agent contact. One that happens after six failed exchanges costs an agent contact plus a customer who is now angry, and the second is worse in every dimension including cost.

A consequence that catches operations out, because it is a second-order effect nobody plans for.

Automation handles the easy contacts, which are the simple documented questions. Those disappear from the agent queue.

What remains is everything else: the complex, the ambiguous, the emotionally difficult, the cases where the customer is already unhappy because the automation failed them first. Agents now face a queue with the easy contacts removed.

Three consequences.

Average handling time rises, and it rises for a good reason: the mix got harder. An operation reporting this as a performance decline has misread its own data, and agents penalised for it are being penalised for automation working.

The work gets harder in a sustained way. Consecutive difficult contacts with no easy ones between them is a materially different job, and the easy contacts were doing something the operation did not notice: providing recovery time. Support burnout is a known problem and removing the breathing space makes it worse.

And the skill requirement rises. A queue of only hard contacts needs more experienced agents, which raises the cost per agent while the headcount falls, so the saving is smaller than the volume reduction suggests.

The planning implication. Reset handling time expectations before deployment rather than after. Build in recovery deliberately, since the mix no longer supplies it. And expect the cost saving to be real and smaller than the contact reduction implies.

Support automation raises the job security question more directly than any other function in this catalogue, and how it is handled determines whether the deployment gets the cooperation it needs.

The reason it is acute. The stated purpose is handling contacts without an agent, and agents can read. Reassurance that this is about augmentation rather than replacement will be assessed against the deflection target on the same slide.

The bounded honesty from the managing-teams cursus applies, and specifically.

Say what is decided. If headcount reduction is planned, saying so is better than the alternative, because the alternative is discovering it. If it is not planned, say what is planned for the capacity instead: more channels, longer hours, faster response, or deeper help on complex cases.

Say what you do not know, which is usually most of the medium term.

And say what the work becomes, which is the mix change from the previous step. Agents should hear that their queue will get harder before they experience it, and that handling time expectations will change accordingly.

The cooperation matters practically rather than only ethically. Agents are the ones who know which contacts automate badly, which knowledge base articles are wrong, and where the escalation triggers should sit. A team that believes the deployment is aimed at them will not volunteer that, and the deployment will be worse for it.

And the corpus curation depends on agent knowledge, which cannot be extracted from people who have concluded they are training their replacement.

Six ways support deployments go wrong, all common enough to plan against.

The corpus was never fixed. The system retrieves stale or contradictory answers and delivers them confidently. This is the most common single cause and it is diagnosed as a model problem for months.

Escalation tuned against the customer. Deflection targets push the threshold high, so customers fight through several failed exchanges. Measured as success by the metric that caused it.

The context-free handoff. Agent receives an escalated contact with no history, customer repeats themselves, and the automation has added cost to an interaction it did not improve.

Invented policy. A fabricated refund term, delivery date or procedure, which is a commitment rather than an error.

The hidden human route. A customer who cannot find how to reach a person concludes you are avoiding them, and that conclusion is durable and gets shared publicly.

And metric collapse. Handling time rises because the mix hardened, the operation reads it as declining performance, and agents are managed against a number that automation caused to move.

The structure across all six: four are caused by optimising deflection, one by skipping the documentation work, and one by not updating the measurement. None is a model capability problem, which is where the attention usually goes.

An order that produces a working deployment rather than a fast one.

Weeks one to four. Cluster contact volume by theme and measure the current baseline: resolution, repeat contact rate, effort, satisfaction. You cannot evaluate anything without this and it is routinely skipped.

Weeks four to ten. Corpus work on the top themes. Find or write the current correct answer, curate ticket-derived material rather than bulk ingesting, mark authority and date, assign owners. This is the longest phase and the one that determines the outcome.

Weeks ten to fourteen. Agent assist first, starting with knowledge surfacing. Internal, low risk, and it validates that retrieval works before any customer sees it. If retrieval is poor here, it would have been poor customer-facing.

Weeks fourteen to twenty. Customer-facing automation on the lowest-stakes documented themes only, with disclosure, generous escalation triggers, a visible human route, and a handoff that carries context.

Then expand by theme, based on measured resolution rather than deflection, with the mix change communicated to the team and handling time expectations reset before it bites.

And throughout, run the contact clustering back into product and documentation, because removing the cause of a recurring contact beats deflecting it.

The sequencing point: the assist deployment is the test of the corpus. An operation that goes customer-facing first discovers its documentation problems in front of customers.

Six claims for a support operation.

Deflection is the wrong objective. It cannot distinguish a solved problem from an abandoned customer, and it has a cost saving attached that makes it easy to report. Pair it with repeat contact rate or do not report it.

The corpus is the product. A support assistant is exactly as good as the documentation behind it, and most of the work in a good deployment is documentation work that organisations budget as a model project.

Grounding is a legal requirement here, not a quality preference, because a support answer is a statement by your company that a customer will act on.

The handoff determines the experience more than the automation does, and the failure path should be designed first.

Agent assist usually pays better than customer-facing automation, at lower risk, and it is less impressive in a demonstration, which is why it gets less attention.

And the volume mix hardens. The easy contacts leave the queue, handling time rises for a good reason, the work gets harder, and the metrics have to be reset before agents are managed against a number automation moved.

The closing observation. The highest-value thing a support operation can do with AI is not answer contacts more cheaply. It is read its own contact volume properly and tell the rest of the organisation what the product keeps getting wrong, which removes contacts entirely and improves things for the customers who never contacted you at all.