The Machinery: Gates, Checklists, and Who Says No

The standard arrangement is a single review before launch. It is the worst available position, because by then the purpose is fixed, the data is chosen, the architecture is built and the date is announced. A reviewer can raise concerns and cannot cause changes, which produces recommendations and a launch.

Three gates work better because each catches what only it can.

At inception, before building: should this exist, for what purpose, affecting whom? This is where the largest decision is available, and it is the gate most organisations do not have.

At design, before implementation is fixed: what data, what objective, what oversight arrangement, what happens when it is wrong? Changes here are cheap.

Before launch: does it meet the thresholds set at design, and what has the evaluation actually shown? This is the release gate from the AI QA cursus with the responsible-AI checks included rather than running alongside.

The economics drive the design. A concern raised at inception costs a conversation. The same concern at launch costs a delay, a rebuild, or an override, and the override is what usually happens.

So an organisation that can only build one gate should build the design gate, not the launch one. It is the last point at which the answer can still change what gets built.

Most checklists produce ticks. The difference is entirely in how the questions are written.

A question answerable with yes produces a tick. Have we considered fairness gets a yes from any team that has thought about it for a minute, and the record shows compliance.

A question requiring a specific answer produces a decision. Which populations does this system affect, and what is our measured performance on each cannot be answered without doing the work, and the answer is inspectable.

So the drafting rule is: no question whose answer can be yes.

Compare directly.

Weak: is the training data representative? Strong: which populations are underrepresented in the training data relative to the deployment population, and what did we do about it?

Weak: is there human oversight? Strong: who reviews, against what criteria, with what authority to override, and how many decisions per hour are they expected to handle?

Weak: have we considered the risks? Strong: what is the worst realistic outcome for someone affected, and what would have to be true for that to happen?

Weak: is the system explainable? Strong: what will we tell a person who challenges a decision, and what information do we retain to be able to?

A checklist of eight strong questions outperforms forty weak ones, and takes less time, because the weak ones consume attention without producing anything.

Each gate has a distinct question and a distinct set of available outcomes, which is what makes them worth having separately.

Inception asks whether this should exist. The available outcomes include not building it, narrowing the purpose, or proceeding with named affected populations recorded. This is the only gate where do not build is genuinely available.

Design asks how it should be built. Outcomes include changing the objective, changing the data, changing the autonomy level, and setting the thresholds that the launch gate will later test against. Everything the launch gate checks is decided here.

Launch asks whether it met the bar. Outcomes are ship, fix, narrow the rollout, or accept a named risk in writing with an accountable owner.

And after launch, the loop continues, because a system whose use changes returns to the inception question with a different answer.

The gate that matters most, and the shortest. Six questions, asked before anything is built.

What decision will this system make or influence, stated specifically enough that a change of use would be visible as a change?

Who is affected by that decision, distinguishing customers, staff and people who are neither and did not choose to be subject to it?

What is the worst realistic outcome for one of those people, described concretely rather than as a risk category?

Would we be comfortable explaining to that person how this decision was made? This is a deliberately informal test and it surfaces more than the formal ones, because discomfort here reliably indicates a design problem people have not yet articulated.

What is the alternative, including not building it, and what does the comparison actually favour?

And does this use fall in a regulated category, which routes it into the AI Act analysis from the high-risk cursus.

Thirty minutes, honestly answered, at a point where the answers can still change everything. The most valuable outcome this gate produces is occasionally deciding not to build something, and a gate that has never produced that outcome should prompt the question from the previous lesson about whether it binds.

Documentation of what a system is and how it behaves has converged on a recognisable format, and it is worth adopting rather than inventing.

Model cards, introduced by Mitchell and colleagues in 2019, proposed a short structured document covering a model's intended use, out-of-scope uses, training data, evaluation data, disaggregated performance across relevant groups, ethical considerations, and known limitations.

The two sections that carry most of the value are the ones teams find least comfortable. Out-of-scope uses forces an explicit statement of what the system should not be used for, which is what makes later misuse identifiable as misuse rather than arguable. And disaggregated performance forces measurement by group, which is where the fairness commitment stops being abstract.

System cards extend the idea from a model to a deployed system, which is usually the more useful unit, since the model is one component alongside retrieval, guardrails, thresholds and human review.

Three practical notes. Write it during development rather than afterwards, for the same reason the company brain cursus gives: reconstructed documentation is slower to produce and less accurate. Keep it versioned with the system. And decide deliberately who sees it, since an internal card can be more candid about limitations than an external one, and candour about limitations is most of its value.

Where the AI Act applies, much of this overlaps the Annex IV technical documentation, so the work is shared rather than duplicated.

Every control in this cursus depends on someone being able to stop a launch, and that is an organisational design question rather than a process one.

Three arrangements, in ascending order of how much they actually constrain.

Advisory. A function that raises concerns and cannot block. Delivery decides. This is the most common arrangement and it produces exactly the outcome you would predict when a concern meets a deadline.

Escalation. A function that cannot block but can force the decision upward, so proceeding requires a named senior person to accept the risk in writing. This is considerably stronger than it sounds. Most overrides do not survive having to be signed.

Veto. A function that can stop a launch outright. Strongest, and it concentrates a great deal of power in a place that may lack the commercial context, so it is usually reserved for defined categories such as a legal breach or an unmitigated high-risk finding.

The practical recommendation is escalation as the default with veto for named categories. It keeps decision-making with people accountable for outcomes while removing the ability to proceed silently.

And whatever the arrangement, the person exercising it must not report to the delivery function whose launch is at stake. That is the same structural point the governance role cursus makes, and no amount of process design substitutes for it.

Applying all of this to every AI feature guarantees the programme is abandoned. The routing rule matters as much as the controls.

Tier by consequence, using the question from the inception gate: what is the worst realistic outcome for an affected person.

Where the answer is nothing much, an internal drafting aid, a summarisation tool with a human reading the source, the control is an inventory entry and nothing more. Most systems land here and should.

Where a person is affected but the consequence is recoverable, a recommendation they can ignore, a routing decision that delays rather than denies, run the design gate and the disaggregated measurement, and skip the rest.

Where the consequence is significant or hard to reverse, employment, credit, education, health, access to services, or anything in an Annex III area, run everything.

Two failure modes this prevents. A uniform heavy process exhausts the organisation on low-stakes systems and has no credibility left when the serious one arrives. And a uniform light process misses the serious one entirely.

The routing decision itself should be recorded, because deciding a system is low-tier is a judgement someone may later question, and the reasoning is what makes it defensible. That is the same pattern as the classification work in the high-risk cursus, applied at a lighter weight.

A separate responsible-AI process competes with delivery and loses. The machinery works when it is inside the process teams already follow.

The inception questions belong in whatever document already justifies building something: the product brief, the project proposal, the funding request. Add a section rather than creating a form.

The design questions belong in the technical design document or architecture review that already exists. Teams that already write design docs will answer six more questions; teams asked to attend a separate review will reschedule it.

The launch checks belong in the release gate from the AI QA cursus, run as part of the same evaluation rather than as a parallel process with its own meeting.

The model or system card belongs in the repository next to the code, versioned with it.

And the inventory entry belongs in whatever the procurement or change process already updates.

The test of whether this has been done well: a team should be able to comply without attending anything new. Every additional meeting is a place the process leaks, and the leak always happens under schedule pressure, which is when the controls matter most.

This is the same principle the company brain cursus applies to knowledge capture, and it holds for the same reason: practices that depend on someone remembering to do an extra thing decay, and practices attached to something already happening do not.