Running the Audit and Acting on What It Finds

An audit that starts by computing metrics produces numbers nobody can act on. The sequence that produces decisions runs in a specific order.

Establish what the system decides and who it affects. Named groups, from the framing work in the first lesson, with the reasoning for inclusion and exclusion recorded.

Choose the criterion and the threshold, before any results exist. Which error harms the person more, therefore which metric, therefore what level is acceptable.

Measure, disaggregated, with confidence intervals and named intersections.

Explain, which is the step that separates an audit from a measurement. For any disparity found, state which cause you believe is operating: lower accuracy for the group, differing base rates, a label encoding past decisions, or a feature acting as a proxy.

Decide, from the options in the next steps.

And document, including the trade-off accepted and by whom.

The explain step is the one most often skipped, and skipping it makes everything downstream arbitrary. A remedy chosen without a cause is a guess, and the most common failure in this work is applying a post-processing fix to a problem that lives in the label.

Mitigations sort by where in the pipeline they act, and the choice should follow the cause you identified.

Pre-processing acts on the data before training: reweighting under-represented groups, resampling, or repairing features that carry proxy signal. Appropriate when the cause is sampling or representation. It leaves the model and the deployment untouched, which makes it the least disruptive.

In-processing acts during training: adding a fairness constraint or a penalty term to the objective, or adversarial approaches that discourage the model from encoding the attribute. Appropriate when the cause is the objective itself. It requires retraining and control of the training pipeline, so it is unavailable for a purchased model.

Post-processing acts on outputs: group-specific thresholds, or calibrating scores per group. Appropriate when you cannot retrain, and it is the only family available for a third-party system. It is also the family with the most difficult legal exposure, since applying an explicitly different threshold by protected group is direct differential treatment in some jurisdictions.

And the fourth option, which is not a mitigation at all and is often correct: change the label, narrow the deployment, or do not use the system for this decision.

There is no free fairness intervention, and pretending otherwise produces surprised stakeholders later.

All of them trade against aggregate accuracy. Constraining a model to satisfy a fairness criterion means it is no longer purely optimising predictive performance, so overall accuracy falls. Usually modestly, sometimes not, and the size should be measured rather than assumed.

They redistribute errors rather than removing them. Equalising the true positive rate across groups means raising it for one group or lowering it for another, and lowering it means real people who would have been selected now are not. Naming who bears the change is part of an honest proposal.

Most of them break calibration, which follows directly from the impossibility result. A system whose scores are consumed by humans as probabilities has a new problem after the fix.

And per-group thresholds carry legal risk that varies sharply by jurisdiction and domain. In some contexts explicitly treating people differently by protected characteristic is unlawful even when done to reduce disparity, so this family in particular needs legal input rather than an engineering decision.

The honest framing for a stakeholder conversation is therefore not we fixed the bias but we chose to accept a smaller disparity on this metric, at this cost in accuracy, having accepted a larger residual on that other metric. That sentence is harder to say and it is the true one.

Some audit findings have no technical remedy, and recognising them prevents months spent on interventions that cannot work.

The label encodes past decisions. Where the model is trained on who was hired rather than who performed, no reweighting or constraint fixes the fact that the target variable is a record of the thing you are trying to avoid reproducing. The remedy is a different label, which means finding or constructing an outcome measure, which is a substantial data project rather than a modelling change.

The data simply does not represent the group. Where a group is a small fraction of training data, techniques can partly compensate and cannot manufacture information that was never collected. The remedy is collecting more data about that group, which takes time.

The disparity reflects the world accurately. Where base rates genuinely differ for reasons outside the system, the model is not wrong and the question is whether the organisation wants to act on an accurate prediction. That is a policy decision, not a technical one, and it belongs with people who can make policy.

And the use case is unsuitable. Sometimes the honest finding is that a prediction problem is not well posed, or that the available data cannot support the decision being made. The remedy is narrowing the deployment or not deploying.

An audit function that never produces one of these findings is probably not looking hard enough at the label.

Most organisations are deployers rather than providers, which constrains the audit substantially and does not excuse skipping it.

What you cannot do. Inspect training data, retrain, apply in-processing constraints, or usually see the model at all.

What you can do, and should.

Measure outcomes on your own population. This is the important one and it is always available. You have the inputs you sent and the decisions that resulted, so you can compute selection rates and, where outcomes eventually arrive, the full confusion matrix per group. A vendor's claims about their system say nothing about how it behaves on your applicants.

Demand disaggregated performance from the provider, which the procurement cursus establishes as a diligence requirement and which the Article 13 instructions for use must contain for a high-risk system, including performance regarding the specific groups it is intended to be used on.

Control what you can: the threshold, which cases route to human review, which populations the system is applied to at all, and whether its output is advisory or determinative.

And remember the input data duty. Where you control the input, ensuring it is relevant and sufficiently representative is a deployer obligation, and feeding a system a population unlike the one it was validated on is your responsibility rather than the provider's.

Because you must choose a criterion and will fail others, the documentation is what makes the choice defensible rather than arbitrary. This is the artefact a regulator, a claimant or a journalist will actually read.

Six things it should contain.

The groups examined, and why those. Including which obvious candidates were not examined and for what reason, since that is the omission someone will notice.

The criterion chosen, and the harm-based reasoning that selected it. Not we used equal opportunity but false negatives exclude qualified applicants from consideration entirely, which we judged the more serious harm, therefore equal opportunity.

The measurements, with intervals, including the metrics you are not optimising. Reporting the residual disparity on the criteria you did not choose is what distinguishes an honest account from a selected one.

The mitigation applied and its measured cost, in accuracy and in redistributed errors.

The residual disparity that remains, stated plainly.

And who accepted it. A named person, on a date. An unmitigated disparity that someone consciously accepted is a governance decision; the same disparity unrecorded is an oversight failure, and the difference is entirely in this document.

Write it for a reader who wants the opposite conclusion, which is the same standard the derogation assessment in the high-risk cursus is held to.

An audit is a measurement at a moment, and every mechanism that produced the original disparity keeps operating.

Four things change underneath a closed finding.

The population shifts. A system validated on one applicant mix meets a different one after a marketing change, an expansion into a new market, or a seasonal swing, and a mitigation tuned to the old distribution may not hold.

The model changes. For a purchased system this happens on the provider's schedule with no deployment on your side, which the AI QA cursus establishes as the change vector nobody's pipeline catches.

Feedback loops form. Where the system's decisions influence the data it later trains on, an initial disparity amplifies. A model that selects fewer people from a group generates fewer positive outcomes for that group, which becomes training data confirming the original pattern.

And the threshold drifts. Someone adjusts it for volume or cost reasons without anyone connecting that to the fairness position it was set to hold.

So the practical arrangement: disaggregated metrics on a schedule, not only at launch. Thresholds that trigger re-audit rather than a dashboard nobody reads. And the fairness check inside the release gate, so a model change re-runs it automatically.

The feedback-loop case deserves particular attention, because it is the one that gets worse on its own while every individual measurement looks acceptable.

Three lessons, five claims.

Bias is three different words. Use disparity for the measurement and argue unfairness separately, because a measured difference can arise from a model defect, from genuinely different base rates, or from a label that records past decisions, and those need different responses.

Removing the protected attribute does not remove its influence and destroys your ability to measure. Retain it for measurement, control its use deliberately.

You must choose a fairness criterion, because Chouldechova and Kleinberg and colleagues proved you cannot have them all. Choose by asking which error harms the person more, and report the residual on the ones you did not choose.

Every mitigation costs something, in accuracy or in redistributed errors or in legal exposure, and some findings have no model-layer fix at all because they live in the label or in the data that was never collected.

And the audit's output is a documented, accepted trade-off with a named owner, not a fixed system. A team claiming to have removed bias has either not measured carefully or is describing one metric while ignoring what moved elsewhere.

The reason to do this work is the one from the responsible AI cursus. The person the system is used on is not in the room, and disaggregated measurement is the only mechanism that lets their experience reach a decision.