An algorithmic bias audit is a structured, evidence-based process for testing whether an AI system produces statistically different outcomes across protected groups before that system reaches production. It combines quantitative fairness metrics, such as demographic parity and equalized odds, with a documented review of training data, feature selection, and model behavior under simulated conditions. Done properly, it produces a defensible record, not just a clean-looking dashboard.
Governance officers are increasingly the ones accountable for this record. Regulators do not ask whether a model is accurate. They ask whether the organization can prove it tested for harm, on what basis, and what it did when the test flagged a problem. This article sets out the technical grounding and the sequence of steps a governance function needs to run a defensible audit.
What Is Disparate Impact, and Why Does It Matter for AI Systems?
Disparate impact is a legal and statistical concept describing a facially neutral policy or system that produces a substantially different outcome rate for one group compared to another, regardless of intent. It originated in US employment law but has become the working standard for AI fairness testing globally, because it does not require proof that the system was designed to discriminate.
The commonly cited threshold is the four-fifths rule: if the selection rate for a protected group falls below 80 percent of the selection rate for the highest-performing group, the system is flagged for further review. The four-fifths rule is a screening heuristic, not a legal safe harbor, and it should never be the only test a governance function runs.
An algorithmic bias audit checklist exists precisely because disparate impact can hide inside a model that performs well on aggregate accuracy. A hiring model can be 92 percent accurate overall and still reject qualified candidates from one demographic group at twice the rate of another. Aggregate metrics conceal exactly the harm the audit is designed to surface.
How Does Disparate Impact Differ From Disparate Treatment?
Disparate treatment is intentional differential handling of a protected group, such as a rule that explicitly scores applicants differently by age. Disparate impact requires no intent. It is the outcome pattern itself, measured after the fact, that creates liability exposure.
Most AI governance failures are disparate impact failures. A model trained on historical hiring data will reproduce historical hiring patterns unless the governance function actively tests for and corrects that reproduction.
What Are the Core Fairness Metrics a Governance Officer Needs to Understand?
Three metrics form the technical backbone of a competent bias audit. None of them is sufficient alone, and in most cases they cannot all be satisfied simultaneously. A governance officer's job is to choose the metric that matches the harm the system can cause, document that choice, and defend it.
Demographic Parity
Demographic parity, also called statistical parity, requires that the positive outcome rate be equal across groups regardless of the true underlying qualification rate. If 20 percent of applicants overall are approved, demographic parity requires that roughly 20 percent of each demographic subgroup be approved.
Demographic parity is easy to compute and easy to explain to a board. It is also blind to legitimate differences in the underlying qualified population, which means enforcing it can require rejecting qualified members of the majority group or approving unqualified members of a minority group purely to hit the ratio. Use it as a first-pass screening metric, not a final compliance test.
Equalized Odds
Equalized odds requires that the true positive rate and the false positive rate be equal across groups, conditional on the actual outcome. In plain terms: among people who are actually qualified, the model should approve them at the same rate regardless of group; among people who are not qualified, the model should reject them at the same rate regardless of group.
Equalized odds is the more rigorous standard for high-stakes decisions such as credit, hiring, and criminal justice risk scoring, because it accounts for the base rate of the outcome rather than just the raw approval rate. It is harder to satisfy and often trades off directly against demographic parity. A governance officer who understands this trade-off, and can explain why one metric was prioritized for a given use case, is doing the job the certification is meant to validate.
Predictive Parity
Predictive parity requires that the positive predictive value, the probability that a positive prediction is correct, be equal across groups. It matters most when the cost of a false positive falls unevenly across the population, such as in fraud flagging or risk scoring where a false flag itself causes harm independent of the final decision.
No system can generally satisfy demographic parity, equalized odds, and predictive parity at the same time when base rates differ across groups. This is a mathematical result, not a modeling failure. The audit process should document which metric was selected as primary, why, and what residual risk the unselected metrics represent.
What Is Proxy Variable Detection, and Why Is It the Hardest Part of the Audit?
A proxy variable is a feature that is not itself a protected characteristic but is statistically correlated with one closely enough to reproduce its effect. Zip code frequently proxies for race and income. Educational institution frequently proxies for socioeconomic status. Names can proxy for ethnicity and gender. Removing the protected attribute from the training data does not remove its influence if a proxy remains.
Proxy detection requires two passes. The first is a correlation analysis: measure the statistical association between each candidate feature and each protected class label in the training data, using the full population, not just the audit sample. The second is an outcome-level test: hold the model constant, vary only the proxy-correlated feature across synthetic profiles that differ only in the protected attribute, and measure whether the prediction shifts.
A feature does not need to be removed simply because it correlates with a protected class. Many legitimate business features do. The governance question is whether the feature's predictive contribution survives once its correlation with the protected class is statistically controlled for. If it does not, the feature is functioning as a proxy and should be flagged for removal or reweighting, with the decision documented.
How Do You Test Protected Classes That Are Not Explicitly Labeled in the Data?
Most production datasets do not include race, gender, disability status, or age as labeled fields, often because collecting them was avoided for compliance reasons. This creates a governance blind spot: you cannot measure disparate impact against a class you cannot observe.
The standard technique is Bayesian Improved Surname Geocoding or a comparable proxy-inference method, which estimates the probability distribution of a protected characteristic from available correlated fields, such as surname and geography, without requiring self-reported labels. This produces an estimate, not ground truth, and the audit documentation should state the method's error margin explicitly rather than presenting inferred demographics as fact.
Where self-reported demographic data can be collected under a lawful basis, such as through voluntary applicant surveys held separately from the decision pipeline, it should be preferred over inference. Inference is a fallback, not a first choice.
The Pre-Launch Bias Audit Checklist
The sequence below reflects the order in which a defensible audit should actually be run. Skipping steps or running them out of order is the most common reason audits fail to hold up under regulator or litigation scrutiny.
- Define the protected classes in scope. Base this on the jurisdiction and use case, not a generic list. Employment decisions in the US trigger different protected classes than credit decisions in the EU.
- Establish the outcome and the harm model. State precisely what a false positive and a false negative each cost, and to whom. This determines which fairness metric takes priority later.
- Audit the training data for representation gaps. Check whether protected subgroups are present in sufficient volume to produce statistically meaningful test results. A subgroup with 40 records in a 2 million record training set cannot support a reliable fairness conclusion.
- Run proxy variable detection on every feature. Correlate each feature against protected class labels or inferred proxies, then test predictive contribution after controlling for that correlation.
- Compute demographic parity, equalized odds, and predictive parity across all protected subgroups. Report all three even if only one is designated primary, so the trade-off is visible on the record.
- Apply the four-fifths rule as a screen, not a verdict. Any subgroup below the 80 percent threshold moves to mandatory secondary review regardless of sample size.
- Run intersectional testing, not just single-attribute testing. A model can pass fairness tests for gender alone and for race alone while still showing significant disparate impact for a specific intersection, such as older women in a particular region. Single-attribute testing systematically misses this.
- Stress-test with synthetic counterfactuals. Hold every feature constant except the protected attribute or its proxy, and measure whether the model's output changes. A stable output under counterfactual variation is stronger evidence than an aggregate metric alone.
- Document the metric selection rationale. Record which fairness definition was prioritized, why, and what the residual exposure is under the metrics not prioritized.
- Set a monitoring cadence for post-launch drift. A model that passes the pre-launch audit can drift into disparate impact as the input population shifts. Define the retest trigger and interval before launch, not after a complaint.
- Assign a named accountable owner and an escalation path. The audit record should show who had authority to block launch on fairness grounds and what happened when that authority was exercised or overridden.
What Documentation Does a Regulator or Board Actually Expect to See?
The documentation standard converging across the EU AI Act, NIST AI RMF, and ISO/IEC 42001 is consistent on one point: the audit has to be reproducible by someone who was not in the room. That means recording the specific dataset version, the metric thresholds set in advance of testing, the subgroup sample sizes, and the actual numeric results, not a narrative summary claiming the system "performed well across groups."
A common failure pattern is running the audit correctly but keeping only the final go/no-go decision on file, with the underlying test data overwritten by the next model version. Regulators and litigation discovery both target the underlying evidence, not the summary conclusion. Retention policy for audit artifacts should match the rigor of the audit itself.
Key Takeaways
- Disparate impact requires no intent to discriminate. It is measured from outcomes, using the four-fifths rule as a screening threshold, not a final legal test.
- Demographic parity, equalized odds, and predictive parity generally cannot all be satisfied at once when base rates differ across groups. Document which metric was prioritized and why.
- Proxy variables such as zip code or educational institution can reproduce a protected characteristic's effect even after that characteristic is removed from the training data. Test predictive contribution after controlling for the correlation.
- Intersectional testing catches disparate impact that single-attribute testing misses. Test combinations of protected attributes, not just each one in isolation.
- A defensible audit record is reproducible by someone who was not in the room: dataset version, pre-set thresholds, subgroup sample sizes, and raw results, retained past the go/no-go decision.
Governance officers who need to run this process with authority, not just familiarity, are the audience for AICA's CCAIGO, the Certified Chief AI Governance Officer credential, which covers AI governance frameworks and operating models, global AI regulation including the EU AI Act, NIST AI RMF, and ISO/IEC 42001, AI risk management and assurance, responsible AI policy design and enforcement, audit readiness and documentation, and board and regulator engagement.