The Audit Finding Nobody Wants on the Dashboard

The request is to suppress a known risk, and the stated reason is self-defeating. If the finding is genuinely not validated, the correct response to 'this is not validated' is to validate it or to report it with an explicit confidence caveat - not to make it invisible to the people accountable for the risk. 'Small sample' is an argument about how to report, not about whether to.

Why it matters beyond procedure. A 27-point differential in a selection tool is a potential human rights exposure, and the board is the body accountable for it. Removing it also removes the board's ability to give informed direction on the expansion the VP wants to approve next month. Withholding it is not a formatting choice; it is a decision taken on the board's behalf without their knowledge.

What I do, in order:

1. Do not alter the record. The finding stays; the supporting analysis is retained and version-controlled.
2. Respond in writing, proposing the alternative: keep the item on the dashboard with the sample size, confidence interval and the caveat stated plainly. That satisfies the director's concern about overstating the result without concealing it.
3. Escalate appropriately if the instruction is repeated - to the audit committee, the privacy officer, or legal, depending on governance structure. Escalation is not disloyalty; it is the mechanism that protects the director as well.
4. Separate the person from the pressure. Assume good faith: the director may be worried about an unvalidated number reaching the board. Give them a version that reports the risk accurately, and let them own it.

The professional frame: integrity obligations do not have an exception for inconvenient findings, and 'not validated' is not a category that permits omission of a known potential adverse impact.

First, test the finding rather than assert it.

• Sample size and confidence. Obtain the underlying n for each group; a 27-point gap on a modest sample may still be statistically meaningful, and an interval will show it. Do not let 'small sample' be asserted in either direction.
• The four-fifths heuristic. 41 ÷ 68 = 0.60, well below the conventional 0.80 screening threshold. Treat that as a red flag that triggers investigation, not as a legal verdict.
• Check the obvious confound first: geography and job-relatedness. If those two colleges simply supply applicants for a different shift or location, the gap may reflect operational variables. If it survives that, the model is the problem.
• Selection-rate analysis by stage. Is the gap created by the model, the underlying data, or by human decisions upstream? A screener trained on three years of hires has learned whatever pattern the managers applied historically, including any bias in them.

Conditions for any further deployment:

1. Vendor transparency or no deal. Contractual rights to audit, to explain model inputs, and to test the model on synthetic applicants. 'Trade secret' is not an answer to a discrimination question; if the vendor will not provide it, terminate.
2. Human decision-maker retained, with the screener used only to rank for human review, never to auto-reject.
3. Candidate notice and an appeal route, plus a privacy assessment of using past candidates' data as training material.
4. Ongoing adverse-impact monitoring by role, with a named owner and a documented stop rule.
5. Legal and human-rights review against applicable obligations - several jurisdictions now impose specific requirements on automated hiring tools, and the requirements differ.
6. A control comparison before scaling: run parallel screening on one job family to test the model against human review.

Give them a route to their goal that is faster than the one they are on. Refusal without an alternative is what makes HR look like a blocker; a phased path with a date on it does the opposite.

Practical framing:

• Distinguish expansion from continuation. Stop expanding to new roles; permit continued use in the pilot role only with human review of every screened-out candidate and adverse-impact monitoring in place. That is a change of configuration, not a shutdown, and it can happen this week.
• Set a 60-day gate with named deliverables: vendor audit rights agreed, a statistical read on the gap with confidence intervals, an appeal mechanism live, and a stop rule signed off. If the vendor will not cooperate, the gate fails on day 30, not on rollout day, and the decision is made on evidence.
• Put a decision owner above both of you. This is not a dispute between Talent and HR to be won; it is a risk decision for the executive team. Escalate jointly - the VP and I present one page together with the finding, the exposure and the options.
• Name the asymmetry. Missing a rollout date is a delay. Deploying a selection tool with a known 27-point differential into every role is a class of risk the agency cannot remediate retroactively for people already screened out. Frame it that way once, calmly, and in writing.
• Protect the relationship. The VP's underlying goal - faster hiring at volume - is legitimate and probably overdue. Offer the compliant version of it, and be seen to be solving their problem rather than policing it.