Same System, Opposite Verdicts: Metric Discretion in AI Ethics Audits and the Limits of Disclosure

<title>Abstract</title> <p> Regulators increasingly require organisations to report a number for the fairness of an artificial intelligence system, and no standard fixes how that number is computed. This paper measures what that gap is worth. It introduces Alternative Ethics Measures, a construct modelled on the alternative performance measures that financial reporting has governed for three decades. Study 1 audits four decision systems, one deployed instrument and three canonical benchmarks, across the full range of defensible specifications, 91,572 in all. Within that range the verdict flips in all 7 pairings of a system with a protected attribute, the group identified as disadvantaged reverses, and the reported figure moves 6 to 30 times as far as sampling error. A validation exercise reproduces the error rates published in the COMPAS dispute and traces them to a reference population the original analysis never stated. Study 2 reads 6,797 published disclosure documents from two regimes, one mandatory and one voluntary, and finds only 38 reporting a quantified fairness measure. None justifies the selection, names the alternative not chosen, or reports a prior-period figure. Study 3 estimates what disclosure would recover. Reconciliation to a stated reference computation would close 78 per cent of the discretion window, adding consistency disclosure would bring the total to 82 per cent, and a measured residual would survive both. Governing the discretion therefore does not require settling which fairness metric is correct. Those choices can be made legible now. <bold>JEL classification:</bold> M42, M48, K23, O33, M14 </p>