Bias Audit: ResumeRank v2
advanced_to_interviewKey findings
1. Impact ratios (NYC Local Law 144 summary of results)
Impact ratio = category selection rate ÷ highest category selection rate. Categories below 2% of the sample are excluded from the impact-ratio calculation but still reported. Significance: Fisher's exact test vs. the reference group, Holm-adjusted within each table (α = 0.05).
sex
| Category | Number assessed | Selection rate | Impact ratio | 95% CI | Adj. p | Status |
|---|---|---|---|---|---|---|
| Female | 5,719 | 27.6% | 0.85 | 0.80–0.90 | 6.2e-09 | Monitor |
| Male | 6,119 | 32.6% | 1.00 | — | — | Reference group |
Individuals with unknown category (not included above): 162
race_ethnicity
| Category | Number assessed | Selection rate | Impact ratio | 95% CI | Adj. p | Status |
|---|---|---|---|---|---|---|
| Black or African American | 1,741 | 22.6% | 0.66 | 0.60–0.72 | 1.7e-20 | Adverse impact |
| Hispanic or Latino | 1,996 | 24.2% | 0.71 | 0.65–0.77 | 1.1e-16 | Adverse impact |
| Two or More Races | 418 | 26.6% | 0.78 | 0.66–0.91 | 0.0027 | Adverse impact |
| Asian | 1,220 | 33.1% | 0.97 | 0.89–1.06 | 0.49 | Pass |
| White | 6,315 | 34.2% | 1.00 | — | — | Reference group |
| Native American or Alaska Native | 148 | 25.7% | — | — | — | Excluded (small group) |
| Native Hawaiian or Pacific Islander | 44 | 36.4% | — | — | — | Excluded (small group) |
Individuals with unknown category (not included above): 118
Intersectional: sex × race_ethnicity
| Category | Number assessed | Selection rate | Impact ratio | 95% CI | Adj. p | Status |
|---|---|---|---|---|---|---|
| Female / Black or African American | 802 | 21.1% | 0.56 | 0.49–0.65 | 1.6e-18 | Adverse impact |
| Male / Black or African American | 913 | 23.9% | 0.64 | 0.56–0.72 | 4.2e-14 | Adverse impact |
| Female / Hispanic or Latino | 963 | 24.0% | 0.64 | 0.57–0.72 | 2.6e-14 | Adverse impact |
| Male / Hispanic or Latino | 1,004 | 24.7% | 0.66 | 0.59–0.74 | 2.2e-13 | Adverse impact |
| Female / White | 3,011 | 30.4% | 0.81 | 0.76–0.87 | 1.2e-08 | Monitor |
| Female / Asian | 586 | 32.9% | 0.88 | 0.78–1.00 | 0.081 | Pass |
| Male / Asian | 626 | 33.2% | 0.89 | 0.79–1.00 | 0.081 | Pass |
| Male / White | 3,216 | 37.4% | 1.00 | — | — | Reference group |
| Female / Two or More Races | 212 | 19.3% | — | — | — | Excluded (small group) |
| Male / Two or More Races | 199 | 33.2% | — | — | — | Excluded (small group) |
| Male / Native American or Alaska Native | 74 | 28.4% | — | — | — | Excluded (small group) |
| Female / Native American or Alaska Native | 72 | 23.6% | — | — | — | Excluded (small group) |
| Male / Native Hawaiian or Pacific Islander | 24 | 33.3% | — | — | — | Excluded (small group) |
| Female / Native Hawaiian or Pacific Islander | 19 | 36.8% | — | — | — | Excluded (small group) |
Individuals with unknown category (not included above): 279
2. Proxy and leakage analysis
How well the model inputs jointly predict each protected attribute (cross-validated AUC; 0.5 = no information, 1.0 = fully encoded):
- sex: 0.63
- race_ethnicity: 0.66
| Feature | Protected attribute | Assoc. with attribute | Assoc. with outcome | Proxy risk | Level |
|---|---|---|---|---|---|
| employment_gap_months | sex | 0.31 | 0.20 | 0.25 | strong |
| zip_region | race_ethnicity | 0.40 | 0.14 | 0.24 | strong |
| commute_distance_km | race_ethnicity | 0.26 | 0.13 | 0.18 | moderate |
| years_experience | sex | 0.02 | 0.39 | 0.08 | weak |
| skills_assessment | race_ethnicity | 0.01 | 0.54 | 0.07 | weak |
| years_experience | race_ethnicity | 0.01 | 0.39 | 0.07 | weak |
| employment_gap_months | race_ethnicity | 0.01 | 0.20 | 0.05 | weak |
| skills_assessment | sex | 0.00 | 0.54 | 0.00 | weak |
| education_level | sex | 0.00 | 0.19 | 0.00 | weak |
| education_level | race_ethnicity | 0.00 | 0.19 | 0.00 | weak |
| zip_region | sex | 0.00 | 0.14 | 0.00 | weak |
| commute_distance_km | sex | 0.00 | 0.13 | 0.00 | weak |
Association = bias-corrected Cramér's V (numeric features binned into deciles). Proxy risk = geometric mean of the two associations.
3. Explanation fairness
Explanation disparity = Jensen-Shannon distance between a group's average feature-importance profile and the population's (0 = identical reasoning). Stability deviation = how much less (+) or more (−) consistent explanations are for the group. Score penalty = feature whose contribution is most lower for the group than average.
sex
| Group | n | Explanation disparity | Stability deviation | Largest score penalty |
|---|---|---|---|---|
| Female | 5,719 | 0.020 | +0.007 | employment_gap_months (-1.16) |
| Male | 6,119 | 0.019 | -0.007 | skills_assessment (-0.05) |
race_ethnicity
| Group | n | Explanation disparity | Stability deviation | Largest score penalty |
|---|---|---|---|---|
| Black or African American | 1,741 | 0.072 | -0.001 | zip_region (-2.21) |
| Native Hawaiian or Pacific Islander | 44 | 0.062 | +0.006 | zip_region (-0.81) |
| Hispanic or Latino | 1,996 | 0.037 | -0.006 | zip_region (-1.68) |
| Native American or Alaska Native | 148 | 0.033 | -0.006 | zip_region (-0.39) |
| Asian | 1,220 | 0.029 | -0.004 | education_level (-0.15) |
| White | 6,315 | 0.027 | -0.002 | education_level (-0.01) |
| Two or More Races | 418 | 0.018 | +0.004 | zip_region (-0.47) |
sex × race_ethnicity
| Group | n | Explanation disparity | Stability deviation | Largest score penalty |
|---|---|---|---|---|
| Female / Black or African American | 802 | 0.089 | +0.009 | zip_region (-2.19) |
| Male / Black or African American | 913 | 0.060 | -0.012 | zip_region (-2.22) |
| Female / Native American or Alaska Native | 72 | 0.057 | +0.007 | employment_gap_months (-1.75) |
| Female / Hispanic or Latino | 963 | 0.044 | +0.003 | zip_region (-1.65) |
| Male / Asian | 626 | 0.041 | -0.013 | skills_assessment (-0.51) |
| Male / White | 3,216 | 0.041 | -0.008 | education_level (-0.03) |
| Male / Hispanic or Latino | 1,004 | 0.039 | -0.017 | zip_region (-1.69) |
| Female / Two or More Races | 212 | 0.029 | -0.002 | employment_gap_months (-1.03) |
| Female / Asian | 586 | 0.029 | +0.003 | employment_gap_months (-0.96) |
| Female / White | 3,011 | 0.023 | +0.003 | employment_gap_months (-0.99) |
| Male / Two or More Races | 199 | 0.019 | +0.013 | zip_region (-0.52) |
| Male / Native American or Alaska Native | 74 | 0.010 | -0.016 | zip_region (-0.15) |
4. Fairness drift over time
sex: impact ratio by month
race_ethnicity: impact ratio by month
5. Less discriminatory alternatives
Each alternative was retrained on the supplied features with 3-fold cross-validation and selects the same share of people as the current tool (30.3%). Validity = precision at that selection rate, measured against ground truth: job_success. Fairness = the worst impact ratio across every single and intersectional category.
| Alternative | Precision | AUC | Worst impact ratio | Worst group | Adverse groups | Uses protected data in training |
|---|---|---|---|---|---|---|
| Current tool (as deployed) | 79.2% | — | 0.56 | Female / Black or African American (sex × race_ethnicity) | 7 | no |
| Retrained, same features | 85.5% | 0.91 | 0.85 | Male / Hispanic or Latino (sex × race_ethnicity) | 0 | no |
| Drop employment_gap_months recommended | 85.6% | 0.91 | 0.86 | Male / Asian (sex × race_ethnicity) | 0 | no |
| Drop zip_region | 85.6% | 0.91 | 0.83 | Male / Hispanic or Latino (sex × race_ethnicity) | 0 | no |
| Drop commute_distance_km | 85.6% | 0.91 | 0.85 | Male / Asian (sex × race_ethnicity) | 0 | no |
| Drop all flagged proxies | 85.5% | 0.91 | 0.86 | Male / Hispanic or Latino (sex × race_ethnicity) | 0 | no |
| Reweighing | 85.5% | 0.91 | 0.84 | Male / Hispanic or Latino (sex × race_ethnicity) | 0 | yes |
| Concept nullspace projection | 84.7% | 0.90 | 0.87 | Male / Hispanic or Latino (sex × race_ethnicity) | 0 | yes |
| Drop proxies + nullspace | 85.5% | 0.91 | 0.86 | Male / Hispanic or Latino (sex × race_ethnicity) | 0 | yes |
- No candidate uses protected attributes or group-specific cut-offs when a decision is made; every candidate selects the same share of people as the current tool.
- Reweighing and nullspace projection use protected attributes during training only. Have counsel review before deploying them.
- In real deployments a job-related outcome is usually observed only for people who were selected (selective labels). Treat validity estimates as directional and confirm with a validation study.
- Candidates are logistic models retrained on the supplied features with 3-fold cross-validation; they show what is achievable, not a drop-in replacement for the vendor's model.
6. Recommendations
- Evaluate the recommended alternative (Drop employment_gap_months) with the model owner; it reduced adverse impact without losing validity in this analysis.
- Investigate the flagged groups with the model owner and search for a less discriminatory alternative (feature removal, re-weighting, or re-training with fairness constraints) that preserves job-relatedness.
- Review proxy features for business necessity. Where a feature mainly carries protected information, remove or transform it and re-run this audit.
- Explanations differ by group: document why the driving features are job-related, and check them for validity on each group.
- Fairness changed over time. Tie audits to model releases and run them continuously, not once a year.
- Do not fix disparities with group-specific score adjustments or cut-offs; adjusting employment-test scores on the basis of race, sex, religion or national origin is prohibited under 42 U.S.C. §2000e-2(l).
7. Method and limitations
- Rates use Wilson 95% intervals; impact-ratio intervals use the Katz log method.
- Intersectional categories are every combination of the selected attributes. Small cells have wide intervals; read status together with the CI.
- Statistical association is not proof of discrimination, and passing the four-fifths rule is not proof of fairness.
- Results describe the data supplied; they are only as complete as its demographic fields.