Machine Learning Fairness on the Adult Census Dataset

Machine learning models are increasingly used in high-stakes decisions, yet standard accuracy metrics can mask systematic biases against demographic groups. The UCI Adult census dataset provides a testbed for exploring these issues in a controlled setting.

The project compares three classifiers — Logistic Regression, Random Forest, and XGBoost — on the task of predicting whether income exceeds $50K/year. Beyond raw performance, the analysis evaluates fairness using demographic parity and equal opportunity metrics.

Feature engineering includes encoding categorical variables and creating interaction terms. Cross-validation ensures performance estimates generalize beyond the test set.

All models achieved acceptable accuracy, but fairness metrics revealed significant disparities. SHAP (SHapley Additive exPlanations) analysis identified which features most influenced predictions, helping distinguish legitimate predictive factors from proxy variables that could encode bias.

The key takeaway: fairness evaluation should be a standard part of any ML pipeline, not an afterthought. Techniques like threshold adjustment and reweighting can mitigate disparities without substantially reducing accuracy.