Abstract
Deep image classifiers have been found to learn biases from datasets. To mitigate the biases, most previous methods require labels of protected attributes (e.g., age, skin tone) as full-supervision, which has two limitations: 1) it is infeasible when the labels are unavailable; 2) they are incapable of mitigating unknown biases—biases that humans do not preconceive. To resolve those problems, we propose Debiasing Alternate Networks (DebiAN), which comprises two networks—a Discoverer and a Classifier. By training in an alternate manner, the discoverer tries to find multiple unknown biases of the classifier without any annotations of biases, and the classifier aims at unlearning the biases identified by the discoverer. While previous works evaluate debiasing results in terms of a single bias, we create Multi-Color MNIST dataset to better benchmark mitigation of multiple biases in a multi-bias setting, which not only reveals the problems in previous methods but also demonstrates the advantage of DebiAN in identifying and mitigating multiple biases simultaneously. We further conduct extensive experiments on real-world datasets, showing that the discoverer in DebiAN can identify unknown biases that may be hard to be found by humans. Regarding debiasing, DebiAN achieves strong bias mitigation performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
In this work, “gender” denotes visually perceived gender, not real gender identity.
References
Agrawal, R., Srikant, R.: Fast algorithms for mining association rules in large databases. In: International Conference on Very Large Data Bases (1994)
Ahmed, F., Bengio, Y., van Seijen, H., Courville, A.: Systematic generalisation with group invariant predictions. In: International Conference on Learning Representations (2021)
Albiero, V., KS, K., Vangara, K., Zhang, K., King, M.C., Bowyer, K.W.: Analysis of gender inequality in face recognition accuracy. In: The IEEE Winter Conference on Applications of Computer Vision Workshops (WACVW) (2020)
Alvi, M., Zisserman, A., Nellaaker, C.: Turning a blind eye: explicit removal of biases and variation from deep neural network embeddings. In: The European Conference on Computer Vision Workshop (ECCVW) (2018)
Antol, S., et al.: VQA: visual question answering. In: The IEEE International Conference on Computer Vision (ICCV) (2015)
Arjovsky, M., Bottou, L., Gulrajani, I., Lopez-Paz, D.: invariant risk minimization. arXiv:1907.02893 [cs, stat] (2020)
Bahng, H., Chun, S., Yun, S., Choo, J., Oh, S.J.: Learning de-biased representations with biased representations. In: International Conference on Machine Learning (2020)
Balakrishnan, G., Xiong, Y., Xia, W., Perona, P.: Towards causal benchmarking of bias in face analysis algorithms. In: The European Conference on Computer Vision (ECCV) (2020)
Buolamwini, J., Gebru, T.: Gender shades: intersectional accuracy disparities in commercial gender classification. In: ACM Conference on Fairness, Accountability, and Transparency (2018)
Cadene, R., Dancette, C., Ben younes, H., Cord, M., Parikh, D.: RUBi: reducing unimodal biases for visual question answering. In: Advances in Neural Information Processing Systems (2019)
Choi, J., Gao, C., Messou, J.C.E., Huang, J.B.: Why can’t i dance in the mall? Learning to mitigate scene bias in action recognition. In: Advances in Neural Information Processing Systems (2019)
Clark, C., Yatskar, M., Zettlemoyer, L.: don’t take the easy way out: ensemble based methods for avoiding known dataset biases. In: Empirical Methods in Natural Language Processing (2019)
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., Huq, A.: Algorithmic decision making and the cost of fairness. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2017)
Creager, E., Jacobsen, J.H., Zemel, R.: Environment inference for invariant learning. In: International Conference on Machine Learning (2021)
Creager, E., et al.: Flexibly fair representation learning by disentanglement. In: International Conference on Machine Learning (2019)
Dhar, P., Gleason, J., Roy, A., Castillo, C.D., Chellappa, R.: PASS: protected attribute suppression system for mitigating bias in face recognition. In: The IEEE International Conference on Computer Vision (ICCV) (2021)
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., Zemel, R.: Fairness through awareness. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference (2012)
Geirhos, R., et al.: Shortcut learning in deep neural networks. Nat. Mach. Intell. 2(11), 665–673 (2020)
Gong, S., Liu, X., Jain, A.K.: Jointly de-biasing face recognition and demographic attribute estimation. In: The European Conference on Computer Vision (ECCV) (2020)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems (2014)
Grgic-Hlaca, N., Zafar, M.B., Gummadi, K.P., Weller, A.: The case for process fairness in learning: Feature selection for fair decision making. In: NIPS Symposium on Machine Learning and the Law (2016)
Hardt, M., Price, E., Srebro, N.: Equality of opportunity in supervised learning. In: Advances in Neural Information Processing Systems (2016)
Hazirbas, C., Bitton, J., Dolhansky, B., Pan, J., Gordo, A., Ferrer, C.C.: Towards measuring fairness in AI: the casual conversations dataset. arXiv:2104.02821 [cs] (2021)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Hendricks, L.A., Burns, K., Saenko, K., Darrell, T., Rohrbach, A.: Women also snowboard: overcoming bias in captioning models. In: The European Conference on Computer Vision (ECCV) (2018)
Jia, S., Meng, T., Zhao, J., Chang, K.W.: Mitigating gender bias amplification in distribution by posterior regularization. In: Annual Meeting of the Association for Computational Linguistics (2020)
Joo, J., Kärkkäinen, K.: Gender slopes: counterfactual fairness for computer vision models by attribute manipulation. In: International Workshop on Fairness, Accountability, Transparency and Ethics in Multimedia (2020)
Kamiran, F., Calders, T.: Data preprocessing techniques for classification without discrimination. Knowl. Inf. Syst. 33, 1–33 (2012). https://doi.org/10.1007/s10115-011-0463-8
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of StyleGAN. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Kim, B., Kim, H., Kim, K., Kim, S., Kim, J.: Learning not to learn: training deep neural networks with biased data. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Kim, E., Lee, J., Choo, J.: BiaSwap: removing dataset bias with bias-tailored swapping augmentation. In: The IEEE International Conference on Computer Vision (ICCV) (2021)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2015)
Krishnakumar, A., Prabhu, V., Sudhakar, S., Hoffman, J.: UDIS: unsupervised discovery of bias in deep visual recognition models. In: British Machine Vision Conference, BMVC (2021)
Kusner, M.J., Loftus, J., Russell, C., Silva, R.: Counterfactual fairness. In: Advances in Neural Information Processing Systems (2017)
Lahoti, P., et al.: Fairness without demographics through adversarially reweighted learning. In: Advances in Neural Information Processing Systems (2020)
Lang, O., et al.: Explaining in style: training a GAN to explain a classifier in StyleSpace. In: The IEEE International Conference on Computer Vision (ICCV) (2021)
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE (1998)
Li, W., et al.: Object-driven text-to-image synthesis via adversarial training. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Li, Y., Li, Y., Vasconcelos, N.: RESOUND: towards action recognition without Representation Bias. In: The European Conference on Computer Vision (ECCV) (2018)
Li, Y., Vasconcelos, N.: REPAIR: removing representation bias by dataset resampling. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Li, Z., Xu, C.: Discover the unknown biased attribute of an image classifier. In: The IEEE International Conference on Computer Vision (ICCV) (2021)
Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: The IEEE International Conference on Computer Vision (ICCV) (2015)
Manjunatha, V., Saini, N., Davis, L.S.: Explicit bias discovery in visual question answering models. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Nam, J., Cha, H., Ahn, S., Lee, J., Shin, J.: Learning from failure: training debiased classifier from biased classifier. In: Advances in Neural Information Processing Systems (2020)
Pleiss, G., Raghavan, M., Wu, F., Kleinberg, J., Weinberger, K.Q.: On fairness and calibration. In: Advances in Neural Information Processing Systems (2017)
Sagawa*, S., Koh*, P.W., Hashimoto, T.B., Liang, P.: Distributionally robust neural networks for group shifts: on the importance of regularization for worst-case generalization. In: International Conference on Learning Representations (2020)
Sarhan, M.H., Navab, N., Albarqouni, S.: Fairness by learning orthogonal disentangled representations. In: The European Conference on Computer Vision (ECCV) (2020)
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: visual explanations from deep networks via gradient-based localization. In: The IEEE International Conference on Computer Vision (ICCV) (2017)
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. Int. J. Comput. Vis. 128(2), 336–359 (2020). https://doi.org/10.1007/s11263-019-01228-7
Singh, K.K., Mahajan, D., Grauman, K., Lee, Y.J., Feiszli, M., Ghadiyaram, D.: Don’t judge an object by its context: learning to overcome contextual bias. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Sohoni, N.S., Dunnmon, J.A., Angus, G., Gu, A., Ré, C.: No subclass left behind: fine-grained robustness in coarse-grained classification problems. In: Advances in Neural Information Processing Systems (2020)
Torralba, A., Efros, A.A.: Unbiased look at dataset bias. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011)
Tzeng, E., Hoffman, J., Darrell, T., Saenko, K.: Simultaneous deep transfer across domains and tasks. In: The IEEE International Conference on Computer Vision (ICCV) (2015)
Verma, S., Rubin, J.: Fairness definitions explained. In: 2018 IEEE/ACM International Workshop on Software Fairness (FairWare) (2018)
Wang, A., Narayanan, A., Russakovsky, O.: REVISE: a tool for measuring and mitigating bias in image datasets. In: The European Conference on Computer Vision (ECCV) (2020a)
Wang, H., He, Z., Lipton, Z.C., Xing, E.P.: Learning robust representations by projecting superficial statistics out. In: International Conference on Learning Representations (2019a)
Wang, J., Liu, Y., Wang, X.E.: Are gender-neutral queries really gender-neutral? mitigating gender bias in image search. In: Empirical Methods in Natural Language Processing (2021a)
Wang, T., Yue, Z., Huang, J., Sun, Q., Zhang, H.: Self-supervised learning disentangled group representation as feature. In: Advances in Neural Information Processing Systems (2021b)
Wang, X., Ang, M.H., Lee, G.H.: Cascaded refinement network for point cloud completion. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2020b)
Wang, Z., et al.: CAMP: cross-modal adaptive message passing for text-image retrieval. In: The IEEE International Conference on Computer Vision (ICCV) (2019b)
Wang, Z., Qinami, K., Karakozis, I.C., Genova, K., Nair, P., Hata, K., Russakovsky, O.: Towards fairness in visual recognition: effective strategies for bias mitigation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2020c)
Yu, F., Seff, A., Zhang, Y., Song, S., Funkhouser, T., Xiao, J.: LSUN: construction of a large-scale image dataset using deep learning with humans in the loop. arXiv:1506.03365 [cs] (2016)
Zhang, B.H., Lemoine, B., Mitchell, M.: Mitigating unwanted biases with adversarial learning. In: AAAI/ACM Conference on AI, Ethics, and Society (2018)
Zhang, Z., Sabuncu, M.: Generalized cross entropy loss for training deep neural networks with noisy labels. In: Advances in Neural Information Processing Systems (2018)
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., Chang, K.W.: Men also like shopping: reducing gender bias amplification using corpus-level constraints. In: Empirical Methods in Natural Language Processing (2017)
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., Torralba, A.: Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1452–1464 (2018)
Acknowledgement
This work has been partially supported by the National Science Foundation (NSF) under Grant 1764415, 1909912, and 1934962 and by the Center of Excellence in Data Science, an Empire State Development-designated Center of Excellence. The article solely reflects the opinions and conclusions of its authors but not the funding agents.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, Z., Hoogs, A., Xu, C. (2022). Discover and Mitigate Unknown Biases with Debiasing Alternate Networks. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13673. Springer, Cham. https://doi.org/10.1007/978-3-031-19778-9_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-19778-9_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19777-2
Online ISBN: 978-3-031-19778-9
eBook Packages: Computer ScienceComputer Science (R0)