Advancements in Fairness Research for Image Recognition

This is the664th original article. Your attention is the greatest encouragement for us!

Keywords: fairness, bias, debiasing learning, image recognition, deep learning

Citation Format:

Wang Mei, Deng Weihong, Su Sen. 2024. Review on fairness in image recognition. Journal of Image and Graphics, 29(07):1814-1833

Wang Mei, Deng Weihong, Su Sen. 2024. Advancements in Fairness Research for Image Recognition. Journal of Image and Graphics, 29(07):1814-1833

[DOI: 10.11834/jig.230226]

http://www.cjig.cn/jig/article/html/230226

Figure: Three Major Sources of Bias

Table: Common Datasets for Image Recognition Fairness

1) Colored MNIST dataset link:

https://github.com/alinlab/LfF

2) Multi-Color MNIST dataset link:

https://github.com/zhihengli-UR/DebiAN

3) Biased MNIST dataset link:

https://github.com/erobic/bias-mitigators

4) Corrupted CIFAR-10 dataset link:

https://github.com/alinlab/LfF/tree/master/data

5) 9-Class ImageNet dataset link:

https://github.com/clovaai/rebias/blob/master/datasets/imagenet.py

6) BAR dataset link:

https://github.com/alinlab/BAR

7) bFFHQ dataset link:

https://github.com/zhihengli-UR/DebiAN/blob/main/datasets/bffhq.py

8) CelebA dataset link:

https://mmlab.ie.cuhk.edu.hk/projects/CelebA.html

9) UTKFace dataset link:

https://susanqq.github.io/UTKFace/

10) IMDB dataset link:

https://github.com/feidfoe/learning-not-to-learn/tree/master/dataset/IMDB

11) Fairface dataset link:

https://github.com/joojs/fairface

12) RFW dataset link:

http://www.whdeng.cn/RFW/index.html

13) BUPT-Balancedface and BUPT-Globalface dataset link:

http://www.whdeng.cn/RFW/index.html

For detailed dataset information, please refer to the full text of the paper.

Figure: Example Datasets for Image Recognition Fairness

In the field of image recognition, existing debiasing algorithms are mainly divided into: reweighting (resampling), image enhancement, feature enhancement, feature decoupling, metric learning, model adaptation, and post-processing.

Table: Recognition Rates on Unbiased Test Sets for Different Methods on Colored MNIST, Corrupted CIFAR-10, and bFFHQ

Table: Debiasing Results Using Different Methods Trained on BUPT-Balancedface on the RFW Dataset

For more experimental results, please refer to the full text of the paper.

Despite significant progress in academic research on fairness in image recognition, various unresolved challenges still require further attention:

1) Datasets and evaluation metrics still need improvement.

2) Fairness algorithms addressing unknown biases urgently need resolution.

3) The trade-off dilemma between accuracy and fairness needs to be overcome.

4) Unique development trends for specific tasks are beginning to emerge.

5) The transition from image fairness to video fairness.

Wang Mei, Associate Professor at the School of Artificial Intelligence, Beijing Normal University, primarily researching computer vision, trustworthy image recognition, and transfer learning.

E-mail: [email protected]

Deng Weihong, Corresponding author, Professor at the School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Doctoral Supervisor, Young Editorial Board Member of The Journal of Image and Graphics. His main research areas include pattern recognition and computer vision, face recognition, emotion recognition, pedestrian re-identification, and fine-grained image recognition.

E-mail: [email protected]

Su Sen, Professor at the School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Doctoral Supervisor, primarily researching data privacy and cloud computing.

E-mail: [email protected]

Editor: Xiu Xiu

Review: Wu Tongjun