research-article

AffectFAL: Federated Active Affective Computing with Non-IID Data

Authors:

Changsheng XuAuthors Info & Claims

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Pages 871 - 882

https://doi.org/10.1145/3581783.3612442

Published: 27 October 2023 Publication History

Abstract

Federated affective computing, which deploys traditional affective computing in a distributed framework, achieves a trade-off between privacy and utility, and offers a wide variety of applications in business and society. However, the expensive annotation cost of obtaining reliable emotion labels at the local client remains a barrier to the effective use of local emotional data. Therefore, we propose a federated active affective paradigm to improve the performance of federated affective computing with a limited annotation budget on the client. A major challenge in federated active learning is the inconsistency between the active sampling goals of global and local models, particularly in scenarios with Non-IID data across clients, which exacerbates the problem. To address the above challenge, we propose AffectFAL, a federated active affective computing framework. It incorporates a Preference-aware Group Aggregation module, which obtains global models representing the different emotional preferences among clients. We also devise a tailored De-biased Federated Active Sampling strategy with an improved vote entropy, facilitating class balancing of labeled samples and alleviating the problem of sampling goals inconsistency between the global and local models. We evaluate AffectFAL on diverse benchmarks (image, video and physiological signal) and experimental settings for affective computing. Thorough comparisons with other active sampling strategies demonstrate our method's advantages in affective computing for Non-IID federated learning.

References

[1]

Sharat Agarwal, Himanshu Arora, Saket Anand, and Chetan Arora. 2020. Contextual diversity for active learning. In European Conference on Computer Vision. Springer, 137--153.

Digital Library

[2]

Umang Aggarwal, Adrian Popescu, and Céline Hudelot. 2020. Active learning for imbalanced datasets. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 1428--1437.

[3]

Jin-Hyun Ahn, Kyungsang Kim, Jeongwan Koh, and Quanzheng Li. 2022. Federated active learning (f-al): An efficient annotation strategy for federated learning. arXiv preprint arXiv:2202.00195 (2022).

[4]

Manoj Ghuhan Arivazhagan, Vinay Aggarwal, Aaditya Kumar Singh, and Sunav Choudhary. 2019. Federated learning with personalization layers. arXiv preprint arXiv:1912.00818 (2019).

[5]

Jordan T Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. 2019. Deep batch active learning by diverse, uncertain gradient lower bounds. arXiv preprint arXiv:1906.03671 (2019).

[6]

William H Beluch, Tim Genewein, Andreas Nürnberger, and Jan M Köhler. 2018. The power of ensembles for active learning in image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition. 9368--9377.

[7]

Christopher Briggs, Zhong Fan, and Peter Andras. 2020. Federated learning with hierarchical clustering of local updates to improve training on non-IID data. In 2020 International Joint Conference on Neural Networks (IJCNN). IEEE, 1--9.

[8]

Yu-Tong Cao, Jingya Wang, Ye Shi, Baosheng Yu, and Dacheng Tao. 2022. Knowledge-Aware Federated Active Learning with Non-IID Data. arXiv preprint arXiv:2211.13579 (2022).

[9]

Razvan Caramalau, Binod Bhattarai, and Tae-Kyun Kim. 2021. Sequential graph convolutional network for active learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 9583--9592.

[10]

Fei Chen, Mi Luo, Zhenhua Dong, Zhenguo Li, and Xiuqiang He. 2018. Federated meta-learning with fast convergence and efficient communication. arXiv preprint arXiv:1802.07876 (2018).

[11]

Jie Chen, Qirong Mao, and Luoyang Xue. 2020. Visual sentiment analysis with active learning. IEEE Access, Vol. 8 (2020), 185899--185908.

[12]

Zhiyong Cheng, Jialie Shen, Lei Zhu, Mohan S Kankanhalli, and Liqiang Nie. 2017. Exploiting Music Play Sequence for Music Recommendation. In IJCAI, Vol. 17. 3654--3660.

Digital Library

[13]

Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai. 2021. Exploiting shared representations for personalized federated learning. In International Conference on Machine Learning. PMLR, 2089--2099.

[14]

Corinna Cortes, Giulia DeSalvo, Mehryar Mohri, Ningshan Zhang, and Claudio Gentile. 2019. Active learning with disagreement graphs. In International Conference on Machine Learning. PMLR, 1379--1387.

[15]

Ido Dagan and Sean P Engelson. 1995. Committee-based sampling for training probabilistic classifiers. In Machine Learning Proceedings 1995. Elsevier, 150--157.

[16]

Zhipeng Deng, Yuqiao Yang, Kenji Suzuki, and Ze Jin. 2022. FedAL: An Federated Active Learning Framework for Efficient Labeling in Skin Lesion Analysis. In 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 1554--1559.

[17]

Moming Duan, Duo Liu, Xianzhang Chen, Yujuan Tan, Jinting Ren, Lei Qiao, and Liang Liang. 2019. Astraea: Self-balancing federated learning for improving classification accuracy of mobile deep learning applications. In 2019 IEEE 37th international conference on computer design (ICCD). IEEE, 246--254.

[18]

Maria Egger, Matthias Ley, and Sten Hanke. 2019. Emotion recognition from physiological signal analysis: A review. Electronic Notes in Theoretical Computer Science, Vol. 343 (2019), 35--55.

Digital Library

[19]

Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. 2020. Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach. Advances in Neural Information Processing Systems, Vol. 33 (2020), 3557--3568.

[20]

Zixiang Fei, Erfu Yang, David Day-Uei Li, Stephen Butler, Winifred Ijomah, Xia Li, and Huiyu Zhou. 2020. Deep convolution network based emotion analysis towards mental health care. Neurocomputing, Vol. 388 (2020), 212--227.

Digital Library

[21]

Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning. PMLR, 1126--1135.

[22]

Yoav Freund, H Sebastian Seung, Eli Shamir, and Naftali Tishby. 1992. Information, prediction, and query by committee. Advances in neural information processing systems, Vol. 5 (1992).

[23]

Yongjian Fu, Xintian Wu, Xi Li, Zhijie Pan, and Daxin Luo. 2020. Semantic neighborhood-aware deep facial expression recognition. IEEE Transactions on Image Processing, Vol. 29 (2020), 6535--6548.

Digital Library

[24]

Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran. 2020. An efficient framework for clustered federated learning. Advances in Neural Information Processing Systems, Vol. 33 (2020), 19586--19597.

[25]

Avishek Ghosh, Justin Hong, Dong Yin, and Kannan Ramchandran. 2019. Robust federated learning in a heterogeneous environment. arXiv preprint arXiv:1906.06629 (2019).

[26]

Filip Hanzely and Peter Richtárik. 2020. Federated learning of a mixture of global and local models. arXiv preprint arXiv:2002.05516 (2020).

[27]

Jennifer A Healey and Rosalind W Picard. 2005. Detecting stress during real-world driving tasks using physiological sensors. IEEE Transactions on intelligent transportation systems, Vol. 6, 2 (2005), 156--166.

Digital Library

[28]

Bingyu Hu, Zheng-Jun Zha, Jiawei Liu, Xierong Zhu, and Hongtao Xie. 2021. Cluster and scatter: A multi-grained active semi-supervised learning framework for scalable person re-identification. In Proceedings of the 29th ACM International Conference on Multimedia. 2605--2614.

Digital Library

[29]

Jian Huang, Ya Li, Jianhua Tao, Zheng Lian, Mingyue Niu, and Minghao Yang. 2018. Multimodal continuous emotion recognition with data augmentation using recurrent neural networks. In Proceedings of the 2018 on Audio/Visual Emotion Challenge and Workshop. 57--64.

Digital Library

[30]

Xin Kang, Yunong Wu, and Fuji Ren. 2018. Progressively improving supervised emotion classification through active learning. In Multi-disciplinary Trends in Artificial Intelligence: 12th International Conference, MIWAI 2018, Hanoi, Vietnam, November 18-20, 2018, Proceedings 12. Springer, 49--57.

[31]

Ioannis Kansizoglou, Loukas Bampis, and Antonios Gasteratos. 2019. An active learning paradigm for online audio-visual emotion recognition. IEEE Transactions on Affective Computing, Vol. 13, 2 (2019), 756--768.

[32]

Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. 2020. Scaffold: Stochastic controlled averaging for federated learning. In International Conference on Machine Learning. PMLR, 5132--5143.

[33]

Hirokatsu Kataoka, Tenga Wakamiya, Kensho Hara, and Yutaka Satoh. 2020. Would mega-scale datasets further enhance spatiotemporal 3D CNNs? arXiv preprint arXiv:2004.04968 (2020).

[34]

Dae Hoe Kim, Wissam J Baddar, and Yong Man Ro. 2016. Micro-expression recognition with expression-state constrained spatio-temporal feature representations. In Proceedings of the 24th ACM international conference on Multimedia. 382--386.

Digital Library

[35]

Dimitrios Kollias and Stefanos Zafeiriou. 2020. Exploiting multi-cnn features in cnn-rnn based dimensional emotion recognition on the omg in-the-wild dataset. IEEE Transactions on Affective Computing, Vol. 12, 3 (2020), 595--606.

[36]

Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. 2020. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, Vol. 2 (2020), 429--450.

[37]

Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, and Qi Dou. 2021. Fedbn: Federated learning on non-iid features via local batch normalization. arXiv preprint arXiv:2102.07623 (2021).

[38]

Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B Allen, Randy P Auerbach, David Brent, Ruslan Salakhutdinov, and Louis-Philippe Morency. 2020. Think locally, act globally: Federated learning with local and global representations. arXiv preprint arXiv:2001.01523 (2020).

[39]

Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi. 2020. Ensemble distillation for robust model fusion in federated learning. Advances in Neural Information Processing Systems, Vol. 33 (2020), 2351--2363.

[40]

Wei Liu, Jie-Lin Qiu, Wei-Long Zheng, and Bao-Liang Lu. 2021. Comparing Recognition Performance and Robustness of Multimodal Deep Learning Models for Multimodal Emotion Recognition. IEEE Transactions on Cognitive and Developmental Systems (2021).

[41]

Xiaofeng Liu, BVK Vijaya Kumar, Ping Jia, and Jane You. 2019. Hard negative generation for identity-disentangled facial expression recognition. Pattern Recognition, Vol. 88 (2019), 1--12.

Digital Library

[42]

Yang Liu, Yan Kang, Chaoping Xing, Tianjian Chen, and Qiang Yang. 2020. A secure federated transfer learning framework. IEEE Intelligent Systems, Vol. 35, 4 (2020), 70--82.

[43]

Guodong Long, Ming Xie, Tao Shen, Tianyi Zhou, Xianzhi Wang, and Jing Jiang. 2023. Multi-center federated learning: clients clustering for better personalization. World Wide Web, Vol. 26, 1 (2023), 481--500.

Digital Library

[44]

Yishay Mansour, Mehryar Mohri, Jae Ro, and Ananda Theertha Suresh. 2020. Three approaches for personalization with applications to federated learning. arXiv preprint arXiv:2002.10619 (2020).

[45]

Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics. PMLR, 1273--1282.

[46]

Prem Melville and Raymond J Mooney. 2004. Diverse ensembles for active learning. In Proceedings of the twenty-first international conference on Machine learning. 74.

Digital Library

[47]

Ali Mollahosseini, Behzad Hasani, and Mohammad H Mahoor. 2017. Affectnet: A database for facial expression, valence, and arousal computing in the wild. IEEE Transactions on Affective Computing, Vol. 10, 1 (2017), 18--31.

Digital Library

[48]

Mahdi Morafah, Saeed Vahidian, Weijia Wang, and Bill Lin. 2023. FLIS: Clustered federated learning via inference similarity for non-IID data distribution. IEEE Open Journal of the Computer Society (2023).

[49]

Louis-Philippe Morency, Jacob Whitehill, and Javier Movellan. 2008. Generalized adaptive view-based appearance model: Integrated framework for monocular head pose estimation. In 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition. IEEE, 1--8.

[50]

Jicai Pan, Shangfei Wang, and Lin Fang. 2022. Representation Learning through Multimodal Attention and Time-Sync Comments for Affective Video Content Analysis. In Proceedings of the 30th ACM International Conference on Multimedia. 42--50.

Digital Library

[51]

Amin Parvaneh, Ehsan Abbasnejad, Damien Teney, Gholamreza Reza Haffari, Anton Van Den Hengel, and Javen Qinfeng Shi. 2022. Active learning by feature mixing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12237--12246.

[52]

Rosalind W Picard. 2000. Affective computing. MIT press.

Digital Library

[53]

Riccardo Presotto, Gabriele Civitarese, and Claudio Bettini. 2022. Semi-supervised and personalized federated activity recognition based on active learning and label propagation. Personal and Ubiquitous Computing, Vol. 26, 5 (2022), 1281--1298.

Digital Library

[54]

Fan Qi, Xiaoshan Yang, and Changsheng Xu. 2018. A unified framework for multimodal domain adaptation. In Proceedings of the 26th ACM international conference on Multimedia. 429--437.

Digital Library

[55]

Fan Qi, Xiaoshan Yang, and Changsheng Xu. 2021. Zero-shot video emotion recognition via multimodal protagonist-aware transformer network. In Proceedings of the 29th ACM International Conference on Multimedia. 1074--1083.

Digital Library

[56]

Fan Qi, Zixin Zhang, Xianshan Yang, Huaiwen Zhang, and Changsheng Xu. 2022. Feeling Without Sharing: A Federated Video Emotion Recognition Framework Via Privacy-Agnostic Hybrid Aggregation. In Proceedings of the 30th ACM International Conference on Multimedia. 151--160.

Digital Library

[57]

Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po-Yao Huang, Zhihui Li, Brij B Gupta, Xiaojiang Chen, and Xin Wang. 2021. A survey of deep active learning. ACM computing surveys (CSUR), Vol. 54, 9 (2021), 1--40.

[58]

Kashfia Sailunaz and Reda Alhajj. 2019. Emotion and sentiment analysis from Twitter text. Journal of Computational Science, Vol. 36 (2019), 101003.

[59]

Felix Sattler, Klaus-Robert Müller, and Wojciech Samek. 2020. Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints. IEEE transactions on neural networks and learning systems, Vol. 32, 8 (2020), 3710--3722.

[60]

Andrey V. Savchenko. 2023. MT-EmotiEffNet for Multi-task Human Affective Behavior Analysis and Learning from Synthetic Data. In Proceedings of the European Conference on Computer Vision (ECCV 2022) Workshops. Springer, 45--59. https://arxiv.org/abs/2207.09508

Digital Library

[61]

Melanie Schreiner, Thomas Fischer, and Rene Riedl. 2021. Impact of content characteristics and emotion on behavioral engagement in social media: literature review and research agenda. Electronic Commerce Research, Vol. 21 (2021), 329--345.

Digital Library

[62]

Ozan Sener and Silvio Savarese. 2017. Active learning for convolutional neural networks: A core-set approach. arXiv preprint arXiv:1708.00489 (2017).

[63]

H Sebastian Seung, Manfred Opper, and Haim Sompolinsky. 1992. Query by committee. In Proceedings of the fifth annual workshop on Computational learning theory. 287--294.

Digital Library

[64]

Feifei Shao, Yawei Luo, Ping Liu, Jie Chen, Yi Yang, Yulei Lu, and Jun Xiao. 2022. Active learning for point cloud semantic segmentation via spatial-structural diversity reasoning. In Proceedings of the 30th ACM International Conference on Multimedia. 2575--2585.

Digital Library

[65]

MyungJae Shin, Chihoon Hwang, Joongheon Kim, Jihong Park, Mehdi Bennis, and Seong-Lyun Kim. 2020. Xor mixup: Privacy-preserving data augmentation for one-shot federated learning. arXiv preprint arXiv:2006.05148 (2020).

[66]

Samarth Sinha, Sayna Ebrahimi, and Trevor Darrell. 2019. Variational adversarial active learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 5972--5981.

[67]

Krishna Somandepalli, Hang Qi, Brian Eoff, Alan Cowen, Kartik Audhkhasi, Josh Belanich, and Brendan Jou. 2022. Federated Learning for Affective Computing Tasks. In 2022 10th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 1--8.

[68]

Bo Sun, Siming Cao, Dongliang Li, Jun He, and Lejun Yu. 2020. Dynamic micro-expression recognition using knowledge distillation. IEEE Transactions on Affective Computing, Vol. 13, 2 (2020), 1037--1043.

[69]

Canh T Dinh, Nguyen Tran, and Josh Nguyen. 2020. Personalized federated learning with moreau envelopes. Advances in Neural Information Processing Systems, Vol. 33 (2020), 21394--21405.

[70]

Ye Lin Tun, Kyi Thar, Chu Myaet Thwal, and Choong Seon Hong. 2021. Federated learning based energy demand prediction with clustered aggregation. In 2021 IEEE International Conference on Big Data and Smart Computing (BigComp). IEEE, 164--167.

[71]

Tiffany Tuor, Shiqiang Wang, Bong Jun Ko, Changchang Liu, and Kin K Leung. 2021. Overcoming noisy and irrelevant data in federated learning. In 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 5020--5027.

[72]

Saeed Vahidian, Mahdi Morafah, Weijia Wang, Vyacheslav Kungurtsev, Chen Chen, Mubarak Shah, and Bill Lin. 2022. Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces. arXiv preprint arXiv:2209.10526 (2022).

[73]

Dong Wang and Yu Shang. 2014. A new active labeling method for deep learning. In 2014 International Joint Conference on Neural Networks (IJCNN). IEEE, 112--119.

[74]

Kai Wang, Xiaojiang Peng, Jianfei Yang, Shijian Lu, and Yu Qiao. 2020b. Suppressing uncertainties for large-scale facial expression recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 6897--6906.

[75]

Kaisiyuan Wang, Qianyi Wu, Linsen Song, Zhuoqian Yang, Wayne Wu, Chen Qian, Ran He, Yu Qiao, and Chen Change Loy. 2020c. Mead: A large-scale audio-visual dataset for emotional talking-face generation. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXI. Springer, 700--717.

[76]

Shuo Wang, Yuexiang Li, Kai Ma, Ruhui Ma, Haibing Guan, and Yefeng Zheng. 2020a. Dual adversarial network for deep active learning. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIV 16. Springer, 680--696.

Digital Library

[77]

Zhaoqiang Xia, Xiaopeng Hong, Xingyu Gao, Xiaoyi Feng, and Guoying Zhao. 2019. Spatiotemporal recurrent convolutional networks for recognizing spontaneous micro-expressions. IEEE Transactions on Multimedia, Vol. 22, 3 (2019), 626--640.

[78]

Huiyuan Yang, Umur Ciftci, and Lijun Yin. 2018. Facial expression recognition by de-expression residue learning. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2168--2177.

[79]

Xiaoshan Yang, Baochen Xiong, Yi Huang, and Changsheng Xu. 2022. Cross-Modal Federated Human Activity Recognition via Modality-Agnostic and Modality-Specific Representation Learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 3063--3071.

[80]

Donggeun Yoo and In So Kweon. 2019. Learning loss for active learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 93--102.

[81]

Tehrim Yoon, Sumin Shin, Sung Ju Hwang, and Eunho Yang. 2021. Fedmix: Approximation of mixup under mean augmented federated learning. arXiv preprint arXiv:2107.00233 (2021).

[82]

Naoya Yoshida, Takayuki Nishio, Masahiro Morikura, Koji Yamamoto, and Ryo Yonetani. 2019. Hybrid-fl: Cooperative learning mechanism using non-iid data in wireless networks. arXiv preprint arXiv:1905.07210 (2019).

[83]

Beichen Zhang, Liang Li, Li Su, Shuhui Wang, Jincan Deng, Zheng-Jun Zha, and Qingming Huang. 2020a. Structural semantic adversarial active learning for image captioning. In Proceedings of the 28th ACM International Conference on Multimedia. 1112--1121.

Digital Library

[84]

Beichen Zhang, Liang Li, Shijie Yang, Shuhui Wang, Zheng-Jun Zha, and Qingming Huang. 2020b. State-relabeling adversarial active learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 8756--8765.

[85]

Chunjiong Zhang, Mingyong Li, and Di Wu. 2022a. Federated Multidomain Learning With Graph Ensemble Autoencoder GMM for Emotion Recognition. IEEE Transactions on Intelligent Transportation Systems (2022).

[86]

Feifei Zhang, Tianzhu Zhang, Qirong Mao, and Changsheng Xu. 2018. Joint pose and expression modeling for facial expression recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3359--3368.

[87]

Feifei Zhang, Tianzhu Zhang, Qirong Mao, and Changsheng Xu. 2020c. Geometry guided pose-invariant facial expression recognition. IEEE Transactions on Image Processing, Vol. 29 (2020), 4445--4460.

[88]

Jie Zhang, Zhiqi Li, Bo Li, Jianghe Xu, Shuang Wu, Shouhong Ding, and Chao Wu. 2022b. Federated learning with label distribution skew via logits calibration. In International Conference on Machine Learning. PMLR, 26311--26329.

[89]

Sicheng Zhao, Shangfei Wang, Mohammad Soleymani, Dhiraj Joshi, and Qiang Ji. 2019. Affective computing for large-scale heterogeneous multimedia data: A survey. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Vol. 15, 3s (2019), 1--32.

Digital Library

[90]

Sicheng Zhao, Hongxun Yao, Yue Gao, Rongrong Ji, Wenlong Xie, Xiaolei Jiang, and Tat-Seng Chua. 2016. Predicting personalized emotion perceptions of social images. In Proceedings of the 24th ACM international conference on Multimedia. 1385--1394.

Digital Library

[91]

Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. 2018. Federated learning with non-iid data. arXiv preprint arXiv:1806.00582 (2018).

[92]

Hangyu Zhu, Jinjin Xu, Shiqing Liu, and Yaochu Jin. 2021. Federated learning on non-IID data: A survey. Neurocomputing, Vol. 465 (2021), 371--390.

Digital Library

[93]

Kangkang Zhu, Zhengyin Du, Weixin Li, Di Huang, Yunhong Wang, and Liming Chen. 2019. Discriminative attention-based convolutional neural network for 3D facial expression recognition. In 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019). IEEE, 1--8.

Digital Library

Cited By

Ling TShi SWang HHu CWang DCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Federated Morozov Regularization for Shortcut Learning in Privacy Preserving Learning with Watermarked Image DataProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681480(4899-4908)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681480
Qi FZhang HYang XXu C(2024)A Versatile Multimodal Learning Framework for Zero-Shot Emotion RecognitionIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.336227034:7(5728-5741)Online publication date: 5-Feb-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3362270
Chan CZheng QXu CWang QHeng P(2024)Adaptive Federated Learning for EEG Emotion Recognition2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650004(1-8)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650004

Index Terms

AffectFAL: Federated Active Affective Computing with Non-IID Data
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
2. Information systems
  1. Information systems applications
    1. Multimedia information systems

Recommendations

Affective computing vs. affective placebo

Relaxation training is an application of affective computing with important implications for health and wellness. After detecting user s affective state through physiological sensors, a relaxation training application can provide the user with explicit ...
The Effect of Different Affective Arousal Levels on Taste Perception
ICMI '20 Companion: Companion Publication of the 2020 International Conference on Multimodal Interaction

The emotions we experience shape our perception, and our emotion is shaped by our perceptions. Taste perception is also influenced by emotions. Positive and negative emotions alter sweetness, sourness, and bitterness perception. However, most previous ...
Affective guidance of intelligent agents: How emotion controls cognition

How do emotions and moods color cognition? In this article, we examine how such reactions influence both judgments and cognitive performance. We argue that many affective influences are due, not to affective reactions themselves, but to the information ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

October 2023

9913 pages

ISBN:9798400701085

DOI:10.1145/3581783

General Chairs:
Abdulmotaleb El Saddik
University of Ottawa, Canada & MBZUAI, UAE
,
Tao Mei
HiDream.ai, China
,
Rita Cucchiara
University of Modena and Reggio Emilia, Italy
,
Program Chairs:
Marco Bertini
University of Florence, Italy
,
Diana Patricia Tobon Vallejo
Unversidad de Medellin, Colombia
,
Pradeep K. Atrey
University at Albany, State University of New York, USA
,
M. Shamim Hossain
M. Shamim Hossain (King Saud University, KSA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China-Liaoning Joint Fund
Tianjin Natural Science Foundation
Inner Mongolia Natural Science Foundation

Conference

MM '23

Sponsor:

SIGMM

MM '23: The 31st ACM International Conference on Multimedia

October 29 - November 3, 2023

Ottawa ON, Canada

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
350
Total Downloads

Downloads (Last 12 months)258
Downloads (Last 6 weeks)18

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ling TShi SWang HHu CWang DCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Federated Morozov Regularization for Shortcut Learning in Privacy Preserving Learning with Watermarked Image DataProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681480(4899-4908)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681480
Qi FZhang HYang XXu C(2024)A Versatile Multimodal Learning Framework for Zero-Shot Emotion RecognitionIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.336227034:7(5728-5741)Online publication date: 5-Feb-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3362270
Chan CZheng QXu CWang QHeng P(2024)Adaptive Federated Learning for EEG Emotion Recognition2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650004(1-8)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650004

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents