research-article

Few-Shot Model Agnostic Federated Learning

Authors:

Xiang GaoAuthors Info & Claims

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

Pages 7309 - 7316

https://doi.org/10.1145/3503161.3548764

Published: 10 October 2022 Publication History

Abstract

Federated learning has received increasing attention for its ability to collaborative learning without leaking privacy. Promising advances have been achieved under the assumption that participants share the same model structure. However, when participants independently customize their models, models suffer communication barriers, which leads the model heterogeneity problem. Moreover, in real scenarios, the data held by participants is often limited, making the local models trained only on private data present poor performance. Consequently, this paper studies a new challenging problem, namely few-shot model agnostic federated learning, where the local participants design their independent models from their limited private datasets. Considering the scarcity of the private data, we propose to utilize the abundant public available datasets for bridging the gap between local private participants. However, its usage also brings in two problems: inconsistent labels and large domain gap between the public and private datasets. To address these issues, this paper presents a novel framework with two main parts: 1) model agnostic federated learning, it performs public-private communication by unifying the model prediction outputs on the shared public datasets; 2) latent embedding adaptation, it addresses the domain gap with an adversarial learning scheme to discriminate the public and private domains. Together with theoretical generalization bound analysis, comprehensive experiments under various settings have verified our advantage over existing methods. It provides a simple but effective baseline for future advancement. The code is available at https://github.com/WenkeHuang/FSMAFL.

Supplementary Material

MP4 File (ACMMM22-FSFL-14-Video.mp4)

Presentation video

Download
59.52 MB

References

[1]

Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. 2007. Analysis of representations for domain adaptation. In Advances in neural information processing systems. 137--144.

[2]

John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman. 2007. Learning bounds for domain adaptation. Advances in neural information processing systems 20 (2007), 129--136.

[3]

Debora Caldarola, Massimiliano Mancini, Fabio Galasso, Marco Ciccone, Emanuele Rodolà, and Barbara Caputo. 2021. Cluster-driven graph federated learning over multiple domains. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2749--2758.

[4]

L Elisa Celis and Vijay Keswani. 2019. Improved adversarial learning for fair classification. arXiv preprint arXiv:1901.10443 (2019).

[5]

Yiqiang Chen, Xin Qin, JindongWang, Chaohui Yu, andWen Gao. 2020. Fedhealth: A federated transfer learning framework for wearable healthcare. IEEE Intelligent Systems (2020).

[6]

Gregory Cohen, Saeed Afshar, Jonathan Tapson, and Andre Van Schaik. 2017. EMNIST: Extending MNIST to handwritten letters. In International Joint Conference on Neural Networks. 2921--2926.

[7]

Changying Du, Changde Du, Xingyu Xie, Chen Zhang, and Hao Wang. 2018. Multi-view adversarially learned inference for cross-domain joint distribution matching. In Proceedings of ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1348--1357.

Digital Library

[8]

Xiuwen Fang and Mang Ye. 2022. Robust Federated Learning with Noisy and Heterogeneous Clients. In Proceedings of the IEEE/CVF International Conference on Computer Vision.

[9]

Rui Feng, Yang Yang, Yuehan Lyu, Chenhao Tan, Yizhou Sun, and Chunping Wang. 2019. Learning fair representations via an adversarial framework. arXiv preprint arXiv:1904.13341 (2019).

[10]

Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised domain adaptation by backpropagation. In International conference on machine learning. 1180--1189.

Digital Library

[11]

Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. The journal of machine learning research 17, 1 (2016), 2096--2030.

[12]

Michelle Goddard. 2017. The EU General Data Protection Regulation (GDPR): European regulation that has a global impact. International Journal of Market Research 59, 6 (2017), 703--705.

[13]

Gautham Krishna Gudur, Bala Shyamala Balaji, and Satheesh K Perepu. 2020. Resource-Constrained Federated Learning with Heterogeneous Labels and Models. arXiv preprint arXiv:2011.03206 (2020).

[14]

Andrew Hard, Kanishka Rao, Rajiv Mathews, Swaroop Ramaswamy, Françoise Beaufays, Sean Augenstein, Hubert Eichner, Chloé Kiddon, and Daniel Ramage. 2018. Federated Learning for Mobile Keyboard Prediction. arXiv preprint arXiv:1811.03604 (2018).

[15]

Andrew Hard, Kanishka Rao, Rajiv Mathews, Swaroop Ramaswamy, Françoise Beaufays, Sean Augenstein, Hubert Eichner, Chloé Kiddon, and Daniel Ramage. 2018. Federated learning for mobile keyboard prediction. arXiv preprint arXiv:1811.03604 (2018).

[16]

https://www.mindspore.cn/. 2020. Mindspore. (2020).

[17]

Wenke Huang, Mang Ye, and Bo Du. 2022. Learn from others and be yourself in heterogeneous federated learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision.

[18]

Yutao Huang, Lingyang Chu, Zirui Zhou, Lanjun Wang, Jiangchuan Liu, Jian Pei, and Yong Zhang. 2021. Personalized Cross-Silo Federated Learning on Non-IID Data. In Proceedings of the AAAI Conference on Artificial Intelligence. 7865--7873.

[19]

Jonathan J. Hull. 1994. A database for handwritten text recognition research. IEEE Transactions on Pattern Analysis and Machine Intelligence 16, 5 (1994), 550--554.

Digital Library

[20]

Ce Ju, Ruihui Zhao, Jichao Sun, Xiguang Wei, Bo Zhao, Yang Liu, Hongshan Li, Tianjian Chen, Xinwei Zhang, Dashan Gao, et al. 2020. Privacy-Preserving Technology to Help Millions of People: Federated Prediction Model for Stroke Prevention. arXiv preprint arXiv:2006.10517 (2020).

[21]

Ce Ju, Ruihui Zhao, Jichao Sun, Xiguang Wei, Bo Zhao, Yang Liu, Hongshan Li, Tianjian Chen, Xinwei Zhang, Dashan Gao, et al. 2020. Privacy-preserving technology to help millions of people: Federated prediction model for stroke prevention. arXiv preprint arXiv:2006.10517 (2020).

[22]

Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. 2019. Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977 (2019).

[23]

Jakub KonečnỴ, H Brendan McMahan, Daniel Ramage, and Peter Richtárik. 2016. Federated Optimization: Distributed Machine Learning for On-Device Intelligence. arXiv preprint arXiv:1610.02527 (2016).

[24]

Jakub KonečnỴ, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. 2016. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492 (2016).

[25]

Jakub KonečnỴ, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. 2016. Federated Learning: Strategies for Improving Communication Efficiency. arXiv preprint arXiv:1610.05492 (2016).

[26]

Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradientbased learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278--2324.

[27]

Daliang Li and Junpu Wang. 2019. Fedmd: Heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581 (2019).

[28]

Qinbin Li, Bingsheng He, and Dawn Song. 2021. Model-Contrastive Federated Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10713--10722.

[29]

Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. 2020. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine 37, 3 (2020), 50--60.

[30]

Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, and Qi Dou. 2021. Fed{BN}: Federated Learning on Non-{IID} Features via Local Batch Normalization. In International Conference on Learning Representations.

[31]

Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B Allen, Randy P Auerbach, David Brent, Ruslan Salakhutdinov, and Louis-Philippe Morency. 2020. Think locally, act globally: Federated learning with local and global representations. In Advances in Neural Information Processing Systems Workshop.

[32]

Edvin Listo Zec, Olof Mogren, John Martinsson, Leon René Sütfeld, and Daniel Gillblad. 2020. Federated learning using a mixture of experts. arXiv e-prints (2020), arXiv--2010.

[33]

Quande Liu, Cheng Chen, Jing Qin, Qi Dou, and Pheng-Ann Heng. 2021. FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1013--1023.

[34]

Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics. 1273--1282.

[35]

Payman Mohassel and Yupeng Zhang. 2017. Secureml: A system for Scalable Privacy-Preserving Machine Learning. In SSP. 19--38.

[36]

Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. 2011. Reading digits in natural images with unsupervised feature learning. (2011).

[37]

Xingchao Peng, Zijun Huang, Yizhe Zhu, and Kate Saenko. 2019. Federated Adversarial Domain Adaptation. In International Conference on Learning Representations.

[38]

Xingchao Peng, Zijun Huang, Yizhe Zhu, and Kate Saenko. 2019. Federated adversarial domain adaptation. arXiv preprint arXiv:1911.02054 (2019).

[39]

Daniel Peterson, Pallika Kanani, and Virendra J Marathe. 2019. Private federated learning with domain adaptation. arXiv preprint arXiv:1912.06733 (2019).

[40]

Frank Rosenblatt. 1961. Principles of neurodynamics. perceptrons and the theory of brain mechanisms. Technical Report. Cornell Aeronautical Lab Inc Buffalo NY.

[41]

Prasun Roy, Subhankar Ghosh, Saumik Bhattacharya, and Umapada Pal. 2018. Effects of degradations on deep neural network architectures. arXiv preprint arXiv:1807.10108 (2018).

[42]

Tao Shen, Jie Zhang, Xinkang Jia, Fengda Zhang, Gang Huang, Pan Zhou, FeiWu, and Chao Wu. 2020. Federated Mutual Learning. arXiv preprint arXiv:2006.16765 (2020).

[43]

Benyuan Sun, Hongxing Huo, Yi Yang, and Bo Bai. 2021. PartialFed: Cross- Domain Personalized Federated Learning via Partial Initialization. Advances in Neural Information Processing Systems 34 (2021).

[44]

Canh T. Dinh, Nguyen Tran, and Josh Nguyen. 2020. Personalized Federated Learning with Moreau Envelopes. In Advances in Neural Information Processing Systems. 21394--21405.

[45]

Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7167--7176.

[46]

Vladimir Vapnik and Vlamimir Vapnik. 1998. Statistical learning theory Wiley. New York 1 (1998), 624.

[47]

Omar AbdelWahab, Azzam Mourad, Hadi Otrok, and Tarik Taleb. 2021. Federated machine learning: Survey, multi-level classification, desirable criteria and future directions in communication and networking systems. IEEE Communications Surveys & Tutorials 23, 2 (2021), 1342--1397.

[48]

Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. 2019. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology 10, 2 (2019), 1--19.

Digital Library

[49]

Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. 2019. Federated Machine Learning: Concept and Applications. ACM Transactions on Intelligent Systems and Technology (2019), 1--19.

Digital Library

[50]

Mang Ye and Jianbing Shen. 2020. Probabilistic Structural Latent Representation for Unsupervised Embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[51]

Mang Ye, Jianbing Shen, Xu Zhang, Pong C Yuen, and Shih-Fu Chang. 2020. Augmentation Invariant and Instance Spreading Feature for Softmax Embedding. IEEE Transactions on Pattern Analysis and Machine Intelligence (2020).

[52]

Edvin Listo Zec, John Martinsson, Olof Mogren, Leon René Sütfeld, and Daniel Gillblad. 2020. Specialized Federated Learning Using Mixture of Experts. arXiv preprint arXiv:2010.02056 (2020).

[53]

Yabin Zhang, Bin Deng, Hui Tang, Lei Zhang, and Kui Jia. 2020. Unsupervised multi-class domain adaptation: Theory, algorithms, and practice. IEEE Transactions on Pattern Analysis and Machine Intelligence (2020).

[54]

Sicheng Zhao, Xiangyu Yue, Shanghang Zhang, Bo Li, Han Zhao, Bichen Wu, Ravi Krishna, Joseph E Gonzalez, Alberto L Sangiovanni-Vincentelli, Sanjit A Seshia, et al. 2020. A review of single-source deep unsupervised visual domain adaptation. IEEE Transactions on Neural Networks and Learning Systems (2020).

[55]

Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. 2018. Federated learning with non-iid data. arXiv preprint arXiv:1806.00582 (2018).

[56]

Weiming Zhuang, Yonggang Wen, Xuesen Zhang, Xin Gan, Daiying Yin, Dongzhan Zhou, Shuai Zhang, and Shuai Yi. 2020. Performance Optimization of Federated Person Re-identification via Benchmark Analysis. In ACM Multimedia. 955--963.

Cited By

HOSHINO YKAWAKAMI HMATSUTANI H(2024)Federated Learning of Neural ODE Models with Different Iteration CountsIEICE Transactions on Information and Systems10.1587/transinf.2023EDP7176E107.D:6(781-791)Online publication date: 1-Jun-2024
https://doi.org/10.1587/transinf.2023EDP7176
Zeng HXu MZhou TWu XKang JCai ZNiyato DCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)One-shot-but-not-degraded Federated LearningProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680715(11070-11079)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680715
Sivasubramanian DNagalapatti LIyer RRamakrishnan G(2024)Gradient Coreset for Federated Learning2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV57701.2024.00263(2636-2645)Online publication date: 3-Jan-2024
https://doi.org/10.1109/WACV57701.2024.00263
Show More Cited By

Index Terms

Few-Shot Model Agnostic Federated Learning
1. Computing methodologies
  1. Distributed computing methodologies
    1. Distributed algorithms

Recommendations

Multimodal federated learning: Concept, methods, applications and future directions
Abstract
Multimodal learning mines and analyzes multimodal data in reality to better understand and appreciate the world around people. However, how to exploit this rich multimodal data without violating user privacy is a key issue. Federated learning is ...
Highlights
- The three different modes in the multimodal federated learning model are summarized.
- Multimodal fusion based on the federated learning framework is also specified.
- The difficulties and some ideas of multimodal federated learning ...
A review of federated learning: taxonomy, privacy and future directions
Abstract
The data generated and stored in mobile devices owned by individuals as well as in various organizations contains a large amount of valuable and important information that can be used to improve service quality, user experience, and satisfaction. ...
Federated Few-Shot Learning for Mobile NLP
ACM MobiCom '23: Proceedings of the 29th Annual International Conference on Mobile Computing and Networking

Natural language processing (NLP) sees rich mobile applications. To support various language understanding tasks, a foundation NLP model is often fine-tuned in a federated, privacy-preserving setting (FL). This process currently relies on at least ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

October 2022

7537 pages

ISBN:9781450392037

DOI:10.1145/3503161

General Chairs:
João Magalhães
NOVA University of Lisbon, Portugal
,
Alberto del Bimbo
University of Florence, Italy
,
Shin'ichi Satoh
National Institute of Informatics, Japan
,
Nicu Sebe
University of Trento, Italy
,
Program Chairs:
Xavier Alameda-Pineda
Inria, Grenoble, France
,
Qin Jin
Renmin University of China, China
,
Vincent Oria
New Jersey Institute of Technology, USA
,
Laura Toni
University College London, UK

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

CCF-NSFOCUS Kun-Peng Scientific Research Fund
National Natural Science Foundation of China
The Science and Technology Major Project of Hubei Province (Next-Generation AI Technologies)
The Key Research and Development Program of Hubei Province

Conference

MM '22

Sponsor:

SIGMM

MM '22: The 30th ACM International Conference on Multimedia

October 10 - 14, 2022

Lisboa, Portugal

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
985
Total Downloads

Downloads (Last 12 months)335
Downloads (Last 6 weeks)28

Reflects downloads up to 25 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

HOSHINO YKAWAKAMI HMATSUTANI H(2024)Federated Learning of Neural ODE Models with Different Iteration CountsIEICE Transactions on Information and Systems10.1587/transinf.2023EDP7176E107.D:6(781-791)Online publication date: 1-Jun-2024
https://doi.org/10.1587/transinf.2023EDP7176
Zeng HXu MZhou TWu XKang JCai ZNiyato DCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)One-shot-but-not-degraded Federated LearningProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680715(11070-11079)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680715
Sivasubramanian DNagalapatti LIyer RRamakrishnan G(2024)Gradient Coreset for Federated Learning2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV57701.2024.00263(2636-2645)Online publication date: 3-Jan-2024
https://doi.org/10.1109/WACV57701.2024.00263
Huang WYe MShi ZWan GLi HDu BYang Q(2024)Federated Learning for Generalization, Robustness, Fairness: A Survey and BenchmarkIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.341886246:12(9387-9406)Online publication date: Dec-2024
https://doi.org/10.1109/TPAMI.2024.3418862
Huang WYe MShi ZDu B(2024)Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity LearningIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.332737346:2(712-728)Online publication date: Feb-2024
https://doi.org/10.1109/TPAMI.2023.3327373
Ma RShi HGao HGuan HIqbal MMumtaz S(2024)cFedDT: Cross-Domain Federated Learning in Digital Twins for Metaverse Consumer Electronic ProductsIEEE Transactions on Consumer Electronics10.1109/TCE.2023.332701070:1(3167-3182)Online publication date: Feb-2024
https://doi.org/10.1109/TCE.2023.3327010
Liu TPan SLi P(2024)Exploring Server-Side Data in Federated Learning: An Empirical Study2024 IEEE/CIC International Conference on Communications in China (ICCC)10.1109/ICCC62479.2024.10681848(1379-1384)Online publication date: 7-Aug-2024
https://doi.org/10.1109/ICCC62479.2024.10681848
Zhang YQi YQi XSenhadji LWei YChen FYang G(2024)Fedsoda: Federated Cross-Assessment and Dynamic Aggregation for Histopathology SegmentationICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10447912(1656-1660)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10447912
Yang XHuang WYe M(2024)FedAS: Bridging Inconsistency in Personalized Federated Learning2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01139(11986-11995)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01139
Jiang YCheng YWang DSong B(2024)RRA-FFSCIL: Inter-intra classes representation and relationship augmentation federated few-shot incremental learningNeurocomputing10.1016/j.neucom.2024.127956597(127956)Online publication date: Sep-2024
https://doi.org/10.1016/j.neucom.2024.127956
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents