Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Uncertainty-aware enhanced dark experience replay for continual learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The replay-based approaches are a notable family of methods among many efforts on Continual Learning, where memory sampling strat- egy and rehearsal mode are two fundamental aspects to alleviate the catastrophic forgetting. However, most existing replay-based approaches focus primarily on exploring the rehearsal mode but neglect the sig- nificant influence of the sampling strategy, especially failing to ade- quately utilize the inherent attributes in samples and the information provided by the old task model. To this end, we propose a novel sam- pling strategy dubbed Uncertainty-Aware Sampling (UAS) strategy, which employs model and data uncertainties as criteria to select sam- ples that are stable to the model and have low noise for rehearsal. Further, we design a dual network to acquire new knowledge while maintaining old knowledge, in which a Convolutional Neural Network (CNN) is applied to continuously learn and consolidate knowledge, and a Bayesian Neural Network (BNN) is employed as a comple- ment to capture the uncertainty while providing additional information for the CNN. Besides, we incorporate a data uncertainty loss into Dark Experience Replay as rehearsal mode to alleviate the catas- trophic forgetting in both CNN and BNN, called Uncertainty-Aware Replay (UAR). Extensive experiments on four benchmark datasets demonstrate that the proposed framework is competitive with state- of-the-art methods under three different continual learning settings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The datasets analysed during the current study are available from the corresponding author on reasonable request.

Code availability

The code for this article will be available after the article is accepted or from the corresponding author on reasonable request.

References

  1. De Lange M, Aljundi R, Masana M, Parisot S, Jia X, Leonardis A, Slabaugh G, Tuytelaars T (2022) A continual learning survey: Defying forgetting in classification tasks. IEEE Trans Pattern Anal Mach Intell 44(7):3366–3385

    Google Scholar 

  2. Zhao K, Fu Z, Yang J (2023) Continual learning via region-aware memory. Appl Intell 53:8389–8401

    Article  Google Scholar 

  3. Fu Y, Cao H, Chen X, Ding J (2022) Task-incremental broad learn- ing system for multi-component intelligent fault diagnosis of machinery. Knowl-Based Syst 246:108730–108744

    Article  Google Scholar 

  4. Li D, Liu S, Gao F, Sun X (2022) Continual learning classification method and its application to equipment fault diagnosis. Appl Intell 52:858–874

    Article  Google Scholar 

  5. Li D, Gu M, Liu S, Sun X, Gong L, Qian K (2022) Continual learning classification method with the weighted k-nearest neighbor rule for time- varying data space based on the artificial immune system. Knowl-Based Syst 240:108145–108160

    Article  Google Scholar 

  6. McCloskey M, Cohen NJ (1989) Catastrophic interference in connectionist networks: The sequential learning problem. Psychol Learn Motiv 24:109–165

    Article  Google Scholar 

  7. Jiang M, Li F, Li L (2022) Continual meta-learning algorithm. Appl Intell 52:4527–4542

    Article  Google Scholar 

  8. Jonathan S, Wojciech C, Jelena L, Agnieszka G-B, Yee WT, Razvan P, Raia H (2018) Progress & compress: a scalable framework for continual learning. In: International conference on machine learning pp 4528–4537

  9. Li Z, Hoiem D (2017) Learning without forgetting. Transactions on pattern analysis and machine intelligence. IEEE 40(12):2935–2947

    Article  Google Scholar 

  10. Zenke F, Poole B, Ganguli S (2017) Continual learning through synaptic intelligence. In: International conference on machine learning pp 3987–3995

  11. Rusu AA, Rabinowitz NC, Desjardins G, Soyer H, Kirkpatrick J, Kavukcuoglu K, Pascanu R, Hadsell R (2016) Progressive neural networks. ArXiv preprint. https://arxiv.org/abs/1606.04671

  12. Serra J, Suris D, Miron M, Karatzoglou A(2018) Overcoming catastrophic forgetting with hard attention to the task. In: International conference on machine learning pp 4548–4557

  13. Ke Z, Liu B, Huang X (2020) Continual learning of a mixed sequence of similar and dissimilar tasks. In: Advances in neural information processing systems pp 18493–18504

  14. Dekhovich A, Tax DMJ, Sluiter MHF, Bessa MA (2023) Continual prune-and-select: class-incremental learning with specialized subnetworks. Appl Intell 1–18

  15. Abati D, Tomczak JM, Blankevoort T, Calderara S, Cucchiara R, Bejnordi BE (2020) Conditional channel gated networks for task-aware continual learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition pp 3931–3940

  16. Aljundi R, Lin M, Goujaud B, Bengio Y (2019) Gradient based sample selection for online continual learning. In: Advances in neural information processing systems pp 11816–11825

  17. Chaudhry A, Ranzato MA, Rohrbach M, Elhoseiny M (2019) Efficient lifelong learning with A-GEM. In: International conference on learning representations pp 1–20

  18. Rebuffi S-A, Kolesnikov A, Sperl G, Lampert CH (2017) Icarl: incremental classifier and representation learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition pp 2001–2010

  19. Benjamin A, Rolnick D, Kording K (2018) Measuring and regularizing networks in function space. In: International conference on learning representations pp 1–18

  20. Chaudhry A, Gordo A, Dokania PK, Torr P, Lopez-Paz D (2020) Using hindsight to anchor past knowledge in continual learning. arXiv preprint arXiv:2002.08165

  21. Buzzega P, Boschini M, Porrello A, Abati D, Calderara S (2020) Dark experience for general continual learning: a strong, simple baseline. In: Advances in neural information processing systems pp 1–24

  22. Ji Z, Liu J, Wang Q, Zhang Z (2021) Coordinating experience replay: A harmonious experience retention approach for continual learning. Knowl-Based Syst 234:107589–107601

    Article  Google Scholar 

  23. Buzzega P, Boschini M, Porrello A, Calderara S (2021) Rethinking experience replay: a bag of tricks for continual learning. In: Proceedings of the international conference on pattern recognition pp 2180–2187

  24. Quang P, Chenghao L, Steven H (2021) DualNet: continual learning, fast and slow. In: Proceedings of the advances in neural information processing systems pp 16131–16144

  25. Simon C, Koniusz P, Harandi M (2021) On learning the geodesic path for incremental learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition pp 1591–1600

  26. Aljundi R, Belilovsky E, Tuytelaars T, Charlin L, Caccia M, Lin M, Page-Caccia L (2019) Online continual learning with maximal interfered retrieval. In: Proceedings of the advances in neural information processing systems pp 1–15

  27. Gal Y (2016) Uncertainty in deep learning. PhD thesis, University of Cambridge

  28. Mobiny A, Yuan P, Moulik SK, Garg N, Wu CC, Van Nguyen H (2021) Dropconnect is effective in modeling uncertainty of bayesian deep networks. Sci Rep 11(1):1–14

    Article  Google Scholar 

  29. Robins A (1995) Catastrophic forgetting, rehearsal and pseudorehearsal. Con- nection Science 7(2):123–146

    Article  Google Scholar 

  30. MacKay DJC (1992) A practical bayesian framework for backpropagation networks. Neural Comput 4(3):448–472

    Article  Google Scholar 

  31. Blundell C, Cornebise J, Kavukcuoglu K, Wierstra D (2015) Weight uncertainty in neural network. In: International conference on machine learning pp 1613–1622

  32. Hernandez-Lobato J, Li Y, Rowland M, Bui T, Hernández-Lobato D, Turner R (2016) Black-box alpha divergence minimization. In: International conference on machine learning pp 1511–1520

  33. Zhang Z, Lan C, Zeng W, Chen Z, Chang S-F (2020) Uncertainty-aware few-shot image classification. Int Joint Conf Artif Intell pp 1–9

  34. Mukherjee S, Awadallah AH (2019) Uncertainty-aware self-training for text classification with few labels. In: Advances in neural information processing systems pp 1–14

  35. Nguyen CV, Li Y, Bui TD, Turner RE (2018) Variational continual learning. In: International conference on learning representations pp 1–18

  36. Kochurov M, Garipov T, Podoprikhin D, Molchanov D, Ashukha A, Vetrov DP (2018) Bayesian incremental learning for deep neural networks. ArXiv preprint. https://arxiv.org/abs/1802.07329

  37. Kurmi VK, Patro BN, Subramanian VK, Namboodiri VP (2021) Do not forget to attend to uncertainty while mitigating catastrophic forgetting. In: 2021 IEEE winter conference on applications of computer vision pp 736–745

  38. Ebrahimi S, Elhoseiny M, Darrell T, Rohrbach M (2020) Uncertainty-guided continual learning with bayesian neural networks. In: International conference on learning representations pp 1–16

  39. Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 1–30

  40. Houlsby N, Huszár F, Ghahramani Z, Lengyel M (2011) Bayesian active learning for classification and preference learning. ArXiv preprint. https://arxiv.org/abs/1112.5745

  41. Gal Y, Islam R, Ghahramani Z (2017) Deep bayesian active learning with image data. In: International conference on machine learning pp 1183–1192

  42. Van de Ven GM, Tolias AS (2019) Three scenarios for continual learning. ArXiv preprint. https://arxiv.org/abs/1904.07734

  43. Krizhevsky A, Hinton G et al (2009) Learning multiple layers of features from tiny images. Citeseer

  44. Stanford (2015) Tiny imagenet challenge (CS231n). https://tiny-imagenet.herokuapp.com

  45. Delange M, Aljundi R, Masana M, Parisot S, Jia X, Leonardis A, Slabaugh G, Tuytelaars T (2021) Continual learning: A comparative study on how to defy forgetting in classification tasks. IEEE Trans Pattern Anal Mach Intell 42(3):99

    Google Scholar 

  46. He KM, Zhang XY, Ren SQ, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition pp 770–778

  47. Cha H, Lee J, Shin J (2021) Co2L: contrastive continual learning. In: Proceedings of the IEEE/CVF International conference on computer vision (ICCV) pp 9516–9525

  48. Vitter JS (1985) Random sampling with a reservoir. ACM Transactions on Mathematical Software 11(1):37–57

    Article  MathSciNet  Google Scholar 

Download references

Funding

This work was supported by the National Key Research and Development Program of China (Grant No. 2022ZD0160403) and the National Natural Science Foundation of China (Grant No. 62176178).

Author information

Authors and Affiliations

  1. School of Electrical and Information Engineering, Tianjin University, Tianjin, 300072, China

    Qiang Wang, Zhong Ji & Yanwei Pang

  2. Shanghai Artificial Intelligence Laboratory, Shanghai, 200232, China

    Qiang Wang, Zhong Ji & Yanwei Pang

  3. Computer Science Department, Binghamton University, NY, 13902, USA

    Zhongfei Zhang

Authors
  1. Qiang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhong Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanwei Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhongfei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study concep- tion and design. Qiang Wang: Methodology, Writing, Software. Zhong Ji: Conceptualization, Writing, Funding acquisition. Yanwei Pang: Conceptual- ization, Writing review and editing. Zhongfei Zhang: Methodology, Writing review and editing.

Corresponding author

Correspondence to Zhong Ji.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Q., Ji, Z., Pang, Y. et al. Uncertainty-aware enhanced dark experience replay for continual learning. Appl Intell 54, 7135–7150 (2024). https://doi.org/10.1007/s10489-024-05488-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05488-w

Keywords

Search

Navigation