Abstract
A Human can immediately add new items to its set of known objects, whereas a computer, using traditional computer vision algorithms, would typically have to go almost back to the start and re-learn the all collection of objects (classes) from scratch. The reason the network must be re-trained is due to a phenomenon named Catastrophic Forgetting, where the changes made to the system during the acquisition of new knowledge brings about the loss of previous knowledge. In this paper, we explore the Continual Learning problem by proposing a way to deal with Catastrophic Forgetting. Our proposal includes a framework capable of learning new information without having to start from scratch and even “improve” its knowledge on what it already knows. With the above in mind, we present the Modular Dynamic Neural Network (MDNN), a network primarily made up of modular sub-networks that progressively grows in a tree shape and re-arranges itself as it learns continuously. The network is divided into two main blocks: (a) the feature extraction block, which is based on a ResNet50; and (b) the modular dynamic classification block, which is made up of sub-networks structured in such a way that its internal components function independently from one another. This structure allows that when new information is learned only specific sub-networks are altered in a way that old information is not forgotten. Tests show promising results with a set of ImageNet classes and also with a set of our own classes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
McCarthy, J.: Programs with common sense. In: RLE and MIT Computation Center (1960). http://jmc.stanford.edu/articles/mcc59.html. Accessed 30 Nov 2020
Parisi, G.I., Kemker, R., Part, J.L., Kanan, C., Wermter, S.: Continual lifelong learning with neural networks: a review. Neural Netw. 113, 54–71 (2019)
Ebrahimi, S., Meier, F., Calandra, R., Darrell, T., Rohrbach, M.: Adversarial continual learning. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12356, pp. 386–402. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58621-8_23
Aljundi, R., Rohrbach, M., Tuytelaars, T.: Selfless sequential learning. In: Proceedings of 7th International Conference on Learning Representations, arXiv preprint arXiv:1806.05421 (2019)
Chen, C.P., Liu, Z.: Broad learning system: an effective and efficient incremental learning system without the need for deep architecture. IEEE Trans Neural Netw. Learn. Syst. 29(1), 10–24 (2017)
De Lange, M., et al.: Continual learning: a comparative study on how to defy forgetting in classification tasks. arXiv preprint arXiv:1909.08383 (2019)
She, Q., et al.: OpenLORIS-Object: a dataset and benchmark towards lifelong object recognition. In: Proceedings of International Conference on Robotics and Automation, pp. 4767–4773 (2020)
Lomonaco, V., Maltoni, D.: CORe50: a new dataset and benchmark for continuous object recognition. In: Proceedings of 1st Conference on Robot Learning, in PMLR, vol. 78, pp. 17–26 (2017)
Requeima, J., Gordon, J., Bronskill, J., Nowozin, S., Turner, R.E.: Fast and flexible multi-task classification using conditional neural adaptive processes. Adv. Neural Inf. Process. Syst. 33, 7957–7968 (2019)
Rebuffi, S.A., Kolesnikov, A., Sperl, G., Lampert, C.H.: iCaRL: incremental classifier and representation learning. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017)
Mallya, A., Lazebnik, S.: PackNet: adding multiple tasks to a single network by iterative pruning. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018)
van de Ven, G.M., Tolias, A.S.: Three continual learning scenarios and a case for generative replay In: Proceedings of 7th International Conference on Learning Representations, arXiv preprint arXiv:1904.07734 (2019)
Pellegrini, L., Graffieti, G., Lomonaco, V., Maltoni, D.: Latent replay for real-time continual learning. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, arXiv preprint arXiv:1912.01100 (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Sharma, N., Jain, V., Mishra, A.: An analysis of convolutional neural networks for image classification. Procedia Comput. Sci. 132, 377–384 (2018)
Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., Fei-Fei, L.:. ImageNet: a large-scale hierarchical image database. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of International Conference on Learning Representations arXiv preprint arXiv:1409.1556 (2015)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)
Tan, M., Le, Q.V.: EfficientNet: rethinking model scaling for convolutional neural networks. In: Proceedings of 26th International Conference on Machine Learning, arXiv preprint arXiv:1905.11946 (2019)
Fang, W., et al.: Recognizing global reservoirs from landsat 8 images: a deep learning approach. IEEE J. Sel. Top. Appl. Earth Obser. Remote Sens. 12(9), 3168–3177 (2019)
Wan, A., et al.: NBDT: neural-backed decision trees. arXiv preprint arXiv:2004.00221 (2020)
Hensman, P., Masko, D.: The impact of imbalanced training data for convolutional neural networks. Degree Project in Computer Science, KTH Royal Institute of Technology (2015)
Acknowledgements
This work was supported by the Portuguese Foundation for Science and Technology (FCT), project LARSyS - FCT Project UIDB/50009/2020 and project WELSAFE.DV (AAC 15/SI/2020) Portugal 2020, CRESC 2020, FEDER.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Turner, D., Cardoso, P.J.S., Rodrigues, J.M.F. (2021). Continual Learning for Object Classification: A Modular Approach. In: Antona, M., Stephanidis, C. (eds) Universal Access in Human-Computer Interaction. Access to Media, Learning and Assistive Environments. HCII 2021. Lecture Notes in Computer Science(), vol 12769. Springer, Cham. https://doi.org/10.1007/978-3-030-78095-1_39
Download citation
DOI: https://doi.org/10.1007/978-3-030-78095-1_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-78094-4
Online ISBN: 978-3-030-78095-1
eBook Packages: Computer ScienceComputer Science (R0)