Abstract
Bad disks are replaced as the disks in the storage system are iteratively updated. However, there are few reference data for minority class disks, and it is difficult to make good failure prediction using traditional machine learning methods. Aiming at the problem of low recognition rate due to insufficient number of samples of a few types of disks in large-scale storage systems, a method for predicting disk failures based on transfer learning is proposed. First, we select the disk data of different models with a large number of samples, use the maximum mean difference as the standard to select the disk model data with small distribution difference as the source domain, use the selected source domain to train the feature extraction network and transfer the pretrained model to the target domain for failure prediction. Experimental results show that the proposed method can improve the failure prediction ability in the case of a few types of disks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ghemawat, S., Gobioff, H., Leung, S.T.: The google file system. In: Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, pp. 29–43 (2003)
Borthakur, D.: The Hadoop Distributed File System: Architecture and Design (2007)
Hughes, G., Murray, J., Kreutz-Delgado, K., Elkan, C.: Improved disk-drive failure warnings. IEEE Trans. Reliab. 51(3), 350–357 (2002)
Wang, Y., Miao, Q., Ma, E.W., Tsui, K.L., Pecht, M.G.: Online anomaly detection for hard disk drives based on Mahalanobis distance. IEEE Trans. Reliab. 62(1), 136–145 (2013)
Hamerly, G., Elkan, C.: Bayesian Approaches to Failure Prediction for Disk Drives (2003)
Zhu, B., et al.: Proactive drive failure prediction for large scale storage systems. In: Mass Storage Systems and Technologies (2013)
Ying, J., et al.: Optimization and choice of hard drive failure prediction models based on adaboost and genetic algorithm, p. 7 (2014)
Lima, F., Amaral, G., Leite, L., Gomes, J., Machado, J.: Predicting failures in hard drives with LSTM networks. In: Brazilian Conference on Intelligent Systems, pp. 222–227 (2017)
Long, K.: Research on hard disk failure prediction technology based on deep learning. Ph.D. Thesis, Xidian University (2019)
Peng, L., et al.: Disk failure prediction model based on adaptive weighted bagging-GBDT algorithm under imbalanced dataset. Microelectron. Comput. 37(3), 14–19 (2020)
Xin, L., Fei, T.: Disk failure prediction and characteristic analysis based on xgboost. J. Chifeng Univ. Nat. Sci. Ed. 37(11), 12–18 (2021)
Bin, Z., Yue, L.: Application of disk failure prediction of domestic artificial intelligence platform. Electronic World, pp. 19–20 (2021)
Ling, D., Zhen, S., Zhi, M.: Prediction method of hard disk remaining service life based on data screening. Comput. Eng. Des. 41(8), 2252–2258 (2020)
Yong, D., Huang, J., Tong, L., Qiang, Z.: Comparison of machine learning methods for disk failure prediction. Comput. Eng. Sci. 37(12), 2200–2207 (2015)
Sheng, L., et al.: Image recognition of Camellia oleifera diseases based on convolutional neural network and transfer learning. Trans. Chin. Soc. Agric. Eng. 34(18), 194–201 (2018)
Ting, S., et al.: Application of deep transfer learning in image recognition of peanut leaf diseases. J. Shandong Agric. Univ. Nat. Sci. Ed. 50(5), 5 (2019)
Pei, S.: Intelligent analysis of pathological characteristics of CT images of pulmonary nodules and research on key technologies of information retrieval based on image features (2017)
Hui, F., et al.: Automatic identification of small intestinal polyps in wireless capsule endoscopy images. Chin. J. Biomed. Eng. 38(5), 522–532 (2019)
Feng, H., Chao, Z., Feng, Z., Zhen, C.: Bamboo chip defect recognition based on transfer learning. J. Northwest Forest. Univ. 36(5), 190–196 (2021)
Yan, S., et al.: Small-sample day-ahead power load forecasting for integrated energy systems based on feature transfer learning. Control Theory Appl. (2021)
Acknowledgements
The research work in this paper was supported by the Shandong Provincial Natural Science Foundation of China (Grant No. ZR2019LZH003). Peng Wu is the author to whom all correspondence should be addressed.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gao, G., Wu, P., Li, H., Zhang, T. (2022). Disk Failure Prediction Based on Transfer Learning. In: Huang, DS., Jo, KH., Jing, J., Premaratne, P., Bevilacqua, V., Hussain, A. (eds) Intelligent Computing Theories and Application. ICIC 2022. Lecture Notes in Computer Science, vol 13394. Springer, Cham. https://doi.org/10.1007/978-3-031-13829-4_54
Download citation
DOI: https://doi.org/10.1007/978-3-031-13829-4_54
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-13828-7
Online ISBN: 978-3-031-13829-4
eBook Packages: Computer ScienceComputer Science (R0)