Abstract
In this study, state-of-the-art AI models are employed to classify aerospace maintenance records into categories based on the fault descriptions of avionic components. The classification is performed using short natural language text descriptions provided by specialised repair engineers. The primary goal is to conduct a more comprehensive analysis of a complex and lengthy maintenance dataset, with the objective of determining the likelihood of failures in non-critical components of airplanes.
Various methodologies are used, including two vectorisation models to natural language representation, as well as several machine learning algorithms such as BiGRU and BiLSTM, to identify repair and replacement event likelihood from the provided corpora. The resulting performance of the deployed models provide a very high F1 score overall, indicating models’ ability to learn repair patterns from the, typically complex, engineering description of components with high confidence.
Two case studies are conducted. The first for a binary classification, with several models achieving an average F1 score of around \(95\%\). In the second case, a multi-class classification is performed for four different classes, with the BiLSTM model achieving the highest performance, accurately predicting the validation set with a \(95.2\%\) F1 score.
The misclassified samples were manually inspected, and it was found that in many cases, the relevant information was simply missing from the text due to errors or omissions by description authors. Only \(12\%\) of the misclassified samples were found to be due to errors made by the model, resulting in an effective accuracy rate of \(\sim 99.4\%\).
Supported by GE Aerospace: www.geaerospace.com.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
References
Agovic, A., Shan, H., Banerjee, A.: Analyzing aviation safety reports: from topic modeling to scalable multi-label classification. In: Proceedings of the 2010 Conference on Intelligent Data Understanding, CIDU 2010, pp. 83–97. Citeseer (2010)
Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017). https://doi.org/10.1162/tacl_a_00051
Candell, O., Karim, R., Söderholm, P.: eMaintenance-Information logistics for maintenance support (2009). https://doi.org/10.1016/j.rcim.2009.04.005
Charte, F., Rivera, A.J., Del Jesus, M.J., Herrera, F.: MLSMOTE: approaching imbalanced multilabel learning through synthetic instance generation. Knowl. Based Syst. 89, 385–397 (2015). https://doi.org/10.1016/j.knosys.2015.07.019
Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002). https://doi.org/10.1613/jair.953, http://arxiv.org/abs/1106.1813
Devaney, M., Ram, A., Qiu, H., Lee, J.: Preventing failures by mining maintenance logs with case-based reasoning (2005)
Do, B.H., Wu, A.S., Maley, J., Biswal, S.: Automatic retrieval of bone fracture knowledge using natural language processing. J. Digit. Imaging 26(4), 709–713 (2013). https://doi.org/10.1007/s10278-012-9531-1
Elhadad, N., Zhang, S., Driscoll, P., Brody, S.: Characterizing the sublanguage of online breast cancer forums for medications, symptoms, and emotions. In: AMIA ... Annual Symposium Proceedings / AMIA Symposium. AMIA Symposium 2014, pp. 516–525 (2014). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4419934/
Ford, E., Carroll, J.A., Smith, H.E., Scott, D., Cassell, J.A.: Extracting information from the text of electronic medical records to improve case detection: a systematic review. J. Am. Med. Inform. Assoc. 23(5), 1007–1015 (2016). https://doi.org/10.1093/jamia/ocv180
Grivel, L.: Customer feedbacks and opinion surveys analysis in the automotive industry. text mining and its applications to intelligence. CRM Knowl. Manage. 249–257 (2005). https://doi.org/10.2495/978-1-85312-995-7/13
Heinze, D.T., Morsch, M.L., Holbrook, J.: Mining free-text medical records. Proceedings. In: AMIA Symposium, pp. 254–258 (2001). https://doi.org/10.1016/j.procir.2019.02.098
Jensen, K., et al.: Analysis of free text in electronic health records for identification of cancer patient trajectories. Sci. Rep. 7(1), 46226 (2017). https://doi.org/10.1038/srep46226
Kang, N., Singh, B., Afzal, Z., van Mulligen, E.M., Kors, J.A.: Using rule-based natural language processing to improve disease normalization in biomedical text. J. Am. Med. Inform. Assoc. 20(5), 876–881 (2013). https://doi.org/10.1136/amiajnl-2012-001173
Lucini, F.R., et al.: Text mining approach to predict hospital admissions using early medical records from the emergency department. Int. J. Med. Inform. 100, 1–8 (2017). https://doi.org/10.1016/j.ijmedinf.2017.01.001
Lyall-Wilson, B., Kim, N., Hohman, E.: Modeling human factors topics in aviation reports (2019). https://doi.org/10.1177/1071181319631095
Maguire, F.B., et al.: A text-mining approach to obtain detailed treatment information from free-text fields in population-based cancer registries: a study of non-small cell lung cancer in California. PLoS ONE 14(2), e0212454 (2019). https://doi.org/10.1371/journal.pone.0212454
Marafino, B.J., Davies, J.M., Bardach, N.S., Dean, M.L., Dudley, R.A.: N-gram support vector machines for scalable procedure and diagnosis classification, with applications to clinical free text data from the intensive care unit. J. Am. Med. Inform. Assoc. 21(5), 871–875 (2014). https://doi.org/10.1136/amiajnl-2014-002694
Marev, K., Georgiev, K.: Automated aviation occurrences categorization. In: ICMT 2019–7th International Conference on Military Technologies, Proceedings, pp. 1–5 (2019). https://doi.org/10.1109/MILTECHS.2019.8870055
McKenzie, A., Matthews, M., Goodman, N., Bayoumi, A.: Information extraction from helicopter maintenance records as a springboard for the future of maintenance text analysis. In: García-Pedrajas, N., Herrera, F., Fyfe, C., Benítez, J.M., Ali, M. (eds.) IEA/AIE 2010. LNCS (LNAI), vol. 6096, pp. 590–600. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13022-9_59
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: 1st International Conference on Learning Representations, ICLR 2013 - Workshop Track Proceedings (2013). https://arxiv.org/abs/1301.3781
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems cs.CL, pp. 1–9 (2013). https://arxiv.org/abs/1310.4546
Moreno Sandoval, A., Díaz, J., Campillos Llanos, L., Redondo, T.: Biomedical term extraction: NLP techniques in computational medicine. Int. J. Interact. Multimedia Artif. Intell. 5(4), 51 (2019). https://doi.org/10.9781/ijimai.2018.04.001
Navinchandran, M., Sharp, M.E., Brundage, M.P., Sexton, T.B.: Studies to predict maintenance time duration and important factors from maintenanceworkorder data. In: Proceedings of the Annual Conference of the Prognostics and Health Management Society, PHM, vol. 11 (2019). https://doi.org/10.36001/phmconf.2019.v11i1.792
Nguyen, A., Moore, D., McCowan, I., Courage, M.J.: Multi-class classification of cancer stages from free-text histology reports using support vector machines. In: Annual International Conference of the IEEE Engineering in Medicine and Biology - Proceedings, vol. 2007, pp. 5140–5143. IEEE, United States (2007). DOIurl10.1109/IEMBS.2007.4353497
Paul, S.: NLP tools used in civil aviation: a survey (2018). https://doi.org/10.26483/ijarcs.v9i2.5559
Pelt, M., Stamoulis, K., Apostolidis, A.: Data analytics case studies in the maintenance, repair and overhaul (MRO) industry. In: MATEC Web of Conferences, vol. 304, p. 04005 (2019). https://doi.org/10.1051/matecconf/201930404005
Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: EMNLP 2014–2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, pp. 1532–1543. Association for Computational Linguistics, Doha, Qatar (2014). https://doi.org/10.3115/v1/d14-1162
Robinson, S.D., Irwin, W.J., Kelly, T.K., Wu, X.O.: Application of machine learning to mapping primary causal factors in self reported safety narratives (2015). https://doi.org/10.1016/j.ssci.2015.02.003
Sexton, T., Hodkiewicz, M., Brundage, M.P., Smoker, T.: Benchmarking for keyword extraction methodologies in maintenance work orders. In: Proceedings of the Annual Conference of the Prognostics and Health Management Society, PHM. Philadelphia, PA (2018). https://doi.org/10.36001/phmconf.2018.v10i1.541
Tanguy, L., Tulechki, N., Urieli, A., Hermann, E., Raynal, C.: Natural language processing for aviation safety reports: from classification to interactive analysis. Comput. Ind. 78, 80–95 (2016). https://doi.org/10.1016/j.compind.2015.09.005
Wang, J., Li, C., Han, S., Sarkar, S., Zhou, X.: Predictive maintenance based on event-log analysis: a case study. IBM J. Res. Dev. 61(1), 121–132 (2017). https://doi.org/10.1147/JRD.2017.2648298
Zhang, K., Xu, J., Min, M.R., Jiang, G., Pelechrinis, K., Zhang, H.: Automated IT system failure prediction: a deep learning approach. In: Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016, pp. 1291–1300. IEEE (2016). https://doi.org/10.1109/BigData.2016.7840733
Acknowledgements
The completion of this research project would not have been possible without the contributions and support of many individuals at UWE and GE. I am grateful to all those who played a role in the success.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mayhew, P.J., Ihshaish, H., Deza, I., Del Amo, A. (2023). Maintenance Automation Using Deep Learning Methods: A Case Study from the Aerospace Industry. In: Iliadis, L., Papaleonidas, A., Angelov, P., Jayne, C. (eds) Artificial Neural Networks and Machine Learning – ICANN 2023. ICANN 2023. Lecture Notes in Computer Science, vol 14263. Springer, Cham. https://doi.org/10.1007/978-3-031-44204-9_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-44204-9_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44203-2
Online ISBN: 978-3-031-44204-9
eBook Packages: Computer ScienceComputer Science (R0)