Evaluating Deep Learning Networks Versus Hybrid Network for Smart Monitoring of Hydropower Plants
<p>An overview of the structure of the autoencoder network structure.</p> "> Figure 2
<p>An overview of the structure of 1D convolutional neural network.</p> "> Figure 3
<p>An overview of the structure of an LSTM cell.</p> "> Figure 4
<p>An overview on the structure of Hybrid 1D CNN LSTM AE model.</p> "> Figure 5
<p>MAE related to AE structure related to Plant A. The cyan line denotes the threshold defined as three times the average value of MAE in the training phase: error values above this line indicate that a fault has been detected.</p> "> Figure 6
<p>MAE related to CNN structure related to Plant A. The cyan line denotes the threshold defined as three times the average value of MAE in the training phase: error values above this line indicate that a fault has been detected.</p> "> Figure 7
<p>MAE related to LSTM structure related to Plant A. The cyan line denotes the threshold defined as three times the average value of MAE in the training phase: error values above this line indicate that a fault has been detected.</p> "> Figure 8
<p>MAE related to Hybrid structure related to Plant A. The cyan line denotes the threshold defined as three times the average value of MAE in the training phase: error values above this line indicate that a fault has been detected.</p> "> Figure 9
<p>MAE related to AE structure related to Plant B. The cyan line denotes the threshold defined as three times the average value of MAE in the training phase: error values above this line indicate that a fault has been detected.</p> "> Figure 10
<p>MAE related to CNN structure related to Plant B. The cyan line denotes the threshold defined as three times the average value of MAE in the training phase: error values above this line indicate that a fault has been detected.</p> "> Figure 11
<p>MAE related to LSTM structure related to Plant B. The cyan line denotes the threshold defined as three times the average value of MAE in the training phase: error values above this line indicate that a fault has been detected.</p> "> Figure 12
<p>MAE related to Hybrid structure related to Plant B. The cyan line denotes the threshold defined as three times the average value of MAE in the training phase: error values above this line indicate that a fault has been detected.</p> ">
Abstract
:1. Introduction
1.1. Motivation
1.2. State of the Art
- Model-Based Methods:These algorithms rely on creating a mathematical model of the system being monitored. The model represents the nominal behavior of the system, and deviations from this behavior may indicate the presence of faults. Model-based methods include techniques such as observer-based methods, state estimation methods, and parameter estimation.
- Data-Driven Methods:These algorithms analyze the data collected from sensors or other monitoring devices and try to identify a nominal behavior of the system. They use statistical analysis, machine learning, and pattern recognition techniques to detect anomalies or deviations from the nominal behavior. Data-driven methods are often preferred when the underlying system is complex or poorly understood as they can adapt to various operating conditions, embed any kind of non-linearity behavior, and detect faults without prior knowledge of the system’s dynamics. They can apply to inherently high-dimensional datasets collected from a multitude of sensors over a specified time frame. Therefore, they encompass diverse parameters with varying scales and feature extraction is extremely important when working with these systems [8]. Historically, feature extraction in data science literature has relied on statistical methods or machine learning techniques like Principal Component Analysis (PCA). With the advent of deep learning algorithms, feature extraction has transformed, enabling the learning of feature representations directly from the data through neural networks. This helps improve the efficiency of network anomaly detection [9].
- Statistical Analysis:Statistical process control (SPC), control charts, and hypothesis testing are examples of statistical methods used for fault detection. The main basis of these methods is monitoring the statistical properties of the data and identifying deviations from dominant behavior, but it should be noticed that traditional methods like vibration analysis have limitations in terms of cost and complexity.
- Machine Learning and Deep Learning Techniques:Various machine learning algorithms, such as support vector machines (SVM), neural networks (NN), decision trees, random forests, k-nearest neighbors (k-NN), and ensemble methods like AdaBoost and gradient boosting, are employed for fault detection by learning patterns and anomalies from historical data. These networks are great at finding and exploring hidden features in the attribute representation space [10]. Among traditional methods, the support vector machine (SVM) technique for condition monitoring and fault diagnostics (CMFD) has been noteworthy [11]. The authors in [12] have been pioneers in the intelligent implementation of energy production networks, studying neural networks for unsupervised learning. A self-organizing map was applied in Italian hydropower plants, leading to the proposal of a new key performance indicator (KPI). Also benefitting from sensor data, an expert system was established for online temperature monitoring systems in [13] and focuses on the predictive maintenance of hydropower plants using a multi-agent system (MAS) and artificial neural network (ANN). The result shows that this network is successful in monitoring, identifying, and diagnosing dynamic performance online. The authors in [14] detect early faults by analyzing temperature rise. Two types of statistical methods and machine learning are used for data processing and the results show that the feed-forward neural network (FFNN) method works better than Hotelling’s multivariate control chart in detecting fault samples.Deep learning techniques have shown superior results compared to classical machine learning methods, especially when the data volume increases. AI algorithms such as autoencoders (AEs), convolutional neural networks (CNN), and Long Short-Term Memory (LSTM) have demonstrated remarkable ability in discerning complex patterns and relationships within extensive datasets. Anomaly detection systems based on deep learning algorithms are increasingly popular and widely applied in both academic and industrial environments. The nature of the collected data plays an important role in the selection of the neural network. Autoencoder networks are a semisupervised learning model proven effective for fault detection [9,15]. The next relevant project employed a long short-term memory (LSTM) neural network for anomaly detection of variables like bearing temperatures and vibration in a 56 MW pumped storage hydroelectric power station in Norway. The LSTM network was utilized to predict the temperature one hour in advance [16]. The paper [17] explores the use of IoT technology to enhance predictive maintenance within the framework of Industry 4.0. By integrating IoT sensors with machine learning, this study demonstrates how real-time data analysis can predict potential equipment failures, thereby reducing downtime and associated costs across various industries. The study utilizes multiple case examples, including applications in aviation, manufacturing, and energy sectors, to illustrate the advantages of IoT-driven predictive maintenance. These examples showcase how predictive maintenance optimizes performance, minimizes unnecessary repairs, and extends equipment lifespan, thus offering competitive advantages in efficiency and cost-effectiveness for industries. These innovations reduce the workload associated with traditional alarm handling and enhance predictive maintenance capabilities by analyzing key trends in equipment parameters. Ref. [18] investigates the use of anomaly detection and explainability algorithms for improving predictive maintenance in hydroelectric power plants (HPPs). The study compares various machine learning models, with the autoencoder model emerging as the most effective for identifying anomalies within operational data. Additionally, the authors employ SHapley Additive exPlanations (SHAP) to provide insights into root causes, enabling experts to pinpoint which features contribute to anomalies. This research demonstrates the potential of machine learning in enhancing operational reliability and reducing costs, making HPPs more attractive to investors and advancing the transition to renewable energy sources. Recently, [19] presents an advanced monitoring system for hydropower stations, focusing on intelligent fault detection and alarm management. This study proposes an intelligent SCADA (Supervisory Control and Data Acquisition) system that integrates real-time monitoring, predictive alarms, and fault diagnosis to streamline data processing and improve decision-making for maintenance. By implementing an equipment alarm model and utilizing machine learning techniques for anomaly detection, the system can rapidly locate and diagnose faults.
- Pattern Recognition Methods:These methods use pattern recognition techniques such as clustering, Principal Component Analysis, Independent Component Analysis (ICA), and Self-Organizing Maps (SOM) to identify abnormal patterns or clusters in the data indicative of faults [20,21].Most of the research of recent years uses hybrid networks that use two or more methods in combination. The authors in [22] proposed two approaches for predictive maintenance in the Peña Blanca hydroelectric power plant. The first approach used logistic regression for classifying various types of failures, while the second approach combined recurrent LSTM networks with an autoencoder. Ref. [23] presents a hybrid model that combines the strengths of ARIMA and Bi-LSTM models, increasing the accuracy and robustness of the forecast. The LSTM model was optimal for detecting high temperatures in generator bearings. A condition monitoring method based on LSTM algorithms has been introduced in [24]. This method establishes correlations between prior known information and current environmental data. In some contexts, the location or type of fault is crucial for subsequent actions. So fault location or diagnosis is a pivotal consideration in this case. Ref. [25] proposes an unsupervised anomaly detection method based on variational modal decomposition (VMD) and a deep autoencoder. The autoencoder based on a convolutional neural network is used to complete unsupervised learning, and the reconstruction residual is used for anomaly detection. Refs. [15,26] merge Principal Component Analysis (PCA) with clustering-based autoencoders (CAE). This strategy enhances CAE’s capacity to detect latent representations of normal data.Despite all the advantages outlined for data-driven methods, the implementation of these systems usually faces many challenges, including the large volume of data and the emergence of database security concerns [5,27]. The next point is the asymmetry in the dataset. Detecting specific anomalous behaviors is challenging due to the rarity, heterogeneity, and low frequency of defective data compared to nominal data [28]. One widely deployed solution is to use deep neural networks trained using only healthy data in a one-class training manner [29].
1.3. Contribution and Organization
- Several methodologies, as outlined in the previous section, have been used for the application of smart monitoring and fault detection, also in the field of hydropower plants. It is usually hard to compare different methodologies on the same problem since each one of them may be more convenient under a different perspective: e.g., the ability to handle massive amounts of data; the ability to distinguish faulty operations more clearly; the ability to recognize incipient faults several days before the actual occurrence is noticed on the field; or the ability to avoid false positives, which may cause unnecessary maintenance actions of parts of hydropower plants. By adopting real data collected from two hydropower plants, this study provides a thorough comparison between classic machine learning algorithms and more recent techniques, most notably those in the class of deep learning algorithms.
- A methodology is introduced, specifically a hybrid deep learning model that, to the best of the authors’ knowledge, has not been used for this particular application. In particular, hybrid deep learning networks combine different architectures with the ultimate goal of leveraging the combined strengths of CNNs, LSTMs, and autoencoders. Indeed, this methodology appears to be slightly superior to the single ones in our case study.
2. Case Study
2.1. Hydropower Plant
2.2. Dataset
3. Methodology
3.1. Autoencoder
3.2. 1D CNN
3.3. Multi-Input LSTM
3.4. Hybrid Network: 1D CNN LSTM AE
4. Proposed Network Structure
4.1. Tuning Hyperparameters and Optimized Structure
4.2. Structure of Implemented AE Network
4.3. Structure of Implemented 1D CNN Network
4.4. Structure of Implemented LSTM Network
4.5. Structure of Implemented Hybrid Network
5. Results
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Gøtske, E.K.; Victoria, M. Future operation of hydropower in Europe under high renewable penetration and climate change. iScience 2021, 24, 102999. [Google Scholar] [CrossRef] [PubMed]
- Betti, A.; Crisostomi, E.; Paolinelli, G.; Piazzi, A.; Ruffini, F.; Tucci, M. Condition monitoring and predictive maintenance methodologies for hydropower plants equipment. Renew. Energy 2021, 171, 246–253. [Google Scholar] [CrossRef]
- Spencer, B., Jr.; Nagarajaiah, S. State of the art of structural control. J. Struct. Eng. 2003, 129, 845–856. [Google Scholar] [CrossRef]
- Li, S.; Pozzi, M. What makes long-term monitoring convenient? A parametric analysis of the value of information in infrastructure maintenance. Struct. Control Health Monit. 2019, 26, e2329. [Google Scholar] [CrossRef]
- Nagarajaiah, S.; Yang, Y. Modeling and harnessing sparse and low-rank data structure: A new paradigm for structural dynamics, identification, damage detection and health monitoring. Struct. Control Health Monit. 2017, 24, e1851. [Google Scholar] [CrossRef]
- Jana, D.; Patil, J.; Herkal, S.; Nagarajaiah, S.; Duenas-Osorio, L. CNN and Convolutional Autoencoder (CAE) based real-time sensor fault detection, localization, and correction. Mech. Syst. Signal Process. 2022, 169, 108723. [Google Scholar] [CrossRef]
- Velasquez, V.; Flores, W. Machine Learning Approach for Predictive Maintenance in Hydroelectric Power Plants. In Proceedings of the 2022 IEEE Biennial Congress of Argentina (ARGENCON), San Juan, Argentina, 7–9 September 2022; pp. 1–6. [Google Scholar] [CrossRef]
- Amini, N.; Zhu, Q. Fault detection and diagnosis with a novel source-aware autoencoder and deep residual neural network. Neurocomputing 2022, 488, 618–633. [Google Scholar] [CrossRef]
- Chalapathy, R.; Chawla, S. Deep learning for anomaly detection: A survey. arXiv 2019, arXiv:1901.03407. [Google Scholar]
- Mehrotra, K.; Mohan, C.; Huang, H. Anomaly Detection; Springer International Publishing: Berlin/Heidelberg, Germany, 2017; pp. 21–32. [Google Scholar]
- Xayyasith, S.; Promwungkwa, A.; Ngamsanroaj, K. Application of Machine Learning for Predictive Maintenance Cooling System in Nam Ngum-1 Hydropower Plant. In Proceedings of the 2018 16th International Conference on ICT and Knowledge Engineering (ICT&KE), Bangkok, Thailand, 21–23 November 2018; pp. 1–5. [Google Scholar]
- Selak, L.; Butala, P.; Sluga, A. Condition monitoring and fault diagnostics for hydropower plants. Comput. Ind. 2014, 65, 924–936. [Google Scholar] [CrossRef]
- Jiang, W. Research on Predictive Maintenance for Hydropower Plant Based on MAS and NN. In Proceedings of the 2008 Third International Conference on Pervasive Computing and Applications, Alexandria, Egypt, 6–8 October 2008; pp. 604–609. [Google Scholar]
- Jain, S.; Barmada, E.; Crisostomi, E.; Romano, F.; Tavano, F.; Tucci, M. Indirect monitoring and early detection of faults in trains’ motors. IET Electr. Syst. Transp. 2018, 8, 86–94. [Google Scholar] [CrossRef]
- Nguyen, V.; Nguyen, V.; Hoang, T.; Shone, N. A Novel Deep Clustering Variational Auto-Encoder for Anomaly-based Network Intrusion Detection. In Proceedings of the 14th International Conference on Knowledge and Systems Engineering (KSE), Nha Trang, Vietnam, 19–21 October 2022; pp. 1–7. [Google Scholar]
- Buaphan, I.; Premrudeepreechacharn, S. Development of an expert system for fault diagnosis of an 8-MW bulb turbine downstream irrigation hydropower plant. In Proceedings of the 6th International Youth Conference on Energy (IYCE), Budapest, Hungary, 21–24 June 2017; pp. 1–6. [Google Scholar]
- Sharma, A.; Aslekar, A. IoT-Based Predictive Maintenance in Industry 4.0. In Proceedings of the 2022 IEEE International Interdisciplinary Humanitarian Conference for Sustainability (IIHC-2022), Bengaluru, India, 18–19 November 2022; pp. 143–145. [Google Scholar] [CrossRef]
- Fanan, M.; Baron, C.; Carli, R.; Divernois, M.A.; Marongiu, J.C.; Susto, G.A. Anomaly Detection for Hydroelectric Power Plants: A Machine Learning-based Approach. In Proceedings of the 2023 IEEE 21st International Conference on Industrial Informatics (INDIN), Lemgo, Germany, 18–20 July 2023. [Google Scholar] [CrossRef]
- Li, Z.; Wang, Y.; Wang, B.; Zhang, H. Research and Design of Intelligent Hydropower SCADA. In Proceedings of the 3rd International Conference on Energy and Power Engineering, Control Engineering (EPECE), Chengdu, China, 23–25 February 2024; pp. 1–5. [Google Scholar] [CrossRef]
- Garbea, R.; Grigoras, G. Clustering-Using Data Mining-based Application to Identify the Hourly Loading Patterns of the Generation Units from the Hydropower Plants. In Proceedings of the 2022 International Conference and Exposition on Electrical And Power Engineering (EPE), Iasi, Romania, 20–22 October 2022; pp. 426–431. [Google Scholar]
- Calvo-Bascones, P.; Sanz-Bobi, M.; Welte, T. Anomaly detection method based on the deep knowledge behind behavior patterns in industrial components. Application to a hydropower plant. Comput. Ind. 2021, 125, 103376. [Google Scholar] [CrossRef]
- Li, M.; Francis, E.; Hinkle, S.; Ajjarapu, A.; Zhang, C. Preconception and prenatal nutrition and neurodevelopmental disorders: A systematic review and meta-analysis. Nutrients 2019, 11, 1628. [Google Scholar] [CrossRef] [PubMed]
- Malhan, P.; Mittal, M. A novel ensemble model for long-term forecasting of wind and hydropower generation. Energy Convers. Manag. 2022, 251, 114983. [Google Scholar] [CrossRef]
- Qian, P.; Tian, X.; Kanfoud, J.; Lee, J.; Gan, T. A novel condition monitoring method of wind turbines based on long short-term memory neural network. Energies 2019, 12, 3411. [Google Scholar] [CrossRef]
- Wang, H.; Liu, X.; Ma, L.; Zhang, Y. Anomaly detection for hydropower turbine unit based on variational modal decomposition and deep autoencoder. Energy Rep. 2021, 7, 938–946. [Google Scholar] [CrossRef]
- Qian, J.; Song, Z.; Yao, Y.; Zhu, Z.; Zhang, X. A review on autoencoder based representation learning for fault detection and diagnosis in industrial processes. Chemom. Intell. Lab. Syst. 2022, 231, 104711. [Google Scholar] [CrossRef]
- Gârbea, R.; Scarlatache, F.; Grigoraș, G.; Neagu, B.C. Extracting the Operating Characteristics of Hydropower Plants Using a Clustering-based Efficient Methodology. In Proceedings of the 9th International Conference on Modern Power Systems (MPS), Cluj-Napoca, Romania, 16–17 June 2021; pp. 1–4. [Google Scholar]
- Pang, G.; Cao, L.; Aggarwal, C. Deep learning for anomaly detection: Challenges, methods, and opportunities. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, Virtual, 8–12 March 2021; pp. 1127–1130. [Google Scholar]
- Nguyen, V.; Nguyen, V.; Le-Khac, N.; Cao, V. Clustering-based deep autoencoders for network anomaly detection. In Proceedings of the Future Data and Security Engineering: 7th International Conference, FDSE 2020, Quy Nhon, Vietnam, 25–27 November 2020; Proceedings 7. Springer International Publishing: Berlin/Heidelberg, Germany, 2020; pp. 290–303. [Google Scholar]
- Nagarajaiah, S.; Erazo, K. Structural monitoring and identification of civil infrastructure in the United States. Struct. Monit. Maint. 2016, 3, 51. [Google Scholar] [CrossRef]
- Theodoridis, S.; Koutroumbas, K. Pattern Recognition; Elsevier: Amsterdam, The Netherlands, 2006. [Google Scholar]
- Harish, A.; Prince, A.; Jayan, M.V. Fault detection and classification for wide area backup protection of power transmission lines using weighted extreme learning machine. IEEE Access 2022, 10, 82407–82417. [Google Scholar] [CrossRef]
- Yan, W.; Wang, J.; Lu, S.; Zhou, M.; Peng, X. A Review of Real-Time Fault Diagnosis Methods for Industrial Smart Manufacturing. Processes 2023, 11, 369. [Google Scholar] [CrossRef]
- Hajimohammadali, F.; Fontana, N.; Tucci, M.; Crisostomi, E. Autoencoder-based Fault Diagnosis for Hydropower Plants. In Proceedings of the 2023 IEEE Belgrade PowerTech, Belgrade, Serbia, 25–29 June 2023; pp. 1–6. [Google Scholar]
- Zhu, X.; Tuia, D.; Mou, L.; Xia, G.; Zhang, L.; Xu, F.; Fraundorfer, F. Deep learning in remote sensing: A comprehensive review and list of resources. IEEE Geosci. Remote Sens. Mag. 2017, 5, 8–36. [Google Scholar] [CrossRef]
- Adari, S.K.; Alla, S. Beginning Anomaly Detection Using Python-Based Deep Learning: Implement Anomaly Detection Applications with Keras and PyTorch; Apress: New York, NY, USA, 2024; pp. 393–398. [Google Scholar]
- Graves, A.; Liwicki, M.; Fernández, S.; Bertolami, R.; Bunke, H.; Schmidhuber, J. A novel connectionist system for unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2008, 31, 855–868. [Google Scholar] [CrossRef]
- Koleva, R.; Babunski, D.; Zaev, E.; Tuneski, A.; Trajkovski, L. New Approach in Hydropower Plant Control Based on Neural Networks. Energ. Ekon. Ekol. 2022, XXIV, 39–46. [Google Scholar] [CrossRef]
Component Name | Measured Signals |
---|---|
Generation Units | Vibrations |
HV Transformer | Temperatures, Gasses levels |
Turbine | Pressures, Flows, Temperatures |
Oleo-dynamic system | Pressures, Temperatures |
Supports | Temperatures |
Alternator | Temperatures |
Validation Loss | latentNeuron | Learning Rate | Batch Size | Best |
---|---|---|---|---|
0.0304704045 | 6 | 16 | ||
0.0062245624 | 27 | 32 | ||
0.0027203022 | 5 | 8 | ✔ | |
0.0039545825 | 24 | 8 | ||
0.0306758715 | 12 | 16 | ||
0.0042250373 | 46 | 8 | ||
0.0070399883 | 19 | 16 | ||
0.0042112886 | 40 | 8 | ||
0.0183390009 | 7 | 32 | ||
0.0072042440 | 49 | 16 |
Case Study | Plant ID | Warning Name | Fault Date |
---|---|---|---|
1 | A | HV transformer gasses | 1 June 2018 |
2 | B | Generator Temperature | 1 October 2018 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Hajimohammadali, F.; Crisostomi, E.; Tucci, M.; Fontana, N. Evaluating Deep Learning Networks Versus Hybrid Network for Smart Monitoring of Hydropower Plants. Energies 2024, 17, 5670. https://doi.org/10.3390/en17225670
Hajimohammadali F, Crisostomi E, Tucci M, Fontana N. Evaluating Deep Learning Networks Versus Hybrid Network for Smart Monitoring of Hydropower Plants. Energies. 2024; 17(22):5670. https://doi.org/10.3390/en17225670
Chicago/Turabian StyleHajimohammadali, Fatemeh, Emanuele Crisostomi, Mauro Tucci, and Nunzia Fontana. 2024. "Evaluating Deep Learning Networks Versus Hybrid Network for Smart Monitoring of Hydropower Plants" Energies 17, no. 22: 5670. https://doi.org/10.3390/en17225670
APA StyleHajimohammadali, F., Crisostomi, E., Tucci, M., & Fontana, N. (2024). Evaluating Deep Learning Networks Versus Hybrid Network for Smart Monitoring of Hydropower Plants. Energies, 17(22), 5670. https://doi.org/10.3390/en17225670