Open AccessArticle

Photovoltaic Panels Classification Using Isolated and Transfer Learned Deep Neural Models Using Infrared Thermographic Images

Waqas Ahmed

Aamir Hanif

¹,

Karam Dad Kallu

²,

Abbas Z. Kouzani

Muhammad Umair Ali

^4,*

and

Amad Zafar

^5,*

Department of Electrical Engineering, University of Wah, Wah Cantt 47040, Pakistan

Department of Robotics and Intelligent Machine Engineering (RIME), School of Mechanical and Manufacturing Engineering (SMME), National University of Sciences and Technology (NUST) H-12, Islamabad 44000, Pakistan

School of Engineering, Deakin University, Geelong, VIC 3216, Australia

⁴

Department of Unmanned Vehicle Engineering, Sejong University, Seoul 05006, Korea

⁵

Department of Electrical Engineering, Islamabad Campus, University of Lahore, Islamabad 54590, Pakistan

Authors to whom correspondence should be addressed.

Sensors 2021, 21(16), 5668; https://doi.org/10.3390/s21165668

Submission received: 8 June 2021 / Revised: 17 August 2021 / Accepted: 17 August 2021 / Published: 23 August 2021

(This article belongs to the Special Issue Condition Monitoring, Field Inspection and Fault Diagnostic Methods for Photovoltaic Systems)

Download

Browse Figures

Versions Notes

Abstract

Defective PV panels reduce the efficiency of the whole PV string, causing loss of investment by decreasing its efficiency and lifetime. In this study, firstly, an isolated convolution neural model (ICNM) was prepared from scratch to classify the infrared images of PV panels based on their health, i.e., healthy, hotspot, and faulty. The ICNM occupies the least memory, and it also has the simplest architecture, lowest execution time, and an accuracy of 96% compared to transfer learned pre-trained ShuffleNet, GoogleNet, and SqueezeNet models. Afterward, ICNM, based on its advantages, is reused through transfer learning to classify the defects of PV panels into five classes, i.e., bird drop, single, patchwork, horizontally aligned string, and block with 97.62% testing accuracy. This proposed approach can identify and classify the PV panels based on their health and defects faster with high accuracy and occupies the least amount of the system’s memory, resulting in savings in the PV investment.

Keywords:

deep convolution neural network; PV panels; infrared images; hotspots

1. Introduction

Electrical energy fulfils the human demand in every sector, i.e., industry, commercial, agriculture, education, etc.; therefore, it has an ever-growing demand [1,2]. However, the majority of the energy production sources across the globe are fossil fuel-based power plants, even though fossil fuels are limited in nature [1,2,3]. For example, Pakistan is heavily dependent upon conventional thermal power plants for its energy needs [3,4]. Moreover, the burning of fossil fuels such as oil, gas, coal, etc., emits greenhouse gases such as carbon dioxide, methane, nitrous oxide, water vapours, etc., which trap solar radiation in the earth’s atmosphere and cause significant environmental degradation such as climate change [2,3,5,6]. As per the literature, electrical energy generation and consumption account for two-thirds of global stack emissions [7].

To achieve green energy sustainability, solar energy, a green, low carbon renewable energy source, is one of the best energy production alternatives with 26–217 gCO2eq/kWh emissions compared to the 530 gCO2eq/kWh emissions of thermal plants on a global average [2]. As per the literature, solar photovoltaics (PV) along with wind energy sources are expected to meet 88% of global energy demand by 2050 [1]. However, investment in a PV system is high, and the payback time defines PV system economic viability, which is highly dependent upon PV system performance and lifetime [3,8]. Therefore, PV system manufacturers must guarantee their long-term performance and lifespan, which is reflected in the PV datasheet [9]. PV system performance suffers with time due to aging, which is a loss process, or due to defects such as component breakdown, which has an immediate impact [10]. Moreover, in the case of PV system defects, PV efficiency and output suffer from a noticeable impact on payback time [3,8].

1.1. Losses in PV System

The power loss in a PV system arises due to different factors, including manufacturing defects, transportation, installation, short circuit, open circuit, partial shading, shading, line to line fault, arc fault, bird drops, dust accommodation, module mismatch, environmental degradation, ageing, cell cracking due to mechanical stresses, etc., in real-time operations [1,8,11,12,13,14,15,16,17]. Details on PV system defects are provided in Basnet et al. [18]. These defects may limit 50% of power output per module [11]. Generally, all defects are treated as faults in the system. However, defects such as bird drops, shadows, dust accommodation, etc., reduce the PV output by localised heating of the PV cell (hotspot formation) due to the operation in the reverse region and dissipating healthy cell current due to series connection [1,14,19]. If such issues remain undetected and persist on the PV surface, these issues result in permanent failure/faults of PV panels [20]. Moreover, defects/hotspots formation on PV panels’ occurrence frequency, duration of stay, and intensity are unpredictable and are purely natural parameters [21]. In addition, only a few defects are identifiable through naked eye/visual inspection, such as corrosion, burned and broken cells, bubbles, etc. [19]. Therefore, a fast, low-storage, and accurate identification system will ensure the safe operation of the PV system during its lifetime by the timely identification of hotspots/defects/faults in the PV system during its normal operation [22]. However, it is important that if defects do not affect the output or system safety, the defect is not treated as a failure [8].

1.2. Health Monitoring Approaches of PV System

To prolong the performance of PV panels, an accurate and efficient approach to identify and classify PV panels because of defects is inevitable [1,11]. Generally, PV system health and defects are identified using two broad approaches: through electrical signal (voltage and current characteristics of PV panels) [16,17,18] and non-invasive image-based approaches, such as electroluminescence (EL) images [23], infrared (IR) thermography of PV panels [1,8,11,12,19], fluorescence images [24], photoluminescence images [25] etc. However, IR thermography of PV panels is widely used in the literature due to its speed, low cost, large-scale outdoor applications, user friendliness, and accuracy [8,11,12,19].

1.3. Non-Invasive Images-Based Classification of PV Panels’ Health and Defects: Literature Review

Different image processing-based machine learning and deep learning approaches are utilised to identify and classify the PV system defects based on images. Niazi et al. [26] used the naïve-Bayes approach to classify PV panels into two classes, defective and non-defective, with IR thermograph texture features with a 98.4% mean recognition rate. Meanwhile, Ali et al. [1] used a support vector machine on IR images after extracting image features and classified PV panels into three classes based on health with 92% accuracy. Naïve Bayes was also used to classify PV panels into three categories (defective, non-defective with hotspots, and non-defective without hotspots) using an IR thermograph dataset with a 94.1% mean recognition rate [27].

However, in deep learning, i.e., using neural networks, Dunderdale et al. [11] used IR images to classify PV panels into defective and non-defective with 91.2% accuracy and defects (block, patchwork, single, and string) identification with 89.5% accuracy using a scale-invariant feature transform feature descriptor, spatial pyramid matching, and deep learning approaches. Akram et al. [12] developed an isolated light convolution isolated neural network trained on PV system EL thermographs, and they used a transfer learning approach on PV system IR thermography for defect detection with 99.23% accuracy. Similarly, the isolated convolution neural networks to classify IR images based on dust and hotspots with 98% accuracy were proposed by Cipriani et al. [13]. Kurukuru et al. [28] used IR thermographs of eight different panels (one healthy and seven faulty), extracted texture features, and used a scaled conjugate gradient back propagation algorithm to adjust the weight of the neural network classifier and achieved 91.7% testing accuracy. Bommes et al. [29] used aerial IR videos of PV modules and a pre-trained convolutional neural network, ResNet-50, to classify the 10 common module abnormalities with more than 90% testing accuracy.

The PV systems have received worldwide acceptance as a green energy solution. However, the defects/hotspots/faults in the PV systems that arise due to the environment, manufacturing, transportation, operation, etc., greatly reduce their efficiency and performance. In this study, an isolated convolution neural model (ICNM) was built from scratch to classify PV panels based on their health into three categories—healthy, hotspot, and faulty—using IR images. The hotspot PV class is the class suffering from dust, shadows, bird drop issues, etc., and a timely solution to these issues may revert their state to first class or healthy. Afterward, the results of the isolated model are compared with transfer learned pre-trained networks such as ShuffleNet, GoogleNet, and SqueezeNet in terms of storage space through their architecture complexity, fastness, true positive rates (TPR), false negative rates (FNR), positive predictive values (PPV), false discovery rate (FDR), and accuracy. Finally, ICNM advantages are utilised using the transfer learning approach to classify five defects in PV panels—namely, bird drops, block, single, patchwork, and horizontally aligned (HA) string—which arise due to different environmental issues and were initially adjusted as two broad health classes, i.e., hotspot and faulty through neurons weights.

The rest of the study is structured as follows. Section 2 explains the research approach. Section 3 presents the results followed by discussion, Section 4 and finally, the study conclusion is provided in Section 5.

2. Research Approach

This study utilises an IR image dataset of a 42.24 kW solar photovoltaic system located in Lahore, Pakistan [27]. The acquired images are pre-processed for uniformity by removing unwanted noise. Afterward, the dataset is first segregated into sub-classes, i.e., based on their health, i.e., healthy, hotspot, and faulty. Secondly, data of two classes, i.e., hotspot and faulty, are further segregated into five sub-classes of defects, i.e., bird drops, single, block, patchwork, and HA string. After segregation, pre-processed images are split into training data comprised of 80% randomly selected images and 20% testing data comprised of the remaining images. Both training and testing datasets are created with an equal proportion of images to train the networks properly. A validation dataset is a 20% dataset created to test the models’ design to avoid over-fitting and a performance check of models by further splitting the training dataset. Finally, a three-class ICNM is built, trained, and validated against transfer-learned pre-trained networks, i.e., 50-layered ShuffleNet, 22-layered GoogleNet, and 18-layered SqueezeNet. Afterward, the advantages of ICNM, fast response, simple architecture, and accuracy are used through a transfer learning approach to classify the five different defects in PV panels. The proposed approach is presented in Figure 1. The study was carried out on MATLAB 2020a with the following system specifications: i5 7th generation, 16 GB DDR4 RAM with a 500 GB SSD hard disk, 2GB Ati Radeon Graphics Card, 64-bit operating system, and x64-based processor.

2.1. PV System

PV system output and performance vary due to multiple parameters such as geographical parameters, i.e., ambient temperature, precipitation, daily solar radiation, etc., and orientation angles [2]. For this study, a previously published PV dataset [27], a PV system installed in Lahore, Pakistan, was obtained and used to classify PV panels based on health and defects to restrain power losses. The per annum average of geographical parameters for Lahore city is provided in Table 1. The data is retrieved from the NASA Meteorological Database using RETScreen Expert software [30].

The literature suggests that bodies with a temperature above absolute zero, zero kelvin, possess thermodynamic energy and emit electromagnetic radiation in the infrared region, with a wavelength of 8 to 12 mm [1]. However, some literature suggests that wavelengths are 700 nm to 1 mm with a 430 THz to 300 GHz frequency and 1.7 eV to 1.24 meV photonic energy [31]. Infrared thermography was used to examine the PV system’s health, heat dissipation, and defects based on the thermal properties. The infrared camera, PV system, and on-site geographical parameters are provided in Table 2. Details of the experimental setup are provided in Niazi et al. [27].

2.2. Pre-Processing

Raw infrared image datasets of 42.24 kW PV systems are pre-processed to increase classification accuracy and reduce unwanted mismatches. Firstly, PV system datasets were cleaned by removing unwanted background objects such as ground, trees, railings, etc., as much as possible. Afterward, images were resized to obtain uniformity. A dataset of 315 infrared images based on PV system health was used. For each sub-class, data is provided in Table 3 and images are reflected in Figure 2.

Moreover, different PV defects on inspection of a thermal image are easily identifiable [11]. However, it is not possible for a large-scale PV system scenario. However, in defects, birds drop defects that have no specific shape and location, as they are highly unpredictable. The single defect is identifiable as a small rectangular shape. A patchwork defect consists of multiple single defects scattered on the PV surface. The HA string defect is a single defect in a row. Moreover, block defects almost cover up to one-third of PV panels. All these defects are visible due to hotter regions compared to the rest of the PV panels, as shown in Figure 3. Extensive details are provided in [11,32]. A dataset of 213 infrared images for defects on PV panels is provided in Table 4.

2.3. Isolated and Trained Transfer Learning-Model-Based Classifiers

The classifiers are rule-based systems capable of data transfer with a parallel processing pool. Classifiers continuously receive new information and process it based on capabilities already learned [33]. In this era, deep neural network models are one of the most powerful tools utilised for classification [23]. Based on deep neural network classifiers, there are mainly two approaches: isolated neural network and transfer learning [12].

In an isolated neural network model, first, a new model is prepared and trained from scratch without any previous knowledge, without adjusting the weights of its neurons to any other problem [12]. In the transfer learning approach, a neural network model has been trained on a base image dataset for a given problem; the model-learned knowledge, i.e., neurons’ weight adjustment, layers, and connections (architecture) advantages, are reutilized, or learning is transferred to a new and different image dataset. The architecture of models concisely trained on different classification problems through transfer learning is reused for new problem solutions [12]. The advantage transfer learning offers is that its architecture, i.e., layers, connections, and weight adjustment, is reutilized for a new problem solution rather than building a new model from scratch that is trained iteratively for the desired performance, which results in fast convergence and less data for training. Two main approaches to transfer learning are pre-trained model-based transfer learning and a model that is built from scratch, trained, and afterwards utilize the transfer learning approach for a new issue [12,34]. However, pre-trained deep networks are publicly available, such as GoogleNet [35], SqueezeNet [36], Resnet50 [23,37], MobileNet v2 [11,23], Inception V3 [23], ShuffleNet [38], etc., and are widely used for classification problems.

The generic architecture of an ICNM for features extraction and classification of input images based on features is provided in Figure 4. In this study, IR images were used to classify PV panels based on their health. For this purpose, different architectures of ICNM are created from scratch, trained, and validated to avoid over-fitting. Secondly, pre-trained deep neural networks, i.e., SqueezeNet, GoogleNet, and ShuffleNet, after the transfer learning approach, are utilised on the same dataset to validate the performance of ICNMs in terms of storage space due to architecture complexity, TPR, FNR, PPV, FDR, execution time, and accuracy. Finally, the best-isolated neural model is reutilised by the transfer learning approach to re-adjust the neurons’ weights to classify the five different defects of PV panels into their sub-classes, which were initially classified as hotspots or faulty classes.

2.4. Training and Testing Dataset

Pre-processed IR image datasets are divided into training and testing data. The training dataset is formed by randomly distributing 80% of the images, which are provided in Table 3 and Table 4, into new datasets to train the deep neural networks. The remaining 20% of the image datasets are utilised to test the efficiency and accuracy of the models. Moreover, each dataset is split into both categories randomly and in equal proportion to ensure the proper training and testing of the deep neural network models.

3. Results

Training and validation parameters for all schemes were kept constant for comparison, provided in Table 5.

3.1. Isolated Neural Network and Transfer Learned Pre-Trained Networks for PV Classification Based on Health

Firstly, the classification of PV panels based on their health is conducted using ICNMs. Different architectures of the ICNM were built by varying convolution layers, with batch normalisation, with and without activation functions, etc. ICNM based on higher validation accuracy, TPR, FNR, PPV, FDR, and validation loss is selected and compared with transfer learned, SqueezeNet, GoogleNet, and ShuffleNet, pre-trained networks from open sources, MathWorks. Detailed comparisons of models are provided based on training loss, training accuracy, validation loss, validation accuracy, execution time, and architecture complexity. Table 6 provides the information of ICNMs, whereas Table 7 reports the information on the pre-trained networks. All models were validated on the IR images dataset that was not utilised in the training phase. Figure 5 and Figure 6 show the training loss of different layer ICN models and a pre-trained network for health classification of the PV system over 60 epochs, respectively. Training loss is provided on the vertical axis, and the epochs/experience are provided on horizontal axis. Training loss diagnoses the condition of model; for instance, six and seven-layered ICNMs loss is minimising, which represents their good learning compared to eight and nine-layered ICNMs and pre-trained networks. Moreover, seven-layered ICNMs model training loss reaches stable points with the least variations, which means good fit learning.

3.2. Transfer Learning on an Isolated Model for Defects Classification

The seven-layered ICN model initially trained on IR images to classify PV panels into three classes based on health was re-utilised after the transfer learning approach to classify PV panels defects among two health sub-classes, i.e., hotspot and faulty, due to its training accuracy, training loss, validation loss, validation accuracy, execution time, simple architecture, and lower storage requirement. The testing accuracy of the seven-layered transfer learned model for five defects is provided in Table 8, while training loss is illustrated in Figure 7. Pre-trained network results on five defects classification are provided in Table 9. It is noted that TPR, FNR, PPV, and FDR are based on testing results on trained networks.

3.3. Transfer Learning on Augmented Dataset

The defects-based dataset is imbalanced since defects occurrence and frequency is a natural parameter [21]. A new balanced augmented dataset of defects based on 275 images (with 20% of each class) is created using scaling and shifting (0–20°) with noise [39]. Afterwards, data are split randomly in an 80:20 training and testing ratio but with equal proportions, and models are trained. The training loss of the model with augmented datasets of defects is provided in Figure 8, and detailed results of the models’ performance are provided in Table 10. TPR, FNR, PPV, and FDR are based on testing accuracy. MATLAB 2020a with core™ i5, 8 GB Ram was used for this section.

4. Discussion

PV systems are widely used globally because of their green renewable energy nature, while defects blocking the PV cell’s radiation absorption capability result in current dissipation, localised heating, and in turn energy and investment loss. Defects that remain undetected lead to the total faultiness of PV panels. Therefore, different methods are used to monitor the condition of PV panels/modules for prediction-based maintenance. For predicting maintenance, different approaches such as electroluminescence, thermographs, UV-fluorescence, transmission methods, etc. are used [40]. Therefore, in this study, IR thermographs are carried out for prediction-based maintenance.

In the literature, linear and nonlinear, feature-based, deep network-based classifiers, etc. are extensively used for PV system classification [1,11,12,13,26,27,28,29]. Linear and feature-based classifiers such as SVM, naïve Bayes, etc., are used to differentiate PV panels based on their health or defects into two and three categories. These classifiers have the advantage of simple structure, less complexity, storage requirement, and execution time. Moreover, their accuracy increases with the most appropriate feature selection in the dataset. In contrast, nonlinear and deep network-based classifiers such as convolution neural networks offer an advantage in terms of multi-classes-based classification because of their complex architecture and nonlinearity, resulting in higher accuracy. These classifiers extract the most relevant features from the dataset and adjust the neuron’s weight iteratively to classify multiple defects.

However, linear and feature-based classifier accuracy suffers from increased output classes because of their simple architecture. Moreover, their accuracy is highly dependent upon the most relevant feature extraction. The image’s dataset in different environmental conditions may require different features, resulting in further limitations of these classifiers. On the other hand, nonlinear and deep network-based classifiers, due to their complex architecture, need more storage and advanced computing systems for their long execution time, even in fewer classes-based differentiation problems.

In this study, multi-classes-based classification with less execution time, high accuracy, less architectural complexity, and storage requirements issues are addressed. Firstly, the IR images dataset was split into three health-based categories using a simple seven-layered ICNM with 96% accuracy. Afterward, isolated trained ICNM was re-utilised using the transfer learning approach to classify the five defects of the PV system with 97.63% testing accuracy in eight minutes. Moreover, a new augmented dataset of five-class defects was created using scale and shift, with associated noise to balance the dataset. A seven-layer transfer learned model and pre-trained networks were re-trained, validated, and tested on an augmented images dataset. The seven-layered transfer learned model has the highest validation accuracy, with a testing accuracy of 96.36% in the presence of a noisy augmented dataset. The seven-layer transfer learned model has the advantage of the least computational time requirement, storage requirement, architecture complexity, and computing system requirements compared to pre-trained networks. It is important that ICNM 7-layered includes input and output not only hidden layers.

Moreover, the direct classification of a PV system into six classes based on health and five defects requires modifications in ICNM to extract its most relevant features, which increases the complexity, storage requirement, and execution time. However, in this approach, initially, three health-based classes adjusted the neurons’ weights in eight minutes, and through transfer learning, a seven-layered ICNM trained model after neurons weight re-adjustment classified five defects with higher accuracy in less time compared to other pre-trained networks with no increase in architectural complexity and through a simple computing system.

5. Conclusions

PV panels are widely used across the globe as a promising solution to global climate change, introducing reliability, investment, and output loss in the system due to hotspots and faultiness. The timely monitoring of PV panel health and defects can avoid their permanent failure. In this paper, a fast, low-storage, and simple architecture of an isolated deep convolution-based neural model was built and utilised with 96% accuracy to classify PV panels into three categories based on health. Afterward, a transfer learning approach was utilised on the same isolated built model to classify the PV panels’ defects, initially placed in hotspot, and faulty with 97.62% testing accuracy on a new IR images dataset not used in training and validation in eight minutes. However, an augmented images dataset of defects resulted in 96.36% testing accuracy. The isolated neural network results are validated against pre-trained high storage, complex architecture, and high execution time neural networks. This approach can classify PV panels to restrain the power loss and ensure the shortest payback time possible. Future work includes the testing of ICN models on big datasets without the need of data augmentation.

Author Contributions

Conceptualization, W.A., M.U.A. and A.Z.; Data curation, K.D.K.; Formal analysis, K.D.K. and M.U.A.; Funding acquisition, A.Z.K.; Investigation, W.A.; Methodology, W.A., M.U.A. and A.Z.; Project administration, A.H. and A.Z.K.; Resources, A.Z.K.; Software, W.A., M.U.A. and A.Z.; Supervision, A.H. and A.Z.; Validation, M.U.A.; Writing—original draft, W.A. and K.D.K.; Writing—review and editing, A.H., A.Z.K., M.U.A. and A.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding authors.

Conflicts of Interest

The authors declare no conflict of interest.

References

Ali, M.U.; Khan, H.F.; Masud, M.; Kallu, K.D.; Zafar, A. A machine learning framework to identify the hotspot in photovoltaic module using infrared thermography. Sol. Energy 2020, 208, 643–651. [Google Scholar] [CrossRef]
Ahmed, W.; Sheikh, J.A.; Ahmad, S.; Farjana, S.H.; Mahmud, M.P. Impact of PV system orientation angle accuracy on greenhouse gases mitigation. Case Stud. Therm. Eng. 2021, 23, 100815. [Google Scholar] [CrossRef]
Ahmed, W.; Sheikh, J.A.; Kouzani, A.Z.; Mahmud, M.A. The Role of Single End-Users and Producers on GHG Mitigation in Pakistan—A Case Study. Sustainability 2020, 12, 8351. [Google Scholar] [CrossRef]
Pakistan Energy Resources. Available online: http://ntdc.gov.pk/energy (accessed on 12 April 2021).
Goal 13: Take Urgent Action to Combat Climate Change and Its Impacts. Available online: https://www.un.org/sustainabledevelopment/climate-change/ (accessed on 2 April 2021).
IPCC (Intergovernmental Panel on Climate Change). Climate Change 2007: Synthesis Report. 2007. Available online: http://www.ipcc.ch/ (accessed on 2 April 2021).
Shahsavari, A.; Akbari, M. Potential of solar energy in developing countries for reducing energy-related emissions. Renew. Sustain. Energy Rev. 2018, 90, 275–291. [Google Scholar] [CrossRef]
Balasubramani, G.; Thangavelu, V.; Chinnusamy, M.; Subramaniam, U.; Padmanaban, S.; Mihet-Popa, L. Infrared Thermography Based Defects Testing of Solar Photovoltaic Panel with Fuzzy Rule-Based Evaluation. Energies 2020, 13, 1343. [Google Scholar] [CrossRef] [Green Version]
BLD 290-Watt Solar Panels. Available online: https://www.enfsolar.com/pv/panel-datasheet/crystalline/23907 (accessed on 18 December 2020).
Dimitrievska, V.; Pittino, F.; Muehleisen, W.; Diewald, N.; Hilweg, M.; Montvay, A.; Hirschl, C. Statistical Methods for Degradation Estimation and Anomaly Detection in Photovoltaic Plants. Sensors 2021, 21, 3733. [Google Scholar] [CrossRef]
Dunderdale, C.; Brettenny, W.; Clohessy, C.; van Dyk, E.E. Photovoltaic defect classification through thermal infrared imaging using a machine learning approach. Prog. Photovolt. Res. Appl. 2020, 28, 177–188. [Google Scholar] [CrossRef]
Akram, M.W.; Li, G.; Jin, Y.; Chen, X.; Zhu, C.; Ahmad, A. Automatic detection of photovoltaic module defects in infrared images with isolated and develop-model transfer deep learning. Sol. Energy 2020, 198, 175–186. [Google Scholar] [CrossRef]
Cipriani, G.; D’Amico, A.; Guarino, S.; Manno, D.; Traverso, M.; Di Dio, V. Convolutional Neural Network for Dust and Hotspot Classification in PV Modules. Energies 2020, 13, 6357. [Google Scholar] [CrossRef]
Niazi, K.; Khan, H.A.; Amir, F. Hot-spot reduction and shade loss minimization in crystalline-silicon solar panels. J. Renew. Sustain. Energy 2018, 10, 033506. [Google Scholar] [CrossRef]
Lazzaretti, A.E.; Costa, C.H.D.; Rodrigues, M.P.; Yamada, G.D.; Lexinoski, G.; Moritz, G.L.; Oroski, E.; Goes, R.E.D.; Linhares, R.R.; Stadzisz, P.C.; et al. A monitoring system for online fault detection and classification in photovoltaic plants. Sensors 2020, 20, 4688. [Google Scholar] [CrossRef]
Aziz, F.; Haq, A.U.; Ahmad, S.; Mahmoud, Y.; Jalal, M.; Ali, U. A novel convolutional neural network-based approach for fault classification in photovoltaic arrays. IEEE Access 2020, 8, 41889–41904. [Google Scholar] [CrossRef]
Ul-Haq, A.; Sindi, H.F.; Gul, S.; Jalal, M. Modeling and Fault Categorization in Thin-Film and Crystalline PV Arrays Through Multilayer Neural Network Algorithm. IEEE Access 2020, 8, 102235–102255. [Google Scholar] [CrossRef]
Basnet, B.; Chun, H.; Bang, J. An Intelligent Fault Detection Model for Fault Detection in Photovoltaic Systems. J. Sens. 2020, 2020, 6960328. [Google Scholar] [CrossRef]
Gallardo-Saavedra, S.; Hernández-Callejo, L.; Alonso-García, M.D.C.; Muñoz-Cruzado-Alba, J.; Ballestín-Fuertes, J. Infrared Thermography for the Detection and Characterization of Photovoltaic Defects: Comparison between Illumination and Dark Conditions. Sensors 2020, 20, 4395. [Google Scholar] [CrossRef]
Dhimish, M.; Holmes, V.; Mather, P.; Sibley, M. Novel hot spot mitigation technique to enhance photovoltaic solar panels output power performance. Sol. Energy Mater. Sol. Cells 2018, 179, 72–79. [Google Scholar] [CrossRef] [Green Version]
Ahmed, W.; Sheikh, J.A.; Farjana, S.H.; Mahmud, M.A.P. Defects Impact on PV System GHG Mitigation Potential and Climate Change. Sustainability 2021, 13, 7793. [Google Scholar] [CrossRef]
Ahsan, S.; Niazi, K.A.K.; Khan, H.A.; Yang, Y. Hotspots and performance evaluation of crystalline-silicon and thin-film photovoltaic modules. Microelectron. Reliab. 2018, 88, 1014–1018. [Google Scholar] [CrossRef]
Tang, W.; Yang, Q.; Xiong, K.; Yan, W. Deep learning based automatic defect identification of photovoltaic module using electroluminescence images. Sol. Energy 2020, 201, 453–460. [Google Scholar] [CrossRef]
Roeder, B.; Schlothauer, J.; Koehl, M. Fluorescence imaging for analysis of the degradation of PV-modules. In Proceedings of the 2011 37th IEEE Photovoltaic Specialists Conference, Seattle, WA, USA, 19–24 June 2011; pp. 003606–003608. [Google Scholar]
Trupke, T.; Mitchell, B.; Weber, J.W.; McMillan, W.; Bardos, R.A.; Kroeze, R. Photoluminescence imaging for photovoltaic applications. Energy Procedia 2012, 15, 135–146. [Google Scholar] [CrossRef] [Green Version]
Niazi, K.; Akhtar, W.; Khan, H.A.; Sohaib, S.; Nasir, A.K. Binary classification of defective solar PV modules using thermography. In Proceedings of the 2018 IEEE 7th World Conference on Photovoltaic Energy Conversion (WCPEC) (A Joint Conference of 45th IEEE PVSC, 28th PVSEC & 34th EU PVSEC), Waikoloa, HI, USA, 10–15 June 2018; pp. 753–757. [Google Scholar]
Niazi, K.A.K.; Akhtar, W.; Khan, H.A.; Yang, Y.; Athar, S. Hotspot diagnosis for solar photovoltaic modules using a Naive Bayes classifier. Sol. Energy 2019, 190, 34–43. [Google Scholar] [CrossRef]
Kurukuru, V.B.; Haque, A.; Khan, M.A.; Tripathy, A.K. Fault classification for photovoltaic modules using thermography and machine learning techniques. In Proceedings of the 2019 International Conference on Computer and Information Sciences (ICCIS), Sakaka, Saudi Arabia, 3–4 April 2019; pp. 1–6. [Google Scholar]
Bommes, L.; Pickel, T.; Buerhop-Lutz, C.; Hauch, J.; Brabec, C.; Peters, I.M. Computer vision tool for detection, mapping, and fault classification of photovoltaics modules in aerial IR videos. Prog. Photovolt. Res. Appl. 2021, 1–16. [Google Scholar] [CrossRef]
RETScreen International, RETScreen Software Online User Manual, CANMET Energy Technology Centre, Varennes, Quebec, Canada. 2005. Available online: http://www.nrcan.gc.ca/energy/software-tools/7465 (accessed on 2 April 2021).
Haynes, W.M. (Ed.) CRC Handbook of Chemistry and Physics; CRC press: Boca Raton, FL, USA, 2014. [Google Scholar]
Review of Failures of Photovoltaic Modules. Available online: https://iea-pvps.org/wp-content/uploads/2020/01/IEA-PVPS_T1301_2014_Review_of_Failures_of_Photovoltaic_Modules_Final.pdf (accessed on 15 May 2021).
Booker, L.B.; Goldberg, D.E.; Holland, J.H. Classifier systems and genetic algorithms. Artif. Intell. 1989, 40, 235–282. [Google Scholar] [CrossRef]
Brownlee, J. Deep Learning for Computer Vision: Image Classification, Object Detection, and Face Recognition in Python; Machine Learning Mastery: San Juan, Puerto Rico, 2019. [Google Scholar]
Du, B.; He, Y.; He, Y.; Duan, J.; Zhang, Y. Intelligent classification of silicon photovoltaic cell defects based on eddy current thermography and convolution neural network. IEEE Trans. Ind. Inform. 2019, 16, 6242–6251. [Google Scholar] [CrossRef]
Nakamichi, K.; Lu, H.; Kim, H.; Yoneda, K.; Tanaka, F. Classification of circulating tumor cells in fluorescence microscopy images based on SqueezeNet. In Proceedings of the 2019 19th International Conference on Control, Automation and Systems (ICCAS), Jeju, Korea, 15–18 October 2019; pp. 1042–1045. [Google Scholar]
Zhou, Z.; Zhang, B.; Yu, X. Infrared Handprint Classification Using Deep Convolution Neural Network. Neural Process. Lett. 2021, 53, 1065–1079. [Google Scholar] [CrossRef]
Zhang, X.; Zhou, X.; Lin, M.; Sun, J. Shufflenet: An extremely efficient convolutional neural network for mobile devices. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 6848–6856. [Google Scholar]
Shorten, C.; Khoshgoftaar, T.M. A survey on image data augmentation for deep learning. J. Big Data 2019, 6, 1–48. [Google Scholar] [CrossRef]
Malik, H.; Fatema, N.; Iqbal, A. Intelligent Data-Analytics for Condition Monitoring: Smart Grid Applications; Elsevier: Amsterdam, The Netherlands, 2021. [Google Scholar]

Figure 1. Proposed approach.

Figure 2. Solar panels classification based on health.

Figure 3. Solar panels classification based on defects.

Figure 4. Deep neural network generic architecture for PV classification.

Figure 5. Training loss of different-layered ICNMs.

Figure 6. Training loss of pre-trained networks.

Figure 7. Training loss of models.

Figure 8. Training loss of models for augmented datasets.

Table 1. Geographical parameters of Lahore city, Pakistan.

Parameter	Values
Geographical coordinates	31.5 N, 74.4 E
Air temperature	24.4 °C
Relative humidity	61.6%
Precipitation	551.78 mm
Daily solar radiation—horizontal	4.68 kWh/m²/d
Wind speed at 10 m	2.1 m/s

Table 2. Details of experimental setup.

Parameter	Values
PV system	42.24 kW
PV strings	8
PV modules per string	22
PV panel rating	240 W
Thermal camera	FLIR VUE-Pro 640
Thermal camera position	Handheld, horizontal aligned
Ambient temperature	32–40 °C
Wind speed	6.9 m/s
Irradiance level	700 W/m2
Thermal image- bit depth	8-bit
Spatial resolution	640 × 512/pixel

Table 3. Classification of IR images dataset based on health condition.

PV System Health	Images Set
Healthy	32.38%
Hotspot	31.43%
Faulty	36.19%

Table 4. Details of experimental setup.

PV System Health	Images Set
Bird Drop	5.16%
Single	36.62%
Patchwork	5.16%
HA String	12.68%
Block	40.38%

Table 5. Training parameters used in deep convolution neural networks.

Solver	SGDM
Initial learn rate	0.001
Epochs	60
Momentum	0.9
Activation function	ReLU
Learn rate drop factor	0.0
Learn rate drop period	0.0

Table 6. Different layer ICN models.

Layers	Training Loss	Training Accuracy	Validation Loss	Validation Accuracy	Execution Time
6	0	100%	0	96%	6 min 47 s
7	0	100%	0.2	96%	8 min 35 s
8	0.00033	100%	1.13	84%	17 min 18 s
9	0.18	97.66%	0.46	88%	35 min 34 s

Table 7. Pre-trained models.

Model	Training Loss	Training Accuracy	Validation Loss	Validation Accuracy	Execution Time
SqueezeNet	0.043	99.22%	0.037	100%	12 min 45 s
GoogleNet	0.003	100%	0.06	98.41%	28 min 28 s
ShuffleNet	0.003	100%	0.08	100%	23 min 23 s

Table 8. Defects classification using proposed seven-layered transfer learned ICNM.

Class	TPR	FNR	PPV	FDR	Training Loss	Training Accuracy	Validation Loss	Validation Accuracy	Testing Accuracy	Execution Time
Bird drop	100	0	100	0	0	100%	1.97 × 10⁻⁵	100%	97.62%	8 min 10 s
Patchwork	50	50	100	0
Single	100	0	100	0
String	100	0	100	0
Block	100	0	94.4	5.6

Table 9. Defects classification using pre-trained networks.

Network	Class	TPR	FNR	PPV	FDR	Training Loss	Training Accuracy	Validation Loss	Validation Accuracy	Testing Accuracy	Execution Time
Squeeze Net	Bird drop	100	0	100	0	0.009	100%	0.24	94.12%	100%	12 min
	Patchwork	100	0	100	0
	Single	100	0	100	0
	String	100	0	100	0
	Block	100	0	100	0
Google Net	Bird drop	100	0	100	0	0.013	100%	0.07	97.62%	97.62%	28 min 21 s
	Patchwork	50	50	100	0
	Single	100	0	94.1	5.9
	String	100	0	100	0
	Block	100	0	100	0
Shuffle Net	Bird drop	100	0	100	0	0.003	100%	0.2	94.12%	97.62%	21 min 38 s
	Patchwork	100	0	100	0
	Single	100	0	94.1	5.9
	String	80	20	100	0
	Block	100	0	100	0

Table 10. Augmented data-based defects classification.

Network	Class	TPR	FNR	PPV	FDR	Training Loss	Training Accuracy	Validation Loss	Validation Accuracy	Execution Time
Seven-layered transfer learned model	Bird drop	100	0	100	0	7.02 × 10⁻⁵	100%	1.06	93.3%	11 min 1 s
	Patchwork	100	0	91.7	8.3
	Single	100	0	91.7	8.3
	String	90.9	9.1	100	0
	Block	90.9	9.1	100	0
GoogleNet	Bird drop	90.9	9.1	100	0	0.017	99.22%	0.43	92.7%	40 min 20 s
	Patchwork	100	0	100	0
	Single	100	0	84.6	15.4
	String	100	0	100	0
	Block	90.9	9.1	100	0
ShuffleNet	Bird drop	100	0	100	0	0.0028	100%	0.27	91.1%	31 min 19 s
	Patchwork	100	0	100	0
	Single	100	0	91.7	8.3
	String	100	0	100	0
	Block	90.9	9.1	100	0
Squeeze Net	Bird drop	100	0	100	0	0.00085	100%	1.17	89.9%	17 min 56 s
	Patchwork	100	0	100	0
	Single	100	0	73.3	26.7
	String	90.9	9.1	100	0
	Block	72.7	27.3	100	0

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Ahmed, W.; Hanif, A.; Kallu, K.D.; Kouzani, A.Z.; Ali, M.U.; Zafar, A. Photovoltaic Panels Classification Using Isolated and Transfer Learned Deep Neural Models Using Infrared Thermographic Images. Sensors 2021, 21, 5668. https://doi.org/10.3390/s21165668

AMA Style

Ahmed W, Hanif A, Kallu KD, Kouzani AZ, Ali MU, Zafar A. Photovoltaic Panels Classification Using Isolated and Transfer Learned Deep Neural Models Using Infrared Thermographic Images. Sensors. 2021; 21(16):5668. https://doi.org/10.3390/s21165668

Chicago/Turabian Style

Ahmed, Waqas, Aamir Hanif, Karam Dad Kallu, Abbas Z. Kouzani, Muhammad Umair Ali, and Amad Zafar. 2021. "Photovoltaic Panels Classification Using Isolated and Transfer Learned Deep Neural Models Using Infrared Thermographic Images" Sensors 21, no. 16: 5668. https://doi.org/10.3390/s21165668

APA Style

Ahmed, W., Hanif, A., Kallu, K. D., Kouzani, A. Z., Ali, M. U., & Zafar, A. (2021). Photovoltaic Panels Classification Using Isolated and Transfer Learned Deep Neural Models Using Infrared Thermographic Images. Sensors, 21(16), 5668. https://doi.org/10.3390/s21165668

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu