Nothing Special   »   [go: up one dir, main page]
Next Article in Journal
PCO-Based BLE Mesh Accelerator
Previous Article in Journal
A RUL Estimation System from Clustered Run-to-Failure Degradation Signals
You seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 22
Issue 14
10.3390/s22145322
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Deep Learning Approach for Detection of Underground Natural Gas Micro-Leakage Using Infrared Thermal Images

by
Kangni Xiong
1,
Jinbao Jiang
1,*,
Yingyang Pan
1,
Yande Yang
1,
Xuhui Chen
2 and
Zijian Yu
1
1
College of Geoscience and Surveying Engineering, China University of Mining and Technology, Beijing 100083, China
2
Satellite Application Center for Ecology and Environment, Beijing 100094, China
*
Author to whom correspondence should be addressed.
Sensors 2022, 22(14), 5322; https://doi.org/10.3390/s22145322
Submission received: 14 June 2022 / Revised: 7 July 2022 / Accepted: 14 July 2022 / Published: 16 July 2022
(This article belongs to the Section Sensing and Imaging)
Browse Figures
Figure 1
<p>The sketch map of the experimental area; the numbers in a 20 m × 5 m plot are marked leak point, and the 2.5 m × 2.5 m plots with odd numbers represent the experimental plots whereas these with even numbers represent control plots without natural gas.</p> "> Figure 2
<p>(<b>a</b>) Gas concentration data collection using GXH-3050E gas analyzer; (<b>b</b>) Infrared thermal images acquisition using Fluke ti55ft thermal imager.</p> "> Figure 3
<p>The architecture of the GLNB model.</p> "> Figure 4
<p>Architecture of the inception.</p> "> Figure 5
<p>Infrared thermal images of experimental groups: (<b>a</b>) grass; (<b>c</b>) soybean; (<b>e</b>) corn and (<b>g</b>) wheat. Infrared thermal images of control groups: (<b>b</b>) grass; (<b>d</b>) soybean; (<b>f</b>) corn and (<b>h</b>) wheat.</p> "> Figure 6
<p>Temperature profiles of experimental groups: (<b>a</b>) grass; (<b>c</b>) soybean; (<b>e</b>) corn and (<b>g</b>) wheat. Temperature profiles of control groups: (<b>b</b>) grass; (<b>d</b>) soybean; (<b>f</b>) corn and (<b>h</b>) wheat.</p> "> Figure 7
<p>(<b>a</b>) Spatial relationship between methane concentration in soil and stressed vegetation areas. (<b>b</b>) Correlation analysis results of methane concentration and canopy temperature.</p> "> Figure 8
<p>The classification accuracy and cross-entropy loss of the two models: (<b>a</b>) the GLNB; (<b>b</b>) the GLNB-based transfer learning model (GLNB-TL).</p> "> Figure 9
<p>Visualization of the used areas in random input images for classifier: (<b>a</b>) grass; (<b>b</b>) soybean; (<b>c</b>) corn and (<b>d</b>) wheat. Each image from left to right is the visual heat map based on Grad-CAM and the heat map fitting circle.</p> "> Figure 10
<p>Temporal and spatial variation of stressed vegetation range under natural gas micro-leakage: (<b>a</b>) grass; (<b>b</b>) soybean; (<b>c</b>) corn and (<b>d</b>) wheat.</p> ">
Review Reports Versions Notes

Abstract

:
The leakage of underground natural gas has a negative impact on the environment and safety. Trace amounts of gas leak concentration cannot reach the threshold for direct detection. The low concentration of natural gas can cause changes in surface vegetation, so remote sensing can be used to detect micro-leakage indirectly. This study used infrared thermal imaging combined with deep learning methods to detect natural gas micro-leakage areas and revealed the different canopy temperature characteristics of four vegetation varieties (grass, soybean, corn and wheat) under natural gas stress from 2017 to 2019. The correlation analysis between natural gas concentration and canopy temperature showed that the canopy temperature of vegetation increased under gas stress. A GoogLeNet model with Bilinear pooling (GLNB) was proposed for the classification of different vegetation varieties under natural gas micro-leakage stress. Further, transfer learning is used to improve the model training process and classification efficiency. The proposed methods achieved 95.33% average accuracy, 95.02% average recall and 95.52% average specificity of stress classification for four vegetation varieties. Finally, based on Grad-Cam and the quasi-circular spatial distribution rules of gas stressed areas, the range of natural gas micro-leakage stress areas under different vegetation and stress durations was detected. Taken together, this study demonstrated the potential of using thermal infrared imaging and deep learning in identifying gas-stressed vegetation, which was of great value for detecting the location of natural gas micro-leakage.

1. Introduction

Natural gas, as a clean energy source, has continued to attain sustained growth in consumption in recent years [1,2]. With the popularity of natural gas, the increased risk of natural gas leakage during transport and distribution has also attracted widespread attention. As the main mode of natural gas transportation, the buried pipeline may be destroyed to cause leakage by corrosion, construction damage, anthropic activities, and natural disasters [3,4], which subsequently leads to personal injury, financial losses, security, and ecological issues [5]. Therefore, it is of great significance to detect natural gas leakage timely and accurately.
Traditional natural gas leakage detection mainly includes manual inspection [6], sound monitoring [7], gas sampling, soil detection [8], flow monitoring [9], software-based dynamic modeling monitoring [10], etc. However, these methods not only consume a lot of manpower, financial resources and time, but also cause certain damage to pipelines, gas storage, and the surrounding environment [11]. Infrared thermography is increasingly used in non-destructive condition monitoring services as a less labor-intensive and non-contact technique to assess vegetation stress by leaf canopy temperature [12,13,14,15]. Canopy temperature has been widely used to detect stress because the closing of the stomas on the leaves is controlled to retain water during the stress, resulting in changes in stomatal conductance, transpiration, and leaf temperature [16,17,18,19,20,21]. Some research found that the canopy temperature between the stressed vegetation and healthy vegetation was different [22,23,24,25]. Joalland et al. [23] found that sugar beets infected by nematodes exhibited the leaf temperature higher than healthy plants. Tian et al. [26] found that the canopy temperature of vegetation under salt stress increased significantly with increasing salinity in the soil. Maes et al. [24] found that the canopy temperature of mistletoe-infected trees was significantly higher than that of uninfected trees. Therefore, canopy temperature can be supported as a viable indicator of vegetation health [27,28]. Currently, infrared thermography is widely used in water stress [27,29,30,31,32,33,34], salinity stress [26], metal stress [35] and crop disease [36,37]. But little attention is focused on natural gas stress.
The study of canopy temperature is mostly based on the average data measured from areas or points, which may be more useful than individual leaf temperatures in detecting vegetation stress [38]. However, it must also be considered that the canopy temperature of stressed vegetation exhibits great spatial variability [39]. The development of image processing and deep learning technology provides a new method for extracting canopy temperature and predicting vegetation stress [40]. Deep learning methods represented by convolutional neural networks (CNN) avoid the labor-intensive feature extraction process, which show great superiority. Meanwhile, deep learning methods can extract more accurate spatial thermal features of the vegetation canopy from infrared thermal images to achieve higher classification ability, which has been widely used in precision agriculture [39,40]. Maimaitijiang et al. [41] fused multi-source remote sensing data including infrared thermal images to predict soybean yield by the deep learning methods. King and Shellie [42] proposed neural network models to detect the water stress of grapes by the leaf temperature. Deep learning methods provide indicators for vegetation growth monitoring under different environmental conditions. In addition, transfer learning as a method of deep learning can effectively use the features learned from large data sets for applications with small training data sets, which can solve the problem that deep learning models need enough training data [43,44,45]. Jiang et al. [46] proposed a method based on deep learning and thermal imaging, which transfer learning was used to pretrain mask RCNN network for plant segmentation and temperature extraction. Therefore, it is valuable to extract vegetation canopy information from infrared thermal images using a deep learning approach for vegetation stress detection.
The aim of this study was to detect natural gas micro-leakage stress areas in four vegetation varieties using infrared thermal images with deep learning methods. The objectives were as follows: (a) to explore the correlation between gas concentration and vegetation canopy temperature under natural gas micro-leakage stress; (b) to classify the natural gas stress of four species of vegetation based on the CNN; (c) to improve the efficiency of the classification model based on transfer learning; (d) to detect natural gas micro-leakage areas and time series analysis.

2. Materials and Methods

2.1. Study Area and Site Description

The impact of natural gas micro-leakage on vegetation was studied by field simulation experiments to indirectly detect natural gas micro-leakage areas. The experiment field was located in Daxing District, Beijing, with coordinates of 39°39′2.56″ N, 116°34′33.10″ E. As shown in Figure 1, the data were obtained from two different types of plots. The first measurement area was a 20 m × 5 m plot with eight gas leakage points and planted grass. The second measurement area was set up with twenty-four plots of 2.5 m × 2.5 m. Wheat, soybean, grass, and corn were planted alternately with seasons. There were eight plots for each plant species, including four control plots and four experimental plots. Each plot was separated by a narrow road with a width of 50 cm. A 1.5 m wide road was reserved between each row of plots as the data acquisition channel. The plots with odd numbers represented the experimental plots while even numbers represented control plots without natural gas. The leakage rate of each experimental plot was controlled in 1 L/min by the gas mass flow controller for the whole day.
This experiment lasted from 2017 to 2019. Each plot received the same standard agronomic practices such as watering, fertilization, weeding, pest control and uniform seedling during the experiment, which can ensure that the vegetation was unacted on the other controllable factors.

2.2. Test Setup and Experimental Procedure

In this experiment, ten gas concentration sampling points were evenly distributed in the experimental plot to determine the concentration of natural gas in the soil. Three hollow PVC pipes were inserted into each sampling point, with a depth of 60 cm underground. Since the main component of natural gas is methane, a GXH-3050E gas analyzer (JunFangLiHua, Beijing, China) was used to collect the methane concentration of each point every three days during the whole experimental period (Figure 2a). The average of the methane concentration at each point was taken to draw the contour map of methane volume fraction by MATLAB (Mathworks, Natick, MA, USA), which was used to clarify the spatial distribution law of natural gas micro-leakage.
Fluke ti55ft thermal imager (Fluke Corporation, Washington, DC, USA) was used for data acquisition (Figure 2b). The thermal imager can obtain both infrared thermal and visible images with a 320 × 240 fixed pixel array. The thermal sensitivity is less than or equal to 0.05 °C, and the temperature range is from −20 °C to 600 °C. Images were taken from 9:00–11:00 AM local time in cloudless weather and steady solar radiation intensity. The lens of the thermal imager should be kept vertically downward. The vertical distance between the lens and the vegetation canopy was 5 m. Each plot was photographed for approximately 30 s. Two or three images of each plot were collected to obtain the infrared thermal images of the control or experimental plot.

2.3. Image Pre-Processing

The infrared thermal images taken in the field were processed in smartview4.3 software (Fluke Corporation, Washington, DC, USA). The data collected from field experiments needed to be removed for singular values before comparative analysis due to the influence of metal marker points. In addition, deep learning approaches require a large number of training samples to learn the weights and parameters in the training phase [47,48]. Due to the limited infrared thermal data collected in the field, the training dataset was enhanced by using random flip, random angle rotation, cropping, color transformation, and noise transformation. Finally, 7571 images were obtained for model training. 1820 images and 428 images were used as the validation and test sets, respectively.

2.4. GLNB Architectures for Classification

Compared with the traditional model, the deep learning approaches provide better results for the recognition and classification of agricultural images [49]. In this paper, the GLNB model incorporating GoogLeNet and Bilinear pooling was proposed to detect stressed vegetation. The detailed structure is shown in Figure 3. GoogLeNet structure consisted of nine Inception modules connected in series. The structure of Inception is shown in Figure 4. The Inception module added 1 × 1 convolution before 3 × 3 convolution and 5 × 5 convolution, and 1 × 1 convolutional layer after 3 × 3 max pooling. This structure made more efficient use of computational resources by incorporating feature information of different scales, in which different features were extracted with the same amount of computation to improve the training results. Furthermore, the bilinear pool captured the relationship between features by obtaining the outer product of two vectors to model the higher-order statistical information of the features [50]. The outer product captured the pairwise correlation between feature channels in a translation-invariant manner. Bilinear pooling could be optimized end-to-end, which provided a stronger feature representation than linear models. The GLNB model was trained with infrared thermal images for 200 epochs in Python 3.7 using PyTorch backend.

2.5. Implementation of the Transfer Learning

Transfer learning pre-trains the model and retains the parameters of the model. Then, a small amount of target data is used to retrain the fully connected layer. This approach can optimize the learning efficiency of the model and improves the model performance [51,52]. However, not all data are equally informative. In this article, data containing similar partial information were selected as samples for transfer learning. The visible images in this experiment, ImageNet [53] and the public infrared thermal dataset [54,55,56,57] were used as the transfer learning training set for pre-training the GLNB model. Then, the parameters were saved and imported to the model with fine-tuning.

2.6. Visual Explanations and Gradient-Based Localization

Gradient-weighted Class Activation Mapping (Grad-CAM) assigns the gradient information of the last convolution layer to each neuron and generates a location map to highlight the important region in the image, which is the basis of output prediction [58]. The last convolution layer should capture the picture features more specifically. Grad-CAM uses the gradient information from the last convolution layer in CNN to understand the importance of each neuron for category recognition. For category c, the weight ( α k c ) of the k th neuron in the last convolutional layer is calculated as follows:
α k c = 1 Z ∑ i ∑ j ∂ y c ∂ A i j k
Equation (1) can be intuitively understood as: take the input ( y c ) of the softmax layer and calculate the partial derivative of the i j th input pixel value of the k th neuron in the last convolution layer. Then, the k th neuron weight ( α k c ) in the last convolution layer is obtained by the global average pooling. For category c, the classification localization map ( L G r a d − C A M c ) is calculated as follows:
L G r a d − C A M c = R e L U   ( ∑ k α k c A k )
Equation (2) can be intuitively understood as: the input of different neurons in the last convolutional layer are multiplied by their respective neuron weights. Then the corresponding values of all neurons are summed. The R e L U retains the pixel values that have a positive effect on the classification and suppresses the pixel values that have a negative effect on the classification, and then a heat map, i.e., the classification localization map ( L G r a d − C A M c ) is obtained.

2.7. Performance Evaluation

In this paper, the confusion matrix was used to evaluate the classification performance of the CNN model. Accuracy represents the proportion of all correctly matched samples to all samples. However, when the number among different samples is unbalanced, there is a great error in using the accuracy as the index. Multiple evaluation indexes being used to comprehensively evaluate the model can more truly reflect the effect of the model. Therefore, the accuracy, recall and specificity were used to evaluate the performance of the model in this study. The following is the formula:
A c c u r a c y = T P + T N T P + T N + F P + F N
R e c a l l = T P T P + F N  
S p e c i f i c i t y = T N T N + F P  
where, True Positive ( T P ) represents correctly identified experiment plots; False Negative ( F N ) indicates unsuccessfully identified experiment plots; False Positive ( F P ) represents control plots identified as experiment plots; and True Negative ( T N ) indicates correctly identified control plots.

3. Results and Discussion

3.1. Stress Symptoms

The infrared thermal image could not cover the whole plot because of the small field angle of the infrared thermal imager. But it could cover the main stress area in the experimental plot. Infrared thermal images of the four vegetation in the experimental plot and the control plot are shown in Figure 5, which are grass (Cynodon dactylon L.), soybean (Glycine max Merr.), corn (Zea mays L.) and wheat (Triticum aestivum L.), respectively. Soybean, wheat and corn are one of the main crops in China. Grass is a good plant for Embankment Consolidation and soil conservation, which is often used to pave lawn for urban greening. The four types of vegetation have different characteristics and are widely planted in China.
As shown in Figure 5, two cross-sectional lines NE-SW and NW-SE were selected on the image. The temperature value of cross-sectional pixels was extracted to draw the temperature curve for analysis. Figure 6 clearly shows that the temperature variation of NE-SW and NW-SE in the experimental plot was high in the middle and low on both sides, while the temperature in the control plot showed a stable trend. In addition, the mean, standard deviation, maximum and minimum values of temperature in the profiles were counted to compare the differences of vegetation temperature between the experiment plots and control plots.
According to the statistical characteristic values of temperature distribution (Table 1), the mean, standard deviation and maximum temperature of the experimental plot were higher than those of the control plot, but the minimum temperature was close to equal. This indicated that the temperature fluctuation in the experimental plot was more drastic, and little temperature fluctuation in the control plot. The maximum temperature difference of the experimental plots among the four types of vegetation was greater than 10 °C, and the average temperature difference between the experimental plot and the control plot ranged from 3.10 °C to 3.55 °C. Combined with Figure 6, different vegetation affected by natural gas micro-leakage stress produced similar temperature characteristics and spatial characteristics, i.e., the vegetation temperature in the central area affected by natural gas leakage was higher than that in the edge, and the temperature values showed spatial characteristics of high in the middle and low in the sides. This is similar to the research of Noomen [59]. This indicated that the poor growth of vegetation under the stress of natural gas leakage led to great changes in its temperature field, which was consistent with the vegetation growth observed in the field.
The infrared thermal image was matched and superimposed with the gas concentration distribution map as shown in Figure 7a. It was found that the concentration distribution contours of the natural gas diffusing in the soil was roughly quasi-circular. This was because the leakage of natural gas led to the loss of oxygen in the soil, which inhibited the function of roots [60]. Due to the long-term impact of high concentration of natural gas, the vegetation respiration in the area where the gas concentration was more than 30% was weakened and showed stress symptoms such as Leaf Chlorosis and growth retardation, which made the canopy temperature of stressed vegetation higher than that of healthy vegetation and showed a quasi-circular stress area in space. The 236 random points in the superposition part of the infrared thermal image and gas concentration distribution map were selected to extract the corresponding temperature values and gas concentration values for correlation analysis. As shown in Figure 7b, the temperature of the experiment plot was positively correlated with the gas concentration and the root mean squared error was 0.87. Therefore, the canopy temperature distribution characteristics of vegetation were consistent with the gas concentration distribution, i.e., both showed quasi-circular distribution. This further confirmed that the difference in canopy temperature was caused by the natural gas leakage. In addition, the range of stressed vegetation areas might vary due to different plant tolerances to natural gas, but the spatial features remained similar. Therefore, this kind of circular spatial feature can be used for the identification of vegetation stressed by natural gas micro-leakage.

3.2. Performance of the GLNB Method

Infrared thermal images of four plants were input into the GLNB model for 200 epochs training. It was found by many attempts that a large learning rate could cause the loss curve to oscillate and the model failed to fit, while a small learning rate could cause slow convergence of the loss curve and overfitting of the model. Therefore, 0.0001 was applied to the proposed model as the appropriate learning rate. Adam made the model converge faster by comparing the performance between Stochastic Gradient Descent (SGD) and Adam, so Adam was applied to the proposed model. In addition, four classical CNN models (AlexNet [61], VGG16 [62], GoogLeNet [63] and ResNet [64]) were developed for comparison. Results as shown in Table 2, the five CNN models effectively identified stressed vegetation plots under natural gas micro-leakage. The proposed GLNB model with 91.59% average classification accuracy achieved better results than other models. Furthermore, the recall and the specificity of average per class in the proposed model were 92.32% and 91.07%, respectively, which were higher than other models (AlexNet: 90.16% and 86.09%, VGG16: 87.03% and 88.40%, GoogLeNet: 85.87% and 86.97%, and ResNet: 86.51% and 83.59%). This indicated that the proposed GLNB model performed superior to identify the stressed vegetation plots under the natural gas micro-leakage. This is because the model includes Inception structure and bilinear pooling, which integrates feature information of different scales and makes use of the difference of second-order statistical information to improve the performance of the model.
In the proposed GLNB model, grass got a better classification accuracy (95.33%) than other kinds of plants (soybean: 90.65%, corn: 88.79%, and wheat: 91.59%). Similar results were obtained in the other four classical models. Grass had a higher stress recognition rate rather than soybean, corn and wheat, which showed more sensitive to the nature gas. There may be the following reasons. The different canopy structure makes the irregular stress shape of corn, soybean and wheat, which makes large identification errors. Corns are taller with sparse planting spacing, which is easy to produce shadow and causes recognition error. Wheats have tufted and thin stalks, which is easy to cause uneven canopy coverage. Soybeans have strong rhizomes with large coverage of single plant. Meanwhile, the soybean rhizobia are easy to affect the stress symptoms of natural gas. Ferchichi [65] found that soybean rhizobia promoted vegetation growth and helped plants resist the impact of adverse environment, which made stress symptoms less pronounced in soybean. Grass canopy is denser and more uniformly covered, while the shallow root system is more sensitive to the increased gas concentration in the soil. In contrast, other plants have the more developed root system than grass, which could better adapt to the environment. Other researchers had done similar experiments on vegetation stress. Smith, K.L [66] found that wheat and beans were not affected when gas was delivered to well-developed wheat and beans. This might be because the plant had developed a complete root system that allowed its roots to obtain nutrients and water from outside the gas-affected area. Therefore, the grass can be used as a good indicator to detect natural gas micro-leakage areas.

3.3. Comparison with the Transfer Learning

The GLNB-based transfer learning model (GLNB-TL) was developed to use public datasets as the training set to pre-train the model and obtained the corresponding model structure and weights. The iterative curves of the GLNB and GLNB-TL during training are shown in Figure 8. The accuracy and loss curves were used to determine the fitting situation of the training model to optimize the model parameters. As the number of iterations increased, the accuracy of the training and validation sets gradually rose while the loss value gradually dropped. Finally, both accuracy and loss curves tended to be stable. Comparing with the iterative curves of the GLNB and GLNB-TL, it was found that GLNB-TL could converge with fewer epochs and obtained lower loss value and higher accuracy. Therefore, transfer learning makes the training model converge quickly and improved the training efficiency.
The test results of GLNB and GLNB-TL are presented in Table 3. The GLNB-TL achieved the average classification accuracy of 95.33%, which was higher than the GLNB model. The recall and specificity of average per class also performed well with the 95.02% and 95.52% classification scores, respectively. Grass had a better classification accuracy (98.13%) than other kinds of plants (soybean: 96.26%, corn: 91.59%, and wheat: 95.33%). Comparing the performance between GLNB and GLNB-TL, it was found that the GLNB-TL model had the better classification performance in a variety of plants, which showed that the transfer learning method improved the performance and generalization ability of the model. This is because the weights of the pre-trained model store key information to identify the basic feature of the stressed vegetation in the proposed model. The transfer learning could customize its own weights for identifying the stressed vegetation in the infrared thermal dataset faster and easier. Therefore, transfer learning has application value in the identification of the natural gas micro-leakage stressed vegetation.

3.4. Detection and Time-Series Analysis

Grad-CAM was used to generate the heatmap of class activation in this paper, which visualized the classification features of the input image. The last convolutional layer captured more specific features of the image, so Grad-CAM used the gradient information from the final convolutional layer of the model to generate heat map to understand the importance of each neuron for category recognition. Figure 9 shows the visualization results. The proposed GLNB model in this paper correctly extracted the features of stressed vegetation from the infrared thermal image and located the micro-leakage area of natural gas. Therefore, this proposed model can identify stressed vegetation and detect natural gas micro-leakage.
Infrared thermal images of five periods were selected for each vegetation type. The stressed vegetation areas caused by natural gas micro-leakage were extracted by Grad-CAM and segmented based on the adaptive threshold method. Vegetation stress areas were consistent with natural gas micro-leakage areas, which had been demonstrated in Section 3.1. So the natural gas micro-leakage areas were located by drawing fitted circles based on the quasi-circular spatial distribution rules of gas stressed areas. The temporal and spatial variation of the range of stressed vegetation areas under natural gas micro-leakage is shown in Figure 10. Due to the influence of continuous natural gas micro-leakage, grass and wheat showed growth retardation and even death. The concentration of natural gas in soil gradually accumulated and diffused with time, so the stress range of grass and wheat showed a trend of gradually expanding. The range of stress in soybean showed a gradual decrease with the duration of stress. This is because both the canopy and the root system of the soybean seedling stage were not well developed, which leaded to its sensitive to the external environment. Therefore, the stress range of soybean was large in the early stage. With the growth and development of soybean, the developed root systems leaded to the stable growth situation, which improved the tolerance to the adverse external growth environment. Meanwhile, the effect of soybean rhizobia was also one of the reasons for the decreasing stress range. The stress range of corn first increased and then decreased with the duration of stress. This was because the developed root system of corn made a high tolerance to the adverse growth environment, resulting in the slow increase of the stress range. But in the later stage, the tall stalks and the large leaf of the corn covered the stress area and generated shadows to affect the temperature extraction by the thermal imager, making a decreasing trend of the stress range on the infrared thermal image.
Combined with the stress range of vegetation in different periods and their mean stress radii, the mean stress radius of grass was the largest. And the stress range showed a trend of steady expansion, which was consistent with the diffusion and accumulation pattern of natural gas leaked into the soil. The large density and the uniform canopy structure of grass and the more obvious stress symptoms make it easier to detect the stress caused by natural gas micro-leakage in the wild. Furthermore, the rhizomes of grass are highly spreading and well adapted to the field environment, which can reduce the time and cost of manual maintenance. Therefore, grass can be used as a good indicator to detect natural gas micro-leakage.

4. Conclusions

In this study, infrared thermal images and natural gas concentration data from 2017, 2018, and 2019 were used to evaluate vegetation growth during the natural gas micro-leakage in the Daxing District of Beijing. It was found that the spatial distribution of natural gas in the soil was basically consistent with the temperature features of the stressed vegetation in the infrared thermal images. A deep learning method was proposed to identify stressed vegetation, thereby the area of natural gas micro-leakage can be detected indirectly. The proposed GLNB model for stress detection on grass, corn, soybean and wheat presented the better performance comparing with four classical CNN models. Furthermore, transfer learning was used to pretrain the GLNB model and the result showed that this method improved classification efficiency and detection accuracy. Finally, the range of stressed vegetation under natural gas micro-leakage was located based on GLNB and Grad-CAM. The time series analysis results showed that the generality and high efficiency of the proposed GLNB model made it applicable to multiple periods and various field plants, which overcame the challenge in terms of irregularity in the outdoor environment. However, there is still a scope for improvement. The response of stressed vegetation in canopy temperature is slow, so hyperspectral remote sensing technology can be added to study the spectral reflectance of vegetation for the early detection of natural gas micro-leakage areas. In addition, large-scale detection of natural gas micro-leakage can be realized by UAV technology as a future scope.

Author Contributions

Conceptualization, J.J. and K.X.; methodology, K.X.; software, K.X.; validation, K.X., Y.Y. and X.C.; formal analysis, J.J.; investigation, J.J.; resources, J.J.; data curation, Y.P.; writing—original draft preparation, K.X.; writing—review and editing, K.X.; visualization, K.X.; supervision, Z.Y.; project administration, J.J.; funding acquisition, J.J. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by National Natural Science Foundation of China (41871341, 41571412).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Acknowledgments

The authors would like to express their gratitude to the funding agencies, the editor and reviewers for leveraging the quality of this work.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Rui, Z.; Han, G.; Zhang, H.; Wang, S.; Pu, H.; Ling, K. A new model to evaluate two leak points in a gas pipeline. J. Nat. Gas Sci. Eng. 2017, 46, 491–497. [Google Scholar] [CrossRef]
  2. Bu, F.; Liu, Y.; Liu, Y.; Xu, Z.; Chen, S.; Jiang, M.; Guan, B. Leakage diffusion characteristics and harmful boundary analysis of buried natural gas pipeline under multiple working conditions. J. Nat. Gas Sci. Eng. 2021, 94, 104047. [Google Scholar] [CrossRef]
  3. Parvini, M.; Gharagouzlou, E. Gas leakage consequence modeling for buried gas pipelines. J. Loss Prev. Process Indust. 2015, 37, 110–118. [Google Scholar] [CrossRef]
  4. van den Brand, D.; Kutrowski, K.H. Safety in European gas transmission pipelines EGIG: European gas pipeline incident data group. In Proceedings of the 23rd World Gas Conference, Amsterdam, The Netherlands, 5–9 June 2006. [Google Scholar]
  5. Badida, P.; Balasubramaniam, Y.; Jayaprakash, J. Risk evaluation of oil and natural gas pipelines due to natural hazards using fuzzy fault tree analysis. J. Nat. Gas Sci. Eng. 2019, 66, 284–292. [Google Scholar] [CrossRef]
  6. Boaz, L.; Kaijage, S.; Sinde, R. An overview of pipeline leak detection and location systems. In Proceedings of the 2nd Pan African International Conference on Science, Computing and Telecommunications (PACT 2014), Arusha, Tanzania, 14–18 July 2014. [Google Scholar]
  7. Jin, H.; Zhang, L.; Liang, W.; Ding, Q. Integrated leakage detection and localization model for gas pipelines based on the acoustic wave method. J. Loss Prev. Process Indust. 2014, 27, 74–88. [Google Scholar] [CrossRef]
  8. Yang, C.; Romanak, K.D.; Reedy, R.C.; Hovorka, S.D.; Trevino, R.H. Soil gas dynamics monitoring at a CO2-EOR site for leakage detection. Geomech. Geophys. Geo-Energy Geo-Resour. 2017, 3, 351–364. [Google Scholar] [CrossRef]
  9. Abdulshaheed, A.; Mustapha, F.; Ghavamian, A. A pressure-based method for monitoring leaks in a pipe distribution system: A Review. Renew. Sustain. Energy Rev. 2017, 69, 902–911. [Google Scholar] [CrossRef]
  10. dos Santos, P.L.; Azevedo-Perdicoulis, T.-P.; Ramos, J.A.; de Carvalho, J.L.M.; Jank, G.; Milhinhos, J. An LPV Modeling and Identification Approach to Leakage Detection in High Pressure Natural Gas Transportation Networks. IEEE Trans. Control Syst. Technol. 2011, 19, 77–92. [Google Scholar] [CrossRef]
  11. Xu, J.; Nie, Z.; Shan, F.; Li, J.; Luo, Y.; Yuan, Q.; Chen, H. Leak Detection Methods Overview and Summary. In Proceedings of the ICPTT 2012: Better Pipeline Infrastructure for a Better Life, Wuhan, China, 19–22 October 2012; pp. 1034–1050. [Google Scholar]
  12. King, B.A.; Tarkalson, D.D.; Sharma, V.; Bjorneberg, D.L. Thermal Crop Water Stress Index Base Line Temperatures for Sugarbeet in Arid Western U.S. Agric. Water Manag. 2021, 243, 106459. [Google Scholar] [CrossRef]
  13. Banerjee, K.; Krishnan, P. Normalized Sunlit Shaded Index (NSSI) for characterizing the moisture stress in wheat crop using classified thermal and visible images. Ecol. Indic. 2020, 110, 105947. [Google Scholar] [CrossRef]
  14. Poirier-Pocovi, M.; Volder, A.; Bailey, B.N. Modeling of reference temperatures for calculating crop water stress indices from infrared thermography. Agric. Water Manag. 2020, 233, 106070. [Google Scholar] [CrossRef]
  15. Appiah, S.A.; Li, J.; Lan, Y.; Darko, R.O.; Alordzinu, K.E.; Al Aasmi, A.; Asenso, E.; Issaka, F.; Afful, E.A.; Wang, H.; et al. Real-Time Assessment of Mandarin Crop Water Stress Index. Sensors 2022, 22, 4018. [Google Scholar] [CrossRef] [PubMed]
  16. Zia, S.; Romano, G.; Spreer, W.; Sanchez, C.; Cairns, J.; Araus, J.L.; Müller, J. Infrared Thermal Imaging as a Rapid Tool for Identifying Water-Stress Tolerant Maize Genotypes of Different Phenology. J. Agron. Crop Sci. 2013, 199, 75–84. [Google Scholar] [CrossRef]
  17. Leinonen, I.; Jones, H.G. Combining thermal and visible imagery for estimating canopy temperature and identifying plant stress. J. Exp. Bot. 2004, 55, 1423–1431. [Google Scholar] [CrossRef] [Green Version]
  18. Ballester, C.; Jiménez-Bello, M.A.; Castel, J.R.; Intrigliolo, D.S. Usefulness of thermography for plant water stress detection in citrus and persimmon trees. Agric. For. Meteorol. 2013, 168, 120–129. [Google Scholar] [CrossRef]
  19. Sobejano-Paz, V.; Mikkelsen, T.N.; Baum, A.; Mo, X.; Liu, S.; Köppl, C.J.; Johnson, M.S.; Gulyas, L.; García, M. Hyperspectral and Thermal Sensing of Stomatal Conductance, Transpiration, and Photosynthesis for Soybean and Maize under Drought. Remote Sens. 2020, 12, 3182. [Google Scholar] [CrossRef]
  20. Xu, R.; Li, C.; Bernardes, S. Development and Testing of a UAV-Based Multi-Sensor System for Plant Phenotyping and Precision Agriculture. Remote Sens. 2021, 13, 3517. [Google Scholar] [CrossRef]
  21. Alordzinu, K.E.; Li, J.; Lan, Y.; Appiah, S.A.; Al Aasmi, A.; Wang, H. Rapid Estimation of Crop Water Stress Index on Tomato Growth. Sensors 2021, 21, 5142. [Google Scholar] [CrossRef]
  22. Joalland, S.; Screpanti, C.; Varella, H.; Reuther, M.; Schwind, M.; Lang, C.; Walter, A.; Liebisch, F. Aerial and Ground Based Sensing of Tolerance to Beet Cyst Nematode in Sugar Beet. Remote Sens. 2018, 10, 787. [Google Scholar] [CrossRef] [Green Version]
  23. Joalland, S.; Screpanti, C.; Liebisch, F.; Varella, H.V.; Gaume, A.; Walter, A. Comparison of visible imaging, thermography and spectrometry methods to evaluate the effect of Heterodera schachtii inoculation on sugar beets. Plant Methods 2017, 13, 73. [Google Scholar] [CrossRef]
  24. Maes, W.; Huete, A.; Avino, M.; Boer, M.; Dehaan, R.; Pendall, E.; Griebel, A.; Steppe, K. Can UAV-Based Infrared Thermography Be Used to Study Plant-Parasite Interactions between Mistletoe and Eucalypt Trees? Remote Sens. 2018, 10, 2062. [Google Scholar] [CrossRef] [Green Version]
  25. Smigaj, M.; Gaulton, R.; Suarez, J.; Barr, S. Use of Miniature Thermal Cameras for Detection of Physiological Stress in Conifers. Remote Sens. 2017, 9, 957. [Google Scholar] [CrossRef] [Green Version]
  26. Tian, F.; Hou, M.; Qiu, Y.; Zhang, T.; Yuan, Y. Salinity stress effects on transpiration and plant growth under different salinity soil levels based on thermal infrared remote (TIR) technique. Geoderma 2020, 357, 113961. [Google Scholar] [CrossRef]
  27. Mangus, D.L.; Sharda, A.; Zhang, N. Development and evaluation of thermal infrared imaging system for high spatial and temporal resolution crop water stress monitoring of corn within a greenhouse. Comput. Electron. Agric. 2016, 121, 149–159. [Google Scholar] [CrossRef]
  28. Savian, F.; Martini, M.; Ermacora, P.; Paulus, S.; Mahlein, A.-K. Prediction of the Kiwifruit Decline Syndrome in Diseased Orchards by Remote Sensing. Remote Sens. 2020, 12, 2194. [Google Scholar] [CrossRef]
  29. Buitrago, M.F.; Groen, T.A.; Hecker, C.A.; Skidmore, A.K. Changes in thermal infrared spectra of plants caused by temperature and water stress. ISPRS J. Photogramm. Remote Sens. 2016, 111, 22–31. [Google Scholar] [CrossRef]
  30. Egea, G.; Padilla-Díaz, C.M.; Martinez-Guanter, J.; Fernández, J.E.; Pérez-Ruiz, M. Assessing a crop water stress index derived from aerial thermal imaging and infrared thermometry in super-high density olive orchards. Agric. Water Manag. 2017, 187, 210–221. [Google Scholar] [CrossRef] [Green Version]
  31. Ondimu, S.; Murase, H. Water stress detection in Sunagoke moss (Rhacomitrium canescens) using combined thermal infrared and visible light imaging techniques. Biosys. Eng. 2008, 100, 4–13. [Google Scholar] [CrossRef]
  32. Hu, T.; van Dijk, A.I.J.M.; Renzullo, L.J.; Xu, Z.; He, J.; Tian, S.; Zhou, J.; Li, H. On agricultural drought monitoring in Australia using Himawari-8 geostationary thermal infrared observations. Int. J. Appl. Earth Obs. Geoinf. 2020, 91, 102153. [Google Scholar] [CrossRef]
  33. Neinavaz, E.; Schlerf, M.; Darvishzadeh, R.; Gerhards, M.; Skidmore, A.K. Thermal infrared remote sensing of vegetation: Current status and perspectives. Int. J. Appl. Earth Obs. Geoinf. 2021, 102, 102415. [Google Scholar] [CrossRef]
  34. Struthers, R.; Ivanova, A.; Tits, L.; Swennen, R.; Coppin, P. Thermal infrared imaging of the temporal variability in stomatal conductance for fruit trees. Int. J. Appl. Earth Obs. Geoinf. 2015, 39, 9–17. [Google Scholar] [CrossRef]
  35. Liu, M.; Liu, X.; Zhang, B.; Ding, C. Regional heavy metal pollution in crops by integrating physiological function variability with spatio-temporal stability using multi-temporal thermal remote sensing. Int. J. Appl. Earth Obs. Geoinf. 2016, 51, 91–102. [Google Scholar] [CrossRef]
  36. Hatton, N.; Sharda, A.; Schapaugh, W.; van der Merwe, D. Remote thermal infrared imaging for rapid screening of sudden death syndrome in soybean. Comput. Electron. Agric. 2020, 178, 105738. [Google Scholar] [CrossRef]
  37. Zhu, W.; Chen, H.; Ciechanowska, I.; Spaner, D. Application of infrared thermal imaging for the rapid diagnosis of crop disease. IFAC-PapersOnLine 2018, 51, 424–430. [Google Scholar] [CrossRef]
  38. Grant, O.M.; Tronina, L.; Jones, H.G.; Chaves, M.M. Exploring thermal imaging variables for the detection of stress responses in grapevine under different irrigation regimes. J. Exp. Bot. 2007, 58, 815–825. [Google Scholar] [CrossRef] [PubMed]
  39. Luan, Y.; Xu, J.; Lv, Y.; Liu, X.; Wang, H.; Liu, S. Improving the performance in crop water deficit diagnosis with canopy temperature spatial distribution information measured by thermal imaging. Agric. Water Manag. 2021, 246, 106699. [Google Scholar] [CrossRef]
  40. Zhou, Z.; Majeed, Y.; Diverres Naranjo, G.; Gambacorta, E.M.T. Assessment for crop water stress with infrared thermal imagery in precision agriculture: A review and future prospects for deep learning applications. Comput. Electron. Agric. 2021, 182, 106019. [Google Scholar] [CrossRef]
  41. Maimaitijiang, M.; Sagan, V.; Sidike, P.; Hartling, S.; Esposito, F.; Fritschi, F.B. Soybean yield prediction from UAV using multimodal data fusion and deep learning. Remote Sens. Environ. 2020, 237, 111599. [Google Scholar] [CrossRef]
  42. King, B.A.; Shellie, K.C. Evaluation of neural network modeling to predict non-water-stressed leaf temperature in wine grape for calculation of crop water stress index. Agric. Water Manag. 2016, 167, 38–52. [Google Scholar] [CrossRef]
  43. Zhang, M.; Wang, D.; Lu, W.; Yang, J.; Li, Z.; Liang, B. A Deep Transfer Model With Wasserstein Distance Guided Multi-Adversarial Networks for Bearing Fault Diagnosis under Different Working Conditions. IEEE Access 2019, 7, 65303–65318. [Google Scholar] [CrossRef]
  44. Li, X.; Zhang, W.; Ma, H.; Luo, Z.; Li, X. Partial transfer learning in machinery cross-domain fault diagnostics using class-weighted adversarial networks. Neural. Netw. 2020, 129, 313–322. [Google Scholar] [CrossRef] [PubMed]
  45. Yang, B.; Lee, C.-G.; Lei, Y.; Li, N.; Lu, N. Deep partial transfer learning network: A method to selectively transfer diagnostic knowledge across related machines. Mech. Syst. Signal Process. 2021, 156, 107618. [Google Scholar] [CrossRef]
  46. Jiang, Y.; Shuang, L.; Li, C.; Paterson, A.H.; Robertson, J. Deep learning for thermal image segmentation to measure canopy temperature of Brassica oleracea in the field. In Proceedings of the 2018 ASABE Annual International Meeting, Detroit, MI, USA, 29 July–1 August 2018. [Google Scholar]
  47. Esgario, J.G.M.; Krohling, R.A.; Ventura, J.A. Deep learning for classification and severity estimation of coffee leaf biotic stress. Comput. Electron. Agric. 2020, 169, 105162. [Google Scholar] [CrossRef] [Green Version]
  48. Nasiri, A.; Taheri-Garavand, A.; Omid, M.; Carlomagno, G.M. Intelligent fault diagnosis of cooling radiator based on deep learning analysis of infrared thermal images. Appl. Therm. Eng. 2019, 163, 114410. [Google Scholar] [CrossRef]
  49. Singh, A.K.; Ganapathysubramanian, B.; Sarkar, S.; Singh, A. Deep Learning for Plant Stress Phenotyping: Trends and Future Perspectives. Trends Plant Sci. 2018, 23, 883–898. [Google Scholar] [CrossRef] [Green Version]
  50. Tenenbaum, J.B.; Freeman, W.T. Separating style and content with bilinear models. Neural Comput. 2000, 12, 1247–1283. [Google Scholar] [CrossRef] [PubMed]
  51. Lu, G.; Yan, Y.; Ren, L.; Saponaro, P.; Sebe, N.; Kambhamettu, C. Where am I in the dark: Exploring active transfer learning on the use of indoor localization based on thermal imaging. Neurocomputing 2016, 173, 83–92. [Google Scholar] [CrossRef]
  52. Li, W.; Huang, R.; Li, J.; Liao, Y.; Chen, Z.; He, G.; Yan, R.; Gryllias, K. A perspective survey on deep transfer learning for fault diagnosis in industrial scenarios: Theories, applications and challenges. Mech. Syst. Signal Process. 2022, 167, 108487. [Google Scholar] [CrossRef]
  53. Deng, J.; Dong, W.; Socher, R.; Li, L.J.; Kai, L.; Li, F.-F. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 20–25 June 2009; pp. 248–255. [Google Scholar]
  54. Alexander, T.; Maarten, A.H. Progress in color night vision. Opt. Eng. 2012, 51, 010901. [Google Scholar] [CrossRef]
  55. Portmann, J.; Lynen, S.; Chli, M.; Siegwart, R. People detection and tracking from aerial thermal views. In Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 31 May–7 June 2014; pp. 1794–1800. [Google Scholar]
  56. Davis, J.W.; Keck, M.A. A Two-Stage Template Approach to Person Detection in Thermal Imagery. In Proceedings of the 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION’05), Breckenridge, CO, USA, 5–7 January 2005; Volume 1, pp. 364–369. [Google Scholar]
  57. Davis, J.W.; Sharma, V. Background-subtraction using contour-based fusion of thermal and visible imagery. Comput. Vis. Image Underst. 2007, 106, 162–182. [Google Scholar] [CrossRef]
  58. Selvaraju, R.R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; Batra, D. Grad-CAM: Visual Explanations From Deep Networks via Gradient-Based Localization. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 618–626. [Google Scholar] [CrossRef] [Green Version]
  59. Noomen, M.F.; Smith, K.L.; Colls, J.J.; Steven, M.D.; Skidmore, A.K.; Van Der Meer, F.D. Hyperspectral indices for detecting changes in canopy reflectance as a result of underground natural gas leakage. Int. J. Remote Sens. 2008, 29, 5987–6008. [Google Scholar] [CrossRef]
  60. Smith, K.L.; Steven, M.D.; Colls, J.J. Plant spectral responses to gas leaks and other stresses. Int. J. Remote Sens. 2007, 26, 4067–4081. [Google Scholar] [CrossRef]
  61. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet Classification with Deep Convolutional Neural Networks. Commun. ACM 2017, 60, 84–90. [Google Scholar] [CrossRef]
  62. Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
  63. Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going Deeper with Convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015; pp. 1–9. [Google Scholar]
  64. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
  65. Ferchichi, N.; Toukabri, W.; Boularess, M.; Smaoui, A.; Mhamdi, R.; Trabelsi, D. Isolation, identification and plant growth promotion ability of endophytic bacteria associated with lupine root nodule grown in Tunisian soil. Arch. Microbiol. 2019, 201, 1333–1349. [Google Scholar] [CrossRef]
  66. Smith, K.L.; Steven, M.D.; Colls, J.J. Use of hyperspectral derivative ratios in the red-edge region to identify plant stress responses to gas leaks. Remote Sens. Environ. 2004, 92, 207–217. [Google Scholar] [CrossRef]
Sensors 22 05322 g001 550
Figure 1. The sketch map of the experimental area; the numbers in a 20 m × 5 m plot are marked leak point, and the 2.5 m × 2.5 m plots with odd numbers represent the experimental plots whereas these with even numbers represent control plots without natural gas.
Figure 1. The sketch map of the experimental area; the numbers in a 20 m × 5 m plot are marked leak point, and the 2.5 m × 2.5 m plots with odd numbers represent the experimental plots whereas these with even numbers represent control plots without natural gas.
Sensors 22 05322 g001
Sensors 22 05322 g002 550
Figure 2. (a) Gas concentration data collection using GXH-3050E gas analyzer; (b) Infrared thermal images acquisition using Fluke ti55ft thermal imager.
Figure 2. (a) Gas concentration data collection using GXH-3050E gas analyzer; (b) Infrared thermal images acquisition using Fluke ti55ft thermal imager.
Sensors 22 05322 g002
Sensors 22 05322 g003 550
Figure 3. The architecture of the GLNB model.
Figure 3. The architecture of the GLNB model.
Sensors 22 05322 g003
Sensors 22 05322 g004 550
Figure 4. Architecture of the inception.
Figure 4. Architecture of the inception.
Sensors 22 05322 g004
Sensors 22 05322 g005 550
Figure 5. Infrared thermal images of experimental groups: (a) grass; (c) soybean; (e) corn and (g) wheat. Infrared thermal images of control groups: (b) grass; (d) soybean; (f) corn and (h) wheat.
Figure 5. Infrared thermal images of experimental groups: (a) grass; (c) soybean; (e) corn and (g) wheat. Infrared thermal images of control groups: (b) grass; (d) soybean; (f) corn and (h) wheat.
Sensors 22 05322 g005
Sensors 22 05322 g006 550
Figure 6. Temperature profiles of experimental groups: (a) grass; (c) soybean; (e) corn and (g) wheat. Temperature profiles of control groups: (b) grass; (d) soybean; (f) corn and (h) wheat.
Figure 6. Temperature profiles of experimental groups: (a) grass; (c) soybean; (e) corn and (g) wheat. Temperature profiles of control groups: (b) grass; (d) soybean; (f) corn and (h) wheat.
Sensors 22 05322 g006
Sensors 22 05322 g007 550
Figure 7. (a) Spatial relationship between methane concentration in soil and stressed vegetation areas. (b) Correlation analysis results of methane concentration and canopy temperature.
Figure 7. (a) Spatial relationship between methane concentration in soil and stressed vegetation areas. (b) Correlation analysis results of methane concentration and canopy temperature.
Sensors 22 05322 g007
Sensors 22 05322 g008 550
Figure 8. The classification accuracy and cross-entropy loss of the two models: (a) the GLNB; (b) the GLNB-based transfer learning model (GLNB-TL).
Figure 8. The classification accuracy and cross-entropy loss of the two models: (a) the GLNB; (b) the GLNB-based transfer learning model (GLNB-TL).
Sensors 22 05322 g008
Sensors 22 05322 g009 550
Figure 9. Visualization of the used areas in random input images for classifier: (a) grass; (b) soybean; (c) corn and (d) wheat. Each image from left to right is the visual heat map based on Grad-CAM and the heat map fitting circle.
Figure 9. Visualization of the used areas in random input images for classifier: (a) grass; (b) soybean; (c) corn and (d) wheat. Each image from left to right is the visual heat map based on Grad-CAM and the heat map fitting circle.
Sensors 22 05322 g009
Sensors 22 05322 g010 550
Figure 10. Temporal and spatial variation of stressed vegetation range under natural gas micro-leakage: (a) grass; (b) soybean; (c) corn and (d) wheat.
Figure 10. Temporal and spatial variation of stressed vegetation range under natural gas micro-leakage: (a) grass; (b) soybean; (c) corn and (d) wheat.
Sensors 22 05322 g010
Table
Table 1. Statistical characteristic values of temperature in the section of experimental and control plots.
Table 1. Statistical characteristic values of temperature in the section of experimental and control plots.
Vegetation TypePlotsMean/°CStandard Deviation/°CMinimum/°CMaximum/°C
GrassExperimental plot31.073.3526.0037.13
Control plot27.530.6726.0629.44
SoybeanExperimental plot35.364.0729.6346.38
Control plot32.020.9630.0634.88
CornExperimental plot30.563.5126.1338.38
Control plot27.460.5225.4429.00
WheatExperimental plot30.982.9725.3138.38
Control plot27.581.4825.3134.81
Table
Table 2. Classification efficiency comparison among different CNN models.
Table 2. Classification efficiency comparison among different CNN models.
Vegetation TypeEvaluation IndicatorsAlexNetVGG16GoogLeNetResNetGLNB
GrassAccuracy (%)94.3990.6589.7291.5995.33
Recall (%)96.1592.3188.4690.3896.15
Specificity (%)92.7389.0990.9192.7394.55
SoybeanAccuracy (%)85.0582.2486.9279.4490.65
Recall (%)85.7176.7983.9378.5789.29
Specificity (%)84.3188.2490.2080.3992.16
CornAccuracy (%)85.9886.9281.3183.1888.79
Recall (%)90.6387.5081.2590.6390.63
Specificity (%)84.0086.6781.3380.0088.00
WheatAccuracy (%)85.9890.6587.8584.1191.59
Recall (%)88.1491.5389.8386.4493.22
Specificity (%)83.3389.5885.4281.2589.58
Average per-classAccuracy (%)87.8587.6286.4584.5891.59
Recall (%)90.1687.0385.8786.5192.32
Specificity (%)86.0988.4086.9783.5991.07
Table
Table 3. Comparison of the classification performance between GLNB and GLNB-TL.
Table 3. Comparison of the classification performance between GLNB and GLNB-TL.
Vegetation TypeEvaluation IndicatorsGLNBGLNB-TL
GrassAccuracy (%)95.3398.13
Recall (%)96.1598.08
Specificity (%)94.5598.18
SoybeanAccuracy (%)90.6596.26
Recall (%)89.2996.43
Specificity (%)92.1696.08
CornAccuracy (%)88.7991.59
Recall (%)90.6390.63
Specificity (%)88.0092.00
WheatAccuracy (%)91.5995.33
Recall (%)93.2294.92
Specificity (%)89.5895.83
Average per-classAccuracy (%)91.5995.33
Recall (%)92.3295.02
Specificity (%)91.0795.52
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Xiong, K.; Jiang, J.; Pan, Y.; Yang, Y.; Chen, X.; Yu, Z. Deep Learning Approach for Detection of Underground Natural Gas Micro-Leakage Using Infrared Thermal Images. Sensors 2022, 22, 5322. https://doi.org/10.3390/s22145322

AMA Style

Xiong K, Jiang J, Pan Y, Yang Y, Chen X, Yu Z. Deep Learning Approach for Detection of Underground Natural Gas Micro-Leakage Using Infrared Thermal Images. Sensors. 2022; 22(14):5322. https://doi.org/10.3390/s22145322

Chicago/Turabian Style

Xiong, Kangni, Jinbao Jiang, Yingyang Pan, Yande Yang, Xuhui Chen, and Zijian Yu. 2022. "Deep Learning Approach for Detection of Underground Natural Gas Micro-Leakage Using Infrared Thermal Images" Sensors 22, no. 14: 5322. https://doi.org/10.3390/s22145322

APA Style

Xiong, K., Jiang, J., Pan, Y., Yang, Y., Chen, X., & Yu, Z. (2022). Deep Learning Approach for Detection of Underground Natural Gas Micro-Leakage Using Infrared Thermal Images. Sensors, 22(14), 5322. https://doi.org/10.3390/s22145322

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop