CN118691989A - A method for detecting aquatic plant coverage based on airborne hyperspectral - Google Patents

Abstract

The present application discloses a method for detecting waterweed coverage based on airborne hyperspectral, which relates to the field of hyperspectral technology. The method obtains sample hyperspectral images through a hyperspectral imaging device carried by an unmanned aerial vehicle, converts the sample hyperspectral images into sample reflectance spectrum curves, and performs band selection to screen out characteristic bands to achieve data dimension reduction, and combines spectral continuum removal transformation to construct fusion features of fused spectral features and spectral indexes, and uses the fusion features to train a waterweed recognition model. The waterweed recognition model obtained by training can be used to perform pixel-level classification detection on the hyperspectral image, thereby quickly obtaining the waterweed coverage detection result. Hyperspectral images can obtain rich spectral information, and the fusion features enable accurate and rapid pixel-level classification even in complex backgrounds, and can automatically and quickly distinguish waterweeds and other aquatic plants with similar colors, so as to achieve waterweed coverage detection, with high automation and accuracy.

Description

A method for detecting aquatic plant coverage based on airborne hyperspectral

Technical Field

The present application relates to the field of hyperspectral technology, and in particular to a method for detecting aquatic plant coverage based on airborne hyperspectral technology.

Background Art

Aquatic plants play an important role in crab pond farming. Not only do they serve as an important food source and shelter for crabs, but lush aquatic plants also help increase the dissolved oxygen content in the water and purify the water quality, effectively preventing soil erosion and reducing water pollution, providing solid support for the healthy development of the crab pond ecosystem. Therefore, testing the growth of aquatic plants is helpful in evaluating the development of the crab pond ecosystem.

Traditional methods for detecting the growth of aquatic plants mainly rely on manual observation, which is time-consuming and labor-intensive, and is subject to significant subjective influences. With the development of technology, remote sensing methods based on spectral analysis technology have also been gradually applied to the field of aquatic plant growth detection. However, the chlorophyll in aquatic plants will make the water body appear green overall. The colors of different aquatic plants in RGB images or multispectral images obtained by remote sensing are very similar, and it is often difficult to accurately analyze and identify aquatic plants from the image, resulting in low detection accuracy and difficulty in popularization.

Summary of the invention

In response to the above problems and technical requirements, this application proposes a method for detecting aquatic plant coverage based on airborne hyperspectral. The technical solution of this application is as follows:

A method for detecting aquatic plant coverage based on airborne hyperspectral, the method comprising:

The sample hyperspectral images of the water area with aquatic plants are taken by the hyperspectral imaging equipment carried by the drone, and the classification results of each pixel point in the sample hyperspectral images are annotated;

The radiation amount of each pixel in the sample hyperspectral image is converted into surface reflectance to obtain the corresponding sample reflectance spectrum curve;

Screen out characteristic bands according to curve characteristics of the sample reflectance spectrum curve;

The spectral index is calculated after performing continuum removal transformation on the sample reflectance spectral curve;

The sample hyperspectral image is divided into several training sample images, and the spectral features of each training sample image in the characteristic band and the spectral index of each training sample image are extracted respectively; the spectral features and spectral index of each training sample image are spliced in the channel dimension to obtain the fusion features of the training sample image;

Taking the fusion features of each training sample image as input and the classification results of the pixels in the training sample image as output, a waterweed recognition model is obtained based on the neural network model training;

The aquatic plant recognition model is used to determine the classification results of each pixel in the hyperspectral image of the water area to be detected, and the proportion of the number of pixels belonging to the aquatic plant category in the total number of pixels is calculated to obtain the aquatic plant coverage of the water area to be detected.

The beneficial technical effects of this application are:

The present application discloses a method for detecting aquatic plant coverage based on airborne hyperspectral. The method uses an unmanned aerial vehicle equipped with a hyperspectral imaging device to collect sample hyperspectral images, performs band selection on the sample hyperspectral images to screen out characteristic bands to achieve data dimension reduction, and combines the spectral continuum removal transformation to construct a fusion feature of fused spectral features and spectral indexes, and uses the fusion features to train an aquatic plant recognition model. The aquatic plant recognition model can be used to perform pixel-level classification detection on the hyperspectral image, thereby quickly obtaining the aquatic plant coverage detection result. The hyperspectral image can obtain rich spectral information. The detection method based on hyperspectral images in the present application can accurately distinguish aquatic plants and other aquatic plants with similar colors, and the application of fusion features enables accurate and rapid pixel-level classification even in complex backgrounds. The method has a high degree of automation and high detection accuracy, and has high practical application value.

This method uses the step-by-step band selection method to screen feature bands, and based on the idea that category separability is inversely proportional to feature correlation, introduces an improved comprehensive JM distance calculation method to evaluate the distinguishability between different categories, which can enhance the difference and separability between aquatic plants and other categories, thereby improving classification accuracy and improving the detection accuracy of aquatic plant coverage.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a method for detecting aquatic plant coverage based on airborne hyperspectral according to an embodiment of the present application.

FIG. 2 is a flow chart of a method for screening characteristic bands using a step-by-step band selection method in one embodiment.

FIG. 3 is a model structure diagram of a neural network model used to train an aquatic plant recognition model.

DETAILED DESCRIPTION

The specific implementation of the present application is further described below in conjunction with the accompanying drawings.

The present application discloses a method for detecting waterweed coverage based on airborne hyperspectral. Please refer to the flowchart shown in FIG1 . The method for detecting waterweed coverage includes:

Step 1: Use the hyperspectral imaging device carried by the drone to capture sample hyperspectral images of water areas with aquatic plants, and annotate the classification results of each pixel in the sample hyperspectral image. The categories to which the pixel points in the sample hyperspectral image belong include aquatic plants and several other categories. The other categories mainly include various other aquatic plant categories, aquatic animal categories, underwater obstacle categories, etc.

Generally, the water area is large, and the hyperspectral imaging equipment often finds it difficult to directly obtain the required sample hyperspectral images in the entire domain. Therefore, in one embodiment, the drone is controlled to fly along multiple parallel routes in sequence, and the single-route remote sensing images of the water area are vertically captured by the onboard hyperspectral imaging equipment during the flight along each route. The single-route remote sensing images captured when flying along multiple routes are sequentially stitched to obtain the sample hyperspectral images. The specific image stitching method is a commonly used image processing technology, and this step will not be repeated.

Step 2, converting the radiation of each pixel in the sample hyperspectral image into surface reflectance to obtain the corresponding sample reflectance spectrum curve. The method of converting the radiation into surface reflectance is a common method in the field and will not be repeated in this step.

Step 3: select characteristic bands according to the curve characteristics of the sample reflectance spectrum curve.

Since there are many bands in the sample hyperspectral image, the amount of data processing will be large, so the characteristic bands are first screened to achieve dimensionality reduction processing of the sample hyperspectral image. In one embodiment, a step-by-step band selection method is used to screen the characteristic bands, including the following steps, please refer to the flowchart of Figure 2:

(1) According to the curve characteristics of the sample reflectance spectrum curve, several representative bands containing large amounts of spectral information are selected, including:

(a) According to the curve characteristics of the sample reflectance spectrum curve, the sample reflectance spectrum curve is divided into a plurality of band subsets, including:

Firstly, the bands are roughly divided based on the peaks and troughs of the sample reflectance spectrum curve to obtain a number of original bands, each of which includes the band range between adjacent peaks and troughs.

Then, according to the curve slope of the sample reflectance spectrum curve in each original band, the original band is subdivided into several band subsets, including: taking the position where the curve slope of the sample reflectance spectrum curve in the original band reaches the slope threshold as the division position, the original band is divided into several band subsets, each band subset includes a continuous band where the curve slope of the sample reflectance spectrum curve in the original band does not exceed the slope threshold. Among them, the slope threshold is a custom preset value.

The smaller the slope of the sample reflectance spectrum curve, the greater the correlation between adjacent bands. In this case, they can be merged into the same band subset. Conversely, the larger the slope of the sample reflectance spectrum curve, the smaller the correlation between the bands. They need to be divided into different band subsets. In this way, each original band can be further divided into multiple band subsets according to the degree of change of the sample reflectance spectrum curve.

(b) Calculate the reflectance standard deviation of the sample reflectance spectral curve in each band subset. The reflectance standard deviation reflects the distribution degree of the sample reflectance spectral curve in the band subset. The larger the reflectance standard deviation, the greater the amount of information and the larger the pixel value distribution range. Therefore, several band subsets with the largest reflectance standard deviation are selected as candidate bands. The number of retained candidate bands can be customized.

(c) Calculate the correlation coefficients between different candidate bands, select several candidate bands with smaller correlation coefficients as representative bands, and filter out multiple candidate bands with larger correlation coefficients. The number of retained representative bands can be customized.

(2) Perform full permutation and combination on multiple representative bands to obtain multiple band combinations, each of which includes several representative bands.

(3) Extract the multispectral image of the sample hyperspectral image under each band combination, and calculate the OIF value of the multispectral image and the comprehensive JM distance J _new .

The OIF (optimal index factor) value of the multispectral image can be calculated using a standard calculation formula, which will not be described in detail in this embodiment. The comprehensive JM distance J _new of the multispectral image is used to characterize the degree of distinguishability between the aquatic plant category and other categories. The larger the comprehensive JM distance J _new , the higher the degree of distinguishability between the aquatic plant category and other categories.

The comprehensive JM distance J _new of the multispectral image is calculated based on the JM distance between the aquatic plant category and any other j-th category. Therefore, the JM distance JM _j between the aquatic plant category and any other j-th category is first calculated according to the multispectral image. The specific calculation formula adopts the standard calculation formula, which will not be repeated in this embodiment.

When dealing with multi-class problems, traditional methods usually calculate the average value of the JM distances between different classes to characterize the degree of distinguishability between different classes. However, the method of calculating the average value of the JM distances will comprehensively consider the JM distances between each class, and only evaluate the degree of distinguishability from the overall perspective, which conceals the problem that the separation between classes with smaller JM distances is smaller. In order to solve this problem, this embodiment calculates the comprehensive JM distance J _new based on the idea that the degree of class distinguishability is inversely proportional to the feature correlation:

Where n is the total number of categories contained in the sample hyperspectral image. The i-th category represents the waterweed category, so j≠i. _Rj is the Pearson correlation coefficient between the waterweed category determined based on the multispectral image and the j-th category, and:

Wherein, _Xik is the sum of the pixel values of the pixels belonging to the aquatic plant category in the k-th band of the multispectral image, and _Xjk is the sum of the pixel values of the pixels belonging to the j-th category in the k-th band of the multispectral image; is the average value of _Xik in each band of the multispectral image, is the average value of X _jk of each band in the multispectral image, and K is the total number of bands contained in the multispectral image.

(4) Determine the representative band in the band combination corresponding to the multispectral image whose comprehensive JM distance J _new reaches the distance threshold and whose OIF value is the largest as the characteristic band.

Step 4, calculate the spectral index after performing a continuum removal transformation on the sample reflectance spectral curve. The continuum removal transformation of the spectrum can enhance the reflectance spectral difference between aquatic plants and other categories. This application analyzes the characteristics of the aquatic plant growth environment and selects the vegetation index with a greater contribution to calculate the spectral index, including: respectively calculating the enhanced vegetation index, photochemical reaction index, normalized water index and green light normalized vegetation index, and merging them to obtain a high-dimensional matrix as the spectral index.

Step 5: Divide the sample hyperspectral image into several training sample images, extract the spectral features of each training sample image in the characteristic band and the spectral index of each training sample image, and then concatenate the spectral features and spectral index of each training sample image in the channel dimension to obtain the fusion features of the training sample image.

In one embodiment, a sliding cutting window method is used to divide the training sample image, which includes: defining the side length of the cutting window and the sliding step length to be x. Then, the sample hyperspectral image containing H rows of pixels is added Row background pixels, add to the sample hyperspectral image containing W columns of pixels The background pixels are added to obtain the expanded sample hyperspectral image, and the surface reflectance of the added background pixels is 0. Finally, the expanded sample hyperspectral image is subjected to sliding cutting using a cutting window, and the image within the cutting window is extracted as a training sample image. Express Round up, Express Round up.

Step 6, using the fusion features of each training sample image as input and the classification results of the pixels in the training sample image as output, a waterweed recognition model is obtained based on neural network model training.

In one embodiment, the model structure of the neural network model used to train the aquatic plant recognition model is shown in FIG3 . The neural network model includes a downsampling module, a feature extraction module, an upsampling feature fusion module, and a segmentation prediction module:

The downsampling module consists of three cascaded convolutional layers. The downsampling module is used to extract features from the input image and obtain three sets of feature maps at different levels, with sizes of the input image and The first two layers of feature maps and the original input image are used as inputs to the three branches of the upsampling feature fusion module, and the last layer of feature maps with the smallest size is input to the feature extraction module.

The feature extraction module includes multiple cascaded transformer modules. The core of each transformer module is a multi-head self-attention module that integrates channel attention. The channel attention module mainly extracts pixel relationships in different bands, while the multi-head self-attention module mainly improves the model's ability to express global pixel relationships. Then, the output feature map of the channel attention module, the multi-head self-attention module, and the input feature map are integrated to generate an intermediate feature map, and the transformer module feature map is output again through the MLP module.

The upsampling feature fusion module integrates multi-scale feature map information to improve the resolution of high-level feature maps while reconstructing image detail features. This module is mainly divided into local and global branches, which learn the local and global pixel relationships of feature maps respectively. For the local branch, a linear transformation is performed in the width dimension, and the same feature points are densely distributed in the local area after the dimension is adjusted. For the global branch, a linear transformation is performed in the height dimension, and the feature points are evenly distributed in the global range after the dimension is adjusted. The feature maps of the two branches are fused and the channel dimension is adjusted to output the feature map of the upsampling feature fusion module.

The segmentation prediction module adjusts the feature map channel dimension to the predicted category through the convolution layer and outputs the pixel classification result.

Step 7, using the aquatic plant recognition model to determine the classification results of each pixel in the hyperspectral image of the water area to be detected, and calculating the proportion of the number of pixels belonging to the aquatic plant category in the total number of pixels to obtain the aquatic plant coverage of the water area to be detected.

The method for acquiring the hyperspectral image of the water area to be detected is similar to the method for acquiring the sample hyperspectral image in step 1, and this embodiment will not be repeated. After acquiring the hyperspectral image of the water area to be detected, the fusion features of the hyperspectral image are extracted. The method for extracting the fusion features is similar to the method for obtaining the fusion features of each training sample image in the training stage. The fusion features of the hyperspectral image of the water area to be detected are input into the aquatic plant recognition model to obtain the prediction results of the category to which each pixel point in the hyperspectral image of the water area to be detected belongs.

The above is only a preferred embodiment of the present application, and the present application is not limited to the above embodiments. It is understood that other improvements and changes directly derived or associated by those skilled in the art without departing from the spirit and concept of the present application should be considered to be included in the scope of protection of the present application.

Claims (10)

1. The aquatic weed coverage detection method based on the airborne hyperspectrum is characterized by comprising the following steps of:

shooting a sample hyperspectral image of a water area growing with waterweeds through hyperspectral imaging equipment carried by an unmanned aerial vehicle, and marking classification results of all pixel points in the sample hyperspectral image;

Converting the radiation quantity of each pixel point in the sample hyperspectral image into the surface reflectivity to obtain a corresponding sample reflectivity spectrum curve;

screening out characteristic wave bands according to curve characteristics of the sample reflectivity spectrum curve;

calculating a spectrum index after continuously removing and transforming the sample reflectivity spectrum curve;

Dividing the sample hyperspectral image into a plurality of training sample images, and respectively extracting spectral characteristics of each training sample image under the characteristic wave band and spectral indexes of each training sample image; splicing the spectral features and the spectral indexes of each training sample image in the channel dimension to obtain the fusion features of the training sample images;

taking the fusion characteristic of each training sample image as input, and the classification result of the pixel points in the training sample image as output, and training based on a neural network model to obtain a water plant recognition model;

And determining a classification result of each pixel point in the hyperspectral image of the water area to be detected by using the water grass recognition model, and calculating the ratio of the number of the pixel points belonging to the water grass category in the total number of the pixel points to obtain the water grass coverage of the water area to be detected.

2. The method for detecting aquatic weed coverage according to claim 1, wherein the screening out the characteristic band according to the curve characteristic of the sample reflectance spectrum curve comprises:

Screening out a plurality of representative wave bands with larger spectral information content according to the curve characteristics of the sample reflectivity spectrum curve;

Performing full permutation and combination on the plurality of representative wave bands to obtain a plurality of wave band combinations, wherein each wave band combination comprises a plurality of representative wave bands;

Extracting a multispectral image of the sample hyperspectral image under each band combination, and calculating an OIF value of the multispectral image and a comprehensive JM distance J _new, wherein the comprehensive JM distance J _new is used for representing the distinguishable degree between the aquatic weed category and other categories, and the greater the comprehensive JM distance is, the higher the distinguishable degree between the aquatic weed category and other categories is;

and determining a representative wave band in a wave band combination corresponding to the multispectral image with the largest OIF value, wherein the distance J _new of the comprehensive JM reaches a distance threshold value, as a characteristic wave band.

3. The aquatic weed coverage detection method according to claim 2, wherein calculating the integrated JM distance J _new for each multispectral image comprises:

according to the multispectral image, JM distance JM _j between the aquatic weed category and any other j-th category is calculated;

Calculating the comprehensive JM distance of the multispectral image as N is the total number of categories contained in the sample hyperspectral image, R _j is the pearson correlation coefficient between the aquatic weed category determined based on the multispectral image and any other j-th category, and the i-th category is the aquatic weed category.

4. A method of detecting aquatic weed coverage according to claim 3, characterized in that the pearson correlation coefficient R _j between the aquatic weed category determined based on the multispectral image and any other j-th category is:

Wherein, X _ik is the sum of the pixel values of the pixel points belonging to the aquatic weed class in the kth wave band of the multispectral image, and X _jk is the sum of the pixel values of the pixel points belonging to the jth class in the kth wave band of the multispectral image; Is the average value of X _ik for each band in the multispectral image, Is the average value of X _jk of each band in the multispectral image, and K is the total number of the bands contained in the multispectral image.

5. The method for detecting the coverage of aquatic weeds according to claim 2, wherein the step of screening out a plurality of representative bands with larger spectral information content according to the curve characteristics of the sample reflectivity spectrum curve comprises the following steps:

Performing band division on the sample reflectivity spectrum curve into a plurality of band subsets according to the curve characteristics of the sample reflectivity spectrum curve;

Calculating the standard deviation of the reflectivity of the sample reflectivity spectrum curve in each band subset, and screening out a plurality of band subsets with the maximum standard deviation of the reflectivity as candidate bands;

and calculating correlation coefficients among different candidate wave bands, and screening out a plurality of candidate wave bands with smaller correlation coefficients as representative wave bands.

6. The aquatic weed coverage detection method of claim 5, wherein band division of the sample reflectance spectrum curve into a plurality of band subsets comprises:

Performing band coarse division based on peaks and troughs of the sample reflectivity spectrum curve to obtain a plurality of original bands, wherein each original band comprises a band range between adjacent peaks and troughs;

and according to the curve slope of the sample reflectivity spectrum curve in each original wave band, carrying out wave band subdivision on the original wave band into a plurality of wave band subsets.

7. The aquatic weed coverage detection method of claim 6 wherein band subdivision for each original band comprises:

Dividing the original wave band into a plurality of wave band subsets by taking the position, where the slope of the sample reflectivity spectrum curve in the original wave band reaches the slope threshold, as a dividing position, wherein each wave band subset comprises continuous wave bands, where the slope of the sample reflectivity spectrum curve in the original wave band does not exceed the slope threshold.

8. The aquatic weed coverage detection method of claim 1, wherein calculating a spectral index comprises:

And respectively calculating an enhanced vegetation index, a photochemical reaction index, a normalized water index and a green light normalized vegetation index, and then merging to obtain a high-dimensional matrix as a spectrum index.

9. The aquatic weed coverage detection method according to claim 1, wherein the photographing of the sample hyperspectral image of the aquatic weed growing water area by the hyperspectral imaging device mounted on the unmanned aerial vehicle comprises:

And controlling the unmanned aerial vehicle to fly along a plurality of parallel airlines in sequence, vertically shooting single-airline remote sensing images of a water area through carried hyperspectral imaging equipment in the flying process along each airline, and sequentially performing image stitching on the single-airline remote sensing images shot during flying along the plurality of airlines to obtain the sample hyperspectral image.

10. The aquatic weed coverage detection method according to claim 1, wherein dividing the sample hyperspectral image into a number of training sample images comprises:

defining the side length of a cutting window and the sliding step length as x;

adding to a sample hyperspectral image containing H rows of pixels Row background pixel points, adding to a sample hyperspectral image containing W columns of pixel pointsThe background pixel points are listed to obtain an expanded sample hyperspectral image, and the surface reflectivity of the added background pixel points is 0; wherein, Representation pairThe round-up is carried out upwards,Representation pairRounding upwards;

and performing sliding cutting on the extended sample hyperspectral image by using a cutting window, and extracting an image in the cutting window as a training sample image.