CN118736436A - A crop recognition method based on multispectral satellite images - Google Patents

Abstract

The present invention provides a crop recognition method based on multispectral satellite images, and relates to the field of image processing technology. The present invention first obtains the original sample set, uses the adaptive filtering method for preprocessing, uses the hyperspectral remote sensing image as the label image, performs spatial and spectral downsampling on the hyperspectral and multispectral images, and obtains a training data set. The present invention combines hyperspectral and multispectral images, makes full use of spectral and spatial information, and can improve the accuracy of crop recognition.

Description

A crop recognition method based on multispectral satellite images

Technical Field

The present invention relates to the technical field of image processing, and in particular to a crop recognition method based on multispectral satellite images.

Background Art

Hyperspectral remote sensing images (HSI) are widely used in environmental monitoring, land classification, and disaster detection due to their rich spectral resolution. However, HSI usually has low spatial resolution due to the physical conditions of the sensor.

In comparison, multispectral remote sensing images (MSI) have lower spectral resolution but higher spatial resolution, and can more completely describe the morphology and distribution of land cover.

The invention patent with application number 201810901457.8 discloses a crop identification method based on multispectral satellite images, including the following steps: S1, collecting crop samples; S2, obtaining multispectral satellite image data of the crop samples; S3, determining the pixel corresponding to the crop sample on the multispectral satellite image data through the collection position of the crop sample; S4, using the time series spectral information of the pixel and the crop type of the crop sample as input to train the machine learning model; S5, using the trained machine learning model to classify crops in other sampling areas. This patent uses the time series spectral information of the pixel as the input for training the machine learning model, which not only greatly expands the amount of crop spectral information and solves the problem of insufficient crop spectral information at a single moment, but also identifies crops from the spectral information of the entire growth cycle of the crop, which is more accurate than the identification at a single moment, thereby improving the efficiency of crop identification. However, it only uses multispectral satellite image data, and fails to consider the rich spectral resolution in hyperspectral remote sensing images, resulting in inaccurate identification results.

Summary of the invention

In order to overcome the deficiencies of the prior art, the object of the present invention is to provide a crop identification method based on multispectral satellite images.

To achieve the above object, the present invention provides the following solutions:

A crop identification method based on multispectral satellite images, comprising:

Acquire an original sample set; the original sample set includes hyperspectral remote sensing images and multispectral remote sensing images corresponding to each sample crop area;

Preprocessing the original sample set by using an adaptive filtering method to obtain a processed sample set;

The hyperspectral remote sensing image in the processing sample set is used as a label image, and the hyperspectral remote sensing image and the multispectral remote sensing image in the processing sample set are downsampled in terms of space and spectrum respectively to obtain a training data set for network parameter adjustment;

Inputting the training data set into the initial network model for training to obtain a trained crop recognition model;

The data to be tested is input into the crop recognition model to obtain a recognition result.

Preferably, the original sample set is preprocessed using an adaptive filtering method to obtain a processed sample set, including:

A square window is taken with each pixel point of the hyperspectral remote sensing image and the multispectral remote sensing image in the original sample set as the center, and the change amount of each pixel point under the square window is detected; the calculation formula for detecting the change amount of each pixel point under the square window is: Wherein, _fx ( _xi , _yj ) represents the gradient value of the hyperspectral remote sensing image and the multispectral remote sensing image in the original sample set in the x direction, _fy ( _xi , _yj ) represents the gradient value of the hyperspectral remote sensing image and the multispectral remote sensing image in the original sample set in the y direction, f( _xi , _yj ) represents the grayscale value of the hyperspectral remote sensing image and the multispectral remote sensing image in the original sample set at the position ( _xi , _yj ), and r represents the change amount of the pixel point f( _xi , _yj );

When the change amount of the pixel point is greater than the preset threshold, the median value of the pixel point in the square window is used to smooth the hyperspectral remote sensing image and the multispectral remote sensing image in the corresponding square window to obtain the smoothed pixel point;

The square window is continuously moved until the smoothing process of the hyperspectral remote sensing image and the multispectral remote sensing image in the entire original sample set is completed to obtain a smoothed image; the output formula of the smoothed pixel point is: I _denoised (x, y) = median (I _original (xh, yh), ..., I _original (x + h, y + h)); wherein, I _denoised (x, y) is the smoothed pixel point, h is the side length of the square window, and I _original (x, y) is the value of the pixel point on the hyperspectral remote sensing image and the multispectral remote sensing image in the original sample set at the position (x, y);

constructing a pixel enhancement function based on the convolution value of the smoothed image;

The pixel enhancement function is used to perform image enhancement processing on the images in the original sample set to obtain the processed sample set.

Preferably, constructing a pixel enhancement function based on the convolution value of the smoothed image comprises:

Performing convolution processing on the smoothed image to obtain a convolved image;

Determine the enhancement coefficient based on the pixel value of the convolved image;

A pixel enhancement function is constructed based on the enhancement coefficient; wherein the pixel enhancement function is:

Wherein, S(x,y) represents the image in the processing sample set, E(x,y) represents the enhancement coefficient, G(x,y) represents the pixel value of the convolved image at (x,y), I(x,y) represents the smoothed image, and σ represents the mean square error between the smoothed image and the convolved image.

Preferably, the hyperspectral remote sensing image in the processing sample set is used as a label image, and the hyperspectral remote sensing image and the multispectral remote sensing image in the processing sample set are downsampled in space and spectrum respectively to obtain a training data set for network parameter adjustment, including:

According to a preset Wald protocol, Gaussian filtering is performed on the hyperspectral remote sensing image and the multispectral remote sensing image in the processing sample set to obtain a filtered image;

The filtered image is then downsampled by a corresponding multiple using a bilinear interpolation method so that the processed image is used as a hyperspectral remote sensing image and a multispectral remote sensing image that simulate a low-resolution input, and the original hyperspectral image is retained as a reference image.

Preferably, the method for constructing the crop identification model includes:

Performing image processing on the acquired training data set to obtain a sample image;

The sample image is input into a CNN network with a multi-layer residual block for feature extraction, and a maximum pooling layer is used for dimensionality reduction to obtain a local feature map after dimensionality reduction;

The local feature map after dimensionality reduction is input into the feature map embedding module to obtain a one-dimensional vector with position information;

Input the one-dimensional vector into the Transformer block and repeat 24 times to obtain global feature fusion data, and adjust the global feature fusion data to obtain a two-dimensional global feature map;

Inputting the two-dimensional global feature map into a feature pyramid module to perform multi-scale feature extraction and fusion on the two-dimensional global feature map to obtain a feature map that fuses features of different scales;

The local feature map extracted by the CNN network with residual blocks in the last layer is concatenated with the feature map that fuses features of different scales, and then aggregated by two-dimensional convolution. Then, the bilinear interpolation upsampling method is used to expand the aggregated feature map size space to the same size as the sample image to obtain the predicted image segmentation mask.

With the goal of minimizing the multi-classification loss function, a neural network model based on a CNN-Transformer and a feature pyramid module is trained to obtain the trained crop recognition model.

Preferably, the multi-classification loss function is an asymmetric loss function.

According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

The present invention provides a crop recognition method based on multispectral satellite images, comprising: obtaining an original sample set; the original sample set includes hyperspectral remote sensing images and multispectral remote sensing images corresponding to each sample crop area; preprocessing the original sample set using an adaptive filtering method to obtain a processed sample set; using the hyperspectral remote sensing image in the processed sample set as a label image, and performing spatial and spectral downsampling on the hyperspectral remote sensing image and the multispectral remote sensing image in the processed sample set to obtain a training data set for network parameter adjustment; inputting the training data set into an initial network model for training to obtain a trained crop recognition model; inputting the data to be tested into the crop recognition model to obtain a recognition result. The present invention utilizes a combination of hyperspectral remote sensing images and multispectral remote sensing images. The hyperspectral image provides rich spectral information and can finely distinguish the spectral characteristics of different substances, while the multispectral image provides a higher spatial resolution. Combining these two data types, the detailed spectral information of the hyperspectral image and the spatial details of the multispectral image can be fully utilized, thereby improving the accuracy and reliability of crop recognition.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

FIG1 is a flow chart of a method provided by an embodiment of the present invention.

DETAILED DESCRIPTION

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

The purpose of the present invention is to provide a crop identification method based on multispectral satellite images to improve the accuracy and reliability of crop identification.

In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

FIG1 is a flow chart of a method provided by an embodiment of the present invention. As shown in FIG1 , the present invention provides a crop identification method based on multispectral satellite images, including:

Step 100: Acquire an original sample set; the original sample set includes a hyperspectral remote sensing image and a multispectral remote sensing image corresponding to each sample crop area;

Step 200: preprocessing the original sample set using an adaptive filtering method to obtain a processed sample set;

Step 300: using the hyperspectral remote sensing image in the processing sample set as a label image, downsampling the hyperspectral remote sensing image and the multispectral remote sensing image in the processing sample set in terms of space and spectrum respectively, to obtain a training data set for network parameter adjustment;

Step 400: inputting the training data set into the initial network model for training to obtain a trained crop recognition model;

Step 500: input the data to be tested into the crop recognition model to obtain the recognition result.

Preferably, the original sample set is preprocessed using an adaptive filtering method to obtain a processed sample set, including:

A square window is taken with each pixel point of the hyperspectral remote sensing image and the multispectral remote sensing image in the original sample set as the center, and the change amount of each pixel point under the square window is detected; the calculation formula for detecting the change amount of each pixel point under the square window is: Wherein, _fx ( _xi , _yj ) represents the gradient value of the hyperspectral remote sensing image and the multispectral remote sensing image in the original sample set in the x direction, _fy ( _xi , _yj ) represents the gradient value of the hyperspectral remote sensing image and the multispectral remote sensing image in the original sample set in the y direction, f( _xi , _yj ) represents the grayscale value of the hyperspectral remote sensing image and the multispectral remote sensing image in the original sample set at the position ( _xi , _yj ), and r represents the change amount of the pixel point f( _xi , _yj );

When the change amount of the pixel point is greater than the preset threshold, the median value of the pixel point in the square window is used to smooth the hyperspectral remote sensing image and the multispectral remote sensing image in the corresponding square window to obtain the smoothed pixel point;

The square window is continuously moved until the smoothing process of the hyperspectral remote sensing image and the multispectral remote sensing image in the entire original sample set is completed to obtain a smoothed image; the output formula of the smoothed pixel point is: I _denoised (x, y) = median (I _original (xh, yh), ..., I _original (x + h, y + h)); wherein, I _denoised (x, y) is the smoothed pixel point, h is the side length of the square window, and I _original (x, y) is the value of the pixel point on the hyperspectral remote sensing image and the multispectral remote sensing image in the original sample set at the position (x, y);

constructing a pixel enhancement function based on the convolution value of the smoothed image;

The pixel enhancement function is used to perform image enhancement processing on the images in the original sample set to obtain the processed sample set.

Specifically, in the actual remote sensing application of this embodiment, since noise is inevitable in the image acquisition process, and the noise is generally very different from the surrounding pixels, this embodiment can find the noise on the image by detecting the change of each pixel under the square window, and at the same time use the median of the pixel sequence under the square window as the smoothed pixel point, which can better maintain the detail information of the spectral remote sensing image while removing the noise.

Preferably, constructing a pixel enhancement function based on the convolution value of the smoothed image comprises:

Performing convolution processing on the smoothed image to obtain a convolved image;

Determine the enhancement coefficient based on the pixel value of the convolved image;

A pixel enhancement function is constructed based on the enhancement coefficient; wherein the pixel enhancement function is:

Wherein, S(x,y) represents the image in the processing sample set, E(x,y) represents the enhancement coefficient, G(x,y) represents the pixel value of the convolved image at (x,y), I(x,y) represents the smoothed image, and σ represents the mean square error between the smoothed image and the convolved image.

Specifically, the present invention can quantify the degree of difference between the original image and the convolved image by calculating the mean square error between the two. When the mean square error is large, it means that the original image and the image after convolution are very different and need to be enhanced; when the mean square error is small, it means that the original image and the image after convolution are small and no enhancement can be considered. In image enhancement, whether the image needs to be enhanced can be determined based on the size of the mean square error, so that the image can achieve a better visual effect.

Preferably, the hyperspectral remote sensing image in the processing sample set is used as a label image, and the hyperspectral remote sensing image and the multispectral remote sensing image in the processing sample set are downsampled in space and spectrum respectively to obtain a training data set for network parameter adjustment, including:

According to a preset Wald protocol, Gaussian filtering is performed on the hyperspectral remote sensing image and the multispectral remote sensing image in the processing sample set to obtain a filtered image;

The filtered image is then downsampled by a corresponding multiple using a bilinear interpolation method so that the processed image is used as a hyperspectral remote sensing image and a multispectral remote sensing image that simulate a low-resolution input, and the original hyperspectral image is retained as a reference image.

Specifically, this embodiment takes into account the actual situation that since the fusion image does not exist, the sample construction method specified in the Wald protocol is adopted. Specifically, the hyperspectral image is used as the label image, and then the hyperspectral image and the multispectral image are downsampled in space and spectrum respectively to obtain the data set required for network parameter adjustment.

Preferably, the method for constructing the crop identification model includes:

Performing image processing on the acquired training data set to obtain a sample image;

The sample image is input into a CNN network with a multi-layer residual block for feature extraction, and a maximum pooling layer is used for dimensionality reduction to obtain a local feature map after dimensionality reduction;

The local feature map after dimensionality reduction is input into the feature map embedding module to obtain a one-dimensional vector with position information;

Input the one-dimensional vector into the Transformer block and repeat 24 times to obtain global feature fusion data, and adjust the global feature fusion data to obtain a two-dimensional global feature map;

Inputting the two-dimensional global feature map into a feature pyramid module to perform multi-scale feature extraction and fusion on the two-dimensional global feature map to obtain a feature map that fuses features of different scales;

The local feature map extracted by the CNN network with residual blocks in the last layer is concatenated with the feature map that fuses features of different scales, and then aggregated by two-dimensional convolution. Then, the bilinear interpolation upsampling method is used to expand the aggregated feature map size space to the same size as the sample image to obtain the predicted image segmentation mask.

With the goal of minimizing the multi-classification loss function, a neural network model based on a CNN-Transformer and a feature pyramid module is trained to obtain the trained crop recognition model.

Optionally, this embodiment repeats 24 layers of Transformer layers, and each layer performs self-attention calculation and multi-layer perceptron (MLP) processing. Deep feature fusion enables the model to capture more complex data relationships and patterns, and improves the data expression ability. In addition, this embodiment uses inter-layer residual connections in each Transformer layer (for example, the first layer of global feature data is added and fused with the data input by the first LayerNorm layer). This embodiment can avoid the common gradient vanishing problem in deep networks, while allowing the model to maintain information flow in the deep structure.

Furthermore, the design of the feature pyramid pooling module of this embodiment includes four MCBR layers (MaxPool, convolution, batch normalization, ReLU), an upsampling layer, a 1×1 convolution layer, and a jump connection. The special structure of this embodiment helps to capture features at different scales and enhance the adaptability of the model to scale changes. In addition, the 1×1 convolution of this embodiment is used to adjust the number of channels, the upsampling layer is used to restore the spatial resolution, and the jump connection helps to retain more original feature information.

Preferably, the multi-classification loss function is an asymmetric loss function.

Specifically, this embodiment considers that when the total number of labels is large, there will be an imbalance between positive and negative samples, resulting in inaccurate classification. Most of the images in the data set only contain a small part of the label class, which means that no matter which category, the number of positive samples is far less than the number of negative samples. Multi-label classification often uses a binary cross entropy function as a loss function, but this cannot solve the problem of imbalance between positive and negative samples. Therefore, this embodiment adopts a simplified asymmetric loss function. Through the asymmetric loss function, the imbalance between positive and negative samples in the data set can be balanced as much as possible, so that the model can better learn the characteristics of positive samples in the data.

Furthermore, the recognition result of this embodiment includes:

Crop type recognition: This is the most direct recognition result, that is, identifying the types of various crops. For example, the model can distinguish different crops such as wheat, corn, rice, cotton, etc.

Crop health: By analyzing multispectral and hyperspectral images, crop recognition models can not only identify crop species, but also assess the health of crops. For example, the model can detect areas of crops affected by diseases, nutrient deficiencies or excesses, etc.

Crop coverage area: The model of this embodiment can estimate the coverage area of various crops in a specific area, which is very important for crop yield prediction and land use efficiency analysis.

Crop growth stage: This embodiment can also identify different growth stages of crops, such as sowing period, growth period, maturity period, etc., by combining multispectral and hyperspectral image analysis of time series. This helps agricultural producers make reasonable irrigation, fertilization and harvesting decisions.

Soil conditions: Although not a direct crop identification result in this embodiment, through the analysis of multispectral and hyperspectral images related to crop growth, the model can indirectly provide information about soil conditions, such as soil moisture, pH value, etc., which are important factors affecting crop growth.

The beneficial effects of the present invention are as follows:

(1) The present invention utilizes a combination of hyperspectral remote sensing images and multispectral remote sensing images. Hyperspectral images provide rich spectral information that can finely distinguish the spectral characteristics of different substances, while multispectral images provide higher spatial resolution. By combining these two data types, the present invention can fully utilize the detailed spectral information of hyperspectral images and the spatial details of multispectral images, thereby improving the accuracy and reliability of crop identification.

(2) When processing the original sample set, the present invention uses an adaptive filtering method for preprocessing. The adaptive filtering of the present invention can dynamically adjust the filtering parameters according to the image content, effectively reducing noise while retaining important edge and detail information. The preprocessing step of the present invention provides clearer and more accurate input data for subsequent image analysis and feature extraction.

(3) In the process of preparing the training data set, the present invention uses hyperspectral remote sensing images as label images. That is, the rich spectral information of the hyperspectral images of the present invention is used to directly guide the learning process of the multispectral images, ensuring the high accuracy and relevance of the learning objectives. The present invention can significantly improve the performance of multispectral images in crop recognition tasks.

(4) The present invention performs spatial and spectral downsampling on the images in the processing sample set, aiming to reduce the size of the training data set, thereby reducing the computing resources and time required for model training. The downsampling strategy of the present invention is carefully designed to ensure that while reducing the amount of data, sufficient information is still retained for effective learning.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

This article uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims (6)

1. A crop identification method based on multispectral satellite images, comprising: Acquire an original sample set; the original sample set includes hyperspectral remote sensing images and multispectral remote sensing images corresponding to each sample crop area; Preprocessing the original sample set by using an adaptive filtering method to obtain a processed sample set; The hyperspectral remote sensing image in the processing sample set is used as a label image, and the hyperspectral remote sensing image and the multispectral remote sensing image in the processing sample set are downsampled in terms of space and spectrum respectively to obtain a training data set for network parameter adjustment; Inputting the training data set into the initial network model for training to obtain a trained crop recognition model; The data to be tested is input into the crop recognition model to obtain a recognition result. 2. The crop identification method based on multispectral satellite images according to claim 1 is characterized in that the original sample set is preprocessed by an adaptive filtering method to obtain a processed sample set, comprising: A square window is taken with each pixel point of the hyperspectral remote sensing image and the multispectral remote sensing image in the original sample set as the center, and the change amount of each pixel point under the square window is detected; the calculation formula for detecting the change amount of each pixel point under the square window is: Wherein, _fx ( _xi , _yj ) represents the gradient value of the hyperspectral remote sensing image and the multispectral remote sensing image in the original sample set in the x direction, _fy ( _xi , _yj ) represents the gradient value of the hyperspectral remote sensing image and the multispectral remote sensing image in the original sample set in the y direction, f( _xi , _yj ) represents the grayscale value of the hyperspectral remote sensing image and the multispectral remote sensing image in the original sample set at the position ( _xi , _yj ), and r represents the change amount of the pixel point f( _xi , _yj ); When the change amount of the pixel point is greater than the preset threshold, the median value of the pixel point in the square window is used to smooth the hyperspectral remote sensing image and the multispectral remote sensing image in the corresponding square window to obtain the smoothed pixel point; The square window is continuously moved until the smoothing process of the hyperspectral remote sensing image and the multispectral remote sensing image in the entire original sample set is completed to obtain a smoothed image; the output formula of the smoothed pixel point is: I _denoised (x, y) = median (I _original (xh, yh), ..., I _original (x + h, y + h)); wherein, I _denoised (x, y) is the smoothed pixel point, h is the side length of the square window, and I _original (x, y) is the value of the pixel point on the hyperspectral remote sensing image and the multispectral remote sensing image in the original sample set at the position (x, y); constructing a pixel enhancement function based on the convolution value of the smoothed image; The pixel enhancement function is used to perform image enhancement processing on the images in the original sample set to obtain the processed sample set. 3. The crop identification method based on multispectral satellite images according to claim 2, characterized in that the pixel enhancement function is constructed based on the convolution value of the smoothed image, comprising: Performing convolution processing on the smoothed image to obtain a convolved image; Determine the enhancement coefficient based on the pixel value of the convolved image; A pixel enhancement function is constructed based on the enhancement coefficient; wherein the pixel enhancement function is: Wherein, S(x,y) represents the image in the processing sample set, E(x,y) represents the enhancement coefficient, G(x,y) represents the pixel value of the convolved image at (x,y), I(x,y) represents the smoothed image, and σ represents the mean square error between the smoothed image and the convolved image. 4. The crop recognition method based on multispectral satellite images according to claim 2 is characterized in that the hyperspectral remote sensing image in the processing sample set is used as a label image, and the hyperspectral remote sensing image and the multispectral remote sensing image in the processing sample set are spatially and spectrally downsampled to obtain a training data set for network parameter adjustment, including: According to a preset Wald protocol, Gaussian filtering is performed on the hyperspectral remote sensing image and the multispectral remote sensing image in the processing sample set to obtain a filtered image; The filtered image is then downsampled by a corresponding multiple using a bilinear interpolation method so that the processed image is used as a hyperspectral remote sensing image and a multispectral remote sensing image that simulate a low-resolution input, and the original hyperspectral image is retained as a reference image. 5. The crop identification method based on multispectral satellite images according to claim 2, characterized in that the method for constructing the crop identification model comprises: Performing image processing on the acquired training data set to obtain a sample image; The sample image is input into a CNN network with a multi-layer residual block for feature extraction, and a maximum pooling layer is used for dimensionality reduction to obtain a local feature map after dimensionality reduction; The local feature map after dimensionality reduction is input into the feature map embedding module to obtain a one-dimensional vector with position information; Input the one-dimensional vector into the Transformer block and repeat 24 times to obtain global feature fusion data, and adjust the global feature fusion data to obtain a two-dimensional global feature map; Inputting the two-dimensional global feature map into a feature pyramid module to perform multi-scale feature extraction and fusion on the two-dimensional global feature map to obtain a feature map that fuses features of different scales; The local feature map extracted by the CNN network with residual blocks in the last layer is concatenated with the feature map that fuses features of different scales, and then aggregated by two-dimensional convolution. Then, the bilinear interpolation upsampling method is used to expand the aggregated feature map size space to the same size as the sample image to obtain the predicted image segmentation mask. With the goal of minimizing the multi-classification loss function, a neural network model based on a CNN-Transformer and a feature pyramid module is trained to obtain the trained crop recognition model. 6. The crop identification method based on multispectral satellite images according to claim 5 is characterized in that the multi-classification loss function is an asymmetric loss function.