CN105654479A - Multispectral image registering method and multispectral image registering device - Google Patents

Abstract

The present invention provides a multi-spectral image registration method and device, wherein the multi-spectral image registration method includes: calculating the EOH descriptor corresponding to each feature point according to the acquired reference image and the feature points of the image to be registered; Calculate the Euclidean distance between each feature point of the reference image and each feature point of the image to be registered according to the EOH descriptor; match each feature point in the reference image with all feature points in the image to be registered according to the Euclidean distance And the matching of each feature point in the image to be registered with all feature points in the reference image to obtain a matching pair of feature points between the reference image and the image to be registered; according to the matching pair of feature points, the reference image and the image to be registered are matched Image registration. The multispectral image registration method provided by the present invention improves the accuracy of image registration through bidirectional matching of feature points in the reference image and the image to be registered.

Description

Multispectral image registration method and device

technical field

The invention relates to the field of image processing, in particular to a multi-spectral image registration method and device.

Background technique

Image registration is the process of matching and superimposing two or more images acquired at different times, different sensors (imaging devices) or under different conditions (weather, illumination, camera position and angle, etc.). Among them, the registration of multispectral images has been more and more widely used. The so-called registration of multispectral images refers to the registration of multiple multispectral images generated by sensors (imaging devices) in different frequency bands, such as : Register the visible light image with the infrared light image.

At present, the registration algorithm based on feature points is the most common method in image registration. The algorithm flow is as follows: first, feature extraction is performed on multiple images to obtain feature points; feature point pairs; finally, image registration is performed through the matched feature point pairs. Among them, for the matching of feature points, one-way matching is usually used. Specifically, two images are defined as the reference image and the image to be registered, and the Euclidean distance between all the feature points in the two images is calculated. For each feature point T in the quasi-image, the feature point with the smallest Euclidean distance to the feature point T in the reference image is S ₁ , the Euclidean distance is D ₁ , and the feature point with the second smallest Euclidean distance to the feature point T in the reference image The point is S ₂ , the Euclidean distance is D ₂ , if D ₁ <α·D ₂ , 0<α<1, then the feature point S ₁ is the feature point that matches the feature point T, forming a feature point pair (S ₁ , T).

However, in multi-mode images, due to different imaging devices, the texture structure and pixel differences between multi-mode images are very large, and the detected feature points are not stable. Through the one-way matching of feature points, the reference image The feature point S1 is the optimal matching point _of the feature point T in the image to be registered, but conversely, the feature point T in the image to be registered is not necessarily the optimal matching point _of the feature point S1 in the reference image, so The obtained matching pair of feature points (S ₁ , T) will be inaccurate, and then image registration is performed based on the matching pair of feature points with lower accuracy, which reduces the accuracy of image registration.

Contents of the invention

The present invention provides a multi-spectral image registration method and device, through bidirectional matching of feature points in a reference image and an image to be registered, the accuracy of matching pairs of feature points is improved, thereby improving the accuracy of image registration.

The multispectral image registration method provided by the present invention includes:

Determine the feature points of the reference image and the image to be registered respectively;

Calculate the edge orientation histogram EOH descriptor corresponding to each feature point in the reference image and the image to be registered according to the feature points of the reference image and the image to be registered respectively;

Calculate the Euclidean distance between each feature point in the reference image and each feature point in the image to be registered according to the EOH descriptor;

Matching each feature point in the reference image with all feature points in the image to be registered and matching each feature point in the image to be registered with all feature points in the reference image according to the Euclidean distance Matching, obtaining matching pairs of feature points between the reference image and the image to be registered;

performing image registration on the reference image and the image to be registered according to the feature point matching pair.

The multispectral image registration device provided by the present invention includes:

A feature point determination module is used to determine the feature points of the reference image and the image to be registered respectively;

A descriptor generating module, configured to calculate an edge direction histogram EOH descriptor corresponding to each feature point in the reference image and the image to be registered according to the feature points of the reference image and the image to be registered, respectively;

A calculation module, configured to calculate the Euclidean distance between each feature point in the reference image and each feature point in the image to be registered according to the EOH descriptor;

A first matching module, configured to match each feature point in the reference image with all feature points in the image to be registered according to the Euclidean distance and match each feature point in the image to be registered with the Matching of all feature points in the reference image to obtain matching pairs of feature points between the reference image and the image to be registered;

The second matching module is configured to perform image registration on the reference image and the image to be registered according to the feature point matching pair.

The present invention provides a multi-spectral image registration method and device, wherein the multi-spectral image registration method includes: respectively determining the feature points of the reference image and the image to be registered, respectively according to the feature points of the reference image and the image to be registered Calculate the EOH descriptor corresponding to each feature point in the reference image and the image to be registered, and calculate the Euclidean distance between each feature point in the reference image and each feature point in the image to be registered according to the EOH descriptor, according to the Euclidean distance Match each feature point in the reference image with all feature points in the image to be registered and match each feature point in the image to be registered with all feature points in the reference image to obtain the distance between the reference image and the image to be registered The feature point matching pair performs image registration on the reference image and the image to be registered according to the feature point matching pair. The multi-spectral image registration method provided by the present invention obtains the matching pair of feature points through bidirectional matching between the feature points in the reference image and the image to be registered, which ensures that both the reference image and the image to be registered are qualified The optimal matching relationship improves the accuracy of feature point matching pairs, thereby improving the accuracy of image registration.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained according to these drawings without any creative effort.

FIG. 1 is a flowchart of a multispectral image registration method provided by Embodiment 1 of the present invention;

FIG. 2 is a flowchart of a multispectral image registration method provided by Embodiment 2 of the present invention;

FIG. 3 is a schematic structural diagram of a multispectral image registration device provided by Embodiment 1 of the present invention.

detailed description

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

FIG. 1 is a flowchart of a multispectral image registration method provided by Embodiment 1 of the present invention. As shown in Figure 1, the multispectral image registration method provided by the present invention may include:

Step 101, respectively determine the feature points of the reference image and the image to be registered.

The so-called feature points refer to pixel points or local areas with certain pattern characteristics that are not easy to change in the image, and are also called interest points, salient points or key points. Feature points are the most abstract description of image content, and have good adaptability to image noise, grayscale changes, image deformation, and occlusion.

There are many ways to obtain the feature points of the image, which are not limited in this embodiment.

Optionally, an implementation of this step may be:

Scale-invariant feature transform (SIFT for short) algorithm is used to obtain the feature points of the reference image and the image to be registered respectively.

The feature points obtained by the SIFT algorithm are feature points based on local appearance, and have a high tolerance to scaling, rotation, brightness changes, noise, and viewing angle changes. The feature points are highly significant and easy to obtain, which is conducive to the stability of image registration. sex.

Optionally, another implementation of this step may be:

Determine the feature line segments of the reference image and the image to be registered respectively.

Extend the feature line segments of the reference image and the image to be registered respectively, take the intersection point of the extension line of the feature line segment in the reference image as the feature point of the reference image, and use the intersection point of the extension line of the feature line segment in the image to be registered as the image to be registered feature points.

In a multimodal image, the pixel and texture structure differences between images are large, and the feature line segments will be more stable than individual feature points. Therefore, the feature points obtained through the feature line segments will be more accurate, which is conducive to image registration. accuracy.

Step 103: Calculate the EOH descriptor corresponding to each feature point in the reference image and the image to be registered according to the feature points of the reference image and the image to be registered respectively.

The so-called descriptor refers to a feature description established for a feature point, also called a descriptor. An ideal descriptor should be invariant to transformations such as scale, rotation, and even affine, and the correlation between descriptors corresponding to different feature points should be small, so that different feature points can be effectively distinguished.

In this step, the edge orientation histogram (EdgeOrientationHistogram, referred to as EOH) descriptor is obtained through the calculation of the feature points of the image. More stable, the EOH descriptor uses a more stable edge region in the image, which is conducive to the accuracy of multispectral image registration.

Step 105. Calculate the Euclidean distance between each feature point in the reference image and each feature point in the image to be registered according to the EOH descriptor.

In this step, it is necessary to traverse all the feature points in the reference image and the image to be registered, and calculate the Euclidean distance between a feature point in the reference image and a feature point in the image to be registered.

For example, if there are 3 feature points in the reference image and 2 feature points in the image to be registered, then through the calculation in this step, a total of 3×2=6 Euclidean distances between feature points can be obtained.

Step 107: Match each feature point in the reference image with all feature points in the image to be registered and match each feature point in the image to be registered with all feature points in the reference image according to the Euclidean distance, and obtain the reference image and all feature points in the image to be registered. Match pairs of feature points between registered images.

In this step, the two-way matching of the feature points in the reference image and the image to be registered is carried out through the Euclidean distance between the feature points, that is, each feature point in the reference image is matched with the feature point in the image to be registered. At the same time, each feature point in the image to be registered is also matched with the feature point in the reference image.

In the prior art, the matching pair of feature points is obtained through one-way matching of feature points. Through the one-way matching of feature points, the feature point S ₁ in the reference image is the optimal matching point of feature point T in the image to be registered , but conversely, the feature point T in the image to be registered is not necessarily the optimal matching point for the feature point S ₁ in the reference image. In this step, through the two-way matching between the feature points, taking into account the mutual matching between the feature points in the reference image and the feature points in the image to be registered, it can be ensured that the matching pairs of feature points obtained after matching are suitable for the reference image and the image to be registered. Both sides of the registration image are the optimal matching relationship that meets the conditions, which improves the accuracy of feature point matching pairs, and further improves the accuracy of image registration.

Step 109 , performing image registration on the reference image and the image to be registered according to the feature point matching pair.

Since an accurate feature point matching pair is obtained through the bidirectional matching of feature points, matching the reference image and the image to be registered according to the accurate feature point matching pair can improve the registration between the reference image and the image to be registered. quasi-accurate.

This embodiment provides a multi-spectral image registration method, including: respectively determining the feature points of the reference image and the image to be registered, and calculating the The EOH descriptor corresponding to each feature point, calculate the Euclidean distance between each feature point in the reference image and each feature point in the image to be registered according to the EOH descriptor, and perform the Euclidean distance between each feature point in the reference image and each feature point in the image to be registered according to the Euclidean distance The matching of all feature points in the image to be registered and the matching of each feature point in the image to be registered with all feature points in the reference image, obtaining the matching pair of feature points between the reference image and the image to be registered, according to the feature point matching Image registration is performed on the reference image and the image to be registered. The multi-spectral image registration method provided in this embodiment obtains the matching pair of feature points through the two-way matching between the feature points in the reference image and the image to be registered, which ensures that both the reference image and the image to be registered meet the conditions The optimal matching relationship improves the accuracy of feature point matching pairs, thereby improving the accuracy of image registration.

FIG. 2 is a flow chart of the multispectral image registration method provided by Embodiment 2 of the present invention. On the basis of Embodiment 1, this embodiment provides a specific implementation manner of step 107 in Embodiment 1. As shown in Figure 2, step 107 matches each feature point in the reference image with all feature points in the image to be registered and matches each feature point in the image to be registered with all feature points in the reference image according to the Euclidean distance, Obtaining matching pairs of feature points between the reference image and the image to be registered may include:

Step 201. For each feature point S _i in the reference image, sort the Euclidean distance d(S _i , T _j ) between all feature points T _j and feature point S _i in the image to be registered from small to large, The feature point pairs (S _i , T _j ) corresponding to the first k Euclidean distances are used as the first set of feature point pairs.

Among them, S _i is the feature point in the reference image, T _j is the feature point in the image to be registered, 1≤k≤min(m,n), m is the number of feature points in the reference image, and n is the feature point to be registered The number of feature points in the quasi-image.

For each feature point S _i in the reference image, 1≤i≤m, its Euclidean distance to all feature points in the image to be registered is D _i ={d(S _i ,T ₁ ),d(S _i , T ₂ ),...,d(S _i ,T _n )}, sort D _i from small to large, and select the set C _i ={(S _i ,T _{j_1} ) consisting of feature point pairs corresponding to the first k Euclidean distances ,(S _i ,T _{j_2} ),…,(S _i ,T _{j_k} )} is the feature point set that may match the feature point S _i , where, 1≤j_k≤k, all feature points S _i in the reference image correspond to The set of feature points _Ci constitutes the first set of feature point pairs.

Through this step, all matching pairs of feature points that match the feature points S _i in the reference image are obtained, and the one-way feature point matching on one side of the reference image is actually completed.

Step 203. For each feature point T _j in the image to be registered, sort the Euclidean distance d(S _i , T _j ) between all feature points S _i and feature point T _j in the reference image in ascending order , taking the feature point pairs (S _i , T _j ) corresponding to the first k Euclidean distances as the second set of feature point pairs.

For each feature point T _j in the image to be registered, 1≤j≤n, its Euclidean distance to all feature points in the image to be registered is D _j ={d(S ₁ ,T _j ),d(S ₂ ,T _j ),...,d(S _m ,T _j )}, sort D _j from small to large, and select the set C _j ={(S _{i_1} ,T _j ),(S _{i_2} ,T _j ),…,(S _{i_k} ,T _j )} are feature point sets that may match feature point T _j , where, 1≤i_k≤k, all features in the image to be registered The feature point set C _j corresponding to the point T _j forms the second feature point pair set. ,

Through this step, all matching pairs of feature points that match the feature points T _j in the image to be registered are obtained, and the one-way feature point matching on one side of the image to be registered is actually completed.

It should be noted that step 203 may also be before step 201, and the order of step 201 and step 203 is not limited in this embodiment.

Step 205, taking the intersection of the first set of feature point pairs and the second set of feature point pairs as a matching pair of feature points between the reference image and the image to be registered.

Through step 201 and step 203, the first feature point pair set and the second feature point pair set of one-way matching are obtained, and the intersection of the first feature point pair set and the second feature point pair set is taken into account, taking into account the reference image The mutual matching between the feature points in the image to be registered and the feature points in the image to be registered can ensure that the matching pair of feature points obtained after matching is the optimal matching relationship that meets the conditions for both the reference image and the image to be registered, improving the feature The accuracy of point matching pairs is improved, which in turn improves the accuracy of image registration.

Preferably, when k=1, one-to-one matching pairs of feature points can be obtained.

This embodiment provides a multispectral image registration method, including: for each feature point S _i in the reference image, the Euclidean distance d between all feature points T _j and feature point S _i in the image to be registered (S _i , T _j ) are sorted from small to large, and the feature point pairs (S _i , T _j ) corresponding to the first k Euclidean distances are used as the first feature point pair set. For each feature point in the image to be registered T _j , sort the Euclidean distance d(S _i , T _j ) between all feature points S _i and feature point T _j in the reference image from small to large, and sort the feature point pairs (S _i , T _j ) as the second feature point pair set, and the intersection of the first feature point pair set and the second feature point pair set is used as the feature point matching pair between the reference image and the image to be registered. The multi-spectral image registration method provided in this embodiment obtains the matching pair of feature points through the two-way matching between the feature points in the reference image and the image to be registered, which ensures that both the reference image and the image to be registered meet the conditions The optimal matching relationship improves the accuracy of feature point matching pairs, thereby improving the accuracy of image registration.

Fig. 3 is a schematic structural diagram of a multispectral image registration device provided in Embodiment 1 of the present invention. The multispectral image registration device provided in this embodiment is used to perform the multispectral image registration provided in the embodiments shown in Fig. 1 to Fig. 2 standard method. The multispectral image registration device provided in this embodiment may include:

The feature point determination module 11 is used to determine the feature points of the reference image and the image to be registered respectively.

The descriptor generation module 13 is configured to calculate the EOH descriptor corresponding to each feature point in the reference image and the image to be registered according to the feature points of the reference image and the image to be registered.

The calculation module 15 is used to calculate the Euclidean distance between each feature point in the reference image and each feature point in the image to be registered according to the EOH descriptor.

The first matching module 17 is used for matching each feature point in the reference image with all feature points in the image to be registered and matching each feature point in the image to be registered with all feature points in the reference image according to the Euclidean distance, Obtain matching pairs of feature points between the reference image and the image to be registered.

The second matching module 19 is configured to perform image registration on the reference image and the image to be registered according to the matching pairs of feature points.

This embodiment provides a multi-spectral image registration device, including: a feature point determination module, used to determine the feature points of the reference image and the image to be registered respectively, and a descriptor generation module, used to The feature points of the quasi-image calculate the EOH descriptor corresponding to each feature point in the reference image and the image to be registered, and the calculation module is used to calculate each feature point in the reference image and each feature in the image to be registered according to the EOH descriptor The Euclidean distance between points, the first matching module is used to match each feature point in the reference image with all the feature points in the image to be registered according to the Euclidean distance and each feature point in the image to be registered with the reference image The matching of all feature points is to obtain matching pairs of feature points between the reference image and the image to be registered, and the second matching module is used to perform image registration on the reference image and the image to be registered according to the matching pairs of feature points. The multi-spectral image registration device provided in this embodiment obtains matching pairs of feature points through bidirectional matching between the feature points in the reference image and the image to be registered, which ensures that both the reference image and the image to be registered meet the conditions The optimal matching relationship improves the accuracy of feature point matching pairs, thereby improving the accuracy of image registration.

Optionally, the feature point determination module 11 is specifically used for:

The SIFT algorithm is used to obtain the feature points of the reference image and the image to be registered respectively.

Optionally, the feature point determination module 11 is specifically used for:

Determine the feature line segments of the reference image and the image to be registered respectively;

The feature line segments of the reference image and the image to be registered are respectively extended, and the intersection points of the extension lines of the feature line segments of the reference image and the image to be registered are used as feature points of the reference image and the image to be registered.

Optionally, the first matching module 17 is specifically used for:

For each feature point S _i in the reference image, sort the Euclidean distance d(S _i , T _j ) between all feature points T _j and feature point S _i in the image to be registered from small to large, and sort the previous The feature point pairs (S _i , T _j ) corresponding to k Euclidean distances are used as the first feature point pair set.

For each feature point T _j in the image to be registered, the Euclidean distance d(S _i , T _j ) between all feature points S _i and feature point T _j in the reference image is sorted from small to large, and the previous The feature point pairs (S _i , T _j ) corresponding to k Euclidean distances are used as the second feature point pair set.

The intersection of the first feature point pair set and the second feature point pair set is used as a feature point matching pair between the reference image and the image to be registered.

Among them, S _i is the feature point in the reference image, T _j is the feature point in the image to be registered, 1≤k≤min(m,n), m is the number of feature points in the reference image, and n is the feature point to be registered The number of feature points in the quasi-image.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present invention. scope.

Claims (8)

1. A multispectral image registration method, characterized in that, comprising: Determine the feature points of the reference image and the image to be registered respectively; Calculate the edge orientation histogram EOH descriptor corresponding to each feature point in the reference image and the image to be registered according to the feature points of the reference image and the image to be registered respectively; Calculate the Euclidean distance between each feature point in the reference image and each feature point in the image to be registered according to the EOH descriptor; Matching each feature point in the reference image with all feature points in the image to be registered and matching each feature point in the image to be registered with all feature points in the reference image according to the Euclidean distance Matching, obtaining matching pairs of feature points between the reference image and the image to be registered; performing image registration on the reference image and the image to be registered according to the feature point matching pair. 2. The multispectral image registration method according to claim 1, wherein the matching of each feature point in the reference image with all feature points in the image to be registered is carried out according to the Euclidean distance And the matching of each feature point in the image to be registered with all the feature points in the reference image, obtaining a matching pair of feature points between the reference image and the image to be registered, including: For each feature point S _i in the reference image, the Euclidean distance d(S _i , T _j ) between all feature points T _j in the image to be registered and the feature point S _i is calculated from small to Large sorting, using the feature point pairs (S _i , T _j ) corresponding to the first k Euclidean distances as the first feature point pair set; For each feature point T _j in the image to be registered, the Euclidean distance d(S _i , T _j ) between all feature points S _i in the reference image and the feature point T _j is calculated from small to Large sorting, using the feature point pairs (S _i , T _j ) corresponding to the first k Euclidean distances as the second feature point pair set; Using the intersection of the first feature point pair set and the second feature point pair set as a matching pair of feature points between the reference image and the image to be registered; Among them, S _i is the feature point in the reference image, T _j is the feature point in the image to be registered, 1≤k≤min(m,n), m is the number of feature points in the reference image, and n is the feature point to be registered The number of feature points in the quasi-image. 3. The multispectral image registration method according to claim 1 or 2, wherein said determining the feature points of the reference image and the image to be registered respectively comprises: The feature points of the reference image and the image to be registered are obtained respectively by adopting a scale-invariant feature transformation SIFT algorithm. 4. The multispectral image registration method according to claim 1 or 2, wherein said determining the feature points of the reference image and the image to be registered respectively comprises: Determining the feature line segments of the reference image and the image to be registered respectively; Prolonging the feature line segments of the reference image and the image to be registered respectively, using the intersection point of the extension line of the feature line segment in the reference image as a feature point of the reference image, and taking the feature line segment in the image to be registered The intersection of the extension lines of is used as the feature point of the image to be registered. 5. A multi-spectral image registration device, characterized in that, comprising: A feature point determination module is used to determine the feature points of the reference image and the image to be registered respectively; A descriptor generating module, configured to calculate an edge direction histogram EOH descriptor corresponding to each feature point in the reference image and the image to be registered according to the feature points of the reference image and the image to be registered, respectively; A calculation module, configured to calculate the Euclidean distance between each feature point in the reference image and each feature point in the image to be registered according to the EOH descriptor; A first matching module, configured to match each feature point in the reference image with all feature points in the image to be registered according to the Euclidean distance and match each feature point in the image to be registered with the Matching of all feature points in the reference image to obtain matching pairs of feature points between the reference image and the image to be registered; The second matching module is configured to perform image registration on the reference image and the image to be registered according to the feature point matching pair. 6. The multispectral image registration device according to claim 5, wherein the first matching module is specifically used for: For each feature point S _i in the reference image, the Euclidean distance d(S _i , T _j ) between all feature points T _j in the image to be registered and the feature point S _i is calculated from small to Large sorting, using the feature point pairs (S _i , T _j ) corresponding to the first k Euclidean distances as the first feature point pair set; For each feature point T _j in the image to be registered, the Euclidean distance d(S _i , T _j ) between all feature points S _i in the reference image and the feature point T _j is calculated from small to Large sorting, using the feature point pairs (S _i , T _j ) corresponding to the first k Euclidean distances as the second feature point pair set; Using the intersection of the first feature point pair set and the second feature point pair set as a matching pair of feature points between the reference image and the image to be registered; Among them, S _i is the feature point in the reference image, T _j is the feature point in the image to be registered, 1≤k≤min(m,n), m is the number of feature points in the reference image, and n is the feature point to be registered The number of feature points in the quasi-image. 7. The multispectral image registration device according to claim 5 or 6, wherein the feature point determination module is specifically used for: The feature points of the reference image and the image to be registered are obtained respectively by adopting a scale-invariant feature transformation SIFT algorithm. 8. The multispectral image registration device according to claim 5 or 6, wherein the feature point determination module is specifically used for: Determining the feature line segments of the reference image and the image to be registered respectively; Prolonging the feature line segments of the reference image and the image to be registered respectively, using the intersection point of the extension line of the feature line segment in the reference image as a feature point of the reference image, and taking the feature line segment in the image to be registered The intersection of the extension lines of is used as the feature point of the image to be registered.