CN110414558B

CN110414558B - Feature point matching method based on event camera

Info

Publication number: CN110414558B
Application number: CN201910551377.9A
Authority: CN
Inventors: 余磊; 陈欣宇; 杨文�; 杨公宇; 叶琪霖; 王碧杉; 周立凤
Original assignee: Wuhan University WHU
Current assignee: Wuhan University WHU
Priority date: 2019-06-24
Filing date: 2019-06-24
Publication date: 2021-07-20
Anticipated expiration: 2039-06-24
Also published as: CN110414558A

Abstract

The invention discloses a method for extracting descriptors for detected feature points, and uses the generated descriptors to match the feature points. The purpose of the present invention is to solve the problem that the traditional feature point descriptor algorithm may not be stably applicable to the event camera. The present invention uses the timestamp information of the event camera to extract the descriptor for the feature point, which can better utilize the advantages of the event camera and make the description Sub-information is richer, making matching results more accurate.

Description

Feature point matching method based on event camera

Technical Field

The invention belongs to the field of image processing, and is used for generating a feature point descriptor based on an event camera and performing feature point matching.

Background

Machine vision relies primarily on frame-based cameras, which acquire entire frames at a fixed temporal exposure and frame rate, store and process image information in a matrix form. Such a simple image storage format may not be ideal for image processing and feature extraction, in large part because grayscale-based images contain much redundant information. Pixel intensity information is useful for human eye identification and retrieval, but adds to the difficulty of machine-based image processing. At the same time, sequential image readout can adversely affect image processing hardware because a large amount of unnecessary data must be processed before the desired features are obtained. Meanwhile, a common camera is sensitive to illumination change, and is easy to be too dark or overexposed when facing a high-dynamic illumination scene, and motion blur or poor imaging effect generated by a common optical image in a high-speed motion state seriously affects the image quality. The imaging of a common optical camera in a high-speed motion state is shown in fig. 1.

Event cameras have gained more and more attention in the field of machine vision, and are a novel visual sensor simulating human retina, the pixel array surface of the event camera triggers and outputs the pixel point position, time and intensity of light intensity change through the light intensity change, so the output of the event camera is not the video frame sequence of a standard camera but a series of asynchronous event streams.More specifically, at t_jTime of day u_j＝(x_j，y_j) The brightness increment at the pixel position reaches a threshold value + -c (c > 0), then an event e_j＝(x_j，y_j，t_j，p_j) To be triggered, p_jE { +1, -1} indicates that the polarity of the event is positive indicating an increase in brightness and negative indicating a decrease in brightness, so the event camera outputs an asynchronous stream of events whose absolute brightness values are no longer directly visible since the events only record incremental changes. The cameras are not limited by traditional exposure time and frame rate, the time coordinate precision can reach microsecond level, background redundant information can be effectively filtered, the camera motion information can be concentrated and captured, the data transmission bandwidth is saved, the data storage pressure is reduced, and the cameras have the advantages of high dynamic, low time delay and low power consumption, so that the cameras can provide reliable visual information during high-speed motion or in scenes characterized by high dynamic range. The imaging of the event camera in the high motion state is shown in fig. 2.

In some difficult scenes, such as high-speed motion or a state with severe change of illumination conditions, the motion blur or poor imaging effect generated by a common optical image causes great difficulty in feature point detection, feature point description and feature point matching. However, the event camera can acquire a real-time state of a high-speed moving object and possess a high dynamic range.

Since the event camera outputs individually isolated points, the points appearing in the event frame at different moments in time in different motion states may not be stable for the same object. Therefore, the conventional feature point descriptor algorithm may not be stably applicable. Meanwhile, compared with a common optical image, the event point of the event output also contains respective time information. This information is not well exploited by using the conventional descriptor algorithm directly. Therefore, extracting descriptors for event feature points from the perspective of timestamp information would better take advantage of the advantages of an event camera.

Disclosure of Invention

The invention generates the feature point descriptor based on the event camera and matches the feature points according to the descriptor. Under the inspiration of a shape context algorithm, a feature point descriptor generation method aiming at the imaging characteristics of an event image is provided, and feature points are matched according to the generated descriptor. The method for generating the feature point descriptors is slightly different from the traditional method, and because the time camera can obtain the position, the time and the polarity of the event point, the invention utilizes the timestamp information, which can better utilize the advantages of the event camera.

The technical scheme provided by the invention is a feature point descriptor matching method based on an event camera, which comprises the following specific steps:

step 1, using characteristic point p_iN concentric circles are established at logarithmic distance intervals in a local area with the circle center as R1 as the outermost circle radius and R2 as the innermost circle radius, i.e., (log10(R1), log10(R2)) are logarithmically (i.e., logspace) equally divided into N elements, and then the region is equally divided in the circumferential direction M to generate a grid, i.e., bins, as shown in fig. 4.

Step 2, comparing the characteristic points p_iTime stamp t of_piAnd time stamp t of each event point in the local area_qiIf t is_pi＜t_qiIf t is equal to 1, the point is set to_pi＞t_qiThen the point is set to "0".

Step 3, counting the characteristic points p_iThe number of "1" in each bins in the local area, i.e. the statistical distribution histogram h of these points in the bins_i(k) Called feature point p_iThe descriptor of (1), wherein the descriptor size is M x N.

And 4, traversing all the feature points to obtain descriptors corresponding to all the feature points.

Step 5, according to h_i(k) And calculating the similarity between every two feature point sets, namely a cost value, and then counting the corresponding relation of a group of point sets with the lowest overall cost value by using the Hungarian algorithm, thereby obtaining the feature point matching relation. The cost value is calculated by the formula:

wherein k is the kth bit in the descriptor, h_i(k)，h_j(k) Respectively represent characteristic points p_iAnd q is_jThe description of (1).

And 6, removing mismatching by using a vector consistency method to obtain the best matching.

Compared with the prior art, the invention has the advantages and beneficial effects that: the event points output by the event cameras also contain respective timestamp information, which is not utilized by descriptor extraction in existing matching algorithms. The advantages of the event camera can be better utilized by extracting the descriptors from the feature points by utilizing the timestamp information of the event camera, so that the descriptor information is richer, and the matching result is more accurate.

Drawings

Fig. 1 is a general optical camera imaging.

Fig. 2 is an event camera imaging.

Fig. 3 is a flowchart describing a child generation method and feature point matching.

FIG. 4 is a feature point description sub-meshing diagram.

Fig. 5 is a feature point matching result.

Detailed Description

The invention is mainly based on an event camera and processes event stream data. In consideration of the characteristics of the event camera, a method for extracting descriptors of feature points is provided, and feature point matching is performed according to the descriptors. In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following examples. It should be understood that the specific examples described herein are intended to be illustrative only and are not intended to be limiting.

As shown in fig. 3, the feature point descriptor matching method based on the event camera provided in the embodiment of the present invention includes the following specific implementation steps:

step 1, setting N to 8, M to 12, R1 to 12, and R2 to 1. By a characteristic point p_iEstablishing N concentric circles at intervals of logarithmic distance in a local area with the circle center as R1 as the outermost circle radius and R2 as the innermost circle radius, and equally dividing the area along the circumferential direction M to generate a netAnd (4) grid.

Step 2, comparing the characteristic points p_iTime stamp t of_piAnd time stamp t of each event point in the local area_qiIf t is_pi＜t_qiIf t is equal to 1, the point is set to_pi＞t_qiThen the point is set to "0", i.e. intra-trellis coded.

Step 3, counting the characteristic points p_iThe number of "1" in each bins in the local area, i.e. the statistical distribution histogram h of these points in the bins_i(k) Called feature point p_iThe description of (1).

And 5, calculating a cost value between every two feature point sets, wherein the cost value is X2 test statistic (chi-square test statistic, and the deviation degree between the actual observed value and the theoretical inferred value of the statistical sample). And (3) counting the corresponding relation of a group of point sets with the lowest overall cost value by using the Hungarian algorithm, thereby obtaining the characteristic point matching relation.

The result of using the vector consistency method to remove the mismatching is shown in fig. 5, the initial matching is 1642 pairs, the number of the mismatching removed is 1101 pairs, and the ratio is 0.6705, and as can be seen from fig. 5, the matching performed by the method of the present invention has many feature points, the matching connecting lines between the two graphs are basically in the same direction, and the matching result is accurate.

The specific embodiments described herein are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.

Claims

1. The feature point matching method based on event camera, is characterized in that, comprises the following steps:

Step 1: Establish N concentric circles at logarithmic distance intervals in the local area with the feature point p _i as the center, R1 as the outermost circle radius, and R2 as the innermost circle radius, and then divide this area into equal parts along the circumferential direction M, generate grid;

Step 2: Compare the size of the timestamp t _pi at the feature point _pi with the timestamp t _qi of each event point in the local area. If t _pi <t _qi , the point is set to “1”, and if t _pi >t _qi , then the point is set to "0";

Step 3, count the number of "1"s in each bin of the feature point p _i , that is, the statistical distribution histogram h _i (k) of these points in the bins, which is called the descriptor of the feature point p _i ;

Step 4, traverse all feature points to obtain descriptors corresponding to all feature points;

Step 5: Calculate the similarity between each two feature point sets according to h _i (k), that is, the cost value, and then use the Hungarian algorithm to count the corresponding relationship of a set of point sets with the lowest overall cost value, so as to obtain the feature point matching relationship. ;

The formula for calculating the cost value in step 5 is,

Among them, k is the kth position in the descriptor, h _i (k), h _j (k) represent the descriptors of the feature points p _i and q _j respectively;

Step 6, use the vector consistency method to remove the mismatch to obtain the best match.