WO2023236508A1

WO2023236508A1 - Image stitching method and system based on billion-pixel array camera

Info

Publication number: WO2023236508A1
Application number: PCT/CN2022/141925
Authority: WO
Inventors: 袁潮; 邓迪旻; 温建伟
Original assignee: 北京拙河科技有限公司
Priority date: 2022-06-07
Filing date: 2022-12-26
Publication date: 2023-12-14
Also published as: CN114841862B; CN114841862A

Abstract

The present application is specifically applied to the field of image processing. Provided are an image stitching method and system based on a billion-pixel array camera. The method comprises: a billion-pixel array camera photographing a target object, so as to acquire a first image to be subjected to stitching and a second image to be subjected to stitching; respectively performing feature point extraction on said first image and said second image, so as to determine first feature points and second feature points; acquiring a first moving speed of the billion-pixel array camera; acquiring a second moving speed of the target object; calculating a deviation rate on the basis of the first moving speed and the second moving speed; adjusting the first feature points and the second feature points on the basis of the deviation rate, so as to obtain first corrected feature points and second corrected feature points; matching the first corrected feature points and the second corrected feature points, so as to obtain a plurality of optimal feature point pairs; and performing screening on the plurality of optimal feature point pairs, and stitching said first image and said second image, so as to obtain a complete image. In this way, the present invention can improve the splicing efficiency and accuracy of images.

Description

An image stitching method and system based on a gigapixel array camera

Technical field

The present application relates to the field of image processing, and more specifically, to an image splicing method and system based on a gigapixel array camera.

Background technique

The array camera replaces the shooting effect of one large lens with multiple small lenses. Its principle is to control multiple cameras at the same time for shooting. Compared with traditional cameras, the 100-megapixel array camera has a wider field of view and is more convenient for shooting. The resulting photos are also larger, while being smaller in size.

Image registration is the key to image splicing. Image registration aims to find the same area in two images to calculate the coordinate changes between images. The accuracy of image registration directly determines the quality of image splicing.

In the existing technology, image registration is usually achieved by performing grayscale processing, angle transformation, edge processing, etc. on the image itself, ignoring the image deviation caused by the movement of the shooting target and the shooting device, resulting in low image stitching accuracy.

Contents of the invention

The purpose of the embodiments of the present invention is to provide an image splicing method and system based on a gigapixel array camera, which determines the offset rate based on the movement of the gigapixel array camera and the target object, and corrects the feature points through the offset rate to avoid movement. The resulting image deviation improves the efficiency and accuracy of image stitching. The specific technical solutions are as follows:

In a first aspect of the embodiment of the present invention, an image splicing method based on a gigapixel array camera is provided, including: acquiring image data captured by a gigapixel array camera; the image data is a first image of a target object to be spliced and the second image to be spliced; perform feature point extraction on the first image to be spliced and the second image to be spliced respectively to determine the first feature point and the second feature point; obtain the third feature point of the gigapixel array camera a moving speed; obtaining the second moving speed of the target object; calculating a deviation rate based on the first moving speed and the second moving speed; adjusting the first feature point and the first moving speed based on the deviation rate Use the second feature point to obtain the first modified feature point and the second modified feature point; match the first modified feature point and the second modified feature point to obtain multiple optimal feature point pairs; filter the multiple optimal feature point pairs. The optimal feature point pair is used to splice the first image to be spliced and the second image to be spliced to obtain a total image.

Optionally, the first image to be spliced and the second image to be spliced are two consecutive frame images.

Optionally, performing feature point extraction on the first image to be spliced and the second image to be spliced respectively, and determining the first feature point and the second feature point include: based on the first image to be spliced or For each pixel on the second image to be spliced, with the pixel as the center, calculate the grayscale difference of n adjacent pixels; if the grayscale difference of adjacent pixels that meet the preset conditions is greater than n /2, then the pixel point is determined as the first feature point or the second feature point.

Optionally, calculating the offset rate based on the first moving speed and the second moving speed includes: calculating a first offset amount based on the first moving speed and the second moving speed respectively. s ₁ and the second offset s ₂ ; if the gigapixel array camera and the target object move in the same direction, the offset rate R is calculated by the following formula:

If the gigapixel array camera and the target object move in opposite directions, the offset rate R is calculated by the following formula:

Among them, t ₂ represents the shooting time of the second image to be spliced, and t ₁ represents the shooting time of the first image to be spliced, then the shooting time difference between the first image to be spliced and the second image to be spliced is Δt=|t ₂ -t ₁ |, vx ₁ represents the moving speed of the gigapixel array camera at time t ₁ , vd ₁ represents the moving speed of the target object at time t ₁ .

Optionally, calculating the first offset s ₁ and the second offset s ₂ based on the first moving speed and the second moving speed respectively includes: calculating the first offset through the following formula s ₁ and second offset s ₂ :

s ₁ =|vx ₁ -vx ₂ |*Δt

s ₂ =|vd ₁ -vd ₂ |*Δt

Among them, vx ₁ represents the moving speed of the gigapixel array camera at time t ₁ , vx ₂ represents the moving speed of the gigapixel array camera at time t ₂ , vd ₁ represents the moving speed of the target object at time t ₁ , vd ₂ represents The moving speed of the target object at time t ₂ , Δt represents the shooting time difference between the first image to be spliced and the second image to be spliced.

Optionally, adjusting the first feature point based on the offset rate to obtain a first corrected feature point includes: constructing a three-dimensional coordinate system; obtaining the coordinate value of any one of the plurality of first feature points ( x ₁ ,y ₁ ,z ₁ ); perform coordinate transformation on the feature point through the following formula:

in,

Represents the coordinate conversion value,

Represents the internal parameter matrix of the gigapixel array camera, f _x and f _y represent the length of the focal length in the x-axis and y-axis directions, respectively, in millimeters; c _x and c _y respectively represent the optical center, in pixels; Q represents the rotation matrix, T represents the translation matrix, [Q T] represents the external parameter matrix of the gigapixel array camera; R represents the offset rate,

represents the corrected coordinate value of the feature point; traverses all feature points in the first feature point and repeats the above steps; maps the first corrected feature point based on the corrected coordinate value of all feature points in the first feature point.

Optionally, adjusting the second feature point based on the offset rate to obtain a second modified feature point includes: obtaining the coordinate value (x ₂ , y ) of any one of the plurality of second feature points. ₂ ,z ₂ ); perform coordinate transformation on the feature point through the following formula:

in,

Represents the coordinate conversion value,

Represents the internal parameter matrix of the gigapixel array camera, f _x and f _y represent the length of the focal length in the x-axis and y-axis directions, respectively, in millimeters; c _x and c _y respectively represent the optical center of the camera, in pixels; Q represents rotation Matrix, T represents the translation matrix, [Q T] represents the external parameter matrix of the gigapixel array camera; R represents the offset rate,

represents the corrected coordinate value of the feature point; traverses all the feature points in the second feature point and repeats the above steps; and maps the second corrected feature point based on the corrected coordinate value of all the feature points in the second feature point.

Optionally, matching the first modified feature point and the second modified feature point to obtain multiple optimal feature point pairs includes: using Hamming distance to match the first modified feature point and the third modified feature point. 2. Correct each feature point in the feature points to obtain multiple optimal feature point pairs with the shortest Hamming distance.

Optionally, the splicing of the first image to be spliced and the second image to be spliced to obtain a total image includes: using a weighted fusion algorithm to obtain a spliced total image.

In another aspect of the embodiment of the present invention, an image splicing system based on a gigapixel array camera is provided, including: an image data acquisition module for acquiring image data captured by the gigapixel array camera; the image data is a target The first image to be spliced and the second image to be spliced of the object; a feature point determination module, configured to extract feature points of the first image to be spliced and the second image to be spliced respectively, and determine the first feature point and the second image to be spliced. Two feature points; a speed determination module, used to obtain the first moving speed and the second moving speed of the gigapixel array camera and the target object; an offset rate calculation module, used to calculate the first moving speed and the second moving speed based on the first moving speed and the second moving speed of the target object; The second moving speed is used to calculate the offset rate; the feature point correction module is used to adjust the first feature point and the second feature point based on the offset rate to obtain the first corrected feature point and the second corrected feature point. Feature points; an optimal feature point pair acquisition module, used to match the first corrected feature point and the second corrected feature point to obtain multiple optimal feature point pairs; an image splicing module, used to filter the multiple The optimal feature point pair is used to splice the first image to be spliced and the second image to be spliced to obtain a total image.

Optionally, the feature point determination module is further configured to: calculate n neighboring pixels based on each pixel on the first image to be spliced or the second image to be spliced, with the pixel as the center. The grayscale difference; if there are more than n/2 adjacent pixels whose grayscale difference meets the preset conditions, then the pixel is determined to be the first feature point or the second feature point.

Optionally, the offset rate calculation module is further configured to: calculate the first offset s ₁ and the second offset s ₂ based on the first moving speed and the second moving speed respectively; if If the gigapixel array camera and the target object move in the same direction, the offset rate R is calculated by the following formula:

Among them, t ₂ represents the shooting time of the second image to be spliced, and t ₁ represents the shooting time of the first image to be spliced, then the shooting time difference between the first image to be spliced and the second image to be spliced is Δt = |t ₂ - t ₁ | , vx ₁ represents the moving speed of the gigapixel array camera at time t ₁ , vd ₁ represents the moving speed of the target object at time t ₁ .

s ₁ =|vx ₁ -vx ₂ |*Δt

s ₂ =|vd ₁ -vd ₂ |*Δt

Optionally, the feature point correction module is further used to: construct a three-dimensional coordinate system; obtain the coordinate value (x ₁ , y ₁ , z ₁ ) of any one of the plurality of first feature points; use the following formula Perform coordinate transformation on the feature point:

in,

Represents the coordinate conversion value,

represents the corrected coordinate value of the feature point; traverses all feature points in the first feature point and repeats the above steps; and maps the first corrected feature point based on the corrected coordinate value of all feature points in the first feature point.

Optionally, the feature point correction module is further configured to: obtain the coordinate value (x ₂ , y ₂ , z ₂ ) of any one of the plurality of second feature points; and calculate the feature point through the following formula Perform coordinate transformation:

in,

Represents the coordinate conversion value,

Optionally, the optimal feature point pair acquisition module is further configured to: use Hamming distance to match each feature point in the first modified feature point and the second modified feature point, and obtain the shortest Hamming distance Multiple optimal feature point pairs.

Optionally, the image splicing module is further used to: use a weighted fusion algorithm to obtain the spliced total image.

Beneficial effects:

The present invention uses a gigapixel array camera to capture a target object and obtain the first image to be spliced and the second image to be spliced; feature point extraction is performed on the first image to be spliced and the second image to be spliced respectively, and the first feature point and the second image to be spliced are determined. Two feature points; obtain the first moving speed of the gigapixel array camera; obtain the second moving speed of the target object; calculate the offset rate based on the first moving speed and the second moving speed; adjust the first feature point based on the offset rate and the second feature point to obtain the first modified feature point and the second modified feature point; match the first modified feature point and the second modified feature point to obtain multiple optimal feature point pairs; screen multiple optimal feature point pairs, Splice the first image to be spliced and the second image to be spliced to obtain a total image. Among them, the calculation of the offset rate takes into account the errors that the moving direction and speed of the camera and the subject may cause to the image stitching. At the same time, a three-dimensional coordinate system is introduced to combine the camera parameters with the offset rate to correct the feature points of the two images; through The image data collected in this way is of higher quality and can improve the efficiency and accuracy of image stitching.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without exerting creative efforts.

Figure 1 is a schematic flow chart of an image stitching method based on a gigapixel array camera provided by an embodiment of the present application;

Figure 2 is a schematic structural diagram of an image stitching system based on a gigapixel array camera provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments These are part of the embodiments of this application, but not all of them. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations.

Accordingly, the following detailed description of the embodiments of the application provided in the appended drawings is not intended to limit the scope of the claimed application, but rather to represent selected embodiments of the application. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

Embodiments of the present application provide an image splicing method and system based on a gigapixel array camera, which includes photographing a target object with a gigapixel array camera, acquiring a first image to be spliced and a second image to be spliced; and separately processing the first image to be spliced. Extract feature points from the image and the second image to be spliced to determine the first feature point and the second feature point; obtain the first moving speed of the gigapixel array camera; obtain the second moving speed of the target object; based on the first moving speed and Second moving speed, calculate the offset rate; adjust the first feature point and the second feature point based on the offset rate to obtain the first corrected feature point and the second corrected feature point; match the first corrected feature point and the second corrected feature point , obtain multiple optimal feature point pairs; filter multiple optimal feature point pairs, splice the first image to be spliced and the second image to be spliced, and obtain the total image. Through the above method, the present invention can improve the efficiency and accuracy of image stitching.

The image splicing method and system based on a gigapixel array camera can be integrated into electronic equipment, and the electronic equipment can be terminals, servers, and other equipment. Among them, the terminal can be a light field camera, a vehicle camera, a mobile phone, a tablet, a smart Bluetooth device, a laptop, or a personal computer (PC); the server can be a single server or composed of multiple servers. server cluster. The above examples should not be construed as limitations of this application.

Figure 1 shows a schematic flowchart of an image stitching method based on a gigapixel array camera provided by an embodiment of the present application. Please refer to Figure 1, which specifically includes the following steps:

S110. Obtain the image data captured by the gigapixel array camera; the image data is the first image to be spliced and the second image to be spliced of the target object.

Among them, the gigapixel array camera is a cross-scale imaging camera that combines a main lens and an array of N micro lenses. The micro lenses can form different focal lengths according to different optical path designs. When multiple lenses work in parallel, they can capture Different images from near and far.

Therefore, collecting image data through a gigapixel array camera can not only greatly increase the order of magnitude of the collected data and imaging range, but also obtain multiple focus points to achieve both a large field of view and details.

In one implementation, the first image to be spliced and the second image to be spliced may be two consecutive frame images.

In another implementation, the first image to be spliced and the second image to be spliced may be two frames of images acquired within a preset time interval; for example, the starting time is 17 o'clock and the preset time interval is 5 seconds, the image taken at 17 o'clock will be used as the first image to be stitched, and the image taken at 17:00 minutes and 5 seconds will be used as the second image to be stitched.

Optionally, the computer device receives the image data collected by the gigapixel array camera, and the image data can be transmitted through the fifth generation mobile communication technology or through the wifi network. Among them, the image data can be portraits, large animals, small animals, vehicles, plants, etc.

S120: Extract feature points from the first image to be spliced and the second image to be spliced respectively, and determine the first feature point and the second feature point.

Wherein, according to each pixel point on the first image to be spliced or the second image to be spliced, with the pixel point as the center, the grayscale difference of n adjacent pixel points on a circle with a radius of d can be calculated. value; if there are more than n/2 adjacent pixels whose grayscale difference value meets the preset condition, then the pixel is determined to be the first feature point or the second feature point.

S130. Obtain the first moving speed of the gigapixel array camera.

S140. Obtain the second moving speed of the target object.

S150. Calculate a deflection rate based on the first moving speed and the second moving speed.

In one implementation, step S150 may specifically include the following steps:

S151. Calculate the first offset s ₁ and the second offset s ₂ based on the first moving speed and the second moving speed respectively. The formula is as follows:

s ₁ =|vx ₁ -vx ₂ |*Δt

s ₂ =|vd ₁ -vd ₂ |*Δt

S152. Calculate the offset rate according to the moving direction of the gigapixel array camera and the target object.

Specifically, if the gigapixel array camera and the target object move in the same direction, the offset rate R is calculated by the following formula:

Therefore, considering the errors that the moving direction and speed of the camera and the subject may cause to image stitching, a calculation formula for the offset rate is introduced to improve the accuracy of image stitching.

S160. Adjust the first feature point and the second feature point based on the offset rate to obtain first modified feature points and second modified feature points.

In one implementation, step S160 may specifically include the following steps:

S161. Construct a three-dimensional coordinate system.

S162. Obtain the coordinate value (x ₁ , y ₁ , z ₁ ) of any one of the plurality of first feature points.

S163. Use the following formula to transform the coordinates of the feature point:

in,

Represents the coordinate conversion value,

Indicates the corrected coordinate value of the feature point.

S164. Traverse all the feature points in the first feature point and repeat steps S162-S163.

S165. Map the first corrected feature points based on the corrected coordinate values of all feature points in the first feature points.

S166. Obtain the coordinate value (x ₂ , y ₂ , z ₂ ) of any one of the plurality of second feature points.

S167. Use the following formula to transform the coordinates of the feature point:

in,

Represents the coordinate conversion value,

Indicates the corrected coordinate value of the feature point.

S168: Traverse all the feature points in the second feature point and repeat steps S166-S167.

S169. Based on the corrected coordinate values of all the feature points in the second feature points, map the second corrected feature points.

The calculation of the offset rate takes into account the errors that may be caused by the moving direction and speed of the camera and the subject in image stitching.

This implementation method introduces a three-dimensional coordinate system and innovatively combines the camera's internal parameter matrix and external parameter matrix with the offset rate caused by movement to correct the feature points of the two images and reduce the occurrence of splicing errors.

S170. Match the first modified feature point and the second modified feature point to obtain multiple optimal feature point pairs.

Wherein, Hamming distance is used to match each feature point in the first modified feature point and the second modified feature point to obtain a plurality of optimal feature point pairs with the shortest Hamming distance.

Specifically, the required number of optimal feature point pairs can be preset, or the Hamming distance threshold can be preset, which is not specifically limited here.

S180. Screen the multiple optimal feature point pairs, splice the first image to be spliced and the second image to be spliced, and obtain a total image.

Among them, a weighted fusion algorithm can be used to obtain the total image after stitching.

As a result, on the basis of image data that takes into account both large field of view and details, image deviations caused by movement of the camera and subject can be avoided, and image stitching efficiency and accuracy can be improved.

In order to implement the above method embodiments, this embodiment also provides an image stitching system based on a gigapixel array camera. As shown in Figure 2, the system includes:

The image data acquisition module 210 is used to acquire image data captured by a gigapixel array camera; the image data is the first image to be spliced and the second image to be spliced of the target object.

The feature point determination module 220 is configured to extract feature points from the first image to be spliced and the second image to be spliced respectively, and determine the first feature point and the second feature point.

The speed determination module 230 is configured to obtain the first moving speed and the second moving speed of the gigapixel array camera and the target object respectively.

The deviation rate calculation module 240 is configured to calculate a deviation rate based on the first moving speed and the second moving speed.

The feature point correction module 250 is configured to adjust the first feature point and the second feature point based on the offset rate to obtain first corrected feature points and second corrected feature points.

The optimal feature point pair acquisition module 260 is used to match the first modified feature point and the second modified feature point to obtain multiple optimal feature point pairs.

The image splicing module 270 is used to screen the multiple optimal feature point pairs, splice the first image to be spliced and the second image to be spliced, and obtain a total image.

Optionally, the feature point determination module 220 is further configured to: calculate n neighboring pixels based on each pixel on the first image to be spliced or the second image to be spliced, with the pixel as the center. The grayscale difference value of the point; if there are more than n/2 adjacent pixel points whose grayscale difference value meets the preset condition, then the pixel point is determined to be the first feature point or the second feature point.

Optionally, the offset rate calculation module 240 is further configured to: calculate the first offset s ₁ and the second offset s ₂ based on the first moving speed and the second moving speed respectively; If the gigapixel array camera and the target object move in the same direction, the offset rate R is calculated by the following formula:

s ₁ =|vx ₁ -vx ₂ |*Δt

s ₂ =|vd ₁ -vd ₂ |*Δt

Optionally, the feature point correction module 250 is further used to: construct a three-dimensional coordinate system; obtain the coordinate value (x ₁ , y ₁ , z ₁ ) of any one of the plurality of first feature points; through the following The formula performs coordinate transformation on the feature point:

in,

Represents the coordinate conversion value,

Optionally, the feature point correction module 250 is further configured to: obtain the coordinate value (x ₂ , y ₂ , z ₂ ) of any one of the plurality of second feature points; Point coordinate transformation:

in,

Represents the coordinate conversion value,

Optionally, the optimal feature point pair acquisition module 260 is further configured to use Hamming distance to match each feature point in the first modified feature point and the second modified feature point to obtain the shortest Hamming distance. multiple optimal feature point pairs.

Optionally, the image splicing module 270 is further configured to use a weighted fusion algorithm to obtain a spliced total image.

When calculating the offset rate, this system takes into account the errors that the moving direction and speed of the camera and the subject may cause in image stitching. It also introduces a three-dimensional coordinate system, combines camera parameters with the offset rate, and corrects the feature points of the two images. , which can improve the efficiency and accuracy of image stitching.

Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the modules/units/subunits/components in the above-described device can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. Repeat.

In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiment provided by this application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program code. .

It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need further definition and explanation in subsequent drawings. In addition, the terms "first", "second", "third", etc. are only used to distinguish descriptions and shall not be understood as indicating or implying relative importance.

Finally, it should be noted that the above-mentioned embodiments are only specific implementation modes of the present application, and are used to illustrate the technical solutions of the present application, but not to limit them. The protection scope of the present application is not limited thereto. Although refer to the foregoing The embodiments describe the present application in detail. Those of ordinary skill in the art should understand that any person familiar with the technical field can still modify the technical solutions recorded in the foregoing embodiments within the technical scope disclosed in the present application. It is possible to easily think of changes or equivalent substitutions of some of the technical features; however, these modifications, changes or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application. All are covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

An image stitching method based on a gigapixel array camera, which is characterized by including:

Obtain the image data captured by the gigapixel array camera; the image data is the first image to be spliced and the second image to be spliced of the target object;

Perform feature point extraction on the first image to be spliced and the second image to be spliced respectively, and determine the first feature point and the second feature point;

Obtain the first moving speed of the gigapixel array camera;

Obtain the second moving speed of the target object;

Calculate a deflection rate based on the first movement speed and the second movement speed;

Calculating the offset rate based on the first moving speed and the second moving speed includes:

Calculate the first offset s 1 and the second offset s 2 based on the first moving speed and the second moving speed respectively;

If the gigapixel array camera and the target object move in the same direction, the offset rate R is calculated by the following formula:

If the gigapixel array camera and the target object move in opposite directions, the offset rate R is calculated by the following formula:

Among them, t 2 represents the shooting time of the second image to be spliced, and t 1 represents the shooting time of the first image to be spliced, then the shooting time difference between the first image to be spliced and the second image to be spliced is Δt = |t 2 - t 1 | , vx 1 represents the moving speed of the gigapixel array camera at time t 1 , vd 1 represents the moving speed of the target object at time t 1 ;

Adjust the first feature point and the second feature point based on the offset rate to obtain first modified feature points and second modified feature points;

Match the first modified feature point and the second modified feature point to obtain multiple optimal feature point pairs;

Screen the multiple optimal feature point pairs, splice the first image to be spliced and the second image to be spliced, and obtain a total image.
The image splicing method according to claim 1, wherein the first image to be spliced and the second image to be spliced are two consecutive frames of images.
The image splicing method according to claim 1, wherein the first feature point extraction and the second feature point extraction are performed on the first image to be spliced and the second image to be spliced respectively, and the first feature point and the second feature point are determined, include:

According to each pixel on the first image to be spliced or the second image to be spliced, with the pixel as the center, calculate the grayscale differences of n adjacent pixels;

If there are more than n/2 adjacent pixels whose grayscale differences meet the preset conditions, then the pixel is determined to be the first feature point or the second feature point.
The image splicing method according to claim 1, wherein the first offset s 1 and the second offset s 2 are calculated based on the first moving speed and the second moving speed respectively, include:

The first offset s 1 and the second offset s 2 are calculated by the following formulas:

s 1 =|vx 1 -vx 2 |*Δt

s 2 =|vd 1 -vd 2 |*Δt

Among them, vx 1 represents the moving speed of the gigapixel array camera at time t 1 , vx 2 represents the moving speed of the gigapixel array camera at time t 2 , vd 1 represents the moving speed of the target object at time t 1 , vd 2 represents The moving speed of the target object at time t 2 , Δt represents the shooting time difference between the first image to be spliced and the second image to be spliced.
The image splicing method according to claim 1, characterized in that adjusting the first feature point based on the offset rate to obtain the first corrected feature point includes:

Construct a three-dimensional coordinate system;

Obtain the coordinate value (x 1 , y 1 , z 1 ) of any one of the plurality of first feature points;

Use the following formula to transform the coordinates of the feature point:

in,
Represents the coordinate conversion value,
Represents the internal parameter matrix of the gigapixel array camera, f x and f y represent the length of the focal length in the x-axis and y-axis directions, respectively, in millimeters; c x and c y respectively represent the optical center, in pixels; Q represents the rotation matrix, T represents the translation matrix, [Q T] represents the external parameter matrix of the gigapixel array camera; R represents the offset rate,
Indicates the corrected coordinate value of the feature point;

Traverse all feature points in the first feature point and repeat the above steps;

Based on the corrected coordinate values of all the feature points in the first feature points, the first corrected feature points are mapped.
The image stitching method according to claim 5, wherein adjusting the second feature point based on the offset rate to obtain a second corrected feature point includes:

Obtain the coordinate value (x 2 , y 2 , z 2 ) of any one of the plurality of second feature points;

Use the following formula to transform the coordinates of the feature point:

in,
Represents the coordinate conversion value,
Represents the internal parameter matrix of the gigapixel array camera, f x and f y represent the length of the focal length in the x-axis and y-axis directions, respectively, in millimeters; c x and c y respectively represent the optical center of the camera, in pixels; Q represents rotation Matrix, T represents the translation matrix, [Q T] represents the external parameter matrix of the gigapixel array camera; R represents the offset rate,
Indicates the corrected coordinate value of the feature point;

Traverse all feature points in the second feature point and repeat the above steps;

Based on the corrected coordinate values of all the feature points in the second feature points, the second corrected feature points are mapped.
The image splicing method according to claim 1, wherein the matching of the first corrected feature point and the second corrected feature point to obtain a plurality of optimal feature point pairs includes:

Hamming distance is used to match each feature point in the first modified feature point and the second modified feature point, and a plurality of optimal feature point pairs with the shortest Hamming distances are obtained.
The image splicing method according to claim 1, characterized in that, splicing the first image to be spliced and the second image to be spliced to obtain a total image includes:

A weighted fusion algorithm is used to obtain the spliced total image.
An image stitching system based on a gigapixel array camera, which is characterized by including:

An image data acquisition module is used to acquire image data captured by a gigapixel array camera; the image data is the first image to be spliced and the second image to be spliced of the target object;

A feature point determination module, configured to extract feature points from the first image to be spliced and the second image to be spliced respectively, and determine the first feature point and the second feature point;

A speed determination module, configured to obtain the first moving speed and the second moving speed of the gigapixel array camera and the target object respectively;

A deviation rate calculation module, configured to calculate a deviation rate based on the first moving speed and the second moving speed;

The offset rate calculation module is further configured to calculate the first offset s 1 and the second offset s 2 based on the first moving speed and the second moving speed respectively;

If the gigapixel array camera and the target object move in the same direction, the offset rate R is calculated by the following formula:

If the gigapixel array camera and the target object move in opposite directions, the offset rate R is calculated by the following formula:

Among them, t 2 represents the shooting time of the second image to be spliced, and t 1 represents the shooting time of the first image to be spliced, then the shooting time difference between the first image to be spliced and the second image to be spliced is Δt = |t 2 - t 1 | , vx 1 represents the moving speed of the gigapixel array camera at time t 1 , vd 1 represents the moving speed of the target object at time t 1 ;

A feature point correction module, configured to adjust the first feature point and the second feature point based on the offset rate to obtain first corrected feature points and second corrected feature points;

An optimal feature point pair acquisition module is used to match the first modified feature point and the second modified feature point to obtain multiple optimal feature point pairs;

An image splicing module is used to screen the plurality of optimal feature point pairs, splice the first image to be spliced and the second image to be spliced, and obtain a total image.