The content of the invention
In order to solve the above problems, the present invention proposes a kind of video image splicing apparatus and its joining method, energy
The video for enough realizing embedded architecture gathers in real time, splicing and display.
Concrete technical scheme proposed by the present invention is:A kind of video image splicing apparatus is provided, including:Image
Acquisition module, for obtaining target video image and video image to be transformed, the target video image and institute
The subregion for stating video image to be transformed is overlapping;Image pre-processing module, for waiting to become described in reception
Change video image and carry out image preconditioning, to obtain preprocessed video image;Image mosaic accelerating module,
For the overlapping region of the target video image of reception and the preprocessed video image to be accelerated
Splicing, to obtain spliced video image.
Further, described image splicing accelerating module includes:First image format conversion unit, for pair
The target video image and the preprocessed video image enter row format conversion respectively;Filter unit, it is used for
Target video image and preprocessed video image after being changed respectively to form are filtered, to obtain the first filter
Ripple image and the second filtering image;Downsampling unit, for respectively to first filtering image and described
Two filtering images carry out down-sampling processing, to construct the first gaussian pyramid and the second gaussian pyramid;Difference
Unit, for respectively between the adjacent layer of first gaussian pyramid and second gaussian pyramid
Adjacent layer between make difference processing, to obtain the first laplacian pyramid and the second laplacian pyramid;
Computing unit, for numerical value corresponding to overlapping region of first laplacian pyramid and described the
The numerical value corresponding to overlapping region of two laplacian pyramids carries out successively weighted average, flat to obtain weighting
Laplacian pyramid after;Unit is up-sampled, for Laplce's gold word after the weighted average
Tower carries out up-sampling processing, to obtain the video image for treating form inverse conversion;Second image format conversion unit,
For entering row format inverse conversion to the video image for treating form inverse conversion, to obtain spliced video figure
Picture.
Further, described image pretreatment module includes:Feature extraction unit, for extracting the target
The characteristic point of video image and the video image to be transformed;Characteristic matching unit, for according to Hamming distance
The characteristic point of characteristic point and the video image to be transformed to the target video image match and basis
Homography matrix is calculated in matching relationship;Image transforming unit, for being treated using the homography matrix
Convert video image and carry out image preconditioning, to obtain preprocessed video image.
Further, in addition to be connected to described image pretreatment module and described image splicing accelerating module it
Between memory cell (DDR), the DDR be used for store target video image and preprocessed video image.
Further, in addition to described image splicing accelerating module the display module being connected, the display mould
Block is used to show spliced video image.
Present invention also offers a kind of joining method of video image, including:
Obtain target video image and video image to be transformed;Wherein, target video image and video to be transformed
The subregion of image is overlapping;
Image preconditioning is carried out to video image to be transformed, to obtain preprocessed video image;
The overlapping region of target video image and preprocessed video image is spliced, it is spliced to obtain
Video image.
Further, the method spliced to the overlapping region of target video image and preprocessed video image
Including:
Enter row format conversion respectively to target video image and preprocessed video image;
Target video image and preprocessed video image after being changed respectively to form using gaussian kernel function are carried out
Filtering, to obtain the first filtering image and the second filtering image;
Down-sampling processing is carried out to the first filtering image and the second filtering image respectively, to construct the first Gauss gold
Word tower and the second gaussian pyramid;
Respectively to making between the adjacent layer of the first gaussian pyramid and between the adjacent layer of the second gaussian pyramid
Difference processing, to obtain the first laplacian pyramid and the second laplacian pyramid;
Numerical value and second laplacian pyramid to the first laplacian pyramid corresponding to overlapping region
Successively weighted average is carried out corresponding to the numerical value of overlapping region, to obtain Laplce's gold word after weighted average
Tower;
Up-sampling processing is carried out to the laplacian pyramid after weighted average, form inverse conversion is treated to obtain
Video image;
The video image for treating form inverse conversion enters row format inverse conversion, to obtain spliced video image.
Further, image preconditioning is carried out to video image to be transformed, to obtain preprocessed video figure
The method of picture includes:
Extract the characteristic point of target video image and video image to be transformed;
According to Hamming distance to the characteristic point of target video image and the characteristic point progress of video image to be transformed
Match somebody with somebody;
Homography matrix is calculated according to matching relationship;
Image preconditioning is carried out to video image to be transformed using homography matrix, regarded with obtaining pretreatment
Frequency image.
Further, image preconditioning is being carried out to video image to be transformed, to obtain preprocessed video
After image, target video image and preprocessed video image are stored in DDR;
Before splicing to the overlapping region of target video image and preprocessed video image, from DDR
Read target video image and preprocessed video image.
Further, after obtaining spliced video image, the video image after display splicing.
Video image splicing apparatus and its joining method proposed by the present invention, the video image splicing apparatus bag
Image mosaic accelerating module is included, image mosaic accelerating module is User Defined IP kernel, and image mosaic accelerates mould
Block can splice according to user-defined merging algorithm for images to video image, and VDMA units,
Image mosaic accelerating module and output unit realized by FPGA, by software and hardware collaborative design,
So as to realize the video of embedded architecture gather in real time, splicing and display.
Embodiment
Hereinafter, with reference to the accompanying drawings to embodiments of the invention are described in detail.However, it is possible to many different
Form implements the present invention, and the present invention should not be construed as limited to the specific embodiment that illustrates here.
Conversely, there is provided these embodiments are in order to explain the principle and its practical application of the present invention, so that this area
Others skilled in the art it will be appreciated that various embodiments of the present invention and being suitable for the various of specific intended application and repairing
Change.
Video image splicing apparatus provided by the invention is the embedded video figure based on Xilinx ZYNQ platforms
As splicing apparatus, it includes processing system (Processing System, PS) and programmable system
(Programmable Logic, PL), PS is arm processor here.PS and PL collection in the present invention
Into on one chip.
Reference picture 1, the video image splicing apparatus that the present embodiment provides include image capture module 10, PS, PL
And it is connected to the memory cell (DDR) 30 between the PS and PL, it is preferred that DDR 30 is double
Times speed synchronous DRAM.Wherein PS includes image pre-processing module 20, and PL spells including image
Connect accelerating module 40 and display module 50.
Image capture module 10 is used to obtain target video image and video image to be transformed, the target video
Image is overlapping with the subregion of the video image to be transformed, for example, image capture module 10 is camera
Image capture module 10 is connected with image pre-processing module 20, for example, image capture module 10 passes through USB
Interface is attached with image pre-processing module 20, and image pre-processing module 20 is used for being treated described in reception
Convert video image and carry out image preconditioning, to obtain preprocessed video image.DDR 30 is connected to figure
As between pretreatment module 20 and image mosaic accelerating module 40, locating for storing target video image with pre-
Manage video image.Image mosaic accelerating module 40 is connected with the display module 50 again after being connected with DDR 30,
Image mosaic accelerating module 40 is used to be added the overlapping region for marking video image and preprocessed video image
Speed Pinyin connects, to obtain spliced video image, wherein, image mosaic accelerating module 40 is User Defined
IP kernel.The display module 50 is used to be shown spliced video image.In order to obtain joining quality
Preferable video image, the area of overlapping region is face in both target video image or video image to be transformed
Product less 20%~80%;Consider amount of calculation problem simultaneously, it is preferred that the area of overlapping region is target
Area less 40%~60% in both video image or video image to be transformed.Reference picture 2, image is located in advance
Feature extraction unit 21, characteristic matching unit 22 and the image conversion that reason module 20 includes being sequentially connected are single
Member 23.Feature extraction unit 21 is connected with image capture module 10, and feature extraction unit 21 is used to receive mesh
Mark video image and video image to be transformed and extract the spy of target video image and video image to be transformed respectively
Levy point, characteristic matching unit 22 is used for according to Hamming distance to the characteristic point of the target video image and described
The characteristic point of video image to be transformed is matched and homography matrix, image is calculated according to matching relationship
Converter unit 23 is used to carry out image preconditioning to video image to be transformed using the homography matrix,
To obtain preprocessed video image.Image transforming unit 23 is also connected with DDR 30, for pretreatment to be regarded
Frequency image is sent to DDR 30 and stored.
Reference picture 3, PL systems also include VDMA units 41 and output unit 42.VDMA units
41 are connected between DDR 30 and image mosaic accelerating module 40, and VDMA units 41 pass through HP interfaces
Enter row data communication between DDR 30, VDMA units 41 are used to extract target video from DDR 30
Image and preprocessed video image simultaneously pass through reality between AXI4-Stream interfaces and image mosaic accelerating module 40
Existing data transfer.Output unit 42 is connected between image mosaic accelerating module 40 and display module 50, defeated
Go out unit 42 to be used to enter spliced video image row format conversion, to cause spliced video image
The display format of form and display module 50 matches.For example, display module 50 is HDMI display,
The color mode for the spliced video image that output unit 42 receives from image mosaic accelerating module 40 is
The color mode of spliced video image is converted to YCbCr by RGB, output unit 42, then by display
Module 50 is shown.
In the present embodiment, VDMA units 41, image mosaic accelerating module 40 and output unit 42 are logical
FPGA realizations are crossed, it is of course also possible to realize by other means, example is merely possible to here and shows simultaneously
It is not used in and limits the invention.
Reference picture 4, image mosaic accelerating module 40 include the first image format conversion unit being sequentially connected
100th, filter unit 101, downsampling unit 102, difference unit 103, computing unit 104, up-sampling are single
The image format conversion unit 106 of member 105 and second.First image format conversion unit 100 is gone back and VDMA
Unit 41 connects, and the second image format conversion unit 106 is single with output again after being connected with up-sampling unit 105
Member 42 connects.
First image format conversion unit 100 be used for respectively by the target video image of AXIvideo forms and
Pretreated video image is converted to Mat forms, and filter unit 101 is used for respectively to the mesh of Mat forms
Mark video image and preprocessed video image are filtered, to obtain the first filtering image and the second filtering image,
Downsampling unit 102 is used to carry out down-sampling to the first filtering image and the second filtering image respectively, with construction
First gaussian pyramid and the second gaussian pyramid.Difference unit 103 is used for respectively to the first Gauss gold word
Difference processing is carried out between the adjacent layer of tower and between the second gaussian pyramid adjacent layer, is drawn with obtaining first
This pyramid of pula and the second laplacian pyramid, computing unit 104 are used to draw pula to described first
This pyramidal numerical value corresponding to overlapping region corresponds to overlay region with second laplacian pyramid
The numerical value in domain carries out successively weighted average, and to obtain the laplacian pyramid after weighted average, up-sampling is single
Member 105 is used to carry out up-sampling processing to the laplacian pyramid after the weighted average of reception, to obtain
The video image of form inverse conversion must be treated, the second image format conversion unit 106 is used to treat that form is inverse to described
The video image of conversion enters row format inverse conversion, to obtain spliced video image, i.e., by Mat forms
Spliced video image is converted to the video image of AXIvideo forms.
Reference picture 5, the present embodiment additionally provide the joining method of the video image splicing apparatus, methods described
Including:
Step S1, target video image and video image to be transformed are obtained;Wherein, target video image and treat
The subregion for converting video image is overlapping.In order to obtain the preferable video image of joining quality, overlapping region
Area be both target video image or video image to be transformed in area less 20%~80%;Examine simultaneously
Consider amount of calculation problem, it is preferred that the area of overlapping region is target video image or video image to be transformed
Area less 40%~60% in both.
Step S2, image preconditioning is carried out to video image to be transformed, to obtain preprocessed video image.
Reference picture 6, specifically, step S2 includes:
S21, extraction target video image and video image to be transformed characteristic point, wherein, feature point extraction is calculated
Method uses ORB (ORiented Brief) algorithm.
S22, according to Hamming distance the characteristic point of target video image and the feature of video image to be transformed are clicked through
Row matching.
S23, homography matrix is calculated according to matching relationship.
S24, using homography matrix image preconditioning is carried out to video image to be transformed, to obtain pre- place
Manage video image.
Referring again to Fig. 5, target video image and the storage of preprocessed video image step S3, are arrived into DDR 30
In.
Step S4, the overlapping region of target video image and preprocessed video image is spliced, to obtain
Spliced video image.
Reference picture 7, specifically, step S4 includes:
S41, target video image and preprocessed video image are read from DDR 30, to target video image
Enter row format conversion respectively with preprocessed video image, wherein, form is changed for example by AXIvideo forms
Target video image and preprocessed video image are converted to target video image and the pretreatment of Mat forms
Video image.S42, using gaussian kernel function respectively to form change after target video image and pretreatment regard
Frequency image is filtered, to obtain the first filtering image and the second filtering image.
S43, down-sampling processing is carried out to the first filtering image and the second filtering image respectively, it is high with construction first
This pyramid and the second gaussian pyramid.
S44, respectively between the adjacent layer of the first gaussian pyramid and the second gaussian pyramid adjacent layer it
Between make difference processing, to obtain the first laplacian pyramid and the second laplacian pyramid.
S45, the numerical value and second Laplce's gold word corresponding to overlapping region to the first laplacian pyramid
The numerical value corresponding to overlapping region of tower carries out successively weighted average, to obtain the Laplce after weighted average
Pyramid.
S46, up-sampling processing is carried out to the laplacian pyramid after weighted average, treat that form reverses to obtain
The video image changed, wherein, the video image for treating form inverse conversion is Mat forms.
S47, the video image for treating form inverse conversion enter row format inverse conversion, to obtain spliced video figure
Picture, wherein, spliced video image is AXIvideo forms.
Referring again to Fig. 5, step S5, spliced video image is shown.
Video image splicing apparatus and its joining method proposed by the present invention, the video image splicing apparatus bag
Image mosaic accelerating module is included, image mosaic accelerating module is User Defined IP kernel, and image mosaic accelerates mould
Block can splice according to user-defined merging algorithm for images to video image, and VDMA units,
Image mosaic accelerating module and output unit realized by FPGA, by software and hardware collaborative design,
So as to realize the video of embedded architecture gather in real time, splicing and display.
Described above is only the embodiment of the application, it is noted that for the common of the art
For technical staff, on the premise of the application principle is not departed from, some improvements and modifications can also be made,
These improvements and modifications also should be regarded as the protection domain of the application.