CN111179165B - Panoramic image generation method and device - Google Patents
Panoramic image generation method and device Download PDFInfo
- Publication number
- CN111179165B CN111179165B CN201911202669.8A CN201911202669A CN111179165B CN 111179165 B CN111179165 B CN 111179165B CN 201911202669 A CN201911202669 A CN 201911202669A CN 111179165 B CN111179165 B CN 111179165B
- Authority
- CN
- China
- Prior art keywords
- image
- spliced
- point
- position offset
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 16
- 238000001914 filtration Methods 0.000 claims abstract description 19
- 238000003384 imaging method Methods 0.000 claims abstract description 8
- 238000004364 calculation method Methods 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 claims description 9
- 230000005540 biological transmission Effects 0.000 claims description 7
- 230000006835 compression Effects 0.000 claims description 7
- 238000007906 compression Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000004927 fusion Effects 0.000 claims description 6
- 238000003702 image correction Methods 0.000 claims description 6
- 230000005236 sound signal Effects 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 3
- 230000001131 transforming effect Effects 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 5
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
- Stereoscopic And Panoramic Photography (AREA)
Abstract
The invention discloses a panoramic image generation method, which comprises the following steps: collecting original images through a plurality of groups of imaging units; correcting the original image into a longitude and latitude image; acquiring a region to be spliced between adjacent image pairs; calculating a position offset field between image pairs to be spliced point by point; performing median filtering treatment on the position offset field; and fusing all the areas to be spliced according to the filtered position offset field to generate a panoramic image. The invention also discloses a panoramic image generation device. The invention can align images point by point and fuse the images to achieve the effect of seamless splicing, thereby providing high-quality immersed panoramic experience for viewers.
Description
Technical Field
The invention relates to a panoramic image generation method and device.
Background
Because the virtual reality technology brings a unique panoramic picture or video display mode, the visual experience of being on the spot is brought to people, and therefore, the virtual reality technology is greatly touted and applied in the fields of travel, house watching, education and the like. Panoramic pictures are formed by splicing pictures acquired by a plurality of groups of imaging units, and improper splicing can lead to uneven alignment of splicing areas, thereby causing ghost images, picture missing and the like, and directly affecting the watching quality and visual feeling of a viewer.
Therefore, a new panoramic image generation method is required to solve the above-described problems.
Disclosure of Invention
The invention aims to solve the problems in the prior art and provides a panoramic image generation method.
In order to achieve the above object, the panoramic image generation method of the present invention may adopt the following technical scheme:
a panoramic image generation method, comprising the steps of:
1) Collecting a plurality of original images through a plurality of groups of imaging units;
2) Converting the original image in the step 1) into a longitude and latitude image;
3) Acquiring a region to be spliced between adjacent longitude and latitude images, wherein the region to be spliced comprises a left image to be spliced and a right image to be spliced;
4) Calculating the position offset field between the left image to be spliced and the right image to be spliced point by point;
5) Performing median filtering treatment on the position offset field;
6) And fusing all the areas to be spliced according to the filtered position offset field to generate a panoramic image.
Further, the step 2) of converting the original image in the step 1) into the longitude and latitude image includes the following steps:
21. obtaining a conversion model for converting an original image into a longitude and latitude image;
22. and converting the original image into a longitude and latitude image according to the conversion model.
Further, the step 3) of obtaining the band stitching region between the adjacent longitude and latitude images includes the following steps:
31. extracting feature points in adjacent longitude and latitude images by using a feature extraction algorithm, and carrying out feature point matching according to a matching rule to obtain at least one pair of successfully matched feature point pairs; if the feature points of the successful pairing are not found, repeating the steps 1) -3);
32. from the positional relationship of the feature point pairs P1 (x 1, y 1) and P2 (x 2, y 2), a homography matrix h= { H is calculated ij The formula is as follows:
transforming one image in the adjacent longitude and latitude images into a coordinate system where the other image is located according to the homography matrix H, and calculating the width of the overlapping area;
33. and respectively intercepting the overlapping width images in the adjacent longitude and latitude images according to the width of the overlapping region to obtain the image pair with splicing.
Further, in step 32), the adjacent longitude and latitude images are composed of a left image and a right image, the right image is transformed to a coordinate system where the left image is located according to the homography matrix, a first abscissa of a rightmost column of points of the left image is obtained, a second abscissa of a leftmost point of gray scale non-0 value of the right image in the left image coordinate system is obtained, and a difference between the first abscissa and the second abscissa is the width of the overlapping area.
Further, in step 33), the right-most overlapping width image of the left image is cut, and the left-most overlapping width image of the right image is cut, so as to obtain the image pair to be spliced.
Further, the calculating the position offset field between the left graph to be spliced and the right graph to be spliced point by point in the step 4) includes the following steps:
41. dividing grids point by point on the left graph to be spliced, wherein the number N=w×h of grid points, w is the width of the left image, and h is the height of the left image;
42. taking the grid points as the centers, and intercepting an area with the width of sub-pixel width, wherein the area is a reference image subarea;
43. moving the reference image subarea in the right image to be spliced, finding out the area most similar to the reference image subarea, wherein the central point of the area is the matching point of the grid point of the left image, and recording the position information of the matching point;
44. respectively making differences between the positions of the right graph matching points and the positions of the left graph grid points in the x and y directions to obtain the position offset in the x and y directions;
45. steps 41-44 are repeated until all grid points are obtained with respect to the x-and y-direction positional offsets, thereby obtaining the x-and y-direction positional offset fields.
Further, the median filtering of the position offset field in step 5) includes the steps of:
and respectively carrying out median filtering on the position offset fields in the x and y directions obtained by point-by-point calculation.
The beneficial effects are that: the panoramic image generation method can align images point by point and fuse the images to achieve the effect of seamless splicing, and provides high-quality immersed panoramic experience for viewers.
The invention also discloses a panoramic image generation device which comprises an image acquisition module, an image correction module, a zone splicing calibration module, a position offset field calculation module, a filtering module and a fusion module,
the image acquisition module is used for acquiring original images through a plurality of groups of imaging units;
the image correction module is used for converting an original image into a longitude and latitude image;
the region to be spliced calibration module is used for acquiring a region to be spliced between adjacent longitude and latitude images, and the region to be spliced comprises a left image to be spliced and a right image to be spliced;
the position offset field calculation module is used for calculating the position offset field between the left image to be spliced and the right image to be spliced point by point;
the filtering module is used for carrying out median filtering treatment on the position offset field;
and the fusion module is used for fusing all the areas to be spliced according to the filtered position offset field to generate a panoramic image.
Still further, still include audio acquisition module and storage module, audio acquisition module and storage module are used for synchronous collection audio signal when recording the video, storage module is used for storing image and audio data.
Still further, the terminal further comprises a compression coding module and a transmission module, wherein the compression coding module is used for compressing the image data and the audio signal, and the transmission module is used for transmitting the image and/or the audio data to other terminals. The compression coding module is used for reducing the data transmission bandwidth or the storage space requirement.
The beneficial effects are that: the panoramic image generation device can align images point by point and fuse the images to achieve the effect of seamless splicing, and provides high-quality immersed panoramic experience for viewers.
Drawings
FIG. 1 is a flow chart of a panoramic image generation method;
FIG. 2 is a flow chart of a position offset field;
fig. 3 is a schematic diagram of the structure of the image generating apparatus.
Detailed Description
The present invention will be further illustrated with reference to the accompanying drawings and specific examples, which are to be understood as being merely preferred embodiments of the invention, and various modifications thereof will be apparent to those skilled in the art after reading the present invention without departing from the principles of the invention, which are defined in the appended claims.
Referring to fig. 1, 2 and 3, the panoramic image generation method of the present invention includes the following steps:
1) Collecting a plurality of original images through a plurality of groups of imaging units;
2) Converting the original image in the step 1) into a longitude and latitude image;
preferably, the step 2) of converting the original image in the step 1) into the latitude and longitude image includes the following steps:
21. obtaining a conversion model for converting an original image into a longitude and latitude image;
22. and converting the original image into a longitude and latitude image according to the conversion model.
3) Acquiring a region to be spliced between adjacent longitude and latitude images, wherein the region to be spliced comprises a left image to be spliced and a right image to be spliced;
preferably, the acquiring the region to be spliced between the adjacent longitude and latitude images in the step 3) includes the following steps:
31. extracting feature points in adjacent longitude and latitude images by using a feature extraction algorithm, and carrying out feature point matching according to a matching rule to obtain at least one pair of successfully matched feature point pairs; if the feature points of the successful pairing are not found, repeating the steps 1) -3);
32. from the positional relationship of the feature point pairs P1 (x 1, y 1) and P2 (x 2, y 2), a homography matrix h= { H is calculated ij The formula is as follows:
transforming one image in the adjacent longitude and latitude images into a coordinate system where the other image is located according to the homography matrix H, and calculating the width of the overlapping area;
33. and respectively intercepting the overlapping width images in the adjacent longitude and latitude images according to the width of the overlapping region to obtain the image pair with splicing.
Preferably, in step 32), the adjacent longitude and latitude images are composed of a left image and a right image, the right image is transformed to a coordinate system where the left image is located according to a homography matrix, a first abscissa of a rightmost column of points of the left image is obtained, a second abscissa of a leftmost point of gray scale non-0 value of the right image in the left image coordinate system is obtained, and a difference between the first abscissa and the second abscissa is the width of the overlapping area.
Preferably, in step 33), the rightmost overlapping width image of the left image is cut, the leftmost overlapping width image of the right image is cut, and the image pair to be spliced is obtained.
4) Calculating a position offset field between the left image to be spliced and the right image to be spliced point by point;
preferably, the calculating the position offset field between the left graph to be spliced and the right graph to be spliced point by point in the step 4) includes the following steps:
41. dividing grids point by point on the left graph to be spliced, wherein the number N=w×h of grid points, w is the width of the left image, and h is the height of the left image;
42. taking the grid points as the centers, and intercepting an area with the width of sub-pixel width, wherein the area is a reference image subarea;
43. moving the reference image subarea in the right image to be spliced, finding out the area most similar to the reference image subarea, wherein the central point of the area is the matching point of the grid point of the left image, and recording the position information of the matching point;
44. respectively making differences between the positions of the right graph matching points and the positions of the left graph grid points in the x and y directions to obtain the position offset in the x and y directions;
45. steps 41-44 are repeated until all grid points are obtained with respect to the x-and y-direction positional offsets, thereby obtaining the x-and y-direction positional offset fields.
5) Performing median filtering treatment on the position offset field;
preferably, the median filtering of the position offset field in step 5) includes the steps of:
and respectively carrying out median filtering on the position offset fields in the x and y directions obtained by point-by-point calculation.
6) And fusing all the areas to be spliced according to the filtered position offset field to generate a panoramic image.
The panoramic image generation method can align images point by point and fuse the images to achieve the effect of seamless splicing, and provides high-quality immersed panoramic experience for viewers.
Referring to fig. 3, the invention also discloses a panoramic image generation device, which comprises an image acquisition module, an image correction module, a calibration module with a splicing area, a position offset field calculation module, a filtering module and a fusion module,
the image acquisition module is used for acquiring original images through a plurality of groups of imaging units;
the image correction module is used for converting the original image into a longitude and latitude image;
the region to be spliced calibration module is used for acquiring a region to be spliced between adjacent longitude and latitude images, wherein the region to be spliced comprises a left image to be spliced and a right image to be spliced;
the position offset field calculation module is used for calculating the position offset field between the left image to be spliced and the right image to be spliced point by point;
the filtering module is used for carrying out median filtering treatment on the position offset field;
and the fusion module is used for fusing all the areas to be spliced according to the filtered position offset field to generate a panoramic image.
Preferably, the system further comprises an audio acquisition module and a storage module, wherein the audio acquisition module and the storage module are used for synchronously acquiring audio signals when video is recorded, and the storage module is used for storing images and audio data.
Preferably, the terminal further comprises a compression coding module and a transmission module, wherein the compression coding module is used for compressing the image data and the audio signal, and the transmission module is used for transmitting the image and/or the audio data to other terminals. The compression coding module is used for reducing the data transmission bandwidth or the storage space requirement.
The panoramic image generation device can align images point by point and fuse the images to achieve the effect of seamless splicing, and provides high-quality immersed panoramic experience for viewers.
Claims (3)
1. A panoramic image generation method is characterized in that a panoramic image generation device is adopted, the panoramic image generation device comprises an image acquisition module, an image correction module, a region calibration module to be spliced, a position offset field calculation module, a filtering module and a fusion module,
the image acquisition module is used for acquiring original images through a plurality of groups of imaging units;
the image correction module is used for converting an original image into a longitude and latitude image;
the region to be spliced calibration module is used for acquiring a region to be spliced between adjacent longitude and latitude images, and the region to be spliced comprises a left image to be spliced and a right image to be spliced;
the position offset field calculation module is used for calculating the position offset field between the left image to be spliced and the right image to be spliced point by point;
the filtering module is used for carrying out median filtering treatment on the position offset field;
the fusion module is used for fusing all the areas to be spliced according to the filtered position offset field to generate a panoramic image;
the method comprises the following steps:
1) Collecting a plurality of original images through a plurality of groups of imaging units;
2) Converting the original image in the step 1) into a longitude and latitude image;
the step 2) of converting the original image in the step 1) into the longitude and latitude image comprises the following steps:
21. obtaining a conversion model for converting an original image into a longitude and latitude image;
22. according to the conversion model, converting the original image into a longitude and latitude image;
3) Acquiring a region to be spliced between adjacent longitude and latitude images, wherein the region to be spliced comprises a left image to be spliced and a right image to be spliced;
the step 3) of acquiring the region to be spliced between the adjacent longitude and latitude images comprises the following steps:
31. extracting feature points in adjacent longitude and latitude images by using a feature extraction algorithm, and carrying out feature point matching according to a matching rule to obtain at least one pair of successfully matched feature point pairs; if the feature points of the successful pairing are not found, repeating the steps 1) -3);
32. calculating homography matrix according to the position relation of the characteristic point pair P1 (x 1, y 1) and P2 (x 2, y 2)The formula is as follows:
transforming one image in the adjacent longitude and latitude images into a coordinate system where the other image is located according to the homography matrix H, and calculating the width of the overlapping area;
in the step 32), adjacent longitude and latitude images are composed of a left image and a right image, the right image is transformed to a coordinate system where the left image is located according to a homography matrix H, a first abscissa of a rightmost column of points of the left image is obtained, a second abscissa of a leftmost point of gray scale non-0 value of the right image under the left image coordinate system is obtained, and the difference between the first abscissa and the second abscissa is the width of a combined area;
33. according to the width of the overlapping region, overlapping width images in adjacent longitude and latitude images are respectively intercepted, and an image pair to be spliced is obtained; step 33), intercepting the rightmost overlapping width image of the left image, intercepting the leftmost overlapping width image of the right image, and obtaining an image pair to be spliced;
4) Calculating the position offset field between the left image to be spliced and the right image to be spliced point by point;
the step 4) of calculating the position offset field between the left graph to be spliced and the right graph to be spliced point by point comprises the following steps:
41. dividing grids point by point on the left graph to be spliced, wherein the number N=w×h of grid points, w is the width of the left image, and h is the height of the left image;
42. taking the grid points as the centers, and intercepting an area with the width of sub-pixel width, wherein the area is a reference image subarea;
43. moving the reference image subarea in the right image to be spliced, finding out the area most similar to the reference image subarea, wherein the central point of the area is the matching point of the grid point of the left image, and recording the position information of the matching point;
44. respectively making differences between the positions of the right graph matching points and the positions of the left graph grid points in the x and y directions to obtain the position offset in the x and y directions;
45. repeating the steps 41-44 until the position offset of all grid points in the x and y directions is obtained, so that the position offset fields in the x and y directions are obtained;
5) Performing median filtering treatment on the position offset field;
the median filtering of the position offset field in step 5) includes the steps of:
respectively carrying out median filtering on the position offset fields in the x and y directions obtained by point-by-point calculation;
6) And fusing all the areas to be spliced according to the filtered position offset field to generate a panoramic image.
2. The panoramic image generation method of claim 1, further comprising an audio acquisition module and a storage module, the audio acquisition module and the storage module for synchronously acquiring audio signals while recording video, the storage module for storing image and audio data.
3. The panoramic image generation method of claim 1, further comprising a compression encoding module for compressing the image data and the audio signal, and a transmission module for transmitting the image and/or the audio data to other terminals.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911202669.8A CN111179165B (en) | 2019-11-29 | 2019-11-29 | Panoramic image generation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911202669.8A CN111179165B (en) | 2019-11-29 | 2019-11-29 | Panoramic image generation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111179165A CN111179165A (en) | 2020-05-19 |
| CN111179165B true CN111179165B (en) | 2023-07-28 |
Family
ID=70657163
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911202669.8A Active CN111179165B (en) | 2019-11-29 | 2019-11-29 | Panoramic image generation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111179165B (en) |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105894449B (en) * | 2015-11-11 | 2019-11-08 | 法法汽车(中国)有限公司 | Overcome the method and system of color change in image co-registration |
| CN107274336B (en) * | 2017-06-14 | 2019-07-12 | 电子科技大学 | A kind of Panorama Mosaic method for vehicle environment |
| CN108769578B (en) * | 2018-05-17 | 2020-07-31 | 南京理工大学 | Real-time panoramic imaging system and method based on multiple cameras |
| CN109246477B (en) * | 2018-08-17 | 2021-04-27 | 南京泓众电子科技有限公司 | Panoramic video frame interpolation method and device |
| CN110246161B (en) * | 2019-06-04 | 2022-10-28 | 哈尔滨工程大学 | Method for seamless splicing of 360-degree panoramic images |
-
2019
- 2019-11-29 CN CN201911202669.8A patent/CN111179165B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111179165A (en) | 2020-05-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111580765B (en) | Screen projection method, screen projection device, storage medium, screen projection equipment and screen projection equipment | |
| US10595004B2 (en) | Electronic device for generating 360-degree three-dimensional image and method therefor | |
| CN112017222B (en) | Video panorama stitching and three-dimensional fusion method and device | |
| CN109863754B (en) | Virtual reality 360-degree video camera system for live streaming | |
| EP3457697A1 (en) | Method and apparatus for encoding and decoding video image | |
| CN109587401B (en) | Electronic cloud deck multi-scene shooting implementation method and system | |
| CN105959620A (en) | Panorama video synchronization display method and panorama video synchronization display device | |
| CN202524495U (en) | Course recording system based on fixed camera position, and video processing device | |
| CN103248855A (en) | Fixed seat-based lesson recording system, video processing device and lesson recording method | |
| JP2019514313A (en) | Method, apparatus and stream for formatting immersive video for legacy and immersive rendering devices | |
| CN107197135B (en) | Video generation method and video generation device | |
| JP5454569B2 (en) | 3D image data processing apparatus and 3D image data processing method | |
| CN111179165B (en) | Panoramic image generation method and device | |
| CN111083368A (en) | Simulation physics cloud platform panoramic video display system based on high in clouds | |
| JP2019149785A (en) | Video conversion device and program | |
| WO2024022086A1 (en) | Video communication method and system based on three-dimensional displaying | |
| CN104488024A (en) | Image generation method | |
| CN115278189A (en) | Image tone mapping method and apparatus, computer readable medium and electronic device | |
| CN111629194B (en) | Method and system for converting panoramic video into 6DOF video based on neural network | |
| CN102802003A (en) | Real-time shooting and real-time free stereoscopic display system based on both GPU and network cameras | |
| CN114004773A (en) | Monocular multi-view video synthesis method based on deep learning and reverse mapping | |
| CN112261441A (en) | Video data processing method, system, device and storage medium | |
| CN115442580B (en) | Naked eye 3D picture effect processing method for portable intelligent equipment | |
| CN112712474B (en) | Perspective correction method and system for video streaming images | |
| CN108282609B (en) | Panoramic video distribution monitoring system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |