CN103489214A - Virtual reality occlusion handling method, based on virtual model pretreatment, in augmented reality system - Google Patents
Virtual reality occlusion handling method, based on virtual model pretreatment, in augmented reality system Download PDFInfo
- Publication number
- CN103489214A CN103489214A CN201310409925.7A CN201310409925A CN103489214A CN 103489214 A CN103489214 A CN 103489214A CN 201310409925 A CN201310409925 A CN 201310409925A CN 103489214 A CN103489214 A CN 103489214A
- Authority
- CN
- China
- Prior art keywords
- profile
- coordinate
- dummy model
- dimensional
- depth
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Processing Or Creating Images (AREA)
Abstract
The invention relates to a virtual reality occlusion handling method, based on a virtual model pretreatment, in augmented reality system. The method comprises the steps of utilizing a depth camera KINECT to obtain a color image and a gray level image representing depth information; converting the color image into a bitmap image capable of being identified and tracked by an augmented reality occlusion system and registering a virtual model in a three-dimension mode; combining the three-dimensional registration position of the virtual model and the self depth of the virtual model to conduct threshold treatment on the gray level image and extracting the peripheral contour of a real object; in the render scene of the virtual model, registering a contour coordinate system in a three-dimensional mode; switching a two-dimensional outline vertex coordinate system into a three-dimensional coordinate corresponding to the actual size, drawing in the contour coordinate system, and using a re-drawn contour as a three-dimensional model to shield the virtual model; combining the color image and the treated virtual model, filling a real object image into an outline internal area, namely, the occlusion part of the virtual model, so as to obtain a virtual reality occlusion effect. According to the method, pre-modeling and comparing the depth information of the virtual model pixel by pixel are not needed, so that the method is suitable for environments with unknown changes and can meet instantaneity requirements.
Description
Technical field:
The present invention relates in a kind of augmented reality system based on the pretreated mutual occlusion disposal route of dummy model, profile is extracted for it and dummy model is three-dimensional registers and Rendering combines, and is applied in the augmented reality system with degree of depth camera KINECT.The present invention compares the dummy model depth information without modeling in advance and individual element, is applicable to the environment of unknown variations, can requirement of real time.Belong to virtual reality, image processing and display technique field.
Background technology:
Augmented reality, in the application in the fields such as the distant operation of robot for space, requires dummy object and real-world object true reappearance mutual alignment relation.The augmented reality system directly is superimposed upon the dummy object image marker position in scene image on composograph, object in real scene is blocked by dummy object all the time, cause the observer on the sense organ direction lose with locus on entanglement, can't correctly judge the relative position relation of actual situation object.Therefore, between the actual situation object, the correct performance of hiding relation is that the augmented reality system drops into practical application problem in the urgent need to address.
Existing researchist has launched research to the mutual occlusion problem both at home and abroad, and the method for head it off mainly contains two kinds at present: the method based on model and the method based on the degree of depth.Mutual occlusion method ultimate principle based on model is: three-dimensional reconstruction blocks the real-world object of dummy model, and in the scene that is added on corresponding position.Document " Object Calibration for Augmented Reality " (Whitaker Ross, Crampton Chris, Breen David, Computer Graphics Forum, 1995,14 (3): 15-27) by used for multi-vision visual and three-dimensional rebuilding method, set up the three-dimensional model of real-world object, recover two-dimentional block surface, realize occlusion effect.But in this method, the three-dimensional reconstruction real-world object is more difficult, and calculated amount is large, and to block precision not high at the reconstruction model edge.
Mutual occlusion method ultimate principle based on the degree of depth is: the degree of depth of each pixel on computed image, and compare with the dummy model depth information, if dummy model is blocked and is not drawn by real-world object, when composograph, only have the dummy model partial display that is not blocked, or repaint after composograph on real-world object pixel the composograph that is added to.Document " Stereo vision based video see-through mixed reality " (Yokoya, N., Takemura, H., Okuma, T, Proceedings of the 1st International Symposia on Mixed Reality, 1999:85-94) propose to utilize the binocular stereo vision principle to calculate the depth information of object on the real scene image, superposed positions and the degree of depth according to dummy model, complete mutual occlusion.Document " Real-Time Occlusion Handling in Augmented Reality Based on an Object Tracking Approach " (Yuan Tian, Tao Guan, Cheng Wang, Sensors, 2010,10 (4): mutual occlusion 2885-2900) proposed based on the profile tracking is processed framework, on the composograph of hiding relation mistake, adopt OpenGL to redraw technology and repaint the pixel of blocking the object interior zone, complete mutual occlusion and process.When these class methods solve depth information, calculated amount is large, and the image stack time need to realize the Realistic Rendering to true shelter, when the complex-shaped and shared zone of true shelter is large, realizes that effect is not very desirable.
Summary of the invention:
Based on above-mentioned, the present invention propose a kind of be applied to under the augmented reality system of degree of depth camera KINECT based on the pretreated mutual occlusion disposal route of dummy model, the method is not only applicable to fully unknown working environment, and requirement of real time, can realize good mutual occlusion effect.
In order to realize this purpose, the technical solution used in the present invention is:
In a kind of augmented reality system, based on the pretreated mutual occlusion disposal route of dummy model, it is characterized in that comprising the following steps:
1) utilize degree of depth camera KINECT to obtain the coloured image of scene and the gray level image of sign depth information;
2) this coloured image is converted to bitmap images the three-dimensional registration dummy model of augmented reality mutual occlusion system energy recognition and tracking;
3) position of the three-dimensional registration of combined with virtual model and dummy model self degree of depth are carried out threshold process to the gray level image that characterizes depth information and are extracted the peripheral profile of the real-world object that blocks dummy model;
4) play up in scene three-dimensional registration profile coordinate system at dummy model; The two-dimensional silhouette apex coordinate is converted to corresponding actual size real-world object profile summit three-dimensional coordinate and in the profile coordinate system, draw, make its projection on screen and the contour convergence of real-world object projection, the profile repainted can block dummy model as " three-dimensional model ", completes the dummy model pre-service;
5) by coloured image as a setting with process after the dummy model composograph, the profile interior zone is that the dummy model part that is blocked is filled by the real-world object image, obtains the mutual occlusion effect.
Further: the concrete steps of extracting the peripheral profile of real-world object that blocks dummy model in described step 3) are: 3.1) to the gray level image noise reduction process; 3.2) calculate the real-world object degree of depth of blocking dummy model with depth and dummy model self depthometer of the three-dimensional registration of dummy model, and using that this processes gray level image as threshold value, the pixel grey scale that is greater than this threshold value is set to unified value, and the pixel that is less than or equal to this threshold value is disallowable; 3.3) extract profile on gray level image after threshold process, to the outline polygon approximation process, reduce profile summit number, alleviate calculated amount.
Further: in described step 4), make draw outline projection in screen coordinate system with the concrete steps that real-world object projected outline overlaps be: 4.1) the profile coordinate system of the peripheral profile of real-world object is drawn in three-dimensional registration, and its initial point is overlapped with the screen coordinate system initial point in the projection of screen; 4.2) convert the two-dimensional silhouette summit pixel coordinate of extraction to the voxel coordinate according to the pose of profile coordinate system registration; 4.3) the actual physics value corresponding to the profile summit voxel coordinate figure after conversion convert to, and draw out under the profile coordinate system, interior zone is set and is filled.
Further: described step 4.1), the method for three-dimensional registration profile coordinate origin is: attitude is identical with the intrinsic coordinates of the scene of playing up dummy model; The depth value that the degree of depth of registered location is threshold process, physical values corresponding to center of gravity pixel coordinate that the displacement of all the other two change in coordinate axis direction of registered location is screen coordinate system, can be obtained by the depth calculation of intrinsic parameters of the camera and profile coordinate origin.
Further: described step 4.2), the method for transformed profile apex coordinate is: the coordinate figure of two coordinate axis parallel with screen coordinate system in the corresponding profile coordinate system of two-dimensional pixel coordinate, profile is without thickness, and it is 0 at the coordinate figure of the 3rd coordinate axis that summit is set.
Further: the method that described step 4.3), the transformed profile apex coordinate is the actual physics value is: calibrating camera obtains the intrinsic parameters of the camera matrix, obtain physical values corresponding to pixel cell size, according to the pin hole projection model, in conjunction with the profile coordinate system degree of depth, can calculate profile summit actual physics coordinate.
The invention has the advantages that: be not only applicable to fully unknown working environment, and requirement of real time, can realize good mutual occlusion effect.
The accompanying drawing explanation:
Figure is overall construction drawing of the present invention.
Embodiment:
The present invention be in a kind of augmented reality system based on the pretreated mutual occlusion disposal route of dummy model, its adopts register the shelter profile of extraction and the method for drafting is strictly according to the facts processed the mutual occlusion of augmented reality in playing up the scene of dummy model.
As shown in Figure of description.The general steps of the inventive method is: utilize degree of depth camera KINECT to obtain the coloured image of scene and the gray level image of sign depth information; This coloured image is converted to bitmap images the three-dimensional registration dummy model of augmented reality mutual occlusion system energy recognition and tracking; The position of the three-dimensional registration of combined with virtual model and dummy model self degree of depth are carried out threshold process to the gray level image that characterizes depth information and are extracted the peripheral profile of the real-world object that blocks dummy model; Play up in scene three-dimensional registration profile coordinate system at dummy model; The two-dimensional silhouette apex coordinate is converted to corresponding actual size real-world object profile summit three-dimensional coordinate and in the profile coordinate system, draw, make its projection on screen and the contour convergence of real-world object projection, the profile repainted can block dummy model as " three-dimensional model ", completes the dummy model pre-service; By coloured image as a setting with process after the dummy model composograph, the profile interior zone is that the dummy model part that is blocked is filled by the real-world object image, realizes that mutual occlusion processes.
The real-world object that the present invention is blocked participation, without priori, directly extracts scene video stream, mainly is divided into image and processes and dummy model pre-service two parts in the realization of program; After the coloured image that degree of depth camera KINECT is extracted and gray level image are processed, obtain the bitmap images that required real-world object profile and augmented reality mutual occlusion system can Tracking Recognition; The profile obtained is repainted in playing up the dummy model scene to the pre-service dummy model; Coloured image after synthetic pretreated dummy model and processing, obtain the mutual occlusion image.
Below the present invention is described in further detail.
(1) image is processed
Utilize degree of depth camera KINECT to obtain the gray level image of scene coloured image and sign depth information simultaneously, be treated to respectively next step dummy model pre-service profile information and bitmap images are provided.Image is processed and is divided into the following steps:
1) coloured image conversion.The coloured image format conversion is become to the bitmap images that augmented reality mutual occlusion system can Tracking Recognition, and three-dimensional registration dummy model;
(three-dimensional registration refers to by the computer graphical Epidemiological Analysis, obtain the transformation relation of unique point coordinate system and camera coordinate system, by dummy model the pose definite with this transformation relation binding be added to shot by camera to real scene in, reach the accurate seamless fusion of real scene and dummy model.Three-dimensional registration comprises that the transformation relation that unique point coordinate system and camera coordinate system and indicator screen coordinate system, unique point coordinate are tied to camera coordinate system is the required three-dimension varying matrix of three-dimensional registration)
2) gray level image threshold process.At first to the gray level image noise reduction process, then combined with virtual model registered location and self depth information are determined the real-world object depth range that blocks dummy model, and using this as the threshold process gray level image, the image that setting is greater than this threshold value retains and unifies gray scale, and the image that is less than or equal to this threshold value is disallowable;
3) profile extracts.The real-world object profile extracted on gray level image also passes through the polygon approximation process, reduces profile summit number when guaranteeing contour accuracy, reduces the workload that profile repaints.
(2) dummy model pre-service
The inventive method adopts the real-world object profile to replace the real-world object model of three-dimensional reconstruction, the two-dimensional silhouette of extracting is registered and drawn with dummy model as one " three-dimensional model " in Same Scene, convert the occlusion issue of two three-dimensional models to, the dummy model part be blocked does not show, obtains the dummy model pretreating effect.Pretreated dummy model does not show partly on composograph is filled by the real-world object image, completes mutual occlusion and processes.The key of this part is that the projection of profile on screen that registration is drawn overlaps with the real-world object image outline.The dummy model pre-service is divided into the following steps:
1) camera calibration.Employing settles scaling method at one go and obtains intrinsic parameters of the camera.
2) coordinate system of registration draw outline.For guaranteeing that the projection of profile on screen of drawing overlaps with the real-world object image outline, profile coordinate origin after registration is overlapped with the screen coordinate system initial point at screen prjection, and now the contour pixel coordinate conversion becomes after D coordinates value the real-world object profile coordinate figure of full-size(d) under direct corresponding profile coordinate system.The profile coordinate system is with respect to the depth displacement of camera coordinate system the threshold value that the gray level image threshold process adopts, in the corresponding screen barycentric coordinates value of evolution difference of two other change in coordinate axis direction.Adopt the pin-hole imaging principle to calculate the parameter of two coordinate system transformation relations, profile coordinate system change in coordinate axis direction is identical with the scene intrinsic coordinates of playing up dummy model, by calculating the three-dimensional registration matrix of profile coordinate system.
3) profile is drawn.Position relationship according to screen coordinate system and profile coordinate system converts two-dimensional silhouette summit pixel coordinate to the three-D profile apex coordinate, convert the actual physics value to by pin-hole imaging principle and intrinsic parameters of the camera simultaneously, draw the profile summit after all converting in the profile coordinate system.Profile after drafting blocks dummy model in scene, and its part that is blocked is not shown, completes the dummy model pre-service.
(3) occlusion effect realizes
By pretreated dummy model and the coloured image after processing synthetic, on composograph, pretreated dummy model does not show that part filled by the real-world object image, obtains augmented reality mutual occlusion effect.
A whole set of augmented reality mutual occlusion handling procedure runs on the PC of 3.3GHz tetra-core CPU, utilizes degree of depth camera KINECT to gather video flowing, and gathering image resolution ratio is 640 * 480.A red cone is registered in real scene and carries out the mutual occlusion experiment, adopt the OpenCV function library process image and extract profile, virtual three-dimensional model is played up with OpenSceneGraph, and the actual situation fusion realizes based on ARToolKit.
Claims (6)
- In an augmented reality system based on the pretreated mutual occlusion disposal route of dummy model, it is characterized in that comprising the following steps:1) utilize degree of depth camera KINECT to obtain the coloured image of scene and the gray level image of sign depth information;2) this coloured image is converted to bitmap images the three-dimensional registration dummy model of augmented reality mutual occlusion system energy recognition and tracking;3) position of the three-dimensional registration of combined with virtual model and dummy model self degree of depth are carried out threshold process to the gray level image that characterizes depth information and are extracted the peripheral profile of the real-world object that blocks dummy model;4) play up in scene three-dimensional registration profile coordinate system at dummy model; The two-dimensional silhouette apex coordinate is converted to corresponding actual size real-world object profile summit three-dimensional coordinate and in the profile coordinate system, draw, make its projection on screen and the contour convergence of real-world object projection, the profile repainted can block dummy model as " three-dimensional model ", completes the dummy model pre-service;5) by coloured image as a setting with process after the dummy model composograph, the profile interior zone is that the dummy model part that is blocked is filled by the real-world object image, obtains the mutual occlusion effect.
- In augmented reality system as claimed in claim 1 based on the pretreated mutual occlusion disposal route of dummy model, it is characterized in that:The concrete steps of extracting the peripheral profile of real-world object that blocks dummy model in described step 3) are:3.1) to the gray level image noise reduction process;3.2) calculate the real-world object degree of depth of blocking dummy model by depth and dummy model self depthometer of the three-dimensional registration of dummy model, and using that this processes gray level image as threshold value, the pixel grey scale that is greater than this threshold value is set to unified value, and the pixel that is less than or equal to this threshold value is disallowable;3.3) extract profile on gray level image after threshold process, to the outline polygon approximation process, reduce profile summit number, alleviate calculated amount.
- In augmented reality system as claimed in claim 1 based on the pretreated mutual occlusion disposal route of dummy model, it is characterized in that:In described step 4), make the projection of draw outline on screen with the concrete steps that real-world object projected outline overlaps be:4.1) the three-dimensional profile coordinate system of drawing the peripheral profile of real-world object of registering, its initial point is overlapped with the screen coordinate system initial point in the projection of screen;4.2) convert the two-dimensional silhouette summit pixel coordinate of extraction to the voxel coordinate according to the pose of profile coordinate system registration;4.3) the actual physics value corresponding to the profile summit voxel coordinate figure after conversion convert to, and draw out under the profile coordinate system, interior zone is set and is filled.
- In augmented reality system as claimed in claim 3 based on the pretreated mutual occlusion disposal route of dummy model, it is characterized in that:Described step 4.1) in, the method for three-dimensional registration profile coordinate origin is: attitude is identical with the intrinsic coordinates of the scene of playing up dummy model; The depth value that the degree of depth of registered location is threshold process, physical values corresponding to center of gravity pixel coordinate that the displacement of all the other two change in coordinate axis direction of registered location is screen coordinate system, can be obtained by the depth calculation of intrinsic parameters of the camera and profile coordinate origin.
- In augmented reality system as claimed in claim 3 based on the pretreated mutual occlusion disposal route of dummy model, it is characterized in that:Described step 4.2) in, the method for transformed profile apex coordinate is: the coordinate figure of two coordinate axis parallel with screen coordinate system in the corresponding profile coordinate system of two-dimensional pixel coordinate, and profile is without thickness, and it is 0 at the coordinate figure of the 3rd coordinate axis that summit is set.
- In augmented reality system as claimed in claim 3 based on the pretreated mutual occlusion disposal route of dummy model, it is characterized in that:Described step 4.3) method that in, the transformed profile apex coordinate is the actual physics value is: calibrating camera obtains the intrinsic parameters of the camera matrix, obtain physical values corresponding to pixel cell size, according to the pin hole projection model, in conjunction with the profile coordinate system degree of depth, can calculate profile summit actual physics coordinate.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310409925.7A CN103489214A (en) | 2013-09-10 | 2013-09-10 | Virtual reality occlusion handling method, based on virtual model pretreatment, in augmented reality system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310409925.7A CN103489214A (en) | 2013-09-10 | 2013-09-10 | Virtual reality occlusion handling method, based on virtual model pretreatment, in augmented reality system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103489214A true CN103489214A (en) | 2014-01-01 |
Family
ID=49829410
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310409925.7A Pending CN103489214A (en) | 2013-09-10 | 2013-09-10 | Virtual reality occlusion handling method, based on virtual model pretreatment, in augmented reality system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103489214A (en) |
Cited By (57)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103785174A (en) * | 2014-02-26 | 2014-05-14 | 北京智明星通科技有限公司 | Method and system for displaying tens of thousands of people on same screen of game |
| CN104504671A (en) * | 2014-12-12 | 2015-04-08 | 浙江大学 | Method for generating virtual-real fusion image for stereo display |
| CN105005970A (en) * | 2015-06-26 | 2015-10-28 | 广东欧珀移动通信有限公司 | Augmented reality implementation method and apparatus |
| CN105354819A (en) * | 2015-09-29 | 2016-02-24 | 上海图漾信息科技有限公司 | Depth data measurement system, depth data determination method and apparatus |
| CN105719233A (en) * | 2016-01-21 | 2016-06-29 | 李应樵 | Novel vertex position lateral shifting method of converting planar game to stereo game |
| CN105809667A (en) * | 2015-01-21 | 2016-07-27 | 瞿志行 | Shading effect optimization method based on depth camera in augmented reality |
| CN105975085A (en) * | 2016-06-01 | 2016-09-28 | 云南滇中恒达科技有限公司 | Novel medium AR interactive projection system |
| CN106296573A (en) * | 2016-08-01 | 2017-01-04 | 深圳迪乐普数码科技有限公司 | A kind of method realizing virtual screen curtain wall and terminal |
| CN106296789A (en) * | 2016-08-05 | 2017-01-04 | 深圳迪乐普数码科技有限公司 | A kind of it is virtually implanted method and the terminal that object shuttles back and forth in outdoor scene |
| CN106530215A (en) * | 2016-11-02 | 2017-03-22 | 浙江中新电力发展集团有限公司 | Virtual scene coordinate converting method and apparatus |
| CN106548519A (en) * | 2016-11-04 | 2017-03-29 | 上海玄彩美科网络科技有限公司 | Augmented reality method based on ORB SLAM and the sense of reality of depth camera |
| CN106803286A (en) * | 2017-01-17 | 2017-06-06 | 湖南优象科技有限公司 | Mutual occlusion real-time processing method based on multi-view image |
| CN106856013A (en) * | 2017-01-12 | 2017-06-16 | 深圳市彬讯科技有限公司 | The method and system that a kind of augmented reality identification figure off card shows |
| CN107341827A (en) * | 2017-07-27 | 2017-11-10 | 腾讯科技（深圳）有限公司 | A kind of method for processing video frequency, device and storage medium |
| CN107562197A (en) * | 2017-08-24 | 2018-01-09 | 北京灵犀微光科技有限公司 | Display methods and device |
| CN107665508A (en) * | 2016-07-29 | 2018-02-06 | 成都理想境界科技有限公司 | Realize the method and system of augmented reality |
| CN107749083A (en) * | 2017-09-28 | 2018-03-02 | 联想(北京)有限公司 | The method and apparatus of image shows |
| CN107886532A (en) * | 2017-11-20 | 2018-04-06 | 北京小米移动软件有限公司 | The laying method and device of dummy object based on augmented reality |
| CN108182730A (en) * | 2018-01-12 | 2018-06-19 | 北京小米移动软件有限公司 | Actual situation object synthetic method and device |
| CN108431872A (en) * | 2016-12-27 | 2018-08-21 | 华为技术有限公司 | A kind of method and apparatus of shared virtual reality data |
| CN108421257A (en) * | 2018-03-29 | 2018-08-21 | 网易（杭州）网络有限公司 | Determination method, apparatus, storage medium and the electronic device of invisible element |
| CN108491081A (en) * | 2018-03-23 | 2018-09-04 | 联想(北京)有限公司 | A kind of data processing method and device based on neural network |
| CN108537149A (en) * | 2018-03-26 | 2018-09-14 | 广东欧珀移动通信有限公司 | Image processing method, device, storage medium and electronic equipment |
| CN108615261A (en) * | 2018-04-20 | 2018-10-02 | 深圳市天轨年华文化科技有限公司 | The processing method, processing unit and storage medium of image in augmented reality |
| CN108805985A (en) * | 2018-03-23 | 2018-11-13 | 福建数博讯信息科技有限公司 | Virtual Space method and apparatus |
| CN108830940A (en) * | 2018-06-19 | 2018-11-16 | 广东虚拟现实科技有限公司 | Hiding relation processing method, device, terminal device and storage medium |
| CN108830804A (en) * | 2018-05-23 | 2018-11-16 | 长春理工大学 | Virtual reality fusion Fuzzy Consistent processing method based on line spread function standard deviation |
| CN108876878A (en) * | 2017-05-08 | 2018-11-23 | 腾讯科技（深圳）有限公司 | Head portrait generation method and device |
| CN109003301A (en) * | 2018-07-06 | 2018-12-14 | 东南大学 | A kind of estimation method of human posture and rehabilitation training system based on OpenPose and Kinect |
| CN109584377A (en) * | 2018-09-04 | 2019-04-05 | 亮风台（上海）信息科技有限公司 | A kind of method and apparatus of the content of augmented reality for rendering |
| CN109685907A (en) * | 2017-10-18 | 2019-04-26 | 深圳市掌网科技股份有限公司 | Image combination method and system based on augmented reality |
| CN109712246A (en) * | 2018-12-06 | 2019-05-03 | 江苏科技大学 | A kind of augmented reality image painting methods based on generation confrontation network technology |
| CN109828671A (en) * | 2014-01-23 | 2019-05-31 | 索尼公司 | Image display device and image display method |
| CN110288707A (en) * | 2019-07-04 | 2019-09-27 | 北京伟杰东博信息科技有限公司 | A kind of method and its system of Three-Dimensional Dynamic modeling |
| CN110443898A (en) * | 2019-08-12 | 2019-11-12 | 北京枭龙科技有限公司 | A kind of AR intelligent terminal target identification system and method based on deep learning |
| CN110599432A (en) * | 2018-06-12 | 2019-12-20 | 光宝电子(广州)有限公司 | Image processing system and image processing method |
| TWI682658B (en) * | 2018-08-21 | 2020-01-11 | 國立清華大學 | Method and system of virtual footwear try-on with improved occlusion |
| US10573075B2 (en) | 2016-05-19 | 2020-02-25 | Boe Technology Group Co., Ltd. | Rendering method in AR scene, processor and AR glasses |
| CN110880161A (en) * | 2019-11-21 | 2020-03-13 | 大庆思特传媒科技有限公司 | Depth image splicing and fusing method and system for multi-host multi-depth camera |
| CN111063015A (en) * | 2019-12-13 | 2020-04-24 | 重庆首厚智能科技研究院有限公司 | Method and system for efficiently drawing point locations |
| CN111325984A (en) * | 2020-03-18 | 2020-06-23 | 北京百度网讯科技有限公司 | Sample data acquisition method and device and electronic equipment |
| CN111383343A (en) * | 2018-12-29 | 2020-07-07 | 江苏赞奇科技股份有限公司 | Home decoration-oriented augmented reality image rendering and coloring method based on generation countermeasure network technology |
| CN111408133A (en) * | 2020-03-17 | 2020-07-14 | 腾讯科技（深圳）有限公司 | Interactive property display method, device, terminal and storage medium |
| CN111897422A (en) * | 2020-07-14 | 2020-11-06 | 山东大学 | Real object interaction method and system for real-time fusion of virtual and real objects |
| CN111951352A (en) * | 2020-08-11 | 2020-11-17 | 深圳市前海手绘科技文化有限公司 | Bitmap-based freehand drawing material generation method |
| CN112044062A (en) * | 2020-08-27 | 2020-12-08 | 腾讯科技（深圳）有限公司 | Game picture rendering method, device, terminal and storage medium |
| CN112365516A (en) * | 2020-11-11 | 2021-02-12 | 华中科技大学 | Virtual and real occlusion processing method in augmented reality |
| CN112862981A (en) * | 2021-02-05 | 2021-05-28 | 北京房江湖科技有限公司 | Method and apparatus for presenting a virtual representation, computer device and storage medium |
| CN113066125A (en) * | 2021-02-27 | 2021-07-02 | 华为技术有限公司 | Augmented reality method and related equipment thereof |
| CN113240692A (en) * | 2021-06-30 | 2021-08-10 | 北京市商汤科技开发有限公司 | Image processing method, device, equipment and storage medium |
| CN113419630A (en) * | 2021-06-28 | 2021-09-21 | 西北工业大学 | Projection AR-based adaptive occlusion elimination method |
| CN113643357A (en) * | 2021-07-12 | 2021-11-12 | 杭州易现先进科技有限公司 | AR portrait photographing method and system based on 3D positioning information |
| CN113763569A (en) * | 2021-08-30 | 2021-12-07 | 之江实验室 | Image annotation method and device used in three-dimensional simulation and electronic equipment |
| CN116542847A (en) * | 2023-07-05 | 2023-08-04 | 海豚乐智科技(成都)有限责任公司 | Low-small slow target high-speed image simulation method, storage medium and device |
| CN116541923A (en) * | 2023-04-07 | 2023-08-04 | 中国民用航空飞行学院 | VR-based indoor installation foundation positioning method for equipment with support |
| CN117853694A (en) * | 2024-03-07 | 2024-04-09 | 河南百合特种光学研究院有限公司 | Virtual-real combined rendering method of continuous depth |
| US12048880B2 (en) | 2020-03-17 | 2024-07-30 | Tencent Technology (Shenzhen) Company Limited | Method and apparatus for displaying interactive item, terminal, and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2174297A1 (en) * | 2007-06-29 | 2010-04-14 | Imperial Innovations Limited | Non-photorealistic rendering of augmented reality |
| CN102129708A (en) * | 2010-12-10 | 2011-07-20 | 北京邮电大学 | Fast multilevel imagination and reality occlusion method at actuality enhancement environment |
| US20110176722A1 (en) * | 2010-01-05 | 2011-07-21 | Mikhail Sizintsev | System and method of processing stereo images |
| CN102509342A (en) * | 2011-09-22 | 2012-06-20 | 北京航空航天大学 | Collaborative virtual and actual sheltering treatment method in shared enhanced real scene |
| CN102509343A (en) * | 2011-09-30 | 2012-06-20 | 北京航空航天大学 | Binocular image and object contour-based virtual and actual sheltering treatment method |
| CN102510506A (en) * | 2011-09-30 | 2012-06-20 | 北京航空航天大学 | Virtual and real occlusion handling method based on binocular image and range information |
| CN102568026A (en) * | 2011-12-12 | 2012-07-11 | 浙江大学 | Three-dimensional enhancing realizing method for multi-viewpoint free stereo display |
| US20130215230A1 (en) * | 2012-02-22 | 2013-08-22 | Matt Miesnieks | Augmented Reality System Using a Portable Device |
-
2013
- 2013-09-10 CN CN201310409925.7A patent/CN103489214A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2174297A1 (en) * | 2007-06-29 | 2010-04-14 | Imperial Innovations Limited | Non-photorealistic rendering of augmented reality |
| US20110176722A1 (en) * | 2010-01-05 | 2011-07-21 | Mikhail Sizintsev | System and method of processing stereo images |
| CN102129708A (en) * | 2010-12-10 | 2011-07-20 | 北京邮电大学 | Fast multilevel imagination and reality occlusion method at actuality enhancement environment |
| CN102509342A (en) * | 2011-09-22 | 2012-06-20 | 北京航空航天大学 | Collaborative virtual and actual sheltering treatment method in shared enhanced real scene |
| CN102509343A (en) * | 2011-09-30 | 2012-06-20 | 北京航空航天大学 | Binocular image and object contour-based virtual and actual sheltering treatment method |
| CN102510506A (en) * | 2011-09-30 | 2012-06-20 | 北京航空航天大学 | Virtual and real occlusion handling method based on binocular image and range information |
| CN102568026A (en) * | 2011-12-12 | 2012-07-11 | 浙江大学 | Three-dimensional enhancing realizing method for multi-viewpoint free stereo display |
| US20130215230A1 (en) * | 2012-02-22 | 2013-08-22 | Matt Miesnieks | Augmented Reality System Using a Portable Device |
Non-Patent Citations (3)
| Title |
|---|
| M.-0.BERGER: "《Resolving Occlusion in Augmented Reality : a Contour Based Approach without 3D Reconstruction》", 《COMPUTER VISION AND PATTERN RECOGNITION》 * |
| 张金玲 等: "增强现实中的多层次遮挡算法", 《湖南大学学报(自然科学版)》 * |
| 田元: "增强现实中的虚实遮挡处理方法研究", 《中国博士学位论文全文数据库 信息科技辑（月刊）》 * |
Cited By (91)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109828671A (en) * | 2014-01-23 | 2019-05-31 | 索尼公司 | Image display device and image display method |
| CN109828671B (en) * | 2014-01-23 | 2022-02-22 | 索尼公司 | Image display device and image display method |
| CN103785174B (en) * | 2014-02-26 | 2016-08-17 | 北京智明星通科技有限公司 | A kind of same screen of playing shows the method and system of ten thousand people |
| CN103785174A (en) * | 2014-02-26 | 2014-05-14 | 北京智明星通科技有限公司 | Method and system for displaying tens of thousands of people on same screen of game |
| CN104504671A (en) * | 2014-12-12 | 2015-04-08 | 浙江大学 | Method for generating virtual-real fusion image for stereo display |
| CN104504671B (en) * | 2014-12-12 | 2017-04-19 | 浙江大学 | Method for generating virtual-real fusion image for stereo display |
| CN105809667A (en) * | 2015-01-21 | 2016-07-27 | 瞿志行 | Shading effect optimization method based on depth camera in augmented reality |
| CN105809667B (en) * | 2015-01-21 | 2018-09-07 | 瞿志行 | Shading effect optimization method based on depth camera in augmented reality |
| CN105005970A (en) * | 2015-06-26 | 2015-10-28 | 广东欧珀移动通信有限公司 | Augmented reality implementation method and apparatus |
| CN105005970B (en) * | 2015-06-26 | 2018-02-16 | 广东欧珀移动通信有限公司 | The implementation method and device of a kind of augmented reality |
| CN105354819A (en) * | 2015-09-29 | 2016-02-24 | 上海图漾信息科技有限公司 | Depth data measurement system, depth data determination method and apparatus |
| CN105354819B (en) * | 2015-09-29 | 2018-10-09 | 上海图漾信息科技有限公司 | Depth data measuring system, depth data determine method and apparatus |
| CN105719233B (en) * | 2016-01-21 | 2018-09-04 | 万云数码媒体有限公司 | A kind of plane game turns the vertex position transverse direction transform method of solid |
| CN105719233A (en) * | 2016-01-21 | 2016-06-29 | 李应樵 | Novel vertex position lateral shifting method of converting planar game to stereo game |
| US10573075B2 (en) | 2016-05-19 | 2020-02-25 | Boe Technology Group Co., Ltd. | Rendering method in AR scene, processor and AR glasses |
| CN105975085A (en) * | 2016-06-01 | 2016-09-28 | 云南滇中恒达科技有限公司 | Novel medium AR interactive projection system |
| CN107665508B (en) * | 2016-07-29 | 2021-06-01 | 成都理想境界科技有限公司 | Method and system for realizing augmented reality |
| CN107665508A (en) * | 2016-07-29 | 2018-02-06 | 成都理想境界科技有限公司 | Realize the method and system of augmented reality |
| CN106296573B (en) * | 2016-08-01 | 2019-08-06 | 深圳迪乐普数码科技有限公司 | A kind of method and terminal for realizing virtual screen curtain wall |
| CN106296573A (en) * | 2016-08-01 | 2017-01-04 | 深圳迪乐普数码科技有限公司 | A kind of method realizing virtual screen curtain wall and terminal |
| CN106296789B (en) * | 2016-08-05 | 2019-08-06 | 深圳迪乐普数码科技有限公司 | It is a kind of to be virtually implanted the method and terminal that object shuttles in outdoor scene |
| CN106296789A (en) * | 2016-08-05 | 2017-01-04 | 深圳迪乐普数码科技有限公司 | A kind of it is virtually implanted method and the terminal that object shuttles back and forth in outdoor scene |
| CN106530215A (en) * | 2016-11-02 | 2017-03-22 | 浙江中新电力发展集团有限公司 | Virtual scene coordinate converting method and apparatus |
| CN106548519A (en) * | 2016-11-04 | 2017-03-29 | 上海玄彩美科网络科技有限公司 | Augmented reality method based on ORB SLAM and the sense of reality of depth camera |
| CN108431872A (en) * | 2016-12-27 | 2018-08-21 | 华为技术有限公司 | A kind of method and apparatus of shared virtual reality data |
| CN106856013A (en) * | 2017-01-12 | 2017-06-16 | 深圳市彬讯科技有限公司 | The method and system that a kind of augmented reality identification figure off card shows |
| CN106803286A (en) * | 2017-01-17 | 2017-06-06 | 湖南优象科技有限公司 | Mutual occlusion real-time processing method based on multi-view image |
| CN108876878A (en) * | 2017-05-08 | 2018-11-23 | 腾讯科技（深圳）有限公司 | Head portrait generation method and device |
| CN108876878B (en) * | 2017-05-08 | 2021-11-09 | 腾讯科技（深圳）有限公司 | Head portrait generation method and device |
| CN107341827A (en) * | 2017-07-27 | 2017-11-10 | 腾讯科技（深圳）有限公司 | A kind of method for processing video frequency, device and storage medium |
| CN107562197A (en) * | 2017-08-24 | 2018-01-09 | 北京灵犀微光科技有限公司 | Display methods and device |
| CN107749083A (en) * | 2017-09-28 | 2018-03-02 | 联想(北京)有限公司 | The method and apparatus of image shows |
| CN109685907A (en) * | 2017-10-18 | 2019-04-26 | 深圳市掌网科技股份有限公司 | Image combination method and system based on augmented reality |
| CN107886532A (en) * | 2017-11-20 | 2018-04-06 | 北京小米移动软件有限公司 | The laying method and device of dummy object based on augmented reality |
| US11636653B2 (en) | 2018-01-12 | 2023-04-25 | Beijing Xiaomi Mobile Software Co., Ltd. | Method and apparatus for synthesizing virtual and real objects |
| CN108182730B (en) * | 2018-01-12 | 2022-08-12 | 北京小米移动软件有限公司 | Virtual and real object synthesis method and device |
| CN108182730A (en) * | 2018-01-12 | 2018-06-19 | 北京小米移动软件有限公司 | Actual situation object synthetic method and device |
| WO2019137006A1 (en) * | 2018-01-12 | 2019-07-18 | 北京小米移动软件有限公司 | Virtual and real object synthesis method and apparatus |
| CN108805985A (en) * | 2018-03-23 | 2018-11-13 | 福建数博讯信息科技有限公司 | Virtual Space method and apparatus |
| CN108805985B (en) * | 2018-03-23 | 2022-02-15 | 福建数博讯信息科技有限公司 | Virtual space method and device |
| CN108491081B (en) * | 2018-03-23 | 2020-09-25 | 联想(北京)有限公司 | Data processing method and device based on neural network |
| CN108491081A (en) * | 2018-03-23 | 2018-09-04 | 联想(北京)有限公司 | A kind of data processing method and device based on neural network |
| CN108537149A (en) * | 2018-03-26 | 2018-09-14 | 广东欧珀移动通信有限公司 | Image processing method, device, storage medium and electronic equipment |
| CN108421257A (en) * | 2018-03-29 | 2018-08-21 | 网易（杭州）网络有限公司 | Determination method, apparatus, storage medium and the electronic device of invisible element |
| CN108615261B (en) * | 2018-04-20 | 2022-09-09 | 深圳市天轨年华文化科技有限公司 | Method and device for processing image in augmented reality and storage medium |
| CN108615261A (en) * | 2018-04-20 | 2018-10-02 | 深圳市天轨年华文化科技有限公司 | The processing method, processing unit and storage medium of image in augmented reality |
| CN108830804A (en) * | 2018-05-23 | 2018-11-16 | 长春理工大学 | Virtual reality fusion Fuzzy Consistent processing method based on line spread function standard deviation |
| CN108830804B (en) * | 2018-05-23 | 2023-03-10 | 长春理工大学 | Virtual-real fusion fuzzy consistency processing method based on line spread function standard deviation |
| CN110599432B (en) * | 2018-06-12 | 2023-02-24 | 光宝电子(广州)有限公司 | Image processing system and image processing method |
| CN110599432A (en) * | 2018-06-12 | 2019-12-20 | 光宝电子(广州)有限公司 | Image processing system and image processing method |
| CN108830940A (en) * | 2018-06-19 | 2018-11-16 | 广东虚拟现实科技有限公司 | Hiding relation processing method, device, terminal device and storage medium |
| CN109003301B (en) * | 2018-07-06 | 2022-03-15 | 东南大学 | Human body posture estimation method based on OpenPose and Kinect and rehabilitation training system |
| CN109003301A (en) * | 2018-07-06 | 2018-12-14 | 东南大学 | A kind of estimation method of human posture and rehabilitation training system based on OpenPose and Kinect |
| TWI682658B (en) * | 2018-08-21 | 2020-01-11 | 國立清華大學 | Method and system of virtual footwear try-on with improved occlusion |
| US10943365B2 (en) | 2018-08-21 | 2021-03-09 | Kneron, Inc. | Method and system of virtual footwear try-on with improved occlusion |
| CN109584377B (en) * | 2018-09-04 | 2023-08-29 | 亮风台（上海）信息科技有限公司 | Method and device for presenting augmented reality content |
| CN109584377A (en) * | 2018-09-04 | 2019-04-05 | 亮风台（上海）信息科技有限公司 | A kind of method and apparatus of the content of augmented reality for rendering |
| CN109712246B (en) * | 2018-12-06 | 2023-07-25 | 江苏科技大学 | Augmented reality image coloring method based on generation countermeasure network technology |
| CN109712246A (en) * | 2018-12-06 | 2019-05-03 | 江苏科技大学 | A kind of augmented reality image painting methods based on generation confrontation network technology |
| CN111383343A (en) * | 2018-12-29 | 2020-07-07 | 江苏赞奇科技股份有限公司 | Home decoration-oriented augmented reality image rendering and coloring method based on generation countermeasure network technology |
| CN111383343B (en) * | 2018-12-29 | 2024-01-16 | 江苏赞奇科技股份有限公司 | Home decoration design-oriented augmented reality image rendering coloring method based on generation countermeasure network technology |
| CN110288707A (en) * | 2019-07-04 | 2019-09-27 | 北京伟杰东博信息科技有限公司 | A kind of method and its system of Three-Dimensional Dynamic modeling |
| CN110443898A (en) * | 2019-08-12 | 2019-11-12 | 北京枭龙科技有限公司 | A kind of AR intelligent terminal target identification system and method based on deep learning |
| CN110880161A (en) * | 2019-11-21 | 2020-03-13 | 大庆思特传媒科技有限公司 | Depth image splicing and fusing method and system for multi-host multi-depth camera |
| CN111063015B (en) * | 2019-12-13 | 2023-07-21 | 重庆首厚智能科技研究院有限公司 | Method and system for efficiently drawing point positions |
| CN111063015A (en) * | 2019-12-13 | 2020-04-24 | 重庆首厚智能科技研究院有限公司 | Method and system for efficiently drawing point locations |
| CN111408133A (en) * | 2020-03-17 | 2020-07-14 | 腾讯科技（深圳）有限公司 | Interactive property display method, device, terminal and storage medium |
| US12048880B2 (en) | 2020-03-17 | 2024-07-30 | Tencent Technology (Shenzhen) Company Limited | Method and apparatus for displaying interactive item, terminal, and storage medium |
| CN111325984A (en) * | 2020-03-18 | 2020-06-23 | 北京百度网讯科技有限公司 | Sample data acquisition method and device and electronic equipment |
| CN111897422A (en) * | 2020-07-14 | 2020-11-06 | 山东大学 | Real object interaction method and system for real-time fusion of virtual and real objects |
| CN111951352A (en) * | 2020-08-11 | 2020-11-17 | 深圳市前海手绘科技文化有限公司 | Bitmap-based freehand drawing material generation method |
| WO2022033138A1 (en) * | 2020-08-11 | 2022-02-17 | 深圳市前海手绘科技文化有限公司 | Bitmap-based method for generating hand-drawn materials |
| CN112044062A (en) * | 2020-08-27 | 2020-12-08 | 腾讯科技（深圳）有限公司 | Game picture rendering method, device, terminal and storage medium |
| CN112365516A (en) * | 2020-11-11 | 2021-02-12 | 华中科技大学 | Virtual and real occlusion processing method in augmented reality |
| CN112365516B (en) * | 2020-11-11 | 2022-09-27 | 华中科技大学 | Virtual and real occlusion processing method in augmented reality |
| WO2022166092A1 (en) * | 2021-02-05 | 2022-08-11 | Realsee (Beijing) Technology Co., Ltd. | Mathod and apparatus for presenting virtual representation, computer device, and storage medium |
| CN112862981B (en) * | 2021-02-05 | 2021-11-23 | 贝壳找房(北京)科技有限公司 | Method and apparatus for presenting a virtual representation, computer device and storage medium |
| CN112862981A (en) * | 2021-02-05 | 2021-05-28 | 北京房江湖科技有限公司 | Method and apparatus for presenting a virtual representation, computer device and storage medium |
| CN113066125A (en) * | 2021-02-27 | 2021-07-02 | 华为技术有限公司 | Augmented reality method and related equipment thereof |
| CN113419630B (en) * | 2021-06-28 | 2022-12-13 | 西北工业大学 | Projection AR-based adaptive occlusion elimination method |
| CN113419630A (en) * | 2021-06-28 | 2021-09-21 | 西北工业大学 | Projection AR-based adaptive occlusion elimination method |
| CN113240692B (en) * | 2021-06-30 | 2024-01-02 | 北京市商汤科技开发有限公司 | Image processing method, device, equipment and storage medium |
| CN113240692A (en) * | 2021-06-30 | 2021-08-10 | 北京市商汤科技开发有限公司 | Image processing method, device, equipment and storage medium |
| CN113643357A (en) * | 2021-07-12 | 2021-11-12 | 杭州易现先进科技有限公司 | AR portrait photographing method and system based on 3D positioning information |
| CN113763569A (en) * | 2021-08-30 | 2021-12-07 | 之江实验室 | Image annotation method and device used in three-dimensional simulation and electronic equipment |
| CN113763569B (en) * | 2021-08-30 | 2024-10-01 | 之江实验室 | Image labeling method and device used in three-dimensional simulation and electronic equipment |
| CN116541923A (en) * | 2023-04-07 | 2023-08-04 | 中国民用航空飞行学院 | VR-based indoor installation foundation positioning method for equipment with support |
| CN116541923B (en) * | 2023-04-07 | 2023-12-19 | 中国民用航空飞行学院 | VR-based indoor installation foundation positioning method for equipment with support |
| CN116542847A (en) * | 2023-07-05 | 2023-08-04 | 海豚乐智科技(成都)有限责任公司 | Low-small slow target high-speed image simulation method, storage medium and device |
| CN116542847B (en) * | 2023-07-05 | 2023-10-10 | 海豚乐智科技(成都)有限责任公司 | Low-small slow target high-speed image simulation method, storage medium and device |
| CN117853694A (en) * | 2024-03-07 | 2024-04-09 | 河南百合特种光学研究院有限公司 | Virtual-real combined rendering method of continuous depth |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103489214A (en) | Virtual reality occlusion handling method, based on virtual model pretreatment, in augmented reality system | |
| CN109003325B (en) | Three-dimensional reconstruction method, medium, device and computing equipment | |
| CN103400409B (en) | A kind of coverage 3D method for visualizing based on photographic head attitude Fast estimation | |
| CN104933718B (en) | A kind of physical coordinates localization method based on binocular vision | |
| CN103810685B (en) | A kind of super-resolution processing method of depth map | |
| CN107292965A (en) | A kind of mutual occlusion processing method based on depth image data stream | |
| CN110458939A (en) | The indoor scene modeling method generated based on visual angle | |
| CN102968809B (en) | The method of virtual information mark and drafting marking line is realized in augmented reality field | |
| CN105279789B (en) | A kind of three-dimensional rebuilding method based on image sequence | |
| CN112053447B (en) | Augmented reality three-dimensional registration method and device | |
| CN104050859A (en) | Interactive digital stereoscopic sand table system | |
| CN109920000B (en) | Multi-camera cooperation-based dead-corner-free augmented reality method | |
| CN104123751A (en) | Combined type measurement and three-dimensional reconstruction method combing Kinect and articulated arm | |
| CN104165750A (en) | Measurement method for pose of wind tunnel model combining stereoscopic vision with gyroscope | |
| CN104376596A (en) | Method for modeling and registering three-dimensional scene structures on basis of single image | |
| CN108492017B (en) | Product quality information transmission method based on augmented reality | |
| CN104835144A (en) | Solving camera intrinsic parameter by using image of center of sphere and orthogonality | |
| CN102222333B (en) | Method and device of mobile augmented reality of underground engineering based on mixed registration | |
| CN105184857A (en) | Scale factor determination method in monocular vision reconstruction based on dot structured optical ranging | |
| CN104034269A (en) | Monocular vision measuring method and monocular vision measuring device | |
| CN104794748A (en) | Three-dimensional space map construction method based on Kinect vision technology | |
| CN102768767B (en) | Online three-dimensional reconstructing and locating method for rigid body | |
| CN106558017A (en) | Spherical display image processing method and system | |
| Deng et al. | Registration of multiple rgbd cameras via local rigid transformations | |
| CN110619661A (en) | Method for measuring volume of outdoor stock ground raw material based on augmented reality |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20140101 |
|
| WD01 | Invention patent application deemed withdrawn after publication |