CN113891101A - Live broadcast method for real-time three-dimensional image display - Google Patents
Live broadcast method for real-time three-dimensional image display Download PDFInfo
- Publication number
- CN113891101A CN113891101A CN202010972436.2A CN202010972436A CN113891101A CN 113891101 A CN113891101 A CN 113891101A CN 202010972436 A CN202010972436 A CN 202010972436A CN 113891101 A CN113891101 A CN 113891101A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- image
- video
- dimensional image
- user terminal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000013473 artificial intelligence Methods 0.000 claims abstract description 20
- 238000012545 processing Methods 0.000 claims abstract description 20
- 238000004364 calculation method Methods 0.000 claims abstract description 11
- 230000003993 interaction Effects 0.000 claims description 9
- 230000000694 effects Effects 0.000 claims description 8
- 230000003321 amplification Effects 0.000 claims description 2
- 238000012937 correction Methods 0.000 claims description 2
- 238000001914 filtration Methods 0.000 claims description 2
- 210000003205 muscle Anatomy 0.000 claims description 2
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 2
- 238000003384 imaging method Methods 0.000 claims 1
- 238000004088 simulation Methods 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 11
- 230000005540 biological transmission Effects 0.000 description 10
- 238000006243 chemical reaction Methods 0.000 description 7
- 238000003702 image correction Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000007726 management method Methods 0.000 description 5
- 230000002194 synthesizing effect Effects 0.000 description 5
- 230000015572 biosynthetic process Effects 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000002457 bidirectional effect Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 239000002537 cosmetic Substances 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 238000002156 mixing Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 230000007306 turnover Effects 0.000 description 2
- 230000003796 beauty Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000002932 luster Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0641—Shopping interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T13/00—Animation
- G06T13/20—3D [Three Dimensional] animation
- G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/361—Reproducing mixed stereoscopic images; Reproducing mixed monoscopic and stereoscopic images, e.g. a stereoscopic image overlay window on a monoscopic image background
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Finance (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Accounting & Taxation (AREA)
- Physics & Mathematics (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Databases & Information Systems (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The invention discloses a live broadcast method for real-time three-dimensional image display, which comprises the following steps: taking an image of an entity and acquiring a three-dimensional image of the entity; processing the image of the three-dimensional image through a three-dimensional live broadcast calculation module; generating a video allocation combination through an artificial intelligence learning module according to at least one video selection condition of a user terminal; optimizing the three-dimensional image according to the use environment of the video allocation combination and the live broadcast system; and the optimized three-dimensional image is matched with the video allocation combination of the user terminal, and the corresponding three-dimensional image is displayed in the user terminal.
Description
Technical Field
The invention relates to a live broadcast method, in particular to a live broadcast method for displaying a real-time three-dimensional image.
Background
Since the invention of real-time communication software, through many years of development and technological progress, from the beginning of pure text two-party communication and chat, the communication and chat mode of video and characters has been entered now, and gradually becomes a part of modern people's life. After the expression modes of expressions, self-timer and filters of the real-time communication are evolved into the social trend of 'all pictures and videos', a new field of self-expression of live broadcast is emerging.
The shopping mode of the network live broadcast is characterized in that live broadcast can broadcast product introduction contents to the network through a short time gap of a background, so that a user terminal or a viewer can see the product introduction contents in real time and can interact with a main broadcast in real time. As shown in fig. 1, it shows a schematic diagram that the host 12 live broadcasts the sold goods to the user terminal 11 through the software (APP) of the mobile device on the internet, and this transaction and communication is different from the traditional tv shopping and online shopping, and the tv shopping is played in a pre-recorded and re-selected manner, so the shopping is not real-time; the traditional online store adopts a hanging network for placing product introduction pictures on the network and then completes product transaction by matching with real-time service of an online store operator.
The direct broadcast visibility and real-time interaction mode is gradually pursued by modern people, so that the time consumed by people to purchase articles outside can be greatly saved by applying the mode to an online shopping mode, the entertainment interest is full in the shopping process, and the real-time property of shopping is also improved. However, in the real-time shopping mode through the network, although a host or a professional introduces a product on the live broadcasting site, a shopper watches a 2D product image on a display screen in a remote indoor environment or in a mobile situation to determine whether to shop, because the shopper does not observe the product on the live broadcasting site, the 2D product introduction image seen by a mobile phone or an intelligent television of a user terminal generates an error with an actual article, and when the consumer receives the physical product, the shopper finds a difference with psychological expectation.
The direct broadcast chat, entertainment and shopping modes are popular and popular with people, besides the visibility of real-time and vivid pictures and the driving of the atmosphere of the anchor, an observer can interact with the anchor or leave a message, and the character interaction results of the observer and the anchor can be directly displayed on a display screen. Therefore, the range of the platform applicable to live broadcast is wider and more direct compared with the prior various real-time communication software, television shopping and plane type network shopping, and the boundary between the prior platforms is broken.
However, the current live broadcast system 10, as shown in fig. 1, still relies on conventional two-dimensional (2D) video and image display technology, and therefore, the viewer or shopping user's understanding of the anchor 12 and the live shopping still differs greatly from the real world. For example, when a live broadcasting party or a selling party is performing an explanation scene of cosmetics, as shown in fig. 2, since the image 20 broadcasted on the conventional 2D video does not have deep stereoscopic information, the selling party has painted the color and shape of the cosmetics on the live broadcasting site, and a remote viewer has difficulty in feeling bright stereoscopic color and luster on the live broadcasting site due to the limitation of image formation, which greatly reduces the display effect of the product. This is because such products do not have a three-dimensional stereoscopic light and a three-dimensional image with depth information when displayed, and cannot show the complete features of the products. Similar to such application examples, many limitations have been imposed on the current two-dimensional image display.
Therefore, how to solve the flat display limitation of the live broadcast system and improve the communication efficiency through the improvement of the design has become one of the important issues to be solved by the business.
Disclosure of Invention
The technical problem to be solved by the present invention is to provide a live broadcasting method capable of displaying three-dimensional images, which does not cause the reduction of image quality or the delay problem when users watch the images, aiming at the defects of the prior art.
In order to solve the above technical problem, one of the technical solutions adopted by the present invention provides a live broadcast method for displaying a real-time three-dimensional image, including: taking an image of an entity and acquiring a three-dimensional image of the entity; processing the image of the three-dimensional image through a three-dimensional live broadcast calculation module; generating a video allocation combination through an artificial intelligence learning module according to at least one video selection condition of a user terminal; optimizing the three-dimensional image according to the video allocation combination; and displaying the corresponding three-dimensional image in the user terminal by matching the optimized three-dimensional image with the video allocation combination of the user terminal.
One of the advantages of the present invention is that the live broadcasting method provided by the present invention adopts a method of two-dimensional and three-dimensional coexistence and mixed editing of images to reduce the real-time flow of data, so that a viewer can place a focus on a product or a person needing attention to view an optimal and most needed three-dimensional image, and the manner of coexistence of the two-dimensional and three-dimensional images can select and set two-dimensional image viewing or three-dimensional image viewing on a user interface, i.e. the user can change the position of the three-dimensional image needing to be viewed at any time, and can set the three-dimensional image viewing in a regional manner or a full-image three-dimensional display manner. The two-dimensional image can be a left image or a right image in the three-dimensional left-right type image as a display image of the two-dimensional image. In addition, besides the stereoscopic display mode of the user terminal device, the control end of the anchor end or the cloud background can also be provided with stereoscopic display equipment so as to provide stereoscopic display preview of the anchor end and interactive stereoscopic display with the user terminal. In addition, the user terminal or the main broadcasting terminal can select the horizontal or vertical mode through the horizontal and vertical control device of the three-dimensional live video to carry out the three-dimensional live broadcasting display.
For a better understanding of the features and technical content of the present invention, reference should be made to the following detailed description of the invention and accompanying drawings, which are provided for purposes of illustration and description only and are not intended to limit the invention.
Drawings
Fig. 1 is a schematic diagram of a conventional live system.
Fig. 2 is a schematic diagram of an existing live software implementation.
Fig. 3 is a flowchart illustrating steps of a live broadcast method with a real-time three-dimensional video display function according to the present invention.
Fig. 4A is a schematic diagram of image adjustment of a three-dimensional image according to the present invention.
FIG. 4B is a diagram illustrating an artificial intelligence learning module learning usage habits of a user terminal according to the present invention.
Fig. 5 is a system block diagram of a live broadcast system with real-time three-dimensional video display function according to the present invention.
Fig. 6 is a schematic diagram of image processing of the three-dimensional live broadcast operation module according to the present invention.
Fig. 7A is a schematic diagram of displaying a three-dimensional image by using the live broadcasting system of the present invention.
Fig. 7B is a schematic diagram of interaction between a main broadcast and a user terminal by applying the live broadcast system of the present invention.
Fig. 8 is a block diagram of a three-dimensional live video control device according to the present invention.
Fig. 9 is a flowchart of a determination step of two-dimensional live video and three-dimensional live video conversion of the three-dimensional live video control device according to the present invention.
[ notation ] to show
10: live broadcast system
11: user terminal
12: anchor (R)
APP: software
20: image forming method
S301-S305: step (ii) of
40: three-dimensional image
50: live broadcast system
51: image acquisition module
52: three-dimensional live broadcast calculation module
521: image correction unit
522: image separating unit
523: image synthesizing unit
524: image correction unit
525: image allocation unit
526: image management unit
527: video coding unit
53: live broadcast system server
54: artificial intelligence learning module
55: three-dimensional decoder
56: three-dimensional display
71: anchor (R)
72: mobile device
73: gift
74: display screen
75: user terminal
80: transverse and longitudinal control device for three-dimensional live broadcast image
81: direction detector
82: control interface
83: live video adjuster
S901-S905: step (ii) of
Detailed Description
The following is a description of the embodiments of the present disclosure relating to a live broadcast method for real-time three-dimensional image display, and those skilled in the art can understand the advantages and effects of the present disclosure from the disclosure of the present disclosure. The invention is capable of other and different embodiments and its several details are capable of modifications and various changes in detail, all without departing from the spirit and scope of the present invention. The drawings of the present invention are for illustrative purposes only and are not intended to be drawn to scale. The following embodiments will further explain the related art of the present invention in detail, but the disclosure is not intended to limit the scope of the present invention. In addition, the term "or" as used herein should be taken to include any one or combination of more of the associated listed items as the case may be.
[ example of the inventive direct broadcast method ]
Fig. 3 is a flowchart showing steps of a live broadcast method for real-time three-dimensional image display according to the present invention, fig. 4A is a schematic diagram showing image adjustment of a three-dimensional image according to the present invention, fig. 4B is a schematic diagram showing an artificial intelligence learning module learning usage habit of a user terminal according to the present invention, and fig. 5 is a system block diagram showing a live broadcast system with a real-time three-dimensional image display function according to the present invention.
As shown in fig. 3 and with reference to the component labels of fig. 5, a live broadcast method for real-time three-dimensional video display according to an embodiment of the present invention includes the following steps. In step S301, an entity is captured, a three-dimensional image of the entity is obtained, and an image obtaining module 51 is used to photograph the anchor, where the image obtaining module 51 may be a three-dimensional camera or a two-dimensional camera, and any electronic device capable of photographing may be the image obtaining module 51 of the present invention.
In the preferred embodiment of the present invention, the image capturing module 51 is a three-dimensional camera, and to solve the problem of the display information of the three-dimensional image, in the preferred embodiment of the present invention, the capturing and the compiling of the three-dimensional image are utilized, and the three-dimensional image capturing is to capture the three-dimensional image by using a real-time embedded three-dimensional dual-lens camera. Alternatively, in different embodiments, a two-dimensional camera may be applied to obtain a two-dimensional image of a main broadcast, and then convert the two-dimensional image into a three-dimensional image by using a two-dimensional to three-dimensional image conversion technique, and the technique for how to obtain a three-dimensional image or how to convert a two-dimensional image into a three-dimensional image is well known to those skilled in the art and will not be described herein again. In order to reduce the real-time flow of data, a method of coexisting two-dimensional images and three-dimensional images and performing mixed editing is adopted, so that a user terminal can place a focus on a product or a person needing attention when watching so as to watch the optimal and most needed three-dimensional images, and the mode of coexisting the two-dimensional images and the three-dimensional images can be used for selectively setting two-dimensional image watching or three-dimensional image watching on a user interface, namely, the user terminal can change the position of the three-dimensional images needing to be watched at any time and can be set in a regional mode or a full-image three-dimensional display mode. In addition, it should be noted that, the data amount required to be transmitted by the three-dimensional video is larger than the data amount required to be transmitted by the two-dimensional video, and the invention can transmit a smaller data amount during video playback than during full-view three-dimensional display by including the three-dimensional image and the two-dimensional image in the video at the same time, so that the smoothness of playback during live broadcast can be maintained and the occurrence of data transmission delay (lag) can be reduced.
In step S302, a three-dimensional live broadcast operation module 52 performs image processing on the three-dimensional video. After the image obtaining module 51 of the present invention receives the three-dimensional image 40, image processing is performed, because the mobile devices used by the user terminals are different, and the locations or environments of the user terminals are different, different mobile devices have different data transmission speeds, and the locations are different, and the data transmission speeds thereof are also different, so that in the step of optimizing the three-dimensional image, the three-dimensional image is cut and adjusted, and the three-dimensional image is optimized through the artificial intelligence learning module 54 according to the video allocation combination.
The live broadcast method with the real-time three-dimensional image display function can adjust images according to different data transmission speeds. In the existing stereoscopic vision technology, an entire image is processed, so that data volume at least twice as much as that of a two-dimensional image is often generated, such data transmission volume creates a considerable obstacle for a live broadcast system of real-time transmission, and the most frequently encountered situations are frame dropping (frame), mosaic or serious delay of the image, so that the use interest and patience are reduced. Therefore, in the present invention, the image is automatically detected, and the foreground, background and image segmentation are adapted, as shown in fig. 4A, the three-dimensional image 40 is further subjected to image fault tolerance and correction, and a plurality of objects in the three-dimensional image 40 are detected and separated. If the object is a person, image processing such as image softening and filtering, background blurring, filter lens effect or skin and muscle beautifying is carried out on the object; if the object is an object, performing image processing such as background stripping, edge enhancement, contrast enhancement or image amplification on the object; if the object is the same as the human object, the color, beauty and figure edge optimization calculation is carried out on the image. Background stripping is to separate the background in the three-dimensional image 40 from the image, so that different backgrounds can be replaced in the subsequent steps according to the usage habits of the user terminal. Finally, the optimized three-dimensional image 40 after image processing is output.
In addition, the live broadcasting method of the invention further comprises the step of taking images of at least one user terminal to obtain a three-dimensional image of the user terminal, so as to execute a three-dimensional display mode of multi-person interaction. The live broadcasting method of the invention can not only display three-dimensionally, but also use a one-to-many display mode of the anchor broadcast, besides a general one-to-one mode, a three-dimensional display mode of synchronous interaction of a plurality of people can be used, therefore, the live broadcasting method of the invention not only can provide intuition, but also can improve entertainment fun, and can improve pleasure of human life besides avoiding errors of shopping.
Through the image processing of the three-dimensional image, the user terminal can see the target display area at a glance, and the data loading capacity of the network bandwidth can be effectively reduced. In the present invention, the three-dimensional image can be processed by a three-dimensional live-broadcasting calculation module 52. The reason why the data load of the network bandwidth can be effectively reduced by the image processing of the three-dimensional image is that the local three-dimensional image data is much smaller than the traditional full-image three-dimensional data, and the data compression technology only has a plurality of bytes (bytes) to a plurality of kilobytes of data than the data of a general full-image two-dimensional image.
In step S303, a video allocation combination is generated by an artificial intelligence learning module 54 according to at least one video selection condition of a user terminal. The at least one video selection condition of the user terminal is the use habit of the user terminal to a live broadcast system, the use habit of each user terminal is different, and the live broadcast method displays the favorite play mode of the user terminal according to the use habit of each user terminal when the user terminal uses the live broadcast software of the invention through an artificial intelligent learning module 54. The habit of the user terminal is recorded by the learning mode of the artificial intelligence learning module 54, and the area of the habit of the user terminal can be automatically displayed or the full-image stereoscopic picture can be displayed when the same or similar picture is opened by the user terminal next time.
Further, each user terminal has a personal usage habit, so that when the user terminal watches the live broadcast, the artificial intelligence learning module 54 of the present invention records the usage habit of each user terminal when watching the live broadcast. For example, as shown in fig. 4B, when the user terminal is watching a live broadcast, the artificial intelligence learning module 54 records the usage pattern of the user terminal, for example, which gift the user terminal likes to send, and the user terminal switches between two-dimensional image and three-dimensional image for that particular person, object or person and object. The live broadcasting method of the invention records the use habit of each user terminal when watching live broadcasting, and the artificial intelligent learning module 54 displays a specific live broadcasting picture to the user terminal according to the recorded use habit when the user terminal watches the live broadcasting next time, so that the user terminal has better live broadcasting watching use experience.
In addition, in step S304, the three-dimensional video is optimized according to the video deployment combination and the usage environment of the live system. The optimization of the three-dimensional video may be based on the usage environment of the live broadcast system, such as the number of viewers and the network speed. In addition, in the live broadcasting method, when people or products are shot, the people or products can be shot in real time, the people or products can also be added in real time through the pre-recorded background, the use scenes of virtual reality are increased, and the user terminal can conveniently change the use scenes to improve the pleasure of environment use. In the live broadcasting method, different video allocation combinations are provided for each user terminal to generate an optimized three-dimensional image, and different background patterns can be selected according to different user terminals. In addition, in order to avoid the wrong or inappropriate video or video stream from being played to the terminal, no matter whether the user terminal chats or communicates with the main broadcast, or enjoys the program performance of the main broadcast, or watches the products sold by the main broadcast, after the video is taken into the human body or the object in real time by the camera, except for real-time three-dimensional editing and compression of the video, the video is automatically checked through artificial intelligence or manual check, after the check is finished, the video is correctly corresponding to the background system according to the setting or the requirement of the user terminal, and then the corresponding code stream (Data Streaming) is sent to the mobile phone or the terminal interaction device of the user terminal, and the user can see the display result corresponding to the setting and the requirement of the terminal.
In step S305, the optimized three-dimensional image is displayed in the user terminal in accordance with at least one video selection condition of the user terminal. The live broadcasting method of the invention not only provides the cutting and display setting of the foreground and the background required by the user terminal at the background end according to the setting of the user terminal, but also sends the corresponding three-dimensional image to the device of the user terminal according to the setting and the requirement of the user terminal. For example, when the user terminal purchases a gift, the live broadcast system sends a stereoscopic display image or video of the gift to the device of the anchor according to the corresponding gift purchased by the user terminal, so that the anchor can view the video or image of the corresponding stereoscopic gift on the display screen of the device of the anchor.
According to the live broadcast method with the real-time three-dimensional image display function, the output quality of the image can be adjusted according to different data transmission speeds, different experience feelings can be provided according to different user terminals, and each user terminal can obtain better watching experience or shopping experience.
[ embodiment of live broadcast System of the present invention ]
Fig. 5 is a system block diagram of a live broadcast system for real-time three-dimensional video display according to the present invention. As shown in fig. 5, the live broadcasting system 50 for real-time three-dimensional image display according to the present invention includes an image acquisition module 51, a three-dimensional live broadcasting calculation module 52, a live broadcasting system server 53, an artificial intelligence learning module 54, a three-dimensional decoder 55 and a three-dimensional display 56.
The image capturing module 51 may be a three-dimensional camera, a two-dimensional camera, or the like, and any electronic device capable of taking images may be used as the image capturing module 51 of the present invention. In the preferred embodiment of the present invention, the image capturing module 51 is a three-dimensional camera, which is to solve the problem of insufficient display information of three-dimensional images, in the preferred embodiment of the present invention, the three-dimensional image is captured and compiled, and the three-dimensional image is captured by a real-time embedded three-dimensional dual-lens camera, such a camera, embedded in a device used by the anchor or in a device of the user terminal, such as a smartphone, a tablet computer, etc., is, due to its embedded type, therefore, the cameras are built in the electronic device, and the two three-dimensional cameras are connected with the electronic device through interfaces, such as Mobile Industry Processor Interface (MIPI), serial Interface, and the like, and is directly connected with a handheld device end, and then is connected with a live broadcast system server 53 through a three-dimensional live broadcast calculation module 52 of the invention, so as to send three-dimensional images and videos to a user terminal in real time. For a personal computer end or a handheld device, if an embedded three-dimensional camera is not installed, an external monocular camera or a binocular stereo camera can be connected with a host through a Universal Serial Bus (USB) interface or a Wireless (WIFI) mode, and after the direct broadcasting system is connected, data processing is carried out according to a processing mode or flow of converting two-dimensional to three-dimensional or stereo cameras.
Alternatively, in different embodiments, a two-dimensional camera may be applied to obtain a two-dimensional image of a main broadcast, and then convert the two-dimensional image into a three-dimensional image by using a two-dimensional to three-dimensional image conversion technique, and the technique for how to obtain a three-dimensional image or how to convert a two-dimensional image into a three-dimensional image is well known to those skilled in the art and will not be described herein again. The image obtaining module 51 may be installed on the devices of the anchor and the user terminal, and may obtain the three-dimensional images of the anchor and the user terminal at the same time, so as to facilitate the subsequent one-to-one mode or one-to-many stereoscopic display mode of the anchor and the at least one user terminal.
The three-dimensional live broadcast operation module 52 is connected to the image acquisition module 51, and is configured to receive the three-dimensional image and optimize the three-dimensional image, where the three-dimensional live broadcast operation module 52 may be disposed in live broadcast software of a main broadcast or a user terminal, or the three-dimensional live broadcast operation module 52 may also be installed in the live broadcast system server 53, which is not limited herein. Further, as shown in fig. 6 and referring to fig. 5, the three-dimensional live broadcasting operation module 52 includes an image correction unit 521, an image separation unit 522, an image synthesis unit 523, an image modification unit 524, an image allocation unit 525, an image management unit 526 and a video coding unit 527. The image correction unit 521 is connected to the image acquisition module 51, receives the three-dimensional image, and is configured to correct the received three-dimensional image, so that a small number of image errors or video and audio asynchronism may occur during the process of two-dimensional to three-dimensional conversion of the acquired three-dimensional image or the image, and errors generated during the acquisition or conversion of the three-dimensional image are corrected by the image correction unit 521. The image separating unit 522 is connected to the image correcting unit 521, and divides the foreground, background, person or object in the corrected three-dimensional image, so as to perform specific image processing on any person or object in the three-dimensional image, for example, the background in the three-dimensional image is cut, so that a specific background pattern can be added to the background in the subsequent image processing, or the specific object or image is three-dimensionally displayed in a specific live broadcast operation.
The image synthesizing unit 523 is connected to the image separating unit 522, the separated three-dimensional image may be subjected to image processing on a specific pattern, a person or an object, respectively, the processed specific pattern, person or object may be combined by the image synthesizing unit 523, the image synthesizing unit 523 may combine an individual three-dimensional pattern, person or object with another two-dimensional image, instead of reducing the original three-dimensional image, or may add a specific background pattern to the background by the image synthesizing unit 523. The image modifying unit 524 is connected to the image allocating unit 525, and the image modifying unit 524 may select and adjust the viewing angle of the image according to the image capture of different patterns, or the image modifying unit 524 may perform an image special effect rendering effect on the gift image in the three-dimensional image, and the image modifying unit 524 may also include functions such as general image edge enhancement, image turning, image background blurring, filter effect, or image area enlargement.
The image allocating unit 525 is connected to the image modifying unit 524, and the image allocating unit 525 adjusts the image output of the three-dimensional image according to different network speeds or the performance of the mobile device. For example, when the network speed is high, the three-dimensional image can be completely output, when the network speed is low, the three-dimensional image can be partially displayed, and the rest of the images are displayed in two dimensions. The video management unit 526 is connected to the video allocation unit 525, integrates and outputs the three-dimensional video after image processing, and the video encoding unit 527 is connected to the video management unit 526, converts the three-dimensional video into a data signal through the video encoding unit 527, and further can transmit the data signal with the three-dimensional video to the live broadcast system server 53 in a wired or wireless manner.
The live broadcast system server 53 is wirelessly connected to a three-dimensional live broadcast calculation module 52, which may also be referred to as a live broadcast system cloud, and the live broadcast system server 53 is configured to receive a data signal with a three-dimensional image, in other words, live broadcast content of each anchor broadcast is transmitted to the live broadcast system server 53 and then is pushed to a device of the user terminal through the live broadcast system server 53. The artificial intelligence learning module 54 is connected to the live broadcast system server 53, in a preferred embodiment of the present invention, the artificial intelligence learning module 54 may be disposed on the live broadcast system server 53, or in a different embodiment, the artificial intelligence learning module 54 may be disposed on a different server or a different computer host, and then connected to the live broadcast system server 53 in a wireless or wired connection manner, which is not limited herein. The screen selection condition or usage habit of each user terminal is recorded by the learning mode of the artificial intelligence learning module 54, and a video deployment combination is output. Next time the user terminal turns on the same or similar picture, the artificial intelligence learning module 54 can automatically output the video allocation combination according to the video selection condition, in other words, display the display area to which the user terminal is accustomed or display the stereoscopic picture of the full picture.
Therefore, no matter the user terminal chats or communicates with the anchor, or enjoys the program performance of the anchor, or watches the products sold by the anchor, after the images are taken into human body or object by the camera in real time, except for real-time three-dimensional editing and compression of the images, the images are automatically checked by the artificial intelligent learning module 54 in the background of the live broadcasting system server 53, and after the check is completed, the corresponding code stream (Data) is sent to the mobile phone or the terminal interaction device of the user terminal according to the setting or the requirement of the user terminal and after the correct correspondence of the background system, the user terminal can see the display result set by the terminal and corresponding to the requirement.
The three-dimensional decoder 55 is connected to the live broadcast system server 53, or the three-dimensional decoder 55 may be installed on the live broadcast system server 53 and connected to the artificial intelligence learning module 54, and the three-dimensional decoder 55 can encode and decode a three-dimensional image, and in the live broadcast system 50 of the present invention, the three-dimensional decoder 55 can be applied to allow a main broadcast to perform an interactive stereoscopic display mode with a plurality of user terminals. The live broadcasting system 50 displays the stereoscopic images or videos of the gifts 73 on the mobile device 72 of the anchor 71 according to the corresponding gifts purchased by the user terminals, as shown in fig. 7A, so that the anchor 71 can view the videos or images of the corresponding stereoscopic gifts 73 on the display screen 74 of the mobile device 72. The three-dimensional display 56 is provided on the devices of the anchor and the user terminal, and the anchor and the user terminal receive the three-dimensional video pushed from the live system server 53 in a wireless transmission manner and display the three-dimensional video on the devices of the anchor and the user terminal through the three-dimensional display 56. The live broadcast system 50 of the present invention has a three-dimensional bidirectional encoding and decoding technology through the three-dimensional decoder 55, as shown in fig. 7B, on the mobile device 72, in addition to the anchor 71 can send a stereoscopic video and an image to the user terminal 75 to receive and watch the stereoscopic video and the image, the user terminal 75 can also send a stereoscopic image or a video to the anchor 71 through the bidirectional stereoscopic display technology, so that the anchor 71 can also receive the stereoscopic image information of the user terminal 75 in real time, the requirement of the user terminal 75 can be rapidly judged, and the purpose of real-time interaction is achieved.
In addition, referring to fig. 6, the user terminal also includes a function of displaying a three-dimensional video at the anchor terminal, and the device of the user terminal also includes a function of the live broadcasting system 50 of the present invention, because the user terminal can also live a video of itself to the anchor terminal for viewing when viewing the live broadcast, or the user terminal can also automatically or manually switch between two-dimensional and three-dimensional when receiving the video at the anchor terminal. Therefore, when the user terminal receives the two-dimensional or three-dimensional image, if the received two-dimensional image can be converted into a three-dimensional image, the three-dimensional image of the user terminal is also subjected to image processing by the components such as the image correction unit 521, the image separation unit 522, the image synthesis unit 523, the image modification unit 524, the image allocation unit 525, the image management unit 526 and the like, so that the three-dimensional image of the user terminal can be displayed as a panoramic three-dimensional image, or can be displayed as a three-dimensional image for a specific person, object or person and object, and the display of the two-dimensional or three-dimensional image on the client terminal can be actively or manually switched. How to process the image through the three-dimensional live broadcast operation module 52 is described in the previous paragraphs, and therefore, details of the image processing will not be described herein.
For example, when the data transmission speed of the user terminal is not fast, the three-dimensional live broadcast calculation module 52 may automatically display only a portion of the three-dimensional image on the three-dimensional display 56 of the user terminal, but not display the panoramic three-dimensional image, or, when the network speed is too low, the three-dimensional live broadcast calculation module 52 may automatically switch the three-dimensional image to the two-dimensional image. In addition, the three-dimensional images of the user terminal can be switched automatically or manually, and the user terminal can manually designate a specific person, object or person and object to display the three-dimensional images. In other words, the three-dimensional image of the present invention is not limited to the full-image three-dimensional image, and the three-dimensional image of the present invention may be a partial three-dimensional image in which a specific object (person, object, or person and object) is displayed in a three-dimensional manner and the remaining images are displayed in a two-dimensional manner. By the image presentation mode, the transmission quantity of data during live broadcasting can be reduced, and the probability of generation of live broadcasting display delay can be reduced.
[ horizontal or vertical display embodiment of three-dimensional live video of the invention ]
In addition, in the preferred embodiment of the present invention, a horizontal and vertical control device 80 for three-dimensional live video can be further provided on the devices of the anchor and the user terminal. Because the device used by the user terminal may have the habit of using vertical display to view or horizontal live display to view, the invention also provides the horizontal and vertical control device 80 of the three-dimensional live video, and for the display direction setting of the mobile device, the setting of the display direction can be automatically detected or manually set on the APP software of the user terminal, and is also suitable for the monitoring display of the display terminal and the cloud terminal of the anchor program. Therefore, the apparatus 80 for controlling the horizontal and vertical directions of a three-dimensional live video of the present invention comprises a direction detector 81 and a control interface 82.
The direction detector 81 is capable of sensing a signal of a gyroscope or a direction sensor inside the mobile device or the handheld device in the direction of automatically detecting the live display, and the direction of the live display is adjusted by the device of the anchor or the user terminal according to the direction signal fed back by the terminal device. The adjustment of the direction can be matched with the steering of the three-dimensional live video or video to perform display conversion of the three-dimensional live video, and the display conversion comprises adjustment of length, width ratio or interlaced ratio (Interlace) of screen display, adjustment of software keys and functional display positions, and the like of steering setting related to 3D live video. The aforementioned settings also include live display function settings for the three-dimensional display 56. The control interface 82 is provided on the mobile device or handheld device of the anchor and the user terminal, and further, the control interface 82 may be a controller such as a button, etc., or the control interface 82 may also be a software interface including functions of the controller. The anchor or the user terminal can automatically turn over the screen for live display through the direction detector 81, or can automatically turn over the screen for live display through the control interface 82, so that the vertical live display viewing or the horizontal live display viewing is realized.
The horizontal and vertical control device 80 for three-dimensional live video of the present invention has a function of displaying three-dimensional live video in both horizontal and vertical directions, and has automatic or manual steering detection and display adjustment. For a stereoscopic display terminal only supporting one direction, the invention further comprises a live video adjuster 83 in the horizontal and vertical control device 80 of the three-dimensional live video, the live video adjuster 83 is connected with the direction detector 81, the step of the live video adjuster 83 judging whether to adjust the three-dimensional live video can be as shown in fig. 9, in step S901, the direction detector 81 detects the change of the setting direction of the mobile device, or, in step S902, the user rotates the live video through the control interface 82, in step S903, the live video adjuster 83 judges whether the mobile device supports three-dimensional live video display horizontally or vertically, if not, in step S904, the live video adjuster 83 can remind the user terminal to inform that the turned stereoscopic display is not supported, and the live video adjuster 83 changes into a two-dimensional plane display mode to display on the display of the user terminal, and the user terminal can see the stereoscopic live broadcast display until the user terminal turns to display the stereoscopic live broadcast display again. On the contrary, in step S905, the live video adjuster 83 adjusts the optimized three-dimensional live video after the turning.
[ advantageous effects of the embodiments ]
The live broadcasting system and the live broadcasting method provided by the invention have the beneficial effects that in order to reduce the real-time flow of data, a method of mixing and editing two-dimensional images and three-dimensional images is adopted, so that a viewer can place a focus on a product or a person needing attention to view the optimal and most-needed three-dimensional image, and the mode of mixing the two-dimensional images and the three-dimensional images can select and set two-dimensional image viewing or three-dimensional image viewing on a user interface, namely, the user can change the position of the three-dimensional image needing to be viewed at any time, and the three-dimensional image can be set in a regional mode or a full-image three-dimensional display mode. The two-dimensional image can be a left image or a right image in the three-dimensional left-right type image as a display image of the two-dimensional image. In addition, besides the stereoscopic display mode of the user terminal device, the control end of the anchor end or the cloud background can also be provided with stereoscopic display equipment so as to provide stereoscopic display preview of the anchor end and interactive stereoscopic display with the user terminal. In addition, the user terminal or the main broadcasting terminal can easily select the horizontal or vertical mode to carry out the three-dimensional live broadcasting display through the horizontal and vertical control device of the three-dimensional live broadcasting image.
The disclosure is only a preferred embodiment of the invention and should not be taken as limiting the scope of the invention, which is defined by the appended claims.
Claims (10)
1. A live broadcast method for real-time three-dimensional image display comprises the following steps:
taking an image of an entity and acquiring a three-dimensional image of the entity;
processing the image of the three-dimensional image through a three-dimensional live broadcast calculation module;
generating a video allocation combination through an artificial intelligence learning module according to at least one video selection condition of a user terminal;
optimizing the three-dimensional image according to the video allocation combination and a use environment of the user terminal; and
the optimized three-dimensional image is matched with at least one video selection condition of the user terminal, and the corresponding three-dimensional image is displayed in the user terminal;
the three-dimensional image is a full-image three-dimensional image or a local three-dimensional image.
2. The live broadcasting method of real-time three-dimensional image display according to claim 1, wherein in the step of capturing an image of an entity, a two-dimensional image is obtained through at least one lens, and then the two-dimensional image is converted into the three-dimensional image through a three-dimensional image simulation module.
3. The live broadcasting method of real-time three-dimensional video display of claim 1, wherein in the step of imaging an entity, the three-dimensional video is directly obtained through a plurality of lenses.
4. The live broadcasting method of real-time three-dimensional video display according to claim 1, wherein in the step of performing image processing of the three-dimensional video, the three-dimensional video is subjected to video tolerance and correction, and a plurality of objects in the three-dimensional video are detected and separated.
5. The live broadcasting method of real-time three-dimensional image display according to claim 4, wherein if the object is a human, the object is subjected to image softening and filtering, background blurring, filter effect or skin and muscle beautifying.
6. The live broadcasting method of real-time three-dimensional video display according to claim 4, wherein if the object is an object, the object is subjected to background peeling, edge enhancement, contrast enhancement or video amplification.
7. A live method of displaying three-dimensional images in real time as claimed in claim 1, wherein the at least one video selection condition of the user terminal is a usage habit of the user terminal to a live system.
8. The live broadcasting method of real-time three-dimensional video display according to claim 1, wherein in the step of optimizing the three-dimensional video, the three-dimensional video is optimized according to the number of viewers and the network speed of the live broadcasting system.
9. A live method of real-time three-dimensional video presentation as claimed in claim 1, further comprising: and taking images of at least one user terminal to acquire the three-dimensional image of the user terminal, so as to execute a three-dimensional display mode of multi-person interaction.
10. The live broadcasting method for displaying real-time three-dimensional images according to claim 1, wherein in the step of optimizing the three-dimensional images, the three-dimensional images are adjusted by cutting, and are optimized by the artificial intelligence learning module according to the video deployment combination.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010972436.2A CN113891101A (en) | 2020-09-16 | 2020-09-16 | Live broadcast method for real-time three-dimensional image display |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010972436.2A CN113891101A (en) | 2020-09-16 | 2020-09-16 | Live broadcast method for real-time three-dimensional image display |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113891101A true CN113891101A (en) | 2022-01-04 |
Family
ID=79012102
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010972436.2A Pending CN113891101A (en) | 2020-09-16 | 2020-09-16 | Live broadcast method for real-time three-dimensional image display |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113891101A (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20070009899A (en) * | 2005-07-14 | 2007-01-19 | 학교법인연세대학교 | Apparatus and method for 3-dimensional modeling using multiple stereo cameras |
| TW201125355A (en) * | 2010-01-08 | 2011-07-16 | Chunghwa Picture Tubes Ltd | Method and system for displaying 2D and 3D images simultaneously |
| US20120069151A1 (en) * | 2010-09-21 | 2012-03-22 | Chih-Hsiang Tsai | Method for intensifying identification of three-dimensional objects |
| CN106060520A (en) * | 2016-04-15 | 2016-10-26 | 深圳超多维光电子有限公司 | Display mode switching method, display mode switching device, and intelligent terminal |
| US20170200315A1 (en) * | 2016-01-07 | 2017-07-13 | Brendan Lockhart | Live stereoscopic panoramic virtual reality streaming system |
| WO2018009427A1 (en) * | 2016-07-07 | 2018-01-11 | Google Llc | 2d video with option for projected viewing in modeled 3d space |
| KR20180076495A (en) * | 2016-12-28 | 2018-07-06 | 광운대학교 산학협력단 | A method of separating objects and estimating their three dimension position, using stereo vision |
| CN109889855A (en) * | 2019-01-31 | 2019-06-14 | 南京理工大学 | Intelligent panoramic net cast networked shopping system and method based on mobile APP |
| CN110278445A (en) * | 2019-07-05 | 2019-09-24 | 浙江开奇科技有限公司 | Image treatment method, mobile terminal and system based on 5G network |
| CN111316650A (en) * | 2017-10-27 | 2020-06-19 | 松下电器（美国）知识产权公司 | Three-dimensional model encoding device, three-dimensional model decoding device, three-dimensional model encoding method, and three-dimensional model decoding method |
| CN111541907A (en) * | 2020-04-23 | 2020-08-14 | 腾讯科技（深圳）有限公司 | Article display method, apparatus, device and storage medium |
-
2020
- 2020-09-16 CN CN202010972436.2A patent/CN113891101A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20070009899A (en) * | 2005-07-14 | 2007-01-19 | 학교법인연세대학교 | Apparatus and method for 3-dimensional modeling using multiple stereo cameras |
| TW201125355A (en) * | 2010-01-08 | 2011-07-16 | Chunghwa Picture Tubes Ltd | Method and system for displaying 2D and 3D images simultaneously |
| US20120069151A1 (en) * | 2010-09-21 | 2012-03-22 | Chih-Hsiang Tsai | Method for intensifying identification of three-dimensional objects |
| US20170200315A1 (en) * | 2016-01-07 | 2017-07-13 | Brendan Lockhart | Live stereoscopic panoramic virtual reality streaming system |
| CN106060520A (en) * | 2016-04-15 | 2016-10-26 | 深圳超多维光电子有限公司 | Display mode switching method, display mode switching device, and intelligent terminal |
| WO2018009427A1 (en) * | 2016-07-07 | 2018-01-11 | Google Llc | 2d video with option for projected viewing in modeled 3d space |
| KR20180076495A (en) * | 2016-12-28 | 2018-07-06 | 광운대학교 산학협력단 | A method of separating objects and estimating their three dimension position, using stereo vision |
| CN111316650A (en) * | 2017-10-27 | 2020-06-19 | 松下电器（美国）知识产权公司 | Three-dimensional model encoding device, three-dimensional model decoding device, three-dimensional model encoding method, and three-dimensional model decoding method |
| CN109889855A (en) * | 2019-01-31 | 2019-06-14 | 南京理工大学 | Intelligent panoramic net cast networked shopping system and method based on mobile APP |
| CN110278445A (en) * | 2019-07-05 | 2019-09-24 | 浙江开奇科技有限公司 | Image treatment method, mobile terminal and system based on 5G network |
| CN111541907A (en) * | 2020-04-23 | 2020-08-14 | 腾讯科技（深圳）有限公司 | Article display method, apparatus, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106789991B (en) | Multi-person interactive network live broadcast method and system based on virtual scene | |
| CN106792246B (en) | Method and system for interaction of fusion type virtual scene | |
| CN113347405B (en) | Scaling related method and apparatus | |
| CN106303289B (en) | Method, device and system for fusion display of real object and virtual scene | |
| CN106165415B (en) | Stereoscopic viewing | |
| CN105264876B (en) | The method and system of inexpensive television production | |
| CN102246529B (en) | Image based 3D video format | |
| CN106730815B (en) | Somatosensory interaction method and system easy to realize | |
| US20160041662A1 (en) | Method for changing play mode, method for changing display mode, and display apparatus and 3d image providing system using the same | |
| CN106101741A (en) | Internet video live broadcasting platform is watched the method and system of panoramic video | |
| CN104335243B (en) | A kind of method and device for handling panorama | |
| CN109218630B (en) | Multimedia information processing method and device, terminal and storage medium | |
| CN109872400B (en) | Panoramic virtual reality scene generation method | |
| JP6934052B2 (en) | Display control device, display control method and program | |
| CN111147883A (en) | Live broadcast method and device, head-mounted display equipment and readable storage medium | |
| US12112426B2 (en) | Augmenting a view of a real-world environment with a view of a volumetric video object | |
| TWI774063B (en) | Horizontal/vertical direction control device for three-dimensional broadcasting image | |
| CN116016977A (en) | Live broadcast-based virtual same-platform wheat connection interaction method, computer equipment and medium | |
| TWI855158B (en) | Live broadcasting system for real time three-dimensional image display | |
| CN113891101A (en) | Live broadcast method for real-time three-dimensional image display | |
| CN116962745A (en) | Mixed drawing method, device and live broadcast system of video image | |
| TWI836141B (en) | Live broadcasting method for real time three-dimensional image display | |
| CN113891100A (en) | Live broadcast system for real-time three-dimensional image display | |
| CN113891099A (en) | Transverse and longitudinal control device for three-dimensional live broadcast image | |
| TW202213990A (en) | Live broadcasting system for real time three-dimensional image display |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20240724 Address after: 518000, 1101, Tianliao Building, Tangchang Road, Tanglang Community, Taoyuan Street, Nanshan District, Shenzhen City, Guangdong Province Applicant after: Shenzhen Vanguard Technology Co.,Ltd. Country or region after: China Address before: 518000 room 311, North block, wandelai building, Gaoxin South 1st Road, Yuehai street, Nanshan District, Shenzhen City, Guangdong Province Applicant before: Shenzhen Bohao Photoelectric Technology Co.,Ltd. Country or region before: China |