KR102566072B1 - Portrait gradual positioning type remote meeting method - Google Patents

Abstract

The present invention relates to a video conference performed by a plurality of computers 10 connected through the Internet, and a virtual screen 32 synthesized with a actually photographed person and a pre-stored virtual background is output to the computer 10 screen. , The position of the person on the virtual screen 32 can be appropriately adjusted despite the actual movement of the person.
Through the present invention, it is possible to implement a stable screen in the course of a video conference in which the virtual screen 32 is implemented, thereby improving the effectiveness and efficiency of the video conference and securing a high degree of immersion in performing the video conference.

Description

Person positioning slow adjustment video conference method {PORTRAIT GRADUAL POSITIONING TYPE REMOTE MEETING METHOD}

The present invention relates to a video conference performed by a plurality of computers 10 connected through the Internet, and a virtual screen 32 synthesized with a actually photographed person and a pre-stored virtual background is output to the computer 10 screen. , The position of the person on the virtual screen 32 can be appropriately adjusted despite the actual movement of the person.

Due to the improvement in the speed of Internet communication and the performance of personal computers, multilateral video conferences through the Internet and personal computers have become possible without the construction of dedicated communication lines and expensive video equipment and signal processing systems. User convenience has been dramatically improved as a video conference can be conducted in a way that a browser installed in a computer participating in a conference accesses a web page established for the purpose of video conference.

In addition, as the spread of wireless networks such as high-speed mobile communication networks and wireless LANs expands and the use of portable information devices such as smartphones and tablet PCs capable of accessing wireless networks and processing high-speed and large-capacity information becomes common, computer and Internet-based image The temporal and spatial constraints on conducting meetings are being dramatically relaxed.

That is, breaking away from the conventional video conference that was conducted in a fixed space such as an office or conference room based on a desktop computer, various information devices capable of accessing a wireless network such as a smartphone or tablet PC as shown in FIG. 1 are utilized for video conferences. Accordingly, video conferences can be performed in various environments such as homes, public places, or outdoors.

As such, as the temporal and spatial constraints on video conferencing are alleviated and video conferencing is conducted in various places, the use of video conferencing is expanding not only in the industrial field but also in everyday life. In particular, the recent non-face-to-face trend and telecommuting Due to the spread of video conferencing services, interest and demand for video conferencing services are explosively increasing.

As described above, as the spatial restrictions in video conferencing have been alleviated and video conferencing is being conducted in various spaces, problems such as exposure of privacy due to unfiltered transmission of actual shooting images have emerged, and accordingly, actual shooting Among the imaging information constituting the real screen 31, the image information of the actual background except for the person information 50 is erased, or the person information 50 from which the actual background has been erased is combined with the pre-stored virtual background information 40. Thus, a technology for implementing the virtual screen 32 is being utilized.

That is, in configuring the screen of the computer 10 during a video conference, only the person part is extracted, and a pre-stored background, such as a beautiful scenery, is synthesized and output, not the actual photographed background. Not only can meeting participants' privacy be protected because they cannot be identified, but also user satisfaction can be improved by enabling users to select various backgrounds according to their preferences.

In particular, in image processing for extracting a person from captured information, as various techniques such as machine learning are applied by breaking away from traditional outline recognition techniques such as histogram smoothing, the accuracy and speed of extraction of person information 50 in the captured information is drastically improved. It is being developed, and accordingly, the realization of a more natural and beautiful virtual image is being realized.

However, the conventional background synthesis type videoconferencing service simply overlaps the image information from which the actual shooting background has been deleted and the pre-stored virtual background, and the actual movement of the person is reflected in the virtual image without addition or subtraction. As a result, various A problem arises.

First of all, since the actual motion of the person is reflected in the virtual image as it is, a distracting screen is inevitable. Compared to the motion of the person in the actual captured image, the motion of the person implemented in the background of the virtual image causes distortion of perspective and It causes serious visual confusion by causing various optical illusion effects.

Particularly, due to the nature of videoconferencing, conference participants are inevitably exposed to such distorted images for a long time, and visual fatigue inevitably accumulates and, in severe cases, psychological instability may be induced.

In addition, due to the distracting configuration of the screen, it is inevitable to interrupt the meeting immersion of the participants in the video conference, and as a result, the effect of communication and exchange of opinions between participants through the video conference cannot but be seriously damaged.

The present invention was devised in view of the above-described problems, and in a video conference method performed by a plurality of computers 10 connected through the Internet, a conference program loaded in the computer 10 captures an image of the actual screen 31 An execution step (S10) of extracting the person information 50 from the information and fetching the background information 40 from the background database 90, and the meeting program specifying the location of all images constituting the person information 50 In the setting step (S21) of setting the reference point 55, the background information 40 and the person information 50 so that the reference point 55 coincides with the target point 45 preset in the background information 40 of the conference program. It is a person position adjustment type video conference method characterized in that it consists of an overlapping step (S22) of generating a virtual screen 32 by overlapping.

In addition, in the regular overlapping step (S22) of the present invention, the background information 40 and the person information 50 are overlapped so that the reference point 55 coincides with the target point 45 preset in the background information 40 of the conference program. to create a virtual screen 32, but set a plurality of waypoints 65 on a linear path connecting the reference point 55 and the target point 45 before overlapping, and the reference point 55 is the waypoint 65 ) It is a person position adjustment type video conference method characterized by matching with the target point 45 via.

In addition, in the present invention, after the overlapping step (S22) is performed, as the object corresponding to the person information 50 on the virtual screen 32 is manipulated, the position of the target point 45 on the background information 40 is changed. It is a person position adjustment type video conference method characterized in that.

Through the present invention, it is possible to implement a stable screen in the course of a video conference in which the virtual screen 32 is implemented, thereby improving the effectiveness and efficiency of the video conference and securing a high degree of immersion in performing the video conference.

In particular, as the position of the person on the output screen is stably maintained, visual fatigue of the videoconference participant can be reduced and psychological stability can be promoted.

1 is an exemplary view of a computer connection state of the present invention
Figure 2 is an example of a computer screen of the present invention on which an actual screen is output
3 is an exemplary view of a computer screen of the present invention on which a virtual screen is output;
Figure 4 is an example of a smartphone screen that is a computer of the present invention on which a virtual screen is output
5 is a comparison diagram of a real screen and a virtual screen of the present invention
6 is an explanatory diagram of screen and image information step-by-step in the prior art;
7 is a screen explanatory diagram when moving a person in the prior art
8 is a screen explanatory diagram when moving a person of the present invention
9 is a step-by-step screen and image information explanatory diagram of the present invention
10 is a screen explanatory diagram for each viewpoint when a person moves according to the present invention
11 is a screen explanatory diagram when moving a person in a slow-moving embodiment of the present invention
Fig. 12 is an explanatory diagram of waypoints in the embodiment of Fig. 11;
13 is a screen explanatory diagram when moving a person in a target point adjustable embodiment of the present invention

The detailed configuration and execution process of the present invention will be described through the accompanying drawings.

First of all, FIG. 1 shows the components for performing the video conference of the present invention based on the Internet and the connection state between the components. As shown, the video conference of the present invention basically has video recording and audio input/output functions. It is performed by a plurality of computers 10 mounted thereon, and these plurality of computers 10 are interconnected through the Internet.

In addition, as shown in phantom lines in the same figure, a web page 21 or a server 20 may be configured and the computer 10 participating in a video conference may be connected via these web pages 21 or the server 20. The computer 10 or the server 20 is connected to a background database 90 containing background information 40 to be described later.

Here, the background database 90 is a database in which virtual background information 40 used in implementing the virtual screen 32 to be described later is recorded, and the connection between the computer 10 and the background database 90 or the server 20 The connection of the background database 90 with the background database 90 in the storage device of the computer 10 or server 20 is built, or the background database 90 is built in an external storage device or remote storage to establish a communication path It covers the state of being connected to the computer 10 or the server 20 through

In addition, in the present invention, the computer 10 performing the Internet-based video conference includes not only a conventional desktop computer 10 or laptop computer 10 equipped with a camera and capable of voice input and output, but also an image pickup as illustrated in FIG. Various information devices such as tablet PCs or smart phones equipped with functions and capable of accessing the Internet can be applied.

As in other Internet-based videoconferencing methods, the main transmission and reception information between the computers 10 participating in the videoconferencing in the present invention is also video information and audio information, and these information is provided through a communication path established between the computers 10. The video conference proceeds as a conference program that performs input and output and distribution of video and audio information as well as various image processing to be described later is installed and executed in the computer 10.

The conference program of the present invention is a browser equipped with a function of transmitting and receiving video and audio information, a browser equipped with a plug-in for transmitting and receiving video and audio information, or a communication channel for the purpose of video conferencing and information It can have various formats, such as a separate application program responsible for transmission and reception.

In particular, as shown by a phantom line in FIG. 1, a web page 21 connected to the server 20 is built and a conference program of the computer 10 is connected to the server 20 via the web page 21. The method can also be applied to the present invention. In this case, the above-described browser can be applied as a conference program, and since the browser is a program familiar to general Internet users, it is advantageous to secure user convenience when the browser is applied as a conference program.

2 and 3 illustrate the screen of the computer 10 on which the video conference of the present invention is performed. As shown, the screen of the computer 10 displays the video transmitted from each participant computer 10 of the video conference. Information, that is, an image captured by each participant's own computer 10 is output, and the image information transmitted by each computer 10 is a speaker screen 11, a person's screen 12, and a listener screen 13 are separated and output.

The speaker screen 11 is a screen that captures the main speaker speaking at the time, and the main speaker screen 12 is an image of the user of the computer 10 captured by the camera installed in the computer 10. As a screen for outputting, the speaker screen 11 and the user screen 12 are output with an area far exceeding the area occupied by other individual listener screens 13.

In addition, FIG. 4 also illustrates the screen of the computer 10 of the present invention. In the illustrated embodiment, a smartphone, a portable Internet access information device, is applied as the computer 10, and in this case, the speaker screen 11 and The main screen 12 exceeds the area of the blue screen 13.

As described above, in the screen output during a video conference, the present invention does not output an actual captured image without adding or subtracting, but extracts a person from the captured image and outputs it by synthesizing it with a virtual background. The actual screen 31 is output as the original screen 12 by outputting the image as it is, and FIGS. The state of being output as is exemplified. This virtual screen 32 output is provided as a function that the user can select, and as illustrated in FIGS. 2 to 5, the user clicks an object such as an icon implemented on the screen. Alternatively, the function of the virtual screen 32 may be performed through a touch operation.

That is, as shown in FIG. 5, when the user clicks the icon at the top right in the actual screen 31 output state as shown in the upper part of the drawing, the screen is converted to the virtual screen 32, and as shown in the lower part of the same drawing Even in the virtual screen 32 output state, it is possible to return to the actual screen 31 by clicking the icon at the top right.

As in the conventional video conferencing system as shown in FIGS. 6 and 7, the implementation of the virtual screen 32 extracts person information 50, which is image information corresponding to a person, from an actual shot image, and person information ( After removing the erased area 59, which is the image information of the actual screen 31 except for 50), the person information 50 from which the erased area 59 has been removed is overlapped with the background information 40 that is pre-stored and secured. This is done in a synthetic way.

Of course, extraction of the person information 50 has no choice but to follow the actual motion of the person, and therefore, in the prior art, as shown in FIG. 7, the motion of the person is reflected on the virtual screen 32 without adjustment, The movement of the person in 31) is also output as it is on the virtual screen 32.

In general, video conference participants have slight differences depending on individual inclinations, but they cannot help but move due to various factors such as changing their posture or moving their location in the process of conducting a video conference for a long time. In the prior art, as shown in FIG. 7, the user The actual movement of is reflected in the person information 50 as it is, and the person on the screen has no choice but to move on the virtual screen 32 finally implemented.

As described above, the movement of the person on the virtual screen 32 causes considerable visual fatigue and psychological anxiety compared to the movement on the real screen 31, which seriously damages the immersion and effectiveness of the video conference will do

Therefore, in the present invention, as shown in FIG. 8, despite the movement of the person on the actual screen 31, on the virtual screen 32, the person can be output while maintaining the correct position. As a video conference method performed by a plurality of computers 10 that are directly connected to each other through, or indirectly connected to each other via a web page 21 built on the Internet or a server 20 connected thereto, as shown in FIG. , The conference program loaded in the computer 10 is executed to conduct a video conference, the conference program extracts the person information 50 from the image information of the actual screen 31, and the conference program extracts the background data from the background database 90. It starts with the execution step (S10) of fetching the information (40).

The present invention is basically a video conferencing method of extracting a person part from an actual photographed image and synthesizing it with a virtual background, as illustrated in FIGS. 2 to 4, the computer 10 user's own screen ( 12) is generally applicable.

Here, the background information 40 is not a background image actually captured at the time, but image information pre-photographed or collected and stored in the background database 90, which is usually beautiful scenery, elegant indoor space, or other user's preferences. Depending on the selected background can be applied.

Among the captured images constituting the real screen 31, the method of extracting the person information 50, as described above, may be applied with various techniques such as image processing through image extraction through machine learning as well as outside extraction, and in FIG. As described above, according to the extraction of the person information 50, a portion of the captured image constituting the actual screen 31 excluding the person portion is set as an erase area 59 and removed.

Thereafter, a setting step (S21) is performed in which the conference program sets a reference point 55 for specifying the position of all images constituting the person information 50 in the extracted person information 50, where the person information 50 The entire image constituting ? refers to an area of a captured image of a real person excluding the erased area 59 in FIG.

In addition, the reference point 55 for specifying the position of the entire image constituting the person information 50 is a point having a characteristic of maintaining a proper position within the area of the pure person image, and is usually the center point of the area, that is, the outer edge of the area. It can be set as the center of gravity of the closed curve forming . By subdividing the area of this person information (50), the facial outline can be extracted separately from the body outline. In this case, the center of gravity of the closed curve outside the face can be applied as the reference point (55). there is.

Subsequently, as shown in the lower part of FIG. 9, the meeting program overlaps the background information 40 and the character information 50 so that the target point 45 preset in the background information 40 and the reference point 55 coincide. As the overlapping step (S22) of creating the screen 32 is performed, as shown in FIG. 10, even if the movement of the person on the real screen 31 occurs, the position of the person on the virtual screen 32 is in the center of the screen. can remain stable.

In addition, the above-described execution steps (S10) to overlapping steps (S22) are continuously and repeatedly performed in the process of conducting a video conference applying the present invention, so that the position of a person output on the screen can be stably adjusted and managed even if the movement of a real person continues. can

Meanwhile, FIGS. 11 and 12 are diagrams for explaining a slow-type embodiment of the present invention. In the slow-type embodiment of the present invention, the reference point 55 of the person information 50 is changed to the background information according to the above-described overlapping step (S22). In the process of following the target point 45 of (40), it is possible to suppress the rapid change of the output virtual screen 32 due to supercharged tracking and implement a more comfortable virtual screen 32.

That is, the person on the virtual screen 32 temporarily moves according to the movement of the person on the real screen 31, and then in the process of adjusting the position to the center of the screen, it can move smoothly as shown in FIG. 11 without abrupt movement. In performing the overlapping step (S22), the conference program sets the background information 40 and the person information 50 so that the target point 45 preset in the background information 40 and the reference point 55 coincide. ) to create a virtual screen 32, but as shown in FIG. 12, a plurality of waypoints 65 are set on a linear path connecting the reference point 55 and the target point 45 before overlapping. This is achieved by allowing the reference point 55 to coincide with the target point 45 via these via points 65 sequentially.

Here, the linear path connecting the reference point 55 and the target point 45 is a straight line or curved path connecting the reference point 55 and the target point 45 without retrogression, as in the dictionary meaning, In the drawing of FIG. 12, the waypoint 65 is set on a horizontal line connecting the reference point 55 and the target point 45.

In addition, FIG. 13 describes an adjustable embodiment of the target point 45 of the present invention, and shows a screen when a person moves in the same embodiment. As shown, the target point 45 can be moved and set according to the user's manipulation. As a result, the user can freely change the position of the person by breaking away from the uniform screen in which the person is placed in the center, which not only secures the aesthetics of the entire screen, but also provides data necessary for conducting a video conference with the person. It can be usefully used in situations where it is accompanied by output.

In the target point 45 adjustable embodiment of the present invention, as shown in FIG. 13, after the overlapping step (S22) is performed, as the object corresponding to the person information 50 on the virtual screen 32 is manipulated, The location of the target point 45 on the background information 40 is changed. In the illustrated embodiment, the user places the mouse cursor in the area where the character information 50 is output and drags the target point 45 through a drag operation. In addition, various methods such as implementing and manipulating a separate marker or icon can be applied.

10 : computer
11: Talker screen
12 : My screen
13: celadon screen
20: server
21: web page
31: actual screen
32: virtual screen
40: background information
45: target point
50: person information
55: reference point
59: erase area
65: transit point
90: background database
S10: Execution step
S21: Setting step
S22: nesting step

Claims (2)

As a videoconferencing method performed by a plurality of computers 10 connected through the Internet, a conference program installed in the computer 10 extracts person information 50 from image information of an actual screen 31, and a background database ( 90), an execution step (S10) of fetching the background information (40), and a setting step (S21) of setting a reference point (55) for specifying the location of all images constituting the person information (50) by the meeting program; An overlapping step in which the conference program creates a virtual screen 32 by overlapping the background information 40 and the person information 50 so that the target point 45 preset in the background information 40 and the reference point 55 coincide ( S22) and the overlapping step (S22) are performed, and as the object corresponding to the person information 50 on the virtual screen 32 is manipulated, the person whose position of the target point 45 on the background information 40 is changed. In the position adjustment type video conference method,
In the overlapping step (S22), the conference program overlaps the background information 40 and the person information 50 so that the target point 45 preset in the background information 40 and the reference point 55 coincide. ), but set a plurality of waypoints 65 on a linear path connecting the reference point 55 and the target point 45 before overlapping, and the reference point 55 passes through these waypoints 65 to the target Character position slow adjustment video conference method, characterized in that it coincides with the point (45). delete

Images