JPH05183811A - Tv telephone set - Google Patents

Abstract

PURPOSE:To obtain a TV telephone set with excellent operability capable of pointing with a sense of pointing out a part of a screen from a distant place. CONSTITUTION:The position of a human image in a video signal imaged by a portrait camera 10 is portrait position detection circuit 11 and a pointing area that all the display areas of screens displayed on a video monitor 30 are mapped within the screen of the portrait image as reference of the position the portrait image is designated by a pointing area designation circuit 12. From the position of the pointing object within the designated pointing area, the pointing position is detected in a pointing position detection circuit 13, a marker to be displayed on the detected pointing position is generated in a marker generation circuit 14 and this marker is superimposed on a monitor video to be being displayed on the video monitor 30 by a superimpose circuit 19.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video telephone having a function of pointing the position of a displayed image.

[0002]

2. Description of the Related Art A videophone is a communication device for making a call by video and voice, and its basic function is to send and receive a person's image of each other and talk while looking at the face of the other party. There may be a case where a person wants to explain to the other party by using an image such as a calligraphy or VTR. For this reason, in addition to a person camera that captures a person image, an external input terminal is often provided so that an image input to this external input terminal can be transmitted instead of the person image as needed. When a call or video of a VTR is transmitted by a videophone having such an external input terminal and a call is being made, there is a case where it is desired to indicate a specific portion of the screen. In order to meet this demand, there is an example in which the position on the screen can be pointed by a mouse, a touch sensor, an arrow key, or the like.

[0003]

Although the above-mentioned video telephone having the pointing function can point a specific portion of the display screen, it is necessary to directly operate a pointing device such as a mouse, a touch sensor, and arrow keys. is there. Among the pointing devices mentioned above, the one that attaches a transparent touch sensor to the display surface and directly touches the position on the displayed screen is the most operable because it matches the action pointing the screen as a pointing operation. There is. Therefore, this method is an excellent pointing method for a comparatively small desk-top type video telephone or the like that looks at the screen at a close range within the reach of the screen.

However, in a videophone having a large-screen display device embedded on a wall or a stationary type of several tens of inches or more, the viewing position is also away from the screen and is out of reach. With the conventional technique, a pointing device such as a mouse is at hand. There is no choice but to extend it to the pointing operation.
For this reason, when the screen becomes large, the position of the pointing operation and the position of the pointing operation greatly shift,
There is a problem of poor operability.

The object of the present invention is to solve this problem.
(EN) It is possible to provide a videophone with good operability that can be pointed as if pointing a part of the screen from a distant position.

[0006]

To achieve the above object, the present invention provides a means for detecting the position of a human image in a video signal captured by a human camera for capturing a human image, and the position of the human image. A means for designating a pointing area in which the entire display area of the screen displayed on the video monitor is mapped within the screen of the person image based on the above, and the pointing position is detected from the position of the pointing object within the designated pointing area. Means, first marker generating means for generating a marker to be displayed at the detected pointing position, means for transmitting pointing data indicating the pointing position to a partner terminal, and receiving pointing data sent from the partner terminal And a second marker generation means for generating a marker at the position indicated by the received pointing data. When, and means for superimposing a marker generated by the first marker generating means and the second marker generating means in the monitor image being displayed on the video monitor.

[0007]

According to the present invention, by moving a pointing object within a pointing area provided near a human image in a video signal captured by a human camera, a specific portion of the video displayed on the video monitor is pointed. In particular, it is possible to realize a very excellent pointing means in a large-screen TV phone.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a diagram showing an embodiment of the present invention, in which 10 is a person camera, 11 is a person position detecting circuit, 12 is a pointing area designating circuit, 13 is a pointing position detecting circuit, and 14 and 15 are first, respectively. , A second marker generating circuit, 16 a pointing data transmitting circuit, 17 a pointing data receiving circuit, 18 a monitor display image selection switch (SW), 19 a superimposing circuit, 20
Is an external video input terminal, 21 is a transmission video selection switch (S
W), 22 are network (NW) interface circuits, and 30 is a video monitor.

Video telephones are most often used for sending and receiving person images to and from each other and talking while looking at the faces of the other parties, and are generally set in this state. In this state, the transmission video selection switch 21 is on the a side, the monitor display video selection switch 21 is also on the a side, and the videophone is the person camera 10
The video signal from the partner terminal, which is transmitted through the NW interface circuit 22 to the partner terminal and is received via the NW interface circuit 22, is the video signal from the partner terminal.
Is displayed.

However, it often happens that during the call, it is desired to send an image of the VTR or use a document camera to send a document and use the image to explain. For this reason, it is general that the external video input terminal 20 is provided and the function of sending the video from the VTR or the like is provided. When transmitting a video from this external video input terminal 20, a transmission video selection S
It goes without saying that the state of W21 is set to the b side, but in general it is a conversation that talks about the video being transmitted, and the state of the monitor display video selection SW18 is controlled to the b side to communicate with the other party. Generally, the same image is viewed.

For convenience of explanation, it is assumed that the terminal (self-terminal) of interest is in the above-described state in which both the transmission video selection SW 21 and the monitor display video selection SW 18 are on the b side. That is, this is a case where both the transmitted image and the image displayed on the image monitor 30 are images input to the external image input terminal 20. On the other hand, it is assumed that the other terminal displays the image transmitted by the own terminal on the image monitor 30 and both are watching the same image. That is, the monitor display video selection SW 18 indicates that
Shall be on the side.

The video signal output from the person camera 10 is input to the person position detection circuit 11. There are various methods for detecting the position of a person. For example, according to "1991 Shin-Gakkai Gaikoken Material IE91-51", the center of gravity of the facial region is detected as the position of the human image. The person position output circuit 11 used here outputs a position of a specific portion of a person image as person position information as in this example.

The person position information output from the person position detecting circuit 11 is input to the pointing area designating circuit 12. The pointing area designating circuit 12 sets a pointing area in the screen imaged by the person camera 10 with the position of the person position information input from the person position detecting circuit 11 as a reference point, and outputs it as pointing area information. This pointing area is used by the video monitor 3
This is a mapping of the entire area displayed at 0, and is set so that the position of the image displayed on the image monitor 30 can be uniquely specified in correspondence with each point in the pointing area.

Pointing position detection circuit 13 is supplied with pointing area designation information from pointing area designation circuit 12 and a video signal input from person camera 10. When the pointing object can be detected in the area designated by the pointing area designation information in the video signal input from the human camera 10, the pointing position detection circuit 13 detects the position on the screen and the position of the pointing area. The pointing coordinates on the video monitor are obtained from the information and output as pointing position information.

Depending on what is used for the pointing object,
Although the detection method is different, the simplest method is to use a small light emitting object such as a light emitting diode. In a normal state, when a light emitting object exists within the angle of view of the camera, the image level of the output video signal from the camera becomes a very high level in the light emitting object portion. Therefore, it is possible to set a threshold value of a considerably high level value and set a portion having a level equal to or higher than the threshold value as the pointing position. As a concrete example of the shape, a short rod having a light emitting diode at its tip, a finger cap, or the like can be considered.

The first marker generation circuit 14 generates a marker to be superimposed on the position on the screen of the video monitor 30 indicated by the coordinates designated by the pointing position information output from the pointing position detection circuit 13, and superimposes it. The circuit 19 superimposes the marker generated by the first marker generation circuit 14 on the screen being displayed on the video monitor 30 and displays it.

The pointing data transmission circuit 16 transmits the pointing data to the partner terminal via the NW interface circuit 22 every time the pointing position information output from the pointing position detection circuit 13 changes.

By the above operation, a marker indicating the designated position is provided on the screen displayed on the video monitor 30 at the position designated by the pointing position information detected from the video signal output from the person camera 10. The superimposed data is displayed, and the pointing data indicating the pointing position is transmitted to the partner terminal.

Next, the operation when the other terminal receives the pointing data transmitted from its own terminal will be described.

Since the other terminal has the monitor display video selection switch 18 on the side a, the video monitor 30 displays the video input from the external terminal to the external video input terminal 20, that is, the same video as the self terminal. .. The pointing data transmitted from the pointing data transmitting circuit 16 of the own terminal is received by the pointing data receiving circuit 17 through the NW interface circuit 22 of the other terminal.

The pointing data receiving circuit 17 decodes the received pointing data, reproduces it as pointing position information, and inputs it to the second marker generating circuit 15.

The pointing position information reproduced here is exactly the same as that output from the pointing position detection circuit 13 on the terminal side. The second marker generation circuit 15 operates in exactly the same manner as the first marker generation circuit 14 and generates a marker to be superimposed and displayed on the screen position of the video monitor 30 indicated by the pointing position information input from the pointing data reception circuit 17. , Is superimposed and displayed on the image being displayed by the superimposing circuit 19.

As is clear from the above description, the images displayed on the image monitors 30 of the own terminal and the other terminal are exactly the same (in this example, the external image input terminal 20 on the own terminal side).
Input image), and the marker position superimposed on the displayed image is also the same. Therefore, the pointing state on the self-terminal side is transmitted to the partner terminal as it is, and the pointing position can be clearly communicated to the partner.

Although the above description has described only the pointing on the side of the own terminal, the pointing operation on the side of the own terminal and the pointing operation on the side of the partner terminal are exactly the same. In this example, the other terminal side has the monitor display image selection switch 18 on the a side, and the image from the other terminal is displayed on the image monitor 30. Therefore, the pointing operation on the partner terminal side becomes the pointing on the video on the partner terminal side, and the marker indicating the position is displayed on the video monitor 30 of the own terminal while being superimposed and displayed on the display video on the video monitor 30 on the partner terminal side. A marker indicating the position is also displayed on the inside image. When both are pointing at the same time, two markers are displayed at the same time. In this case, by changing the shapes and colors of the markers generated by the first marker generation circuit 14 and the second marker generation circuit 15, respectively, it is possible to easily identify which side the pointing is. You can do it.
As a result, it becomes possible for both parties to have a conversation while pointing at a specific location in the same way while seeing the same screen on the terminal side and the partner terminal side.

Next, the operation of the monitor display image selection SW 18 is independent of the above pointing operation. Therefore, when both the own terminal side and the partner terminal side perform pointing while displaying the transmitted video signal on the video monitor 30, the position of the marker indicating the pointing position that is displayed in a superimposed manner is a position for different videos, which is inconvenient. ..

As a method of preventing this, the pointing data transmitted from the pointing data transmission circuit 16 is added with pointing screen identification data indicating the state of the monitor display image selection switch 18 at the time of transmission, and the pointing data is transmitted on the receiving side. The content of the pointing screen identification data received by the receiving circuit 17 is the video monitor 3
If it does not match the image being displayed at 0, it is possible to prevent the above-mentioned inconvenience from occurring by stopping the operation of the second marker generation circuit 15. That is,
When the received pointing screen identification data indicates that the monitor display video selection switch 18 is on the side a, pointing to the received video signal on the terminal side that transmitted the pointing data, that is, sent out from the terminal on the side that received the pointing data Since the pointing is for an image, the terminal side receiving the pointing data operates the second marker generation circuit 15 only when the monitor display image selection SW 18 is on the b side. Further, when the received pointing screen identification data indicates that the monitor display video selection SW is on the side b, it is pointing to the video transmitted by the terminal that transmitted the pointing data, and the terminal that received the pointing data. On the side, the second marker generating circuit 15 is operated only when the monitor display video selection switch 18 is on the side a.

FIG. 2 is a diagram for explaining the pointing operation according to the present invention. 100 is a person image output from the person camera 10, 110 is a pointing area designated by the pointing area designating circuit 12, and 120 is an external image input. An external input image input from the terminal 20, 130 a marker output from the marker generation circuit 14, 140 a marker space in which the marker can exist, 150 an image monitor 3
It is a monitor display image displayed at 0.

The example of FIG. 2 shows a case where both the transmission video selection SW 21 and the monitor display video selection SW 18 of FIG. 1 are in the b side.

A person image 10 picked up by the person camera 10.
0 is input to the person position detection circuit 11 to detect the position of the person, and the pointing area 110 is set in the screen of the person image 100 by the method determined by the pointing area designating circuit 12 based on the detected person position. ..

Next, the pointing position detection circuit 13 finds the position of the pointing object in the pointing area set in the person image 100, and the marker generation circuit 14 outputs the external input terminal 20 corresponding to the found position.
A marker 130 to be displayed in a superimposed manner is generated at a position on the screen of the externally input video 120 that is input more.

Subsequently, the superimposing circuit 19 creates a monitor display image 150 in which the marker 130 is superimposed on the external input image 120 and displays it on the image monitor 30.

As shown in FIG. 2, each point in the pointing area 110 set on the screen of the person image 100 is input to the external input image 12 via the virtual marker space 140.
There is a one-to-one correspondence with each point within 0, and the marker generation circuit 14
The position of the marker 130 generated at 1 is generated at the corresponding position in the external input image 120.

The above description is for the case of pointing to the external input video 120 input to the external input terminal 20, but when the monitor display video selection SW 18 is on the side a, the video displayed on the video monitor 30 is the other party. It is the video sent from the terminal, and in this case, it is obvious that the pointing is to the partner video.

FIG. 3 is a diagram for explaining a method of obtaining pointing position information output from the pointing position detection circuit 13.

FIG. 3 shows an example in which the pointing area 110 is set on the right side of the pointing person.

The position of the person image 100 on the screen is represented by coordinates (x, y), and the origin is at the lower right of the screen.
The height of the screen is V and the width is H.

It is assumed that the person position output from the person position detecting circuit 11 indicates the center of gravity of the face area, and that position is M
And The person position detection circuit 11 outputs the position information of the person position M at coordinates (x _m , y _m ).

First, the pointing area designating circuit 12 first receives the coordinates (x
_m, the origin P coordinates _(P h of the pointing area 110 specified by the following equation from the _{y m),} determine the _{P v).}

[0039] In _{P h = x m + a (} 1) P v = y m + b (2) where, a, b is x from M points of the respective point P, a distance in the y-direction, right or from the point M, respectively The upward direction is the positive direction.

The horizontal and vertical magnifications of the pointing area are 1 / n _h and 1 / n _v , respectively, and the coordinates of the detection position of the pointing object detected by the pointing position detection circuit 13 are (x _p , y _p ). Then, x _p , y _p
The movement range of is as follows.

P _h <x _p <p _h + H / n _h (3) P _v <y _p <p _v + V / n _v (4) Next, the position within the screen of the display image of the image monitor to be pointed As in the case of a person image, assuming that the lower left corner is the origin and the pointing coordinates are (P _x , P _v ), the coordinates (x _p , y _p ) of the detection position of the pointing object in the screen of the person image are used to calculate the coordinates by the following equation. The coordinates can be calculated.

P _x = n _h (x _p −P _h ) (5) P _v = n _v (y _p −P _v ) (6) The magnification of the pointing area may be any value, but it is a 1 / integer. Is more convenient. Moreover, although this magnification depends on the angle of view of the person camera, 1/3 or 1/4 is suitable.

In addition, a constant a that determines the distance between the points M and P,
Although the value of b varies depending on the angle of view of the portrait camera and the size of the pointing object, assuming that the fingertip is used as the pointing object, the absolute value of a is 3 to 4 times the face width of the portrait image of the portrait image, and b It is appropriate that the absolute value of is about 1 times the face width.

In the video telephone, the person camera is generally fixed once set, and the magnification of the pointing area and the values of the constants a and b are set according to the use conditions when the video telephone is installed. You can

As is clear from the above description, the size of the pointing area and the relative position of the person and the pointing area are constant. That is, the user can perform pointing without being conscious of the location within the angle of view of the portrait camera. However, some of the pointing area when the value of P _h and P _v obtained by the formula (1) and (2) has become such a value below is off screen, it can be set pointing region.

P _v <0 (7) P _h > H-H / n _h (8) P _v <0 (9) P _v > V-V / n _v (10) In each of these cases, Correspondingly, the above problems can be avoided by setting _Ph and _Pv as described below.

In the case of the formula (7): P _h = 0 In the case of the formula (8): P _h = H−H / n _{h In} the case of the formula (9): P _v = 0 In the case of the formula (10): P _v = V-V / n _v

[0048]

As described above, in the present invention, the pointing of a specific portion of the image displayed on the image monitor is realized by detecting the position of the pointing object from the image signal output from the person camera. .. Therefore, even when using a large-screen video monitor whose viewing position is far from the screen, it is possible to point while looking at the screen without having to use a pointing device such as a mouse.
This has the effect of eliminating the need to sometimes remove the line of sight from the screen. Also, since the effective area for pointing is automatically set based on the position of the human image, you can point with the same feeling anywhere within the angle of view of the human camera, so you can use the same pointing method as when using a mouse, etc. The place is no longer restricted, and the effect of being able to point on the spot when necessary while acting naturally is produced.

[Brief description of drawings]

FIG. 1 is a diagram showing an embodiment of the present invention.

FIG. 2 is a diagram illustrating a pointing operation according to the present invention.

FIG. 3 is a diagram illustrating a method of obtaining a pointing position according to the present invention.

[Explanation of symbols]

10 ... Person camera, 11 ... Person position detecting circuit, 12 ... Pointing area designating circuit, 13 ... Pointing position detecting circuit, 14, 15 ... Marker generating circuit, 16 ... Pointing data transmitting circuit, 17 ... Pointing data receiving circuit, 18 ... Monitor display video selection SW, 19 ... Superimpose circuit, 20 ... External video input terminal, 21 ... Transmission video selection SW, 22 ... NW interface circuit, 30
... video monitor, 100 ... portrait video, 110 ... pointing area, 120 ... external input video, 130 ... marker, 1
40 ... Marker space, 150 ... Monitor display image.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Hiroshi Sakai 1-1-6 Uchisaiwaicho, Chiyoda-ku, Tokyo Nihon Telegraph and Telephone Corporation

Claims (1)

[Claims] 1. A means for detecting the position of a human image in a video signal captured by a human camera for capturing a human image, and a means for displaying the image of the human image on a video monitor within the screen of the human image based on the position of the human image. Means for designating a pointing area mapping the entire display area of the screen, a means for detecting the pointing position from the position of the pointing object in the designated pointing area, and a marker for displaying at the detected pointing position. First marker generating means, means for transmitting pointing data indicating the pointing position to a partner terminal, means for receiving pointing data sent from the partner terminal, and generating a marker at the position indicated by the received pointing data. Second marker generating means, the first marker generating means and the second marker generating means A videophone comprising: means for superimposing a marker generated by the live means on a monitor image being displayed on an image monitor.