JP5178543B2 - Video conference system and control method thereof - Google Patents

Description

  The present invention relates to a TV conference system using a TV conference apparatus, and facilitates a plurality of TV conferences held by a small number of people, and allows a start time and an end time of each TV conference to be set individually. About.

  2. Description of the Related Art Video conferencing systems are widely used in which conference rooms at two or more points separated from each other are connected by a network and images and voices of conference participants are transmitted and received between the conference rooms to perform a conference. FIG. 11 shows a typical example of a conventional TV conference system. This TV conference system 30 is connected to a TV conference apparatus at other points via a network, and includes a camera 31, a monitor 32, a microphone 33, a speaker 34, and A round table 35 is provided. In this case, the microphone 33 and the speaker 34 are configured as an integrated device.

  In this TV conference system 30, images of conference participants A 11, A 12, A 13 taken by the camera 31 at the point A and the sound collected by the microphone 33 at the point A are transmitted to the point B via the TV conference device. Transmit to the TV conference device. At the same time, image information and collected sound of the conference participants B11 and B12 are transmitted from the TV conference device at the B point, and are output to the monitor 32 and the speaker 34 at the A point, so that the two points between the A point and the B point. The video and audio of conference participants can now be sent and received.

  In a normal TV conference system 30, each conference participant in each conference room proceeds with the TV conference while grasping the status of other conference rooms via the monitor 32. The image taken by the camera 31 is a vaguely taken picture of the situation of the other party's conference room, and is monotonous, so that there is a lack of realism and it is difficult to obtain a sense of unity among the entire conference participants. Therefore, by using the round table 35, all the conference participants (A11, A12, A13, B11, B12) at both the A point and the B point make the round table 35 surrounded by the car seat and feel unity. Devised to get.

FIG. 12 shows an embodiment in which the advantage of the round table 35 is used to surround the vehicle seat at the point A. Japanese Patent Application Laid-Open No. 2004-133867 has a configuration that allows the change to a form as shown in FIG. As described above, there are proposals for changing convenience according to the form of the TV conference and the situation of the TV conference place, and making it possible to grasp the state of all the conference participants, the atmosphere in the conference room, and the like, thereby improving convenience. .
Further, Patent Document 2 proposes that the size of the round table 35 corresponding to the number of participants in the conference can be changed. It is possible to arbitrarily change the relative orientation and posture of the desk body according to the purpose of use and the space.

JP 2008-085930 A Japanese Patent No. 02894633

  However, since these TV conference systems are configured only for one TV conference, the next second TV conference cannot be held unless the one held TV conference is completed. In the TV conference system, a dedicated conference room in which a TV conference device is installed is necessary at each point, and there are circumstances in which many dedicated conference rooms cannot be provided. In addition, it is difficult for the TV conference system to effectively improve the movable rate by adjusting the holding time of each conference.

  Furthermore, the dedicated conference room of the TV conference system does not have the flexibility to expand or narrow according to the number of participants in the conference. A dedicated conference room is set according to the size of the conference, which is expected to have a relatively large number of participants. For this reason, in a TV conference with a small number of participants, the conference room is too wide and lacks a sense of reality, making it difficult to obtain a sense of unity among the entire conference participants. In particular, it is difficult for a desk or the like on which equipment such as a TV conference device is installed to move according to the scale of the conference.

In order to solve the above problems, the present invention provides a photographing means for photographing a plurality of conference groups at a first location as a ring image or a partial image thereof, and a plurality of conference groups at the first location as each conference group. The first grasping means for grasping the position of the first partition to be divided in the image photographed by the photographing means and a plurality of conference groups at the second point facing the display means at the second point are assigned to each conference group. A second grasping means for grasping a position of the second partition to be partitioned; and a position of the first partition in an image photographed by the photographing means coincides with a position of the second partition at the second point. JP further comprising a correction means for correcting the image, and a display control means for displaying the image in which the correction means is corrected to the display means in the second point in To provide a TV conference system or the like to be.

  In the TV conference system of the present invention, a plurality of small-scale TV conferences can be held simultaneously on one video line in a dedicated conference room for TV conferences. Furthermore, each TV conference can be held separately by starting up and closing time by dedicating only the voice line. In addition, the space of each conference can be changed arbitrarily and flexibly by moving the partition wall plate according to the number of conference groups and the number of participants in each conference.

As a result, the video conference system can be effectively used to the maximum extent, and the availability of the video conference system is improved.
In addition, it is possible to perform an interview method such as a collective match form by having individuals sit at points A and B and changing their order.

A TV conference system according to an embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a view of the TV conference system of the present invention as viewed obliquely from above. In the figure, reference numeral 10 denotes an overall outline of the video conference system of the present invention. Reference numeral 11 denotes a round table provided at the center of the conference room at each point of the TV conference system 10. The round table 11 is equipped with equipment of a TV conference apparatus described below, but has no special shape or configuration. Reference numeral 12 denotes an omnidirectional camera that captures panoramic images (annular images) in 360 ° in all directions in the horizontal direction. This camera can take a ring image of a subject or a partial image thereof. In addition, this video conference system is provided with the communication means which enables transmission and reception of a ring image to each point.

  The panoramic image shooting lens of the omnidirectional camera 12 is formed of optical glass, a transparent resin for the lens, and the like, is formed rotationally symmetric around the optical axis at the center of the lens, and swells so that light can enter from the entire 360 ° circumference. And an annular light incident surface formed in a substantially convex lens shape. Reference numeral 13 denotes an angle of view indicating a range that can be captured by the omnidirectional camera 12 including the vertical direction. For ease of understanding, only 180 ° in the horizontal direction is shown in the figure.

  Reference numeral 14 denotes an omnidirectional projector that projects a panoramic image in 360 ° in all directions in the horizontal direction. The panoramic image projection lens of the omnidirectional projector 14 is formed of optical glass, a transparent resin for the lens, and the like, is formed rotationally symmetric around the optical axis at the center of the lens, and swells so that light can be emitted from the entire 360 ° circumference. And an annular light emitting surface formed in a substantially convex lens shape. That is, the panoramic image projection lens of the omnidirectional projector 14 and the panoramic image photographing lens of the omnidirectional camera 12 use the same type of lens, and the same lens can be used. Reference numeral 15 denotes an angle of view indicating the range that can be projected by the omnidirectional projector 14 including the vertical direction. Similarly, only 180 ° in the horizontal direction is shown in the figure for easy understanding. Reference numeral 16 denotes a 360 ° cylindrical screen that displays a panoramic image projected in all directions by the omnidirectional projector 14. The 360 ° cylindrical screen 16 having this configuration is a transmissive (real projection) image projection, and an image photographed by the omnidirectional camera 12 and a projected image are horizontally reversed. Therefore, although not shown in the figure, it is possible to easily cope with the control means incorporated in the conference apparatus server or the like by switching the left and right coordinates when converting the captured panoramic image into the projected panoramic image.

A11 is a conference participant who sits around the round table 11 and looks into each TV conference. Here, in order to distinguish and explain the conference participants below, the conference participants are described with a three-digit number. The first digit of the conference participant's number indicates where it is, the second digit of the number indicates which conference group, and the third digit of the number indicates the number of people.
Example 1) The conference participant A11 means the first of the first conference group at the point A.
Example 2) Conference participant B22 means the second person in the second conference group at point B.

  Reference numeral 17 denotes a headset that is used by the conference participant A11 for the TV conference and that integrates a microphone and a speaker. The headset 17 is worn by all members, and a dedicated audio line is connected to each individual TV conference group so that only conference participants in each conference group can talk. Reference numeral 18 denotes a partition wall plate for separating each conference group. As will be described later, an identification mark for identifying each partition wall plate 18 is provided on the end of the partition wall plate 18 facing the omnidirectional camera 12.

FIG. 2 is a vertical cross-sectional view of the video conference system of the present invention.
The omnidirectional camera 12 is located at the center of the round table 11 and above the omnidirectional projector 14, and is supported by a member extending from the upper end annular frame of the cylindrical screen 16. The vertical angle of view 13 that can be captured by the omnidirectional camera 12 can be captured by the bust image of the conference participant A11 without being blocked by the cylindrical screen 16. The omnidirectional projector 14 is also located on the round table 11 and at the center inside the cylindrical screen 16.

  The vertical view angle 15 that can be projected by the omnidirectional projector 14 does not protrude from the cylindrical screen 16, and the captured image at point B is projected and displayed as a panoramic image. Thus, the conference participant A11 can view the conference participant on the other side of the point B in front of himself / herself.

FIG. 3 is a detailed view showing that an identification mark for identifying each partition wall plate 18 is provided.
An identification mark for identifying each partition wall plate is provided on an end of the partition wall plate 18 separating each conference group facing the omnidirectional camera 12. For example, the first partition wall plate 18a and the second partition wall plate 18b have different intervals between the circles of the identification marks. By photographing and identifying this identification mark by the omnidirectional camera 12, the control means incorporated in the conference server or the like allows the position, quantity, arrangement angle of the partition wall plate 18 and the relative position of each conference group. Relationships can also be grasped.

FIG. 4 is a view of the point A of the pattern 1 where a plurality of TV conferences are held as viewed obliquely from above.
A video conference at a point A of pattern 1 in a certain time zone is shown. Currently, four TV conferences are being held at points A and B. The first conference group includes two conference participants A11 and B11. The second conference group includes four participants, conference participants A21, A22, B21, and B22. The third conference group includes three conference participants A31, A32, and B31. The fourth conference group includes two conference participants A41 and B41. Looking at the round table 11 from diagonally above, it sits in the order of conference participants A11, A21, A22, A41, A31, A32 in the counterclockwise direction. This is photographed by the omnidirectional camera 12 at the point A and transmitted to the point B.

  The images on the cylindrical screen 16 are displayed in the order of conference participants B11, B21, B22, B41, and B31 in the same counterclockwise direction. (Conference participants B41 and B31 are not shown in this figure because they are displayed on the back side of the cylindrical screen 16.) The relative angles of the partition wall plates 18a to 18d separating each conference group at point A are as follows. In consideration of the second conference group and the third conference group, each of whom is participating at the point A, it is set wider than the first conference section loop and the fourth conference group. A dedicated voice line (communication means) is connected to each conference group, all of them are equipped with headsets 17 that allow only the conference participants of each conference group to talk, and a TV conference is in progress. Group conversations do not adversely affect other conference groups.

FIG. 5 is a diagram of the point B of the pattern 1 where a plurality of TV conferences are held as viewed obliquely from above.
Similarly, a TV conference at point B of pattern 1 in a certain time zone is shown. Looking at the round table 11 from diagonally above, the conference participants B11, B22, B21, B41, and B31 are sitting in the counterclockwise order. Similarly, this is photographed by the omnidirectional camera 12 at the point B and transmitted to the point A.

  The images on the cylindrical screen 16 are displayed in the order of conference participants A11, A22, A21, A41, A32, and A31 in the same counterclockwise direction. (Conference participants A41, A32, A31 are not shown in this figure because they are displayed on the back side of the cylindrical screen 16.) Here, the order of the conference participants at point B is It can be seen that the display order on the cylindrical screen 16 is different. Similarly, it can be seen that the order of conference participants at point A is different from the display order on the cylindrical screen 16 at point B. The relative angles of the partition wall plates 18a to 18d separating the conference groups at the point B are set equally regardless of the conference group, and are different from the relative angles at the point A.

Therefore, simply transmitting panoramic images in all directions taken at the points A and B to each other and displaying them on the cylindrical screen 16 at the points A and B respectively, And the partition wall plate 18 may be inconsistent.
Moreover, there is a possibility that the adjacent order of the conference group is shifted and the image to be faced is lost. Therefore, it is necessary to partially correct the display image for each conference group according to the situation.

FIG. 6 is a diagram in which the display image is partially corrected for each conference group.
Reference numeral 19 denotes a circular image taken by the omnidirectional camera 12 at the point B. When panoramic images in 360 ° in the horizontal direction are photographed through a panoramic image photographing lens, the omnidirectional camera 12 captures a planar circular image 19, and the graphic image becomes narrower in the center. The order of the photographed conference participants is B11, B22, B21, B41, B31 in the clockwise direction, and all the conference participants described in FIG. 5 are photographed from the front. In addition, the identification mark of each partition wall plate 18 is also photographed, and the position, quantity, and arrangement angle of the partition wall plate 18 and the relative positional relationship of each conference group are also controlled by the control means incorporated in the conference apparatus server or the like. To grasp. These image data are transmitted to point A through a video line.

  Reference numeral 20 denotes an image obtained by developing a panoramic image projected in 360 ° in all directions on the cylindrical screen 16 at the point A. The planar circular image 19 is expanded and reduced for each detailed portion, and is expanded as a panoramic image like a developed image 20a. Here, since the cylindrical screen 16 is a transmissive (real projection) image, the left and right coordinates of the captured image and the projected image are reversed for each conference group. That is, the order of the conference participants in the photographed circular image 19 is B11, B22, B21, B41, B31. However, the order of the conference participants in the developed image 20a is B11, B21, B22, B41, B31, which is the order of the conference participants on the cylindrical screen 16 described in FIG.

  Next, the relative angle of the partition wall plates 18a to 18d separating the conference group at the point A and the relative angle of the partition wall plates 18a to 18d separating the conference group at the point B are irrelevant. If an image is projected on the cylindrical screen 16 at the point A with the developed image 20a subjected to the image conversion correction at the interval between the partition wall plates 18a to 18d at the point B, the partition wall plates 18a to 18d at the point A are projected. The interval and the image display area do not match. Therefore, the image display area is corrected from the developed image 20a to the developed image 20b by the control means incorporated in the conference apparatus server or the like at intervals between the partition wall plates 18a to 18d at the point A. At this time, the display magnification or the like may be changed and displayed as in the conference participant B31.

  Further, the adjoining order of the conference groups at point A and the adjoining order of the conference groups at point B are not necessarily the same. Therefore, the image adjacency order is corrected from the developed image 20b to the developed image 20c, so that the image order to be faced corresponds to the arrangement order of each conference group. The order of the partition wall plates 18a to 18d separating the conference groups is denoted by the same reference numerals at the points A and B for easy understanding of the explanation in FIGS. Here, if the position of each other conference group is input to the control means incorporated in the conference apparatus server or the like by the operation unit, the constraints such as the relationship between the conference group and the partition wall plates 18 and the order of the partition wall plates 18. There is no. The control means recognizes the identification mark of each partition wall plate 18 by the omnidirectional camera 12, and image correction is performed as in the developed image 20b and the developed image 20c.

  Image conversion correction similar to these is performed, and the image data taken by the omnidirectional camera 12 at the point A is also transmitted to the point B via the video line, and the image is displayed on the cylindrical screen 16 at the point B.

FIG. 7 is a diagram of the point A of the pattern 2 where a plurality of TV conferences are held as viewed obliquely from above. Similarly, FIG. 8 is a view of the point B of the pattern 2 where a plurality of TV conferences are held as viewed obliquely from above.
The holding situation of the pattern 1 TV conference changes from moment to moment, and shows the pattern 2 TV conference over time. The number of video conferences is three, and the number of conference participants for each conference group is changed between the A point and the B point. In the first conference group, the number of participants increases from two to six, and the space expands. In the second conference group, the number of conference participants decreases from four to three, and the location is slightly changed as the space of the first conference group is expanded. The third conference group is continuing and can be left as it is. The fourth conference group ends the conference.

  The relative angle of the partition wall plate 18 that separates each conference group at the point A and the point B is set in consideration of the number of conference participants at each point and the ongoing video conference. 7 and 8, the partition wall plate 18c is removed. Different partition wall plates 18 may be used at the points A and B if the positions of the conference groups are input to the control means incorporated in the conference device server or the like by the operation unit. Moreover, it is also possible to recognize it as one sheet by overlapping two or more partition wall plates 18.

  In the same manner as described with reference to FIGS. 4 to 6, each conference group can hold a TV conference simultaneously on one video line. Furthermore, each TV conference can be held separately by starting up and closing time by dedicating only the voice line.

FIG. 9 is a diagram simply showing changes in the pattern 1 and the pattern 2 over time.
As described with reference to FIGS. 4 to 8, if a dedicated voice line and space for a TV conference can be secured, a new TV conference can be held independently of other conference groups. In addition, the space of each conference can be changed arbitrarily and flexibly by moving the partition wall plate according to the number of conference groups and the number of participants in each conference. In FIG. 9, it can be seen that the fifth conference group is scheduled to be held. As a result, the video conference system can be effectively used to the maximum extent, and the availability of the video conference system is improved.

  As described above, the video conference system at the two points of the A point and the B point has been described. However, although it is a little complicated, it can be easily understood that the video conference system at the three points or more can also be supported.

FIG. 10 is a flowchart showing a simple operation procedure in the video conference system of the present invention.
When starting a video conference, it is determined whether video transmission / reception lines at each point of the video conference system are connected (step S1). If the lines are not connected, the video lines at each point are connected to each other.
Next, a dedicated voice line is connected, and it is determined whether the number of conference groups is equal to the number of dedicated voice lines (step S2). If the number of voice lines is insufficient, the number of dedicated voice lines corresponding to the number of conference groups is mutually secured.

Next, in consideration of each conference group, it is determined whether the position of the separate partition wall plate has been changed (step S3). When the position is changed or set again, the relative position information of the partition wall plate at each point is transmitted to each other.
Next, it is determined whether the image area and magnification displayed on the cylindrical screen of each conference group are appropriate at each point (step S4). If not appropriate, the displayed image area and magnification are corrected corresponding to each conference group at each point.
Next, it is determined whether the images to be faced displayed for each conference group at each point are appropriate (step S5). If not appropriate, the order of the displayed image is corrected corresponding to each conference group at each point.
Next, it is determined whether the number of conference groups, the number of conference participants, and the like have been changed as time progresses as the TV conference progresses (step S6). If there is a change, the process returns to step 2 and is corrected so as to be appropriate.
When all the TV conferences at each point are closed (step S7), the TV conference system is terminated.

FIG. 13 is a view of the point B of the pattern 3 where the TV conference is held as seen obliquely from above.
In the figure, reference numeral 21 denotes an arc-shaped screen obtained by cutting out a part of the cylindrical shape. The basic system is the same as that shown in FIGS. 4 to 5 and 7 to 8 described above. Further, the TV conference is further performed with a small number of people, a small group, and a space-saving, and shows a TV conference at a point B of Pattern 3 which is a certain time zone. As seen from diagonally above, the conference participants B11, B22, and B21 are sitting in the counterclockwise order. Similarly, this is photographed by the omnidirectional camera 12 at the point B and transmitted to the point A. The images on the arc-shaped screen 21 are displayed in the order of conference participants A11, A22, A21 in the same counterclockwise direction.

  In this way, when a part of the cylindrical shape is cut out and used as an arc shape, there is a method of using it more effectively than simply transmitting panoramic images of all directions taken at the points A and B to each other. is there. The image quality range of the photographing camera (omnidirectional camera 12) and the projection projector (omnidirectional projector 14) is partially corrected in accordance with the situation, the photographing image range to be photographed through the panoramic image photographing lens.

FIG. 14 is a diagram in which the captured image range is partially corrected.
In the figure, reference numeral 22a denotes an image range that is actually captured by an image sensor such as a CCD of the imaging camera (omnidirectional camera 12). This image sensor has a horizontal / vertical aspect ratio of 16: 9 compatible with a wide screen. At this time, when panoramic images in 360 ° in the horizontal direction are photographed through the panoramic image photographing lens, they are photographed as planar circular images 19 by the omnidirectional camera 12, and the graphic images are spaced closer to the center. It narrows. The planar circular image 19 is stretched and contracted for each detailed portion to expand the development view as a panoramic image.
Reference numeral 23 denotes an image range in which the image is invalid in the actual captured image range 22. As can be seen from FIG. 13, the circular image 19 includes a portion where the conference participant is not photographed.

  Here, if only the part of the circular image 19 where the conference participant is photographed is photographed via the panorama image photographing lens, the image area 22b actually photographed by the image sensor is obtained, and the number of effective pixels. Many will be available.

24 b is an arc-shaped image corresponding to 180 ° of a part of the circular image 19. For example, in the conference participant B11, the actual captured image range 22b is captured at a size approximately 1.7 times larger than the actual captured image range 22a. As a result, the image quality is further improved and the capability of the imaging camera can be further utilized. Similarly, the capability of the projection projector can be further utilized.
It is also possible to take a larger picture in a space-saving TV conference such as when a TV conference is held only.
24 c is an arc-shaped image corresponding to 90 ° of a part of the circular image 19. Here, in the conference participant B11, the actual captured image range 22b is captured at a size approximately 2.3 times the actual captured image range 22a.

It is the figure which looked at the video conference system of this invention from diagonally upward. It is the figure which showed the video conference system of this invention in the longitudinal cross-section. It is a figure which shows the identification mark of the partition wall board which concerns on this invention in detail. It is the figure which looked at the A point of the pattern 1 which concerns on this invention from diagonally upward. It is the figure which looked at B point of the pattern 1 which concerns on this invention from diagonally upward. It is a figure which shows the state which correct | amended the display image partially for every meeting group which concerns on this invention. It is the figure which looked at A point of the pattern 2 which concerns on this invention from diagonally upward. It is the figure which looked at B point of the pattern 2 which concerns on this invention from diagonally upward. It is a figure which shows the change by the time passage of the pattern 1 and the pattern 2 which concern on this invention. It is a flowchart which shows the operation | movement procedure of the video conference system of this invention. It is a figure which shows the typical example of the conventional video conference system. It is a figure which shows the representative example which can change the form of the conventional TV conference. It is the figure which looked at the B point of the pattern 3 from diagonally upward. It is a figure which shows the state which correct | amended the display image partially for every meeting group.

10: TV conference system 11 of the present invention 11: Round table 12: Omnidirectional camera 13: Angle of view indicating a shootable range 14: Omnidirectional projector 15: Angle of view indicating a projectable range 16: Cylindrical screen 17: Headset 18: Partition wall plate 19: Circular image captured by omnidirectional camera 20: Expanded image of cylindrical screen 21: Arc-shaped screen 22: Actual captured image range 23: Invalid image range 24: Arc-shaped image 30: Conventional TV conference system 31: Conventional camera 32: Conventional monitor 33: Conventional microphone 34: Conventional speaker 35: Conventional round table

Claims (5)

Photographing means for photographing a plurality of conference groups at the first point as a ring image or a partial image thereof;
First grasping means for grasping the position of the first partition that partitions the plurality of conference groups at the first point into each conference group in the image photographed by the photographing means;
A second grasping means for grasping a position of a second partition for partitioning a plurality of conference groups at the second point facing the display means at the second point into each conference group;
Correction means for correcting the image so that the position of the first partition in the image photographed by the photographing means coincides with the position of the second partition at the second point;
Display control means for displaying the image corrected by the correction means on the display means at the second point;
TV conference system which is characterized in that it has a. The correction unit is configured such that an image of the conference group at the first point taken by the photographing unit is displayed on the display unit so as to face the conference group at the second point corresponding to the conference group. 2. The video conference system according to claim 1, wherein the adjacent order of the images of each conference group in the image captured by the image capturing unit is corrected. Claims, characterized in that the chromatic and conference group at the first point, a communication means for communicating via a dedicated voice line speech between the conference group in the second point corresponding to the conference group 1. The video conference system according to 1. An imaging step in which the imaging means captures a plurality of conference groups at the first point as a ring image or a partial image thereof;
A first grasping step in which the first grasping means grasps the position of the first partition that partitions the plurality of conference groups at the first point into each conference group in the image photographed in the photographing step;
A second grasping step in which the second grasping means grasps the position of the second partition for partitioning the plurality of conference groups at the second point facing the display means at the second point into the respective conference groups;
A correcting step for correcting the image so that the position of the first partition in the image captured in the capturing step coincides with the position of the second partition at the second point;
A display control step for causing the display means to display the image corrected in the correction step on the display means at the second point;
A control method for a TV conference system, comprising: The photographing means takes a picture so that the image of the conference group at the first point taken in the photographing step is displayed on the display means facing the conference group at the second point corresponding to the conference group. 5. The control method for a TV conference system according to claim 4, wherein the adjacent order of the images of each conference group in the selected image is corrected in the correction step.

Images