JP4513667B2 - VIDEO INFORMATION INPUT / DISPLAY METHOD AND DEVICE, PROGRAM, AND STORAGE MEDIUM CONTAINING PROGRAM - Google Patents

Description

  The present invention relates to a video information input / display method and apparatus, a program, and a storage medium storing the program, and in particular, in a communication method via video, information is input to a video that a user is watching in real time. The present invention relates to a video information input / display method and apparatus, a program, and a storage medium storing the program, which can be easily input to a desired place and can easily input information even when the video is viewed in non-real time.

  In recent years, a communication method using video has attracted attention in communication between users performed on a network.

  For example, there is a technique in which a plurality of users watch a specific TV program (for example, a soccer program) at the same time, and communicate by writing comments in real time on a bulletin board on a network (hereinafter referred to as the first conventional technique). ). In this way, by using a TV broadcast receiver and a real-time communication system in combination, it is possible to communicate while sharing the excitement of a distant user at the same time as if watching a TV program with a family. The utility value is particularly high in a popular live broadcast program (for example, see Non-Patent Document 1).

  Such a method using both a television broadcast receiver and a bulletin board / chat system requires the use of two types of devices: a television broadcast receiver and an input terminal such as a personal computer. Inconvenience users, such as having to install a personal computer nearby.

  On the other hand, there is a technique for facilitating real-time communication while viewing a television broadcast by including a television broadcast receiving system in the user's terminal (hereinafter referred to as a conventional second technique). In this technique, communication is performed by transmitting information input while viewing a television broadcast video to other user terminals via a server in real time. In addition, by accumulating time information synchronized with input information and video, it is possible to output information that goes back in the past (see, for example, Patent Document 1).

  In addition, there is a communication system using video media that is broadcast / distributed in real time, not limited to television broadcasting (hereinafter referred to as a conventional third technique) (for example, see Non-Patent Document 2).

  In the system as described above that communicates while watching video in real time, it can create a sense of presence and unity that you are enjoying video with multiple people, so you can enjoy communication and video viewing more. However, since there is a restriction that the same video must be viewed at the same time, the utility value is low for those who cannot view the video.

On the other hand, in the above-described second conventional technique, not only a user who is watching in real time can receive and record a television broadcast and record input information and time information. Information sharing is also possible for viewers who view the recorded video.
JP 2001-148841 A http://scoccer.pos.to/ Atsushi Morita, Hiroyuki Kanaya, Kazushi Nishimoto, Susumu Kunifuji “TV.com $ ^ 2 $: Internet Street TV that Activates Communication,” Multimedia, Distributed, Collaboration and Mobile (DICOMO2002) Symposium Proceedings, Information Processing Society of Japan, Symposium Series, Vol. 2002, No.9, pp. 429-432, 2002

  The system using real-time video as described above is a conventional method that uses a TV broadcast receiver and a real-time communication system together, improving the video viewing environment on the Internet and improving the functions of general-purpose terminals such as personal computers. It was only combined and expanded to match. For example, the user's information browsing / input environment is not different from conventional ones. For example, input information is only listed along the input time.

  In such a conventional real-time video information browsing / input environment, video time elapses every moment. For example, when a long sentence is input, a completely different video is broadcast / distributed when the input is completed. The situation can happen easily. Therefore, although the user wants to write the moment that he / she just watched, the impression / information about the scene, the problem of losing the timing to convey the intention and information due to the time lag occurs. .

  In addition, despite the occurrence of a time lag in input, information is managed according to the input time, so information with different contents is enumerated while being mixed, resulting in poor readability and listing. . The most common situation is when a topic from a few minutes ago is suddenly presented, or a large number of users input information at a time, and it is difficult to understand at a glance what kind of information is written. The case where it ends.

  As described above, when real-time communication is performed while viewing a video, information that is appropriate for the appropriate location of the video can be obtained because a time lag occurs in the input and the information is managed according to the input time. Cannot be entered. There arises a problem that appropriate information cannot be organized and viewed at an appropriate place in the video. In the conventional second technology described above, communication is possible not only in real time but also using recorded video. However, information input in real time in the same system has the same problem, so it has been recorded. Even when viewing video, it is difficult to browse and input appropriate information.

  The present invention has been made in view of the above points, and when communicating via video on a network, it is easy to know where the comment information input is input on the video. An object of the present invention is to provide a video information input / display method and apparatus, a program, and a storage medium storing the program.

  FIG. 1 is a diagram for explaining the principle of a video information input / display method according to the present invention.

The present invention (Claim 1) is a video information input / display method in communication related to video on a network,
A video information acquisition procedure (step 1) for acquiring video information for identifying video;
This is video structured information related to the video. When it is necessary to calculate the section from the start to the end of the display or when it takes calculation time, the primary video structured information with undecided items is calculated and the video structure is calculated. Video structuring calculation procedure (step 2) to be stored in the conversion information DB;
A video structured information acquisition procedure (step 3) for acquiring primary video structured information from the video structured information DB;
A video structured information specifying procedure (step 4) for presenting the primary video structured information acquired in the video structured information acquiring procedure (step 3) on the display device and allowing the user to select the primary video structured information ;
Comment information input procedure (step 5) for allowing the user to input comment information related to the video corresponding to the selected primary video structured information ;
When comment information is input from the user, the comment information and the video structuring information stored in the comment information / video structuring information DB in association with the comment information and the primary video structuring information selected when the comment information is input. Information accumulation procedure (step 6);
Comment information acquisition that acquires comment information associated with the comment information / video structured information DB using the video information acquired in the video information acquisition procedure and the primary video structured information acquired in the video structured information acquisition procedure Procedure (step 7);
An information display procedure (step 8) for associating and displaying the comment information acquired in the comment information acquisition procedure and the primary video structured information on the display means;
Video structuring information correction procedure for changing the primary video structuring information to video structuring information and storing it in the comment information / video structuring information DB when an undecided item of primary video structuring information is confirmed; Do.

  According to the above procedure, first, video information for identifying a video is acquired, and video structured information related to the video is calculated one after another from real-time video such as television broadcast video and network distribution video. The video structured information is, for example, a section / place where a scene break or character (telop character) is displayed, a person's name or place / appearance section in the video, a spoken audio section / written in characters It is information that structures video using various units that describe things and events related to the video. As a simple example, an image is divided into several sections and structured by switching points (cut points) of cameras.

  Next, the calculated video structuring information is acquired, and when the user inputs comment information, the video structuring information for inputting the comment information is designated. As described above, the video is structured by using the video structuring information. For example, when using a cut point to consider a video as multiple continuums, the user wants to enter comment information at the location he is currently looking at, the current section, or past information Specify the corresponding section of.

  Next, the input comment information and the designated video structuring information are stored in association with each other. Since the comment information is stored in association with the video structuring information, for example, even if a new cut point is detected and the video section where the comment information has started to be input becomes a past video section, the input video section Are held in association with each other. Therefore, the user can input information at an appropriate location even when a time lag occurs in the input.

  Next, the accumulated comment information and video structuring information are acquired and presented in association with each other. As described above, since the comment information and the video structured information are associated with each other, it is possible to determine at a glance what information is input in which place of the video. For example, as the video structured information, When the various information described above is used, information input about a specific person in the video, information added about character information in the video, and the like can be organized and presented using the video structured information.

  Therefore, if a comment is input / displayed based on these procedures, the comment can be appropriately input / displayed even when viewing a video in real time, and communication between users is smoothly performed.

  The method of the claim calculates primary video structured information when calculating the video structured information in a situation where the video structured information cannot be calculated or has not been determined. By this method, for example, even if it takes time to calculate the video structured information in the video structured information calculation procedure, the presentation of the video structured information and the presentation / input of the comment information immediately using the primary video structured information. Is possible.

  The method of the claim can specify the primary video structured information in the situation where the video structured information is not calculated or determined when the video structured information is designated, and the primary video structure is specified. Accumulation information is accumulated. By this method, for example, in the video structured information calculation procedure, even when it takes time to calculate the video structured information, the user can input information immediately using the temporary video structured information, and the primary video structured It is possible to browse and input comment information using information.

  The method of the claim can arbitrarily change the calculated video structuring information, and converts the stored comment information / video structuring information according to the changed video structuring information. By this method, for example, when using temporary video structuring information or when the erroneously detected result is included in the automatically calculated video structuring information, even if the video could not be structured with an appropriate granularity, The video structured information is appropriately corrected, and information can be browsed and input in an appropriate state according to the corrected result.

  FIG. 2 is a principle configuration diagram of the present invention.

The present invention (Claim 2 ) is a video information input / display apparatus in communication related to video on a network,
A video structuring DB 3 for storing video structuring information;
Comment information / video structured information DB 8 that stores the comment information and the video structured information in which the comment information is designated in association with each other;
Video information acquisition means 1 for acquiring video information for specifying video;
This is video structured information related to the video. When it is necessary to calculate the section from the start to the end of the display or when it takes calculation time, the primary video structured information with undecided items is calculated and the video structure is calculated. Video structured information calculating means 2 stored in the structured information DB 3;
Video structured information acquisition means 4 for acquiring primary video structured information from the video structured information DB 3;
Video structured information specifying means 5 for presenting the primary video structured information acquired by the video structured information acquiring means 4 on a display device, and allowing the user to select primary video structured information ;
Comment information input means 6 for allowing the user to input comment information related to the video corresponding to the selected primary video structured information ;
When comment information is input from the user, the comment information and the structured video information stored in the comment information / video structured information DB 8 are associated with the comment information and the primary video structured information selected when the comment information is input. Information storage means 7;
A comment for acquiring comment information associated with the comment information / video structured information DB 8 using the video information acquired by the video information acquiring unit 1 and the primary video structured information acquired by the video structured information acquiring unit 4. Information acquisition means 9;
Information display means 10 for associating and displaying the comment information acquired by the comment information acquisition means 9 and the primary video structured information on the display means;
Video structuring information correction means for changing the primary video structuring information to video structuring information and storing it in the comment information / video structuring information DB when undecided items of primary video structuring information are confirmed; Have

The present invention (Claim 3 ) is a video information input / display program for causing a computer to execute the procedure of the video information input / display method according to Claim 1 .

The present invention (Claim 4 ) is a storage medium storing the video information input / display program according to Claim 3 .

  In order to achieve the above object, the present invention is characterized in that video structured information is calculated from real-time video such as television broadcast video and network distribution video. In addition, the video structured information is used to input and present comment information such as impressions about the video input by the user in association with the video structured information.

  With the above characteristics, the position where comment information is input can be specified using the video structured information. Therefore, even when a time lag occurs, information can be appropriately input at a place where information is desired to be input. Also, by presenting the comment information in association with the video structured information, it becomes possible to organize and browse the information, unlike the method using only the time information as found in the conventional method.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  In the description of the embodiment of the present invention, video (video content) is not limited to content including video information and sound information, and any content including at least image information is referred to as video.

  The comment information is not limited to character information, but includes information that can be used for communication, such as URLs pointing to contents on the Web, images expressing emotions, related documents, sounds, and music.

[First Embodiment]
The video information input / display system according to the present embodiment is used in real time on a network when viewing a television broadcast video in real time using a television broadcast receiver or the like, or using video playback software independent of this system. It is intended for viewing distributed video, and by sharing the comment information input by each user via the server while viewing the video, communication regarding the video is performed between the users. .

  In the following example, for the sake of simplification of explanation, a description will be given based on the most common communication method in which a television broadcast receiver is used to communicate using a terminal such as a personal computer while watching a television broadcast. .

  In addition, this Embodiment has shown the specific example at the time of implementing Claim 1 and Claim 5. FIG.

  FIG. 3 is a system configuration diagram according to the first embodiment of the present invention.

  The video comment input / display system shown in FIG. 1 includes a plurality of client apparatuses 100 and a server apparatus 200, and the client apparatus 100 and the server apparatus 200 are connected via a network such as the Internet, for example. The connection here refers to a state in which each terminal is logically connected, and any connection method such as a telephone line, FTTH, wireless LAN or the like can be used.

  Although there are a plurality of client devices 100, the following description will be made using one client device 100 for the sake of simplicity. In addition, it is assumed that the user who owns each client device 100 is viewing a video using a television broadcast receiver as described above.

  The server device 200 acquires video information that specifies a video to be communicated. In the case of this embodiment, since a television broadcast video is a target, a television broadcast station (channel) and a broadcast time are used as video information. As another example, EPG information, iEPG information, a G code, or the like may be used as long as a received television broadcast video can be specified. Note that unlike this embodiment, when a live distribution video on the network is assumed, a URL indicating the position of the video is used as the video information.

  The server apparatus 200 includes a video structured information calculation unit 202 that calculates video structured information from a video specified by the video information obtained by the video information acquisition unit 201. Specifically, the video structured information calculation unit 202 includes a device such as a television tuner board that converts a television broadcast video into a signal that can be processed by a computer, a device such as a memory that stores the signal, and a video from the converted signal. It consists of a device or a program for calculating structured information. Unlike the present embodiment, when targeting live distribution video on a network, a device such as a memory for temporarily storing video, a device for calculating video structuring information from the stored video, or a program Etc. In general, the video structured information calculation unit 202 includes a device or program for converting / accumulating a target video into a processable form, a device, a device / program for calculating the video structured information, and the like. As long as the structured information can be calculated, the form is not limited.

  Further, in this embodiment, for simplification, description will be made only for one channel of a television broadcast video or a network distribution video, but processing may be performed by simultaneously receiving a plurality of channels. In that case, the video information acquisition unit 201 and the video structured information calculation unit 202 exist in parallel for the number of channels, and acquire video information and calculate video structured information.

  The image structured information calculated by the image structured information calculating unit 202 includes color information on the screen, scene switching points, camera work, captured objects, characters, utterance contents, recognition results of speakers and music. This is information indicating physical or semantic characteristics on the video such as time information extracted at regular intervals, and is information for structuring the video using various units. Video structured information is information that is widely used in fields such as video management and video search, and some of the information can be obtained using known video analysis techniques. For example, scene switching points and telop display sections are described in Japanese Patent No. 2839132, Japanese Patent Laid-Open No. 9-238298, Japanese Patent Laid-Open No. 11-178007, and other feature quantities. Various studies have been conducted.

  In general, calculation / acquisition of video structuring information requires processing time for analyzing video. However, with the improvement of computer performance and the proposal and improvement of various analysis algorithms, for example, calculation of light information using physical features such as cut points that are camera switching points is delayed by current computers. It can be done without it. In the present embodiment, in consideration of such a light process in the present situation and a reduction in future calculation time, it is assumed that there is no delay in the time required for the process. A situation where a delay occurs will be described later in the second embodiment.

  The video structuring information calculation unit 202 acquires video structuring information one after another from videos broadcast / distributed in real time, and sequentially stores them in the video structuring information DB 203. FIG. 4 shows an example of storing the video structuring information in the video structuring information DB 203 when a camera switching point is used as the video structuring information. It should be noted that a state temporarily stored in a memory or the like may be used as the video structured information DB 203 without using a DB implemented as software or a device.

  In the example in the present embodiment, when the calculated time on the video of the switching point of the camera is tn, an image representing a section divided by each tn is calculated as the representative image n. The representative image is also one piece of information that is effective when a structured video is presented in an easy-to-understand manner. As the simplest calculation method of the representative image n, there is a method of acquiring a still image at tn which is the start time of each section, but a method of creating a panoramic representative image from the movement of the camera in the section, There is also a method of acquiring a representative image from the time when the face is shown on the screen using the face recognition technology. These methods are not particularly limited, and a representative image is acquired using a known technique.

  As shown in FIG. 4, each video structured information acquired from the video A has an identification ID for uniquely identifying, a video identifier consisting of the video information acquired by the video information acquiring unit 201, and video structured information. The video time and the representative image ID for identifying the representative image of each section are defined, and the video structuring information is uniquely specified by the information.

  By using the above means, on the server device 200, video structured information is calculated and accumulated one after another from videos broadcast / distributed in real time.

  The client device 100 includes a video information acquisition unit 101 that acquires video information for specifying a video to be communicated. In the case of the present embodiment, since the user views the video using a television broadcast receiver or the like, the video information acquisition unit 101 broadcasts with the television broadcast station (channel) of the television broadcast video being viewed, for example. It is assumed that the time and the like are manually input or selected from a TV program guide prepared separately and acquired. In the video information acquisition unit 101, the video information acquired needs to be matched with the video information acquired in the video information acquisition unit 201 on the server device 200. Since it is only necessary to be able to identify videos with the same information, when handling different information (for example, iEPG information in the client device 100 and G code in the server device 200), implicit conversion such as converting both into the same video information. Processing may be included.

  The client device 100 includes a video structured information acquisition unit 102 that acquires the video structured information calculated by the video structured information calculation unit 202. Specifically, the video structuring information acquisition unit 102 uses the video information acquired by the video information acquisition unit 101 to inquire the video structuring information DB 203, and the video structuring information about the video specified by the video information. Get information. In the case of FIG. 4 as an example, the video structured information can be easily obtained by using the video identifier in each video structured information.

  The client device 100 includes a video structured information designating unit 103 that designates video structured information, which is an input destination of comment information, when the user wants to input comment information such as an impression on the video. FIG. 5 shows an example of the method for specifying the video structured information in the video structured information specifying unit 103.

  FIG. 5 is an example in the case of using the video structuring information shown in FIG. In addition, a change with time of display in the video structured information specifying unit 103 is shown. As the video A is viewed, the video structured information is calculated by the video structured information calculating unit 202 one after another. Therefore, when viewing the time ta on the video, the video structuring information calculated so far is updated, and when viewing the time tb, the newly calculated video structuring information is successively added. It will be presented. By presenting the video structuring information calculated in this way one after another, the user can not only enter comment information for the video section currently being viewed, but also in the previous or previous section. You can also enter comment information. The user selects video information structured information for inputting comment information by selecting one of the presented video structured information (pressing a “comment input” button in FIG. 5).

  The client apparatus 100 includes a comment information input unit 104 that allows the user to input comment information for the video structured information specified by the video structured information specifying unit 103. A screen example of the comment information input unit 104 is shown in FIG. The comment information input unit 104 presents a window as shown in FIG. 6 and prompts the user to input comment information when the video structured information is specified by the user in the video structured information specifying unit 103. By using such an interface, the user can input comments such as impressions on the specified structured video information, so that the video content changes over time as shown in FIG. Even in the event of a failure, comment information can be continuously input to an appropriate video section specified by the video structuring information.

  The client device 100 includes a comment information / video structured information storage unit 105 that stores the video structured information specified by the video structured information specifying unit 103 and the comment information input from the comment information input unit 104 in association with each other. . In the case of the present embodiment, the comment information and the video structuring information are immediately stored in the comment information / video structuring information DB 204 on the server device 200 when the input is confirmed in the comment information input unit 104 in FIG. A specific example of the comment information / video structured information accumulation DB 204 in this embodiment is shown in FIG.

  In the case of the present embodiment, as the video structured information stored in the comment information / video structured information storage DB 204, an identification ID serving as a unique identifier for identifying each video structured information in the video structured information DB 203 is used. . As the comment information, the user name of the user who input the comment information, the input date and time, and the input comment are used. The comment information / video structured information DB 204 accumulates the information in association with each other as shown in FIG. 8 so that it can be easily obtained what comment information is assigned to which video structured information. Become.

  The client apparatus 100 uses the video information acquired by the video information acquisition unit 101 and the video structured information acquired by the video structured information acquisition unit 102 to acquire related comment information from the comment information / video structured information DB 204. A comment information acquisition unit 106 for performing In the case of the present embodiment, as shown in FIG. 8, all the comment information input to the corresponding video is acquired by referring to the identification ID of the acquired video structured information.

  The client apparatus 100 includes an information display unit 107 that presents the comment information acquired by the comment information acquisition unit 106 in association with the acquired video structured information. In the case of the present embodiment, the image structured information specifying unit 103 and the display unit are used together and presented as shown in FIG.

  On the left side of FIG. 9, the information display unit 107 displays the video structured information acquired at a certain time ta and the comment information input at that time in association with each video structured information. In addition, on the right side of FIG. 9, new comment information input between time ta and tb is additionally displayed together with video structured information newly acquired at a certain time tb.

  In this way, the information display unit 107 displays the comment information in association with each video structured information, so that the comment information can be appropriately presented to the information input destination even when there is a time lag in the input. At the same time, by displaying together with the video structured information, it is possible to easily understand which comment information is attached to which video portion.

  According to the present embodiment, by using the above-described means, it becomes possible to input comment information to an appropriate place using video structured information calculated one after another from the real-time video being viewed. By presenting it in association with the video structured information, it is possible to easily grasp where each comment information is input on the video. Therefore, unlike conventional information input / display methods that present comment information one after another along the input time, information can be input / displayed appropriately even when viewing video in real time, and communication between users is Performed smoothly.

[Second Embodiment]
This embodiment is intended for TV broadcast video broadcast in real time, live distribution video on a network, and the like. Unlike the first embodiment, it is for video viewing prepared in this system. Communication is performed while viewing video using a device or a program.

  In the following example, for simplification of description, a description will be given assuming that communication is performed on a live distribution video on a network. In this Embodiment, the specific example at the time of implementing Claims 1-4 and 5-8 is shown.

  FIG. 10 is a system configuration diagram according to the second embodiment of the present invention.

  The system shown in FIG. 1 includes a plurality of client devices 100, a server device 200, and a live video distribution server 300 that distributes real-time live video. The client device 100 and each server device 200 are, for example, the Internet. Connected via network. Although there are a plurality of client apparatuses 100, the following description will be given using one client apparatus 100 for simplification of description.

  In the following, the same components as those in FIG.

  The live video distribution server 300 includes a video distribution unit 301 that distributes live video from a camera or recorded video in real time through a network. The connection destination of the video distributed from the video distribution unit 301 can be specified by a URL or the like, and the user can view the video in real time by inputting the information and connecting to the video reproduction program or the like. In the present embodiment, live distribution on the network is assumed. However, when a television broadcast image is targeted, the live image distribution server device 300 can be replaced with a broadcast station.

  The client device 100 includes a video information acquisition unit 101 that acquires video information for specifying a video to be communicated. As the video information for specifying the live distribution video in the present embodiment, for example, a URL, an identification ID, a video file name uniquely determined by the video distribution source, or information that combines them is used.

  The client device 100 includes a video structured information calculation unit 108 that calculates video structured information from a video specified by the video information obtained by the video information acquisition unit 101. Since the specific video structured information calculation unit 108 is already the same as the video structured information calculation unit 202 of the first embodiment, detailed description thereof is omitted. However, in the case of the present embodiment, since the video structured information is calculated in the client device 100, it is assumed that the calculation takes time depending on the video structured information to be used and the performance of the client device. Therefore, the following explanation is based on the assumption that a time lag occurs in the calculation of the video structuring information in the video structuring information calculation unit 108. Also, the video structured information calculated by the client device 100 is different. That is, a situation is assumed in which video structured information calculated by a user is distributed and processed, such as a client device that calculates a camera switching point, a client device that recognizes an object, and the like. In the first embodiment described above, an example is shown in which all video structured information is calculated on the server device 200 side, but the video structured information is calculated on the client device 100 side as in the present embodiment. Thus, communication using various video structured information other than the video structured information calculated on the server device 200 side becomes possible.

  Depending on the client device 100, the video structured information may not be calculated. In that case, the video structured information calculation unit 108 does not store anything in the video structured information DB 203.

  The server apparatus 200 includes a video structured information DB 203 that stores the video structured information calculated by the video structured information calculating unit 108. An example of the stored video structured information is shown in FIG.

  In the present embodiment, unlike the first embodiment, by recognizing not only the camera switching point but also various information in the video, for example, an object or a human face reflected in the video, In-image character information called telop characters is also extracted. As shown in FIG. 11, the video structuring information includes a video identifier for identifying a video to be calculated (consisting of video information as in FIG. 4), a calculated start time / end time, a representative image, and the like. For example, in the case of face recognition, it is stored together with auxiliary information such as the position of the recognized face area on the screen and the recognized person name (in FIG. 11, it is shown in a simplified manner as “auxiliary information”). ing). Note that these pieces of information can be appropriately corrected by the video structured information correction unit 206 included in the server device 200. The video structured information correction unit 206 will be described later.

  Here, as a feature relating to claims 2 and 6, the video structured information calculation unit 108 can calculate primary video structured information. Here, the primary video structured information is information that requires calculation of the section from the start to the end of the display, such as characters displayed on the screen, or video that requires calculation time, such as facial image recognition. Since structured information exists, it refers to video structured information that is primarily calculated and used for such video structured information.

  When the corresponding video structuring information is determined after the section is determined or after the calculation is completed, the primary video structuring information is replaced with the determined video structuring information. For example, as shown in FIG. 14, when the position information on the screen input by the user is the primary video structured information and the area information corresponding to the position information is confirmed as shown in FIG. 15, the position information is converted into the area information. Replaced.

  The information described above includes information that can be calculated immediately at a certain time, such as a camera switching point, and information that needs to be calculated from the display start to the end, such as characters displayed on the screen. There are those that require calculation time, such as face image recognition. The video structured information calculation unit 108 can view and input comment information immediately by calculating the primary video structured information for the video structured information that takes time to calculate and confirm, such as the latter. And An example of primary video structuring information is shown in FIG. In FIG. 12, for the object and telop character that are not fixed at the time ta, the primary video structured information is calculated by setting the undetermined item as “*”, and is stored in the video structured information DB 203. Specific examples of the effect using the primary video structured information will be described in the description of the video structured designating unit 103 and the information display unit 107.

  When the video structuring information is calculated / confirmed, the video structuring calculation unit 108 immediately updates the corresponding information in the video structuring information DB 203.

  The client device 100 includes a video structured information acquisition unit 102 that acquires the video structured information calculated by the video structured information calculation unit 108. Specifically, the video structured information acquisition unit 102 makes an inquiry to the video structured information DB 203 of the server device 200 using the video information acquired by the video information acquisition unit 101, and sets the video specified by the video information. Get related video structured information. Specifically, when FIG. 11 is taken as an example, video structured information in which the video identifier is determined to be the same as the video information is acquired using the video identifier in each video structured information. However, in the case of having the features related to claims 2, 3, 6 and 7, the acquired video structured information includes primary video structured information.

  The client apparatus 100 includes a video structured information specifying unit 103 that allows the user to specify video structured information that is an input destination of comment information when inputting comment information such as an impression of the video. FIG. 13 shows an example of the method for specifying the video structured information in the video structured information specifying unit 103. Similarly to the method described in FIG. 5, video structured information calculated in accordance with video viewing is presented one after another (showing the state at time ta in FIG. 12), and each video structured information can be designated. Prepare buttons. As described above, in this embodiment, the video structured information calculation unit 108 can acquire primary video structured information. In FIG. 13, although there are undecided items at the stage of time ta in the object and telop characters, the video structured information can be specified by using the primary video structured information.

  Here, as a feature related to claims 3 and 7, the video structured information designating unit 103 can designate the primary video structured information. The video structured information includes information that requires calculation time until recognition, such as facial image recognition. Therefore, when viewing a video in real time, the primary video structured information by the video structured information calculation unit 108 is used. There is a situation where the calculation of is not in time for the user's input. The video structured information specifying unit 103 also avoids such a problem by making the primary video structured information available like the video structured information calculating unit 108. FIG. 14 shows a method of specifying primary video structured information by the video structured information specifying unit 103.

  By specifying the information in the video using a mouse cursor or the like for the video being played back and providing means for selecting the type, it is possible to specify the primary video structured information as shown in FIG. More specifically, the start time is the time when a certain place in the video is specified by the mouse cursor, the type is the type selected by the selection means, the auxiliary information is the position on the screen of the mouse cursor, and the representative image is the start time A still image at the time specified as can be specified.

  By providing a means for specifying such primary video structured information, the video structured information designating unit 108 can immediately use the primary video structured information for the video structured information to be calculated. Allow comment information to be entered.

  The client device 100 includes a comment information input unit 104 that inputs comment information for the video structured information specified by the video structured information specifying unit 103. Since the comment information input unit 104 has been described with reference to FIG. 6 of the first embodiment, description thereof will be omitted.

  The client device 100 associates the video structured information specified by the user in the video structured information specifying unit 103 and the comment information input by the user in the comment information input unit 104 with the comment information / video structured information DB 204. It has a comment information / video structured information storage unit 105 to be stored. Since the information accumulated by the comment information / video structured information accumulation unit 105 has already been described with reference to FIG. 8 of the first embodiment, a detailed description thereof will be omitted.

  In addition, when it has the characteristics regarding Claims 3 and 7, the primary video structured information specified by the video structured information specifying unit 103 is stored in the video structured information DB 203.

  The client apparatus 100 uses the video information acquired by the video information acquisition unit 101 and the video structured information acquired by the video structure information acquiring unit 108 to acquire related comment information from the comment information / video structured information DB 204. A comment information acquisition unit 106 is included. In the case of the present embodiment, as shown in FIG. 8, the comment information input to the corresponding video is acquired by referring to the identification ID of the acquired video structured information. Further, only the comment information associated with the specific video structured information can be acquired using the identification ID of the video structured information.

  The client device 100 includes an information display unit 107 that presents the comment information acquired by the comment information acquisition unit 106 in association with the acquired video structured information. The information display unit 107 has already been described with reference to FIG. However, in the case of having features related to claims 2, 3, 6, and 7, comment information can be browsed using primary video structured information even for video structured information that has not been calculated or determined.

  The client device 100 includes a video reproduction unit 109 that views a video broadcast / distributed from the video distribution unit 301. In the case where the features relating to claims 3 and 7 are provided, the video reproduction unit 109 can be used for designating the primary video structured information in the video structured information designating unit 103 as shown in FIG.

  In the present invention, when the video structured information is changed, the comment information / video structured information storage unit 105 changes the comment information / video structured information when the video structured information is changed. The comment information / video structured information DB 204 of the server device 200 is stored. Therefore, in the present embodiment, the server apparatus 200 includes the video structured information correction unit 206 that performs such processing.

  The video structured information correcting unit 206 immediately corrects the related information in the comment information / video structured information DB 204 when the content of the video structured information DB 203 is changed. For example, in the case where the features related to claims 2 and 6 are included, the video structured information DB 203 stores the primary image structured information calculated by the image structured information calculating unit 108, and the image structured information calculating unit 108. Since the video structuring information is updated when the video structuring information is confirmed, the information is corrected accordingly. In the case of the present embodiment, as shown in FIG. 8, the comment information / video structured information DB 204 holds the video structured information as an identification ID, so there is no need to modify it, but depending on the embodiment, Since there is a possibility that information other than the identification ID may be held, correction is performed when the corresponding information is updated.

  When the video structured information correction unit 206 has the characteristics related to claims 3 and 7, the video structured information calculation unit 108 performs the video structured information calculation unit 108 on the primary video structured information specified by the user by the video structured information specifying unit 103. The calculated video structured information is compared, and when it is determined that they are the same video structured information, they are integrated and the corresponding comment information / video structured information is corrected. FIG. 15 shows an example of a method for comparing the primary video structured information and the calculated video structured information.

  FIG. 15 compares the primary video structured information x designated by the user by the video structured information designating unit 103 in FIG. 14 and the video structured information y calculated by the video structured information calculating unit 108. . In order to determine whether or not both indicate the same video structured information, first, after comparing that the types are the same (face), the start time at x (the time when the primary video structured information is designated) Are included in the start and end times in y. If included, the face area (region tf0) in the auxiliary information of y is compared to determine whether or not the information designation position (position i0) of x is included. It is determined that it is pointing. If they are determined to be the same, since x is primary video structured information, it is discarded from the video structured information DB 203 and comment information / video structured with reference to the identification IDx in the comment information / video structured information DB 204 Rewrite the identification ID to y for all the information. By taking such procedures into account, the primary video structuring information and the calculated video structuring information are not duplicated, and the comment information can be browsed and input correctly.

  The video structured information correction unit 206 can also be used when the video structured information DB 203 is corrected by using human hands or the like. When the start / end time is different from the actual time due to erroneous detection, or when viewing / inputting comments becomes troublesome because the camera switching point has been recognized too finely, the items in the video structured information DB 203 are appropriately set. By correcting and integrating / adding / deleting some video structured information, and modifying information in the comment information / video structured information DB 204, information can be browsed / input in an appropriate state. . Also, by making such correction possible, it is possible to improve the viewability of the video structured information and comment information, so that the information can be used effectively even after real-time communication. For example, it can be used when recording / accumulating live distribution video and communicating using the same system at a later date.

  By using the means as described above, according to the present embodiment, the method according to claims 1 to 4 and the apparatus according to claims 5 to 8 are used to calculate one after another from the real-time video being viewed. By using the structured video information, comment information can be input at an appropriate location, and information can be immediately viewed and input even when it takes time to calculate the structured video information. In addition, since the information can be viewed and input appropriately even when the video structured information is modified, the information can be input and displayed properly even when viewing the video in real time. Performed smoothly.

  In addition, the operations in the first embodiment and the second embodiment described above can be constructed as a program and installed in a computer or distributed via a network.

  Note that the video information acquisition unit 101, the video structured information acquisition unit 102, the video structured information designation unit 103, the comment information input unit 104, the comment information / video structured information storage unit of the client device 100 according to the first embodiment. Each operation of the comment information acquisition unit 106 and the information display unit 107 can be constructed as a program for the client device, and can be installed and executed on a computer used as the client device.

  In addition, the operations of the video information acquisition unit 201 and the video structured information calculation unit 202 of the server device 200 can be constructed as a server device program, and can be installed and executed on a computer used as the server device.

  Also, the video information acquisition unit 101, the video structured information calculation unit 108, the video structured information acquisition unit 102, the video structured information designation unit 103, the comment information input unit 104, the comment of the client device 100 according to the second embodiment. Each operation of the information / video structured information storage unit 105, the comment information acquisition unit 106, the information display unit 107, and the video playback unit 109 is constructed as a program for the client device, and is installed and executed on a computer used as the client device It is possible to make it.

  Further, the operation of the video structured information correction unit 206 of the server device 200 can be constructed as a program for the server device, and can be installed and executed on a computer used as the server device.

  Further, the constructed program can be stored in a portable storage medium such as a hard disk device, a flexible disk, or a CD-ROM, and installed in a computer to be executed or distributed.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention is applicable to a system that uses a television broadcast receiver and a real-time communication system in combination, or a system that sends information such as comments in real time to other user terminals via a server.

It is principle explanatory drawing of the video information input and display method of this invention. It is a principle block diagram of the video information input / display apparatus of this invention. It is a system configuration figure in a 1st embodiment of the present invention. It is an example of accumulation of picture structure information in a 1st embodiment of the present invention. It is an example of the designation | designated method of the video structure information in the video structure information designation | designated part in the 1st Embodiment of this invention. It is an example of a screen of the comment information input part in the 1st Embodiment of this invention. It is a relationship diagram between the change of the image content with time passage and the comment information input unit in the first embodiment of the present invention. It is a specific example of the comment information / video structured information storage DB in the first exemplary embodiment of the present invention. It is an example of the screen of the information display part in the 1st Embodiment of this invention. It is a system configuration figure in a 2nd embodiment of the present invention. It is an example of the video structure information accumulate | stored in the video structure information DB in the 2nd Embodiment of this invention. It is an example of the primary image structure information in the image structure information calculation part in the 2nd Embodiment of this invention. It is an example of the designation | designated method of the video structure information of the video structure information designation | designated part in the 2nd Embodiment of this invention. It is the designation | designated method of primary image | video structured information by the image | video structured information designation | designated part in the 2nd Embodiment of this invention. It is an example of the comparison method of the primary image structure information and the calculated image structure information in the second embodiment of the present invention.

Explanation of symbols

1 Video information acquisition means 2 Video structured information calculation means 3 Video structured information DB
4 video structured information acquisition means 5 video structured information designation means 6 comment information input means 7 comment information / video structured information storage means 8 comment information / video structured information DB
9 Comment Information Acquisition Unit 10 Information Display Unit 100 Client Device 101 Video Information Acquisition Unit 102 Video Structured Information Acquisition Unit 103 Video Structured Information Designation Unit 104 Comment Information Input Unit 105 Comment Information / Video Structured Information Storage Unit 106 Comment Information Acquisition Unit 107 information display unit 108 video structured information calculation unit 109 video playback unit 200 server device 201 video information acquisition unit 202 video structured information calculation unit 203 video structured information DB
204 Comment Information / Video Structured Information DB
206 Video Structured Information Correction Unit 300 Live Video Distribution Server Device 301 Video Distribution Unit

Claims (4)

A video information input / display method in communication related to video on a network,
Video information acquisition procedure for acquiring video information for identifying video;
The video structuring information related to the video , and when it is necessary to calculate the section from the start to the end of the display or when calculation time is required, the primary video structuring information in which the undetermined items are set is calculated, and the video A video structuring calculation procedure stored in the structured information DB;
A video structured information acquisition procedure for acquiring primary video structured information from the video structured information DB;
A video structured information designation procedure for presenting the primary video structured information acquired in the video structured information acquisition procedure on a display device and allowing a user to select primary video structured information ;
Comment information input procedure for allowing the user to input comment information related to the video corresponding to the selected primary video structured information ;
When comment information is input from the user, the comment information and video structure stored in the comment information / video structured information DB in association with the comment information and the primary video structured information selected when the comment information is input Information storage procedure,
Using the video information acquired in the video information acquisition procedure and the primary video structured information acquired in the video structured information acquisition procedure, comment information associated from the comment information / video structured information DB is obtained. The comment information acquisition procedure to be acquired,
An information display procedure for displaying the comment information acquired in the comment information acquisition procedure and the primary video structured information in association with each other on the display means;
Video structuring information correction procedure for changing the primary video structuring information to video structuring information and storing it in the comment information / video structuring information DB when undecided items of the primary video structuring information are confirmed; ,
A video information input / display method characterized by: A video information input / display device in communication related to video on a network,
A video structuring DB for storing video structuring information;
A comment information / video structured information DB for associating and storing comment information and video structured information in which the comment information is designated;
Video information acquisition means for acquiring video information for identifying video;
The video structuring information related to the video , and when it is necessary to calculate the section from the display start to the end, or when calculation time is required, calculate the primary video structuring information in which undetermined items are set , Video structured information calculating means for storing in the video structured information DB;
Video structured information acquisition means for acquiring primary video structured information from the video structured information DB;
Video structured information specifying means for presenting the primary video structured information acquired by the video structured information acquiring means on a display device, and allowing a user to select primary video structured information ;
Comment information input means for allowing the user to input comment information related to the video corresponding to the selected primary video structured information ;
When comment information is input from the user, the comment information and video stored in the comment information / video structured information DB in association with the comment information and the primary video structured information selected when the comment information is input. Structured information storage means;
Using the video information acquired by the video information acquisition means and the primary video structured information acquired by the video structured information acquisition means, comment information associated with the comment information / video structured information DB is obtained. Comment information acquisition means to acquire;
Information display means for associating and displaying the comment information acquired by the comment information acquisition means and the primary video structured information on the display means;
Video structured information correcting means for changing the primary video structured information to video structured information and storing it in the comment information / video structured information DB when an undecided item of the primary video structured information is confirmed; ,
A video information input / display device characterized by comprising: A video information input / display program which causes a computer to execute the procedure of the video information input / display method according to claim 1 . 4. A storage medium storing a video information input / display program according to claim 3, wherein said video information input / display program is stored.

Images