JP2000244889A - Method and system for multimedia communication control and recording medium where program thereof is recorded - Google Patents

Abstract

PROBLEM TO BE SOLVED: To synchronize alteration of attached information with the reproduction of video and voice by distributing and receiving the attached information separately from video and voice information and allowing a server device and a client device to inform each other of the distribution and alteration of the attached information. SOLUTION: When the client device 20 sends a video and voice information distribution request to the server device 10, the server device 10 distributes both the video information and attached information to the client device 20 before the client device 20 begins to reproduce video and voice. The attached information is distributed periodically from the server device 10 to the client device 20, which receives the attached information periodically. Consequently, if the attached information is altered, the client device 20 obtains the altered attached information from the server device 10 and can synchronize the altered attached information with the reproduction of the video and voice on itself.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multimedia communication control method for distributing and reproducing video and audio information on a communication network, and a system therefor.

[0002]

2. Description of the Related Art At present, in a technology for distributing or reproducing video / audio information on a network, for video / audio to be distributed / reproduced, setting of a link anchor between a plurality of video / audio, video / audio. There is a method for specifying a change in the setting of the anchor with the lapse of time from the start of reproduction of a file and a method of using the functions of other programs or devices.

[0003] As the method, there are a method represented by WIL Consortium (W3C) standard SMIL for controlling a plurality of images and sounds in an image and sound distribution / reproduction technology on a network, and an HTM displayed on a WWW browser.
In L, there is a control method using a programming language typified by JavaScript or CGI (Common Gateway Interface) for controlling a plurality of images and sounds.

[0004] In the above method, a method of storing information accompanying video / audio information (hereinafter referred to as “attached information”), such as a method of setting a hyperlink for video / audio to be distributed / reproduced or a method of using another device, If the processing method in the distribution method is shown, there is no difference between the processing method of the designated information of the video / audio to be distributed / reproduced and the processing method of the accompanying information.

For example, in SMIL, the designation information of video / audio to be distributed / reproduced and the accompanying information are described in the same place. In JavaScript and CGI, as a database referred to by a program, information for specifying video / audio to be distributed / reproduced and accompanying information are described as the same information in the same program.

[0006]

As described above, in the method of reproducing video / audio information and the method of handling accompanying information, when reproducing video and audio, especially when broadcasting live broadcasts or radio broadcasts over a network, However, the anchor information of the link cannot be changed in the client device.

For example, in SMIL, a client device obtains information on all accompanying information from a server device at the start of video playback. Therefore, when the accompanying information is changed after the start of the reproduction, it is not possible to notify the client apparatus that has already acquired the accompanying information from the server apparatus and reproduces the video and audio of the change.

In JavaScript, since the WWW browser acquires the contents of all accompanying information when acquiring HTML information, it is not possible to notify the client device of the changed contents of the accompanying information for the same reason.

When CGI is used, WWW
By specifying the CGI program as the link destination in the display content of the browser, it was possible to notify the client device of the change in the accompanying information. Synchronization with the timing of video and audio playback was not achieved.

SUMMARY OF THE INVENTION The present invention solves the problem that occurs when video and audio information is distributed over a communication network using conventional techniques, and that the accompanying information cannot be changed during video and audio reproduction. It is an object of the present invention to notify a client device of a change in incidental information even during reproduction of video and audio, and to synchronize the reproduction of video and audio with the change of incidental information.

[0011]

In order to achieve the above object, the present invention provides a video / audio distribution method for distributing and reproducing video / audio information from a server device to a client device over a communication network. It distributes and receives additional information separately from video and audio information, and can distribute the additional information and notify changes between the server and client even during playback of video and audio. I do.

In order to enable distribution and modification of accompanying information between the server device and the client device even during the reproduction of video and audio as described above, for example, means having the following functions are provided by the server. Provided on the device and the client device.

In the server device, accompanying information including information on how to supplement the contents of the video / audio information by using the function of the external device is attached to the video / audio information. Means for storing and storing ancillary information, means for receiving an accompanying information delivery request from a client device, means for delivering ancillary information, management of the accompanying information, and management of the accompanying information. An additional information distribution control unit for managing distribution is provided.

The client device has an additional information distribution request transmitting means for requesting the server device to distribute additional information based on a user's request, and a video / audio reproduction position when the additional information distribution request is transmitted. Video / audio playback position extracting means for acquiring the accompanying information, accompanying information receiving means for receiving the accompanying information distributed from the server device, decoding the received accompanying information, and determining the corresponding video / audio playback position of the accompanying information. Information control means for synchronizing with the reproduced video / audio position being reproduced, auxiliary information display means for displaying the received auxiliary information, input of a distribution request from the user of the auxiliary information, and auxiliary information display means And an external device control unit for activating the external device based on an input from the auxiliary information operation unit.

The operation of the present invention is as follows. In the server device and the client device having the above-mentioned means, the accompanying information distribution request transmitting means of the client device and the accompanying information distribution request receiving means of the server device perform the distribution of the accompanying information independently of the video / audio information distribution request. The request is transmitted from the client device to the server device. In the server device, the accompanying information accumulating means, the accompanying information distribution control means, and the accompanying information distribution means perform the accompanying information distribution processing, thereby distributing the accompanying information independently of the video / audio information distribution processing. I have.

On the other hand, in the client device, by providing the accompanying information receiving means and the accompanying information control means, the receiving processing of the accompanying information is performed independently of the receiving processing of the video / audio information. By providing means, associated information operating means, and external device control means, an external device can be used independently of video / audio reproduction.

Therefore, according to the present invention, it is possible to distribute or receive the accompanying information independently of the process of distributing or reproducing the video / audio information, and the server device and the client device can distribute the video / audio information. Alternatively, even during the reproduction, it is possible to notify the client device of the change of the accompanying information, and the client device can synchronize the reproduction of the video / audio and the change of the accompanying information.

A program for realizing each processing means of the server device and the client device by a computer can be stored in an appropriate recording medium such as a portable medium memory, a semiconductor memory, and a hard disk which can be read by the computer. .

[0019]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram illustrating a configuration example of a server device according to the present invention, and FIG. 2 is a diagram illustrating a configuration example of a client device according to the present invention. Server device 1
0 and the client device 20 are connected to the communication network 10
0 is connected. Generally, client device 2
0 is connected to a plurality of communication networks 100.

As shown in FIG. 1, the server device 10 comprises a video / audio information storage unit 101 for storing video / audio information.
A video / audio information distribution control means 102 for controlling the distribution of video / audio information; a video / audio information distribution request receiving means 103 for receiving a video / audio information distribution request from the client device 20; A video / audio information distributing means 104 for distributing the audio information; and an external device 30 for supplementing the content of the video / audio information by using the function of the external device 30 (not shown) accompanying the video / audio information. Additional information, including information on how to use
Ancillary information storage means 111 for accumulating and holding each part of video / audio information, ancillary information distribution control means 112 for managing the accumulated ancillary information and managing the distribution of the ancillary information, and ancillary information from the client device 20 The apparatus includes an accompanying information delivery request receiving unit 113 for receiving a delivery request and an accompanying information delivery unit 114 for delivering accompanying information. The above video
Instead of the audio information storage means 101, or in addition to the video / audio information storage means 101, video / audio information input means from a camera or the like can be used.

Here, the use of the function of the external device 30 is not necessarily limited to the one described below. For example, during reproduction of a certain video or audio, identification of a part related to the part being reproduced is performed. Use the Internet phone of the other party, access a specific homepage with a WWW browser, or use a specific FTP (File Transfer Protocol).
ocol) Accessing a server to transfer or acquire a file.

As shown in FIG. 2, the client device 20
Controls playback video / audio output means 201 for outputting the reproduced video / audio information, video / audio information operation means 202 for operating the distributed video / audio information, and controls the reproduction of the distributed video / audio information Playback control means 203 for
A video / audio information distribution request transmitting unit 204 for transmitting the audio / video information distribution request to the server device 10; a video / audio information receiving unit 205 for receiving the distributed video / audio information; When an information acquisition request is issued, a video / audio reproduction position extraction unit 206 for acquiring a video / audio reproduction position, and an external device used for inputting a distribution request for accompanying information from a user and using the accompanying information display unit 212 30; an accompanying information display means 212 for displaying the received accompanying information; decoding the received attached information; and a video / audio playback position corresponding to the attached information and an actual playback control means 203. Information control means 21 for synchronizing with the video / audio reproduction position being reproduced by
3 and an accompanying information distribution request transmitting unit 214 for requesting the server device 10 to distribute the accompanying information based on the user's request.
An accompanying information receiving unit 215 for receiving the accompanying information distributed from the server device 10;
External device control means 216 for activating the external device 30 based on the input from the external device.

For example, as accompanying information accompanying the part of video / audio information being reproduced, (a) destination information when using an Internet telephone for an external device, and (b) WW for an external device.
When the connection destination information when using the W browser and (c) the connection destination information when using the FTP server are set in the external device, the accompanying information operation unit 211 inputs a use request of the accompanying information from the user. Then, the accompanying information display means 21
2, based on the accompanying information at that time, according to the notification from the accompanying information operating means 211, as available external devices, (a) an Internet telephone, (b) a WWW browser,
(c) Display that there is an FTP server. When the user selects an external device using the accompanying information operating means 211, the accompanying information and the user's selection information are notified to the external device control means 216, and the external device control means 216 is notified.
Calls the external device 30 selected by the user based on the notified accompanying information.

Server device 10 and client device 2
As means for distributing video and audio information in
For example, the following (1) to (3) can be implemented.

(1) The video / audio information distribution request transmitting means 204 of the client device 20 transmits a video / audio information distribution request to the server device 10. Server device 10
Before the reproduction control means 203 of the client device 20 starts reproduction of the video / audio information, the video / audio information distribution means 104 of the client device 2 transmits all the information of the video / audio information to the client device 2.
Deliver to 0.

(2) The video / audio information distribution request transmitting means 204 of the client device 20 transmits a video / audio information distribution request to the server device 10. Reproduction control means 2
In step 03, when the server device 10 starts distributing the video / audio information, the client device 20 receives a part of the video / audio information and starts reproducing the video / audio at the same time.

(3) The video / audio information distribution request transmitting means 204 of the client device 20 transmits a video / audio information distribution request to the server device 10. Server device 10
The video / audio information distribution means 104 distributes video / audio information of the same content to one or a plurality of client devices 20 that have issued a video / audio information distribution request to the server device 10.

As means relating to distribution of accompanying information in the server device 10 and the client device 20,
For example, the following (1) to (5) can be implemented.

(1) The accompanying information distribution request transmitting means 214 of the client device 20 transmits a distribution request of the accompanying information to the server device 10 at the start of the reproduction of the video / audio information. Upon receiving the accompanying information distribution request, the accompanying information distribution control unit 112 and the accompanying information distribution unit 114 of the server device 10 transmit the necessary accompanying information to the client device 20 before the client device 20 starts reproducing the video and audio. To deliver.

(2) The associated information distribution control means 112 and the associated information distribution means 114 of the server device 10 continue to distribute the associated information at regular intervals. Further, the accompanying information control means 213 of the client device 20 compares the accompanying information distributed at regular intervals with the accompanying information acquired at the time of transmitting the video / audio delivery information request, and changes the accompanying information if they are different. .

(3) The accompanying information distribution request transmitting means 214 of the client device 20 issues a request for distribution of accompanying information to the server device 10 at the start of reproduction of video / audio information. Upon receiving the accompanying information delivery request, the accompanying information delivery control unit 112 and the accompanying information delivery unit 114 of the server device 10 perform the delivery of the video / audio information and the delivery of the accompanying information in synchronization with each other, and The associated information is delivered to the client device 20.

(4) The video / audio reproduction position extracting means 206 of the client device 20 acquires the video / audio reproduction position information when the user issues a request for acquiring the accompanying information during the reproduction of the video / audio information. Further, the accompanying information distribution request transmitting unit 214 transmits a distribution request of the accompanying information accompanied with the acquired video / audio reproduction position information to the server device 10. Upon receiving the distribution request, the accompanying information distribution unit 114 of the server device 10 distributes the accompanying information corresponding to the received video / audio reproduction position.

(5) When the accompanying information changes, the accompanying information distribution control means 112 of the server device 10 recognizes that the change has occurred, and sends the accompanying information distribution means 114 to the accompanying information distribution means 114. Instruct the delivery of. The accompanying information distribution unit 114 distributes the changed accompanying information to the client device 20 when receiving the notification of the change of the accompanying information from the accompanying information distribution control unit 112.

According to the present invention, the server device 10 distributes video / audio information, and the client device 20 reproduces video / audio. An additional information distribution function of transmitting an additional information distribution request to the server device 10 and distributing the additional information by the server device 10 that has received the distribution request, and an external device activation in which the client device 20 calls the external device 30 based on the additional information. Has functions. Hereinafter, the flow of processing in each embodiment will be described.

[First Embodiment] FIG. 3 is a diagram for explaining a processing flow in the first embodiment.
The flow of processing shown in FIG. 3 is such that when a video / audio information distribution request is transmitted from the client device 20 to the server device 10 between the server device 10 and the client device 20, the client device 20 reproduces the video / audio. Before starting the process, all information of the video information and the accompanying information is distributed from the server device 10 to the client device 20, and the accompanying information is periodically distributed from the server device 10 to the client device 20. This is for using the accompanying information in a method of changing the accompanying information when a change occurs.

When the video / audio information distribution request transmitting means 204 of the client device 20 transmits the video / audio information distribution request signal and the accompanying information distribution request signal to the server device 10, the respective signals are transmitted to the video image of the server device 10. -The audio information distribution request receiving means 103 and the accompanying information distribution request receiving means 113 receive (S101, S102). When the server device 10 receives the video / audio information distribution request signal,
All necessary information of the requested video / audio information is distributed from the video / audio information distribution means 104 to the client device 20 (S103). At this time, all of the accompanying information accompanying the video / audio information to be delivered is simultaneously delivered from the accompanying information delivery unit 114 to the client device 20 (S10).
4).

In the client device 20, all video / audio information and all accompanying information are respectively transmitted to the video / audio information receiving means 2.
05 and the accompanying information receiving means 215, and the reproduction of video / audio is started (S105). However, if the requested video / audio information or accompanying information has not been delivered, the client device 20 sends a video / audio information / attached information retransmission request to the server device 10 respectively.
The request is transmitted from the audio information distribution request transmitting unit 204 and the auxiliary information distribution request transmitting unit 214, and requests the server device 10 to distribute the video / audio information or the auxiliary information again.

The server device 10 distributes the accompanying information from the accompanying information distribution means 114 to the client device 20 at regular intervals. Associated information control means 21 of client device 20
3 compares the additional information acquired at the time of the video / audio information distribution request with the newly received additional information every time the additional information is received by the additional information receiving means 215, and acquires the additional information at the time of the video / audio information distribution request. If the accompanying information is different from the newly received accompanying information, the held accompanying information is replaced with the newly received accompanying information (S106).

The user confirms the reproduced video / audio and, when using the function of the external device 30, inputs a request to use the function of the external device 30 from the associated information operating means 211 to the client device 20 (S107). When a request to use the function of the external device 30 is input, the playback control means 203 temporarily stops the playback of the video / audio and outputs the playback stop position information to the video / audio.
(S108, S10)
9). At the same time, the reproduction stop position information is also passed to the accompanying information control means 213.

When the playback stop position information is received, the accompanying information control means 213 refers to the accompanying information received from the server device 10 and compares the information with the playback stop position information.
The action to be executed by the external device 30 (hereinafter referred to as an external information activation condition) is determined (S110).

When the external information activation condition is determined, the accompanying information control means 213 passes the external information activation condition to the external device control means 216. Upon receiving the external information activation condition, the external device control means 216 activates the external device 30 based on the external information activation condition.

For example, taking the accompanying information as shown in FIG. 4 as an example, when each part of video and audio is recognized by the elapsed time from the start of reproduction, the elapsed time is assumed to be Lt (0 <Lt <L1). Judge from the specific information of Location and Location, informatio
n [0] 401 becomes valid. At this time, when the user
If WWW browser is selected as 0, WW
The W browser follows the parameters of “Browser [0] = http: //www.*********” written in information [0] 401 of the accompanying information, and http: //www.** Open the URL of ******* (S
111).

When the external information activation condition is determined, the accompanying information control means 213 passes the external information activation condition to the external device control means 216. Upon receiving the external information activation condition, the external device control means 216 activates the external device 30 based on the external information activation condition (S112).

When the user stops using the external device 30, the external device control means 216 detects an end signal of the external device 30 (S113). External device control means 216
Sends a reproduction restart request to the reproduction control means 203 upon detecting the end signal of the external device 30. Upon receiving the reproduction restart request, the reproduction control unit 203 acquires the reproduction stop position information from the video / audio reproduction position extraction unit 206, and restarts the reproduction from the reproduction stop position (S114).

As described above, in this embodiment, the client device 20 periodically receives the accompanying information. Thus, when the accompanying information is changed, the client device 20 acquires the changed accompanying information from the server device 10 and can synchronize the reproduction of the video / audio in the client device 20 with the changed accompanying information. Becomes

[Second Embodiment] FIG. 5 is a diagram for explaining the flow of processing in the second embodiment.
The flow of the process shown in FIG. 5 is such that when a video / audio information distribution request is transmitted from the client device 20 to the server device 10 between the server device 10 and the client device 20, the video information This is a case where the accompanying information is used when the reproduction of the video / audio is started at the same time when the client device 20 receives the accompanying information.

When the video / audio information distribution request transmitting means 204 of the client device 20 transmits the video / audio information distribution request signal and the accompanying information distribution request signal to the server device 10, these signals are transmitted to the server device 10 respectively. Are received by the video / audio information distribution request receiving means 103 and the accompanying information distribution request receiving means 113 (S201, S20).
2). Upon receiving the video / audio information distribution request signal, the server device 10 continuously distributes the requested video / audio information from the video / audio information distribution means 104 to the client device 20 (S203). At this time, the accompanying information accompanying the video / audio information to be delivered is
14 to the client device 20 continuously (S
204).

The client device 20 transmits the video / audio information and the accompanying information to the video / audio information receiving means 205, respectively.
And the associated information receiving means 215, the reproduction of the video / audio is started simultaneously with the reception (S205). However, it is checked whether or not the information transmitted on the communication network 100 has been lost. If the information loss is recognized, the client device 20 sends a request for retransmission of the lost information to the server device 10 using video / audio. The video / audio information distribution request transmitting means 204 and the auxiliary information distribution request transmitting means 214 transmit each of the information and the accompanying information.

Further, it is necessary to synchronize the distributed video / audio information with the accompanying information before reproducing the video / audio. Therefore, it is necessary to obtain the video / audio information and the accompanying information before playing the video / audio.
Performed prior to video / audio playback. For example, if a time of 3 seconds is required for processing of an information retransmission request and synchronizing video / audio information and accompanying information, the client device 20 transmits at least 3 seconds of video / audio information and accompanying information to video / audio. Earn before playing.

When using the functions of the external device 30 after confirming the reproduced video and audio, the user inputs a request to use the functions of the external device 30 to the client device 20 from the associated information operating means 211 (S206). When a request to use the function of the external device 30 is input, the reproduction control unit 203 passes the reproduction position information at the time when the request to use the function of the external device 30 is input to the accompanying information control unit 213 (S207, S2).
08).

Upon receiving the playback position information, the accompanying information control means 213 determines the external information activation condition by referring to the accompanying information received from the server device 10 and comparing it with the playback position information (S209).

Taking the accompanying information shown in FIG. 4 as an example, when recognizing each part of video and audio based on the elapsed time from the start of reproduction, if the elapsed time is Lt (0 <Lt <L1), info
rmation [0] 401 becomes valid. At this time, if the user has selected the WWW browser as the external device 30, the WWW browser transmits information [0] 401 of the accompanying information.
"Browser [0] = http: //www.*********" written in
The URL of http: //www.********* is opened according to the parameters (S210).

When the external information activation condition is determined, the accompanying information control means 213 passes the external information activation condition to the external device control means 216. Upon receiving the external information activation condition, the external device control means 216 activates the external device 30 based on the external information activation condition (S211).

When the user stops using the external device 30, the external device control means 216 detects the end signal of the external device 30 and confirms that the external device 30 has been ended (S212).

As described above, in the present embodiment, the client device 20 continuously receives the accompanying information. Thus, when the accompanying information is changed, the client device 20 always obtains the changed accompanying information from the server device 10, and the reproduction of the video / audio in the client device 20 is synchronized with the changed attached information. It becomes possible.

[Third Embodiment] FIG. 6 is a diagram for explaining the flow of processing in the third embodiment.
The flow of processing shown in FIG. 6 is such that when a video / audio information distribution request is transmitted from the client device 20 to the server device 10 between the server device 10 and the client device 20, the client device 20 reproduces the video / audio. Before starting the process, all information of the video information is distributed from the server device 10 to the client device 20, and the accompanying information is distributed to the server device 10 only when a request to use the function of the external device 30 is input from the user. This is for using the accompanying information in a method of transmitting a request and distributing the request to the client device 20.

When a video / audio information distribution request signal is transmitted from the video / audio information distribution request transmission means 204 of the client device 20 to the server device 10, the server device 1
The video / audio information distribution request receiving means 103 receives the video / audio information distribution request signal (S301). Upon receiving the video / audio information distribution request signal, the server device 10 distributes the requested video / audio information information from the video / audio information distribution unit 104 to the client device 20 (S3).
02).

When all the video / audio information is received by the video / audio information receiving means 205, the client device 20
The sound reproduction is started (S303). However, when the requested video / audio information has not been delivered, the client device 20 sends a video / audio information distribution request transmission means 204 to the server device 10 to retransmit the video / audio information and the accompanying information. Associated information distribution request transmission means 21
4 and again requests the server device 10 to deliver video / audio information or accompanying information.

The user confirms the reproduced video / audio and, when using the function of the external device 30, inputs a use request for the function of the external device 30 from the associated information operating means 211 to the client device 20 (S304). When a request to use the function of the external device 30 is input, the reproduction control unit 203 passes the video / audio reproduction position information at the time when the request to use the function of the external device 30 is input to the accompanying information control unit 213 (S305). , S306).

Upon receiving the reproduction position information, the accompanying information control means 213 transmits the video / audio reproduction position information to the server device 10 and transmits a distribution request for the accompanying information from the accompanying information distribution request transmitting means 214. Yes (S30
7). When the video / audio playback position information and the accompanying information distribution request are received by the accompanying information distribution request receiving unit 113, the server device 10 determines the accompanying information to be distributed to the client device 20 based on the video / audio playback position information. The accompanying information is delivered from the accompanying information delivery means 114 (S308). However, if the requested accompanying information has not been delivered, the client device 20 transmits a request for retransmission of the accompanying information to the server device 10 from the accompanying information delivery request transmitting means 214, and sends the request to the server device 10 again. Request the distribution of accompanying information.

When the incidental information is received by the incidental information receiving means 215, an external information activation condition is determined based on the incidental information (S309). When the external device activation condition is determined, the accompanying information control unit 213 passes the external information activation condition to the external device control unit 216. Upon receiving the external information activation condition, the external device control means 216 activates the external device 30 based on the external information activation condition.

Taking the accompanying information shown in FIG. 4 as an example, when recognizing each part of video and audio based on the elapsed time from the start of reproduction, if the elapsed time is Lt (0 <Lt <L1), info
rmation [0] 401 becomes valid. At this time, if the user has selected the WWW browser as the external device, the W
The WW browser follows the parameter “Browser [0] = http: //www.*********” written in information [0] 401 of the accompanying information, and http: //www.** The URL of ******* is opened (S310).

When the external information activation condition is determined, the accompanying information control means 213 passes the external information activation condition to the external device control means 216. Upon receiving the external information activation condition, the external device control means 216 activates the external device 30 based on the external information activation condition (S311).

When the user stops using the external device 30, the external device control means 216 detects the end signal of the external device 30 and confirms that the external device 30 has been ended (S312).

As described above, in the present embodiment, although the client device 20 does not hold the accompanying information in advance, the client device 20 requests the server device 10 to distribute the accompanying information in response to a request from the user. To win. As a result, when a change occurs in the accompanying information, the client device 20
The changed accompanying information of the server device 10 is acquired, and the reproduction of the video and audio in the client device 20 can be synchronized with the changed attached information.

[Fourth Embodiment] FIG. 7 is a diagram for explaining the flow of processing in the fourth embodiment.
The flow of the process shown in FIG. 7 is such that when a video / audio information distribution request is transmitted from the client device 20 to the server device 10 between the server device 10 and the client device 20, the video information distributed from the server device 10 is transmitted. At the same time as the reception, the client device 20 starts the reproduction of the video / audio, and the accompanying information acquires all the necessary information from the server device 10 before the client device 20 starts the reproduction of the video / audio. When the change occurs, the change information is used when the server device 10 uses the accompanying information when distributing the change information to the client device 20.

When a video / audio information distribution request signal and an accompanying information distribution request signal are transmitted from the video / audio information distribution request transmission unit 204 of the client device 20 to the server device 10, the video / audio information Information distribution request receiving means 103 and associated information distribution request receiving means 1
13 receives (S401, S402). Server device 1
0 receives the video / audio information distribution request signal, and transmits the requested video / audio information to the video / audio information distribution unit 104.
From the client device 20. At this time, all information of the accompanying information accompanying the video / audio information to be delivered is also delivered from the accompanying information delivery unit 114 to the client device 20 at the same time (S403, S404).

When the video / audio information is received by the video / audio information receiving means 205, the client device 20 starts reproducing the video / audio (S405). However, it is checked whether or not the information transmitted on the communication network 100 has been lost. If the information loss is recognized, the client device 20 sends a request for retransmission of the lost information to the server device 10 using video / audio. The information and the accompanying information are transmitted from the video / audio information delivery request sending unit 204 and the accompanying information delivery request sending unit 214.

Further, it is necessary to synchronize the distributed video / audio information with the accompanying information before reproducing the video / audio. Therefore, the video / audio information and the auxiliary information need to be obtained before reproducing the video / audio, and the acquisition of the video / audio information and the auxiliary information is performed prior to the reproduction of the video / audio. For example, if a time of 3 seconds is required for processing of an information retransmission request and synchronizing video / audio information and accompanying information, the client device 20 transmits at least 3 seconds of video / audio information and accompanying information to video / audio. Earn before playing.

The server device 10 includes an accompanying information storage unit 11
When the accompanying information distribution control unit 112 detects a change in the accompanying information stored in the storage unit 1, the accompanying information distribution unit 114 distributes the changed accompanying information to the client device 20. When the client device 20 receives the changed accompanying information by the accompanying information receiving unit 215, the client device 20 compares the accompanying information acquired at the time of the video / audio information distribution request with the newly received accompanying information, and issues the video / audio information distribution request. If the acquired additional information is different from the newly received additional information, the stored additional information is replaced with the newly received additional information (S406).

When the user confirms the reproduced video / audio and uses the function of the external device, the user requests the use of the function of the external device 30 from the associated information operating means 211 to the client device 2.
0 is input (S407).

When a request to use the function of the external device 30 is input, the reproduction control means 203 transmits the video / audio reproduction position information at the time when the request to use the function of the external device 30 is input to the accompanying information control means 213. (S408, S40
9).

When the reproduction position information is received, the accompanying information control means 213 determines the external information activation condition by referring to the accompanying information received from the server device 10 and comparing it with the reproduction position information (S410).

Taking the accompanying information shown in FIG. 4 as an example, when recognizing each part of video and audio based on the elapsed time from the start of reproduction, if the elapsed time is Lt (0 <Lt <L1), info
rmation [0] 401 becomes valid. At this time, if the user has selected the WWW browser as the external device 30, the WWW browser transmits information [0] 401 of the accompanying information.
"Browser [0] = http: //www.*********" written in
The URL of http: //www.********* is opened in accordance with the parameters (S411).

When the external information activation condition is determined, the accompanying information control means 213 passes the external information activation condition to the external device control means 216. Upon receiving the external information activation condition, the external device control means 216 activates the external device 30 based on the external information activation condition (S412).

When the user stops using the external device 30, the external device control means 216 detects the end signal of the external device 30 and confirms that the external device 30 has been ended (S413).

As described above, in the present embodiment, the client device 20 receives the new accompanying information changed when the accompanying information is changed. As a result, when a change occurs in the accompanying information, the client device 20 acquires the changed accompanying information from the server device 10 and synchronizes the reproduction of the video / audio in the client device 20 with the changed accompanying information. It becomes possible.

In the above description, it goes without saying that the video and audio may be either video or audio, or both. In the description of the above embodiment, an example of distributing video / audio information stored in advance has been described. However, for example, video / audio information having a time constraint from information input to distribution, such as a live broadcast, is described. The present invention can be similarly applied to distribution.

[0079]

As described above, by using the present invention, when video and audio are distributed over a communication network, the distribution processing of accompanying information is performed independently of the video or audio information distribution or reproduction processing. Becomes possible. Therefore,
Even when the accompanying information is changed during the reproduction of the video / audio, the changed information of the accompanying information can be distributed from the server device to the client device.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a configuration example of a server device.

FIG. 2 is a diagram illustrating a configuration example of a client device.

FIG. 3 is a diagram for explaining a processing flow in the first embodiment.

FIG. 4 is a diagram illustrating an example of accompanying information.

FIG. 5 is a diagram illustrating a flow of a process according to the second embodiment.

FIG. 6 is a diagram illustrating a flow of a process according to a third embodiment.

FIG. 7 is a diagram illustrating a flow of a process according to a fourth embodiment.

[Explanation of symbols]

 Reference Signs List 10 server apparatus 101 video / audio information storage means 102 video / audio information distribution control means 103 video / audio information distribution request receiving means 104 video / audio information distribution means 111 auxiliary information storage means 112 auxiliary information distribution control means 113 auxiliary information distribution request Receiving unit 114 Associated information distribution unit 20 Client device 201 Reproduced video / audio output unit 202 Video / audio information operation unit 203 Reproduction control unit 204 Video / audio information distribution request transmission unit 205 Video / audio information receiving unit 206 Video / audio reproduction position Extraction unit 211 Associated information operation unit 212 Associated information display unit 213 Associated information control unit 214 Associated information distribution request transmission unit 215 Associated information reception unit 216 External device control unit 100 Communication network

Continued on the front page (72) Inventor Akira Ono 3-192-2 Nishi-Shinjuku, Shinjuku-ku, Tokyo Nippon Telegraph and Telephone Corporation (72) Inventor Yasuhito Hayashi 3- 19-2 Nishi-Shinjuku, Shinjuku-ku, Tokyo Nippon Telegraph Within Telephone Co., Ltd. (72) Inventor Hiroshi Koyano 3-19-2, Nishi-Shinjuku, Shinjuku-ku, Tokyo F-Term within Nippon Telegraph and Telephone Corporation (reference) 5B089 GA11 GA21 JA33 JB06 JB22 KC28 KC30 5C064 BA01 BA07 BB07 BB10 BC06 BC16 BC18 BC20 BC25 BC27 BD01 BD02 BD07 BD08 BD09

Claims (7)

[Claims] 1. A server device for distributing video / audio information, a client device for receiving and reproducing video / audio information distributed from the server device, and a communication network connecting the server device and the client device. In the multimedia communication control method in the system comprising: the server device is information accompanying the video / audio information, and the external device for supplementing the content of the video / audio information by utilizing the function of the external device. Is stored for each part of the video / audio information to be distributed, and when a request for distribution of the auxiliary information is received from the client device, the client device sends the video / audio information to the client device. The client device delivers necessary accompanying information before reproduction, and thereafter delivers the accompanying information periodically. A request for distribution of video and audio information is transmitted to the server device, and a request for distribution of accompanying information is transmitted to the server device. Receiving the additional information, periodically receiving the additional information distributed from the server device, replacing the old additional information with the new additional information, and using the function of the external device, the latest distributed information is used. A multimedia communication control method, wherein an external device is called based on accompanying information. 2. A server device for distributing video / audio information, a client device for receiving and reproducing video / audio information distributed from the server device, and a communication network connecting the server device and the client device. In the multimedia communication control method in the system comprising: the server device is information accompanying the video / audio information, and the external device for supplementing the content of the video / audio information by utilizing the function of the external device. Is stored for each part of the video / audio information to be distributed, and when a request for distribution of the accompanying information is received from the client device, the video / audio to be distributed is transmitted to the client device. The accompanying information corresponding to each part of the audio information is continuously performed in synchronization with the distribution of each part of the video / audio information, and The client device transmits a distribution request for video / audio information to the server device, transmits a distribution request for accompanying information to the server device, and continuously transmits each part of the distributed video / audio information. , And reproduces the information associated with each part.
When the video / audio information of each part is received before reproduction and the function of the external device is used, the external device is called based on the accompanying information corresponding to the video / audio information part being reproduced. Multimedia communication control method. 3. A server device for distributing video / audio information, a client device for receiving and reproducing the video / audio information distributed from the server device, and a communication network connecting the server device and the client device. In the multimedia communication control method in the system comprising: the server device is information accompanying the video / audio information, and the external device for supplementing the content of the video / audio information by utilizing the function of the external device. When the client device receives a distribution request for the accompanying information specifying the video / audio reproduction position information from the client device, the client device stores the associated information including information on a method of using the video / audio information. Deliver accompanying information corresponding to the video / audio playback position information to the requesting client device. The client device transmits a video / audio information distribution request to the server device, reproduces the distributed video / audio information, and uses the function of the external device during reproduction of the video / audio information. When the accompanying information is needed to transmit the information, a request for distribution of the accompanying information with the video / audio reproduction position information at that time is transmitted to the server device, and the accompanying information is received from the server device;
A multimedia communication control method, wherein an external device is called based on the received accompanying information. 4. A server device for distributing video / audio information, a client device for receiving and reproducing video / audio information distributed from the server device, and a communication network connecting the server device and the client device. In the multimedia communication control method in the system comprising: the server device is information accompanying the video / audio information, and the external device for supplementing the content of the video / audio information by utilizing the function of the external device. Is stored for each part of the video / audio information to be distributed, and when a request for distribution of the auxiliary information is received from the client device, the client device sends the video / audio information to the client device. Necessary information is distributed before playback, and after that, it is monitored whether or not changes have been made to the accompanying information. When the change is detected, the accompanying information that has changed is delivered to the client device. The client device sends a video / audio information delivery request to the server device, and sends the accompanying information to the server device. Transmitting an information distribution request, receiving necessary accompanying information from the server device before reproducing the distributed video / audio information, and, when the changed accompanying information is distributed from the server device, old accompanying information Is replaced with new additional information, and the external device is called based on the latest additional information distributed when using the function of the external device. 5. A server device for distributing video / audio information, a client device for receiving and reproducing video / audio information distributed from the server device, and a communication network connecting the server device and the client device. In the multimedia communication control system comprising: the server device is information accompanying video / audio information,
Means for holding, for each part of the video / audio information to be distributed, accompanying information including information on a method of using the external device to supplement the content of the video / audio information by using the function of the external device; Means for receiving a distribution request for video / audio information from the client device, and for continuously distributing the video / audio information collectively or for each part, and means for receiving a distribution request for accompanying information from the client device And in response to a request for distribution of accompanying information from the client device, at least necessary latest accompanying information is delivered before reproduction of the video / audio information or each part thereof in the client device, or In response to a request for distribution of accompanying information specifying video / audio playback position information from a client device, the corresponding video / audio playback position information And means for delivering the incidental information, the client device, the video and to the server device
Means for transmitting a distribution request for audio information and reproducing the distributed video / audio information, and distributing the accompanying information to the server device before reproducing the distributed video / audio information or reproducing each part thereof; Means for transmitting a request or transmitting a request for distribution of accompanying information accompanied by video / audio reproduction position information to the server device when accompanying information is required during reproduction of the distributed video / audio information; And an external device control means for calling an external device based on the latest accompanying information distributed from the server device when using a function of the external device. 6. A recording medium storing a program to be executed by a server device in a multimedia communication control system for distributing video / audio information to a client device via a communication network, wherein the recording medium stores a program of the video / audio information to be distributed. Information accompanying video / audio information stored for each part, including information on how to use the external device to supplement the contents of the video / audio information by using the functions of the external device. A process for managing information and reading it out at the time of a request for distribution of accompanying information from the client device, and receiving a request for distribution of video and audio information from the client device, and continuing the video and audio information collectively or for each part. And a process of receiving a request for distribution of accompanying information from the client device, and a process of receiving a request from the client device. In response to a request for distribution of pertinent information, at least necessary latest accompanying information is distributed before reproduction of video / audio information or each part thereof in the client device, or video / audio from the client device. Multimedia communication control characterized by recording a program for causing a computer to execute a process of distributing accompanying information corresponding to the video / audio playback position information in response to a delivery request of the accompanying information specifying the playback position information A recording medium that records a server program in a system. 7. A recording medium on which a program executed by a client device in a multimedia communication control system for receiving and reproducing video / audio information distributed from a server device for distributing video / audio information via a communication network is described. Transmitting a video / audio information distribution request to the server device, and reproducing the distributed video / audio information; and reproducing the distributed video / audio information before or each portion thereof. Previously, information accompanying the video / audio information held by the server device for each part of the video / audio information, which is used to supplement the content of the video / audio information by using the function of the external device. A distribution request for accompanying information including information on a method of using an external device is transmitted to the server device, or the distributed video / audio information is transmitted to the server device. A process of transmitting a request for distribution of accompanying information accompanied by video / audio playback position information to the server device when accompanying information is required during playback; A program for causing a computer to execute a process of calling an external device based on the latest accompanying information delivered from a computer, and a program for a client device in a multimedia communication control system.