JP2002108929A - System and method for retrieving related information, and computer readable recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system and a method for retrieving related information, and a computer readable recording medium, with which related information on which the present state of a user is reflected can be presented. SOLUTION: In the system for presenting first image information to user terminals 70 and 72 so as to refer to the information, retrieving related information corresponding to audio information transmitted from the user, who refers to the first image information, and presenting the related information to the user, a control means 2a decides the state of the user on the basis of the audio information analyzed by an analyzing means (audio recognizing part) 4, retrieves the related information stored in relation to the state of the user from a related information database 8 on the basis of the decided result and presents the related information to the user.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a system and a method for presenting related information other than image information to a user, and a computer-readable recording medium.

[0002]

2. Description of the Related Art In recent years, with the spread of the Internet, it has become possible for users to access various Web sites and browse predetermined contents (image information such as Web pages). Also, it is possible to add related information such as a banner advertisement to the Web page and present it to the user terminal to perform advertisement.

[0003]

However, in the case of the banner advertisement described above, since the association (link) with the Web page is predetermined, it is difficult to present the advertisement according to the user's hobbies and preferences. is there. for that reason,
At present, the advertising effect is not enough. Recently, the age, gender, and the like of a user are registered on a Web page, and an advertisement corresponding to the registered age and gender is also distributed. However, even in such a case, the advertisement is based on user information registered in the past. However, it has not been possible to present an advertisement according to the user's current mental state or the like in real time.

[0004] The present invention has been made in view of the above-described problems, and relates to related information reflecting the current state of a user in accordance with audio information or character information transmitted from a user who has viewed predetermined image information. It is an object to provide a related information search system and method that can be presented, and a computer-readable recording medium.

[0005]

In order to achieve the above object, a related information retrieval system according to the present invention presents first image information to a user terminal of a user so as to be able to browse the first image information. Searching for related information corresponding to the voice information or character information transmitted from the user who browsed the image information, and controlling means, an analyzing means for analyzing the voice information or character information, and the related information, the state of the user And a related information database that stores the related information in association with the related information. The control unit determines the state of the user based on the analyzed voice information or character information, and based on the determination result, the related information corresponding to the user from the related information database. Is searched.

In the related information search system of the present invention,
The control means may present the searched related information to the user.

[0007] It is preferable that the control means determines the state of the user on the basis of either the speed at which the voice is emitted or the tone of the voice in the analyzed voice information. The related information is further stored in association with a predetermined word, the analysis unit extracts a word from the voice information, and the control unit reads corresponding relevant information from the related information database based on the extracted word. It is preferable to search.

In the related information search system according to the present invention, the related information search system is connected to an operator terminal of a predetermined operator, and can exchange voice information or character information between the user and the operator. It is characterized by being.

Further, in the related information search system according to the present invention, the user terminal can transmit the voice information or character information via a network, and the related information search system can send the user information to the user terminal via the network. It is characterized by being provided with an information separating means which is connected and separates image information and audio information or image information and character information. Further, in the related information search system of the present invention, the character information transmitted from the user is an electronic mail.

A related information search method according to the present invention presents first image information to a user terminal of the user so that the first image information can be browsed, and transmits voice information or character information transmitted from the user who browsed the first image information. Searching for related information according to, the step of storing the related information in association with the state of the user, the step of analyzing the voice information or character information, and the step of analyzing the voice information or character information based on the analyzed voice information or character information The method includes a step of determining a state of the user and a step of searching for relevant information based on the determination result.

[0011] The related information search method of the present invention further comprises a step of presenting the searched related information to the user.

A computer-readable recording medium according to the present invention presents first image information to a user terminal of a user so that the first image information can be browsed, and also stores audio information or voice information transmitted from the user who browsed the first image information. Recording a method of searching for related information according to character information, storing the related information in association with the state of the user, analyzing the voice information or character information, and analyzing the analyzed voice information or A program for causing a computer to execute a step of determining the state of the user based on character information and a step of searching for relevant information based on the determination result is recorded.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to FIG. FIG. 1 is a related information retrieval system of the present invention (hereinafter referred to as “this system” as appropriate))
50 is a configuration block diagram showing an embodiment of a voice / IP conversion server 2 having a control unit 2a for controlling the entire system, a voice recognition unit (analysis unit, IVR (Interact
ive Voice Response), a first image information database 6, and a related information database 8. Voice / I
The P conversion server 2 also has an input / output control unit 2b for transmitting and receiving information to and from the network 100, and an information (synthesis) separation unit 2c for combining and separating image information and audio information. The control unit 2a and the information separation unit 2c can be realized as a central processing unit of the voice / IP conversion server 2.
Further, this system constitutes a Web site as a whole.

Here, the first image information database 6
Is the user terminal 70, 72 or the operator terminal 20
And stores first image information composed of, for example, a Web page. The first image information is composed of predetermined images (photographs, pictures, etc.) so that a user who has viewed the first image information can make a voice response. The related information database 8 stores related information including content (advertisement, entertainment information) and the like in association with a user state described later. Note that the related information may be audio information, electronic mail, or the like, in addition to a Web page describing the content. In addition,
“Present related information” refers to transmitting related information including a web page, an e-mail, and voice information to a user terminal. Further, the content itself may be provided in a database separate from the present system (for example, an advertisement database) including only the identification information of the predetermined content as the related information, and the content may be acquired based on the identification information. Then, such a case is also referred to as related information.

In this embodiment, the present system 50 further includes an operator system 50a for communicating with a user, but the operator system 50a is not essential to the present system. And the operator system 50
a is an operator terminal 20 for browsing image information separated by an information separating unit 2c and image information transmitted from the first image information database 6, and exchanges voice information separated by the information separating unit 2c with an exchange (PBX). ) 30 and a link unit 24 for associating each terminal with each telephone when there are a plurality of operators. The link unit 24 is a program operated by the control unit of the terminal 20. As the operator terminal 20, for example, a personal computer is used.

The system 50 is connected to user terminals 70 and 72 of the user via a network 100. Here, the user terminal 70 is composed of a personal computer or the like, and includes a headset 70a for exchanging voice information. The user terminal 70 has an Internet telephone function described later. On the other hand, the user terminal 72 is a mobile phone that can be connected to the Internet, and is connected to the network 100 via the mobile communication network 110. Note that each of the above terminals 2
Browsers capable of browsing Web pages are mounted on 0, 70, and 72. As the network 100, a dedicated line, a LAN (Local Area) is used instead of the Internet.
Network), WAN (Wide Area Network) or the like may be used.

FIG. 2 shows a mode in which information is exchanged between a user terminal 70 as an Internet telephone and the present system 50 (and operator A) via a network (Internet) 100. In this figure, a user terminal 70 includes a keyboard / input device, a screen for displaying an image, a synthesis / separation unit 7 including a sound board, and the like.
0c, and the synthesis / separation unit 70c includes an analog / digital conversion unit 70d. Then, after the audio information emitted by the user A is received as an analog signal by the headset 70a, the audio information is converted into a digital signal by the analog / digital converter 70d and further compressed. On the other hand, the image information (for example, selection information for the displayed screen or the like) input by the user A from the keyboard of the terminal 70 is a digital signal, and the synthesizing / separating unit 70c packetizes the image information and the audio information respectively, and The packet is transmitted to the system 50 by packet communication using a communication protocol (for example, TCP / IP).

In the present system 50, the received digital signal is separated into image information and audio information by the information (synthesis) separation section 2c. The control means 2a keeps the image information as a digital signal.
And the control means 2a acquires the corresponding image from the first image information database 6 and transmits it to the operator terminal 20 for browsing. The information (synthesis) separation unit 2c also connects and decompresses the packets of the audio information to perform analog /
The data is passed to the digital conversion unit 2d, and the analog / digital conversion unit 2d converts the signal into an analog signal, transmits the analog signal to the telephone set 22 via the exchange 30, and transmits the analog signal to the voice recognition unit 4.

On the other hand, the voice information issued by the operator A is
The image information (for example, image selection information, etc.) digitized and compressed by the analog / digital conversion unit 2d and input by the operator A from the input unit (keyboard or the like) of the terminal 20 together with the information processed by the control unit 2a together with the information (Composition) The packet is transmitted to the user terminal 70 by the packet communication in the separation unit 2c.

Next, a processing flow in the related information search system 50 will be described with reference to FIG. In the figure, first, the user specifies a URL (Uniform Resource Locator) to access the system (Web site) from the user terminal (step S200). At this time, the user may log in by appropriately inputting authentication information. The control unit 2a of the present system transmits the top page to the user terminal (Step S300). A predetermined "phone button" is provided on the top page displayed on the user terminal, and by selecting this, the user makes a call to the present system (step S210). More specifically, the control unit 2a receiving the selection information of the "telephone button"
Searches for a telephone that can be connected via the exchange 30.
If there is a connectable telephone, control means 2
a connects the telephone and the user terminal (step S
310).

The operator uses the telephone to answer the telephone with the user and instructs the user to select a screen (step S400). The instruction to select a screen is, for example, "Click the menu on the top page to see the next screen." The user selects a screen according to the instruction (step S220), and upon receiving the selection information, the control unit 2a searches the first image information database for a corresponding selection screen, and transmits the selected screen to the user terminal and the operator terminal, respectively ( Step S320).

Next, the operator uses the telephone to instruct the user to make a voice response to the browsing of the selection screen by telephone (step S410). This instruction is, for example, "Are you interested in the screen you are looking at?" The user makes a voice response and transmits the voice information via the user terminal (step S230). In step S230, since the user views the first image information (selection screen) and emits the audio information, the user's true information based on the impression of looking at the screen can be provided without giving a preconception or time to reflect. Voice information reflecting a psychological state or the like can be acquired, and reliability of determination of a user state described later can be improved. The voice information is transmitted to the telephone of the operator, and also transmitted to the voice recognition unit 4 of the present system, where the voice is analyzed (step S330).

The control means 2a acquires the analysis result and determines the state of the user based on the analysis result (step S34).
0). Further, the relevant information is searched from the relevant information database 8 based on the determination result (step S35).
0), and transmit them to the user terminal and the operator terminal, respectively (step S360). Related information is displayed on the user terminal and the operator terminal so as to be browsed.

FIG. 4 is a block diagram showing the configuration of the voice recognition unit 4. As shown in FIG. In this figure, the voice recognition unit 4
Are a word extraction unit 4a for extracting words from voice information, a voice speed calculation unit 4b for calculating voice speed from the number of words extracted per conversation time, a sound pressure measurement unit 4c for measuring sound pressure from voice information, A sound pitch measuring section 4d for measuring the pitch of the sound from the information, and a start / end specifying section 4e for specifying the start / end timing of the processing in each section 4a to 4d are provided. Among these, the sound pressure and the sound pitch are indices indicating the “voice tone”, and any of them may be used, or both may be used as the voice tone.

In determining the state of the user, which will be described later, it is sufficient if there is any data of voice speed or voice tone, or the determination may be made by referring to both data. Furthermore, in the present invention, it is not essential to extract a word from audio information, but when a word is extracted, related information can be searched based on the extracted word, in addition to the above-described state of the user. Search accuracy is improved.

The word extracting section 4a stores a sound pattern database 4f for storing a sound waveform pattern corresponding to each word.
, A word is extracted by performing speech recognition from the speech information. The start / end designation unit 4e receives an instruction from the operator via the operator terminal, and starts a user response,
Specify the end. Instead of the operator's instruction, a natural language processing unit 4g may be provided to estimate the meaning of the user response from the extracted word group based on predetermined grammar / grammar data, and to specify the start and end. The voice speed calculation unit 4b, the sound pressure measurement unit 4c, and the pitch measurement unit 4d calculate and measure each item at predetermined time intervals, which will be described later.

Next, the processing flow of the voice recognition unit 4 in step S330 will be described with reference to FIG. First, start
The end specifying unit 4e issues an instruction to acquire voice information (step S500), and the voice recognizing unit 4 acquires the user voice information (step S510). The word extraction unit 4a performs word extraction (step S520),
The voice speed calculation unit 4b acquires the word data extracted from the word extraction unit 4a and calculates the voice speed (step S530). The sound pressure measuring section 4c or the pitch measuring section 4d measures the sound pressure (pitch) (step S54).
0). Hereinafter, while the word extraction is constantly performed, the calculation of the voice speed and the measurement of the sound pressure (the measurement of the pitch of the sound) are performed at regular intervals (interval time to be described later) (step S550). Then, in response to a termination instruction from the start / end designation unit 4e, word extraction, voice speed calculation, and sound pressure (pitch) measurement are terminated (step S560), and each data is passed to the control unit 2a (step S570).

FIG. 6 shows a specific mode of performing voice analysis according to the flow of FIG. First, the user's voice information (waveform)
Are as shown in FIG. From this waveform,
For example, voice recognition of "I want to buy a new car" is performed, and predetermined words (excluding connectives and the like) "car" and "new replacement" are extracted from the speech (FIG. 6B). The timing for calculating and measuring the voice speed and the sound pressure (pitch of the sound) is as shown in FIG. 6C. First, a predetermined time from the start of the conversation is set as an initial period. Calculate and measure the value during the period and use it as the base (reference) value. After that, each item at that time is calculated / measured every fixed time (section time) until the end of the conversation. For example,
The voice speed is set to “XX words (words) /
Minutes ”, and then, along with the section time,
(Word (word) / minute) (FIG. 6)
(D)). Similarly, the sound pressure is “○ →→ ▲ → ×××” (P
To a), the pitch of the sound changes to “○ → △ → ×” (Hz) (FIGS. 6E and 6F).

Next, the processing flow of the control means 2a in step S340 will be described with reference to FIG. First, the control unit 2a receives the extracted word, the voice speed,
And data of sound pressure (steps S600 and S61)
0). In the following description, a case in which the “voice tone” is obtained based on the sound pressure will be described. However, the voice tone may be based on the pitch instead of the sound pressure. You may ask.

Next, the control means 2a calculates a change per section time described later for the voice speed and the sound pressure (step S620), and determines the state of the user (speaker) based on the results (step S620). Step S630). Then, the control unit 2a searches the related information database (DB) 8 for information corresponding to the extracted word acquired in S610 and the determination result in S630 (step S640).
The control unit 2a transmits the searched information to the user terminal (Step S650).

FIG. 8 shows an example of a method of calculating the time change in step S620. In this figure, the audio speed for each of the initial period and the subsequent section time is obtained. The change in voice speed during the initial period and the first section time is + x1, the change in voice speed during the first and second section times is + x2, and the change in voice speed during the second and third section times. Is -x3, and the time change x of the audio speed shown in FIG. 7 is obtained by the average of the change of each audio speed, that is, x = (+ x1 + x2-x3) / 3 (1). The same applies to the time change of the sound pressure.
Note that the calculation method is not limited to the above, and for example, the one having the largest absolute value may be selected from the values of the change in the audio speed. Examples of the unit of the time change are “word / minute / section time” in the case of the voice speed, and “dB /
Minutes / section time ", and for the pitch," Hz / section time ". dB is a decibel, which is a logarithmic representation of the ratio of the sound pressure in a predetermined section time to the initial period.

FIG. 9 shows an example of the criterion in step S630. This reference is stored in a predetermined computer program executed by the control unit 2a. In this figure, if the above-mentioned temporal change of the voice speed is positive (+), if the speaker is in the state of “interested (excitement, attention)” with respect to the extracted words “car” and “replacement,” the negative ( If it is −), it is determined that “not interested (calm, ignored)”. When a plurality of items (e.g., sound speed and sound pressure) are measured with respect to sound, there is no limitation on the method of calculating the sign of the time change. For example, whether the sum of the time changes of each item is positive or negative is determined. The user's determination may be made, and the time change of each item may be used to discriminate between positive and negative by adopting the one with the larger absolute value.

FIG. 10 shows an example of a method for searching for related information from the related information database (DB) in step S640. In this figure, the control means 2a searches for related information data 8a having word information 1 (car) corresponding to the extracted word "car" as a key. Further, the related information data 8a is searched for word information 2 (new car) corresponding to the extracted word "replacement" and information having attributes of the speaker state "interesting". In this embodiment, the finally retrieved information is “advertisement A” and “advertisement B”, and the control unit 2a transmits these pieces of information to the user terminal, and the user terminal displays the information as an advertisement so as to be able to be viewed.

Next, FIGS. 11 and 12 show examples of display screens on the user terminals 70 and 72 accessing the system. In FIG. 11A, a top page 1000 is displayed on the user terminal that has accessed the system (corresponding to step S300 in FIG. 3). Here, by selecting the “phone” button 1000a, the user is notified to the operator. (Step S in FIG. 3)
210). The user operates the menu 1000 on the top page 1000 according to an instruction from the operator.
b, a predetermined menu (for example, “advice for car”) is selected, and a page 1100 is displayed on the user terminal according to the selection result (step S320 in FIG. 3).
Corresponding to). Here, conversation about the car is performed with the operator, and a screen of a predetermined car is displayed on page 1100.
It is assumed that it is displayed in. Then, the user enters page 11
While browsing 00, the operator says "What do you think about this car?" And instructs a voice response to page 1100 (corresponding to step S410 in FIG. 3). When the voice information "I want to buy a new car (this car). This car is cool" is transmitted as a response from the user, the system transmits the advertisement A to the user terminal by the above-described processing. Advertisement screen 12 of Advertisement A in the upper half
A screen 1200 in which 00b is divided and displayed in the lower half is displayed.

FIG. 12 is an example of a display screen on the user terminal when a menu ("psychological test and counseling") is selected on the top page. A page 2100 is displayed on the user terminal according to the selection (corresponding to step S320 in FIG. 3). Here, a picture of a hut in the forest is displayed. Then, the operator says, "There is a hut in the forest. You have entered the hut. What is inside?" And instructs the voice response to page 2100 (step in FIG. 3). (Corresponds to S410). For example, when voice information such as “There is a person inside” is transmitted as a response from the user, the system determines the state of the user by the above-described processing, and transmits the corresponding advertisement C to the user terminal. On the user terminal, a screen 2200 in which the user's mental state “○ ×” screen 2200a based on the user's response is displayed in the upper half and the advertisement screen 2200b of the advertisement C is displayed in the lower half is displayed.

In the embodiment shown in FIG. 12, words are not extracted in the user response, but the state of the user is determined based only on the voice speed, and the related information (advertisement C) corresponding to the state is searched. . That is, for this psychological test, for example, if the change in the voice speed is +, the association is made such that predetermined related information is searched regardless of the word spoken by the user. However, a further search may be performed in accordance with the word spoken by the user. For example, when the word "person (ga)" is spoken in the user response, predetermined related information, for example, the word "wolf" )), Another related information may be searched for.

The present invention is not limited to the above embodiment. For example, when searching for related information, in the above embodiment, the search was performed based on the extracted word in addition to the state of the user determined, but the user emits voice information in addition to or instead of the extracted word. The search may be performed based on the browsing screen (first image information) at the time of the search. In this way, the search accuracy of the related information is increased. in this case,
For example, the screen selection information transmitted from the user terminal in S220 of FIG. 3 may be recorded.

In the above embodiment, the case where the user terminal has an Internet telephone function has been described, but the present invention is not limited to this. FIG. 13 is a configuration block diagram of the present system 50 'when the user terminal and the telephone are different.

In this figure, the present system 50 'has substantially the same configuration as the system 50 shown in FIG. 1, and these components are distinguished by attaching "'" to the reference numerals of the respective components in FIG. . The function of each component is the same as that of FIG. However, the present system 50 'is connected to the network 100 and the public telephone network 130. Since the image information is exchanged separately via the network 100 and the voice information is exchanged separately via the public telephone network 130, the server 2 'is different from the case of FIG. 1 in that it does not have an information separation unit. The related information stored in the related information database 8 ′ does not include the advertisement content itself, and the advertisement databases 90 and 9 provided outside the present system.
Only the link information to 2, 94 is included.

On the other hand, the user has a user terminal 74 for exchanging only image information and a telephone 76 for exchanging voice information. The user terminal 74 is connected to the server 2 ′ via the network 100, and the telephone 76 is connected to the voice recognition unit 4 ′ and the exchange 3 via the public telephone network 130.
0 '. Then, the control unit 2a ', based on the link information from the related information database 8' according to the state of the user, etc.,
The contents of the advertisement are acquired from 0, 92, 94, and the user terminal 7
4 or the telephone 76 as image information or audio information.

In the embodiment shown in FIG. 1, an exchange (PBX) is required. However, an exchange is not required in a telephone system called an IP phone, and FIG. 14 shows the configuration of the present system 50 '' in this case. FIG. In this figure, the present system 50 "has a predetermined router 90" and a hub 92 ", and the router 90"
0, and a plurality of operator terminals 20 ″ are connected to the hub 92 ″. The operator terminal 20 ″ is composed of a personal computer or the like, and its control part has the same configuration as the server 2 shown in FIG.
a '', an input / output control means 2b '', and an information (synthesis) separation section 2c ''). The audio information is branched into a handset 96 ″ and an IVR 4 ″ by a branching unit 94 ″.
4 '' performs the same operation as in FIG. 1, and returns the processing result to the control means 2a ''.

In each of the above embodiments, the voice information of the user is analyzed. However, the character information of the user may be analyzed. The system in this case may have the same configuration as that of FIG. 1, but the part corresponding to the information separating unit 2c in FIG.
Image information and character information are separated based on the header information among the packets transmitted and received from the network. Then, instead of the voice recognition unit, a predetermined character recognition unit analyzes the corresponding character code from the character information and returns the code to the control unit. Further, information is stored in the related information database corresponding to the character code, and the control means searches for predetermined related information in accordance with the information. The transmission of the character information from the user may be achieved by transmitting an e-mail from the user terminal or performing chat communication with the operator terminal.

FIG. 15 shows an example of an electronic mail. FIG.
In the mail 3000 shown in (1), the character input by the user includes a character string 3000a of "angry", and information related to the character code of "angry" is stored in the related information database. Therefore, when the character code is obtained, the related information is searched. In the mail 3100 shown in FIG. 15 (2), the character input by the user includes the emoticon "@
/ "3000a, and information related to these character codes is stored in the related information database. Therefore, when this character code (“＼” “/”) is obtained,
Related information is searched.

The related information search system of the present invention can be realized by a computer, various peripheral devices such as a communication device, and a software program executed by the computer. It can be distributed via a computer-readable storage medium or a communication line.

[0045]

As described above, according to the present invention,
The state of the user is determined according to the voice information and character information transmitted from the user who has viewed the first image information, and related information reflecting the determination result in real time can be presented to the user. Related information that is useful or easily accepted by the user can be provided. For example,
The advertising effect given to the user increases.

Since the user utters voice information and character information after browsing the first image information, the user's true mental state based on the impression of looking at the screen can be provided without giving a preconception or time for reflection. It is possible to acquire voice information and character information reflecting the above, etc., and it is possible to enhance the reliability of the determination of the state of the user.

Further, when the related information is searched based on the word extracted from the voice information of the user in addition to the above-mentioned state of the user, the search accuracy of the related information is improved.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a related information search system of the present invention.

FIG. 2 is a diagram showing exchange of information between a user terminal and the present system.

FIG. 3 is a diagram showing a processing flow in the related information search system.

FIG. 4 is a block diagram illustrating a configuration of a voice recognition unit.

FIG. 5 is a diagram showing a processing flow in a voice recognition unit.

FIG. 6 is a diagram showing a mode of performing voice analysis from voice information.

FIG. 7 is a diagram illustrating a processing flow for determining a state of a user.

FIG. 8 is a diagram illustrating a method for calculating a temporal change in audio speed.

FIG. 9 is a diagram showing criteria for judging a state of a user.

FIG. 10 is a diagram showing a configuration of related information data.

FIG. 11 is a diagram showing a display screen on a user terminal.

FIG. 12 is another diagram showing a display screen on the user terminal.

FIG. 13 is a block diagram showing another configuration of the related information search system of the present invention.

FIG. 14 is a block diagram showing still another configuration of the related information search system of the present invention.

FIG. 15 is a diagram showing an example of an electronic mail.

[Explanation of symbols]

Reference Signs List 2 voice / IP conversion server 2a control means 2c information (synthesis) separation unit (information separation means) 4 analysis means (speech recognition unit) 6 first image information database 8 related information database 50 related information search system 70, 72, 74 User terminal 100 network 110 mobile communication network

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G10L 15/10 G10L 3/00 531N 15/18 537G 15/00 551P 15/28

Claims (10)

[Claims] 1. A method for presenting first image information to a user terminal of a user so that the first image information can be browsed, and searching for related information corresponding to voice information or character information transmitted from the user who browsed the first image information. A control system, comprising: a control unit; an analysis unit that analyzes the voice information or the character information; and a related information database that stores the related information in association with the state of the user. A related information search system that determines the state of the user based on the obtained voice information or character information, and searches the relevant information database for relevant information based on the determination result. 2. The related information search system according to claim 1, wherein the control unit presents the searched related information to the user. 3. The apparatus according to claim 1, wherein the control unit determines the state of the user based on one of a speed at which a voice is emitted and a tone of the voice in the analyzed voice information. Related information search system described in. 4. The related information is further stored in association with a predetermined word, the analysis unit extracts a word from the voice information, and the control unit reads the word from the related information database based on the extracted word. The related information search system according to any one of claims 1 to 3, wherein the related information is searched. 5. The related information search system is connected to an operator terminal of a predetermined operator, and is capable of exchanging voice information or character information between the user and the operator. 5. The related information search system according to any one of 1 to 4. 6. The user terminal is capable of transmitting the voice information or character information via a network, the related information search system is connected to the user terminal via the network, and includes image information and voice information. 6. The related information search system according to claim 1, further comprising an information separating unit configured to separate image information and character information. 7. The related information search system according to claim 1, wherein the character information transmitted from the user is an electronic mail. 8. Presenting first image information to a user terminal of the user in a viewable manner, and searching for related information corresponding to voice information or character information transmitted from the user who browsed the first image information. Storing the related information in association with the state of the user; analyzing the voice information or character information; and the state of the user based on the analyzed voice information or character information. And a step of retrieving relevant information based on the result of the determination. 9. The related information search method according to claim 8, further comprising a step of presenting the searched related information to the user. 10. Presenting first image information to a user terminal of the user in a viewable manner, and searching for related information corresponding to voice information or character information transmitted from the user who browsed the first image information. A computer-readable recording medium recording a method of performing the method, wherein the step of storing the related information in association with the state of the user; the step of analyzing the audio information or the character information; and the analyzed audio information or A computer readable recording for recording a program for causing a computer to execute a step of determining the state of the user based on character information and a step of searching for relevant information based on the determination result. Medium.