WO2017060994A1

WO2017060994A1 - System and method for controlling information presented to user who refers to content

Info

Publication number: WO2017060994A1
Application number: PCT/JP2015/078535
Authority: WO
Inventors: ミャオメイレイ; 利昇三好; 芳樹丹羽; 佐藤　大樹
Original assignee: 株式会社日立製作所
Priority date: 2015-10-07
Filing date: 2015-10-07
Publication date: 2017-04-13
Also published as: JPWO2017060994A1; US10699057B2; JP6469242B2; US20180285351A1

Abstract

This system for controlling information presented to a user who refers to a content comprising a plurality of elements: outputs the content to an output apparatus; obtains biological information about the user at the time when each of the elements included in the content is outputted to the output apparatus; calculates user's understanding level of each of the elements included in the content on the basis of the corresponding obtained biological information; and, when determining, on the basis of the calculated understanding level, that the user does not understand the content, determines an understanding type on the basis of the calculated understanding level and a difficulty level indicated by difficulty level information of the element of which the understanding level is equal to or lower than a first threshold, and outputs, to the output apparatus, presentation information corresponding to the understanding type that has been determined.

Description

System and method for controlling information to be presented to user browsing content

The present invention relates to a system and method for controlling information to be presented to a user who refers to content.

In recent years, with the development of technology for visualizing the brain, not only physiological knowledge about the brain has been enriched, but also the estimation of the human state from brain measurement signals has been performed.

Methods for non-invasive measurement of brain activity include electroencephalogram measurement, functional nuclear magnetic resonance imaging (fMRI), magnetoencephalography, near infrared light measurement. Method (NIRS: Near-InfraRed Spectroscopy).

As a background art in this technical field, there is JP-A-2011-150408 (Patent Document 1). In this publication, “in the workplace using a machine, the internal state (visual attention, working memory, skill proficiency, etc.) from the biological measurement signal of the operator is estimated by applying a machine learning algorithm. When it is estimated that human error is likely to occur and it is estimated that there is a high risk of negligence, it is possible to eliminate the negligence by feeding back to the operator by visual, auditory, or tactile sensation, or a combination thereof. “Prevent it in advance” (see summary).

JP 2011-150408 A

The technique described in Patent Literature 1 describes a method for estimating an internal state from a user's biological measurement signal, but it is necessary to identify the cause of the internal state and to improve the internal state. It cannot be controlled.

That is, the technique described in Patent Literature 1 can estimate the user's degree of understanding of the information from the biological measurement signal of the user who refers to the presented information. However, in the state where the degree of understanding is low, the user Cannot identify the cause of the inability to understand the information. As a result, the technique described in Patent Document 1 cannot present information according to a cause that is not understood.

Also, for example, it is often difficult for the user himself / herself to identify the contents that the user cannot understand in the presented information and the cause that the user cannot understand when the degree of understanding of the user is low. Thus, according to one embodiment of the present invention, an objective understanding level of a user for presented information and a cause when the information cannot be understood are specified, and information corresponding to the understanding level and the cause is presented.

The following configuration is adopted in one aspect of the present invention. A system for controlling information to be presented to a user who refers to content composed of a plurality of elements, including a processor, a storage device, and an output device, wherein the storage device includes the content and the plurality Holding difficulty level information indicating the difficulty level of each of the elements, and presentation information corresponding to each understanding type indicating a reason why the user cannot understand the content, and the processor outputs the content to the output device Then, the user's biological information at the time when each element included in the content is output to the output device is acquired, and the user's understanding of each element included in the content is acquired based on each of the acquired biological information The user does not understand the content based on the calculated understanding level. And determining the understanding type based on the calculated understanding level and the level of difficulty indicated by the difficulty level information of the element whose understanding level is equal to or less than the first threshold, and presenting corresponding to the determined understanding type A system for outputting information to the output device.

According to one aspect of the present invention, the user's objective understanding of the presented information and the cause when the information cannot be understood are specified, and the information corresponding to the understanding and the cause is presented. can do.

Issues, configurations, and effects other than those described above will be clarified by the following description of the embodiments.

1 is a block diagram illustrating a configuration example of an interactive device in Embodiment 1. FIG. 4 is an example of text included in text data in the first embodiment. 6 is a flowchart illustrating an example of information presentation processing according to the first exemplary embodiment. 3 is an example of a content selection screen according to the first embodiment. 10 is a flowchart illustrating an example of a difficulty level analysis process for presentation information according to the first exemplary embodiment. 4 is an example of a difficulty level analysis result in Example 1. FIG. 3 is an example of hemoglobin concentration data in Example 1. It is a block diagram which shows the structural example of the measurement system in Example 1. FIG. It is a block diagram which shows the structural example of the dictionary generation apparatus for comprehension degree discrimination | determination in Example 1. FIG. 6 is a flowchart illustrating an example of an understanding level determination dictionary generation process according to the first exemplary embodiment. 6 is a flowchart illustrating an example of an understanding level determination process according to the first embodiment. It is an example of the understanding level discrimination | determination result in Example 1. FIG. 10 is a flowchart illustrating an example of a collation process between a content difficulty level and a user understanding level determination result according to the first exemplary embodiment. It is an example of the collation result in Example 1. It is an example of the time synchronization result in Example 1. 6 is a flowchart illustrating an example of an understanding type determination process according to the first embodiment. It is an example of the understanding type result in Example 1. 6 is a flowchart illustrating an example of a presentation information control process according to the first embodiment. 3 is an example of control content in the first embodiment. It is an example of the warning message output on the touch panel in Example 1. In Example 1, it is an example of a user interface in case a user does not understand a word is an understanding type. FIG. 10 is an example of a user interface in the first embodiment when the user does not understand the syntax is an understanding type. FIG. FIG. 10 is an example of a user interface when the understanding type is that the user does not understand the topic in the first embodiment. FIG. 10 is an example of a user interface when the voice speed is the understanding type in the first embodiment. FIG. FIG. 10 is an example of a user interface in the case where the understanding type indicates that there is a voice in the first embodiment. FIG.

This embodiment describes an interactive system. The interactive system presents information to the user, and calculates the user's degree of understanding of the information from the user's biological information referring to the information. The dialogue system compares the calculated degree of understanding with the degree of difficulty of the presented information to identify an understanding type that indicates the cause that the user cannot understand the information, and then presents it according to the degree of understanding and the understanding type. To control what information.

Fig. 1 shows a configuration example of a dialogue system. The dialogue system 101 includes, for example, a dialogue device 102, a touch panel 103, and a biological information measuring instrument 104. The interactive device 102 is configured by a computer including a processor (CPU) 121, an auxiliary storage device 105, a memory 106, an input / output interface 122, and a communication interface 123, for example.

The processor 121 executes a program stored in the memory 106. The memory 106 includes a ROM that is a nonvolatile storage element and a RAM that is a volatile storage element. The ROM stores an immutable program (for example, BIOS). The RAM is a high-speed and volatile storage element such as a DRAM (Dynamic Random Access Memory), and temporarily stores a program executed by the processor 121 and data used when the program is executed.

The auxiliary storage device 105 is a large-capacity non-volatile storage device such as a magnetic storage device (HDD) or a flash memory (SSD), and stores a program executed by the processor 121 and data used when the program is executed. To do.

The input / output interface 122 is an interface to which the touch panel 103 or the like is connected, receives an input from an operator or the like, and outputs the execution result of the program in a format that can be visually recognized by the operator or the like. The touch panel 103 receives character input and voice input from the user, and outputs character information and voice information. The input / output interface 122 may be connected to input devices such as a keyboard, a mouse, and a microphone, and output devices such as a display device, a printer, and a speaker.

The communication interface 123 is a network interface device that controls communication with other devices according to a predetermined protocol. The communication interface 123 includes a serial interface such as USB. For example, the communication interface 123 is connected to the biological information measuring instrument 104 that measures the biological information of the user. A device that measures a change in cerebral blood volume, which is an example of a brain function by near infrared spectroscopy, is an example of the biological information measuring device 104. Moreover, the biological information measuring device 104 may acquire brain function information by another measurement method such as magnetic field measurement.

The program executed by the processor 121 is provided to the interactive device 102 via a removable medium (CD-ROM, flash memory, etc.) or a network, and is stored in the nonvolatile auxiliary storage device 105 that is a non-temporary storage medium. Good. For this reason, the dialogue apparatus 102 may have an interface for reading data from a removable medium.

The interactive device 102 is a computer system configured on a single computer or a plurality of computers configured logically or physically, and operates in a separate thread on the same computer. Alternatively, it may operate on a virtual machine constructed on a plurality of physical computer resources.

The auxiliary storage device 105 stores, for example, text data 107, audio data 108, and understanding level determination dictionary data 109. The text data 107 includes a plurality of texts. For example, a news article, a book that the user is interested in, a textbook or a reference book for each subject of elementary school, junior high school, and high school are examples of each text included in the text data 107. Further, the contents of advertisements in marketing and the contents that the administrator wants to advertise are examples of each text included in the text data 107. Details of each text included in the text data 107 will be described later.

The voice data 108 includes a voice associated with each of a plurality of texts included in the text data 107. Each voice included in the voice data includes content equivalent to the corresponding text. Each of the sounds included in the sound data is, for example, a synthesized sound that can adjust speed and beat. Further, the dialogue apparatus 102 may have a function of newly adding, deleting, and editing the text data 107 and the voice data 108 as necessary. The understanding level discrimination dictionary data 109 stores in advance the understanding level discrimination dictionary generated by the understanding level discrimination dictionary generation process described later. Details of the comprehension determination dictionary will be described later.

The memory 106 includes an information presentation unit 110, a difficulty level analysis unit 111, a biological information acquisition unit 112, an understanding level determination unit 113, a matching unit 114, an understanding type determination unit 115, and an information control unit 116, which are programs.

The program is executed by the processor 121 to perform a predetermined process using a storage device and a communication port (communication device). Therefore, the description with the program as the subject in the present embodiment may be an explanation with the processor 121 as the subject. Alternatively, the process executed by the program is a process performed by a computer and a computer system on which the program operates.

The processor 121 operates as a functional unit (means) that realizes a predetermined function by operating according to a program. For example, the processor 121 functions as an information presentation unit (information presentation unit) by operating according to the information presentation unit 110, and functions as a difficulty level analysis unit (difficulty level analysis unit) by operating according to the difficulty level analysis unit 111. . The same applies to other programs. Further, the processor 121 also operates as a functional unit (means) that realizes each of a plurality of processes executed by each program. A computer and a computer system are an apparatus and a system including these functional units (means).

The information presentation unit 110 outputs the content of the text data 107 and / or the audio data 108 selected according to the instruction from the user, for example, to the touch panel 103 as presentation information. The content consists of a plurality of elements. For example, when the content is text data, each word included in the text data is an example of a content element. The biometric information acquisition unit 112 acquires a time series of the user's biometric information measured by the biometric information measuring device 104 when the user understands the presentation information output by the information presentation unit 110.

User understanding activity refers to an activity in which the user understands the presentation information with one of the five senses. For example, reading the presentation information in the text format by the user and listening to the presentation information in the voice format are examples of the user's understanding activities. In addition, the time series of biological information in the present embodiment indicates biological information at two or more time points. In addition, each time series of biological information includes signals of one or more channels. The brain activity signal is an example of biological information.

The difficulty analysis unit 111 analyzes the difficulty of the text content included in the text data and the audio content included in the audio data at the presentation time. The understanding level determination unit 113 refers to the understanding level determination dictionary included in the understanding level determination dictionary data 109, and from the time series of the biological information at the time of the user's understanding activity on the presentation information acquired by the biological information acquisition unit 112, The user's degree of understanding of the presentation information at each time is calculated.

The collation unit 114 collates the user's understanding level calculated by the understanding level determination unit 113 with the difficulty level of the presentation information analyzed by the difficulty level analysis unit 111, and synchronizes the time. The understanding type determination unit 115 determines the understanding type of the user based on the time synchronization result of the matching unit 114. The user understanding type indicates a reason why the user cannot understand the content when the user does not understand the content. The information control unit 116 controls information to be next presented to the user based on the user's understanding type determined by the understanding type determination unit 115.

FIG. 2 is an example of text included in the text data 107. A text 200 is an example of a text of a news article in English learning for a Japanese user. The text 200 includes, for example, a content type 201, a background 202, content 203, and a word interpretation 204. The content type 201 includes a keyword indicating the type of content of the text 200. In the example of FIG. 2, the content of the text 200 is written in English about brain science news.

The background 202 includes background knowledge of the content of the text 200. The background 202 may include background knowledge of multiple types of languages. In the example of FIG. 2, the background 202 includes background knowledge of the English version and the Japanese version. Further, the background 202 may include image data such as a photograph related to the background knowledge of the content.

The content 203 includes a plurality of versions of content. In the example of FIG. 2, the content 203 includes a plurality of versions of content corresponding to the word level and syntax level used in the content. Although not particularly illustrated, the content 203 holds information indicating a time (for example, a time based on the output start time) at which a word included in the content is output via the touch panel 103.

For example, “Version 1: Easy Level” in FIG. 2 is a word having 1000 or less vocabulary, “Version 2: Intermediate Level” is a word having 1000 to 5000 vocabulary, and “Version 3: Advanced Level” is a word having 5000 or more vocabulary. As described above, a level is assigned in advance to each version of content. The word interpretation 204 includes, for example, the meaning of a word having a high difficulty level included in the corresponding version of the content.

The difficulty level analysis unit 111 may assign a level to each version of content included in the content 203. The difficulty level analysis unit 111 assigns a level with reference to a corpus, for example. The corpus is, for example, a text database in a text format that includes texts in English textbooks from elementary schools to universities, English news articles, and the like. The corpus may be stored in the auxiliary storage device 105, for example, or may be stored in another computer connected to the interactive device 102.

The difficulty level analysis unit 111 acquires, for example, a predetermined number or more of sentences from the corpus, sorts words included in the acquired sentences in descending order of appearance frequency, and sets a plurality of sorted word sets for each predetermined number of words. Divide into hierarchies. Hereinafter, a hierarchy having a low appearance frequency is defined as a hierarchy corresponding to a high level.

The difficulty level analysis unit 111 determines that the content corresponding to the Nth hierarchy is included in the Nth hierarchy when words of a predetermined ratio or more included in the content are included in the Nth hierarchy and not included in the (N−1) th hierarchy. Is granted. For example, when the lower limit of the Nth hierarchy is 1000 words and the upper limit is 2000 words, the level corresponding to the Nth hierarchy indicates a level of 1000 to 2000 words.

FIG. 3 shows an example of information presentation processing performed by the information presentation unit 110. The information presentation unit 110 selects the text format content stored in the text data 107 and / or the audio format content corresponding to the content stored in the audio data 108 in accordance with an input from the user via the touch panel 103. (S301).

Specifically, the information presentation unit 110 accepts input of content type and version. The information presentation unit 110 selects the text having the input content type 201 from the text data 107 and selects the input version of the content from the text content 203.

When there are a plurality of texts having the input content type 201, the information presentation unit 110 may select one text from the plurality of texts at random, or present the plurality of texts to the user. The text may be selected in accordance with an input from the user. The information presentation unit 110 acquires content in an audio format corresponding to the selected content from the audio data 108.

The information presentation unit 110 selects the presentation format of the content selected in step S301 in accordance with an input from the user via the touch panel 103 (S302). Specifically, for example, the information presenting unit 110 receives and inputs information indicating whether the content is presented in either text format or audio format, or both text format and audio format. Select the presentation format according to the information.

Hereinafter, in the present embodiment, an example in which the information presentation unit 110 selects both the text format and the audio format will be described. However, the processing when either the text format or the audio format is selected is also described below. It is the same. Subsequently, the information presentation unit 110 outputs the content selected in step S301 to the touch panel 103 according to the time information included in the content 203 corresponding to the selected content in the presentation format selected in step S302. (S303).

In steps S301 and S302, the information presentation unit 110 may select the content and the presentation format from the content included in the text data 107 and / or the content included in the audio data 108, for example, at random.

FIG. 4 shows an example of a content selection screen which is a user interface for the user to select content. The content selection screen 400 includes, for example, a content type selection section 401, a version selection section 402, and a presentation format selection section 403.

The content type selection section 401 receives an input of content type. In the example of FIG. 4, the user can select a content type from subject-specific, foreign language, format, and topic selection in the content type selection section 401. Further, the content type selection section 401 may accept an input of content type by accepting an input of a keyword.

The version selection section 402 accepts version input. In the example of FIG. 4, the user can select a version from beginner, intermediate, and advanced. The presentation format selection section 403 receives an input for selecting a presentation format.

FIG. 4 shows an example in which a middle-level text content including foreign language, English, news article, and brain science in the content type 201 is selected, and a text and audio presentation format is selected. Note that the information presentation unit 110 may display the content type of the content that the user may be interested in “recommended” in the content type selection section 401 based on the content selection history by the user.

FIG. 5 shows an example of the difficulty level analysis process of the presentation information performed by the difficulty level analysis unit 111. The difficulty level analysis unit 111 scores the difficulty level of text data and voice data presented by the information presentation unit 110.

First, the difficulty level analysis unit 111 acquires the content presented by the information presentation unit 110 and the output time of each word included in the content from the information presentation unit 110 (S501). Here, it is assumed that the difficulty level analysis unit 111 receives both text content input and audio content input. The difficulty level analysis unit 111 determines the difficulty level of the input text format content and audio format content (S502).

The difficulty level analysis unit 111 determines the difficulty level of text content based on, for example, criteria such as words and syntax used in the content. Further, the difficulty level analysis unit 111 determines the difficulty level of the content in the audio format based on, for example, a reference such as a speed and a hit of the content to be reproduced. A specific example of the difficulty level determination method will be described later. The difficulty level analysis unit outputs the difficulty level analysis result generated in step S502 and stores it in the memory 106, for example (S503).

FIG. 6 is an example of the difficulty analysis result generated by the difficulty analysis unit 111. FIG. 6 is an example of the difficulty level analysis result of English content used for English education. The difficulty analysis result 600 includes, for example, a time 601, a word 602, a word difficulty 603, a syntax difficulty 604, a speed difficulty 605, and a hit difficulty 606. The time 601 indicates the time when the corresponding word 602 is displayed on the touch panel 103 as text and output from the touch panel 103 as sound, for example.

The word 602 indicates a word included in the content. The words included in the word 602 are extracted from the sentences included in the content by the difficulty level analysis unit 111 using, for example, morphological analysis. Further, the content included in the text data 107 and the audio data 108 may be divided in units of words in advance.

The word difficulty level 603 indicates the difficulty level of each corresponding word 602. Hereinafter, an example of processing in which the difficulty level analysis unit 111 calculates the word difficulty level 603 of each word will be described. The difficulty level analysis unit 111 calculates, for example, the appearance rate of each word in the corpus (number of appearances of the word in the corpus / total number of words including duplication in the corpus). A difficulty level analysis unit, for example, determines a value obtained by substituting the calculated appearance rate into a predetermined decreasing function having a range of 0.0 to 1.0 as the word difficulty level 603 of the word. Thereby, a word having a low appearance rate in the corpus is regarded as a word having a high word difficulty, that is, a difficult word, and a word having a high appearance rate is regarded as having a low word difficulty, that is, a simple word.

The syntax difficulty level 604 indicates the difficulty level of each corresponding word 602. Hereinafter, an example of processing in which the difficulty level analysis unit 111 calculates the difficulty level of the syntax of each word will be described. The difficulty level analysis unit 111 searches for a syntax to which each word belongs using means such as syntax analysis. A noun phrase, a verb phrase, a verb phrase with an auxiliary verb, a continuous modification phrase, a combination modification phrase, a proper expression, etc. are examples of syntax. A word may belong to more than one syntax or may not belong to any syntax.

The difficulty analysis unit 111 calculates the appearance rate in the corpus of each syntax to which each word belongs (number of appearances in the corpus of the syntax / total number of syntaxes including duplication in the corpus). For each word, the difficulty level analysis unit 111 selects, for example, the lowest appearance rate from the appearance rate of the syntax to which the word belongs. A value obtained by assigning the selected appearance rate to a predetermined decreasing function having a range of 0.0 to 1.0 is determined as the syntax difficulty 604 of the word. Further, the difficulty level analysis unit 111 determines the syntax difficulty level 604 of a word that does not belong to any syntax, for example, 0.0. Thereby, a word belonging to a syntax having a low appearance rate in the corpus is regarded as a word having a high syntax difficulty level, and a word having a high appearance rate is regarded as a word having a low syntax difficulty level.

The difficulty level analysis unit 111 may, for example, select an appearance rate at random instead of selecting the lowest appearance rate from the appearance rate of the syntax to which the word belongs, or the appearance rate of the syntax to which the word belongs. An average value or the like may be calculated.

The speed difficulty 605 indicates the difficulty according to the speed when the corresponding word 602 is read out as audio content. The speed difficulty level 605 is given by, for example, a deviation from the speed of the standard word speech. Hereinafter, an example of processing in which the difficulty level analysis unit 111 calculates the speed difficulty level 605 of each word will be described.

The difficulty level analysis unit 111 acquires the same word as each word from, for example, a speech corpus. The speech corpus is a text database in a speech format that includes, for example, a voice that is read out in English as a standard language from an elementary school to a university, or a voice that is read out in English as a news article in English. The voice corpus may be stored, for example, in the auxiliary storage device 105 or may be stored in another computer connected to the interactive device 102.

The difficulty level analysis unit 111 calculates, for example, the average speed of the same word as the word acquired from the speech corpus for each word, and determines the deviation from the average speed as the speed of the word. For example, a speed slower than the average speed is given as a negative value, and a speed faster than the average speed is given as a positive value. The difficulty level analysis unit 111 determines the value obtained by substituting the determined speed into a predetermined increase function having a value range of 0.0 to 1.0, for example, as the speed difficulty level 605. Thereby, a word with high speed is regarded as a word with high speed difficulty, and a word with low speed is regarded as a word with low speed difficulty.

The difficulty level 606 indicates the difficulty level due to the speech when the corresponding word 602 is read out as audio content. Hereinafter, an example of processing in which the difficulty level analysis unit 111 calculates the difficulty level 606 of each word will be described. The difficulty level analysis unit 111 generates a parameter indicating the accent of each word, for example. For example, for each word, the difficulty level analysis unit 111 calculates the average parameter of the accent of the same word acquired from within the speech corpus, and uses the absolute value of the deviation from the average parameter as the beat of the word. decide. The difficulty analysis unit 111 determines a value obtained by substituting the determined beat for a predetermined increase function having a range of 0.0 to 1.0, for example, as the beat difficulty 606. Thereby, a word having a large deviation from the accent of the standard word is regarded as a word having a high difficulty level, and a word having a small difference is regarded as a word having a low difficulty level.

The difficulty level analysis unit 111 determines the difficulty level of each type of each word through the above processing. In addition, the difficulty level of each kind in a present Example shows that the parameter | index which the said difficulty level of a kind shows is so difficult that a value is large, and it shows that the parameter | index which the difficulty level of the said kind shows is so easy that a value is small.

FIG. 7 is an example of hemoglobin concentration data that is an example of biological information acquired by the biological information acquisition unit 112. The example of the hemoglobin concentration data in FIG. 7 shows a time series of the oxygenated hemoglobin concentration and the reduced hemoglobin concentration of the user who performs the understanding activity. For example, the near-infrared light measurement device, which is an example of the biological information measuring instrument 104, uses near-infrared spectroscopy, and oxyhemoglobin concentration and / or reduced hemoglobin concentration in blood at a plurality of measurement sites on the surface of the user's brain. Measure the time series of.

The biological information measuring device 104 may measure, for example, the hemoglobin concentration in the whole brain, or may measure the hemoglobin concentration in only the frontal lobe where the language field that understands the language and the cognitive activity. The biological information measuring device 104 irradiates the living body with near infrared light, for example. The irradiated light enters the living body, is scattered and absorbed in the living body, and the living body information measuring device 104 detects the light that has propagated out.

Note that the biological information measuring instrument 104 performs measurement using, for example, a method described in Patent Document 1 for obtaining a blood flow change in the brain from an internal state when the user performs an understanding activity. The biological information acquisition unit 112 acquires the hemoglobin concentration measured by the biological information measuring device 104 and the hemoglobin concentration when the user performs an understanding activity.

FIG. 8 shows a configuration example of a measurement system that measures brain measurement data used to generate an understanding level discrimination dictionary stored in the understanding level discrimination dictionary data 109. The measurement system 801 includes a measurement management device 802, a touch panel 803, and a biological information measuring instrument 804. The measurement management device 802 includes, for example, a computer including a processor (CPU) 821, an auxiliary storage device 805, a memory 806, an input / output interface 822, and a communication interface 823.

The touch panel 803, the biological information measuring instrument 804, the processor (CPU) 821, the auxiliary storage device 805, the memory 806, the input / output interface 822, and the communication interface 823 are each described in the touch panel 103, the biological information measuring instrument 104, and the processor (CPU). 121, the auxiliary storage device 105, the memory 106, the input / output interface 122, and the communication interface 123 are the same as those described above, and will not be described.

The memory 806 includes an information presentation unit 811 and a biological information acquisition unit 812, each of which is a program. The auxiliary storage device 805 stores, for example, text data 807, voice data 808, and biometric data 810. The description of the text data 807 and the sound data 808 is the same as the description of the text data 107 and the sound data 108, and will not be repeated. The content included in the text data 807 and the audio data 808 may be the same as or different from the content included in the text data 107 and the audio data 108. The biological data 810 stores the biological information acquired by the biological information acquisition unit 812.

The information presentation unit 811 selects content from the text data 807 and / or audio data 808 in accordance with an instruction from the user, for example, and presents the selected content to the user via the touch panel 803.

Specifically, for example, the information presentation unit 811 selects content that is well understood by the user and content that is difficult to understand in accordance with an instruction from the user. Thereby, both the biological information in a state in which the user understands the information presented by the biological information acquisition unit 812 and the biological information in a state in which the user does not understand can be acquired. For example, if the user is Japanese, the text written in Japanese words at the elementary school level is an example of the content understood by the user, and written in a foreign language that the user has never studied. A sentence is an example of content that is difficult for the user to understand.

The biometric information acquisition unit 812 acquires biometric information of a user who performs an understanding activity on the presentation information presented by the information presentation unit 811 from the biometric information measuring device 804, and uses the acquired biometric information as time-series data. Biometric information for content that is understood and biometric information for content that is difficult for the user to understand may be stored separately in biometric data 810. The biometric information acquisition unit 812 may add a user identifier to each acquired biometric information and store the biometric information in the biometric data 810.

FIG. 9 shows an example of the configuration of an understanding level discrimination dictionary generating apparatus that generates an understanding level discrimination dictionary stored in the understanding level discrimination dictionary data 109. The comprehension level determination dictionary generation device 902 is configured by a computer including a processor (CPU) 921, an auxiliary storage device 905, a memory 906, an input / output interface 922, and a communication interface 923, for example.

The processor (CPU) 921, auxiliary storage device 905, memory 906, input / output interface 922, and communication interface 923 are each described with respect to the processor (CPU) 121, auxiliary storage device 105, memory 106, input / output interface 122, and communication interface. The description of each of 123 is omitted because it is the same as that described above.

The memory 906 includes an understanding level discrimination dictionary generation unit 911 which is a program. The understanding level discrimination dictionary generation unit 911 uses the information stored in the biometric data 910 to generate an understanding level discrimination dictionary, and stores the generated understanding level discrimination dictionary in the understanding level discrimination dictionary data 909.

The auxiliary storage device 905 stores the understanding level discrimination dictionary data 909 and the biometric data 910. The understanding level determination dictionary data 909 generates the understanding level determination dictionary created by the understanding level determination dictionary generation unit 911. The biometric data 910 stores biometric information equivalent to the biometric information stored in the biometric data 810 in advance. In the present embodiment, the dialogue system 101, the measurement system 801, and the understanding level determination dictionary generation device 902 are described as separate systems, but may be a single system.

FIG. 10 shows an example of a dictionary generation process for understanding degree discrimination. From the biometric data 910, the understanding level discriminating dictionary generation unit 911 includes a plurality of time series of biometric information for content that is well understood by the user and a plurality of time series of biometric information for content that is difficult for the user to understand. Obtain (S1001). In the following processing, the understanding level determination dictionary generation unit 911 generates an understanding level determination dictionary from the training data generated from the acquired data.

The understanding level determination dictionary generation unit 911 performs preprocessing on the signals of the respective channels included in the time series of the biological information acquired in step S1001 (S1002). For example, a near-infrared light measurement device, which is an example of the biological information measuring instrument 804, performs measurement using a non-invasive cerebral hemodynamic measurement method using light. Signals related to brain activity and information related to systemic hemodynamics such as heart rate variability are included.

Therefore, the understanding level determination dictionary generation unit 911 performs preprocessing for removing noise, which is information not related to the level of understanding of content, on the acquired biometric information, thereby improving the accuracy of the understanding level determination dictionary. Can be increased. The understanding level discrimination dictionary generation unit 911 performs preprocessing in step S1002 using an algorithm such as a frequency bandpass filter, polynomial baseline correction, principal component analysis, or independent component analysis.

Subsequently, the understanding level discrimination dictionary generation unit 911 extracts only essential features necessary for understanding level understanding from each time series of the preprocessed biological information (S1003). Specifically, for each time series that has undergone the preprocessing in step S1002, the understanding level determination dictionary generation unit 911 obtains a feature vector at one or more sample times from the signal of the channel included in the time series. calculate. Each sample time is, for example, a predetermined time interval.

種々 There are various methods for calculating the feature vector. Hereinafter, an example in which the understanding level determination dictionary generation unit 1011 calculates a feature vector at a sample time t from a time series having biological information will be described. The time at the start of understanding activities is set to zero.

First, the understanding level discrimination dictionary generation unit 911 sets a time window including the sample time t. Here, the understanding level discrimination dictionary generation unit 911 sets a time window from t−δ to t + δ for a predetermined positive number δ. The comprehension determination dictionary generation unit 911 cuts out a signal from time t−δ to time t + δ from the signal of each channel included in the time series subjected to the preprocessing in step S1002.

The understanding level discrimination dictionary generation unit 911 calculates a predetermined basic statistic (for example, peak amplitude value, average value, variance, gradient, skewness, kurtosis, etc.) from the extracted signal of each channel. The comprehension determination dictionary generation unit 911 selects a channel signal with the highest sensitivity based on the calculated basic statistics. A highly sensitive channel refers to a channel that strongly reflects the characteristics of the signal in the time window. The f value of dispersion, the peak amplitude value, and the slope are examples of sensitivity. For example, the understanding level determination dictionary generation unit 911 generates a feature vector having one or more types of basic statistics as elements.

Subsequently, the understanding level discrimination dictionary generation unit 911 generates a discrimination function by optimizing the parameters of the classification algorithm using the generated feature vector (S1004). The identification function is a function that receives a feature vector at a certain time generated from a time series of biological information at the time of the user's understanding activity and outputs the degree of understanding of the user at the understanding activity at the time. In this embodiment, it is assumed that the degree of understanding is given by a numerical value of 0 or more and 1.0 or less, that is, the value range of the discrimination function is 0 or more and 1.0 or less. The higher the value of the understanding level, the more the user understands the content that is the object of the understanding activity at the time.

The optimization of the parameters of the classification algorithm in step S1004 is performed by the user so that the degree of understanding becomes as high as possible when, for example, a feature vector corresponding to a time series of biological information with respect to content that the user understands well is input. This indicates that the parameters of the classification algorithm are determined so that the understanding level is as low as possible when a feature vector corresponding to a time series of biological information for difficult content is input.

The support vector machine based on the margin maximization principle, linear discriminant analysis, sparse logistic regression, logistic regression, nonlinear classification algorithm hidden Markov model, neural network, etc. are examples of classification algorithms.

Subsequently, the understanding level discrimination dictionary generation unit 911 includes the generated discriminant function in the understanding level discrimination dictionary, and stores the understanding level discrimination dictionary in the understanding level discrimination dictionary data 909 (S1005). Further, the understanding level discrimination dictionary generation unit 911 calculates a class (understood / not understood) to which each feature vector belongs by collating the generated feature vector with the generated identification function, The correspondence between each feature vector and the class to which it belongs may be included in the understanding level determination dictionary. Note that if the degree of understanding corresponding to the feature vector is greater than or equal to a predetermined value, the feature vector belongs to the “understanding” class, and if the degree of understanding corresponding to the feature vector is less than the predetermined value, the feature vector is Assume that you belong to a class that you do not understand.

It should be noted that the above-described understanding level determination dictionary is a general-purpose understanding level determination dictionary that can be applied to any user, but biological information such as brain measurement results has a characteristic pattern for each user. Therefore, for example, when the user identifier is given to the time series of the biological information stored in the biological data 910, the understanding level determination dictionary generation unit 911 creates an understanding level determination dictionary for each user. May be. In the understanding level determination process to be described later, the understanding level determination unit 113 calculates an understanding level using the understanding level determination dictionary for each user, so that a higher level of understanding can be obtained.

FIG. 11 shows an example of understanding level determination processing in the dialogue system 101. The understanding level determination unit 113 acquires, from the biological information acquisition unit 112, a time series of the user's biological information when performing an understanding activity on the presented information (S1101). The understanding level determination unit 113 performs preprocessing on the signal of each channel included in the acquired time series of biological information, for example, in the same manner as in step S1002 (S1102).

The understanding level determination unit 113 extracts only essential features necessary for understanding level identification from the time series of the preprocessed biological information (S1103). Specifically, the understanding level determination unit 113 generates a feature vector at each time point in time series of biological information that has been preprocessed, for example, in the same manner as in step S1003.

The understanding level determination unit 113 assigns the feature vector at each time to the identification function included in the understanding level determination dictionary stored in the understanding level determination dictionary data 109, and calculates the understanding level of the user at each time ( S1104). Note that, when the understanding level determination dictionary stores an identification function for each user, it is preferable that the understanding level determination unit 113 uses an identification function corresponding to the user in step S1104. Subsequently, the understanding level determination unit 113 outputs, for example, the calculated understanding level at each time and stores it in the memory 106 (S1105).

FIG. 12 is an example of the understanding level determination result. The understanding level determination result in FIG. 12 indicates the understanding level at each time calculated by the understanding level determination unit 113.

FIG. 13 shows an example of a collation process between the difficulty level of the content and the user's understanding level determination result by the collation unit 114. The collation unit 114 collates the difficulty level of the voice data analyzed by the difficulty level analysis unit 111 with the understanding level analyzed by the understanding level determination unit 113 on a time axis, and synchronizes the time (S1301). Subsequently, the collation unit 114 collates the difficulty level of the text analyzed by the difficulty level analysis unit 111 with the understanding level analyzed by the understanding level determination unit 113 on a time axis, and synchronizes the time (S1302).

In step S1301 and step S1302, the collation unit 114 synchronizes the difficulty level and the understanding level with time, for example, in increments within a certain period of time. Subsequently, the collation unit 114 outputs the difficulty level of the time-synchronized voice data, the difficulty level of the text data, and the understanding level, and stores them in, for example, the memory 106 (S1303).

FIG. 14 is an example of a collation result by the collation unit 114. The collation result shows the correspondence between the degree of understanding at each time, the word difficulty and the syntax difficulty of the text data presented by the user at each time, and the speed and utterance of the speech data presented by the user at each time. Show.

FIG. 15 shows an example of the time synchronization result of the text difficulty level, the voice difficulty level, and the understanding level. For example, the collation unit 114 collates the understanding level determination result, the text difficulty level, and the voice difficulty level in each section of a predetermined cycle such as 1000 ms at each time, for example.

FIG. 16 shows an example of an understanding type determination process by the understanding type determination unit 115. The understanding type determination unit 115 acquires the time synchronization result generated by the collation unit 114 (S1601). For example, when the time when the comprehension level is equal to or less than the first threshold exceeds the first percentage of the comprehension activity time, the comprehension type determination unit 115 determines the text difficulty level information and the voice at each time when the comprehension level is equal to or less than the first threshold value. With reference to the difficulty level information, the understanding type at the time of the user's understanding activity is determined (S1602). When the time during which the degree of understanding is equal to or less than the first threshold is equal to or less than the first ratio of the understanding activity time, the understanding type determination unit 115 determines that the user understands the content.

Hereinafter, a specific example of the understanding type determination process in step S1602 will be described. For example, when the comprehension level is 50% or less at 80% or less of the comprehension activity time, the comprehension type discriminating unit 115 causes the text difficulty level information and the voice at each time of the time group whose comprehension level is 50% or less. And the difficulty level information.

For example, when there is a time in the time group in which the word difficulty is equal to or higher than the second threshold, the understanding type determination unit 115 determines that the understanding type includes that the user does not understand the word. For example, when there is a time in the time group whose syntax difficulty level is equal to or greater than the third threshold, the understanding type determination unit 115 determines that the understanding type is included because the user does not understand the syntax.

For example, when there is a time in the time group in which the speed difficulty is equal to or higher than the fourth threshold, the understanding type determination unit 115 determines that the understanding type includes that the speed of the voice is high. For example, when there is a time in the time group in which the difficulty level of speaking is greater than or equal to the fifth threshold, the understanding type determination unit 115 determines that the understanding type includes that the sound is speaking.

For example, when the understanding level is equal to or less than a predetermined threshold at a time equal to or higher than the second rate of the understanding activity time (the second rate is equal to or lower than the first rate), the understanding type determination unit 115 causes the topic of the content to be the user. Is determined to be included in the understanding type.

Note that, when there are a plurality of understanding types, the understanding type determination unit 115 may narrow down the understanding type, that is, the cause that the user cannot understand the content. For example, the priority is determined in advance for each understanding type, and the understanding type determination unit 115 narrows the plurality of understanding types to a predetermined number of understanding types according to the priority.

Further, for example, the understanding type determination unit 115 may determine the priority of each of the plurality of understanding types. When the understanding type includes that the user does not understand the topic, the understanding type determination unit 115 indicates that the user does not understand the topic, for example, a value obtained by multiplying the average value of the understanding level by the first weight. The priority corresponding to is determined.

When the understanding type includes that the user does not understand the word, the understanding type determination unit 115 sets, for example, the average value of the word difficulty level that is equal to or higher than the second threshold value of the word whose understanding level is equal to or lower than the first threshold value. The value multiplied by the second weight is determined as the priority corresponding to the fact that the user does not understand the word. When the understanding type includes that the user does not understand the syntax, the understanding type determination unit 115 sets, for example, an average value of the syntax difficulty level that is equal to or higher than the third threshold value of a word whose understanding level is equal to or lower than the first threshold value. The value multiplied by the third weight is determined as the priority corresponding to the fact that the user does not understand the syntax.

When the understanding type includes that the speed of speech is high, the understanding type determination unit 115 uses, for example, the fourth weight to the average value of the speed difficulty level that is equal to or higher than the fourth threshold value of the word whose understanding level is equal to or lower than the first threshold value. The value obtained by multiplying is determined as the priority corresponding to the high speed of the voice. When the understanding type includes that the voice is spoken, for example, the understanding type determination unit 115 sets the fifth weight to the average value of the difficulty of speaking that is equal to or higher than the fifth threshold of words whose understanding is equal to or lower than the first threshold. A value obtained by multiplying by is determined as a priority corresponding to the voice being uttered.

FIG. 17 shows an example of the understanding type result. In the example of FIG. 17, the current user's understanding type is that the word cannot be understood, and it is highly possible that the user cannot understand the entire presented information because the word cannot be understood. That is, the information control unit 116 can improve the degree of understanding of the user by controlling the information to be presented based on the understanding type.

FIG. 18 shows an example of presentation information control processing by the information control unit 116. The information control unit 116 acquires the understanding type result determined by the understanding type determination unit 115 (S1801). That is, the understanding type determination unit 115 acquires information indicating whether or not the user understands the content, and further understands the understanding type when the user does not understand the content.

The information control unit 116 determines whether the user understands the content according to the acquired understanding type result (S1802). When it is determined that the user does not understand the content (S1802: NO), the information control unit 116 controls the information to be presented according to the understanding type indicated by the understanding type result (S1803), and presents the next information. (S1804). A specific example of the next information in step S1804 in the case of going through step S1803 will be described later. When it is determined that the user understands the content (S1802: YES), the information control unit 116 presents the next information, for example, another content (S1804).

FIG. 19 shows an example of control contents in step S1603. The information control unit 116 performs a control process in accordance with a predetermined control content corresponding to the understanding type. In the example of FIG. 19, when the understanding type is a word, the control is “rephrase with a simple word having a vocabulary number of 1000 or less”, and when the understanding type is a syntax, the control is “rephrase with a simple syntax”. An example in which the control when the topic is "Introduction of background knowledge", the control when the understanding type is speed is "Slowly rephrase", and the control when the understanding type is Speak is "Rephrase with standard words" Show.

FIG. 20 is an example of a warning message output from the information control unit 116 to the touch panel 103. For example, if the information control unit 116 determines in step S1802 that the user does not understand the content, the information control unit 116 outputs the warning message in FIG.

When the user selects the check box corresponding to “as is”, for example, the information control unit 116 presents the same information as in step S1804 when the user understands the content in step S1802.

When the user selects the check box corresponding to “information control”, in step S1803, the information control unit 116 controls information to be presented next according to the understanding type. Hereinafter, a specific example of the step S1803 and the step S1804 performed by the user display screen and the information control unit 116 when the user selects the check box corresponding to “information control” will be described.

FIG. 21A is an example of a user interface that the information control unit 116 outputs to the touch panel in step S1803 when the user does not understand the word is an understanding type. The user interface displays, for example, a message indicating that the user does not understand the word and a countermeasure, and text-format content in which a word having a word difficulty level equal to or greater than the second threshold is surrounded by a thick frame. Note that words whose degree of understanding is not more than the first threshold and whose word difficulty is not less than the second threshold may be surrounded by a thick frame.

When the user selects a check box corresponding to “meaning of word”, for example, the information control unit 116 acquires the meaning of the word surrounded by a thick frame from the word interpretation 204 and outputs it to the touch panel 103. Thereby, the user can understand the meaning of the word with a low understanding level.

Also, when the user selects the check box corresponding to “Rephrase with simple version”, for example, the information control unit 116 selects a simpler version of the content from the content 203 and outputs it to the touch panel 103.

FIG. 21B is an example of a user interface that the information control unit 116 outputs to the touch panel in step S1803 when the user does not understand the syntax is an understanding type. The user interface displays, for example, a message indicating that the user does not understand the syntax and a countermeasure, and text-format content in which words having a syntax difficulty level equal to or greater than the third threshold value are surrounded by a thick frame. Note that words whose degree of understanding is not more than the first threshold and whose syntax difficulty is not less than the third threshold may be surrounded by a thick frame.

When the user selects the check box corresponding to “as is”, the information control unit 116 performs the same processing as when the understanding type does not include syntax in the next information presentation. When the user selects a check box corresponding to “easy version”, the information control unit 116 selects a simpler version of the content from the content 203 and outputs it to the touch panel 103.

FIG. 22A is an example of a user interface that the information control unit 116 outputs to the touch panel in step S1803 when the user does not understand the topic is an understanding type. The user interface displays, for example, a message indicating that the user does not understand the topic and a countermeasure, and text content.

When the user selects the check box corresponding to “Japanese”, the information control unit 116 selects the background knowledge of the Japanese version of the content from the background 202 and outputs it to the touch panel 103. When the user selects the check box corresponding to “English”, the information control unit 116 selects the background knowledge of the English version of the content from the background 202 and outputs it to the touch panel 103. When the user selects a check box corresponding to “photo”, the information control unit 116 selects a photo from the background 202 of the content and outputs it to the touch panel 103.

FIG. 22B is an example of a user interface that the information control unit 116 outputs to the touch panel in step S1803 in the case where the understanding type is that the voice speed is high. The user interface displays, for example, a message indicating that the user does not understand the voice due to the high speed of the voice and a countermeasure and a text format content.

When the user selects the check box corresponding to “as is”, the information control unit 116 performs the same processing as when the understanding type does not include speed in the next information presentation. When the user selects the check box corresponding to “slow”, the information control unit 116 decreases the speed of, for example, all the words in the presented content or the words whose speed difficulty is the fourth threshold value or more by a predetermined value. Alternatively, the voice data lowered to a predetermined value is created and output.

FIG. 22C is an example of a user interface that is output from the information control unit 116 to the touch panel in step S1803 in the case where the understanding type is that there is a voice. The user interface displays, for example, a message indicating that the user does not understand the voice due to speech and a countermeasure indicating the countermeasure, and text content.

When the user selects the check box corresponding to “as is”, the information control unit 116 performs the same processing as when the understanding type does not include speed in the next information presentation. When the user selects the check box corresponding to “standard word”, the information control unit 116, for example, accents the standard word of all the words in the presented content or the word whose difficulty of speaking is the fifth threshold value or more. Create and output audio data with a deviation from a predetermined value lowered or reduced to a predetermined value.

Up to this point, in step S1803, the information control unit 116 outputs the screens as shown in FIGS. 20 to 22C, the user selects the information in the screen, and determines the next presentation information according to the selected information. explained.

In step S1803, the information control unit 116 does not output the screens shown in FIGS. 20 to 22C, and automatically determines any of the information corresponding to the check boxes in FIGS. 20 to 22C as the next presentation information. May be. In addition, for example, when the information control unit 116 is of the understanding type that the user does not understand the syntax, the ratio of the words whose word difficulty in the presented content is greater than or equal to the second threshold is predetermined. If the value is greater than or equal to the value, the meaning of a word having a word difficulty level equal to or greater than the second threshold may be output, or a simpler version of the content may be output.

As described above, the interactive system 101 according to the present embodiment compares the biological information at the time of the user's understanding activity on the presentation information with the difficulty level of the presentation information. Can be accurately identified. In addition, the dialogue system 101 according to the present embodiment determines the information to be presented next based on the degree of understanding and the understanding type, thereby having the same contents as the presentation information and in accordance with the degree of understanding of the user. Information can be presented. Specifically, for example, the dialogue system 101 can increase the user's understanding level by presenting information in which elements having a low degree of understanding and a high degree of difficulty are replaced with elements having a low level of difficulty. .

As described above, the dialogue system 101 can determine the degree of understanding of the user with respect to the presentation information. Therefore, the dialogue system 101 is applied not only to the content related to education as described above but also to content in fields such as advertising, marketing, and medical care. You can also. The contents of these fields may be stored in the text data 107 and the audio data 108 in advance.

In addition, the memory 106 may include a voice recognition unit that performs speech language recognition. For example, instead of the information presentation unit 110 and the information control unit 116 receiving the above-described user input, the voice recognition unit receives a voice language received from the user. Is received and transmitted to the information presentation unit 110 and the information control unit 116. As a result, the dialogue system 101 can communicate with humans in natural language.

In the present embodiment, the information presentation unit 110 has described an example in which one or both of text and voice are selected in the presentation format selection in step S302. However, data in other presentation formats such as music, images, and videos is supplemented. When stored in the storage device 105, the other presentation format may be selected.

Further, the difficulty level analysis unit 111 of the present embodiment analyzes the difficulty level, but may accept a direct input of the difficulty level from the user. For example, in the case of conducting a market survey on the degree of understanding of the moving image of an advertisement using the interactive system 101, the researcher sets the difficulty level at a predetermined time of the moving image, for example, so that the interactive system 101 performs an investigation at the predetermined time. The subject's understanding type can be output.

In the first embodiment, an example in which the biological information measuring instrument 104 and the biological information measuring instrument 804 use near-infrared spectroscopy as a brain function measuring method has been described. However, the biological information measuring instrument 104 and the biological information measuring instrument 804 have an electroencephalogram. Alternatively, techniques such as functional magnetic resonance imaging may be used.

Moreover, the biological information measuring instrument 104 and the biological information measuring instrument 804 may further include an eye tracking device, a camera, and the like, and may observe a line of sight and a facial expression. At this time, the biological information acquisition unit 112 and the biological information acquisition unit 812 further acquire the line-of-sight information and facial expression information recognized by the biological information measuring instrument 104 and the biological information measuring instrument 804.

The understanding level determination unit 113 and the understanding level determination dictionary generation unit 911 perform feature extraction from biological information including line-of-sight information and face information. As described above, the interactive system 101 of this embodiment can obtain a higher degree of understanding.

In addition, this invention is not limited to the above-mentioned Example, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of a certain embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of a certain embodiment. Further, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

In addition, each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit. Each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor. Information such as programs, tables, and files for realizing each function can be stored in a memory, a hard disk, a recording device such as an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD.

Also, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. Actually, it may be considered that almost all the components are connected to each other.

Claims

A system for controlling information to be presented to a user who refers to content composed of a plurality of elements,
A processor, a storage device, and an output device;
The storage device
The content;
Difficulty information indicating the difficulty of each of the plurality of elements;
And presenting information corresponding to each understanding type indicating a reason why the user cannot understand the content,
The processor is
Outputting the content to the output device;
Obtaining the biometric information of the user at the time when each element included in the content is output to the output device;
Based on each of the acquired biometric information, to calculate the degree of understanding of the user for each element included in the content,
When it is determined that the user does not understand the content based on the calculated understanding level,
Based on the calculated degree of understanding and the degree of difficulty indicated by the difficulty level information of the element whose degree of understanding is equal to or less than a first threshold,
A system for outputting presentation information corresponding to the determined understanding type to the output device.
The system of claim 1, comprising:
The storage device holds a plurality of types of difficulty information,
When it is determined that the user does not understand the content based on the calculated understanding level,
Selecting a first element having an understanding level equal to or less than the first threshold;
Obtaining the difficulty level of the selected element, which is equal to or higher than a threshold corresponding to the difficulty level information to which the difficulty level belongs, from the plurality of types of difficulty level information;
Based on the acquired difficulty comparison result, one or more difficulty information is selected from the plurality of types of difficulty information,
A system for determining an understanding type based on the selected one or more difficulty information.
The system of claim 1, comprising:
The presentation information includes first presentation information, which is the content obtained by replacing each element of the first element group including one or more elements of the content with an element having a lower difficulty level than the element,
The first presentation information corresponds to a first understanding type,
The processor is
When it is determined that the user does not understand the content based on the calculated understanding level,
For each element of the first element group, it is determined whether or not the first condition that the degree of understanding is the first threshold value or less and the difficulty level indicated by the difficulty level information is the second threshold value or more is satisfied.
The system which determines an understanding type to the said 1st understanding type when it determines with all the elements of the said 1st element group satisfy | filling the said 1st condition.
The system of claim 1, comprising:
The presentation information includes second presentation information indicating background knowledge of the content,
The second presentation information corresponds to a second understanding type,
The processor determines the understanding type as the second understanding type when it is determined that an understanding degree equal to or greater than a predetermined percentage of the calculated understanding degree is equal to or less than a third threshold value.
The system of claim 1, comprising:
The content includes text data in text format,
Each of the plurality of elements is a word included in the sentence data,
The system in which the difficulty level information indicates at least one of a difficulty level of each word included in the sentence data and a difficulty level of a syntax to which each word included in the sentence data belongs.
6. The system according to claim 5, wherein
The difficulty level information indicates the difficulty level of each word included in the sentence data,
The presentation information includes third presentation information indicating interpretation of a first word included in the sentence data,
The third presentation information corresponds to a third understanding type,
The processor is
Based on the calculated degree of understanding, it is determined that the user does not understand the content, the degree of understanding of the first word is less than or equal to the first threshold, and the degree of difficulty of the first word is greater than or equal to a fourth threshold. If it is determined that
A system for determining an understanding type as the third understanding type.
6. The system according to claim 5, wherein
The difficulty level information indicates the difficulty level of each word included in the sentence data,
The processor is
Obtain first sentence data in text format containing multiple words,
The system which determines the difficulty level of the said word based on the appearance rate in the said 1st sentence data of each word contained in the said sentence data.
6. The system according to claim 5, wherein
The difficulty level information indicates the difficulty level of the syntax to which each word included in the sentence data belongs,
The processor is
Acquire second text data in text format that contains multiple words,
A system for determining a difficulty level of a syntax to which the word belongs based on an appearance rate in the second sentence data of a syntax to which each word included in the text data belongs.
The system of claim 1, comprising:
The content includes text data in audio format,
Each of the plurality of elements is a word included in the sentence data,
The difficulty level information includes the difficulty level of the speed when each word included in the sentence data is output to the output device, and the resentment level when each word included in the sentence data is output to the output device. A system that shows at least one of difficulty.
10. The system according to claim 9, wherein
The difficulty level information indicates a difficulty level of speed when each word included in the sentence data is output to the output device,
The processor is
Acquire third sentence data in audio format containing multiple words,
Based on the comparison result between the speed at which each word included in the sentence data is output to the output device and the speed at which the word included in the third sentence data is output to the output device. A system for determining a difficulty level of speed when each word included in the sentence data is output to the output device.
10. The system according to claim 9, wherein
The difficulty level information indicates the difficulty level of the hit when each word included in the sentence data is output to the output device,
The processor is
Acquire the fourth sentence data in audio format that contains multiple words,
Based on the comparison result between the utterance when each word included in the sentence data is output to the output device and the utterance when the word included in the fourth sentence data is output to the output device. The system for determining the difficulty level of swearing when each word included in the sentence data is output to the output device.
The system of claim 1, comprising:
The processor is
Outputting a first element that the user can understand and a second element that the user cannot understand to the output device;
Obtaining the first biometric information of the user referring to the first element and the second biometric information of the user referring to the second element;
Based on the training data including the first biological information and the second biological information, an understanding level determination dictionary that indicates the correspondence between the biological information of the user and the understanding level is generated,
A system that calculates the user's degree of understanding of each element included in the content based on each of the acquired biological information and the understanding level determination dictionary.
The system of claim 1, comprising:
The biological information includes a brain function information, line-of-sight information, and facial expression information.
A method for controlling information to be presented to a user who refers to content composed of a plurality of elements by a system including an output device,
The system
The content;
Difficulty information indicating the difficulty of each of the plurality of elements;
And presenting information corresponding to each understanding type indicating a reason why the user cannot understand the content,
The method
The system is
Outputting the content to the output device;
Obtaining the biometric information of the user at the time when each element included in the content is output to the output device;
Based on each of the acquired biometric information, to calculate the degree of understanding of the user for each element included in the content,
When it is determined that the user does not understand the content based on the calculated understanding level,
Based on the calculated degree of understanding and the degree of difficulty indicated by the difficulty level information of the element whose degree of understanding is equal to or less than a first threshold,
A method of outputting presentation information corresponding to the determined understanding type to the output device.