JP6712027B1 - Learning support system - Google Patents

Abstract

[Problem] To more appropriately send to students. A student-oriented transmitting unit 101 transmits student-oriented information. The language operation unit 102 receives an operation for inputting language information by a student, and transmits the language information input by the received operation to the server device 20 as a response of the student. The non-language sensing unit 103 senses non-language information emitted by the student, and transmits the sensed non-language information to the server device 20 as a response from the student. The response information acquisition unit 201 acquires the response of the student including the required time and the correctness of the answer. The emotion estimation unit 204 estimates the current emotion of the student based on the acquired response. The transmission information determination unit 205 determines the content of the student-oriented information transmitted to the student based on the estimated emotion of the student. [Selection diagram] Fig. 6

Description

The present invention relates to computer-aided education.

As a technique related to computer-aided education, for example, in Patent Document 1, when a user is authenticated based on personal data read from a student ID, data of a learning problem corresponding to the personal data is data. A technique is disclosed in which an image forming unit receives an image from a providing apparatus and an image forming unit forms an image of a test question based on the received data.

JP, 2014-29572, A

As computer-aided education, for example, a new question is transmitted based on the correctness of the student's answer and the time required. However, even the same student changes his/her attitude to tackle the problem depending on the state at that time. I have something to do.
Therefore, an object of the present invention is to appropriately transmit information to a student according to the state of the student.

To achieve the above object, the present invention provides a response information acquisition means for acquiring a response of the student with respect to information transmitted with respect to the student, the current of the student based on the response to the response information acquisition unit acquires and estimating means for estimating the emotion, and determining means for determining a information originating to the student based on the emotion that the estimating means has estimated, transmits the information to the determining means has determined for the student And a supplementary information acquisition unit for acquiring information about the student as additional information from an information source other than the student , wherein the estimation unit is configured to transmit the student based on the additional information acquired by the additional information acquisition unit. to provide a learning support system that estimates the current feelings of.

Further, the determining means may determine the information to be transmitted to the student, using the additional information acquired by the additional information acquiring means .

Further, the additional information acquisition means may acquire information regarding the schedule of the student or the organization to which the student belongs as additional information from an external device via a network .
Further, the additional information acquisition means may acquire information transmitted by the student via the Social Networking Service as additional information.

Further, the deciding means decides the information to be transmitted to the related person of the student based on the emotion estimated by the estimating means, and is for the related person to transmit the information decided by the deciding means to the concerned person. A transmission means may be provided .

In addition, the determining means provides the student with information based on the information transmitted by the student transmitting means before the response information acquiring means acquires the information used by the estimating means to estimate the current emotion of the student. You may decide the information to transmit .

Further, the present invention provides a response information acquisition unit that acquires the response of the student to the information transmitted to the student, and an estimation unit that estimates the current emotion of the student based on the response acquired by the response information acquisition unit. A learning support system including: a determining unit that determines information to be transmitted to the student based on the emotion estimated by the estimating unit; and a transmitting unit for a student that transmits the information determined by the determining unit to the student. A robot having a movable part imitating a part of a human body, and a server device communicatively connected to the robot, wherein the student-originating means is arranged in the robot, the response information acquiring means, The estimating means and the determining means are arranged in both the robot and the server device, and the estimating means arranged in the robot is shorter than the estimating means arranged in the server device in the student. When the current emotion is estimated and the emotion estimated by the estimation means arranged in the robot has changed to promptly transmit, the determination means arranged in the robot is stored in the server device. A learning support system for deciding information to be transmitted to the student without waiting for the decision of the deciding means arranged .

According to the present invention, it is possible to appropriately transmit to a student according to the state of the student.

The figure showing an example of the whole composition of the learning support system concerning an example. Diagram showing the appearance of the robot Diagram showing an example of the hardware configuration of a robot The figure showing an example of the hardware constitutions of a server apparatus. The figure showing an example of the hardware constitutions of a participant terminal. Diagram showing the functional configuration of the learning support system Illustration representing the circle of emotions of Prutic The figure showing an example of the structure of the table used for estimating a student's emotion based on the content of a student's response The figure showing an example of the structure of the table used for the determination of the information for students according to the emotion of a student The figure showing an example of the structure of the table used for estimating a student's emotion based on the additional information obtained from the student's person concerned The figure showing an example of the operation procedure in transmission processing. The figure showing an example of another operation procedure in transmission processing. A diagram showing a functional configuration of a learning support system according to a modification. A diagram showing a functional configuration of a learning support system according to a modification. A diagram showing a functional configuration of a learning support system according to a modification.

[1] Example FIG. 1 shows an example of the overall configuration of a learning support system 1 according to an example. The learning support system 1 is a system that supports learning for students. The learning support system 1 includes a network 2, a wearable device 3, a robot 10, a server device 20, a teacher terminal 30, and a guardian terminal 40.

The network 2 is a communication system including a mobile communication network, the Internet, and the like, and relays data exchange between devices that access the own system. The server device 20 is connected to the network 2 by wire communication, and the robot 10, the teacher terminal 30, and the guardian terminal 40 are connected by wireless communication. The communication between each device and the network 2 may be wired or wireless. The wearable device 3 is connected to the robot 10 by wireless communication.

The wearable device 3 is a device that is worn and used by a student, and measures the amount of activity of the student. The wearable device 3 is, for example, a wristwatch-type device worn on the wrist of a student, and measures motion information (moving direction and speed, etc.) of the hand of the student, heart rate, and the like as activity amounts. The wearable device 3 transmits the measured amount of activity to the robot 10.

The robot 10 has a function of communicating with students, and sends, for example, information for students regarding student learning (hereinafter referred to as “student information”) and receives student responses to the sent student information. To do. The robot 10 is installed on, for example, a study desk in a classroom, and receives various types of operations by accepting the operation of the student or detecting the movement of the student based on the amount of activity measured by the wearable device 3. Take action.

The server device 20 acquires the response of the student received by the robot 10 and estimates the emotion of the student and determines the content of the student-oriented information based on the acquired response. When determining the content of the student-oriented information, the server device 20 notifies the determined content to the robot 10. The robot 10 transmits the notified information for students to the students.

The teacher terminal 30 is a terminal used by a teacher who teaches students, and the parent terminal 40 is a terminal used by parents of the students. Teachers and parents are concerned with the students. Hereinafter, the teacher and the guardian will be collectively referred to as “student related persons”, and the teacher terminal 30 and the guardian terminal 40 will be collectively referred to as “related person terminals 4”. The server device 20 determines not only the information for the students but also the information for the related persons of the students (hereinafter referred to as “information for related persons”).

The related party terminal 4 transmits the determined related party information to the related parties. In addition, the related party terminal 4 is operated by a related person of the student, and information about the student (for example, information about a state observed by a guardian at home before a student goes to the classroom, or a current student observed by the teacher in the classroom). (Information regarding the state of 1) is generated and transmitted to the server device 20. The server device 20 acquires the transmitted information as information added to the response of the student (hereinafter referred to as “additional information”) and utilizes it for estimating the emotion of the student.

FIG. 2 shows the appearance of the robot 10. Although various robots can be adopted as the robot 10, the case where “UNIBO” (registered trademark) sold by Uni Robot Co., Ltd. is adopted as the robot 10 will be described as a suitable example in the following description. To do. The robot 10 is roughly composed of a body portion 1001, a head portion 1002, a right arm portion 1003R and a left arm portion 1003L, a right leg portion 1004R and a left leg portion 1004L.

A touch display 1005 is arranged on the front surface of the head 1002. A camera 1006 is arranged on the touch display 1005 (forehead part). Further, a microphone 1007 is arranged on the upper surface (zenith portion) of the head 1002. Further, a speaker 1008 is arranged on the front surface (belly portion) of the body portion 1001.

The touch display 1005 includes a stacked touch panel 1005A and a display 1005B. The touch panel 1005A detects a touch of a user's finger or a stylus, and the display 1005B displays an image or the like. The camera 1006 is a so-called digital camera, which captures an image of the surroundings of the robot 10 and generates image data indicating the captured image. The microphone 1007 is a so-called microphone, and generates sound data indicating the sound around the robot 10 picked up by itself. The speaker 1008 emits various sounds including the synthesized voice.

The head portion 1002 can rotate (pan) around a vertical axis and rotate (tilt) around a horizontal axis with respect to the body portion 1001. Therefore, the robot 10 can make various gestures by the movement of the head, such as nodding and shaking the head.

The right arm portion 1003R is rotatable about the left and right axes with respect to the body portion 1001 and slightly inclined in the upper right direction, and the left arm portion 1003L is rotatable about the left and right axes with respect to the body portion 1001 and slightly inclined in the upper left direction. It is possible. Therefore, the robot 10 can make various gestures by moving the arms, such as shaking both hands up and down to emphasize the speech.

On the touch display 1005, when a question is given to the student or a commentary is given, texts or diagrams showing the contents of the question are displayed, but while the question is given or the commentary is not given, Images representing the left and right eyes are displayed. The robot 10 can create various facial expressions by changing the shape, position, size, etc. of the left and right eyes. As a result, the student feels the personality of the robot 10, which is a machine, so that the student can easily express his/her feelings toward the robot 10. As a result, the efficiency of learning support described later can be significantly improved.

FIG. 3 shows an example of the hardware configuration of the robot 10. The robot 10 includes a computer 11, an input device 12, an output device 13, and a drive mechanism 14.

The computer 11 includes a processor 111, a memory 112, a storage 113, a communication IF (Interface) 114, an input/output IF (Interface) 115, and a bus 119.

The processor 111 is, for example, a central processing unit (CPU), and controls the entire robot 10 by performing arithmetic processing according to programs such as an operating system and application programs read from the storage 113 to the memory 112.

The memory 112 is a recording medium that records data in a computer-readable manner, and is, for example, a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), a RAM (Random Access Memory), or the like. is there. The storage 113 is also a recording medium that records data so that it can be read by a computer, but generally has a storage capacity larger than that of the memory 112, but has a slower data read/write speed than the memory 112. The storage 113 is, for example, a hard disk drive, a solid state drive, a flash memory, or the like.

The communication IF 114 is an interface for communicating with an external device via a network (at least one of wired and wireless).

The input/output IF 115 is an interface for exchanging data with an external device in a wired or wireless manner without using a ketwork.

The bus 19 is a communication line for other components of the computer 11 (processor 111, memory 112, etc.) to mutually pass data.

The input device 12, the output device 13, and the drive mechanism 14 are connected to the input/output IF 115.

The input device 12 includes a touch panel 1005A, a camera 1006, a microphone 1007, and a wearable device 3. The output device 13 includes a display 1005B and a speaker 1008.

The drive mechanism 14 includes a drive control device 141 that controls the operation of the motor under the control of the computer 11, a head pan motor 142 that operates under the control of the drive control device 141, a head tilt motor 143, and a right arm rotation device. A motor 144 and a left arm rotation motor 145 are provided.

The head pan motor 142 is arranged at a connecting portion between the body portion 1001 and the head portion 1002, and drives the head portion 1002 around the vertical axis with respect to the body portion 1001. The head tilt motor 143 is arranged at the connecting portion between the body portion 1001 and the head portion 1002, and drives the head portion 1002 around the left and right axes with respect to the body portion 1001.

The right arm rotation motor 144 is arranged at a connecting portion between the body portion 1001 and the right arm portion 1003R, and drives the right arm portion 1003R around the axis with respect to the body portion 1001. The left arm rotation motor 145 is arranged at the connecting portion between the body portion 1001 and the left arm portion 1003L, and drives the left arm portion 1003L around the axis with respect to the body portion 1001.

FIG. 4 shows an example of the hardware configuration of the server device 20. The server device 20 includes a computer 21. The computer 21 includes a processor 211, a memory 212, a storage 213, a communication IF 214, an input/output IF 215, and a bus 219. The processor 211 and the like in which the hardware of the same name is represented in FIG. 3 is the same type of hardware as that of FIG. 3 although there are differences in performance and specifications.

FIG. 5 shows an example of the hardware configuration of the related party terminal 4. The party terminal 4 includes a computer 41, an input device 42, and an output device 43. The computer 41 includes a processor 411, a memory 412, a storage 413, a communication IF 414, an input/output IF 415, and a bus 419. The processor 411 etc. in which the hardware of the same name is represented in FIG. 3 is the same kind of hardware as in FIG. 3 although there are differences in performance and specifications.

The input device 42 includes, for example, a keyboard for a person concerned with the student to input text by typing, a microphone for inputting text by voice, a touch panel for inputting text by touch operation, selection of options, and the like. Is included. The output device 43 includes, for example, a display for displaying characters, figures and the like to the students concerned, a speaker for producing sounds such as synthesized voice, and the like.

The functional configuration of the learning support system 1 shown in FIG. 6 is realized by each processor of the robot 10, the server device 20, and the related person terminal 4 executing a program stored in the device itself to control each part of the device itself. To be done. The functional configuration of the learning support system 1 will be described below.

The robot 10 includes a student-oriented transmitting unit 101, a language operating unit 102, and a non-language detecting unit 103. The server device 20 includes a response information acquisition unit 201, a required time calculation unit 202, a correctness determination unit 203, an emotion estimation unit 204, a transmission information determination unit 205, and an additional information acquisition unit 206. The related party terminal 4 includes a related party transmitting section 301 and an additional information operating section 302.

The student-oriented transmitting unit 101 of the robot 10 transmits the above-described student-oriented information to the students. The student transmitting unit 101 is an example of the "transmitting unit" and the "first transmitting unit" of the present invention.

The information for students is, for example, a problem to be applied to the student, advice for solving the problem to the student, a cheering voice that inspires the student, and a display of a smile that pleases the student's answer. The student transmitting unit 101 transmits the student information by displaying the student information on the touch display 1005 of the robot 10 or by uttering the student information from the speaker 1008 of the robot 10.

When the above-mentioned student-oriented information is transmitted from the robot 10, the student responds to the student-oriented information. For example, when the information for students is a question to be given to the students, a response of answering the question is given. When the information for students is advice or shouts to the students, a response such as a reply to the advice or shouts, a change in facial expression, a gesture, a change in posture, a change in voice color, or a change in the above-mentioned activity amount is performed.

Responses by students are roughly divided into verbal information and non-verbal information. The linguistic information is information represented by using a language such as an answer to a question and a reply. The non-verbal information is information expressed without using language, such as facial expression, posture, voice color, speaking speed, gesture, and the above-mentioned activity amount (heart rate, motion information, gaze direction, etc.). The language operation unit 102 receives an operation for inputting language information by a student. The language operation unit 102 is an example of the “operation means” of the present invention.

The language operation unit 102 receives, for example, an operation of inputting a character or a sentence using the touch display 1005 as an operation for inputting language information. In addition, the language operation unit 102 receives an operation of voice input using the microphone 1007 as an operation for inputting language information. The language operation unit 102 adds time information (time stamp) indicating the input time to the language information input by the received operation and transmits it to the server device 20.

The non-verbal sensing unit 103 senses non-verbal information emitted by the student. The non-language sensing unit 103 is an example of the "sensing device" in the present invention. The non-language sensing unit 103 senses, for example, a student's state (a student's facial expression, posture, gesture, gaze direction, etc.) recognized from an image captured by the camera 1006 as non-language information. The non-language sensing unit 103 senses the student's state (voice color, speaking speed, etc.) recognized from the voice picked up by the microphone 1007 as non-language information. Since the camera 1006 and the microphone 1007 detect the non-verbal information by detecting the facial expression, posture, and voice color of the student, the camera 1006 and the microphone 1007 also constitute a part of the non-verbal sensor 103.

Further, the activity amount of the student measured from the wearable device 3 is transmitted to the robot 10 as described above. The non-language sensing unit 103 senses a student's state (heart rate, motion information, etc.) recognized from the amount of activity measured by the wearable device 3 as non-language information. Since the wearable device 3 detects non-verbal information by detecting the state of the student, the wearable device 3 also constitutes a part of the non-verbal sensing unit 103. The non-language sensing unit 103 adds time information (time stamp) indicating the sensing time to the non-language information sensed as described above, and transmits it to the server device 20.

The transmitted language information and non-language information are supplied to the response information acquisition unit 201 of the server device 20. The response information acquisition unit 201 acquires the response of the student to the information transmitted to the student. The response information acquisition unit 201 is an example of the “acquisition unit” and the “first acquisition unit” of the present invention. The response information acquisition unit 201 uses the supplied information, that is, the language information input by the operation accepted by the language operation unit 102 and the non-language information sensed by the non-language sensing unit 103, as a student response. get.

The response information acquisition unit 201 supplies, to the required time calculation unit 202 and the correctness/incorrectness determination unit 203, the answer to the question transmitted as the student-oriented information among the acquired responses. The required time calculation unit 202 calculates the required time required for the student to solve the problem. The required time calculating unit 202 calculates the required time from the interval of the input time of the answer to each question. The required time calculation unit 202 supplies the calculated required time to the response information acquisition unit 201.

The correctness determination unit 203 determines whether the answer to the question input by the student is correct. The server device 20 stores, for example, question data in which a question to be transmitted as student information and a correct answer are associated with each other. The correctness determination unit 203 determines the correctness of the supplied student's answer based on the question data. The correctness determination unit 203 supplies the determined correctness of each answer to the response information acquisition unit 201.

The response information acquisition unit 201 acquires the required time and correctness of the answer supplied as described above as the response of the student. The response information acquisition unit 201 supplies the acquired student response to the emotion estimation unit 204. The emotion estimation unit 204 estimates the current emotion of the student based on the response acquired by the response information acquisition unit 201. The emotion estimation unit 204 is an example of the “estimation unit” in the present invention. In the present embodiment, the emotion estimating unit 204 expresses the emotion in the form of a set of numerical values indicating the height of each of the plurality of parameters that follow the ring of emotions of the plutik.

FIG. 7 shows the circle of emotions of Plutik. Pulchic's emotional ring is represented by eight basic emotions: joy, trust, fear, surprise, sadness, disgust, anger, and prediction. Pulchic's emotional ring represents stronger emotion as it gets closer to the inside. For example, "enthusiasm", "joy" and "comfort" are all classified as "joy", which is one of the eight basic emotions. Of these emotions, "enthusiasm" is the strongest emotion. , "Comfort" is the weakest emotion. Further, a new emotion generated from two emotions is arranged between the outermost emotions. For example, "comfort" and "acceptance" give rise to "love."

Two basic emotions (for example, "joy" and "sorrow"), which are arranged at opposite positions in the Purtic emotional ring, are paired as opposite basic emotions. Therefore, the eight basic emotions are represented by the following four axes (hereinafter referred to as “emotion axes”).
First emotional axis: "joy"-"sadness"
Second emotional axis: "trust"-"dislike"
Third emotional axis: "fear"-"anger"
Fourth emotional axis: "Surprise"-"Prediction"

In the present embodiment, the emotion of the student is expressed by a group of four numerical values (for example, numerical values within the range of -3 to +3) corresponding to each of the above four emotional axes. Hereinafter, a group of these four numerical values will be referred to as a “feeling index value group”.

FIG. 8 shows an example of the configuration of a table used by the emotion estimation unit 204 to estimate the emotion index value group representing the emotion of the student. The table of FIG. 8 shows various response contents (for example, a pattern of change in correctness and an incorrect answer to a problem over time, a pattern of facial expressions recognized from an image captured by the camera 1006, a voice color recognized from a voice picked up by the microphone 1007). Pattern), and each record stores an emotion index value group (numerical value of each of the first emotion axis to the fourth emotion axis) corresponding to the response content of the record. ..

Every time information is supplied from the response information acquisition unit 201, the emotion estimation unit 204 reads the emotion index value group corresponding to the response content represented by the information from the table in FIG. 8, and the information supplied from the response information acquisition unit 201. It is stored in the log table in association with the time information associated with. Then, the emotion estimation unit 204 sets, for each of the first emotion axis to the fourth emotion axis, a large number of emotion index value groups stored in the log table, from the time information associated with them to the current time. The weighted average value is calculated using the weight (the smaller the weight, the longer the time). The collection of the weighted average values of the first emotion axis to the fourth emotion axis thus calculated means a group of student emotion index values estimated at that time. Hereinafter, this emotion index value group is referred to as a “student's current emotion index value group”. The emotion estimation unit 204 supplies the updated emotion index value group to the transmission information determination unit 205 every time the current emotion index value group of the student is updated.

The transmission information determination unit 205 determines the content of the student-oriented information transmitted to the student based on the current emotion index value group of the student supplied from the emotion estimation unit 204. The outgoing information determining unit 205 is an example of the “determining means” in the present invention.

FIG. 9 shows an example of the configuration of a table used by the transmission information determination unit 205 to determine the content of the student-oriented information transmitted to the students. The table in FIG. 9 is a collection of records according to combinations of various conditions and emotion index value groups, and each record stores transmission contents according to combinations of conditions of the records and emotion index value groups. Has been done.

The conditions stored in the table of FIG. 9 are conditions relating to, for example, the type of the transmitted content transmitted immediately before, the type of the response of the student immediately before (whether or not the answer to the question, etc.), etc. It is used to narrow down the content choices that are appropriate for the current situation. Therefore, the transmission information determining unit 205 first extracts from the table of FIG. 9 a record that stores the condition satisfied by the current situation. Then, the transmission information determining unit 205 searches the extracted records for a record storing an emotion index value group that is the closest to the current emotion index value group of the student, and transmits the retrieved record. The content is determined as the content of the student information to be sent to the student at that time. The transmission information determination unit 205 generates student information having the determined content. In addition, when the question to be asked is determined as the content of the student-oriented information, the transmission information determination unit 205 refers to the question data stored in the device itself and generates information indicating the determined question as the student-oriented information.

The transmission information determination unit 205 transmits the generated student-oriented information to the robot 10. The student transmission unit 101 of the robot 10 transmits the transmitted student information, that is, the student information determined by the transmission information determination unit 205 to the student. When the student-oriented information indicates a problem, the student-oriented transmitting unit 101 transmits the student-oriented information by displaying the problem on the display of the robot 10.

Up to this point, the transmission of the information for the students has been described, but below, the transmission of the information for the related persons, which is the information for the related persons of the student, will be described. The information for related parties is, for example, the current emotional state of the student, the learning evaluation of the student on that day (whether the answer to the question is correct or not and its commentary, etc.), the suggestion of a voice call to the student, and the like. The related party transmitting unit 301 of the related party terminal 4 transmits related party information to the related person of the student. The related party transmitting unit 301 is an example of the “second transmitting unit” in the present invention.

The related party transmitting unit 301 transmits the related person information by displaying the related person information on the display of the related person terminal 4 or by uttering the related person information from the speaker of the related person terminal 4. ..

As described above, the emotion estimation unit 204 estimates the current emotion of the student based on the information regarding the response of the student, which is mainly supplied from the response information acquisition unit 201. However, in the present embodiment, the emotion estimation unit 204 uses the additional information regarding the student transmitted from the related person of the student for estimating the current emotion of the student in addition to the information regarding the response of the student.

The student related person inputs the information about the student as additional information in response to the related party information transmitted from the related person terminal 4 as described above. The additional information entered by the parents of the students includes, for example, whether or not there is any disagreement between the students and their family members at home, the dietary intake before going to the classroom, the current physical condition of the students, and the information to be taken before going to the classroom. This is information related to the activities (learning, etc.) that were being conducted. The additional information input by the teacher is, for example, information about how the student notices in the classroom, whether the relationship with other students is good or bad, and the like.

The additional information operation unit 302 of the related person terminal 4 receives an operation for inputting additional information by a related person of the student. The additional information operating unit 302 receives, for example, an operation of inputting a character or a sentence using a keyboard as an operation for inputting additional information. Further, the additional information operation unit 302 accepts a voice input operation using a microphone as an operation for inputting additional information. The additional information operation unit 302 adds the time information (time stamp) indicating the input time to the additional information input by the received operation, and transmits the additional information to the server device 20.

The transmitted additional information is supplied to the additional information acquisition unit 206 of the server device 20. The additional information acquisition unit 206 acquires information about the student from an information source that is not a student. The additional information acquisition unit 206 is an example of the “second acquisition unit” in the present invention. In the present embodiment, the information source that is not a student is a teacher or a guardian who uses the related person terminal 4. The additional information acquisition unit 206 acquires the supplied additional information as information about a student from an information source that is not a student, and supplies it to the emotion estimation unit 204.

The emotion estimation unit 204 estimates the current emotion of the student based on the supplied additional information, that is, the information about the student acquired by the additional information acquisition unit 206. FIG. 10 shows an example of the configuration of a table used by the emotion estimation unit 204 to estimate the emotion index value group representing the emotion of the student based on the additional information. The table of FIG. 10 differs from the table of FIG. 8 in that the content of the first column is “additional information” instead of “response content”. Every time the additional information acquisition unit 206 supplies additional information, the emotion estimation unit 204 searches the table in FIG. 10 for an emotion index group according to the additional information, and adds the retrieved emotion index value group to the additional information. The associated time information is associated and stored in the log table described above. Then, the emotion estimation unit 204 weights and averages a large number of emotion index groups stored in the log table using weights according to the elapsed time, and calculates a current emotion index value group of the student. As described above, the transmission information determining unit 205 determines the student-oriented information and the related-person information according to the emotion index value group calculated in this way.

Each device included in the learning support system 1 performs a transmission process of transmitting information for students and information for related persons based on the above configuration.

FIG. 11 shows an example of an operation procedure in the calling process. The operation procedure shown in FIG. 11 is started, for example, when the student-oriented information is transmitted to the robot 10. First, the robot 10 (student sending unit 101) sends the sent student information (step S11).

Next, the robot 10 (language operation unit 102) accepts an operation for inputting language information by the student (step S12), and transmits the language information input by the accepted operation to the server device 20 as a response of the student. (Step S13). Subsequently, the robot 10 (non-language detecting unit 103) detects the non-language information emitted by the student (step S14), and transmits the detected non-language information to the server device 20 as a response of the student (step S15). The order of steps S12 and S13 and S14 and S15 may be reversed or may be performed in parallel.

The server device 20 (required time calculation unit 202) calculates the required time required for the student to solve the problem based on the transmitted student response (step S21). Next, the server device 20 (correctness determination unit 203) determines the correctness of the answer to the question input by the student based on the transmitted response of the student (step S22). Then, the server device 20 (response information acquisition unit 201) acquires the response of the student including the calculated required time and the correctness of the determined answer (step S23).

Next, the server device 20 (emotion estimation unit 204) estimates the current emotion of the student based on the acquired response (step S24). Subsequently, the server device 20 (transmission information determination unit 205) determines the content of the student-oriented information transmitted to the student based on the estimated emotion of the student (step S25), and outputs the determined student-oriented information. It is transmitted to the robot 10 (step S26). The robot 10 (transmission unit 101 for students) returns to step S11 and transmits the transmitted information for students.

FIG. 12 shows an example of another operation procedure in the calling process. The operation procedure illustrated in FIG. 12 is started, for example, when the related information is transmitted to the related party terminal 4. First, the related party terminal 4 (related party sending unit 301) sends the sent related party information (step S31).

Next, the related party terminal 4 (additional information operation unit 302) receives an operation for inputting additional information by a related person (step S32), and transmits the additional information input by the received operation to the server device 20. (Step S33). The server device 20 (additional information acquisition unit 206) acquires the transmitted additional information as information about the student from an information source that is not a student (step S41).

Next, the server device 20 (emotion estimation unit 204) estimates the current emotion of the student based on the acquired additional information (step S42). Subsequently, the server device 20 (the transmission information determination unit 205) determines the content of the information for the related persons to be transmitted to the related persons based on the estimated emotion of the student (step S43), and the related persons of the determined content The target information is transmitted to the related party terminal 4 (step S44). The related party terminal 4 (related party sending unit 301) returns to step S31 and sends the sent related party information.

In this embodiment, the emotion of the student is estimated based on the student's response to the student information transmitted as described above, and the content of the next student information is determined based on the estimated emotion. Therefore, for example, to a student who is absorbed in the subject being studied and a student who is disgusted by the subject, the subject matter that suits each emotion is transmitted as student-oriented information. As a result, more appropriate information can be transmitted to the students as compared with the case where the information for the students is transmitted regardless of the emotions of the students.

Further, in this embodiment, both the verbal information and the non-verbal information of the student are acquired as the response of the student and used for estimating the emotion. Since the emotions of the students also appear in facial expressions and gestures, the non-verbal information can be used to more accurately estimate the emotions of the students as compared to the case where the non-verbal information is not used. Further, by detecting non-verbal information using a camera, a microphone, or an activity meter, it is possible to estimate the emotion of the student represented by the appearance, voice, or movement of the student.

Teachers and parents who are involved in the students are in a position to have a great influence on their emotions. In the present embodiment, the emotion of the student is estimated based on the additional information acquired by using the related person of the student as the information source. Therefore, the emotion of the student is more accurately estimated as compared with the case where the additional information is not used. be able to. Moreover, since the information for the related persons is transmitted to those related persons, it is possible to control the influence of the related persons on the students.

[2] Modifications Each of the above-described embodiments is merely an example of the embodiment of the present invention, and may be modified as follows. In addition, each embodiment and each modification may be combined with each other as necessary.

[2-1] Information to be transmitted In the embodiment, the learning support system 1 transmits the information for students indicating a problem and the information for related persons indicating advice to the related persons. The student-originated transmitting unit 101 and the related-person-related transmitting unit 301 that transmit the language information are both examples of the “language transmitting unit” of the present invention. It should be noted that the student-originated transmitting unit 101 and the related-person-related transmitting unit 301 may transmit not only linguistic information but also non-linguistic information, and in that case, they are examples of the "non-language transmitting means" of the present invention.

The robot 10 can display a face of an anthropomorphic device on the touch display 1005 and change the facial expression of the face. Therefore, the transmission information determination unit 205 determines the type of facial expression of the robot 10 (such as a smile or a disappointing face) as the student information, and the student transmission unit 101 displays the face of the determined type of expression. Therefore, non-verbal information for students may be transmitted. Further, the robot 10 can move the head 1002 and the right arm 1003R and the left arm 1003L. Therefore, the transmission information determination unit 205 may determine the movement of the robot 10 as the student information, and the student transmission unit 101 may transmit the non-language information for the student by performing the determined movement. .. By sending non-verbal information, students can be told nuances that cannot be conveyed by verbal information alone.

For example, if the correctness of the answers of the students solving the same level of calculation problem changes in the order of correct answer→correct answer→wrong answer→correct answer, the third wrong answer is presumed to be careless miss. In such a case, the robot 10 does not simply tell the student whether the answer is “incorrect” or “correct”, for example, while displaying an expression of doubt, “Eh?” “Is this really the final answer?” In this way, by sending a combination of linguistic information and non-linguistic information, it is possible to encourage the students to voluntarily review and promote further growth.

[2-2] Information Transmission Method The information transmission method in the learning support system 1 is not limited to the one described in the embodiment. For example, instead of displaying the question, which is information for students, on the display of the robot 10, it may be printed by a printer and transmitted to the students as information on a paper medium.

Further, the transmission information determining unit 205 may transmit the generated student-oriented information to the related party terminal 4 instead of the robot 10, and the related party terminal 4 may transmit the student-related information to the related parties. In that case, a person (teacher or guardian) using the person concerned terminal 4 may transmit the transmitted information for students to the student verbally or using the person concerned terminal 4.

[2-3] Function Realization Method The method for realizing each function shown in FIG. 6 is not limited to the method described in the embodiment. For example, the server device 20 may realize the function realized by the robot 10 (for example, the language operation unit 102 or the like), or the function realized by the server device 20 (for example, the transmission information determination unit 205 or the like) may be realized by the robot 10. May be. Further, the functions realized by the robot 10 may be realized by a terminal such as a tablet terminal or a smartphone.

Further, the functions realized by the robot 10 or the server device 20 by one device may be shared by two or more devices. Further, the operation executed by each function may be shared by two or more functions and executed. For example, separate functions may execute the operations such as the information determination and generation performed by the transmission information determination unit 205. In either case, each function shown in FIG. 6 may be realized in the learning support system 1 as a whole.

FIG. 13 is a diagram illustrating the functional configuration of the learning support system 1 of the configuration example in which the robot 10 has all the functions of the server device 20 of the embodiment.

FIG. 14 is a diagram illustrating a response information acquisition unit 201, a required time calculation unit 202, a correctness determination unit 203, an emotion estimation unit 204, and a transmission information determination unit 205 that are included in the server device 20 of the embodiment and have the same function. FIG. 3 is a diagram showing a functional configuration of a learning support system 1 of a configuration example in which the robot 10 includes a unit 104, a required time calculation unit 105, a correctness determination unit 106, an emotion estimation unit 107, and a transmission information determination unit 108. In the learning support system 1 having the configuration shown in FIG. 14, for example, the robot 10 estimates the current emotion of the student in the emotion estimation unit 107 in a shorter time than the emotion estimation unit 204 of the server device 20, although the estimation is rough. Then, when a change in the student's current feelings that should be appropriately transmitted is found, the robot 10 does not wait for the transmission information determination unit 205 of the server device 20 to determine the student-oriented information, and transmits the transmission information. The deciding unit 108 decides the student-oriented information, and the student-oriented information is transmitted by the student-oriented transmitting unit 101.

On the other hand, in the robot 10, when there is no change in the current feelings of the student that should be appropriately and promptly transmitted, the student-oriented sending unit 101 sends the student-oriented information sent from the outgoing information determination unit 205 of the server device 20. Send information.

According to the learning support system 1 having the configuration of FIG. 14, for example, the information for students who need to promptly send information such as grouping can be determined and sent by the robot 10, so that more appropriate sending can be performed for the students. It will be possible.

FIG. 15 is a diagram illustrating a functional configuration of the learning support system 1 of a configuration example in which the server device 20 includes part of the functions of the non-language sensing unit 103 included in the robot 10 in the embodiment. In the learning support system 1 having the configuration shown in FIG. 15, for example, the non-language sensing unit 103 of the robot 10 uses the non-language information to calculate the feature amount (for example, if the non-language information is an image, coordinates indicating the position of the edge of the image, etc.). If the non-verbal information is voice, the volume of each syllable, the time between syllables, etc.) is extracted, and the characteristic amount information indicating the extracted characteristic amount is transmitted to the server device 20. The non-verbal detection unit 207 of the server device 20 means the meaning of the non-verbal information based on the feature amount information received from the robot 10 (for example, if the non-verbal information is an image of a student's face, the image represents a smile). If the non-linguistic information is voice, it means that the voice has a tense tone color).

According to the learning support system 1 having the configuration of FIG. 15, the robot 10 does not need to perform the process for specifying the meaning of the non-verbal information. Further, according to the learning support system 1 having the configuration of FIG. 15, the amount of data to be communicated is significantly reduced as compared with, for example, the case of transmitting image data or audio data from the robot 10 to the server device 20.

[2-4] Arrangement Position of Robot In the embodiment, the robot 10 is arranged in the classroom. The position where the robot 10 is arranged is not limited to the classroom. For example, the robot 10 may be placed at home, or the robot 10 may move as a student moves.

[2-5] Form of Student Terminal In the embodiment, the robot 10 is used as a terminal used by a student. If the terminal used by the student is a robot having a physically movable part as in the embodiment, it is desirable because it can transmit non-verbal information by movement. On the other hand, a robot provided with a movable part is generally expensive and long compared to a terminal such as a tablet type personal computer having no movable part. Therefore, in consideration of price and mobility, as the terminal used by the student, a terminal having no movable part may be adopted. Even in that case, the communication with the student can be enriched by using the non-verbal information such as the facial expression displayed on the display of the terminal and the voice color of the voice uttered from the speaker.

[2-6] Type of Wearable Device and Necessity of Use In the embodiment, as the wearable device 3, a wristwatch type device worn on the wrist of the student is used. The form of the wearable device 3 is not limited to the wristwatch type, and may be, for example, a necklace type device, a glasses type device, or the like.

The wearable device 3 is not always necessary. For example, if the movement of the student and the heart rate can be estimated from the image captured by the camera 1006, the wearable device 3 for measuring them is unnecessary.

[2-7] Types of Measuring Means Included in Robot In the embodiment, the robot 10 includes a touch panel 1005A, a camera 1006, and a microphone 1007 as measuring means for measuring information for estimating a student's emotions. .. The types of measuring means included in the robot 10 (or the terminal used by the student) are not limited to these. For example, the robot 10 may be connected to an infrared camera that generates a thermograph representing the distribution of the body temperature of the student, and the thermograph generated by the infrared camera may be used to estimate the emotion of the student.

[2-8] Movable Part of Robot In the embodiment, the movable part of the robot 10 is the head 1002, the right arm 1003R, and the left arm 1003L, but the movable part of the robot 10 is not limited to these. For example, a configuration may be adopted in which the right leg 1004R and the left leg 1004L move.

Further, the moving direction of the movable part of the robot 10 is not limited to that shown in the embodiment. For example, a configuration may be adopted in which the head 1002 moves in the roll direction in addition to the pan and tilt. Further, the right arm 1003R and the left arm 1003L may be provided with joints on the sides, elbows and wrists, and the joints may be used to bend the right arm 1003R and the left arm 1003L in various directions.

[2-9] AI
In the embodiment, the current emotion of the student is estimated based on the information stored in the tables having the configurations illustrated in FIGS. 8 and 10. Alternatively, when the learning support system 1 inputs the content of the response of the student and the additional information supplied from the information source other than the student, the emotion index value group representing the emotion of the student according to the input information. It is also possible to provide an artificial intelligence model that outputs, and estimate the emotion of the student using the artificial intelligence model.

Further, in the embodiment, the student-oriented information corresponding to the current emotion of the student is determined based on the information stored in the table having the configuration illustrated in FIG. Instead of this, when the learning support system 1 inputs the current emotion index value group of the student, the past student response, and the student-oriented information transmitted to the student in the past, the learning support system 1 responds to the input information. An artificial intelligence model that outputs student-oriented information may be provided, and the student-oriented information may be determined by the artificial intelligence model.

[2-10] Information Source of Additional Information In the embodiment, the information provided by the related person of the student is used as the information (additional information) of the information source which is not the student and is used for estimating the current emotion of the student. The information source of the information used for estimating the current emotion of the student is not limited to this. For example, when the student's schedule is managed on the cloud, the learning support system 1 may acquire information about the student's schedule from the cloud as additional information and use it for estimating the current emotion of the student. For example, if there is a schedule for a lesson presentation in the schedule in the near future, the learning support system 1 may be anxious about failing in the lesson presentation if the student lacks concentration in learning (3rd By increasing the index value of the emotion axis, the accuracy of estimating the emotion of the student can be improved.

In addition, when the student's schedule is managed not on the cloud but on a terminal such as a smartphone carried by the student, the learning support system 1 communicates with the student's terminal and adds information about the student's schedule. The information may be acquired from the student terminal.

Further, for example, if the schedule of the organization to which the student belongs (school, lesson classroom, etc.) is managed on the cloud, instead of or in addition to the schedule of the student, the learning support system 1 schedules the organization to which the student belongs. It is also possible to obtain the information regarding the above from the cloud as additional information and use it for estimating the current emotion of the student.

In addition, for example, when a student transmits information to an unspecified number of people via SNS (Social Networking Service), the learning support system 1 acquires the information transmitted from the student via SNS as additional information. However, it may be used to estimate the current feelings of the student.

[2-11] Use of additional information In the embodiment, information (additional information) of an information source (student-related person) who is not a student is used to estimate the current emotion of the student. Alternatively or additionally, additional information may be used to determine student-oriented information. For example, when the additional information indicates the evaluation of learning at home, the learning support system 1 may send the student-oriented information corresponding to the evaluation of learning at home to the students in the classroom by the robot 10. ..

[2-12] Category of the Invention The present invention can be understood as an information processing device such as the robot 10 and the server device 20 as well as an information processing system including the information processing devices (the learning support system 1 is one example). .. It can also be understood as an information processing method for realizing the processing executed by those information processing apparatuses and a program for causing a computer that controls these apparatuses to function. These programs may be provided in the form of a recording medium such as an optical disc in which they are stored, or may be downloaded to a computer via a network such as the Internet and installed and made available. May be provided.

1... Learning support system, 3... Wearable device, 4... Person terminal, 10... Robot, 1006... Camera, 1007... Microphone, 20... Server device, 30... Teacher terminal, 40... Parent terminal, 101... Transmitter, 102...Language operation unit, 103...Non-language detection unit, 201...Response information acquisition unit, 202...Required time calculation unit, 203...Correctness determination unit, 204...Emotion estimation unit, 205...Transmission information determination unit, 206...Additional information Acquisition unit, 301... Sending unit, 302... Additional information operation unit.

Claims (7)

Response information acquisition means for acquiring the response of the student to the information transmitted to the student,
And estimating means for estimating a current emotion of the student based on the response to the response information acquisition means has acquired,
Determining means for determining the information for transmitting to the student based on the emotion that the estimating means estimated,
And students for transmitting means for transmitting the information that the determining means has determined for the student,
An additional information acquisition unit for acquiring information about the student as additional information from an information source that is not the student ,
It said estimating means, learning support system that estimates the current emotion of the student based on the additional information acquisition unit acquires the additional information. The determining means determines information to be transmitted to the student using the additional information acquired by the additional information acquiring means.
The learning support system according to claim 1. The additional information acquisition unit acquires information about the schedule of the student or the organization to which the student belongs as additional information from an external device via a network.
The learning support system according to claim 1. The additional information acquisition unit acquires the information transmitted by the student via the Social Networking Service as additional information.
The learning support system according to any one of claims 1 to 3. It said determining means determines the information originating against officials of the pupils based on the emotion that the estimating means estimated,
Learning System according to any one of claims 1 to 4 comprising a transmitting means for parties that transmits information to said determining means has determined for the parties. The determining means transmits the information used by the estimating means to estimate the current emotion of the student to the student based on the information transmitted by the student transmitting means before the response information acquiring means acquires the information. Determine information
The learning support system according to any one of claims 1 to 5. Response information acquisition means for acquiring the response of the student to the information transmitted to the student,
Estimating means for estimating the current emotion of the student based on the response acquired by the response information acquiring means,
Determining means for determining information to be transmitted to the student based on the emotion estimated by the estimating means;
A sending means for students, which sends the information decided by the deciding means to the students,
A learning support system comprising:
A robot having a movable part simulating a part of a human body; and a server device communicatively connected to the robot,
The transmitting means for students is arranged in the robot,
The response information acquisition means, the estimation means and the determination means are arranged in both the robot and the server device,
The estimating means arranged in the robot estimates the current emotion of the student in a shorter time than the estimating means arranged in the server device, and the estimating means arranged in the robot estimates the present emotion. When a change in the emotion that should be transmitted promptly is observed, the determination unit arranged in the robot transmits information to be transmitted to the student without waiting for the determination of the determination unit arranged in the server device. decide
Learning support system.

Images