JP7381054B2 - Speech training system, speech training method and program - Google Patents

Description

The present invention relates to a speech training system, a speech training method, and a program.

JP-A-7-319380 (Patent Document 1) discloses a vocal training device. In this vocal training device, instructions based on the difference between the articulation method of the trainee's vocalization and the articulation method of the model vocalization are fed back to the trainee. According to this vocal training device, the trainee can effectively correct his or her articulation method by proceeding with the training according to the instructions (see Patent Document 1).

Japanese Patent Application Publication No. 7-319380

In the voice training device disclosed in Patent Document 1, feedback is provided to the trainee based on the voice uttered by the trainee. However, the present inventor has discovered that the trainee's speech training is not necessarily effectively performed by feedback based only on the voice uttered by the trainee.

The present invention has been made to solve such problems, and its purpose is to provide a speech training system, a speech training method, and a program that can more effectively train trainees to speak. It is to provide.

A speech training system according to an aspect of the present invention is used for speech training of a trainee. The speech training system includes an imaging means and a display means. The imaging means images the trainee's face and generates moving image data. The display means displays a moving image indicated by the moving image data. The display means displays an image indicating the amount of mouth movement of the trainee superimposed on the moving image.

The present inventor has discovered that when speech training is performed with an awareness of making large movements of the muscles around the mouth, the range of motion of the speech organs is expanded and the clarity of the speech uttered by the trainee is improved. According to this speech training system, an image showing the amount of the trainee's mouth movement is superimposed on the video image, so the trainee can visually recognize whether or not the mouth movement is insufficient. can be done. As a result, according to this speech training system, the trainee can perform speech training while being conscious of making large movements of the muscles around the mouth, so that the trainee's speech training can be carried out more effectively. I can do it.

In the above speech training system, the display means may further display sentences for the trainee to read aloud.

According to this speech training system, since sentences for the trainee to read aloud are displayed, the trainee can perform speech training simply by reading the displayed sentences aloud.

In the above speech training system, the display means may further display evaluation results regarding the trainee's speech.

According to this speech training system, since the evaluation results regarding the trainee's speech are displayed, the trainee can perform speech training while checking the evaluation results.

In the above speech training system, the display means may further display a warning message when the trainee's speech does not meet predetermined requirements.

According to this speech training system, a warning message is displayed if the trainee's speech does not meet the predetermined requirements, so the trainee can visually recognize that his or her speech does not meet the predetermined requirements. can do.

In the above speech training system, the image indicating the amount of mouth movement may be a line indicating a trajectory of the mouth movement.

The speech training system may further include a calculation unit that calculates an optical flow based on the moving image data, and a generation unit that generates an image indicating the amount of mouth movement based on the optical flow.

A speech training method according to another aspect of the invention trains a trainee in speech. The speech training method includes the steps of capturing an image of the trainee's face and generating moving image data, displaying a moving image represented by the moving image data, and converting an image showing the amount of mouth movements of the trainee into a moving image. and displaying the image superimposed on the image.

According to this speech training method, an image showing the amount of the trainee's mouth movements is superimposed on the video image, so the trainee can visually recognize whether or not the mouth movements are insufficient. can be done. As a result, according to this speech training method, it is possible to more effectively train the trainee's speech.

A program according to another aspect of the invention is used for speech training of a trainee. The program includes the steps of causing the imaging means to image the trainee's face and generating moving image data, causing the displaying means to display a moving image represented by the moving image data, and causing the displaying means to capture the trainee's face. The computer is caused to execute a step of superimposing and displaying an image indicating the amount of mouth movement on a moving image.

When this program is executed by a computer, an image showing the amount of mouth movement of the trainee is displayed superimposed on the moving image, so that the trainee can visually see if the mouth movement is insufficient. can be recognized. As a result, according to this program, the trainee can be trained to speak more effectively.

According to the present invention, it is possible to provide a speech training system, a speech training method, and a program that can more effectively train a trainee to speak.

FIG. 2 is a diagram showing an example of a speech training scene using a smartphone. It is a diagram showing an example of the hardware configuration of a smartphone. FIG. 3 is a diagram illustrating an example of the relationship between software modules realized by a control unit. It is a flowchart which shows the execution procedure of video display processing. FIG. 3 is a diagram showing an example of an image displayed on a display. 7 is a flowchart illustrating an execution procedure of optical flow display processing. It is a flowchart which shows the execution procedure of muscle activity amount display processing. 3 is a flowchart illustrating an execution procedure of audio feature quantity display processing. 3 is a flowchart illustrating an execution procedure of a warning message display process. FIG. 3 is a diagram showing an example of an image displayed on a display. It is a figure showing the amplitude of the voice recorded before and after training. FIG. 3 is a diagram showing the range of change in the fundamental frequency of voices recorded before and after training. It is a figure showing VAS measured before and after training.

Embodiments of the present invention will be described in detail below with reference to the drawings. In addition, the same reference numerals are attached to the same or corresponding parts in the drawings, and the description thereof will not be repeated.

[1. overview]
A survey conducted by the present inventors revealed that approximately 30% of healthy university and graduate students are aware of difficulty in speaking. When the present inventor tried various speech training methods, it was found that a high training effect could be obtained by performing speech training with the trainee holding a thin stick in his or her mouth. In particular, the present inventor has found that a high training effect can be obtained when the trainee makes a loud voice and firmly moves the facial muscles during speech training.

FIG. 1 is a diagram showing an example of a speech training scene using a smartphone 100 according to the present embodiment. As shown in FIG. 1, during speech training, the trainee 10 speaks while holding a stick 20 in his/her mouth. The trainee 10 performs speech training while viewing images displayed on the smartphone 100. Although details will be described later, an image is displayed on the smartphone 100 that urges the trainee 10 to speak loudly and firmly move the muscles of the face. Details of the smartphone 100 will be described below.

[2. Hardware configuration]
FIG. 2 is a diagram showing an example of the hardware configuration of the smartphone 100. As shown in FIG. 2, the smartphone 100 includes a camera 130, a display 140, a microphone 150, a speaker 160, a control section 170, a storage section 180, and a communication module 190. Each component included in smartphone 100 is electrically connected via a bus.

Camera 130 is configured to capture an image of a subject and generate image data. The camera 130 captures an image of the trainee 10 (FIG. 1), for example, and generates moving image data. Camera 130 includes, for example, an image sensor such as a CMOS image sensor or a CCD image sensor.

Display 140 is configured to display images. Display 140 displays a moving image represented by moving image data generated by camera 130, for example. The display 140 is configured by a display such as a liquid crystal display or an organic EL display, for example.

Microphone 150 is configured to generate audio data based on sounds surrounding microphone 150. The microphone 150 generates audio data based on the voice uttered by the trainee 10, for example.

The speaker 160 is configured to output the sound indicated by the sound data. The speaker 160 outputs, for example, a sound indicated by audio data generated based on the voice of the trainee 10.

The control unit 170 includes a CPU (Central Processing Unit) 172, a RAM (Random Access Memory) 174, a ROM (Read Only Memory) 176, etc., and is configured to control each component according to information processing. .

The storage unit 180 is, for example, a memory such as a flash memory. The storage unit 180 is configured to store a control program 181, for example. The control program 181 is a control program for the smartphone 100 that is executed by the control unit 170. When the control unit 170 executes the control program 181, the control program 181 is loaded into the RAM 174. The control unit 170 controls each component by causing the CPU 172 to interpret and execute the control program 181 loaded in the RAM 174.

Communication module 190 is configured to communicate with external devices. The communication module 190 includes, for example, an LTE (Long Term Evolution) module, a wireless LAN module, and the like.

[3. Software configuration]
FIG. 3 is a diagram illustrating an example of the relationship between each software module realized by the control unit 170. As shown in FIG. 3, a face area extraction section 131, a pixel movement amount calculation section 132, a face movement amount correction section 133, a muscle activity amount estimation section 134, a first judgment section 135, an audio feature extraction section 151, and a second judgment section. Each of the units 152 is a software module, and is realized by the control unit 170 executing the control program 181.

The face region extraction unit 131 is configured to extract a region corresponding to the face of the trainee 10 based on the moving image data generated by the camera 130. Various known methods can be used to extract the face area.

The pixel movement amount calculation unit 132 is configured to calculate the optical flow of each area based on the moving image data generated by the camera 130. Various known methods are used to calculate the optical flow. Here, each region may be composed of each pixel included in the image, or may be composed of multiple pixels included in the image. Further, the pixel movement amount calculation unit 132 generates an image indicating the calculated optical flow size for each area, and outputs the generated image to the display 140.

The face movement amount correction unit 133 is configured to calculate the movement amount and movement direction of the face area extracted by the face area extraction unit 131, and subtract it from the optical flow calculated by the pixel movement amount calculation unit 132. . As a result, it is possible to calculate an optical flow indicating the movement of facial muscles by subtracting the amount of movement of the face.

The muscle activity amount estimating unit 134 is configured to estimate the amount of facial muscle movement of the trainee 10 by calculating the sum of the magnitudes of optical flows in each region. That is, the muscle activity amount estimating unit 134 is configured to estimate the amount of mouth movement of the trainee 10. The estimated facial muscle movement amount (the sum of the optical flow magnitudes of each region) is output to the display 140.

The first determining unit 135 is configured to determine whether the amount of facial muscle movement estimated by the muscle activity amount estimating unit 134 remains smaller than a first predetermined amount for a predetermined period of time. The first predetermined amount is a value at which the expected speech training effect cannot be obtained if the amount of facial muscle movement is smaller than this. If the amount of movement of the facial muscles continues to be smaller than the first predetermined amount for a predetermined period of time, a first warning image is output to the display 140.

The voice feature extraction unit 151 is configured to extract the feature amount of the voice uttered by the trainee 10 based on the voice data generated by the microphone 150. The voice feature extraction unit 151 extracts, for example, the volume of the voice uttered by the trainee 10. The voice feature extraction unit 151 also generates an image indicating the extracted voice volume, and outputs the generated image to the display 140.

The second determination unit 152 is configured to determine whether a state in which the voice feature amount extracted by the voice feature extraction unit 151 is smaller than a second predetermined amount continues for a predetermined period of time. The second predetermined amount is a value at which the expected speech training effect cannot be obtained when the voice feature amount is smaller than this. A second warning image is output to the display 140 when the voice feature amount remains smaller than the second predetermined amount for a predetermined period of time.

[4. motion]
In smartphone 100 according to the present embodiment, control unit 170 executes video display processing, optical flow display processing, muscle activity amount display processing, audio feature amount display processing, and warning message display processing in parallel. Each process will be explained in turn below.

(4-1. Video display processing)
FIG. 4 is a flowchart showing the procedure for executing moving image display processing. The processing shown in this flowchart is executed at predetermined intervals.

Referring to FIG. 4, the control unit 170 controls the camera 130 to capture a moving image including the face of the trainee 10 and generate moving image data, and also to generate audio data including the voice of the trainee 10. and the microphone 150 (step S100). The control unit 170 extracts a face area included in the video based on the generated video data (step S110). The control unit 170 controls the display 140 to display the text to be read by the trainee 10, a frame surrounding the extracted face area, and a moving image indicated by the moving image data in a superimposed manner (step S120). Text data indicating a sentence to be read by the trainee 10 is stored in advance in the storage unit 180 (FIG. 2), for example.

FIG. 5 is a diagram showing an example of an image displayed on the display 140. As shown in FIG. 5, the display 140 displays a video including the trainee 10, a face frame 200 surrounding the face area of the trainee 10, and a sentence 210 to be read by the trainee 10. . According to the smartphone 100, the sentence 210 to be read aloud by the trainee 10 is displayed on the display 140, so the trainee 10 can practice speaking by simply reading the displayed sentence aloud.

(4-2. Optical flow display processing)
FIG. 6 is a flowchart showing the execution procedure of optical flow display processing. The processing shown in this flowchart is executed at predetermined intervals.

Referring to FIG. 6, control unit 170 calculates the optical flow of each area based on the moving image data generated in the moving image display process (step S200). The control unit 170 generates an image indicating the size and direction of optical flow for each region (step S210). The control unit 170 controls the display 140 to display the generated image superimposed on the video (step S220).

Referring again to FIG. 5, on the display 140, an image of a line 240 (optical flow) indicating the locus of movement of the mouth of the trainee 10 is displayed superimposed on a moving image. According to the smartphone 100, an image indicating the amount of mouth movement of the trainee 10 is displayed superimposed on a moving image, so that the trainee 10 can visually recognize whether or not the mouth movement is insufficient. can be done. As a result, according to the smartphone 100, the trainee 10 can perform speech training while being conscious of making large movements of the muscles around the mouth, so that the trainee 10 can perform speech training more effectively. I can do it.

(4-3. Muscle activity amount display processing)
FIG. 7 is a flowchart showing the execution procedure of muscle activity amount display processing. The processing shown in this flowchart is executed at predetermined intervals.

Referring to FIG. 7, control unit 170 extracts the facial area of trainee 10 and the movement (size and direction) of the facial area based on the video data generated in the video display process. (Step S300). The control unit 170 corrects the optical flow by subtracting the movement of the face area extracted in step S300 from the optical flow calculated in the optical flow display process (step S310). The control unit 170 estimates the amount of facial muscle movement (hereinafter also referred to as "muscle activity amount") of the trainee 10 by calculating the sum of the corrected optical flow magnitudes of each region. (Step S320). The control unit 170 generates an image showing the estimated facial muscle movement amount (the sum of the optical flow magnitudes of each area), and controls the display 140 to display the image (step S330).

Referring again to FIG. 5, on display 140, an image showing the amount of movement of facial muscles, such as level meter 220, is displayed superimposed on a moving image. According to the smartphone 100, the evaluation results regarding the utterances of the trainee 10 (for example, the amount of movement of facial muscles including the mouth) are displayed on the display 140, so the trainee 10 can speak while checking the evaluation results. Training can be carried out.

(4-4. Audio feature amount display processing)
FIG. 8 is a flowchart showing the execution procedure of the audio feature value display process. The processing shown in this flowchart is executed at predetermined intervals.

Referring to FIG. 8, control unit 170 extracts the feature amount (for example, loudness) of the voice of trainee 10 based on the audio data generated in the video display process (step S400). The control unit 170 generates an image showing the extracted voice feature amount, and controls the display 140 to display the image (step S410).

Referring again to FIG. 5, on the display 140, an image indicating the feature amount of the voice, such as the level meter 230, is displayed superimposed on the moving image. According to the smartphone 100, the evaluation results (for example, the volume of voice) regarding the utterances of the trainee 10 are displayed on the display 140, so the trainee 10 can perform speech training while checking the evaluation results. can.

(4-5. Warning message display processing)
FIG. 9 is a flowchart showing the execution procedure of the warning message display process. The processing shown in this flowchart is executed at predetermined intervals.

Referring to FIG. 9, control unit 170 determines whether the state in which the amount of muscle activity estimated in the muscle activity amount display process is smaller than the first predetermined amount continues for a predetermined period of time (step S500). If it is determined that the amount of muscle activity is equal to or greater than the first predetermined amount (NO in step S500), the process moves to step S510. On the other hand, if it is determined that the state in which the amount of muscle activity is smaller than the first predetermined amount continues for a predetermined time (YES in step S500), the control unit 170 controls the display 140 to display the first warning image. (Step S510).

FIG. 10 is a diagram showing an example of an image displayed on the display 140. As shown in FIG. 10, if the muscle activity remains smaller than the first predetermined amount for a predetermined period of time, a first warning image 250 (“Move your mouth more!”) is displayed on the display 140. Ru. According to the smartphone 100, the first warning image 250 is displayed when the utterance of the trainee 10 does not meet the predetermined requirements, so the trainee 10 can visually see that his or her utterance does not meet the predetermined requirements. can be recognized visually.

Referring again to FIG. 9, next, the control unit 170 determines whether the state in which the audio feature amount extracted in the audio feature amount display process is smaller than the second predetermined amount continues for a predetermined period of time (step S520). If it is determined that the audio feature amount is equal to or greater than the second predetermined amount (NO in step S520), the process moves to step S500. On the other hand, if it is determined that the state in which the audio feature amount is smaller than the second predetermined amount continues for a predetermined period of time (YES in step S520), the control unit 170 controls the display 140 to display the second warning image. (Step S530).

Referring again to FIG. 10, if the voice feature amount continues to be smaller than the second predetermined amount for a predetermined period of time, a second warning image 260 (“Speak louder!”) is displayed on display 140. . According to the smartphone 100, the second warning image 260 is displayed when the utterance of the trainee 10 does not meet the predetermined requirements, so the trainee 10 can visually see that his or her utterance does not meet the predetermined requirements. can be recognized visually.

[5. Features]
As described above, in the smartphone 100 according to the present embodiment, the display 140 displays an image indicating the amount of mouth movement of the trainee 10 superimposed on a moving image. According to the smartphone 100, an image indicating the amount of mouth movement of the trainee 10 is displayed superimposed on a moving image, so that the trainee 10 can visually recognize whether or not the mouth movement is insufficient. can be done. As a result, according to the smartphone 100, the trainee 10 can perform speech training while being conscious of making large movements of the muscles around the mouth, so that the trainee 10 can perform speech training more effectively. I can do it.

Note that the smartphone 100 is an example of a "speech training system," the camera 130 is an example of an "imaging means," and the display 140 is an example of a "display means." Further, the pixel movement amount calculating unit 132 is an example of a "calculating means" and a "generating means".

[6. experiment]
The inventor conducted the following experiment. This experiment was conducted in a soundproof room. Prior to the experiment, we explained the experiment to the participants. Next, we explained to the experiment participants the significance of speech training and asked them to be willing to participate in the experiment. Regarding the volume and speed of speech, participants were asked to imagine reciting aloud in a high school classroom. Audio recording and speaking training were performed in a standing position. During training, a PC (Personal Computer) display (EIZO EV2450) was placed in front of the experiment participants' faces so that they could practice facing forward. Note that in this experiment, the application implemented in the smartphone 100 according to the above embodiment is installed on the PC.

In the experiment, participants first recorded their pre-training speech and asked them to self-evaluate how well they were able to produce the utterances using a visual analog scale (VAS). Next, the subjects practiced for 3 minutes using the PC (speech training system) while biting disposable chopsticks with their front teeth. Thereafter, the post-training audio was recorded and the VAS was measured again. Audio was recorded using a condenser microphone (SonyECM-77B) and a recorder (Marantz PMD671) at a sampling frequency of 16 kHz and quantization of 16 bits.

FIG. 11 is a diagram showing the amplitude of audio recorded before and after training. FIG. 12 is a diagram showing the range of change in the fundamental frequency of voices recorded before and after training. These results show the distribution of the average values of the 14 sentences for each experimental participant. As shown in FIGS. 11 and 12, it can be seen that both the amplitude and the fundamental frequency change width tend to increase as a result of training using the speech training system. Comparing the median values before and after training, the amplitude increased by 4.3 dB, and the range of change in fundamental frequency increased by 1.19 semitones. Although not shown, the average fundamental frequency also tended to increase after training.

FIG. 13 is a diagram showing VAS measured before and after training. As shown in Figure 13, almost all participants in the experiment felt that they had become able to read well through training. After the experiment, we received many positive comments such as ``I can now speak clearly,'' ``My sa and ta lines have improved,'' and ``The muscles on the sides of my mouth can move more easily.'' By providing feedback on facial movements, it is thought that even just three minutes of practice was effective in improving mouth movements.

[7. Modified example]
Although the embodiments have been described above, the present invention is not limited to the above embodiments, and various changes can be made without departing from the spirit thereof. Modifications will be described below.

(7-1)
In the embodiment described above, the line 240 is used as an image indicating the amount of muscle activity (an image indicating the amount of mouth movement). However, the image showing the amount of muscle activity is not limited to line 240. The image indicating the amount of muscle activity may be, for example, an arrow indicating the direction and magnitude of movement. Furthermore, in order to indicate the amount of muscle activity, for example, parts with a large amount of movement and parts with a small amount of movement may be colored differently. For example, an area with a large amount of movement may be expressed in red, and an area with a small amount of movement may be expressed in blue.

(7-2)
In the embodiment described above, the optical flow of the entire moving image captured by the camera 130 is calculated. However, the range in which optical flow is calculated is not limited to this. For example, the optical flow of only the face region of the trainee 10 may be calculated, the optical flow of only the lower half region of the trainee 10's face may be calculated, or the optical flow of only the lower half region of the trainee 10's face may be calculated. The optical flow of only the region may be calculated. By narrowing down the area in which the optical flow is calculated, the amount of calculation by the control unit 170 can be reduced.

(7-3)
In the embodiment described above, speech training was performed using images and voices of the trainee 10. However, the voice of the trainee 10 does not necessarily need to be used for speech training.

(7-4)
In the embodiment described above, the optical flow is calculated to obtain the amount of mouth movement of the trainee 10. However, optical flow does not necessarily need to be calculated. For example, the amount of movement of the trainee's 10 mouth may be obtained by simply calculating the difference between frames in a moving image.

(7-5)
In the embodiments described above, the speech training system is implemented on a smartphone, but the speech training system according to the present invention may also be implemented on, for example, a PC, a tablet, or the like.

(7-6)
In the embodiment described above, an instructor's model video may also be displayed on the display 140 during speech training.

(7-7)
In the embodiment described above, the optical flow is calculated for each region of the trainee's 10 face. Therefore, for example, it is also possible to calculate which region of the trainee's 10 face is lacking in movement. For example, a warning image indicating which region of the trainee's 10 face is lacking in movement may be displayed on the display 140.

(7-8)
In the above embodiment, for example, the history of speech training of the trainee 10 may be sequentially stored in the storage unit 180. Thereby, for example, when the trainee 10 performs a new speech training, it is possible to inform the trainee 10 which parts have improved or which parts have worsened compared to the previous time.

Reference Signs List 10 trainee, 20 bar, 100 smartphone, 130 camera, 131 face area extraction unit, 132 pixel movement amount calculation unit, 133 face movement amount correction unit, 134 muscle activity amount estimation unit, 135 first determination unit, 140 display, 150 microphone, 151 audio feature extraction unit, 152 second determination unit, 160 speaker, 170 control unit, 172 CPU, 174 RAM, 176 ROM, 180 storage unit, 181 control program, 190 communication module, 200 face frame, 210 text, 220, 230 level meter, 240 line, 250 first warning image, 260 second warning image.

Claims (7)

A speech training system used for speech training of a trainee,
an imaging means for imaging the trainee's face and generating moving image data;
Display means for displaying a moving image indicated by the moving image data;
a muscle activity amount estimation unit that estimates a muscle activity amount, which is the amount of movement of the facial muscles of the trainee, based on the video image data;
a muscle activity amount determination unit that determines whether a state in which the muscle activity amount estimated by the muscle activity amount estimation unit is smaller than a first predetermined amount continues for a predetermined time;
a voice feature extraction unit that extracts a feature amount of the voice uttered by the trainee based on voice data generated from the voice uttered by the trainee by a microphone;
a voice feature determining unit that determines whether a state in which the voice feature extracted by the voice feature extracting unit is smaller than a second predetermined amount continues for the predetermined time;
Equipped with
The display means displays an image indicating the amount of mouth movement of the trainee superimposed on the moving image,
The display means is
When the muscle activity amount determining unit determines that the muscle activity amount is smaller than the first predetermined amount for the predetermined period of time, further displaying a first warning message;
further displaying a second warning message when the voice feature amount determining unit determines that the voice feature amount is smaller than the second predetermined amount for the predetermined period of time;
Speech training system. The speech training system according to claim 1, wherein the display means further displays sentences to be read aloud by the trainee. 3. The speech training system according to claim 1, wherein the display means further displays evaluation results regarding the trainee's speech. The speech training system according to any one of claims 1 to 3, wherein the image showing the amount of movement of the mouth is a line showing a trajectory of movement of the mouth. Calculation means for calculating optical flow based on the moving image data;
The speech training system according to any one of claims 1 to 4, further comprising: generating means for generating an image indicating the amount of movement of the mouth based on the optical flow. A speech training method for training a trainee regarding speech, the method comprising:
capturing an image of the trainee's face and generating moving image data;
Displaying a moving image indicated by the moving image data;
estimating the amount of muscle activity, which is the amount of movement of the facial muscles of the trainee, based on the video image data;
a first determination step of determining whether a state in which the amount of muscle activity estimated in the step of estimating the amount of muscle activity is smaller than a first predetermined amount continues for a predetermined time;
superimposing and displaying an image indicating the amount of mouth movement of the trainee on the moving image;
extracting a feature amount of the voice uttered by the trainee based on voice data generated from the voice uttered by the trainee by a microphone;
a second determination step of determining whether a state in which the extracted feature amount of the voice is smaller than a second predetermined amount continues for the predetermined time;
If it is determined in the first determination step that the state in which the amount of muscle activity is smaller than the first predetermined amount has continued for the predetermined time, further displaying a first warning message; further displaying a second warning message when it is determined that the voice feature amount is smaller than the second predetermined amount for the predetermined period of time;
speech training methods, including A program used for speech training of a trainee,
causing an imaging means to image the trainee's face and generate moving image data;
displaying a moving image indicated by the moving image data on a display means;
estimating the amount of muscle activity, which is the amount of movement of the facial muscles of the trainee, based on the video image data;
a first determination step of determining whether a state in which the amount of muscle activity estimated in the step of estimating the amount of muscle activity is smaller than a first predetermined amount continues for a predetermined time;
displaying on the display means an image indicating the amount of mouth movement of the trainee, superimposed on the moving image;
extracting a feature amount of the voice uttered by the trainee based on voice data generated from the voice uttered by the trainee by a microphone;
a second determination step of determining whether a state in which the extracted feature amount of the voice is smaller than a second predetermined amount continues for the predetermined time;
If it is determined in the first determination step that the state in which the amount of muscle activity is smaller than the first predetermined amount has continued for the predetermined time, further displaying a first warning message; further displaying a second warning message when it is determined that the voice feature amount is smaller than the second predetermined amount for the predetermined period of time;
A program that causes a computer to execute.