JP2016157097A - Reading-aloud evaluation device, reading-aloud evaluation method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a reading-aloud evaluation device, a reading-aloud evaluation method, and a program capable of presenting overall evaluation including a reading-aloud speed and interval to a speaker.SOLUTION: A reading-aloud evaluation device compares the time length of a speaker interval section specified based on the speaker voice waveform data with the time length of a model interval section specified based on the model voice waveform data to evaluate a reading-aloud speed for each phrase, and compares the time length of a speaker phrase section specified based on speaker voice waveform data with the time length of a model phrase section specified based on model voice waveform data to evaluate an interval. The reading-aloud evaluation device carries out overall evaluation for sentence reading-aloud based on at least the evaluation of the reading-aloud speed and the evaluation of the interval to display at least one of the evaluation of the reading-aloud speed, the evaluation of the interval and the overall evaluation for sentence reading-aloud.SELECTED DRAWING: Figure 5

Description

  The present invention relates to a technical field such as a system that evaluates reading of a sentence based on speech uttered when a speaker reads out a sentence.

  In recent years, various systems for the purpose of supporting language learning and the like have been proposed. In these systems, learning is supported mainly by focusing on pitches (inflections) and smooth tongues in units of words and sentences. For example, in the technique disclosed in Patent Document 1, the pitch of the model voice and the pitch of the learner's voice are displayed together for each word, thereby easily knowing the difference between the learner's voice and the model voice. Techniques that can be used are disclosed.

JP 2007-139868 A

  However, for learning such as announcements and readings, it is important not only to use inflection and smooth tongue, but also to speak (read aloud) “speed” and “interval”. Has not been evaluated.

  The present invention has been made in view of the above points, and provides a reading aloud evaluation apparatus, a reading aloud evaluation method, and a program capable of presenting a speaker with a comprehensive evaluation including reading speed and interval.

  In order to solve the above-described problem, the invention according to claim 1 is specified for each phrase based on first speech waveform data indicating a waveform of a speech that serves as a model when a sentence including a plurality of phrases is read aloud. A first phrase section from the start timing to the end timing of the phrase, and a first interval section specified based on the first speech waveform data, and among the plurality of phrases Storage means for storing a first interval section from the end timing of any one of the phrases to the start timing of the next phrase, and a second voice waveform indicating a waveform of a voice uttered when the speaker reads the sentence aloud Based on the input means for inputting data and the second speech waveform data, the second phrase from the start timing to the end timing of the phrase Specifying the interval for each phrase, and specifying the second interval section from the end timing of any one of the phrases to the start timing of the next phrase, and storing in the storage unit Speed evaluation means for comparing the time length of the first phrase section and the time length of the second phrase section specified by the specifying means to evaluate the speed at which the sentence is read aloud for each phrase; A time period for evaluating the time when the sentence is read aloud by comparing the time length of the first interval section stored in the storage means with the time length of the second interval period specified by the specifying means. Based on the evaluation means, and at least the evaluation of the speed by the speed evaluation means and the evaluation of the gap by the gap evaluation means At least one of comprehensive evaluation means for performing comprehensive evaluation on the reading of the sentence aloud, evaluation of the speed by the speed evaluation means, evaluation of the space by the space evaluation means, and the comprehensive evaluation by the comprehensive evaluation means Display control means for displaying one evaluation.

According to a second aspect of the present invention, in the reading aloud evaluation apparatus according to the first aspect, the sound information of at least one of pitch and volume is calculated at predetermined time intervals based on the first speech waveform data. Calculating means; and second calculating means for calculating sound information of at least one of pitch and volume based on the second sound waveform data at a predetermined time interval, wherein the display control means All or part of the first graph representing the time-series change of the sound information calculated by the one calculating means, and the second graph representing the time-series change of the sound information calculated by the second calculating means. All or part of the information is displayed on one screen so as to be comparable.

  According to a third aspect of the present invention, in the reading aloud evaluation device according to the second aspect, the display control means includes a first graph portion corresponding to at least one first phrase section in the first graph, and A second graph portion corresponding to at least one second phrase section in the second graph is displayed on one screen for comparison, and the first graph displayed on the one screen in response to a display switching instruction. The graph portion is a graph portion that is not displayed on the one screen and is switched to a graph portion corresponding to the other first phrase section in the first graph, and is displayed on the one screen. The second graph portion that is not displayed on the one screen and is switched to a graph portion corresponding to the other second phrase section in the second graph. To.

  According to a fourth aspect of the present invention, in the reading aloud evaluation device according to the third aspect, the display control means includes the first interval section stored in the storage means and the second specified by the specifying means. The first graph portion and the second graph portion are displayed on one screen so that the interval section can be compared.

  The invention according to claim 5 is the reading aloud evaluation device according to claim 2, wherein the display control means includes a first graph portion corresponding to at least one first phrase section in the first graph, and the first graph section. The first interval interval following the first phrase interval, the second graph portion corresponding to at least one second phrase interval in the second graph, and the second interval interval following the second phrase interval. The first graph portion and the second graph portion are displayed on a single screen so that they can be compared.

  The invention according to claim 6 is the reading aloud evaluation device according to any one of claims 3 to 5, wherein the display control means includes a start position of the first phrase section and a start position of the second phrase section. And the first graph portion and the second graph portion are displayed on one screen by matching them in a predetermined direction.

  According to a seventh aspect of the present invention, in the reading aloud evaluation device according to any one of the first to sixth aspects, the display control means includes the total evaluation in one of the sections and the total in the plurality of sections. The evaluation is displayed on the screen of the same hierarchy.

  According to an eighth aspect of the present invention, in the reading aloud evaluation device according to any one of the first to sixth aspects, the display control means includes the comprehensive evaluation in one of the sections and the total in the plurality of sections. The evaluation is displayed on a screen of a different hierarchy.

  According to a ninth aspect of the present invention, in the reading aloud evaluation device according to any one of the first to eighth aspects, the storage unit stores the first speech waveform data, and the specifying unit is the storage unit. The first phrase section is specified for each phrase based on the first speech waveform data stored in the section, and the first interval section is specified.

  The invention according to claim 10 is a speech reading evaluation method executed by one or more computers, wherein the first speech waveform data indicates a speech waveform as a model when a sentence including a plurality of phrases is read aloud. A first phrase section specified for each phrase based on the first phrase section from the start timing to the end timing of the phrase, and a first interval section specified based on the first speech waveform data A storage step of storing in the storage means a first interval from the end timing of any one of the plurality of phrases to the start timing of the next phrase, and when the speaker reads the sentence aloud An input step of inputting second voice waveform data indicating a waveform of the emitted voice, and the frame based on the second voice waveform data. A second phrase interval from the start timing to the end timing is specified for each phrase, and a second interval interval from the end timing of any one of the phrases to the start timing of the next phrase is determined. The speed of reading the sentence aloud by comparing the specifying step, the time length of the first phrase section stored in the storage means, and the time length of the second phrase section specified by the specifying step The speed evaluation step for performing the evaluation for each phrase, the time length of the first interval section stored in the storage means, and the time length of the second interval section specified by the specifying step are compared with the sentence. A gap evaluation step for evaluating a gap when reading aloud, and at least the speed evaluation step. Based on the evaluation of the speed by the evaluation and the evaluation of the interval by the interval evaluation step, an overall evaluation step for performing an overall evaluation on the reading of the sentence, the speed evaluation by the speed evaluation step, and the interval by the interval evaluation step And a control step of displaying at least one of the comprehensive evaluations in the comprehensive evaluation step.

  The invention according to claim 11 is the first phrase section specified for each phrase based on the first voice waveform data indicating the waveform of the voice that serves as a model when reading a sentence including a plurality of phrases. A first phrase interval from the start timing to the end timing of the phrase and a first interval interval specified based on the first speech waveform data, and the end timing of any of the phrases among the plurality of phrases A storage step of storing in the storage means a first interval section from the start timing of the next phrase to the start timing of the next phrase, and an input for inputting second speech waveform data indicating a waveform of speech uttered when the speaker reads the sentence aloud And a second phrase section from the start timing to the end timing of the phrase based on the step and the second speech waveform data A specific step of specifying for each raise and specifying a second interval section from the end timing of any one of the phrases to the start timing of the next phrase, and stored in the storage means Comparing the time length of the first phrase section with the time length of the second phrase section specified in the specifying step, and evaluating the speed at which the sentence is read aloud for each phrase; and the storage A time evaluation step of evaluating the time when the sentence is read aloud by comparing the time length of the first interval section stored in the means with the time length of the second interval section specified by the specifying step. And at least the speed evaluation by the speed evaluation step and the preliminary evaluation step Comprehensive evaluation step for performing comprehensive evaluation on the reading of the sentence based on evaluation of the interval, evaluation of the speed by the speed evaluation step, evaluation of the interval by the interval evaluation step, and the overall evaluation by the overall evaluation step And a control step of displaying at least one of the evaluations.

  The invention according to claim 12 is the first sentence element specified for each sentence element based on the first voice waveform data indicating a waveform of a voice that serves as a model when a sentence including a plurality of sentence elements is read aloud. A first sentence element section from the start timing to the end timing of the sentence element and a first interval section specified based on the first speech waveform data, and any one of the plurality of sentence elements Storage means for storing a first interval section from the end timing of the sentence element to the start timing of the next sentence element, and a second voice waveform indicating a waveform of a voice uttered when the speaker reads the sentence aloud Based on the input means for inputting data and the second speech waveform data, a second sentence element section from the start timing to the end timing of the sentence element is specified for each sentence element, and the plurality of the sentences A specifying means for specifying a second interval section from the end timing of any one of the sentence elements to a start timing of the next sentence element, and a time length of the first sentence element section stored in the storage means Comparing the time length of the second sentence element section specified by the specifying means and evaluating the speed at which the sentence is read aloud for each sentence element, and the storage means stored in the storage means A time evaluation means for comparing the time length of the first interval section and the time length of the second interval section specified by the specifying means to evaluate the time when the sentence is read aloud, and at least the speed evaluation Based on the evaluation of the speed by means and the evaluation of the gap by the gap evaluation means, comprehensive evaluation means for performing a comprehensive evaluation on the reading of the sentence; Display control means for displaying at least one of the evaluation of the speed by the speed evaluation means, the evaluation of the gap by the gap evaluation means, and the comprehensive evaluation by the comprehensive evaluation means, To do.

  The invention according to claim 13 is a reading aloud evaluation method executed by one or more computers, and a first voice waveform showing a waveform of a voice serving as a model when reading a sentence including a plurality of sentence elements aloud. A first sentence element section identified for each sentence element based on data, the first sentence element section from the start timing to the end timing of the sentence element, and identified based on the first speech waveform data A storage step of storing in the storage means a first interval section from the end timing of any one of the plurality of sentence elements to the start timing of the next sentence element in the storage section; An input step of inputting second voice waveform data indicating a waveform of a voice uttered when the sentence is read aloud, and a start timing of the sentence element based on the second voice waveform data And a second interval from the end timing of any one of the plurality of sentence elements to the start timing of the next sentence element. The step of specifying a section, the time length of the first sentence element section stored in the storage means, and the time length of the second sentence element section specified by the specifying step are compared to determine the sentence. A speed evaluation step for evaluating the speed of reading aloud for each sentence element, a time length of the first interval section stored in the storage means, and a time length of the second interval section specified by the specifying step A time evaluation step for evaluating the time when the sentence is read aloud in comparison, and at least the speed evaluation by the speed evaluation step and the A comprehensive evaluation step for performing a comprehensive evaluation on the reading of the sentence based on the evaluation of the clearance according to the clearance evaluation step; an evaluation of the speed by the speed evaluation step; an evaluation of the clearance by the clearance evaluation step; and the comprehensive evaluation And a control step of displaying at least one of the overall evaluations in the step.

  The invention according to claim 14 is the first sentence element specified for each sentence element based on the first voice waveform data indicating a waveform of a voice that serves as a model when a sentence including a plurality of sentence elements is read aloud. A first sentence element section from the start timing to the end timing of the sentence element and a first interval section specified based on the first speech waveform data, and any one of the plurality of sentence elements A storage step of storing in the storage means a first interval section from the end timing of the sentence element to the start timing of the next sentence element; and a second waveform showing a waveform of a voice uttered when the speaker reads the sentence aloud A second sentence element section from the start timing to the end timing of the sentence element is specified for each sentence element based on the input step of inputting two voice waveform data and the second voice waveform data; and A specifying step of specifying a second interval section from an end timing of any one of the plurality of sentence elements to a start timing of the next sentence element; and the first sentence stored in the storage unit A speed evaluation step for evaluating the speed at which the sentence is read aloud for each sentence element by comparing the time length of the element section and the time length of the second sentence element section specified in the specifying step; A time evaluation step of evaluating the time when the sentence is read aloud by comparing the time length of the first interval section stored in the time interval with the time length of the second interval section specified in the specifying step; , Based on at least the evaluation of the speed by the speed evaluation step and the evaluation of the clearance by the clearance evaluation step. Display of at least one of a comprehensive evaluation step for performing a comprehensive evaluation on reading, an evaluation of the speed by the speed evaluation step, an evaluation of the clearance by the gap evaluation step, and a comprehensive evaluation by the comprehensive evaluation step And a control step for causing the computer to execute the control step.

  According to the invention described in claims 1, 9, 10, 11, 12, 13, and 14, for speakers who conduct speech training such as language learning, announcements and readings, etc. It is possible to present at least one of a reading speed evaluation, a gap evaluation, and a comprehensive evaluation that considers these evaluations. It becomes possible to make the person aware and to carry out effective practice and training.

  According to the second aspect of the present invention, it is possible to make the speaker aware of whether the pitch and the sound volume are appropriate in addition to the reading speed and how to make a space, thereby enabling effective practice and training.

  According to the third aspect of the present invention, even if the first graph and the second graph do not fit on one screen in the time axis direction, it is possible to display whether or not the pitch or volume is appropriate by switching the display. The speaker can be grasped over a period of time.

  According to the fourth and fifth aspects of the present invention, the first graph and the second graph allow the speaker to grasp at a glance whether the layout is appropriate.

  According to the sixth aspect of the invention, it is possible to make the speaker understand at a glance whether or not the reading speed is appropriate.

  According to the invention described in claim 7, it is possible to allow the speaker to simultaneously grasp the comprehensive evaluation in one section and the comprehensive evaluation in a plurality of sections.

  According to the invention described in claim 8, it is possible to make the speaker understand the comprehensive evaluation in one section and the comprehensive evaluation in a plurality of sections separately and easily.

It is a figure which shows the example of a schematic structure of the reading aloud evaluation apparatus S which concerns on this embodiment. It is a figure which shows an example of example phrase area F11-F18, example interval area I11-I17, speaker phrase area F21-F26, and speaker interval area I21-I25. (A) is a figure which shows the comparative example with the time length of a sample phrase area, and the time length of a speaker phrase area, (B) is the time length of a sample interval area, and a speaker interval area. It is a figure which shows the comparative example with time length. It is a flowchart which shows the evaluation display process of the control part 3 in the reading aloud evaluation apparatus S. It is a figure which shows an example of an area scroll screen. It is a figure which shows an example of an area selection screen. It is a figure which shows an example of the screen which displays the detail of the selected phrase area. It is a figure which shows an example of the screen which displays the detail of the selected interval area.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, a first embodiment of the invention will be described with reference to the drawings.

[1. Configuration and function of reading aloud evaluation device S]
First, the configuration and function of the reading aloud evaluation apparatus S according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram illustrating a schematic configuration example of the reading aloud evaluation apparatus S according to the present embodiment. In addition, a personal computer, a portable information terminal (smartphone, etc.) etc. are mentioned as an example of a reading aloud evaluation apparatus. As shown in FIG. 1, the reading aloud evaluation device S includes a communication unit 1, a storage unit 2, a control unit 3, an operation unit 4, an interface (IF) unit 5, and the like. It is connected to the. The operation unit 4 receives an operation instruction from the user and outputs a signal corresponding to the received operation to the control unit 3. The interface unit 5 is connected to a microphone M, a display D, and the like. The microphone M collects a sound that is produced when a speaker who performs language learning, speech utterance training such as announcement, reading, etc. reads a sentence (sentence) including a plurality of phrases aloud. Here, a phrase is a unit that is read at a time when reading a sentence, and sometimes includes a punctuation mark “.” Or “.” Indicating the end of the sentence, or a punctuation mark “,” or “,”. . A phrase is sometimes called a syllable. Examples of sentences to be read aloud include, for example, sentences used for language learning, announcement training, reading training, and lyrics used for singing. The display D displays voice information provided to the speaker on the screen in accordance with a display command from the control unit 3. The speech information is information such as an evaluation for reading aloud a sentence and a graph representing a time-series change in sound information of at least one of pitch (also referred to as pitch or intonation) and volume. In addition, the evaluation includes reading speed evaluation, interval evaluation, pitch evaluation, sound volume evaluation, smooth tongue evaluation, comprehensive evaluation, and the like. Here, the reading speed, interval, pitch, volume, and smooth tongue are referred to as evaluation items. The microphone M and the display D may be integrated with the reading aloud evaluation device S or may be separate.

  The communication unit 1 communicates with a server or the like by connecting to a network (not shown) by wire or wireless. The storage unit 2 includes, for example, a hard disk drive and stores an OS (Operating System), an evaluation display processing program (an example of the program of the present invention), and the like. The evaluation display process program is a program that causes the control unit 3 as a computer to execute an evaluation display process to be described later. The evaluation display processing program may be downloaded from a predetermined server as an application, or may be provided by being stored in a recording medium such as a CD or a DVD. In addition, the storage unit 2 includes text data of a sentence including a plurality of phrases, and first voice waveform data (hereinafter referred to as “example voice waveform data”) indicating a waveform of a voice as a model when reading the sentence aloud. ) Is memorized. Here, the text data includes, for example, the sound generation timing of each character (for example, the elapsed time from the start of sound generation) in association with each character. The model voice waveform data is stored in a predetermined voice file format.

The control unit 3 includes a CPU (Center Processing Unit) and a ROM (Read
Only Memory), RAM (Random Access Memory), and the like. The control unit 3 functions as a voice processing unit 31, a reading aloud evaluation unit 32, and a display processing unit 33 according to the evaluation display processing program. The voice processing unit 31 is an example of an input unit, a specifying unit, a first calculation unit, and a second calculation unit in the present invention. The reading aloud evaluation unit 32 is an example of a speed evaluation unit, a gap evaluation unit, and a comprehensive evaluation unit. The display processing unit 33 is an example of display control means in the present invention. The RAM in the storage unit 2 or the control unit 3 is an example of a storage unit in the present invention.

  The voice processing unit 31 inputs model voice waveform data stored in a predetermined voice file format from the storage unit 2. The speech processing unit 31 is a second speech waveform data (hereinafter referred to as “speaker speech waveform data”) indicating a waveform of speech collected by the microphone M, which is a speech produced when the speaker reads the sentence aloud. ”). The model voice waveform data and the speaker voice waveform data are collectively referred to as voice waveform data. The voice waveform data is discretized time-series sound pressure waveform data, for example, sampling rate 44.1 kHz, quantization 16 bits, and monaural waveform data. Note that the sound pressure refers to a change (Pa) in air pressure due to sound waves. In this embodiment, the sound pressure level (dB) that represents the magnitude of effective sound pressure (Pa), which is the root mean square (RMS) of instantaneous sound pressure (Pa), is expressed as a numerical value that is easy to handle in calculation. To do. The sound pressure level (dB) is also called volume in a broad sense.

  For example, the sound processing unit 31 calculates a sound pressure level (dB) as sound pressure from data cut out from the sound waveform data every predetermined time (for example, 10 ms) at predetermined time intervals (at predetermined time intervals). Further, the voice processing unit 31 calculates a fundamental frequency (Hz) from data cut out at predetermined time intervals from the voice waveform data, and calculates the calculated fundamental frequency (Hz) at predetermined time intervals as a pitch. For the pitch calculation method, for example, a known method such as a zero cross method or vector autocorrelation can be applied. In addition, the voice processing unit 31 calculates a feature amount (acoustic characteristic) indicating a vocal tract characteristic used for evaluation of the smooth tongue for each phrase. For example, the voice processing unit 31 cuts out voice waveform data for each phrase (phrase section), divides the cut-out phrase data into windows (also referred to as frames) (for example, frames every 25 ms), and Fourier analysis ( The amplitude spectrum is obtained by performing FFT. Then, the speech processing unit 31 multiplies the obtained amplitude spectrum by a mel filter bank and performs a discrete cosine transform (DCT) on a logarithmic value of the output of the mel filter bank to obtain a MFCC (mel frequency cepstrum coefficient). It is calculated for each phrase as a feature amount indicating the characteristic.

  In addition, the speech processing unit 31 specifies, for each phrase, a first phrase section (hereinafter referred to as “example phrase section”) from the start timing to the end timing of each phrase based on the model speech waveform data, and A first interval section (hereinafter referred to as “example interval section”) from the end timing of one of the phrases to the start timing of the next phrase is specified. The data of the model phrase section and the model interval section specified in this way are stored in the storage unit 2 in association with the voice file of the model voice waveform data, for example. Note that the phrase section and the interval section are represented by, for example, a time range from the waveform start time (for example, 01: 00-03: 00). Further, the speech processing unit 31 specifies, for each phrase, a second phrase section (hereinafter referred to as “speaker phrase section”) from the start timing to the end timing of each phrase based on the speaker speech waveform data, and A second interval section (hereinafter referred to as “speaker interval section”) from the end timing of one of the phrases to the start timing of the next phrase is specified.

  Here, the start timing and the end timing may be recognized from the sound waveform, or may be recognized from the sound pressure level (dB) calculated as described above. For example, the voice processing unit 31 recognizes the time point when the amplitude width of the voice waveform becomes a predetermined value or more as the start timing. Alternatively, the sound processing unit 31 recognizes the time point when the sound pressure level (dB) becomes a predetermined value or more as the start timing. Further, for example, the voice processing unit 31 recognizes the time point when the amplitude width of the voice waveform is less than a predetermined value as the end timing. Alternatively, the sound processing unit 31 recognizes the time point when the sound pressure level (dB) becomes less than a predetermined value as the end timing.

  FIG. 2 is a diagram illustrating an example of model phrase sections F11 to F18 and model interval sections I11 to I17, speaker phrase sections F21 to F26, and speaker interval sections I21 to I25. FIG. 2 is an example in which the start timing and end timing are determined using a predetermined sound pressure level as a threshold. In the example of FIG. 2, the horizontal axis (X) indicates time, and the vertical axis (Y) indicates sound pressure. In the time-series change of sound pressure, the sample phrase intervals F11 to F18 and the sample interval. The sections I11 to I17, the speaker phrase sections F21 to F26, and the speaker interval sections I21 to I25 are specified. Serial numbers are assigned to the specified example phrase sections F11 to F18 and example interval sections I11 to I17, for example, in order from the top. Similarly, serial numbers are assigned to the speaker phrase sections F21 to F26 and the speaker interval sections I21 to I25, for example, sequentially from the top. Thereby, a speaker phrase section (for example, F21) corresponding to the model phrase section (for example, F11) can be specified, and a speaker interval section (for example, I21) corresponding to the model interval section (for example, I11) can be specified. ) Can be specified.

  Next, the reading aloud evaluation unit 32 compares the time length (temporal length) of the model phrase section with the time length of the speaker phrase section, and evaluates the speed of reading the sentence (reading speed) by the phrase ( That is, it is performed for each phrase section. FIG. 3A is a diagram illustrating a comparative example of the time length of the model phrase section and the time length of the speaker phrase section. In the example of FIG. 3A, the time length of the sample phrase section F11 is compared with the time length of the speaker phrase section F21 corresponding to the sample phrase section F11 (for example, the serial numbers match), and the model phrase is compared. The time length of the section phrase F12 is compared with the time length of the speaker phrase section F22 corresponding to the model phrase section F12, and the time length of the model phrase section F13 and the speaker phrase section F23 corresponding to the model phrase section F13 are compared. The length of time is compared. The phrase section following the example phrase section F13 and the phrase section following the speaker phrase section F23 are similarly compared. The reading aloud evaluation unit 32 calculates, for example, the time difference between the time length of the model phrase section and the time length of the speaker phrase section for each phrase (phrase section) as a comparison result of time lengths, and the absolute value of this time difference The reading speed is evaluated by calculating the evaluation score based on the above. For example, the closer the absolute value of the time difference is to 0, the higher the evaluation (that is, the higher the evaluation score). That is, since the absolute value of the time difference increases as the speaker's reading speed is faster or slower than the reading speed of the model, the evaluation becomes lower. On the other hand, since the absolute value of the time difference becomes smaller as the speaker's reading speed approaches the reading speed of the model, the evaluation becomes higher. In this way, the reading speed is evaluated for each phrase (that is, an evaluation score is calculated). Moreover, the reading aloud evaluation part 32 evaluates the reading speed in all the phrase sections based on the evaluation of the reading speed for each phrase section. In the evaluation of the reading speed in all the phrase sections, for example, the average value of the reading speed evaluation points calculated for each phrase section is calculated as the total evaluation score of the reading speed in all the phrase sections.

  Moreover, the reading aloud evaluation unit 32 compares the time length of the sample interval section with the time length of the speaker interval section, and evaluates the time when the sentence is read aloud. FIG. 3B is a diagram illustrating a comparative example of the time length of the sample interval section and the time length of the speaker interval section. In the example of FIG. 3B, the time length of the sample interval section I11 is compared with the time length of the speaker interval section I21 corresponding to the sample interval section I11 (for example, the serial numbers match), and the sample interval is compared. The time length of the section I12 and the time length of the speaker interval section I22 corresponding to the sample interval section I12 are compared, and the time length of the sample interval section I13 and the speaker interval section I23 corresponding to the sample interval section I13 are compared. The length of time is compared. The comparison is also made in the same way for the sample interval section following the sample interval section I13 and the speaker interval section following the speaker interval section I23. The reading aloud evaluation unit 32 calculates, for example, a time difference between the time length of the sample interval section and the time length of the speaker interval section as a comparison result of time lengths, and calculates an evaluation score based on the absolute value of this time difference. To evaluate the time. This evaluation score is calculated so that the evaluation is higher (that is, the evaluation score is higher), for example, as the absolute value of the time difference is closer to 0, similarly to the evaluation of the reading speed. In other words, the absolute value of the time difference becomes larger as the way of taking between speakers is longer or shorter than the way of taking between models, so the evaluation becomes lower. On the other hand, since the absolute value of the time difference becomes smaller as the approach between speakers approaches the approach between models, the evaluation becomes higher. In this way, the evaluation of the time interval is made for each interval section (that is, the evaluation score is calculated). Moreover, the reading aloud evaluation part 32 evaluates the intervals in all interval intervals based on the evaluation of intervals for each interval interval. In the evaluation of the interval in all interval intervals, for example, the average value of the interval evaluation points calculated for each interval interval is calculated as the total evaluation score of intervals in all interval intervals.

  Moreover, the reading aloud evaluation part 32 compares the pitch of a model phrase area with the pitch of the speaker phrase area corresponding to a model phrase area, and performs pitch evaluation for every phrase. The reading aloud evaluation unit 32 calculates, for example, the difference between the pitch of the sample phrase section and the pitch of the speaker phrase section as a comparison result of the pitches, and calculates an evaluation score based on this difference. Make a high evaluation. For example, the evaluation score is calculated such that the evaluation is higher (that is, the evaluation score is higher) as the difference is closer to zero. That is, since the difference becomes larger as the pitch of the speaker is higher or lower than the pitch of the model, the evaluation becomes lower. On the other hand, the evaluation becomes higher because the difference becomes smaller as the pitch of the speaker approaches the pitch of the model. In this way, the pitch is evaluated for each phrase section (that is, an evaluation score is calculated). By the way, when comparing the pitches calculated every predetermined time in each phrase section, the pitch evaluation is performed by matching the start time positions of the model phrase section and the speaker phrase section, and the phrase section (phrase It is recommended that the time length of the phrase section be adjusted by expanding and contracting the unit. At this time, the length may be adjusted by simply expanding or contracting, or the position to be dynamically evaluated in the phrase section may be adjusted using a technique such as DP matching. However, the pitches to be compared may be an average value of pitches calculated every predetermined time in each phrase section, and in this case, the time position of the start of the phrase section and the time length of the phrase section need not be matched. Good. Moreover, the reading aloud evaluation part 32 evaluates the pitch in all the phrase sections based on the evaluation of the pitch for each phrase section. In the evaluation of pitches in all phrase sections, for example, an average value of pitch evaluation points calculated for each phrase section is calculated as a total evaluation score of pitches in all phrase sections.

  Moreover, the reading aloud evaluation part 32 compares the volume of a model phrase area with the volume of the speaker phrase area corresponding to a model phrase area, and performs volume evaluation for every phrase. As the volume to be compared, for example, the above-described sound pressure level (dB) is used. The reading aloud evaluation unit 32 calculates, for example, the difference between the volume of the example phrase section and the volume of the speaker phrase section as a comparison result of the volume, and evaluates the volume by calculating an evaluation score based on this difference. Do. For example, the evaluation score is calculated such that the evaluation is higher (that is, the evaluation score is higher) as the difference is closer to zero. That is, since the difference increases as the speaker volume is larger or smaller than the model volume, the evaluation is lower. On the other hand, the evaluation becomes higher because the difference becomes smaller as the volume of the speaker approaches the volume of the model. In this way, the sound volume is evaluated for each phrase section (that is, an evaluation score is calculated). Similarly to the case of the pitch, the evaluation of the volume is performed by adjusting the time position of the phrase section by expanding and contracting the phrase section (phrase unit) by matching the start time positions of the sample phrase section and the speaker phrase section. It is good to be done. Alternatively, the volume to be compared may be an average value of the volumes calculated every predetermined time in each phrase section. Moreover, the reading aloud evaluation part 32 evaluates the sound volume in all the phrase sections based on the evaluation of the sound volume for each phrase section. In the evaluation of the sound volume in all phrase sections, for example, the average value of the sound volume evaluation points calculated for each phrase section is calculated as the total sound volume evaluation score in all phrase sections.

  Moreover, the aloud reading evaluation unit 32 compares the smooth tongue of the sample phrase section with the smooth tongue of the speaker phrase section corresponding to the sample phrase section, and evaluates the smooth tongue for each phrase. The reading aloud evaluation unit 32 uses, for example, a feature value (MFCC) indicating vocal tract characteristics calculated for each phrase in the smooth tongue evaluation. The reading aloud evaluation unit 32 calculates, for example, the similarity between the feature amount of the model phrase section and the feature amount of the speaker phrase section as a smoothing tongue comparison result, and calculates an evaluation score based on this similarity degree. The smooth tongue is evaluated. For example, the evaluation score is calculated such that the higher the similarity, the higher the evaluation (that is, the higher the evaluation score). In this way, the smooth tongue is evaluated for each phrase section (that is, an evaluation score is calculated). Moreover, the reading aloud evaluation part 32 evaluates the smooth tongue in all the phrase areas based on the evaluation of the smooth tongue for every phrase area. In the evaluation of the smooth tongue in all the phrase sections, for example, the average value of the smooth tongue evaluation points calculated for each phrase section is calculated as the total evaluation score of the smooth tongue in all the phrase sections.

  And the aloud reading evaluation part 32 performs the comprehensive evaluation with respect to the aloud of the sentence based on at least the evaluation of the aloud reading speed and the evaluation of the interval. In this comprehensive evaluation, for example, an average value (or a total value) of an evaluation point of reading aloud speed in one phrase section and an evaluation score of the interval in at least one interval section before and after this phrase section. Is calculated as a comprehensive evaluation score for one section. Alternatively, the reading aloud evaluation unit 32, for example, evaluates the reading speed in one phrase section, evaluates the interval in at least one interval section before and after this phrase section, and evaluates the pitch in this phrase section. Then, based on at least one of the evaluation of the sound volume in the phrase section and the evaluation of the smooth tongue in the phrase section, a comprehensive evaluation of one section may be performed for the reading of the sentence. In this comprehensive evaluation, for example, an average value (or a total value) of evaluation points calculated in at least any two evaluations is calculated as a comprehensive evaluation point of one section.

  Moreover, the reading aloud evaluation part 32 performs the comprehensive evaluation with respect to the reading aloud of a sentence based on the evaluation of the reading speed in all the phrase sections and the evaluation of the interval in all the interval sections, for example. In this comprehensive evaluation, for example, an average value (or a total value) of a total evaluation score of reading aloud speed in all phrase sections and a total evaluation score of intervals in all interval sections may be calculated as a total evaluation score of all sections. The Alternatively, the reading aloud evaluation unit 32, for example, evaluates reading speed in all phrase intervals, evaluation of intervals in all interval intervals, evaluation of pitches in all phrase intervals, and evaluation of volume in all phrase intervals. Based on at least one of the evaluations of the tongues in all the phrase sections, a comprehensive evaluation for reading aloud sentences may be performed. In this comprehensive evaluation, for example, an average value (or a total value) of the total evaluation points calculated in at least any two evaluations is calculated as a total evaluation point for all sections.

  The display processing unit 33 displays at least one of the evaluation of the reading speed, the evaluation of the interval, and the comprehensive evaluation on the screen of the display D. As a result, it is possible to make the speaker aware of whether the speed of reading aloud or how to take a space is appropriate, and to allow effective practice and training. For example, as an evaluation of the reading speed, an evaluation point of the reading speed for each phrase section may be displayed, or a total evaluation point of the reading speed in all the phrase sections may be displayed. In addition, as evaluation of the interval, an evaluation score for the interval for each interval interval may be displayed, or a total evaluation score for the interval in all interval intervals may be displayed. Moreover, as a comprehensive evaluation, a comprehensive evaluation score of one section may be displayed, or a comprehensive evaluation score of all sections may be displayed. In addition to the above evaluation, the display processing unit 33 may display at least one of the evaluation of the pitch, the evaluation of the sound volume, and the evaluation of the tongue on the screen of the display D. In this case as well, the evaluation score for each phrase section may be displayed as the evaluation of pitch, volume, or smooth tongue, or the total evaluation score in all phrase sections may be displayed.

  Further, the display processing unit 33 is a first graph (hereinafter referred to as “example graph”) showing a time-series change of at least one of pitch and volume calculated at predetermined time intervals based on the example voice waveform data. )) And a second graph (hereinafter referred to as “speech”) representing a time-series change of at least one of pitch and volume calculated every predetermined time based on all or part of the speech waveform data. All or a part of the “person graph” is displayed on a single screen for comparison. As a result, in addition to the reading speed and how to set the interval, it is possible to make the speaker aware of whether the pitch and volume are appropriate, and to perform effective practice and training.

[2. Example of operation of the reading aloud evaluation device S]
Next, with reference to FIG. 4 etc., an example of operation | movement of the reading aloud evaluation apparatus S which concerns on 1st Embodiment is demonstrated. FIG. 4 is a flowchart showing the evaluation display process of the control unit 3 in the reading aloud evaluation apparatus S. As a premise of the evaluation display process shown in FIG. 4, calculation is performed at predetermined intervals based on the data of the model phrase section and the model interval section specified based on the model voice waveform data and the model voice waveform data. The vocal tract feature value (MFCC) data calculated for each sample phrase section based on the sampled sound pressure and pitch data and the model speech waveform data is, for example, a speech file of the model speech waveform data. And stored in the storage unit 2.

  The process shown in FIG. 4 is started, for example, when a speaker designates a desired voice file to be a model of a reading target and gives a reading start instruction via the operation unit 4. When the process shown in FIG. 4 is started, the control unit 3 turns on the microphone input, and the example phrase interval, the example interval interval, the sound pressure, the pitch, and the associated voice file are specified. Data of the vocal tract feature value (MFCC) is read from the storage unit 2 (step S1). The read data is stored in the RAM. A serial number is assigned to each of the speaker phrase section and the speaker interval section. When the speaker starts reading the sentence, the voice generated during the reading of the sentence is collected by the microphone M, and the speaker voice waveform data indicating the waveform of the collected voice is sent to the interface unit 5. Is input to the reading aloud evaluation device S.

  As described above, the control unit 3 of the reading aloud evaluation apparatus S stores (records) the input speaker voice waveform data in the storage unit 2 and, based on the input speaker voice waveform data, as described above. Sections and speaker interval sections are sequentially identified (step S2). Serial numbers are assigned to the data of the specified speaker phrase section and speaker interval section, and are stored in the RAM. The data of each speaker phrase section and each speaker interval section stored in this way are used for evaluation described later.

  Next, as described above, the control unit 3 calculates the sound pressure and the pitch for each predetermined time based on the input speaker voice waveform data, and the vocal tract feature amount for each example phrase section. (MFCC) is calculated (step S3). The calculated sound pressure, pitch, and vocal tract feature value (MFCC) data are stored in the RAM. The sound pressure, pitch, and vocal tract feature value (MFCC) data stored in this way are used for evaluation described later.

  Next, the control unit 3 compares the time length of the model phrase section and the time length of the speaker phrase section in order of serial numbers, and evaluates the reading speed (step S4). As a result of the evaluation of reading aloud speed, for example, as described above, the reading score evaluation score for each phrase section and the total reading speed evaluation score for all phrase sections are calculated.

  Next, the control unit 3 compares the time length of the sample interval section and the time length of the speaker interval section in the order of serial numbers and evaluates the time interval (step S5). As a result of the evaluation of the interval, for example, as described above, an evaluation score for the interval for each interval interval and a total evaluation score for the interval in all interval intervals are calculated.

  Next, the control unit 3 evaluates the pitch by comparing the pitch of the model phrase section and the pitch of the speaker phrase section in the order of serial numbers (step S6). As a result of the evaluation of the pitch, for example, as described above, the evaluation score of the pitch for each phrase section and the total evaluation score of the pitch in all the phrase sections are calculated.

  Next, the control unit 3 compares the volume of the model phrase section and the volume of the speaker phrase section in the order of serial numbers, and evaluates the volume (step S7). As a result of the evaluation of the sound volume, for example, as described above, a sound volume evaluation point for each phrase section and a total sound volume evaluation score for all phrase sections are calculated as the evaluation results.

  Next, the control unit 3 compares the feature quantity (MFCC) indicating the vocal tract characteristics of the model phrase section and the feature quantity (MFCC) showing the vocal tract characteristics of the speaker phrase section in the order of serial numbers, thereby evaluating the smooth tongue. Is performed (step S8). As described above, for example, as described above, a smooth tongue evaluation score for each phrase section and a total evaluation score for the smooth tongue in all phrase sections are calculated by the smooth tongue evaluation.

  Next, the control unit 3 performs a comprehensive evaluation on the reading of the sentence based on the reading result of the reading speed, the evaluation result of the interval, the evaluation result of the pitch, the evaluation result of the volume, and the evaluation result of the smooth tongue (step S9). ). As a result of the comprehensive evaluation, for example, as described above, a comprehensive evaluation score for one section and a comprehensive evaluation score for all sections are calculated as the comprehensive evaluation result.

  Next, the control unit 3 displays a section scroll screen (step S10). The section scroll screen is a screen that displays the model graph and the speaker graph in a scrollable manner and displays information based on the evaluation results of Steps S4 to S9.

  FIG. 5 is a diagram illustrating an example of a section scroll screen. The section scroll screen shown in FIG. 5 includes a graph display field 51, an evaluation result display field 52, a “switch” key 53, a scroll bar 54, and a “close” key 55. In the upper part of the graph display column 51, a pitch example graph (line shape) and a volume example graph (bar shape) corresponding to some example phrase sections in the example graph are displayed. In the lower part of the column 51, a speaker graph (pitch) of pitches corresponding to a part of speaker phrase sections in the speaker graph and a speaker graph (bars) of volume are displayed. Note that volume and pitch are assigned to the vertical axis (Y-axis) of the graph display column 51, and time is assigned to the horizontal axis (X-axis) of the graph display column 51. Further, the display area in the graph display column 51 is scroll-displayed in the time axis direction in response to a display switching instruction by operating the scroll bar 54, for example. Alternatively, the upper display area and the lower display area of the graph display field 51 may be separated and scroll-displayed independently.

  Further, in the graph display column 51, the comprehensive evaluation for each phrase section is displayed as a pattern 51a, 51b. The pattern 51a is represented by a double circle, which means that the overall evaluation score of one section is between 70 and 100 points, for example. Moreover, the pattern 51b is represented by a triangle, which means that the overall evaluation score of one section is between 30 and 49 points, for example. It should be noted that a comprehensive evaluation score for one section may be represented by a pattern representing weather such as sunny, cloudy, rainy, etc., or a comprehensive evaluation may be represented by a comprehensive evaluation score. In addition, the model phrase section and the speaker phrase section in the graph display column 51 are displayed in association with the character strings included in the phrases, respectively. In addition, the position of each character displayed in association with the speaker phrase section is determined by, for example, a labeling process. The labeling process is based on text data of sentences including phrases, speaker voice waveform data, and frequency spectrogram of speaker voice waveform data. This is processing for specifying the boundary position. Since various known methods can be applied to the labeling process, detailed description thereof is omitted.

  In the evaluation result display column 52, a total evaluation point (total: 78) for all sections, a total evaluation point for pitch (pitch: 82), a total evaluation point for volume (volume: 89), and a total smoothing tongue An evaluation point (smooth tongue: 68), a total evaluation point for reading aloud speed (speed: 81), and a total evaluation point for shortage (distance: 90) are displayed. That is, in this case, in the section scroll screen shown in FIG. 5, the overall evaluation (52) of all sections (an example of a plurality of sections) and the overall evaluation (51a or 51b) of one section are displayed on the same level screen ( That is, it is displayed without any key operation. Thereby, the speaker can be made to grasp simultaneously the comprehensive evaluation in all the sections and the comprehensive evaluation in one section.

  In such a section scroll screen display state, the control unit 3 determines whether or not the “switch” key 53 has been designated (step S11). For example, when the speaker designates the “switch” key 53 via the operation unit 4 and the control unit 3 determines that the “switch” key 53 is designated (step S11: YES), the process proceeds to step S15. On the other hand, when it is determined that the “switch” key 53 is not designated (step S11: NO), the control unit 3 proceeds to step S12.

  In step S12, the control unit 3 determines whether or not the scroll bar 54 has been operated. For example, when the control unit 3 determines that the scroll bar 54 has been operated by operating the scroll bar 54 (moving in the horizontal direction) via the operation unit 4 (step S12: YES), a graph is displayed. The display area in the column 51 is updated (step S13), and the section scroll screen in which the display area in the graph display column 51 is updated is displayed (scrolled) (step S10). That is, the control unit 3 includes a model graph portion corresponding to at least one model phrase interval in the sample graph, and a speaker graph portion corresponding to at least one speaker phrase interval in the speaker graph. Are displayed on one screen for comparison. At this time, the control unit 3 displays the model graph portion and the speaker graph portion on one screen so that the sample interval section and the speaker interval section can be compared. Then, in response to the display switching instruction by the operation of the scroll bar 54, the control unit 3 changes the model graph part displayed on one screen to the model graph part not displayed on one screen (that is, the model graph). The speaker graph portion that is displayed on one screen is changed to the speaker graph portion that is not displayed on one screen (i.e., the sample graph portion corresponding to another example phrase section). In the speaker graph, a speaker graph portion corresponding to another speaker phrase section is switched and displayed. As a result, even if the model graph and speaker graph do not fit on one screen in the time axis direction, the speaker can grasp whether the pitch and volume are appropriate for a long period of time by switching display. be able to. On the other hand, when it determines with the control part 3 not being operated with the scroll bar (step S12: NO), it progresses to step S14.

  In step S <b> 14, the control unit 3 determines whether or not the “close” key 55 is designated. When it is determined that the “close” key 55 is not designated (step S14: NO), the control unit 3 returns to step S11. On the other hand, when it is determined that the “close” key 55 is designated (step S14: YES), the control unit 3 ends the process illustrated in FIG.

  In step S15, the control unit 3 displays a section selection screen. The section selection screen is a screen that displays a phrase section and an interval section in a selectable manner, and displays information based on the evaluation results of steps S4 to S9.

  FIG. 6 is a diagram illustrating an example of a section selection screen. The section selection screen shown in FIG. 6 includes a graph display field 61, an evaluation result display field 62, a “switch” key 63, a “detail” key 64, and a “close” key 65. In the graph display column 61, a pitch sample graph (linear) and a volume sample graph (bar) corresponding to all the sample phrase sections in the sample graph are displayed. In the lower row, a speaker graph (pitch) of pitches and a speaker graph (bars) of volume corresponding to all speaker phrase sections in the speaker graph are displayed. Further, each phrase section and each interval section in the display area in the graph display column 61 can be selected by the speaker using the cursor via the operation unit 4, for example. In the example of FIG. 6, the second example phrase section 61a from the beginning is selected by the cursor, and by moving the cursor, for example, the second example interval section 61b or the third example phrase section 61a is selected. This example phrase section 61c is selected. In addition, although the comprehensive evaluation for every phrase area is displayed on the lowermost end part 61d of the graph display column 61 with the design, it does not need to be displayed. The display contents of the evaluation result display column 62 are the same as those of the evaluation result display column 52.

  In such a section selection screen display state, the control unit 3 determines whether or not the “switch” key 63 is designated (step S16). When it is determined that the “switch” key 63 is designated (step S16: YES), the control unit 3 returns to step S10 and displays the section scroll screen. On the other hand, when it is determined that the “switch” key 63 is not designated (step S16: NO), the control unit 3 proceeds to step S17. In step S <b> 17, the control unit 3 determines whether or not the “detail” key 64 has been designated. For example, when the control unit 3 determines that the “detail” key 64 is designated by selecting a “detail” key 64 after the speaker selects any section with the cursor via the operation unit 4 ( Step S17: YES), the process proceeds to Step S19. On the other hand, when it is determined that the “detail” key 64 is not designated (step S17: NO), the control unit 3 proceeds to step S18.

  In step S18, the control unit 3 determines whether or not the “close” key 65 is designated. When it is determined that the “close” key 65 is not designated (step S18: NO), the control unit 3 returns to step S16. On the other hand, when it is determined that the “close” key 65 is designated (step S18: YES), the control unit 3 ends the process shown in FIG.

  In step S19, the control unit 3 displays a section detail screen. There are two types of section detail screens: a screen that displays the details of the selected phrase section and a screen that displays the details of the selected interval section. FIG. 7 is a diagram illustrating an example of a screen that displays details of a selected phrase section, and FIG. 8 is a diagram illustrating an example of a screen that displays details of a selected interval section.

  The section detail screen shown in FIG. 7 includes a graph display field 71, an evaluation result display field 72, a “next” key 73, a “previous” key 74, and a “return” key 75. In the upper part of the graph display column 71, a sample graph portion (line shape) of the pitch and a sample graph portion (bar shape) of the volume corresponding to the sample phrase section F12 selected in the graph display column 61 are displayed. In the lower part of the display field 71, the speaker graph part (linear) of the pitch corresponding to the speaker phrase period F22 corresponding to the model phrase period F12 selected in the graph display field 61 and the speaker graph of the volume are shown. The part (bar shape) is displayed. At this time, as shown in FIG. 7, the control unit 3 matches the start position 71 b of the model phrase section with the start position 71 b of the speaker phrase section in the time axis direction. Thereby, the speaker can grasp at a glance whether the reading speed is appropriate.

  The evaluation result display field 72 includes an overall evaluation score (total: 52) for one section corresponding to the selected example phrase section F12, an evaluation score for the pitch (pitch: 78) in this one section, Evaluation point of sound volume in one section (volume: 56), evaluation point of smooth tongue in this one section (smooth tongue: 68), evaluation point of reading speed in this one section (speed: 78), and interval in this one section Evaluation points (interval: 90) are displayed. That is, in this case, the overall evaluation (62) of all the sections (an example of a plurality of sections) on the section selection screen shown in FIG. 6 is different from the overall evaluation (72) of one section on the section detail screen shown in FIG. Displayed on the hierarchy screen. Thereby, it is possible to make the speaker understand the comprehensive evaluation in all sections and the comprehensive evaluation in one section separately and easily.

  In such a section selection screen display state, the control unit 3 determines whether or not the “next” key 73 is designated (step S20). When it is determined that the “next” key 73 is designated (step S20: YES), the control unit 3 selects the next section (phrase section in the example of FIG. 7) (step S21), and selects the selected section ( In the example of FIG. 7, the section detail screen corresponding to the model phrase section F13 and the speaker phrase section F23) is displayed (step S19). On the other hand, when it is determined that the “next” key 73 is not designated (step S20: NO), the control unit 3 proceeds to step S22.

  In step S <b> 22, the control unit 3 determines whether or not the “Previous” key 74 is designated. If it is determined that the “Previous” key 74 has been designated (step S22: YES), the control unit 3 selects the previous section (phrase section in the example of FIG. 7) (step S23), and selects the selected section ( In the example of FIG. 7, a section detail screen corresponding to the model phrase section F11 and the speaker phrase section F21) is displayed (step S19). On the other hand, when it is determined that the “Previous” key 74 is not designated (step S22: NO), the control unit 3 proceeds to step S24.

  In step S24, the control unit 3 determines whether or not the “return” key 75 has been designated. When it is determined that the “return” key 75 is not designated (step S24: NO), the control unit 3 returns to step S20. On the other hand, when it determines with the "return" key 75 having been designated (step S24: YES), the control part 3 displays an area selection screen again (step S15).

  On the other hand, the section detail screen shown in FIG. 8 includes a graph display field 81, an evaluation result display field 82, a “next” key 83, a “previous” key 84, and a “return” key 85. In the upper part of the graph display column 81, a sample graph portion (line shape) of the pitch before and after the sample interval section I11 selected in the graph display column 61 and a sample graph portion (bar shape) of the volume are displayed. In the lower part of the display field 81, the speaker graph part (linear) of the pitch before and after the speaker interval section I21 corresponding to the sample interval section I11 selected in the graph display field 61 and the speaker graph of the volume level are displayed. The part (bar shape) is displayed. That is, the control unit 3 includes a sample graph portion corresponding to the sample phrase interval, a sample interval interval following the sample phrase interval, a speaker graph portion corresponding to the speaker phrase interval, and the speaker phrase interval. The speaker graph portion and the speaker graph portion are displayed on one screen so that the subsequent speaker interval section can be compared. At this time, as shown in FIG. 8, the control unit 3 matches the start position 81b of the sample interval section with the start position 81b of the speaker interval section in the time axis direction. This allows the speaker to grasp at a glance whether the layout is appropriate.

  The evaluation result display field 82 displays a total evaluation score (total: 62) for one section corresponding to the selected sample interval section I11 and an evaluation score for the interval in this one section (interval: 62). ing. Note that the overall evaluation score in this case is calculated based only on the evaluation of the availability, and therefore coincides with the evaluation score of the availability.

  In such a section selection screen display state, when the control unit 3 determines that the “next” key 83 is designated (step S20: YES), the control section 3 selects the next section (interval section in the example of FIG. 8). Selection is made (step S21), and a section detail screen corresponding to the selected section (in the example of FIG. 8, the sample interval section I12 and the speaker interval section I22) is displayed (step S19). After this, for example, when it is determined that the “Previous” key 84 has been designated (step S22: YES), the control unit 3 selects the previous section (interval section in the example of FIG. 8) (step S23). The section detail screen corresponding to the selected section is displayed (step S19). If it is determined that the “return” key 85 is designated (step S24: YES), the control unit 3 displays the section selection screen again (step S15).

  As described above, according to the above embodiment, the time length of the model phrase section specified based on the model voice waveform data and the time length of the speaker phrase section specified based on the speaker voice waveform data are calculated. The reading speed is evaluated for each phrase in comparison, and the time length of the sample interval section specified based on the model voice waveform data and the time length of the speaker interval section specified based on the speaker voice waveform data And at least one of a reading speed evaluation, a gap evaluation, and a comprehensive evaluation based on at least a reading speed evaluation and a reading evaluation. One evaluation was configured to be displayed on the screen. Therefore, not only inflection and smooth tongue, but also speech reading speed evaluation, interval evaluation, and comprehensive evaluation that considers these evaluations for speakers who conduct speech learning training such as language learning, announcements and readings. At least one of the evaluations can be presented, and as a result, it is possible to make the speaker aware of whether the speed of reading aloud or how to set the interval is appropriate and to make effective practice and training.

[3. Second embodiment]
Next, the configuration, function, and the like of the reading aloud evaluation apparatus S according to the second embodiment of the present invention will be described. The basic configuration and functions of the reading aloud evaluation device S according to the first embodiment are also applied to the reading aloud evaluation device S according to the second embodiment. A configuration, functions, and the like different from the reading aloud evaluation apparatus S according to the embodiment will be described. In the second embodiment, the speech processing unit 31 of the reading aloud evaluation device S includes a hand text element section (first sentence element section from the start timing to the end timing of sentence elements constituting a sentence based on the model speech waveform data. For each sentence element, and for each sentence element, a speaker sentence element section (an example of the second sentence element section) from the start timing to the end timing of the sentence element is specified based on the speaker speech waveform data. You may comprise. Here, the sentence element is a unit constituting a sentence. Examples of sentence elements include the phrases, phrases, words, combined phrases, and the like described above. The combined phrase is formed by combining a plurality of phrases. Moreover, the phrase in other embodiment is comprised from one or more clauses. That is, one phrase may be composed of one phrase, and one phrase may be composed of a plurality of phrases. A phrase is a group of one or more words, for example. Words include independent words such as nouns, verbs, adjectives, adverbs, and conjunctions (parts of speech that can constitute a phrase alone), adjuncts such as auxiliary verbs and particles (parts of speech that cannot constitute a phrase alone), and the like. Examples of sentences that can be read aloud include sentences used in language learning, announcements, and reading.

  Also in the second embodiment, the start timing and the end timing may be recognized from the sound waveform, or may be recognized from the sound pressure level (dB) calculated as described above. For example, the voice processing unit 31 recognizes the time point when the amplitude of the voice waveform has reached a predetermined value or more as the start timing. Alternatively, the sound processing unit 31 recognizes the time point when the sound pressure level (dB) becomes a predetermined value or more as the start timing. Further, for example, the voice processing unit 31 recognizes the time point when the amplitude width of the voice waveform is less than a predetermined value as the end timing. Alternatively, the sound processing unit 31 recognizes the time point when the sound pressure level (dB) becomes less than a predetermined value as the end timing. For example, only when the time from when the sound pressure level (dB) becomes less than a predetermined value until the time when the sound pressure level (dB) becomes more than a predetermined value (silence time) is equal to or more than a threshold value, A point in time when the sound pressure level (dB) becomes less than a predetermined value is recognized as the end timing, and a point in time when the sound pressure level (dB) exceeds the predetermined value may be recognized as the start timing (sound waveform waveform). The same applies to the amplitude). This means that when the silent time is shorter than the threshold value, the sentence element is not divided in the section. By the way, for example, if there is a sample voice waveform data that is separated and read slowly, “Please do not call (wait) on a (pause) mobile phone in the car” (pause) in the manner mode. By recognizing the start timing and end timing, it is possible to respond to each of the four phrases such as “in the car”, “cell phone”, “set to manner mode”, and “please refrain from talking”. It is specified by being divided into body element sections. In addition, when the speaker reads the same sentence at the same interval as the model, it recognizes the start timing and end timing by the above method, so that "in the car", "cell phone" Each of the four phrases is specified by being divided into corresponding speaker sentence element sections, such as “on manner mode” and “please refrain from talking”. On the other hand, for example, in the above sentence, the speaker quickly reads “Please set the manner mode” and “Please refrain from talking”, and quickly “Please refrain from talking after setting the manner mode”. When reading aloud, this part becomes a phrase to be read at a breath. When the start timing and the end timing are recognized by the above method, the speaker sentence element section is specified without particularly dividing the part. As described above, when a plurality of phrases read aloud by the model correspond to one phrase read aloud by the speaker, the hand text element section corresponding to the phrase read aloud by the model and the speaker It becomes difficult to compare with the speaker sentence element section of the phrase read aloud. For this reason, in such a case, the speech processing unit 31 sets a plurality of phrases so that a phrase read aloud by a speaker ("Please refrain from calling after setting the manner mode") is matched with a phrase aloud by a model. The speaker sentence element section may be specified by segmenting into phrases or words.

  More specifically, the voice processing unit 31 calculates, for example, a cumulative value of sound pressure (hereinafter referred to as a “sample sound pressure cumulative value”) from a voice waveform indicated by stored model voice waveform data. It is calculated in advance for each section, and stored in a RAM or the like in association with each hand text element section. The sample sound pressure cumulative value is calculated in the order of phrases read aloud, for example, “In the car” → “Mobile phone is” → “Set to manner mode” → “Please refrain from talking”. That is, the first example sound pressure accumulated value, the second example sound pressure accumulated value, and so on are calculated. In addition, the speech processing unit 31 calculates, in advance, the number of phonemes (hereinafter referred to as “example phonemes”) from the speech waveform indicated by the stored example speech waveform data for each hand text element section. Each text element section is associated with each other and stored in a RAM or the like. Examples of phonemes include three vowels only, consonants only, and combinations of consonants and vowels. There are five vowels: a (a), i (i), u (u), e (e), and o (o). Consonants include sound components other than vowels (for example, k, s, t, n, h, m, y, r, w...). The number of model phonemes is calculated in the order of phrases read aloud, for example, “In the car” → “Mobile phone is” → “Set to manner mode” → “Please refrain from talking”. That is, the number of first example phonemes, the number of second example phonemes, and so on are specified. The phoneme identification method is well-known as a labeling method and the like, so detailed description thereof will be omitted.

  Then, the voice processing unit 31 inputs speaker voice waveform data indicating a voice waveform when the speaker reads the sentence aloud, and generates a sound in time series from the voice waveform indicated by the input speaker voice waveform data. The pressure is integrated and the number of phonemes is specified in time series. During this period, the speech processing unit 31 determines a delimiter timing (corresponding to a start timing or an end timing) by making a composite determination such as how much the integrated value of the sound pressure has been reached and how many phonemes have been specified. Identify. For example, the sound processing unit 31 compares the integrated value of the sound pressure with the sample sound pressure accumulated value (comparison in the order of the first sample sound pressure accumulated value, the second sample sound pressure accumulated value,...) The first timing when the difference is within the threshold is compared with the number of specified phonemes and the number of example phonemes (number of first example phonemes, number of second example phonemes,... The separation timing is sequentially specified from the second timing when the difference is within the threshold. For example, the time between the first timing (for example, the elapsed time from the start position of the sound waveform) and the second timing is specified as the division timing. Then, the speech processing unit 31 sequentially specifies the final speaker sentence element section by further dividing the speaker sentence element section specified by the sound pressure level or the like as described above at the division timing. In this case, for example, the section of “Please refrain from calling after setting manner mode” is divided into “Please set to manner mode” and “Please refrain from calling”, so that the sentence element section is specified. Will be. In other words, for example, the segment timing is the end timing of the speaker sentence element section corresponding to “After setting the manner mode” and the start timing of the speaker sentence element section corresponding to “Please refrain from talking”. . Along with this, the text that says "Please refrain from calling after setting manner mode" will be divided into text that represents "after setting to manner mode" and text that represents "Please refrain from calling" Become. Thereby, the hand text element section corresponding to the phrase read aloud by the model and the speaker sentence element section corresponding to, for example, a phrase (a phrase in which the phrase is divided into a plurality of phrases) read aloud by the speaker are compared. It will be.

  Contrary to the above, for example, if there is a model voice waveform data to be read aloud quickly, for example, "Please refrain from talking on the phone in the manner mode in the car" By recognizing the start timing and end timing in, it is divided into hand text element sections corresponding to every two phrases, such as “in the car” and “Please refrain from calling after setting the mobile phone to silent mode” Identified. On the other hand, if the speaker reads “Let's refrain from calling (waiting) with (waiting) mobile phone set in (waiting)” in the car and separating it slowly, start timing and end using the above method. By recognizing timing, speaker sentence element sections corresponding to each of four phrases such as “in the car”, “cell phone”, “set to manner mode”, “please refrain from talking” It will be specified by being separated by. As described above, even when one phrase read aloud by the model corresponds to a plurality of phrases read aloud by the speaker, the hand text element section corresponding to the phrase read aloud by the model and the speaker This makes it difficult to compare with the speaker sentence element section of the phrase read aloud. For this reason, in such a case, the voice processing unit 31, for example, “cell phone”, “set to manner mode”, and “please refrain from talking” so as to match the phrase read aloud by the model. It is good to specify the speaker sentence element area corresponding to the combined phrase containing these three phrases.

  Also in this case, as described above, the speech processing unit 31 stores, for example, the model sound pressure accumulated value and the number of model phonemes for each model body element section. Then, the voice processing unit 31 inputs speaker voice waveform data indicating a voice waveform when the speaker reads the sentence aloud, and generates a sound in time series from the voice waveform indicated by the input speaker voice waveform data. The pressure is integrated and the number of phonemes is specified in time series. For example, the sound processing unit 31 compares the integrated value of the sound pressure with the sample sound pressure accumulated value, and the timing at which the difference falls within the threshold (for example, the integrated value of the sound pressure has reached the sample sound pressure accumulated value). (Timing) and the number of identified phonemes compared to the number of example phonemes, and the difference is within a threshold (for example, the timing when the number of identified phonemes reaches the number of example phonemes) Specify the phrase end timing. Note that the start timing of the combined phrase corresponds to the start timing of the first phrase included in the combined phrase. Then, the speech processing unit 31 specifies the section from the start timing to the end timing of the combined phrase as the final hand text element section. In this case, for example, three phrases including “cell phone is set to silent mode” and “please refrain from talking” are combined with the phrase “cell phone should be set to silent mode and refrain from talking” The speaker sentence element section is specified as the phrase. Along with this, the text indicating "mobile phone is", the text indicating "Set to manner mode", and the text indicating "Please refrain from talking" Please refrain from "". As a result, the hand text element section corresponding to the phrase read aloud by the model and the speaker sentence element section corresponding to the combined phrase read aloud by the speaker are compared.

  In the second embodiment, as in the first embodiment, the voice processing unit 31 uses the sound pressure level (dB) as the sound pressure from data cut out from the voice waveform data, for example, every predetermined time. calculate. Further, the voice processing unit 31 calculates a fundamental frequency (Hz) from data cut out at predetermined time intervals from the voice waveform data, and calculates the calculated fundamental frequency (Hz) at predetermined time intervals as a pitch. The voice processing unit 31 calculates a feature amount (acoustic characteristic) indicating a vocal tract characteristic used for the evaluation of the smooth tongue for each sentence element. For example, the voice processing unit 31 cuts voice waveform data for each sentence element section, divides the cut sentence element data by windowing, and obtains an amplitude spectrum by performing Fourier analysis. Then, the speech processing unit 31 multiplies the obtained amplitude spectrum by a mel filter bank and performs a discrete cosine transform (DCT) on a logarithmic value of the output of the mel filter bank to obtain a MFCC (mel frequency cepstrum coefficient). It is calculated for each sentence element as a feature amount indicating the characteristic.

  Further, the speech processing unit 31 specifies, for each sentence element, a hand text element section from the start timing to the end timing of each sentence element based on the model speech waveform data, and any one of the plurality of sentence elements The sample interval section from the end timing of the sentence element to the start timing of the next sentence element is specified. In addition, the speech processing unit 31 specifies, for each sentence element, a speaker sentence element section from the start timing to the end timing of each sentence element based on the speaker voice waveform data, and any one of a plurality of sentence elements The speaker interval interval from the end timing of the next sentence element to the start timing of the next sentence element is specified.

  Next, the reading aloud evaluation unit 32 compares the time length of the hand text element section with the time length of the speaker sentence element section and evaluates the speed at which the sentence is read aloud (speech reading speed) for each sentence element section. The reading aloud evaluation unit 32 calculates, for example, the time difference between the time length of the hand text element section and the time length of the speaker sentence element section for each sentence element section as a comparison result of time lengths, and the absolute value of this time difference is calculated. The reading speed is evaluated by calculating the evaluation score based on this. Moreover, the reading aloud evaluation part 32 evaluates the reading aloud speed in all the sentence element sections based on the reading speed evaluation for each sentence element section. In the evaluation of the reading speed in all sentence element sections, for example, the average value of the reading speed evaluation points calculated for each sentence element section is calculated as the total evaluation point of the reading speed in all sentence element sections.

  In the second embodiment, as in the first embodiment, the reading evaluation unit 32 compares the time length of the sample interval section with the time length of the speaker interval section and reads the sentence aloud. Evaluate time interval. Moreover, the reading aloud evaluation unit 32 compares the pitch of the hand text element section with the pitch of the speaker sentence element section corresponding to the hand text element section, and evaluates the pitch for each sentence element. The reading aloud evaluation unit 32 calculates, for example, a difference between the pitch of the hand text element section and the pitch of the speaker sentence element section as a comparison result of the pitches, and calculates an evaluation score based on the difference. Evaluate the pitch. Moreover, the reading aloud evaluation part 32 evaluates the pitch in all the sentence element sections based on the evaluation of the pitch for each sentence element section. In the evaluation of the pitches in all sentence element sections, for example, the average value of the pitch evaluation points calculated for each sentence element section is calculated as the total evaluation score of the pitches in all sentence element sections. Moreover, the reading aloud evaluation unit 32 compares the volume of the hand text element section with the volume of the speaker sentence element section corresponding to the hand text element section, and evaluates the volume for each sentence element. Moreover, the reading aloud evaluation part 32 evaluates the sound volume in all the sentence element sections based on the evaluation of the sound volume for each sentence element section. In the evaluation of the sound volume in all sentence element sections, for example, the average value of the sound volume evaluation points calculated for each sentence element section is calculated as the total sound volume evaluation score in all sentence element sections.

  Moreover, the reading aloud evaluation unit 32 compares the smooth tongue of the hand text element section with the smooth tongue of the speaker sentence element section corresponding to the hand text element section, and evaluates the smooth tongue for each sentence element. The reading aloud evaluation unit 32 uses, for example, a feature amount (MFCC) indicating vocal tract characteristics calculated for each sentence element in the evaluation of the smooth tongue. The reading aloud evaluation unit 32 calculates, for example, the similarity between the feature amount of the hand text element section and the feature amount of the speaker sentence element section as a smoothing tongue comparison result, and calculates an evaluation score based on the similarity. The tongue is evaluated. Moreover, the reading aloud evaluation part 32 evaluates the smooth tongue in all the sentence element sections based on the evaluation of the smooth tongue for each sentence element section. In the evaluation of the smooth tongue in all sentence element sections, for example, the average value of the smooth tongue evaluation points calculated for each sentence element section is calculated as the total evaluation score of the smooth tongue in all sentence element sections.

  And the reading aloud evaluation part 32 performs the comprehensive evaluation with respect to the reading aloud of a sentence based on the evaluation of at least reading aloud speed and evaluation of a gap, like 1st Embodiment. In this comprehensive evaluation, for example, an average value (or a total value) of an evaluation score of reading aloud speed in one sentence element section and an evaluation score of a gap in at least one interval section before and after this sentence element section. Good) is calculated as the overall evaluation score of one section. Alternatively, the reading aloud evaluation unit 32, for example, evaluates the reading speed in one sentence element section, evaluates the interval in at least one interval section before and after this sentence element section, and sounds in the sentence element section. Based on at least one of the high evaluation, the sound volume evaluation in this sentence element section, and the smooth tongue evaluation in this sentence element section, a comprehensive evaluation of one section for reading aloud sentences may be performed. . In this comprehensive evaluation, for example, an average value (or a total value) of evaluation points calculated in at least any two evaluations is calculated as a comprehensive evaluation point of one section.

  In addition, as in the first embodiment, the reading aloud evaluation unit 32 performs a comprehensive evaluation on reading aloud sentences based on, for example, at least the evaluation of reading speed in all sentence element sections and the evaluation of interval in all interval sections. Do. In this comprehensive evaluation, for example, an average value (or may be a total value) of a total evaluation score of reading aloud speed in all sentence element sections and a total evaluation score of intervals in all interval sections is calculated as a total evaluation score of all sections. Is done. Alternatively, the reading aloud evaluation unit 32, for example, evaluates reading speed in all sentence element sections, evaluation of intervals in all interval sections, evaluation of pitches in all sentence element sections, and in all sentence element sections. Based on at least one of the evaluation of the volume and the evaluation of the smooth tongue in all sentence element sections, a comprehensive evaluation for reading aloud sentences may be performed. In this comprehensive evaluation, for example, an average value (or a total value) of the total evaluation points calculated in at least any two evaluations is calculated as a total evaluation point for all sections.

  Similar to the first embodiment, the display processing unit 33 displays at least one of the evaluation of the reading speed, the evaluation of the interval, and the comprehensive evaluation on the screen of the display D. As a result, it is possible to make the speaker aware of whether the speed of reading aloud or how to take a space is appropriate, and to allow effective practice and training. For example, as the reading speed evaluation, a reading speed evaluation score for each sentence element section may be displayed, or a total reading speed evaluation score for all sentence element sections may be displayed. In addition, as evaluation of the interval, an evaluation score for the interval for each interval interval may be displayed, or a total evaluation score for the interval in all interval intervals may be displayed. Moreover, as a comprehensive evaluation, a comprehensive evaluation score of one section may be displayed, or a comprehensive evaluation score of all sections may be displayed. In addition to the above evaluation, the display processing unit 33 may display at least one of the evaluation of the pitch, the evaluation of the sound volume, and the evaluation of the tongue on the screen of the display D. In this case as well, the evaluation score for each sentence element section may be displayed as the evaluation of pitch, volume, or smooth tongue, or the total evaluation score in all sentence element sections may be displayed. Further, as in the first embodiment, the display processing unit 33 is a model representing a time-series change in at least one of a pitch and a volume calculated every predetermined time based on the model voice waveform data. All or part of the graph, and all or part of the speaker graph representing a time-series change of at least one of the pitch and the volume calculated every predetermined time based on the speaker voice waveform data. Display on one screen for comparison. As a result, in addition to the reading speed and how to set the interval, it is possible to make the speaker aware of whether the pitch and volume are appropriate, and to perform effective practice and training.

  In addition, the display processing unit 33 performs display processing similar to that of the first embodiment, and at least one example graph portion corresponding to at least one example body text element section in the example graph and at least one in the speaker graph. The speaker graph portion corresponding to one speaker sentence element section is displayed on one screen so as to be comparable. At this time, the display processing unit 33 displays the model graph portion and the speaker graph portion on one screen so that the sample interval section and the speaker interval section can be compared. Then, in response to the display switching instruction, the display processing unit 33 converts the model graph portion displayed on one screen to the sample graph portion not displayed on the one screen (that is, other model graphs in the sample graph). The speaker graph portion that is displayed on one screen is switched to the speaker graph portion that is not displayed on one screen (that is, the speaker graph portion of the speaker graph). Among them, the display is switched to the speaker graph portion corresponding to another speaker sentence element section.

DESCRIPTION OF SYMBOLS 1 Communication part 2 Memory | storage part 3 Control part 4 Operation part 5 Interface part 6 Bus 31 Sound processing part 32 Reading aloud evaluation part 33 Display processing part S Reading aloud evaluation apparatus

Claims (14)

A first phrase section specified for each phrase based on first voice waveform data indicating a waveform of a voice that serves as a model when a sentence including a plurality of phrases is read aloud, from the start timing to the end timing of the phrase From the end timing of one of the plurality of phrases to the start timing of the next phrase, the first phrase section up to and a first interval section specified based on the first speech waveform data Storage means for storing the first interval section;
Input means for inputting second speech waveform data indicating a waveform of a speech uttered when the speaker reads the sentence aloud;
Based on the second audio waveform data, a second phrase section from the start timing to the end timing of the phrase is specified for each phrase, and the next timing is determined from the end timing of any of the phrases. A specifying means for specifying the second interval section until the start timing of the phrase;
The time length of the first phrase section stored in the storage means and the time length of the second phrase section specified by the specifying means are compared, and the speed at which the sentence is read aloud is evaluated for each phrase. Speed evaluation means,
A time period for evaluating the time when the sentence is read aloud by comparing the time length of the first interval section stored in the storage means with the time length of the second interval period specified by the specifying means. An evaluation means;
Comprehensive evaluation means for performing comprehensive evaluation on the reading of the sentence based on at least the evaluation of the speed by the speed evaluation means and the evaluation of the space by the space evaluation means;
Display control means for displaying at least one of the evaluation of the speed by the speed evaluation means, the evaluation of the gap by the gap evaluation means, and the comprehensive evaluation by the comprehensive evaluation means;
A reading aloud evaluation apparatus comprising: First calculation means for calculating sound information of at least one of pitch and volume at predetermined time intervals based on the first sound waveform data;
Second calculating means for calculating sound information of at least one of pitch and volume based on the second sound waveform data at predetermined time intervals;
Further comprising
The display control means includes all or part of the first graph representing the time-series change of the sound information calculated by the first calculation means, and the time-series of the sound information calculated by the second calculation means. 2. The reading aloud evaluation apparatus according to claim 1, wherein all or part of the second graph representing a change is displayed on a single screen so as to be comparable.   The display control means includes a first graph portion corresponding to at least one first phrase interval in the first graph, and a second graph portion corresponding to at least one second phrase interval in the second graph. The graph portion is displayed on one screen so that the graph portion can be compared, and in response to a display switching instruction, the first graph portion displayed on the one screen is a graph portion not displayed on the one screen and the first graph portion is displayed. In the graph, the graph portion corresponding to the other first phrase section is switched and displayed, and the second graph portion displayed on the one screen is a graph portion not displayed on the one screen. The reading aloud evaluation apparatus according to claim 2, wherein the second graph is switched to a graph portion corresponding to the other second phrase section in the second graph.   The display control means is configured to compare the first graph portion and the second graph portion so that the first interval section stored in the storage means and the second interval section specified by the specifying means can be compared. The reading aloud evaluation apparatus according to claim 3, wherein the reading aloud evaluation apparatus is displayed on a screen.   The display control means includes a first graph portion corresponding to at least one first phrase section in the first graph, the first interval section following the first phrase section, and the second graph. The first graph portion and the second graph portion are displayed on one screen so that the second graph portion corresponding to at least one of the second phrase intervals and the second interval interval following the second phrase interval can be compared. The reading aloud evaluation apparatus of Claim 2 characterized by the above-mentioned.   The display control means displays the first graph part and the second graph part on one screen by matching the start position of the first phrase section and the start position of the second phrase section in a predetermined direction. The reading aloud evaluation apparatus according to claim 3, wherein the reading aloud evaluation apparatus is characterized.   The said display control means displays the said comprehensive evaluation in one said area, and the said comprehensive evaluation in the said some area on the screen of the same hierarchy, The Claim 1 thru | or 6 characterized by the above-mentioned. Reading device.   The said display control means displays the said comprehensive evaluation in the said one area, and the said comprehensive evaluation in the said some area on the screen of a different hierarchy, The Claim 1 thru | or 6 characterized by the above-mentioned. Reading device. The storage means stores the first speech waveform data,
The specifying means specifies the first phrase section for each phrase based on the first speech waveform data stored in the storage means, and specifies the first interval section. The reading aloud evaluation apparatus as described in any one of claim | item 1 thru | or 8. A reading aloud evaluation method executed by one or more computers,
A first phrase section specified for each phrase based on first voice waveform data indicating a waveform of a voice that serves as a model when a sentence including a plurality of phrases is read aloud, from the start timing to the end timing of the phrase From the end timing of one of the plurality of phrases to the start timing of the next phrase, the first phrase section up to and a first interval section specified based on the first speech waveform data A storage step for storing the first interval section in the storage means;
An input step of inputting second voice waveform data indicating a waveform of a voice uttered when the speaker reads the sentence aloud;
Based on the second audio waveform data, a second phrase section from the start timing to the end timing of the phrase is specified for each phrase, and the next timing is determined from the end timing of any of the phrases. A specifying step of specifying a second interval section until the start timing of the phrase;
The time length of the first phrase section stored in the storage means is compared with the time length of the second phrase section specified in the specifying step, and the speed of reading the sentence is evaluated for each phrase. A speed evaluation step,
A time interval for evaluating the time when the sentence is read aloud by comparing the time length of the first interval interval stored in the storage means with the time length of the second interval interval specified by the specifying step. An evaluation step;
A comprehensive evaluation step for performing a comprehensive evaluation on the reading of the sentence based on at least the evaluation of the speed by the speed evaluation step and the evaluation of the space by the space evaluation step;
A control step for displaying at least one of the evaluation of the speed by the speed evaluation step, the evaluation of the interval by the interval evaluation step, and the overall evaluation by the comprehensive evaluation step;
A reading aloud evaluation method characterized by including: A first phrase section specified for each phrase based on first voice waveform data indicating a waveform of a voice that serves as a model when a sentence including a plurality of phrases is read aloud, from the start timing to the end timing of the phrase From the end timing of one of the plurality of phrases to the start timing of the next phrase, the first phrase section up to and a first interval section specified based on the first speech waveform data A storage step for storing the first interval section in the storage means;
An input step of inputting second voice waveform data indicating a waveform of a voice uttered when the speaker reads the sentence aloud;
Based on the second audio waveform data, a second phrase section from the start timing to the end timing of the phrase is specified for each phrase, and the next timing is determined from the end timing of any of the phrases. A specifying step of specifying a second interval section until the start timing of the phrase;
The time length of the first phrase section stored in the storage means is compared with the time length of the second phrase section specified in the specifying step, and the speed of reading the sentence is evaluated for each phrase. A speed evaluation step,
A time interval for evaluating the time when the sentence is read aloud by comparing the time length of the first interval interval stored in the storage means with the time length of the second interval interval specified by the specifying step. An evaluation step;
A comprehensive evaluation step for performing a comprehensive evaluation on the reading of the sentence based on at least the evaluation of the speed by the speed evaluation step and the evaluation of the space by the space evaluation step;
A control step for displaying at least one of the evaluation of the speed by the speed evaluation step, the evaluation of the interval by the interval evaluation step, and the overall evaluation by the comprehensive evaluation step;
A program that causes a computer to execute. A first sentence element section specified for each sentence element based on first voice waveform data indicating a waveform of a voice that serves as a model when a sentence including a plurality of sentence elements is read aloud, and the start of the sentence element The first sentence element section from the timing to the end timing, and the first interval section specified based on the first speech waveform data, and from the end timing of any one of the plurality of sentence elements to the next Storage means for storing a first interval section until the start timing of the sentence element;
Input means for inputting second speech waveform data indicating a waveform of a speech uttered when the speaker reads the sentence aloud;
Based on the second speech waveform data, a second sentence element section from the start timing to the end timing of the sentence element is specified for each sentence element, and any one of the sentence elements of the plurality of sentence elements is specified. A specifying means for specifying a second interval section from the end timing to the start timing of the next sentence element;
The time length of the first sentence element section stored in the storage means is compared with the time length of the second sentence element section specified by the specifying means to evaluate the speed at which the sentence is read aloud. Speed evaluation means to be performed every time,
A time period for evaluating the time when the sentence is read aloud by comparing the time length of the first interval section stored in the storage means with the time length of the second interval period specified by the specifying means. An evaluation means;
Comprehensive evaluation means for performing comprehensive evaluation on the reading of the sentence based on at least the evaluation of the speed by the speed evaluation means and the evaluation of the space by the space evaluation means;
Display control means for displaying at least one of the evaluation of the speed by the speed evaluation means, the evaluation of the gap by the gap evaluation means, and the comprehensive evaluation by the comprehensive evaluation means;
A reading aloud evaluation apparatus comprising: A reading aloud evaluation method executed by one or more computers,
A first sentence element section specified for each sentence element based on first voice waveform data indicating a waveform of a voice that serves as a model when a sentence including a plurality of sentence elements is read aloud, and the start of the sentence element The first sentence element section from the timing to the end timing, and the first interval section specified based on the first speech waveform data, and from the end timing of any one of the plurality of sentence elements to the next A storage step of storing in the storage means a first interval section up to the start timing of the sentence element;
An input step of inputting second voice waveform data indicating a waveform of a voice uttered when the speaker reads the sentence aloud;
Based on the second speech waveform data, a second sentence element section from the start timing to the end timing of the sentence element is specified for each sentence element, and any one of the sentence elements of the plurality of sentence elements is specified. A specifying step of specifying a second interval section from the end timing to the start timing of the next sentence element;
The time length of the first sentence element section stored in the storage means and the time length of the second sentence element section specified by the specifying step are compared to evaluate the speed at which the sentence is read aloud. A speed evaluation step to be performed every time,
A time interval for evaluating the time when the sentence is read aloud by comparing the time length of the first interval interval stored in the storage means with the time length of the second interval interval specified by the specifying step. An evaluation step;
A comprehensive evaluation step for performing a comprehensive evaluation on the reading of the sentence based on at least the evaluation of the speed by the speed evaluation step and the evaluation of the space by the space evaluation step;
A control step for displaying at least one of the evaluation of the speed by the speed evaluation step, the evaluation of the interval by the interval evaluation step, and the overall evaluation by the comprehensive evaluation step;
A reading aloud evaluation method characterized by including: A first sentence element section specified for each sentence element based on first voice waveform data indicating a waveform of a voice that serves as a model when a sentence including a plurality of sentence elements is read aloud, and the start of the sentence element The first sentence element section from the timing to the end timing, and the first interval section specified based on the first speech waveform data, and from the end timing of any one of the plurality of sentence elements to the next A storage step of storing in the storage means a first interval section up to the start timing of the sentence element;
An input step of inputting second voice waveform data indicating a waveform of a voice uttered when the speaker reads the sentence aloud;
Based on the second speech waveform data, a second sentence element section from the start timing to the end timing of the sentence element is specified for each sentence element, and any one of the sentence elements of the plurality of sentence elements is specified. A specifying step of specifying a second interval section from the end timing to the start timing of the next sentence element;
The time length of the first sentence element section stored in the storage means and the time length of the second sentence element section specified by the specifying step are compared to evaluate the speed at which the sentence is read aloud. A speed evaluation step to be performed every time,
A time interval for evaluating the time when the sentence is read aloud by comparing the time length of the first interval interval stored in the storage means with the time length of the second interval interval specified by the specifying step. An evaluation step;
A comprehensive evaluation step for performing a comprehensive evaluation on the reading of the sentence based on at least the evaluation of the speed by the speed evaluation step and the evaluation of the space by the space evaluation step;
A control step for displaying at least one of the evaluation of the speed by the speed evaluation step, the evaluation of the interval by the interval evaluation step, and the overall evaluation by the comprehensive evaluation step;
A program that causes a computer to execute.