JP2015068863A - Voice synthesis device, voice synthesis method, and voice synthesis program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a voice synthesis device capable of generating a synthetic voice having a stable rhythm close to a natural voice.SOLUTION: A voice synthesis device 1 includes: approximate pitch pattern output means 11 for inputting a pitch pattern including a fine structure, and for generating an approximate pitch pattern for approximately expressing the pitch pattern; and pitch pattern connection means 12 for inputting an original utterance pitch pattern based on the registered pitch pattern and the approximate pitch pattern, and for determining a connection point between the pitch pattern including the fine structure and the original utterance pitch pattern on the basis of the approximate pitch pattern, and for connecting the pitch pattern including the fine structure with the original utterance pitch pattern at the connection point to generate a connected pitch pattern.

Description

  The present invention relates to a speech synthesizer, a speech synthesis method, and a speech synthesis program using a waveform editing method, and more particularly to a speech synthesizer, a speech synthesis method, and a speech synthesis program that can generate natural prosody.

  In speech synthesis technology using a method of synthesizing speech by editing a short-time waveform called a segmental waveform (waveform editing method), prosody information expressed by pitch pattern, phoneme duration, etc. is generated. The unit waveform is edited so as to reproduce the prosodic information, and the entire waveform is constructed. In order to correctly convey the linguistic meaning through synthesized speech, it is important to accurately reproduce prosodic information. In general, when accurately reproducing prosodic information, it is necessary to change the pitch frequency of the segment waveform extracted from the recorded speech, which is known to degrade the quality of the synthesized speech that is generated. Yes.

  Patent Document 1 discloses a speech synthesizer that prevents deterioration in sound quality due to a change in pitch frequency by collating text data to be synthesized with the content of the original utterance of data stored in the segment waveform database. Yes. The speech synthesizer of Patent Document 1 connects segment waveforms without editing the pitch pattern as much as possible in the section where the data stored in the segment waveform database matches the text data to be synthesized. Also, in a section where the data stored in the segment waveform database and the text data to be synthesized do not match, synthesized speech is generated by using a standard pitch pattern.

  According to a statistical method such as HMM as shown in Non-Patent Document 1, it is possible to generate a pitch pattern that represents even a fine structure (HMM: Hidden Markov Model). The fine structure of the pitch pattern refers to a steep variation in pitch frequency affected by the articulation style of phonemes, the shape of the vocal tract of the speaker, and the like typified by microprosody.

JP 2009-535999 A

Keiichi Tokuda, “Application of Hidden Markov Model to Speech Synthesis”, IEICE Technical Report, SP99-61, pp. 47-54, 1999.

  According to the speech synthesizer of Patent Document 1, it is possible to generate a synthesized speech with high sound quality while maintaining the naturalness and stability of the prosody of the entire sentence. However, in a section where the text data to be synthesized does not match the original utterance, a standard pitch pattern in which the fine structure is smoothed is used. Therefore, the sound quality of the synthesized speech generated is lower than that of the section in which the original utterance is reproduced, and there is a problem that the uniformity of the prosody naturalness in all the sections is lost.

  According to Non-Patent Document 1, the uniformity of prosody naturalness can be obtained by generating a pitch pattern including a fine structure and using the fine structure of the pitch pattern in a section that does not match the original utterance.

  However, the pitch pattern including the fine structure information is not stable in outline for each accent phrase as compared with the standard pitch pattern. For example, in Japanese standard accents, a pitch peak generally appears at the first or second mora of an accent phrase, but a local peak may appear rearward because a fine structure is expressed. If such a pitch pattern and a pitch pattern that reproduces the original utterance are partially connected, the outline of the pitch pattern in the accent phrase will be destroyed. As a result, there is a problem that the synthesized speech is heard with an incorrect accent. In addition, in the pitch pattern generated by HMM or the like, there is a problem that information on the unvoiced sound section is missing, and no information on the outline exists in the soundless section.

  An object of the present invention is to provide a speech synthesizer capable of generating synthesized speech having a stable prosody close to a real voice.

  The speech synthesizer of the present invention has an input of a pitch pattern including a fine structure, an approximate pitch pattern output means for generating an approximate pitch pattern that approximately represents the pitch pattern, and an original utterance pitch based on the registered pitch pattern A connection point between the pitch pattern including the fine structure and the original utterance pitch pattern is determined based on the approximate pitch pattern and the pitch pattern including the fine structure and the original utterance pitch pattern are input at the connection point. And pitch pattern connection means for generating a connection pitch pattern.

  In the speech synthesis method of the present invention, a pitch pattern including a fine structure is input to generate an approximate pitch pattern that approximately represents the pitch pattern, and an original utterance pitch pattern and an approximate pitch pattern based on the registered pitch pattern are generated. To determine the connection point between the pitch pattern including the fine structure and the original utterance pitch pattern based on the approximate pitch pattern, and connecting the pitch pattern including the fine structure and the original utterance pitch pattern at the connection point Generate a pattern.

  The speech synthesis program of the present invention includes a process for generating an approximate pitch pattern that approximates a pitch pattern by inputting a pitch pattern including a fine structure, and an original utterance pitch pattern and an approximate pitch based on the registered pitch pattern. The connection point between the pitch pattern including the fine structure and the original utterance pitch pattern is determined based on the approximate pitch pattern, and the pitch pattern including the fine structure and the original utterance pitch pattern are connected at the connection point. And causing the computer to execute processing for generating a connection pitch pattern.

  According to the present invention, a synthesized speech having a stable prosody close to a real voice is generated by smoothly connecting a pitch pattern having a fine structure and an original utterance pitch pattern so as to maintain a pitch pattern outline in an accent phrase. It becomes possible to do.

It is a functional block diagram of the speech synthesizer concerning an embodiment of the present invention. It is a functional block diagram of the speech synthesizer concerning an embodiment of the present invention. 1 is a functional block diagram of a speech synthesizer according to a first embodiment of the present invention. It is a flowchart for demonstrating operation | movement of the speech synthesizer which concerns on the 1st Embodiment of this invention. It is a functional block diagram of the speech synthesizer concerning a 2nd embodiment of the present invention. It is a flowchart for demonstrating operation | movement of the speech synthesizer which concerns on the 2nd Embodiment of this invention. It is a functional block diagram of the speech synthesizer concerning Example 1 of the present invention. It is a figure for demonstrating the speech synthesis | combination in the speech synthesizer based on Example 1 of this invention. It is a figure for demonstrating the speech synthesis | combination in the speech synthesizer based on Example 1 of this invention. It is a figure for demonstrating the speech synthesis | combination in the speech synthesizer based on Example 1 of this invention. It is a figure for demonstrating the speech synthesis | combination in the speech synthesizer which concerns on Example 2 of this invention. It is a figure for demonstrating the speech synthesis | combination in the speech synthesizer which concerns on Example 2 of this invention. It is a figure for demonstrating the speech synthesis | combination in the speech synthesizer which concerns on Example 2 of this invention. It is a functional block diagram of the speech synthesizer concerning Example 3 of the present invention. It is a functional block diagram of the approximate pitch pattern output means which concerns on Example 3 of this invention.

  EMBODIMENT OF THE INVENTION Below, the form for implementing this invention is demonstrated using drawing. However, the embodiments and examples described below are technically preferable for carrying out the present invention, but the scope of the invention is not limited to the following. In addition, about each embodiment, the same code | symbol is attached | subjected to the same component and description is abbreviate | omitted suitably.

  FIG. 1 is a block diagram showing a functional configuration of a speech synthesizer 1 according to an embodiment of the present invention.

  The speech synthesizer 1 generates an approximate pitch pattern from the input pitch pattern information (hereinafter referred to as pitch pattern). Then, the speech synthesizer 1 connects the original utterance pitch pattern information (hereinafter referred to as the original utterance pitch pattern) and the pitch pattern based on the generated approximate pitch pattern, and the connection pitch pattern information (hereinafter referred to as the connection pitch pattern). Is generated.

  The pitch pattern input to the speech synthesizer 1 is a time change of the fundamental frequency (pitch frequency). The pitch pattern is generated based on context information generated by performing linguistic analysis on the input text. The context information is a collection of phoneme strings corresponding to the input text to be synthesized, and information associated with each phoneme such as the environment and position of the phoneme.

  The pitch pattern is a pattern in which patterns corresponding to each phoneme constituting the phoneme string included in the context information are arranged, and includes a fine structure that finely changes in a short time. In addition, although a pitch pattern is obtained for voiced sound, a pitch pattern cannot be obtained for unvoiced sound.

  The original utterance pitch pattern is a pattern that faithfully reproduces a pitch pattern including a minute change in pitch frequency of recorded voice. The original utterance pitch pattern is expressed by a node having numerical values of time and pitch frequency. The original utterance pitch pattern is stored in advance in a database or the like.

  The speech synthesizer 1 includes approximate pitch pattern output means 11 and pitch pattern connection means 12.

(Approximate pitch pattern output means)
The approximate pitch pattern output means 11 generates an approximate pitch pattern that approximately represents the input pitch pattern (P1).

  The approximate pitch pattern output unit 11 may have a function of generating an approximate pitch pattern by itself or a function of selecting an approximate pitch pattern stored in a database or the like.

  The approximate pitch pattern output means 11 generates an approximate pitch pattern from the pitch pattern by, for example, polynomial approximation or spline interpolation curve approximation using representative points. However, since the pitch pattern is not generated for the unvoiced sound section, the pitch pattern is obtained by interpolation.

  When obtaining an approximate curve, features related to prosody for each language may be added as constraints. For example, in the case of Japanese, a feature that the peak rises at the first to second mora in the accent phrase, then gently falls to the accent fall, and then suddenly drops near the accent fall mora can be used as the constraint condition.

  In addition, when the unvoiced sound section is continuous, the peak position, frequency value, and starting pitch frequency value may not be clarified. In such a case, a temporary pitch pattern may be estimated. For example, a temporary pitch pattern can be estimated by replacing unvoiced phonemes in the input text with voiced phonemes that are close to the features of the unvoiced phonemes. For example, it is possible to select voiced / unvoiced sounds with similar articulation structures, such as unvoiced gum burst sound “t” and voiced gum burst sound “d”.

  Depending on the language, there may be unvoiced phonemes that do not have voiced phonemes with similar articulatory structures. In such a case, it may be replaced with an appropriate voiced phoneme such as a phoneme (for example, “n”) that does not easily include a sharp change in pitch pattern. In addition, the unvoiced vowel may be replaced with the corresponding voiced vowel.

  Further, the approximate pitch generation means 10 may generate an approximate pitch pattern by selecting a pitch pattern created in advance. For example, a large amount of audio data is stored in a database, and a pitch pattern is extracted for each accent phrase from the audio data. The approximate pitch pattern output unit 11 may generate an approximate pitch pattern from the extracted pitch pattern.

(Pitch pattern connection means)
The pitch pattern connection means 12 generates a connection pitch pattern obtained by connecting the original utterance pitch pattern and the pitch pattern based on the approximate pitch pattern, and outputs the generated connection pitch pattern.

  The pitch pattern connection means 12 determines a connection point that is a frequency value for connecting the original utterance pitch pattern and the pitch pattern based on the approximate pitch pattern.

  For example, when an original utterance pitch pattern having a syllable string and an accent position that matches the syllable string included in the pitch pattern is searched, a boundary line is defined between the phoneme to which the original utterance pitch pattern is connected and the original utterance pitch pattern. To do. And the frequency value of the intersection of an approximate pitch pattern and a boundary line can be determined as a connection point.

  Further, an intersection point between the approximate pitch pattern and the boundary line may be used as a reference value, and a point different from the intersection point between the approximate pitch pattern and the boundary line may be used as the connection point. For example, the intersection of the approximate pitch pattern and the boundary line, the average value of the frequency values on the boundary line of the pitch pattern and the original utterance pitch pattern may be used as the connection point. In addition, when the pitch pattern and the original utterance pitch pattern change sufficiently smoothly on the boundary line, the frequency value on the boundary line of either the pitch pattern or the original utterance pitch pattern may be used as the connection point.

  At the connection point, the pitch pattern and the original utterance pitch pattern may be connected as they are, but if the pitch changes sharply at the connection point, it may cause noise when a waveform is generated later. Therefore, it is desirable to perform smoothing in the vicinity of the connection point between the pitch pattern and the original utterance pitch pattern. For example, two phonemes adjacent to the connection point may be set as a smoothing section, and smoothing may be performed in the smoothing section. If the pitch pattern and the original utterance pitch pattern change sufficiently smoothly in the vicinity of the boundary line, the pitch pattern or the original utterance pitch pattern can be connected by translating in the frequency axis direction without performing smoothing. I do not care.

  Further, the pitch pattern connecting means 12 connects the pitch pattern and the original utterance pitch pattern in order to balance the pitch pattern of the whole sentence, and then, based on the peak value of the connection pitch pattern in the accent phrase, The whole is corrected in the frequency direction. Specifically, the entire pitch pattern is translated in the frequency direction by the difference between the connected pitch pattern and the peak value of the approximate pitch pattern. However, for example, when the difference is small, this process may not be performed.

  Here, referring to FIG. 2, the speech synthesizer 2 in which components are added to the approximate pitch pattern output means 11 and the pitch pattern connection means 12 in order to realize the function of the speech synthesizer 1 according to the embodiment of the present invention. Will be described. In FIG. 2, the data flow is omitted.

  The speech synthesizer 2 includes the above-described approximate pitch pattern output means 11 and approximate pitch pattern connection means 12. The speech synthesizer 2 also includes a language analysis unit 13, a pitch pattern generation unit 14, a phoneme duration generation unit 15, an original utterance pitch pattern selection unit 16, a unit waveform selection unit 17, and a speech waveform generation unit 18. And comprising. Furthermore, the speech synthesizer 2 includes a pitch pattern generation model storage unit 21, a phoneme duration generation model storage unit 22, an original utterance pitch pattern storage unit 23, and a unit waveform storage unit 24.

  Here, each component will be described. Detailed description will be given in the description of embodiments and examples described later.

  When the input text is input, the language analysis unit 13 generates context information related to the input text using language analysis technology. The context information is a collection of phoneme strings corresponding to the input text and information associated with each phoneme, for example, the surrounding environment and position of the phoneme.

  The pitch pattern generation unit 14 generates pitch pattern information using the pitch pattern generation model stored in the pitch pattern generation model storage unit 21 based on the generated context information.

  The phoneme duration generation unit 15 generates a phoneme duration using the phoneme duration generation model stored in the phoneme duration generation model storage unit 22.

  The approximate pitch pattern output means 11 generates an approximate pitch pattern based on the pitch pattern information.

  The original utterance pitch pattern selection means 16 searches the original utterance pitch pattern based on the syllable string information stored in the original utterance pitch pattern storage means 23, and includes a pitch pattern that matches at least a part of the syllable string information. Select original utterance pitch pattern data. Then, the original utterance pitch pattern selection means 16 searches the selected original utterance pitch pattern data for a portion where the relevant / front / rear syllable string and accent position match as the original utterance pitch pattern. Further, the original utterance pitch pattern selection means 16 determines a position where the original utterance pitch pattern is connected to a pitch pattern having a fine structure based on the search result.

  The pitch pattern connection unit 12 connects the original utterance pitch pattern selected by the original utterance pitch pattern selection unit 16 and the pitch pattern generated by the pitch pattern generation unit 14 based on the approximate pitch pattern, and connects the connection pitch pattern. Is generated. Note that the information of the connection pitch pattern generated by connecting the original utterance pitch pattern and the pitch pattern having a fine structure is a target value of the pitch frequency.

  The unit waveform selection means 17 selects the unit waveform data stored in the unit waveform storage means 24 based on the prosodic information that combines the connection pitch pattern and the generated phoneme duration. In the waveform editing type speech synthesis method in which unit selection is performed, the pitch frequency is expressed by adjusting the interval (pitch interval) at which the selected unit waveform data is superimposed.

  The speech waveform generation means 18 generates a synthesized speech waveform by editing the selected unit waveform data so as to reproduce the generated prosodic information.

  The pitch pattern generation model storage unit 21 stores a pitch pattern generation model. The phoneme duration generation model storage unit 22 stores a phoneme duration generation model. Note that the pitch pattern generation model and the phoneme duration generation model may be stored in the same database.

  The original utterance pitch pattern storage means 23 stores syllable string data indicating the original utterance pitch pattern and utterance content.

  The unit waveform storage means 24 stores unit waveform data.

  The above is the description of the speech synthesizer 1 according to the embodiment of the present invention.

  As described above, according to this embodiment, it is possible to smoothly connect pitch patterns including a fine structure and generate a synthesized speech having natural prosodic information.

  Next, a speech synthesizer according to an embodiment that embodies the speech synthesizer 1 according to the present embodiment will be described.

(First embodiment)
FIG. 3 is a block diagram illustrating a configuration example of the speech synthesizer 10 according to the first embodiment of the present invention. FIG. 4 is a flowchart showing an example of the operation of this embodiment.

  Referring to FIG. 3, the speech synthesizer 10 according to the present embodiment includes an approximate pitch pattern generation unit 111 that is an approximate pitch pattern output unit and a pitch pattern connection unit 120 that is a pitch pattern connection unit. ing.

  The operation of this embodiment will be described with reference to FIGS.

  The approximate pitch pattern generation unit 111 generates an approximate pitch pattern that approximately represents the pitch pattern (P1) (step S11). P1 is a pitch pattern including a fine structure, and the approximate pitch pattern is a non-linear approximate curve expressing the outline of P1.

  The pitch pattern connection unit 120 determines a connection point between P1 and the pitch pattern (P2) to be connected (step S12). P2 corresponds to the original utterance pitch pattern.

  And the pitch pattern connection part 120 connects P1 and P2 in a connection point (step S3).

  As described above, according to the first embodiment, it is possible to generate an approximate pitch pattern, smoothly connect pitch patterns including a fine structure, and generate a synthesized speech having natural prosodic information. .

(Second Embodiment)
Subsequently, a second embodiment of the present invention will be described.

  FIG. 5 is a block diagram showing a configuration example of the speech synthesizer 20 according to the second embodiment of the present invention. FIG. 6 is a flowchart showing an example of the operation of this embodiment.

  Referring to FIG. 5, the speech synthesizer 20 according to this embodiment includes an approximate pitch pattern selection unit 112 and an approximate pitch pattern storage unit 113 that are approximate pitch pattern output means, and a pitch pattern connection unit 120 that is a pitch pattern connection unit. And is configured. The speech synthesis apparatus 20 according to the second embodiment has a configuration in which the approximate pitch pattern generation unit 111 of the speech synthesis apparatus 10 according to the first embodiment is replaced with an approximate pitch pattern selection unit 112 and an approximate pitch pattern storage unit 113. Take.

  The operation of this embodiment will be described with reference to FIGS.

  The approximate pitch pattern storage unit 113 stores in advance at least one approximate pitch pattern that does not include a fine structure.

  The approximate pitch pattern selection unit 112 calculates the distance between the pitch pattern (P1) including the fine structure and each approximate pitch pattern stored in the approximate pitch pattern storage unit 113 (step S21).

  The approximate pitch pattern selection unit 112 selects, as the approximate pitch pattern, the shortest distance among the approximate pitch patterns stored in the approximate pitch pattern storage unit 113 (step S22).

  The pitch pattern connection unit 120 determines a connection point between P1 and the pitch pattern (P2) to be connected (step S23). P2 corresponds to the original utterance pitch pattern.

  And the pitch pattern connection part 120 connects P1 and P2 in a connection point (step S24).

  As described above, according to the second embodiment, it is possible to connect pitch patterns using approximate pitch patterns without sequentially generating approximate pitch patterns.

  Next, the speech synthesis apparatus and speech synthesis system according to the embodiment of the present invention will be described with reference to examples.

Example 1
FIG. 7 is a diagram showing an overview of the speech synthesis apparatus 100 of Example 1 according to the first embodiment of the present invention. In FIG. 7, data elements exchanged between the constituent elements are illustrated between the constituent elements.

  The speech synthesizer 100 according to the first embodiment includes an approximate pitch pattern generation unit 111 and a pitch pattern connection unit 120. The speech synthesizer 100 also includes a language analysis unit 130, a pitch pattern generation unit 140, a phoneme duration generation unit 150, an original utterance pitch pattern selection unit 160, a pitch pattern generation model storage unit 211, and an original utterance pitch. A pattern storage unit 213 and a phoneme duration generation model storage unit 212 are provided. Furthermore, the speech synthesizer 100 includes a unit waveform selection unit 170, a speech waveform generation unit 180, and a unit waveform storage unit 214. In Example 1, the same reference numerals as those in the first embodiment are used for the approximate pitch pattern generation unit 111.

  In the first embodiment, the prosodic information is treated as a time change of the fundamental frequency (pitch frequency), that is, information including two pieces of information of the pitch pattern and the phoneme duration. As the prosodic information, it is conceivable that the prosodic information includes a temporal change in the short-time power of the speech waveform.

  In the first embodiment, it is assumed that an HMM speech synthesis technique is used as a technique for generating a pitch pattern including a fine structure (HMM: Hidden Markov Model). Of course, as long as a pitch pattern including a fine structure can be generated, a voice synthesis technique other than the HMM voice synthesis technique may be used.

  Next, each component of the speech synthesizer 100 according to the first embodiment will be described in detail.

  When the input text is input, the language analysis unit 130 generates context information regarding the input text.

  The pitch pattern generation unit 140 generates pitch pattern information using the pitch pattern generation model stored in the pitch pattern generation model storage unit 211 based on the generated context information.

  The phoneme duration generation unit 150 generates a phoneme duration by using the phoneme duration generation model stored in the phoneme duration generation model storage unit 212.

  In the first embodiment, the pitch pattern and the phoneme continuation length are generated separately. However, information in the form of a combination of both may be generated.

  The pitch pattern generated by the pitch pattern generation unit 140 includes a fine structure that changes finely in a short time as in a phoneme. However, since there is no pitch for the unvoiced sound section such as “k” or “s”, a pitch pattern is not generated.

  FIG. 8 shows a specific example (solid line) of the pitch pattern generated when the input text is “please go out”. In FIG. 8, pitch patterns (solid lines) are obtained for voiced sound sections “ode”, “a”, “e”, “uda”, and “ai”. On the other hand, a pitch pattern cannot be obtained for “k” and “s” which are unvoiced sound sections.

  The approximate pitch pattern generation unit 111 generates an approximate pitch pattern that is approximately expressed based on the generated pitch pattern information.

  FIG. 9 shows a specific example of the generated approximate pitch pattern. In FIG. 9, an approximate pitch pattern (one-dot chain line) is generated based on the pitch pattern obtained in the voiced sound section.

  The original utterance pitch pattern storage unit 213 stores syllable string data indicating the original utterance pitch pattern and utterance content. The original utterance pitch pattern is a pattern that faithfully reproduces a pitch pattern including a minute change in the pitch frequency of the recorded voice, and is expressed by a node having numerical values of time and pitch frequency.

  Further, in the first embodiment, it is assumed that the original utterance pitch pattern expressing the recorded voice of the utterance content “o he N ji ku da sa @ i (please change)” is stored. Here, the vowel immediately before “@” indicates the accent position in the standard word.

  The original utterance pitch pattern selection unit 160 searches the original utterance pitch pattern storage unit 213 based on the syllable string information stored in the original utterance pitch pattern storage unit 213, and the original utterance pitch pattern storage unit 213 matches the at least part of the syllable string information. Select the utterance pitch pattern. For example, when the input text “please go out (please go out)” is input as shown in FIG. 8, the syllable string becomes “o de ka ke ku da sa @ i” from the context information.

  The original utterance pitch pattern selection unit 160 searches the original utterance pitch pattern data in the original utterance pitch pattern storage unit 213 for a portion where the corresponding / forward / rear syllable string and accent position match. The original utterance pitch pattern selection unit 160 determines a position to connect the original utterance pitch pattern based on the search result.

  In the case of the first embodiment, the “da sa @ i” portion of “o he N ji ku da sa @ i” is included in “o de kake ku da sa @ i” in both the syllable string and the accent position. Matches the syllable string to be played. Therefore, “da sa @ i” corresponds to the search result and can be used as the original utterance pitch pattern. In this way, the original utterance pitch pattern in the accent phrase is selected.

  In addition, the original utterance pitch pattern selection unit 160 sets the boundary line connecting the syllable string “da sa @ i” as “ku” and “ku” that are syllables that constitute the syllable string “o he Nji ku da sa @ i”. "da".

The pitch pattern connection unit 120 connects the selected original utterance pitch pattern (hereinafter referred to as P _o ) and the pitch pattern (hereinafter referred to as P _g ) generated by the pitch pattern generation unit 140 and connects the accent phrases. Generate a pitch pattern.

  Here, a specific example of the pitch pattern connection method by the pitch pattern connection unit 120 will be described with reference to FIG.

First, pitch pattern connecting unit 120, based on the generated approximated pitch contour, to determine the connection point is a frequency value which connects the P _o and P _g.

As determined by the original utterance pitch pattern selection unit 160, the connection between P _o and P _g is a boundary line between the syllables “ku” and “da”. The pitch pattern connection unit 120 determines the frequency value of the intersection of the generated approximate pitch pattern curve and the boundary line as a connection point.

Alternatively, the connection point may be determined using the intersection point as a reference value without using the intersection point of the boundary line and the approximate pitch pattern curve as the connection point. For example, the intersection of the generated approximate pitch pattern curve and the boundary line, and the average value of the frequency values on the boundary line of “P _o ” and “P _g ” may be used as the connection point. In addition, when P _o and P _g change sufficiently smoothly in the vicinity of the boundary line, the frequency value on the boundary line of either P _o or P _g may be used as the connection point.

Note that _Po and _Pg may be connected as they are at the connection point, but the pitch may change sharply at the connection point, which may cause abnormal noise to be mixed when a waveform is generated later. Therefore, it is desirable to smooth the pitch pattern in a range including the connection point.

In FIG. 10, the two phonemes “u” and “d” adjacent to the connection point are set as the smoothing section, and the smoothing is performed within this section. When P _o and P _g change smoothly enough in the vicinity of the boundary line, etc., P _o or P _g is translated in the frequency axis direction (vertical axis upward direction) without performing smoothing. You may connect with.

Further, the pitch pattern connecting portion 120, in order to balance the pitch pattern of the entire sentence, after connecting the P _o and P _g, based on the peak value of the connection pitch pattern in the accent phrase, the entire connection pitch pattern Processing to correct in the frequency direction is performed.

  Specifically, the entire pitch pattern is translated in the frequency direction by the difference between the connection pitch pattern and the peak value of the approximate pitch pattern. However, if the difference is small, this process may not be performed.

  The unit waveform selection unit 170 selects unit waveform data stored in the unit waveform storage unit 214 based on the prosodic information that combines the connection pitch pattern and the generated phoneme duration. The unit waveform data is generated in advance from recorded speech and indicates a speech waveform that is a minimum unit constituting a synthesized sound. Here, for the original utterance pitch pattern section in the pitch pattern, the corresponding unit waveform data of the original utterance is used.

  Finally, the speech waveform generation unit 180 generates a synthesized speech waveform by editing the selected unit waveform data so as to reproduce the generated prosodic information. Regarding the generation of the synthesized speech waveform, for example, unit waveforms may be arranged on the basis of the pitch pattern and superimposed.

  In the first embodiment, the original utterance pitch pattern selection unit 160 selects one original utterance pitch pattern. However, when there are a plurality of pieces of data satisfying conditions such as phoneme sequences, a plurality of original utterance pitches are selected. A pattern may be cited as a candidate.

  When there are a plurality of pieces of data that satisfy a condition such as a phoneme string, the pitch pattern connection unit 120 determines one optimal original utterance pitch pattern from the listed original utterance pitch pattern candidates. As a method of determining one original utterance pitch pattern from the candidates, a method of selecting the one having the smallest amount of change in pitch frequency at the time of connection can be considered based on the pitch frequency information of the original utterance pitch pattern. This is because the smaller the change amount of the pitch frequency, the more the deterioration of the sound quality can be suppressed when the synthesized waveform is generated by the speech waveform generator 180.

  According to the first embodiment, since the pitch pattern having a fine structure and the original utterance pitch pattern are smoothly connected so as to maintain the outline of the pitch pattern in the accent phrase, the synthesized speech having a stable prosody close to the real voice Can be generated.

  The above is the description of Example 1 according to the first embodiment of the present invention. Next, Example 2 according to the first embodiment of the present invention will be described.

(Example 2)
The second embodiment is based on the speech synthesizer 100 having the same configuration (FIG. 7) as the first embodiment. In the second embodiment, it is assumed that the input text is “Hyantashi Sutemuga (the criticized system)”. A specific example of the pitch pattern generated by the pitch pattern generation unit 140 in the second embodiment is shown in FIG.

  The syllable string corresponding to the input text is “hI # ha N shI # ta / shI # sU # te muga”.

  Here, “/” indicates an accent phrase delimiter. Also, phonemes with a # in uppercase vowels such as I # indicate unvoiced vowels.

  As can be seen from FIG. 11, no pitch pattern is generated in the unvoiced sound section (three places “hI # h”, “shI # t”, and “shI # sU # t”), and the peak position and frequency The pitch frequency value at the start point is not clear. For this reason, it is not possible to obtain information on the overall pitch pattern of the accent phrase. Therefore, in the second embodiment, it is necessary to estimate the temporary pitch pattern of the unvoiced sound section.

  As a method for estimating the pitch pattern of an unvoiced sound section, a method of using a pitch pattern generated after replacing an unvoiced phoneme with a voiced phoneme can be considered. Specifically, first, context information (voiced context information) is generated by replacing unvoiced phonemes in the input text with voiced phonemes having similar characteristics. It is conceivable to select phonemes having similar features such as voiced / unvoiced phonetic structures such as unvoiced gum burst sound “t” and voiced gum burst sound “d”.

  However, depending on the language, there may be no corresponding phoneme. For this reason, if there is no voiced phoneme corresponding to the unvoiced phoneme, the voiced phoneme is replaced with a voiced phoneme that does not easily contain a sharp change in pitch pattern such as “n”. For the unvoiced vowel, it is conceivable to use a corresponding voiced phoneme as “I #” → “i”.

  In the second embodiment, for example, “ni na zi da / ji zu de mu ga” is the voiced context information. The pitch pattern generation unit 140 generates a temporary pitch pattern of an unvoiced sound section based on the generated voiced context information. FIG. 12 shows a specific example (dotted line) of the generated temporary pitch pattern.

  The approximate pitch pattern generation unit 111 generates an approximate pitch pattern so as to approximately represent the generated temporary pitch pattern. FIG. 13 shows a specific example (one-dot chain line) of the generated approximate pitch pattern.

  After generating the approximate pitch pattern, the speech synthesizer 100 generates synthesized speech by the same processing as in the first embodiment.

  Note that whether or not to use the processing for replacing unvoiced phonemes and voiced phonemes shown in this embodiment may be switched depending on the situation. For example, the replacement process is performed when 50% or more of the phonemes included in the context information corresponding to the input text are unvoiced, and the replacement process is not performed when the number is less than 50%. It may be switched according to such a judgment criterion.

  According to the second embodiment, even when there are many unvoiced phonemes and it is difficult to obtain outline information of the pitch pattern, it is possible to estimate an appropriate outline of the pitch pattern. Therefore, it is possible to generate synthesized speech having a more stable prosody. .

(Example 3)
Next, Example 3 according to the second embodiment of the present invention will be described. FIG. 14 is a diagram illustrating an overview of the speech synthesis apparatus 300 according to the third embodiment of the present invention. In FIG. 14, data elements exchanged between the constituent elements are illustrated between the constituent elements.

  The speech synthesizer 300 according to the third embodiment includes an approximate pitch pattern selection unit 112 and an approximate pitch pattern storage unit 113 instead of the approximate pitch pattern generation unit 111 in the configuration of the first embodiment. Yes. In the third embodiment, the constituent elements other than the approximate pitch pattern selection unit 112 and the approximate pitch pattern storage unit 113 are the same as those in the first embodiment, and thus are denoted by the same reference numerals. In the third embodiment, the same reference numerals as those in the second embodiment are used for the approximate pitch pattern selection unit 112 and the approximate pitch pattern storage unit 113.

  The approximate pitch pattern storage unit 113 stores a plurality of approximate pitch patterns created in advance. The speech synthesizer 300 may generate an approximate pitch pattern by itself and store it in the approximate pitch pattern storage unit 113.

  Here, a method of generating an approximate pitch pattern in the speech synthesizer 300 will be described with reference to FIG. In the third embodiment, an example of generating an approximate pitch pattern by the approximate pitch pattern forming apparatus 310 (approximate pitch pattern forming means) of FIG. 15 is shown. Note that the approximate pitch pattern forming device 310 may not be included in the configuration of the speech synthesizer 300 as long as the approximate pitch pattern storage unit 113 can acquire the approximate pitch pattern from the outside.

  The approximate pitch pattern forming apparatus 310 includes an audio data storage unit 311, a pitch pattern extraction unit 312, and an approximate pitch pattern formation unit 313.

  The audio data storage unit 311 stores a plurality of audio data.

  The pitch pattern extraction unit 312 extracts a pitch pattern for each accent phrase from the audio data stored in the audio data storage unit 311 using a fundamental frequency extraction technique.

  The approximate pitch pattern forming unit 313 has the same function as the approximate pitch pattern generation unit 111 of the first embodiment, and generates an approximate pitch pattern based on the extracted pitch pattern. The approximate pitch pattern forming unit 313 stores the generated approximate pitch pattern in the approximate pitch pattern storage unit 113.

  In order to increase the choices of the approximate pitch pattern, it is desirable that the audio data is large. In order to process a large amount of audio data, it is desirable that processing such as approximate pitch pattern generation be performed automatically by a computer.

  Further, after the number of audio data is reduced, an accurate approximate pitch pattern may be manually created and stored in the approximate pitch pattern storage unit 113. If the number of audio data can be reduced, the processing time can be shortened by adjusting the scale of the approximate pitch pattern storage unit 113.

  The approximate pitch pattern selection unit 112 selects an approximate pitch pattern having a shape most similar to the pitch pattern generated by the pitch pattern generation unit 140 from the approximate pitch patterns stored in the approximate pitch pattern storage unit 113. . As a method of selecting an approximate pitch pattern by the approximate pitch pattern selection unit 112, a method of selecting a distance measure having the smallest least square error such as a Euclidean distance or a Mahalanobis distance may be considered.

  After the selection of the approximate pitch pattern, the speech synthesizer 300 generates synthesized speech by performing the same processing as in the first embodiment.

  According to the third embodiment, since the approximate pitch pattern generated from the pitch pattern extracted from the actual voice is used, it is possible to generate a synthesized voice having a prosody closer to the real voice. In addition, since it is not necessary to sequentially generate approximate curves, there is an advantage that the processing time can be shortened compared with the first embodiment by adjusting the scale of the approximate pitch pattern storage unit 113.

  As mentioned above, although this invention has been described with reference to the embodiments and examples, the present invention is not limited to the above-described embodiments and examples. Various changes that can be understood by those skilled in the art within the scope of the present invention can be made to the configuration and details of the present invention with respect to, for example, the method of deriving an approximate curve, the prosody information generation method, and the speech synthesis method.

  As described above, the speech synthesizer of the present invention can be suitably applied when constructing a speech synthesis system that expresses natural and stable intonation.

  For example, the speech synthesizer of the present invention is suitably applied to a general text reading system such as news articles and automatic response sentences.

A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.
(Appendix 1)
Approximate pitch pattern output means for receiving a pitch pattern including a fine structure and generating an approximate pitch pattern that approximately represents the pitch pattern;
Input an original utterance pitch pattern based on a registered pitch pattern and the approximate pitch pattern, and determine a connection point between the pitch pattern including the fine structure and the original utterance pitch pattern based on the approximate pitch pattern, A speech synthesizer comprising: pitch pattern connection means for generating a connection pitch pattern by connecting a pitch pattern including the fine structure and the original utterance pitch pattern at the connection point.
(Appendix 2)
The pitch pattern connecting means is:
The speech synthesizer according to appendix 1, wherein a connection point is a point where a connection boundary line between the original utterance pitch pattern and the pitch pattern intersects with the approximate pitch pattern.
(Appendix 3)
The pitch pattern connecting means is:
The speech synthesizer according to appendix 1 or 2, wherein smoothing processing is performed on the pitch pattern and the original utterance pitch pattern at the connection point.
(Appendix 4)
The pitch pattern connecting means is:
After connecting the pitch pattern and the original utterance pitch pattern, the entire connection pitch pattern is corrected in the frequency direction based on a peak value of the connection pitch pattern in an accent phrase. The speech synthesizer as described in any one of Claims.
(Appendix 5)
The approximate pitch pattern output means includes:
An approximate pitch pattern storage means for storing the approximate pitch pattern;
5. An approximate pitch pattern selection unit that selects the approximate pitch pattern stored in the approximate pitch pattern storage unit based on the pitch pattern, according to any one of Supplementary notes 1 to 4. Speech synthesizer.
(Appendix 6)
The approximate pitch pattern output means includes:
Approximate pitch pattern forming means for generating an approximate pitch pattern based on the pitch pattern extracted from the recorded voice,
The approximate pitch pattern storage means stores a plurality of approximate pitch patterns generated by the approximate pitch pattern formation means,
The approximate pitch pattern selection means includes:
6. The speech synthesizer according to appendix 5, wherein the approximate pitch pattern stored in the approximate pitch pattern storage means is selected.
(Appendix 7)
The approximate pitch pattern selection means includes:
The speech synthesizer according to appendix 5 or 6, wherein the approximate pitch pattern having a distance close to the pitch pattern including the fine structure is selected in a feature amount space having at least a pitch pattern as a feature amount.
(Appendix 8)
Language analysis means for generating context information including phoneme string information corresponding to the input text and information associated with each phoneme;
Pitch pattern generation means for inputting the context information, generating a pitch pattern including the fine structure based on a pitch pattern generation model stored in advance, and outputting the generated pitch pattern to the approximate pitch pattern output means; The speech synthesizer according to any one of appendices 1 to 7, further comprising:
(Appendix 9)
The speech synthesizer according to any one of appendices 1 to 7, further comprising language analysis means for generating context information including phoneme string information corresponding to an input text and information associated with each phoneme.
(Appendix 10)
The speech synthesizer according to appendix 9, further comprising pitch pattern generation means for inputting the context information and generating a pitch pattern including the fine structure based on a pitch pattern generation model stored in advance.
(Appendix 11)
The speech synthesizer according to appendix 9 or 10, further comprising phoneme duration generation means for inputting the context information and generating a phoneme duration using a phoneme duration generation model stored in advance.
(Appendix 12)
The approximate pitch pattern output means includes:
The speech synthesizer according to any one of appendices 8 to 11, wherein an unvoiced sound section is estimated from a voiced sound section included in the input text.
(Appendix 13)
The approximate pitch pattern output means includes:
A temporary pitch pattern is generated by replacing unvoiced phonemes constituting the pitch pattern with voiced phonemes close to the characteristics of the unvoiced phoneme, and the approximate pitch pattern is generated after interpolating the pitch pattern of the unvoiced sound section with the temporary pitch pattern The speech synthesizer according to any one of appendices 8 to 12, wherein:
(Appendix 14)
The approximate pitch pattern output means includes:
14. The speech synthesizer according to any one of appendices 1 to 13, wherein the approximate pitch pattern is generated by adding a feature related to a prosody for each language to a constraint condition.
(Appendix 15)
Searching the original utterance pitch pattern based on pre-registered syllable string information, selecting original utterance pitch pattern data including syllable string information that matches at least a part of the syllable string information, and selecting the selected original utterance pitch Additional utterance pitch pattern selection means for retrieving, as a utterance pitch pattern, a portion where the corresponding / forward / backward syllable string and accent position coincide with each other from the pattern data is provided. The speech synthesizer described.
(Appendix 16)
Unit waveform selection means for selecting unit waveform data stored in advance based on prosodic information that combines the connection pitch pattern and the phoneme duration;
The synthesis according to appendix 14 or 15, further comprising: speech waveform generation means for generating a synthesized speech waveform by editing the selected unit waveform data so as to reproduce the generated prosodic information. Audio device.
(Appendix 17)
An input of a pitch pattern including a fine structure to generate an approximate pitch pattern that approximately represents the pitch pattern,
Input the original utterance pitch pattern based on the registered pitch pattern and the approximate pitch pattern, determine a connection point between the pitch pattern including the fine structure and the original utterance pitch pattern based on the approximate pitch pattern, and A speech synthesis method characterized in that a connection pitch pattern is generated by connecting a pitch pattern including the fine structure and the original utterance pitch pattern at a connection point.
(Appendix 18)
A process of generating an approximate pitch pattern that approximates the pitch pattern by inputting a pitch pattern including a fine structure;
By inputting the original utterance pitch pattern based on the registered pitch pattern and the approximate pitch pattern, a connection point between the pitch pattern including the fine structure and the original utterance pitch pattern is determined based on the approximate pitch pattern. A speech synthesis program that causes a computer to execute a process of generating a connection pitch pattern by connecting the pitch pattern including the fine structure and the original utterance pitch pattern at the connection point.

1, 2, 10, 20, 100, 300 Speech synthesis apparatus 11 Approximate pitch pattern output means 12 Pitch pattern connection means 13 Language analysis means 14 Pitch pattern generation means 15 Phoneme duration generation means 16 Original utterance pitch pattern selection means 17 Unit waveform Selection means 18 Speech waveform generation means 21 Pitch pattern generation model storage means 22 Phoneme duration generation model storage means 23 Original utterance pitch pattern storage means 24 Unit waveform storage means 111 Approximate pitch pattern generation section 112 Approximate pitch pattern selection section 113 Approximation pitch pattern selection section Storage unit 120 Pitch pattern connection unit 130 Language analysis unit 140 Pitch pattern generation unit 160 Original utterance pitch pattern selection unit 150 Phoneme duration generation unit 170 Unit waveform selection unit 180 Speech waveform generation unit 211 Pitch pattern generation model storage unit 212 Phoneme continuation Long generation model storage unit 213 Original utterance pitch pattern storage unit 214 Unit waveform storage unit 310 Approximate pitch pattern formation device 311 Speech data storage unit 312 Pitch pattern extraction unit 313 Approximate pitch pattern formation unit

Claims (10)

Approximate pitch pattern output means for receiving a pitch pattern including a fine structure and generating an approximate pitch pattern that approximately represents the pitch pattern;
Input an original utterance pitch pattern based on a registered pitch pattern and the approximate pitch pattern, and determine a connection point between the pitch pattern including the fine structure and the original utterance pitch pattern based on the approximate pitch pattern, A speech synthesizer comprising: pitch pattern connection means for generating a connection pitch pattern by connecting a pitch pattern including the fine structure and the original utterance pitch pattern at the connection point. The pitch pattern connecting means is:
The speech synthesizer according to claim 1, wherein a connection point is a point where a connection boundary line between the original utterance pitch pattern and the pitch pattern intersects with the approximate pitch pattern. The approximate pitch pattern output means includes:
An approximate pitch pattern storage means for storing the approximate pitch pattern;
The speech synthesis apparatus according to claim 1, further comprising: an approximate pitch pattern selection unit that selects the approximate pitch pattern stored in the approximate pitch pattern storage unit based on the pitch pattern. The approximate pitch pattern selection means includes:
4. The speech synthesizer according to claim 3, wherein the approximate pitch pattern having a distance close to a pitch pattern including the fine structure is selected in a feature amount space having at least a pitch pattern as a feature amount. Language analysis means for generating context information including phoneme string information corresponding to the input text and information associated with each phoneme;
Pitch pattern generation means for inputting the context information, generating a pitch pattern including the fine structure based on a pitch pattern generation model stored in advance, and outputting the generated pitch pattern to the approximate pitch pattern output means; The speech synthesizer according to any one of claims 1 to 4, further comprising: The approximate pitch pattern output means includes:
A temporary pitch pattern is generated by replacing unvoiced phonemes constituting the pitch pattern with voiced phonemes close to the characteristics of the unvoiced phoneme, and the approximate pitch pattern is generated after interpolating the pitch pattern of the unvoiced sound section with the temporary pitch pattern The speech synthesizer according to claim 5. The approximate pitch pattern output means includes:
7. The speech synthesizer according to claim 1, wherein the approximate pitch pattern is generated by adding a feature related to prosody for each language to a constraint condition.   Searching the original utterance pitch pattern based on pre-registered syllable string information, selecting original utterance pitch pattern data including syllable string information that matches at least a part of the syllable string information, and selecting the selected original utterance pitch 8. An original utterance pitch pattern selection means for searching a part of the pattern data where the corresponding / forward / backward syllable string and accent position coincide with each other as an original utterance pitch pattern. The speech synthesizer described in 1. An input of a pitch pattern including a fine structure to generate an approximate pitch pattern that approximately represents the pitch pattern,
Input the original utterance pitch pattern based on the registered pitch pattern and the approximate pitch pattern, determine a connection point between the pitch pattern including the fine structure and the original utterance pitch pattern based on the approximate pitch pattern, and A speech synthesis method characterized in that a connection pitch pattern is generated by connecting a pitch pattern including the fine structure and the original utterance pitch pattern at a connection point. A process of generating an approximate pitch pattern that approximates the pitch pattern by inputting a pitch pattern including a fine structure;
Input an original utterance pitch pattern based on a registered pitch pattern and the approximate pitch pattern, and determine a connection point between the pitch pattern including the fine structure and the original utterance pitch pattern based on the approximate pitch pattern, A speech synthesis program that causes a computer to execute a process of generating a connection pitch pattern by connecting a pitch pattern including the fine structure and the original utterance pitch pattern at the connection point.

Images