JP2011242465A - Speech element database creating device, alternative speech model creating device, speech synthesizer, speech element database creating method, alternative speech model creating method, program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To create a complete speech element database by creating an alternative speech model even when all the speech models can not be created from a speech waveform database.SOLUTION: A speech element database creating device comprises: a phoneme-diphone section converting section 1200 receiving the input of speech waveform data and outputting a diphone section and a diphone label; a speech parameter series converting section 1300 receiving the input of the speech waveform data and outputting a speech parameter series; a speech model creating section 1400 receiving the input of the speech parameter series and outputting a speech model; a missing diphone label output section 1600 receiving the input of the diphone label and a defined diphone label list 1500 and outputting a missing diphone label; a half-phone creating section 1800 receiving the input of the speech model and the diphone label and outputting a half-phone; and an alternative speech model creating section 1900 receiving the input of the half-phone and the missing diphone label and outputting an alternative speech model.

Description

  The present invention relates to a speech segment database creation device that creates a speech segment database that can be used in a text speech synthesis technique from a speech waveform database that records speech uttered by a human.

  2. Description of the Related Art There is known a technique for generating synthesized speech by connecting speech synthesis units using speech units shorter than words such as phonemes, syllables, or phonological chains as speech synthesis units. The speech synthesis unit is collected from the speech waveform database that records the speech that the speaker has read out the sentence list. The following are known as speech synthesis units (see Non-Patent Documents 1 and 2).

  Phoneme units: Vowels and consonants are used as speech synthesis units. The total number of speech synthesis units that must be collected is small. However, since the sound articulation combination information is not included, the sound quality of the synthesized speech is low.

  Syllable (CV) unit: A combination of consonant and vowel is used as a speech synthesis unit. It is suitable for Japanese syllables, and preserves articulation combinations when changing from consonants to vowels. The total number of speech synthesis units that must be collected is small. However, the tone quality of the synthesized speech is still low because the articulation combination information around the syllable (CV) unit is not included.

  Diphone unit: A combination of two phonemes such as CV, VC, and VV is used as a speech synthesis unit. Connection between speech synthesis units is performed at the center of phonemes. Since all the articulation combination information appearing in Japanese is included, the total number of necessary speech synthesis units is larger than the syllable (CV) unit, but the synthesized speech has high quality.

  A method of using a phoneme unit and a diphone unit in combination (see Non-Patent Document 3): When connecting vowels, a diphone unit is used as a speech synthesis unit, but a phoneme unit is used for other connections. When diphone units are used, the speech synthesis units are connected at the center of the phoneme. When phoneme units are used, the speech synthesis units are connected at the phoneme boundary. When vowels are connected, a smoother speech synthesis result is obtained when connecting at the phoneme center than when connecting at the phoneme boundary. Therefore, according to this method, the synthesized speech units can be connected at a connection point where the phoneme boundary part and the phoneme center part can be connected more smoothly. More natural synthesis is possible than when only phoneme units are used as speech synthesis units, and the synthesized speech has high quality. However, there is a problem in that the amount of segment search processing during speech synthesis is large.

  In addition to this, a variable length synthesis unit for selecting an appropriate speech synthesis unit from a speech phone with environment (triphone) or a speech corpus for each synthesis has been proposed.

Abe Yoshinobu, “Trends in Corpus-Based Speech Synthesis Technology [II]”, IEICE Journal, The Institute of Electronics, Information and Communication Engineers, February 2004, Vol. 87, No. 2, pp129-134. Koike Tsunehiko, "Voice Information Engineering", NTT Advanced Technology, 1987, pp66-67. Toda Tomoki, Kawai Tsune, Tsuzaki Minoru, Shikahiro Shikano, “Fragment Selection Based on Phoneme Units and Diphone Units in Segment-Connected Japanese Text-to-Speech Synthesis”, IEICE Transactions, The Institute of Electronics, Information and Communication Engineers, Heisei Heisei December 2014, D-II, vol.J85-D-II, no.12, pp 1760-1770.

  If the diphone unit is used as the speech synthesis unit, the total number of required speech synthesis units is not so much compared with the case where the syllable (CV) unit is used as the speech synthesis unit, and high-quality synthesized speech can be obtained. Can do. However, when collecting speech synthesis units necessary for speech synthesis in units of diphones from a speech waveform database that records the speech that the speaker has read out the sentence list, the scale of the sentence list is not sufficient and is necessary from the speech waveform database. It may not be possible to collect all speech synthesis units. In this case, it is not possible to create a speech segment database that holds all necessary speech synthesis units as speech models. Depending on this incomplete speech segment database, synthesized speech can be created without missing speech. Can not do it.

  According to the above-described method using both phoneme units and diphone units, the total number of speech synthesis units can be reduced because the speech synthesis unit necessary for speech synthesis by diphone units is limited to the connection of vowels. Therefore, even if the sentence list is small, it is easy to obtain all necessary speech synthesis units. However, since the range of segment search at the time of speech synthesis is expanded, the segment search processing amount is increased.

  According to the present invention, there is provided a speech unit database creation device capable of generating a substitute speech model and generating a complete speech unit database when all necessary speech models cannot be generated from the speech waveform database. The The speech segment database creation device of the present invention includes a phoneme-diphone section converter, a speech parameter series converter, a speech model generator, a missing diphone label output unit, a half phone generator, and an alternative speech model generator. A part.

  The phoneme-diphone section conversion unit receives speech waveform data to which a phoneme label is assigned for each phoneme section length, and inputs the latter half of the preceding phoneme section and the subsequent phonemes of any two adjacent phoneme sections. The first half of the section is connected to form a diphone section, the phoneme label of the previous phoneme section and the phoneme label of the subsequent phoneme section are connected to form a diphone label, and the diphone section and the diphone label are Output in association.

  The speech parameter series conversion unit receives the speech waveform data, the diphone label, and the diphone section, and converts the speech waveform data into speech parameters for each diphone section and for each fixed frame length. Each voice parameter column is set as a voice parameter series, and the voice parameter series is output in association with the diphone section.

  The voice model generation unit receives the voice parameter series, the diphone label, and the diphone section, and selects one or more voice parameters from the voice parameter series associated with the diphone section for each diphone section. As a representative pattern, a voice model composed of the representative pattern is generated, and a diphone label associated with the diphone section and the voice model are output in association with each other.

  The missing diphone label output unit receives the diphone label and a defined diphone label list, and is present in the defined diphone label list but is not input as the diphone label. Is output as a missing diphone label.

  The half phone generation unit receives the voice model and the diphone label as input, divides the voice model into a first half and a second half to make both half phones, and a die associated with the divided voice model. The first half of the phone label is set as a half phone label and output in association with the half phone consisting of the first half of the divided voice model, and the second half of the diphone label associated with the divided voice model is half phone. As a label, it is output in association with a half phone comprising the latter half of the divided speech model.

  The alternative speech model generation unit inputs the half phone, the half phone label, and the missing diphone label, and associates the half phone label with the same or similar half phone label as the first half of any missing diphone label. The half phone and the half phone associated with the same or similar half phone label as the latter half of the missing diphone label are connected and output as an alternative voice model.

  As a result, half phone is connected to generate an alternative speech model, so that even if all required speech models cannot be generated from the speech waveform database, a complete speech unit database can be generated, Using the speech segment database, synthesized speech with no missing speech can be created. In addition, since an appropriate alternative speech model is generated in advance when the speech unit database is created, an increase in the amount of segment search processing can be avoided.

  The alternative speech model generation unit in the speech unit database creation device of the present invention includes a half phone associated with the same half phone label as the first half of any missing diphone label, and the latter half of the missing diphone label. A combination that minimizes the F0 gap between the half phone in the first half and the half phone in the second half when there is a plurality of half phones associated with the same half phone label. You may select as a connection object of a missing diphone label.

  As a result, the F0 change amount at the connection portion of the alternative speech model is reduced, and the synthesized speech using the alternative speech model becomes high quality.

  The alternative speech model generation unit in the speech unit database creation device of the present invention includes a half phone associated with the same half phone label as the first half of any missing diphone label, and the latter half of the missing diphone label. When there are a plurality of halfphones associated with the same halfphone label, the first halfphone and the second halfphone are separated by a predefined F0 range. The combination of the first half half phone and the second half half phone classified into two or more categories and the same or close categories may be selected as the connection target of the missing diphone label. .

  As a result, the F0 change amount at the connection portion of the alternative speech model is reduced, and the synthesized speech using the alternative speech model becomes high quality. In addition, the speech unit database created in this way has a high possibility of having a plurality of alternative speech models having the same average F0 (the same number as the number of categories in the maximum case) for the same missing diphone label. Therefore, when speech synthesis processing is performed using the speech unit database, it is highly possible that a speech model having an F0 value approximated to F0 generated by prosody generation can be selected from the speech unit database. As the amount of change is reduced, the synthesized speech is of higher quality. In addition, since the half-phones classified into the same or close categories are combined into an alternative voice model, the combination of all the half-phones is stored in the voice unit database as compared with the case where all the half-phone combinations are stored as alternative voice models. The total number of alternative speech models to be performed can be significantly reduced, and an increase in segment search time and an increase in database size can be avoided.

  The alternative speech model generation unit in the speech unit database creation device of the present invention is a half phone associated with the same half phone label as the first half of any missing diphone label, or the latter half of any missing diphone label. If there is no halfphone associated with the same halfphone label as the part, the halfphone with the smallest interphoneme distance matrix is connected to the missing diphone label instead of the nonexistent halfphone. You may select as an object.

  As a result, an alternative speech model can be generated even for a phoneme that does not exist in the speech waveform database. Therefore, a speech unit database having all necessary speech models can be generated, and the speech unit Using the database, synthesized speech without missing speech can be created.

  Also, if a speech model to which a diphone label is assigned in advance is input, the phoneme-diphone section converter, the speech parameter series converter, and the speech model generator can be omitted. Therefore, in the present invention, an alternative speech model creation device including a missing diphone label output unit, a half phone generation unit, and an alternative speech model generation unit is provided. The function of each part in this alternative speech model creation apparatus is the same as the function of each part having the same name in the speech segment database creation apparatus.

  As a result, half phone is connected to generate an alternative speech model, so that even if all required speech models cannot be generated from the speech waveform database, a complete speech unit database can be generated, Using the speech segment database, synthesized speech with no missing speech can be created. In addition, since an appropriate alternative speech model is generated in advance when the speech unit database is created, an increase in the amount of segment search processing can be avoided.

  The present invention also provides a speech synthesizer that synthesizes speech from text using the speech segment database created by the speech segment database creation device or the alternative speech model creation device. The speech synthesizer includes a text analysis unit, a prosody generation unit, a speech model selection unit, and a speech synthesis unit.

  The text analysis unit takes text as input and outputs readings, accents, and phoneme sequences. The prosody generation unit receives readings and accents, and outputs F0, power, and phoneme length. The speech model selection unit receives F0, power, and phoneme series as input, and selects and outputs a speech model from the speech unit database. The speech synthesizer receives the speech model, F0, power, and phoneme length, and outputs synthesized speech.

  As a result, half phone is connected to generate an alternative speech model and a speech unit database is created. Therefore, even if all necessary speech models cannot be generated from the speech waveform database, a complete speech unit database is created. Can be generated, and using the speech segment database, synthesized speech with no missing speech can be created. In addition, since an appropriate alternative speech model is generated in advance when the speech unit database is created, an increase in the amount of segment search processing can be avoided.

  As described above, in the present invention, even when all necessary speech models cannot be generated from the speech waveform database, a complete speech segment database can be generated by generating an alternative speech model, Using the speech segment database, synthesized speech with no missing speech can be created. In addition, since an appropriate alternative speech model is generated in advance when the speech unit database is created, an increase in the amount of segment search processing can be avoided.

The block diagram which shows the structure of an audio | voice element database production apparatus and an alternative audio | voice model production apparatus. The flowchart which shows operation | movement of a speech segment database creation apparatus and an alternative speech model creation apparatus. The figure which shows the example of an output of the phoneme segmentation part of a speech segment database preparation apparatus, and a phoneme-diphone area conversion part. The figure which shows the example of an output of the speech parameter series conversion part of a speech unit database production apparatus, and a speech model production | generation part. The block diagram which shows the structure of the alternative speech model production | generation part of a speech unit database preparation apparatus. The figure which shows the example of an output of the half phone selection means of the alternative speech model production | generation part of a speech unit database production apparatus. The block diagram which shows the structure of a speech synthesizer. The flowchart which shows operation | movement of a speech synthesizer. The figure which shows the output example of the speech synthesis part of a speech synthesizer. The table | surface which shows the example of the speech model memorize | stored in a speech unit database, and an alternative speech model.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the structure part which has the same function, and duplication description is abbreviate | omitted.

  The speech segment database creation apparatus and speech segment database creation method of the present invention will be described with reference to FIGS. The speech unit database creation apparatus 1000 shown in FIG. 1 includes a phoneme segmentation unit 1100, a phoneme-diphone section conversion unit 1200, a speech parameter series conversion unit 1300, a speech model generation unit 1400, and a defined diphone label list. 1500, a missing diphone label output unit 1600, a half phone generation unit 1800, and an alternative speech model generation unit 1900. The phoneme segmentation unit 1100 shown in FIG. 3 includes a phoneme segment dividing unit 1110 and a phoneme label providing unit 1120. The phoneme-diphone section conversion unit 1200 includes a diphone section dividing unit 1210 and a diphone label providing unit 1220. The alternative speech model generation unit 1900 illustrated in FIG. 5 includes a half phone placement unit 1910, a missing diphone label list 1920, a decision tree determination unit 1930, an interphoneme distance matrix table 1940, a half phone selection unit 1950, Half phone connection means 1960.

  2 and 3, in phoneme segmentation section 1100, speech waveform data 1111 in speech waveform database 91 is input, and phoneme segment dividing means 1110 divides speech waveform data 1111 into phoneme segments 1112 to generate speech waveforms. Data 1111 and phoneme section 1112 are output in association with each other. With the phonetic waveform data 1111 and the phoneme segment 1112 as inputs, the phoneme label assigning means 1120 assigns the phoneme label 1121 for each phoneme segment 1112, and associates the speech waveform data 1111, the phoneme segment 1112, and the phoneme label 1121. Are output (S1100). For this processing, a conventional method known as a method of automatically performing segmentation (Reference Patent Document 1: Japanese Patent Application Laid-Open No. 2004-77901) can be used.

  In the phoneme-diphone section conversion unit 1200, the speech waveform data 1111 provided with the phoneme label 1121 for each phoneme section 1112 is input, and the diphone section dividing unit 1210 is the preceding phoneme section of any two adjacent phoneme sections. Are connected to the first half of the subsequent phoneme section to form a diphone section 1212, and the diphone section 1212 and the phoneme label 1121 are output in association with the speech waveform data 1111. The diphone section 1212, the phoneme label 1121, and the speech waveform data 1111 are input, and the diphone label attaching unit 1220 connects the phoneme label of the previous phoneme section and the phoneme label of the subsequent phoneme section to connect the diphone section 1212 and the phoneme label. The phone label 1221 is used, and the diphone section 1212 and the diphone label 1221 are output in association with each other (S1200).

  FIG. 3 illustrates the operations of the phoneme segmentation unit 1100 and the phoneme-diphone section conversion unit 1200 when the speech waveform data “ONSe” is input to the phoneme segmentation unit 1100. “Sil” in FIG. 3 means a phoneme label in a silent section. The silent section given “Sil” is not divided into the first half and the second half, and the entire silent section is connected to the first half of the subsequent phoneme section or the second half of the previous phoneme section to generate the diphone section 1212. It shall be. In FIG. 3, “Sil: O”, “O: N”, “N: S”, “S: e”, and “e: Sil” are all diphone labels 1221.

  Referring to FIGS. 2 and 4, speech parameter series conversion section 1300 receives speech waveform data 1111, diphone label 1221, and diphone section 1212 as input, and uses speech waveform data 1111 for each diphone section 1212 at a fixed frame. Every length (for example, 5 ms) is converted into speech parameter series 1301-1 to N composed of N speech parameters, and the speech parameter series 1301-1 to 130-N are output in association with the diphone section 1212 (S1300). As a speech parameter expression method, for example, a cepstrum (see Non-Patent Document 2) or the like can be used.

  The voice model generation unit 1400 receives the voice parameter series 1301-1 to 130-1 to N, the diphone label 1221 and the diphone section 1212 as input, and for each diphone section 1212, the voice parameter series associated with the diphone section 1212 The voice parameters are selected one by one from the states 1 to 3 and set as representative patterns 1401-1 to 3. A three-state voice model 1402 composed of these three representative patterns is generated, and the diphone label 1221 associated with the diphone section 1212 and the voice model 1402 are output in association with each other (S1400). In this embodiment, the number of states is set to 3, but the number of states may be other values. For example, a diphone section including a long phoneme length such as a long vowel is set to 5 states, and the number of representative patterns to be selected. It is good also as producing | generating the audio | voice model which consists of 5 states with 5 being. The length of each state may be obtained by equally dividing the diphone section, or by performing non-uniform division such as densely dividing the central portion where the voice parameter rapidly changes. In addition, as a method for selecting a representative pattern in each state, a method for selecting a speech parameter in a frame in a temporally central portion of each state, a method using an average value of speech parameters of all frames in each state, this average value There is a method of selecting the voice parameter closest to the number from each state.

  In FIG. 4, voice waveform data “ONSe” divided into diphone sections, corresponding diphone labels “Sil: O”, “O: N”, “N: S”, “S: e”, “e: Sil ”Is input, for example, the operation of the speech parameter sequence conversion unit 1300 and the speech model generation unit 1400 in the speech parameter sequence 1301-1 to 130-N associated with the diphone label“ N: S ”is illustrated.

  Referring to FIGS. 1 and 2, missing diphone label output unit 1600 receives diphone label 1221 and defined diphone label list 1500 as input, and exists in defined diphone label list 1500. A diphone label that is not input as the label 1221 is output as a missing diphone label (S1600). The defined diphone label list 1500 is a list in which diphone labels of all diphones necessary for speech synthesis are generated in advance as a list.

  Note that the missing diphone label output unit 1600 determines that the voice waveform database 91 is insufficient in scale and does not include all the required diphones in the voice waveform database 91. It exists for the purpose of specifying and outputting as. Therefore, the missing diphone label output unit 1600 is missing by comparing all the diphone labels associated with all the audio waveform data included in the audio waveform database 91 with the defined diphone label list 1500. A diphone is identified and output as a missing diphone label. Attention should be paid to this point.

  Here, when there is a missing diphone label, the operation of the half phone generation unit 1800 (S1800) and the operation of the alternative speech model generation unit 1900 (S1900) are executed. S1800 and S1900 are not performed (S1700). Hereinafter, each operation of the half phone generation unit 1800 and the alternative speech model generation unit 1900 when a missing diphone label exists will be described.

  The half phone generation unit 1800 receives the voice model 1402 and the diphone label 1221 as input, divides the voice model 1402 into the first half and the second half and sets both as half phones, and a die associated with the divided voice model. The first half of the phone label is set as a half phone label and output in association with the half phone consisting of the first half of the divided voice model, and the second half of the diphone label associated with the divided voice model is half phone. A label is output in association with a half phone comprising the latter half of the divided speech model (S1800). For example, when the number of states of the voice model 1402 is L, the half phone in the first half holds the representative pattern from the first state to the (L / 2) state (rounded up after the decimal point), deletes the remaining states, The half phone in the latter half is generated by holding the representative patterns from the (L / 2) +1 state to the Lth state (decimal point down) and deleting the remaining states. When L is an odd number, the representative pattern in a state located just in the middle of the number of states L of the speech model 1402 is held in both the first half phone and the second half phone. Therefore, when the number of states of the voice model 1402 is 3, the first half of the half phone holds the representative patterns of the first and second states of the voice model, and the second half of the half phone holds the second state of the voice model. Hold the representative pattern in the third state. The representative pattern in the state 2 that is located in the middle of the three states is held in both the first half phone and the second half phone. Note that the half phone generating unit 1800 generates a half phone by dividing a speech model generated from all speech waveform data included in the speech waveform database 91. Therefore, it should be noted that half-phones corresponding to all voice waveform data included in the voice waveform database 91 are generated.

  2 and 5, the alternative speech model generation unit 1900 receives the half phone, the half phone label, and the missing diphone label as input, and is the same as the first half of any missing diphone label or The half phone associated with the similar half phone label and the half phone associated with the same or similar half phone label as the second half of the missing diphone label are connected and output as an alternative voice model (S1900).

  Specifically, in the alternative speech model generation unit 1900, the half phone placement unit 1910 receives the half phone and the half phone label as input, and places the input half phone in a predetermined decision tree. The decision tree is generated by generating a phoneme environment before and after phonemes as a context, and one for each phoneme. The aforementioned half phone is arranged at a leaf node of the decision tree prepared for the same phoneme as the phoneme indicated in the half phone label. The half phone arrangement unit 1910 arranges all the half phones generated by the half phone generation unit 1800 at the leaf nodes of the decision tree prepared for each phoneme. That is, all the half phones generated based on all the speech waveform data included in the speech waveform database 91 are arranged in the leaf nodes of the decision tree. Attention should be paid to this point.

  The missing diphone label list 1920 receives the missing diphone label and stores the inputted missing diphone label in itself. The decision tree decision means 1930 receives the missing diphone label list 1920 and the decision tree in which halfphones are arranged as inputs, and decides decision trees to be referred to for the first half and the second half of all missing diphone labels. Each of them is output as a first half half phone decision tree and a second half half phone decision tree in association with a missing diphone label. Here, when no half phone is arranged in the leaf node of the decision tree to be referred to, the interphoneme distance matrix 1940 is referred to and the distance between the phonemes of the decision tree to be referred to and the phoneme distance is the shortest. Determine the phoneme decision tree as an alternative decision tree. Here, the inter-phoneme distance matrix is a matrix table defined in advance in consideration of discrimination features such as a tuning point and a tuning method.

  By using a decision tree as described above, there is an advantage that narrowing down in consideration of context information becomes easy. For example, if a set of halfphones is configured in units of phonemes instead of using a decision tree, halfphones whose phoneme environments before and after do not match too much are selected as connection targets. For this reason, clustering is performed in consideration of the phoneme environment before and after using a decision tree, and candidates for connection are narrowed down. Since the decision tree is created in units of phonemes, the same number and number of decision trees as the phonemes included in the speech database 91 are created. Phoneme context information (such as matching phoneme environments before and after) is used for the decision tree branch condition question. For example, “Does it correspond to the first half of the diphone?”, “Is the subsequent phoneme a vowel?”, “Is the subsequent phoneme / A /”, “Is the preceding phoneme a burst consonant”, “Is the preceding phoneme / P /”, etc. It is configured to gradually narrow down from rough classification to detailed classification.

  Referring to FIG. 6, half phone selection means 1950 receives a missing diphone label list 1920, a first half half phone decision tree 1951, and a second half half phone decision tree 1952, and inputs the first half half for each missing diphone label. A half phone included in a leaf node whose context before and after the semiphoneme is matched is determined as a connection target for each one from the phone decision tree 1951 and the latter half phone decision tree 1952, and is output in association with the missing diphone label.

  Here, when there are a plurality of halfphones in at least one of the leaf nodes to be referred to in the first halfphone decision tree 1951 and the second halfphone decision tree 1952, and the phonemes of both halfphone labels are voiced sounds To select the combination that minimizes the F0 gap between the half phone existing in the first half-phone decision tree and the half phone existing in the second half-phone decision tree as the half phone to be connected to the missing diphone label. It is good. The F0 gap is obtained from the difference between the F0 average values of the half phones. Here, the F0 average value of the half phone is a value obtained by averaging the F0 values of the representative patterns in each state of the first half phone, and an average value of the F0 values of the representative patterns of each state of the second half phone. Use.

  Further, the halfphones existing in the leaf node to be referred to are classified into two or more F0 categories 1953 separated by a predefined F0 range, and combinations consisting of halfphones classified into the same or adjacent categories are combined. You may select as a connection object of a missing diphone label. Here, the F0 category is a classification of the F0 values by the quantization width D. The quantization width D of the F0 value is, for example, 50 Hz, and can be classified into 6 categories consisting of less than 100 Hz, 100 Hz to less than 150 Hz, 150 Hz to less than 200 Hz, 200 Hz to less than 250 Hz, 250 Hz to less than 300 Hz, and 300 Hz or more. As another method, the quantization width D may be set in the logarithmic F0 region. The quantization width D is determined so that appropriate classification can be performed according to the F0 deformation tolerance of the signal processing method in the speech synthesis process. When there are a plurality of half-phones included in the same F0 category, the F0 average values are compared, and the combination with the smallest difference in the F0 average value is selected as the connection target. When either half phone is not included in the same F0 category, the half phones of the F0 category adjacent to each other are selected as the connection targets. In this way, it is possible to obtain a combination of at least one half phone and at most equal to the number of F0 categories. In addition, the selection of the connection target may be based on a difference in acoustic parameters (for example, a distance between spectra, for example) instead of the average F0.

  Note that the halfphones to be connected are determined based on the average F0 only when all the halfphones to be connected are voiced sounds. If either or both of the combined segments are unvoiced sounds, the centroid (one voice model closest to the average (center of gravity)) of all the halfphone sets included in the leaf node is used as a representative pattern. Or it is good also as selecting the combination with which both acoustic parameters (for example, distance between spectra, etc.) are the nearest about all the half phones contained in a leaf node. An unvoiced sound does not have F0 information because it is a sound without periodic vibration. Therefore, since it is not necessary to consider the F0 gap, the connection method differs between connection of voiced sounds and connection including unvoiced sounds.

  Returning to FIG. 5, the halfphone connection unit 1960 receives the halfphone to be connected and the missing diphone label list 1920 as input, and connects the halfphone to be connected for each missing diphone label to substitute voice. A model is output in association with the missing diphone label.

  When the number of states of the alternative speech model after connection is an odd number, the half phone is connected to the representative pattern of the second state from the first state to the last half phone of the first half and the half phone of the second half. The connection is made using the representative patterns from the second state to the last state counted from the beginning. For the last state of the first half phone and the first half phone state that are located in the middle after the connection, values obtained by weighted addition using the internal values in both states are used. . For example, a sigmoid function can be used as the internal ratio.

  For example, when the number of representative patterns of the speech model is 3, the representative pattern of the first state of the first half phone is used as the representative pattern of the first state of the alternative speech model after connection, The representative pattern in the last state is used as the representative pattern in the third state of the alternative speech model after connection. For the second state of the alternative speech model after concatenation, a value obtained by weighted addition using the internal values of the last state of the first half phone and the first state of the second half phone is used. . When the number of states of the alternative speech model is an even number, connection may be performed using the representative patterns of the states of the first half phone and the states of the second half phone.

  Referring to FIGS. 1 and 10, the speech unit database 92 receives a speech model and an alternative speech model, and stores the input speech model and the alternative speech model (S92). FIG. 10 is a table illustrating a speech model and an alternative speech model stored in the speech segment database 92. The speech segment database 92 stores an average frequency F0 (Hz), a slope (Hz / ms) of the average frequency F0, power (dB), and speech parameters for each diphone label.

  In the case where a speech model to which a diphone label is assigned in advance is used as an input, only a missing diphone label list 1500, a missing diphone label output unit 1600, a half phone generation unit 1800, and an alternative speech model generation unit 1900 are provided. The alternative voice model creating apparatus 10000 may create a substitute voice model.

  The function of each part in the alternative speech model creation apparatus 10000 is the same as the function of each part having the same name in the speech segment database creation apparatus 1000. The voice model with the diphone label input to the alternative voice model creation device 10000 is created in advance by another device using all the voice waveform data included in the voice waveform database prepared in advance. And Note that the operation of the missing diphone label output unit 1600 in the alternative speech model creation device 10000 is the same as that of the missing diphone label output unit 1600 in the speech segment database creation device 1000. All diphone labels generated from all included audio waveform data are input, and a missing diphone is identified and output as a missing diphone label by comparison with a predefined diphone label list 1500. Similarly, the half phone generation unit 1800 of the alternative speech model creation device 10000 creates a half phone using all speech waveform data included in the speech waveform database, and the half phone placement unit 1910 of the alternative speech model creation device 10000 All the half phones generated by the half phone generation unit 1800 are arranged in the leaf nodes of the decision tree prepared for each phoneme. Attention should be paid to this point.

  According to the speech unit database creation apparatus 1000 of the present embodiment, half phone is connected to generate an alternative speech model. Therefore, even when all necessary speech models cannot be generated from the speech waveform database, the speech unit database creation device 1000 is completely A speech segment database can be generated, and a synthesized speech free of speech loss can be created using the speech segment database. In addition, since an appropriate alternative speech model is generated in advance when the speech unit database is created, an increase in the amount of segment search processing can be avoided.

  In addition, when the combination of half phones with the smallest F0 gap is selected as the half phone to be connected to the missing diphone label, the amount of change in F0 at the connection portion of the alternative voice model is reduced, and the alternative voice model is selected. The synthesized speech used is of high quality.

  In addition, when two or more F0 categories separated by a predefined F0 range are classified, and a combination of halfphones classified into the same or adjacent categories is selected as a connection target of missing diphone labels. The F0 change amount at the connection portion of the alternative speech model is reduced, and the synthesized speech using the alternative speech model becomes high quality. In addition, the speech unit database created in this way has a high possibility of having a plurality of alternative speech models having the same average F0 (the same number as the number of categories in the maximum case) for the same missing diphone label. Therefore, when speech synthesis processing is performed using the speech unit database, it is highly possible that a speech model having an F0 value approximated to F0 generated by prosody generation can be selected from the speech unit database. As the amount of change is reduced, the synthesized speech is of higher quality. In addition, since the half-phones classified into the same or close categories are combined into an alternative voice model, the combination of all the half-phones is stored in the voice unit database as compared with the case where all the half-phone combinations are stored as alternative voice models. The total number of alternative speech models to be performed can be significantly reduced, and an increase in segment search time and an increase in database size can be avoided. At the same time, since the aforementioned F0 category is set from an appropriate quantization width according to the F0 deformation tolerance, an alternative speech model that is optimal for speech synthesis is still stored in the speech unit database. High quality.

  In addition, when there is no halfphone in any phoneme, the halfphone having the smallest predefined phoneme distance matrix is selected as the connection target of the missing diphone label instead of the non-existing halfphone. If so, since an alternative speech model can be generated even for phonemes that are not present in the speech waveform database, it is possible to generate a speech segment database that holds all necessary speech models. Using the speech segment database, synthesized speech with no missing speech can be created.

  The speech synthesis apparatus and speech synthesis method of the present invention will be described with reference to FIGS. A speech synthesizer 7000 shown in FIG. 7 includes a text analysis unit 7100, a text analysis dictionary 7200, a prosody generation unit 7300, a speech model selection unit 7400, and a speech synthesis unit 7600. The text analysis unit 7100 receives the text, and outputs a reading, accent, and phoneme sequence using the text analysis dictionary 7200 (S7100). The prosody generation unit 7300 receives readings and accents, and outputs F0, power, and phoneme length (S7300). The speech model selection unit 7400 receives F0, power, and phoneme series as input, and selects and outputs a speech model from the speech unit database (S7400). The speech synthesizer 7600 receives the speech model, F0, power, and phoneme length, and outputs synthesized speech (S7600). Specifically, the speech synthesis unit 7600 shown in FIG. 9 includes speech parameter series generation means 7610, speech parameter series interpolation means 7620, and synthesized speech waveform generation means 7630.

  The speech parameter series generation unit 7610 repeatedly connects the representative patterns of the input speech model 1402 according to the input phoneme length. In the example of FIG. 9, each of the speech parameters, which are the three representative patterns of the speech model 1402, is repeatedly duplicated and connected by a length obtained by dividing the phoneme length into three equal parts. The voice parameter duplication / concatenation processing is performed for all the input voice models, and the voice parameter series that has undergone the duplication / concatenation processing for each voice model is all concatenated for each corresponding diphone label order. For example, when the number of states of the speech model is P and the number of frames calculated from the phoneme length is Q, the speech parameter that is the representative pattern of the jth state is the frame number (j−1) × (Q / P) +1. To j × (Q / P) number are repeated and connected.

  The voice parameter series interpolation means 7620 interpolates the voice parameter series so as to make a smooth transition. As an interpolation method here, for example, a method of generating a maximum likelihood parameter sequence from a speech parameter distribution sequence (reference non-patent document 1: Keiichi Tokuda, Takashi Masuko, Takao Kobayashi, Kiyoshi Imai, “Dynamic feature value The speech parameter generation algorithm from the used HMM ”, Journal of the Acoustical Society of Japan, Acoustical Society of Japan, March 1997, Vol. 53, No. 3, pp192-200) can be applied.

  The synthesized speech waveform generating unit 7630 generates a synthesized speech waveform from the speech parameter series. As a synthetic speech waveform generation method here, for example, the STRAIGHT method (reference non-patent document 2: Hideki Kawahara, Ikuyo Masuda-Katsuse and Alain de Cheveigne, “Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instanta- neous-frequency-based F0 extraction: Possible role of a repetitive structure in sounds ”, Speech Communication, 27, 3-4, pp.187-207 (1999)).

  According to the speech synthesizer 7000 of this embodiment, when a half phone is connected to generate an alternative speech model and a speech segment database is created, all necessary speech models cannot be generated from the speech waveform database. In addition, it is possible to generate a complete speech unit database, and it is possible to create a synthesized speech without speech loss using the speech unit database. In addition, since an appropriate alternative speech model is generated in advance when the speech unit database is created, an increase in the amount of segment search processing can be avoided.

  In addition, the various processes described above are not only executed in time series according to the description, but may be executed in parallel or individually according to the processing capability of the apparatus that executes the processes or as necessary. Needless to say, other modifications are possible without departing from the spirit of the present invention.

  Further, when the above-described configuration is realized by a computer, processing contents of functions that each device should have are described by a program. The processing functions are realized on the computer by executing the program on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, the present apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

Claims (9)

A speech unit database creation device for creating a speech unit database from speech waveform data to which a phoneme label is assigned for each phoneme section length,
The speech waveform data is input, and the second half of the preceding phoneme section and the first half of the subsequent phoneme section of any two adjacent phoneme sections are connected to form a diphone section, and the phonemes of the preceding phoneme section are connected. A phoneme-diphone section conversion unit that links the label and the phoneme label of the subsequent phoneme section to form a diphone label, and associates and outputs the diphone section and the diphone label;
The voice waveform data, the diphone label, and the diphone section are input, the voice waveform data is converted into voice parameters for each diphone section and at a fixed frame length, and a voice parameter column for each diphone section is converted into voice. A voice parameter series conversion unit that outputs the voice parameter series in association with the diphone section as a parameter series;
The voice parameter series, the diphone label, and the diphone section are input, and for each diphone section, one or more voice parameters are selected from the voice parameter series associated with the diphone section as a representative pattern, A voice model generation unit that generates a voice model including a representative pattern, outputs a corresponding diphone label associated with the diphone section, and the voice model;
Missing output of the diphone label and the defined diphone label list as input, and the diphone label that is present in the defined diphone label list but not entered as the diphone label is output as a missing diphone label A diphone label output section;
The voice model and the diphone label are input, the voice model is divided into a first half and a second half to make both half phones, and the first half of the diphone label associated with the divided voice model is a half phone. The phone label is output in association with the half phone consisting of the first half of the divided voice model, and the second half of the diphone label associated with the divided voice model is used as the half phone label. A half-phone generating unit that outputs the half-phone corresponding to the second half of the speech model;
The half phone that has the half phone, the half phone label, and the missing diphone label as inputs and is associated with a half phone label that is the same as or similar to the front half of any missing diphone label, and the missing diphone An alternative speech model generation unit that connects a half phone corresponding to the same or similar half phone label to the latter half of the label and outputs as an alternative speech model;
A speech unit database creation apparatus comprising: The speech unit database creation device according to claim 1,
The alternative speech model generation unit
At least one of the half phone corresponding to the same half phone label as the first half of any missing diphone label and the half phone corresponding to the same half phone label as the second half of the missing diphone label When there are a plurality of speech unit databases, a combination that minimizes the F0 gap between the first half phone and the second half phone is selected as a connection target of the missing diphone label. Creation device. The speech unit database creation device according to claim 1,
The alternative speech model generation unit
At least one of the half phone corresponding to the same half phone label as the first half of any missing diphone label and the half phone corresponding to the same half phone label as the second half of the missing diphone label When there are a plurality, the first half phone and the second half phone are classified into two or more categories separated by a predefined F0 range, and are classified into the same or adjacent categories. A speech unit database creation apparatus, wherein a combination of a first half phone and a second half phone is selected as a connection target of the missing diphone label. The speech unit database creation device according to any one of claims 1 to 3,
The alternative speech model generation unit
If there is no half phone associated with the same half phone label as the first half of any missing diphone label, or no half phone associated with the same half phone label as the second half of any missing diphone label, A speech unit database creation apparatus, wherein a half phone having a minimum distance matrix between phonemes is selected as a connection target of the missing diphone label instead of the half phone that does not exist. An alternative speech model creation device for creating an alternative speech model from a speech model assigned a diphone label,
Missing output of the diphone label and the defined diphone label list as input, and the diphone label that is present in the defined diphone label list but not entered as the diphone label is output as a missing diphone label A diphone label output section;
The voice model and the diphone label are input, the voice model is divided into a first half and a second half to make both half phones, and the first half of the diphone label associated with the divided voice model is a half phone. The phone label is output in association with the half phone consisting of the first half of the divided voice model, and the second half of the diphone label associated with the divided voice model is used as the half phone label. A half-phone generating unit that outputs the half-phone corresponding to the second half of the speech model;
The half phone that has the half phone, the half phone label, and the missing diphone label as inputs and is associated with a half phone label that is the same as or similar to the front half of any missing diphone label, and the missing diphone An alternative speech model generation unit that connects a half phone corresponding to the same or similar half phone label to the latter half of the label and outputs as an alternative speech model;
An alternative speech model creation device comprising: A speech synthesizer that synthesizes speech from text,
A text analysis unit that takes text as input and outputs a reading, accent, and phoneme sequence;
A prosody generation unit that inputs reading, accent, and outputs F0, power, and phoneme length;
A speech model selection unit that receives F0, power, and phoneme sequence as input, and selects and outputs a speech model from a speech unit database;
A speech synthesizer that receives a speech model, F0, power, and phoneme length as input, and outputs a synthesized speech;
And the speech unit database is created by the apparatus according to any one of claims 1 to 5. A speech segment database creation method for creating a speech segment database from speech waveform data to which a phoneme label is assigned for each phoneme section length,
The speech waveform data is input, and the second half of the preceding phoneme section and the first half of the subsequent phoneme section of any two adjacent phoneme sections are connected to form a diphone section, and the phonemes of the preceding phoneme section are connected. A phoneme-to-diphone section conversion step of connecting the label and the phoneme label of the subsequent phoneme section to form a diphone label, and associating and outputting the diphone section and the diphone label;
The speech waveform data, the diphone label, and the diphone section are input, and the speech waveform data is converted into speech parameters for each diphone section and for each fixed frame length, and a speech parameter column for each diphone section is converted to speech. A voice parameter series conversion step for outputting the voice parameter series in association with the diphone section as a parameter series;
The voice parameter series, the diphone label, and the diphone section are input, and for each diphone section, one or more voice parameters are selected from the voice parameter series associated with the diphone section as a representative pattern, A speech model generation step of generating a speech model composed of a representative pattern, and outputting a corresponding diphone label associated with the diphone section and the speech model;
Missing output of the diphone label and the defined diphone label list as input, and the diphone label that is present in the defined diphone label list but not entered as the diphone label is output as a missing diphone label A diphone label output step;
The voice model and the diphone label are input, the voice model is divided into a first half and a second half to make both half phones, and the first half of the diphone label associated with the divided voice model is a half phone. The phone label is output in association with the half phone consisting of the first half of the divided voice model, and the second half of the diphone label associated with the divided voice model is used as the half phone label. A half phone generation step of outputting in association with a half phone comprising the second half of the voice model;
The half phone that has the half phone, the half phone label, and the missing diphone label as inputs and is associated with a half phone label that is the same as or similar to the front half of any missing diphone label, and the missing diphone An alternative speech model generation step of connecting a half phone corresponding to the same or similar half phone label to the latter half of the label and outputting as an alternative speech model;
A speech unit database creation method comprising: An alternative speech model creation method for creating an alternative speech segment database from a speech model assigned a diphone label,
Missing output of the diphone label and the defined diphone label list as input, and the diphone label that is present in the defined diphone label list but not entered as the diphone label is output as a missing diphone label A diphone label output step;
The voice model and the diphone label are input, the voice model is divided into a first half and a second half to make both half phones, and the first half of the diphone label associated with the divided voice model is a half phone. The phone label is output in association with the half phone consisting of the first half of the divided voice model, and the second half of the diphone label associated with the divided voice model is used as the half phone label. A half phone generation step of outputting in association with a half phone comprising the second half of the voice model;
The half phone that has the half phone, the half phone label, and the missing diphone label as inputs and is associated with a half phone label that is the same as or similar to the front half of any missing diphone label, and the missing diphone An alternative speech model generation step of connecting a half phone corresponding to the same or similar half phone label to the latter half of the label and outputting as an alternative speech model;
An alternative speech model creation method comprising:   The program for functioning a computer as an apparatus in any one of Claims 1-6.

Images