CN101685633A - Voice synthesizing apparatus and method based on rhythm reference - Google Patents

Abstract

The present invention provides a speech synthesis device and method based on prosodic reference. The speech synthesis device includes: a prosody parameter acquisition unit, which analyzes the recording files obtained by natural persons reading the text to be synthesized aloud, or the prosody obtained by marking the text to be synthesized with prosody parameters according to a predetermined labeling standard Parameter annotation file is analyzed to obtain natural rhythm parameters or approximate natural rhythm parameters; and the sound generation part, which uses the natural rhythm parameters or approximate natural rhythm parameters as a reference, selects the corresponding text from the pre-recorded voice library for the text to be synthesized A speech synthesis unit, and splicing and synthesizing the speech synthesis unit to generate a synthesized speech file corresponding to the text to be synthesized. According to the speech synthesis device and method of the present invention, it is possible to generate highly natural synthesized speech that is full of emotion and whose intonation is very close to natural speech according to user requirements.

Description

Speech synthesis device and method based on prosodic reference

technical field

The present invention relates to a device and method for speech synthesis based on prosodic reference. More specifically, the present invention relates to using the prosodic features obtained from natural speech or prosodic feature annotation files based on specific standards as a reference to synthesize speech with A device and method for synthesizing speech with high naturalness.

Background technique

Speech synthesis (Text-To-Speech, TTS for short) is a technology for converting text to speech, specifically, a technology for converting arbitrary text information into standard, fluent speech. Speech synthesis involves many cutting-edge high-tech technologies such as natural language processing, prosody, speech signal processing, and sound perception. It spans multiple disciplines such as acoustics, linguistics, and digital signal processing. It is a cutting-edge technology in the field of Chinese information processing. .

Speech synthesis technology can be widely used in telecommunications, finance, electric power, postal services, government and other industries. Speech synthesis technology can make it easier for users to send and receive emails, get stock market quotes, understand weather, traffic and road conditions, and in the near future, it will provide more comprehensive and more valuable application services.

The speech synthesis system is used to synthesize speech with high intelligibility and high naturalness.

Generally speaking, the speech synthesis system first selects certain basic synthesis units, such as phonemes in English, half-syllables or toned syllables in Chinese, etc., and then under the guidance of the prediction results of the prosody model (sound length and fundamental frequency, etc.) , search for the globally optimal synthesis unit from the pre-recorded and marked standard speech library, use specific waveform generation technology (such as TD-PSOLA algorithm) to adjust and modify the prosodic characteristics of the selected speech segment, and finally splicing and synthesizing Voice required.

After more than ten years of research, the voice quality synthesized by the current speech synthesis system has reached a practical level, and the intelligibility can meet the actual needs of the application, but the naturalness is not high enough, and it is still not as good as human natural speech. There is a big gap.

Most of the speech synthesis systems at the present stage use splicing and synthesis technology based on large-scale speech databases, that is, using the method of probability and statistics, under the guidance of the predicted prosody parameters, search from the pre-recorded speech database that matches the global best for the input text. An excellent synthesis unit, and then perform splicing and synthesis of waveform adjustment according to the predicted rhythm parameters.

Generally speaking, a speech synthesis system includes the following three modules: a text analysis module, a prosody parameter prediction module, and a backend synthesis module. The functions of the Chinese text analysis module include word segmentation, part-of-speech tagging, phonetic notation, prosodic structure prediction, etc. The prosodic parameter prediction module predicts acoustic parameters such as sound length, fundamental frequency, and energy based on text analysis results. The back-end synthesis module is generally composed of a unit selection sub-module and a waveform generation sub-module. Among them, the unit selection sub-module searches for the globally optimal synthesis for the input text from the speech library through the method of probability and statistics under the guidance of prosody parameters. Unit, the waveform generation sub-module uses a specific waveform generation technology (such as TD-PSOLA algorithm) to adjust and modify the prosodic characteristics of the selected speech segment, and finally stitches and synthesizes the speech that meets the requirements.

Pitch length (phoneme length) is one of the most important prosodic features, which has important implications for the perceived naturalness of synthesized speech. The change of sound length can help people to recognize the phoneme itself, and also help people to determine the division of words and phrases in a continuous speech flow, thereby improving the naturalness and intelligibility of speech.

Fundamental frequency is also one of the most important prosodic features, which is especially important for Chinese because Chinese is a tonal language. The fundamental frequency is of great significance to the perceived naturalness and intelligibility of synthesized speech.

In natural language flow, the duration and fundamental frequency of a phoneme are highly correlated with the context before and after the phoneme. Many contextual factors, such as the type of the phoneme itself, the type of the front and back phonemes, the level of front and back prosodic boundaries, stress or not, etc. all have a restrictive effect on the duration and fundamental frequency of the phoneme. The basic purpose of research on sound length prediction and fundamental frequency prediction is to try to describe the impact of these contextual factors on phoneme duration and fundamental frequency, so as to improve the naturalness of the speech synthesis system.

However, prosodic parameter prediction is constrained by many aspects. In addition to the problems in the model itself, it is also constrained by limited training data and the accuracy of front-end text analysis. Although various prosody parameter prediction techniques try to take into account various phenomena in coarticulation, the prosodic rhythm of synthetic speech still cannot be as cadenced as in natural speech.

The key defect of the above-mentioned speech synthesis system at this stage is that it pays attention to the part and ignores the whole, resulting in the synthesis of sentences that are ineffective and lifeless, thus hindering the current speech synthesis system from being widely used in the audio electronic book market, etc. .

The current audio e-book market uses real people to read aloud. It can be difficult to find a speaker who has a beautiful timbre and can work well with emotive cadences. If you find a professional announcer to complete the recording, the cost will definitely be high.

In a nutshell, the traditional speech synthesis system first analyzes the text (such as word segmentation, part-of-speech tagging, digital symbol processing, phonetic notation, prosodic structure analysis, etc.), and then performs prosodic parameters such as sound length, fundamental frequency, and energy on this basis. etc., and then under the guidance of these prosodic parameters, search for the globally optimal synthesis unit from the pre-recorded speech library, and then perform splicing and synthesis of waveform adjustment. Because there are still many unavoidable problems in text analysis and prosody prediction, the traditional speech synthesis system cannot accurately grasp the content of the text and the prediction of prosody parameters, and cannot well control the connection between various speech synthesis units. Controlling the cadence of the synthesized voice well will eventually lead to the user's unsatisfactory voice. For the wide application such as the production of audio e-books, in order to reduce the cost and obtain synthetic speech files with beautiful timbre and natural rhythm, there is an urgent need for a speech synthesis system that can synthesize synthetic speech with high naturalness .

Some literature on research in this area is listed below and is hereby incorporated by reference as if fully set forth herein.

[1]Meron and Joram, US Patent publication No.6,829,581, July 31, 2001, Method for prosody generation by unit selection from an imitations speech database;

[2]Baraff and David R., US Patent publication No.6,795,807, August 17, 2000, Method and means for creating prosody in speech regeneration for laryngectomees;

[3] Holm, Frode, Hata and Kazue, US Patent publication No.6,260,016, November 25, 1998, Speech synthesis employing prosody templates;

[4] Holm, Frode, Hata and Kazue, US Patent publication No.6,185,533, March 15, 1999, Generation and synthesis of prosody templates;

[5] Shih, C.L., "The Prosodic Domain of Tone Sandhi in Mandarin Chinese", PhD Dissertation, UC San Diego, 1986;

[6] Chu M. and Qian Y., "Locating boundaries for prosodic constituents in unrestricted Mandarin texts", Journal of Computational Linguistics and Chinese Language Processing, 6(1), 61-82, 2001;

[7]E.Moulines and F.Charpentier., "Pitch-synchronous waveformprocessing techniques for text-to-speech synthesis using diphones", SpeechCommunication, 9: 453-467, 1990;

[8] Qing Guo, Nobuyuki Katae, "Statistical Prosody Generation in Mandarin TTS System", OCOCOSDA 2007;

[9] Qing Guo, Nobuyuki Katae, Hao Yu, Hitoshi Iwamida, "DecisionTree based Duration Prediction in Mandarin TTS System", Journal of Chinese Language and Computing;

[10] Guo Qing, Nobuyuki Katae, Yu Hao, Hitoshi Iwamida, "High Quality Prosody Generation in a Text-to-speech System", Journal of Chinese Information Processing, Vol.22 No.2: 110-115, 2008;

[11] Qing Guo, Jie Zhang, Nobuyuki Katae, "Prosodic Word Grouping with Global Probability Estimation Method", Speech Prosody, 2008.

Contents of the invention

In view of the above-mentioned problems existing in the traditional speech synthesis technology, the present invention is proposed. The purpose of the present invention is to provide a speech synthesis system based on prosodic reference, which skillfully overcomes the problems of word segmentation, semantic analysis, and prosodic parameter prediction in text analysis that are difficult to solve in traditional speech synthesis technology, and can obtain prosodic rhythm and Good synthesized voice effect that is very close to natural voice and has beautiful timbre. By adopting the present invention, arbitrary voice files can be generated according to user's requirements, and the degree of intonation and setback is very close to natural voice. By combining the standard sweet-sounding reading and the not-so-good-sounding or non-standard pronunciation but emotional reading, it is possible to create a synthesized voice work with a sweet and cadenced voice by reading aloud by anyone. Therefore, the production cost of the audio electronic book can be greatly reduced, and the audio electronic book with the same voice feature and full of rhythm can be produced in batches.

In order to achieve the above object, according to the first aspect of the present invention, a speech synthesis device for performing speech synthesis based on prosodic reference is provided, which includes: a prosody parameter acquisition unit, which obtains the text to be synthesized by reading the text to be synthesized to a natural person The audio recording file of the text is analyzed, or the prosodic parameter labeling file obtained by labeling the prosodic parameter of the text to be synthesized with the predetermined labeling standard is analyzed to obtain the natural rhythm parameter or the approximate natural rhythm parameter; and the sound generation part, which uses the natural Prosodic parameters or approximate natural prosody parameters are used as a reference, select the corresponding speech synthesis unit from the pre-recorded speech bank for the text to be synthesized, and splicing and synthesizing the speech synthesis unit to generate the synthesized speech corresponding to the text to be synthesized document.

According to a second aspect of the present invention, there is provided the speech synthesis device described in the first aspect, wherein the sound generation unit includes: a speech unit selection unit that takes the natural rhythm parameter or the approximate natural rhythm parameter as a reference, and selects from the Selecting the globally optimal speech synthesis unit for the text to be synthesized from the pre-recorded speech library; and a waveform generation unit, based on the natural rhythm parameters or approximate natural rhythm parameters, splicing and synthesizing the speech synthesis units selected by the speech unit selection unit Waveform adjustment is performed on the synthesized voice file to obtain a synthesized voice file with high naturalness corresponding to the text to be synthesized.

According to a third aspect of the present invention, the speech synthesis device described in the second aspect is provided, wherein the prosody parameter acquisition unit includes: a recording unit, which obtains a recording file of the text to be synthesized by a natural person reading the text to be synthesized ; and a prosody parameter extraction unit, which obtains prosody parameters from the waveform data of the recording file, and the prosody parameters include sound length, audio frequency and energy.

According to a fourth aspect of the present invention, there is provided the speech synthesis device described in the second aspect, wherein the prosodic parameter acquisition unit includes: a prosodic parameter labeling unit, which refers to a knowledge base defining labeling rules for prosodic parameters and uses the predetermined Annotation standard carries out prosodic parameter labeling on the text to be synthesized to obtain a prosodic parameter labeling file; and a prosodic parameter generation unit analyzes the prosodic parameter labeling file to obtain approximate prosodic parameters, the approximate prosodic parameters including sound length, audio frequency and energy.

According to a fifth aspect of the present invention, there is provided the speech synthesis device described in the first aspect, wherein the recording file of the text to be synthesized is obtained by pronunciation of an electronic cartoon.

According to a sixth aspect of the present invention, there is provided a method for speech synthesis based on prosodic reference, which includes the following steps: a prosodic parameter acquisition step, which is obtained by reading the audio file of the text to be synthesized by a natural person reading the text to be synthesized Analyzing, or analyzing the prosodic parameter labeling file obtained by labeling the prosodic parameter of the text to be synthesized with a predetermined labeling standard, to obtain natural prosodic parameters or approximate natural prosodic parameters; Prosodic parameters are used as a reference, and the corresponding speech synthesis unit is selected from the pre-recorded speech library for the text to be synthesized, and the speech synthesis units are spliced and synthesized to generate a synthesized speech file corresponding to the text to be synthesized.

According to a seventh aspect of the present invention, there is provided the method described in the sixth aspect, wherein the voice generating step includes: using the natural rhythm parameter or an approximate natural rhythm parameter as a reference, selecting a voice for the voice from the pre-recorded voice library The globally optimal speech synthesis unit of the text to be synthesized; and based on the natural rhythm parameter or approximate natural rhythm parameter, the selected speech synthesis unit is spliced and synthesized, and the waveform adjustment is performed on the synthesized speech file, so as to obtain the text corresponding to the text to be synthesized Synthetic speech files with high naturalness for synthesized text.

According to an eighth aspect of the present invention, there is provided the method described in the seventh aspect, wherein the step of obtaining the prosody parameters includes: obtaining a recording file of the text to be synthesized by a natural person reading the text to be synthesized; and from the recording file The prosodic parameters are obtained from the waveform data, and the prosodic parameters include sound duration, audio frequency and energy.

According to a ninth aspect of the present invention, there is provided the method described in the seventh aspect, wherein the step of obtaining prosodic parameters includes: referring to a knowledge base defining labeling rules for prosodic parameters and using the predetermined labeling standard to process the text to be synthesized prosodic parameter labeling to obtain a prosodic parameter labeling file; and analyzing the prosodic parameter labeling file to obtain approximate prosodic parameters, the approximate prosodic parameters including sound duration, audio frequency and energy.

According to a tenth aspect of the present invention, there is provided the method described in the sixth aspect, wherein the recording file of the text to be synthesized is obtained by means of electronic cartoon pronunciation.

According to an eleventh aspect of the present invention, there is provided a computer program including computer instruction codes. When the computer program is loaded into a computer and the computer executes the computer instruction codes included in the computer program, the computer program as described above can be realized. The above-mentioned method for speech synthesis based on prosodic reference according to any one of the sixth aspect to the tenth aspect of the present invention.

According to a twelfth aspect of the present invention, there is provided a computer-readable recording medium carrying the computer program as described in the eleventh aspect, the computer-readable recording medium can be read by a computer to load the computer program into The computer executes the computer instruction code included in the computer program by the computer, so as to realize the method for speech synthesis based on prosodic reference according to any one of the sixth aspect to the tenth aspect of the present invention as described above .

According to the speech synthesis device and method of the above aspects of the present invention, the computer program for realizing the speech synthesis method, and the computer-readable recording medium carrying the computer program, it is possible to generate arbitrary synthesized speech whose cadence is very close to natural speech according to the user's requirements. Speech, enabling improved naturalness of synthesized speech.

Both the foregoing summary and the following detailed description are exemplary descriptions of the invention, not limitations of the invention. After reading the disclosure content of this application, those skilled in the art can completely conceive various other forms of implementation based on the essence of the present invention, but as long as these implementations include the following technical features, that is, read aloud by a real person The text to be synthesized or the recording file obtained in other ways (for example, through electronic cartoon pronunciation) or the prosodic parameter labeling file obtained by labeling the text to be synthesized with reference to the prosodic parameter labeling rule knowledge base with predetermined labeling standards, is analyzed and obtained Natural rhythm parameters or approximate rhythm parameters, and using these natural rhythm parameters or approximate rhythm parameters to refer to the pre-recorded voice library to synthesize a synthesized voice file with high naturalness, obviously these implementations should fall within the scope of the present invention. The protection scope of the present invention is specifically defined by the claims.

Specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

Description of drawings

The included drawings are used to provide a further understanding of the present invention, which constitute a part of the specification, illustrate preferred embodiments of the present invention, and together with the text description, are used to explain the principle of the present invention, wherein for the same elements, Always identified with the same reference signs. In the attached picture:

1 is a block diagram illustrating a configuration example of a speech synthesis system according to a first embodiment of the present invention;

2 is a block diagram illustrating a configuration example of a natural rhythm parameter acquisition section in the speech synthesis system according to the first embodiment of the present invention;

3 is a block diagram illustrating an example of the configuration of a voice generating unit in the speech synthesis system according to the first embodiment of the present invention;

4 is a block diagram illustrating a configuration example of a speech synthesis system according to a second embodiment of the present invention;

5 is a block diagram illustrating a configuration example of an approximate natural rhythm parameter acquisition section in a speech synthesis system according to a second embodiment of the present invention; and

FIG. 6 is a diagram showing an example of a prosodic parameter labeling rule knowledge base.

Detailed ways

These and other aspects of the invention will become apparent with reference to the following description and drawings. In these descriptions and drawings, some specific embodiments of the present invention are specifically disclosed to represent some ways of implementing the principles of the present invention, but it should be understood that the scope of the present invention is not limited thereto. On the contrary, the invention includes all changes, modifications and equivalents coming within the spirit and scope of the appended claims.

Features described and/or exemplified for one embodiment can be used in the same or similar manner in one or more other embodiments, and/or in combination with or instead of features of other embodiments .

It should be emphasized that the word "comprising" when used in this specification is used to refer to the presence of stated features, integers, steps or components, but does not exclude the presence of one or more other features, integers, steps, components or The presence or increase of their combinations.

An exemplary detailed description of an embodiment of the present invention is given with reference to FIGS. 1 to 6 .

FIG. 1 is a block diagram illustrating a configuration example of a speech synthesis system according to a first embodiment of the present invention. The figure also shows the working flow of the speech synthesis system according to the first embodiment, that is, the signal flow relationship among various components.

The speech synthesis system according to the first embodiment includes a text to be synthesized providing unit 101, a natural rhythm parameter acquisition unit 102, a natural rhythm parameter storage unit 103, a phoneme list storage unit 104, a speech library 105, a voice generation unit 106, and a synthesized speech storage unit 107.

The text-to-be-synthesized providing unit 101 provides any text to be synthesized by speech, and can preprocess the text to eliminate various defects and errors in it, make it standardized and accurate, and facilitate the production of good recording files.

The natural prosody parameter acquisition unit 102 generates the natural prosody parameters and phoneme list corresponding to the text to be synthesized based on the text to be synthesized provided by the text to be synthesized providing unit 101, and provides them to the natural prosody parameter storage unit 103 and the phoneme list storage unit respectively. The unit 104 stores it.

Natural prosody parameters refer to the acoustic prosody parameters corresponding to each phoneme in speech, generally including sound length, fundamental frequency and energy. For example, sound duration describes the length of time a phoneme corresponds to in the speech flow.

The phoneme list refers to a sequence obtained through language analysis and phonetic analysis at the front end of the speech synthesis system. Generally speaking, it corresponds to a text sentence. For example, for Chinese, after being processed by the front-end of the speech synthesis system, the following information is included: Chinese characters, word segmentation information, part-of-speech information, Chinese pinyin (syllable, semi-syllable) information, prosodic boundary level information, etc.

Below is an example of a list of possible phonemes. This phoneme list contains participle, pinyin, part of speech and prosodic structure information.

There are (you3)/v one (yi1)/m times (ci4)/q|||we (wo3 men5)/r|| and (he2)/p|outer (wai4)/f school (xiao4)/ng| | engage in (gao3)/v fellowship (lian2 yi4)/v| climb (pa2)/v Xiangshan (xiang1 shan1)/ns|||our (wo3 men5)/r's (de5)/u|student (xue2 sheng1) /n||no(mei2 you3)/vone(yi2 ge4)/m|left behind(diao4 dui4)/v(de5)/u|||噢噌噌(ceng1 ceng1 ceng1)/o||just(jiu4 )/d climb (pa2)/v up (shang4)/v up (le5)/u|top of the mountain (shan1 ding3)/n@.

Since there is no mark between Chinese words and words, in order to facilitate subsequent processing, lexical analysis including word segmentation and part-of-speech tagging is the primary issue. Generally speaking, machine automatic word segmentation uses a dictionary to complete. At present, the main automatic word segmentation methods are forward maximum matching method, backward maximum matching method, language model method, hidden Markov model and maximum entropy model. In the above example, one or more characters before "/" form a word, and the English letters after "/" indicate the part of speech of the word. For example, "have" is a verb ("v"), "we" is a pronoun ("r"), etc.

Among them, "|", "||", "|||" represent prosodic phrase, prosodic phrase and intonation phrase respectively. "@" is used to mark the end of a sentence. In this way, "one time", "we and", "outside school", "engaging in friendship", "climbing Xiangshan" and so on are all rhyming words. Furthermore, "we and other schools" and "climbing Xiangshan in friendship" are prosodic phrases, and "we are climbing Xiangshan in friendship with other schools" are intonation phrases.

The other is pinyin information. For example: "you3" is the pinyin of the word "you", and "wo3 men54" is the pinyin of the word "we".

Based on the natural prosody parameters stored in the natural prosody parameter storage unit 103 and the phoneme list stored in the phoneme list storage unit 104, the voice generator 106 selects a plurality of phonetic units corresponding to the text to be synthesized from the pre-recorded voice bank 105, And perform waveform synthesis and adjustment on multiple selected speech units to generate a final synthesized speech file, and provide the synthesized speech file to the synthesized speech storage unit 107 for storage.

The speech library 105 may be a speech library for a general speech synthesis system. The voice generator 106 may be a back-end module of a general-purpose speech synthesis system. The natural prosody parameter storage unit 103, the phoneme list storage unit 104 and the synthesized speech storage unit 107 can all be commonly used readable and writable storage devices in computer systems, such as RAM, flash memory, hard disk, readable and writable optical disk, magneto-optical disk, etc., or Dedicated storage server. Although these storage units are described here as separate components, in practice, they may share the same physical storage device in essence.

FIG. 2 is a block diagram illustrating a specific configuration example of the natural rhythm parameter acquisition unit 103 .

The natural prosody parameter acquisition unit 103 includes a recording unit 201 , a natural prosody parameter extraction unit 202 and a phoneme list generation unit 203 .

The recording unit 201 is configured to generate a recording file according to the text to be synthesized provided by the text to be synthesized providing unit 101 . Here, a natural person can be asked to read the text to be synthesized with an emotional intonation suitable for the content of the text, or a voice that can make a sound similar to an ordinary electronic cartoon or can make other special styles of sound. The text, and make a corresponding record to form a recording file. Alternatively, the recording material corresponding to or containing the content of the text may be searched from the existing recording material library, and the corresponding editing process may be performed to obtain the required recording file. The specific implementation of the recording unit 201 can adopt various methods fully described in the prior art, which will not be repeated here.

Then, based on the recording file provided by the recording unit 201, the natural prosody parameter extraction part 202 extracts the prosody parameters corresponding to the specific content (word or phrase unit) of the text by performing digital signal processing on the waveform data of the recording file, such as the sound length, audio, energy, etc.

The phoneme list generating unit 203 analyzes the text to be synthesized based on the analysis result of the prosodic parameter extracting unit 202 to obtain a phoneme list including a series of phonemes constituting the continuous speech flow of the text. Here, considering that there may be certain problems in the automatic generation of the phoneme list by the machine, such as the processing of polyphonic characters, etc., manual verification can be added to the processing of the phoneme list generation unit 203 to generate the correct phoneme list corresponding to the text to be synthesized .

The natural prosody parameters extracted by the prosody parameter extracting unit 202 and the phoneme list generated by the phoneme list generating unit 203 are stored in the prosody parameter storage unit 103 and the phoneme list storage unit 104 , respectively.

FIG. 3 illustrates an example of the configuration of the sound generator 106 .

The sound generation unit 106 includes a unit selection unit 301 and a waveform generation unit 302 . The unit selection unit 301 uses the natural prosody parameters stored in the natural prosody parameter storage unit 103 as a reference, and searches for the phoneme list stored in the phoneme list storage unit 104 from the various phonetic units stored in the speech bank 105 that is in line with the global optimum. A list of speech synthesis units that includes an array of speech synthesis units.

The description of the global optimum is as follows.

Given a sentence, suppose to consist of N phonemes. For each phoneme, according to its prosodic parameter characteristics, the most similar sample to its prosodic parameter can always be found in the speech library as its synthesis unit. However, simply splicing together the N synthetic units thus found is not optimal for a sentence. In fact, in addition to hoping that the prosody parameters of each phoneme synthesis are as similar as possible to the sample prosody parameters, the sound quality loss caused by the inconsistency of the frequency spectrum at the splicing of adjacent synthesis units should also be considered. Based on this consideration, we call this unit selection strategy the global optimal unit selection.

Then, the waveform generation unit 302 splices and synthesizes the speech synthesis unit list selected by the unit selection unit 301, and refers to the natural rhythm parameters in the natural rhythm parameter storage unit 103, and performs waveform adjustment on the spliced and synthesized files to obtain a speech synthesis unit list with high natural rhythm. degree of synthetic voice file, and the synthetic voice file is stored in the synthetic voice storage unit 107.

According to the first embodiment of the present invention, using the pronunciation standard provided by the speech database and the speech material with beautiful timbre, combined with the natural rhythm parameters obtained by analyzing the recording files obtained by reading aloud by real people or other methods, it is possible to synthesize beautiful and timbre. Synthetic voice files with natural rhythms.

The speech synthesis system according to the second embodiment of the present invention will be described below with reference to FIGS. 4 to 6 . The difference from the first embodiment is that the second embodiment obtains approximate natural rhythm parameters by annotating prosodic parameters of the text to be synthesized and analyzing the tagged file.

As shown in Figure 4, the speech synthesis system according to the second embodiment of the present invention includes a text to be synthesized providing unit 101, an approximate natural rhythm parameter acquisition unit 401, an approximate natural rhythm parameter storage unit 402, a phoneme list storage unit 104, and a speech library 105 , the voice generation unit 106 and the synthesized voice storage unit 107 . Among them, the to-be-synthesized text providing unit 101 , phoneme list storage unit 104 , speech library 105 , voice generation unit 106 and synthesized speech storage unit 107 are the same as those in the first embodiment, and will not be described again here.

The approximate natural rhythm parameter acquiring unit 401 tags the text provided by the text to be synthesized providing unit 101 by referring to predetermined prosodic parameter tagging rules, and then analyzes the tagged text to obtain approximate natural rhythm parameters. The obtained approximate natural rhythm parameters are stored in the approximate natural rhythm parameter storage unit 402 .

FIG. 5 illustrates a configuration example of the approximate natural rhythm parameter acquisition unit 401 .

The approximate natural parameter acquisition unit 401 includes a prosodic parameter labeling rule knowledge base 501 , a prosodic parameter labeling unit 502 , a prosodic parameter labeling file storage unit 503 , a prosodic parameter generating unit 504 and a phoneme list generating unit 505 .

The prosodic parameter labeling rule knowledge base 501 stores data related to various rules of prosodic parameter labeling. For example, FIG. 6 shows an example of a data structure of prosodic parameter labeling rules. In the prosodic parameter labeling rule, at least prosodic parameter name (for example, duration, audio frequency, speed, etc.), label symbol (for example, P, D, S) and label interval (for example, 0-9) etc. should be stored.

The prosodic parameter labeling unit 502 refers to the prosodic parameter labeling rules stored in the prosodic parameter labeling rule knowledge base 501 to perform prosodic parameter labeling on the text provided by the text to be synthesized providing unit 101, and stores the labeled file in the prosodic parameter labeling file storage unit 503 middle.

In fact, adopting prosodic labeling rules can be combined with automatic prosodic parameter prediction. Specifically, firstly, the automatic prosodic parameter prediction module predicts the prosodic parameters of each phoneme in the input phoneme list. On this basis, according to the needs of the application, some of the parameters can be modified by humans. For example, according to the needs of the context and emotion, one or several phonemes can be protracted or tone accentuated. The former can be used It is operated with the symbol D, and the latter can be operated with the symbol P. For example, assuming that "D5" means maintaining the original sound length, "D6, D7, D8, D9" can represent the lengthening of the sound length by 10%, 20%, 30%, and 40% respectively on the original basis. . On the contrary, "D4, D3, D2, D1, D0" can represent that the sound length is accelerated by 10%, 20%, 30%, 40%, and 50% on the original basis.

The prosodic parameter generation unit 504 analyzes the prosodic parameter labeling file stored in the prosodic parameter labeling file storage unit 503 to obtain various quantized prosody parameters, and stores them in the approximate natural prosody parameter storage unit 402 . Although these rhythm parameters cannot be completely equal to the natural rhythm parameters obtained in the first embodiment, they still have a good degree of naturalness.

The phoneme list generating unit 505 is configured to analyze the prosodic parameter labeling file stored in the prosodic parameter labeling file 503 to generate a phoneme list corresponding to the file, and store the generated phoneme list in the phoneme list storage unit 104 .

Finally, as described in the first embodiment, the voice generation unit 106 uses the approximate natural rhythm parameters stored in the approximate natural rhythm parameter storage unit 402 as a reference, and selects phoneme The phoneme list stored in the list 104 searches for a globally optimal speech synthesis unit list including a series of speech synthesis units, splicing and synthesizing the speech synthesis units, and adjusting the waveform of the synthesized file with reference to the approximate natural rhythm parameters, to obtain synthesized speech files with improved naturalness.

According to the second embodiment of the present invention, not only compared with the speech synthesis system of the prior art, can obtain the synthesized speech file of improved naturalness, and compared with the first embodiment of the present invention, do not need to read aloud with real person, This increases efficiency and saves costs.

It should be further explained that although the natural prosody parameter storage unit 103, the phoneme list storage unit 104, the synthesized speech storage unit 107, the approximate natural prosody parameter storage unit 402, etc. have been separately described above, these components can share the same physical storage unit in essence. device. The speech database 105 and the prosodic parameter labeling rule knowledge base 501 may use a dedicated storage device, but may also use a part of a general storage device.

The speech synthesis system of the present invention can be realized by a general-purpose computer system coupled with corresponding computer programs. However, it is not limited thereto, and each component can also be implemented as a dedicated electronic device (such as firmware, etc.), and a complete speech synthesis system can be realized by integrating them.

The present invention provides a speech synthesis system capable of improving the naturalness of synthesized speech. By adopting the speech synthesis system of the present invention, it is possible to generate a synthesized speech with beautiful timbre and cadence very close to natural speech according to the user's requirements.

By combining the standard sweet-sounding reading and the not-so-good-sounding or non-standard pronunciation but emotional reading, it is possible to create a synthesized voice work with a sweet and cadenced voice by reading aloud by anyone. Therefore, the production cost of the audio electronic book can be greatly reduced, and the audio electronic book with the same voice feature and full of rhythm can be produced in batches.

By adopting the speech synthesis system of the present invention, the original sweet voice can still be emitted when the voice of the voice changes. For example, when an announcer has a cold, he can use this system to broadcast in a voice that is not much different from the past. When women are old, they can still make their youthful voices through this system.

Although only the preferred embodiments have been chosen to illustrate the present invention, those skilled in the art can easily make various changes and modifications based on the disclosure herein without departing from the scope of the invention defined by the appended claims. The descriptions of the above embodiments are illustrative only, and do not constitute limitations on the invention defined by the appended claims and their equivalents.

Claims (10)

1, a kind of based on the rhythm with reference to carrying out the speech synthetic device of phonetic synthesis, it comprises:

The prosodic parameter acquisition unit, it is by to by comprising that the recording file that the nature person reads aloud this text to be synthesized that the mode of text to be synthesized obtains analyzes, perhaps the prosodic parameter mark file that obtains treat synthesis text to carry out the prosodic parameter mark with predetermined labeled standards is analyzed, and obtains nature prosodic parameter or approximate rhythm of nature parameter; With

Sound sound makes portion, its with this rhythm of nature parameter or approximate rhythm of nature parameter as reference, from the sound bank of pre-recording, select corresponding phonetic synthesis unit at this text to be synthesized, and described phonetic synthesis unit is spliced synthetic, to produce synthetic speech file corresponding to this text to be synthesized.

2, speech synthetic device as claimed in claim 1, wherein, this sound sound portion of making comprises:

The voice unit selection portion, it as reference, selects the phonetic synthesis unit at this text to be synthesized global optimum with this rhythm of nature parameter or approximate rhythm of nature parameter from this sound bank of pre-recording; With

The waveform generating unit, it is based on this rhythm of nature parameter or approximate rhythm of nature parameter, the phonetic synthesis unit that this voice unit selection portion is selected splices synthetic and the voice document after synthetic is carried out the waveform adjustment, to obtain the synthetic speech file with high naturalness corresponding to this text to be synthesized.

3, speech synthetic device as claimed in claim 2, wherein, this prosodic parameter acquisition unit comprises:

Recording portion, it reads aloud the recording file that this text to be synthesized obtains this text to be synthesized by the nature person; With

The prosodic parameter extraction unit, it obtains prosodic parameter from the Wave data of this recording file, and this prosodic parameter comprises the duration of a sound, audio frequency and energy.

4, speech synthetic device as claimed in claim 2, wherein, this prosodic parameter acquisition unit comprises:

Prosodic parameter mark portion, its knowledge base with reference to the mark rule that has defined prosodic parameter is carried out the prosodic parameter mark with this predetermined labeled standards to this text to be synthesized, to obtain prosodic parameter mark file; With

The prosodic parameter generating unit, it is analyzed this prosodic parameter mark file, and to obtain approximate prosodic parameter, this approximate prosodic parameter comprises the duration of a sound, audio frequency and energy.

5, speech synthetic device as claimed in claim 1, wherein, the recording file of this text to be synthesized is to obtain by the mode that the electronics cartoon is pronounced.

6, a kind of based on the rhythm with reference to carrying out the method for phonetic synthesis, it may further comprise the steps:

The prosodic parameter obtaining step, it is by to by comprising that the recording file that the nature person reads aloud this text to be synthesized that the mode of text to be synthesized obtains analyzes, perhaps the prosodic parameter mark file that obtains treat synthesis text to carry out the prosodic parameter mark with predetermined labeled standards is analyzed, and obtains nature prosodic parameter or approximate rhythm of nature parameter; With

Sound sound makes step, its with this rhythm of nature parameter or approximate rhythm of nature parameter as reference, from the sound bank of pre-recording, select corresponding phonetic synthesis unit at this text to be synthesized, and described phonetic synthesis unit is spliced synthetic, to produce synthetic speech file corresponding to this text to be synthesized.

7, method as claimed in claim 6, wherein, this sound sound makes step and comprises:

As reference, from this sound bank of pre-recording, select phonetic synthesis unit with this rhythm of nature parameter or approximate rhythm of nature parameter at this text to be synthesized global optimum; With

Based on this rhythm of nature parameter or approximate rhythm of nature parameter, to splicing synthetic and the voice document after synthetic is carried out the waveform adjustment in selected phonetic synthesis unit, to obtain synthetic speech file with high naturalness corresponding to this text to be synthesized.

8, method as claimed in claim 7, wherein, this prosodic parameter obtaining step comprises:

Read aloud the recording file that this text to be synthesized obtains this text to be synthesized by the nature person; With

Obtain prosodic parameter from the Wave data of this recording file, this prosodic parameter comprises the duration of a sound, audio frequency and energy.

9, method as claimed in claim 7, wherein, this prosodic parameter obtaining step comprises:

Knowledge base with reference to the mark rule that has defined prosodic parameter is carried out the prosodic parameter mark with this predetermined labeled standards to this text to be synthesized, to obtain prosodic parameter mark file; With

This prosodic parameter mark file is analyzed, and to obtain approximate prosodic parameter, this approximate prosodic parameter comprises the duration of a sound, audio frequency and energy.

10, method as claimed in claim 6 wherein, obtains the recording file of this text to be synthesized by the mode of electronics cartoon pronunciation.

Images