JP5552797B2 - Speech synthesis apparatus and speech synthesis method - Google Patents

Description

  The present invention relates to a technique for synthesizing voice (typically singing sound).

  Conventionally, a technique for synthesizing a desired speech using a set of a plurality of segment data (hereinafter referred to as “speech library”) indicating speech segments has been proposed (for example, Patent Document 1). The speech library is created by collecting and analyzing actual speech and then segmenting and analyzing each speech unit.

JP 2002-202790 A

  Under the technique of Patent Document 1, a separate audio library is required for each characteristic of audio to be synthesized. Therefore, in order to synthesize a voice having characteristics different from those of an existing voice library (for example, a singer's singing sound), it is necessary to create a new voice library. In addition, since a large number of voice libraries are used for synthesizing various voices, there is a problem that the capacity of a storage device necessary for storing these voice libraries increases. In view of the above circumstances, an object of the present invention is to synthesize various voices while reducing the effort for creating a voice library and the capacity required for storing the voice library.

In order to solve the above problems, the speech synthesizer according to the first aspect of the present invention includes a speech library including a plurality of segment data indicating speech segments, and segment usage information that defines the usage of the segment data. but storage means for storing the attribute information set in the target each of the plurality of the unit one or more fragment data in the speech library as a unit, designated sounds (designated as the synthesis of the target) The segment selection means for sequentially selecting segment data of the audio library according to the music information indicating the time series, and each segment data selected by the segment selection means is set as the segment data in the attached information Segment processing means for processing according to the segment usage information, and synthesis processing means for synthesizing speech from the segment data processed by the segment processing means. In the above configuration, synthesized sound is generated by applying the attached information to the audio library, so the acoustic characteristics are different from the synthesized sound from the existing audio library without adding a new audio library. It is possible to generate a synthesized sound. That is, it is possible to synthesize various voices while reducing the effort required to create a voice library and the capacity required for storing the voice library.

The storage means is a concept including a single recording medium for storing the audio library and the attached information, and a plurality of separate recording media for separately storing the audio library and the attached information. The storage means and the means for storing music information may be separate recording media or separate storage areas set on a single recording medium.

  In a preferred aspect of the present invention, the attached information includes section information indicating a section used for speech synthesis in the piece data, and the piece processing means includes the piece data selected by the piece selection means. The section indicated by the section information is extracted. In the above aspect, it is possible to generate various synthesized sounds by making the usage interval of each piece data different from the piece data of the existing speech library. In another aspect, the attached information includes characteristic information indicating a feature amount in the speech unit corresponding to the piece data, and the piece processing means is a feature amount of the piece data selected by the piece selection means. Is controlled according to the characteristic information. In the above aspect, various synthesized sounds can be generated by making the change in the feature amount of each piece data different from the piece data of the existing speech library. Specific examples of the above aspects will be described later as the first embodiment.

In a preferred aspect of the present invention, the storage means stores a plurality of sound libraries, the attached information indicates a mixture ratio of each piece data of the plurality of sound libraries, and the piece selection means has a plurality of sound libraries. The segment data is selected from each of the libraries, and the segment processing unit mixes the segment data selected from each speech library by the segment selection unit at a mixing ratio indicated by the attached information. In the above aspect, since the segment data selected from each speech library is mixed at the mixing ratio specified by the attached information, a synthesized sound reflecting the characteristics of each segment data of the plurality of speech libraries is generated. It is possible. A specific example of the above aspect will be described later as a second embodiment.

  The speech synthesizer according to the second aspect of the present invention includes a plurality of speech libraries each including a plurality of segment data indicating speech units, and setting values of control variables for each segment data of the plurality of speech libraries. First storage means for storing the attached information to be shown, second storage means for storing music information indicating the time series of the designated sound, variable instruction means for sequentially indicating the instruction values of the control variables, and a plurality of audio libraries Among the segment data corresponding to the music information, the segment selection means for selecting the segment data whose set value in the attached information is close to the instruction value by the variable instruction means, and the segment data selected by the segment selection means Synthesizing means for synthesizing speech from. In the above configuration, the synthesized sound is generated using the segment data selected from each speech library. Therefore, the synthesized speech from one existing speech library can be generated without adding a new speech library. Can generate synthesized sounds with different acoustic characteristics. That is, it is possible to synthesize various voices while reducing the effort required to create a voice library and the capacity required for storing the voice library. A specific example of the above aspect will be described later as a third embodiment.

The speech synthesizer according to each aspect described above is realized by hardware (electronic circuit) such as a DSP (Digital Signal Processor) dedicated to speech synthesis and general-purpose arithmetic processing such as a CPU (Central Processing Unit). This is also realized by cooperation between the apparatus and the program. The program according to the first aspect of the present invention includes a speech library including a plurality of segment data indicating speech segments, and segment usage information for defining the usage of the segment data in one or more segments in the speech library. In a computer having storage means for storing auxiliary information set for each of a plurality of units with a piece of data as a unit, the piece of speech library data is stored in accordance with the music information indicating the time series of the specified sound. Segment selection processing for sequentially selecting, segment processing for processing each segment data selected in the segment selection processing according to the segment usage information set in the segment data in the attached information, And synthesizing processing for synthesizing speech from segment data after processing by segment processing. According to the above program, the same operation and effect as the speech synthesizer according to the first aspect are realized.

  The program according to the second aspect of the present invention includes a plurality of speech libraries each including a plurality of segment data indicating speech segments, and setting values of control variables for each segment data of the plurality of speech libraries. Variable instruction processing for sequentially instructing instruction values of control variables to a computer having first storage means for storing attached information and second storage means for storing music information indicating a time series of a specified sound; In each of the plurality of audio libraries, segment selection processing for selecting segment data in which the set value in the attached information is close to the instruction value by the variable instruction processing among the segment data corresponding to the music information, and the segment selection processing And a synthesis process for synthesizing speech from the selected segment data. According to the above program, the same operation and effect as the speech synthesizer according to the second aspect are realized.

  The program according to each aspect of the present invention is provided to the user in a form stored in a computer-readable recording medium and installed in the computer, and is also provided from the server device in the form of distribution via a communication network. Installed on the computer.

It is a block diagram of the speech synthesizer concerning a 1st embodiment. It is a schematic diagram of the waveform of a speech element. It is a schematic diagram of attached information. It is a schematic diagram of an edit image. It is a schematic diagram for demonstrating the process by a speech synthesizer. It is a schematic diagram for demonstrating the process of the speech synthesizer in 2nd Embodiment. It is a schematic diagram for demonstrating the process of the speech synthesizer in 3rd Embodiment. It is a schematic diagram of the edited image in 3rd Embodiment. It is a schematic diagram which shows the structure of the virtual library in a modification. It is a schematic diagram which shows the structure of the virtual library in a modification. It is a schematic diagram which shows the structure of the virtual library in a modification.

<A: First Embodiment>
FIG. 1 is a block diagram of a speech synthesizer 100 according to the first embodiment of the present invention. The speech synthesizer 100 is a device that synthesizes various sounds such as singing sounds (hereinafter referred to as “synthesized sounds”). As shown in FIG. 1, the control device 10, the storage device 12, the input device 14, and the display device 16. And a sound emitting device 18. In the following, it is assumed that the speech synthesizer 100 is used for synthesizing a song singing sound.

  The control device (CPU) 10 realizes a plurality of functions (display control unit 22, information generation unit 24, speech synthesis unit 26) necessary for generating the audio signal SOUT by executing the program PG stored in the storage device 12. To do. The audio signal SOUT is a signal representing the waveform of the synthesized sound. A configuration in which each function of the control device 10 is realized by a dedicated electronic circuit (DSP) or a configuration in which each function of the control device 10 is distributed over a plurality of integrated circuits may be employed.

  The input device 14 is a device (for example, a mouse or a keyboard) that receives an instruction from a user. The display device (for example, a liquid crystal display device) 16 displays an image instructed from the control device 10. A sound emitting device (for example, a speaker or a headphone) 18 emits a sound wave corresponding to the sound signal SOUT generated by the control device 10.

  The storage device 12 stores a program PG executed by the control device 10 and various data (music information DS, audio library L, attached information A) used by the control device 10. A known recording medium such as a semiconductor recording medium or a magnetic recording medium or a combination of a plurality of types of recording media is arbitrarily employed as the storage device 12. A configuration in which the program PG and each data (DS, L, A) are distributed and stored in a plurality of recording media is also employed.

  The music information DS is information (score data) indicating a time series of musical notes (hereinafter referred to as “designated sound”) constituting the music. Specifically, the music information DS designates the pitch (note number) of the designated sound, the pronunciation period, and the pronunciation character for each designated sound in the music. The sound generation period is defined by, for example, the time when sound generation starts and the length of time during which sound generation is continued. The pronunciation character is a character indicating the content (lyrics) of the pronunciation in syllable units.

  The speech library L in FIG. 1 is a set of a plurality of segment data V corresponding to different speech segments. Each piece data V is used as a synthetic sound material. The phoneme segment is, for example, a phoneme corresponding to a minimum unit that can be audibly distinguished from a voice, or a phoneme chain in which a plurality of phonemes are connected. A phoneme chain is typically a chain of two phonemes (consonant-vowel, vowel-consonant, consonant-consonant, vowel-vowel), but a chain of three or more phonemes (eg consonant-vowel-consonant). It is also a concept that encompasses syllables.

  As shown in FIG. 1, the segment data V of each speech unit constituting the speech library L includes the waveform W of the speech unit (hereinafter referred to as “segment waveform”) W and the initial unit waveform W. And section information Q indicating a section to be used. The used section corresponds to a section of the unit waveform W that is actually used for generating a synthesized sound. FIG. 2 is a schematic diagram of a unit waveform W of a speech unit (phoneme chain) [a_s] in which a vowel phoneme [a] and a consonant phoneme [s] are continuous. The section information Q includes the initial start point of the used section (hereinafter referred to as “initial start point”) qS and the initial end point of the used section (hereinafter “ QE) (referred to as "initial end point").

  The attached information A in FIG. 1 is applied to the processing of each piece data V in the audio library L. As shown in FIG. 3, the attached information A includes the section information P and the characteristics for each speech unit that the user has instructed to edit among a plurality of speech units (segment data V) recorded in the speech library L. It is configured to include at least one of the information F. The segment information P and the characteristic information F are not included in the attached information A for the speech unit that the user does not instruct to edit.

  The section information P is information for designating a use section that is actually used for speech synthesis in the unit waveform W indicated by the unit data V in the speech library L. As shown in FIG. 2, the section information P variably indicates the start point pS and the end point pE of the use section. The starting point pS specified by the section information P may be different from the initial starting point qS specified by the section information Q in the audio library L. Similarly, the end point pE of the section information P may be different from the initial end point qE. The start point pS is specified as a change amount with respect to the initial start point qS, and the end point pE is specified as a change amount with respect to the initial end point qE.

  The characteristic information F in FIG. 3 indicates a feature amount (particularly temporal change) in the segment waveform W indicated by each segment data V in the audio library L. Specifically, the temporal change in volume, pitch, formant frequency, or timbre in the segment waveform W is indicated by the characteristic information F. The temporal change in timbre is defined by, for example, the transition of a spectrum or MFCC (mel-frequency cepstrum coefficient).

  The display control unit 22 in FIG. 1 causes the display device 16 to display an edited image 40 visually recognized by the user for generating and editing the music information DS and the attached information A. FIG. 4 is a schematic diagram of the edited image 40. As illustrated in FIG. 4, the edited image 40 includes a score area 42 that displays a time series of designated sounds, and an edit area 44 that is used for editing the attached information A.

  The score area 42 is a piano roll type image area in which a vertical axis (pitch axis) corresponding to pitch and a horizontal axis (time axis) corresponding to time are set. The user designates the pitch, pronunciation period, and pronunciation character of the designated sound by appropriately operating the input device 14 while visually recognizing the score area 42. The display control unit 22 arranges the sound indicator 51 corresponding to the designated sound instructed by the user in the score area 42. The position of the sound indicator 51 in the direction of the pitch axis is determined according to the pitch instructed by the user, and each end point of the sound indicator 51 in the direction of the time axis is the start point and end point of the sound generation period instructed by the user. It corresponds to. In addition, pronunciation characters instructed by the user are added to each sound indicator 51. It is also possible to adopt a configuration in which an image of a musical score in which a musical score of a designated sound is recorded in a staff score is arranged in the musical score area 42.

  The information generation unit 24 in FIG. 1 stores the pitch of the designated sound, the pronunciation period, and the pronunciation characters specified by the user with respect to the score area 42 in the music information DS of the storage device 12. By repeating the above processing, music information DS indicating the time series of the designated sound instructed by the user is generated in the storage device 12, and the time series of the sound indicator of each designated sound is shown in FIG. As shown in the musical score area 42.

  The editing area 44 includes a waveform area 441 and a characteristic area 443. The display control unit 22 arranges the segment waveform W of each segment data V used for synthesizing the designated sound designated by the user in the waveform area 441 in time series. The segment data V in which the segment waveform W is displayed in the waveform area 441 is selected according to the pronunciation character designated by the designated sound. For example, when “ASA (morning)” is designated as a pronunciation character as illustrated in FIG. 4, the display control unit 22 causes the speech element [#_a] (“#” means silence), [a ], [A_s], [s_a], [a], and [a_ #], the segment data V corresponding to each of them is acquired from the speech library L of the storage device 12, and each segment waveform W is stored in the waveform region 441. Arrange in time series.

  The display control unit 22 arranges, for each unit waveform W, a start point indicator 532 indicating the start point of the use interval and an end point indicator 534 indicating the end point of the use interval in the unit waveform W. The initial position of the start point indicator 532 is set to the initial start point qS indicated by the section information Q of the segment data V in the audio library L. Similarly, the initial position of the end point indicator 534 is set to the initial end point qE indicated by the segment information Q of the segment data V. The user appropriately operates the input device 14 to select one of the plurality of segment waveforms W arranged in the waveform region 441, and at the same time, the start point indicator 532 and the end point indicator of the segment waveform W are selected. It is possible to instruct the movement of 534. The display control unit 22 moves the start point indicator 532 and the end point indicator 534 within the range from the start point s0 to the end point e0 of each segment waveform W in accordance with an instruction from the user to the input device 14.

  Further, the display control unit 22 arranges the characteristic transition image 55 showing the transition of the feature amount of the segment data V used for the synthesis of each designated sound in the characteristic region 443 for each segment waveform W. For example, as illustrated in FIG. 4, the display control unit 22 causes the display device 16 to display a graph (line graph) indicating the temporal transition of the volume of each piece data V as the characteristic transition image 55. The characteristic transition image 55 corresponding to each segment waveform W is displayed in common with the segment waveform W in the waveform region 441 and the time axis. The user can instruct editing (change) of the characteristic transition image 55 by appropriately operating the input device 14. The display control unit 22 edits the characteristic transition image 55 in accordance with an instruction from the user.

  The information generation unit 24 updates the attached information A in response to an instruction from the user for the editing area 44. Specifically, the information generation unit 24 specifies the positions of the start point indicator 532 and the end point indicator 534 instructed by the user with respect to the unit waveform W of each unit data V, and the start point indicator Information indicating the start point pS corresponding to the position of 532 and the end point pE corresponding to the position of the end indicator 534 is generated as section information P of the segment data V and stored in the attached information A. That is, while the use section designated by the section information Q in the audio library L is maintained, the use section designated by the section information P of the attached information A is variably set according to an instruction from the user.

  Further, the information generation unit 24 generates characteristic information F from the characteristic transition image 55 edited by the user for the segment waveform W of each segment data V and stores it in the attached information A. That is, the feature quantity instructed by the characteristic information F of the attached information A for the segment waveform W while the segment waveform W of each segment data V in the audio library L is maintained corresponds to the instruction from the user. Set to variable. As described above, even when the user instructs the movement of the start point indicator 532 or the end point indicator 534 or the change of the characteristic transition image 55, only the attached information A is updated, Each piece data V is not changed at all.

  The voice synthesizer 26 in FIG. 1 synthesizes a designated sound indicated by the music information DS stored in the storage device 12 to generate a voice signal SOUT. Schematically, the speech synthesizer 26 generates the speech signal SOUT by processing the segment data V selected in accordance with the music information DS in the speech library L in accordance with the attached information A and connecting them together. . As shown in FIG. 5, the speech synthesis unit 26 includes a segment selection unit 32, a segment processing unit 34, and a synthesis processing unit 36. The segment selection unit 32 sequentially selects the segment data V of each speech unit corresponding to the pronunciation character designated by each designated sound in the music information DS from the speech library L of the storage device 12.

  The segment processing unit 34 processes each segment data V selected by the segment selection unit 32. When the segment information P or the characteristic information F is not included in the attached information A for the segment data V selected by the segment selection unit 32, the segment processing unit 34 includes the segment waveform W indicated by the segment data V. The use section indicated by the section information Q of the audio library L (the section extending between the start point qS and the end point qE in FIG. 2) is adjusted to the pitch and the sound generation period indicated by the music information DS.

  On the other hand, when the segment information P is included in the attached information A for the segment data V selected by the segment selection unit 32, the segment processing unit 34 includes the segment information in the segment waveform W indicated by the segment data V. The use section indicated by P (the section extending from the start point pS and the end point pE) is extracted, and the use section after extraction is adjusted to the pitch and sound generation period indicated by the music information DS. That is, the processing (extraction of the used section) according to the section information P of the attached information A is performed on the piece data V. A known technique is arbitrarily employed for adjusting the pitch and sound generation period of the segment data V. In addition, a configuration may be employed in which a use section corresponding to the section information P is extracted after adjustment of the pitch or sound generation period of the segment data V.

  Further, in the case where the characteristic information F is included in the attached information A for the segment data V selected by the segment selection unit 32, the segment processing unit 34 determines the usage period of the segment waveform W indicated by the segment data V, The music information DS is adjusted to the pitch and sound generation period indicated, and processed according to the characteristic information F. Specifically, the segment processing unit 34 sets the feature amount of the segment waveform W in accordance with the property information F so that the property indicated by the property information F is added to the use period of the segment waveform W. Control. For example, when the characteristic information F indicates a time series of volume, the segment processing unit 34 processes the segment data V so that the volume in the use section of the segment waveform W is along the time series of the volume of the characteristic information F. To do. Of the segment data V, the usage interval to be processed according to the characteristic information F is the usage interval indicated by the interval information Q of the audio library L for the segment data V in which the attached information A does not include the interval information P. The segment data V in which the attached information A includes the section information P is the use section indicated by the section information P.

  The synthesis processing unit 36 in FIG. 5 generates the audio signal SOUT by connecting the piece data V processed by the piece processing unit 34 to each other on the time axis. As can be understood from the above description, the sound signal SOUT of the singing sound in which the music composed of the notes indicated by the sound indicators 51 in the score area 42 is sung with the pronunciation characters of the designated sounds is generated. A known technique is arbitrarily employed for generating the audio signal SOUT using the segment data V.

  As described above, in the first embodiment, since the synthesized sound is generated by applying the auxiliary information A to the existing sound library L, a sound library separate from the sound library L is actually prepared. Without being performed, it is possible to generate an audio signal SOUT equivalent to the case of using a new audio library (virtual audio library) whose recorded sound characteristics are different from those of the audio library L. That is, it is possible to synthesize sounds having different characteristics without preparing a separate sound library L for each sound. Therefore, there is an advantage that various voices can be synthesized as in the case of creating and using a new voice library L while reducing the effort for creating the voice library L. Further, since the attached information A has a smaller data amount than the audio library L, there is an advantage that the capacity required for the storage device 12 is reduced as compared with the case where a new audio library L is prepared.

<B: Second Embodiment>
Next, a second embodiment of the present invention will be described. In the following examples, elements having the same functions and functions as those of the first embodiment are denoted by the same reference numerals, and detailed descriptions thereof are omitted as appropriate.

  FIG. 6 is a schematic diagram for explaining speech synthesis in the second embodiment. As shown in FIG. 6, the storage device 12 of the second embodiment stores a plurality (two in the following example) of audio libraries L (L1, L2). Each speech library L (L1, L2) is a set of segment data V (V1, V2) for each speech segment, as in the first embodiment. The audio library L1 and the audio library L2 are generated from audio having different characteristics. For example, the voice speaker from which each speech element is extracted differs between the voice library L1 and the voice library L2. Note that the voice library L1 and the voice library L2 may be generated from each of a plurality of voices uttered by a single speaker with different characteristics.

  The segment selection unit 32 in FIG. 6 sequentially selects the segment data V of each speech unit corresponding to the phonetic character designated for each designated sound in the music information DS from each of the speech library L1 and the speech library L2. To do. Accordingly, the segment data V1 in the speech library L1 and the segment data V2 in the speech library L2 are sequentially selected for each speech segment corresponding to each phonetic character.

  As shown in FIG. 6, the attached information A of the second embodiment is that, for each of a plurality of speech units recorded in the speech library L1 and the speech library L2, the segment data V1 and speech library L2 in the speech library L1. The mixing ratio R with the segment data V2 is indicated. Each mixing ratio R indicated by the auxiliary information A is variably set according to an instruction from the user to the input device 14, for example.

  The segment processing unit 34 is instructed by the attached information A to the segment data V1 selected from the speech library L1 by the segment selection unit 32 and the segment data V2 selected from the speech library L2. The segment data VA is generated by mixing (adding) at the mixing ratio R. Note that the adjustment of the pitch and the sound generation period according to the music information DS is executed by the method exemplified in the first embodiment before or after mixing by the segment processing unit 34. The synthesis processing unit 36 generates an audio signal SOUT from the segment data VA processed (after mixing) by the segment processing unit 34.

  In the above embodiment, the segment data V1 of the audio library L1 and the segment data V2 of the audio library L2 are mixed at the mixing ratio R of the auxiliary information A and then used to generate the audio signal SOUT. Therefore, without creating a new audio library L, an audio library reflecting the characteristics of both the audio library L1 and the audio library L2 (that is, the unit data V of the audio library L1 and the unit data V of the audio library L2) It is possible to generate an audio signal SOUT equivalent to the case of using a virtual audio library composed of segment data V mixed with. That is, as in the first embodiment, it is possible to synthesize various voices as in the case of preparing a new voice library L while reducing the effort for creating the voice library L and the capacity required for the storage device 12. There is.

<C: Third Embodiment>
FIG. 7 is a schematic diagram for explaining speech synthesis in the third embodiment of the present invention. As shown in FIG. 7, the storage device 12 of the third embodiment is similar to the second embodiment in that a plurality (two in the following example) of audio libraries L (L1, L2) is stored.

  Further, the storage device 12 stores variable information DP indicating a change in the numerical value of a control variable (control parameter) X applied to speech synthesis. The control variable X is a variable for controlling a musical expression given to the synthesized sound. Specifically, the sound intensity (velocity), volume (dynamics) of the specified sound, breathness intensity (breathness), clarity (brightness, clearness), opening degree during pronunciation, gender ( genderfactor), time point of continuous pitch change (portamento) (portamento-timing), minute pitch change (pitch-bend), maximum pitch change (pitch-bend sensitivity), etc. A known variable applied to the above is arbitrarily adopted as the control variable X. Further, a configuration in which a combination of a plurality of variables selected from the above examples (for example, a plurality of variables instructed by the user) is designated as a new (virtual) control variable X by the variable information DP may be employed.

  FIG. 8 is a schematic diagram of an edited image 40 in the third embodiment. As illustrated in FIG. 8, the display control unit 22 converts an image 57 (hereinafter referred to as “variable transition image”) 57 indicating the time series of the control variable X indicated by the variable information DP into each sound indicator 51 in the score area 42. Are arranged in the editing area 46 on the same time axis as that of the time series. Specifically, a graph (for example, a line graph) showing the numerical transition of the control variable X is displayed as the variable transition image 57. The display control unit 22 changes the variable transition image 57 as needed in accordance with an instruction from the user to the input device 14. The information generation unit 24 updates the variable information DP of the storage device 12 to the content indicating the time series of the control variable X corresponding to the variable transition image 57 after the change. That is, the variable information DP is variably set according to an instruction from the user.

  As shown in FIG. 7, the auxiliary information A of the third embodiment includes numerical values of control variables X (hereinafter “set values”) for each of the segment data V1 of the audio library L1 and the segment data V2 of the audio library L2. XA) is designated for each speech unit. A set value xA1 is instructed to each piece data V1, and a set value xA2 is instructed to each piece data V2. The set value xA (xA1, xA2) is different between the segment data V1 and the segment data V2 that share the speech segment. For example, in FIG. 7, the set value xA1 of the segment data V1 of the speech unit [a_s] is set to 0.2, and the set value xA2 of the segment data V2 of the same speech unit [a_s] is set to 0.6. Illustrated. Each set value xA1 and each set value xA2 are variably set according to an instruction from the user to the input device 14, for example.

  As shown in FIG. 7, the speech synthesis unit 26 according to the third embodiment includes a variable instruction unit 38 in addition to the segment selection unit 32, the segment processing unit 34, and the synthesis processing unit 36. The variable instruction unit 38 sequentially specifies the numerical value (hereinafter referred to as “instruction value”) xB of the control variable X to the element selection unit 32. Specifically, the variable instruction unit 38 sequentially acquires the numerical value of the control variable X designated by the variable information DP in time series as the instruction value xB from the storage device 12 and instructs the element selection unit 32.

  The segment selection unit 32 selects one of the speech segment data V (segment data V1 and segment data V2) corresponding to the pronunciation character of the music information DS in each of the speech library L1 and the speech library L2. For each of them, the setting value xA (xA1, xA2) defined in the attached information A and the instruction value xB from the variable instruction section 38 are selected sequentially. Specifically, the segment selection unit 32 designates the set values xA (xA1, xA2) defined in the attached information A among the segment data V1 and the segment data V2 corresponding to the phonetic characters as the variable instructions. The segment data V close to the instruction value xB from the unit 38 is selected as the segment data VA.

  For example, a case is assumed in which the synthesis of the speech unit [a_s] is instructed in the situation where the attached information A is set to the content illustrated in FIG. When the instruction value xB from the variable instruction unit 38 is 0.3, for example, the segment selection unit 32 sets the set value xA1 (0.2) and the set value xA2 (0.6) defined in the attached information A for the speech unit [a_s]. Of these, the segment data V1 corresponding to the set value xA1 close to the indicated value xB (0.3) is selected from the speech library L1 as the segment data VA. On the other hand, when the instruction value xB from the variable instruction unit 38 is 0.5, for example, the element selection unit 32 indicates the instruction value among the setting value xA1 (0.2) and the setting value xA2 (0.6) related to the speech element [a_s]. The segment data V2 corresponding to the set value xA1 close to xB (0.5) is selected from the speech library L2 as the segment data VA. Therefore, the audio library L that is the target of the selection of the segment data VA by the segment selection unit 32 is a voice according to the instruction value xB from the variable instruction unit 38 (time series of the control variable X defined by the variable information DP). The library L1 and the audio library L2 are sequentially changed from one to the other.

  The segment processing unit 34 adjusts the segment data VA selected by the segment selection unit 32 to the pitch and sound generation period indicated by the music information DS. In addition, the method similar to 1st Embodiment can be employ | adopted for the process of the segment data V by the segment processing part 34. FIG. Further, the synthesis processing unit 36 generates an audio signal SOUT by concatenating the segment data DS processed by the segment processing unit 34.

  As described above, in the third embodiment, a plurality of audio libraries L (L1, L2) are selected according to the setting value xA defined by the audio library and the instruction value xB from the variable instruction unit 38. The segment data VA is selected and used to generate a synthesized sound. Therefore, without creating a new audio library L, an audio library reflecting the characteristics of both the audio library L1 and the audio library L2 (that is, alternatively extracted for each audio unit from the audio library L1 and the audio library L2). It is possible to generate various audio signals SOUT equivalent to the case where a virtual audio library composed of segment data V is used. That is, as in the first embodiment, it is possible to synthesize various voices as in the case of preparing a new voice library L while reducing the effort for creating the voice library L and the capacity required for the storage device 12. There is.

<D: Modification>
Each of the above forms can be variously modified. Specific modifications are exemplified below. Two or more aspects arbitrarily selected from the following examples can be appropriately combined.

(1) Modification 1
In each of the above embodiments, the configuration in which the speech library L includes one unit data V for each speech unit is illustrated for convenience, but a configuration in which the unit data V is further subdivided may be employed. For example, a configuration may be employed in which a plurality of segment data V having different acoustic attributes (hereinafter referred to as “segment attributes”) such as pitch (frequency) and volume are included in the speech library L for each speech segment. . The segment selection unit 32 selects the segment data V (for example, the segment attribute specified for the designated sound from among the plurality of segment data V corresponding to the speech segment of the phonetic character specified by the specified sound. The pitch unit data V) indicated by the music information DS is selected.

  According to the above configuration, there is an advantage that various synthesized sounds can be generated as compared with a configuration in which one unit data V is prepared for each speech unit. On the other hand, as the number of segment data V increases, the amount of data in the speech library L increases. Therefore, the effect of each embodiment that the synthesized speech can be diversified while reducing the capacity required for the storage device 12 is effective for speech segments. In addition, it is particularly prominent under the modified example 1 in which the segment data V is prepared according to the segment attributes (pitch and volume).

(2) Modification 2
In each of the above forms, information for defining the use of the segment data V in the speech synthesizer 26 (selection by the segment selection unit 32 or processing by the segment processing unit 34) (hereinafter referred to as “segment usage information”). Is set in the attached information A for each segment data V (for each speech segment). However, as illustrated below, a configuration in which the segment usage information is set in units of a plurality of segment data V may be employed. The element usage information is a concept that includes the section information P and the characteristic information F in the first embodiment, the mixing ratio R in the second embodiment, and the set value xA of the control variable X in the third embodiment.

  For example, a configuration is used in which segment usage information is set in the attached information A for each speech segment classification. As the classification of speech elements, classification based on phoneme structure (phoneme unit / phoneme chain), or classification based on the presence or absence of vowels or consonants is assumed. For example, for each of a set of segment data V of a plurality of speech units composed of a single phoneme and a set of segment data V of a plurality of speech units composed of a phoneme chain, element A includes Single usage information is set. The same unit usage information is applied to the unit data V of a plurality of speech units having a common classification.

  Further, a configuration in which common segment usage information is set for all the segment data V in the audio library L may be employed. For example, in the first embodiment, one section information P and one characteristic information F are commonly applied to all the piece data V. In the second embodiment, the segment data V1 in the speech library L1 and the segment data V2 in the speech library L2 are mixed at a common mixing ratio R regardless of the speech segment. In the third embodiment, either the segment data V1 of the speech library L1 or the segment data V2 of the speech library L2 is selected according to the common set value xA (xA1, xA2) regardless of the speech segment. Is done.

  When preparing the segment data V for each segment attribute such as pitch and volume as in the first modification, a configuration in which segment usage information is set for each speech segment regardless of the segment attribute, A configuration in which element usage information is set for each attribute is adopted. In the former configuration, common unit usage information is applied to a plurality of unit data V having a common speech unit regardless of the unit attribute. In the latter configuration, separate segment usage information is applied to each segment data V having different segment attributes even if the speech segment is common.

(3) Modification 3
The first to third embodiments can be appropriately merged as illustrated below. In the following description, a virtual audio library L composed of segment data VA obtained by applying attached information A to each segment data V in the audio library L will be referred to as a “virtual library” for convenience. As can be understood from the description of each embodiment, a set of segment data VA (speech library) corresponding to all speech units is not actually generated in the storage device 12 (that is, the speech library L). In consideration of the fact that the segment data VA is sequentially generated by applying the auxiliary information A to each segment data V of FIG.

  For example, in the first embodiment, a set of a plurality of segment data VA that can be generated on the assumption that the attached information A is applied to all the segment data V of the existing audio library L is stored in the virtual library LV. Equivalent to. In the second embodiment, the process of mixing the segment data V1 in the speech library L1 and the segment data V2 in the speech library L2 with the mixing ratio R is performed by using the segment data V1 and the segment data in which the speech segments are common. A set of a plurality of segment data VA obtained when it is assumed that all pairs with V2 are executed corresponds to the virtual library LV. Similarly, in the third embodiment, the process of selecting either the segment data V1 in the speech library L1 or the segment data V2 in the speech library L2 according to the set value xA (xA1, xA2) is performed. A set of a plurality of segment data VA obtained on the assumption that the processing is executed for all pairs of the segment data V1 and the segment data V2 having the same segment corresponds to the virtual library LV.

  First, as shown in FIG. 9, the virtual library LV3 is configured by applying the auxiliary information A3 of the second embodiment or the third embodiment to the virtual library LV1 and the virtual library LV2. The virtual library LV1 is configured, for example, by applying the auxiliary information A1 of the first embodiment including the section information P and the characteristic information F to the existing audio library L1. Similarly, the virtual library LV2 is configured by applying the auxiliary information A2 of the first embodiment to the existing audio library L2. Further, as shown in FIG. 10, the virtual library LV4 is configured by applying the auxiliary information A4 of the second embodiment or the third embodiment to the virtual library LV1 and the existing audio library L2. As described above, various virtual libraries LV corresponding to voices having various characteristics can be constructed by appropriately merging the first to third embodiments.

  Further, as shown in FIG. 11, a configuration in which a plurality of attached information A (A1, A2) is prepared for an existing audio library L may be employed. The virtual library LV1 is constructed by applying the attached information A1 to the audio library L, and the virtual library LV2 is constructed by applying the attached information A2 to the audio library L. That is, a virtual library LV corresponding to the number of attached information A is generated from one audio library L.

(4) Modification 4
In the first embodiment, the case where the section information P of the accessory information A designates the use section of the segment waveform W of the segment data V is exemplified. However, for example, continuous pronunciation is instructed by the music information DS. A configuration in which the section information P designates a section of the segment data V used for vowel interpolation (crossfade) is also employed. For example, the voice of the phonetic character “Asa (morning)” is converted into speech units [#_a], [a], [a_s], [s_a], [a], [a_g], [g_a], [a_ Assuming the case of generating from [#], interpolation between the backward segment indicated by the segment information P in the speech unit [s_a] and the forward segment indicated by the segment information P in the speech unit [a_g] [A] is synthesized.

(5) Modification 5
In the third embodiment, the case where the variable information DP is stored in the storage device 12 is exemplified, but the method in which the variable instruction unit 38 indicates the instruction value xB of the control variable X is appropriately changed. For example, a configuration in which the variable instruction unit 38 instructs the instruction value xB in time series according to an input to the input device 14, or the variable instruction unit 38 sequentially receives the instruction value xB sequentially received from the communication network. A configuration instructed to 32 is also employed. That is, the variable instruction unit 38 is included as an element that sequentially indicates the instruction value xB of the control variable X, and the configuration in which the variable information DP is prepared in advance and stored in the storage device 12 may be omitted.

(6) Modification 6
In each of the above embodiments, the case where the segment data V indicates the segment waveform W is exemplified, but the content of the segment data V is appropriately changed. For example, information indicating the result of analyzing a speech unit by a known method (for example, information on the frequency spectrum of the speech unit) may be used as the unit data V.

(7) Modification 7
In each of the above embodiments, the music information DS is edited in response to an instruction from the user, but the editing of the music information DS can be omitted. That is, a configuration in which music information DS stored in advance in the storage device 12 or music information DS provided to the speech synthesizer 100 via a portable recording medium or communication network is used to generate synthesized sound is also employed. . Therefore, the information generation unit 24 in each of the above forms can be omitted.

100 ... speech synthesizer, 10 ... control device, 12 ... storage device, 14 ... input device, 16 ... display device, 18 ... sound emitting device, 22 ... display control unit, 24 ... information generation , 26... Speech synthesis unit, 32... Segment selection unit, 34... Segment processing unit, 36.

Claims (6)

And audio library containing a plurality of segment data indicating a speech segment, segment usage information defining the use of fragment data, a plurality of the units of one or more segment data as a unit in the audio library Storage means for storing attached information set for each of the objects ;
Segment selection means for sequentially selecting the segment data of the audio library according to music information indicating a time sequence of a designated sound;
Each piece data selected by the piece selection means is processed according to the piece usage information set in the piece data in the attached information;
A speech synthesizer comprising: synthesis processing means for synthesizing speech from the piece data processed by the piece processing means. The time series of the segment waveform of each segment data corresponding to the specified sound specified by the music information, the start point indicator indicating the start point of the used segment used for speech synthesis in each segment waveform, and the used segment An end point indicator indicating the end point of the display, and a display control means for moving each of the start point indicator and the end point indicator in accordance with an instruction from a user,
The segment usage information set for each segment data in the attached information includes segment information indicating a usage segment defined by the start point indicator and the end point indicator in the segment waveform of the segment data. ,
The speech synthesis apparatus according to claim 1, wherein the segment processing unit extracts a segment indicated by the segment information from segment data selected by the segment selection unit. The speech synthesis apparatus according to claim 2, wherein the display control unit displays an image indicating a time series of a designated sound indicated by the music information on the display device in parallel with the time series of the segment waveforms. The display control means displays a characteristic transition image showing a transition of the feature amount of each piece data corresponding to the designated sound specified by the music information, for each piece waveform, on a time axis common to the piece waveform. And display on the display device, edit the characteristic transition image according to instructions from the user,
The segment usage information set for each segment data in the attached information includes characteristic information indicating a feature amount according to a characteristic transition image of the segment data,
The speech synthesis apparatus according to claim 2 or 3, wherein the segment processing unit controls a feature amount of the segment data selected by the segment selection unit according to the characteristic information. The storage means stores the accessory information in which the unit usage information is set for each speech unit classification of each unit data in the speech library,
2. The segment processing means commonly applies the segment usage information set in the one classification in the attached information to processing the segment data of each speech segment belonging to one classification. The speech synthesizer according to claim 4. And audio library containing a plurality of segment data indicating a speech segment, segment usage information defining the use of fragment data, a plurality of the units of one or more segment data as a unit in the audio library A computer comprising storage means for storing auxiliary information set for each object ,
According to the music information indicating the time series of the specified sound, the unit data of the voice library is sequentially selected,
Each selected piece data is processed according to the piece use information set in the piece data in the attached information,
A speech synthesis method for synthesizing speech from the processed segment data.

Images