JP6477096B2 - Input device and sound synthesizer - Google Patents

Description

  The present invention relates to an input device using a multi-touch panel that can be operated by simultaneously touching a plurality of points.

  As a user interface for accepting a user input operation on an electronic device, a touch panel is generally popular. The touch panel is configured by attaching a transparent sheet-like contact detection sensor to a display surface of a display device such as a liquid crystal display. The user of the electronic device performs an intuitive and easy-to-understand operation such as touching the surface (hereinafter referred to as an operation detection surface) of the transparent contact detection sensor with a finger or a pen tip according to the guidance by the image displayed on the display device. Various inputs can be made. When a touch panel is used as a user interface for an electronic device, it is necessary to convert data representing the operation contents of the user, that is, data representing the touch position of the user into data corresponding to an application executed on the electronic device. For example, in the case of a singing voice synthesizing device that synthesizes a singing voice, the user's touch position is converted into data representing the pronunciation of the singing voice to be synthesized (for example, data representing a phoneme, pitch, volume, etc.), and the conversion result The singing voice is synthesized according to the situation.

“Angry Bird”, Internet, <URL: http://www.angrybirds.jp/>

  Conventional touch panels have mainly been those that can detect only one point even if the user touches a plurality of locations on the operation detection surface at the same time, and those that cannot correctly detect a contact location. However, in recent years, multi-touch panels capable of detecting contact at a plurality of locations are becoming popular. When a multi-touch panel is used as a user interface of an electronic device, it is expected that an application that has not existed before can be realized. For example, a collaborative application can be realized by a plurality of people operating simultaneously. In addition, if new information is made to correspond to the positional relationship between a plurality of touch positions, or the processing contents of the application can be changed according to changes in the touch positions, the range of applications to which the user interface can be applied is increased. spread. However, there has been no technology that makes new information correspond to the positional relationship of a plurality of touch positions, or changes the processing contents of an application in accordance with changes over time of the plurality of touch positions.

  The present invention has been made in view of the problems described above, and makes it possible to make information correspond to the relationship between a plurality of touch positions on the operation detection surface, and to change the positional relationship over time. An object of the present invention is to provide a technology that makes it possible to realize an application that changes processing contents in response.

  In order to solve the above-described problems, the present invention provides operation detection means for outputting operation content data indicating each of a plurality of touch positions by a touch operation performed by a user on an operation detection surface; When the data is converted into data representing the information and output, and one of the touch positions indicated by the operation content data is a start point and one of the touch positions other than the start point is an end point The predetermined position of the start point and the end point in the above, the other direction viewed from the predetermined direction, and the distance between them is converted into three types of information according to a predetermined rule, at least There is provided an input device comprising conversion means for outputting data representing a plurality of pieces of information including the three types of information.

  A specific example of the operation detection means is a multi-touch panel. If the input device of the present invention is used as an input means for an electronic device, in addition to the information determined according to the position of the start point or the end point, it corresponds to the other direction as viewed from a predetermined one of the start point and the end point. It is possible to output information determined according to the distance between them and information determined according to the distance between them (that is, information determined according to the positional relationship between the two) according to the operation on the operation detection surface. The operation detection means outputs the operation content data every time at least one touch position is updated, and the conversion means performs the conversion and output each time the operation content data is received. If so, an operation for changing the positional relationship between the start point and the end point with time, i.e., tracing either one or both of the fingertip indicating the start point and the fingertip indicating the end point on the operation detection surface. The information is updated over time according to the operation to be moved, and the processing content can be changed according to the update of the information on the application side that performs some processing using the information. In the computer game disclosed in Non-Patent Document 1, it is possible to specify the initial speed and direction when throwing the character by dragging while touching the character, but it is possible to specify changes in the initial speed and direction over time. It is not a translation but a technique different from the present invention.

  As a more preferable aspect, the reference value or recommended value relating to at least one of the predetermined position of the start point and the end point, the other direction viewed from the predetermined direction, and the distance between the two is set by the user. It is conceivable to provide the input device with notification means for informing the input device. This is because, according to such an aspect, the user can perform an operation of designating the start point or the end point while referring to the reference value or the recommended value notified by the notification unit.

  In another preferred embodiment, the operation detection means generates operation content data representing the touch start time in addition to each touch position, and the conversion means causes the time difference between the start touch start time and the end touch start time. Can be converted into fourth information different from the three types of information and output. According to such an aspect, it becomes possible to make information correspond to the positional relationship and the time relationship of a plurality of touch positions on the operation detection surface, and it becomes possible to perform more diverse information input. In addition, you may utilize the said time difference for determination of a starting point instead of making it respond | correspond to 4th information. For example, if one of the two touch positions specified with a time difference is set as the start point and the other is set as the end point, if the time difference is less than a predetermined threshold, the previously specified one is set as the start point, and the time difference is If is greater than or equal to a predetermined threshold, the one specified later is used as the starting point.

  In the present invention, the type of information associated with a predetermined one of the starting point and the ending point is determined according to the type of electronic device used as the input device of the present invention. Examples of the electronic device to which the input device of the present invention is applied include a sound synthesizer such as a singing synthesizer and a computer game machine. Of course, the input device of the present invention may be used as an input device for a map application or an input device for a computer game. For example, an aspect in which the input device of the present invention is used as input means of a sound synthesizer such as a synthesizer, in other words, a singing voice or a musical instrument performance sound according to the input device of the present invention and output data of the input device. In the aspect of providing a sound synthesizer having a sound synthesizer for synthesizing the sound, the position of the predetermined point of the start point and the end point, the other direction viewed from the predetermined direction, and the distance between the two are determined. It is only necessary to associate with a plurality of pieces of information defining at least the synthesis target sound (at least three pieces of information of tone color or pronunciation, pitch, and volume). Further, in the aspect in which the time difference between the touch times of the start point and the end point is associated with the fourth information, the fourth information may be associated with the velocity at the time of outputting the synthesized sound.

  In an aspect in which the input device of the present invention is used as an input unit of a sound synthesizer, a pitch is associated with a direction in which the other one of the start point and the end point is viewed from the predetermined point, and the conversion unit has a predetermined pitch. Correspondence between the direction and pitch to convert the data indicating the position of the start point and end point to data indicating the pitch so that the pitch changes by one octave each time the other rotates by a predetermined angle. It is preferable to perform the attachment. The other direction seen from the predetermined one of the start point and the end point can be expressed by an angle in polar coordinates centered on the predetermined direction, and the expression by the angle is in music theory such as equal temperament. This is because it has high affinity with pitch expression.

  As another aspect for solving the above-described problem, a program that causes a computer such as a CPU (Central Processing Unit) to function as the conversion unit and the sound synthesis unit, that is, the computer functions as the sound synthesis device of the present invention. An aspect of providing a program is conceivable. Similarly, an aspect of providing a program that causes a computer to function as the conversion means, that is, a program that causes the computer to function as the input device of the present invention is also conceivable. As specific provision modes of these programs, the above programs are written on a computer-readable recording medium such as a CD-ROM (Compact Disk-Read Only memory), a DVD (registered trademark: Digital Versatile Disc), or a flash ROM. A distribution mode or a distribution mode via a telecommunication line such as the Internet can be considered.

It is a block diagram which shows the structural example of the song synthesis apparatus 10 which is one Embodiment of the sound synthesizer of this invention. It is a figure which shows an example of the screen displayed on the display means 110a of the user interface part 110 of the song synthesizing | combining apparatus 10. FIG. It is a figure for demonstrating input operation with respect to the song synthesizing | combining apparatus. It is a figure for demonstrating the state transition implement | achieved when the control part 100 of the song synthesizing | combining apparatus 10 operate | moves according to the song synthesis program 124a. It is a flowchart which shows the flow of the process performed by the detection of a touch start among the processes which the control part 100 performs according to the song synthesis program 124a. It is a flowchart which shows the flow of the process performed by the detection of completion | finish of a touch among the processes which the control part 100 performs according to the song synthesis program 124a. It is a figure which shows an example of the source code of the program which implement | achieves angle unwrapping It is a figure for demonstrating a modification (3). It is a figure for demonstrating a modification (4). It is a figure for demonstrating a modification (8). It is a figure for demonstrating a modification (8). It is a figure for demonstrating a modification (8). It is a figure for demonstrating a modification (8).

Embodiments of the present invention will be described below with reference to the drawings.
(A: Configuration)
FIG. 1 is a diagram illustrating a configuration example of a singing voice synthesizing apparatus 10 according to an embodiment of the sound synthesizing apparatus of the present invention. The singing voice synthesizing apparatus 10 is obtained by installing a singing voice synthesizing program on a tablet computer device having a size (for example, B5 size) that a user can hold and operate. The user of the song synthesizing device 10 can perform various operations with the other hand while holding the song synthesizing device 10 with one hand. The singing voice synthesizing apparatus 10 synthesizes a singing voice according to a user operation and emits the sound. As shown in FIG. 1, the singing voice synthesizing apparatus 10 includes a control unit 100, a user interface (hereinafter referred to as “I / F”) unit 110, a storage unit 120, and a bus 130 that mediates data exchange between these components. have.

  The control unit 100 is a CPU, for example. The control unit 100 functions as a control center of the singing voice synthesizing apparatus 10 by executing the singing voice synthesis program 124a stored in the storage unit 120. Details of processing executed by the control unit 100 in accordance with the song synthesis program 124a will be clarified later.

  The user I / F unit 110 includes a display unit 110a, a sound output unit 110b, and an operation detection unit 110c. The display means 110a is a liquid crystal panel having substantially the same size as the casing of the singing voice synthesizing apparatus 10. The sound output means 110b outputs a sound corresponding to sound data given from the control unit 100 (for example, a sample sequence representing a sound sampling waveform: a sample sequence of singing voice in this embodiment). Although not shown in FIG. 1, the sound output means 110b includes a D / A converter that converts the sound data into an analog signal and a speaker that is driven by the output signal of the D / A converter.

  The operation detection unit 110c is a transparent sheet-like contact detection sensor (ie, a transparent contact detection sensor slightly smaller than the B5 size) attached so as to cover the entire display surface of the display unit 110a. Since the operation detection unit 110c is transparent, the user can visually recognize the display content of the display unit 110a through the operation detection unit 110c. Although details will be described later, a screen that prompts the user to use the singing voice synthesizing device 10 under the control of the control unit 100 is displayed on the display unit 110a. The user performs an operation on the operation detection surface in response to guidance on the screen displayed on the display unit 110a. The operation detection unit 110c outputs operation content data representing the operation content when a touch start operation or touch end operation is performed on the surface (that is, the operation detection surface) by the user's fingertip or the like. Note that the touch end operation refers to releasing the touched fingertip or the like from the operation detection surface.

  The operation content data includes position data indicating a touch start position or a touch end position, and operation type data indicating an operation type (touch start or touch end). In this embodiment, as position data, the origin of the touch start in the two-dimensional coordinate system when the upper left corner of the operation detection surface is the origin, the horizontal scanning direction of the display unit 110a is one coordinate axis, and the vertical scanning direction is the other coordinate axis. Data indicating coordinates such as position is used. When the touch on another position is started in a state where a touch is made on a position on the operation detection surface, the operation detection unit 110c performs position data indicating the latter position and operation type data indicating the touch start. Operation contents data including is output. That is, the operation detection unit 110c and the display unit 110a form a multi-touch panel.

  As illustrated in FIG. 1, the storage unit 120 includes a volatile storage unit 122 and a nonvolatile storage unit 124. The volatile storage unit 122 is, for example, a RAM (Random Access Memory). The volatile storage unit 122 is used by the control unit 100 as a work area when executing various programs. The nonvolatile storage unit 124 is a flash ROM or a hard disk. In addition to the song synthesis program 124a, the nonvolatile storage unit 124 stores OS software (not shown in FIG. 1) for causing the control unit 100 to implement an OS (Operating System).

  The control unit 100 reads the OS software from the non-volatile storage unit 124 to the volatile storage unit 122 and starts executing it when a power source (not shown in FIG. 1) is turned on or reset. The control unit 100 operating according to the OS software and realizing the OS can execute another program in accordance with a user instruction. When an instruction to execute the song synthesis program 124a is given via the user I / F unit 110, the control unit 100 reads the song synthesis program 124a from the nonvolatile storage unit 124 to the volatile storage unit 122 and starts its execution.

  The control unit 100 operating in accordance with the singing synthesis program 124a is configured to generate the singing voice according to the synthesis instruction data instructing the pronunciation of the singing voice to be synthesized (in this embodiment, the phoneme of the singing voice), the pitch, and the volume. It functions as speech synthesis means for synthesizing waveform data and outputting it to the sound output means 110b. In addition, the control unit 100 operating according to the singing synthesis program 124a gives the user a plurality of pieces of information (phonemes, pitches, and volumes) that define the singing voice to be synthesized by operating the operation detecting unit 110c. A screen for designating is displayed on the display means 110a.

  More specifically, the control unit 100 operating according to the song synthesis program 124a displays the screen shown in FIG. 2 on the display means 110a in order to allow the user to specify a plurality of pieces of information that define the song voice to be synthesized. Let As shown in FIG. 2, a small circle A01 to A05 and a great circle A06 passing through the center of each of the small circles A01 to A05 are drawn on this screen so as to be the vertices of a regular pentagon. In each of the small circles A01 to A05, letters “a”, “e”, “i”, “o”, and “u” representing vowels are drawn. The user who visually recognizes the screen shown in FIG. 2 defines the singing voice to be synthesized by performing an operation for designating one vector on the operation detection plane (that is, an operation for designating the start point and end point of the vector). Multiple information can be specified.

  In the present embodiment, a plurality of vectors cannot be specified simultaneously for the operation detection unit 110c. That is, even when a touch operation is performed on another position in a state where one vector is specified (a state where the user holds the start point and end point of the vector with a fingertip), the touch operation is ignored. In the present embodiment, the user performs a touch operation in a state in which no vector is specified or in a state in which only the end point is specified (a state in which the start point and the end point are specified to cancel the start point specification). When done, the touch position is interpreted as the starting point of the vector.

  More specifically, the user can designate a vowel corresponding to the small circle as a phoneme of the singing voice to be synthesized by designating one of the small circles A01 to A05 as a starting point. In the present embodiment, a small circle to which a vowel is assigned is used as a representation that prompts the user to specify the phoneme of the singing voice to be synthesized. However, other representations may be used. In addition, the user can specify the pitch in the direction of the end point viewed from the start point, and the volume in the distance between the start point and the end point. For example, in order to instruct the synthesis of a singing voice that utters the vowel “a” at a certain pitch and volume, a vector as indicated by the arrow B01 in FIG. 3 is designated (that is, the start point and the end point are designated). What is necessary is just to designate the vector as shown by arrow B02 of the same figure, when instruct | indicating the synthesis | combination of the loud sound of singing. Then, when specifying the synthesis of a singing voice having a pitch higher than that of the singing voice indicated by the vector indicated by the arrow B01 in FIG. 3, a vector represented by the arrow B03 in FIG. 3 may be specified.

  When an operation is performed on the operation detection unit 110c in a state where the screen illustrated in FIG. 2 is displayed on the display unit 110a, the control unit 100 writes data corresponding to the operation content in the volatile storage unit 122. As a result, the internal state of the singing voice synthesizing apparatus 10 sequentially changes. As the internal state of the singing voice synthesizing apparatus 10, there are four states shown in FIG. That is, the initial state S01 in which neither the start point nor the end point is specified, the start point stored state S02 in which only the start point is specified, the end point stored state S03 in which only the end point is specified, both the start point and the end point are specified There are four types of states: a start point / end point stored state S04. The control unit 100 executes the process shown in FIG. 5 triggered by the detection of the touch start on the operation detection surface of the operation detection unit 110c, while executing the process shown in FIG. 6 triggered by the detection of the touch end. Realize transitions between types of states.

  FIG. 5 is a flowchart showing a flow of processing executed in response to detection of touch start. The control unit 100 that has detected the start of touch by analyzing the operation content data received from the operation detection unit 110c first determines whether or not the start point position data indicating the start point is stored in the volatile storage unit 122 (step). SA100). In the present embodiment, when the operation type data included in the operation content data means “touch start”, the control unit 100 detects the touch start and performs the determination in step SA100.

  If the determination result in step SA100 is “No”, the control unit 100 sets the position indicated by the position data included in the operation content data as the start point position, and stores the start point position data indicating the position of the start point. The starting point position data is written in the volatile storage unit 122 (step SA110), and the process ends. Thereby, the transition T01 from the initial state S01 to the start point stored state S02 or the transition T07 from the end point stored state S03 to the start / end point stored state S04 is realized. On the other hand, when the determination result in step SA100 is “Yes”, the control unit 100 determines whether or not the end point position data indicating the end point position is stored in the volatile storage unit 122 ( Step SA120).

  If the determination result in step SA120 is “No”, the control unit 100 sets the position indicated by the position data included in the operation content data as the end point position, and stores end point position data indicating the end point position (step SA130). Then, the process ends. Thereby, the transition T02 from the start point stored state S02 to the start point / end point stored state S04 is realized. If the determination result in step SA120 is “Yes”, the control unit 100 ends the process without storing the touch start period position. This is for ignoring the touch operation even if another position is touched while the start point and the end point are specified.

  FIG. 6 is a flowchart showing a flow of processing executed in response to detection of the end of touch. The control unit 100 that has detected the end of the touch by analyzing the operation content data received from the operation detection unit 110c first determines whether the touch at the start point has been released (step SB100). If the determination result in step SB100 is “Yes”, the control unit 100 deletes the starting point position data from the volatile storage unit 122 (step SB110), and ends the process. Thereby, the transition T04 from the start point stored state S02 to the initial state S01 or the transition T05 from the start point / end point stored state S04 to the end point stored state S03 is realized.

  On the other hand, when the determination result of step SB100 is “No”, the control unit 100 determines whether or not the end point touch is released (step SB120). If the determination result in step SB120 is “Yes”, the control unit 100 deletes the end point position data from the volatile storage unit 122 (step SB130), and ends the process. Thereby, the transition T03 from the start point / end point stored state S04 to the start point stored state S02 or the transition T06 from the end point stored state S03 to the initial state S01 is realized. If the determination result in step SB120 is “No”, the control unit 100 ends the process. In the present embodiment, even if a touch operation that designates another point other than the start point and the end point is performed, the touch operation is ignored, so that the determination result in step SB120 does not become “No”.

  When the transition T02 from the start point stored state S02 to the start point / end point stored state S04 or the transition T07 from the end point stored state S03 to the start point / end point stored state S04 occurs, one vector is determined. For this reason, when the transition to the start point / end point stored state S04 occurs, the control unit 100 converts the data representing the vector into synthesis instruction data according to a predetermined rule, and gives it to the sound synthesis means. That is, the control unit 100 operating according to the song synthesis program 124a functions as a conversion unit that converts operation content data into synthesis instruction data according to a predetermined rule.

  The conversion means of this embodiment uses the position of the start point stored in the start point / end point stored state S04 as the phoneme of the singing voice to be synthesized, the direction of the end point viewed from the start point as the pitch of the singing voice, Is converted into the volume of the singing voice, and synthesis instruction data for instructing synthesis of the singing voice having the phoneme, the pitch and the volume is output to the sound synthesis means. As a result, the singing voice having the above phoneme, pitch and volume is output from the sound output means. With respect to the singing sound that has started to be output in this manner, the control unit 100 stops the output when the state transition from the start point / end point stored state S04 to another state occurs.

  Various data conversion rules in the conversion means can be considered, and specific examples are as follows. First, regarding the conversion rule to the phoneme of the starting point position, range data (on the circumference in the two-dimensional coordinate system described above) on the circumference of each of the small circles A01 to A02 on the screen shown in FIG. And a conversion rule table storing vowel data indicating vowels corresponding to the small circle in association with the small circle) is incorporated in the singing synthesis program 124a, and the stored contents of this conversion rule table and the start point position data What is necessary is just to implement | achieve by contrast with the starting point position which shows. For example, when the starting point position is on the circumference of one of the small circles or in a region, the conversion unit converts the vowel represented by the vowel data associated with the small circle in the conversion rule table to be synthesized. This is the phoneme of the singing voice. When the starting point position does not belong to or within the circumference of the small circles A01 to A05, it may be interpreted that the vowel corresponding to the closest small circle is designated, and the closest small circle and the second It may be interpreted that a phoneme between vowels corresponding to each of small circles close to is designated. When the starting point position does not belong to or within the circumference of the small circles A01 to A05, the sum of the reciprocals of the distance between the center of the small circle corresponding to each vowel and the starting point is 1. The phonetic pronunciation obtained by morphing each vowel with the normalized value as the weight of each vowel is interpreted as instructed, and the position of the starting point is converted to the phoneme (phoneme obtained by the morphing) Also good.

For the conversion to the pitch in the direction of the end point viewed from the start point, and the conversion to the volume of the distance between the start point and the end point, the conversion means first sets the position of the end point to polar coordinates (r, r, θ). When the coordinates of the start point are (x1, y1) and the coordinates of the end point are (x2, y2: where x2 ≠ x1, y2 ≠ y1), the conversion means performs polar coordinate conversion according to the following equations 1 and 2. Realize. In Equation 2, atan () is an arc tangent function, x = x 2 −x 1, y = y 2 −y 1, and π is a circumference ratio. Next, the conversion means converts the value of r calculated according to Equation 1 into a volume using a conversion rule table (that is, a conversion rule table that associates the value of r with the volume), and θ calculated according to Equation 2 Is converted into a pitch using a conversion rule table (that is, a conversion rule table that associates the value of θ with the pitch). These conversion rule tables may be incorporated in the song synthesis program 124a in advance.

  The value of θ calculated according to Equation 2 is limited to a range of −π to π. Here, if the time interval of touch position detection is sufficiently small, it can be considered that no change in angle beyond the threshold θth occurs between the previous touch position detection and the current touch position detection. Under such circumstances, the value of θ can be increased in both positive and negative directions by applying so-called unwrapping with the threshold θth = π. In addition, as a concrete implementation algorithm of unwrapping, for example, an algorithm shown by the source code of FIG. By applying such unwrapping, the pitch changes by one octave every time the end point is rotated by a predetermined angle with the start point as the center (for example, when rotating by a predetermined angle counterclockwise) By setting the above angle θ and pitch conversion rules so that the pitch becomes 1 octave higher and the pitch becomes 1 octave lower when rotated clockwise by a predetermined angle) It is possible to express pitches with high affinity with music theory. In consideration of the fact that a range of about two octaves is required when playing a normal music piece, it is preferable to set the predetermined angle to π radians. This is because an intuitively easy-to-understand operation such as turning the end point around the start point makes it possible to secure a 2-octave range necessary for normal music performance. When playing a musical piece that requires a wider range, the predetermined angle is set to π / 2 radians (that is, an angle corresponding to a quarter turn) or π / 4 radians (that is, 1 / Needless to say, it may be set to an angle of 8 laps.

  As described above, according to the present embodiment, a plurality of pieces of information defining the singing voice (phonemes of lyrics, pitches and volumes when sounding the lyrics) are displayed on a plurality of touch positions (books). In the embodiment, it is possible to correspond to the positional relationship between the two points of the start point and the end point. Further, as the operation detection unit 110c, one that outputs the operation content data every time at least one touch position is updated is used, and the start point or the end point is updated according to the operation content data, and the updated start point is obtained. If the control unit 100 is caused to execute the process of generating and outputting the synthesis instruction data according to the position or the like, either one or both of the fingertip that indicates the start point and the fingertip that indicates the end point will be performed. It is possible to update the plurality of information over time according to the operation to move the operation detection surface, that is, to change the singing voice over time, and to easily apply acoustic effects such as vibrato. It becomes possible to do.

  In the present embodiment, the application of the present invention to the singing voice synthesizing apparatus has been described. However, the present invention may be applied to an apparatus that synthesizes a reading voice such as a sentence or an apparatus that synthesizes a performance sound of a musical instrument. For example, in the case of a sound synthesizer that synthesizes performance sounds of musical instruments, a timbre may be associated with the position of the starting point. Even in a sound synthesizer that synthesizes a text-to-speech speech or a sound synthesizer that synthesizes a performance sound of a musical instrument, the tone color or phoneme, the pitch and volume specification, and the time This is because it is preferable that the update can be performed by an intuitive and easy-to-understand operation.

(B: Deformation)
Although the embodiment of the present invention has been described above, the following modifications may be added.
(1) In the above embodiment, the sound generation start time is defined as the generation timing of the transition from the start point stored state S02 to the start point / end point stored state S04 or the transition from the end point stored state S03 to the start point / end point stored state S04. However, the generation timing of the transition from the initial state S01 to the start point stored state S02 may be set as the sound generation start time. In the start point stored state S02, the end point is not specified, and the pitch and volume of the singing voice to be synthesized cannot be determined. For this reason, in the aspect in which the generation timing of the transition from the initial state S01 to the start point stored state S02 is set as the sound generation start time, the pitch and volume may be set in advance, or a pseudo random number or the like may be used. It may be a predetermined value.

  In addition, when the singing voice synthesizing apparatus 10 has other sensors such as an acceleration sensor or other operation detection means such as an operator, the pitch is determined according to the data acquired through the other operation detection means. Also, the volume may be determined. Similarly, for the sound generation end time, the start point / end point stored state is not set as the sound generation end time when the transition from the start point / end point stored state S04 to the start point stored state S02 or the end point stored state S03 occurs. The timing of transition from S04 to the initial state S01 through either the start point stored state S02 or the end point stored state S03 may be set as the sound generation end time. When the start point designation is canceled in the start point / end point stored state S04, the previous end point may be changed to the start point stored state S02 as a new start point.

(2) In the above embodiment, the position of the start point specified by the touch operation on the operation detection unit 110c is the phoneme of the singing voice to be synthesized, the direction of the end point viewed from the start point is the pitch of the singing voice, and the start point and the end point Although the distance between is made to correspond to the volume of singing voice, it is not necessarily limited to this. For example, the position of the start point may correspond to the volume, the direction of the end point viewed from the start point may correspond to the phoneme, the distance between the start point and the end point may correspond to the pitch, or the roles of the start point and end point may be interchanged good. Also, the one specified earlier in time may be the end point, and the one specified later may be the start point. When a start point and an end point are specified, a single vector is specified. Therefore, a plurality of pieces of information that define the vector (information indicating the magnitude, direction, and position of the vector) are defined as the singing voice to be synthesized. Any mode may be used as long as it corresponds to a plurality of information. In addition to the start point and the end point, the user may designate a third point that is different from the end point and different from the end point, and the positional relationship between these three points may correspond to a plurality of pieces of information. Also, specify the above vector such that the time interval when sequentially specifying the start point and end point corresponds to the velocity of the singing voice (for example, the velocity is increased as the time interval when specifying each of the start point and end point is shorter). Information other than the information to be used may be used for the synthesis of the singing voice.

  In general, a singing voice takes a certain amount of time from the start of pronunciation until it reaches an acoustically steady state. It takes some time. Velocity is information used mainly for the purpose of transmitting transient characteristics at the start and end of pronunciation in the MIDI standard used for communication of music performance information. Conventionally, in a musical instrument application or the like for a tablet terminal such as a smartphone, when the tablet terminal has an acceleration sensor, the output value of the acceleration sensor is often used as a velocity. However, in such a mode, there is a problem that an intended velocity cannot be input when playing while moving the tablet terminal itself, and an input value changes depending on how the tablet terminal is held. On the other hand, if the time interval for sequentially specifying the start and end points of the vector is made to correspond to the velocity, there is no problem even if the device itself is moved, and the influence of how the device itself is held. Can be reduced.

(3) In the above-described embodiment, the pronunciation of the singing voice to be synthesized is designated within the range of vowels. However, more various pronunciations (or timbres) may be designated by using buttons together. For example, when the screen shown in FIG. 8 is displayed on the display unit 110a and the touch operation is performed in which the point in the small circle A01 corresponding to the vowel [a] is specified as the start point while the [S] button A08 is touched. Alternatively, consonant / s / may be added to synthesize a sound with / sa / pronunciation. In addition, in order to enable sound synthesis using various lyrics (or timbres), the phoneme (or timbre) candidates to be displayed on the display means 110a are switched each time the transition to the start / end point stored state S04 occurs. May be. Further, when the present invention is applied to a singing voice synthesizing apparatus, the pronunciation of the singing voice to be synthesized may be designated by phonetic symbols instead of phonemes. In this case, a phonetic symbol may be displayed instead of a character indicating a phoneme inside the small circle A01 or the like. In short, any mode may be used as long as a pronunciation character (phoneme character or phonetic symbol) representing pronunciation is displayed on the small circle A01 or the like to prompt the user to specify a pronunciation. Further, when the pronunciation of the singing voice to be synthesized is designated, the pronunciation may be designated in syllable units, and the pronunciation may be designated in mora units if the pronunciation is designated in the Japanese range.

(4) The sound synthesizer (that is, the song synthesizer 10) of the above embodiment includes an operation detection unit that detects an operation that specifies a plurality of information that defines a singing voice, and an operation that is detected by the operation detection unit. It had conversion means for converting data representing contents into synthesis instruction data, sound synthesis means for synthesizing singing voice according to the synthesis instruction data, and sound output means. However, the sound output means may be an external element as shown in FIG. 9A, such as using headphones as the sound output means. That is, the sound output means is not an essential component of the sound synthesizer of the present invention. As for the operation detecting means, the converting means, and the sound synthesizing means, any one of these may be mounted on different devices, and all three of them may be executed on different devices. May be. For example, as shown in FIG. 9 (b), the operation detection means and the conversion means form an input device that converts data representing the user's operation contents into synthesis instruction data and outputs the combined instruction data. Synthesis instruction data may be given from the apparatus to the voice synthesis means. Further, the input device in FIG. 9B may be manufactured and sold alone.

  Further, as shown in FIG. 9 (c), the first computer device having the operation detecting means 110c and the sound output means 110b is connected to an electric communication line such as the Internet and functions as the converting means and the voice synthesizing means. Connecting the second computer device to the telecommunications line, converting the operation content data transmitted from the first computer device via the telecommunications line into a synthesis instruction, and the waveform data corresponding to the synthesis instruction You may make it make the said 2nd computer apparatus perform a synthesis | combination and the reply to the 1st computer apparatus of the waveform data. FIG. 9C illustrates the case where two first computer devices are connected to a telecommunication line. According to such an aspect, real-time synthesis of singing voice can be provided by a so-called ASP communication service. A plurality of song synthesizing apparatuses 10 are connected to an electric communication line, and one of the plurality of song synthesizing apparatuses 10 is made to function as a first computer apparatus and a second computer apparatus, and the other one is set as a first one. The same effect can be obtained when the computer apparatus is made to function.

  Further, a plurality of the first computer devices are connected to an electric communication line to perform simultaneously, and a vector input to each of the first computer devices is changed to a second one other than the device input with the vector. The information may be displayed on one computer device, and the reply from the second computer device may be sent by multicast to the plurality of computer devices. By doing so, it becomes possible for each user of the plurality of first computer devices to visually grasp the musical relationship of the singing voice designated by each user through the positional relationship of each vector. .

  For example, if pitches and vector angles are associated with each other so that a rotation of 2π radians coincides with a pitch change of 1 octave (ie, 1200 cent), (i) vectors having equal pitches are parallel. (Ii) Two vectors having a pitch ratio of octave, that is, a multiple of 1200 cent, are parallel, (iii) Vectors having the same timbre have the same visual aesthetics. It is done. More generally speaking, with respect to the pitch, there is an effect that it is possible to visually grasp that the relationship between the pitches is a special value by the angle made by the vector. The occurrence of any of the above events (i) to (iii) is detected by the second computer device and notified to each of the plurality of first computer devices, and a notification (vector display) corresponding to the notification is made. Notification that appeals visually such as changing the aspect, notification by sound, or notification by vibration may be performed.

(6) In the above embodiment, a substantially B5 size sheet-like contact detection sensor is used as the operation detection unit 110c. However, a larger contact detection sensor such as one tatami mat may be used. When using such a large contact detection sensor, the start point and the end point may be designated by placing the contact detection sensor on the floor or the like and stepping on the operation detection surface with a foot. In such an aspect, the contact detection sensor does not need to be transparent, and may be anything in which a plurality of representations (small circles A01 to A05 in FIG. 2) showing different pronunciations are drawn. According to such an aspect, it becomes possible to specify a plurality of information defining the singing voice to be synthesized by an operation such as stepping or dancing on the contact detection sensor, and singing is performed in a game sense. It becomes possible. Moreover, this aspect and the aspect of the above-described embodiment may be used in combination, and among the phonemes constituting the lyrics, the vowel may be specified according to this aspect, and the consonant may be specified according to the aspect of the above-described embodiment.

(7) In the above embodiment, the input device of the present invention (that is, the input device having the operation detecting means and the converting means) is used as the information input means to the sound synthesizer, but the portable game machine or the stationary type is used. You may use as an information input means to a game machine, and you may use as an information input means to a map application. For example, when the input device of the present invention is used as an information input means for a map application or an information input means for a role playing game, the end point viewed from the start point when the start point and the end point are specified on the map. An image of a landscape in the direction can be displayed at a zoom ratio corresponding to the distance between the start point and the end point, and road guidance can be performed.

(8) Information serving as a reference for designating a vector according to the type of application may be displayed. Specifically, a representation based on a certain rule such as a lattice as shown in FIG. 10 or a combination of a cross line and a concentric circle as shown in FIG. 11 is displayed as information indicating the reference. According to such an aspect, it is possible to perform an input operation while allowing the user to grasp the reference or reference interval for the position, size, and angle of the vector. In addition, as shown in FIG. 12, the display position of the representation indicating the reference for the size and angle follows the position of the vector being input, such as by positioning the center of the combination of the crosshair and the concentric circle at the starting point of the vector being input. You may let them. Further, the radius of each circle constituting the concentric circle may be gradually reduced (or increased) according to the elapsed time from the designated time at the start point.

  Further, as shown in FIG. 13, a symbol indicating a recommended value for any one of (or a plurality of) the position, direction, and length of the vector may be displayed. When the vector represented by is sufficiently close, it may be notified to the user. Specifically, depending on the degree of approximation between the vector being input and the vector represented by the representation, a mode or sound in which either (or both) the display mode of the vector being input and the vector represented by the representation is different is displayed. A mode in which the notification is made by vibration or vibration can be considered. Further, the display mode of the representation representing the recommended value may be changed according to the elapsed time from the designated time of the starting point of the vector being input.

(9) In the above embodiment, the converting means and the sound synthesizing means are realized by software modules, but may be realized by hardware such as an electronic circuit. Moreover, in the said embodiment, the program which functions the control part 100 as a conversion means was contained in the song synthesis program 124a, However, You may provide with the program single body which functions the control part 100 as a conversion means. This is because by operating the control unit of the computer device having a multi-touch panel in accordance with the program, the computer device can function as the input device of the present invention. As a specific mode of providing the program, a mode in which the program is recorded on a computer-readable recording medium such as a CD-ROM or a DVD and a mode in which the program is distributed by downloading via a telecommunication line are considered. It is done.

  DESCRIPTION OF SYMBOLS 10 ... Singing synthesis apparatus, 100 ... Control part, 110 ... User I / F part, 110a ... Display means, 110b ... Sound output means, 110c ... Operation detection means, 120 ... Memory | storage part, 122 ... Volatile memory part, 124 ... Non-volatile storage unit, 124a ... singing synthesis program, 130 ... bus.

Claims (6)

Operation detection means for outputting operation content data indicating each of a plurality of touch positions by a touch operation performed by the user on the operation detection surface;
Means for converting the operation content data output at the predetermined time interval into data representing other information and outputting the data, wherein a start point and an end point of a vector are designated among a plurality of touch positions indicated by the operation content data is a has been touched position starts in a state not the start point of the vector, the start point and the end point when the touch position is started in a state where the starting point of the vector other than the starting point is specified as the end point of the vector Of the predetermined direction , the other direction θ viewed from the predetermined direction , and the distance r between the two , and unwrapping the direction θ to obtain the value of the direction θ. Conversion means for expanding both positive and negative , converting into three types of information according to a predetermined rule, and outputting data representing a plurality of information including at least the three types of information; An input device comprising: Informing means for informing a user of a reference value or a recommended value relating to at least one of a predetermined position of the start point and the end point, the other direction viewed from the predetermined direction, and a distance between the two. The input device according to claim 1, further comprising: The operation detection means outputs operation content data representing a touch start time in addition to the touch position,
The input according to claim 1, wherein the conversion unit converts the time difference between the touch start time at the start point and the touch start time at the end point into fourth information different from the three types of information. apparatus. The converting means converts the predetermined position, the other direction viewed from the predetermined direction, and the distance between the two according to a predetermined rule according to a timbre or pronunciation, pitch and volume. The input device according to claim 3, wherein the time difference between the touch start time at the start point and the touch start time at the end point is converted into a velocity at the time of outputting the tone color or the sound of the pronunciation. A pitch is associated with the direction of viewing the other from the predetermined one of the start point and the end point,
The converting means is configured to position the start point and the end point so that the pitch changes by one octave each time the other of the start point and the end point rotates around a predetermined angle. The input device according to claim 4, wherein the data indicating the pitch is converted into the data indicating the pitch. In a sound synthesizer that synthesizes and outputs a timbre or sound corresponding to a user's operation as a sound having a pitch and volume according to the operation,
Operation detection means for outputting operation content data indicating each of a plurality of touch positions by a touch operation performed by the user on the operation detection surface;
A means for converting the operation content data into data representing other information and outputting the data, wherein one of a plurality of touch positions indicated by the operation content data is set as a start point and one of the touch positions other than the start point. The tone of each of the starting point and the predetermined position of the end point, the other direction as viewed from the predetermined direction, and the distance between the two when the two points are the end points. Or conversion means for converting to pronunciation, pitch and volume, and outputting synthesis instruction data for instructing to synthesize the tone or tone with the pitch and volume;
Sound synthesis means for synthesizing sound according to synthesis instruction data output by the conversion means;
A sound synthesizer characterized by comprising:

Images