JP5604824B2 - Tempo information output device, sound processing system, and electronic musical instrument - Google Patents

Description

  The present invention relates to a tempo information output device that outputs tempo information indicating a performance tempo, an audio processing system using the tempo information output device, and an electronic musical instrument.

  Conventionally, MIDI has been widely used in the field of music production. Since MIDI data includes tempo information, it is easy to adjust the playback time (tempo). When synchronizing audio data and MIDI data, the audio data is recorded in synchronization with the MIDI data. Further, when using existing audio data, it is necessary to manually adjust the tempo information of the MIDI data so as to match the tempo of the audio data. However, when the tempo is changed in the middle of the audio data, it is very laborious to manually adjust the tempo information of the MIDI data.

  On the other hand, in recent years, a technique for using audio data and MIDI data in association with each other has also been developed. For example, Patent Document 1 shows an example in which digital audio data and MIDI data are used in association with each other.

  In recent years, it has also become possible to adjust the tempo of audio data by using a signal processing technique such as time stretching (see Patent Document 2).

JP 2003-316356 A JP 2003-280664 A

  However, in Patent Document 1, since MIDI data is stored in the LSB (Least Significant Bit) of audio data, it is associated when converted to compressed audio such as MP3 or emitted as an analog audio signal. There was a problem that information that was lost. Although there are applications that handle audio data and MIDI data, there is no general-purpose data format, which is not convenient.

  On the other hand, the time stretch as in Patent Document 2 is to extract beats from audio data and change the tempo of the entire song at the absolute beat timing. Tempo is not reflected. That is, as shown in FIG. 6 (A), in the actual performance, the performer does not perform the performance in accordance with the absolute beat timing, but performs a little early or conversely. . For this reason, if a beat is extracted from audio data and time stretched, and the tempo of the entire song is changed at an absolute beat timing as shown in FIG. 5B, the nuance of the performance is lost.

  Therefore, an object of the present invention is to provide a tempo information output device, a sound processing system, and an electronic musical instrument that output tempo information reflecting the performance tempo.

  The tempo information output device according to the present invention includes output means for outputting an audio signal generated in response to a performance operation of the performer. Tempo information indicating the performance tempo of the performer is superimposed on the audio signal. The tempo information output device superimposes the tempo information so that the modulation component of the tempo information is included in a band different from the frequency component of the audio signal. Tempo information is superimposed as beat information (tempo clock) such as MIDI clock. This beat information is always output by an automatic performance system (sequencer).

  Therefore, the tempo information output device can output tempo information that reflects the performance tempo of the performer in the audio signal (in one line). Further, since the output audio signal can be handled in the same manner as a normal audio signal, it can be recorded by a recorder or the like and used as general-purpose audio data. Furthermore, the time difference from the actual performance timing can be obtained from the tempo information, and the nuance of the performance does not change even when the playback time is adjusted by time stretching or the like. Note that the tempo information output device is an aspect incorporated in an electronic musical instrument such as an electronic piano, an aspect in which an audio signal is input from an existing musical instrument, an aspect in which an acoustic instrument or singing sound is collected by a microphone and an audio signal is input , Etc.

  In the above invention, it is also possible to input a reference tempo signal as a reference of the performance tempo from the outside, such as a metronome, and extract tempo information based on the reference tempo signal. It is also possible to use a foot switch or the like to input a beat that the player performs as a reference tempo signal. In this case, tempo information can be extracted even if the tempo information cannot be generated by itself, such as an acoustic instrument.

  In addition, a mode is also possible in which an audio processing system further includes a decoding device that decodes the tempo information using the tempo information output device according to claim 1 or 2. The superimposing means of the tempo information output device superimposes the tempo information by superimposing pseudo noise on the audio signal at a timing based on the performance tempo. As the pseudo noise, a signal having high autocorrelation such as a PN code is used. The tempo information output device generates a signal with high autocorrelation at a timing based on the performance tempo (for example, for each beat) and superimposes the signal on the audio signal. Therefore, even if sound is output as an analog audio signal, the superimposed tempo information is not lost.

  The decoding apparatus includes input means for inputting the audio signal and decoding means for decoding tempo information. The decoding unit obtains a correlation between the audio signal input to the input unit and the pseudo noise, and decodes the tempo information based on the peak generation timing of the correlation. Since the pseudo noise superimposed on the audio signal is a signal having a very high autocorrelation, when the correlation between the audio signal and the pseudo noise is obtained by the decoding device, a correlation peak is extracted at each beat timing. . Therefore, the correlation peak generation timing indicates the performance tempo.

  Pseudo noise with high autocorrelation such as PN code can extract the peak of correlation even at low level, so it is highly accurate even though it is sound that is not uncomfortable on hearing (sound that is difficult to hear). Tempo information can be superimposed and decoded. Further, if pseudo noise is superimposed only on a high frequency band of 20 kHz or higher, it is possible to make it more difficult to hear.

  Furthermore, in the above invention, the tempo information extracting means extracts a plurality of different tempo information (for example, beat timing, measure timing) according to each timing of the performance tempo, and the superimposing means superimposes a plurality of different pseudo noises. In this way, the plurality of different tempo information can be superposed respectively. In this case, the decoding unit of the decoding apparatus obtains correlations between the audio signal input to the input unit and the plurality of different pseudo noises, and determines the plurality of different tempos based on the peak generation timing of each correlation. Decrypt information. In other words, by superimposing different patterns of pseudo-noise at beat timing and measure timing, it is possible to superimpose and decode beat timing and measure timing separately with high accuracy without interfering with each other's pseudo-noise.

  In addition, when superimposing tempo information using pseudo noise, the tempo information output device can also perform encoding by phase-modulating M-sequence pseudo noise (PN code) with tempo information. The frequency band for superimposing the tempo information is desirably a non-audible range of 20 kHz or higher. However, in the case where the non-audible range cannot be used due to D / A conversion or compression audio encoding, for example, a high frequency band of 15 kHz or higher is used. The tempo information is superimposed on the sound to reduce the effect on hearing.

  According to the present invention, tempo information that is a reference for the performance of the performer is extracted and superimposed on the audio signal, so that tempo information reflecting the performance tempo can be output.

It is a block diagram which shows the structure of the tempo information output device which concerns on embodiment of this invention. It is a block diagram which shows the structure of a decoding apparatus. It is a block diagram which shows the structure of the tempo information output device which concerns on an application example, and a decoding apparatus. It is a block diagram which shows the structure of the electronic piano which incorporated the sequencer. It is a figure which shows the example at the time of attaching a tempo information output device to an acoustic guitar. It is a figure explaining time stretch.

  FIG. 1A is a block diagram showing a configuration of a tempo information output apparatus according to an embodiment of the present invention. FIG. 1A shows an example in which an electronic musical instrument (electronic piano) also serves as a tempo information output device. An electronic piano 1 shown in FIG. 1A includes a control unit 11, a performance information acquisition unit 12, a musical sound generation unit 13, a data superposition unit 14, an output interface (I / F) 15, a tempo clock generation unit 16, and a metronome sound generation. Unit 17, mixer unit 18, and headphone I / F 19.

  The performance information acquisition unit 12 acquires performance information according to the performance operation of the performer. The performance information includes, for example, information on the pressed key (note number), key pressing timing (note on, note off), key pressing speed (velocity), and the like. Which performance information is output (which music information is used to generate a musical sound) is instructed by the control unit 11.

  The musical sound generator 13 has a built-in sound source, and generates musical sounds (audio signals) by inputting performance information from the performance information acquisition unit 12 in accordance with an instruction (setting of a volume or the like) from the control unit 11. The audio signal of the present invention is not limited to an analog signal, and may be a digital signal.

  The tempo clock generation unit 16 generates a tempo clock corresponding to the set tempo. The tempo clock is, for example, a clock conforming to a MIDI clock (24 clocks per 4th note) and is always output. The tempo clock generation unit 16 outputs the generated tempo clock to the data superimposition unit 14 and the metronome sound generation unit 17. The metronome sound generator 17 generates a metronome sound according to the input tempo clock. The metronome sound is mixed with the musical sound produced by the performer in the mixer unit 18 and output to the headphone I / F 19. The performer performs while listening to the metronome sound (tempo) heard from the headphones.

  The electronic piano 1 is provided with an operator dedicated to inputting tempo information (a tap switch or the like, a tempo information input unit indicated by a broken line in the figure), and a beat engraved by the performer is input as a reference tempo signal to extract tempo information. It is also possible to configure. When performing automatic accompaniment on a musical instrument equipped with an automatic performance system (sequencer), the tempo clock generator 16 also outputs the tempo clock to the automatic performance system (see, for example, FIG. 4).

  The data superimposing unit 14 superimposes the tempo clock on the audio signal input from the musical sound generating unit 13. The superimposing method uses a technique that makes it difficult to hear the superimposed signal. For example, by phase-modulating a high-frequency carrier signal with tempo information (a data code string indicating code 1 at the clock timing), the frequency of the tempo information is different from the frequency component (acoustic signal component) of the audio signal. Ingredients should be included.

  Alternatively, a technique of superimposing pseudo noise such as a PN code (M series) at a weak level that does not cause a sense of incongruity in hearing may be used. At this time, the band on which the pseudo noise is superimposed may be limited to a band outside the audible range (20 kHz or more). Since pseudo-noise such as M-sequence has a very high autocorrelation, the tempo clock can be extracted by obtaining the correlation between the audio signal and the same code as the superimposed pseudo-noise on the decoding side. it can. In addition, you may use other random numbers, such as not only M series but Gold series. Each time the tempo clock is input from the tempo clock generation unit 16, the data superimposing unit 14 generates a predetermined length of pseudo noise, superimposes it on the audio signal, and outputs it to the output I / F 15.

  Furthermore, when using pseudo noise, the following spectrum spread can be used. FIG. 1B is a block diagram illustrating an example of the configuration of the data superimposing unit 14 when using spread spectrum.

  In this example, the multiplier 265 multiplies the M-sequence pseudo noise code (PN code) output from the spread code generator 144 and the tempo information (0, 1 data code string), thereby spreading the tempo information. To do. The spread tempo information is input to the XOR circuit 146. The XOR circuit 146 outputs an exclusive OR of the code input from the multiplier 145 and the output code one sample before input via the delay unit 147, and differentially encodes the spread tempo information. The signal after differential encoding is binarized by -1,1. By outputting the differential code binarized to −1 and 1, the decoding side can extract the tempo information after spreading by multiplying two consecutive differential codes.

  The differentially encoded tempo information is band-limited within the baseband by an LPF (Nyquist filter) 148 and input to the multiplier 150. Multiplier 150 multiplies the carrier signal output from carrier signal generator 149 (a carrier signal higher in frequency than the acoustic signal component) and the output signal of LPF 148, and frequency-shifts the tempo information after differential encoding to the passband. . Note that the tempo information after differential encoding may be frequency shifted after upsampling. The tempo information after the frequency shift is gain-adjusted by the gain adjuster 151, mixed with the audio signal by the adder 143, and output to the output I / F 27.

  The audio signal output from the tone generator 13 is input to the adder 143 after the passband band is cut by the LPF 141 and the gain is adjusted by the gain adjuster 142. However, the LPF 141 is not indispensable. It is not necessary to completely divide the signal component and the modulated signal component (frequency component of the superimposed tempo information). For example, if the carrier signal is about 20 to 25 kHz, even if the acoustic signal component and the modulation signal component are slightly overlapped, the SN ratio is such that the listener cannot hear the modulation signal and can decode the tempo information. Can be secured. Note that the frequency band for superimposing the tempo information is preferably set to a non-audible range of 20 kHz or more. However, in a configuration in which the non-audible range cannot be used due to D / A conversion, compressed audio encoding, or the like, the frequency band is set to, for example, 15 kHz or higher. By superimposing tempo information on the frequency band, it is possible to reduce the influence on hearing.

  The audio signal on which the tempo information is superimposed as described above is output from the output I / F 15 which is an audio output terminal.

  The audio signal output from the output I / F 15 is input to the decoding device 2 as shown in FIG. The decoding device 2 has a function as a recorder that records an audio signal, a function as a playback device that reproduces the audio signal, and a function as a decoder that decodes tempo information superimposed on the audio signal. ing. Since the audio signal output from the electronic piano 1 can be handled in the same manner as a normal audio signal, it can be recorded with another general recording device. Further, since the recorded audio data is general-purpose audio data, it can be reproduced by a general audio player.

  Here, the decoding apparatus 2 will be described mainly with respect to a function for decoding tempo information superimposed on an audio signal and a usage mode of the decoded tempo information.

  In FIG. 2A, the decoding device 2 includes an input I / F 21, a control unit 22, a storage unit 23, and a tempo clock extraction unit 24. The control unit 22 records the audio signal input from the input I / F 21 and records it as general-purpose audio data in the storage unit 23. In addition, the control unit 22 reads the audio data recorded in the storage unit 23 and outputs it to the tempo clock extraction unit 24.

  The tempo clock extraction unit 24 generates the same pseudo noise as the pseudo noise generated by the data superimposing unit 14 of the electronic piano 1 and obtains a correlation with the reproduced audio signal. Since the pseudo noise superimposed on the audio signal is a signal having a very high autocorrelation, when the correlation between the audio signal and the pseudo noise is obtained, as shown in FIG. Peaks are extracted. This correlation peak generation timing indicates the performance tempo (tempo clock).

  When using the spread spectrum described with reference to FIG. 1B, the tempo clock extraction unit 24 decodes the tempo information and extracts the tempo clock as follows. FIG. 2C is a block diagram illustrating an example of the configuration of the tempo clock extraction unit 24. The input audio signal is input to the HPF 241. The HPF 241 is a filter for removing acoustic signal components. The output signal of the HPF 241 is input to the delay unit 242 and the multiplier 243. The delay amount of the delay unit 242 is set to a time corresponding to one sample of the above-described differential code. When the differential code is upsampled, it is set to the time for one sample after the upsampling. The multiplier 243 multiplies the signal input from the HPF 241 by the signal one sample before output from the delay unit 242, and performs delay detection processing. The differentially encoded signal is binarized to −1 and 1, and indicates the phase change from the code one sample before. Therefore, by multiplying the signal one sample before, the difference is obtained. Tempo information (code after spreading) before moving encoding is extracted.

  The output signal of the multiplier 243 is extracted as a baseband signal through the LPF 244 that is a Nyquist filter, and is input to the correlator 245. The correlator 245 obtains the correlation with the input signal using the same pseudo noise code as the pseudo noise code output from the spread code generator 244. From the correlation value output by the correlator 245, the peak detector 246 extracts positive and negative peak components in the pseudo-noise period (data code period). The code determination unit 247 decodes each peak component as a data code (0, 1) of tempo information. In this way, the tempo information superimposed on the audio signal is decoded. Note that the superposition-side differential encoding process and the decoding-side delay detection process are not essential.

  The tempo clock extracted as described above can be used for automatic performance by a sequencer as long as it conforms to the MIDI clock. For example, an automatic performance reflecting its own performance tempo can be realized by a sequencer.

  Furthermore, as shown in FIG. 4, in the electronic piano 5 incorporating the sequencer 101, if the sequencer 17 is configured to perform automatically based on the tempo information, the music played by the performer is synchronized with the music played automatically. Can take. Therefore, the performer can generate an audio signal in which the musical sound of his / her performance and the musical sound of the automatic performance are synchronized only by performing a performance operation. Moreover, it is also possible to synchronize with a video signal like a karaoke apparatus.

  Also, the extracted tempo clock can be used as a reference clock when performing time stretching of audio data, and the complexity at the time of editing can be greatly reduced. As shown in FIG. 6C, the correction time is calculated from the difference between the tempo information and the performance information included in the original audio data to be time-stretched, and the time-stretched audio data corresponding to the new tempo is obtained. By adding the correction time, it is possible to change the tempo without losing the nuance of the performance. For example, if the difference between each beat of the tempo information and the note-on timing is α, the base tempo is T1, and the tempo after time stretching is T2, the correction time is α × (T2 / T1). Therefore, the nuance of the performance does not change even when time stretching is performed.

  In addition, in the case of a superimposition method using pseudo noise such as an M series, the following application examples are also possible. FIG. 3 is a block diagram showing configurations of a tempo information output device and a decoding device according to an application example. In addition, about the structure which is common in FIG. 1 and FIG. 2, the same code | symbol is attached | subjected and the description is abbreviate | omitted.

  The electronic piano 3 according to the application example includes a strong beat tempo clock generator 161 and a weak beat tempo clock generator 162 instead of the tempo clock generator 16. The decoding device 4 includes a strong tempo clock extraction unit 241 and a weak beat tempo clock extraction unit 242 instead of the tempo clock extraction unit 24.

  The strong beat tempo clock generator 161 generates a tempo clock for each strong beat (measure) timing. In addition, the weak beat tempo clock generation unit 162 generates a tempo clock for each weak beat (beat) timing.

  The data superimposing unit 14 generates pseudo noise every time a tempo clock is input from the strong beat tempo clock generating unit 161 and every time a tempo clock is input from the weak beat tempo clock generating unit 162, and is superimposed on the audio signal. . The data superimposing unit 14 has different patterns of pseudo-noise (pseudo-pseudo noise and strong-pseudo pseudo-noise and timing) when the tempo clock is input from the strong beat tempo clock generation unit 161 and when the tempo clock is input from the weak beat tempo clock generation unit 162. Pseudo noise for weak beats) is generated.

  The strong beat tempo clock extraction unit 241 and the weak beat tempo clock extraction unit 242 of the decoding device 4 generate the same pseudo noise as the strong beat pseudo noise and the weak beat pseudo noise generated in the data superimposing unit 14, respectively. The correlation with the reproduced audio signal is obtained.

  In the audio signal, the strong beat pseudo noise is superimposed at every bar timing and the weak beat pseudo noise is superimposed at every beat timing. Since these are signals with very high autocorrelation, the audio signal and pseudo noise are When the correlation is obtained, a steep peak is periodically extracted as shown in FIG. The peak generation timing extracted by the strong beat tempo clock extraction unit 241 indicates the bar timing (strong beat tempo clock), and the peak generation timing extracted by the weak beat tempo clock extraction unit 242 indicates the beat timing (weak beat tempo clock). Will be shown. Since these pseudo noises use different patterns, correlations can be calculated with high accuracy without interference between the pseudo noises.

  As for the bar timing, the period of four beats is four times the beat timing, so that the noise length of the pseudo noise can be set to four times. Therefore, the S / N ratio can be ensured correspondingly, and the level of pseudo noise can be lowered.

  Further, by using a larger number of pseudo noise patterns, different pseudo noises can be superimposed at each beat timing, and various tempos such as composite time signatures can be handled. In particular, when the Gold sequence is used as the pseudo noise, it is possible to generate many types of code sequences. Therefore, even when there are a large number of composite time signatures and time signatures, different code sequences may be used for each beat. Is possible.

  Even in the case of using the spread spectrum described with reference to FIGS. 1B and 2C, the tempo information can be spread using pseudo-noise that differs at each beat timing and measure timing.

  Note that the tempo information output device of the present invention is not limited to a mode built in an electronic musical instrument, and can be attached to an existing musical instrument later. FIG. 5 is a diagram showing an example when the tempo information output device is attached to a guitar. In the same figure, an explanation will be given on an electric guitar that outputs an analog audio signal. In addition, about the structure which is common in FIG. 1, the same code | symbol is attached | subjected and the description is abbreviate | omitted.

  As shown in FIGS. 1A and 1B, the tempo information output device 9 includes an audio input I / F 51 and a foot switch 52, and the audio input I / F 51 has a line output of the guitar 7. Terminal is connected.

  The audio input I / F 51 inputs a performance sound (audio signal) from the guitar 7 and outputs it to the data superimposing unit 14. The foot switch 52 is an operator dedicated to inputting tempo information, and inputs a beat engraved by the performer as a reference tempo signal. The tempo clock generator 16 inputs a reference tempo signal from the foot switch 52 and extracts tempo information.

  In this way, any musical instrument having an existing audio output terminal can use the tempo information output device of the present invention, and can superimpose tempo information reflecting the performance tempo of the performer on the audio signal. it can.

  Note that the tempo information output device of the present invention is not limited to an example attached to an electronic piano or an electric acoustic guitar. If musical sounds are collected by a normal microphone, the tempo information output device of the present invention can be used even for an acoustic instrument without a line output terminal. Further, not only musical instruments but also singing sounds are included in the technical scope of audio signals generated according to the performance operation in the present invention, and singing sounds are collected by a microphone and tempo information is superimposed. Is also possible.

1-electronic piano 11-control unit 12-performance information acquisition unit 13-music sound generation unit 14-data superimposition unit 15-output I / F
16-tempo clock generator

Claims (5)

An audio processing system comprising a tempo information output device and a decoding device for decoding tempo information,
The tempo information output device includes tempo information extracting means for extracting the tempo information indicating the beat timing,
Superimposing means for superimposing the tempo information on the audio signal so that a modulation component of the tempo information is included in the audio signal generated according to the performance operation of the performer;
An output means for outputting an audio signal on which the tempo information is superimposed;
Equipped with a,
The superimposing means of the tempo information output device superimposes the tempo information by superimposing pseudo noise on the audio signal at a timing based on the beat timing,
The decoding device includes input means for inputting the audio signal;
Decoding means for obtaining a correlation between the audio signal input to the input means and the pseudo noise, and decoding the tempo information based on a peak occurrence timing of the correlation;
The tempo information extraction means of the tempo information output device extracts a plurality of different tempo information according to each timing of the beat timing,
The superimposing means superimposes the plurality of different tempo information by superimposing a plurality of different pseudo noises,
The decoding means of the decoding device obtains the correlation between the audio signal input to the input means and the plurality of different pseudo noises, and obtains the plurality of different tempo information based on the peak occurrence timing of each correlation. A speech processing system for decoding. 2. The sound processing system according to claim 1, wherein the tempo information extraction unit inputs a reference tempo signal as a reference for a performance tempo from the outside, and extracts the tempo information based on the reference tempo signal. The superimposing means of the tempo information output device includes a spreading code generator that generates the pseudo noise as a spreading code having a predetermined period;
Based on the tempo information, a modulation unit that phase-modulates the spreading code for each period;
A synthesis unit that synthesizes a modulation signal generated based on the phase-modulated spreading code with the audio signal in a frequency band higher than the frequency of the audio signal, and outputs the synthesized signal as a synthesized signal. The speech processing system according to claim 1 or 2 . Tempo information extracting means for extracting tempo information indicating the performance tempo;
And superimposing means for superimposing the tempo information to contain modulated component of the tempo information to the audio signal generated in response to the player's performance operation on the audio signal,
An output means for outputting an audio signal on which the tempo information is superimposed;
A tempo information output device comprising:
The superimposing means superimposes the tempo information by superimposing pseudo noise on the audio signal at a timing based on the performance tempo,
The tempo information extracting means extracts a plurality of different tempo information according to each timing of the performance tempo,
A superimposing unit superimposes the plurality of different tempo information by superimposing a plurality of different pseudo noises, respectively. An electronic musical instrument incorporating the tempo information output device according to any one of claims 1 to 4 .

Images