KR101108955B1 - A method and an apparatus for processing an audio signal - Google Patents

Abstract

The present invention provides, by an audio processing apparatus, type information indicating a specific band extension scheme for a current frame of an audio signal among a plurality of band extension schemes including a first band extension scheme and a second band extension scheme, and a low frequency band. Receiving spectral data of; When the type information indicates the first band extension scheme with respect to the current frame, generating spectral data of the high frequency band of the current frame by using the spectral data of the low frequency band by performing the first band extension scheme. step; And when the type information indicates the second band extension scheme with respect to the current frame, performs spectral data of the high frequency band of the current frame by using the spectral data of the low frequency band by performing a second band extension scheme. Generating a first data area of the spectral data in the low frequency band, and the second band extension method in the second data area of the spectral data in the low frequency band. Based on

Audio, voice, band extension

Description

Audio signal processing method and apparatus {A METHOD AND AN APPARATUS FOR PROCESSING AN AUDIO SIGNAL}

The present invention relates to a signal processing method and apparatus capable of encoding or decoding an audio signal.

In general, an audio signal has a correlation between a signal of a low frequency band and a signal of a high frequency band within a frame. Based on this similarity, the spectral data of a high frequency band is used by using spectral data of a low frequency band. The audio signal is compressed using a band extension technique for encoding data.

Conventionally, when the similarity between the signal of the low frequency band and the signal of the high frequency band is low, if the audio signal is compressed by applying the band extension method, there is a problem that the sound quality of the audio signal is deteriorated.

In particular, in case of sibilant or the like, since the correlation is not high, the band extension method of the audio signal has an inappropriate problem.

Meanwhile, there may be various types of band extension schemes, and the type of band extension schemes applied to the audio signal may vary according to time. In this case, sound quality may deteriorate instantaneously in a section in which other types change.

The present invention has been made to solve the above problems, and to provide an audio signal processing method and apparatus that can selectively apply a band extension method according to the characteristics of the audio signal.

It is still another object of the present invention to provide an audio signal processing method and apparatus which can adaptively apply an appropriate method instead of a band extension method according to the characteristics of an audio signal for each frame.

It is still another object of the present invention to provide an audio signal processing method and apparatus which can maintain sound quality by avoiding application of a band extension method when the characteristics of an audio signal are close to hissing sound after analyzing the characteristics of the audio signal. have.

It is still another object of the present invention to provide an audio signal processing method and apparatus for applying various types of band extension schemes according to characteristics of an audio signal.

Another object of the present invention is to provide an audio signal processing method and apparatus for reducing artifacts in a section in which a type of band extension scheme is changed in applying various types of band extension schemes.

The present invention provides the following effects and advantages.

First, since the band extension scheme is selectively applied for each frame according to the characteristics of the signals for each frame, the sound quality can be improved without significantly increasing the number of bits.

Second, for frames deemed to contain high-energy sounds such as sibilant sounds, a linear predictive coding (LPC) method suitable for a speech signal or a high band extension instead of a band extension method is used. ), The loss of the sound quality can be minimized by using the newly proposed method (PSDD).

Third, according to the characteristics of the audio signal, it is possible to apply various types of band extension schemes by time, and furthermore, in applying various types of band extension schemes, artifacts are reduced in a section in which the type of band extension schemes is changed. Because of this, the sound quality can be improved while applying the band extension method.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Prior to this, terms or words used in the specification and claims should not be construed as having a conventional or dictionary meaning, and the inventors should properly explain the concept of terms in order to best explain their own invention. Based on the principle that can be defined, it should be interpreted as meaning and concept corresponding to the technical idea of the present invention. Therefore, the embodiments described in the specification and the drawings shown in the drawings are only the most preferred embodiment of the present invention and do not represent all of the technical idea of the present invention, various modifications that can be replaced at the time of the present application It should be understood that there may be equivalents and variations.

In the present invention, the following terms may be interpreted based on the following criteria, and terms not described may be interpreted according to the following meanings. Coding can be interpreted as encoding or decoding in some cases, and information is a term that encompasses values, parameters, coefficients, elements, and so on. It may be interpreted otherwise, but the present invention is not limited thereto.

Here, the audio signal is a concept that is broadly distinguished from the video signal, and refers to a signal that can be visually identified during reproduction. The narrow signal is a concept that is distinguished from a speech signal. Means a signal with little or no characteristics. The audio signal in the present invention should be interpreted broadly and can be understood as a narrow audio signal when used separately from a voice signal.

First, FIG. 1 is a diagram illustrating a configuration of an audio signal processing apparatus according to an embodiment of the present invention. Referring to FIG. 1, the encoder side 100 of the audio signal processing apparatus includes a sibilant detecting unit 110, a first encoding unit 122, and a second encoding unit. 124, a multiplexing unit 130. The decoder side 200 of the audio signal processing apparatus may include a de-multiplexer 210, a first decoding unit 222, and a second decoding unit 224. have.

The encoder side 100 of the audio signal processing apparatus determines whether to apply a band extension scheme according to the characteristics of the audio signal, and generates coding scheme information according to the determination, The decoder side 200 selects whether to apply a band extension scheme for each frame according to coding scheme information.

The sibilant detecting unit 110 detects a sibilant proportion with respect to the current frame of the audio signal, and based on the sibilant degree, coding indicating whether a band extension scheme is applied to the current frame. Create method information. Here, the sibilant proportion indicates the degree of sibilant in the current frame. Sibilant is a sound produced by the friction generated when air is passed through a narrow gap toward the teeth in general when pronunciation of a touch sound. For example, s in Korean and s in English are examples. On the other hand, the affricate is a sound produced by closing the airflow to prevent the flow of air and then opening it little by little, without passing it through, and passing air through narrow gaps. E.g. The sibilant herein refers to a phone that is not limited to a specific phone and that a peak band having the maximum energy belongs to a higher frequency band than other phones. The detailed configuration of the sibilant sound detection unit 110 will be described later with reference to FIG. 2.

As a result of the detection of the hissing sound level, the audio signal is encoded by the first encoding unit 122 in the frame determined to be low in the hissing sound, and the frame determined to be the high hissing sound in the second encoding unit ( The audio signal is encoded by a second encoding unit (124).

The first encoding unit 122 is a component for encoding an audio signal according to a frequency domain based band extension scheme. In this case, the band extension scheme based on the frequency domain means that the spectral data corresponding to the higher band of the spectral data of the wide band includes all or part of the narrow band ( encoding using all or portion). This method can reduce the number of bits based on the similarity between the high frequency band and the low frequency band. In this case, the band extension scheme is based on the frequency domain, and the spectral data is frequency-converted by a quadrature mirror filter (QMF) filter bank. The decoder recovers spectral data of a higher band from narrowband spectral data using the band extension information. Here, the high band is a band equal to or higher than the boundary frequency, and the narrow band is a band equal to or lower than the boundary frequency and is composed of consecutive bands. Such a frequency domain based band extension scheme may be based on Spectral Band Replication (SBR) or Enhanced Spectral Band Replication (eSBR) standards, but the present invention is not limited thereto.

Meanwhile, the frequency domain based band extension scheme is based on the similarity between the high frequency band and the low frequency band, which may be strong or weak depending on the characteristics of the audio signal. In particular, in the case of the sibilant described above, since the similarity is weak, when the band extension scheme is applied to a frame corresponding to the sibilant, sound quality may be degraded. The relationship between the energy characteristics of sibilant and the application of the band extension scheme based on the frequency domain will be described in detail later with reference to FIGS. 3 and 4. The first encoding unit 122 may be a concept including an audio signal encoder to be described later together with FIG. 8, but the present invention is not limited thereto.

The second encoding unit 124 is a unit for encoding an audio signal without using the frequency domain based band extension scheme. Here, not all types of band extension schemes are used, but a specific frequency domain based band extension scheme applied in the first encoding unit 122 is not used. The second encoding unit 124 firstly corresponds to a speech signal encoder applying a linear predictive coding (LPC) scheme, or secondly, further includes a module according to a time domain based band extension scheme as well as the speech encoder. In addition, the present invention may further include a module according to the PSDD (Partial Spectral Data Duplication) scheme, which will be described later with reference to FIGS. 5 to 8. On the other hand, the second time domain based band extension scheme may be based on the High Band Extension (HBE) scheme applied to the Adaptive Multi Rate WideBand (AMR-WB) standard, but the present invention is not limited thereto.

The multiplexer 130 multiplexes the audio signal encoded by the first encoding unit 122 and the non-band extension encoding unit 124 and the coding scheme information generated by the sibilant detecting unit 110. Generate one or more bitstreams.

The demultiplexer 210 on the decoder side extracts the coding scheme information from the bitstream, and based on the coding scheme information, decodes the audio signal of the current frame in a first decoding unit 222 or a second decoding unit (second). the decoding unit 224). The first decoding unit 222 decodes the audio signal according to the band extension scheme described above, and the second decoding unit 224 uses the LPC scheme (and HBE / PSDD scheme) to decode the audio signal.

FIG. 2 is a diagram illustrating a detailed configuration of a sibilant detecting unit in FIG. 1, a view for explaining the principle of the sibilant detection of FIG. 3, and FIG. 4 is a non-sibilant case. It is an example of the energy spectrum in the case of sibilant. First, referring to FIG. 2, the sibilant detection unit 110 includes a transform part 112, an energy estimating part 114, and a sibilant deciding part 116. Include.

The transform part 112 converts the audio signal in the time domain into a signal in the frequency domain by performing frequency transformation. In this case, a fast Fourier transform (FFT) or a Modified Discrete Cosine Transform (MDCT) may be used for the frequency conversion, but the present invention is not limited thereto.

The energy estimating part 114 bundles the audio signal of the frequency domain into several bands and calculates band-specific energy for the current frame. Then, it is determined which peak band B _max has the largest energy E _max among the entire bands. The sibilant deciding part 116 determines whether the band B _max having the largest energy is higher or lower than the threshold band B _th , and detects the sibilant degree of the current frame. . This is based on the characteristic that voiced sound has maximum energy at low frequency while sibilant sound has maximum energy at high frequency. Here, the threshold band B _th may be a predetermined value as a default value or a value calculated according to the characteristics of the input audio.

Referring to FIG. 3, it can be seen that there is a wide band including a narrow band and a high band. The peak band B _max with the highest energy E _max may be higher or lower than the threshold band B _th . Meanwhile, referring to FIG. 4, it can be seen that an energy peak of a non-sibilant signal exists in a low frequency band, and an energy peak of a sibilant signal exists in a relatively high frequency band. . Referring back to FIG. 3, in case of (A), since the energy peak is relatively low, it is determined to be non-sibilant, and in case of (B), the energy peak is relative. Because it exists at a high frequency can be determined as sibilant (sibilant).

On the other hand, the aforementioned frequency domain based band extension scheme encodes a higher band higher than the boundary frequency by using a narrow band lower than the boundary frequency. This method is based on the similarity between narrow band spectral data and high band spectral data. However, for signals with energy peaks at high frequencies, the correlation is relatively low. Therefore, applying a frequency domain-based band extension method that predicts spectral data of a higher band as narrow band spectral data may degrade sound quality. Therefore, for the current frame determined to be hissing sound, it is preferable to apply another method instead of the frequency domain based band extension method.

Referring back to FIG. 2, the sibilant deciding part 116 is a non-sibilizing current frame when the peak band B _max at the energy peak is lower than the threshold band B _th . sibilant, so that the audio signal is encoded according to the band extension scheme of the frequency domain in the first encoding unit, and in the opposite case, the audio signal is determined as a sibilant (second encoding unit). To be encoded in an alternative way.

FIG. 5 is an example of a detailed configuration diagram of a second encoding unit and a second decoding unit in FIG. 1. Referring to FIG. 5A, the second encoding unit 124a according to the first embodiment includes an LPC encoding part 124a-1, and the first embodiment The second decoding unit 224a in accordance with the LPC decoding part 224a-1. The LPC encoding part and the LPC decoding part are components that encode or decode an audio signal for an entire band in a linear prediction coding (LPC) scheme. Linear prediction coding (LPC) is a method of predicting a current sample value by multiplying a predetermined number of past sample values by a coefficient and adding the sum, and short term prediction for processing a speech signal based on a time domain. (STP) is a representative example. When the LPC coefficients (not shown) encoded in the LPC method are generated in the LPC encoding part 124a-1, the LPC decoding part 224a-1 reconstructs the audio signal using the LPC coefficients. .

Meanwhile, the second encoding unit 124b according to the second embodiment includes an HBE encoding part 124b-1 and an LPC encoding part 124b-2. In addition, the second decoding unit 224b according to the second embodiment includes an LPC decoding part 224b-1 and an HBE decoding part 224b-2. do. The HBE encoding part 124b-1 and the HBE decoding part 224b-2 are components for encoding / decoding an audio signal according to the HBE scheme. The HBE (High Bnad Extension) method is a kind of time domain based band extension method. The encoder generates HBE information, that is, spectral envelope modeling information and frame energy information, for the high frequency signal, and generates an excitation signal for the low frequency signal. In this case, the spectral envelope modeling information may correspond to the LP coefficient generated through linear prediction (LP) analysis, which is time domain-based, is converted into an implicit spectral pair (ISP). The frame energy information may correspond to information determined by comparing the original energy and the synthesized energy every 64 sub-frames. The decoder generates a high frequency signal by shaping an excitation signal of a low frequency signal using the spectral envelope modeling information and the frame energy information. This HBE scheme is distinguished from the frequency domain based band extension scheme described above in that it is based on the time domain. Sibilant is a very complex and random noise-like signal from a time-base waveform that can be very inaccurate when band-extended based on the frequency domain. Since HBE is based on the time domain, it can handle sibilants appropriately. Meanwhile, when the HBE method further includes post-processing for reducing buzzness of the high frequency excitation signal, performance may be further improved for the sibilant frame.

On the other hand, the LPC encoding part 124b-2 and the LPC decoding part 224b-1 are components 124a-1 and 224a-1 of the same name in the first embodiment. Performs the same function as However, in the first embodiment, the linear prediction encoding / decoding is performed for the entire band of the current frame, whereas in the second embodiment, the narrow band (or low frequency band) after the HBE is performed instead of the entire band. (lower band), linear prediction encoding is performed, and linear prediction decoding is performed on a narrow band, and then HBE decoding is performed.

Meanwhile, the second encoding unit 124c according to the third embodiment includes a PSDDD encoding part 124c-1 and an LPC encoding part 124c-2. The second encoding unit 224c according to the third embodiment includes an LPC decoding part 224c-1 and a PSDD decoding part 224c-2. . A frequency domain based band extension scheme performed in the first encoding unit 122 of FIG. 1 uses a part or all of a narrow band composed of a low frequency band. On the other hand, PSDD (Partial Spectral Data Duplication) is to encode a target band adjacent to the copy band by using a copy band distributed discretely in the low frequency and high frequency bands. . Details thereof will be described later with reference to FIGS. 6 to 9.

Meanwhile, the LPC encoding part and the LPC decoding part described above with reference to FIGS. 5A to 5C are speech signal encoders to be described with reference to FIGS. 9 to 12. encoder 440) and speech signal decoder 630, respectively.

FIG. 6 is a diagram for describing first to second embodiments of a partial spectral data duplication (PSDD) scheme, which is an example of a second encoding / decoding scheme.

First, referring to FIG. 6A, there are n scale factor bands sfb ₀ to sfb _{n −1} from low to high frequencies, that is, from 0th to n−1th, and each scale factor band sfb. Spectral data corresponding to ₀ ,, sfb _n-1 ). The spectral data (sd _i ) belonging to a specific band may mean a set of spectral data (sd _{i _0} to sd _{i _m-1} ), and the number of spectral data (m _i ) is a spectral data unit. Can be generated corresponding to a band unit or more units.

In this case, the band through which data is transmitted to the decoder includes a low frequency band (sfb ₀ ,, sfb _{s -1} ) and a copy band (cb) (sfb _s , sfb _n ) among the entire bands (sfb ₀ ,, sfb _{n -1} ). _-4 , sfb _{n -2} ). The copy bands are bands starting from the start band (sb) or the start frequency (start frequency), and of the target band (tb) (sfb _{s +1} , sfb _{n -3} , sfb _{n -1} ) A band used for prediction, and a target band is a band predicted using a copy band, and no spectral data is transmitted to the decoder.

As shown in FIG. 6A, the copy band is not concentrated in the low frequency band, but also exists in the high frequency band and is adjacent to the target band. Similarity can be maintained. Meanwhile, gain information g, which is a difference between the spectral data of the copy band and the spectral data of the target band, may be generated. Even when a target band is predicted using a copy band, it is possible to minimize the degradation of sound quality without increasing the bitrate as compared with the band extension method.

6A illustrates an example in which the bandwidth of the copy band and the bandwidth of the target band are the same, and FIG. 6B illustrates an example in which the bandwidth of the copy band and the bandwidth of the target band are different. Referring to FIG. 6B, the bandwidth of the target band is more than twice the bandwidth of the copy band (tb, tb '), where the left band (tb) of consecutive bands forming the target band is included. Different gains g _s and g _{s + 1} may be applied to the right band tb '.

7 and 8 are diagrams for describing a method in which frames have different lengths in the PSDD scheme. 7 is a case where the number (N _t) of the spectral data of the target band, the large copy number of band spectral data of (N _c) than, Figure 8 is a view for explaining a small case.

First, referring to FIG. 7A, it can be seen that the number of spectral data N _t of the target band sfb _i is 36, and the number of spectral data N _c of the copy band sfb _s is 24. Can be. The larger the number of data, the longer the horizontal length of the band. Since the data of the target band is larger, the data of the copy band can be used more than once. For example, as shown in (B1) of FIG. 7, first, 24 data of the copy band are filled from the low frequency of the target band, and as shown in (B2) of FIG. 7, the front part 12 of the copy band is shown. The dog or the back 12 can be filled in the rest of the target band. Of course, it is also possible to apply the transmitted gain information.

Meanwhile, referring to FIG. 8A, the number of spectral data N _t of the target band sfb _i is 24, and the number of spectral data N _c of the copy band sfb _s is 36. Able to know. Since the number of data in the target band is smaller, only some of the data in the copy band can be used. For example, as shown in FIG. 8B, only 24 spectral data of the front portion of the copy band sfb _s are used, or as shown in FIG. 8C, the back of the copy band sfb _s . Using only 24 spectral data in the region, the spectral data of the target band sfb _i can be generated.

9 is a first example of an audio signal encoding apparatus to which an audio signal processing apparatus according to an embodiment of the present invention is applied, and FIG. 10 is a second example. The first example is an encoding apparatus to which the first embodiment 124a of the second encoding unit described with reference to FIG. 5A is applied, and the second example includes FIGS. 5B and 5B. The encoding device to which the second embodiment 124b or the third embodiment 124c of the second encoding unit described with reference to (C) is applied.

Referring first to FIG. 9, referring to FIG. 9, an audio signal encoding apparatus 300 includes a multi-channel encoder 305, a sibilant detection unit 310, a first encoding unit 322, and audio. It may include a signal encoder 330, a voice signal encoder 340, and a multiplexer 350. The sibilant detecting unit 310 and the first encoding unit 320 may be identical to the functions of the components 110 and 122 of the same name described with reference to FIG. 1.

The multi-channel encoder 305 receives a plurality of channel signals (two or more channel signals) (hereinafter, referred to as a multi-channel signal) and performs downmixing to generate a mono or stereo downmix signal and multi-channel the downmix signal. Generates spatial information needed for upmixing to a signal. The spatial information may include channel level difference information, interchannel correlation information, channel prediction coefficients, downmix gain information, and the like. If the audio signal encoding apparatus 300 receives the mono signal, the multi-channel encoder 305 may bypass the mono signal without downmixing.

The sibilant detecting unit 310 detects the hissing degree of the current frame and transmits the audio signal to the first encoding unit 322 in the case of the non-sibilant sound, and in the case of the sibilant sound, the audio signal. The first encoding unit 322 is bypassed and passed to the speech signal encoder 340. As a result of indicating whether the band extension scheme is applied to the current frame, band extension information is generated and transmitted to the multiplexer 350.

The first encoding unit 322 generates a narrow band of spectral data and band extension information by applying a frequency domain based band extension method as described above with reference to the wideband audio signal. do.

The audio signal encoder 330 encodes the downmix signal according to an audio coding scheme when a specific frame or a specific segment of the downmix signal has a large audio characteristic. Here, the audio coding scheme may be based on an AAC standard or a high efficiency advanced audio coding (HE-AAC) standard, but the present invention is not limited thereto. The audio signal encoder 340 may correspond to a modified discrete transform (MDCT) encoder.

A speech signal encoder 340 encodes the downmix signal according to a speech coding scheme when a specific frame or a segment of the downmix signal has a large speech characteristic. Here, the speech coding scheme may be based on an adaptive multi-rate wide-band (AMR-WB) standard, but the present invention is not limited thereto. Meanwhile, the voice signal encoder 350 may further include linear prediction coding (LPC) encoding parts 124a-1, 124b-1, and 124c-1 as described above with reference to FIG. 5. When the harmonic signal has high redundancy on the time axis, the harmonic signal may be modeled by linear prediction that predicts the current signal from the past signal. In this case, the linear prediction coding method may increase coding efficiency. Meanwhile, the voice signal encoder 340 may correspond to a time domain encoder.

The multiplexer 350 generates an audio signal bitstream by multiplexing spatial information, coding scheme information, bandwidth extension information, and spectral data.

10 is an encoding to which the second embodiment 124b or the third embodiment 124c of the second encoding unit described with reference to FIGS. 5B and 5C is applied as mentioned above. An apparatus, which is substantially the same as the first example described with reference to FIG. 9, but before an audio signal corresponding to the entire band is encoded by the speech signal encoder 440, an HBE encoding part 424 (or PSDD). The difference is that the encoding part is encoded by the HBE method or the PSDD method. As described above with reference to FIG. 5, the HBE encoding part 424 encodes an audio signal according to a time domain based band extension scheme to generate HBE information. The HBE encoding part 424 may be replaced with a PSDD encoding part 424. The PSDD encoding part 424 is described with reference to FIGS. As described above, a target band is encoded using information of a copy band, and as a result, PSDD information for reconstructing a target band is generated. The speech signal encoder 440 encodes the result encoded by the HBE method or the PSDD method into the speech signal method. Of course, the voice signal encoder 440 may further include an LPC encoding part as in the first example.

11 is a first example of an audio signal decoding apparatus to which an audio signal processing apparatus according to an embodiment of the present invention is applied, and FIG. 12 is a second example. The first example is an encoding apparatus to which the first embodiment 224a of the second decoding unit described with reference to FIG. 5A is applied, and the second example includes FIGS. 5B and 5B. An encoding apparatus to which the second embodiment 224b or the third embodiment 224c of the second decoding unit described with reference to (C) is applied.

Referring to FIG. 11, the audio signal decoding apparatus 500 includes a demultiplexer 510, an audio signal decoder 520, a voice signal decoder 530, a first decoding unit 540, and a multi-channel decoder. 550.

The demultiplexer 510 extracts spectral data, coding scheme information, bandwidth extension information, spatial information, and the like from the audio signal bitstream. The audio signal corresponding to the current frame is transmitted to the audio signal decoder 520 or the voice signal decoder 530 according to the coding scheme information. Specifically, when the coding scheme information indicates that the band extension scheme has been applied to the current frame, the audio signal is transmitted to the audio signal decoder 520, and the coding scheme information indicates that the band extension scheme has not been applied to the current frame. In this case, the audio signal is transmitted to the voice signal decoder 530.

The audio signal decoder 520 decodes the spectral data by audio coding when the spectral data corresponding to the downmix signal has a large audio characteristic. As described above, the audio coding scheme may be based on the AAC standard and the HE-AAC standard. The audio signal decoder 520 may include an inverse quantizer (not shown) and an inverse transformer (not shown). Accordingly, the audio signal decoder 520 may perform inverse quantization and inverse transformation on spectral data and scale factors transmitted through the bitstream.

A speech signal decoder 530 decodes the downmix signal using a speech coding scheme when the spectral data has a large speech characteristic. As described above, the speech coding scheme may conform to the adaptive multi-rate wide-band (AMR-WB) standard, but the present invention is not limited thereto. As described above, the voice signal decoder 530 may include LPC decoding parts 224a-1, 224b-1, and 224c-1 as described with reference to FIG. 5.

The first decoding unit 540 decodes the bandwidth extension information bitstream and generates the high frequency band audio signal by applying the frequency domain based band extension scheme described above to the audio signal using the information.

When the decoded audio signal is downmixed, the multichannel decoder 550 generates an output channel signal of a multichannel signal (including a stereo signal) using spatial information.

12 is a decoding to which the second embodiment 224b or the third embodiment 224c of the second decoding unit described with reference to FIGS. 5B and 5C is applied as mentioned above. As an apparatus, it is almost the same as the first example described with reference to FIG. However, after the audio signal corresponding to the entire band is decoded by the speech signal encoder 630, the HBE decoding part 635 (or the PSDD decoding part) is applied to the HBE method or the PSDD method. The difference is that it is decoded by. As described above, the HBE decoding part 635 generates a high frequency signal by shaping a low frequency excitation signal using the HBE information. Meanwhile, the PSDD decoding part 635 reconstructs a target band using information of a copy band and PSDD information. The speech signal decoder 635 decodes the result encoded by the HBE scheme or the PSDD scheme into the speech signal scheme. Of course, the voice signal encoder 635 may further include LPC decoding parts 224a-1, 224b-1, and 224c-1 as in the first example.

The audio signal processing apparatus according to the present invention can be included and used in various products. These products can be broadly divided into stand alone and portable groups, which can include TVs, monitors and set-top boxes, and portable groups include PMPs, mobile phones, and navigation. can do.

FIG. 13 is a diagram illustrating a relationship between products in which an audio signal processing device according to an embodiment of the present invention is implemented. First, referring to FIG. 13, the wired / wireless communication unit 710 receives a bitstream through a wired / wireless communication method. In more detail, the wired / wireless communication unit 710 may include at least one of a wired communication unit 710A, an infrared communication unit 710B, a Bluetooth unit 710C, and a wireless LAN communication unit 710D.

The user authentication unit 720 receives user information and performs user authentication, and includes one or more of a fingerprint recognition unit 720A, an iris recognition unit 720B, a face recognition unit 720C, and a voice recognition unit 720D. The fingerprint, iris information, facial contour information, and voice information may be input, converted into user information, and the user authentication may be performed by determining whether the user information matches the existing registered user data. .

The input unit 730 is an input device for a user to input various types of commands, and may include one or more of a keypad unit 730A, a touch pad unit 730B, and a remote controller unit 730C. It is not limited.

The signal coding unit 740 encodes or decodes an audio signal and / or a video signal received through the wired / wireless communication unit 710 and outputs an audio signal of a time domain. An audio signal processing device 745, which corresponds to an embodiment of the present invention described above, and thus, the audio processing device 745 and the signal coding unit including the same may be implemented by one or more processors.

The controller 750 receives input signals from the input apparatuses and controls all processes of the signal decoding unit 740 and the output unit 760. The output unit 760 is a component that outputs an output signal generated by the signal decoding unit 740, and may include a speaker unit 760A and a display unit 760B. When the output signal is an audio signal, the output signal is output to the speaker, and when the output signal is a video signal, the output signal is output through the display.

14 is a relationship diagram of products in which an audio signal processing apparatus according to an embodiment of the present invention is implemented. FIG. 14 illustrates a relationship between a terminal and a server corresponding to the product illustrated in FIG. 13. Referring to FIG. 14A, the first terminal 700. 1 and the second terminal 700. It can be seen that the data to the bitstream can be bidirectionally communicated through the wired / wireless communication unit. Referring to FIG. 14B, it can be seen that the server 800 and the first terminal 700.1 may also perform wired or wireless communication with each other.

15 is a block diagram of an audio signal processing apparatus according to another embodiment of the present invention. Referring to FIG. 15, the encoder side 1100 of the audio signal processing apparatus includes a type determination unit 1110, a first band extension encoding unit 1120, a second band extension encoding unit 1122, and a multiplexer 1130. Include. The decoder side silver 1200 of the audio signal processing apparatus includes a demultiplexer 1210, a first band extension decoding unit 1220, and a second band extension decoding unit 1222.

The type determination unit 1110 analyzes the input audio signal to detect a transient proportion. The type determination unit 1110 distinguishes between a stationary section and a transient section, and based on the result, determines a specific type of band extension scheme for the current frame among two or more band extension schemes. And type information for identifying the determined manner. A detailed configuration of the type determination unit will be described later with reference to FIG. 16.

The first band extension encoding unit 1120 encodes the corresponding frame according to the first type of band extension method, and the second band extension encoding unit 1122 encodes the corresponding frame according to the second type of band extension method. do. The first band extension encoding unit 1122 may perform bandpass filtering, time stretching processing, decimation processing, or the like. A band extension method of the first type and a band extension method of the second type will be described in detail with reference to FIG. 16 and the like.

The multiplexer 1130 is configured to display spectral data of the lower band generated by the first and second band extension encoding units 1120 and 1122, type information generated by the type determination unit 1110, and the like. The multiplexer generates an audio signal bitstream. The demultiplexer 1210 of the decoder side 1200 extracts spectral data and type information of a low frequency band from the audio signal bitstream. The demultiplexer 1210 then transfers the current frame to the first band extension decoding unit 1220 or the second band extension decoding unit 1222, depending on which type of band extension scheme the type information indicates. The first band extension decoding unit 1220 decodes the current frame in reverse according to the first type of band extension scheme encoded by the first band extension encoding unit 1120. Furthermore, the first band extension decoding unit 1222 may perform bandpass filtering, time stretching processing, and decimation processing. Similarly, the second band extension decoding unit 1222 generates the high frequency band of spectral data by using the low frequency band of spectral data by decoding the current frame according to the second type of band extension method.

FIG. 16 is a diagram illustrating a detailed configuration of the type determination unit 1110 in FIG. 15. The type determination unit 1110 includes a transient detection part 1112 and a type information generation part 1114, and is associated with a coding scheme determination part 1140.

The transient detection part 1112 analyzes the energy of the input audio signal to distinguish between the stationary section and the transient section. The stationary section may be a section in which the energy of the audio signal is flat, and the transient section may be a section in which the energy of the audio signal changes rapidly. Since the transient section is a section in which energy changes rapidly, it is difficult for the listener to recognize artifacts that occur as the type of the band extension scheme changes. On the other hand, since the stationary section is a section in which sound flows smoothly, if the type of the band extension system is changed in this section, it may feel like the sound suddenly stops momentarily. Therefore, when it is necessary to change the type of the band extension method from the first type to the second type, if the type is changed in the transient section instead of the stationary section, the type is similar to the masking effect according to the psychoacoustic model. You can hide artifacts from changes.

As described above, the type information generation part 1114 determines a specific type of band extension method among two or more band extension methods, and generates type information indicating the determined band extension method for the current frame. More than one band extension method will be described later with reference to FIG. 18.

In order to determine a specific band extension scheme, first, the type of the band extension scheme is temporarily determined with reference to the coding scheme received from the coding scheme determination part 1140, and the information received from the transient detection part 1112 is referred to. In this case, the type of the band extension method is decided. Hereinafter, this will be described in detail with reference to FIG. 17.

17 is a diagram for describing a process of determining a type of a band extension method. Referring to FIG. 17, there are several frames f _i , f _n , and f _t along the time axis. A frequency domain based audio coding scheme (coding scheme 1) and a time domain based speech coding scheme (coding scheme 2) may be determined for each frame. That is, according to this coding scheme, a type of band extension scheme suitable for the coding scheme can be temporarily determined. For example, with respect to the frames f _i to f _{n -2} corresponding to the audio coding scheme 1, the band extension scheme of the first type is a frame corresponding to the speech coding scheme 2. For f _{n -1} to f _t ), the first type of band extension may be temporarily determined. Then, the type of the band extension scheme is finally determined by correcting the temporarily determined type with reference to whether the audio signal is a stationary section or a transient section. For example, as shown in FIG. 17, when the temporarily determined type of the band extension scheme is changed at the frame boundary of f _{n -2} to f _{n -1} , the f _{n -2} frame and the f _{n -1} frame are staggered. Because it is an awkward section, artifacts due to the change of the type of the band extension method are not hidden. Therefore, the type of the band extension scheme that is temporarily determined to be changed in the transition period f _n , f _{n +1} is corrected. In other words, since the f _{n -1} frame and the f _n frame are stationary intervals, the type of the band extension method is maintained as the first type as before, and the band extension method of the second type is applied from the f _{n +1} frame. will be. In other words, the temporarily determined type is retained in other than f _{n -1} frames and f _n frames, and is modified in the last step only for the frames.

18 is a diagram for describing various types of band extension methods. The band extension scheme of the first type to be described herein corresponds to the first band extension scheme described with reference to FIG. 15, and the band extension scheme to the second type to be described herein is the second band extension scheme described with reference to FIG. 15. It may correspond to. On the contrary, the first type of band extension scheme to be described herein corresponds to the second band extension scheme described above with reference to FIG. 15, and the second type of band extension scheme described above is described with reference to FIG. 15. It may correspond to a band extension method.

As described above, the band extension method generates wide-spectrum spectral data using narrow-band spectral data, where the narrow-band may correspond to a low frequency band, and the newly generated band may be a high frequency band ( higher band). First, referring to FIG. 18A, an example of the first type of band extension method is illustrated. The first band extension scheme restores a high frequency band by using a narrow band (or low frequency band) first data area as a copy band. Here, the first data region may be all of the received narrowband or may be a plurality of portions, where one portion may correspond to the second data region to be described below, and the first data region is the second data region. Can be greater than

18 (B) -1 and (B) -2, a first example (type 2-1) and a second example (type 2-2) of the second band extension scheme are shown. In the second type of band extension, a second data area is used as a copy band to restore the high frequency band. Here, the second data area may be a portion of the received narrow band and may be a smaller band than the first data area. Meanwhile, in the case of the first example of the second type, the copy bands cb used to generate the high frequency band are continuous, and in the case of the second example of the second type, the copy bands are not continuous and discrete. It is distributed.

19 is a diagram illustrating a configuration of an audio signal encoding apparatus to which an audio signal processing apparatus according to another embodiment of the present invention is applied. Referring to FIG. 19, an audio signal encoding apparatus 1300 may include a multichannel encoder 1305, a type determination unit 1310, a first band extension encoding unit 1320, a second band extension decoding unit 1322, and an audio signal. An encoder 1330, a voice signal encoder 1340, and a multiplexer 1350. Here, the type determination unit 1310, the first band extension encoding unit 1320, and the second band extension encoding unit 1322 correspond to the functions of the components 1110, 1120, and 1122 of the same name described with reference to FIG. 15. May be the same.

The multi-channel encoder 1305 receives a plurality of channel signals (two or more channel signals) (hereinafter, referred to as a multi-channel signal) and performs downmixing to generate a mono or stereo downmix signal and multi-channel the downmix signal. Generates spatial information needed for upmixing to a signal. The spatial information may include channel level difference information, interchannel correlation information, channel prediction coefficients, downmix gain information, and the like. If the audio signal encoding apparatus 1300 receives a mono signal, the multi-channel encoder 1305 may bypass the mono signal without downmixing.

The type determination unit 1310 determines the type of the band extension scheme to be applied to the current frame and generates type information indicating the type. The type determination unit 1310 transmits an audio signal to the first band extension encoding unit 1320 when the first band extension scheme is applied to the current frame, and a second band extension when the second band extension scheme is applied. The audio signal is passed to the encoding unit 1322. The first band extension encoding unit 1320 and the second band extension encoding unit 1322 generate band extension information for reconstructing a high frequency band by using a low frequency band by applying a band extension scheme according to each type. The signal encoded in the band extension method is then encoded by the audio signal encoder 1330 or the voice signal encoder 134 according to the characteristics of the signal, regardless of the type of the band extension method. The coding scheme information according to the characteristics of the signal may be information generated by the coding scheme determination part 1140 described above with reference to FIG. 16, which may be transmitted to the multiplexer 1350 like other information.

The audio signal encoder 1330 encodes the downmix signal according to an audio coding scheme when a specific frame or a specific segment of the downmix signal has a large audio characteristic. Here, the audio coding scheme may be based on an AAC standard or a high efficiency advanced audio coding (HE-AAC) standard, but the present invention is not limited thereto. The audio signal encoder 1340 may correspond to a modified disc transform transform (MDCT) encoder.

The speech signal encoder 1340 encodes the downmix signal according to a speech coding scheme when a specific frame or a segment of the downmix signal has a large speech characteristic. Here, the speech coding scheme may be based on an adaptive multi-rate wide-band (AMR-WB) standard, but the present invention is not limited thereto. Meanwhile, the speech signal encoder 1350 may further include a linear prediction coding (LPC) encoding part. When the harmonic signal has high redundancy on the time axis, the harmonic signal may be modeled by linear prediction that predicts the current signal from the past signal. In this case, the linear prediction coding method may increase coding efficiency. Meanwhile, the voice signal encoder 1340 may correspond to a time domain encoder.

The multiplexer 1350 generates an audio signal bitstream by multiplexing spatial information, coding scheme information, bandwidth extension information, and spectral data.

20 is a diagram illustrating a configuration of an audio signal decoding apparatus to which an audio signal processing apparatus according to another embodiment of the present invention is applied. Referring to FIG. 20, the audio signal decoding apparatus 1400 may include a demultiplexer 1410, an audio signal decoder 1420, a voice signal decoder 1430, a first band extension decoding unit 1440, and a second band extension decoding unit ( 1442, and a multichannel decoder 1450.

The demultiplexer 1410 extracts spectral data, coding scheme information, type information, bandwidth extension information, and spatial information from the audio signal bitstream. The audio signal corresponding to the current frame is transmitted to the audio signal decoder 1420 or the voice signal decoder 1430 according to the coding scheme information.

When the spectral data corresponding to the downmix signal has a large audio characteristic, the audio signal decoder 1420 decodes the spectral data by an audio coding method. As described above, the audio coding scheme may be based on the AAC standard and the HE-AAC standard. The audio signal decoder 1420 may include an inverse quantizer (not shown) and an inverse transformer (not shown). Accordingly, the audio signal decoder 1420 may perform inverse quantization and inverse transformation on spectral data and scale factors transmitted through the bitstream.

The speech signal decoder 1430 decodes the downmix signal using a speech coding scheme when the spectral data has a large speech characteristic. As described above, the speech coding scheme may conform to the adaptive multi-rate wide-band (AMR-WB) standard, but the present invention is not limited thereto. The voice signal decoder 1430 may include an LPC decoding part.

As described above, the audio signal is transmitted to the first band extension decoding unit 1440 or the second band extension decoding unit 1442 according to the type information indicating specific extension information among two or more band extension schemes. The first and second band extension decoding units 1440 and 1442 reconstruct wideband spectral data using some or all of narrowband spectral data according to a band extension scheme of the type.

When the decoded audio signal is downmixed, the multichannel decoder 1450 generates an output channel signal of a multichannel signal (including a stereo signal) using spatial information.

The audio signal processing apparatus according to the present invention can be included and used in various products. These products can be broadly divided into stand alone and portable groups, which can include TVs, monitors and set-top boxes, and portable groups include PMPs, mobile phones, and navigation. can do.

FIG. 21 is a view illustrating a schematic configuration of a product in which an audio signal processing device is implemented according to another embodiment of the present invention, and FIG. 22 is a relation diagram of products in which the audio signal processing device according to another embodiment of the present invention is implemented. to be. First, referring to FIG. 21, a communication unit 1510, a user authentication unit 1520, an input unit 1530, a signal coding unit 1540, a control unit 1550, and an output unit 1560 may be included. Each component except the unit 1540 performs the same function as the component of the same name described above with reference to FIG. 12. Meanwhile, the signal coding unit 1540 encodes or decodes an audio signal and / or a video signal received through the wired / wireless communication unit 1510 and outputs an audio signal of a time domain. The signal coding unit includes an audio signal processing device 1545, which corresponds to another embodiment of the present invention described above with reference to FIGS. 15 to 20, and thus, the audio processing device 1545 and the signal coding unit including the same. May be implemented by one or more processors.

22 is a relationship diagram of products in which an audio signal processing device according to an embodiment of the present invention is implemented. FIG. 22 illustrates a relationship between a terminal and a server corresponding to the product illustrated in FIG. 21. Referring to FIG. 22A, the first terminal 1500.1 and the second terminal 1500.2 may be referred to as the respective terminals. It can be seen that the data to the bitstream can be bidirectionally communicated through the wired / wireless communication unit. Referring to FIG. 22B, it can be seen that the server 1600 and the first terminal 1500.1 may also perform wired or wireless communication with each other.

The audio signal processing method according to the present invention can be stored in a computer-readable recording medium which is produced as a program for execution in a computer, and multimedia data having a data structure according to the present invention can also be stored in a computer-readable recording medium. Can be stored. The computer readable recording medium includes all kinds of storage devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, which are also implemented in the form of a carrier wave (for example, transmission over the Internet). It also includes. In addition, the bit stream generated by the encoding method may be stored in a computer-readable recording medium or transmitted using a wired / wireless communication network.

As described above, although the present invention has been described by way of limited embodiments and drawings, the present invention is not limited thereto and is intended by those skilled in the art to which the present invention pertains. Of course, various modifications and variations are possible within the scope of equivalents of the claims to be described.

The present invention can be applied to encoding and decoding audio signals.

1 is a block diagram of an audio signal processing apparatus according to an embodiment of the present invention.

FIG. 2 is a detailed configuration diagram of the sibilant detecting unit in FIG. 1. FIG.

A diagram for explaining the principle of hissing sound detection in FIG. 3.

4 is an example of energy spectra in the case of non-sibilant and in case of sibilant.

5 is an example of a detailed configuration diagram of a second encoding unit and a second decoding unit in FIG. 1.

FIG. 6 is a diagram for explaining first to second embodiments of a partial spectral data duplication (PSDD) scheme, which is an example of a non-sibilant encoding / decoding scheme; FIG.

7 and 8 are diagrams for explaining the method when the length of the frame is different from each other in the PSDD method.

9 is a first example of an audio signal encoding apparatus to which an audio signal processing apparatus according to an embodiment of the present invention is applied.

10 is a second example of an audio signal encoding apparatus to which an audio signal processing apparatus according to an embodiment of the present invention is applied.

11 is a first example of an audio signal decoding apparatus to which an audio signal processing apparatus according to an embodiment of the present invention is applied.

12 is a second example of an audio signal decoding apparatus to which an audio signal processing apparatus according to an embodiment of the present invention is applied.

FIG. 13 is a schematic structural diagram of a product implemented with an audio signal processing device according to an embodiment of the present invention; FIG.

14 is a relationship diagram of products in which an audio signal processing apparatus according to an embodiment of the present invention is implemented.

15 is a block diagram of an audio signal processing apparatus according to another embodiment of the present invention.

16 is a detailed configuration diagram of a type determination unit 1110 in FIG. 15.

17 is a diagram for explaining a process of determining a type of a band extension method.

18 is a diagram for describing various types of band extension methods.

19 is a block diagram of an audio signal encoding apparatus to which an audio signal processing apparatus according to another embodiment of the present invention is applied.

20 is a block diagram of an audio signal decoding apparatus to which an audio signal processing apparatus according to another embodiment of the present invention is applied.

21 is a schematic structural diagram of a product implemented with an audio signal processing device according to another embodiment of the present invention;

FIG. 22 is a relational view of products in which an audio signal processing device according to another embodiment of the present invention is implemented. FIG.

Claims (19)

By the audio processing apparatus, among a plurality of band extension schemes including a first band extension scheme and a second band extension scheme, type information indicating a specific band extension scheme for the current frame of the audio signal, and spectral of a low frequency band Receiving data; When the type information indicates the first band extension scheme with respect to the current frame, generating spectral data of the high frequency band of the current frame by using the spectral data of the low frequency band by performing the first band extension scheme. step; And, When the type information indicates the second band extension scheme with respect to the current frame, the spectral data of the high frequency band of the current frame is generated by using the spectral data of the low frequency band by performing a second band extension scheme. Including steps The first band extension scheme is based on a first data region of spectral data in the low frequency band, the second band extension scheme is based on a second data region of spectral data in the low frequency band, And the second data area is larger than the first data area. The method according to claim 1, And the first data area is a portion of spectral data in the low frequency band, and the second data area is a plurality of portions including the portion of the spectral data in the low frequency band. The method of claim 1, And the first data area is a portion of spectral data in the low frequency band, and the second data area is all of the spectral data in the low frequency band. delete The method of claim 1, And the high frequency band includes one or more bands equal to or higher than the boundary frequency, and the low frequency band includes one or more bands equal to or lower than the boundary frequency. The method according to claim 1, The first band extension method is performed using at least one of bandpass filtering, time stretching and decimation processing. The method of claim 1, Receiving band extension information including envelope information; The first band extension method or the second band extension method is performed using the band extension method. The method of claim 1, Decoding the spectral data of the low frequency band using an audio coding scheme in a frequency domain or a speech coding scheme in a time domain; The spectral data of the high frequency band is generated using the decoded spectral data of the low frequency band. A plurality of band extension schemes including a first band extension scheme and a second band extension scheme, the demultiplexer for receiving type information indicating a specific band extension scheme for a current frame of an audio signal and spectral data of a low frequency band; When the type information indicates the first band extension scheme with respect to the current frame, generating spectral data of the high frequency band of the current frame by using the spectral data of the low frequency band by performing the first band extension scheme. A first band extension decoding unit; And, When the type information indicates the second band extension scheme with respect to the current frame, the spectral data of the high frequency band of the current frame is generated by using the spectral data of the low frequency band by performing a second band extension scheme. A second band extension decoding unit, The first band extension scheme is based on a first data region of spectral data in the low frequency band, the second band extension scheme is based on a second data region of spectral data in the low frequency band, And the second data area is larger than the first data area. The method of claim 9, And the first data area is a portion of spectral data in the low frequency band, and the second data area is a plurality of portions including the portion of the spectral data in the low frequency band. The method of claim 9, And the first data area is a portion of spectral data in the low frequency band, and the second data area is all of the spectral data in the low frequency band. delete The method of claim 9, And the high frequency band includes one or more bands equal to or higher than the boundary frequency and the low frequency band includes one or more bands equal to or lower than the boundary frequency. The method of claim 9, The first band extension method is performed using at least one of bandpass filtering, time stretching and decimation processing. The method of claim 9, The demultiplexer further receives band extension information including envelope information, The first band extension method or the second band extension method is performed using the band extension method. The method of claim 9, An audio signal decoder for decoding the spectral data of the low frequency band using an audio coding scheme in a frequency domain; And, And a speech signal decoder for decoding the low frequency spectral data using a speech coding scheme in a time domain. The spectral data of the high frequency band is generated using spectral data of the decoded low frequency band. Detecting, by the audio processing apparatus, the degree of transient for the current frame of the audio signal; Determining a specific band extension scheme for the current frame among a plurality of band extension schemes including a first band extension scheme and a second band extension scheme based on the transient degree; Generating type information indicating the specific band extension method; Generating spectral data of a high frequency band by using spectral data of a low frequency band by performing the first band extension method when the specific band extension method is the first band extension method with respect to the current frame; Generating the spectral data of the high frequency band by using the spectral data of the low frequency band by performing the second band extension method when the specific band extension method is the second band extension method with respect to the current frame; And Transmitting spectral data of the type information and the low frequency band, The first band extension scheme is based on a first data region of spectral data in the low frequency band, and the second band extension scheme is based on a second data region of spectral data in the low frequency band. Audio signal processing method. A transient detection part that detects a transient degree of a current frame of an audio signal; A type of determining a specific band extension scheme for the current frame among a plurality of band extension schemes including a first band extension scheme and a second band extension scheme, and indicating the specific band extension scheme based on the transient degree. A type information generation part for generating information; When the specific band extension scheme is the first band extension scheme for the current frame, a first band for generating spectral data of a high frequency band using spectral data of a low frequency band by performing the first band extension scheme An extension encoding unit; When the specific band extension scheme is the second band extension scheme for the current frame, performing the second band extension scheme to generate spectral data of the high frequency band using spectral data of a low frequency band; Band extension encoding unit; And A multiplexer for transmitting the type information and the spectral data of the low frequency band, The first band extension scheme is based on a first data region of spectral data in the low frequency band, and the second band extension scheme is based on a second data region of spectral data in the low frequency band. Audio signal processing device. Receiving, among a plurality of band extension schemes including a first band extension scheme and a second band extension scheme, type information indicating a specific band extension scheme for a current frame of an audio signal, and spectral data of a low frequency band; When the type information indicates the first band extension scheme with respect to the current frame, generating spectral data of the high frequency band of the current frame by using the spectral data of the low frequency band by performing the first band extension scheme. step; And, When the type information indicates the second band extension scheme with respect to the current frame, the spectral data of the high frequency band of the current frame is generated by using the spectral data of the low frequency band by performing a second band extension scheme. Including steps The first band extension scheme is based on a first data region of spectral data in the low frequency band, the second band extension scheme is based on a second data region of spectral data in the low frequency band, The second data area is larger than the first data area, Instructions are stored, wherein the instructions, when executed by a processor, cause the processor to perform an operation.

Images