RU2822612C1 - Harmonic conversion based on subband block, amplified by cross products - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: group of inventions relates to audio signal sources encoding systems, which apply a harmonic conversion method for high-frequency reconstruction (HFR) in digital effects processors, such as exciters, which generate harmonic distortion to add brightness to the processed signal, and in devices for stretching the time scale, which increase the duration of the signal while preserving the spectral composition. Disclosed method realizes high-frequency reconstruction (HFR), in which a new component with frequency QΩ+rΩ₀ is generated based on existing components with frequencies Ω and Ω+Ω₀. In the method, harmonic conversion is performed based on a subband block, where a time block of complex-valued discrete values of subbands is processed by well-known phase modification. Superposition of several modified discrete values gives resultant effect of limiting undesirable combination components, whereby it is possible to use coarser frequency resolution and/or less degree of oversampling. In one embodiment, the invention further comprises a window function suitable for use with a cross-product-amplified subband block-based HFR. Also disclosed is a system and a persistent data medium which implements the disclosed method.

EFFECT: creation of a more efficient implementation of HFR amplified by cross products, in particular creation of a method which reproduces an audio signal with high accuracy under the condition of reduction of computational costs.

3 cl, 9 dwg

Description

TECHNICAL FIELD

The present invention relates to audio source encoding systems that employ harmonic transformation techniques for high frequency reconstruction (HFR) in digital effects processors such as exciters that generate harmonic distortion to add brightness to the processed signal, and in time stretchers that increase signal duration while maintaining the spectral composition.

BACKGROUND OF THE INVENTION

In WO 98/57436, the concept of conversion was established as a method of recreating a high frequency band from a low frequency band of an audio signal. By using this concept, significant bit rate savings can be achieved when encoding an audio signal. In an HFR-based audio encoding system, a low-bandwidth signal is passed to a base waveform encoder and higher frequencies are regenerated using a transform and additional auxiliary information transmitted at a very low bit rate that describes the target spectral shape at the decoder side . For low bit rates, where the bandwidth of the underlying encoded signal is narrow, it becomes increasingly important to recreate the high frequency band with visually pleasing characteristics. The harmonic transformation defined in WO No. 98/57436 performs very well for complex musical material in low transition frequency situations. The principle of harmonic transformation is that a sinusoid with frequency ω is mapped into a sinusoid with frequency Q _ϕ ω , where Q _ϕ greater than 1 - an integer that determines the conversion order. In comparison, HFR based on single sideband signal modulation maps a sine wave of frequency ω to a sine wave of frequency ω+Δω , where Δω is a fixed frequency offset. For any given low-bandwidth base signal, the resulting SSB conversion will produce a dissonant ringing artifact.

In order to achieve the best possible audio signal quality, state-of-the-art high-quality harmonic HFR techniques use complex modulated filter banks with very high frequency resolution and a high degree of oversampling to achieve the required audio quality. High resolution is necessary to avoid unwanted intermodulation distortion resulting from nonlinear processing of sums of sine waves. If the subbands are sufficiently narrow, high-quality methods strive to ensure that each subband contains no more than one sine wave. A high degree of time oversampling is necessary to avoid undersampling distortion, and a certain degree of frequency oversampling is necessary to avoid leading echoes for transient signals. The obvious disadvantage is that the computational complexity becomes very high.

Another well-known disadvantage associated with harmonic transformations appears for signals with a pronounced periodic structure. These signals are superpositions of harmonically related sinusoids with frequencies Ω, 2Ω, 3Ω … ., where Ω is the fundamental frequency. With a harmonic transformation of order Q _ϕ, the output sinusoids have frequencies Q _ϕ Ω, 2 Q _ϕ Ω, 3 Q _ϕ Ω .... , which in the case of Q _ϕ greater than 1 represents a strict subset of the desired complete harmonic series. In terms of the resulting sound quality, a "spurious" fundamental tone will typically be perceived corresponding to the converted fundamental frequency Q _ϕ Ω. Often harmonic transformation leads to a “metallic” character in the sound of the encoded and decoded audio signal.

In WO 2010/081892, which is incorporated by reference herein, a cross-product method was developed to address the above-described problem of a “spurious” pitch in the case of high-quality conversion. For a given partial or complete transmitted information about the value of the fundamental frequency of the predominant harmonic part of the signal to be converted with high accuracy, nonlinear modifications of the subbands are supplemented by nonlinear combinations of at least two different analyzed subbands, where the distances between the indices of the analyzed subbands are related to the fundamental frequency. As a result, missing harmonics of the converted output signal are generated, which, however, occurs at significant computational costs.

BRIEF DESCRIPTION OF THE INVENTION

In view of the above-described shortcomings of available HFR methods, the purpose of the present invention is to provide a more efficient cross-product-enhanced implementation of HFR. In particular, the aim is to create said method which would allow the reproduction of an audio signal with high accuracy, provided that the computational cost is reduced compared to available methods.

The present invention achieves at least one of these objects by providing devices and methods according to independent claims.

In a first aspect, the invention provides a system configured to generate a time-stretched and/or frequency-converted signal from an input signal. The system contains:

• a block of analyzing filters configured to obtain from the input signal a certain number of Y signals of analyzed subbands, where each signal of the analyzed subband includes a number of complex-valued analyzed discrete values, each of which has a phase and amplitude;

• a subband processing module configured to determine a synthesized subband signal based on the Y signals of the analyzed subbands using a subband transformation coefficient Q and a subband stretching coefficient S , where at least one of the coefficients Q and S is greater than one, wherein the subband processing module includes:

○ block extractor configured for:

■ i) generating Y frames consisting of L input discrete values, where each frame is extracted from a specified series of complex-valued discrete values in the signal of the analyzed subband, and the frame length L is greater than 1; And

■ ii) applying the jump value of a block of h samples to the specified series of analyzed samples before generating the next frame consisting of L input samples, whereby a sequence of input sample frames is generated;

○ a nonlinear frame processing module configured to generate, based on Y corresponding frames of input samples generated in the block extractor, a frame of processed samples by determining the phase and amplitude for each processed sample of the frame, where for at least one processed sample:

■ i) the phase of the processed sample is based on the corresponding phases of the corresponding input sample in each of the Y input sample frames; And

■ ii) the amplitude of the processed sample is based on the amplitude of the corresponding input sample in each of the Y input sample frames; And

○ an overlay and addition module configured to determine the signal of the synthesized subband by overlaying and adding discrete values from a sequence of frames of processed discrete values; And

• a synthesis filter bank configured to generate a time-stretched and/or frequency-converted signal from the synthesis subband signal.

The system can operate for any positive integer value of Y. However, it is valid for at least Y =2.

In a second aspect, the invention provides a method for generating a time-stretched and/or frequency-converted signal from an input signal. The method includes:

• obtaining from the input signal a certain number of Y >2 signals of the analyzed subranges, where each signal of the analyzed subrange includes a number of complex-valued analyzed discrete values, each of which has a phase and amplitude;

• formation of Y frames consisting of L input discrete values, where each frame is extracted from a specified number of complex-valued analyzed discrete values in the signal of the analyzed subband, and the frame length L is more than 1;

• applying a jump value of a block of h samples to a specified series of analyzed samples before obtaining the next frame consisting of L input samples, whereby a sequence of frames of input samples is generated;

• generating, based on Y, the corresponding frame input samples of the frame of processed samples by determining the phase and amplitude for each processed sample of the frame, where for at least one processed sample:

○ the phase of the processed discrete value is based on the corresponding phases of the corresponding input discrete value in at least one of the Y input discrete value frames; And

○ the amplitude of the processed discrete value is based on the amplitude of the corresponding input discrete value in each of the Y frames of input discrete values;

• determination of the synthesized subband signal by superimposing and adding discrete values from a sequence of frames of processed discrete values; And

• generation of a time-stretched and/or frequency-converted signal from a synthesized sub-band signal.

Here Y is an arbitrary integer greater than one. The system according to the first aspect operates to implement the method at least for Y =2.

A third aspect of the invention provides a computer program product that includes a computer-readable storage medium (or information storage medium) in a memory of which software instructions are stored for causing a programmable computer to execute a method according to the second aspect.

The invention is based on the understanding that the general concept of HFR, enhanced by cross products, will provide improved results when processing data arranged into blocks that consist of complex discrete subband values. Among other things, this makes it possible to apply a frame-by-frame phase shift to discrete values, which has been found to weaken the Raman components in some situations. It is also possible to use amplitude adjustment, which can lead to similar beneficial effects. The implementation of cross-product enhanced HFR according to the invention involves harmonic transformation based on a block of subbands, which can significantly attenuate the combinational components. Therefore, while maintaining high perceived quality, a filter bank (such as a QMF filter bank) with coarser frequency resolution and/or less oversampling may be used. In subband block processing, a time block of complex subband samples is processed by conventional phase modification, and the superposition of several modified samples to form an output subband sample has the cumulative effect of suppressing the combinational components that would otherwise occur when the input subband signal consists of multiple sinusoids. . Conversion based on block-based subband processing has much less computational complexity than high-resolution converters and achieves almost the same quality for many signals.

For the purpose of this disclosure, note that in embodiments of the invention where Y >2, the nonlinear processing module uses as input Y "corresponding" frames of input samples in the sense that the frames are synchronous or nearly synchronous. For example, the discrete values in the corresponding frames may refer to time periods that have significant time overlap between frames. The term "corresponding" is also used in relation to discrete values to indicate that they are or approximately are synchronous. Additionally, the term "frame" will be used interchangeably with the term "block". Accordingly, the "block jump amount" may be equal to the frame length (possibly adjusted for downsampling, if applicable) or may be less than the frame length (possibly adjusted for downsampling, if applicable), in which case consecutive frames overlap in the sense that an input sample can belong to more than one frame. The system optionally generates each processed sample in a frame by determining its phase and amplitude based on the phase and amplitude of all Y corresponding frames of input samples; Without departing from the invention, the system may generate the phase and/or amplitude of some of the processed samples based on a smaller number of corresponding input samples or based on only one input sample.

In one embodiment of the invention, the analysis filter bank is a quadrature mirror filter (QMF) bank, or pseudo-QMF bank, with any number of links and points. For example, it could be a 64-point QMF block. The analysis filter block can also be selected from a class of windowed discrete Fourier transforms or wavelet transforms. Advantageously, the synthesis filter block is coordinated with the analysis filter block, being, respectively, an inverse QMF block, an inverse pseudo-QMF block, etc. It is known that these filter banks may have a fairly coarse frequency resolution and/or a relatively low degree of oversampling. Unlike the current state of the art, the invention can be implemented using these relatively simpler components without necessarily suffering from a reduction in output quality; Thus, these embodiments of the invention have an economic advantage over the current state of the art.

In one embodiment of the invention, one or more of the following statements are true for the analysis filter bank:

• time step of analysis - Δ t _A ;

• separation of analyzed frequencies - Δƒ _A ;

• the block of analyzing filters includes N >1 analyzed subranges, indexed by the index of the analyzed subrange n=0,..., N-1 ;

• the analyzed subband is associated with one of the frequency bands of the input signal.

In one embodiment of the invention, one or more of the following statements are true for a synthesis filter bank:

• synthesis time step - Δt _s ;

• separation of synthesized frequencies - Δ ƒ _s ;

• the block of synthesis filters includes M >1 synthesized subbands, indexed by the index of the synthesized subband m=0,..., M-1 ;

• the synthesized sub-range is associated with one of the frequency bands of the signal, stretched in time and/or converted in frequency.

In one embodiment of the invention, the nonlinear frame processing module is adapted to input two frames ( Y =2) to generate one frame of processed samples, and the subband processing module includes a cross-processing control module for generating cross-processing control data. By defining quantitative and/or qualitative characteristics of subband processing in this way, the invention achieves flexibility and adaptability. The control data may define subbands (eg, identified by indices) that differ in frequency from the fundamental frequency of the input signal. In other words, the indices identifying the subbands may differ by an integer, which serves as an approximation of the quotient of the specified fundamental frequency, divided by the separation of the analyzed frequencies. This will result in a psychoacoustically pleasing output signal since the new spectral components generated by the harmonic transformation will be compatible with a range of natural harmonics.

In a further development of the previous embodiment of the invention, the indices of the (input) analyzed and (output) synthesized subbands are selected so as to satisfy Equation (16) below. The parameter σ that appears in this equation makes it applicable to both non-uniformly and uniformly arranged filter banks. If the subband indices are obtained as an approximate (eg least squares) solution of equation (16), the new spectral component obtained by the harmonic transformation is likely to be compatible with a number of natural harmonics. Thus, HFR is likely to provide a faithful reconstruction of the original signal from which high-frequency content has been removed.

A further development of the foregoing embodiment of the invention provides a method for selecting the parameter r appearing in equation (16) and representing the order of the cross-product transformation. For a given index m of the output subrange, each value of the order r of the transformation will determine two indices n ₁ , n ₂ of the analyzed subranges. This further development evaluates the amplitudes of the two specified subranges for a certain number r of options and selects the value that maximizes the minimum of the two amplitudes of the analyzed subranges. This method of selecting indices can avoid the need to recreate sufficient amplitude by amplifying weak components of the input signal, which can lead to poor quality output. In this regard, the amplitudes of the subbands can be calculated in a manner that is known per se, such as the square root of the squares of the input samples forming a frame (block) or part of a frame. The subband amplitude can also be calculated as the amplitude of the central, or closest to the central, sample value in the frame. Such a calculation can create a simple, but at the same time adequate quantitative indicator of the amplitude.

In a further development of the previous embodiment of the invention, the synthesized subband can receive contributions from harmonic transformation events according to both direct processing and cross product processing. In this regard, decision criteria may be applied to determine whether the special ability to recover the missing harmonic through cross-product processing should be used. For example, said further development may be adapted so that it refrains from using one subband cross-processing module in the case where one of the following conditions is met:

a) the ratio of the amplitude M _s of the member of the analyzed subband from the direct source leading to the synthesized subband, and at least the amplitude M _c in the optimal pair of terms from the cross source leading to the synthesized subband, is greater than a predetermined constant;

b) the synthesized subband already receives a significant contribution from the direct processing module;

c) the fundamental frequency Ω ₀ is less than the frequency separation of the block of analyzing filters Δ ƒ _A .

In one embodiment of the invention, the invention includes downsampling (decimation) of the input signal. Moreover, one or more frames of input samples may be determined by downsampling the complex-valued samples in a subrange, which may be performed by a block extractor.

In a further development of the foregoing embodiment, the downsampling factors to be applied satisfy Equation (15) below. It is not allowed for both downsampling factors to be zero because this corresponds to a trivial case. Equation (15) determines the relationship of the downsampling coefficients D ₁ , D ₂ with the subband stretching coefficient S and the subband conversion coefficient Q , as well as with the phase coefficients T ₁ , T ₂ appearing in expression (13) to determine the phase of the processed discrete value. This ensures that the phase of the processed discrete values is consistent with other components of the input signal, to which the processed discrete values are subject to addition.

In one embodiment of the invention, frames of processed discrete values are windowed before they are superimposed and added. The windowing module can be adapted to be applied to processed discrete values of a finite length windowing function. Suitable window functions are listed in the accompanying claims.

The inventor realized that the cross product methods disclosed in WO No. 2010/08892 are not inherently compatible with subband block based processing methods. Although this method can be satisfactorily applied to one of the samples in a block, it may lead to aliasing artifacts if directly extended to other samples in the block. To this end, one embodiment of the invention employs window functions comprising windowed discrete values which, when weighted by complex weighting factors and offset by a jump amount, are reduced to a substantially constant sequence. The jump size may be the product of the block jump size h and the subband stretch factor S. Using these windowing functions reduces the impact of aliasing artifacts. Alternatively or in addition, these window functions may also allow other artifact mitigation measures such as phase alternations of the processed discrete values.

Preferably, the successive complex weighting factors that are applied to the windowed samples for the purpose of estimating their state differ only by a fixed phase rotation. It is also preferable that said fixed phase rotation be proportional to the fundamental frequency of the input signal. The phase alternation may also be proportional to the cross product transformation order to be applied and/or the physical transformation parameter and/or the difference in downsampling factors and/or the time step of the analysis. The phase alternation may take the form of equation (21), at least in an approximate sense.

In one embodiment, the present invention enables cross-product-enhanced harmonic transformation by modifying the synthesis windowing in response to a fundamental frequency parameter.

In one embodiment of the invention, successive frames of processed discrete values are added with a certain overlap. To perform a suitable overlay, the frames of the processed sample values are suitably shifted by a jump amount, which is the block jump amount h multiplied by the subband stretch factor S. Thus, if the overlap of successive frames of input samples is Lh , then the overlap of successive frames of processed samples may be S(Lh) .

In one embodiment of the invention, the system of the invention operates not only to generate a process sample based on the Y =2 input samples, but also based on the Y =1 sample only. Thus, the system can recover the missing harmonics not only through a cross-product approach (such as Equation (13)), but also through a direct subband approach (such as Equation (5) or (11) )). Preferably, the control module is configured to control the operation of the system, including which approach should be used to restore a particular missing harmonic.

In a further development of the previous embodiment of the invention, the system is also adapted to generate a processed discrete value based on more than three discrete values, i.e. for Y >3. For example, the processed sample value may be obtained by contributing to the processed sample value from multiple cross product harmonic transform events, by multiple direct subband processing events, or by a combination of cross product transform and direct transform. This ability to adapt the conversion method provides a productive and multifunctional HFR. Accordingly, this embodiment of the invention operates to implement the method according to the second aspect of the invention for Y = 3, 4, 5 and the like.

One embodiment of the invention is configured to define a processed sample value as a complex number having an amplitude that is the average of the corresponding amplitudes of the corresponding input samples. Said average may be a (weighted) arithmetic, (weighted) geometric, or (weighted) harmonic average of two or more input samples. In the case of Y =2, the average is based on two complex input samples. Preferably, the amplitude of the processed sample value is a geometric weighted average value. More preferably, the geometric value, as shown in equation (13), is weighted by the parameters ρ and 1-ρ. Here, the geometric amplitude weighting parameter ρ is a real number inversely proportional to the subband conversion factor Q. The parameter ρ can also be inversely proportional to the stretching coefficient S.

In one embodiment of the invention, the system is adapted to define the sample value being processed as a complex number having a phase that is a linear combination of the corresponding phases of the corresponding input samples in the input sample frames. In particular, the linear combination may include phases related to two input discrete values ( Y =2). The linear combination of two phases can apply integer non-zero coefficients whose sum is equal to the stretch factor S multiplied by the subband conversion factor Q. Optionally, the phase obtained by said linear combination is further corrected by a fixed phase correction parameter. The phase of the processed discrete value can have the form of equation (13).

In one embodiment of the invention, a block extractor (or a similar step in the method of the invention) is adapted to interpolate two or more analysis samples from the analysis subband signal to produce a single input sample to be included in a frame (block). Said interpolation may enable downsampling of the input signal by a non-integer factor. The analyzed discrete values to be interpolated may or may not be consecutive to each other.

In one embodiment of the invention, the subband processing configuration may be controlled by control data delivered externally to the module performing the processing. The control data may relate to the instantaneous acoustic properties of the input signal. For example, the system itself may contain a section adapted to determine instantaneous acoustic properties of the signal, such as the (dominant) fundamental frequency of the signal. Knowing the fundamental frequency provides guidance in selecting the analyzed subranges from which to obtain the processed discrete values. Accordingly, the separation of the analyzed subbands is proportional to the specified fundamental frequency of the input signal. Alternatively, the control data may also be delivered from outside the system, preferably by being included in an encoding format suitable for transmission as a bit stream over a digital communications network. In addition to control data, said encoding format may contain information related to low frequency components of the signal (eg, components at 701 in FIG. 7). However, in the interest of saving bandwidth, the format preferably does not include complete information related to the high frequency components (item 702) that can be recovered according to the invention. The invention may, in particular, provide a decoding system with a control data receiving module configured to receive said control data, either included in a received bit stream that also encodes the input signal, or received as a separate signal or bit stream.

One embodiment of the invention provides a method for efficiently performing the calculations required by the method according to the invention. To this end, the hardware implementation may include a pre-normalizer for rescaling the amplitudes of the corresponding samples in some of the Y frames on which the frame of samples being processed will be based. After said rescaling, the processed sample value can be calculated as the (weighted) complex product of the input sample values subject to rescaling and possibly unscaled values. An input discrete value that appears in a product as a rescaled coefficient typically does not reappear as an unscaled coefficient. With the possible exception of the phase correction parameter θ, one can evaluate equation (13) as the product of complex input discrete values (possibly rescaled). This provides a computational advantage over processing the amplitude and phase of the sampled value separately.

In one embodiment of the invention, a system configured for the Y =2 case includes two block extractors adapted to each generate one frame of input samples in parallel operation.

In a further development of embodiments of the invention representing Y >3, the system may comprise a number of subband processing units, each of which is configured to determine an intermediate synthesized subband signal using a different subband conversion factor and/or a different subband stretch factor, and/or conversion method, characterized in that it is cross product based or direct. For parallel operation, subband processing modules can be arranged in parallel. In this embodiment of the invention, the system may also include a fusion module located behind the subband processing modules and in front of the synthesis filter bank. The fusion module may be adapted to merge (eg, by mixing) corresponding intermediate synthesis subband signals to produce a synthesis subband signal. As already noted, the intermediate synthesized subband that undergoes merging can be obtained either by direct harmonic transformation or by cross-product transformation. The system according to this embodiment of the invention may also include a core decoder for decoding the bit stream into an input signal. It may also include an HFR processing module adapted to apply the spectral band information, especially by performing spectrum shaping. The action of the HFR processing module may be controlled by information encoded in the bitstream.

One embodiment of the invention provides for HFR of multi-dimensional signals, for example, in a system designed to reproduce audio in a stereo format containing Z channels such as left, right, center, surround, etc. In one possible implementation of multi-channel input signal processing, the processed samples of each channel are based on the same number of input samples, although the stretch factor S and the transform factor Q for each band may vary between channels. To this end, the implementation may include an analysis filter bank configured to generate Y analysis subband signals from each channel, a subband processing module configured to generate Z subband signals, and a synthesis filter bank configured to generate Z time-stretched and/or frequency-converted signals that form the output signal.

In variations of the previous embodiment, the output signal may comprise output channels that are based on different numbers of analyzed subband signals. For example, it may be desirable to transfer more processing resources to the HFR for acoustically more pronounced channels; for example, channels intended to be reproduced by sound sources located in front of the listener may be given preference over surround or rear channels.

It should be emphasized that the invention relates to all combinations of the above characteristic features, even if they are set out in different claims.

BRIEF DESCRIPTION OF GRAPHIC MATERIALS

The present invention will be described below by way of illustrative examples, without limiting the scope or spirit of the invention, with reference to the accompanying drawings.

Fig. 1 illustrates the principle of harmonic transformation based on a block of subbands.

Fig. 2 illustrates the operation of nonlinear processing of a subband block with one input subband.

Fig. 3 illustrates the operation of nonlinear processing of a subband block with two input subbands.

Fig. 4 illustrates the operation of a harmonic transform based on a cross-product enhanced subband block.

Fig. 5 illustrates an example scenario for applying subband block based transform using multiple transform orders in an HFR enhanced audio codec.

Fig. 6 illustrates an example scenario of a multi-order transform operation based on a subband block using a 64-band QMF analysis filter bank.

Fig. 7 and 8 illustrate the experimental results of the described subband block-based conversion method.

Fig. 9 shows details of the nonlinear processing unit according to FIG. 2, including a prenormalizer and a multiplier.

DESCRIPTION OF PREFERRED OPTIONS FOR IMPLEMENTING THE INVENTION

The embodiments described below are solely illustrative of the principles of the present SUB-BAND BLOCK BLOCK HARMONIC CONVERSION ENHANCED BY CROSS PRODUCTS invention. It should be understood that modifications and changes to the circuitry and parts described herein will be apparent to those skilled in the art. It is therefore intended that the invention be limited only by the scope of the appended claims and not by the specific details set forth herein for the purpose of describing and explaining embodiments of the invention.

Fig. 1 illustrates the principle of transformation, time stretching, or a combination of transformation and time stretching based on a block of subbands. The time domain input signal is fed to an analysis filter block 101, which produces several complex-valued subband signals. These signals are supplied to a subband processing module 102, the operation of which can be influenced by control data 104. Each output subband can be obtained by processing either one or two input subbands, or even as a superposition of the result for several specified processed subbands. A plurality of complex-valued output subbands are supplied to a synthesis filter bank 103, which in turn outputs a modified time domain signal. Optional control data 104 describes the configuration and processing parameters of the subbands that can be adapted to the signal to be converted. In the case of cross-product enhanced conversion, this data may carry information related to the dominant fundamental frequency.

Fig. 2 illustrates the operation of nonlinear processing of a subband block with one input subband. Given target physical time stretch and transform values, as well as physical parameters of the analysis and synthesis filter blocks 101 and 103, time stretch and transform subband parameters are output, as well as a source subband index for each target subband index. The purpose of processing a block of subbands is then to implement an appropriate transformation, time stretching, or a combination of transformation and time stretching of the complex-valued signal of the source subband to generate a signal of the target subband.

Block extractor 201 samples a final frame of discrete values from the complex-valued input signal. The frame is determined by the position of the input pointer and the subband conversion factor. This frame undergoes non-linear processing in processing section 202 and is then windowed with windows of finite and possibly variable length in windowing section 203. The resulting samples are added to the previous output samples in an overlay and addition module 204, where the position of the output frame is determined by the position of the output pointer. The input pointer is incremented by a fixed amount, and the output pointer is incremented by the same amount multiplied by the subband stretch factor. Repeating this chain of operations will result in an output signal with a duration that is the duration of the input subband signal multiplied by the stretch factor, up to the length of the synthesis window, and with complex frequencies converted by the subband conversion factor. The control signal 104 may influence each of the three sections 201, 202, 203.

Fig. 3 illustrates the operation of nonlinear processing of a subband block with two subband input signals. Given the target physical time stretch and transform values and the physical parameters of the analysis and synthesis filter blocks 101 and 103, the time stretch and transform subband parameters are output, as well as two source subband indices for each target subband index. In the case where nonlinear processing of a block of subbands is to be used to create missing harmonics by adding cross products, the configuration of sections 301-1, 301-2, 302, 303, as well as the values of the two source subband indices, may depend on the output signal 403 of the crossover control module 404 processing. The purpose of subband block processing is to implement an appropriate transform, time stretching, or a combination of transform and time stretching to combine two complex-valued source subband signals to generate a target subband signal. The first block extractor 301-1 samples a final time frame of discrete values from the first complex-valued source subband, and the second block extractor 301-2 samples a final time frame of discrete values from the second complex-valued source subband. Frames are determined by the overall position of the input pointer and the subband conversion factor. Both frames undergo non-linear processing in section 302 and are then windowed with a window of finite length in windowing section 303. The overlay and addition module 204 may have a similar or identical design to the module shown in FIG. 2. Repeating this chain of operations will result in an output signal with a duration equal to the duration of the two input subband signals multiplied by the subband stretch factor (up to the length of the synthesis window). In the case where both input signals carry the same frequencies, the output signal will have complex frequencies converted by a subband conversion factor. In the case where two input signals carry different frequencies, the present invention specifies that windowing 303 can be adapted to generate an output signal that has a target frequency suitable for generating the missing harmonics in the converted signal.

Fig. 4 illustrates the principle of cross-product enhanced transform based on a block of subbands, time stretching, or a combination of transform with time stretching. The direct subband processing module 401 may be of the type already described with reference to FIG. 2 (section 202) or FIG. 3. The subband cross processing unit 402 is also supplied with a plurality of complex valued subband signals and is influenced by the cross processing control data 403. The subband cross processing unit 402 performs subband block processing related to the dual input subband processing type described in FIG. 3, and the output target subbands are added to the subbands from the forward subband processing 401 in the adder 405. The cross processing control data 403 may vary for each input pointer position and consists of at least the following data:

• selected list of target subrange indices;

• pairs of source subrange indices for each selected target subrange index; And

• synthesis windows of finite length.

The cross-processing control module 404 delivers said cross-processing control data 403 for a given portion of the fundamental frequency control data 104 and a plurality of complex-valued subband output signals from the analysis filter bank 101 . The control data 104 may also carry other signal-dependent configuration parameters that affect cross product processing.

In the following text with reference to FIG. 1-4 and by adding appropriate mathematical terminology, a description of the principles of cross-product enhanced time stretching and subband block based transformation will be given.

The two main configuration parameters of a general harmonic transformer and/or time stretching device in general are:

• S _ϕ - the required coefficient of physical stretching in time, and

• Q _ϕ - the required physical conversion coefficient.

Filter banks 101, 103 may be any type of complex exponential modulated filters, such as QMF or windowed DFT, or wavelet transform. The analysis filter bank 101 and the synthesis filter bank 103 may be uniformly or non-uniformly modulated and determined from a wide range of prototype filters and/or windows. Although all of these second-order options affect subsequent design details such as phase corrections and subband display control, the main system design parameters for subband processing are typically derived from two quotients: Δt _S /Δt _A and Δ ƒ _s /Δ ƒ _A of the following four parameters of filter blocks, where all parameters are measured in physical units. In the above quotients:

• Δt _A - step or time shift of the discrete value of the sub-range of the block 101 of analyzing filters (for example, measured in seconds, [s]);

• Δ ƒ _A - frequency spacing of subbands of the block 101 of analyzing filters (for example, measured in Hertz, [1/s]);

• Δt _S - step or time shift of a discrete value of the sub-band of the synthesis filter block 103 (for example, measured in seconds, [s]); And

• Δf _s is the frequency spacing of the subbands of the synthesis filter block 103 (for example, measured in Hertz, [1/s]).

To configure the subband processing module 102, the following parameters should be calculated:

• S - stretching coefficient of subbands, i.e. a stretch factor that is applied in the subband processing module 102 as the ratio of the input and output samples to achieve an overall physical time stretch of the time domain signal through the coefficient S _ϕ ;

• Q - conversion factor of subbands, i.e. a conversion factor that is applied in the subband processing unit 102 to achieve the overall physical conversion of the signal in the time domain by means of the coefficient Q _ϕ ; And

• correspondence between the indices of the source and target subbands, where n denotes the index of the analyzed subband included in the subband processing unit 102, and m denotes the index of the corresponding synthesized subband as the output signal of the subband processing unit 102.

In order to determine the subband stretching coefficient S , it was observed that the input signal to the analyzing filter block 101, having a physical duration D , corresponds to the number D/Δt _A of discrete values of the analyzed subbands at the input to the subband processing module 102. These D/Δt _A samples will be stretched to S ⋅ D/Δt _A samples by subband processing module 102, which applies the subband stretch factor S. At the output of synthesis filter bank 103, these S⋅D/Δt _A samples result in an output signal having a physical duration Δt _A ⋅S ⋅ D/Δt _A . Since this last duration must correspond to the given value S _ϕ ⋅ D , i.e. Since the duration of the output time-domain signal must be time-stretched relative to the input time-domain signal by _the physical time-stretch factor Sϕ , the following design rule is obtained:

In order to determine the subband conversion factor Q , which is used in the subband processing module 102 to achieve the physical conversion Q _ϕ , it is observed that the input sinusoid to the analysis filter block 101 with a physical frequency Ω will result in a complex signal of the analyzed subband with a time-discrete angular frequency ω=2πΩ⋅Δt _A , and the main contribution comes from the analyzed subrange with the index n≈Ω/Δ ƒ _A . The output sinusoid at the output of the block 103 of synthesis filters with the required converted physical frequency Q _ϕ ⋅Ω will be the result of feeding a complex subband signal with a discrete angular frequency 2πQ _ϕ ⋅Ω⋅Δt _S to the synthesized subband with index m≈Q _ϕ ⋅Ω/Δ ƒ _s . In this context, care must be taken to avoid synthesizing aliased frequencies other than Q _ϕ ⋅Ω . Typically, this can be avoided by making appropriate second-order choices as discussed above, for example by selecting appropriate analysis and/or synthesis filter banks. The discrete frequency 2πQ _ϕ ⋅Ω⋅Δt _S at the output of the subband processing module 102 must correspond to the time-discrete frequency ω=2πΩ⋅Δt _A at the input to the subband processing unit 102, multiplied by the subband conversion factor Q. Those. equating 2πQΩΔt _A to 2πQ _ϕ ⋅Ω⋅Δt _S , we can determine the following relationship between the physical conversion coefficient Q _ϕ and the subband conversion coefficient Q :

Likewise, the corresponding source or analysis subband index n of the subband processing module 102 for a given target or synthesis subband index m must satisfy the following condition:

In one embodiment of the invention, it is true that Δ ƒ _s /Δ ƒ _A =Q _ϕ , i.e. the frequency spacing of the synthesis filter bank 103 corresponds to the frequency spacing of the analysis filter bank 101 multiplied by the physical conversion factor, and a one-to-one mapping of the analysis subband index to the synthesis subband index n = m can be applied. In other embodiments of the invention, the display of subband indices may depend on details of the filter bank parameters. In particular, if the quotient of the frequency separations of the synthesis filter bank 103 and the analysis filter bank 101 differs from the physical conversion coefficient Q _ϕ , then one or two source subbands may be assigned to a given target subband. In the case of two source subbands, it may be preferable to use two adjacent source subbands with indices n , n +1, respectively. That is, the first and second initial subranges have the form either (n(m), n(m)+1), or (n(m)+1, n(m)).

Processing of the subbands presented in FIG. 2, with a single source subband, will now be described as a function of the subband processing parameters S and Q. Let x(k) be the input signal to the block extractor 201, and let h be the input block step. Those. x(k) - complex-valued signal of the analyzed subband with index n . The block extracted by the block extractor 201 can, without loss of generality, be considered as being defined by L=R ₁ +R ₂ discrete values:

where the integer l is the block count index, L is the block length, and R ₁ , R ₂ are non-negative integers. Note that for Q=1 the block is extracted from successive samples, but for Q greater than 1, downsampling is performed so that the input addresses are stretched by a factor of Q . If Q is an integer, this operation is typically straightforward, while interpolation may be required for non-integer Q values. This statement is also true for non-integer values of the increment h , i.e. for the input block step. In one embodiment of the invention, short interpolating filters, such as filters having two filter stages, can be applied to the complex-valued subband signal. For example, if a discrete value with a fractional time index k +0.5 is required, sufficient quality can be provided by two-link interpolation in the form x(k+0.5)≈ax(k)+bx(k+1) , where coefficients a , b may be constant or may depend on the subband index (see, for example, WO No. 2004/G97794 and WO No. 2007/085275).

An interesting special case of formula (4) is R _i =0, R ₂ =1, where the extracted block consists of a single discrete value, i.e. the length of the block is L =1.

In the polar representation of a complex number z =⏐z⏐exp(i∠z), where ⏐z⏐ is the amplitude of the complex number, and ∠z is the phase of the complex number, nonlinear processing module 202 generates output frame y ₁ from input frame x _i , predominantly determined by the phase modification coefficient T=SQ through:

,

where ρ∈ [0,1] is the geometric weight parameter of the amplitude. The case ρ=0 corresponds to a pure modification of the phases of the extracted block. A particularly attractive value for amplitude weighting is ρ=1-1/T , for which some computational complexity is removed regardless of the block length L , and the resulting transient response is somewhat improved relative to the case of ρ=0 . The phase correction parameter θ depends on the details of the filter bank, as well as the indices of the source and target subbands. In one embodiment of the invention, the phase correction parameter θ can be determined experimentally by sweeping a set of input sinusoids. In addition, the phase correction parameter θ can be obtained by studying the phase difference of adjacent complex sinusoids of the target subband or by optimizing the performance on a Dirac pulse type input signal. Finally, with a suitable design of analysis and synthesis filter blocks 101 and 103, the phase correction parameter θ can be set to zero or omitted. The phase modification coefficient T must be an integer so that the coefficients T -1 and 1 are integers in the linear combination of phases in the first line of formula (5). With this assumption, i.e. Under the assumption that the phase modification coefficient T is an integer, the result of nonlinear processing is quite certain even though the phases are ambiguous due to the identification modulus 2π.

In words, formula (5) states that the phase of the discrete value of the output frame is determined by the phase shift of the corresponding discrete value of the input frame by a constant displacement value. The constant amount of offset may depend on the modification factor T , which itself depends on the subband stretch factor and/or on the subband conversion factor. In addition, the constant amount of the offset may depend on the phase of a particular input frame sample from the input frame. The specified specific input frame sample is held constant to determine the phases of all output frame samples for a given block. In the case of formula (5), the phase of the central discrete value of the input frame is used as the phase of the special discrete value of the input frame.

The second line of formula (5) states that the amplitude of the output frame sample can depend on the amplitude of the corresponding input frame sample value. In addition, the amplitude of the output frame sample may depend on the amplitude of a particular input frame sample. This particular input frame sample can be used to determine the amplitude of all output frame samples. In the case of formula (5), the central sample value of the input frame is used as a special discrete value of the input frame. In one embodiment of the invention, the amplitude of an output frame sample may correspond to the geometric average of the amplitude of a corresponding input frame sample and a particular input frame sample.

In windowing module 203, a window w of length L is applied to the output frame, resulting in a windowed output frame:

Finally, all frames are assumed to be padded with zeros, and the overlay and add operation 204 is defined as

where it should be noted that the overlay and addition unit 204 applies the block step Sh , i.e. a time step that is S times larger than the step h of the input block. Due to the indicated difference in time steps according to formulas (4) and (7), the duration of the output signal z(k) is S times greater than the duration of the input signal x(k) , i.e. the synthesized subband signal is stretched by a subband stretch factor S compared to the analyzed subband signal. It should be noted that this observation generally applies when the window length L is negligible compared to the signal duration.

In the case when a complex sinusoid is used as an input signal for processing 102 subbands, i.e. the signal of the analyzed subband corresponds to a complex sinusoid:

Using formulas (4)-(7), we can determine that the output signal for processing 102 subbands, i.e. the corresponding signal of the synthesized subband has the form:

regardless of ρ . Thus, a complex sinusoid with a time-discrete frequency ω will be transformed into a complex sinusoid with a time-discrete frequency Qω , provided that the synthesis window is shifted with a step Sh , which in total for all k leads to the same constant K :

To illustrate, consider the special case of a pure transformation, where S =1 and T = Q. If the input block step h =1 and R ₁ =0, R ₂ =1, all of the above, i.e. Formula (5) is reduced to a pointwise or discrete value-based phase modification rule:

To set certain processing parameters, i.e. block lengths in the block extractors, the subband processing module 102 may use the control data 104.

Below, the description of subband processing will be expanded to cover the case presented in FIG. 3, with two sub-band input signals. Let x ⁽¹⁾ (k) be the input subband signal to the first block extractor 301-1, and let x ⁽²⁾ (k) be the input subband signal to the second block extractor 301-2. Each extractor may use a different downsampling factor, resulting in the blocks extracted:

Nonlinear processing 302 generates an output frame y ₁ and can be defined as

the processing at 303 is again described by formulas (6) and (7), and the processing 204 is identical to the superposition and addition processing described in the context of the single input signal case.

The definition of the non-negative real parameters D ₁ , D ₂ , ρ , the non-negative integer parameters T ₁ , T ₂ and the synthesis window w now depends on the required operating mode. Note that if the same subband is supplied to both inputs, x ⁽¹⁾ (k)=x ⁽²⁾ (k) and D ₁ = Q , D ₂ = Q , T ₁ =1, T ₂ =T- 1, then the operations according to formulas (12) and (13) are reduced to operations according to formulas (3) and (4) in the case of a single input signal.

In one embodiment of the invention, where the ratio of the frequency spacing Δƒ _s of the synthesis filter bank 103 and the frequency spacing Δƒ _A of the analyzing filter bank 101 differs from the required physical conversion coefficient Q _ϕ , it may be useful to determine discrete values of the synthesized subband with index m from the two analyzed subbands , respectively, with indices n , n +1. For a given index m, the corresponding index n can have the form of an integer value obtained by truncating the value n of the analyzed index, which has the form according to formula (3). One of the analysis subband signals, for example, the analysis subband signal corresponding to index n , is supplied to the first block extractor 301-1, and the other analysis subband signal, for example, the signal corresponding to index n +1, is supplied to the second block extractor 301-2. Based on these two signals of the analyzed subbands, the signal of the synthesized subband corresponding to the index m is determined by the processing described above. The assignment of adjacent signals of the analyzed subbands to the two block extractors 301-1 and 302-1 may be based on the remainder that is obtained by truncating the index value according to formula (3), i.e. on the difference between the exact index value, which has the form according to formula (3), and the truncated integer value nobtained according to formula (3). If the remainder is greater than 0.5, the analysis subband signal corresponding to index n may be assigned to the second block extractor 301-2, otherwise, the analysis subband signal may be assigned to the first block extractor 301-1. In a given operating mode, parameters can be designed so that the signals of the input subbands share the same complex frequency ω :

which results in an output subband signal that is a complex sinusoid with a discrete time frequency Qω . It turns out that this happens when the following relations are true:

For the operating mode of generating missing harmonics through cross products, the design criteria are different. Returning to the physical transformation parameter Q _ϕ , the purpose of adding cross products is to generate an output signal at frequencies Q _ϕ Ω+rΩ ₀ , where r =1, ... , Q _ϕ -1, for given input signals at frequencies Ω and Ω+Ω ₀ , where Ω ₀ is the fundamental frequency belonging to the predominant component of the fundamental tone of the input signal. As described in WO 2010/081892, selective addition of these terms will result in the harmonic series being filled and the spurious pitch artifact being significantly reduced.

Below, a design algorithm for the operation of the cross-processing control 404 will be described. For a given index m of the target output subband, parameter r =1, ... , Q _ϕ -1 and fundamental frequency Ω _0, the corresponding indices n ₁ and n ₂ of the source subbands can be obtained by solving in an approximate sense the following system of equations:

where σ =1/2 for non-uniformly spaced filter bank modulation (which is typically used for QMF and MDCT filter banks) and σ=0 for uniformly spaced filter bank modulation (which is typically used for FFT filter banks).

For definitions

• p = Ω ₀ /Δ ƒ _A - fundamental frequency, measured in units of frequency spacing of the block of analyzing filters;

• F=Δ ƒ _s /Δ ƒ _A - the ratio of the frequency separations of the synthesized and analyzed subbands; And

• n ^f =((m+σ)F-rp)⋅Q _ϕ - σ - real-valued target index for the lowest source index with an integer value,

An example of a preferential approximate solution to the system of equations (16) is the choice of n ₁ as the integer closest to n ^f , and n ₂ as the integer closest to n ^f + p .

If the fundamental frequency is less than the frequency spacing of the analysis filter block, i.e. if p is less than 1, it may be preferable to avoid adding cross products.

As stated in WO 2010/081892, the cross product should not be added to an output subband that already contains a significant main contribution from the non-cross product transform. Moreover, at most one of the cases r =1, ... , Q _ϕ -1 should contribute to the cross product. In this specification, these rules can be implemented by performing the following three steps for each index m of the target output subband:

( _​ ​ ​ ​ _​ ​ ^​ ​ ^{​ ​} ​ or in its vicinity), where the original subranges x ⁽¹⁾ and x ⁽²⁾ can have the form of indices n ₁ and n ₂ , as in equation (16);

2. Calculate the corresponding amplitude M _s for the direct source term ⏐ x ⏐ obtained from the source subrange with index n=(F/Q _ϕ )m (cf. equation (3));

3. Operate the winning cross term for M _c in step 1 above only if M _c is greater than qM _s , where q is a predetermined threshold value.

Depending on your specific system configuration, modifications to this procedure may be desirable. One such change is the replacement of the hard threshold in stage 3 with softer rules depending on the quotient M _c /M _s . Another change is to extend the maximization in step 1 to more than Q _ϕ - 1 options, for example, determined by a finite list of candidate values for the fundamental frequency, measured in units of p spacing of the analyzed frequencies. Another change is the use of other quantitative indicators of the amplitudes of the subranges, such as the amplitude of a fixed discrete value, the maximum amplitude, the average amplitude, the amplitude in the sense of the l ^ϕ -norm, etc.

The list of target source bands m selected for adding the cross product, together with the values n ₁ and n ₂ constitutes the main part of the cross processing control data 403. It remains to describe the configuration parameters D ₁ , D ₂ , ρ , the non-negative integer parameters T ₁ , T ₂ that appear during phase alternation (13), and the synthesis window w , intended for use in cross-processing 402 subbands. Inserting a sinusoidal model into a cross-product situation results in the following original subband signals:

where ω=2πΩΔt _A and ω ₀ =2πΩ ₀ Δt _A. Similarly, the required output subband has the form

Calculations reveal that this target output signal can be obtained if condition (15) is satisfied together with

Conditions (15) and (19) are equivalent to the following expression:

which defines the integer coefficients T ₁ , T ₂ for phase modification in (13) and provides some freedom in design when specifying the values of the downsampling coefficients D ₁ , D ₂ . The weighting parameter of the amplitudes can advantageously be chosen as ρ=r/Q _ϕ . As can be seen, these configuration parameters depend only on the fundamental frequency Ω ₀ through the choice of r . However, in order for equation (18) to be true, a new condition arises for the synthesis window w , namely:

A synthesis window w that either exactly or approximately satisfies condition (21) must be provided in the last cross-processing control data element 403.

Note that the above algorithm for calculating cross-processing control data 403 based on input parameters such as target output subband index m and fundamental frequency Ω ₀ is purely illustrative and, as such, does not limit the scope of the invention. Modifications to this disclosure within the knowledge and everyday experience of those skilled in the art, such as an additional subband block-based processing method producing a signal (18) as an output signal in response to input signals (17), are fully within the scope of the present invention.

Fig. 5 illustrates an example scenario for applying subband block based transform using multiple transform orders in an HFR enhanced audio codec. The transmitted bit stream is received by the base decoder 501, which produces a low-bandwidth decoded base signal at the sampling frequency f _s . The decoded low-bandwidth signal is resampled to an output sampling rate of 2 f _s using a 32-band complex modulated QMF analyzer block 502, followed by a 64-band synthetic QMF (inverse QMF) block 505. Both filter banks 502 and 505 share the same physical parameters Δt _s = Δt _A and Δƒ _s =Δƒ _A , and the HFR processing module 504 simply passes the unmodified low-frequency subbands corresponding to the base low-bandwidth signal. The high frequency content of the output signal is obtained by feeding the higher frequency subbands into the 64-band QMF synthesis block 505 with the output bands from the multiple converter module 503, subjecting it to spectrum shaping and modification by the HFR processing module 504. The multiple converter 503 takes as an input the decoded base signal and outputs a plurality of subband signals that represent a 64-band superposition analysis, or combination of several converted signal components. The goal is that, if HFR processing is bypassed, each component corresponds to an integer physical transformation without stretching the underlying signal in time ( Qϕ ₌ 2, 3 ... , and _Sϕ = 1). In the scenario according to the invention, the converter control signal 104 contains data describing the fundamental frequency. This data can either be transmitted via a bit stream from the corresponding audio encoder, or output by pitch detection at the decoder, or obtained from a combination of transmitted and detected information.

Fig. 6 illustrates an example of a multi-order transform operation scenario based on a subband block using a single 64-band QMF analysis filter bank. Here, three transform orders Q _ϕ =2, 3, 4 are subject to generation and delivery into the region of a 64-band QMF operating at a sampling rate of 2 fs . The merge module 603 simply selects and combines the significant subbands from the branches of each of the transform orders into a single set of QMFs. -subbands to be fed to the HFR processing module. The goal in particular is that the processing chain consisting of 64-band QMF analysis 601, Q _ϕ subband processing module 602, and 64-band QMF synthesis 605 results in a physical conversion with Q _ϕ and S _ϕ = 1 (i.e. without stretching). When identifying three of these blocks by 101, 102 and 103 in FIG. 1 it is found that Δ t _A =64fs and Δƒ _A =fs/128 , therefore Δ t _S / Δ t _A =1/2 and F =Δƒ _S / Δƒ _A =2 . The design of specific configuration parameters for the 602- Q _ϕ will be described for each of the Q _ϕ =2, 3, 4 cases separately. For all cases, the analysis step is chosen as h =1, and it is assumed that the normalized fundamental frequency parameter ρ=Ω ₀ /Δƒ _A =128Ω ₀ / ƒ s is known.

First of all, consider the case Q _ϕ =2 . In this case, 602-2 should perform subband stretching with a factor of S =2 and subband conversion with a factor of Q =1 (i.e., no conversion), the correspondence between the source n and target m subbands for direct subband processing is n = m . In the cross product addition scenario of the invention, there is only one type of cross product to consider, namely r =1 (see above discussion after equation (15)), and equations (20) reduce to T ₁ = T ₂ =1 and D ₁ + D ₂ =1. An example solution consists of choosing D ₁ =0 and D ₂ =1 . A rectangular window of even length L=10 with R ₁ =R ₂ =5 can be used as a synthesis window for direct processing, since it satisfies condition (10). For the cross-processing synthesis window, a short window with L =2 links and R ₁ =R ₂ =1 may be used in order to keep the additional complexity of adding cross products to a minimum level. However, the beneficial effect of using a long block to process subbands is most significant in the case of complex audio signals, where unwanted combinational terms are suppressed; in the case of a dominant fundamental tone, the occurrence of these artifacts is less likely. A window with L =2 links is the shortest of those that can satisfy condition (10), since h =1 and S =2. However, according to the present invention, the window preferably satisfies condition (21). Given the existing parameters, this is equivalent to the following condition:

which is done by choosing w (0)=1 and w (-1)=exp(i α )=exp( iπp /2).

For the case of Q _ϕ =3, the technical conditions for 602-3, having the form of conditions (1)-(3), are such that it must perform the stretching of the subband S=2, the transformation of the subband Q =3/2, and the correspondence between the original n and the target m subranges for processing direct terms has the form n≈2m/3 . There are two types of cross product terms - r =1, 2, - and equations (20) reduce to

.

An example solution consists of choosing the downsampling parameters as

• D ₁ =0 and D ₂ =3/2 for r =1;

• D ₁ =3/2 and D ₂ =0 for r =1.

As a synthesis window for direct processing, you can use a rectangular window of even length L = 8 with R ₁ = R ₂ = 4. A short window with L = 2 links and R ₁ = R ₂ = 1, satisfying the condition, can be used as a window for processing cross products

which is done by choosing w (0)=1 and w (-1)=exp(i α ).

In the case of Q _ϕ =4, the technical conditions for 602-4, having the form of conditions (1)-(3), are such that it must perform subband stretching with a coefficient of S=2, subband transformation with a coefficient of Q =2, and correspondence between the original n and target m subranges for processing direct terms is of the form n = 2m . There are three types of cross product terms, - r =1, 2, 3, - and equations (20) reduce to

An example solution is to choose

• D ₁ =0 and D ₂ =2 for r =1;

• D ₁ =0 and D ₂ =1 for r =2;

• D ₁ =2 and D ₂ =0 for r =3.

As a synthesis window for direct processing, you can use a rectangular window of even length L =6 with R ₁ =R ₂ =3. A short window with L = 2 links and R ₁ = R ₂ = 1, satisfying the condition, can be used as a window for processing cross products

which is done by choosing w (0=1) and w (-1)=exp(iα).

In each of the above cases where more than one value of r is applicable, a selection will occur, for example, similar to the three-step procedure described before equation (17).

Fig. 7 shows the amplitude spectrum of a harmonic signal with a fundamental frequency Ω ₀ = 564.7 Hz. The low frequency portion 701 of the signal is to be used as an input signal for a multiple converter. The purpose of the converter is to generate a signal as close as possible to the high frequency portion 702 of the input signal, so that transmission of the high frequency portion 702 becomes unnecessary and the available data bit rate can be used sparingly.

Fig. 8 shows the amplitude spectrum of the output signals from a converter that contains as an input signal the low frequency portion 701 of the signal shown in FIG. 7. A multiple converter is constructed by using 64-band QMF filter banks and an input sampling frequency fs =14400 Hz as described in FIG. 5. However, for clarity, only two orders of transformation Q _ϕ =2,3 are considered. Three different panels 801-803 represent the final output signal obtained by using different cross-processing control data settings.

The top panel 801 depicts the output spectrum obtained when all cross product processing is disabled and only direct subband processing 401 is active. This will be the case where the cross-processing control 404 does not accept the pitch, or p =0. Conversion by Q _ϕ =2 generates an output signal in the range of 4-8 kHz, and conversion by Q _ϕ =3 generates an output signal in the range of 8-12 kHz. As can be seen, the harmonics created are widely and increasingly far apart, and the output signal deviates significantly from the target high frequency signal 702. The resulting audio output signal will have audible double and triple pitch artifacts.

Middle panel 802 depicts the output spectrum obtained when cross product processing is active, using the pitch parameter p = 5 (which serves as an approximation of 128Ω ₀ / f _s = 5.0196), but using a simple two-link synthesis window with w for cross-product processing of subbands (0)= w (-1)=1, satisfying condition (10). This is equivalent to a direct combination of subband block-based processing and cross-product-enhanced harmonic transform. As you can see, the additional components of the output signal compared to 801 do not completely coincide with the required harmonic series. This shows that when using a procedure inherited from direct subband processing to process cross-products, the processing results in sound quality that is insufficient for use.

The bottom panel 803 depicts the output spectrum obtained using the same scenario as for the middle panel 802, but, in this case, with cross-band synthesis windows having the form of the formulas described in the case of Q _ϕ =2.3 according to FIG. 5, that is: a two-tier synthesis window in the form w (0)=1 and w (-1)=exp(i α ), satisfying condition (21), and with the characteristic feature indicated by the present invention, which is that it depends on r . As can be seen, the combined output matches the required harmonic series 702 very well.

Fig. 9 shows the nonlinear processing section of frame processing module 202, including sections configured to receive two input samples u ₁ , u ₂ and to generate from them a processed sample value w , the amplitude of which is the geometric average of the amplitudes of the input values, and the phase of which is is a linear combination of the phases of the input discrete values, i.e.:

According to this description, the processed value w can be obtained by pre-normalizing each of the sample values u ₁ , u ₂ in the corresponding pre-normalizer 901, 902 and multiplying the pre-normalized input sample values v ₁ = u ₁ /⏐ u ₁ ⏐ ^a , v ₂ = u ₂ /⏐ u ₂ ⏐ ^b in the weighted multiplier 910, which outputs w = v ₁ ^α v ₂ ^β . It is clear that the operation of the prenormalizers 901, 902 and the weighted multiplier 910 is determined by the input parameters a, b, α and β. It is easy to verify that equations (22) will be satisfied if α =T ₁ , β=T ₂ , a=1-ρ/T ₁ , b=1-(1-ρ)/T ₂ . Those skilled in the art can easily generalize this circuit to an arbitrary number of N ₀ input samples, where the multiplier is fed N ₀ input samples, some or all of which have been normalized. It will then be found that overall pre-normalization ( a = b , assuming pre-normalizers 901, 902 generate the same results) is possible by setting ρ to ρ= T ₁ /( T ₁ + T ₂ ). This results in a computational advantage when a large number of subbands are considered, since a common pre-normalization step can be performed on all candidate subbands before multiplication. In the preferred hardware implementation, a number of identically functioning prenormalizers are replaced by a single module, which alternates between discrete values from different subranges with a time separation.

Further embodiments of the present invention will become apparent to those skilled in the art upon reading the above description. Although the present description and drawings disclose embodiments and examples of the invention, the invention is not limited to these specific examples. Numerous modifications and changes may be made without departing from the scope of the present invention, which is defined by the accompanying claims.

The systems and methods disclosed herein above may be implemented as software, firmware, hardware, or a combination thereof. Some or all of the components may be implemented as software executed by a digital signal processor or microprocessor, or they may be implemented as hardware or a special purpose integrated circuit. The software may be distributed on machine-readable storage media, which may include computer storage media (or enduring media) and communications media (transitory media). As is well known to those skilled in the art, computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, CD-ROM or other optical disk media, magnetic cassettes, magnetic tape, magnetic disk media or other magnetic storage media. devices, or any other storage medium that can be used to store the required information and for a computer to access it. As is also well known to those skilled in the art, communications media typically include machine-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transmission mechanism, and includes any means of delivering information.

Claims (22)

1. A system configured to generate a time-stretched and/or frequency-converted signal from an input signal, the system comprising one or more processing elements that: obtaining a certain number Y≥1 of analyzed subband signals from the input signal, wherein each analyzed subband signal includes a number of complex-valued analyzed discrete values, each of which has a phase and amplitude; generating a signal of the synthesized subband from Y signals of the analyzed subbands using the subband conversion coefficient Q and the subband stretching coefficient S, wherein at least one of the coefficients Q and S is greater than one, by: generating Y frames consisting of L input discrete values, wherein each frame is extracted from a specified number of complex-valued analyzed discrete values in the signal of the analyzed subband, wherein L is a frame length that is greater than 1, and at least one of the L input discrete values values are obtained by interpolating two or more of a series of complex-valued discrete values being analyzed; applying the jump value of a block of h samples to a specified series of complex-valued analyzed samples before generating the next frame consisting of the L input samples, thereby generating a sequence of frames from the input samples; generating, based on Y corresponding frames of input samples, a frame of processed samples by determining the phase and amplitude for each processed sample of the frame, wherein for at least one processed sample: i) the phase of the processed sample is based on the corresponding phases of the corresponding input samples in each of the Y input sample frames; And ii) the amplitude of the processed sample is determined as the average of the amplitude of the corresponding input sample in the first frame of Y frames of input samples and the amplitude of the corresponding input sample in the second frame of Y frames of input samples; applying a window function to a frame of processed discrete values, wherein the window function is a rectangular window with a length corresponding to the length L of the frame; And determining the signal of the synthesized subband by superimposing and adding the discrete values of the sequence of frames to which the window function was applied, the processed discrete values; And generating a time-stretched and/or frequency-converted signal from the synthesized sub-band signal, wherein the system is operational at least for Y=2. 2. A method for generating a time-stretched and/or frequency-converted signal from an input signal, the method comprising: obtaining a certain number of Y≥2 signals of the analyzed subbands from the input signal, wherein each signal of the analyzed subband includes a number of complex-valued analyzed discrete values, each of which has a phase and amplitude; generating Y frames consisting of L input discrete values, wherein each frame is extracted from a specified number of complex-valued analyzed discrete values in the signal of the analyzed subband, wherein L is a frame length that is greater than 1, and at least one of the L input discrete values values are obtained by interpolating two or more of a series of complex-valued discrete values being analyzed; applying a jump value of a block of h samples to said series of complex-valued analyzed samples before obtaining the next frame consisting of the L input samples, thereby generating a sequence of frames from the input samples; generating, based on Y corresponding frames of input samples, a frame of processed samples by determining the phase and amplitude for each processed sample of the frame, wherein for at least one processed sample: i) the phase of the processed sample is based on the corresponding phases of the corresponding input samples in each of the Y input sample frames; And ii) the amplitude of the processed sample is determined as the average of the amplitude of the corresponding input sample in the first frame of Y frames of input samples and the amplitude of the corresponding input sample in the second frame of Y frames of input samples; determining the signal of the synthesized subband by applying a window function to a frame of processed discrete values, and superimposing and adding the discrete values of a sequence of frames to which the window function was applied, the processed discrete values, wherein the window function is a rectangular window with a length corresponding to the length L of the frame; And generating a time-stretched and/or frequency-converted signal from a synthesized subband signal. 3. A permanent storage medium on which machine-readable commands are stored, intended for executing the method according to claim 2.