DE102017102988B4 - Method and device for processing a digital audio signal for binaural reproduction - Google Patents

Abstract

A method of processing a digital audio signal presented as a 3D sound field description in Higher Order Ambisonics HOA format of order N, producing a left output signal and a right output signal for binaural reproduction via headphones, comprising the steps of: rotating (110) the 3D sound field description of the digital audio signal in Higher Order Ambisonics HOA format according to a rotation control signal obtainable from a head tracker, filtering (120) the rotated 3D sound field description of the digital audio signal with (N + 1) first rendering filters (21L) according to the first one head-related impulse responses (HRIRs) for a left-channel and (N + 1) second rendering filters (21R) according to second head-related impulse responses (HRIRs) for a right channel, the rendering filters corresponding to an approximation of the first and second head-related impulse responses by spherical functions, and the Approximation is limited to the order N, w wherein the spherical functions are the same as in the Higher Order Ambisonics HOA format of the sound field description, summing (130) the outputs of the first rendering filters (21L) to an intermediate signal for the left channel, and summing (130) the outputs of the second rendering filters (21R) to an intermediate signal for the right channel, filtering (140) the intermediate signal for the left channel with a first equalization filter (30L) for changing the timbre, wherein the first equalization filter is automatically adjusted according to the order N, and wherein the left output signal ( 40L) for binaural reproduction, and filtering (150) the right channel intermediate signal with a second equalization filter to alter the timbre, the second equalization filter being automatically adjusted according to the N order, and the right output signal (40R) for binaural reproduction arises.

Description

The invention relates to a method and a device for processing a digital audio signal, which is present as a 3D sound field description, to a suitable for binaural playback via headphones output signal.

background

Higher Order Ambisonics (HOA) is an approach to represent physical sound pressure fields in space. The underlying idea is that any 3-dimensional sound field that is free of sound sources can be represented by superimposing plane waves from all possible directions of incidence [7, Chap.2.4], [11, Chap.7.3.3], similar to the principle the representation of conventional audio signals by superimposing sine waves of different frequencies. In particular, the contributions of each individual plane wave to the sound pressure field at a specific position in space can be expressed by a function defined on the surface of a unit sphere, since each direction of incidence can be mathematically unambiguously assigned to a specific position on the unit sphere. With HOA this function is approximated as a broken series of spherical basis functions, the so-called spherical harmonics (SH). The actual HOA (coefficient) signals are the time sequences of the weights of individual spherical-harmonic functions. The point at which the SH series is broken off, is the so-called order N of the HOA signal, which determines the number of HOA signals to be (N + 1) 2, thereby defining the spatial resolution of the sound field representation. In general, individual directional contributions to the sound field can be resolved the better, the higher the order N is.

In practice, HOA sound field representations can either be artificially generated or acquired using a microphone array (see, e.g., [13], [6]). In the second case, the meaningfully chosen order depends on the number of microphones. In the currently most commercially available spherical arrays, such as mhacoustics' own mike with 32 microphone capsules or the visiSonics audio-visual camera with 64 microphone capsules, the order is not higher than N = 7, often much lower.

The playback of HOA sound field representations via headphones requires special signal processing called binaural rendering. This typically takes into account the fact that the human ear is able to locate the 3-dimensional position of individual sound sources by so-called binaural cues. These include differences in intensity and spectral composition, as well as the temporal offset between the sound pressure signals on the two eardrums. These features are due to the scattering that occurs when sound waves strike the human body, especially the pinna, head and trunk. This scattering is modeled by so-called head-related impulse responses (HRIRs), which usually vary significantly individually.

A sound field represented by HOA by definition contains no information about a scattering caused by the body of the listener. Therefore, this information must be artificially synthesized in binaural rendering. To achieve maximum presence in a virtual acoustic environment, the binaural features must be consistent with the listener's head movements. A head tracker can therefore be used to obtain the relevant information for adapting the rendering process. Another advantage of head trackers in this regard is that they help to improve the ability to localize sound sources by inherently detecting small unconscious head movements [2, Chap.3].

For the binaural rendering of Ambisonics first-order sound field representations, for example, the authors note in [3] that, in principle, only a single linear filter per individual ambisonic signal and ear is required (see [3, chap.4]), resulting in a total of 2 (1 + 1) ² = 8 linear filters for an Ambisonics first-order sound field representation. Specifically, the authors propose a two-stage binaural rendering process to determine the filters [3, Chap. 6.1], where the first stage renders the Ambisonics sound field representation for a group of virtual loudspeaker signals which in the second stage match the respective HRIRs for the left and right ear are filtered. An essential assumption in this connection is that the directions of the HRIRs coincide with the directions of the virtual loudspeaker signals.

To adapt the binaural rendering to a head rotation of the listener, the authors use the fact that there is no difference between a rotation of the listener's head and inverse rotation of the sound field. Therefore, the speakers are not rotated there, so that differences between the directions of the rotated speakers and the given directions of the HRIRs are avoided. Instead, the sound field is rotated directly in the ambisonics area (domain), resulting in the use of quadratic rotation matrices of dimension 4 to the ambisonic signals [4, Chap. 2], [5]. The same technique is used for HOA sound field representations of arbitrary orders N in [3], while the alternative strategy of turning the virtual loudspeakers is used eg in [7, ch. 2], [8, ch.3.6], [9] and [10, ch.3.5]. In doing so, the HRIRs for evaluation have to be elaborately interpolated at the directions of the rotated virtual loudspeaker positions.

In contrast, in [11] a single-stage binaural rendering for HOA sound field representations is suggested, which avoids the concept of virtual speakers by working directly in the SH domain. For this, the directional dependence of the HRIRs is approximated by a series of spherical harmonics, which are then at the same order N how the HOA sound field representation to be reproduced is aborted. Canceling the series not only limits the spatial resolution, but also greatly attenuates high spectral-temporal frequencies, which affects the timbre of the rendered binaural signal. To compensate for this change in timbre, the authors suggest in [12], [13] a downstream filtering of each of the rendered binaural signals, which amplifies the high spectro-temporal frequencies.

Summary of the invention

It is an object of the present invention to provide a method and apparatus for processing a digital audio signal that is a 3D sound field description in HOA format N present, wherein a left and a right output signal for a binaural reproduction are generated via headphones.

A method according to the invention is specified in claim 1. A device according to the invention is specified in claim 11. Another device according to the invention is specified in claim 15.

Such a method according to the invention includes a rotation of the 3D sound field description of the digital audio signal in HOA format according to a direction control signal, filtering the rotated 3D sound field description with head-related impulse response (HRIR) rendering filters for a left channel and a Right channel, a summation of the output signals of the rendering filters to intermediate signals respectively for the left channel and the right channel, and a correction of the intermediate signals, each with an equalization filter to change the timbre, the two equalization filters automatically corresponding to the order N be adjusted. This produces the left and the right output signal for the binaural reproduction.

In one embodiment, an apparatus for processing a digital audio signal according to the invention has at least one processor and memory, the memory storing a program that configures the processor to perform a method as described above.

Further advantageous embodiments are described in the dependent claims 2-10.

list of figures

• 1 einen Überblick über die Verarbeitung eines digitalen Audiosignales im HOA-Format;
• 2 ein vereinfachtes Flussdiagram eines Verfahrens;
• 3 das Prinzip der Dimensionierung der Renderingfilter;
• 4 exemplarische Frequenzgänge der Equalizationfilter und ihrer normierten Eingangssignale; und
• 5 eine skalierbare Filterstruktur der Equalizationfilter.

Further details and advantageous embodiments are shown in the drawings. It shows

• 1 an overview of the processing of a digital audio signal in HOA format;
• 2 a simplified flow diagram of a method;
• 3 the principle of sizing the rendering filters;
• 4 exemplary frequency responses of the equalization filters and their normalized input signals; and
• 5 a scalable filter structure of the equalization filter.

Detailed description of the invention

1 shows an overview of the inventive processing of a digital audio signal, the 3D sound field description in the High Order Ambisonics HOA format of an order N is present, with a left output signal and a right output signal for a binaural reproduction can be generated via headphones.

First, the 3D sound field description of the digital audio signal in HOA format c ^(N) (t) according to a rotation control signal 16 turned 10 , This can eg by a head tracker 15 , a camera or similar be received. The rotation takes place 10 Inverse to the rotation of the user's head, so that the user has the impression that the sound field remains spatially fixed despite head movements.

The rotated 3D sound field description 12 of the digital audio signal becomes (N + 1) ² first rendering filters 21L according to first head-related impulse responses (HRIR) filtered for a left channel. In addition, the rotated 3D sound field description 12 of the digital audio signal also with (N + 1) ² second rendering filters 21R filtered according to second head related impulse responses (HRIRs) for a right channel. The rendering filters 21L . 21R generate from the sound field description 12 a playback format, ie they convert the sound field description into audio channels. The rendering filters 21L . 21R are dimensioned so that their impulse responses the time course of the weights of spherical functions one on the order N represent limited series representation by spherical-harmonic functions of the first and second head-related impulse responses (HRIRs). Due to the approximation, these are on the order N limited. The spherical functions are the same as those used in the format of the sound field description, which facilitates the calculations.

The output signals of the first rendering filter 21L are added up to an intermediate signal for the left channel 25L , and the output signals of the second rendering filter 21R are added up to an intermediate signal for the right channel 25R ,

Then the intermediate signal for the left channel with a first Equalisierungsfilter 30L Filtered to change the timbre, using the first equalization filter 30L automatically according to the order N is adjusted. This creates the left output signal 40L for binaural playback. Similarly, the intermediate signal for the right channel with a second Equalisierungsfilter 30R Filtered to change the timbre, with the second equalization filter automatically according to the order N is adjusted, and where the right output signal 40R for the binaural reproduction arises. The output signals 40L . 40R for binaural playback may optionally be amplified if necessary and for playback on a headphone 50 are given. At this example, a head tracker 15 be attached to receive head movements and the above-mentioned rotation control signal 16 to deliver.

In one embodiment, an order parameter corresponding to the order N set the 3D sound field description, read from a memory or otherwise detected or received. This order parameter is used to configure at least the equalization filters 30L . 30R and the first and second rendering filters 21L . 21R , Optionally, other blocks can also be adapted according to the order parameter.

Optionally, for example, the first and second rendering filters 21L . 21R with first and second adders 25L . 25R for summing their respective output signals to a first processing unit 20 be summarized as in 1 shown. The processing unit 20 can in other variants, the equalization filter 30L . 30R and / or a rotation unit 10 for rotating the 3D sound field description of the digital audio signal.

2 shows a flowchart of a method for processing a digital audio signal, the 3D sound field description in the HOA format of an order N present, wherein a left and a right output signal for a binaural reproduction are generated via headphones. The procedure involves turning 110 the 3D sound field description of the digital audio signal in HOA format according to a rotation control signal, a filtering 120 the rotated 3D sound field description with head-related impulse response (HRIR) rendering filters for a left channel and a right channel, a summation 130 the output signals of the rendering filters to intermediate signals respectively for the left channel and the right channel, and a correction 140 . 150 the intermediate signals, each with an equalization filter to change the timbre, with at least the Equalisierungsfilter and the rendering filter automatically according to the order N be adjusted. This produces the left and the right output signal 40L . 40R for binaural playback.

In one embodiment, the method or portions thereof are implemented as program code so that it may be executed by a computer configured according to the program code.

The correction 140 . 150 the intermediate signals, each with an equalization filter for the left and right sides, are normally taken simultaneously. With sufficient processing capacity, however, the correction of the two intermediate signals can also take place one after the other, if at least the first result is buffered, so that both results can be output simultaneously. The same, respectively adapted hardware filters or software routines can be used for both channels. In this case, the two channels (L, R) can be processed in any order. The same applies in principle also to the first and second rendering filters 21L . 21R ,

In one embodiment, the adjustment of the first and second equalization filters effects or approximates 30L . 30R a change in timbre corresponding to an increase in the order of the 3D sound field description as well as the rendering filter of the order N to a higher order N _H. In other words, the timbre attainable by a higher order N _H is modeled by the elevation enhancement, though the signal is only in the lower order N is present and processed.

The Equalization filter 30L , 30R basically cause an increase in high frequencies. In one embodiment, high frequencies are those frequencies in the audible range obtained by the approximation of the first and second head related impulse responses (HRIRs) for the rendering filters 21L . 21R be steamed. Here, the adaptation of the first and the second Equalisierungsfilters according to the order includes N or the order parameter that the increase is more pronounced for lower orders than for higher orders N (at constant N _H ). Preferably, however, lower frequencies are produced by the equalization filters 30L . 30R not or only very little steamed.

In one embodiment, the order parameter becomes N is set automatically according to the 3D sound field description of the digital audio signal and the number of the first and second rendering filters is determined by the order N or the order parameter modified. It will increase the order N also increases the number of filters, with the lower order filters still being used. This creates a scalable structure, as explained below with reference to 5 explained in more detail.

The coefficients of the first and second equalization filters 30L . 30R result in principle from the rendering filters, in particular from a quotient of the magnitude spectra of the intermediate signals from rendering filters for a higher order N _H and magnitude spectra of the intermediate signals from rendering filters for the actual order N , assuming a diffuse sound field as the input signal.

In one embodiment, the first and second equalization filters are 30L . 30R Infinite Impulse Response (IIR) filter, and both are the same. Their coefficients can be calculated, for example, from an average value of the two sides.

In one embodiment, the impulse responses of the rendering filters place the time history of the weights of spherical functions on order N In one embodiment, during initialization, filter coefficients of the rendering filters are retrieved from a memory containing coefficients for a largest order N _max , the coefficients for smaller orders are a subset of the largest order coefficients N _max (corresponding to the scalable structure described below).

In one embodiment, the digital audio signals are transformed into the frequency domain by means of FFT, the coefficients of the rendering filters are present in the frequency domain, and the output signals are each transformed into the time domain by means of inverse FFT after the summation. In this case, advantageously only one inverse FFT is required per output signal.

In one embodiment, the 3D sound field description in HOA format includes time discrete sample samples and the rotation control signal is also time discrete. In this case, the rotation control signal is temporally interpolated before the rotation in order to correspond to the sampling instants of the audio samples. In the simplest case, interpolation between two values of the rotation control signal, but it can also be more.

In an embodiment where also the 3D sound field description includes time discrete sample samples and the rotation control signal is time discrete, two or more consecutive values of the rotation control signal are applied to calculate two or more rotations and the audio samples of the two or more rotated 3D sound field representations are temporally interpolated ,

In the following, further details of various components and steps mentioned above will be explained.

HOA rotation

aus der ursprünglichen, oben beschriebenen HOA-Schallfelddarstellung c^(N)(t) durch eine einfache Matrixoperation gemäß $$c_{\text{ROT}}^{\left(N\right)}\left(t\right)=\mathbf{R}\cdot {\mathbf{c}}^{\left(N\right)}\left(t\right),$$

erhalten werden, wobei R ∈ ℝ^{(N+1) 2×(N+1) 2} die Rotationsmatrix bedeutet.Diese hat eine blockweise diagonale Struktur (siehe [1, Kap.2], [4]). Typischerweise liefert ein Headtracker zeitdiskrete Rotationsparameter mit einer Abtastrate, die deutlich geringer als die Audio-Abtastrate ist. Prinzipiell muss die Abtastrate des Headtrackers angeglichen werden. Um hier jedoch Diskontinuitäten in der gedrehten HOA-Schallfelddarstellung zu vermeiden, die aus plötzlichen Änderungen der Drehung bzw. Position herrühren können, wird in einer Ausführungsform Interpolation genutzt. Die Interpolation kann im Bereich (domain) der Rotationsmatrizen oder der gedrehten HOA-Schallfelddarstellung angewandt werden.The rotation or an HOA rotation block causes a rotation of an HOA sound field representation (HOA sound field representation), which is inverse to the rotation of the head of a listener. In principle, the rotated HOA sound field representation $c_{\text{RED}}^{\left(N\right)}\left(t\right)$

from the original HOA sound field representation c ^(N) (t) described above by a simple matrix operation according to FIG $$c_{\text{RED}}^{\left(N\right)}\left(t\right)=\mathbf{R}\cdot {\mathbf{c}}^{\left(N\right)}\left(t\right).$$

where R ∈ ℝ ^{(N + 1) 2 × (N + 1) 2} means the rotation matrix. This has a blockwise diagonal structure (see [1, Chap. 2], [4]). Typically, a head tracker provides discrete-time rotation parameters at a sampling rate that is significantly less than the audio sample rate. In principle, the sampling rate of the head tracker must be adjusted. However, to avoid discontinuities in the rotated HOA sound field representation that may result from sudden changes in rotation or position, interpolation is used in one embodiment. The interpolation can be applied in the range (domain) of the rotation matrices or the rotated HOA sound field representation.

One-stage binaural rendering of HOA signals in the SH domain

Normally, (far-field) HRIRs are measured for a certain number Q of different directions of incidence Ω _q (q = 1, ..., Q) for the left and the right ear, respectively. Therefore, they can be expressed as two sets of Q functions in the time domain, h _L (t, Ω _q ) and h _R (t, Ω _q ), with the "L" and "R" indices standing for the left and right ears , For simplicity, these indices are omitted below, as the derivation is basically the same for both ears and it is implicitly assumed that for each ear the corresponding head-related impulse responses (HRIRs) are used for the calculations.

3 shows the principle of dimensioning the rendering filters and the filter coefficients. Sound sources from all spatial directions Ω = {Ω ₁ , Ω ₂ , ...} are superimposed. The first step in the binaural rendering process for HOA signals (ie, signals in the HOA sound field representation format) is to replace the discrete HRIRs (with respect to the direction of incidence) by an N-th order aborted SH-series, which is defined as $$H_N\left(t.\Omega \right)={\displaystyle {\Sigma }_{n=0}^N{\displaystyle {\Sigma }_{m=-n}^n{H}_n^m\left(t\right)S_n^m\left(\Omega \right)}}$$

This is a continuous spherical function and approximates the HRIRs to the measured directions, ie $$H_N\left(t.{\Omega }_q\right)\approx H\left(t.{\Omega }_q\right)\text{For}1\le q\le Q,$$

wird das gerenderte binaurale Zwischensignal erhalten durch $$y_{\text{BIN}\text{,I}}\left(t\right)={\displaystyle {\sum }_{n=0}^N{\displaystyle {\sum }_{m=-n}^n{c}_n^m\left(t\right)\ast h_n^m\left(t\right)}}$$

ähnlich dem Vorschlag in [8], Gl.4. Das heißt insbesondere, dass in einer ersten Stufe jedes HOA-Koeffizientensignal mit dem entsprechenden, aus der SH-Repräsentation der HRIRs erhaltenen Filter gefiltert wird. In einer zweiten Stufe werden alle Ergebnisse der Filterung aufsummiert.Zu beachten ist, dass y_BIN,I (t) auch ausgedrückt werden kann als Filterung des Beitrags jeder einzelnen ebenen Welle mit der HRIR für die entsprechende Einfallsrichtung, gemäß $$y_{\text{BIN}\text{,I}}\left(t\right)={\displaystyle \int {}_{{\delta }^2}{c}_N\left(t,\Omega \right)\ast h_N\left(t,\Omega \right)\text{d}\Omega }$$

Methods for this kind of approximations are described eg in [14]. If the aborted N-order SH representation of the HRIRs is present, $$H^{\left(N\right)}\left(t\right)={\left[\begin{array}{cccc}\begin{array}{cc}\begin{array}{cc}\begin{array}{cc}{H}_0^0\left(t\right)& H_1^{-1}\left(t\right)\end{array}& {\mathrm{<figure-callout\; id="1"\; label="H"\; filenames="DE102017102988B4\_0024.png,DE102017102988B4\_0027.png"\; state="\{\{state\}\}">H</figure-callout>}}_1^0\left(t\right)\end{array}& {\mathrm{<figure-callout\; id="1"\; label="H"\; filenames="DE102017102988B4\_0024.png,DE102017102988B4\_0027.png"\; state="\{\{state\}\}">H</figure-callout>}}_1^1\left(t\right)\end{array}& H_2^{-2}\left(t\right)& ...& H_N^N\left(t\right)\end{array}\right]}^T$$

the rendered binaural intermediate signal is obtained by $$y_{\text{BIN}\text{, I}}\left(t\right)={\displaystyle {\Sigma }_{n=0}^N{\displaystyle {\Sigma }_{m=-n}^n{c}_n^m\left(t\right)*H_n^m\left(t\right)}}$$

similar to the proposal in [8], Eq.4. This means, in particular, that in a first stage each HOA coefficient signal is filtered with the corresponding filter obtained from the SH representation of the HRIRs. In a second stage all results of the filtering are summed up. Note that y _{BIN, I} (t) can also be expressed as filtering the contribution of each individual plane wave to the HRIR for the corresponding direction of incidence, according to $$y_{\text{BIN}\text{, I}}\left(t\right)={\displaystyle \int {}_{{\delta }^2}{c}_N\left(t.\Omega \right)*H_N\left(t.\Omega \right)\text{d}\Omega }$$

Because of their complex structure, the filters derived from the SH representation of the HRIRs typically have a finite impulse response (FIR) that is typically about 256 taps in length at a sampling rate of 48 kHz. Further, an SH representation has a scalable structure, which means that an Nth order representation is a subset of an N + 1 order representation. Therefore, it makes sense to have an SH representation h ^{( N MAX)} (t) of the HRIRs of order N _MAX and to store in a memory. The order N _MAX is assumed to be the maximum order of a HOA sound field representation that can be handled by binaural rendering. The required SH representation h ^(N) (t) can then during an initialization depending on the required order N ≤ N _{MAX are} charged.

Furthermore, through an FFT-based implementation, the processing efficiency for the binaural rendering of HOA signals can be increased. The coefficients of the rendering filters are in the frequency domain. Then the digital audio signals are transformed by means of FFT into the frequency domain and filtered with the rendering filters in the frequency domain by means of a multiplication. Their output signals are each summed up and transformed into the time domain by means of inverse FFT. A reduction in computational capacity results from the fact that only one single inverse FFT is required per binaural signal (i.e., per page) due to the summation of FFT filter results.

Equalization filter to change the timbre

als Fouriertransformierte der Signale $h_n^m\left(t\right)$

der SH-Repräsentation, wobei ω die Winkelfrequenz ist, wird die ideale Übertragungsfunktion des Nachfilters zu $$F_{N\to N_{\text{H}}}\left(\omega \right)=\sqrt{\frac{{\displaystyle {\sum }_{n=0}^{N_{\text{H}}}{\displaystyle {\sum }_{m=-n}^n{\left|H_n^m\left(\omega \right)\right|}^2}}}{{\displaystyle {\sum }_{n=0}^N{\displaystyle {\sum }_{m=-n}^n{\left|H_n^m\left(\omega \right)\right|}^2}}}}$$

Canceling the coefficient series at a particular order to calculate the SH representation has the effect of spectral low-pass filtering [8], [9]. In order to compensate for this effect, a single-channel post-filtering of each of the two binaural signals is performed as in [8], [9]. For dimensioning the post-filters, it is generally believed that the HRIRs are for a large number of directions. This simplifies the calculation of the SH representation aborted at an order N _H (where the order N _{H is} significantly greater than the order N is with $H_n^m\left(\omega \right)$

as Fourier transform of the signals $H_n^m\left(t\right)$

The SH representation, where ω is the angular frequency, becomes the ideal transfer function of the postfilter $$F_{N\to N_{\text{H}}}\left(\omega \right)=\sqrt{\frac{{\displaystyle {\Sigma }_{n=0}^{N_{\text{H}}}{\displaystyle {\Sigma }_{m=-n}^n{\left|H_n^m\left(\omega \right)\right|}^2}}}{{\displaystyle {\Sigma }_{n=0}^N{\displaystyle {\Sigma }_{m=-n}^n{\left|H_n^m\left(\omega \right)\right|}^2}}}}$$

However, this function is valid only on the assumption that the binaural sound field to be reproduced is diffuse. Since the assumption of a diffuse sound field is hardly justifiable in practice, according to the invention the order N _{H is} not set as high as possible, because this generally leads to an excessive amplification of high frequencies. Instead, a lower value is chosen for the higher order N _H , which is also of order N can depend. For example, it has been found experimentally that for an order N = 1, the higher order N _H does not exceed 12 should lie.

In addition, the equalization filters can also be implemented as infinite impulse response (IIR) filters and therefore efficiently. If an individual equalization filter (postfilter) is used for each of the preliminary binaural signals, two different IIR filters with different frequency responses must be designed, which generally have different phase responses. However, this can lead to an impairment of the binaural features (binaural cues) and thus disrupt or worsen the binaural reproduction. Therefore, in one embodiment of the invention, two similar IIR equalization filters are used as post filters. These may e.g. have a transfer function obtained by averaging the two page-dependent transfer functions.

A calculation of the equalization filter 30L . 30R may depend on the order N of an HOA input signal and a given or selected higher order N _H during an initialization or in an initialization stage, respectively. In another embodiment, filter coefficients for different orders N calculated in advance and stored in a memory. The stored coefficients may then be loaded during initialization to configure the rendering filters (ie post-filters). For the reproduction of natural recordings, only a few different sets of filter coefficients are to be stored since the currently available microphones can only make HOA sound field recordings up to an order N = 7, as mentioned above.

4 shows exemplary frequency responses of binaural intermediate signals for each the same exemplary person for a diffuse sound field, which is approximated by an HOA representation of the three different orders N = 1,3,6, normalized to the frequency response of the binaural intermediate signal of order N _H = 12 , as well as the frequency responses F _{N → N H} (ω) of the corresponding equalization filter. If the diffuse sound field is approximated by an HOA representation of the order N = 1, for example, the frequency response drops 410 of the associated binaural intermediate signal relatively quickly. A perception that is significant for perception begins at about the frequency f ₁ and increases in its magnitude for higher frequencies. The corresponding equalization filter has a corresponding frequency response 510 , which increases significantly at about the frequency f ₁ and thus exactly compensates for this waste. At lower frequencies f <f1, the magnitude spectrum of the equalization filter is approximately 0 dB, which corresponds to no or very little damping or amplification. If the diffuse sound field is approximated, for example, by a HOA representation of a higher order N = 3, the associated binaural intermediate signal has a less steeply falling frequency response 420 , The associated Equalisationsfilter has a corresponding inverse frequency response 520 , which brings a noticeable gain only from a higher frequency f2> f1. The recognizable trend in the frequency responses of the binaural intermediate signals 430 and equalization filters 530 continues with an even higher order N = 6. Only the audible frequency range, ie an area below an audible maximum frequency f _{3, is} considered.

5 shows a scalable structure of filters or filter parameters, which in a particularly advantageous variant as a rendering filter 21L . 21R can be used. When the input signal to be reproduced as a digital audio signal as a 3D sound field description in the HOA format of an order N is present, (N + 1) ² first rendering filters 21L and (N + 1) ² second rendering filters 21R used. For example, for a first-order input signal with N = 1, (N + 1) ² = 4 and four filters or sets of filter coefficients F1L, F2 ₁ L, F2 ₂ L, F2 ₃ L and F1R, F2 ₁ R, F2 ₂ R, F2 ₃ R used for each side. On the other hand, with a second-order input signal with N = 2, (N + 1) ² = 9 and nine filters or sets of filter coefficients F1L, F2 ₁ L, F2 ₂ L,..., F2 ₉ L and F1R, respectively. F2 ₁ R, F2 ₂ R, ..., F2 ₉ R used for each side. However, the filter or filter coefficient sets F1L, F2 ₁ L, F2 ₂ L, F2 ₃ L and F1R, F2 ₁ R, F2 ₂ R, F2 ₃ R are the same as for a first order input signal. There are only new filters added. Accordingly, these nine filters or sets of filter coefficients per page are further used in further increasing the order to N = 3 and (N + 1) ² = 16 filters per side. For N = 4, (N + 1) ² = 25 and there are 9 filters or coefficient sets added on each side. This scalable structure is advantageous because fewer filters or sets of filter coefficients need to be kept in order to also allow processing of higher order input signals, especially as the number of filters required in principle increases quadratically with order.

HOA basics and nomenclature

wobei p_GPW(t,x,Ω) den Schalldruckbeitrag zur Zeit t und Position x auf Grund einer ebenen Welle aus der Einfallsrichtung Ω ∈ S² bezogen auf den Koordinatenursprung darstellt, und wobei S² die Einheitskugel im dreidimensionalen Euklidischen Raum darstellt. Die Einfallsrichtung Ω kann durch das Tupel Ω = (θ,ϕ) dargestellt werden, das aus einem von der Polarachse z aus gemessenen Elevationswinkel θ ∈ [0, π] und einem in der x-y-Ebene von der x-Achse aus im Gegenuhrzeigersinn gemessenen Azimuthwinkel ϕ ∈ [-π,π] besteht.The Higher Order Ambisonics (HOA) format is based on a sound field description within a given space that is assumed to be free of sound sources (eg, the space between multiple sound sources). In this case, the spatial-temporal behavior of the sound pressure p (t, x) is at time t and at one position x within the given space completely physically determined by the homogeneous wave equation. In particular, the sound pressure as a superposition of contributions of individual general plane waves (GPWs), the source of which lies outside the given space, can be clearly stated according to $$p\left(t.x\right)={\displaystyle \int {}_{{\delta }^2}}{p}_{\text{GPW}}\left(t.x.\Omega \right)\text{d}\Omega$$

where p _GPW (t, x, Ω) the sound pressure contribution at time t and position x on the basis of a plane wave from the direction of incidence Ω ∈ S ² relative to the origin of the coordinates, and where S ^{2 represents} the unit sphere in the three-dimensional Euclidean space. The direction of incidence Ω can be represented by the tuple Ω = (θ, φ) that originates from one of the polar axis z from measured elevation angle θ ∈ [0, π] and an azimuth angle φ ∈ [-π, π] measured in the xy plane from the x axis in the counterclockwise direction.

die die zeitlichen Veränderungen von allgemeinen ebenen Wellen (GPWs) im Koordinatenursprung beschreibt, die aus allen möglichen Einfallsrichtungen Ω eintreffen. Darüber hinaus kann die Funktion c(t, Ω) dargestellt werden durch eine Reihe sphärischer Basisfunktionen, die sogenannten Sphärisch-Harmonischen (SH), gemäß $$c\left(t,\Omega \right)={\displaystyle {\sum }_{n=0}^{\infty }{\displaystyle {\sum }_{m=-n}^n{c}_n^m\left(t\right)S_n^m\left(\Omega \right)}}$$

wobei $S_n^m(\cdot )$

eine reellwertige sphärisch-harmonische Funktion der Ordnung n und dem Grad m ist. Eine exemplarische Definition wird weiter unten gegeben. Die aktuellen Gewichte $c_n^m\left(t\right)$

der Reihe, als Funktion der Zeit betrachtet, werden als HOA Koeffizientensignale bezeichnet. Aus praktischen Gründen wird die Reihe normalerweise bei einer Ordnung N abgebrochen, was zu einer räumlichen frequenzband-limitieren Approximation $$c_N\left(t,\Omega \right)={\displaystyle {\sum }_{n=0}^N{\displaystyle {\sum }_{m=-n}^n{c}_n^m\left(t\right)S_n^m}}\left(\Omega \right)$$

führt. Der Satz von HOA-Koeffizientensignalen $c_n^m\left(t\right)$

mit dem Ordnungsindex n ≤ N stellt die eigentliche HOA Schallfeldrepräsentation dar, die kompakt als Vektor geschrieben werden kann: $${\mathbf{c}}^{\left(N\right)}\left(t\right)={\left[\begin{array}{cccc}\begin{array}{ccc}{c}_0^0\left(t\right)& c_1^{-1}\left(t\right)& c_1^0\left(t\right)\end{array}& c_1^1\left(t\right)& c_2^{-2}\left(t\right)& \begin{array}{cc}\dots & c_N^N\left(t\right)\end{array}\end{array}\right]}^T$$

In general, the sound pressure of a plane wave (GPW) at any position x can only with knowledge of the direction of incidence Ω and their temporal change at any arbitrary position, so in particular at the coordinate origin, are calculated. Therefore, the sound pressure for the previously considered sound field-free space can be reconstructed exactly, only with knowledge of the function $$c\left(t.\Omega \right)={p_{\text{GPW}}\left(t.x.\Omega \right)|}_{\text{x}=0}$$

which describes the temporal changes of general plane waves (GPWs) in the origin of coordinates, from all possible directions of incidence Ω arrive. In addition, the function c (t, Ω) can be represented by a series of spherical basis functions, the so-called spherical harmonics (SH), according to $$c\left(t.\Omega \right)={\displaystyle {\Sigma }_{n=0}^{\infty }{\displaystyle {\Sigma }_{m=-n}^n{c}_n^m\left(t\right)S_n^m\left(\Omega \right)}}$$

in which $S_n^m(\cdot )$

a real-valued spherical harmonic function of order n and the degree is m. An exemplary definition is given below. The current weights $c_n^m\left(t\right)$

The series, considered as a function of time, are referred to as HOA coefficient signals. For practical reasons, the series usually gets in order N aborted, resulting in a spatial frequency band-limiting approximation $$c_N\left(t.\Omega \right)={\displaystyle {\Sigma }_{n=0}^N{\displaystyle {\Sigma }_{m=-n}^n{c}_n^m\left(t\right)S_n^m}}\left(\Omega \right)$$

leads. The set of HOA coefficient signals $c_n^m\left(t\right)$

with the order index n ≤ N represents the actual HOA sound field representation that can be compact written as a vector: $${\mathbf{c}}^{\left(N\right)}\left(t\right)={\left[\begin{array}{cccc}\begin{array}{ccc}{c}_0^0\left(t\right)& c_1^{-1}\left(t\right)& {\mathrm{<figure-callout\; id="1"\; label="c"\; filenames="DE102017102988B4\_0024.png,DE102017102988B4\_0027.png"\; state="\{\{state\}\}">c</figure-callout>}}_1^0\left(t\right)\end{array}& {\mathrm{<figure-callout\; id="1"\; label="c"\; filenames="DE102017102988B4\_0024.png,DE102017102988B4\_0027.png"\; state="\{\{state\}\}">c</figure-callout>}}_1^1\left(t\right)& c_2^{-2}\left(t\right)& \begin{array}{cc}...& c_N^N\left(t\right)\end{array}\end{array}\right]}^T$$

innerhalb des Vektors c^(N)(t) wird hier gemäß der sogenannten Ambisonics Channel Number (ACN)-Konvention gewählt zu n (n + 1) + 1 + m.The position index of the HOA coefficient signal $c_n^m\left(t\right)$

within the vector c ^(N) (t), here, according to the so-called Ambisonics Channel Number (ACN) convention, one chooses n (n + 1) + 1 + m.

mit sogenannter N3D Normierung kann angegeben werden durch $$S_n^m\left(\left(\theta ,\varphi \right)\right)=\sqrt{\left(2n+1\right)\cdot \frac{\left(n-\left|m\right|\right)\text{!}}{\left(n+\left|m\right|\right)\text{!}}}\cdot P_n^{\left|m\right|}\left(\text{cos}\theta \right)\cdot \{\begin{array}{ll}\sqrt{2}\text{cos}\left(m\varphi \right)\hfill & \text{for }m>0\hfill \\ 1\hfill & \text{for }m=0\hfill \\ -\sqrt{2}\text{sin}\left(m\varphi \right)\hfill & \text{for }m<0\hfill \end{array}$$

wobei $P_n^m(\cdot )$

die zugehörigen Legendre'schen Funktionen sind, die (anders als z.B. in [11, Kap.6.3.2] ohne Benutzung des Condon-Shortley Phasenterms (-1)^m) mittels der Legendrepolynome P_n(·) definiert sind als $$P_n^m\left(x\right)={\left(1-x^2\right)}^{m\text{/}2}\frac{{\text{d}}^m}{\text{d}{x}^m}{P}_n\left(x\right),m\ge 0.$$

An exemplary definition of a real-valued spherical harmonic function $S_n^m(\cdot )$

with so-called N3D normalization can be indicated by $$S_n^m\left(\left(\theta .\phi \right)\right)=\sqrt{\left(2n+1\right)\cdot \frac{\left(n-\left|m\right|\right)\text{!}}{\left(n+\left|m\right|\right)\text{!}}}\cdot P_n^{\left|m\right|}\left(\text{cos}\theta \right)\cdot \{\begin{array}{ll}\sqrt{2}\text{cos}\left(m\phi \right)\hfill & \text{for}m>0\hfill \\ 1\hfill & \text{for}m=0\hfill \\ -\sqrt{2}\text{sin}\left(m\phi \right)\hfill & \text{for}m<0\hfill \end{array}$$

in which $P_n^m(\cdot )$

are the associated Legendre functions, which (unlike, for example, in [11, chap.6.3.2] without the use of the Condon-Shortley phase term (-1) ^m ) are defined by the legend polynomials P _n (·) $$P_n^m\left(x\right)={\left(1-x^2\right)}^{m\text{/}2}\frac{{\text{d}}^m}{\text{d}{x}^m}{P}_n\left(x\right).m\ge \mathrm{0th}$$

The invention can be used advantageously for audio reproduction systems or their components, in particular for binaural reproduction. This is best done via headphones or earphones.

references

1. [1] JR Driscoll and DM Healy. Computing Fourier Transforms and Convolution on the 2-Sphere. Advances in Applied Mathematics, 15 (2): 202-250, 1994 ,
2. [2] R.Duraiswami, E.Grassi, NA Gumerov, Z.Li, DN Zotkin, and LS Davis. High Order Spatial Audio Capture and Its Binaural Head-Tracked Playback Over Headphones with HRTF Cues. In Audio Engineering Society Convention 119, Oct 2005 ,
3. [3] G. Enzner, M. Weinert, S. Abeling, JM Batke, and P. Jax. Advanced system options for binaural rendering of Ambisonic format. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 251-255, May 2013 ,
4. [4] Z. Gimbutas and L. Greengard. A fast and stable method for rotating spherical harmonic expansion. Journal of Computational Physics, 228 (16): 5621-5627, 2009 ,
5. [5] Adam McKeag and David McGrath. Sound Field Format to Binaural Decoder with Head Tracking. Preprint of the Audio Engineering Society for the 6th Australian Regional Convention, (4302), 1996 ,
6. [6] J. Meyer and G. Elko. A highly scalable spherical microphone array based on orthonormal decomposition of the soundfield. In 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing, volume 2, pages 1781-1784, May 2002 ,
7. [7] B. Rafaely. Fundamentals of Spherical Array Processing. Springer, Berlin, 2015 ,
8. [8th] J. Sheaffer and B. Rafaely. Equalization strategies for binaural room impulse response rendering using spherical arrays. In 2014, IEEE 28th Convention of Electrical Electronics Engineers in Israel (IEEEI), pages 1-5, Dec 2014 ,
9. [9] J.Sheaffer, S.Villeval, B.Rafaely. Rendering Binaural Room Impulse Responses from Spherical Microphone Array Recordings Using Timbre Correction. In Proc. of the EAA Joint Symposium on Auralization and Ambisonics, pages 81-85, Berlin, 2014 ,
10. [10] EM Wenceslas. What Perception Implies About Implementation of Interactive Virtual Acoustic Environments. In Audio Engineering Society Convention 101, Nov 1996 ,
11. [11] Earl G. Williams. Fourier Acoustics: Sound Radiation and Nearfield Acoustical Holography. Academic Press, 1 edition, June 1999 ,
12. [12] DN Zotkin, R. Duraiswami, and LS Davis. Creation of virtual auditory spaces. In 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing, Volume 2, pages II-2113-II-2116, May 2002 ,
13. [13] DN Zotkin, R. Duraisvami, and NA Gumerov. Sound field decomposition using spherical microphone arrays. 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 277-280, March 2008 ,
14. [14] DN Zotkin, R. Duraisvami, and NA Gumerov. Regularized HRTF fitting using spherical harmonics. In 2009 IEEE Workshop on Signal Processing to Audio and Acoustics, pages 257-260, Oct 2009 ,

Claims (15)

A method of processing a digital audio signal presented as a 3D sound field description in Higher Order Ambisonics HOA format of order N, producing a left output signal and a right output signal for binaural reproduction via headphones, comprising the steps of: rotating (110) the 3D sound field description of the digital audio signal in the Higher Order Ambisonics HOA format according to a rotation control signal obtainable from a head tracker, filtering (120) the rotated 3D sound field description of the digital audio signal with (N + 1) ² first rendering filters (21L) first head related impulse responses (HRIRs) for a left channel and (N + 1) ² second rendering filters (21R) according to second head related impulse responses (HRIRs) for a right channel, the rendering filters corresponding to approximation of the first and second head related impulse responses by spherical functions and the approximation to the order N limit t, where the spherical functions are the same as in the Higher Order Ambisonics HOA format of the sound field description, summing (130) the outputs of the first rendering filters (21L) into an intermediate signal for the left channel, and summing (130) the outputs of the second rendering filters (21R) to an intermediate signal for the right channel, Filtering (140) the intermediate signal for the left channel with a first equalization filter (30L) for changing the tone color, the first equalization filter being automatically adjusted according to the order N, and the left output signal (40L) being for binaural reproduction, and filtering (150) the right-channel intermediate signal having a second equalization filter for changing the timbre, the second equalization filter being automatically adjusted according to the N-order, and the right-hand output signal (40R) for binaural reproduction. Method according to Claim 1 wherein the fitting of the first and second equalization filters (30L, 30R) causes a change in the timbre corresponding to an increase in the order of the 3D sound field description and the rendering filters of the order N to a higher order N _H. Method according to Claim 1 or 2 wherein the first and second equalization filters (30L, 30R) cause high frequency boosting, high frequencies being those frequencies in the audible range that are attenuated by the approximation of the first and second head related impulse responses (HRIRs), and wherein the matching of the first and the second Equalisierungsfilters according to the order N implies that the increase at lower orders is more pronounced than at higher orders N. Method according to one of Claims 1 - 3 wherein the order parameter N is automatically set according to the 3D sound field description of the digital audio signal and the number of the first and second rendering filters are modified by the order N. Method according to one of Claims 1 - 4 , wherein the first and the second equalization filters are IIR filters and are the same. Method according to one of Claims 1 - 5 in which the impulse responses of the rendering filters represent the time history of the weights of spherical functions of a sequence representation bounded by the order N by spherical-harmonic functions of the first and second head-related impulse responses (HRIRs). Method according to Claim 6 wherein, during initialization, filter coefficients of the rendering filters are retrieved from a memory containing coefficients for a largest order N _max , and wherein the smaller order coefficients are a subset of the largest order coefficients N _max . Method according to one of Claims 1 - 7 in which the digital audio signals are transformed by means of FFT into the frequency domain, the coefficients of the rendering filters are in the frequency domain, the digital audio signals are filtered with the rendering filters in the frequency domain by means of a multiplication and the output signals after the summation are in each case transformed by means of inverse FFT into the time domain, where only one inverse FFT per output is needed. Method according to one of Claims 1 - 8th wherein the 3D sound field description in Higher Order Ambisonics HOA format includes discrete-time sample samples, and wherein the rotation control signal is provided by the head tracker and is also time-discrete and interpolated prior to rotation to correspond to the sample instants of the audio samples. Method according to one of Claims 1 - 8th wherein the 3D sound field description in Higher Order Ambisonics HOA format includes discrete-time sample samples, and wherein the rotation control signal is provided by the head tracker and is also time-discrete, and wherein two or more consecutive values of the rotation control signal are applied to calculate two or more rotations and the audio samples of the two or more rotated 3D sound field representations are temporally interpolated. Apparatus for processing a digital audio signal presented as a 3D sound field description in Higher Order Ambisonics HOA format of order N, wherein a left output signal and a right output signal for binaural reproduction are generated via headphones, comprising a rotation unit (10) for rotating the 3D sound field description according to a rotation control signal describing head movements of a user, first and second rendering filters (21L, 21R) which receive the rotated 3D sound field description and each having a plurality of outputs, first and second adders (25L, 25R), each connected to the outputs of the first and second rendering filters (21L, 21R) for summing their output signals, and first and second equalization filters (30L, 30R), each of which outputs the summed outputs from an adder (25L, 25R) Filter the rendering filter to change its tone, with the Output signals (40L, 40R) for binaural reproduction, and wherein the first and the second equalization filters are automatically adjusted according to the order N, and wherein the timbre is adjusted to correspond to a higher order N _H > N. Device after Claim 11 wherein the first and second equalization filters (30L, 30R) cause high frequency boosting, high frequencies being those frequencies in the audible range that are attenuated by the approximation of the first and second head related impulse responses (HRIRs), and wherein the matching of the first and the second Equalisierungsfilters according to the order N implies that the increase at lower orders is more pronounced than at higher orders N. Device after Claim 11 or 12 , where the first and second equalization filters are IIR filters and are the same. Device according to one of Claims 11 - 13 further comprising an FFT unit and two inverse FFT units, wherein the digital audio signals are FFT transformed into the frequency domain by FFT, the coefficients of the rendering filters are in the frequency domain, the digital audio signals are rendered in frequency domain with the rendering filters Multiplication are filtered and the output signals after summing are each transformed by means of inverse FFT by the inverse FFT unit in the time domain, with only one inverse FFT per output signal is required. Apparatus for processing a digital audio signal having at least a processor and a memory, wherein the memory stores a program that configures the processor to perform a method according to any one of Claims 1 - 10 perform.

Images