CN103854642B - Flame speech synthesizing method based on physics - Google Patents

Abstract

The present invention relates to the fields such as computer game, video display specially good effect, Engineering Simulation, for providing the automatic synthesis method of a kind of flame sound, provide key technology to support for acousto-optic emulation.For this; the present invention adopts the technical scheme that, flame speech synthesizing method based on physics, comprises the following steps: the visual simulation of flame: use blue kernel model to model flame; flame is simulated, and derives velocity field and other dependent field amount such as fuel firing rates of every frame；Speed divergence integral and calculating: calculated by the speed divergence integration of flame front encirclement part by fuel firing rate；Up-sampling based on fast fourier transform: use up-sampling based on FFT that the speed divergence integration tried to achieve is reconstructed；Acoustic pressure exports: the speed divergence integration derivation after up-sampling is obtained final acoustic pressure.Present invention is mainly applied to video, image procossing occasion.

Description

Flame sound synthesis method based on physics

Technical Field

The invention relates to the fields of computer games, movie and television special effects, engineering simulation and the like, in particular to a flame sound synthesis method based on physics.

Technical Field

Traditional flame simulation tends to focus on visual rendering [1] only, ignoring auditory rendering. Like visual rendering, auditory rendering is also an important component of the realism of virtual reality. Even though flame simulation can produce vivid, realistic scenes, the user still experiences unrealistic scenes if there is a lack of auditory rendering, resulting in a poor user experience.

Recording is one of the most direct methods of auditory rendering. The user needs to record a corresponding piece of sound for each specific application. However, since such a form of recording is not conducive to multiplexing, different applications need to re-record, and more people start using short recordings. This type of process is commonly referred to as Granular Synthesis (Granular Synthesis) by editing, manipulating, and synthesizing short recordings to match specific applications. However, sound recordings have two inherent drawbacks: firstly, the recording and the simulated scenes need to be manually synchronized, which is not only a time-consuming and labor-consuming work, but also difficult to achieve complete synchronization; secondly, in some interactive scenes, most of the interactive behaviors are unpredictable in advance, so that the recording method has a large limitation in practical application.

In recent years, with the further development of computer graphics, attention to auditory rendering has been increased, and the results of such studies have been highlighted, and researchers have proposed sound generation methods for such phenomena as vibrating solids [2] [3] [4], aerodynamic acoustic phenomena [5] [6], splashed liquids [7] [8], and fractured solids [9] in succession. But is still relatively blank for the generation of flame sounds. Dobashi et al [5] [6] propose a physics-based method of generating vortex sounds and applying it to flames. Although this approach solves the above problem, the main source of the flame is not from vortices, i.e. vortices are only a small part of the flame sound. Jeffrey and Doug [10] propose methods for generating sound from the combustion of the main source of sound from a flame. Their proposed method, while producing a credible flame sound, is computationally expensive making it unsuitable for complex scenes. Even simple flame scenes, such as the back and forth movement of a candle, can take tens of hours, or even days, of operating time. Lowering the frame rate of the simulation can significantly reduce the simulation time, but at the same time, can significantly affect the reality of the generated sound.

Flames are important components of game scenes, movie special effects, simulation drilling systems (such as fire drilling) and the like. The realistic simulation of the flame comprises a visual simulation part and an auditory simulation part, but the existing work is more focused on the research of the visual simulation part and the auditory simulation part; the auditory simulation can greatly enhance the immersion of the simulated scene, and is not only an indispensable part in a game engine but also an indispensable key technology in a plurality of movie special effects.

[1]Bukowski R,Sequin C.Interactive simulation of fire in virtualbuilding environments.Proceedings of SIGGRAPH,1997.35～44.

[2]O’Brien J F,Cook P R,Essl G.Synthesizing sounds from physicallybased motion.Proceedings of SIGGRAPH,2001.529～536.

[3]O’Brien J F,Shen C,Gatchalian C M.Synthesizing sounds from rigid-body simulations.Proceedings of SIGGRAPH,2002.175～181.

[4]Van D D,Pai D K.Foleyautomatic:Physically based sound effects forinteractive simulation and animation.Proceedings of SIGGRAPH,2001.537～544.

[5]Dobashi Y,Yamamoto T,Nishita T.Real-time rendering of aerodynamicsound using sound textures based on computational fluid dynamics.ACMTransactions on Graphics,2003,22(3):732～740.

[6]Dobashi Y,Yamamoto T,Nishita T.Synthesizing sound from turbulentfields using sound textures for interactive fluid simulation.ComputerGraphics Forum,2004,23(3):736～744.

[7]Zheng C.,James D.L.Harmonic fluids.ACM Transactions on Graphics,2009,28(3):37:1～37:12.

[8]Moss W,Yeh H,Manocha D.Sounding liquids:Automatic sound synthesisfrom fluid simulation.ACM Transactions on Graphics,2010,29(3):21:1～21:13.

[9]Zheng C,James D L.Rigid-body fracture sound with precomputedsoundbanks.ACM Transactions on Graphics,2010,29(3):69:1～69:13.

[10]Chadwick J N,James D L.Animating fire with sound.ACM Transactionson Graphics,2011,30(4):84:1～84:8.。

Disclosure of Invention

In order to overcome the defects of the prior art, the automatic synthesis method of the flame sound is provided, and key technical support is provided for acousto-optic simulation. Therefore, the technical scheme adopted by the invention is that the flame sound synthesis method based on physics comprises the following steps:

visual simulation of the flame: modeling flame by adopting a blue core model, simulating the flame, and deriving a velocity field of each frame and other related field quantities such as fuel combustion rate;

velocity divergence integral calculation: calculating the velocity divergence integral of the portion surrounded by the flame front end from the fuel combustion rate;

fast Fourier Transform (FFT) -based upsampling: reconstructing the obtained velocity divergence integral by adopting FFT-based up-sampling;

sound pressure output: and (4) obtaining final sound pressure by differentiating the velocity divergence integral after the up-sampling.

The simulation of the flame is as follows:

the present invention employs a blue core model to model the flame, which assumes that the gaseous fuel will immediately burn completely when it crosses the interface of the gaseous fuel and the combustion products, releasing a large amount of heat and the corresponding combustion products, the interface being referred to as the flame front, which is explicitly tracked using a level set or similar method, and the gaseous fuel and the combustion products are modeled using the following Navier-Stokes equations, respectively:

$\frac{\∂u}{\∂t}=-(u\·\▿)u-\▿p/\mathrm{\ρ}+f,---\left(1\right)$

$\▿\·u=\mathrm{\Φ},---\left(2\right)$

wherein, formula (1) is a momentum equation, u is a speed, t represents time, p is a pressure, ρ is a density, and f represents an external force including buoyancy or gravity; formula (2) is a mass equation, phi is an optional divergence source, the value of phi is calculated through real-time data obtained through pre-specification or simulation, and when phi is larger than zero, a velocity field diffuses outwards at the point; conversely, when Φ is less than zero, the velocity field contracts inward at that point; in addition to the velocity field, the density field is used to model the state of smoke produced by combustion, and the temperature field is used to model the heat produced by combustion;

the velocity divergence integral calculation specifically comprises:

based on the assumption of a blue core, the heat release is concentrated at the flame front, which is directly proportional to the fuel velocity, and the volumetric integral of the heat release can then be converted into a curved integral of the velocity flux, which yields:

$\∫{\mathrm{qd}}^{3}x={\∫}_{S}u\·\mathrm{nds},---\left(3\right)$

wherein S is the flame front end, u represents the velocity, n is the normal vector, and q represents the heat release amount; to avoid this operation, the surface integral is converted into a volume integral of the divergence according to the gaussian divergence formula, i.e.:

${\∫}_{S}u\·\mathrm{nds}={\∫}_V\▿\·\mathrm{udv},---\left(4\right)$

where V is an arbitrary geometric body surrounded by the flame front, and based on a propagation wave equation describing the sound source and sound, and ignoring time delay and distance attenuation, the sound pressure p (t) is obtained as:

$p\left(t\right)=\frac{d}{\mathrm{dt}}{\∫}_V\▿\·u(x,t)\mathrm{dv}\left(x\right).---\left(5\right)$

the FFT-based upsampling specifically comprises:

signal enhancement is realized by constructing an FFT-based up-sampling operation in a zero-padding mode in a frequency domain, wherein zero padding is used for prolonging the spectrum length, and a spectrum with the length of L is prolonged to N > L by zero padding, as follows:

${\mathrm{ZeroPad}}_{N,L}\left(x\right)=\left\{\begin{array}{cc}x\left(\mathrm{\&omega;}\right),& \left|\mathrm{\&omega;}\right|<L/2\\ 0,& \mathrm{otherwise}\end{array}\right.,---\left(6\right)$

wherein, L is the original spectrum length, N is the spectrum signal length after zero padding operation, generally, N = kL, and k is the expansion multiple; the upsampling operation is to firstly convert the obtained divergence value div (t) from the spatial domain to the frequency domain s (ω) through FFT; keeping the low-frequency part unchanged, and performing zero filling operation on the high-frequency part; finally, an up-sampled time domain signal is obtained by Inverse Fast Fourier Transform (IFFT).

The sound pressure output is specifically:

to determine the sound pressure p (t), the divergence value div (t) needs to be derived: interpolate div (t) using the cubic interpolation function:

$\mathrm{div}\left(t\right)=\underset{n=0}{\overset{N}{\mathrm{\&Sigma;}}}\mathrm{div}\left(\mathrm{n\&Delta;t}\right)k[(t-\mathrm{n\&Delta;t})/\mathrm{\&Delta;t}],---\left(7\right)$

wherein, $k\left(x\right)=\frac{1}{6}\left\{\begin{array}{cc}7{\left|x\right|}^3-12{\left|x\right|}^2+\frac{16}{3},& \left|x\right|<1\\ -\frac{7}{3}{\left|x\right|}^3+12{\left|x\right|}^2-20\left|x\right|+\frac{32}{3},& 1\&le;\left|x\right|<2\\ 0,& \mathrm{otherwise}\end{array}\right..$

then, the sound pressure is derived as:

$p\left(t\right)=\frac{d}{\mathrm{dt}}\mathrm{div}\left(t\right)=\underset{n=0}{\overset{N}{\mathrm{\&Sigma;}}}\mathrm{div}\left(\mathrm{n\&Delta;t}\right)\frac{d}{\mathrm{dt}}k[(t-\mathrm{n\&Delta;t})/\mathrm{\&Delta;t}],---\left(8\right).$

the invention has the following technical effects:

the invention can automatically generate more vivid flame sound effect and can be used in the fields of computer games, movie and television special effect making and the like. In addition, when the method is used in engineering simulation, the sense of reality and the immersion of a simulation environment can be enhanced.

The invention has simple requirements on the operating environment and low implementation cost, and the specific requirements are as follows:

support Windows XP, Windows vista, Windows7 environment, the microcomputer of the memory above 2G, the video memory configuration above 1G.

Drawings

Fig. 1 is a schematic diagram of FFT-based upsampling.

Fig. 2 shows a flame scene and a resulting sound signal generated using the present invention.

FIG. 3 is a flow chart of the present invention.

Detailed Description

The invention provides a flame sound automatic synthesis method based on physics, which mainly comprises the following steps:

visual simulation of the flame: the flame is simulated using a physics-based solution and the velocity field and other relevant field quantities such as fuel burn rate for each frame are derived.

Velocity divergence integral calculation: the velocity dispersion integral of the portion surrounded by the flame front is calculated from the fuel combustion rate.

Fast Fourier Transform (FFT) -based upsampling: and reconstructing the obtained velocity divergence integral by adopting FFT-based up-sampling.

Sound pressure output: and (4) obtaining final sound pressure by differentiating the velocity divergence integral after the up-sampling.

The specific technical scheme of the invention is as follows:

1) visual simulation of flames

The invention adopts a blue core model (reference: Nguyen D, Fedkiwew R, Jensen H. physically based modeling and evaluation of fire. ACM Transactions on Graphics,2002,21(3): 721-728.) to model flame. The model assumes that the gaseous fuel will immediately burn completely as it crosses the interface of the gaseous fuel and the combustion products, releasing a significant amount of heat and the corresponding combustion products. This interface is referred to as the flame front. The flame front is explicitly tracked using a level set or similar method, and the gaseous fuel and combustion products are modeled using the following Navier-Stokes equations, respectively:

$\frac{\&PartialD;u}{\&PartialD;t}=-(u\&CenterDot;\&dtri;)u-\&dtri;p/\mathrm{\&rho;}+f,---\left(1\right)$

$\&dtri;\&CenterDot;u=\mathrm{\&Phi;},---\left(2\right)$

where equation (1) is a momentum equation, u is velocity, t represents time, p is pressure, ρ is density, and f represents an external force such as buoyancy or gravity. Equation (2) is a mass equation, and Φ is an optional divergence source, and the value of Φ can be calculated from real-time data that is specified in advance or modeled. When phi is larger than zero, the velocity field diffuses outwards at the point; conversely, when Φ is less than zero, the velocity field contracts inward at that point. In addition to the velocity field, the density field may be used to model the state of smoke produced by combustion, and the temperature field may be used to model the heat produced by combustion.

2) Velocity divergence integral calculation

Based on the assumption of a blue core, the heat release is concentrated at the flame front. The amount of heat released is directly proportional to the fuel velocity, and the volumetric integral of heat released can be converted into a curved integral of velocity flux, from which:

$\&Integral;{\mathrm{qd}}^{3}x={\&Integral;}_{S}u\&CenterDot;\mathrm{nds},---\left(3\right)$

where S is the flame front, u represents the velocity, n is the normal vector, and q represents the amount of heat released. Although the heat release amount q does not need to be obtained, the conversion inevitably requires the flame front end to be discretized into a triangular patch because the curved surface integral needs to be calculated on the flame front end. Experiments have shown that this is a rather computationally expensive operation, each of which takes on the order of seconds. To avoid this operation, the present invention converts the surface integral into a volume integral of divergence according to the gaussian divergence formula, namely:

${\&Integral;}_{S}u\&CenterDot;\mathrm{nds}={\&Integral;}_V\&dtri;\&CenterDot;\mathrm{udv},---\left(4\right)$

where V is any geometric body surrounded by the flame front. Although the integral is still calculated within the geometry of the flame front envelope, the only thing that needs to be calculated is the divergence of the velocity, which can significantly increase the calculation time. Based on the propagation wave equation describing the sound source and sound (ref: Chrighton D G, Dowling A P.model Methods in analytical acoustics. Berlin: Springer-Verlag, 1992.), neglecting time delay and distance attenuation, a sound pressure p (t) is obtained as:

$p\left(t\right)=\frac{d}{\mathrm{dt}}{\&Integral;}_V\&dtri;\&CenterDot;u(x,t)\mathrm{dv}\left(x\right).---\left(5\right)$

3) FFT-based upsampling

The velocity divergence integral div (t) is found for each frame from the data derived from the flame simulation, but the information from frame to frame is lost. In order to further increase the speed, the analog frame rate needs to be reduced. But lowering the frame rate results in the loss of some key frame information and thus distortion of the generated sound.

The invention constructs an FFT-based up-sampling operation to realize signal enhancement by a zero filling mode in a frequency domain. The zero padding operation uses zero padding to extend the spectral length. By zero-filling, a spectrum of length L can be extended to N > L, as follows:

${\mathrm{ZeroPad}}_{N,L}\left(x\right)=\left\{\begin{array}{cc}x\left(\mathrm{\&omega;}\right),& \left|\mathrm{\&omega;}\right|<L/2\\ 0,& \mathrm{otherwise}\end{array}\right.,---\left(6\right)$

wherein, L is the original spectrum length, and N is the spectrum signal length after zero filling operation. Typically, N = kL, k being the expansion factor. Since such zero-padding operation can provide nearly perfect upsampling information for the time-domain signal, the upsampling operation thus constructed can recover as much as possible the frame information lost by lowering the frame rate. The upsampling operation is shown in fig. 1, and the obtained divergence value div (t) is first transformed from the spatial domain to the frequency domain s (ω) by the FFT. In fig. 1 (c), the low frequency part is kept unchanged, and the zero padding operation is performed on the high frequency part. Finally, an up-sampled time domain signal is obtained by Inverse Fast Fourier Transform (IFFT). Comparing fig. 1 (a) and fig. 1 (d), after the up-sampling operation, the frame information lost between the time steps is recovered.

Sound pressure output

To determine the sound pressure p (t), a derivation of the divergence value div (t) is required. Since div (t) is only known at fixed time steps 0,. DELTA.t, 2. DELTA.t, …, N. DELTA.t, a continuous time-domain signal is obtained by interpolation. The invention uses the cubic interpolation function (reference: Mitchell D P, NervalliA. Reconstruction filters in computer-graphics. proceedings of SIGTRAPH, 1988.221-228.) proposed by Mitchell and Nervali to interpolate div (t):

$\mathrm{div}\left(t\right)=\underset{n=0}{\overset{N}{\mathrm{\&Sigma;}}}\mathrm{div}\left(\mathrm{n\&Delta;t}\right)k[(t-\mathrm{n\&Delta;t})/\mathrm{\&Delta;t}],---\left(7\right)$

wherein, $k\left(x\right)=\frac{1}{6}\left\{\begin{array}{cc}7{\left|x\right|}^3-12{\left|x\right|}^2+\frac{16}{3},& \left|x\right|<1\\ -\frac{7}{3}{\left|x\right|}^3+12{\left|x\right|}^2-20\left|x\right|+\frac{32}{3},& 1\&le;\left|x\right|<2\\ 0,& \mathrm{otherwise}\end{array}\right..$

then, the sound pressure is derived as:

$p\left(t\right)=\frac{d}{\mathrm{dt}}\mathrm{div}\left(t\right)=\underset{n=0}{\overset{N}{\mathrm{\&Sigma;}}}\mathrm{div}\left(\mathrm{n\&Delta;t}\right)\frac{d}{\mathrm{dt}}k[(t-\mathrm{n\&Delta;t})/\mathrm{\&Delta;t}],---\left(8\right)$

the invention relates to solution and vector operation of a large number of equations, and the computational efficiency can be improved by adopting a programming mode based on GPU (graphics processing units).

Claims (5)

1. A method for synthesizing flame sound based on physics is characterized by comprising the following steps:

visual simulation of the flame: modeling flame by adopting a blue core model, simulating the flame, and deriving a speed field of each frame and related field quantity including fuel combustion rate;

velocity divergence integral calculation: calculating the velocity divergence integral of the portion surrounded by the flame front end from the fuel combustion rate;

fast Fourier Transform (FFT) -based upsampling: reconstructing the obtained velocity divergence integral by adopting FFT-based up-sampling;

sound pressure output: and (4) obtaining final sound pressure by differentiating the velocity divergence integral after the up-sampling.

2. A physics-based flame sound synthesis method as claimed in claim 1 wherein the simulating of flames is specifically: the flame was modeled using a blue core model that assumes complete combustion immediately as the gaseous fuel crossed the interface of the gaseous fuel with the combustion products, known as the flame front, using level set explicit tracking for the flame front, and the following Navier-Stokes equations for the gaseous fuel and combustion products, respectively:

wherein, formula (1) is a momentum equation, u is a speed, t represents time, p is a pressure, ρ is a density, and f represents an external force including buoyancy or gravity; formula (2) is a mass equation, phi is an optional divergence source, the value of phi is calculated through real-time data obtained through pre-specification or simulation, and when phi is larger than zero, a velocity field diffuses outwards at the point; conversely, when Φ is less than zero, the velocity field contracts inward at that point; in addition to the velocity field, the density field is used to model the state of smoke produced by combustion, and the temperature field is used to model the heat produced by combustion.

3. The physics-based flame sound synthesis method of claim 1, wherein the velocity divergence integral calculation is specifically:

based on the assumption of a blue core, the heat release is concentrated at the flame front, which is directly proportional to the fuel velocity, and the volumetric integral of the heat release can then be converted into a curved integral of the velocity flux, which yields:

∫qd³x＝∫_Su·nds， (3)

wherein S is the flame front end, u represents the velocity, n is the normal vector, and q represents the heat release amount; the surface integral is converted into the volume integral of the divergence according to the gaussian divergence formula, i.e.:

where V is an arbitrary geometric body surrounded by the flame front, and based on a propagation wave equation describing the sound source and sound, and ignoring time delay and distance attenuation, the sound pressure p (t) is obtained as:

。

4. the physics-based flame sound synthesis method of claim 1, wherein the FFT-based upsampling is specifically: signal enhancement is realized by constructing an FFT-based up-sampling operation in a zero-padding mode in a frequency domain, wherein zero padding is used for prolonging the spectrum length, and a spectrum with the length of L is prolonged to N > L by zero padding, as follows:

wherein, L is the original spectrum length, N is the spectrum signal length after zero padding operation, N equals kL, and k is the expansion multiple; the upsampling operation is to firstly convert the obtained divergence value div (t) from the spatial domain to the frequency domain s (ω) through FFT; keeping the low-frequency part unchanged, and performing zero filling operation on the high-frequency part; finally, an up-sampled time domain signal is obtained by Inverse Fast Fourier Transform (IFFT).

5. The physics-based flame sound synthesis method of claim 1, wherein the sound pressure output is specifically:

to determine the sound pressure p (t), the divergence value div (t) needs to be derived: interpolate div (t) using the cubic interpolation function:

wherein,

then, the sound pressure is derived as:

。