Abstract
The use of surrogate models for approximating computationally expensive simulations has been on the rise for the last two decades. Kriging-based surrogate models are popular for approximating deterministic computer models. In this work, the performance of Kriging is investigated when gradient information is introduced for the approximation of computationally expensive black-box simulations. This approach, known as gradient-enhanced Kriging, is applied to various benchmark functions of varying dimensionality (2D-20D). As expected, results from the benchmark problems show that additional gradient information can significantly enhance the accuracy of Kriging. Gradient-enhanced Kriging provides a better approximation even when gradient information is only partially available. Further comparison between gradient-enhanced Kriging and an alternative formulation of gradient-enhanced Kriging, called indirect gradient-enhanced Kriging, highlights various advantages of directly employing gradient information, such as improved surrogate model accuracy, better conditioning of the correlation matrix, etc. Finally, gradient-enhanced Kriging is used to model 6- and 10-variable fluid–structure interaction problems from bio-mechanics to identify the arterial wall’s stiffness.
Similar content being viewed by others
Notes
MATLAB, The MathWorks, Inc., Natick, MA, USA.
This fact may not be completely true in high dimensional problems where partial set of gradients for a set of high-valued hyper-parameters is required to provide accurate GEK models (Table 8). Again, the size of the set of high-valued hyper-parameters, which is greater than \(5\) in this case, depends on the complexity of the function to be modelled.
References
Brezillon J, Dwight R (2005) Discrete adjoint of the Navier–Stokes equations for aerodynamic shape optimization. In: Evolutionary and deterministic methods for design, optimisation and control with applications to industrial and societal problems (EUROGEN 2005). Munich, Germany
Chung HS, Alonso JJ (2002) Using gradients to construct cokriging approximation models for high-dimensional design optimization problems. Problems., G 40th AIAA Aerospace Sciences Meeting and Exhibit, AIAAReno, NV, pp. 2002–0317
Couckuyt I, Dhaene T, Demeester P (2014) ooDACE toolbox: a flexible object-oriented kriging implementation. J Mach Learn Res 15:3183–3186
Degroote J, Bathe KJ, Vierendeels J (2009) Performance of a new partitioned procedure versus a monolithic procedure in fluid–structure interaction. Comput Struct 87(11–12):793–801
Degroote J, Hojjat M, Stavropoulou E, Wüchner R, Bletzinger KU (2013) Partitioned solution of an unsteady adjoint for strongly coupled fluid–structure interactions and application to parameter identification of a one-dimensional problem. Struct Multidiscip Optim 47(1):77–94
Dwight RP, Han ZH (2009) Efficient uncertainty quantification using gradient-enhanced kriging. In: 11th AIAA Non-Deterministic Approaches Conference. Palm Springs, California, USA
Forrester AI, Sóbester A, Keane AJ (2007) Multi-fidelity optimization via surrogate modelling. Proc R Soc 463:3251–3269
Forrester AI, Sóbester A, Keane AJ (2008) Engineering design via surrogate modelling: a practical guide, 1 edn. Wiley, New York (2008)
Ginsbourger D (2009) Multiples métamodéles pour l’approximation et l’optimisation de fonctions numériques multivariables. Ph.D. thesis, École des Mines de Saint-Étienne
Ginsbourger D, Dupuis D, Badea A, Carraro L, Roustant O (2009) A note on the choice and the estimation of kriging models for the analysis of deterministic computer experiments. Appl Stoch Models Bus Ind 25(2):115–131. Special issue: Computer experiments versus physical experiments
Ginsbourger D, Helbert C, Carraro L (2008) Discrete mixtures of kernels for kriging-based optimization. Qual Reliabil Eng Int 24(6):681–691
Griewank A (2000) Evaluating derivatives: principles and techniques of algorithmic differentiation. No. 19 in Frontiers in Appl Math SIAM, Philadelphia, PA (2000)
Husslage B, Rennen G, van Dam E, den Hertog D (2011) Space-filling latin hypercube designs for computer experiments. Optim Eng 12(4):611–630
Kennedy MC, O’Hagan A (2000) Predicting the output from a complex computer code when fast approximations are available. Biometrika 87(1):1–13
Killeya MRH (2004) Thinking inside the box: using derivatives to improve Bayesian black box emulation of computer simulators with application to compart mental models. Ph.D. thesis, Durham theses, Durham University, England
Kleijnen J (2009) Kriging metamodeling in simulation: a review. Eur J Oper Res 192(3):707–716
Laurenceau J, Meaux M, Montagnac M, Sagaut P (2010) Comparison of gradient-based and gradient-enhanced response-surface-based optimizers. Am Inst Aeronaut Astronaut J 48(5):981–994
Laurenceau J, Sagaut P (2008) Building efficient response surfaces of aerodynamic functions with kriging and cokriging. AIAA 46(2):498–507
Laurent S, Cockcroft J, Van Bortel L, Boutouyrie P, Giannattasio C, Hayoz D, Pannier B, Vlachopoulos C, Wilkinson I, Struijker-Boudier H (2006) Expert consensus document on arterial stiffness: methodological issues and clinical applications. Eur Heart J 27(21):2588–2605
Liu W (2003) Development of gradient-enhanced kriging approximations for multidisciplinary design optimisation. Ph.D. thesis, University of Notre Dame, Notre Dame, Indiana
Lockwood BA, Anitescu M (2010) Gradient-enhanced universal kriging for uncertainty propagation. Preprint ANL/MCS-P1808-1110
Meckesheimer M, Booker AJ, Barton RR, Simpson TW (2002) Computationally inexpensive metamodel assessment strategies. AIAA 40(10):2053–2060
Morris MD, Mitchell TJ, Ylvisaker D (1993) Bayesian design and analysis of computer experiments: use of gradients in surface prediction. Technometrics 35(3):243–255
Näther W, Šimák J (2003) Effective observation of random processes using derivatives. Metrika Springer 58:71–84
Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. The MIT Press, Cambridge
Sacks J, Schiller SB, Welch WJ (1989) Designs for computer experiments. Technometrics 31(1):41–47
Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4(4):409–423
Schneider R (2012) Feins: Finite element solver for shape optimization with adjoint equations. In: Progress in industrial mathematics at ECMI 2010 Conference, pp. 573–580
Shao W, Deng H, Ma Y, Wei Z (2012) Extended Gaussian kriging for computer experiments in engineering design. Eng Comput 28(2):161–178
Šimák J (2002) On experimental designs for derivative random fields. Ph.D. thesis, TU Bergakademie Freiberg, Freiberg, Germany
Simpson T, Poplinski J, Koch PN, Allen J (2001) Metamodels for computer-based engineering design: survey and recommendations. Eng Comput 17(2):129–150
Stein ML (1999) Interpolation of spatial data: some theory for Kriging. Springer, New York
Stephenson G (2010) Using derivative information in the statistical analysis of computer models. Ph.D. thesis, University of Southampton, Southampton, UK
Toal DJ, Forrester AI, Bressloff NW, Keane AJ, Holden C (2009) An adjoint for likelihood maximization. Proc R Soc A 8 465(2111):3267–3287 (2009)
Ulaganathan S, Couckuyt I, Ferranti F, Laermans E, Dhaene T (2014) Performance study of multi-fidelity gradient enhanced Kriging. Struct Multidiscip Optim. doi:10.1007/s00158-014-1192-x
Vignon-Clementel I, Figueroa C, Jansen K, Taylor C (2010) Outflow boundary conditions for 3D simulations of non-periodic blood flow and pressure fields in deformable arteries. Comput Methods Biomech Biomed Eng 13(5):625–640
Wang GG, Shan S (2006) Review of metamodeling techniques in support of engineering design optimization. J Mech Design 129(4):370–380
Yamazaki W, Rumpfkeil MP, Mavriplis DJ (2010) Design optimization utilizing Gradient/Hessian enhanced surrogate model. In: 28th AIAA Applied Aerodynamics Conference, AIAA paper 2010–4363. Chicago, Illinois, USA
Ying X, JunHua X, WeiHua Z, YuLin Z (2009) Gradient-based kriging approximate model and its application research to optimization design. Sci China Technol Sci 52(4):1117–1124
Zhao D, Xue D (2011) A multi-surrogate approximation method for metamodeling. Eng Comput 27(2):139–153
Acknowledgments
This research has been funded by the Interuniversity Attraction Poles Programme BESTCOM initiated by the Belgian Science Policy Office. Additionally, this research has been supported by the Fund for Scientific Research in Flanders (FWO-Vlaanderen). Ivo Couckuyt and Joris Degroote are post-doctoral research fellows of the Research Foundation Flanders (FWO).
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix A: Analytical expressions for likelihood gradients
1.1 Gaussian correlation function
Derivative of correlation function with respect to \(\theta _k\):
Derivatives of cross-correlation functions with respect to \(\theta _k\):
1.2 Matérn \(\frac{3}{2}\) correlation function
Derivative of correlation function with respect to \(\theta _k\):
Derivatives of cross-correlation functions with respect to \(\theta _k\):
where
1.3 Matérn \(\frac{5}{2}\) correlation function
Derivative of correlation function with respect to \(\theta _k\):
Derivatives of cross-correlation functions with respect to \(\theta _k\):
where
Appendix B: Problem description (fluid structure interaction problem)
In 2008, the World Health Organization reported that cardiovascular diseases are the leading cause of death in the world. Aortic stiffening has been linked to many (patho)physiological mechanisms and conditions. In all of them, the stiffness and mechanical properties of the aortic wall are altered. Consequently, non-invasive measurements of the arterial stiffness are needed and assessing the arterial stiffness should be part of the routine clinical diagnosis and follow-up procedures [19]. In this example, the stiffness distribution along the length of an artery is identified using a simplified numerical model. Previously, unconstrained quasi-Newton optimization with line search and a discrete adjoint solver for the calculation of the gradient were applied for this purpose [5].
The numerical model is one-dimensional in an axisymmetric \((r,\phi ,z)\) coordinate system, as depicted in Fig. 23. It consists of \(k-1\) elastic segments, each with its own stiffness. Inside the artery, there is an incompressible blood flow. Furthermore, the interaction between this blood flow and the elastic wall is taken into account.
The blood flow rate at a point can be measured as a function of time using non-invasive techniques. So, the flow rate at the inlet is prescribed as a function of the time \(t\) with a period corresponding to one heart beat \(t_b\). A Windkessel model relates this velocity with the outlet pressure [35]. This Windkessel model (See Fig. 23) represents the remainder of the circulation, downstream from the artery. The capacitor \(C\) represents the compliance of the arterial system, while the resistors \(R_p\) and \(R_d\) model the proximal and distal viscous resistance, respectively.
The goal is to adjust the stiffness parameters of this fluid–structure interaction model so that the displacement of the arterial wall as a function of time matches the displacement data from a non-invasive measurement. The elasticity modulus \(E_i\) of each segment \(i\) (\(i\in \{1,\ldots ,k-1\}\)) is modified by the corresponding parameter \(x_i\) which varies from −1 to 1.
As the Windkessel model also has a significant impact on the wall displacement, the value of \(C\) is modified by the parameter \(x_k\) which varies from −1 to 1.
The parameter \(x_k\) will be identified, together with the parameters \(x_i\) (\(i\in \{1,\ldots ,k-1\}\)). All fixed parameters are listed in Table 13.
The governing flow equations and the structural equations, which are formulated, discretized and linearized in reference [5], are solved separately. Consequently, coupling iterations using the IQN-ILS algorithm [4] need to be performed between the flow equations and the structural equations to obtain the solution of the coupled problem. A cost function \(y({\varvec{{x}}})\) is defined as the sum over all time steps and all segments of the squared difference between the radius in the simulation and in the measurement. This measurement, which would normally be obtained from a non-invasive medical imaging technique such as ultrasound, is mimicked by a simulation with the same model. It is then assumed that the parameter values in this “measurement simulation” have been forgotten and their values are calculated using the parameter identification. The vector \({\varvec{{x}}}\) contains the \(k\) parameters which are defined in 37 and 38. The state vector \({\varvec{{s}}}\) contains the radius in all segments and all time steps. The parameter identification can thus be reformulated as a minimization problem
subject to the governing equations as constraints. As the state vector \({\varvec{{s}}}\) depends on the parameters \({\varvec{{x}}}\), the total derivative of the cost function \(y({\varvec{{x}}},{\varvec{{s}}})=y({\varvec{{x}}},{\varvec{{s}}}({\varvec{{x}}}))\) with respect to the parameters is obtained with the chain rule.
To avoid the direct calculation of \({{\mathrm {d}}{\varvec{{s}}}/{\mathrm {d}}{\varvec{{x}}}}\), the adjoint equations of this unsteady fluid–structure interaction problem are derived and solved, which involves backward time steps. In each of these steps, the adjoint flow equations and adjoint structural equations are coupled using the IQN-ILS algorithm [4], similarly to the forward equations.
The stiffness off each segment is identified by constructing a surrogate model for \(y({\varvec{{x}}})\), followed by a search for its minimum and the corresponding values of \({\varvec{{x}}}\).
Appendix C: Surrogate model accuracy
Rights and permissions
About this article
Cite this article
Ulaganathan, S., Couckuyt, I., Dhaene, T. et al. Performance study of gradient-enhanced Kriging. Engineering with Computers 32, 15–34 (2016). https://doi.org/10.1007/s00366-015-0397-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00366-015-0397-y