Article

Free access

Bootstrapping and validation of metamodels in simulation

Authors:

Jack P. C. Kleijnen,

Ad J. Feelders,

Russell C. H. ChengAuthors Info & Claims

WSC '98: Proceedings of the 30th conference on Winter simulation

Pages 701 - 705

Published: 01 December 1998 Publication History

PDF eReader

References

[1]

Barton, R.R. and L.W. Schruben (1996), Resampling of empirical distributions for simulation output analysis. Working Paper, PennState, 1996

Google Scholar

[2]

Breiman, L. (1992), The little bootstrap and other methods for dimensionality selection inregression: x-fixed prediction error. Journal American Statistical Association, 87, no. 419, pp. 738-754

Crossref

Google Scholar

[3]

Cheng, R.C.H. (1995), Bootstrap methods for computer simulation experiments. Proceedings of the 1995 Winter Simulation Conference, edited by C. Alexopoulos, K. Kang, W.R. Lilegdon, and D. Goldsman.

Digital Library

Google Scholar

[4]

Cheng, R.C.H. and W. Holland (1997), Sensitivity of computer simulation experiments to errors in input data Journal Statistical Computation and Simulation, 57, numbers 1-4.

Google Scholar

[5]

Cheng, R.C.H. and J.P.C. Kleijnen (1998), Improved designs of queueing simulation experiments with highly heteroscedastic responses Operations Research (Earlier version: Optimal design of simulation experiments with nearly saturated queues, Discussion Paper, CentER, no. 9567).

Digital Library

Google Scholar

[6]

Cheng, R.C.H. and J.P.C. Kleijnen (1997) The use of Bayesian methods in regression metamodelling. Working Paper.

Google Scholar

[7]

Efron, B. and R.J. Tibshirani (1993), Introduction to the Bootstrap. Chapman & Hall, New York.

Google Scholar

[8]

Friedman, L.W. and H.H. Friedman (1995), Analyzing simulation output using the bootstrap method. Simulation, 64, no. 2, February 1995, pp. 95-100.

Crossref

Google Scholar

[9]

Kim, Y.B., T.R. Willemain, J. Haddock, and G.C. Runger (1993), The threshold bootstrap: a new approach to simulation output analysis. Proceedings of the 1993 Winter Simulation Conference, edited by G.W. Evans, M. Mollaghasemi, E.C. Russell, and W.E. Biles, pp. 498-502.

Digital Library

Google Scholar

[10]

Kleijnen, J.P.C. (1992), Regression metamodels for simulation with common random numbers: comparison of validation tests and confidence intervals. Management Science, 38, no. 8, August 1992, pp. 1164- 1185.

Digital Library

Google Scholar

[11]

Kleijnen, J.P.C. and B. Bettonvil (1990), Measurement scales and resolution IV designs. American Journal of Mathematical andManagement Sciences 10, nos. 3 & 4, pp. 309-322.

Google Scholar

[12]

Kleijnen, J.P.C. and Sargent (1997), A methodology for the fitting and validation of metamodels in simulation, Working Paper.

Google Scholar

[13]

Kleijnen, J.P.C. and Standridge (1987), Experimental design and regression analysis: an FMS case study. European Journal of Operational Research, 33, no. 3, pp. 257- 261.

Crossref

Google Scholar

[14]

Rao, C.R. (1959), Some problems involving linear hypothesis in multivariate analysis. Biometrika, 46, pp. 49-58.

Crossref

Google Scholar

[15]

Stine, R.A. (1985), Bootstrap prediction intervals for regression. Journal American Statistical Association, 80, no. 392, pp. 1026-1031.

Crossref

Google Scholar

Cited By

View all

Dellino GMeloni CRose OUhrmacher A(2012)Metamodel variability analysis combining bootstrapping and validation techniquesProceedings of the Winter Simulation Conference10.5555/2429759.2430206(1-2)Online publication date: 9-Dec-2012
https://dl.acm.org/doi/10.5555/2429759.2430206
Kleijnen JSanchez SLucas TCioppa T(2005)State-of-the-Art ReviewINFORMS Journal on Computing10.1287/ijoc.1050.013617:3(263-289)Online publication date: 1-Jul-2005
https://dl.acm.org/doi/10.1287/ijoc.1050.0136
Kleijnen JCheng RBettonvil BFishwick PKang KJoines JBarton R(2000)VV&A IVProceedings of the 32nd conference on Winter simulation10.5555/510378.510504(882-892)Online publication date: 10-Dec-2000
https://dl.acm.org/doi/10.5555/510378.510504
Show More Cited By

Index Terms

Bootstrapping and validation of metamodels in simulation
1. Computing methodologies
  1. Modeling and simulation
    1. Model development and analysis
      1. Model verification and validation
    2. Simulation types and techniques
2. Mathematics of computing
  1. Probability and statistics
    1. Probabilistic reasoning algorithms
      1. Markov-chain Monte Carlo methods
      2. Sequential Monte Carlo methods
    2. Statistical paradigms
      1. Survival analysis

Recommendations

Validation of metamodels in simulation: a new metric

Metamodels are used to provide more efficient predictions than the underlying simulation models do, but at the price of reduced prediction accuracy. Statistics used to quantify this prediction accuracy include the root-mean square error (RMSE), the ...
Testing Metamodels
ECMDA-FA '08: Proceedings of the 4th European conference on Model Driven Architecture: Foundations and Applications

In this paper, we deal with errors in metamodels. Metamodels define the abstract syntax of modeling languages. They play a central role in the Model-Driven Architecture. Other artifacts like models or tools are based on them and have to be changed if ...
Validation of metamodels in simulation: a new metric

Metamodels are used to provide more efficient predictions than the underlying simulation models do, but at the price of reduced prediction accuracy. Statistics used to quantify this prediction accuracy include the root-mean square error (RMSE), the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

WSC '98: Proceedings of the 30th conference on Winter simulation

December 1998

1766 pages

ISBN:0780351347

Publisher

IEEE Computer Society Press

Washington, DC, United States

Publication History

Published: 01 December 1998

Check for updates

Qualifiers

Article

Conference

WSC98

Sponsor:

IIE
IEEE-SMCS
INFORMS/CS
ASA
ACM
SCS
SIGSIM
IEEE-CS
NIST

WSC98: Winter Simulation Conference 1998

December 13 - 16, 1998

D.C., Washington, USA

Acceptance Rates

WSC '98 Paper Acceptance Rate 164 of 216 submissions, 76%;

Overall Acceptance Rate 3,413 of 5,075 submissions, 67%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
308
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)5

Reflects downloads up to 23 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Dellino GMeloni CRose OUhrmacher A(2012)Metamodel variability analysis combining bootstrapping and validation techniquesProceedings of the Winter Simulation Conference10.5555/2429759.2430206(1-2)Online publication date: 9-Dec-2012
https://dl.acm.org/doi/10.5555/2429759.2430206
Kleijnen JSanchez SLucas TCioppa T(2005)State-of-the-Art ReviewINFORMS Journal on Computing10.1287/ijoc.1050.013617:3(263-289)Online publication date: 1-Jul-2005
https://dl.acm.org/doi/10.1287/ijoc.1050.0136
Kleijnen JCheng RBettonvil BFishwick PKang KJoines JBarton R(2000)VV&A IVProceedings of the 32nd conference on Winter simulation10.5555/510378.510504(882-892)Online publication date: 10-Dec-2000
https://dl.acm.org/doi/10.5555/510378.510504
Kleijnen J(1999)Validation of modelsProceedings of the 31st conference on Winter simulation: Simulation---a bridge to the future - Volume 110.1145/324138.324450(647-654)Online publication date: 1-Dec-1999
https://dl.acm.org/doi/10.1145/324138.324450

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Validation of metamodels in simulation: a new metric

Testing Metamodels

Validation of metamodels in simulation: a new metric