article

Confidence Intervals for Steady-State Simulations II: A Survey of Sequential Procedures

Authors:

Averill M. Law,

W. David KeltonAuthors Info & Claims

Management Science, Volume 28, Issue 5

Pages 550 - 562

https://doi.org/10.1287/mnsc.28.5.550

Published: 01 May 1982 Publication History

Abstract

We continue our survey of methods for constructing confidence intervals for steady-state means via simulation by studying sequential procedures which determine the length of the simulation during the course of the run. Our goal is to provide the simulation practitioner with some guidance as to which published procedures might actually perform well in practice. Empirical results for a variety of stochastic models with known steady-state means suggest that sequential procedures by Fishman and by Law and Carson provide good performance relative to the criterion probability of coverage.

References

[1]

CHOW, Y. S. AND ROBBINS, H., "On the Asymptotic Theory of Fixed-Width Sequential Confidence Intervals for the Mean," Ann. Math. Statist., Vol. 36 (1965), pp. 457-462.

[2]

CRANE, M. A. AND IGLEHART, D. L., "Simulating Stable Stochastic Systems, I: General Multiserver Queues," J. Assoc. Comput. Mach., Vol. 21 (1974), pp. 103-113.

Digital Library

[3]

CRANE, M. A. AND IGLEHART, D. L., "Simulating Stable Stochastic Systems, II: Markov Chains," J. Assoc. Comput. Mach., Vol. 21 (1974), pp. 114-123.

Digital Library

[4]

CRANE, M. A. AND IGLEHART, D. L., "Simulating Stable Stochastic Systems, III: Regenerative Processes and Discrete-Event Simulations," Operations Res., Vol. 23 (1975), pp. 33-45.

Digital Library

[5]

CRANE, M. A. AND LEMOINE, A. J., An Introduction to the Regenerative Method for Simulation Analysis, Springer-Verlag, New York, 1977.

[6]

FISHMAN, G. S., "Estimating Sample Size in Computer Simulation Experiments," Management Sci., Vol. 18 (1971), pp. 21-38.

[7]

FISHMAN, G. S., Concepts and Methods in Discrete Event Digital Simulation, Wiley, New York, 1973.

[8]

FISHMAN, G. S., "Achieving Specific Accuracy in Simulation Output Analysis," Comm. Assoc. Comput. Mach., Vol. 20(1977), pp. 310-315.

Digital Library

[9]

HEIDELBERGER, P. AND WELCH, P. D., "A Spectral Method for Confidence Interval Generation and Run Length Control in Simulations," Comm. Assoc. Comput. Mach., Vol. 24 (1981), pp. 233-245.

Digital Library

[10]

IGLEHART, D. L., "Simulating Stable Stochastic Systems, V: Comparison of Ratio Estimators," Naval Res. Logist. Quart., Vol. 22 (1975), pp. 553-565.

[11]

KLEIJNEN, J. P. C., Statistical Techniques in Simulation, Part II, Marcel Dekker, New York, 1975.

[12]

LAVENBERG, S. S. AND SAUER, C. H., "Sequential Stopping Rules for the Regenerative Method of Simulation," IBM J. Res. Develop., Vol. 21 (1977), pp. 545-558.

Digital Library

[13]

LAW, A. M., "Confidence Intervals in Discrete Event Simulation: A Comparison of Replication and Batch Means," Naval Res. Logist. Quart., Vol. 24 (1977), pp. 667-678.

[14]

LAW, A. M. AND CARSON, J. S., "A Sequential Procedure for Determining the Length of a Steady-State Simulation," Operations Res., Vol. 27 (1979), pp. 1011-1025.

Digital Library

[15]

LAW, A. M. AND KELTON, W. D., "Confidence Intervals for Steady-State Simulations, I: A Survey of Fixed Sample Size Procedures," Technical Report No. 78-5, Department of Industrial Engineering, University of Wisconsin, 1979.

[16]

MECHANIC, H. AND MCKAY, W., "Confidence Intervals for Averages of Dependent Data in Simulations II," Technical Report No. ASDD 17-202, IBM Corporation, Yorktown Heights, New York, 1966.

[17]

MILLER, R. G., "The Jackknife¿A Review," Biometrika, Vol. 61 (1974), pp. 1-15.

[18]

ROBINSON, D. W., "Determination of Run Length in Simulations of Stable Stochastic Systems," Technical Report No. 86-21, Control Analysis Corporation, Palo Alto, California, 1976.

[19]

SHAPIRO, S. S. AND FRANCIA, R. S., "An Approximate Analysis of Variance Test for Normality," J. Amer. Statist. Assoc., Vol. 67 (1972), pp. 215-216.

[20]

SHAPIRO, S. S. AND WILK, M. B., "An Analysis of Variance Test for Normality (Complete Samples)," Biometrika, Vol. 52 (1965), pp. 591-611.

[21]

STARR, N., "The Performance of a Sequential Procedure for the Fixed-Width Interval Estimation of the Mean," Ann. Math. Statist., Vol. 37 (1966), pp. 36-50.

[22]

WEISBERG, S. AND BINGHAM, C., "An Approximate Analysis of Variance Test for Non-Normality Suitable for Machine Calculation," Technometrics, Vol. 17 (1975), pp. 133-134.

Cited By

Alexopoulos CKelton W(2017)A concise history of simulation output analysisProceedings of the 2017 Winter Simulation Conference10.5555/3242181.3242191(1-16)Online publication date: 3-Dec-2017
https://dl.acm.org/doi/10.5555/3242181.3242191
Singham D(2014)Selecting Stopping Rules for Confidence Interval ProceduresACM Transactions on Modeling and Computer Simulation10.1145/262773424:3(1-18)Online publication date: 25-Jul-2014
https://dl.acm.org/doi/10.1145/2627734
Singham DSchruben LDunkin AIngalls R(2009)Analysis of sequential stopping rulesWinter Simulation Conference10.5555/1995456.1995565(723-730)Online publication date: 13-Dec-2009
https://dl.acm.org/doi/10.5555/1995456.1995565
Show More Cited By

Confidence Intervals for Steady-State Simulations II: A Survey of Sequential Procedures
1. Applied computing
2. Mathematics of computing

Recommendations

How Hard are Steady-State Queueing Simulations?
Special Issue on Don Iglehart

Some queueing systems require tremendously long simulation runlengths to obtain accurate estimators of certain steady-state performance measures when the servers are heavily utilized. However, this is not uniformly the case. We analyze a number of ...
Confidence Intervals for Steady-State Simulations: I. A Survey of Fixed Sample Size Procedures

We consider the problem of constructing a confidence interval for the steady-state mean of a stochastic process by means of simulation, and study the five main methods which have been proposed replication, batch means, autoregressive representation, ...
Robustness of confidence intervals for transient simulations
WSC '77: Proceedings of the 9th conference on Winter simulation - Volume 1

Most of the statistical simulation literature is concerned with steady-state simulations. That is, a simulation where the quantity to be estimated is defined as a limit as time (or the number of customers) goes to infinity. For example, the regenerative ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Management Science

Management Science Volume 28, Issue 5

May 1982

144 pages

ISSN:0025-1909

Issue’s Table of Contents

Publisher

INFORMS

Linthicum, MD, United States

Publication History

Published: 01 May 1982

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Alexopoulos CKelton W(2017)A concise history of simulation output analysisProceedings of the 2017 Winter Simulation Conference10.5555/3242181.3242191(1-16)Online publication date: 3-Dec-2017
https://dl.acm.org/doi/10.5555/3242181.3242191
Singham D(2014)Selecting Stopping Rules for Confidence Interval ProceduresACM Transactions on Modeling and Computer Simulation10.1145/262773424:3(1-18)Online publication date: 25-Jul-2014
https://dl.acm.org/doi/10.1145/2627734
Singham DSchruben LDunkin AIngalls R(2009)Analysis of sequential stopping rulesWinter Simulation Conference10.5555/1995456.1995565(723-730)Online publication date: 13-Dec-2009
https://dl.acm.org/doi/10.5555/1995456.1995565
Bayraksan GMorton DTew J(2007)Sequential sampling for solving stochastic programsProceedings of the 39th conference on Winter simulation: 40 years! The best is yet to come10.5555/1351542.1351629(421-429)Online publication date: 9-Dec-2007
https://dl.acm.org/doi/10.5555/1351542.1351629
Kounev SBuchmann A(2006)SimQPNPerformance Evaluation10.1016/j.peva.2005.03.00463:4(364-394)Online publication date: 1-May-2006
https://dl.acm.org/doi/10.1016/j.peva.2005.03.004
L'Ecuyer PRohrer MMedeiros DGrabau M(2001)Software for uniform random number generationProceedings of the 33nd conference on Winter simulation10.5555/564124.564139(95-105)Online publication date: 9-Dec-2001
https://dl.acm.org/doi/10.5555/564124.564139
McNickle DPawlikowski KEwing G(1996)Experimental evaluation of confidence interval procedures in sequential steady-state simulationProceedings of the 28th conference on Winter simulation10.1145/256562.256690(382-389)Online publication date: 8-Nov-1996
https://dl.acm.org/doi/10.1145/256562.256690
Schruben LGoldsman DMorrice D(1996)Using null effects estimators in the analysis of simulation outputProceedings of the 28th conference on Winter simulation10.1145/256562.256655(367-371)Online publication date: 8-Nov-1996
https://dl.acm.org/doi/10.1145/256562.256655
Schmeiser BSong W(1996)Batching methods in simulation output analysisProceedings of the 28th conference on Winter simulation10.1145/256562.256589(122-127)Online publication date: 8-Nov-1996
https://dl.acm.org/doi/10.1145/256562.256589
(1994)Two-Stage Stopping Procedures Based on Standardized Time SeriesManagement Science10.5555/2725661.272566840:9(1189-1206)Online publication date: 1-Sep-1994
https://dl.acm.org/doi/10.5555/2725661.2725668
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents