article

Free access

Steady-state simulation of queueing processes: survey of problems and solutions

Author:

Krzysztof PawlikowskiAuthors Info & Claims

ACM Computing Surveys (CSUR), Volume 22, Issue 2

Pages 123 - 170

https://doi.org/10.1145/78919.78921

Published: 01 June 1990 Publication History

Abstract

For years computer-based stochastic simulation has been a commonly used tool in the performance evaluation of various systems. Unfortunately, the results of simulation studies quite often have little credibility, since they are presented without regard to their random nature and the need for proper statistical analysis of simulation output data.

This paper discusses the main factors that can affect the accuracy of stochastic simulations designed to give insight into the steady-state behavior of queuing processes. The problems of correctly starting and stopping such simulation experiments to obtain the required statistical accuracy of the results are addressed. In this survey of possible solutions, the emphasis is put on possible applications in the sequential analysis of output data, which adaptively decides about continuing a simulation experiment until the required accuracy of results is reached. A suitable solution for deciding upon the starting point of a steady-state analysis and two techniques for obtaining the final simulation results to a required level of accuracy are presented, together with pseudocode implementations.

References

[1]

ABATE, J., AND WHITT, W. 1987a. Transient behavior of regulated brownian motion, I: Starting at the origin. Adv. Appl. Probab. 19, 560-598.

[2]

ABATE, J., AND WHITT, W. 1987b. Transient behavior of regulated brownian motion, II: Non-zero initial conditions. Adv. Appl. Probab. 19, 599-631.

[3]

ABATE, J., AND WHITT, W. 1987c. Transient behavior of the M/M/1 queue: Starting at the origin. Queue. Syst. 2, 41-65.

[4]

ABATE, J., AND WHITT, W. 1988. Transient behavior of the M/M/1 queue via Laplace transform. Adv. Appl. Probab. 20, 145-178.

[5]

ADAM, N. R. 1983. Achieving a confidence interval for parameters estimated by simulation. Manage. Sci., 29, 856-866.

[6]

AMEMIYA, T. 1973. Generalized least squares with an estimated autocovariance matrix. Econometria 41,723-732.

[7]

ANDERSON, K. 1985. Some steady-state problems in simulation. In Proceedings of the Annual Simulation Symposium (Tampa, Fla.). IEEE Computer Society, New York, 79-93.

[8]

ANDREWS, R. W., AND SCHRIBER, T. Z. 1978. Interactive analysis of output from GPSS-based simulations. In Proceedings of the 1978 Winter Simulation Conference (Miami Beach, Fla.). ACM/IEEE, New York, 267-278.

[9]

ANONUEVO, R., AND NELSON, B. L. 1986. Automated estimation and variance reduction for steady state simulation. In Proceedings of the 1986 Winter Simulation Conference (Washington, D.C.). ACM/IEEE, New York, 871-875.

[10]

Artificial intelligence and simulation: The diversity of applications. 1988. In Proceedings of the SCS Multiconference (San Diego, Calif.). Society for Computer Simulation.

[11]

ASGARKHANI, M., AND PAWLIKOWSKI, K. 1989. Simulation studies of mixed traffic on satellite channels using TDMA-reservation protocol. In Proceedings of the 8th Phoenix International Conference on Computers and Communications (Scottsdale, Ariz.). IEEE, Washington, D.C., 195-200.

[12]

BAIK, D., AND ZEIGLER, B. P. 1985. Performance evaluation of hierarchical distributed simulators. In Proceedings of the 1985 Winter Simulation Conference (San Francisco, Calif.). ACM/IEEE, New York, 421-427.

[13]

BALMER, D. W. 1987. Polishing the analysis of the statistical output of comparative simulation experiments. Simulation, 49, 123-126.

[14]

BANKS, J., AND CARSON, J. S. 1984. Discrete Event System Simulation. Prentice-Hall, Englewood Cliffs, N.J.

[15]

BARTLETT, M. S. 1946. On the theoretical specification of the sampling properties of autocorrelated time series. J. Roy. Star. Soc. Ser. B, 8, 27-41.

[16]

BAUER, K. W., VENKATRAMAN, S., AND WILSON, J. R. 1987. Estimation procedures based on control variates with known covariance matrix. In Proceedings of the 1987 Winter Simulation Conference (Atlanta, Ga.). ACM/IEEE, New York, 334-341.

[17]

BEALL, C. W. 1982. A regression technique or determining steady state conditions in time series simulations. In Proceedings of the 1982 Winter Simulation Conference (San Diego, Calif.). ACM/ IEEE, New York, 439-448.

[18]

BELL, P. C., AND O'KNEEFE, R. M. 1987. Visual interactive simulation: History, recent developments, and major issues. Simulation, 49, 109-116.

[19]

BHARATH-KUMAR, K., AND KERMANI, P. 1984. Performance evaluation tool (PET): An analysis tool for computer communication networks. IEEE J. Select. Areas Commun. SAC-2, 220-225.

[20]

BILLINGSLEY, P. 1968. Convergence of Probability Measures. Wiley, New York.

[21]

BLACKMAN, R. B., AND TUCKEY, J. W. 1958. The measurement of power spectra from the point of view of communications engineering (Part 1 and Part 2). Bell Syst. Tech. J. 37, 185-282, 485-569.

[22]

BLANC, J. P. C. 1985a. The transient behaviour of networks with infinite servers nodes. In Performance '84, E. Gelenbe, Ed. North Holland, Amsterdam, 159-174.

[23]

BLANC, J. P. C. 1985b. The relaxation time of two queueing systems in series. Commun. Statist. Stochastic Models 1, 1-16.

[24]

BLOMQVIST, N. 1967. The covariance function of the M/G/1 queueing system. Skandinavisk Aktuarietidskrift, 6, 157-174.

[25]

BLOMQVIST, i. 1970. On the transient behavior of the GI/G/1 waiting-times. Skandinavisk Aktuarietidshrift, 53, 118-129.

[26]

BLUM, A., ET AL. 1985. Experiments with decomposition of extended queueing network models. In Modelling Techniques and Tools for Performance Analysis, D. Potier, Ed. North-Holland, The Netherlands, 623-638.

[27]

Box, G. E. P., AND JENKINS, G. M. 1970. Time Series Analysis, Forecasting and Control. Holden- Day, San Francisco.

[28]

BRATLEY, P., FOX, B. L., AND SCHRAGE, L. E. 1983. A Guide to Simulation. Springer-Verlag, New York.

[29]

BRILLINGER, D. R. 1973. Estimation of the mean of a stationary time series by sampling. J. Appl. Prob., 10, 419-431.

[30]

BRILLINGER, D. R. 1981. Time Series: Data Analysis and Theory. Holden-Day, San Francisco.

[31]

BROCKWELL, P. J., AND DAVIS, R. A. 1987. Time Series: Theory and Methods. Springer-Verlag, New York.

[32]

BULGREN, W. G. 1982. Discrete System Simulation. Prentice-Hall, Englewood Cliffs, N.J.

[33]

CALVIN, J. M. 1988. Covariance of regenerative mean and variance estimators for the Markov chains. In Proceedings of the 1988 Winter Simulation Conference (San Diego, Calif.). ACM/ IEEE, New York, 473-475.

[34]

CARSON, J. S., AND LAW, A. M. 1980. Conservation Equations and Variance Reduction in Queueing Simulations. Oper. Res., 28, 535-546.

[35]

Catalog of simulation software. 1987. Simulation, 49, 165-181.

[36]

CHANDY, K. M., AND MISRA, J. 1981. Asynchronous distributed simulation via a sequence of parallel computations. Commun. ACM, 24, 198-205.

[37]

CHARNES, J. M., AND KELTON, W. D. 1988. A comparison of confidence region estimators for multivariate simulation output. In Proceedings of the 1988 Winter Simulation Conference (San Diego, Calif.). ACM/IEEE, New York, 458-465.

[38]

CHEN, D. R., AND SEILA, A. F. 1987. Multivariate inference in stationary simulation using batch means. In Proceedings of the 1987 Winter Simulation Conference (Atlanta, Ga.). ACM/IEEE, New York, 302-304.

[39]

CHENG, R. C. H. 1976. A note on the effect of initial conditions on a simulation run. Oper. Res. Quart., 24, 467-470.

[40]

CHENG, R. C. H. 1986. Variance reduction methods. In Proceedings of the 1986 Winter Simulation Conference (Washington, D.C.). ACM/IEEE, New York, 60-68.

[41]

CLARK, G. M. 1986. A Bonferroni selection procedure when using common numbers with unknown variances. In Proceedings of the 1987 Winter Simulation Conference (Washington, D.C.). ACM/ IEEE, New York, 313-315.

[42]

COHEN, J. W. 1982. The Single Server Queue. North- Holland, Amsterdam.

[43]

CONWAY, R. W. 1963. Some tactical problems in digital simulation. Manage. Sci., 10, 47-61.

[44]

CONWAY, R. W., JOHNSON, B. M., AND MAXWELL, W. C. 1959. Some problems of digital systems simulation. Manage. Sci. 6, 92-110.

[45]

COTTRELL, M., FORT, J. C., AND MALGOUYRES, G. 1983. Large deviations and rare events in the study of stochastic algorithms. IEEE Trans. Automatic Control, AC-28, 907-918.

[46]

COX, D. R., AND SMITH, S. W. L. 1961. Queues. Wiley, New York.

[47]

CRANE, M., AND IGLEHART, D. L. 1974. Simulating stable stochastic systems: I. General multiserver queues. J. ACM, 21, 103-113.

[48]

CRANE, M. A., AND IGLEHART, D. L. 1975a. Simulating stable stochastic systems: III. Regenerative processes and discrete-event simulations. Oper. Res., 23, 33-45.

[49]

CRANE, M. A., AND IGLEHART, D. L. 1975b. Simulating stable stochastic systems. IV: Approximation techniques. Manage. Sci., 21, 1215-1224.

[50]

CRANE, M. A., AND LEMOINE, A. J. 1977. An Introduction to the Regenerative Method for Simulation Analysis. Lecture Notes in Control and Information Science, No. 4, A. W. Balakrishnan and M. Thoma, Eds. Springer-Verlag, New York.

[51]

DALEY, D. J. 1968. The serial correlation coefficients of waiting times in a stationary simple server queue. J. Austral. Math. Soc., 8, 683-699.

[52]

DAMERDJI, H. 1987. On strong consistency of the variance estimator. In Proceedings of the 1987 Winter Simulation Conference (Atlanta, Ga.). ACM/IEEE, New York, 305-308.

[53]

DECEGAMA, A. 1987. Parallel processing simulation of large computer networks. In Proceedings of the 1987 Symposium on the Simulation of Computer Networks (Colorado). ACM/IEEE, New York, 51-62.

[54]

DONNELLY, J. N., AND SHANNON, R. E. 1981. Minimum mean-squared error estimates for simulation experiments. Commun. ACM, 24, 253-259.

[55]

DUERSCH, R. R., AND SCHRUBEN, L. W. 1986. An interactive run length control for simulations on PCs. In Proceedings of the 1986 Winter Simulation Conference (Washington, D.C.). ACM/IEEE, New York, 866-870.

[56]

DUKET, S. D., AND PRITSKER, A. A. B. 1978. Examination of simulation output using spectral methods. Math. Comput. Simul., 20, 53-60.

[57]

EMSHOFF, J. R., AND SISSON, R. L. 1970. Design and Use of Computer Simulation Models. Macmillan, New York.

[58]

FISHMAN, G. S. 1967. Problems in the statistical analysis of simulation experiments: The comparison of means and the length of sample records. Commun. A CM, 10, 94-99.

[59]

FISHMAN, G. S. 1971. Estimating sample size in computing simulation experiments. Manage. Sci., 18, 21-38.

[60]

FISHMAN, G. S. 1972. Bias considerations in simulation experiments. Oper. Res., 20, 685-790.

[61]

FISHMAN, G. S. 1973a. Concepts and Methods in Discrete Event Digital Simulation. John Wiley, New York.

[62]

FISHMAN, G. S. 1973b. Statistical analysis for queueing simulation. Manage. Sci., 20, 363-369.

[63]

FISHMAN, G. S. 1974. Estimation in multiserver queueing simulations. Oper. Res., 22, 72-78.

[64]

FISHMAN, G. S. 1977. Achieving specific accuracy in simulation output analysis. Commun. A CM, 20, 310-315.

[65]

FISHMAN, G. S. 1978a. Principles of Discrete Event Simulation. John Wiley, New York.

[66]

FISHMAN, G. S. 1978b. Grouping observations in digital simulation. Manage. Sci., 24, 510-521.

[67]

FISHMAN, G. S., AND KIVIAT, P. J. 1967. The analysis of simulation-generated time series. Manage. ScL, 13, 525-557.

[68]

FOLEY, R. D., AND GOLDSMAN, D. 1988. Confidence intervals and orthonormally weighted standardized time series. In Proceedings of the 1988 Winter Simulation Conference (San Diego, Calif.). ACM/ IEEE, New York, 422-424.

[69]

Fox, B. 1978. Estimation and simulation. Manage. Sci., 24, 860-861.

[70]

FRIEDMAN, L. W., AND FRIEDMAN, H. H. 1986. Comparing simulated alternatives using a distribution-free statistic with blocking by random number stream. Simulation, 48, 68-70.

[71]

FRIEDMAN, a. W. 1984. Multivariate simulation output analysis: Past, present, and future. In Proceedings of the 1984 Winter Simulation Conference (Dallas, Tex.). ACM/IEEE, New York, 277-281.

[72]

FROST, V. S., LAURE, W. W., AND SHANMUGAN, K. S. 1988. Efficient techniques for simulation of computer communication networks. IEEE J. Selected Areas Commun., SAC-8, 146-157.

[73]

FUJIMOTO, R. M. 1988. Performance measurements of distributed simulation strategies. In Proceedings of Distributed Simulation 1988 (San Diego, Calif.). Society for Computer Simulation, 14-20.

[74]

GAFARIAN, A. V., ANCKER C. J., AND MORISAKU, T. 1978. Evaluation of commonly used rules for detecting "steady state" in computer simulation. Naval Res. Logist. Quart. 78, 511-529.

[75]

GATES, B., CAMMARATA, S., AND ROTHENBERG, J. 1988. A demon facility for object-oriented simulation language. In Proceedings of the 1988 Summer Simulation Conference (Seattle, Wash.). Society for Computer Simulation, San Diego, Calif., 667-673.

[76]

GEISLER, M. A. 1964. The size of simulation samples required to compute certain inventory characteristics with stated precision and confidence. Manage. Sci., 10, 261-271.

[77]

GLYNN, P. W. 1982. Coverage error for confidence intervals arising in simulation output analysis. In Proceedings of the 1982 Winter Simulation Conference (San Diego, Calif.). ACM/IEEE, New York, 369-375.

[78]

GLYNN, P. W., AND IGLEHART, D. L. 1985. Largesample theory for standardized time series: An overview. In Proceedings of the 1985 Winter Simulation Conference (San Francisco, Calif.). ACM/ IEEE, New York, 129-134.

[79]

GLYNN, P. W., AND WHITT, W. 1989. Indirect estimation via L = ~ W. Manage. Sci., 36, 82-103.

[80]

GOLDSMAN, D. 1983. Ranking and selection in simulation. In Proceedings of the 1983 Winter Simulation Conference (Arlington, Va.). ACM/ IEEE, New York, 387-393.

[81]

GOLDSMAN, D. 1986. Tutorial on indifference-zone normal means ranking and selection procedures. In Proceedings of the 1986 Winter Simulation Conference (Washington, D.C.). ACM/IEEE, New York, 370-375.

[82]

GOLDSMAN, D., AND MEKETON, M. S. 1985. A comparison between standardized time series and overlapping batched means. In Proceedings of the 1985 Winter Simulation Conference. IEEE, New York, 208-210.

[83]

GOLDSMAN, D., AND MEKETON, M. S. 1990. A comparison of several variance estimators. Oper. Res. (to be published).

[84]

GOLDSMAN, D., KANG, K., AND SARGENT, R. G. 1986. Large and small sample comparisons of various variance estimators. In Proceedings of the 1986 Winter Simulation Conference (Washington, D.C.). ACM/IEEE, New York, 278-284.

[85]

GOLDSMAN, D., AND SCHRUBEN, L. 1984. Asymptotic properties of some confidence interval estimators for simulation output. Manage. Sci., 30, 1217-1225.

[86]

GORDON, G. 1969. System Simulation. Prentice Hall, Englewood Cliffs, N.J.

[87]

GROSS, D., AND HARRIS, C. M. 1974. Fundamentals of Queueing Theory. John Wiley, New York.

[88]

GUNTHER, F. L., AND WOLFF, R. W. 1980. The almost regenerative method for stochastic system simulations. Oper. Res., 28, 375-386.

[89]

HADDOCK, J. 1987. An expert system framework based on a simulation generator. Simulation, 49, 45-53.

[90]

HAHN, G. J. 1985. More intelligent statistical software and statistical expert systems: Future directions. Am. Star., 39, 1-16.

[91]

HAMMING, R. W. 1962. Numerical Methods for Scientists and Engineers. McGraw-Hill, New York.

[92]

HAND, D. J. 1985. Statistical expert systems: Necessary attributes. J. Appl. Stat., 12, 19-27.

[93]

HANNAN, E. J. 1970. Multiple Time Series. John Wiley, New York.

[94]

HARLING, J. 1985. Simulation techniques in operations research: A review. Oper. Res., 33, 307-319.

[95]

HEIDELBERGER, P. 1979. A variance reduction technique that increases regeneration frequency. In Current Issues in Computer Simulation. Academic Press, New York, 257-269.

[96]

HEIDELBERGER, P. 1986. Statistical analysis of parallel simulation. In Proceedings of the I986 Winter Simulation Conference (Washington, D.C.). ACM/IEEE, New York, 290-295.

[97]

HEIDELBERGER, P., AND LEWIS, P. A. W. 1981. Regression: Adjusted estimates for regenerative simulations, with graphics. Commun. A CM, 24, 260-273.

[98]

HEIDELBERGER, P., AND LEWIS, P. A. W. 1984. Quantile estimation in dependent sequences. Oper. Res., 32, 185-209.

[99]

HEIDELBERGER, P., AND WELCH, P. D. 1981a. A spectral method for confidence interval generation and run length control in simulations. Commun. A CM, 24, 233-245.

[100]

HEIDELBERGER, P., AND WELCH, P. D. 1981b. Adaptive spectral methods for simulation output analysis. IBM J. Res. Dev., 25, 860-876.

[101]

HEIDELBERGER, P., AND WELCH, P. D. 1983. Simulation run length control in the presence of an initial transient. Oper. Res., 31, 1109-1144.

[102]

Ho, Y. C. 1987. Performance evaluation and perturbation analysis of discrete event dynamic systems. IEEE Trans. Automat. Control, AC-32, 563-572.

[103]

Ho, Y. C., AND CAO, X. 1983. Perturbation analysis and optimization of queueing networks. J. Optim. Theory Appl., 40, 559-582.

[104]

HO, Y. C., CAO, X., AND CASSANDRAS, C. 1983. Infinitesimal and finite perturbation analysis for queueing networks. Automatica, 4, 439-445.

[105]

IGLEHART, D. L. 1975. Simulating stable stochastic systems. V: Comparison of ratio estimators. Naval Res. Logist. Quart., 22, 553-565.

[106]

IGLEHART, D. L. 1976. Simulating stable stochastic systems VI: Quantile estimation. J. A CM 347-360.

[107]

IGLEHART, D. L. 1978. The regenerative method for simulation analysis. In Current Trends in Programming Methodology. Vol. IIi, Software Modeling, K. M. Chandy and P. T. Yeh, Eds., Prentice Hall, Englewood Cliffs, N.J., 52-71.

[108]

IZYDORCZYK, J., MIERZWA, J., AND WOLISZ, A. 1984. A control variable method for increasing the efficiency of multiple access protocol simulation. In Performance of Computer-Communication Systems, H. Rudin and W. Bux, Eds., North- Holland, Amsterdam, 95-108.

[109]

JACKMAN, J., AND MEDEIROS, D. J. 1988. A graphical methodology for simulating communication networks. IEEE Trans. Commun., A COM-36, 459-464.

[110]

JAIN, R., AND CHLAMTAC, I. 1985. The p2 algorithm for dynamic calculation of quantiles and histograms without storing observations. Commun. ACM, 28, 1076-1085.

[111]

JENKINS, G. M., AND WATTS, D. G. 1968. Spectral Analysis and Its Applications. Holden-Day, San Francisco.

[112]

KANG, K., AND GOLDSMAN, D. 1985. The correlation between mean and variance estimators. In Proceedings of the 1985 Winter Simulation Conference (San Francisco, Calif.). ACM/IEEE, New York, 211-216.

[113]

KELTON, W. D. 1986. Statistical design and analysis. In Proceedings of the 1986 Winter Simulation Conference (Washington, D.C.). ACM/IEEE, New York, 45-51.

[114]

KELTON, W. D. 1985. Transient exponential-Erlang queues and steady-state simulation. Commun. ACM, 28, 741-749.

[115]

KELTON, W. D., AND LAW. A. M. 1985. The transient behaviour of the M/M/S queue, with implications for steady-state simulation. Oper. Res., 33, 378-396.

[116]

KELTON, W. D., AND LAW, A. M. 1984. An analytical evaluation of alternative strategies in steadystate simulation. Oper. Res., 32, 169-184.

[117]

KELTON, W. D., AND LAW, A. M. 1983. A new approach for dealing with the startup problem in discrete event simulation. Naval Res. Logist. Quart., 30, 641-658.

[118]

KERCKHOFFS, E. J. H., AND VANSTEENKISTE, G. C. 1986. The impact of advanced information processing on simulation: An illustrative review. Simulation, 48, 17-26.

[119]

KIEFER, J., AND WOLFOWITZ, J. 1955. On the theory of queues with many servers. Trans. Am. Math. Soc., 78, 1-18.

[120]

KLEIJNEN, J. P. C. 1974. Statistical Techniques in Simulation, Vol. 1. Marcel Dekker, New York.

[121]

KLEIJNEN, J. P. C. 1979. The role of statistical methodology in simulation. In Methodology in Systems Modelling and Simulation, B. P. Zeigler, M. S. Elzas, G. J. Klir, and T. I. Oren, Eds. North-Holland, Amsterdam.

[122]

KLEIJNEN, J. P. C. 1987. Statistical Tools for Simulation Practitioners. Marcel Dekker, New York.

[123]

KLEIJNEN, J. P. C., VAN DER VEN, R., AND SAUNDER, B. 1982. Testing independence of simulation subruns: A note on the power of the yon Neumann test. Eur. J. Oper. Res., 9, 92-93.

[124]

KLEINROCK, L. 1976. Queueing Systems. Vol. 2, Computer Applications. John Wiley, New York.

[125]

KNAPP, V. 1986. The smalltalk simulation environment. In Proceedings of the 1986 Winter Simulation Conference (Washington, D.C.). ACM/IEEE, New York, 125-128.

[126]

KREUTZER, W. 1986. System Simulation. Programming Styles and Languages. Addison Wesley, Sydney.

[127]

KRISHNAMURTHY, M., CHANDRASEKARAN, U., AND SHEPPARD, S. 1985. Two approaches to the implementation of a distributed simulation system. In Proceedings of the 1985 Winter Simulation Conference (San Francisco, Calif.). ACM/IEEE, New York, 435-443.

[128]

KUMAR, D. 1986. A novel approach to sequential simulation. IEEE Softw., 3, 25-33.

[129]

KUROSE, J. F., AND MOUFTAH, H. T. 1988. Computer-aided modeling, analysis, and design of communication networks. IEEE J. Select. Areas Commun., SAC-6, 130-145.

[130]

LAVENBERG, S. S., AND SAUER, C. I-I. 1977. Sequential stopping rules for the regenerative method of simulation. IBM J. Res. Dev., 2I, 545-558.

[131]

LAVENBERG, S. S., AND WELCH, P. D. 1981. A perspective on the use of control variables to increase the efficiency of Monte Carlo simulations. Manage. Sci., 27, 322-335.

[132]

LAVENBERG, S. S., MOELLER, T. L., AND WELCH, P. D. 1982. Statistical results on control variables with application to queueing network simulation. Oper. Res., 30, 182-202.

[133]

LAVENBERG, S. S., ET AL. 1981. The initial transient in steady state simulation (panel discussion). In Proceedings of the 1981 Winter Simulation Conference (Atlanta, Ga.). ACM/IEEE, New York, 113-120.

[134]

LAW, A. M. 1975. Efficient estimators for simulated queueing systems. Manage. Sci., 22, 30-41.

[135]

LAW, A. M. 1977. Confidence interval in discrete event simulation: A comparison of replication and batch means. Naval Res. Logist. Quart., 24, 667-678.

[136]

LAW, A. M. 1980. Statistical analysis of the output data from terminating simulations. Naval Res. Logist. Quart., 27, 131-143.

[137]

LAW, A. M. 1983. Statistical analysis of simulation output data. Oper. Res., 3I, 983-1029.

[138]

LAW, A. M., AND CARSON, J. C. 1979. A sequential procedure for determining the length of a steady state simulation. Oper. Res., 27, 1011-1025.

[139]

LAW, A. M., AND KELTON, W. D. 1982a. Simulation, Modeling and Analysis. McGraw-Hill, New York.

[140]

LAW, A. M., AND KELTON, W. D. 1982b. Confidence intervals for steady state simulations. II: A survey of sequential procedures. Manage. Sci., 28, 550-562.

[141]

LAW, A. M., AND KELTON, W. D. 1984. Confidence intervals for steady state simulations. I: A survey of fixed sample size procedures. Oper. Res. 1221-1239.

[142]

MACNAIR, E. A. 1985. An introduction to the research queueing package. In Proceedings of the 1985 Winter Simulation Conference (San Francisco, Calif.). ACM/IEEE, New York, 257-262.

[143]

MADANSKY, A. 1976. Optimal initial conditions for a simulation problem. Oper. Res., 24, 172-577.

[144]

MARKS, N. B. 1981. Further investigation into spectral analysis for confidence intervals in steady state simulation. In Proceedings of the 1981 Winter Simulation Conference (Atlanta, Ga.). ACM/ IEEE, New York, 461-464.

[145]

MEKETON, M. S. 1980. The variance time-curve: Theory, estimation, and application. Ph.D. dissertation, School of Operations Research and Industrial Engineering, Cornell Univ., Ithaca, N.Y.

[146]

MEKETON, M. S., AND HEIDELBERGER, P. 1982. A renewal theoretic approach to bias reduction on regenerative simulations. Manage. Sci., 28, 173-181.

[147]

MEKETON, i. S., AND SCHMEISER, B. 1984. Overlapping batch means: Something for nothing? In Proceedings of the 1984 Winter Simulation Conference (Dallas, Tex.). ACM/IEEE, New York, 227-230.

[148]

MELLICHAMP, J. M., AND PARK, Y. H. 1989. A statistical expert system for simulation analysis. Simulation, 51,134-139.

[149]

MILLER, R. G. 1974. The Jackknife: A Review. Biometrika, 61, 1-15.

[150]

MILLS, R. 1987. Statistical Analysis of Steady State Simulation Output Data with SIMSCRIPT 11.5. CACI, Los Angeles.

[151]

MINH, D. L. 1987. Simulating GI/G/k queues in heavy traffic. Manage. Sci., 34, 1192-1199.

[152]

MISRA, J. 1986. Distributed discrete event simulation. ACM Comput. Surv., 18, 39-66.

[153]

MITRANI, I. 1982. Simulation techniques for discrete event systems. Cambridge University Press, Cambridge, U.K.

[154]

MORSE, P. M. 1955. Stochastic processes of waiting lines. Oper. Res., 1,255-261.

[155]

MURRAY, J. R., AND KELTON, W. D. 1988. Initializing for bias reduction: Some analytical considerations. In Proceedings of the 1988 Winter Simulation Conference (San Diego, Calif.). ACM/ IEEE, New York, 546-548.

[156]

NELSON, B. L., AND SCHMEISER, B. W. 1986. Decomposition of some well-known variance reduction techniques. J. Stat. Comput. Simul., 23, 183-209.

[157]

NEWELL, G. F. 1971. Applications of Queueing Theory. Chapman & Hall, London.

[158]

NICOL, D. M. 1988. Parallel discrete event simulation of FCFS stochastic queueing networks. In Parallel Programming: Experience with Applications, Languages and Systems, ACM SIGPLAN, 124-137.

[159]

NICOL, D. M. 1988. High performance parallelized discrete-event simulation of stochastic queueing networks. In Proceedings of the 1988 Winter Simulation Conference (San Diego, Calif.). ACM/ IEEE, New York, 237-241.

[160]

NIELSEN, N. R. 1986. Expert systems and simulation. In Computer Networks and Simulation III, S. Schoemaker, Ed., North-Holland, Amsterdam, 41-52.

[161]

ODONI, A. R., AND ROTH, E. 1983. An empirical investigation of the transient behaviour of stationary queueing systems. Oper. Res., 34, 306-314.

[162]

O'KEEFE, R. M. 1986. Simulation and expert systems: A taxonomy and some examples. Simulation, 48, 10-16.

[163]

OREN, T. I., AND ZEIGLER, B. P. 1987. Artificial intelligence in modelling and simulation: Directions to explore. Simulation, 49, 131-134.

[164]

PAREKH, S., AND WALRAND, J. 1989. A quick simulation of excessive backlogs in networks of queues. IEEE Trans. Automatic Control, AC-34, 54-66.

[165]

PARK, S. K., AND MILLER, K. W. 1988. Random number generators: Good ones are hard to find. Commun. ACM, 31, 1192-1201.

[166]

PARZEN, E. 1961. Mathematical consideration in the estimation of spectra. Technometrics, 3, 167-190.

[167]

PAWLIKOWSKI, K., AND ASGARKHANI, M. 1988. Sequential procedures in simulation studies of satellite protocols. In Proceedings of the ITC'12 (Torino, Italy), Int. Advisory Council of ITC vol. 6, 4.3B.3.1-4.3B.3.7. (Also in Teletraffic Science for New Cost-Effective Systems, Systems and Services. Vol. 12, Studies in Telecommunication. North-Holland, Amsterdam, 1989.)

[168]

PAYNE, J. A. 1982. Introduction to Simulation. McGraw-Hill, New York.

[169]

PIDD, M. 1984. Computer Simulation in Management Science. Wiley, Chichester.

[170]

POTIER, D. 1984. New user's introduction to QNAP2. Rapport Technique INRIA, No. 40.

[171]

POTIER, D. 1986. The modeling package QNAP2 and applications to computer networks simulation. In Computer Networks and Simulation IiI. North- Holland, Amsterdam, 235-265.

[172]

PRESS, W. H., FLANNERY, B. P., TENKOLSKY, S. A. AND VATTERLING, W. T. 1986. Numerical Recipes. The Art of Scientific Computing. Cambridge University Press, Cambridge, U.K.

[173]

RAATIKAINEN, K. E. E. 1987. Simultaneous estimation of several percentiles. Simulation, 49, 159-164.

[174]

RAATIKAINEN, K. E. E. 1988. Validating percentiles of response times in queueing network models. In Proceedings of 1988 Summer Simulation Conference (Seattle, Wash.). Society for Computer Simulation, San Diego, Calif., 136-141.

[175]

REDDY, R. 1987. Epistemology of knowledge based simulation. Simulation, 48, 162-166.

[176]

REIMAN, M. I., AND WEISS, A. 1986. Sensitivity analysis via likelihood ratios. In Proceedings of the 1986 Winter Simulation Conference (Washington, D.C.). ACM/IEEE, New York, 285-289.

[177]

REYNOLDS, P. F. 1988. A spectrum of options for parallel simulation. In Proceedings of the 1988 Winter Simulation Conference (San Diego, Calif.). ACM/IEEE, New York, 325-332.

[178]

ROTH, E. 1985. A relaxation time heuristic for reducing initialization bias in simulation of M/Ek/1 queueing systems. In Proceedings of the 1985 Winter Simulation Conference (San Francisco, Calif.). ACM/IEEE, New York, 263-268.

[179]

ROTH, E., AND RUTAN, A. H. 1985. A relaxation time approach for reducing initialization bias in simulation. In Proceedings, Annual Simulation Symp. (Tampa, Fla.). IEEE Computer Society, New York, 189-203.

[180]

RUBINSTEIN, R. Y. 1989. Sensitivity analysis and performance extrapolation for computer simulation models. Oper. Res., 37, 72-81.

[181]

RUBINSTEIN, R. Y., AND MARCUS, R. 1985. Efficiency of multivariate control variates in Monte Carlo simulation. Oper. Res., 33, 661-677.

[182]

RUIz-MIER, S., AND TALAVAGE, J. 1987. A hybrid paradigm for modeling of complex systems. Simulation, 49, 135-141.

[183]

SARGENT, R. G. 1986. The use of graphical models in model validation. In Proceedings of the 1986 Winter Simulation Conference (Washington, D.C.). ACM/IEEE, New York, 237-241.

[184]

SAVER, C. H., MACNAm, E. A., AND KUROSE, J. K. 1984. Queueing network simulations of computer communications. IEEE J. Select. Areas Commun., SAC-l, 203-220.

[185]

SCHMEISER, B. 1982. Batch size effects in the analysis of simulation output. Oper. Res., 30, 556-568.

[186]

SCHMEISER, B., AND SONG, W. T. 1987. Correlation among estimators of the variance of the sample mean. In Proceedings of the 1987 Winter Simulation Conference (Atlanta, Ga.). ACM/IEEE, New York, 309-317.

[187]

SCHMIDT, J. W., AND HO, C. 1988. An algorithm for testing dependence of simulation output data. In Proceedings of the 1988 Winter Simulation Conference (San Diego, Calif.). ACM/IEEE, New York, 532-539.

[188]

SCHRIBER, T. 1974. Simulation Using GPSS. Wiley, New York.

[189]

SCHRIBER, W. J., AND ANDREWS, R. W. 1979. Interactive analysis of simulation output by the method of batch means. In Proceedings o/the 1979 Winter Simulation Conference (San Diego, Calif.). ACM/IEEE, New York, 512-524.

[190]

SCHRIBER, T. J., AND ANDREWS, R. W. 1981. A conceptual framework for research in the analysis of simulation output. Commun. A CM, 24, 218-232.

[191]

SCHRUBEN, L. W. 1980. A coverage function for interval estimators of simulation response. Manage. Sci., 26, 18-27.

[192]

SCHRUBEN, L. W. 1981. Control of initialization bias in multivariate simulation response. Commun. A CM, 24, 246-252.

[193]

SCHRUBEN, L. W. 1982. Detecting initialisation bias in simulation output. Oper. Res. 569-590.

[194]

SCHRUB~N, L. W. 1983. Confidence interval estimation using standardised time series. Oper. Res., 30, 1090-1108.

[195]

SCHRUBEN, L. W. 1985. Overview of standardized time series. In Proceedings of the 1985 Winter Simulation Conference (San Francisco, Calif.). ACM/IEEE, New York, 115-118.

[196]

SCHRUBEN, L. W. 1987. Using simulation to solve problems: A tutorial on the analysis of simulation output. In Proceedings of the 1987 Winter Simulation Conference (Atlanta, Ga.). ACM/IEEE, New York, 40-42.

[197]

SCHRUBEN, L. W., SINGH, H., AND TIERNEY, L. 1983. Optimal tests for initialisation bias in simulation output. Oper. Res., 31, 1167-1178.

[198]

SEILA, A. F. 1982. A batching approach to quantile estimation in regenerative simulation. Manage. Sci., 28, 573-581.

[199]

SEILA, A. F. 1982. Multivariate estimation in regenerative simulation. Oper. Res. Lett., 1,153-156.

[200]

SEILA, A. F. 1983. Multivariate estimation in simulation. In Proceedings of the 1983 Winter Simulation Conference (Arlington, Va.). ACM/IEEE, New York, 693-696.

[201]

SHAHABUDDIN, P., ET AL. 1988. Variance reduction in mean time to failure simulation. In Proceedings of the 1988 Winter Simulation Conference (San Diego, Cali~). ACM/IEEE, New York, 491-499.

[202]

SHANNON, R. E. 1981. Test for the verification and validation of computer simulation models. In Proceedings of the 1981 Winter Simulation Conference (Atlanta, Ga.). ACM/IEEE, New York, 573-577.

[203]

SHANTHIKUMAR, J., AND SARGENT, R. 1983. A unifying view of hybrid simulation/analytic models and modeling. Oper. Res., 31, 1030-1052.

[204]

SHAPIRO, S. S., AND WILK, M. B. 1965. An analysis of variance test for normality (complete) samples. Biometrika, 52, 591-611.

[205]

SHEDLER, G. S. 1987. Regeneration and Networks of Queues. Springer-Verlag, New York.

[206]

SOLOMON, S. L. 1983. Simulation of Waiting-Line Systems. Prentice-Hall, Englewood Cliffs, N.J.

[207]

SONG, W. T. 1988. Estimators of the Variance of the Simple Mean: Quadratic Forms, Optimal Batch Sizes, and Linear Combinations. Ph.D. Dissertation, School of Industrial Engineering, Purdue University, West Lafayette, Indiana, 1988.

[208]

SONG, W. T., AND SCHMEISER, B. 1988. Minimal MSE linear combinations of variance estimators of the sample mean. In Proceedings o/the 1988 Winter Simulation Conference (San Diego, Calif.). ACM/IEEE, New York, 414-421.

[209]

STAIRMAND, M. C., AND KREUTZER, W. 1988. POSE: A process-oriented simulation environment embedded in scheme. Simulation, 50, 143-153.

[210]

SURf, R., AND ZAZANIS, M. A. 1988. Perturbation analysis gives strongly consistent sensitivity estimates for the M/G/1 queue. Manage. Sci., 35, 39-64.

[211]

TOCHER, K. D. 1963. The Art of Simulation. The English Universities Press Ltd., London.

[212]

TRIVEDI, K. S. 1982. Probability and Statistics with Reliability, Queueing, and Computer Science Applications. Prentice-Hall, Englewood Cliffs, N.J.

[213]

TURNQUIST, M. A., AND SUSSMAN, J. M. 1977. Toward guidelines for designing experiments in queuing simulation. Simulation, 28, 137-144.

[214]

UNGER, B. W. 1988. Distributed simulation. In Proceedings of the 1988 Winter Simulation Conference (San Diego, Calif.). ACM/IEEE, New York, 198-205.

[215]

VASSILACOPOULOS, G. 1989. Testing for initialization bias in simulation output. Simulation, 52, 151-153.

[216]

VELAYAS, J. M., AND LEVARY, R. R. 1987. Validation of simulation models using decision theory. Simulation, 48, 87-92.

[217]

VENKATRAMAN, S., AND WILSON, J. R. 1985. Using path control variates in activity network simulation. In Proceedings of the 1985 Winter Simulation Conference (San Francisco, Calif.). ACM/ IEEE, New York, 217-222.

[218]

VENKATRAMAN, S., AND WILSON, J. R. 1986. The efficiency of control variates in multiresponse simulation. Oper. Res. Lett. 5, 37-42.

[219]

YON NEUMANN, J. 1941. Distribution of the ratio of the mean square successive difference to the variance. Ann. Math. Star., 12, 367-395.

[220]

WAGNER, D. B., AND LAZOWSKA, E. D. 1989. Parallel simulation of queueing networks: Limitations and potentials. Perform. Eval. Rev., 17, 146-155.

[221]

WAHBA, G. 1980. Automatic smoothing of the log periodogram, j. Am. Star. Assoc., 57, 122-132.

[222]

WALRAND, J. 1988. Computer Performance and Reliability: Proceedings of the 2nd int. NCPR Workshop (Rome). North-Holland, Amsterdam, 275-286.

[223]

WELCH, P. D. 1983. Statistical Analysis of Simulation Results. In Computer Performance Modelling Handbook, S. S. Lavenberg, Ed. Academic Press, New York, Chap. 6.

[224]

WELCH, P. D. 1987. On the relationship between batch means, overlapping batch means and spectral estimation. In Proceedings of the 1987 Winter Simulation Conference (Atlanta, Ga.). ACM/ IEEE, New York, 320-323.

[225]

WILSON, J. R. 1984. Variance reduction techniques for digital simulation. Am. J. Math. Manage. Sci., 4, 277-312.

[226]

WILSON, J. R. 1983. Variance reduction: The current state. Math. Comput. Simul., 26, 55-59.

[227]

WILSON, J. R., AND PRITSKER, A. A. B. 1978a. A survey of research on the simulation startup problem. Simulation, 31, 55-58.

[228]

WILSON, J. R., AND PRITSKER, A. A. B. 1978b. Evaluation of startup policies in simulation experiments. Simulation, 31, 79-89.

[229]

YANG, W., AND NELSON, B. L. 1988. Multivariate estimation and variance reduction in terminating and steady state simulation. In Proceedings of the I988 Winter Simulation Conference (San Diego, Calif.). ACM/IEEE, New York, 466-472.

[230]

ZEIGLER, B. P. 1987. Hierarchical, modular discreteevent modelling in an object-oriented environment. Simulation, 49, 219-230.

Cited By

Pourbafrani MLücking NLucke Mvan der Aalst W(2023)Steady State Estimation for Business Process SimulationsBusiness Process Management Forum10.1007/978-3-031-41623-1_11(178-195)Online publication date: 1-Sep-2023
https://doi.org/10.1007/978-3-031-41623-1_11
Morato DPerez-Gomara CMagana EIzal M(2022)Network Simulation in a TCP-Enabled Industrial Internet of Things Environment-Reproducibility Issues for Performance EvaluationIEEE Transactions on Industrial Informatics10.1109/TII.2021.308412818:2(807-815)Online publication date: Feb-2022
https://doi.org/10.1109/TII.2021.3084128
Muñoz D(2022)Empirical evaluation of initial transient deletion rules for the steady-state mean estimation problemComputational Statistics10.1007/s00180-022-01243-2Online publication date: 11-Jul-2022
https://doi.org/10.1007/s00180-022-01243-2
Show More Cited By

Recommendations

How Hard are Steady-State Queueing Simulations?
Special Issue on Don Iglehart

Some queueing systems require tremendously long simulation runlengths to obtain accurate estimators of certain steady-state performance measures when the servers are heavily utilized. However, this is not uniformly the case. We analyze a number of ...
Some Queuing Problems with Balking and Reneging---II

This is the second of two papers in which balking refusing to join the queue and reneging leaving the queue after joining are considered. The new element here is that the balking behavior is drastically altered The model assumes 1 Customers arrive from ...
State-dependent M/G/1 queueing systems

We consider a state-dependent $$M_{n}$$Mn/$$G_{n}$$Gn/1 queueing system with both finite and infinite buffer sizes. We allow the arrival rate of customers to depend on the number of people in the system. Service times are also state dependent and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 22, Issue 2

June 1990

87 pages

ISSN:0360-0300

EISSN:1557-7341

DOI:10.1145/78919

Issue’s Table of Contents

Copyright © 1990 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 June 1990

Published in CSUR Volume 22, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

182
Total Citations
View Citations
3,287
Total Downloads

Downloads (Last 12 months)210
Downloads (Last 6 weeks)16

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pourbafrani MLücking NLucke Mvan der Aalst W(2023)Steady State Estimation for Business Process SimulationsBusiness Process Management Forum10.1007/978-3-031-41623-1_11(178-195)Online publication date: 1-Sep-2023
https://doi.org/10.1007/978-3-031-41623-1_11
Morato DPerez-Gomara CMagana EIzal M(2022)Network Simulation in a TCP-Enabled Industrial Internet of Things Environment-Reproducibility Issues for Performance EvaluationIEEE Transactions on Industrial Informatics10.1109/TII.2021.308412818:2(807-815)Online publication date: Feb-2022
https://doi.org/10.1109/TII.2021.3084128
Muñoz D(2022)Empirical evaluation of initial transient deletion rules for the steady-state mean estimation problemComputational Statistics10.1007/s00180-022-01243-2Online publication date: 11-Jul-2022
https://doi.org/10.1007/s00180-022-01243-2
Dobler MSchumacher JBusel PHartmann C(2020)Supporting SMEs in the Lake Constance Region in the Implementation of Cyber-Physical-Systems: Framework and Demonstrator2020 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC)10.1109/ICE/ITMC49519.2020.9198430(1-8)Online publication date: Jun-2020
https://doi.org/10.1109/ICE/ITMC49519.2020.9198430
Viswanath N(2020)Transient study of Markov models with time-dependent transition ratesOperational Research10.1007/s12351-020-00613-222:3(2209-2243)Online publication date: 6-Nov-2020
https://doi.org/10.1007/s12351-020-00613-2
Domżał JWójcik RJajszczyk ADomżał JWójcik RJajszczyk A(2020)Flow-Aware NetworkingGuide to Flow-Aware Networking10.1007/978-3-030-57153-5_2(53-82)Online publication date: 27-Oct-2020
https://doi.org/10.1007/978-3-030-57153-5_2
Zimmermann AHotz T(2019)Integrating Simulation and Numerical Analysis in the Evaluation of Generalized Stochastic Petri NetsACM Transactions on Modeling and Computer Simulation10.1145/332151829:4(1-25)Online publication date: 18-Nov-2019
https://dl.acm.org/doi/10.1145/3321518
Cao PWang JZhang C(2019)Steady-State Interval Detection and Nonlinear Modeling for Automatic Generation Control SystemsIEEE Access10.1109/ACCESS.2019.29431667(139592-139600)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2943166
Wu JXu HJu FTseng T(2019)Adaptive Minimal Confidence Region Rule for Multivariate Initialization Bias Truncation in Discrete-Event SimulationsTechnometrics10.1080/00401706.2019.1665590(1-14)Online publication date: 15-Oct-2019
https://doi.org/10.1080/00401706.2019.1665590
Zhang NYuan JHui Ng S(2019)A data-driven online truncation method for transient bias reduction in steady-state simulationsComputers & Industrial Engineering10.1016/j.cie.2019.06.034Online publication date: Jun-2019
https://doi.org/10.1016/j.cie.2019.06.034
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents