Abstract
In this paper we present the architecture and framework for a benchmark suite that has been developed as part of the DeSiDeRaTa project. The proposed benchmark suite is representative of the emerging generation of distributed, mission-critical, real-time control systems that operate in dynamic environments. Systems that operate in such environments may have unknown worst-case scenarios, may have large variances in the sizes of the data and event sets that they process (and thus, have large variances in execution latencies and resource requirements), and may be very difficult to characterize statically, even by time-invariant statistical distributions. The proposed benchmark suite (called DynBench) is useful for evaluation of the Quality of Service (QoS) management and/or Resource Management (RM) services in distributed real-time systems. As such, DynBench includes a set of performance metrics for the evaluation of the QoS and RM technologies in dynamic distributed real-time systems. The paper demonstrates the successful application of DynBench in evaluation of the DeSiDeRaTa QoS management middle-ware.
Sponsored in part by DARPA/NCCOSC contract N66001-97-C-8250, and by NSWC/NCEE contracts NCEE/A303/41E-96 and NCEE/A303/50A-98.
Preview
Unable to display preview. Download preview PDF.
References
[CS+97] S. Chatterjee, J. Sydir, B. Sabata, and T. Lawrence, “Modeling Applications for Adaptive QoS-based Resource Management,” 2nd IEEE High-Assurance System Engineering Workshop (HASE97), Bethesda, Maryland, August 1997.
[Cur76] H.J. Curnow and B.A. Wichmann, “A synthetic benchmark,” Computer Journal 19(1), January 1976, 43–49.
[Dav80] C.G. Davis, “Ballistic missile defense: a supercomputer challenge,” IEEE Computer, 30, 1980, 37–46.
[DL97] Z. Deng and J. W.-S. Liu, “Scheduling real-time applications in an open environment,” Real-Time Systems Symposium, San Francisco, California, December 1997, pp. 308–319.
[Har94] Robert D. Harrison Jr., “Combat system prerequisites on supercomputer performance analysis,” in Proceedings of the NATO Advanced Study Institute on Real Time Computing, NATO ASI Series F(127), 512–513, Springer-Verlag 1994.
[Hon97] Honeywell Technology, “Adaptive Computing systems Benchmark,” http://www.htc.honeywell.com/projects/acsbench/.
[H+97] J. Huang, etal., “RT-ARM: A real-time adaptive resource management system for distributed mission-critical applications,” Workshop on Middleware for Distributed Real-Time Systems, RTSS-97.
[Kam91] N.I. Kamenoff and N.H. Weiderman, “Hartstone distributed benchmark: requirements and definitions,” in Proceedings of the 12 th IEEE Real-Time Systems Symposium, 1991, 199–208.
[Kar89] Kar, R.P. and Porter, K., “Rhealstone—A Real Time Benchmarking Proposal,” Dr. Dobbs Journal 14(2):4–24, February, 1989
[Kav92] K.M. Kavi and S.M. Yang, “Real-time systems design methodologies: an introduction and a survey,” The Journal of Systems and Software, April 1992, 85–99, Elsevier Science Publishers.
[Kis94] D.L. Kiskis and K.G. Shin, “SWSL: A synthetic workload specification language for real-time systems,” IEEE Transactions on Software Engineering, 20(10), Oct. 1994, 798–811.
[Kis96] D.L. Kiskis and K.G. Shin, “A synthetic workload for a distributed real-time system,” Journal of Real-Time Systems, 11(1), July 1996, 5–18.
[Koob98] G. Koob, “DARPA, Quorum mission statement,” http://www.darpa.mil/ito/research/quorum/index.html.
[Mast95] Michael W. Masters, “Real-time Computing Cornerstones: A System Engineer’s View,” Proceedings of the 3rd Workshop on Parallel and Distributed Real-time Systems, 1995
[Met96] R.C. Metzger, B. VanVoorst, L.S. Pires, R. Jha, W. Au, M. Amin,D.A. Castanon and V. Kumar, “C3I parallel benchmark suite: introduction and preliminary results,” in Supercomputing’96, 1996.
[Mil95] D.L. Mills, “Improved Algorithms for Synchronizing Computer Network Clocks,” IEEE Trans. on Networks, 3, June 1995.
[Mol90] J.J. Molini, S.K. Maimon and P.H. Watson, “Real-time system scenarios,” in Proceedings of The 11 th IEEE Real-Time Systems Symposium, 214–225, IEEE Computer Society Press, 1990.
[Pri89] W.J. Price, “A benchmark tutorial,” IEEE Micro, October 1989, 28–43.
[RLLS97] R. Rajkumar, C. Lee, J. Lehoczky, and D. Siewiorek, “A resource allocation model for qos management,” 18 th IEEE Real-Time Systems Symposium, pages 298–307, 1997.
[Ravin98a] B. Ravindran, L.R. Welch, and B. Shirazi, “A Resource Management Middleware for Dynamic, Dependable Real-Time Systems,” Tech Report, CSE Dept, UTA, Aug. 1998.
[Ravin98b] B. Ravindran, L.R. Welch, C. Bruggeman, B. Shirazi, C. Cavanaugh, “A Resource Management Model for Dynamic, Scalable, Dependable, Real-Time Systems,” Lecture Notes in Computer Science, #1388, Springer Verlag, March 1998, pp. 931–936.
[RSYJ97] D. Rosu, K. Schwan, S. Yalamanchili, and R. Jha, “On adaptive resource allocation for complex real-time applications,” In Proceedings of the 18 th IEEE Real-Time Systems Symposium, 1997, 320–329.
[Ujv97] B.G. Ujvary and N.I. Kamenoff, “Implementation of the Hartstone distributed benchmark for real-time distributed systems: results and conclusions,” in Proceedings of the 5 th International Workshop on Parallel and Distributed Real-Time Systems, 98–103, IEEE Computer Society Press, 1997.
[Wei84] R.P. Weicker, “Dhrystone: a synthetic systems programming benchmark,” Communications of the ACM, 27(10), 1984, 1013–1030.
[Wei89] N.H. Weiderman, “Hartstone: synthetic benchmark requirements for hard real-time applications,” CMU/SEI-89-TR-23, June 89.
[Wei90] R.P. Weicker, “An overview of common benchmarks,” IEEE Computer, 23(12), Dec. 1990, 65–75.
[Wei92] N.H. Weiderman and N.I. Kamenoff, “Hartstone uniprocessor benchmark: definitions and experiments for real-time systems,” Journal of Real-Time Systems, 4(4), Dec. 1992, 353–382.
[Wel96] L. R. Welch, Binoy Ravindran, Robert D. Harrison, Leslie Madden, Michael W. Masters and Wayne Mills, “Challenges in Engineering Distributed Shipboard Control Systems,” in Proceedings of Work-in-Progress Session of The IEEE Real-Time Systems Symposium, December 1996, 19–22.
[Welch97a] L.R. Welch and B. Shirazi, “DeSiDeRaTa: QoS Management Tools for Dynamic, Scalable, Dependable, Real-Time Systems,” IEEE Workshop on Middleware for Distributed Real-Time Systems and Services, Dec. 1997, pp. 164–178.
[Welch98a] L.R. Welch, B. Ravindran, B. Shirazi, C. Bruggeman, “Specification and Modeling of Dynamic, Distributed Real-time Systems,” Proceedings of the 19 th IEEE Real-Time Systems Symposium (RTSS’98), Dec. 1998, pp. 72–81.
[Welch98b] L.R. Welch, B. Shirazi, B. Ravindran, C. Bruggeman, “DeSiDeRaTa: QoS Management Technology for Dynamic, Scalable, Dependable, Real-time Systems,” 15th IFAC Workshop—Distributed Computer Control Systems (DCCS’98), Sept. 1998.
[Welch98c] L.R. Welch, et al., “Adaptive Resource Management for Scalable, Dependable Real-time Systems,” in the proceedings of the Work-In-Progress Session of the Fourth IEEE Real-Time Technology and Applications Symposium (RTAS’98), June 1998, pp. 3–6.
[Welch98d] L.R. Welch and B. Shirazi, “A Distributed Architecture for QoS Management of Dynamic, Scalable, Dependable, Real-Time Systems,” Workshop on Parallel and Distributed Real-Time Systems, April 1998, available on 1998 IPPS/SPDP CD-ROM.
[Welch98e] L.R. Welch, B. Shirazi, B. Ravindran, and F. Kamangar, “Instrumentation, Modeling, and Analysis of Dynamic, Distributed Real-time Systems,” Tech Rep, CSE Dept, UTA, 1998.
[ZBS97] John A. Zinky, David E. Bakken, and Richard Schantz, “Architectural Support for Quality of Service for CORBA Objects,” Theory and Practice of Object Systems, 1997.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1999 Springer-Verlag
About this paper
Cite this paper
Shirazi, B. et al. (1999). DynBench: A dynamic benchmark suite for distributed real-time systems. In: Rolim, J., et al. Parallel and Distributed Processing. IPPS 1999. Lecture Notes in Computer Science, vol 1586. Springer, Berlin, Heidelberg . https://doi.org/10.1007/BFb0098013
Download citation
DOI: https://doi.org/10.1007/BFb0098013
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-65831-3
Online ISBN: 978-3-540-48932-0
eBook Packages: Springer Book Archive