Article

A Performance Prediction Framework for Data Intensive Applications on Large Scale Parallel Machines

Authors:

Tahsin M. Kurç,

Joel H. SaltzAuthors Info & Claims

LCR '98: Selected Papers from the 4th International Workshop on Languages, Compilers, and Run-Time Systems for Scalable Computers

Pages 243 - 258

Published: 28 May 1998 Publication History

Abstract

This paper presents a simulation-based performance prediction framework for large scale data-intensive applications on large scale machines. Our framework consists of two components: application emulators and a suite of simulators. Application emulators provide a parameterized model of data access and computation patterns of the applications and enable changing of critical application components (input data partitioning, data declustering, processing structure, etc.) easily and flexibly. Our suite of simulators model the I/O and communication subsystems with good accuracy and execute quickly on a high-performance workstation to allow performance prediction of large scale parallel machine configurations. The key to efficient simulation of very large scale configurations is a technique called loosely-coupled simulation where the processing structure of the application is embedded in the simulator, while preserving data dependencies and data distributions. We evaluate our performance prediction tool using a set of three data-intensive applications.

References

[1]

A. Acharya, M. Uysal, R. Bennett, A. Mendelson, M. Beynon, J. K. Hollingsworth, J. Saltz, and A. Sussman. Tuning the performance of I/O intensive parallel applications. In Proc. of IOPADS'96. ACM Press, May 1996.

[2]

R. Agrawal and J. Shafer. Parallel mining of association rules. IEEE Transactions on Knowledge and Data Engineering, 8(6):962-9, Dec. 1996.

[3]

R. Bagrodia, S. Docy, and A. Kahn. Parallel simulation of parallel file systems and I/O programs. In Proceedings of the 1997 ACM/IEEE SC97 Conference. ACM Press, Nov. 1997.

[4]

J. Brehm, M. Madhukar, E. Smirni, and L. Dowdy. PerPreT - a performance prediction tool for massively parallel systems. In Proceedings of the Joint Conference on Performance Tools / MMB 1995, pages 284-298. Springer-Verlag, Sept. 1995.

[5]

C. F. Cerco and T. Cole. User's guide to the CE-QUAL-ICM three-dimensional eutrophication model, release version 1.0. Technical Report EL-95-15, US Army Corps of Engineers Water Experiment Station, Vicksburg, MS, 1995.

[6]

C. Chang, B. Moon, A. Acharya, C. Shock, A. Sussman, and J. Saltz. Titan: a high-performance remote-sensing database. In Proceedings of the 13th International Conference on Data Engineering, Apr. 1997.

[7]

M. J. Clement and M. J. Quinn. Using analytical performance prediction for architectural scaling. Technical Report TR BYU-NCL-95-102, Networked Computing Lab, Brigham Young University, 1995.

[8]

R. Ferreira, B. Moon, J. Humphries, A. Sussman, J. Saltz, R. Miller, and A. Demarzo. The Virtual Microscope. In Proc. of the 1997 AMIA Annual Fall Symposium, pages 449-453. American Medical Informatics Association, Oct. 1997.

[9]

A. Gürsoy and L. V. Kalé. Simulating message-driven programs. In Proceedings of Int. Conference on Parallel Processing, volume III, pages 223-230, Aug. 1996.

[10]

S. Herrod. Tango lite: A multiprocessor simulation environment. Technical report, Computer Systems Laboratory, Stanford University, 1993.

[11]

C. L. Mendes. Performance prediction by trace transformation. In Fifth Brazilian Symposium on Computer Architecture, Sept. 1993.

[12]

B. Moon and J. H. Saltz. Scalability analysis of declustering methods for multidimensional range queries. IEEE Transactions on Knowledge and Data Engineering, 10(2):310-327, March/April 1998.

[13]

S. Reinhardt, M. Hill, J. Larus, A. Lebeck, J. Lewis, and D. Wood. The Wisconsin Wind Tunnel: Virtual prototyping of parallel computers. In Proc. of the 1993 ACM SIGMETRICS Conf. on Measuring and Modeling of Computer Systems, 1993.

[14]

M. Rosenblum, S. Herrod, E. Witchel, and A. Gupta. Complete computer system simulation: The SimOS approach. IEEE Parallel and Distributed Technology, 3(4):34-43, Winter 1995.

[15]

J. M. Schopf. Structural prediction models for high-performance distributed applications. In Cluster Computing Conference, 1997.

[16]

J. Simon and J.-M. Wierum. Accurate performance prediction for massively parallel systems and its applications. In Proceedings of Euro-Par'96, volume 1124 of LNCS, pages 675-688. Springer-Verlag, Aug. 1996.

[17]

Y. Yan, X. Zhang, and Y. Song. An effective and practical performance prediction model for parallel computing on non-dedicated heterogeneous NOW. Journal of Parallel and Distributed Computing, 38:63-80, Oct. 1996.

Cited By

Llorà XRothlauf F(2009)Data-intensive computing for competent genetic algorithmsProceedings of the 11th Annual conference on Genetic and evolutionary computation10.1145/1569901.1570087(1387-1394)Online publication date: 8-Jul-2009
https://dl.acm.org/doi/10.1145/1569901.1570087
Khanna GCatalyurek UKurc TSadayappan PSaltz J(2007)Scheduling file transfers for data-intensive jobs on heterogeneous clustersProceedings of the 13th international Euro-Par conference on Parallel Processing10.5555/2391541.2391568(214-223)Online publication date: 28-Aug-2007
https://dl.acm.org/doi/10.5555/2391541.2391568
Vydyanathan NKhanna GCatalyurek UKurc TSadayappan PSaltz J(2006)Scheduling of tasks with batch-shared I/O on heterogeneous systemsProceedings of the 20th international conference on Parallel and distributed processing10.5555/1898953.1899083(159-159)Online publication date: 25-Apr-2006
https://dl.acm.org/doi/10.5555/1898953.1899083
Show More Cited By

A Performance Prediction Framework for Data Intensive Applications on Large Scale Parallel Machines

Recommendations

Performance prediction of large parallel applications using parallel simulations
PPoPP '99: Proceedings of the seventh ACM SIGPLAN symposium on Principles and practice of parallel programming

Accurate simulation of large parallel applications can be facilitated with the use of direct execution and parallel discrete event simulation. This paper describes the use of COMPASS, a direct execution-driven, parallel simulator for performance ...
Performance prediction of large parallel applications using parallel simulations

Accurate simulation of large parallel applications can be facilitated with the use of direct execution and parallel discrete event simulation. This paper describes the use of COMPASS, a direct execution-driven, parallel simulator for performance ...
Simulation-based performance prediction for large parallel machines
Special issue: The next generation software program

We present a performance prediction environment for large scale computers such as the Blue Gene machine. It consists of a parallel simulator, BigSim, for predicting performance of machines with a very large number of processors, and BigNetSim, which ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

LCR '98: Selected Papers from the 4th International Workshop on Languages, Compilers, and Run-Time Systems for Scalable Computers

May 1998

409 pages

ISBN:3540651721

Editor:
David R. O'Hallaron

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 28 May 1998

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Llorà XRothlauf F(2009)Data-intensive computing for competent genetic algorithmsProceedings of the 11th Annual conference on Genetic and evolutionary computation10.1145/1569901.1570087(1387-1394)Online publication date: 8-Jul-2009
https://dl.acm.org/doi/10.1145/1569901.1570087
Khanna GCatalyurek UKurc TSadayappan PSaltz J(2007)Scheduling file transfers for data-intensive jobs on heterogeneous clustersProceedings of the 13th international Euro-Par conference on Parallel Processing10.5555/2391541.2391568(214-223)Online publication date: 28-Aug-2007
https://dl.acm.org/doi/10.5555/2391541.2391568
Vydyanathan NKhanna GCatalyurek UKurc TSadayappan PSaltz J(2006)Scheduling of tasks with batch-shared I/O on heterogeneous systemsProceedings of the 20th international conference on Parallel and distributed processing10.5555/1898953.1899083(159-159)Online publication date: 25-Apr-2006
https://dl.acm.org/doi/10.5555/1898953.1899083
Lee YHall M(2004)A code isolatorProceedings of the 17th international conference on Languages and Compilers for High Performance Computing10.1007/11532378_13(164-178)Online publication date: 22-Sep-2004
https://dl.acm.org/doi/10.1007/11532378_13
Kurc TLee FAgrawal GCatalyurek UFerreira RSaltz JMcGraw J(2003)Optimizing Reduction Computations In a Distributed EnvironmentProceedings of the 2003 ACM/IEEE conference on Supercomputing10.1145/1048935.1050160Online publication date: 15-Nov-2003
https://dl.acm.org/doi/10.1145/1048935.1050160
Yu SWinslett MLee JMa X(2002)Automatic and portable performance modeling for parallel I/OACM SIGMETRICS Performance Evaluation Review10.1145/605521.60552430:3(3-5)Online publication date: 1-Dec-2002
https://dl.acm.org/doi/10.1145/605521.605524
Faerman MSu AWolski RBerman FPancake C(1999)Adaptive performance prediction for distributed data-intensive applicationsProceedings of the 1999 ACM/IEEE conference on Supercomputing10.1145/331532.331568(36-es)Online publication date: 1-Jan-1999
https://dl.acm.org/doi/10.1145/331532.331568
Kurc TChang CFerreira RSussman ASaltz JPancake C(1999)Querying very large multi-dimensional datasets in ADRProceedings of the 1999 ACM/IEEE conference on Supercomputing10.1145/331532.331544(12-es)Online publication date: 1-Jan-1999
https://dl.acm.org/doi/10.1145/331532.331544

View Options

View options

Media

Figures

Other

Tables

View Table of Contents