Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

COMPASSION: A parallel I/O runtime system including chunking and compression for irregular applications

  • 3. Computer Science
  • Conference paper
  • First Online:
High-Performance Computing and Networking (HPCN-Europe 1998)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 1401))

Included in the following conference series:

Abstract

In this paper we present two designs, namely, “Collective I/O” and “Pipelined Collective I/O”, of a runtime library for irregular applications based on the two-phase collective 1/O technique. We also present the optimization of both models by using chunking and compression mechanisms. In the first scheme, all processors participate in compressions and I/O at the same time, making scheduling of I/O requests simpler but creating a possibility of contention at the I/O nodes. In the second approach, processors are grouped into several groups, overlapping communication, compression, and I/O to reduce I/O contention dynamically. Finally, evaluation results are shown that demonstrates that we can obtain significantly high-performance for I/O above what has been possible so far.

This work was supported in part by Sandia National Labs award AV-6193 under the ASCI program, and in part by NSF Young Investigator Award CCR-9357840 and NSF CCR-9509143. Jesus Carretero is a postdoctoral fellow at NWU supported by the NATO Science Fellowships Programme.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alok Choudhary, Rajesh Bordawekar, Michael Harry, Rakesh Krishnaiyer, Ravi Ponnusamy, Tarvinder Singh, and Rajeev Thakur. PASSION: parallel and scalable software for input-output. Technical Report SCCS-636, ECE Dept., NPAC and CASE Center, Syracuse University, September 1994.

    Google Scholar 

  2. T. Mattson and G. Henry. The asci option red supercomputer. In Intel Supercomputer Users Group. Thirteenth Annual Conference, Albuquerque, USA, June 1997.

    Google Scholar 

  3. J. No and A. Choudhary. Techniques to provide run-time support for solving irregular problems. In Proceedings of the International Conference on Parallel and Distributed Systems, Seoul, Korea, 1997.

    Google Scholar 

  4. R. Ponnusamy, J. Saltz, A. Choudhary, Y.-S. Hwang, and G. Fox. Runtime-compilation techniques for data partitioning and communication schedule reuse. In Proc. of Supercomputing'93, Portland, OR., November 1993.

    Google Scholar 

  5. K. E. Seamons and M. Winslett. A data management approach for handling large compressed arrays in high performance computing. In Proceedings of the Fifth Symposium on the Frontiers of Massively Parallel Computation, pages 119–128, February 1995.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Arquitectura y Tecnología de Sistemas Informáticos, Universidad Politécnica de Madrid, Spain

    Jesús Carretero

  2. Dept. of Electrical Engineering and Computer Science, Syracuse University, USA

    Jaechun No

  3. Electrical and Computer Engineering, Northwestern University, USA

    Sung-soon Park

  4. Computer Science Dept., Anyang University, USA

    Alok Choudhary

  5. Sandia National Laboratories, USA

    Pang Chen

Authors
  1. Jesús Carretero
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jaechun No
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sung-soon Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alok Choudhary
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Pang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Peter Sloot Marian Bubak Bob Hertzberger

Rights and permissions

Reprints and permissions

Copyright information

© 1998 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Carretero, J., No, J., Park, Ss., Choudhary, A., Chen, P. (1998). COMPASSION: A parallel I/O runtime system including chunking and compression for irregular applications. In: Sloot, P., Bubak, M., Hertzberger, B. (eds) High-Performance Computing and Networking. HPCN-Europe 1998. Lecture Notes in Computer Science, vol 1401. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0037194

Download citation

  • DOI: https://doi.org/10.1007/BFb0037194

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-64443-9

  • Online ISBN: 978-3-540-69783-1

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation