Article

Free access

A novel application development environment for large-scale scientific computations

Authors:

G. Thiruvathukal,

A. SinghAuthors Info & Claims

ICS '00: Proceedings of the 14th international conference on Supercomputing

Pages 274 - 283

https://doi.org/10.1145/335231.335258

Published: 08 May 2000 Publication History

Abstract

Our results demonstrate that our novel application development environment provides both ease-of-use and high performance for large-scale, I/O-intensive scientific applications.

References

[1]

C. Banl, R. Frost, J. Lopez, R. Marciano, R. Moore, A. Rajasekar, and M. Wan. Meta-data design for a massive data analysis system. In Proc. CASCON'96 Conference, 1996.]]

[2]

C. Baru, R. Moore, A. Rajasekat, and M. Wan. The sdsc storage resource broker. In Proe. CASCON'98 Conference, Dec 1998, Toronto, Canada, 1998.]]

Digital Library

[3]

R. Bennett, K. Bryant, A. Sussman, R. Das, and J. S. Jovian. A framework for optimizing parallel i/o. In Proc. of the 1994 Scalable Parallel Libraries Conference, 1994.]]

[4]

R. Bordawekar, A. Choudhary, K. Kennedy, C. Koelbel, and M. Paleczny. A model and compilation strategy for out-of-core data parallel programs. In Proceedings of the ACM Symposium on Principles and Practice of Parallel Programming, pages 1-10, 1995.]]

Digital Library

[5]

P. Brown, R. Troy, D. Fisher, S. Louis, J. R. McGraw, and R. Musick. Meta-data sharing for balanced performance. In Proc. the First IEEE Meta-data Conference, Silver Spring. Maryland, 1996.]]

[6]

P. Cao, E. Felten, and K. Li. Application-controlled file caching policies. In Proc. the 1994 Summer USENIX Technical Conference, pages 171-182, 1994.]]

Digital Library

[7]

A. Choudhary, R. Bordawekar, M. Harry, R. Krishnalyes, R. Poanusamy, T. Singh, and R. Thaicur. Passion: parallel and scalable software for input-output. In NPAC Technical Report 5CC8-636, 1994.]]

[8]

P. Corbott, D. Feitelson, J.-P. Prost, G. Almasi, S. J. Baylor, A. Bolmarcich, Y. Hsu, J. Satran, M. Snir, R. Colao, B, Herr, L Kavaky, T. Morgan, and A. Zlotek. Parallel file systems for the ibm sp computers. IBM Systems Journal, 34(2):222-248, Jannry 1995.]]

Digital Library

[9]

P. Corbott, D. Fietelson, S. Fineberg, Y. Hsu, B. Nitzberg, J. Prost, M. Snir, B. Tmversat, and P. Wong. Overview of the mpi-io parallel i/o interface. In Proc. Third Workshop on I/0 in Parallel and Distributed Systems, IPPS'95, Santa Barbara, CA, 1995.]]

[10]

P. F. Corbott, D. G. Feitelson, L-P. Prost, and S. J. Baylor. Parallel access to files in the vesta file system. In Prac. Supercomputing'93, pages 472-481, 1993.]]

Digital Library

[11]

T. H. Cormen and D. M. Nieol. Out-of-core fits with parallel disks. In ACM SIGMETRICS Performance Evaluation Review, pages 3-12, 1997.]]

Digital Library

[12]

R.A. Coyne and H. Hulen. An introduction to the mass storage system reference model. In Proc. 12th IEEE Symposium on Mass STorage Systems, Monterey CA, 1993.]]

[13]

R.A. Coyne, H. Hulen, and R. Watson. The high performance storage system. In Proc. Supercomputing 93, Portland, OR, 1993.]]

Digital Library

[14]

J.R. Davis. Datalinks: Managing external data with db2 universal database. In IBM Corporation White Paper, 1997.]]

[15]

J. del Rosario, R. Bordawekar, and A. Clioudhary. Improved parallel i/o via a two-phase ran-time access strategy. In Proc. the 1993 IPPS Workshop on Input/Output in Parallel Computer Systems, 1993.]]

[16]

J. del Rosario and A. Choudhary. High performance i/o for parallel computers: problems and prospects. IEEE Computer, March 1994.]]

Digital Library

[17]

M. Drewry, H. Conover, S. McCoy, and S. Graves. Meta-data: quality vs. quantity. In Proc. the Second IEEE Meta-data Conference, 1997.]]

[18]

C.S. Ellis and D. Kotz. Ptefetchiog in file systems for mired multiprocessors. In Proc. the 1989 International Conference on Parallel Processing, pages 306-314, 1989.]]

[19]

M. Kandaswamy, M. Kandemir, A. Choudhary, and D. Bemholdt. Performance implications of architectural and software techniques on fro-intensive applications. In Proc. the International Conference on Parallel Processing, 1998.]]

Digital Library

[20]

J. E Karpovich, A. S. Grimshaw, and J. C. French. Extensible file systems (elfs): An object*oriented approach to high performance file i/o. In Proc. the Ninth Annual Conference on ObjecuOriented Programming Systems, Languages, and Applications, pages 191-204, 1994.]]

Digital Library

[21]

D. Kotz. Multiprocessorfilesystentintarfaces. InProc. theSecondlnternational Conference on Parallel and Distributed Information Systems, pages 194-201, 1993,]]

Digital Library

[22]

D. Kotz. Disk-directed i/o for mired multiprocessors. In Proc. the 1994 Symposium on Operating Systems Design and Implementation, pages 61-74, 1994.]]

Digital Library

[23]

T. Madhyastha and D. R. Intelligent. adaptive file system policy selection. In Proc. Frontiers of Massively Parallel Computing, pages 172-179, 1996.]]

Digital Library

[24]

Meat. In http://www.npaci, edu/DICE/SRB/mcat.html.]]

[25]

G. Memik, M. Kandemir, A. Choudhary, and V. E. Taylor. April: A run-time library for tape resident data. In the 17th IEEE Symposium oh Mass Storage Systems, 2000.]]

[26]

T. C. Mowry, A. K. Dernke, and O. Krieger. Automatic compiler-inserted i/o prefetching for out-of.core applications. In Proc. the Second Symposium on Operating Systems Design and Implementation, pages 3-17, i 996.]]

Digital Library

[27]

1. Newton. Application of raeta-data standards. In First IEEE Meta-data Conference, 1996.]]

[28]

R. H. Patterson, G. A. Gibson, and M. Satyanarayanan. A status report on research in Transparent informed prefetcning. In ACM Operating Systems Review, pages 21-34, 1993.]]

Digital Library

[29]

B. Runman. Paragon parallel file system. In External Product Specification, Intel Supercomputer Systems Division.]]

[30]

S. Sarawagi. Query processing in tertiary memory databases. In Proc. the 21st VLDB Conference, 1995.]]

Digital Library

[31]

M. Stonebraker. Object-Relational DBMSs : Tracking the Next Great Wave. Morgan Kanfman Publishers, ISBN: 1558604529, 1998.]]

Digital Library

[32]

M. Stonebraker and k A. Rowe. The design of postgres. In Proc. the ACM SIGMOD'86 International Conference on Management of Data, pages 340-355, 1986.]]

Digital Library

[33]

R. Thakur, W. Gropp, and E. LusL On implementing MPl-lO portably and with high performance. Preprint ANL/MCS-P'732-1098, Mathematics and Computer Science Division, Argonne National Laboratory, 1998.]]

[34]

R. Thakur, W. Gropp, and E. Lusk. Data sieving and collective i/o in romio. In Proc. the 7th Symposium on the Frontiers of Massively Parallel Computation, 1999.]]

Digital Library

[35]

S. Toledo and E G. Gnstavson. The design and implementation of solar, a portable library for scalable out-of-cere linear algebra computations. In Proc. Fourth Annual Workshop on 1/0 in Parallel and Distributed Systems, 1996.]]

Digital Library

[36]

UmTree User Guide, Release 2.0. UniTree Software, Inc., 1998.]]

Cited By

Memik GKandemir MLiao WChoudhary A(2006)Multicollective I/OACM Transactions on Storage10.1145/1168910.11689152:3(349-369)Online publication date: 1-Aug-2006
https://dl.acm.org/doi/10.1145/1168910.1168915
(2004)Optimized Management of Large-Scale Data Sets Stored on Tertiary Storage SystemsIEEE Distributed Systems Online10.1109/MDSO.2004.55:5(1)Online publication date: 1-May-2004
https://dl.acm.org/doi/10.1109/MDSO.2004.5
Shen XChoudhary A(2003)A distributed multi-storage I/O system for data intensive scientific computingParallel Computing10.1016/j.parco.2003.05.00929:11-12(1623-1643)Online publication date: 1-Nov-2003
https://dl.acm.org/doi/10.1016/j.parco.2003.05.009
Show More Cited By

Index Terms

Recommendations

Efficient Aggregate Computations in Large-Scale Dense WSN
RTAS '09: Proceedings of the 2009 15th IEEE Symposium on Real-Time and Embedded Technology and Applications

We focus on large-scale and dense deeply embedded systems where, due to the large amount of information generated by all nodes, even simple aggregate computations such as the minimum value (MIN) of the sensor readings become notoriously expensive to ...
Exploring Cloud Computing for Large-Scale Scientific Applications
SERVICES '13: Proceedings of the 2013 IEEE Ninth World Congress on Services

This paper explores cloud computing for large-scale data intensive scientific applications. Cloud computing is attractive because it provides hardware and software resources on-demand, which relieves the burden of acquiring and maintaining a huge amount ...
Acknowledgments in scientific publications: Presence in Spanish science and text patterns across disciplines

The acknowledgments in scientific publications are an important feature in the scholarly communication process. This research analyzes funding acknowledgment presence in scientific publications and introduces a novel approach for discovering text ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICS '00: Proceedings of the 14th international conference on Supercomputing

May 2000

347 pages

ISBN:1581132700

DOI:10.1145/335231

Chairmen:
John Reynders
Los Alamos National Lab, Los Alamos, NM
,
Alex Veidenbaum
Univ. of California at Irvine, Irvine

Copyright © 2000 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGARCH: ACM Special Interest Group on Computer Architecture

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 May 2000

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

ICS00

Sponsor:

SIGARCH

ICS00: International Conference on Supercomputing

May 8 - 11, 2000

New Mexico, Santa Fe, USA

Acceptance Rates

ICS '00 Paper Acceptance Rate 33 of 122 submissions, 27%;

Overall Acceptance Rate 629 of 2,180 submissions, 29%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
351
Total Downloads

Downloads (Last 12 months)42
Downloads (Last 6 weeks)4

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Memik GKandemir MLiao WChoudhary A(2006)Multicollective I/OACM Transactions on Storage10.1145/1168910.11689152:3(349-369)Online publication date: 1-Aug-2006
https://dl.acm.org/doi/10.1145/1168910.1168915
(2004)Optimized Management of Large-Scale Data Sets Stored on Tertiary Storage SystemsIEEE Distributed Systems Online10.1109/MDSO.2004.55:5(1)Online publication date: 1-May-2004
https://dl.acm.org/doi/10.1109/MDSO.2004.5
Shen XChoudhary A(2003)A distributed multi-storage I/O system for data intensive scientific computingParallel Computing10.1016/j.parco.2003.05.00929:11-12(1623-1643)Online publication date: 1-Nov-2003
https://dl.acm.org/doi/10.1016/j.parco.2003.05.009
Memik GKandemir MChoudhary A(2002)Exploiting inter-file access patterns using multi-collective I/OProceedings of the 1st USENIX conference on File and storage technologies10.5555/1973333.1973350(17-17)Online publication date: 28-Jan-2002
https://dl.acm.org/doi/10.5555/1973333.1973350
Memik GKandemir MChoudhary A(2002)Exploiting Inter-File Access Patterns Using Multi-Collective I/OProceedings of the 1st USENIX Conference on File and Storage Technologies10.5555/1083323.1083350(20-es)Online publication date: 30-Jan-2002
https://dl.acm.org/doi/10.5555/1083323.1083350
Xiaohui Shen Choudhary A(2002)MS-I/O: A Distributed Multi-Storage I/O System2nd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID'02)10.1109/CCGRID.2002.1017124(163-163)Online publication date: 2002
https://doi.org/10.1109/CCGRID.2002.1017124
Xiaohui Shen Choudhary A(2001)DPFS: a distributed parallel file systemInternational Conference on Parallel Processing, 2001.10.1109/ICPP.2001.952101(533-541)Online publication date: 2001
https://doi.org/10.1109/ICPP.2001.952101
Liao WShen XChoudhary A(2001)Meta-data Management System for High-Performance Large-Scale Scientific Data AccessHigh Performance Computing — HiPC 200010.1007/3-540-44467-X_26(293-300)Online publication date: 8-Jun-2001
https://doi.org/10.1007/3-540-44467-X_26
Shen XChoudhary A(2000)A distributed multi-storage resource architecture and I/O performance prediction for scientific computingProceedings the Ninth International Symposium on High-Performance Distributed Computing10.1109/HPDC.2000.868631(21-30)Online publication date: 2000
https://doi.org/10.1109/HPDC.2000.868631
Shen XThiruvathukal GLiao WChoudhary ASingh A(2000)A Java graphical user interface for large-scale scientific computations in distributed systemsProceedings Fourth International Conference/Exhibition on High Performance Computing in the Asia-Pacific Region10.1109/HPC.2000.846602(478-484 vol.1)Online publication date: 2000
https://doi.org/10.1109/HPC.2000.846602

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents