research-article

SAGA-based file access application over multi-filesystem middleware

Authors:

Yutaka Kawai,

Go Iwai,

Takashi Sasaki,

Yoshiyuki WataseAuthors Info & Claims

HPDC '10: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing

Pages 622 - 626

https://doi.org/10.1145/1851476.1851568

Published: 21 June 2010 Publication History

Get Access

Abstract

This paper describes practical file access applications based on SAGA -- A Simple API for Grid Applications-- for distributed storage resources over multi-file-system middleware. SAGA provides a high-level programming interface to bridge between network file-system middleware as well as accessing a local file-system. The Computing Research Center of KEK uses many data files from physics experiments. For example, large numbers of bubble chamber image files are shared between KEK and King's College in UK. Our research problem involves displaying a single bubble chamber image file that is divided and stored in the different kinds of file-systems. We use the two types of file-systems that are iRODS --The Integrated Rule-Oriented Data System-- and Gfarm --Grid Data Farm--. To access the middleware, we are developing the SAGA adaptors for the iRODS and Gfarm file-systems. We present the technical details for the user environments and show the usability with real bubble chamber image files.

References

[1]

}}Gfarm -- Grid Data Farm. Online. http://datafarm.apgrid.org/index.en.html.

Google Scholar

[2]

}}T. S. Go Iwai, Yutaka Kawai and Y. Watase. SAGA-based user environment for distributed computing resources: A universal Grid solution over multi-middleware infrastructures. In Accepted for International Conference on Computational Science, ICCS 2010, Amsterdam, Netherlands., 2010.

Google Scholar

[3]

}}T. Goodale et al. SAGA - v1.0 Specification (GFD-R-P.90). Technical report, SAGA-CORE-WG, 2008. http://www.ggf.org/documents/GFD.90.pdf.

Google Scholar

[4]

}}ImageMagick. Online. http://www.imagemagick.org/script/index.php.

Google Scholar

[5]

}}ImageMagick Program Interfaces. Online. http://www.imagemagick.org/script/api.php.

Google Scholar

[6]

}}iRODS -- the Integrated Rule-Oriented Data System. Online. http://www.irods.org.

Google Scholar

[7]

}}S. Jha, H. Kaiser, Y. El Khamra, and O. Weidner. Design and Implementation of Network Performance Aware Applications Using SAGA and Cactus. In Accepted for 3rd IEEE Conference on eScience2007 and Grid Computing, Bangalore, India., 2007. http://saga.cct.lsu.edu/publications/papers/confpapers/Design-and-Implementation-of-Network-Performance-Aware/sagacactusescience.pdf/view.

Digital Library

Google Scholar

[8]

}}S. Matsuoka, S. Shinjo, M. Aoyagi, S. Sekiguchi, H. Usami, and K. Miura. Japanese Computational Grid Research Project: NAREGI. Proceedings of the IEEE, 93(3):522--533, March 2005.

Crossref

Google Scholar

[9]

}}NAREGI -- NAtional REsearch Grid Initiative. Online. http://www.naregi.org/index e.html.

Google Scholar

[10]

}}OGF -- Open Grid Forum. Online. http://www.ogf.org/.

Google Scholar

[11]

}}RENKEI -- REsources liNKage for E-scIence. Online. http://www.e-sciren.org/index-e.html.

Google Scholar

[12]

}}RFC1630 -- Universal Resource Identifiers in WWW. Online. http://www.ietf.org/rfc/rfc1630.txt.

Google Scholar

[13]

}}The SAGA C++ Reference API. Online. http://saga.cct.lsu.edu/cpp/apidoc/.

Google Scholar

Cited By

View all

Nguyen BTran VHluchy L(2013)An Abstraction Model for High-Level Application Programming on the CloudAdvanced Computational Methods for Knowledge Engineering10.1007/978-3-319-00293-4_22(295-306)Online publication date: 2013
https://doi.org/10.1007/978-3-319-00293-4_22
Kawai YIwai GSasaki TWatase Y(2011)Managing Distributed Files with RNS in Heterogeneous Data Grids2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing10.1109/CCGrid.2011.19(494-503)Online publication date: May-2011
https://doi.org/10.1109/CCGrid.2011.19

Index Terms

SAGA-based file access application over multi-filesystem middleware
1. Information systems
  1. Information systems applications
2. Software and its engineering
  1. Software organization and properties
    1. Software system structures
      1. Abstraction, modeling and modularity

Recommendations

A distributed workflow management model for grid middleware
Best Papers from the GCC 2006 Conference

Workflow management enables grid service composition and user collaboration in grid middleware, it includes the workflow of grid service invoking and data exchanging among services. Because of the large requirements of grid service composition, it's ...
SAGA BigJob: An Extensible and Interoperable Pilot-Job Abstraction for Distributed Applications and Systems
CCGRID '10: Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing

The uptake of distributed infrastructures by scientific applications has been limited by the availability of extensible, pervasive and simple-to-use abstractions which are required at multiple levels -- development, deployment and execution stages of ...
Managing Distributed Files with RNS in Heterogeneous Data Grids
CCGRID '11: Proceedings of the 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

This paper describes the management of files distributed in heterogeneous Data Grids by using RNS (Resource Namespace Service). RNS provides hierarchical namespace management for name-to-resource mapping as a key technology to use Grid resources for ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

HPDC '10: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing

June 2010

911 pages

ISBN:9781605589428

DOI:10.1145/1851476

General Chairs:
Salim Hariri
University of Arizona
,
Kate Keahey
University of Chicago

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 June 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

HPDC '10

Sponsor:

University of Arizona
SIGARCH

HPDC '10: The 19th International Symposium on High Performance Distributed Computing

June 21 - 25, 2010

Illinois, Chicago

Acceptance Rates

Overall Acceptance Rate 166 of 966 submissions, 17%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
137
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Nguyen BTran VHluchy L(2013)An Abstraction Model for High-Level Application Programming on the CloudAdvanced Computational Methods for Knowledge Engineering10.1007/978-3-319-00293-4_22(295-306)Online publication date: 2013
https://doi.org/10.1007/978-3-319-00293-4_22
Kawai YIwai GSasaki TWatase Y(2011)Managing Distributed Files with RNS in Heterogeneous Data Grids2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing10.1109/CCGrid.2011.19(494-503)Online publication date: May-2011
https://doi.org/10.1109/CCGrid.2011.19

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

A distributed workflow management model for grid middleware

SAGA BigJob: An Extensible and Interoperable Pilot-Job Abstraction for Distributed Applications and Systems

Managing Distributed Files with RNS in Heterogeneous Data Grids