poster

Poster: scalable infrastructure to support supercomputer resiliency-aware applications and load balancing

Authors:

Yoav Tock,

Benjamin Mandler,

José Moreira,

Terry JonesAuthors Info & Claims

SC '11 Companion: Proceedings of the 2011 companion on High Performance Computing Networking, Storage and Analysis Companion

Pages 9 - 10

https://doi.org/10.1145/2148600.2148606

Published: 12 November 2011 Publication History

Get Access

Abstract

High performance computing systems display increasing complexity and component counts. This trend exposes weaknesses in the underlying clustering infrastructure needed for continuous availability, maximizing utilization, and efficient administration of such systems. To mitigate the problem, we present a highly scalable clustering infrastructure, based on peer-to-peer technologies, for supporting resiliency-aware applications as well as efficient monitoring and load balancing. Supported services include Membership, Publish-subscribe messaging, Convergecast, Attribute replication and a DHT. We present a preliminary evaluation taken from an IBM BlueGene/P, demonstrating scalability up to ~ 256K nodes.

Supplementary Material

PDF File (post128.pdf)

Download
285.93 KB

References

[1]

HPC Colony-II. http://www.hpc-colony.org, 2011.

Google Scholar

[2]

A. Allavena, A. Demers, and J. E. Hopcroft. Correctness of a gossip based membership protocol. In PODC '05: Proceedings of the twenty-fourth annual ACM symposium on Principles of distributed computing, pages 292--301. ACM, 2005.

Digital Library

Google Scholar

[3]

G. Chockler, R. Melamed, Y. Tock, and R. Vitenberg. SpiderCast: a scalable interest-aware overlay for topic-based pub/sub communication. In Proceedings of the 2007 inaugural international conference on Distributed event-based systems, DEBS '07, pages 14--25. ACM, 2007.

Digital Library

Google Scholar

[4]

P. Kogge, K. Bergman, S. Borkar, D. Campbell, W. Carlson, W. Dally, M. Denneau, P. Franzon, W. Harrod, K. Hill, J. Hiller, S. Karp, S. Keckler, D. Klein, R. Lucas, M. Richards, A. Scarpelli, S. Scott, A. Snavely, T. Sterling, R. S. Williams, and K. Yelick. ExaScale Computing Study: Technology Challenges in Achieving ExaScale Systems. Technical report, DARPA IPTO, Air Force Research Labs, Sept. 2008.

Google Scholar

[5]

P. C. Roth, D. C. Arnold, and B. P. Miller. MRNet: A Software-Based Multicast/Reduction Network for Scalable Tools. In Proceedings of the 2003 ACM/IEEE conference on Supercomputing, SC '03. ACM, 2003.

Digital Library

Google Scholar

[6]

I. Stoica, R. Morris, D. Karger, M. F. Kaashoek, and H. Balakrishnan. Chord: A scalable peer-to-peer lookup service for internet applications. SIGCOMM Comput. Commun. Rev., 31(4):149--160, 2001.

Digital Library

Google Scholar

[7]

R. van Renesse, D. Dumitriu, V. Gough, and C. Thomas. Efficient reconciliation and flow control for anti-entropy protocols. In Proceedings of the 2nd Workshop on Large-Scale Distributed Systems and Middleware, LADIS '08. ACM, 2008.

Digital Library

Google Scholar

[8]

J. Varma, C. Wang, F. Mueller, C. Engelmann, and S. L. Scott. Scalable, fault tolerant membership for MPI tasks on HPC systems. In ICS '06: Proceedings of the 20th annual international conference on Supercomputing. ACM, 2006.

Digital Library

Google Scholar

Cited By

View all

Manzalini AMinerva RKaempfer ECallegari FCampi ACerroni WCrespi NDekel ETock YTavernier WCasier KVerbrugge SColle DVilalta RMunoz RCasellas RMartinez RMazzocca NMaini E(2013)Manifesto of edge ICT fabric2013 17th International Conference on Intelligence in Next Generation Networks (ICIN)10.1109/ICIN.2013.6670888(9-15)Online publication date: Oct-2013
https://doi.org/10.1109/ICIN.2013.6670888
Tock YMandler BMoreira JJones T(2013)Design and implementation of a scalable membership service for supercomputer resiliency-aware runtimeProceedings of the 19th international conference on Parallel Processing10.1007/978-3-642-40047-6_37(354-366)Online publication date: 26-Aug-2013
https://dl.acm.org/doi/10.1007/978-3-642-40047-6_37

Index Terms

Poster: scalable infrastructure to support supercomputer resiliency-aware applications and load balancing
1. Computer systems organization
  1. Architectures
    1. Distributed architectures
2. Software and its engineering
  1. Software organization and properties
    1. Software system structures
      1. Distributed systems organizing principles

Recommendations

A hybrid publish subscribe protocol
Companion '08: Proceedings of the ACM/IFIP/USENIX Middleware '08 Conference Companion

Content-based publish/subscribe system performance depends upon the efficient subscription matching and event dissemination to interested subscribers. We propose a hybrid content-based publish/subscribe protocol for large size events wherein a ...
Issues in the Design of Adaptive Middleware Load Balancing

Load balancing middleware is used extensively to improve scalability and overall system throughput in distributed systems. Many load balancing middleware services are simplistic, however, since they are geared only for specic use-cases and environments. ...
SpiderCast: a scalable interest-aware overlay for topic-based pub/sub communication
DEBS '07: Proceedings of the 2007 inaugural international conference on Distributed event-based systems

We introduce SpiderCast, a distributed protocol for constructing scalable churn-resistant overlay topologies for supporting decentralized topic-based pub/sub communication. SpiderCast is designed to effectively tread the balance between average overlay ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

SC '11 Companion: Proceedings of the 2011 companion on High Performance Computing Networking, Storage and Analysis Companion

November 2011

166 pages

ISBN:9781450310307

DOI:10.1145/2148600

Conference Chair:
Scott Lathrop
University of Chicago
,
Program Chairs:
Jim Costa
Sandia National Laboratories
,
William Kramer
National Center for Supercomputing Applications

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 November 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

SC '11

Sponsor:

SIGARCH
IEEE-CS

SC '11: International Conference for High Performance Computing, Networking, Storage and Analysis

November 12 - 18, 2011

Washington, Seattle, USA

Acceptance Rates

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
80
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)1

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Manzalini AMinerva RKaempfer ECallegari FCampi ACerroni WCrespi NDekel ETock YTavernier WCasier KVerbrugge SColle DVilalta RMunoz RCasellas RMartinez RMazzocca NMaini E(2013)Manifesto of edge ICT fabric2013 17th International Conference on Intelligence in Next Generation Networks (ICIN)10.1109/ICIN.2013.6670888(9-15)Online publication date: Oct-2013
https://doi.org/10.1109/ICIN.2013.6670888
Tock YMandler BMoreira JJones T(2013)Design and implementation of a scalable membership service for supercomputer resiliency-aware runtimeProceedings of the 19th international conference on Parallel Processing10.1007/978-3-642-40047-6_37(354-366)Online publication date: 26-Aug-2013
https://dl.acm.org/doi/10.1007/978-3-642-40047-6_37

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

A hybrid publish subscribe protocol

Issues in the Design of Adaptive Middleware Load Balancing

SpiderCast: a scalable interest-aware overlay for topic-based pub/sub communication