article

Free access

Relaxed consistency and coherence granularity in DSM systems: a performance evaluation

Authors:

Jaswinder Pal Sing,

Brian R. Toonen,

Ioannis Schoinas,

David A. WoodAuthors Info & Claims

ACM SIGPLAN Notices, Volume 32, Issue 7

Pages 193 - 205

https://doi.org/10.1145/263767.263788

Published: 21 June 1997 Publication History

Abstract

During the past few years, two main approaches have been taken to improve the performance of software shared memory implementations: relaxing consistency models and providing fine-grained access control. Their performance tradeoffs, however, we not well understood. This paper studies these tradeoffs on a platform that provides access control in hardware but runs coherence protocols in software, We compare the performance of three protocols across four coherence granularities, using 12 applications on a 16-node cluster of workstations. Our results show that no single combination of protocol and granularity performs best for all the applications. The combination of a sequentially consistent (SC) protocol and fine granularity works well with 7 of the 12 applications. The combination of a multiple-writer, home-based lazy release consistency (HLRC) protocol and page granularity works well with 8 out of the 12 applications. For applications that suffer performance losses in moving to coarser granularity under sequential consistency, the performance can usually be regained quite effectively using relaxed protocols, particularly HLRC. We also find that the HLRC protocol performs substantially better than a single-writer lazy release consistent (SW-LRC) protocol at coase granularity for many irregular applications. For our applications and platform, when we use the original versions of the applications ported directly from hardware-coherent shared memory, we find that the SC protocol with 256-byte granularity performs best on average. However, when the best versions of the applications are compared, the balance shifts in favor of HLRC at page granularity.

References

[1]

J.K. Bennett, J.B. Carter, and W. Zwaenepoel. Adaptive Software Cache Management for Distributed Shared Memory Architectures. In Proceedings of the 17th Annual Symposium on Computer Architecture, pages 125-134, May 1990.]]

Digital Library

[2]

B.N. Bershad, M.J. Zekauskas, and W.A. Sawdon. The Midway Distributed Shared Memory System. in Proceedings of the IEEE COMPCON '93 Conference, February 1993.]]

[3]

Nanette J. Boden, Danny Cohen, Robert E. Felderman, Alan E. Kulawik, Charles L. Seitz, Jakov N. Seizovic, and Wen-King Su. Myrinet: A Gigabit-per-Second Local Area Network. IEEE Micro, 15(1):29-36, February 1995.]]

Digital Library

[4]

L. Borrmann and M. tierdieckerhoff. A Coherency Model for Virtual Shared Memory. In Proceedings of the l Oth International Parallel Processing Symposium, June 1990.]]

[5]

J.B. Carter, J.K. Bennett, and W. Zwaenepoel. Implementation and Performance of Munin. In Proceedings of the Thirteenth Symposium on Operating Systems Principles, pages 152-164, October 1991.]]

Digital Library

[6]

David Culler, Lok Tin Liu, Richard Martin, and Chad Yoshikawa. LogP Performance Assessment of Fast Network Interfaces. IEEE Micro, pages 35-43, February 1996.]]

Digital Library

[7]

David Culler, Lok Tin Liu, Richard Martin, and Chad Yoshil#wa. LogP Performance Assessment of Fast Network Interfaces. IEEE Micro, pages 35-43, February 1996.]]

Digital Library

[8]

M. Dubois, J.C. Wang, L.A. Barroso, K. Lee, and Y-S Chen. Delayed Consistency and Its Effects on the Miss Rate of Parallel Programs. In Supercompu#ing '91, pages 197-206, 1991.]]

Digital Library

[9]

A. Erlichson, N. Nuckolls, G. Chesson, and J. Hennessy. Soft- FLASH: Analyzing the Performance of Clustered Distributed Virtual Shared Memory. In The 6th International Conference on Architectural Support for Programming Languages and Operating Systems, October 1996.]]

Digital Library

[10]

K. Gharachorloo, D. Lenoski, J. Laudon, P. Gibbons, A. Gupta, and J. Hennessy. Memory Consistency and Event Ordering in Scalable Shared-Memory Multiprocessors. In Proceedings of the 17th Annual Symposium on Computer Architecture, pages 15-26, May 1990.]]

Digital Library

[11]

L. Iftode, C. Dubnicki, E. W. Felten, and Kai Li. Improving Release-Consistent Shared Virtual Memory using Automatic Update. In The 2nd IEEE Symposium on High-Performance Computer Architecture, February 1996.]]

Digital Library

[12]

L. Iftode, J. P. Singh, and Kai Li. Understanding Application Performance on Shared Virtual Memory. in Proceedings of the Z3rd Annual Symposium on Computer Architecture, May 1996.]]

Digital Library

[13]

L. lftode, J.P. Singh, and K. Li. Scope Consistency: a Bridge Between Release Consistency and Entry Consistency. In Proceedings of the 8th Annual A CM Symposium on Parallel Algorithms and Architectures, June 1996.]]

Digital Library

[14]

P. Keleher, A.L. Cox, S. Dwarkadas, and W. Zwaenepoel. TreadMarks: Distributed Shared Memory on Standard Workstations and Operating Systems. In Proceedings of the Winter USENIX Conference, pages 115-132, January 1994.]]

Digital Library

[15]

P. Keleher, A.L. Cox, and W. Zwaenepoel. Lazy Consistency for Software Distributed Shared Memory. In Proceedings of the i gth Annual Symposium on Computer Architecture, pages 13-21, May 1992.]]

Digital Library

[16]

P.J. Keleher. The Relative Importance of Concurrent Writers and Weak Consistency Models. in Proceedings of the IEEE COMPCON '96 Conference, February 1996.]]

Digital Library

[17]

L. Lamport. How to Make a Multiprocessor Computer That Correctly Executes Multiprocessor Programs. IEEE Transactions on Computers, C-28(9):690--691, 1979.]]

Digital Library

[18]

Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Wolf-Dietrich Weber, Anoop Gupta, John Hennessy, Mark Horowitz, and Monica Lain. The Stanford DASH Multiprocessor. IEEE Computer, 25(3):63-79, March 1992.]]

Digital Library

[19]

K. Li. IVY: A Shared Virtual Memory System for Parallel Computing. In Proceedings of the 1988 International Con- }erence on Parallel Processing, volume II Software, pages 94-101, August 1988.]]

[20]

K. Li and P. Hudak. Memory Coherence in Shared Virtual Memory Systems. In Proceedings of the 5th Annual A CM Symposium on Principles of Distributed Computing, pages 229-239, August 1986.]]

Digital Library

[21]

Scott Pakin, Mario Laura, and Andrew Chien. High Performance Messaging on Workstations: Illinois Fast Messages (FM) for Myrinet. In Proceedings o,f Supercomputing '95, 1995.]]

Digital Library

[22]

Robert W. Pfile. Typhoon-Zero Implementation: The Vortex Module. Technical report, Wisconsin University, CS department, 1995.]]

[23]

S. K. Reinhard, R. W. Pfile, and D. A. Wood. Decoupled Hardware Support for Distributed Shared Memory. In Proceedings of the P3rd Annual Symposium on Computer Architecture, May 1996.]]

Digital Library

[24]

S.K. Reinhardt, J.R. Larus, and D.A. Wood. Tempest and Typhoon: User-Level Shared Memory. In Proceedings of the 21st Annual Symposium on Computer Architecture, pages 325-336, April 1994.]]

Digital Library

[25]

I#OSS Technology, Inc. SPARC RISC User's Guide: hyper- SPARC Edition, September 1993.]]

[26]

D.J. Scales, K. Gharachorloo, and C.A. Thekkath. Shasta: A Low Overhead, SOftware-Only Approach for Supporting Fine-Grain Shared Memory. In The 6th International Conference on Architectural Support .for Programming Languages and Operating Systems, October 1996.]]

Digital Library

[27]

I. Schoinas, B. Falsafi, A.R. Lebeck, S.K. Reinhardt, J.R. Larus, and D.A. Wood. Fine-grain Access for Distributed Shared Memory. In The 6th International Conference on Architectural Support,for Programming Languages and Operating Systems, pages 297-306, October 1994.]]

Digital Library

[28]

Ioannis Schoinas, Babak Falsafi, Mark D. Hill, James R. Larus, Christopher E. Lucas, Shubhendu S. Mukherjee, Steven K. Reinhardt, Eric Schnarr, and David A. Wood. Implementing Fine-Grain Distributed Shared Memory On Commodity SMP Workstations. Technical Report 1307, March 1996.]]

[29]

W. Weber and A. Gupta. Analysis of Cache Invalidation Patterns in Multiprocessors. In The Third International Conference on Architectural Support for Programming Languages and Operating Systems, pages 243-256, April 1989.]]

Digital Library

[30]

Y. Zhou, L. Iftode, and K. Li. Performance Evaluation of Two Home-Based Lazy Release Consistency Protocols for Shared Virtual Memory Systems. In Proceedings of the Operating Systems Design and Implementation Symposium, October 1996.]]

Digital Library

Cited By

An BCha MLee SYang WKim H(2024)Providing scalable single‐operating‐system NUMA abstraction of physically discrete resourcesETRI Journal10.4218/etrij.2023-0056Online publication date: 16-Jan-2024
https://doi.org/10.4218/etrij.2023-0056
Mu SSu MGao PWu YLi KZomaya A(2015)Cloud Storage over Multiple Data CentersHandbook on Data Centers10.1007/978-1-4939-2092-1_24(691-725)Online publication date: 17-Mar-2015
https://doi.org/10.1007/978-1-4939-2092-1_24
Vora KKoduru SGupta R(2014)ASPIREACM SIGPLAN Notices10.1145/2714064.266022749:10(861-878)Online publication date: 15-Oct-2014
https://doi.org/10.1145/2714064.2660227
Show More Cited By

Index Terms

Recommendations

Relaxed consistency and coherence granularity in DSM systems: a performance evaluation
PPOPP '97: Proceedings of the sixth ACM SIGPLAN symposium on Principles and practice of parallel programming

During the past few years, two main approaches have been taken to improve the performance of software shared memory implementations: relaxing consistency models and providing fine-grained access control. Their performance tradeoffs, however, we not well ...
Hiding Relaxed Memory Consistency with Compilers
PACT '00: Proceedings of the 2000 International Conference on Parallel Architectures and Compilation Techniques

We present a compiler technique, which is based on Shasha and Snir's delay set analysis, to hide the underlying relaxed memory consistency model for an optimizing compiler for explicitly parallel programs. The compiler presents programmers with a ...
An interaction of coherence protocols and memory consistency models in DSM systems

Coherence protocols and memory consistency models are two improtant issues in hardware coherent shared memory multiprocessors and softare distributed shared memory(DSM) systems. Over the years, many researchers have made extensive study on these two ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM SIGPLAN Notices

ACM SIGPLAN Notices Volume 32, Issue 7

July 1997

287 pages

ISSN:0362-1340

EISSN:1558-1160

DOI:10.1145/263767

Chairmen:
Rob Schreiber
Hewlett-Packard Labs, Palo Alto, CA
,
Keshav Pingali
Cornell Univ., Ithaca, NY
,
Editor:
A. Michael Berman

Issue’s Table of Contents

PPOPP '97: Proceedings of the sixth ACM SIGPLAN symposium on Principles and practice of parallel programming
June 1997
287 pages
ISBN:0897919068
DOI:10.1145/263764
Chairmen:
Rob Schreiber
Hewlett-Packard Labs, Palo Alto, CA
,
Keshav Pingali
Cornell Univ., Ithaca, NY
,
Editor:
Michael A. Berman

Copyright © 1997 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 June 1997

Published in SIGPLAN Volume 32, Issue 7

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

45
Total Citations
View Citations
538
Total Downloads

Downloads (Last 12 months)93
Downloads (Last 6 weeks)12

Reflects downloads up to 21 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

An BCha MLee SYang WKim H(2024)Providing scalable single‐operating‐system NUMA abstraction of physically discrete resourcesETRI Journal10.4218/etrij.2023-0056Online publication date: 16-Jan-2024
https://doi.org/10.4218/etrij.2023-0056
Mu SSu MGao PWu YLi KZomaya A(2015)Cloud Storage over Multiple Data CentersHandbook on Data Centers10.1007/978-1-4939-2092-1_24(691-725)Online publication date: 17-Mar-2015
https://doi.org/10.1007/978-1-4939-2092-1_24
Vora KKoduru SGupta R(2014)ASPIREACM SIGPLAN Notices10.1145/2714064.266022749:10(861-878)Online publication date: 15-Oct-2014
https://doi.org/10.1145/2714064.2660227
Zhao HShriraman AKumar SDwarkadas SMendelson A(2013)ProtozoaProceedings of the 40th Annual International Symposium on Computer Architecture10.1145/2485922.2485969(547-558)Online publication date: 23-Jun-2013
https://dl.acm.org/doi/10.1145/2485922.2485969
Iosevich VSchuster A(2005)Software Distributed Shared Memory: a VIA-based implementation and comparison of sequential consistency with home-based lazy release consistencySoftware: Practice and Experience10.1002/spe.65635:8(755-786)Online publication date: 2005
https://doi.org/10.1002/spe.656
Kim SChuang HLyerly ROlivier PMin CRavindran B(2020)DeX: Scaling Applications Beyond Machine Boundaries2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS47774.2020.00021(864-876)Online publication date: Nov-2020
https://doi.org/10.1109/ICDCS47774.2020.00021
Cao DHe XNie LWei XHu XWu SChua T(2017)Cross-Platform App Recommendation by Jointly Modeling Ratings and TextsACM Transactions on Information Systems10.1145/301742935:4(1-27)Online publication date: 11-Jul-2017
https://dl.acm.org/doi/10.1145/3017429
Liu XAi WLi HTang JHuang GFeng FMei Q(2017)Deriving User Preferences of Mobile Apps from Their Management ActivitiesACM Transactions on Information Systems10.1145/301546235:4(1-32)Online publication date: 11-Jul-2017
https://dl.acm.org/doi/10.1145/3015462
Vora KKoduru SGupta R(2014)ASPIREACM SIGPLAN Notices10.1145/2714064.266022749:10(861-878)Online publication date: 15-Oct-2014
https://dl.acm.org/doi/10.1145/2714064.2660227
Vora KKoduru SGupta RBlack AMillstein T(2014)ASPIREProceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & Applications10.1145/2660193.2660227(861-878)Online publication date: 15-Oct-2014
https://dl.acm.org/doi/10.1145/2660193.2660227
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents