research-article

Triple-A: a Non-SSD based autonomic all-flash array for high performance storage systems

Authors:

Myoungsoo Jung,

Mahmut Taylan KandemirAuthors Info & Claims

ASPLOS '14: Proceedings of the 19th international conference on Architectural support for programming languages and operating systems

Pages 441 - 454

https://doi.org/10.1145/2541940.2541953

Published: 24 February 2014 Publication History

Abstract

Solid State Disk (SSD) arrays are in a position to (as least partially) replace spinning disk arrays in high performance computing (HPC) systems due to their better performance and lower power consumption. However, these emerging SSD arrays are facing enormous challenges, which are not observed in disk-based arrays. Specifically, we observe that the performance of SSD arrays can significantly degrade due to various array-level resource contentions. In addition, their maintenance costs exponentially increase over time, which renders them difficult to deploy widely in HPC systems. To address these challenges, we propose Triple-A, a non-SSD based Autonomic All-Flash Array, which is a self-optimizing, from-scratch NAND flash cluster. Triple-A can detect two different types of resource contentions and autonomically alleviate them by reshaping the physical data-layout on its flash array network. Our experimental evaluation using both real workloads and a micro-benchmark show that Triple-A can offer a 53% higher sustained throughput and a 80% lower I/O latency than non-autonomic SSD arrays.

References

[1]

http://www.dramexchange.com/.

[2]

SNIA IOTTA repository. URL http://iotta.snia.org/tracetypes/3.

[3]

N. Agrawal, V. Prabhakaran, T. Wobber, J. D. Davis, M. Manasse, and R. Panigrahy. Design tradeoffs for SSD performance. In USENIX ATC, 2008.

Digital Library

[4]

A. Akel, A. M. Caulfield, T. I. Mollov, R. K. Gupta, and S. Swanson. Onyx: a protoype phase change memory storage array. HotStorage, 2011.

Digital Library

[5]

M. Balakrishnan, A. Kadav, V. Prabhakaran, and D. Malkhi. Differential RAID: Rethinking RAID for SSD reliability. Trans. Storage, 2010.

Digital Library

[6]

K. Bates and B. McNutt. Umass Trace Repository. URL traces.cs.umass.edu/index.php/Main/Traces.

[7]

J. Borrill, L. Oliker, J. Shalf, and H. Shan. Investigation of leading hpc i/o performance using a scientific-application derived benchmark. In Proceedings of the 2007 ACM/IEEE conference on Supercomputing, 2007.

Digital Library

[8]

R. Budruk, D. Anderson, and E. Solari. PCI Express System Architecture. Pearson Education, 2003. ISBN 0321156307.

Digital Library

[9]

M. Canim, G. A. Mihaila, B. Bhattacharjee, K. A. Ross, and C. A. Lang. SSD bufferpool extensions for database systems. Proc. VLDB Endow., 2010.

Digital Library

[10]

P. Carns, K. Harms, W. Allcock, C. Bacon, S. Lang, R. Latham, and R. Ross. Understanding and improving computational science storage access through continuous characterization. Mass Storage Systems and Technologies, IEEE / NASA Goddard Conference on, 2011.

Digital Library

[11]

A. M. Caulfield and S. Swanson. Annual international symposium on computer architecture (isca). In QuickSAN: A Storage Area Network for Fast, Distributed, Solid State Disks, 2013.

Digital Library

[12]

Caulfield, Grupp, and Swanson}gordonA. M. Caulfield, L. M. Grupp, and S. Swanson. Gordon: using flash memory to build fast, power-efficient clusters for data-intensive applications. In ASPLOS, 2009.

Digital Library

[13]

Caulfield, Grupp, and Swanson}hpc:gordonA. M. Caulfield, L. M. Grupp, and S. Swanson. Gordon: Using flash memory to build fast, power-efficient clusters for data-intensive applications. In Proceedings of Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2009.

Digital Library

[14]

A. M. Caulfield, J. Coburn, T. Mollov, A. De, A. Akel, J. He, A. Jagatheesan, R. K. Gupta, A. Snavely, and S. Swanson. Understanding the impact of emerging non-volatile memories on high-performance, IO-intensive computing. In Proceedings of SC, 2010.

Digital Library

[15]

A. M. Caulfield, A. De, J. Coburn, T. I. Mollow, R. K. Gupta, and S. Swanson. Moneta: A high-performance storage array architecture for next-generation, non-volatile memories. In MICRO, 2010.

Digital Library

[16]

S. Choudhuri and T. Givargis. Deterministic service guarantees for NAND flash using partial block cleaning. In Proceedings of the International Conference on Hardware-Software Codesign and System Synthesis, 2008.

Digital Library

[17]

B. Debnath, S. Sengupta, and J. Li. Chunkstash: speeding up inline storage deduplication using flash memory. In USENIX ATC, 2010.

Digital Library

[18]

EMC. Vnx5500-f unified storage flash array.

[19]

A. Gupta, Y. Kim, and B. Urgaonkar. DFTL: A flash translation layer employing demand-based selective caching of page-level address mappings. In ASPLOS, 2009.

Digital Library

[20]

J. He, J. Bennett, and A. Snavely. DASH-IO: An empirical study of flash-based IO for HPC. TeraGrid'10, August 2010.

Digital Library

[21]

Y. Hu, H. Jiang, D. Feng, L. Tian, H. Luo, and S. Zhang. Performance impact and interplay of ssd parallelism through advanced commands, allocation strategy and data granularity. In ISC, 2011.

Digital Library

[22]

M. Jung and M. Kandemir. An evaluation of different page allocation strategies on high-speed SSDs. In USENIX HotStorage, 2012.

Digital Library

[23]

M. Jung and J. Yoo. Scheduling garbage collection opportunistically to reduce worst-case I/O performance in solid state disks. In Proceedings of the International Workshop on Software Support for Portable Storage, 2009.

[24]

Jung, Prabhakar, and Kandemir}bgcM. Jung, R. Prabhakar, and M. Kandemir. Taking garbage collection overheads off the critical path in ssds. In Middleware, 2012.

Digital Library

[25]

M. Jung, E. Willson, and M. Kandemir. Physically addressed queueing (PAQ): Improving parallelism in solid state disks. In phISCA, 2012.

Digital Library

[26]

M. Jung, E. H. Wilson, D. Donofrio, J. Shalf, and M. Kandemir. NANDFlashSim: Intrinsic latency variation aware NAND flash memory system modeling and simulation at microarchitecture level. In IEEE Conference on Massive Data Storage, 2012.

[27]

T. Kgil, D. Roberts, and T. Mudge. Improving NAND flash based disk caches. In International Symposium on Computer Architecture, 2008.

Digital Library

[28]

Y. Kim, B. Tauras, A. Gupta, and B. Urgaonkar. Flashsim: A simulator for NAND flash-based solid-state drives. In SIMUL, 2009.

Digital Library

[29]

S. W. Lee, W. K. Choi, and D. J. Park. FAST: An efficient flash translation layer for flash memory. In EUC Workshops Lecture Notes in Computer Science, 2006.

Digital Library

[30]

N. Liu, J. Cope, P. Carns, C. Carothers, R. Ross, G. Grider, A. Crume, and C. Maltzahn. On the role of burst buffers in leadership-class storage systems. In IEEE Conference on Massive Data Storage, 2012.

[31]

Y. Liu, J. Huang, C. Xie, and Q. Cao. Raf: A random access first cache management to improve SSD-based disk cache. NAS, 2010.

Digital Library

[32]

H. V. Madhyastha, J. C. McCullough, G. Porter, R. Kapoor, S. Savage, A. C. Snoeren, and A. Vahdat. scc: cluster storage provisioning informed by application characteristics and slas. FAST'12, 2012.

Digital Library

[33]

N. Master, M. Andrews, J. Hick, S. Canon, and N. Wright. Performance analysis of commodity and enterprise class flash devices. In PDSW, 2010.

[34]

S. Moon and A. L. N. Reddy. Hotstorage). In Dont Let RAID Raid the Lifetime of Your SSD Array, 2013.

Digital Library

[35]

E. H. Nam, B. Kim, H. Eom, and S.-L. Min. Ozone (O3): An out-of-order flash memory controller architecture. Computers, IEEE Transactions on, 2011.

Digital Library

[36]

NERSC. In http://www.nersc.gov/users/computational-systems/carver/.

[37]

NetApp. Netapp EF540 flash array.

[38]

ONFI Working Group. Open nand flash interface 3.0. In http://onfi.org/, 2012.

[39]

Y. Ou, T. Härder, and P. Jin. Cfdc: a flash-aware replacement policy for database buffer management. In the Fifth International Workshop on Data Management on New Hardware, 2009.

Digital Library

[40]

X. Ouyang, S. Marcarelli, and D. K. Panda. Enhancing checkpoint performance with staging I/O and SSD. In International Workshop on Storage Network Architecture and Parallel I/O, 2010.

Digital Library

[41]

C. Park, W. Cheon, Y. Lee, W. C. Myoung-Soo Jung, and H. Yoon. A re-configurable FTL architecture for NAND flash based applications. In IEEE/IFIP International Workshop on Rapid Prototyping, 2007.

Digital Library

[42]

C. Park, E. Seo, J.-Y. Shin, S. Maeng, and J. Lee. Exploiting internal parallelism of flash-based ssds. In Computer Architecture Letters, page 9, January 2010.

Digital Library

[43]

PCI-SIG. PCI express base 3.0 specification. 2012.

[44]

PLX Technology. PLX 8796 PLX expresslane gen 3 PCI Express compliant switches.

[45]

C. Policroniades and I. Pratt. Alternatives for detecting redundancy in storage systems data. In USENIX Annual Technical Conference, 2004.

Digital Library

[46]

Pure Storage. Fa-300 series technical specifications.

[47]

F. B. Schmuck and R. L. Haskin. GPFS: A shared-disk file system for large computing clusters. In FAST, 2002.

Digital Library

[48]

P. Shivam, A. Demberel, P. Gunda, D. Irwin, L. Grit, A. Yumerefendi, S. Babu, and J. Chase. Automated and on-demand provisioning of virtual machines for database applications. In ACM SIGMOD international conference on Management of data, 2007.

Digital Library

[49]

Texas Memory Systems. Texas memory systems RamSan-820.

[50]

Violin Memory. Violin memory 6000 Series Flash Memory Arrays.

[51]

D. Watts and M. Bachmaier. Implementing an IBM System x iDataPlex Solution. Redbooks, 2012.

Digital Library

[52]

O. C. Workgroup". Open nand flash interface specification: NAND connector.

[53]

Z. Zhou, E. Saule, H. Aktulga, C. Yang, E. Ng, P. Maris, J. Vary, and U. Catalyurek. An out-of-core dataflow middleware to reduce the cost of large scale iterative solvers. In Parallel Processing Workshops (ICPPW), 2012 41st International Conference on, 2012.

Digital Library

[54]

B. Zhu, K. Li, and R. H. Patterson. Avoiding the disk bottleneck in the data domain deduplication file system. In FAST, 2008.

Digital Library

Cited By

Wu SDu CLi HJiang HShen ZMao B(2021)CAGC: A Content-aware Garbage Collection Scheme for Ultra-Low Latency Flash-based SSDs2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS49936.2021.00025(162-171)Online publication date: May-2021
https://doi.org/10.1109/IPDPS49936.2021.00025
Kim JLim KJung YLee SMin CNoh SDan TDahlia M(2019)Alleviating garbage collection interference through spatial separation in all flash arraysProceedings of the 2019 USENIX Conference on Usenix Annual Technical Conference10.5555/3358807.3358875(799-812)Online publication date: 10-Jul-2019
https://dl.acm.org/doi/10.5555/3358807.3358875
Zhang JJung MOliveira RFelber PHu Y(2018)FlashabacusProceedings of the Thirteenth EuroSys Conference10.1145/3190508.3190544(1-15)Online publication date: 23-Apr-2018
https://dl.acm.org/doi/10.1145/3190508.3190544
Show More Cited By

Index Terms

Triple-A: a Non-SSD based autonomic all-flash array for high performance storage systems

Recommendations

Triple-A: a Non-SSD based autonomic all-flash array for high performance storage systems
ASPLOS '14

Solid State Disk (SSD) arrays are in a position to (as least partially) replace spinning disk arrays in high performance computing (HPC) systems due to their better performance and lower power consumption. However, these emerging SSD arrays are facing ...
Triple-A: a Non-SSD based autonomic all-flash array for high performance storage systems
ASPLOS '14

Solid State Disk (SSD) arrays are in a position to (as least partially) replace spinning disk arrays in high performance computing (HPC) systems due to their better performance and lower power consumption. However, these emerging SSD arrays are facing ...
Revisiting widely held SSD expectations and rethinking system-level implications
SIGMETRICS '13: Proceedings of the ACM SIGMETRICS/international conference on Measurement and modeling of computer systems

Storage applications leveraging Solid State Disk (SSD) technology are being widely deployed in diverse computing systems. These applications accelerate system performance by exploiting several SSD-specific characteristics. However, modern SSDs have ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ASPLOS '14: Proceedings of the 19th international conference on Architectural support for programming languages and operating systems

February 2014

780 pages

ISBN:9781450323055

DOI:10.1145/2541940

General Chairs:
Rajeev Balasubramonian
University of Utah
,
Al Davis
University of Utah
,
Program Chair:
Sarita Adve
University of Illinois at Urbana-Champ

ACM SIGARCH Computer Architecture News Volume 42, Issue 1
ASPLOS '14
March 2014
729 pages
ISSN:0163-5964
DOI:10.1145/2654822
Editor:
Doug DeGroot
acm dot org
Issue’s Table of Contents
ACM SIGPLAN Notices Volume 49, Issue 4
ASPLOS '14
April 2014
729 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/2644865
Editors:
Mark W. Bailey
Hamilton College, Clinton, NY
,
Rajeev Balasubramonian
University of Utah
,
Al Davis
University of Utah
,
Sarita Adve
University of Illinois at Urbana-Champ
Issue’s Table of Contents

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

SIGBED: ACM Special Interest Group on Embedded Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 February 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ASPLOS '14

Sponsor:

ASPLOS '14: Architectural Support for Programming Languages and Operating Systems

March 1 - 5, 2014

Utah, Salt Lake City, USA

Acceptance Rates

ASPLOS '14 Paper Acceptance Rate 49 of 217 submissions, 23%;

Overall Acceptance Rate 535 of 2,713 submissions, 20%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

32
Total Citations
View Citations
1,324
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)2

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wu SDu CLi HJiang HShen ZMao B(2021)CAGC: A Content-aware Garbage Collection Scheme for Ultra-Low Latency Flash-based SSDs2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS49936.2021.00025(162-171)Online publication date: May-2021
https://doi.org/10.1109/IPDPS49936.2021.00025
Kim JLim KJung YLee SMin CNoh SDan TDahlia M(2019)Alleviating garbage collection interference through spatial separation in all flash arraysProceedings of the 2019 USENIX Conference on Usenix Annual Technical Conference10.5555/3358807.3358875(799-812)Online publication date: 10-Jul-2019
https://dl.acm.org/doi/10.5555/3358807.3358875
Zhang JJung MOliveira RFelber PHu Y(2018)FlashabacusProceedings of the Thirteenth EuroSys Conference10.1145/3190508.3190544(1-15)Online publication date: 23-Apr-2018
https://dl.acm.org/doi/10.1145/3190508.3190544
Boito FInacio EBez JNavaux PDantas MDenneulin Y(2018)A Checkpoint of Research on Parallel I/O for High-Performance ComputingACM Computing Surveys10.1145/315289151:2(1-35)Online publication date: 12-Mar-2018
https://dl.acm.org/doi/10.1145/3152891
Mao BWu SDuan L(2018)Improving the SSD Performance by Exploiting Request Characteristics and Internal ParallelismIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2017.269796137:2(472-484)Online publication date: 1-Feb-2018
https://dl.acm.org/doi/10.1109/TCAD.2017.2697961
Wu SZhu WLiu GJiang HMao B(2018)GC-Aware Request Steering with Improved Performance and Reliability for SSD-Based RAIDs2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS.2018.00039(296-305)Online publication date: May-2018
https://doi.org/10.1109/IPDPS.2018.00039
Mu JSoumagne JTang HByna SKoziol QWarren R(2018)A Transparent Server-Managed Object Storage System for HPC2018 IEEE International Conference on Cluster Computing (CLUSTER)10.1109/CLUSTER.2018.00063(477-481)Online publication date: Sep-2018
https://doi.org/10.1109/CLUSTER.2018.00063
Tang HByna STessier FWang TDong BMu JKoziol QSoumagne JVishwanath VLiu JWarren REl-Araby EEl-Ghazawi TPanda D(2018)Toward scalable and asynchronous object-centric data management for HPCProceedings of the 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing10.1109/CCGRID.2018.00026(113-122)Online publication date: 1-May-2018
https://dl.acm.org/doi/10.1109/CCGRID.2018.00026
Wu SMao BLin YJiang H(2017)Improving Performance for Flash-Based Storage Systems through GC-Aware Cache ManagementIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2017.269275728:10(2852-2865)Online publication date: 1-Oct-2017
https://doi.org/10.1109/TPDS.2017.2692757
Choi WJung MKandemir MDas C(2017)A Scale-Out Enterprise Storage Architecture2017 IEEE International Conference on Computer Design (ICCD)10.1109/ICCD.2017.96(549-556)Online publication date: Nov-2017
https://doi.org/10.1109/ICCD.2017.96
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten