research-article

DFS: A file system for virtualized flash storage

Authors:

William K. Josephson,

David FlynnAuthors Info & Claims

ACM Transactions on Storage (TOS), Volume 6, Issue 3

Article No.: 14, Pages 1 - 25

https://doi.org/10.1145/1837915.1837922

Published: 28 September 2010 Publication History

Abstract

We present the design, implementation, and evaluation of Direct File System (DFS) for virtualized flash storage. Instead of using traditional layers of abstraction, our layers of abstraction are designed for directly accessing flash memory devices. DFS has two main novel features. First, it lays out its files directly in a very large virtual storage address space provided by FusionIO's virtual flash storage layer. Second, it leverages the virtual flash storage layer to perform block allocations and atomic updates. As a result, DFS performs better and is much simpler than a traditional Unix file system with similar functionalities. Our microbenchmark results show that DFS can deliver 94,000 I/O operations per second (IOPS) for direct reads and 71,000 IOPS for direct writes with the virtualized flash storage layer on FusionIO's ioDrive. For direct access performance, DFS is consistently better than ext3 on the same platform, sometimes by 20%. For buffered access performance, DFS is also consistently better than ext3, and sometimes by over 149%. Our application benchmarks show that DFS outperforms ext3 by 7% to 250% while requiring less CPU power.

References

[1]

}}Agrawal, N., Prabhakaran, V., Wobber, T., Davis, J. D., Manasse, M., and Panigrahy, R. 2008. Design tradeoffs for SSD performance. In Proceedings of the USENIX Technical Conference, USENIX Association, Monterey, CA.

Digital Library

[2]

}}Birrell, A., Isard, M., Thacker, C., and Wobber, T. 2007. A design for high-performance flash disks. ACM Oper. Syst. Rev.41, 2 (April).

Digital Library

[3]

}}Brants, T. and Franz, A. 2006. Web 1T 5-gram version 1.

[4]

}}Card, R., T'so, T., and Tweedie, S. 1994. The design and implementation of the second extended filesystem. In Proceedings of the 1st Dutch International Symposium on Linux.

[5]

}}Carlson, A., Mitchell, T. M., and Fette, I. 2008. Data analysis project: Leveraging massive textual corpora using n-gram statistics. Tech. rep. CMU-ML-08-107, Machine Learning Department, Carnegie Mellon University.

[6]

}}Douceur, J. R. and Bolosky, W. J. 1999. A large scale study of file-system contents. In Proceedings of the ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, ACM, New York.

Digital Library

[7]

}}Douglis, F., Caceres, R., Kaashoek, M. F., Li, K., Marsh, B., and Tauber, J. A. 1994. Storage alternatives for mobile computers. In Operating Systems Design and Implementation, 25--37.

Digital Library

[8]

}}FusionIO Corporation. 2009. FusionIO ioDrive specification sheet. http://www.fusionio.com/PDFs/Fusion%20Specsheet.pdf.

[9]

}}Hitz, D., Lau, J., and Malcom, M. 2001. File system design for an nfs file server appliance. Tech. rep. TR-3002, NetAppCorp. Sept.

[10]

}}Intel Corporation. 1998. Understanding the flash translation layer (FTL) specification.

[11]

}}Intel Corporation. 2009. Intel X25-E SATA solid state drive. http://download.intel.com/design/flash/nand/extreme/extreme-sata-ssd-datasheet.pdf.

[12]

}}Jo, H., Kang, J.-U., Park, S.-Y., Kim, J.-S., and Lee, K. 2006. FAB: Flash-aware buffer management policy for portable media players. IEEE Trans. Consumer Electron. 52, 2, 485--493.

Digital Library

[13]

}}Kawaguchi, A., Nishioka, S., and Motoda, H. 1995. A flash-memory based file system. In Proceedings of the Winter USENIX Technical Conference. USENIX Association, Monterey, CA.

Digital Library

[14]

}}Kim, H. and Ahn, S. 2008. BPLRU: A buffer management scheme for improving random writes in flash storage. In Proceedings of the 6th USENIX Conference on File and Storage Technologies. USENIX Association, Monterey, CA.

Digital Library

[15]

}}Kim, J., Kim, J. M., Noh, S. H., Min, S. L., and Cho, Y. 2002. A space-efficient flash translation layer for CompactFlash systems. IEEE Trans. Consumer Electron. 48, 2, 366--375.

Digital Library

[16]

}}Li, K. 1994. Towards a low power file system. Tech. Rep. CSD-94-814, University of California, Berkeley.

Digital Library

[17]

}}Llanos, D. R. 2006. TPCC-UVa: An open-source TPC-C implementation for global performance measurement of computer systems. ACM SIGMOD Record 35, 4, 6--15.

Digital Library

[18]

}}Manning, C. 2002. YAFFS: The NAND-specific flash file system. LinuxDevices.Org.

[19]

}}Marsh, B., Douglis, F., and Krishnan, P. 1994. Flash memory file caching for mobile computers. In Proceedings of the 27th Hawaii International Conference on Architecture.

[20]

}}Mathur, A., Cao, M., Bhattacharya, S., Dilger, A., Tomas, A., and Vivier, L. 2007. The new ext4 filesystem: Current status and future plans. In Proceedings of the Ottowa Linux Symposium.

[21]

}}McKusick, M. K., Joy, W. N., Leffler, S. J., and Fabry, R. S. 1984. A fast file system for UNIX. ACM Trans. Comput. Syst. 2, 3.

Digital Library

[22]

}}Norcott, W. 2009. Iozone filesystem benchmark. http://www.iozone.org.

[23]

}}Park, S.-Y., Jung, D., Kang, J.-U., Kim, J.-S., and Lee, J. 2006. CFLRU: A replacement algorithm for flash memory. In Proceedings of the International Conference on Compilers, Architecture and Synthesis for Embedded Systems.

Digital Library

[24]

}}Prabhakaran, V., Rodeheffer, T. L., and Zhou, L. 2008. Transactional flash. In Proceedings of the 8th USENIX Symposium on Operating Systems Design and Implementation.

Digital Library

[25]

}}Rajimwale, A., Prabhakaran, V., and Davis, J. D. 2009. Block management in solid state devices. Unpublished Tech. rep.

[26]

}}Rosenblum, M. and Ousterhout, J. K. 1992. The design and implementation of a log-structured file system. ACM Trans. Comput. Syst. 10, 1--15.

Digital Library

[27]

}}Tanenbaum, A. S., Herder, J. N., and Bos, H. 2006. File size distribution in UNIX systems: Then and now. ACM SIGOPS Oper. Syst. Rev. 40, 1, 100--104.

Digital Library

[28]

}}Transaction Processing Performance Council. 2008. TPC benchmark H: Decision support. http://www.tpc.org/tpch.

[29]

}}Troyanskaya, O., Cantor, M., Sherlock, G., Brown, P., Hastieevor, T., Tibshirani, R., Botstein, D., and Altman, R. B. 2001. Missing value estimation methods for DNA microarrays. Bioinformatics 17, 6, 520--525.

[30]

}}Tweedie, S. 2000. Ext3, journaling filesystem. In Proceedings of the Ottowa Linux Symposium.

[31]

}}Ulmer, C. and Gokhale, M. 2008. Threading opportunities in high-performance flash-memory storage. In High Performance Embedded Computing.

[32]

}}Woodhouse, D. 2001. JFFS: The journalling flash file system. In Proceedings of the Ottowa Linux Symposium.

[33]

}}Wu, M. and Zwaenepoel, W. 1994. eNVy: A non-volatile, main memory storage system. In Proceedings of the 6th International Conference on Architectural Support for Programming Languages and Operating Systems.

Digital Library

Cited By

Zhang TChen RLi ZGao CWang CShu J(2024)Design and Implementation of Deduplication on F2FSACM Transactions on Storage10.1145/366273520:4(1-50)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3662735
Sun XHe YWu DHuang J(2023)Survey of Distributed Computing Frameworks for Supporting Big Data AnalysisBig Data Mining and Analytics10.26599/BDMA.2022.90200146:2(154-169)Online publication date: Jun-2023
https://doi.org/10.26599/BDMA.2022.9020014
Zhang HZhou YXue YLiu YHuang J(2023)G10: Enabling An Efficient Unified GPU Memory and Storage Architecture with Smart Tensor MigrationsProceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture10.1145/3613424.3614309(395-410)Online publication date: 28-Oct-2023
https://dl.acm.org/doi/10.1145/3613424.3614309
Show More Cited By

Index Terms

DFS: A file system for virtualized flash storage
1. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Operating systems
        File systems management

Recommendations

DFS: a file system for virtualized flash storage
FAST'10: Proceedings of the 8th USENIX conference on File and storage technologies

This paper presents the design, implementation and evaluation of Direct File System (DFS) for virtualized flash storage. Instead of using traditional layers of abstraction, our layers of abstraction are designed for directly accessing flash memory ...
The design and implementation of an extensible network backup system in realtime
ICUIMC '09: Proceedings of the 3rd International Conference on Ubiquitous Information Management and Communication

This paper proposes a backup system based on mirroring filesystem "GMFS." GMFS has been developed to mirror data in realtime on the filesystem layer. The GMFS is a stackable filesystem which flexibly mirrors without changing the existing environment by ...
A unified framework for designing high performance in-memory and hybrid memory file systems

The emerging non-volatile memory technologies provide a new choice for storing persistent data in memory. Therefore, file system structure needs re-studying and re-designing. Our goal is to design a framework that gives high-performance in-memory file ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Storage

ACM Transactions on Storage Volume 6, Issue 3

September 2010

165 pages

ISSN:1553-3077

EISSN:1553-3093

DOI:10.1145/1837915

Issue’s Table of Contents

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 September 2010

Accepted: 01 June 2010

Revised: 01 May 2010

Received: 01 April 2010

Published in TOS Volume 6, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

147
Total Citations
View Citations
1,839
Total Downloads

Downloads (Last 12 months)61
Downloads (Last 6 weeks)9

Reflects downloads up to 28 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang TChen RLi ZGao CWang CShu J(2024)Design and Implementation of Deduplication on F2FSACM Transactions on Storage10.1145/366273520:4(1-50)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3662735
Sun XHe YWu DHuang J(2023)Survey of Distributed Computing Frameworks for Supporting Big Data AnalysisBig Data Mining and Analytics10.26599/BDMA.2022.90200146:2(154-169)Online publication date: Jun-2023
https://doi.org/10.26599/BDMA.2022.9020014
Zhang HZhou YXue YLiu YHuang J(2023)G10: Enabling An Efficient Unified GPU Memory and Storage Architecture with Smart Tensor MigrationsProceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture10.1145/3613424.3614309(395-410)Online publication date: 28-Oct-2023
https://dl.acm.org/doi/10.1145/3613424.3614309
Gu ZLi JPeng YLiu YZhang TGilad YKostic DMoatti YBiran O(2023)Elastic RAID: Implementing RAID over SSDs with Built-in Transparent CompressionProceedings of the 16th ACM International Conference on Systems and Storage10.1145/3579370.3594773(83-93)Online publication date: 5-Jun-2023
https://dl.acm.org/doi/10.1145/3579370.3594773
Sun JLi SSun YSun CVucinic DHuang JAamodt TJerger NSwift M(2023)LeaFTL: A Learning-Based Flash Translation Layer for Solid-State DrivesProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 210.1145/3575693.3575744(442-456)Online publication date: 27-Jan-2023
https://dl.acm.org/doi/10.1145/3575693.3575744
Liang YZeng SWang L(2023)Quantifying Resource Contention of Co-located Workloads with the System-level EntropyACM Transactions on Architecture and Code Optimization10.1145/356369620:1(1-25)Online publication date: 10-Feb-2023
https://dl.acm.org/doi/10.1145/3563696
Shen ZChen FYadgar GJia ZShao Z(2022)Prism-SSD: A Flexible Storage Interface for SSDsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2021.307232641:4(882-896)Online publication date: Apr-2022
https://doi.org/10.1109/TCAD.2021.3072326
Shen ZHan LMa CJia ZLi TShao Z(2021)Leveraging the Interplay of RAID and SSD for Lifetime Optimization of Flash-Based SSD RAIDIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2020.302049540:7(1395-1408)Online publication date: Jul-2021
https://doi.org/10.1109/TCAD.2020.3020495
Hao JChen XQiao YZhang YZhang T(2021)Implementing Flash-Cached Storage Systems Using Computational Storage Drive with Built-in Transparent Compression2021 IEEE International Conference on Networking, Architecture and Storage (NAS)10.1109/NAS51552.2021.9605383(1-8)Online publication date: Oct-2021
https://doi.org/10.1109/NAS51552.2021.9605383
Shu JLi FLi SLu Y(2020)Towards Unaligned Writes Optimization in Cloud Storage With High-Performance SSDsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2020.300665531:12(2923-2937)Online publication date: 1-Dec-2020
https://doi.org/10.1109/TPDS.2020.3006655
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents