Article

Free access

Deciding when to forget in the Elephant file system

Authors:

Douglas S. Santry,

Michael J. Feeley,

Norman C. Hutchinson,

Alistair C. Veitch,

Ross W. Carton,

Jacob OfirAuthors Info & Claims

SOSP '99: Proceedings of the seventeenth ACM symposium on Operating systems principles

Pages 110 - 123

https://doi.org/10.1145/319151.319159

Published: 12 December 1999 Publication History

Abstract

Modern file systems associate the deletion of a file with the immediate release of storage, and file writes with the irrevocable change of file contents. We argue that this behavior is a relic of the past, when disk storage was a scarce resource. Today, large cheap disks make it possible for the file system to protect valuable data from accidental delete or overwrite.This paper describes the design, implementation, and performance of the Elephant file system, which automatically retains all important versions of user files. Users name previous file versions by combining a traditional pathname with a time when the desired version of a file or directory existed. Storage in Elephant is managed by the system using file-grain user-specified retention policies. This approach contrasts with checkpointing file systems such as Plan-9, AFS, and WAFL that periodically generate efficient checkpoints of entire file systems and thus restrict retention to be guided by a single policy for all files within that file system.Elephant is implemented as a new Virtual File System in the FreeBSD kernel.

References

[1]

Digital. Vax/VMS System Software Handbook. Bedford, 1985.]]

[2]

John R. Douceur and William J. Bolosky. A large-scale study of file-system contents. In Proceedings of SIG- METRICS '99, pages 59-69, Atlanta, GA, USA, 1999.]]

Digital Library

[3]

James Griffioen and Randy Appleton. Reducing file system latency using a predictive approach. InPreceedings ef the Usenix Summer Conference, pages 197-208, Boston, MA, June 1994. Usenix.]]

Digital Library

[4]

R. Hagmann. Reimplementing the cedar file system using logging and group commit. Proceedings of the llth ACM Symposium on Operating Systems Principles, 21(5):155-162, November 1987.]]

Digital Library

[5]

Dave Hitz, James Lau, and Michael Malcolm. File system design for a file server appliance. In Proceedings of the 1994 Winter USENIX Technical Conference, pages 235-245, San Francisco, CA, January 1994. Usenix.]]

Digital Library

[6]

John H. Howard, Michael L. Kazar, Sherri G. Menees, David A. Nichols, M. Satyanarayanan, Robert N. Sidebotham, and Michael J. West. Scale and performance in a distributed file system. ACM Transactions on Computer Systems, 6(1):51-81, February 1988.]]

Digital Library

[7]

James J. Kistler and M. Satyanarayanan. Disconnected operation in the Coda file system. In Proceedings of the Thirteenth ACM Symposium on Operating System Principles, pages 213-225, Pacific Grove, CA, October 1991. ACM.]]

Digital Library

[8]

K.Smith and M.Seltzer. A comparison of FFS disk allocation policies. In Prec. 1996 USENIX Conference, pages 15-26, January 1996.]]

Digital Library

[9]

Jeanna Neefe Matthews, Drew Roselli, Adam M. Costello, Randolph Y. Wang, and Thomas E. Anderson. Improving the performance of log-structured file systems with adaptative methods. In Proceedings of the 16th Symposium on Operating Systems Principles, pages 238-252, 1997.]]

Digital Library

[10]

M. McDonald and R. Bunt. Improving file system performance by dynamically restructuring disk space. In Prec. of Phoenix Conference on Computers and Communication, pages 264-269, March 1989.]]

[11]

M. K. McKusick, W. N. Joy, S. J. Leffler, and R. S. Fabry. A fast file system for UNIX. ACM Transactions on Computer Systems, 2(3):181-197, August 1984.]]

Digital Library

[12]

L. McVoy and S. Kleiman. Extent-like performance from a UNIX file system. In Proceedings of the 1990 Summer Usenix, pages 137-144, June 1990.]]

[13]

Lisa Moses. TOPS-20 User's manual. USC/Information Sciences Institute, Internal manual, Marina del Rey, California.]]

[14]

J. K. Ousterhout, H. Da Costa, D. Harrison, J.A. Kunze, M. Kupfer, and J.G. Thompson. A trace-driven analysis of the UNIX 4.2BSD file system. In Proceedings of the 10th Symposium on Operating Systems Principles, pages 15-24, Orcas Island, WA, December 1985.]]

Digital Library

[15]

David Presotto. Plan 9. In Proceedings of the Workshop on Micro-kernels and Other Kernel Architectures, pages 31-38, Seattle, WA, USA, April 1992. USENIX Association.]]

Digital Library

[16]

Sean Quinlan. A cached worm file system. Software-- Practice and Experience, 21(12):1289-1299, December 1991.]]

Digital Library

[17]

Marc J. Rochkind. The source code control system. IEEE Transactions on Software Engineering, 1 (4):364- 370, December 1975.]]

Digital Library

[18]

Drew Roselli. Characteristics of file system workloads. Technical Report UCB//CSD-98-1029, Computer Science Division, University of California, Berkeley, 1998.]]

Digital Library

[19]

Mendel Rosenblum and John K. Ousterhout. The design and implementation of a log-structured file system. ACM Transactions on Computer Systems, 10(1):26-52, February 1992.]]

Digital Library

[20]

Michael D. Schroeder, David K. Gifford, and Roger M. Needham. A caching file system for a programmer's workstation. In Proceedings of the lOth ACM Symposium on Operating Systems Principles, pages 25-34, Orcas Island WA (USA), December 1985. ACM.]]

Digital Library

[21]

M. Seltzer, K. Smith, H. Balakrishnan, J. Chang, S. Mc- Mains, and V. Padmanabhan. File system logging versus clustering. In Prec. 1995 Winter USENIX Conference, pages 249-264, January 1995.]]

Digital Library

[22]

Margo Seltzer, Keith Bostic, Marshall Kirk McKusick, and Carl Staelin. An implementation of a log-structured file system for UNIX. In USENIX Association, editor, Proceedings of the Winter 1993 USENIX Conference: January 25-29, 1993, San Diego, California, USA, pages 307-326, Berkeley, CA, USA, Winter 1993. USENIX.]]

Digital Library

[23]

Michael Stonebraker. The design of the POSTGRES storage system. In Proceedings of the 13th International Conference on Very Large Data Bases, pages 289-300, September 1987.]]

Digital Library

[24]

Walter F. Tichy. RCS: A system for version control. Software- Practice and Experience, 15(7):637-654, July 1985.]]

Digital Library

Cited By

Matos DPardal MCorreia M(2023)Sanare: Pluggable Intrusion Recovery for Web ApplicationsIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2021.313947220:1(590-605)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TDSC.2021.3139472
Vaz DMatos DPardal MCorreia M(2023)MIRES: Intrusion Recovery for Applications Based on Backend-As-a-ServiceIEEE Transactions on Cloud Computing10.1109/TCC.2022.317898211:2(2011-2027)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TCC.2022.3178982
Matos DPardal MSilva ACorreia M(2023)μVerum: Intrusion Recovery for Microservice ApplicationsIEEE Access10.1109/ACCESS.2023.329811311(78457-78470)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3298113
Show More Cited By

Index Terms

Deciding when to forget in the Elephant file system
1. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Operating systems

Recommendations

Deciding when to forget in the Elephant file system

Modern file systems associate the deletion of a file with the immediate release of storage, and file writes with the irrevocable change of file contents. We argue that this behavior is a relic of the past, when disk storage was a scarce resource. Today, ...
Deciding When to Forget in the Elephant File System
Elephant: The File System that Never Forgets
HOTOS '99: Proceedings of the The Seventh Workshop on Hot Topics in Operating Systems

Modern file systems associate the deletion of a file with the release of the storage associated with that file, and file writes with the irrevocable change of file contents. We propose that this model of file system behavior is a relic of the past, when ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SOSP '99: Proceedings of the seventeenth ACM symposium on Operating systems principles

December 1999

300 pages

ISBN:1581131402

DOI:10.1145/319151

Chairmen:
David Kotz
Dartmouth College, Hanover, NH
,
John Wilkes
HP Laboratories

ACM SIGOPS Operating Systems Review Volume 33, Issue 5
Dec. 1999
291 pages
ISSN:0163-5980
DOI:10.1145/319344
Editors:
David Kotz
Dartmout College, Hanover, NH
,
John Wilkes
HP Laboratories
Issue’s Table of Contents

Copyright © 1999 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 December 1999

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

SOSP99

Sponsor:

SIGOPS

SOSP99: 17th Symposium on Operating Systems Principles

December 12 - 15, 1999

South Carolina, Charleston, USA

Acceptance Rates

SOSP '99 Paper Acceptance Rate 19 of 90 submissions, 21%;

Overall Acceptance Rate 131 of 716 submissions, 18%

Upcoming Conference

SOSP '25

Sponsor:
sigops

ACM SIGOPS 31st Symposium on Operating Systems Principles

October 13 - 16, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

224
Total Citations
View Citations
2,135
Total Downloads

Downloads (Last 12 months)285
Downloads (Last 6 weeks)24

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Matos DPardal MCorreia M(2023)Sanare: Pluggable Intrusion Recovery for Web ApplicationsIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2021.313947220:1(590-605)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TDSC.2021.3139472
Vaz DMatos DPardal MCorreia M(2023)MIRES: Intrusion Recovery for Applications Based on Backend-As-a-ServiceIEEE Transactions on Cloud Computing10.1109/TCC.2022.317898211:2(2011-2027)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TCC.2022.3178982
Matos DPardal MSilva ACorreia M(2023)μVerum: Intrusion Recovery for Microservice ApplicationsIEEE Access10.1109/ACCESS.2023.329811311(78457-78470)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3298113
Zhan YConway AJiao YMukherjee NGroombridge IBender MFarach-Colton MJannen WJohnson RPorter DYuan JNoh SWelch B(2020)How to copy filesProceedings of the 18th USENIX Conference on File and Storage Technologies10.5555/3386691.3386699(75-90)Online publication date: 24-Feb-2020
https://dl.acm.org/doi/10.5555/3386691.3386699
Jana RDey SDasgupta P(2020)A Hierarchical HVAC Control Scheme for Energy-aware Smart Building AutomationACM Transactions on Design Automation of Electronic Systems10.1145/339366625:4(1-33)Online publication date: 23-May-2020
https://dl.acm.org/doi/10.1145/3393666
Chatterjee UChatterjee SMukhopadhyay DChakraborty R(2020)Machine Learning Assisted PUF Calibration for Trustworthy Proof of Sensor Data in IoTACM Transactions on Design Automation of Electronic Systems10.1145/339362825:4(1-21)Online publication date: 8-Jun-2020
https://dl.acm.org/doi/10.1145/3393628
Konstantopoulos MChondros NRoussopoulos M(2020)A social link based private storage cloud2020 IEEE 26th International Conference on Parallel and Distributed Systems (ICPADS)10.1109/ICPADS51040.2020.00065(447-455)Online publication date: Dec-2020
https://doi.org/10.1109/ICPADS51040.2020.00065
Matos DPardal MCarle GCorreia MPierre GFerreira PShrira L(2018)RockFSProceedings of the 19th International Middleware Conference10.1145/3274808.3274817(107-119)Online publication date: 26-Nov-2018
https://dl.acm.org/doi/10.1145/3274808.3274817
Rawat PRastello FSukumaran-Rajam APouchet LRountev ASadayappan P(2018)Register optimizations for stencils on GPUsACM SIGPLAN Notices10.1145/3200691.317850053:1(168-182)Online publication date: 10-Feb-2018
https://dl.acm.org/doi/10.1145/3200691.3178500
Chabbi MWen SLiu X(2018)Featherlight on-the-fly false-sharing detectionACM SIGPLAN Notices10.1145/3200691.317849953:1(152-167)Online publication date: 10-Feb-2018
https://dl.acm.org/doi/10.1145/3200691.3178499
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents