invited-talk

Hierarchical Storage from NVMe to Tapes

Author:

Thomas LeiboviciAuthors Info & Claims

EMOSS '22: Proceedings of the 2022 Workshop on Emerging Open Storage Systems and Solutions for Data Intensive Computing

Pages 3 - 4

https://doi.org/10.1145/3526061.3532096

Published: 27 June 2022 Publication History

Get Access

Abstract

Hierarchical storage allows to build both high-performance and high-capacity storage systems by combining various technologies like NVMe, SSDs, HDDs, or tapes.

While most existing systems only manage a limited number of levels and technologies, the IO-project aims to implement a deep storage hierarchy in the context of Exascale workloads. This system will be capable of managing most of the existing storage devices, from NVMe to tapes.

To achieve this, it leverages two open-source object stores: Motr[1], an innovative object store developed by Seagate and designed for Exascale, and Phobos[2], a tape-capable object store developed by CEA.

The resulting system will implement features to allow user and applications to optimize their data placement by specifying "hints" like extended attributes. It will also provide features to migrate data massively according to arbitrary criteria.

The system will be accessible through a native API, as well as a S3 gateway.

References

[1]

GitHub - Seagate/cortx-motr: CORTX Motr is a distributed object and key-value storage system targeting mass capacity storage configurations. It's the core component of CORTX storage system.: 2022. https://github.com/Seagate/cortx-motr/. Accessed: 2022- 05- 17.

Google Scholar

[2]

GitHub - cea-hpc/phobos: Parallel Heterogeneous Object Store: 2022. https://github.com/cea-hpc/phobos. Accessed: 2022-05-17.

Google Scholar

Index Terms

Hierarchical Storage from NVMe to Tapes
1. Information systems
  1. Information storage systems
    1. Record storage systems

Recommendations

File systems unfit as distributed storage backends: lessons from 10 years of Ceph evolution
SOSP '19: Proceedings of the 27th ACM Symposium on Operating Systems Principles

For a decade, the Ceph distributed file system followed the conventional wisdom of building its storage backend on top of local file systems. This is a preferred choice for most distributed file systems today because it allows them to benefit from the ...
Experiences with Hierarchical Storage Management Support in Blue Whale File System
PDCAT '10: Proceedings of the 2010 International Conference on Parallel and Distributed Computing, Applications and Technologies

In order to meet the challenges of significant storage and application growth, as well as shortened backup windows and limited IT resources, more and more organizations embrace Hierarchical Storage Management (HSM). Parts of SAN file systems provide the ...
The Optimization of the Hierarchical Storage System Based on the Hybrid SSD Technology
ISDEA '12: Proceedings of the 2012 Second International Conference on Intelligent System Design and Engineering Application

This paper introduces the hierarchical storage system (HSM) which is adopted in most computers in consideration of achieving a good balance between cost and performance. As the cost of Flash memory rapidly decreases, Flash-based Solid-State Disk (SSD) ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

EMOSS '22: Proceedings of the 2022 Workshop on Emerging Open Storage Systems and Solutions for Data Intensive Computing

July 2022

16 pages

ISBN:9781450393126

DOI:10.1145/3526061

General Chairs:
Sai Narasimhamurthy
Seagate, UK
,
Glenn Lockwood
LBNL, USA

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 June 2022

Check for updates

Author Tags

Qualifiers

Invited-talk

Funding Sources

EuroHPC-JU
Ministry of Education, Youth and Sports of the Czech Republic
BMBF/DLR

Conference

HPDC '22

Sponsor:

HPDC '22: The 31st International Symposium on High-Performance Parallel and Distributed Computing

July 1, 2022

MN, Minneapolis, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
77
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Sep 2024

Other Metrics

View Author Metrics

Citations

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Index Terms

Recommendations

File systems unfit as distributed storage backends: lessons from 10 years of Ceph evolution

Experiences with Hierarchical Storage Management Support in Blue Whale File System

The Optimization of the Hierarchical Storage System Based on the Hybrid SSD Technology