Abstract
System software overheads in the I/O path, including VFS and file system code, become more pronounced with emerging low-latency storage devices. Currently, these overheads constitute the main bottleneck in the I/O path and they limit efficiency of modern storage systems. In this paper we present Iris, a new I/O path for applications, that minimizes overheads from system software in the common I/O path. The main idea is the separation of the control and data planes. The control plane consists of an unmodified Linux kernel and is responsible for handling data plane initialization and the normal processing path through the kernel for non-file related operations. The data plane is a lightweight mechanism to provide direct access to storage devices with minimum overheads and without sacrificing strong protection semantics. Iris requires neither hardware support from the storage devices nor changes in user applications. We evaluate our early prototype and we find that it achieves on a single core up to \(1.7\times \) and \(2.2\times \) better read and write random IOPS, respectively, compared to the xfs and ext4 file systems. It also scales with the number of cores; using 4 cores Iris achieves \(1.84\times \) and \(1.96\times \) better read and write random IOPS, respectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
AMD: Secure Virtual Machine Architecture Reference Manual
Axboe, J.: Flexible I/O Tester (2005). https://github.com/axboe
Baumann, A., Barham, P., Dagand, P.E., Harris, T., Isaacs, R., Peter, S., Roscoe, T., Schüpbach, A., Singhania, A.: The multikernel: a new OS architecture for scalable multicore systems. In: Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles SOSP 2009, pp. 29–44. ACM, New York (2009). http://doi.acm.org/10.1145/1629575.1629579
Belay, A., Bittau, A., Mashtizadeh, A., Terei, D., Mazières, D., Kozyrakis, C.: Dune: Safe user-level access to privileged cpu features. In: 10th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2012), pp. 335–348. USENIX, Hollywood (2012). https://www.usenix.org/conference/osdi12/technical-sessions/presentation/belay
Belay, A., Prekas, G., Klimovic, A., Grossman, S., Kozyrakis, C., Bugnion, E.: IX: a protected dataplane operating system for high throughput and low latency. In: 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2014), pp. 49–65. USENIX Association, Broomfield, October 2014. https://www.usenix.org/conference/osdi14/technical-sessions/presentation/belay
Brodal, G.S., Fagerberg, R.: Lower bounds for external memory dictionaries. In: Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2003, pp. 546–554. Society for Industrial and Applied Mathematics, Philadelphia (2003). http://dl.acm.org/citation.cfm?id=644108.644201
Caulfield, A.M., Mollov, T.I., Eisner, L.A., De, A., Coburn, J., Swanson, S.: Providing safe, user space access to fast, solid state disks. In: Proceedings of the Seventeenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XVII, pp. 387–400. ACM, New York (2012). http://doi.acm.org/10.1145/2150976.2151017
Chen, F., Mesnier, M., Hahn, S.: A protected block device for persistent memory. In: 2014 30th Symposium on Mass Storage Systems and Technologies (MSST), pp. 1–12, June 2014
Coburn, J., Caulfield, A.M., Akel, A., Grupp, L.M., Gupta, R.K., Jhala, R., Swanson, S.: NV-Heaps: making persistent objects fast and safe with next-generation, non-volatile memories. In: Proceedings of the Sixteenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XVI, pp. 105–118. ACM, New York (2011). http://doi.acm.org/10.1145/1950365.1950380
Condit, J., Nightingale, E.B., Frost, C., Ipek, E., Lee, B., Burger, D., Coetzee, D.: Better i/o through byte-addressable, persistent memory. In: Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, SOSP 2009, pp. 133–146. ACM, New York (2009). http://doi.acm.org/10.1145/1629575.1629589
Dieny, B., Sousa, R., Prenat, G., Ebels, U.: Spin-dependent phenomena and their implementation in spintronic devices. In: International Symposium on VLSI Technology, Systems and Applications VLSI-TSA 2008, pp. 70–71, April 2008
DPDK: Data plane development kit (2016). http://dpdk.org/
Dulloor, S.R., Kumar, S., Keshavamurthy, A., Lantz, P., Reddy, D., Sankaran, R., Jackson, J.: System software for persistent memory. In: Proceedings of the Ninth European Conference on Computer Systems, EuroSys 2014, pp. 15:1–15:15. ACM, New York (2014). http://doi.acm.org/10.1145/2592798.2592814
FusioIO: ioDrive2/ioDrive2 Duo Datasheet (2014). http://www.fusionio.com/load/-media-/2rezss/docsLibrary/FIO_DS_ioDrive2.pdf
Ho, Y., Huang, G., Li, P.: Nonvolatile memristor memory: device characteristics and design implications. In: IEEE/ACM International Conference on Computer-Aided Design - Digest of Technical Papers, ICCAD 2009, pp. 485–490, November 2009
Intel: Persistent memory block driver (PMBD) v0.9. (2013). https://github.com/linux-pmbd/pmbd
Kutch, P.: PCI-SIG SR-IOV primer: An introduction to SR-IOV technology, Intel application note, 321211–002 (2011)
Papagiannis, A., Saloustros, G., González-Férez, P., Bilas, A.: Tucana: Design and implementation of a fast and efficient scale-up key-value store. In: 2016 USENIX Annual Technical Conference (USENIX ATC 2016), USENIX Association, Denver, June 2016. https://www.usenix.org/conference/atc16/technical-sessions/presentation/papagiannis
Peter, S., Li, J., Zhang, I., Ports, D.R.K., Anderson, T., Krishnamurthy, A., Zbikowski, M., Woos, D.: Towards high-performance application-level storage management. In: 6th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage 2014). USENIX Association, Philadelphia, June 2014. https://www.usenix.org/conference/hotstorage14/workshop-program/presentation/peter
Peter, S., Li, J., Zhang, I., Ports, D.R.K., Woos, D., Krishnamurthy, A., Anderson, T., Roscoe, T.: Arrakis: the operating system is the control plane. In: 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2014), pp. 1–16. USENIX Association, Broomfield, October 2014. https://www.usenix.org/conference/osdi14/technical-sessions/presentation/peter
Raoux, S., Burr, G., Breitwisch, M., Rettner, C., Chen, Y., Shelby, R., Salinga, M., Krebs, D., Chen, S.H., Lung, H., Lam, C.: Phase-change random access memory: a scalable technology. IBM J. Res. Dev. 52(4.5), 465–479 (2008)
Rodeh, O.: B-trees, shadowing, and clones. Trans. Storage 3(4), 2: 1–2: 27 (2008). http://doi.acm.org/10.1145/1326542.1326544
Uhlig, R., Neiger, G., Rodgers, D., Santoni, A., Martins, F., Anderson, A., Bennett, S., Kagi, A., Leung, F., Smith, L.: Intel virtualization technology. Computer 38(5), 48–56 (2005)
Volos, H., Nalli, S., Panneerselvam, S., Varadarajan, V., Saxena, P., Swift, M.M.: Aerie: Flexible file-system interfaces to storage-class memory. In: Proceedings of the Ninth European Conference on Computer Systems, EuroSys 2014, pp. 14:1–14:14. ACM, New York (2014). http://doi.acm.org/10.1145/2592798.2592810
Volos, H., Tack, A.J., Swift, M.M.: Mnemosyne: lightweight persistent memory. In: Proceedings of the Sixteenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XVI, pp. 91–104, ACM, New York (2011). http://doi.acm.org/10.1145/1950365.1950379
Wu, X., Reddy, A.L.N.: SCMFS: a file system for storage class memory. In: Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2011, pp. 39:1–39:11. ACM, New York (2011). http://doi.acm.org/10.1145/2063384.2063436
Xu, J., Swanson, S.: Nova: a log-structured file system for hybrid volatile/non-volatile main memories. In: 14th USENIX Conference on File and Storage Technologies (FAST 2016), pp. 323–338. USENIX Association, Santa Clara, February 2016. https://www.usenix.org/conference/fast16/technical-sessions/presentation/xu
Acknowledgments
We thankfully acknowledge the support of the European Commission under the Horizon 2020 Framework Programme for Research and Innovation through the ExaNeSt project (grant agreement 671553).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Papagiannis, A., Saloustros, G., Marazakis, M., Bilas, A. (2016). User-Space I/O for \(\mu \)s-level Storage Devices. In: Taufer, M., Mohr, B., Kunkel, J. (eds) High Performance Computing. ISC High Performance 2016. Lecture Notes in Computer Science(), vol 9945. Springer, Cham. https://doi.org/10.1007/978-3-319-46079-6_44
Download citation
DOI: https://doi.org/10.1007/978-3-319-46079-6_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46078-9
Online ISBN: 978-3-319-46079-6
eBook Packages: Computer ScienceComputer Science (R0)