Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3589610.3596276acmconferencesArticle/Chapter ViewAbstractPublication PagescpsweekConference Proceedingsconference-collections
research-article

LUNAR: A Native Table Engine for Embedded Devices

Published: 13 June 2023 Publication History

Abstract

Embedded systems have evolved tremendously in recent years. We perform a study on SQLite and find that the multiple layers of abstraction drastically reduce bandwidth utilization. To minimize the bandwidth loss in the I/O path, we propose Lunar, a novel native table storage engine. Lunar performs a cross-layer design across the database and file system to avoid the pitfalls of multi-layer abstraction while providing SQL-compatible APIs. It employs a type-aware storage layout that considers the access patterns of different data types. Then, Lunar designs a variable-size allocator to reduce fragmentation and optimize RAM and I/O bandwidth usage. Further, considering the limited resources on embedded devices, Lunar employs a modular architecture that enables selecting modules on demand. It also offers optional consistency modes to make a trade-off between resource consumption and consistency. Experiments show that Lunar achieves higher bandwidth utilization, outperforming state-of-the-art approaches while consuming fewer resources.

References

[1]
Nitin Agrawal, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2009. Generating Realistic Impressions for File-System Benchmarking. In 7th USENIX Conference on File and Storage Technologies (FAST). 125–138.
[2]
Armink. 2020. FlashDB. https://github.com/armink/FlashDB
[3]
Joy Arulraj, Matthew Perron, and Andrew Pavlo. 2016. Write-Behind Logging. The VLDB Journal (VLDBJ), 10, 4 (2016), 337–348. https://doi.org/10.14778/3025111.3025116
[4]
Jens Axboe. 2007. Block I/O Layer Tracing. http://linux.die.net/man/8/blktrace
[5]
Jeff Bonwick. 1994. The Slab Allocator: An Object-Caching Kernel Memory Allocator. In USENIX Summer 1994 Technical Conference. 87–98.
[6]
Jeff Bonwick, Matt Ahrens, Val Henson, Mark Maybee, and Mark Shellenbaum. 2003. The Zettabyte File System. In 2nd USENIX Conference on File and Storage Technologies (FAST). 215–228.
[7]
ChaN. 2009. FatFS. http://elm-chan.org/fsw/ff/00index_e.html
[8]
Tseng-Yi Chen, Yuan-Hao Chang, Shuo-Han Chen, Nien-I Hsu, Hsin-Wen Wei, and Wei-Kuan Shih. 2017. On Space Utilization Enhancement of File Systems for Embedded Storage Systems. ACM Transactions on Embedded Computing Systems (TECS), 16, 3 (2017), 83:1–83:28. https://doi.org/10.1145/2820488
[9]
Jiajia Chu, Yunshan Tu, Yao Zhang, and Chuliang Weng. 2020. LATTE: A Native Table Engine on NVMe Storage. In 36th IEEE International Conference on Data Engineering (ICDE). 1225–1236. https://doi.org/10.1109/ICDE48307.2020.00110
[10]
Alexander Conway, Ainesh Bakshi, Yizheng Jiao, William Jannen, Yang Zhan, Jun Yuan, Michael A. Bender, Rob Johnson, Bradley C. Kuszmaul, Donald E. Porter, and Martin Farach-Colton. 2017. File Systems Fated for Senescence? Nonsense, Says Science!. In 15th USENIX Conference on File and Storage Technologies (FAST). 45–58.
[11]
Brian F. Cooper, Adam Silberstein, Erwin Tam, Raghu Ramakrishnan, and Russell Sears. 2010. Benchmarking Cloud Serving Systems with YCSB. In 1st ACM Symposium on Cloud Computing (SoCC). 143–154. https://doi.org/10.1145/1807128.1807152
[12]
Wei Dong, Jiamei Lv, Gonglong Chen, Yihui Wang, Huikang Li, Yi Gao, and Dinesh Bharadia. 2022. TinyNet: a Lightweight, Modular, and Unified Network Architecture for the Internet of Things. In 20th International Conference on Mobile Systems, Applications, and Services (MobiSys). 248–260. https://doi.org/10.1145/3498361.3538919
[13]
Kevin P. Gaffney, Martin Prammer, Laurence C. Brasfield, D. Richard Hipp, Dan R. Kennedy, and Jignesh M. Patel. 2022. SQLite: Past, Present, and Future. The VLDB Journal (VLDBJ), 15, 12 (2022), 3535–3547.
[14]
Sangwook Shane Hahn, Sungjin Lee, Cheng Ji, Li-Pin Chang, Inhyuk Yee, Liang Shi, Chun Jason Xue, and Jihong Kim. 2017. Improving File System Performance of Mobile Storage Systems Using a Decoupled Defragmenter. In 2017 USENIX Annual Technical Conference (ATC). 759–771.
[15]
Kyuhwa Han and Dongkun Shin. 2020. Command Queue-Aware Host I/O Stack for Mobile Flash Storage. Journal of Systems Architecture: Embedded Software Design (JSA), 109, 101758 (2020), https://doi.org/10.1016/j.sysarc.2020.101758
[16]
Sooman Jeong, Kisung Lee, Jungwoo Hwang, Seongjin Lee, and Youjip Won. 2013. Framework for Analyzing Android I/O Stack Behavior: From Generating the Workload to Analyzing the Trace. Future Internet, 5, 4 (2013), 591–610. https://doi.org/10.3390/fi5040591
[17]
Sooman Jeong, Kisung Lee, Seongjin Lee, Seoungbum Son, and Youjip Won. 2013. I/O Stack Optimization for Smartphones. In 2013 USENIX Annual Technical Conference (ATC). 309–320.
[18]
Cheng Ji, Li-Pin Chang, Sangwook Shane Hahn, Sungjin Lee, Riwei Pan, Liang Shi, Jihong Kim, and Chun Jason Xue. 2019. File Fragmentation in Mobile Devices: Measurement, Evaluation, and Treatment. IEEE Transactions on Mobile Computing (TMC), 18, 9 (2019), 2062–2076. https://doi.org/10.1109/TMC.2018.2869737
[19]
Cheng Ji, Li-Pin Chang, Riwei Pan, Chao Wu, Congming Gao, Liang Shi, Tei-Wei Kuo, and Chun Jason Xue. 2021. Pattern-Guided File Compression with User-Experience Enhancement for Log-Structured File System on Mobile Devices. In 19th USENIX Conference on File and Storage Technologies (FAST). 127–140.
[20]
Cheng Ji, Li-Pin Chang, Liang Shi, Chao Wu, Qiao Li, and Chun Jason Xue. 2016. An Empirical Study of File-System Fragmentation in Mobile Storage Systems. In 8th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage).
[21]
Cheng Ji, Riwei Pan, Li-Pin Chang, Liang Shi, Zongwei Zhu, Yu Liang, Tei-Wei Kuo, and Chun Jason Xue. 2020. Inspection and Characterization of App File Usage in Mobile Devices. ACM Transactions on Storage (TOS), 16, 4 (2020), 25:1–25:25. https://doi.org/10.1145/3404119
[22]
Rohan Kadekodi, Se Kwon Lee, Sanidhya Kashyap, Taesoo Kim, Aasheesh Kolli, and Vijay Chidambaram. 2019. SplitFS: Reducing Software Overhead in File Systems for Persistent Memory. In 27th ACM Symposium on Operating Systems Principles (SOSP). 494–508. https://doi.org/10.1145/3341301.3359631
[23]
Saurabh Kadekodi, Vaishnavh Nagarajan, and Gregory R. Ganger. 2018. Geriatrix: Aging What You See and What You Don’t See. A File System Aging Approach for Modern Storage Systems. In 2018 USENIX Annual Technical Conference (ATC). 691–704.
[24]
Woon-Hak Kang, Sang-Won Lee, Bongki Moon, Gi-Hwan Oh, and Changwoo Min. 2013. X-FTL: Transactional FTL for SQLite Databases. In 2013 ACM International Conference on Management of Data (SIGMOD). 97–108. https://doi.org/10.1145/2463676.2465326
[25]
Hyojun Kim, Nitin Agrawal, and Cristian Ungureanu. 2012. Revisiting Storage for Smartphones. ACM Transactions on Storage (TOS), 8, 4 (2012), 14:1–14:25. https://doi.org/10.1145/2385603.2385607
[26]
Simon Kuenzer, Vlad-Andrei Bădoiu, Hugo Lefeuvre, Sharan Santhanam, Alexander Jung, Gaulthier Gain, Cyril Soldani, Costin Lupu, Stefan Teodorescu, Costi Răducanu, Cristian Banu, Laurent Mathy, Răzvan Deaconescu, Costin Raiciu, and Felipe Huici. 2021. Unikraft: Fast, Specialized Unikernels the Easy Way. In 16th European Conference on Computer Systems (EuroSys). 376–394. https://doi.org/10.1145/3447786.3456248
[27]
Changman Lee, Dongho Sim, Joo Young Hwang, and Sangyeun Cho. 2015. F2FS: A New File System for Flash Storage. In 13th USENIX Conference on File and Storage Technologies (FAST). 273–286.
[28]
Borui Li, Hongchang Fan, Yi Gao, and Wei Dong. 2022. Bringing Webassembly to Resource-constrained IoT Devices for Seamless Device-cloud Integration. In 20th Annual International Conference on Mobile Systems, Applications and Services (MobiSys). 261–272. https://doi.org/10.1145/3498361.3538922
[29]
Yu Liang, Cheng Ji, Chenchen Fu, Rachata Ausavarungnirun, Qiao Li, Riwei Pan, Siyu Chen, Liang Shi, Tei-Wei Kuo, and Chun Jason Xue. 2021. iTRIM: I/O-Aware TRIM for Improving User Experience on Mobile Devices. IEEE Transactions on Computer-Aided Design of Integrated Circuits And System (TCAD), 40, 9 (2021), 1782–1795. https://doi.org/10.1109/TCAD.2020.3027656
[30]
Duo Liu, Yi Wang, Zhiwei Qin, Zili Shao, and Yong Guan. 2012. A Space Reuse Strategy for Flash Translation Layers in SLC NAND Flash Memory Storage Systems. IEEE Transactions on Very Large Scale Integration Systems (TVLSI), 20, 6 (2012), 1094–1107. https://doi.org/10.1109/TVLSI.2011.2142015
[31]
Shaun Liu and Jack Heller. 1974. A Record Oriented, Grammar Driven Data Translation Model. In 1974 ACM Workshop on Data Description, Access and Control (SIGMOD). 171–189. https://doi.org/10.1145/800296.811511
[32]
Youyou Lu, Jiwu Shu, and Weimin Zheng. 2013. Extending the Lifetime of Flash-Based Storage Through Reducing Write Amplification from File Systems. In 11th USENIX conference on File and Storage Technologies (FAST). 257–270.
[33]
Matt Mackall. 2005. slob: introduce the SLOB allocator. https://lwn.net/Articles/157944/
[34]
Avantika Mathur, Mingming Cao, Suparna Bhattacharya, Andreas Dilger, Alex Tomas, and Laurent Vivier. 2007. The New Ext4 Filesystem: Current Status and Future Plans. Linux Symposium, 2 (2007), 21–33.
[35]
Marshall K. McKusick, William N. Joy, Samuel J. Leffler, and Robert S. Fabry. 1984. A Fast File System for UNIX. ACM Transactions on Computer Systems (TOCS), 2, 3 (1984), 181–197. https://doi.org/10.1145/989.990
[36]
Akira Moroo. 2018. SQLite3 Benchmark. https://github.com/ukontainer/sqlite-bench
[37]
Daejun Park and Dongkun Shin. 2017. iJournaling: Fine-Grained Journaling for Improving the Latency of Fsync System Call. In 2017 USENIX Annual Technical Conference (ATC). 787–798.
[38]
Hongwei Qin, Dan Feng, Wei Tong, Yutong Zhao, Sheng Qiu, Fei Liu, and Shu Li. 2021. Better Atomic Writes by Exposing the Flash out-of-band Area to File Systems. In 22nd ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems (LCTES). 12–23. https://doi.org/10.1145/3461648.3463843
[39]
Mark Raasveldt and Hannes Mühleisen. 2019. DuckDB: an Embeddable Analytical Database. In 2019 ACM Conference on Management of Data (SIGMOD). 1981–1984. https://doi.org/10.1145/3299869.3320212
[40]
Kai Shen, Stan Park, and Meng Zhu. 2014. Journaling of Journal Is (almost) Free. In 12th USENIX conference on File and Storage Technologies (FAST). 287–293.
[41]
Zhaoyan Shen, Yuanjing Shi, Zili Shao, and Yong Guan. 2019. An Efficient LSM-tree-based SQLite-like Database Engine for Mobile Devices. IEEE Transactions on Computer-Aided Design of Integrated Circuits And System (TCAD), 38, 9 (2019), 1635–1647. https://doi.org/10.1109/TCAD.2018.2855179
[42]
Sipeed. 2021. Maix Bit. https://wiki.sipeed.com/hardware/en/maix/index.html
[43]
RT-Thread Development Team. 2019. RT-Thread. https://www.rt-thread.io/document/site/
[44]
SQLite Development Team. 2017. SQLite. https://www.sqlite.org/index.html
[45]
Runyu Zhang, Duo Liu, Xianzhang Chen, Xiongxiong She, Chaoshu Yang, Yujuan Tan, Zhaoyan Shen, and Zili Shao. 2020. LOFFS: A Low-Overhead File System for Large Flash Memory on Embedded Devices. In 57th ACM/IEEE Design Automation Conference (DAC). 1–6. https://doi.org/10.1109/DAC18072.2020.9218635

Cited By

View all
  • (2024)Structured storage for ubiquitous operating systemsSCIENTIA SINICA Informationis10.1360/SSI-2022-041554:3(461)Online publication date: 12-Mar-2024
  • (2024)TEngine: A Native Distributed Table Storage Engine2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00290(3782-3795)Online publication date: 13-May-2024

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
LCTES 2023: Proceedings of the 24th ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems
June 2023
147 pages
ISBN:9798400701740
DOI:10.1145/3589610
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 June 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. consistency
  2. embedded devices
  3. modularity
  4. storage management

Qualifiers

  • Research-article

Funding Sources

Conference

LCTES '23

Acceptance Rates

Overall Acceptance Rate 116 of 438 submissions, 26%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)70
  • Downloads (Last 6 weeks)1
Reflects downloads up to 23 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Structured storage for ubiquitous operating systemsSCIENTIA SINICA Informationis10.1360/SSI-2022-041554:3(461)Online publication date: 12-Mar-2024
  • (2024)TEngine: A Native Distributed Table Storage Engine2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00290(3782-3795)Online publication date: 13-May-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media