short-paper

Hermes: enabling energy-efficient IoT networks with generalized deduplication

Authors:

Valerio SchiavoniAuthors Info & Claims

DEBS '20: Proceedings of the 14th ACM International Conference on Distributed and Event-based Systems

Pages 133 - 136

https://doi.org/10.1145/3401025.3404098

Published: 15 July 2020 Publication History

Get Access

Abstract

The Internet of Things (IoT) is connecting a massive number of devices that generate a growing amount of data to be transmitted over the network. This traffic growth is expected to continue. Generalized deduplication (GD) is a novel technique to effectively compress the data to (a) reduce the data storage cost by identifying similar data chunks, and (b) reduce the pressure on the network infrastructure. This paper presents Hermes, an application-level protocol for the data-plane that can operate using GD as well as classic deduplication. Hermes significantly reduces the data transmission traffic while effectively decreasing the energy footprint, a key goal in many IoT deployments. We fully implemented Hermes, evaluated its performance using consumer-grade IoT devices (e.g., Raspberry Pi 4B), and highlighted key tradeoffs to be considered to manage real-world workloads. Several fold to several order of magnitude gains over standard compressors and deduplication are achievable.

References

[1]

2013. Indian Dataset for Ambient Water and Energy. http://iawe.github.io/ Accessed: 2020-02-03.

Google Scholar

[2]

Nipun Batra, Manoj Gulati, Amarjeet Singh, and Mani B Srivastava. 2013. It's Different: Insights into home energy consumption in India. In ACM Workshop on Embedded Syst. for Energy-Efficient Build. 1--8.

Digital Library

Google Scholar

[3]

L. Peter Deutsch. 1996. DEFLATE Compressed Data Format Specification version 1.3. RFC 1951. RFC Editor. http://www.rfc-editor.org/rfc/rfc1951.txt http://www.rfc-editor.org/rfc/rfc1951.txt.

Digital Library

Google Scholar

[4]

Christian Göttel, Lars Nielsen, Niloofar Yazdani, Pascal Felber, Daniel E. Lucani, and Valerio Schiavoni. 2020. Hermes: Enabling Energy-efficient IoT Networks with Generalized Deduplication. arXiv:2005.11158

Google Scholar

[5]

Abdallah Jarwan, Ayman Sabbah, and Mohamed Ibnkahla. 2019. Data transmission reduction schemes in WSNs for efficient IoT systems. IEEE Journal on Selected Areas in Comm. 37, 6 (2019), 1307--1324.

Crossref

Google Scholar

[6]

Galit Mendelson. 2004. All you need to know about Power over Ethernet (PoE) and the IEEE 802.3 af Standard. Internet Citation,[Online] Jun (2004).

Google Scholar

[7]

Jorge Castiñeira Moreira and Patrick Guy Farrell. 2006. Essentials of error-control coding. John Wiley & Sons.

Google Scholar

[8]

Lars Nielsen, Rasmus Vestergaard, Niloofar Yazdani, Prasad Talasila, Daniel E Lucani, and Márton Sipos. 2019. Alexandria: A Proof-of-concept Implementation and Evaluation of Generalised Data Deduplication. In IEEE Global Communications Conference (GLOBECOM).

Google Scholar

[9]

Rasmus Vestergaard, Daniel E Lucani, and Qi Zhang. 2019. Generalized Deduplication: Lossless Compression for Large Amounts of Small IoT Data. In European Wireless Conference. IEEE.

Google Scholar

[10]

Terry A. Welch. 1984. A technique for high-performance data compression. Computer 6 (1984), 8--19.

Digital Library

Google Scholar

[11]

Niloofar Yazdani and Daniel E Lucani. 2019. Protocols to Reduce CPS Sensor Traffic using Smart Indexing and Edge Computing Support. In IEEE GLOBECOM - Workshop on Edge Comp. for Cyber Phys. Sys. IEEE.

Google Scholar

[12]

J. Ziv and A. Lempel. 1977. A universal algorithm for sequential data compression. IEEE Transactions on Information Theory 23, 3 (May 1977), 337--343.

Digital Library

Google Scholar

Cited By

View all

Hurst ALucani DZhang Q(2024)PairwiseHist: Fast, Accurate and Space-Efficient Approximate Query Processing with Data CompressionProceedings of the VLDB Endowment10.14778/3648160.364818117:6(1432-1445)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.14778/3648160.3648181
Chen MLu HXu XTang XFan QLou C(2022)Novel Deep Packet Compression For Industrial Internet of Things2022 IEEE Globecom Workshops (GC Wkshps)10.1109/GCWkshps56602.2022.10008589(1628-1633)Online publication date: 4-Dec-2022
https://doi.org/10.1109/GCWkshps56602.2022.10008589
Das AChakraborty SChakraborty S(2022)Where do all my smart home data go? Context-aware data generation and forwarding for edge-based microservices over shared IoT infrastructureFuture Generation Computer Systems10.1016/j.future.2022.03.027134(204-218)Online publication date: Sep-2022
https://doi.org/10.1016/j.future.2022.03.027
Show More Cited By

Index Terms

Hermes: enabling energy-efficient IoT networks with generalized deduplication

Recommendations

HeRMES: High-Performance Reliable MRAM-Enabled Storage
HOTOS '01: Proceedings of the Eighth Workshop on Hot Topics in Operating Systems

Abstract: Magnetic RAM (MRAM) is a new memory technology with access and cost characteristics comparable to those of conventional dynamic RAM (DRAM)and the non-volatility of magnetic media such as disk. Simply replacing DRAM with MRAM will make main ...
TREAT - Two wRongs makE A righT: efficient distributed storage and queries of IoT datasets with erasure coding and compression
DEBS '24: Proceedings of the 18th ACM International Conference on Distributed and Event-based Systems

Erasure coding in distributed multi-cloud data storage increases availability, durability and security, but it also makes data analytics inefficient since the whole dataset must be reconstructed to answer a query, even if the result set is a small ...
HERMES: embedded file system design for A/V application

Embedded systems such as PVR, set-top box, HDTV put unique demand on I/O subsystem design. Underlying software, particularly file system, needs to be elaborately designed so that it can meet tight constraints of consumer electronics platform: ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

DEBS '20: Proceedings of the 14th ACM International Conference on Distributed and Event-based Systems

July 2020

244 pages

ISBN:9781450380287

DOI:10.1145/3401025

Conference Chair:
Julien Gascon-Samson
ÉTS
,
General Chairs:
Kaiwen Zhang
ÉTS Montréal
,
Khuzaima Daudjee
University of Waterloo
,
Bettina Kemme
McGill University

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 July 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Conference

DEBS '20

Sponsor:

DEBS '20: The 14th ACM International Conference on Distributed and Event-based Systems

July 13 - 17, 2020

Quebec, Montreal, Canada

Acceptance Rates

DEBS '20 Paper Acceptance Rate 11 of 43 submissions, 26%;

Overall Acceptance Rate 145 of 583 submissions, 25%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
161
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)1

Reflects downloads up to 24 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Hurst ALucani DZhang Q(2024)PairwiseHist: Fast, Accurate and Space-Efficient Approximate Query Processing with Data CompressionProceedings of the VLDB Endowment10.14778/3648160.364818117:6(1432-1445)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.14778/3648160.3648181
Chen MLu HXu XTang XFan QLou C(2022)Novel Deep Packet Compression For Industrial Internet of Things2022 IEEE Globecom Workshops (GC Wkshps)10.1109/GCWkshps56602.2022.10008589(1628-1633)Online publication date: 4-Dec-2022
https://doi.org/10.1109/GCWkshps56602.2022.10008589
Das AChakraborty SChakraborty S(2022)Where do all my smart home data go? Context-aware data generation and forwarding for edge-based microservices over shared IoT infrastructureFuture Generation Computer Systems10.1016/j.future.2022.03.027134(204-218)Online publication date: Sep-2022
https://doi.org/10.1016/j.future.2022.03.027
Nielsen LLucani D(2021)Hekate a tool for gauging Data Deduplication Performance2021 IEEE 6th International Conference on Smart Cloud (SmartCloud)10.1109/SmartCloud52277.2021.00019(67-72)Online publication date: Nov-2021
https://doi.org/10.1109/SmartCloud52277.2021.00019
Vaucher SYazdani NFelber PLucani DSchiavoni VHan DFeldmann A(2020)ZipLineProceedings of the 16th International Conference on emerging Networking EXperiments and Technologies10.1145/3386367.3431302(399-405)Online publication date: 23-Nov-2020
https://dl.acm.org/doi/10.1145/3386367.3431302
Yazdani NNielsen LLucani D(2020)Memory-aware Online Compression of CAN Bus Data for Future Vehicular SystemsGLOBECOM 2020 - 2020 IEEE Global Communications Conference10.1109/GLOBECOM42002.2020.9348074(1-6)Online publication date: Dec-2020
https://doi.org/10.1109/GLOBECOM42002.2020.9348074

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

HeRMES: High-Performance Reliable MRAM-Enabled Storage

TREAT - Two wRongs makE A righT: efficient distributed storage and queries of IoT datasets with erasure coding and compression

HERMES: embedded file system design for A/V application