Abstract
One of the use cases of mobile networks that can be considered for use in Beyond 5G is a massive IoT environment where many IoT (Internet of Things) terminals with low power consumption and computing power are connected. In order to efficiently use network resources in this environment, it is necessary to compress and reduce the amount of data uploaded by a large number of IoT terminals. In this study, we consider data compression in a Massive IoT environment using edge servers, assuming a Multi-access Edge Computing (MEC) scenario. In particular, we consider the application of “Generalized Deduplication (GD)", a stream data compression method based on duplicate deletion, which has been attracting attention in recent years for its lightweight and efficient compression of IoT sensing data. The basic GD algorithm assumes one-to-one stream transmission and reception. In this report, we propose an extension of the GD algorithm that is suitable for one-to-multi (edge server and IoT terminals) MEC environments and has more efficient performance. Specifically, we investigate dictionary construction for the GD utilization in a one-to-multi environment and show a basic evaluation of the efficiency of the proposed algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
In practice, the dictionary-based replacement is performed simultaneously with the elimination.
- 3.
“registry" is another name of the “dictionary".
- 4.
In practice, a separator bit is required to clearly indicate either the basis data or the index data.
- 5.
In the illustration of the explanation, it is represented as “main”.
- 6.
Individual dictionaries are dictionaries that play a supplementary role because they preserve the relationship between indices and indices, unlike the original dictionaries. Therefore, on the illustration for the explanation, it is denoted as “sub”.
References
Atzori, L., Iera, A., Morabito, G.: The internet of things: a survey. Comput. Netw. 54(15), 2787–2805 (2010). https://doi.org/10.1016/j.comnet.2010.05.010
Bellare, M., Keelveedhi, S., Ristenpart, T.: Message-locked encryption and secure deduplication. In: Johansson, T., Nguyen, P.Q. (eds.) EUROCRYPT 2013. LNCS, vol. 7881, pp. 296–312. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38348-9_18
Kurihara, J.: rust-gd: a rust implementation of generalized deduplication (2022). https://github.com/junkurihara/rust-gd
Minerva, R., Biru, A., Rotondi, D.: Towards a definition of the Internet of Things (IoT). IEEE Internet Initiative 1(1), 1–86 (2015)
Sehat, H., Kloborg, A.L., Mørup, C., Pagnin, E., Lucani, D.E.: Bonsai: a generalized look at dual deduplication. arXiv:2202.13925v2 (2022)
Sehat, H., Pagnin, E., Lucani, D.E.: Yggdrasil: privacy-aware dual deduplication in multi client settings. In: the Proceedings of IEEE International Conference on Communications, pp. 1–6 (2021)
Vaucher, S., Yazdani, N., Felber, P., Lucani, D.E., Schiavoni, V.: Zipline: In-network compression at line speed (2021). arXiv:2101.05323
Vestergaard, R., Lucani, D.E., Zhang, Q.: Generalized deduplication: lossless compression for large amounts of small IOT data. In: European Wireless 2019; 25th European Wireless Conference, pp. 1–5 (2019)
Vestergaard, R., Zhang, Q., Lucani, D.E.: Generalized deduplication: bounds, convergence, and asymptotic properties. In: 2019 IEEE Global Communications Conference (GLOBECOM), pp. 1–6 (2019). https://doi.org/10.1109/GLOBECOM38437.2019.9014012
Vestergaard, R., Zhang, Q., Lucani, D.E.: Lossless compression of time series data with generalized deduplication. In: Proceedings of IEEE GLOBECOM 2019, pp. 1–6 (2019)
Watanabe, R., Kubota, A., Kurihara, J.: Resource authorization patterns on edge computing. In: IN2020-68, vol. 120, pp. 85–90. IEICE (2021)
Xia, W., et al.: A comprehensive study of the past, present, and future of data deduplication. Proc. IEEE 104(9), 1681–1710 (2016)
Zhou, Y., Yu, Z., Gu, L., Feng, D.: An efficient encrypted deduplication scheme with security-enhanced proof of ownership in edge computing. BenchCouncil transactions on benchmarks, Stand. Eval. 2(2), 100,062 (2022). https://doi.org/10.1016/j.tbench.2022.100062
Ziv, J., Lempel, A.: A universal algorithm for sequential data compression. IEEE Trans. Inf. Theory 23(3), 337–343 (1977)
Ziv, J., Lempel, A.: Compression of individual sequences via variable-rate coding. IEEE Trans. Inf. Theory 24(5), 530–536 (1978)
Acknowledgements
This work was supported in part by NICT22401 and JSPS KAKENHI Grant Number JP22K11994, JP21H03442, JP20K23329.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Watanabe, R., Kubota, A., Kurihara, J. (2023). Application of Generalized Deduplication Techniques in Edge Computing Environments. In: Barolli, L. (eds) Advanced Information Networking and Applications. AINA 2023. Lecture Notes in Networks and Systems, vol 655. Springer, Cham. https://doi.org/10.1007/978-3-031-28694-0_55
Download citation
DOI: https://doi.org/10.1007/978-3-031-28694-0_55
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-28693-3
Online ISBN: 978-3-031-28694-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)