research-article

Privacy-Preserved Task Offloading in Mobile Blockchain With Deep Reinforcement Learning

Authors:

Dinh C. Nguyen,

Pubudu N. Pathirana,

Aruna SeneviratneAuthors Info & Claims

IEEE Transactions on Network and Service Management, Volume 17, Issue 4

Pages 2536 - 2549

https://doi.org/10.1109/TNSM.2020.3010967

Published: 01 December 2020 Publication History

Abstract

Blockchain technology with its secure, transparent and decentralized nature has been recently employed in many mobile applications. However, the process of executing extensive tasks such as computation-intensive data applications and blockchain mining requires high computational and storage capability of mobile devices, which would hinder blockchain applications in mobile systems. To meet this challenge, we propose a mobile edge computing (MEC) based blockchain network where multi-mobile users (MUs) act as miners to offload their data processing tasks and mining tasks to a nearby MEC server via wireless channels. Specially, we formulate task offloading, user privacy preservation and mining profit as a joint optimization problem which is modelled as a Markov decision process, where our objective is to minimize the long-term system offloading utility and maximize the privacy levels for all blockchain users. We first propose a reinforcement learning (RL)-based offloading scheme which enables MUs to make optimal offloading decisions based on blockchain transaction states, wireless channel qualities between MUs and MEC server and user’s power hash states. To further improve the offloading performances for larger-scale blockchain scenarios, we then develop a deep RL algorithm by using deep Q-network which can efficiently solve large state space without any prior knowledge of the system dynamics. Experiment and simulation results show that the proposed RL-based offloading schemes significantly enhance user privacy, and reduce the energy consumption as well as computation latency with minimum offloading costs in comparison with the benchmark offloading schemes.

References

[1]

T. M. Fernandez-Carames and P. Fraga-Lamas, “A review on the use of blockchain for the Internet of Things,” IEEE Access, vol. 6, pp. 32979–33001, 2018.

[2]

D. C. Nguyen, P. N. Pathirana, M. Ding, and A. Seneviratne, “Integration of blockchain and cloud of things: Architecture, applications and challenges,” Aug. 2019. [Online]. Available: http://www.arXiv:1908.09058

[3]

K. Suankaewmanee, D. T. Hoang, D. Niyato, S. Sawadsitang, P. Wang, and Z. Han, “Performance analysis and application of mobile blockchain,” in Proc. Int. Conf. Comput. Netw. Commun. (ICNC), Maui, HI, USA, Mar. 2018, pp. 642–646.

[4]

D. C. Nguyen, P. N. Pathirana, M. Ding, and A. Seneviratne, “Blockchain for secure EHRs sharing of mobile cloud based E-health systems,” IEEE Access, vol. 7, pp. 66792–66806, 2019.

[5]

R. Pass and E. Shi, “FruitChains: A fair blockchain,” in Proc. ACM Symp. Principles Distrib. Comput., Washington, DC, USA, Jul. 2017, pp. 315–324.

[6]

W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, “Edge computing: Vision and challenges,” IEEE Internet Things J., vol. 3, no. 5, pp. 637–646, Oct. 2016.

[7]

P. Mach and Z. Becvar, “Mobile edge computing: A survey on architecture and computation offloading,” IEEE Commun. Surveys Tuts., vol. 19, no. 3, pp. 1628–1656, 3rd Quart., 2017.

Digital Library

[8]

Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, “A survey on mobile edge computing: The communication perspective,” IEEE Commun. Surveys Tuts., vol. 19, no. 4, pp. 2322–2358, 4th Quart., 2017.

[9]

H. Guo, J. Liu, J. Zhang, W. Sun, and N. Kato, “Mobile-edge computation offloading for ultradense IoT networks,” IEEE Internet Things J., vol. 5, no. 6, pp. 4977–4988, Dec. 2018.

[10]

M. Chen and Y. Hao, “Task offloading for mobile edge computing in software defined ultra-dense network,” IEEE J. Sel. Areas Commun., vol. 36, no. 3, pp. 587–597, Mar. 2018.

[11]

S. Yu, X. Wang, and R. Langar, “Computation offloading for mobile edge computing: A deep learning approach,” in Proc. 28th Annu. Int. Symp. Pers. Indoor Mobile Radio Commun. (PIMRC), Montreal, QC, Canada, Oct. 2017, pp. 1–6.

[12]

X. He, H. Xing, Y. Chen, and A. Nallanathan, “Energy-efficient mobile-edge computation offloading for applications with shared data,” in Proc. IEEE Global Commun. Conf. (GLOBECOM), Abu Dhabi, UAE, Dec. 2018, pp. 1–6.

[13]

S. Liet al., “Joint admission control and resource allocation in edge computing for Internet of Things,” IEEE Netw., vol. 32, no. 1, pp. 72–79, Jan./Feb. 2018.

[14]

I.-S. Comsaet al., “Towards 5G: A reinforcement learning-based scheduling solution for data traffic management,” IEEE Trans. Netw. Service Manag., vol. 15, no. 4, pp. 1661–1675, Dec. 2018.

Digital Library

[15]

R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning Series), 2nd ed. Cambridge, MA, USA: MIT Press, 2018.

[16]

K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, “Deep reinforcement learning: A brief survey,” IEEE Signal Process. Mag., vol. 34, no. 6, pp. 26–38, Nov. 2017.

[17]

V. Mnihet al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, Feb. 2015.

[18]

X. Chen, H. Zhang, C. Wu, S. Mao, Y. Ji, and M. Bennis, “Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning,” IEEE Internet Things J., vol. 6, no. 3, pp. 4005–4018, Jun. 2019.

[19]

J. Li, H. Gao, T. Lv, and Y. Lu, “Deep reinforcement learning based computation offloading and resource allocation for MEC,” in Proc. IEEE Wireless Commun. Netw. Conf. (WCNC), Barcelona, Spain, Apr. 2018, pp. 1–6.

[20]

M. Min, L. Xiao, Y. Chen, P. Cheng, D. Wu, and W. Zhuang, “Learning-based computation offloading for IoT devices with energy harvesting,” IEEE Trans. Veh. Technol., vol. 68, no. 2, pp. 1930–1941, Feb. 2019.

[21]

Z. Xiong, Y. Zhang, D. Niyato, P. Wang, and Z. Han, “When mobile blockchain meets edge computing,” IEEE Commun. Mag., vol. 56, no. 8, pp. 33–39, Aug. 2018.

Digital Library

[22]

M. Liu, F. R. Yu, Y. Teng, V. C. M. Leung, and M. Song, “Computation offloading and content caching in wireless blockchain networks with mobile edge computing,” IEEE Trans. Veh. Technol., vol. 67, no. 11, pp. 11008–11021, Nov. 2018.

[23]

Y. Wu, X. Chen, J. Shi, K. Ni, L. Qian, L. Huang, and K. Zhang, “Optimal computational power allocation in multi-acces mobile edge computing for blockchain,” Sensors, vol. 18, no. 10, p. 3472, Oct. 2018.

[24]

M. Liu, F. R. Yu, Y. Teng, V. C. M. Leung, and M. Song, “Distributed resource allocation in blockchain-based video streaming systems with mobile edge computing,” IEEE Trans. Wireless Commun., vol. 18, no. 1, pp. 695–708, Jan. 2019.

Digital Library

[25]

J. Zhang, B. Chen, Y. Zhao, X. Cheng, and F. Hu, “Data security and privacy-preserving in edge computing paradigm: Survey and open issues,” IEEE Access, vol. 6, pp. 18209–18237, 2018.

[26]

T. Salman, M. Zolanvari, A. Erbad, R. Jain, and M. Samaka, “Security services using blockchains: A state of the art survey,” IEEE Commun. Surveys Tuts., vol. 21, no. 1, pp. 858–880, 1st Quart., 2019.

[27]

D. K. Tosh, S. Shetty, X. Liang, C. A. Kamhoua, K. A. Kwiat, and L. Njilla, “Security implications of blockchain cloud with analysis of block withholding attack,” in Proc. 17th IEEE/ACM Int. Symp. Clust Cloud Grid Comput. (CCGRID), Madrid, Spain, May 2017, pp. 458–467.

[28]

X. He, J. Liu, R. Jin, and H. Dai, “Privacy-aware offloading in mobile-edge computing,” in Proc. IEEE Global Commun. Conf., Singapore, Dec. 2017, pp. 1–6.

[29]

M. Minet al., “Learning-based privacy-aware offloading for healthcare IoT with energy harvesting,” IEEE Internet Things J., vol. 6, no. 3, pp. 4307–4316, Jun. 2019.

[30]

R. Recabarren and B. Carbunar, “Hardening stratum, the bitcoin pool mining protocol,” Mar. 2017. [Online]. Available: http://www.arXiv:1703.06545

[31]

T. He, E. N. Ciftcioglu, S. Wang, and K. S. Chan, “Location privacy in mobile edge clouds: A chaff-based approach,” IEEE J. Sel. Areas Commun., vol. 35, no. 11, pp. 2625–2636, Nov. 2017.

[32]

A. P. Ozisik, G. Bissias, and B. Levine, “Estimation of miner hash rates and consensus on blockchains (draft),” Jun. 2017. [Online]. Available: http://www.arXiv:1707.00082

[33]

D. C. Nguyen, P. N. Pathirana, M. Ding, and A. Seneviratne, “Blockchain for 5G and beyond networks: A state of the art survey,” J. Netw. Comput. Appl., vol. 166, Sep. 2020, Art. no.

[34]

Z. Li, Z. Yang, S. Xie, W. Chen, and K. Liu, “Credit-based payments for fast computing resource trading in edge-assisted Internet of Things,” IEEE Internet Things J., vol. 6, no. 4, pp. 6606–6617, Aug. 2019.

[35]

N. Houy, “The bitcoin mining game,” Ledger, vol. 1, pp. 53–68, Dec. 2016.

[36]

X. Qiu, L. Liu, W. Chen, Z. Hong, and Z. Zheng, “Online deep reinforcement learning for computation offloading in blockchain-empowered mobile edge computing,” IEEE Trans. Veh. Technol., vol. 68, no. 8, pp. 8050–8062, Aug. 2019.

[37]

D. C. Nguyen, P. N. Pathirana, M. Ding, and A. Seneviratne, “Secure computation offloading in blockchain based IoT networks with deep reinforcement learning,” Aug. 2019. [Online]. Available: http://www.arXiv:1908.07466

[38]

S. Biswas, K. Sharif, F. Li, B. Nour, and Y. Wang, “A scalable blockchain framework for secure transactions in IoT,” IEEE Internet Things J., vol. 6, no. 3, pp. 4650–4659, Jun. 2019.

[39]

Q. Luo, C. Li, T. H. Luan, and W. Shi, “Collaborative data scheduling for vehicular edge computing via deep reinforcement learning,” IEEE Internet Things J., early access, Mar. 26, 2020. 10.1109/JIOT.2020.2983660.

Cited By

Hu XLiu CWan JLi L(2024)Resource Pricing and Allocation for the Internet of Vehicles BlockchainProceedings of the 2024 International Conference on Computer and Multimedia Technology10.1145/3675249.3675325(442-446)Online publication date: 24-May-2024
https://dl.acm.org/doi/10.1145/3675249.3675325
Li CJiang KZhang YJiang LLuo YWan S(2024)Deep Reinforcement Learning-based Mining Task Offloading Scheme for Intelligent Connected Vehicles in UAV-aided MECACM Transactions on Design Automation of Electronic Systems10.1145/365345129:3(1-29)Online publication date: 3-May-2024
https://dl.acm.org/doi/10.1145/3653451
Zhou ZOnireti OXu HZhang LImran M(2024)AI and Blockchain Enabled Future Wireless Networks: A Survey And OutlookDistributed Ledger Technologies: Research and Practice10.1145/36443693:3(1-30)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3644369
Show More Cited By

Index Terms

Privacy-Preserved Task Offloading in Mobile Blockchain With Deep Reinforcement Learning
1. Information systems

Index terms have been assigned to the content through auto-classification.

Recommendations

Deep Reinforcement Learning Based Task Offloading Algorithm for Mobile-edge Computing Systems
ICMAI '19: Proceedings of the 2019 4th International Conference on Mathematics and Artificial Intelligence

Mobile-edge computing(MEC) is deemed to a promising paradigm. By deploying high-performance servers on the mobile access network side, MEC can provide auxiliary computing power for mobile devices, greatly reducing the computing pressure of mobile ...
Modelling Task Offloading Mobile Edge Computing
ICCDE '22: Proceedings of the 2022 8th International Conference on Computing and Data Engineering

With the rapid growth of mobile devices (such as smart phones and IoT devices) and the upcoming 5G era, it has been considered that edge computing will play a significant role, which together with the Cloud server forms the Mobile Edge Computing (MEC) ...
A Deep Reinforcement Learning Approach Towards Computation Offloading for Mobile Edge Computing
Human Centered Computing
Abstract
In order to improve the quality of service for users and reduce the energy consumption of the cloud computing environment, Mobile Edge Computing (MEC) is a promising paradigm by providing computing resources which is close to the end device in ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Network and Service Management

IEEE Transactions on Network and Service Management Volume 17, Issue 4

Dec. 2020

759 pages

ISSN:1932-4537

Issue’s Table of Contents

1932-4537 © 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Press

Publication History

Published: 01 December 2020

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

34
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hu XLiu CWan JLi L(2024)Resource Pricing and Allocation for the Internet of Vehicles BlockchainProceedings of the 2024 International Conference on Computer and Multimedia Technology10.1145/3675249.3675325(442-446)Online publication date: 24-May-2024
https://dl.acm.org/doi/10.1145/3675249.3675325
Li CJiang KZhang YJiang LLuo YWan S(2024)Deep Reinforcement Learning-based Mining Task Offloading Scheme for Intelligent Connected Vehicles in UAV-aided MECACM Transactions on Design Automation of Electronic Systems10.1145/365345129:3(1-29)Online publication date: 3-May-2024
https://dl.acm.org/doi/10.1145/3653451
Zhou ZOnireti OXu HZhang LImran M(2024)AI and Blockchain Enabled Future Wireless Networks: A Survey And OutlookDistributed Ledger Technologies: Research and Practice10.1145/36443693:3(1-30)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3644369
Fan W(2024)Blockchain-Secured Task Offloading and Resource Allocation for Cloud-Edge-End Cooperative NetworksIEEE Transactions on Mobile Computing10.1109/TMC.2023.334281723:8(8092-8110)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TMC.2023.3342817
Wang ZSun YLiu DHu JPang XHu YRen K(2024)Location Privacy-Aware Task Offloading in Mobile Edge ComputingIEEE Transactions on Mobile Computing10.1109/TMC.2023.325455323:3(2269-2283)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1109/TMC.2023.3254553
Habbal AHamouda HAlnajim AKhan SAlrifaie M(2024)Privacy as a LifestyleJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2024.10203936:4Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.jksuci.2024.102039
Zhang HWang JZhang HBu C(2024)Security computing resource allocation based on deep reinforcement learning in serverless multi-cloud edge computingFuture Generation Computer Systems10.1016/j.future.2023.09.016151:C(152-161)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.future.2023.09.016
Zhai ZShen SMao Y(2024)An explainable deep reinforcement learning algorithm for the parameter configuration and adjustment in the consortium blockchainEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107606129:COnline publication date: 16-May-2024
https://dl.acm.org/doi/10.1016/j.engappai.2023.107606
Kumar NAli R(2024)Blockchain-enabled authentication framework for Maritime Transportation System empowered by 6G-IoTComputer Networks: The International Journal of Computer and Telecommunications Networking10.1016/j.comnet.2024.110353244:COnline publication date: 1-May-2024
https://dl.acm.org/doi/10.1016/j.comnet.2024.110353
Allaoui TGasmi KEzzedine T(2024)Reinforcement learning based task offloading of IoT applications in fog computing: algorithms and optimization techniquesCluster Computing10.1007/s10586-024-04518-z27:8(10299-10324)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10586-024-04518-z
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents