BlockDeepNet: A Blockchain-Based Secure Deep Learning for IoT Network

The rapid advancement in 5G-envisioned industrial automation combines IoT and leading mobile wireless connectivity (5G) to deliver high mobility, availability, and connectivity support in industrial automation. It further enables significant growth in data generation on the IoT platform due to the integration of autonomous machine, mobile robot, and many sensors [1,2]. A recent analyst report valued the global IoT data-management market at around $27.13 billion in 2017 and predicted that it would reach approximately $94.47 billion in 2024, growing at Compound Annual Growth Rate (CAGR) of slightly above 19.51 percent between 2018 and 2024 [3]. To support the analysis of large amounts of data and provide valuable information for the prediction, classification, and detection of future events in IoT, the Deep Learning (DL) paradigm is often employed. As a strong analytic tool, DL enables the reliable mining (feature extraction and representation) of IoT big data generated and collected from various complex and noisy environments [4]. It has become a promising approach for various informatics applications, such as autonomous driving, human activity recognition, anomaly detection, bioinformatics, object detection, and face recognition in IoT [5]. The convergence of DL and IoT has been applied in many real-world tasks and considered the most desirable choice to solve the IoT big data problems. For instance, DL can support the accurate prediction of electricity power consumption in the smart home by collecting and processing data from smart meters, which further help in improving the electricity supply from the smart grid.

With the significant growth in data-driven applications and generated data, however, DL is expected to be more resource-intensive and to be the dominant computing workload in IoT in the future. To support efficient big data analysis and handle dominant computing workload in IoT, various DL paradigms have been presented in the recent year. The existing paradigms can be divided into three categories: (1) Cloud intelligence, (2) Fog intelligence, and (3) Edge intelligence. In cloud intelligence, a single DL model is deployed at the central cloud server to perform big data analysis in IoT [6]. Note, however, that the IoT network generates a large amount of data that impose heavy communication and computation tasks on the cloud server, such as raw data collection, pre-processing, features extraction, and analysis; hence, the rapid escalation in the server calculation load and communication frequency that further lead to overwhelming, and sometimes unbearable, load to the centralized servers. To address these challenges, Fog intelligence—wherein the data collection and computation tasks of DL are shifted to the fog layer in a distributed manner—was presented [7]. Each fog node prepares a DL model by training its own private data individually and periodically shares the intermediate parameters of the model to the cloud server. The cloud server then aggregates the received parameter and continuously updates the individual model at each fog node. Consequently, the distributed learning at each fog node lowers the computation and communication overhead for the cloud server. Note, however, that the cloud server has full control over the individual learning model, which leads to single point of failure. On the other hand, edge intelligence has the cloud server offloading the intermediate learning and computation tasks to the edge layer. To offload the processing, edge intelligence distributes the DL model in such a way that the lower layers of the model are trained into edge nodes and the higher layers of the model are processed into the cloud server. This is because parts of the DL layers are offloaded at the edge, which, in turn, reduce the intermediate data computation and processing at the cloud layer [8]. Nonetheless, the cloud server in edge intelligence still has full control on the DL model as higher layers are trained at the cloud server, which might result in single point of failure. In addition, all three paradigms do not maintain the full privacy of the data contributors (IoT devices). This is because IoT devices are excluded from the DL process as the data for the learning process are collected without the permission of the device owner, which may cause a privacy leak. Due to the privacy issue, some data contributors do not share their data to the server; hence the difficulty in accumulating enough training data for the distributed DL model, which, in turn, leads to a weak DL model with low accuracy. Furthermore, all three forms of intelligence may suffer from a data poisoning attack wherein any of the entities in the IoT network may demonstrate adversarial behavior in the distributed DL process, e.g., a malicious server can deliberately implant misleading training data, which may disrupt the distributed DL process and wrongly update the model parameters into distributed DL models. Malicious data poisoning at the cloud server may disrupt the individual DL model at the edge of the IoT network [9].

From the aforesaid discussion of existing DL paradigms, we pointed out four major challenges that require solving to design an effective DL paradigm for IoT: (a) Single point of failure, (b) Privacy leak, (c) Training data insufficiency, and (d) Data poisoning attack. Given the migration of the DL platform on the IoT edge device, the learning task is performed directly on the edge devices without depending on the cloud server. Recently, Matt and Rei et al. [10] deployed a DL model directly on an edge device using Google TensorFlow models and Project Flogo. Moreover, Tang, et al. [11] demonstrated that the DL task on an IoT device can benefit from the tailoring of individual device data on the device itself, thus maintaining data privacy. Moreover, data training performs nearer to the device, which reduces the communication overhead and ensures low latency. In this sense, the DL task on an IoT device can be regarded as advantageous in terms of not only offloading computation overhead, low latency, and cost saving, but also as a solution to the problem of single point of failure and privacy leak.

On the other hand, a blockchain supporting a secure network over the untrusted parties has recently become an attractive choice to provide a secure, scalable IoT network [12]. It delivers the silent features of decentralization and immutable distributed ledger to provide privacy in an IoT network. Decentralization in blockchain supports secure sharing of resources and data among all the member nodes in an IoT network that overcomes many-to-one traffic flows and central control dependency and provides enough data for big data analysis. Consequently, it eliminates the problem of single point of failure and training data insufficiency and reduces the communication delay. In addition, the immutable distributed ledger securely maintains information about all the member nodes of an IoT network in the form of blocks cryptographically controlled by all member nodes. This feature prevents malicious tampering of training data, which, in turn, responds very well to data poisoning attacks in an IoT network.

The DL task on an IoT device (device intelligence) and Blockchain can jointly solve the challenges in the existing DL paradigms for IoT. Therefore, the primary goal of our study is the integration of device intelligence and blockchain technique to support a collaborative, secure DL paradigm for IoT. The research contributions of our study include:

Along with the recent advancement in the IoT, many researchers have presented their methodologies to provide an efficient big data analysis in IoT. The recent methodologies are mainly focused on the use of DL model that typically contains three layers: input layer, hidden layer, and output layer. A multiple hidden layer (called depth of the model) is used in the DL model, wherein each hidden layer contains a certain number of neurons to provide an accurate feature representation and extraction at the different level. Typically, the neurons at each layer are responsible for the learning of hierarchical features from the input data and give a final output at the output layer in terms classification, prediction, and many more [5]. A neuron takes multiple inputs and gives a single output. The neurons at the $l^{th}$ layer take input from the output of the neurons at the previous ${\left(l-1\right)}^{th}$ layer. To connect the neurons from the subsequent layer, weight and bias parameters are used. For instance, a neuron $j$ at the $l^{th}$ layer relates to a neuron $i$ at the previous ${(l-1)}^{th}$ layer using model parameters $\left(w_{ij},b_i\right)$. These model parameters are required to be learned to minimize the model error (i.e., the difference between final model output and targeted value) for accomplishing the DL task successfully. A back-propagation approach that relies on the gradient descent method is used to learn the model parameters efficiently. The detailed process for the back-propagation approach can be found in Abeshu et al. [7].

The abovementioned process to train a multilayer and complex DL model entails a high computational overhead. To mitigate this issue, various novel DL paradigms have been proposed by many researchers at the different layer of IoT such as cloud edge, and fog [4,5,6,7,8]. Zhang et al. [6] presented a DL approach wherein the learning task is performed at the cloud layer to improve the learning efficiency for big data analysis. In Abeshu et al. [7], the authors proposed an innovative distributed deep learning approach called a Fog-to-Things Computing, wherein various fog nodes at the fog layer are responsible for carrying out the DL task to mitigate issue of less scalability and low accuracy in big data analysis. They evaluated their proposed approach using a case study of attack detection in IoT and demonstrated the effectiveness of the distributed DL. Li at al. [8] examined the possibility of DL into the edge computing environment and designed a new offloading methodology to offload the deep learning task from cloud to edge layer in IoT. The authors demonstrated that the DL task can be carried out by training the lower layer of DL at the edge nodes and the higher layer of the DL at the cloud layer. Mohammadi et al. [4] summarized and analyzed existing researches reported of DL in IoT.

As described in the preceding section, the existing research faces major challenges, such as Single point of failure, Privacy leak, Training data insufficiency, and Data poisoning attack. To mitigate these challenges, our research focused on performing the collaborative DL task at the device level with the integration of blockchain technology. In Tang et al. [11], the authors demonstrated that the DL task at the device level supports protecting the data privacy at the device by performing data analysis on the device itself. On the other hand, our recent research, Rathore et al. [12] provides the integration of Blockchain and IoT to support a secure and scalable IoT network, wherein blockchain can overcome the single point of failure and Training data insufficiency by performing the DL task in a decentralized way. As benefited from the integration of device intelligence and blockchain, BlockDeepNet, a Blockchain-based secure DL for IoT network, is introduced in the subsequent section.

The system infrastructure proposed in this paper describes a novel decentralized big data analysis approach wherein the learning task is performed at the device level and distributed by employing blockchain technology. A modern IoT network employs three basic steps for big data analysis: a) the data is collected from the IoT device and preprocessed at the connected server; b) The intelligent learning paradigms are used to analyzed the processed data; and, c) with the help of the analyzed data, the IoT device is remotely controlled. A centralized management and control are supported by the existing big data analysis mechanisms; however, an attacker can exploit a significant amount of data maliciously. The proposed BlockDeepNet system remarkably lowers the possibility of data being manipulated adversely by facilitating a secure and collaborative DL paradigm. Consequently, some components of the IoT network must be reconfigured to support the working procedure of BlockDeepNet.

In order to resolve the issues in the existing works, we introduce a BlockDeepNet system wherein the blockchain network is leveraged to support a collaborative and secure DL task in IoT. The conceptual structure of the BlockDeepNet system consists of edge server and IoT devices. An edge server can be associated with a set of IoT devices, working as a miner in the private blockchain network. The working mechanism of the BlockDeepNet system is summarized as follows. Each IoT device in the BlockDeepNet system trains a local model using DL and uploads the local update (gradient parameters, learning model’s weight) to its associated edge server in the blockchain network. In return, the edge server pays data reward to the IoT device based on the amount of sample data it has. The edge server verifies and processes the local updates of its associated devices to generate a collaborative DL model by aggregating local updates. Once the edge server accomplishes the collaborative DL task, it creates a block wherein the aggregated local updates are stored and receives the processing reward from the blockchain network. Finally, the created block is added to the blockchain network, which is also called a distribute ledger. Each IoT device downloads the distributed ledger and updates its local update to compute a global update, which is an input of the next local update. Since an IoT device locally computes its global update, malfunctioning of the edge server or another IoT device during the learning process does not affect the device global update, thus overcoming single point of failure and ensuring the security and robustness of the overall DL task in IoT. Based on these benefits, in order to design the BlockDeepNet system, we present in this section a more detailed explanation about the collaborative DL process and its deployment in a blockchain environment. We also discuss the latency delay invoked by the collaborative DL and blockchain operation.

In this section, we present an experimental evaluation of the BlockDeepNet system to show its practicability and feasibility in IoT. First, an experimental testbed is described to implement a BlockDeepNet prototype. An application of object detection is then deployed on the BlockDeepNet system to analyze the compatibility of collaborative DL and IoT. In the third part, we show a security analysis of BlockDeepNet, including how it overcomes the security issues in the existing centralized and distributed system. Finally, a feasibility analysis is presented to validate the capabilities of DL and blockchain operations in the BlockDeepNet system with an acceptable performance overhead.

In this paper, we proposed BlockDeepNet, a Blockchain-based secure DL system by combining DL and blockchain. BlockDeepNet provides three contributions in the area of DL for big data analysis in IoT. First, a collaborative DL paradigm that supports DL at the IoT device level to mitigate privacy leak and obtains enough data for DL was presented. Second, collaborative DL was deployed in a blockchain environment to provide secure and reliable exchange of local and global updates. Finally, a protype model of BlockDeepNet was developed to validate its effectiveness in real-time scenarios. We conducted an experimental evaluation of BlockDeepNet with a case study of object detection to demonstrate its feasibility and compatibility in IoT. The evaluation results demonstrated that BlockDeepNet is feasible for big data analysis task in IoT such as object detection and provides an efficiency in terms of accuracy, security analysis, time delay, and computational complexity. Our findings suggest that BlockDeepNet mitigates the existing challenges and obtains higher accuracy with acceptable latency and computational overhead of blockchain operation for DL in IoT. However, DL at the device level can further result in the requirement of higher computation power and devices with lower computation power cannot be benefited with BlockDeepNet. To address this issue, BlockDeepNet can be enhanced with an offloading mechanism wherein devices with low computation power can offload their DL task to the edge server via blockchain transactions.