Introduction

Incredible developments in the routine use of electronic services and applications have led to massive advances in telecommunications networks and the emergence of the concept of the Internet of Things (IoT). The IoT is an emerging communications paradigm in which devices serve as objects or “things” that have the ability to sense their environment, connect with each other, and exchange data over the Internet [1, 2]. By 2022, one trillion IP addresses or objects will be connected to the Internet through IoT networks [3].

The IoT paradigm has recently been used in creating smart environments, such as smart cities and smart homes, with various application domains and related services. The goal of developing such smart environments is to make human life more productive and comfortable by solving challenges related to the living environment, energy consumption, and industrial needs [4]. This goal is directly reflected in the substantial growth in the available IoT-based services and applications across different networks. For example, the Padova Smart City in Italy is a successful example of a smart city based on an IoT system [5].

Smart environments consist of sensors that work together to execute operations. Wireless sensors, wireless communication techniques, and IPv6 assist in the expansion of smart environments. Such environments are wide ranging, from smart cities and smart homes to smart healthcare and smart services. The integration of IoT systems and smart environments makes smart objects more effective. However, IoT systems are susceptible to various security attacks, such as denial-of-service (DoS) attacks and distributed denial-of-service (DDoS) attacks. Such attacks can cause considerable damage to the IoT services and smart environment applications in an IoT network. Consequently, securing IoT systems has become a major concern [1]. For example, on Friday, October 21, 2016, a series of DDoS attacks were launched across the US that exploited the security vulnerabilities in IoT systems [6]. These attacks affected IoT devices, websites and online services such as Twitter, Netflix, and PayPal.

An intrusion detection system (IDS) is a security mechanism that works mainly in the network layer of an IoT system. An IDS deployed for an IoT system should be able to analyze packets of data and generate responses in real time, analyze data packets in different layers of the IoT network with different protocol stacks, and adapt to different technologies in the IoT environment [3]. An IDS that is designed for IoT-based smart environments should operate under stringent conditions of low processing capability, fast response, and high-volume data processing. Therefore, conventional IDSs may not be fully suitable for IoT environments. IoT security is a continuous and serious issue; thus, an up-to-date understanding of the security vulnerabilities of IoT systems and the development of corresponding mitigation approaches are required.

This article offers a comprehensive review of IDSs as a security solution for IoT-based smart environments. The primary goal of this study is to present the most recent designs and approaches for IDSs operating in IoT-based environments. Although related surveys have been published in the literature [3, 7], this article focuses on the important factors that affect IDS performance in smart environments, such as the detection accuracy, false positive rate, energy consumption, processing time, and performance overhead. In addition, this article introduces a solid foundation for the development of IDSs for IoT-based smart environments.

This study offers multiple key contributions. First, a full preliminary analysis of IoT systems, smart environments, and IDSs is presented. Second, the study confirms that traditional IDSs cannot satisfy IoT security requirements due to the large diversity of IoT networks and protocols. For instance, IPv6 over low-power wireless personal area networks (6LoWPAN) is not a protocol that is used in traditional telecommunications networks. Third, the common features that can be ported from traditional IDSs to IoT-based IDSs are emphasized. This third contribution emerges from the integration of the previous surveys [3, 7] to summarize the features, advantages and disadvantages of all IDSs designed for IoT-based systems. Fourth, this work introduces a future outlook on IDSs for IoT environments with a focus on the strengths and weaknesses of the current IDSs. Additionally, this study presents new recommendations for designing IDSs that satisfy the security requirements of IoT-based smart environments.

This survey focuses on IDSs for the IoT paradigm, independent of any specific technology or protocol; however, readers who are interested in learning more about IoT enabling technologies and protocols such as low-power wide-area network (LPWAN) technologies, long range (LoRa) technology, the low power WAN protocol for Internet of Things (LoRaWAN), the 6LoWPAN protocol, or the constrained application protocol (CoAP) may refer to [8–11] for further details.

The remainder of this paper is organized as follows. “The IoT paradigm” section discusses various definitions and architectures relevant to the IoT context. This section also highlights the importance of cloud computing systems for IoT-based smart environments and the challenges of applying this combination of systems in the real world. Definitions, goals and challenges related to smart environments, with a focus on smart cities, are discussed in “IoT and smart environments” section. “Security challenges in IoT-based smart environments” section reviews the security challenges in IoT-based smart environments in relation to the various layers of the IoT architecture and highlights some practical open challenges facing real-world IoT networks. “Intrusion detection systems (IDSs)” section provides preliminary information about the definitions relevant to IDSs, the different types of IDSs and the detection techniques used in these systems. A survey of IoT-oriented IDSs that either can be applied in or are specifically designed for smart environments is presented in “IDSs designed for IoT systems” section.“Discussion and future outlook” section discusses recommendations concerning IDSs implemented for IoT-based smart environments. Finally, conclusions and plans for future work are reported in “Concluding remarks” section. The organization of this paper is presented visually in Fig. 1.

Fig. 1
figure 1

The flow of this article, separated into two levels; level 1 describes the main topics, and level 2 describes the detailed points

Full size image

The IoT paradigm

The IoT concept has been established since the founding of the Auto-ID Center at the Massachusetts Institute of Technology (MIT) in 1999. The Auto-ID Center created the electronic product code (EPC) number, which depends on radio frequency identification (RFID), in 2003. This idea is the crucial technology of the IoT [12].

However, the IoT is a well-established paradigm, and it is defined in several ways from various perspectives. Thiesse et al. [13] defined the IoT as consisting of hardware items and digital information flows based on RFID tags. The IoT definitions and architectures provided by various standards and industrial organizations will be described in the following.

The Institute of Electrical and Electronics Engineers (IEEE) defines the IoT as a collection of items with sensors that form a network connected to the Internet [12, 14]. The International Telecommunication Union (ITU) defines the IoT through three dimensions, as a network that is available anywhere, anytime, and by anything and anyone [15]. The European Telecommunications Standards Institute (ETSI), rather than using the expression “Internet of Things (IoT)”, defines machine-to-machine (M2M) communications as an automated communications system that makes decisions and processes data operations without direct human intervention [16].

The Coordination and Support Action for Global RFID-related Activities and Standardisation (CASAGRAS) project has created a new concept of the IoT that encompasses two viewpoints: the connection of physical objects with virtual objects over a global network without any human intervention to the greatest extent possible [17] and the incredible increase in IoT applications within traditional networks due to the extent of IoT marketing [17].

Moreover, Cisco, an industrial organization, works on IoT technology under the title of the Internet of Everything (IoE). Cisco has summarized the IoE concept as a network that consists of people, data, things, and processes. Thus, information and actions are created in and moved through this network [18].

IoT system architectures

Regarding IoT design, IEEE is working on a project (IEEE P2413) to determine the IoT architectural framework. The scope of this project is to describe the IoT domains and the various applications in these domains [19]. This IoT architecture is divided into three layers: the application layer, the networking and data communications layer, and the sensing layer.

According to [20–22], the general architecture of the IoT is divided into five layers that span three domains, namely, the application domain, the network domain, and the physical domain; thus, the IoT can be customized to fit the needs of different smart environments. The application domain encompasses management and utilization. The network domain is responsible for data transmission. The physical domain is responsible for information collection. The layers of the general IoT architecture are shown in Fig. 2. The functionality of the different layers is discussed in the following.

Fig. 2
figure 2

The general architecture for the IoT. The general architecture for the IoT, which consists of five layers according to [22]

Full size image

The perception layer is a hardware layer that consists of sensors and physical objects in different forms. These hardware elements provide identification, information storage, information collection, and information processing. The information output from this layer is sent to the next layer (the network layer) to be transmitted to the processing system [22].

The network layer is a transmission layer that transfers the information from physical objects or sensors to the processing system over secure lines using a communication system. This communication system can be either wired or wireless and can be based on different technologies, depending on the physical object or sensor components. The information output from this layer is sent to the next layer (the middleware layer) [22].

The middleware layer is responsible for service management over IoT devices to create connections between IoT devices that provide the same service. Moreover, the middleware layer stores the information coming from the network layer in a database to facilitate decision-making on the basis of information processing operations [22].

The application layer is responsible for the global management of IoT applications. The application layer depends on the information processed in the middleware layer. Moreover, the application layer depends on the specifics of the different implemented IoT applications, such as smart industry, building, city, and health applications [22].

The business layer is also responsible for the global management of IoT applications as well as service management over IoT devices. The business layer creates a business model that depends on the information processed in the application layer and on the analysis of the results of these information processing operations [22].

Cloud computing and the IoT

IoT systems connect an enormous number of devices and sensors exchanging an enormous amount of data and supporting a massive number of services. The management and analysis of these data pose certain special requirements, such as powerful processing, massive storage and high-speed networking capabilities [23].

Cloud computing offers high computational power, a massive storage capacity, and configurable resources with virtualization capabilities for manipulating the large amounts of data collected from IoT-based smart environments. With the integration of cloud computing systems and IoT-based smart environments, smart things can be easily accessed and managed at any time and place, and better services can be provided through the IoT model [23, 24].

According to [25], one of the important challenges in employing a cloud computing system for the IoT is the synchronization between different cloud vendors. A second challenge is achieving compatibility between general cloud service environments and IoT requirements. Security challenges are the main factor hindering the adoption of cloud computing by businesses and government organizations [26]. Thus, the ability to respect the necessary security constraints to fulfill the needs of the IoT in a cloud computing platform is a vital requirement. A robust and efficient security solution such as an IDS is one possible option. Moreover, standardization, enhancement, and management for the deployment of IoT systems and their connection to the cloud are additional challenges that should be taken into consideration.

IoT and smart environments

The objective of smart environments is to make human life more comfortable and more efficient by using sensors. IoT-based smart environments enable the effective realization of smart objects. By means of an IoT network, sensors can be monitored and controlled remotely. According to Navigant Research, the global smart city services market is expected to increase from 93.5 billion US dollars in 2017 to 225.5 billion US dollars by 2026 [27].

Ahmed et al. [28] state that “The term smart refers to the ability to autonomously obtain and apply knowledge, and the term environment refers to the surroundings”. A smart city is one type of smart environment. The core element of a smart city is an integrated information center operated by the IoT service provider, which provides information on services such as electricity, water, and gas.

Smart health, smart industry, smart buildings and smart homes are other types of smart environments. The objective of such smart environments is to provide services via smart methods based on the information collected by IoT-enabled sensors. The architecture of such IoT-based smart environments is shown in Fig. 3.

Fig. 3
figure 3

IoT-based smart environments. The architecture of the IoT and the extent of the IoT market according to [14]

Full size image

Smart environments based on the IoT paradigm have certain special characteristics, and hence, special needs arise in the deployment of such environments. For instance, remote monitoring and remote control capabilities are required to allow smart objects to collect and process data and to execute operations remotely. Moreover, the ability to make decisions is an important characteristic in such a system. A smart object should be able to make intelligent decisions without human intervention by using data mining and other techniques for extracting useful data.

By virtue of these characteristics, smart environments offer certain features that can be used to enhance the quality of service (QoS) of user applications. Real-time information is one of these features. Smart objects can collect and analyze data and make intelligent decisions in real time. Moreover, the cost-effectiveness of cloud applications can be used to increase the QoS of smart environment applications. The integration of smart and IoT environments offers new opportunities with respect to the QoS of services and applications.

IoT technology for developing smart cities

Many national governments are working on the information and communication technology (ICT) infrastructure to solve the problems arising in traditional public management affairs. One of the most modern and effective solutions is to establish a smart city [5]. The smart city concept is one facet of the idea of smart environments.

There are many benefits of converting traditional public services and resources into a form that takes advantage of the smart city concept, including increasing the quality of public services and reducing the operating costs of public administration [29]. However, the management and execution of public services in a smart city require a powerful network, such as an IoT network.

Additionally, there are many barriers to the establishment of an IoT-based smart city. The novelty, complexity and technical challenges of IoT systems present the greatest difficulty. Furthermore, in the absence of widely accepted definitions for smart city operations, political and financial barriers prevent the smart city concept from being effectively applied.

The Padova Smart City in Italy is a successful example of a smart city that has overcome these barriers. The main goal of establishing the Padova Smart City is to develop ICT solutions for public administration systems using different types of data and technology [5].

The implementation of the IoT paradigm for creating smart environments, particularly smart cities, faces several technical challenges. Among these, precision, latency and available bandwidth have important effects in many smart environments, such as industrial and healthcare environments. Because of the need to support an increasing number of users and smart objects in IoT networks and the corresponding generation of increasingly large amounts of data, scalable computing platforms, such as cloud computing, are necessary. Such platforms can improve the performance of data management services in IoT systems and the QoS of smart environment applications [30].

Security challenges in IoT-based smart environments

The security of IoT systems is a serious issue due to the increasing numbers of services and users in IoT networks. The integration of IoT systems and smart environments makes smart objects more effective. However, the impacts of IoT security vulnerabilities are very dangerous in critical smart environments used in fields such as medicine and industry. In IoT-based smart environments without robust security systems, applications and services will be at risk. Confidentiality, integrity, and availability are three important security concepts of applications and services in IoT-based smart environments; thus, to address these concerns, information security in IoT systems requires greater research focus [2]. For example, IoT-based smart homes face security and privacy challenges that span all layers of the IoT architecture [31].

The creation of smart environments in the real world faces two notable barriers: the security of IoT systems and the complexity and compatibility of IoT environments. Attacks such as DoS or DDoS attacks on IoT networks affect IoT services and thus affect the services provided by smart environments.

Researchers study the security challenges of the IoT from many different points of view, one of which is the security vulnerability of IoT communication protocols [32]. This survey focuses on IDSs for the IoT paradigm, independent of any specific protocol; thus, this study focuses on the security challenges facing IoT systems on the basis of the IEEE definition and the general IoT architecture.

The security challenges in IoT systems are related to security issues arising in the different IoT layers. Physical damage, hardware failure, and power limitations are challenges faced in the physical layer. DoS attacks, sniffing, gateway attacks, and unauthorized access are challenges relevant to the network layer. Malicious code attacks, application vulnerabilities, and software bugs are challenges faced in the application layer [33].

According to [34], the security-related problems of any IoT system can be categorized into four types: authentication and physical threats, confidentiality risks, data integrity issues and privacy problems. The relations between these groups are shown in Fig. 4. The security problems arising in the different IoT layers are concisely discussed below.

Fig. 4
figure 4

The security challenges in the different IoT layers. The four types of security problems arising in the IoT model are associated with the different layers of the IoT architecture

Full size image

  • The authentication-related problem and physical threats are the first challenges that affect an IoT system. The perception layer includes many IoT devices, such as sensors, that depend on their own security systems; thus, they are susceptible to physical attacks.

  • Confidentiality-related risks arise between IoT devices and the gateways in the network layer. The resource-constrained nature of the low-level devices in IoT systems poses an indirect challenge with regard to the confidentiality of data transmission in IoT networks [35].

  • The third class of security challenges concerns the data integrity between services and applications. Data integrity problems emerge when spoofing attacks or noise affect an IoT system. DoS, DDoS, and probe attacks are arbitrary attacks that can harm IoT applications and services.

  • The challenges of the fourth type are related to privacy. Information privacy is an important aspect of security in IoT systems [36]. Different IoT components use different types of object identification technologies; thus, every object has its own identification tag, which carries personal, location and movement information. Managing and monitoring the applications and services in an IoT system mean placing information privacy at risk; for example, using a system based on a deep packet inspection technique for trusted operations within an IoT system is considered to be a violation of information privacy [37]. Any intrusive accesses to the management system without permission threaten the information privacy of the IoT users [34].

Real-world applications of IoT systems face many open challenges. The open security challenges affecting IDS operations identified by [20, 22, 33, 38] are discussed below.

  • A smart environment that integrates IoT technology is considered to be a complex system because it consists of different products from different companies based on different technologies that do not share a universal language. Therefore, standardization is another important aspect of security in IoT systems. Creating a standard IoT architecture based on one standard technology for all vendors and manufacturers would enhance the interoperability of the security functionalities of all objects and sensors in an IoT system. The success of this integration will depend on collaboration among companies to create a universal standard. Such standardization will greatly facilitate IoT network security.

  • A single successful penetration of one or more end devices can threaten the security of an entire IoT system and cause harm to its applications and services, especially from an industrial point of view [39]. Thus, the implementation of a strong security mechanism in an IoT system depends on the strength of the security for individual IoT devices, which in turn depends on power and memory factors. Consequently, power and memory limitations are considered to pose indirect security challenges in IoT systems. To address these challenges, lightweight security solutions and lightweight encryption and decryption methods are required. These solutions and methods must be applicable in different IoT domains and must satisfy the security requirements without affecting the QoS.

Intrusion detection systems (IDSs)

IDSs: a historical overview

Monitoring and analyzing user information, networks, and services through passive traffic collection and analysis are useful tools for managing networks and discovering security vulnerabilities in a timely manner [40, 41]. An IDS is a tool for monitoring traffic data to identify and protect against intrusions that threaten the confidentiality, integrity, and availability of an information system [42]. The operations of an IDS are schematically illustrated in Fig. 5.

Fig. 5
figure 5

IDS operations. IDS operations can be divided into the monitoring stage, the analysis stage and the detection stage

Full size image

The operations of an IDS can be divided into three stages. The first stage is the monitoring stage, which relies on network-based or host-based sensors. The second stage is the analysis stage, which relies on feature extraction methods or pattern identification methods. The final stage is the detection stage, which relies on anomaly or misuse intrusion detection. An IDS captures a copy of the data traffic in an information system and then analyzes this copy to detect potentially harmful activities [43].

The concept of an IDS as an information security system has evolved considerably over the past 30 years. During these years, researchers have proposed various methods and techniques for protecting different types of systems using IDSs. In 1987, Denning presented an intrusion detection model that could compare malicious attack behavior against the normal model for the system of interest [44].

In 2000, Axelsson [45] surveyed 20 research projects on IDSs. He listed fourteen IDSs relying on host-based methods, two IDSs relying on network-based methods and three IDSs relying on both host-based and network-based methods. However, the IDS model used in those studies was out of date and depended on the local machine more than on the network traffic during the analysis stage.

In 2013, Ganapathy et al. [46] presented a survey on intelligent techniques for feature selection and classification-based intrusion detection in networks. This survey considered fuzzy techniques, neural networks, genetic algorithms, neuro-genetic algorithms, particle swarm intelligence and rough sets for Internet security protection and QoS enhancement. Moreover, these authors proposed new feature selection and classification algorithms. In their experiment, they used 19 flow-based features comprising basic features, packet content features and traffic features.

In 2014, Mitchell and Chen [47] surveyed 60 papers on IDSs designed for wireless environments. Their survey revealed the strengths and weaknesses of IDS techniques for wireless local area networks (WLANs), wireless mesh networks (WMNs), wireless personal area networks (WPANs), wireless sensor networks (WSNs), cyber-physical systems (CPSs), ad hoc networks and mobile telephony.

Mitchell and Chen [47] proved that an anomaly-based IDS is the most suitable design for mobile telephony systems. However, such IDSs face challenges in terms of their high false positive rate and computational complexity [47]. High false positive and false negative rates reduce the QoS of a mobile network system. If any user packet is dropped by mistake, the user will suffer a billing error, and the user packet will be delayed [47]. Anomaly-based IDSs also face challenges with regard to illegal analysis methods, such as packet-based methods, that infringe on user privacy [47]. This survey proposed the detection latency as a critical metric for use in future research.

Also in 2014, Butun et al. [48] surveyed 18 papers focusing on mobile ad hoc networks (MANETs) and 17 papers focusing on WSNs in their survey on IDSs in WSNs. These authors discussed the feasibility of using systems designed for MANETs in WSNs. The possible security attacks against WSNs were divided into two categories: passive attacks and active attacks. These authors proved that IDSs are very important for the security of WSNs and that an IDS designed for a WSN must have certain special characteristics, including low power consumption. A WSN is a resource-constrained environment, so the effectiveness of an IDS in a WSN depends on its effect on the energy consumption of the network.

Butun et al. [48] recommended the use of a hierarchical IDS model to solve the energy consumption issue in WSNs. In accordance with the relevant application requirements, Butun et al. [48] recommended using a distributed IDS scheme for mobile applications, a centralized IDS scheme for stationary applications and a hierarchical IDS scheme for cluster-based applications.

IDSs: types and methods

The implementation of an IDS depends on the environment. A host-based intrusion detection system (HIDS) is designed to be implemented on a single system and to protect that system from intrusions or malicious attacks that will harm its operating system or data [49].

A HIDS generally depends on metrics in the host environment, such as the log files in a computer system [50]. These metrics or features are used as input to the decision engine of the HIDS. Thus, feature extraction from the host environment serves as the basis for any HIDS. The operational structure of a HIDS and its location in the network are shown in Fig. 6.

Fig. 6
figure 6

Generic architecture of a host-based IDS (HIDS). The operational structure of a HIDS and its location in the network

Full size image

A network-based intrusion detection system (NIDS) sniffs network traffic packets to detect intrusions and malicious attacks [50]. A NIDS can be either a software-based system or a hardware-based system. For example, Snort NIDS is a software-based NIDS [51]. The operational structure of a NIDS and its location in the network are shown in Fig. 7.

Fig. 7
figure 7

Generic architecture of a network-based IDS (NIDS). The operational structure of a NIDS and its location in the network

Full size image

Network expansion and increasing traffic volumes necessitate the implementation of IDSs as hardware systems, such as a smart sensor architecture [52]. For example, field programmable gate arrays (FPGAs) can be used as the basis of a hardware-based NIDS. The special characteristics of FPGAs, such as their ability to support high-speed interfaces, dynamic reprogramming and very high-volume data processing, make FPGAs very suitable for use in NIDSs [53].

IDSs: detection techniques

An IDS depends on algorithms for implementing the various stages of intrusion detection. There are a vast number of algorithms for all IDS types and methods. Some of these IDS algorithms will be discussed briefly in the section titled ‘IDSs Designed for IoT Systems’.

Additionally, some of these IDS algorithms can be used for multiple different detection techniques. Thus, this section focuses on lightweight anomaly-based IDS algorithms that can be used in IoT-based environments depending on the complexity, execution time and detection time requirements. Principal component analysis (PCA) is a lightweight algorithm that can be used for various detection techniques in IDSs; thus, the PCA algorithm will be discussed as a representative example in the following.

Mori et al. [54] have stated that “principal component analysis (PCA) is a commonly used descriptive multivariate method for handling quantitative data and can be extended to deal with mixed measurement level data.” Thus, PCA has been widely applied in various fields. As described by [55], PCA generates a set of variables depending on the variance-covariance structure of the original variables. These new variables are linear combinations of the original variables and are fewer in number than the original variables.

In IDSs, PCA is used as a dimensionality reduction and detection technique. Elrawy et al. [56] used the PCA approach to create an anomaly-based statistical and data mining IDS that depends on the division of the principal components into the most and least significant principal components. In this system, the detection stage relies on the major principal component score and the minor principal component score. In addition, PCA has been used in intrusion detection techniques based on payload modeling, statistical modeling, data mining and machine learning [56–58].

Misuse-based intrusion detection

A misuse-based intrusion detection technique uses a database of known signatures and patterns of malicious codes and intrusions to detect well-known attacks [59]. Network packet overload, the high cost of signature matching, and the large number of false alarms are three disadvantages of misuse-based IDSs [60]. In addition, the severe memory constraints in some types of networks, such as WSNs, result in low performance of misuse-based IDSs because of their need to store a large database of attack signatures [61].

Moreover, the signature and pattern databases in signature-based IDSs and pattern-matching IDSs need to be continuously updated. Such misuse-based IDSs are designed to detect malicious attacks and intrusions based on previous knowledge.

Anomaly-based intrusion detection

In an anomaly-based intrusion detection technique, a normal data pattern is created based on data from normal users and is then compared against current data patterns in an online manner to detect anomalies [62]. Such anomalies arise due to noise or other phenomena that have some probability of being created by hacking tools.

Thus, anomalies are unusual behaviors caused by intruders that leave footprints in the computing environment [63]. These footprints are detected in order to identify attacks, particularly unknown attacks.

An anomaly-based IDS operates by creating a model of the normal behavior in the computing environment, which is continuously updated, based on data from normal users and using this model to detect any deviation from normal behavior [64]. The advantages and disadvantages of various anomaly-based intrusion detection techniques are shown in Table 1. These techniques will be discussed in the following.

Table 1 A short comparison between anomaly IDS techniques with a focus on the advantages and disadvantages of each technique
Full size table

  • A data mining approach is a means for extracting knowledge from a large amount of data, analogous to extracting gold from numerous rocks and sand [65]. The extracted knowledge is defined as interesting patterns in the data [66]. Such a pattern can describe the behavior of data from users or networks in a computing environment. The ability to automatically generate models that depend on the traffic description is one of the advantages of the data mining approach. Moreover, this approach can be applied in generalized IDSs and in any computing environment [67]. The data mining approach works perfectly for an online data stream that is unbounded, continuous and rapidly increasing in volume [68]. A procedure consisting of a rule learning stage, a clustering stage, a classification stage, and a regression stage is applied in the design of an IDS based on this approach [68].

  • Machine learning is a technique that depends on two stages: the training or learning stage and the detection or testing stage [69]. The training stage depends on mathematical algorithms or functions that use normal data as a reference input to learn the characteristics of the computing environment. Then, in the detection stage, these characteristics are used for detection and classification [70]. Supervised learning is one type of machine learning technique in which the characteristics of the training dataset are used in the learning phase to create a classification model, which is then used to classify new unseen instances [71]. Unsupervised learning is a type of machine learning technique that depends on the features of the data without using clustered training data [71].

    A pattern classification method in machine learning depends on pattern recognition, whereas a single classifier method depends on a single machine learning algorithm [72, 73].

  • The statistical model approach depends on statistical mathematical operations [74]. The statistics of historical user behavior are used to create a normal model, and any deviations from this model are then detected. These deviations are considered abnormal data. The statistical model approach uses statistical mathematical operations applied to a training dataset to detect abnormal traffic from the observed traffic patterns [75].

  • The rule model approach depends on the creation of rules for the computing environment. These rules are extracted from data traffic patterns. A rule-model-based IDS detects any anomalous data traffic that breaks these rules and considers any such anomaly as an attack [76]. The rule creation process depends on the historical system behavior. Thus, the system must be monitored for a long time to avoid an excessively high false positive rate.

  • The payload model approach depends on the packet traffic of a specific port or user for a given application. In a signature-based IDS, the payload model is based on pattern matching to identify attack packets with specific characteristics [77]. By contrast, an anomaly-based IDS that uses the payload model approach creates a model that depends on bytes or calculations from bytes that describe the normal characteristics of the packet payload [57].

  • The protocol model approach depends on monitoring protocols in different layers of the computing environment. An IDS based on this approach detects anomalies associated with a specific protocol or a protocol that is not present in the normal model. A specification-based approach, a parser-based approach or an approach based on application protocol keywords can be used to analyze the protocols in a computing environment [78].

  • The signal processing model approach depends on traffic analysis using signal processing methods. An IDS based on this approach creates a normal pattern by capturing the statistics of normal data traffic and the data distribution over time, and any deviation from this pattern is considered to be an anomaly [79].

Specification-based intrusion detection

The concept of a specification-based IDS was proposed by Ko et al. [80] in 1997. They proposed a monitoring and detection system based on security specifications that determine the normal behavior of the system to be protected. These security specifications are created based on the functions and security policies for this system. Thus, operating sequences that are not included in the system behavior are considered security violations [81].

The most important challenge in designing a robust specification-based IDS is creating a formalism that captures the valid operating sequences of the system. Therefore, the cost of defining the specification “trace policy” and the difficulty of evaluating and verifying the specifications limit the real-world applicability of specification-based IDSs. A specification-based IDS learns the root characteristics of attacks and detects known attacks like a misuse-based IDS, and it also has the ability of anomaly-based IDSs to detect unknown attacks, such as operating sequences that are not included in the normal behavior of the system [82–84].

IDSs: performance evaluation

The measures used to assess IDS performance depend on four factors, namely, the numbers of true positives (α), true negatives (δ), false positives (γ) and false negatives (β), as described in Table 2. Following [85, 86], these factors and the performance metrics for IDSs are described below.

Table 2 The two-dimensional confusion matrices that define the main factors considered in IDS performance evaluation when predicting the anomaly and normal classes
Full size table

When predicting the anomaly class, a true positive (αA) is a correct classification that indicates an intrusion. A true negative (δA) is a correct classification that indicates no intrusion. A false positive (γA) is an incorrect classification that indicates an intrusion when there is no intrusion. A false negative (βA) is an incorrect classification that indicates no intrusion when there is an intrusion. The true positive rate (TPR), which describes the probability of detecting intrusions, is calculated as:

$$ TPR=\frac{\alpha_{A}}{\alpha_{A}+\beta_{A}} $$
(1)

The false positive rate (FPR), which describes the probability of incorrectly identifying normal behavior as an intrusion, is calculated as:

$$ FPR=\frac{\gamma_{A}}{\gamma_{A}+\delta_{A}} $$
(2)

The recall (R), which describes the percentage of the total relevant records in a database that are retrieved by searching, is calculated in the same way as the TPR. The precision (P), which describes the percentage of relevant records among the records retrieved, is calculated as:

$$ P=\frac{\alpha_{A}}{\alpha_{A}+\gamma_{A}} $$
(3)

The F-score (F), which describes the balance between P and R, is calculated as:

$$ F=\frac{2*P*R}{P+R} $$
(4)

The overall success rate, which describes the percentage of correct classifications, is calculated as:

$$ Success Rate=\frac{\alpha_{A}+\delta_{A}}{\alpha_{A}+\delta_{A}+\gamma_{A}+\beta_{A}} $$
(5)
$$ \;\:\: Error Rate=1- Success Rate $$
(6)

When predicting the normal class, the same definitions and equations can be used, except with the parameters αN, βN, γN and δN.

IDSs designed for IoT systems

Due to the security challenges facing IoT systems, methods that can proactively identify new attacks are most suitable for protecting IoT networks. Thus, a robust IDS that can detect new attacks in IoT-based smart environments is required. An overview of the IDSs that have been proposed for IoT systems is shown in Table 3.

Table 3 A comparison of IDSs designed for IoT systems, with a focus on the types, techniques and features of these systems with respect to their adaptability to IoT-based smart environments
Full size table

According to the recommendations in recent surveys of IDSs for IoT systems [3, 7], this paper focuses on the features of all IDS methods for the IoT that can be applied in smart environments. IoT systems require special security measures with particular characteristics that are not offered by traditional IDSs.

Liu et al. [87] proposed an artificial immune IDS for IoT networks. This system can adapt to the IoT environment and automatically learn new attacks. The system is based on machine learning and a signature-based model. The adopted machine learning approach is designed after the mechanisms of artificial immune systems. The objective of the system is to increase the security of the IoT network; thus, it is a network IDS. This system has two main features: self-adaptation to new environments and self-learning of new attacks.

Kasinathan et al. [88] proposed an IDS that detects DoS attacks based on 6LoWPAN in IoT networks. They proposed a DoS detection architecture that consists of an IDS probe, a DoS protection manager and a Suricata IDS [89]. They designed this system based on a study of the vulnerabilities present in IP-based WSNs. The Suricata [89] IDS runs on a host computer; thus, the advantage of this system is that it can overcome the problem of power consumption, thus conserving power resources in WSNs.

Moreover, Kasinathan et al. [90] proposed an enhanced IDS for detecting DoS attacks based on 6LoWPAN in IoT networks. This system depends on the DoS detection architecture presented in [88]; its main new elements are a frequency agility manager (FAM) and security incident and event management system (SIEM). These elements together create a monitoring system that can monitor large networks.

Jun and Chi [91] proposed an IDS integrated with complex event-processing (CEP) technology. The benefit of CEP technology is the ability to identify complex patterns via real-time data processing. The event-processing IDS architecture consists of an event filtering unit, an event database unit, a CEP unit and an action engine unit. The system depends on an event-processing model that uses the rule model approach to detect intrusions.

The main features of this system are that it operates in real time and shows high performance in detecting intrusions in an IoT system using an event-processing mechanism.

Krimmling and Peter [92] proposed a NIDS that depends on machine learning for anomaly-based and signature-based intrusion detection. The system framework is designed for smart public transport applications that use CoAP. The main features of this system are its applicability to CoAP applications and its reliance on a lightweight algorithm.

Butun et al. [93] proposed a NIDS for WSNs that combines the statistical model approach and the rule model approach. The system is based on a downward-IDS and an upward-IDS in accordance with the hierarchical WSN structure. The downward-IDS detects abnormal behavior of the member nodes, and the upward-IDS detects abnormal behavior of the cluster heads. The main features of this system are its applicability to hierarchical WSNs and its dependence on WSN clustering.

Surendar and Umamakeswari [82] proposed a constraint-based specification IDS for IoT networks using 6LoWPAN. This system maintains efficiency in terms of QoS metrics while detecting sinkhole attacks. The system isolates malicious nodes and reconstructs the network without these nodes. This IDS is a specification-based IDS that depends on behavioral rules and uses the protocol model approach.

The main features of this system are that it detects sinkhole attacks, preserves QoS and isolates malicious nodes.

In addition, Le et al. [83] proposed a specification-based IDS for IoT networks using 6LoWPAN for the detection of several topology attacks against the IPv6 Routing Protocol for Low-Power and Lossy Networks (RPL), such as sinkhole, rank, local repair, neighbor, and destination oriented directed acyclic graph (DODAG) information solicitation (DIS) attacks. In a DIS attack, for example, the attacker increases the overhead in the network by using malicious nodes to send DIS messages [94] with fake IP addresses to the malicious nodes’ neighbors, forcing these other nodes to generate DODAG information objective (DIO) messages [83, 94]. The proposed IDS depends on analyzing the protocol behavior from trace files to learn the route establishment and maintenance procedures for a stable topology. The main features of this system are its high efficiency in detecting RPL topology attacks in an energy-efficient manner and its applicability to large-scale networks.

Moreover, Bostani and Sheikhan [84] proposed a hybrid IDS for IoT networks using 6LoWPAN for the detection of several RPL attacks. This system depends on specification-based intrusion detection modules, serving as IDS agents, in the router nodes and an anomaly-based intrusion detection module, serving as the main IDS, in the root node. The main features of this system are a reduction in the number of communication messages due to the lack of additional control messages or monitor nodes in the IDS design and its applicability to large-scale networks.

Garcia-Font et al. [95] proposed a NIDS for WSNs that depends on a machine learning approach and a signature model. They used a signature-based detection engine and an anomaly-based detection engine to improve the detection rate and the FPR. The system is designed to help smart city administrators detect intrusions using the IDS and an attack classification schema. The objective of the system is to detect intrusions in WSNs in different smart city environments. The main feature of this system is its applicability to large-scale WSNs.

Fu et al. [96] proposed a NIDS that depends on signature-based and protocol-based anomaly detection. The proposed IDS structure focuses on detecting attacks on IoT networks without being affected by the heterogeneity of such a network. The detection method depends on comparing the abstracted action flows in the data packets against three databases based on the protocol type information for each packet. These databases are a standard protocol library, an abnormal action library and a normal action library. The proposed approach consists of an event monitor, an event database, an event analyzer, and a response unit. This approach provides a uniform intrusion detection method for IoT networks based on automata theory. The main features of this system are the classification of attacks into three categories and the development of graphical user interface (GUI) tools to graphically present the abstract action flows and detect possible intrusions.

Deng et al. [97] proposed a NIDS that depends on the fuzzy c-means clustering (FCM) algorithm and the PCA algorithm. The system combines a machine learning approach with a data mining approach to improve the accuracy of intrusion detection for IoT networks. The PCA algorithm is used for feature selection and reduction. The FCM algorithm is used as a clustering method. The KDD-CUP99 dataset [98] was used to evaluate the proposed system. The main features of this system are its light weight and its ability to improve the detection efficiency by achieving a low FPR.

Amouri et al. [99] proposed a NIDS based on the protocol model approach and machine learning. This system consists of two detection stages. In the first stage, namely, local detection, network behavior data are collected by dedicated sniffers to generate a set of correctly classified instances (CCIs) using a supervised learning approach based on decision trees. In the second stage, namely, global detection, CCIs are collected by supernodes to generate time-based profiles called the accumulated measures of fluctuation (AMoFs) for malicious and normal nodes separately.

The main features of this system are its low computational complexity and low resource requirements.

Liu et al. [100] proposed a NIDS that depends on the suppressed fuzzy clustering (SFC) algorithm and the PCA algorithm. This system combines machine learning and data mining to improve the accuracy of intrusion detection in high-dimensional space. The PCA algorithm is used for feature extraction. A novel prejudgment-based intrusion detection method using PCA and SFC is applied that divides the dimension-reduced data into high-risk and low-risk data. The main feature of this system is its adaptability to high-dimensional spaces, such as the IoT space. Moreover, the efficiency and effectiveness of the IDS are optimized by reducing the detection time and increasing the accuracy by means of a frequency self-adjustment algorithm.

Abhishek et al. [101] proposed a NIDS that depends on the packet drop probability (PDP) in an IoT device to monitor gateways and detect malicious gateways. The system uses the statistical model approach for anomaly-based intrusion detection using a likelihood ratio test to detect malicious gateways, which corrupt the communications between IoT devices and access points. One of the disadvantages of this system is that it can only detect malicious getaways that affect downlink packets; it does not consider malicious getaways that affect uplink packets from IoT devices. The main features of this system are that it is based on theoretical foundations instead of requiring training data and that it can detect malicious gateways in real time.

Oh et al. [102] proposed a lightweight malicious-pattern-matching IDS. They stressed that traditional IDSs are not applicable for smart objects due to the limited memory size and battery life of these objects. Thus, a powerful and lightweight IDS is required because of these restrictions. Oh et al. proposed the auxiliary skipping (AS) algorithm, the early decision with boundary searching (EBS) algorithm, and an approach that uses both AS and EBS (AS-EBS).

These algorithms reduce the number of matching operations that must be performed [102]. The system depends on a pattern-matching approach based on signature detection. The advantage of this system is that it can be applied to smart objects with limited memory size and battery life. The main features of this system are the reduced memory size required for matching operations, the reduced workload for processing on smart objects, the increased speed of processing, and its scalable performance for a large number of patterns.

Summerville et al. [103] proposed an ultralightweight deep packet anomaly detection approach that can be implemented on small IoT devices. The system is designed with a dependence on the bitwise AND operation and uses a payload model approach for anomaly-based intrusion detection. The main features of this system are its low latency, high throughput and ultralight weight.

Mohan et al. [37] proposed a HIDS that depends on signature-based and rule-based anomaly detection. The system uses the traditional signature-based technique in combination with Snort-rule-based intrusion detection. Thus, the system can detect known attacks using the signature database and unknown attacks using SNORT rules. The main challenge of this system is privacy because the system uses a deep packet inspection technique to detect attacks. The main features of this system are its simplicity and self-learning capability.

Arrignton et al. [104] proposed a HIDS that depends on a machine learning approach for anomaly-based intrusion detection. The machine learning approach is based on the mechanisms of artificial immune systems. The main features of this system are its use of a behavioral modeling IDS (BMIDS) to decide whether behavior is acceptable and its increased detection sensitivity achieved by canceling out environmental noise.

Gupta et al. [105] confirmed that the threat of attacks in IoT systems affects not only the computational environment but also human life and the economy. For this reason, they proposed a computational-intelligence-based IDS for wireless communications and IoT systems. They proposed a three-tier architecture as the basis of an intelligent IDS suitable for wireless networks; this architecture consists of an information storage unit, a computational intelligence and optimization unit, and a clustering and intrusion reporting unit. This system depends on a machine learning approach for anomaly-based intrusion detection.

The machine learning approach applied in this IDS is based on the swarm intelligence (SI) paradigm, which is a specific type of computational intelligence (CI) paradigm [105]. The system targets IP addresses to detect attacks; thus, it has the disadvantage that it cannot be applied in regions of WSNs that do not use the TCP/IP protocol. The main feature of this system is its ability to operate as both a NIDS and a HIDS.

Raza et al. [106] proposed a hybrid-based IDS for IoT networks using 6LoWPAN for the detection of several RPL attacks. They proposed the SVELTE IDS, which consists of a 6LoWPAN mapper unit, an intrusion detection unit and a mini-firewall unit. The 6LoWPAN mapper unit collects information about the RPL network. The intrusion detection unit analyzes the data from the 6LoWPAN mapper unit to detect intrusions. The mini-firewall unit filters unwanted traffic. This system is designed for distributed and centralized IDS placement strategies. The main features of this system are its light weight and energy efficiency.

Khan and Herrmann [107] proposed three algorithms based on the protocol model approach using a trust management mechanism for IoT networks. One of these algorithms is neighbor based trust dissemination (NBTD), which can be used to implement a NIDS in a border router using the centralized approach. The second algorithm is tree based trust dissemination (TTD), which can be used to implement a HIDS in a small network with extra-high communication costs using the distributed approach. The third algorithm is clustered neighbor based trust dissemination (CNTD), which can be used to implement a NIDS using the distributed approach to reduce the number of packet exchanges compared with the NBTD algorithm. The main features of these algorithms are their light weight, energy efficiency and applicability in healthcare environments.

Performance analysis

In this section, a descriptive statistical analysis is applied to the reviewed papers based on several performance metrics: the TPR, FPR, energy consumption, processing time and performance overhead. The suitability of an IDS for IoT-based smart environments depends on these metrics; thus, they are the focus of this study.

In a traditional communication network, the performance of an IDS depends on the TPR and FPR only. In an IoT-based smart environment, the energy consumption, processing time and performance overhead of an IDS are also of critical interest. Because of the power and memory limitations of IoT devices, these metrics are very important to the QoS of an IoT system. Therefore, they are important performance metrics for an IDS designed for IoT-based smart environments.

Table 4 summarizes the technical details of the surveyed papers in terms of the important performance metrics of the proposed IDSs and their effects on smart environments. In Table 4, the symbol - indicates that no experimental result is available for the corresponding metric. The symbol * indicates that no numerical result was determined for this metric. The terms MAX and MAX range refer to the maximum result or maximum range result, respectively, for this metric.

Table 4 A descriptive statistical analysis of the surveyed IDSs in terms of the important performance metrics affecting IDS suitability in IoT systems
Full size table

Table 4 merely summarizes the experimental results from the surveyed papers; it is not intended as a comparison of these results. The experiments reported in these papers were performed under different conditions and using databases of different sizes. Thus, a standard IoT workbench with a standard IoT database would be required to conduct a fair comparison.

Tables 3 and 4 illustrate that the majority of researchers focus on three parameters during the testing stage: the TPR, FPR and processing time. Thus, IDSs are designed based on four features. First, such a system should be compatible with IoT systems. Second, it should be able to detect attacks in real time. Third, it should depend on lightweight algorithms. Fourth, it should be scalable.

Tables 3 and 4 show that some previous IDS studies did not present experimental results, whereas others presented only system methodologies that were not subjected to real experiments. Only a few previous IDS studies have presented practical results for the performance metrics that characterize the suitability of IDSs for IoT-based smart environments. Thus, the development of IDSs for IoT-based smart environments is still in an incipient phase.

Discussion and future outlook

Integrity, confidentiality, and availability are three important factors in IoT systems. In most cases, applications that use the IoT model are considered to be vital, such as industrial and medical applications. On the one hand, these applications can be real-time applications; thus, network delay and latency directly affect their performance. On the other hand, attacks such as DoS, DDoS, probing, and RPL attacks can degrade the usability of these applications. Thus, security issues can be considered a life-threatening concern in e-health systems, for example [108]. Consequently, powerful security measures are required in IoT networks. Such a security mechanism must protect the IoT network and its resources without impacting the system’s performance or user privacy.

Moreover, IoT-based smart environments consist of a wide range of devices, sensors and IoT objects from different vendors and based on different IoT platforms. Thus, interoperability issues prevent the emergence of IoT technology at a large scale [109]. Interoperability and standardization issues must be considered in designing IDSs for IoT-based smart environments.

IoT networks suffer from power efficiency problems; thus, a lightweight IDS that requires only a small number of computational operations is needed. In a HIDS, the IoT devices must simultaneously perform the necessary computational operations for the IDS and for IoT services. Thus, power resources and battery life must be considered in HIDS designs. Because of the power and memory limitations of IoT systems, the energy consumption, processing time and performance overhead of an IDS are important performance metrics. Thus, these metrics must be considered when designing IDSs for IoT-based smart environments. These issues should receive greater focus in research on HIDSs for such environments.

Privacy is another important factor in IoT systems. Deep packet inspection techniques are considered a violation of privacy. Such techniques and other techniques with similar characteristics are therefore undesirable. Moreover, the blocking of normal data packets affects IoT applications and services. This effect is very harmful, particularly for vital and real-time applications, such as industrial and medical applications. Therefore, introducing a smart system without deep packet inspection requires trusting that the operations in the IoT system will prevent any unauthorized access to IoT objects, thus helping to solve the user privacy problem. A new IDS design with a very low FPR and a very high detection accuracy is required for application in vital and real-time applications because traditional IDSs cannot satisfy these requirements.

An IDS based on a hybrid intrusion detection technique is required to detect different types of attacks from different computational environments. The IDS must be compatible with the 6LoWPAN protocol to detect attacks in WSNs in IoT networks. Furthermore, an autonomous IDS that can detect intrusions without human intervention is required for application in the IoT environment.

IDS placement is also a serious issue that must be considered when designing any type of IDS, whether it is a NIDS or a HIDS. The placement of the IDS in the IoT network will affect the overall efficiency of the IDS. There are two general IDS placement strategies: centralized and distributed. The centralized strategy offers the advantage of centralized management but can also lead to system processing overload, which may affect the QoS in IoT networks. The distributed strategy has the advantages of reducing the amount of monitored traffic and increasing the processing capacity. However, implementing an IDS in different regions of an IoT network is a challenge due to the associated management issues.

Finally, there is a need for both normal and anomaly databases that are up to date and integrated with IoT applications and services. These databases will be very useful for testing different IDS types and techniques in IoT environments. The ability to perform successful and meaningful IDS comparisons will depend on these databases.

Concluding remarks

As the numbers of IoT users, services, and applications increase, an urgent need for a robust and lightweight security solution that is suitable for use in IoT environments is emerging. Furthermore, IoT networks are the basis of smart environments; thus, any deficiencies in the security of these IoT networks will directly influence the smart environments on which they are based. Attacks such as DoS, DDoS, probing, and RPL attacks affect the services and applications offered in IoT-based smart environments; thus, the security of IoT environments is a very serious issue. An IDS is one possible solution for this issue. This has paper presented a survey of IDSs designed for IoT environments. Recommendations for designing a robust and lightweight IDS were also discussed.

In this survey, several papers were investigated. These papers mainly study the design and implementation of IDSs for use in the IoT paradigm that can be applied in smart environments. The features of all IDS methods presented in these papers were summarized. Moreover, this paper proposed some recommendations that must be considered when designing an IDS for the IoT, such as the need for a powerful and lightweight system with a suitable placement strategy that does not adversely affect the integrity, confidentiality, and availability of the IoT environment. This study showed that there is a need to design an integrated IDS that can be applied in IoT-based smart environments. This design will need to be tested on a unified IoT database. The question of the placement strategy must be considered in this design.

Future work will investigate the design of a high-performance hybrid IDS specifically designed for IoT-based smart environments based on the recommendations of this study. Moreover, the security vulnerabilities of IoT enabling technologies and protocols will be considered in the IDS design. In addition, the IDS will be implemented on programmable reconfigurable hardware devices, such as FPGAs, to facilitate adaptation to IoT-based smart environments. The design should be suitable for both distributed and centralized placement strategies and have the ability to detect different types of attacks.