CN114915446A - Intelligent network security detection method fusing priori knowledge - Google Patents
Intelligent network security detection method fusing priori knowledge Download PDFInfo
- Publication number
- CN114915446A CN114915446A CN202210340432.1A CN202210340432A CN114915446A CN 114915446 A CN114915446 A CN 114915446A CN 202210340432 A CN202210340432 A CN 202210340432A CN 114915446 A CN114915446 A CN 114915446A
- Authority
- CN
- China
- Prior art keywords
- vulnerability
- environment
- information
- host
- tested
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1416—Event detection, e.g. attack signature detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
- G06N5/022—Knowledge engineering; Knowledge acquisition
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1433—Vulnerability analysis
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Computer Hardware Design (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Debugging And Monitoring (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides an intelligent network security detection method fusing prior knowledge, which comprises the following steps: defining a characterization form of a vulnerability body, storing and extracting the vulnerability knowledge, and forming a vulnerability knowledge base; acquiring information from an environment to be measured; constructing a state information matrix; taking a state information matrix as an input of an agent and the vulnerability knowledge base; constructing an intelligent agent; determining a behavior strategy of an agent based on a state information matrix acquired by the environment to be tested and the potential vulnerability information acquired by the vulnerability knowledge base; and based on the behavior strategy of the intelligent agent, executing specific behaviors, calculating reward information by a reward module according to the execution result of the behaviors and the influence on the environment, feeding the reward information back to the intelligent agent, and guiding the intelligent agent to update the strategy. According to the scheme of the invention, the network security detection of the network environment is automatically realized, the efficiency of the network security detection is improved, and the problem that the automatic network security detection is difficult to apply in a complex environment is solved.
Description
Technical Field
The invention relates to the field of network space security, in particular to an intelligent network security detection method fusing priori knowledge.
Background
Periodic security testing is an important process for assessing asset resiliency and compliance, particularly confidentiality, availability, and integrity. Network security detection (networking Testing) is widely recognized as the best method to assess the security of digital assets by identifying and exploiting vulnerabilities. The safety expert faces the problems of complex environment, repeated operation and the like in the process of PT, and the PT task automation is obviously a method which saves manpower and resources and is efficient. Early research focused on improving PT systems by optimizing the planning phase, which was modeled as an attack graph or decision tree problem, reflecting the practical nature of continuous decision making. Nevertheless, due to the static nature of the method and its limitations on the planning phase, most of the work is related to vulnerability assessment, not PT.
In recent years, Machine Learning (ML) has opened up a new approach to effectively solve complex problems. ML has been demonstrated to be able to handle difficult problems faster and more accurately than humans in some cases. There are three types of ML: supervised learning, unsupervised learning, and reinforcement learning. Supervised and unsupervised learning has been used for intrusion detection, malware detection, privacy protection systems, and the like. Preparing large-scale data sets for training is a prerequisite for making security solutions. However, in such a real-time, continuous environment of network security detection, it is difficult to preprocess the behavior data set, and therefore, neither supervised nor unsupervised learning is suitable for solving the automated PT problem. Reinforcement Learning (RL) is a type of machine Learning that learns by exploration of the environment and accumulation of experience, and the intelligence of the RL can adapt itself to a real-time, continuous environment without a priori data sets.
In 2013, Sarraute et AL established a 4AL decomposition algorithm, which divides a large network into smaller networks according to the network structure, and solves one by one through a Partially Observable Markov Decision Process (POMDP). In 2014, Durkota et al proposed an algorithm for calculating an optimal attack strategy with an attack graph of action cost and failure probability, which converts the optimal path planning problem of the attack graph into a Markov Decision Process (MDP) and generates the optimal attack strategy to guide network security detection. In 2017, Shmaryahu et al modeled PT as a partially observable contingency problem and designed a contingency planning tree algorithm to plan an attack path. In the same year, Alexander preschner introduced POMDP to industrial control systems in an attempt to automatically verify the security of the industrial control system. In 2018, ghamem and Chen modeled the system as POMDP and tested using an external POMDP solver. In 2019, Weekly et al describe PT as an MDP process, and propose an attack planning (NIG-AP) algorithm based on network information gain. And acquiring rewards by utilizing network information, guiding the agent to select the optimal response action and finding a hidden attack path from the perspective of an intruder. In 2020, Hu et al constructs an automatic network security detection framework based on deep reinforcement learning, and automatically finds the optimal attack path of a given topological structure. In 2021, Zennaro et al formalized the simple CTF topic as a network security detection problem, which was solved based on model-free reinforcement learning.
The research result based on the POMDP confirms the hypothesis that the reinforcement learning can improve the accuracy and reliability of the network security detection. However, because the number of hosts in the network security detection environment is large, the configuration of the hosts is complex, and it is very difficult to accurately solve the POMDP. MDP-based reinforcement learning may in principle allow model-less learning, but may in practice rely on some form of a priori knowledge to solve the problem.
Disclosure of Invention
In order to solve the technical problems, the invention provides an intelligent network security detection method fusing priori knowledge, which is used for solving the technical problems that an automatic network security detection method in the prior art is low in efficiency, low in practicability and difficult to apply to a real large-scale network scene.
According to a first aspect of the present invention, there is provided an intelligent network security detection method incorporating prior knowledge, the method comprising the steps of:
step S1: defining a characterization form of a vulnerability ontology, wherein the vulnerability ontology is characterized based on the concept, attribute and relationship of the vulnerability ontology; extracting vulnerability knowledge from the acquired knowledge source based on the acquired knowledge source and the characterization form of the vulnerability ontology; storing the extracted vulnerability knowledge to form a vulnerability knowledge base;
step S2: the method comprises the following steps of constructing an environment information acquisition module, wherein the environment information acquisition module is used for acquiring the following basic information from an environment to be tested: operating a host IP, an operating system, a survival port and service information, and storing the acquired basic information according to the category number;
step S3: judging whether a preset target is reached, if so, ending the method; if not, go to step S4; the preset target is used for realizing network security detection on a specific target;
step S4: acquiring basic information of the environment to be detected based on the environment information acquisition module; acquiring a network topology structure, host authority and host configuration information of the environment to be tested based on the serial number information, and constructing a state information matrix; taking the state information matrix as the input of an agent and the vulnerability knowledge base;
step S5: constructing an intelligent agent; determining a behavior strategy of the agent based on the environment state information matrix and the potential vulnerability information acquired by the vulnerability knowledge base;
step S6: based on the behavior strategy of the intelligent agent, executing specific behaviors, acting the specific behaviors on the environment to be tested, calculating reward information by a reward module according to the execution result of the behaviors and the influence on the environment, feeding the reward information back to the intelligent agent, and guiding the intelligent agent to update the strategy; the process advances to step S3.
Further, the vulnerability ontology is characterized based on the concept, attribute and relationship of the vulnerability ontology, the vulnerability ontology refers to defects existing in the specific implementation of hardware, software and protocols or system security strategies, the attribute of the vulnerability ontology refers to potential conditions of the vulnerability, and the relationship of the vulnerability ontology refers to an interaction relationship among vulnerabilities, wherein the attribute of the vulnerability ontology includes the utilization mode of the vulnerability, the effect and influence of the vulnerability, whether vulnerability is utilized, whether the vulnerability contains the service of the vulnerability and the operating system corresponding to the service containing the vulnerability, and the relationship includes an intersection relationship, an inheritance relationship and an attribute relationship.
Further, the step S2, wherein:
the preset target is to realize network security detection of a specific target, and comprises the network security detection of a specific host in a network environment and/or the network security detection of a single host from a certain starting host.
Further, the environment state information matrix to be measured is defined as follows:
wherein h is i h j The connection relation between the ith host computer to be tested and the jth host computer to be tested is represented, 0 represents that the host computers to be tested are not communicated, 1 represents that the host computers to be tested are communicated, and h represents that the host computers to be tested are communicated i h i Representing the authority level acquired on the host i to be tested, and setting the number of the nodes to be a fixed value p due to different network node numbers in different network security detection environments k (h i ) Indicating whether the host i to be tested contains an attribute with the number k, privilege (h) i ) Indicating the authority of the agent on the host i to be tested.
According to a second aspect of the present invention, there is provided an intelligent network security detection apparatus fusing prior knowledge, the apparatus comprising:
a vulnerability knowledge base construction module: the vulnerability analysis method comprises the steps that a characterization form of a vulnerability ontology is configured to be defined, and the vulnerability ontology is characterized based on the concept, the attribute and the relation of the vulnerability ontology; extracting vulnerability knowledge from the acquired knowledge source based on the acquired knowledge source and the characterization form of the vulnerability ontology; storing the extracted vulnerability knowledge to form a vulnerability knowledge base;
a detection module: the environment information acquisition module is configured to construct an environment information acquisition module, and the environment information acquisition module is used for acquiring the following basic information from the environment to be measured: operating a host IP, an operating system, a survival port and service information, and storing the acquired basic information according to the category number;
a judging module: the method comprises the steps of configuring to judge whether a preset target is reached, wherein the preset target is to realize network security detection on a specific target;
the state information matrix construction module: the environment information acquisition module is configured to acquire basic information of an environment to be measured; acquiring a network topology structure, host authority and host configuration information of the environment to be tested based on the serial number information, and constructing a state information matrix; taking the state information matrix as the input of an agent and the vulnerability knowledge base;
a behavior determination module: configured to build an agent; determining a behavior strategy of the agent based on the environment state information matrix and the potential vulnerability information obtained by the vulnerability knowledge base;
an update module: the intelligent agent management system is configured to execute specific behaviors based on behavior strategies of the intelligent agent, act the specific behaviors on the environment to be tested, calculate reward information by a reward module according to the execution result of the behaviors and the influence on the environment, feed the reward information back to the intelligent agent and guide the intelligent agent to update the strategies; and a triggering judgment module.
According to a third aspect of the present invention, an intelligent network security detection system fusing prior knowledge is provided, which includes:
a processor for executing a plurality of instructions;
a memory to store a plurality of instructions;
wherein the plurality of instructions are for storage by the memory and for loading and executing the method as previously described by the processor.
According to a fourth aspect of the present invention, there is provided a computer readable storage medium having a plurality of instructions stored therein; the plurality of instructions for being loaded by a processor and performing the method as described above.
According to the scheme of the invention, the method aims at realizing automatic and intelligent network security detection, integrates prior knowledge, and provides a network security detection method based on knowledge graph and reinforcement learning. The method combines the relevant technology of reinforcement learning in the field of artificial intelligence and knowledge graph to construct an automatic model, and intelligent network security detection is realized. The invention aims to solve the problems that the network environment is complex, the types and the number of vulnerabilities are various, and the automatic network security detection is difficult to realize. The method of the invention adopts a method of combining knowledge map and reinforcement learning to solve the problems and realize the following effects: (1) the vulnerability knowledge base is constructed based on the knowledge graph, so that the potential vulnerability information in the current host and the network can be managed and inquired conveniently, and meanwhile, the convenience is provided for the intelligent agent to analyze the optional effective behaviors; (2) the method provided by the invention changes the behavior selection mode of the intelligent agent, changes the intelligent agent from randomly exploring available behaviors to selecting the optimal behavior from the available behaviors, and greatly improves the learning efficiency of the intelligent agent; (3) the method can realize the automatic and intelligent network security detection of the network environment.
The foregoing description is only an overview of the technical solutions of the present invention, and in order to make the technical solutions of the present invention more clearly understood and to implement them in accordance with the contents of the description, the following detailed description is given with reference to the preferred embodiments of the present invention and the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention. In the drawings:
FIG. 1 is a flow chart of an embodiment of the invention of an intelligent network security detection method incorporating prior knowledge;
FIG. 2 is a schematic structural diagram of an intelligent network security detection model with a priori knowledge fused according to an embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating a vulnerability knowledge base construction method according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a vulnerability ontology according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of an intelligent agent neural network architecture according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an intelligent network security detection device incorporating prior knowledge according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the specific embodiments of the present invention and the accompanying drawings. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
First, a flowchart of an intelligent network security detection method incorporating a priori knowledge according to an embodiment of the present invention is described with reference to fig. 1-2. As shown in fig. 1-2, the method comprises the steps of:
step S1: defining a characterization form of a vulnerability ontology, wherein the vulnerability ontology is characterized based on the concept, attribute and relationship of the vulnerability ontology; extracting vulnerability knowledge from the acquired knowledge source based on the acquired knowledge source and the characterization form of the vulnerability ontology; storing the extracted vulnerability knowledge to form a vulnerability knowledge base;
step S2: the method comprises the following steps of constructing an environment information acquisition module, wherein the environment information acquisition module is used for acquiring the following basic information from an environment to be tested: operating a host IP, an operating system, a survival port and service information, and storing the acquired basic information according to the category number;
step S3: judging whether a preset target is reached, if so, ending the method; if not, go to step S4; the preset target is used for realizing network security detection on a specific target;
step S4: acquiring basic information of the environment to be detected based on the environment information acquisition module; based on the number information, acquiring a network topology structure, host authority and host configuration information of the environment to be tested, and constructing a state information matrix; taking the state information matrix as the input of an agent and the vulnerability knowledge base;
step S5: constructing an intelligent agent; determining a behavior strategy of the agent based on the environment state information matrix and the potential vulnerability information acquired by the vulnerability knowledge base;
step S6: based on the behavior strategy of the intelligent agent, executing specific behaviors, acting the specific behaviors on the environment to be tested, calculating reward information by a reward module according to the execution result of the behaviors and the influence on the environment, feeding the reward information back to the intelligent agent, and guiding the intelligent agent to update the strategy; the process advances to step S3.
Because the network security detection is a sequential decision problem, the task decomposition is to judge the network security detection behavior to be adopted according to the current state, so that the network security detection can be modeled as a Markov decision process, and the automatic network security detection is realized based on the relevant technology of reinforcement learning. The intelligent agent for reinforcement learning can accumulate experience in the training process and make a decision like a security expert, however, the cost is that a large amount of training is used, the network security detection environment is complex, the types and the number of vulnerabilities are too many, and the intelligent agent is difficult to realize quick and effective learning, so that the vulnerability knowledge base is added into an automatic model, the vulnerability knowledge base is introduced to serve as a library of the intelligent agent, after the intelligent agent obtains the state, feasible operation is inquired in the vulnerability knowledge base, and finally adopted behaviors are selected from the feasible behaviors according to the learned experience, so that the learning efficiency is improved.
The method constructs the relevant information of the knowledge graph storage vulnerability, and realizes matching the vulnerability which possibly exists according to the current state information; the automatic network security detection method based on reinforcement learning is realized, the state information acquired by an intelligent agent from the environment is firstly subjected to vulnerability matching, and then the next action is selected from the matched actions.
The step S1, wherein:
the vulnerability information has the characteristics of quantization, decentralization and fragmentation, and the important condition for constructing the vulnerability knowledge base is to collect the vulnerability information. The method mainly adopts internet query and acquisition, and currently, the international relatively known Vulnerability related standards and specifications include CVE (Common virtualization & deployment), cpe (Common platform authorization), cvss (Common virtualization profiling system), and cnnvd (Chinese National virtualization Database of Information security). The CVE is the security vulnerability disclosure and release unit with the most public credibility in the world at present; CPE is a standardized method for describing and identifying the applications, operating systems and hardware devices present in enterprise computing assets; CVSS is an industry public standard used for evaluating the severity of a vulnerability and helping to determine the urgency and importance of response; the CNNVD is a domestic authoritative vulnerability listing platform. Collecting vulnerability information from the CVE, the CPE, the CVSS and the CNNVD, integrating the vulnerability information, and taking the obtained information as a knowledge source.
The method comprises the steps that a characterization form of a vulnerability body is defined based on a knowledge graph, the vulnerability body is characterized based on the concept, the attribute and the relation of the vulnerability body, the vulnerability body is a defect existing in hardware, software and protocol specific implementation or system security strategies, the attribute of the vulnerability body is a potential condition of the vulnerability, the relation of the vulnerability body is an interaction relation between vulnerabilities, wherein the attribute of the vulnerability body comprises the utilization mode of the vulnerability, the effect and the influence of the vulnerability, whether vulnerability utilization exists, whether the vulnerability contains the service of the vulnerability and an operating system corresponding to the service containing the vulnerability, and the relation comprises an intersection relation, an inheritance relation and an attribute relation.
In this embodiment, the defined vulnerability ontology is shown in fig. 4. The vulnerability information is reasonably represented, therefore, a vulnerability body is constructed for the association between a common operating system, common services and common vulnerabilities according to expert experience, the association between vulnerability related knowledge can be better described, the structure of the vulnerability body is shown in figure 4, wherein the serial number is an internal serial number, and the serial number is consistent with the serial number in the environmental state information, so that the intelligent agent can conveniently match the vulnerability according to the input state information.
And extracting vulnerability knowledge from the acquired knowledge source based on the acquired knowledge source and the characterization form of the vulnerability ontology. In this embodiment, the information in the knowledge source comes from different specifications and standards, and may include repeated or structurally different information, so that the information in the knowledge source is extracted and cleaned by using a named entity recognition technology, and then the relation extraction is performed, and finally the knowledge extraction of the vulnerability information is realized.
And storing and extracting the vulnerability knowledge to form a vulnerability knowledge base, namely selecting the vulnerability knowledge with the importance degree greater than a preset threshold value, and storing the vulnerability knowledge related to the state of the network to be tested and the host to be tested into a database by adopting Neo4j to realize the construction of the vulnerability knowledge base.
When the security expert performs network security detection, the security expert can judge possible bugs in the current environment according to the scanned network and host state information, and realize network security detection by using the bugs. In the process, the judgment of the expert is based on the accumulation of self knowledge, so that the lack of expert experience is one of the great challenges facing the current automatic network security detection. Therefore, the invention constructs a priori knowledge base similar to expert experience, collects vulnerability information, wherein the vulnerability information comprises vulnerability numbers, vulnerability grades, vulnerability sources, functions which can be realized by utilizing vulnerabilities and the like, extracts useful information through a named entity identification technology, processes the information to normalize the information, constructs a knowledge reasoning model and realizes the management of the vulnerability information.
The step S3, wherein:
the preset target is to realize the network security detection of a specific target in the network to be detected, and comprises the network security detection of a specific host in the network environment and/or the network security detection of a single host from a host in the network to be detected.
The step S4, wherein:
and based on the environment information acquisition module, acquiring basic information of the environment to be detected, classifying and numbering according to information content, constructing an environment state matrix based on the numbered environment information as input information of the intelligent agent, wherein the environment state matrix comprises information such as a network topology structure, host configuration and the like explored by the intelligent agent at present. According to the information scanned from the environment by the expert in the actual network security detection process, the to-be-detected environment state information matrix of the automatic network security detection model is defined as follows:
wherein h is i h j The connection relation between the ith host computer to be tested and the jth host computer to be tested is represented, 0 represents that the host computers to be tested are not communicated, 1 represents that the host computers to be tested are communicated, h represents that the host computers to be tested are communicated i h i Representing the authority level acquired on the host i to be tested, and setting the number of the nodes to be a fixed value p due to different network node numbers in different network security detection environments k (h i ) Indicating whether the host i to be tested contains an attribute with the number k, privilege (h) i ) Indicating the authority of the agent on the host i to be tested.
The state information matrix is used as the input of the intelligent agent and the leak library, and the state matrix is used as the input of the intelligent agent to provide scene information for the intelligent agent; and the state information matrix is used as the input of the vulnerability database to predict the potential vulnerability of the environment to be tested, so that a more accurate behavior space is provided for the intelligent agent.
The step S5, wherein:
the method comprises the steps that an intelligent agent determines behaviors based on the output of a vulnerability prior knowledge base, wherein the behaviors are the output of the intelligent agent and represent the decision made by the intelligent agent aiming at the current environment, the output of the intelligent agent comprises the connection behaviors between hosts in a network to be tested and vulnerability utilization behaviors, and the connection behaviors between the hosts refer to the behaviors of the intelligent agent in transverse movement between the hosts; the vulnerability exploitation is characterized in that the vulnerability information in the vulnerability knowledge base is matched through analysis of an input state, the vulnerability possibly existing in a network environment to be tested is judged, the vulnerability corresponding to the possible vulnerability is combined with connection behaviors between hosts found in the environment to be tested to form a behavior library which can be selected by an intelligent body, in order to guarantee stability of a model in a learning process, the size of the behavior library is set to be a fixed value, actions are selected and executed from the given behavior library, and the method is different from the method that the intelligent body freely explores in all actions in the past.
In the reinforcement learning-based automatic network security detection model, the input received by the agent is state information acquired from the environment, and the probability of taking each action for the agent is output. Due to the complexity of the network security detection environment, the size of the state space exponentially increases along with the network scale and the host configuration, and the traditional tabular method such as the Q-learning method is not suitable for realizing automatic network security detection, so that the invention introduces a deep reinforcement learning technology, and can effectively solve the problem of overlarge state space based on the fitting Q function of the neural network. However, due to the introduction of a priori knowledge, the behavior that the agent can take in different states is different, resulting in inconsistent number of output nodes. Aiming at the situations, a reinforcement learning algorithm needs to be redesigned to realize intelligent decision.
The neural network structure for updating the Q value of the intelligent agent is shown in fig. 5, wherein the neural network model comprises three convolutional layers, and the third convolutional layer is connected with a full connection layer; since the values in the state matrix are mostly 0 or 1, and the matrix is sparse, no pooling layer is used. And taking the environment state matrix acquired from the environment to be measured as the input of the first convolution layer, and outputting the characteristics of the environment state matrix by the full-connection layer. In this embodiment, the maximum network node number is set to 100, the host configuration includes information such as services and ports, and the number is set to 100, so the input matrix size is 100 x 200. And matching the received state information with a priori knowledge base, selecting the vulnerability corresponding to the vulnerability at the top 10 of the ranking to construct a behavior space of the agent, and adding connection behaviors besides the vulnerability behaviors to indicate the behaviors of moving from the current host to other hosts, so that the size of the behavior space output by the agent is set to be 11. In addition, hyper-parameters such as the number of convolutional layers, the number of convolutional kernels per layer, and the sizes of the convolutional kernels of the respective layers need to be determined by experiments. And selecting proper output layer activation function and loss function according to task requirements.
The step S6, wherein:
the reward is feedback on the behavior of the intelligent agent, is important for reinforcement learning, determines the learning direction and the convergence speed of the intelligent agent, and influences the correctness and the effectiveness of the decision of the intelligent agent, and the reward is divided into a positive feedback part and a negative feedback part: positive rewards resulting from successful performance of the agent and negative rewards resulting from failed performance of the agent. And the intelligent agent receives the reward, adjusts the neural network parameters according to the reward value, guides the intelligent agent to update the strategy and makes more accurate prediction. The steps are repeated until the target in step S3 is satisfied.
Fig. 6 is a schematic structural diagram of an intelligent network security detection apparatus fusing a priori knowledge according to an embodiment of the present invention, as shown in fig. 6, the apparatus includes:
a vulnerability knowledge base construction module: the vulnerability analysis method comprises the steps that a characterization form of a vulnerability ontology is configured to be defined, and the vulnerability ontology is characterized based on the concept, the attribute and the relation of the vulnerability ontology; extracting vulnerability knowledge from the acquired knowledge source based on the acquired knowledge source and the characterization form of the vulnerability ontology; storing the extracted vulnerability knowledge to form a vulnerability knowledge base;
a detection module: the method comprises the following steps of configuring an environment information acquisition module, wherein the environment information acquisition module is used for acquiring the following basic information from an environment to be tested: operating a host IP, an operating system, a survival port and service information, and storing the acquired basic information according to the category number;
a judging module: the method comprises the steps of configuring to judge whether a preset target is reached, wherein the preset target is to realize network security detection on a specific target;
the state information matrix construction module: the environment information acquisition module is configured to acquire basic information of an environment to be measured; based on the number information, acquiring a network topology structure, host authority and host configuration information of the environment to be tested, and constructing a state information matrix; taking the state information matrix as the input of an agent and the vulnerability knowledge base;
a behavior determination module: configured to build an agent; determining a behavior strategy of the agent based on the environment state information matrix and the potential vulnerability information acquired by the vulnerability knowledge base;
an update module: the intelligent agent management system is configured to execute specific behaviors based on behavior strategies of the intelligent agent, act the specific behaviors on the environment to be tested, calculate reward information by a reward module according to the execution result of the behaviors and the influence on the environment, feed the reward information back to the intelligent agent and guide the intelligent agent to update the strategies; and a triggering judgment module.
The embodiment of the invention further provides an intelligent network safety detection system fusing prior knowledge, which comprises:
a processor for executing a plurality of instructions;
a memory to store a plurality of instructions;
wherein the plurality of instructions are for storage by the memory and for loading and executing the method as previously described by the processor.
The embodiment of the invention further provides a computer readable storage medium, wherein a plurality of instructions are stored in the storage medium; the plurality of instructions for being loaded by a processor and performing the method as described above.
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
In the embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions in actual implementation, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer-readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a physical machine Server, or a network cloud Server, etc., and needs to install a Windows or Windows Server operating system) to perform some steps of the method according to various embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any way, and any simple modification, equivalent change and modification made to the above embodiment according to the technical spirit of the present invention are still within the scope of the technical solution of the present invention.
Claims (7)
1. An intelligent network security detection method fused with prior knowledge is characterized by comprising the following steps:
step S1: defining a characterization form of a vulnerability ontology, wherein the vulnerability ontology is characterized based on the concept, the attribute and the relation of the vulnerability ontology; extracting vulnerability knowledge from the acquired knowledge source based on the acquired knowledge source and the characterization form of the vulnerability ontology; storing the extracted vulnerability knowledge to form a vulnerability knowledge base;
step S2: the method comprises the following steps of constructing an environment information acquisition module, wherein the environment information acquisition module is used for acquiring the following basic information from an environment to be tested: operating a host IP, an operating system, a survival port and service information, and storing the acquired basic information according to the category number;
step S3: judging whether a preset target is reached, if so, ending the method; if not, go to step S4; the preset target is used for realizing network security detection on a specific target;
step S4: acquiring basic information of an environment to be detected based on the environment information acquisition module; based on the number information, acquiring a network topology structure, host authority and host configuration information of the environment to be tested, and constructing a state information matrix; taking the state information matrix as the input of an agent and the vulnerability knowledge base;
step S5: constructing an intelligent agent; determining a behavior strategy of the agent based on the environment state information matrix and the potential vulnerability information obtained by the vulnerability knowledge base;
step S6: based on the behavior strategy of the intelligent agent, executing specific behaviors, acting the specific behaviors on the environment to be tested, calculating reward information by a reward module according to the execution result of the behaviors and the influence on the environment, feeding the reward information back to the intelligent agent, and guiding the intelligent agent to update the strategy; the process advances to step S3.
2. The method of claim 1, wherein the vulnerability ontology is characterized based on concepts, attributes and relationships of the vulnerability ontology, the vulnerability ontology refers to defects existing in specific implementation of hardware, software and protocols or system security policies, the attributes of the vulnerability ontology refer to potential conditions for the existence of vulnerabilities, and the relationships of the vulnerability ontology refer to interaction relationships among vulnerabilities, wherein the attributes of the vulnerability ontology include vulnerability exploitation modes, vulnerability generation effects and influences, whether vulnerability exploitation exists, whether services including the vulnerability and operating systems corresponding to the services including the vulnerability, and the relationships include intersection relationships, inheritance relationships and attribute relationships.
3. The method of claim 2, wherein said step S2, wherein:
the preset target is to realize network security detection of a specific target, and comprises the network security detection of a specific host in a network environment and/or the network security detection of a single host from a certain starting host.
4. The method of claim 3, wherein the environment state information matrix to be measured is defined as follows:
wherein h is i h j The connection relation between the ith host computer to be tested and the jth host computer to be tested is represented, 0 represents that the host computers to be tested are not communicated, 1 represents that the host computers to be tested are communicated, h represents that the host computers to be tested are communicated i h i The permission level obtained on the host i to be tested is shown, and the number of the nodes is set to be a fixed value p due to different network node numbers in different network security detection environments k (h i ) Indicating whether the host i to be tested contains an attribute with the number k, privilege (h) i ) Indicating the authority of the agent on the host i to be tested.
5. An intelligent network security detection device fusing prior knowledge, the device comprising:
a vulnerability knowledge base construction module: the vulnerability analysis method comprises the steps that a characterization form of a vulnerability ontology is configured to be defined, and the vulnerability ontology is characterized based on the concept, the attribute and the relation of the vulnerability ontology; extracting vulnerability knowledge from the acquired knowledge source based on the acquired knowledge source and the characterization form of the vulnerability ontology; storing the extracted vulnerability knowledge to form a vulnerability knowledge base;
a detection module: the method comprises the following steps of configuring an environment information acquisition module, wherein the environment information acquisition module is used for acquiring the following basic information from an environment to be tested: operating a host IP, an operating system, a survival port and service information, and storing the acquired basic information according to the category number;
a judging module: the method comprises the steps of configuring to judge whether a preset target is reached, wherein the preset target is to realize network security detection on a specific target;
the state information matrix construction module: the environment information acquisition module is configured to acquire basic information of an environment to be measured; acquiring a network topology structure, host authority and host configuration information of the environment to be tested based on the serial number information, and constructing a state information matrix; taking the state information matrix as the input of an agent and the vulnerability knowledge base;
a behavior determination module: configured to build an agent; determining a behavior strategy of the agent based on the environment state information matrix and the potential vulnerability information obtained by the vulnerability knowledge base;
an update module: the intelligent agent management system is configured to execute specific behaviors based on behavior strategies of the intelligent agent, act the specific behaviors on the environment to be tested, calculate reward information by a reward module according to the execution result of the behaviors and the influence on the environment, feed the reward information back to the intelligent agent and guide the intelligent agent to update the strategies; and a triggering judgment module.
6. An intelligent network security detection system fusing prior knowledge, comprising:
a processor for executing a plurality of instructions;
a memory to store a plurality of instructions;
wherein the plurality of instructions are for storage by the memory and for loading and execution by the processor of the method of any of claims 1-4.
7. A computer-readable storage medium having stored therein a plurality of instructions; the plurality of instructions for being loaded by a processor and for performing the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210340432.1A CN114915446B (en) | 2022-04-02 | 2022-04-02 | Intelligent network security detection method integrating priori knowledge |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210340432.1A CN114915446B (en) | 2022-04-02 | 2022-04-02 | Intelligent network security detection method integrating priori knowledge |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114915446A true CN114915446A (en) | 2022-08-16 |
| CN114915446B CN114915446B (en) | 2023-08-29 |
Family
ID=82763246
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210340432.1A Active CN114915446B (en) | 2022-04-02 | 2022-04-02 | Intelligent network security detection method integrating priori knowledge |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114915446B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117097627A (en) * | 2023-10-19 | 2023-11-21 | 中国人民解放军国防科技大学 | Permeation test agent training and verification environment construction method and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108933793A (en) * | 2018-07-24 | 2018-12-04 | 中国人民解放军战略支援部队信息工程大学 | The attack drawing generating method and its device of knowledge based map |
| CN111639344A (en) * | 2020-07-31 | 2020-09-08 | 中国人民解放军国防科技大学 | Vulnerability detection method and device based on neural network |
| CN113919485A (en) * | 2021-10-19 | 2022-01-11 | 西安交通大学 | Multi-agent reinforcement learning method and system based on dynamic hierarchical communication network |
-
2022
- 2022-04-02 CN CN202210340432.1A patent/CN114915446B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108933793A (en) * | 2018-07-24 | 2018-12-04 | 中国人民解放军战略支援部队信息工程大学 | The attack drawing generating method and its device of knowledge based map |
| CN111639344A (en) * | 2020-07-31 | 2020-09-08 | 中国人民解放军国防科技大学 | Vulnerability detection method and device based on neural network |
| CN113919485A (en) * | 2021-10-19 | 2022-01-11 | 西安交通大学 | Multi-agent reinforcement learning method and system based on dynamic hierarchical communication network |
Non-Patent Citations (2)
| Title |
|---|
| ANDREW M. SAXE: "A mathematical theory of semantic development in deep neural networks", PNAS * |
| LAURA VON RUEDEN: "Informed Machine Learning – A Taxonomy and Survey of Integrating Prior Knowledge into Learning Systems", IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117097627A (en) * | 2023-10-19 | 2023-11-21 | 中国人民解放军国防科技大学 | Permeation test agent training and verification environment construction method and electronic equipment |
| CN117097627B (en) * | 2023-10-19 | 2023-12-22 | 中国人民解放军国防科技大学 | Permeation test agent training and verification environment construction method and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114915446B (en) | 2023-08-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Anton et al. | Anomaly-based intrusion detection in industrial data with SVM and random forests | |
| CN111523119B (en) | Vulnerability detection method and device, electronic equipment and computer readable storage medium | |
| EP3490223B1 (en) | System and method for simulating and foiling attacks on a vehicle on-board network | |
| CN112131882A (en) | Multi-source heterogeneous network security knowledge graph construction method and device | |
| WO2019175880A1 (en) | Method and system for classifying data objects based on their network footprint | |
| Vokorokos et al. | Intrusion detection system using self organizing map | |
| US11106801B1 (en) | Utilizing orchestration and augmented vulnerability triage for software security testing | |
| US11625483B2 (en) | Fast identification of trustworthy deep neural networks | |
| Berghout et al. | EL-NAHL: Exploring labels autoencoding in augmented hidden layers of feedforward neural networks for cybersecurity in smart grids | |
| CN115102705A (en) | Automatic network security detection method based on deep reinforcement learning | |
| CN114329455B (en) | User abnormal behavior detection method and device based on heterogeneous graph embedding | |
| CN114036531A (en) | Multi-scale code measurement-based software security vulnerability detection method | |
| CN115296876A (en) | Network security early warning system of self-adaptation mimicry technique | |
| CN113965497B (en) | Server abnormity identification method and device, computer equipment and readable storage medium | |
| Zarai | Recurrent Neural Networks & Deep Neural Networks Based on Intrusion Detection System | |
| CN114915446A (en) | Intelligent network security detection method fusing priori knowledge | |
| CN112822184B (en) | Unsupervised autonomous attack detection method in endogenous security system | |
| CN113886829A (en) | Method and device for detecting defect host, electronic equipment and storage medium | |
| Maseer et al. | Meta‐analysis and systematic review for anomaly network intrusion detection systems: Detection methods, dataset, validation methodology, and challenges | |
| CN113434857A (en) | User behavior safety analysis method and system applying deep learning | |
| CN117828586A (en) | Power data attack tracking and tracing method and system | |
| Alagrash et al. | Machine learning and recognition of user tasks for malware detection | |
| CN112436969A (en) | Internet of things equipment management method, system, equipment and medium | |
| Woodward et al. | Uncertainty quantification for trusted machine learning in space system cyber security | |
| CN115454473A (en) | Data processing method based on deep learning vulnerability decision and information security system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |