CN105681462A - Cluster system based on message router, and data communication transfer method - Google Patents

Abstract

The invention discloses a cluster system based on message routing and a data communication transfer method. The cluster system includes at least one sending node, at least one receiving node, and message routing, wherein the message routing includes an interface processing module, a queue management module, and a message channel Agent module, message queue module, security management module. The data communication transfer method implements message forwarding between nodes based on message routing. This method solves the problems of huge network overhead caused by data communication in the cluster system, improves the efficiency of the entire system, and ensures the healthy and stable operation of the server. At the same time, the specific process of data communication is described. As the middleware of computing node communication, message routing avoids direct communication between nodes, greatly reduces the coupling degree between nodes and the risk of node access, prevents nodes from being illegally invaded, and ensures the cluster system safety.

Description

A cluster system and data communication transfer method based on message routing

technical field

The invention relates to a cluster system based on message routing and a data communication transfer method, and belongs to the technical field of information technology data management applications.

Background technique

A cluster system is a parallel or distributed processing system, which uses multiple homogeneous or heterogeneous computers as independent computing nodes to associate with each other through a local area network or other networks, and cooperate to complete some tasks, so that the resources of each node can be effectively utilized. . Each computing node has its own CPU, memory, and independent operating system. The cluster system encapsulates the underlying implementation details and unifies the resources of all nodes into a collection. From the perspective of users, the cluster system is a single system, while non-computer groups.

As early as the 1970s and 1980s, some companies began to research and develop cluster systems. Cluster systems have the characteristics of high scalability, high availability, and high performance. They can provide powerful computing capabilities and handle complex calculation problem. Generally speaking, cluster systems are mainly divided into high-availability clusters and high-performance clusters. The main function of the high-availability cluster ensures uninterrupted service. When a node in the cluster fails, it automatically and quickly handles the failed node without human intervention, and transfers node tasks so that users will not be aware of the failure. The high-performance cluster is mainly manifested in processing power. It uses the network to connect multiple common computing nodes and makes them work together. The processing power can reach the performance of a mainframe.

With the continuous improvement of the performance and availability of cluster systems, high-performance clusters have gradually replaced traditional supercomputers due to their high cost performance and strong scalability, and have become the focus of research in the computing field. However, the cluster system needs to provide services 7×24 hours without interruption, and the computing nodes need to run continuously all year round. During this period, the nodes continuously exchange information. As the number of nodes in the cluster increases, the communication between nodes becomes more and more frequent, which will easily lead to the transmission of a large amount of node information on the network, resulting in network congestion. As the size of the cluster expands, the cluster system gradually begins to show performance problems. The traditional inter-node communication method in the cluster will generate a huge amount of communication between the networks, resulting in a large amount of additional network communication burden and reducing the efficiency of the entire system. In addition, if the computing nodes communicate directly, this will undoubtedly increase the access risk of the nodes and easily affect the performance of the nodes. The increase of node accessible objects means that the node security index decreases.

In order to solve the above problems, the current method is usually to improve the system configuration of the computing nodes, increase the network bandwidth, and ensure the high performance and high availability of the server, which increases the system cost and does not fundamentally solve the problem.

Contents of the invention

The technical problem to be solved by the present invention is: to provide a cluster system and data communication transfer method based on message routing, using message routing as the middleware for communication between computing nodes, avoiding illegal intrusion of nodes, and greatly reducing the access risk of nodes, Ensure the healthy and stable operation of the server.

The present invention adopts the following technical solutions for solving the problems of the technologies described above:

A cluster system based on message routing, including at least one sending node, at least one receiving node, and message routing, wherein the message routing includes an interface processing module, a queue management module, a message channel agent module, and a message queue module;

The interface processing module is used to accept the service request of the sending node, and analyze the service request as a queue change request or a message delivery request, and call the corresponding queue management module or message channel proxy module according to different request types;

The message queue module is used to save the independent message queue corresponding to each node, and the message sent by the corresponding node is stored in the message queue, and each node only subscribes to the message queue corresponding to itself;

The queue management module is used to operate the related message queues in the message queue module according to the queue change request fed back by the interface processing module;

The message channel proxy module is used to complete message push or message extraction to the message queue module according to the message delivery request fed back by the interface processing module. If it is a message push, it extracts the queue ID from the message, and pushes the message into The corresponding message queue; if it is a message extraction, the message is extracted from the corresponding message queue, and fed back to the interface processing module, and then sent to the receiving node by the interface processing module.

Further, the message routing further includes a security management module, which is used to encrypt and decrypt messages when the interface processing module, the message channel agent module, and the message queue module transmit messages.

A data communication transfer method based on message routing, the method utilizes message routing to realize communication between a sending node and a receiving node, and the communication steps between the sending node and the receiving node are as follows:

Step 1, the sending node sends a signaling message including the message queue identifier to the message router, and requests the receiving node subscribed to the message queue to respond;

Step 2: After receiving the signaling message, the message router analyzes and extracts the queue identifier, and pushes the signaling message into the corresponding message queue according to the queue identifier;

Step 3. The receiving node subscribes to the message queue and continuously monitors the message queue. Once a message is pushed into the message queue, it immediately extracts the message, and sends a response message to the message router after extracting the message;

Step 4: After the message routing receives the response message, it pushes the message into the message queue corresponding to the sending node according to the queue identifier contained in the message, and the sending node subscribes to its own corresponding message queue, extracts the message from the message queue, and thus obtains the received Node's response message.

Further, before step 1, it is judged whether there is a message queue corresponding to the sending node and the receiving node in the message queue module of the message routing. Create an independent message queue corresponding to it.

Preferably, when the sending node and the receiving node respectively establish a communication connection with the message router, a session mode and a message confirmation mechanism are attached.

Preferably, the session mode is a non-transactional session, and the message confirmation mechanism is automatic confirmation.

Further, after step 4, the sending node sets the time to obtain the response message. If the response message is not obtained within the set time, the sending node calls the judgment function to judge whether the receiving node is invalid, and when the receiving node fails, report to the management Nodes report failures.

Preferably, the set time is 1 second.

Compared with the prior art, the present invention adopts the above technical scheme and has the following technical effects:

1. The cluster system and data communication transfer method based on message routing in the present invention make it unnecessary for the source node to know the address of the destination node and the state of the destination node, and only communicate with the message routing middleware, without interfering with each other between nodes, greatly reducing Coupling between nodes.

2. The cluster system and data communication transfer method based on message routing in the present invention can play an important role in the case of a sharp increase in the amount of messages, and unprocessed messages continue to be stored in the message queue to ensure that nodes process them sequentially in the queue message, will not overload the system and crash.

3. The cluster system and data communication transfer method based on message routing in the present invention cancel direct access between nodes, and indirect communication through message routing reduces the risk of node access.

4. The cluster system and data communication transfer method based on message routing in the present invention, the message queue reduces the coupling degree between nodes, when a node fails, the processed message is stored in the message queue, and will not be lost due to node failure lost. When the node recovers, it can continue to extract unprocessed messages from the message queue.

5. In the cluster system and data communication transfer method based on message routing of the present invention, sometimes the processing order of messages during communication between nodes is very important. First-in-first-out is the most basic form of message queues, which ensures that messages can be processed in a specific order are processed sequentially.

Description of drawings

Fig. 1 is a structural diagram of message routing in the present invention.

Fig. 2 is a diagram of the communication establishment process inside the message routing in the present invention.

Fig. 3 is a schematic diagram of communication between a sending node and a receiving node through message routing in the present invention.

Fig. 4 is a schematic diagram of communication failure between nodes in the present invention.

Fig. 5 is a flowchart of node failure judgment in the present invention.

detailed description

Embodiments of the invention are described in detail below, examples of which are illustrated in the accompanying drawings. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

The problem solved by the invention is the problem of information transmission between nodes under the cluster system. The data communication transfer method proposed by the present invention uses the message routing as the middleware for node information transfer, and the message routing as the core component for communication between all nodes and modules in the cluster system, maintains the message queue, and realizes the transfer of communication messages. Each computing node reports the message to the message router, and the message router pushes the message into the message queue so that it can be known by other nodes subscribing to the queue. Compared with the computing nodes in the traditional cluster system that directly transmit messages to each other, setting up message routing can avoid direct communication between nodes, effectively reduce the coupling between them, ensure network security, and reduce the number of accesses of computing nodes, reducing the A large amount of additional network communication burden, thereby improving the work efficiency of the entire system.

1. Message routing model

Message routing is essentially a message-oriented middleware, which is a middleware technology that provides cross-platform efficient information transmission for distributed applications. The process is connected to the middleware through the interface provided by the middleware, and the data exchange is carried out through the message transmission mechanism and the message queue service. There are two transmission modes: synchronous transmission and asynchronous transmission.

As shown in Figure 1, message routing mainly uses the following five components to achieve communication between nodes:

(1) Interface processing module

This module accepts the service request sent by the application process, analyzes the service request, distinguishes whether the request is a queue change request or a message delivery request, and calls the corresponding module according to different request types.

(2) Message queue module

The message queue module that exists on the message routing acts as a container for storing messages during message transmission, and plays an intermediate role in the transmission of messages from source to target. Each node on the message routing corresponds to an independent message queue, and the node is bound to a specific message queue, sends a message to the message routing, and then pushes it into the corresponding message queue.

(3) Queue management module

The queue management module operates on related message queues according to the queue change instructions fed back by the interface processing module, such as adding new queues or deleting existing message queues.

(4) Message channel agent module

The message channel agent module mainly completes the two tasks of message push and message extraction, and is responsible for obtaining information from the interface processing module and analyzing task categories. If it is a push operation, the queue ID is extracted from the message, and the message is pushed into the corresponding message queue according to the queue ID. If it is an extraction operation, the message is extracted from the corresponding message queue and fed back to the interface processing module.

(5) Security management module

The security management module is responsible for the security of the entire message transmission process, and encrypts and decrypts the messages.

2. The workflow of message routing

The message routing adopts the subscription/publishing mode of operation in the message delivery process. In the subscription/publishing mode, the initiating process initiates a communication request to the message routing. After the message routing accepts the request, it judges the communication mode and establishes a corresponding message agent on the message routing. The initiating process ends the communication after sending the message to the message broker, and the message broker sends the message to the process of subscribing to the message.

As shown in Figure 2, it depicts the communication establishment process inside the message routing when the message is transferred between nodes A and B. The main steps are as follows:

Step 1: Node A first initiates a connection request to the message router, and the connection producer allocates a connection for this request after receiving the request.

Step 2: Node A initiates a session request to the connection, along with transactional and confirmation mechanism parameters of the session, and establishes a corresponding session after the connection receives the request.

Step 3: Node A sends communication mode information to the session layer, and specifies the name of its corresponding message queue, and the session layer establishes a corresponding queue after receiving the request.

Step 4: Node A negotiates with the queue on its message sending mode, message nature and storage method. The message is then reported to the message router and pushed into the message queue.

Step 5: Node B initiates a connection request to the message router, and the connection producer allocates a connection for this request after receiving the request.

Step 6: Node B initiates a session request to the connection, and attaches transactional and confirmation mechanism parameters of the session at the same time, and establishes a corresponding session after the connection receives the request.

Step 7: Node B sends communication mode information to the session layer, and specifies the name of the message queue it subscribes to, and the session layer binds the corresponding queue after receiving the request.

Step 8: Node B continuously monitors the message queue. Once a message is pushed into the queue, the message is immediately extracted and obtained by Node B.

3. Application in network management

In order to illustrate the content of the present invention in more detail, the present invention is applied to network management. Simply put, network management is a series of methods and measures implemented on the network for the healthy, stable and efficient operation of the network system. Operations such as monitoring, configuring and controlling network resources can guarantee system performance and service quality. In addition, when a fault occurs on the network, it can be found in time and dealt with quickly. The following describes the specific application scenarios of the present invention in network management for the two aspects of the inter-node communication mode and the identification of failed nodes.

(1) Inter-node communication method

As shown in Figure 3, nodes A and B respectively establish communication with the message router through the above methods, and at the same time stipulate that their conversation mode is non-transactional conversation, the message confirmation mechanism is automatic confirmation (AUTO_ACKNOWLEDGE), and the message format is text message (TextMessage). After nodes A and B respectively establish a connection with the message router, the two nodes can communicate with each other. The communication process between nodes is as follows:

Step 1: When two nodes need to communicate, first determine whether there are message queues corresponding to the two nodes on the message route. If there is no corresponding message queue, the daemon process on the node initiates a request to the message routing to apply for the establishment of an independent message queue corresponding to it, and each node only subscribes to the message queue corresponding to itself.

Step 2: Node B sends a signaling message to the message router, the signaling message contains the message queue identifier, and requests the nodes subscribing to the queue to respond.

Step 3: After receiving the signaling message, the message router analyzes and extracts the queue identifier, and pushes the signaling message into the corresponding message queue according to the identifier.

Step 4: Node A subscribes to the message queue and continuously monitors the queue. As soon as a message is pushed into the queue, the message is fetched immediately. After analyzing the signaling message, node A sends a response message to the message router according to the requirements of the signaling.

Step 5: The message routing receives the message, and pushes the message into the corresponding message queue of Node B according to the queue identifier included in the message.

Step 6: Node B subscribes to its corresponding message queue, extracts messages from the message queue, and obtains a response message from node A.

(2) Fault node identification

Fault management is one of the most basic functions in network management. In the process of network management, the administrator needs to know the status of each component in real time. When a component fails, it must quickly find the fault and deal with it in time to ensure a reliable network. computer network. Using message routing as a message transfer method in network management can quickly discover failed nodes and improve the stability of the network system. As shown in Figure 4, when node A fails due to failure, the fault identification process is as follows:

Step 1: Node B sends a signaling message to the message router, and the message router forwards it to the corresponding message queue of node A.

Step 2: Since node A has failed, it no longer listens to the message queue, let alone processes signaling messages in the queue, and cannot respond as required.

Step 3: Node B continues to monitor its message queue. When there is no message in the queue for a period of time, it immediately calls the judgment function to judge that node A is invalid, and reports the failure to the management node, as shown in Figure 5.

The above embodiments are only to illustrate the technical ideas of the present invention, and can not limit the protection scope of the present invention with this. All technical ideas proposed in accordance with the present invention, any changes made on the basis of technical solutions, all fall within the protection scope of the present invention. Inside.

Claims (8)

1. A cluster system based on message routing, characterized in that it includes at least one sending node, at least one receiving node, and message routing, and the message routing includes an interface processing module, a queue management module, a message channel agent module, and a message queue module ; The interface processing module is used to accept the service request of the sending node, and analyze the service request as a queue change request or a message delivery request, and call the corresponding queue management module or message channel agent module according to different request types; The message queue module is used to save the independent message queue corresponding to each node, and the message sent by the corresponding node is stored in the message queue, and each node only subscribes to the message queue corresponding to itself; The queue management module is used to operate the related message queues in the message queue module according to the queue change request fed back by the interface processing module; The message channel proxy module is used to complete message push or message extraction to the message queue module according to the message delivery request fed back by the interface processing module. If it is a message push, it extracts the queue ID from the message, and pushes the message into The corresponding message queue; if it is a message extraction, the message is extracted from the corresponding message queue, and fed back to the interface processing module, and then sent to the receiving node by the interface processing module. 2. The cluster system based on message routing according to claim 1 , characterized in that, the message routing also includes a security management module, and the security management module is used to perform processing in the interface processing module, the message channel proxy module, and the message queue module. Encrypt and decrypt messages during message transmission. 3. A data communication transfer method based on message routing, the method utilizes message routing to realize the communication between the sending node and the receiving node, it is characterized in that, the communication steps between the sending node and the receiving node are as follows: Step 1, the sending node sends a signaling message including the message queue identifier to the message router, and requests the receiving node subscribed to the message queue to respond; Step 2: After receiving the signaling message, the message router analyzes and extracts the queue identifier, and pushes the signaling message into the corresponding message queue according to the queue identifier; Step 3: The receiving node subscribes to the message queue and continuously monitors the message queue. Once a message is pushed into the message queue, it immediately extracts the message, and sends a response message to the message router after extracting the message; Step 4: After the message routing receives the response message, it pushes the message into the message queue corresponding to the sending node according to the queue identifier contained in the message, and the sending node subscribes to its own corresponding message queue, extracts the message from the message queue, and obtains the received Node's response message. 4. The data communication transfer method based on message routing according to claim 3 , wherein, before step 1, it is judged whether there is a message queue corresponding to the sending node and the receiving node in the message queue module of message routing, if not , then the node that does not have a corresponding message queue initiates a request to the message router to apply for establishing an independent message queue corresponding to it. 5. The data communication transfer method based on message routing according to claim 4 , characterized in that, when the sending node and the receiving node respectively establish communication connections with the message routing, a session mode and a message confirmation mechanism are attached. 6. The data communication transfer method based on message routing according to claim 5 , wherein the session mode is a non-transactional session, and the message confirmation mechanism is automatic confirmation. 7. The data communication transfer method based on message routing according to claim 3 , characterized in that, after step 4, the sending node sets the time to obtain the response message, if no response message is obtained within the set time, the sending node calls Judgment function, to judge whether the receiving node is invalid, and report the failure to the management node when the receiving node is invalid. 8. The data communication transfer method based on message routing according to claim 7 , wherein the set time is 1 second.