CN103235820B - Date storage method and device in a kind of group system - Google Patents

Abstract

The invention discloses a method and device for storing data in a cluster system. The method includes: parsing the collected server operation data; Determine the corresponding initial mapping relationship table; perform the conversion step based on the data collected by the IPMI protocol to remove data heterogeneity; establish a target mapping relationship table between the values of all collected sensors and table field names; according to the target mapping A relational table, generating an SQL statement to store the collected server operation data into a database. In the present invention, the heterogeneity of data is eliminated by setting the initial mapping relationship table of different servers and establishing the server target mapping relationship table, and centrally stores all server operation data, which is convenient for unified management of data and maintenance of clusters.

Description

Data storage method and device in a cluster system

technical field

The present invention relates to the technical field of cluster communication, and more specifically relates to a data storage method and device in a cluster system.

Background technique

In order to obtain whether the server cluster is running normally in real time, it is necessary to monitor the server cluster. The monitoring of the server cluster mainly includes collecting server operating data (encapsulated data for easy transmission) according to the communication protocol, storing the collected server operating data, Whether the server running data analysis cluster is running normally. Wherein the operation data may further include status data and performance data, the status data refers to some data with fixed values, such as CPU model; the performance data may include some dynamically changing data such as CPU usage rate and network traffic usage information .

When collecting server operation data, if the server models in the cluster are different, the data collected based on the SNMP protocol has the same dimension and can be stored and managed in a unified manner. However, based on the IPMI protocol, data is mainly collected through sensors in the server, and the collected data dimensions are generally different. For example, the names of the sensors in the server are different, and the types of sensors are different. These will lead to the heterogeneity of the collected data and make it difficult to store the data. At present, Internet companies use the IPMI management system provided by the server manufacturer. If the servers forming the cluster come from multiple manufacturers, then there will be multiple sets of IPMI management systems when monitoring the server cluster, that is, different types of servers run Data is kept in different databases. When maintenance personnel want to detect the running status of the cluster through the running data of all servers, they need to retrieve and compare data between different IPMI management systems. When the cluster scale is large, the amount of stored server running data is large , making cluster maintenance more difficult and less efficient.

Contents of the invention

The embodiment of the present application provides a method and device for storing data in a cluster system to solve the problem of decentralized management and maintenance of the cluster as a result of the heterogeneity of server operating data in a cluster composed of different types of servers in the prior art. To solve the technical problems, provide a unified data storage method for heterogeneous data, store all types of server operation data in the same table in the database, improve the concentration of server operation data and the overall maintenance efficiency of the cluster.

In order to solve the above technical problems, the present invention provides a data storage method and device in a cluster system. The present invention provides the following technical solutions:

A data storage method in a cluster system, the method comprising:

Analyze the collected server operation data;

If the parsed data is based on the data collected by the intelligent platform management IPMI protocol, determine the corresponding initial mapping table of the server according to the parsed server identification ID, and the initial mapping table records at least one sensor name in the server The mapping relationship with the table field name, the same table field name is used in the initial mapping table corresponding to different servers;

Execute the data conversion step collected based on the IPMI protocol to remove data heterogeneity, the described data conversion step collected based on the IPMI protocol includes: judging whether the names of all collected sensors analyzed are recorded in the initial mapping table, If so, then set up the target mapping relationship table between the value of the collected sensor and the table field name according to the initial mapping relationship table; if not, add a new table field name in the initial mapping relationship table, and set up the new table The mapping relationship between the field name and the unrecorded collected sensor name; perform the step of establishing the target mapping relationship table between the value of the collected sensor and the table field name according to the initial mapping relationship table;

According to the target mapping relationship table, an SQL statement is generated to store the data collected based on the IPMI protocol into a database.

Preferably, the determining the initial mapping relationship table corresponding to the server according to the parsed server ID includes:

Determine the server model identifier by means of database retrieval according to the server identifier ID;

An initial mapping relationship table corresponding to the server is determined according to the server model identifier.

Preferably, the initial mapping table is established in the form of a hash index table, then determining the initial mapping table corresponding to the server according to the model of the server includes: according to the identification corresponding to the server model, searching for the Pre-stored initial mapping relationship table.

Preferably, the parsing of the collected server operation data includes:

Obtain a queue composed of readable file descriptors according to the Epoll method;

Obtain the file descriptor indicated by the head of the queue from the readable file descriptor queue;

Obtaining the original server operation data according to the file descriptor indicated by the queue head includes: reading a preset number of bytes from the Socket to determine the length of the server operation data described by the file descriptor indicated by the queue head; According to the determined data length, read the data of the length value from the file described by the file descriptor; read the original data of the server from the data of the length according to the preset data format.

Preferably, if the parsed data is based on the data collected by the Simple Network Management SNMP protocol, perform a storage step based on the data collected by the SNMP protocol;

The storage step of the data collected based on the SNMP protocol includes: establishing a mapping relationship table between the data analyzed and the corresponding parameter name of the data analyzed; according to the state data in the mapping relationship table, through the SQL statement Updating the value of the state data in the database; storing the performance data in the mapping relationship table into the database through SQL statements.

Preferably, the database is a relational database.

The present invention also provides a data storage device in a cluster system, wherein the device includes:

The data analysis module is used to analyze the collected server operation data;

The initial mapping table acquisition module is used to determine the initial mapping table corresponding to the server according to the parsed server identification ID when the parsed data is data collected based on the IPMI protocol, and the initial mapping table records the server In the mapping relationship between at least one sensor name and table field name, the same table field name is used in the initial mapping table corresponding to different servers;

The first execution module is used to perform the conversion of the data collected based on the IPMI protocol to remove the heterogeneity of the data, and the first execution module includes: a judging sub-module, used to judge the names of all collected sensors in the initial mapping Whether relational table has record; Generate submodule, when the result for described judging submodule is yes, set up target mapping relational table between the numerical value of collected sensor and table field name according to initial mapping relational table; Table field name new Adding module, when the result of the judging submodule is No, add a new table field name in the initial mapping relationship table, and establish the relationship between the new table field name and the unrecorded collected sensor name Mapping relationship; a triggering submodule, used to trigger the generating submodule.

The data storage module is configured to generate an SQL statement to store the data collected based on the IPMI protocol into a database according to the target mapping relationship table.

Preferably, the initial mapping table acquisition module includes:

The server model identification determination submodule is used to determine the server model identification in a database retrieval manner according to the parsed server identification ID;

The initial mapping relationship table determination submodule is configured to determine the initial mapping relationship table corresponding to the server according to the server model identifier.

Preferably, the initial mapping relationship table is established in the form of a hash index table, and the initial mapping relationship table determination submodule is specifically used to search for the pre-stored initial mapping relationship through hashing according to the identification corresponding to the server model surface.

Preferably, the data analysis module includes:

The descriptor queue acquisition submodule is used to acquire a queue composed of readable file descriptors according to the Epoll method;

The descriptor reading submodule is used to obtain the file descriptor indicated by the head of the queue from the readable file descriptor queue;

The raw data reading sub-module is used to obtain the server running raw data according to the file descriptor indicated by the queue head, specifically including: reading a preset number of bytes from the Socket to determine the file description indicated by the queue head The length of the server running data described by the descriptor; according to the determined data length, read the data of the length value from the file described by the file descriptor; read the original data of the server from the data of the length according to the preset data format data.

Preferably, the device also includes:

The second execution module is used to store the data collected based on the SNMP protocol when the parsed data is based on the data collected by the SNMP protocol;

The second execution module includes: a mapping relationship table establishment submodule, which is used to establish a mapping relationship table between the analyzed data and the parameter names corresponding to the data when the parsed data is data collected based on the SNMP protocol; The state data storage submodule is used to update the value of the state data in the database through SQL statements according to the state data in the mapping relationship table; the performance data storage submodule is used to store the performance data in the mapping relationship table The data is stored in the database through SQL statements.

In the technical solution of the present invention, each server in the cluster has an initial mapping relationship table between corresponding sensor names and table field names, and then establishes a target mapping relationship table according to the initial relationship mapping table and the collected sensor values. What is recorded in the target mapping relationship table is the relationship between the table field name and the value of the collected sensor. By comparing the initial mapping relationship table with the target mapping relationship table, it can be seen that only the sensor There is no sensor name for the mapping relationship between the corresponding value and the field name of the table. At the same time, the table field names used in the initial mapping table corresponding to each server in the present invention are the same, so far, the data recorded in the target mapping relationship table has no heterogeneity. If the table field name is regarded as the parameter name, the corresponding sensor value is the parameter value, which is a simple one-to-one correspondence. When storing, just create a table with the above table field name, and then the corresponding The collection value can be stored under the corresponding table field name. It's just that for different servers, the physical meaning of the parameter value may be different. The target mapping relationship table has removed the data heterogeneity caused by the different sensor names and numbers. At this time, the collected server operation data can be stored in the database by using a unified SQL statement.

Through the technical solution of the present invention, the data of different types of servers based on the IPMI protocol can be uniformly stored in one table. For maintenance personnel, when it is necessary to monitor the overall operating status of the cluster, they only need to read and compare the data in the same table. Even if the scale of the cluster is large, the speed of searching and comparing data in the same table is relatively fast, which reduces the difficulty of cluster maintenance and improves the maintenance efficiency of the cluster.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments described in this application. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is the flowchart of embodiment 1 of data storage method in a kind of cluster system of the present invention;

Fig. 2 is a flow chart of Embodiment 2 of a data storage method in a cluster system of the present invention

Fig. 3 is a flow chart of Embodiment 3 of a data storage method in a cluster system of the present invention;

4 is a schematic structural diagram of Embodiment 1 of a data storage device in a cluster system of the present invention;

FIG. 5 is a schematic structural diagram of the first execution module 403 in Embodiment 1 of the device of the present invention;

FIG. 6 is a schematic structural diagram of an initial mapping table acquisition module 402 in Embodiment 1 of the device of the present invention;

7 is a schematic structural diagram of Embodiment 2 of a data storage device in a cluster system of the present invention;

FIG. 8 is a schematic structural diagram of the data parsing module 401 in Embodiment 2 of the device of the present invention;

FIG. 9 is a structural diagram of an embodiment of the present invention in an actual application scenario.

detailed description

In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

All embodiments of the present invention are described on the premise that the cluster system includes servers of different models. Referring to FIG. 1, it is a flow chart of Embodiment 1 of a data storage method in a cluster system provided by the present invention. This embodiment may specifically include:

Step 101: Parse the collected server operation data.

In actual application scenarios, the server operation data is collected according to a certain collection protocol. After the server operation data is collected, the collected data needs to be encapsulated, and the encapsulated data can be transmitted in the channel. Therefore, after receiving the server operation data, the data analysis must be performed first to obtain the original server operation data collected. Regarding data parsing, reference may be made to an implementation manner of data parsing in the prior art, which will not be repeated here.

Step 102: If the parsed data is data collected based on the IPMI protocol, determine an initial mapping table corresponding to the server according to the parsed server ID.

If the parsed data is collected based on the IPMI protocol (Intelligent Platform Management Interface, Intelligent Platform Management Interface), then the data collected based on the IPMI protocol for different types of servers is heterogeneous. In order to remove the data heterogeneity, The first step is to obtain the initial mapping relationship table corresponding to the server. Specifically, according to the parsed data collected based on the IPMI protocol, the ID (identification) of the server can be obtained through data parsing, and then the server model (or model ID) corresponding to the server ID can be obtained through database retrieval. Finally, According to the determined server model, the initial mapping relationship table corresponding to the model server is searched by means of hash.

Preferably, the initial mapping relationship table is established in the form of a hash index table, and an initial mapping relationship table can be pre-stored for each server. In actual operation, a corresponding initial mapping relationship table can be pre-stored for each type of server , and the same table field names are used in the initial mapping tables corresponding to different models of servers. Alternatively, after parsing the data each time, a mapping relationship table between the parsed sensor names and table field names is established as an initial mapping relationship table. Take the pre-stored initial mapping relationship table corresponding to each type of server, and two different types of servers included in the cluster as an example. For the convenience of description, the models are respectively recorded as the first model and the second model. Refer to Table 1 shows the initial mapping relationship table A1 corresponding to the first model server established by the hash index table method; Table 2 is the initial mapping relationship table A2 corresponding to the second model server established by the hash index table method.

Table 1 Initial mapping relationship table A1

table field name sensor name Col1 Name1 example Col2 Name2 example

Table 2 Initial mapping relationship table A2

table field name sensor name Col1 Name3 example Col2 Name4 example

The names of the sensors in Table 1 and Table 2 are identified by Name1, Name2, Name3 and Name4. The sensor names in Table 1 and Table 2 are not necessarily the names of all the sensors included in the server of the corresponding model, but may be the names of several sensors among all the sensors in the server of the specific model. The factory time of each server is different. For example, for the server of the first model, it is assumed that the number of sensors it includes is 3, specifically a temperature sensor, a voltage sensor and a current sensor. However, there may be situations such as server upgrades in actual use. For example, if a new sensor is added or a certain sensor is deleted on the original basis, the names of the remaining original sensors remain unchanged, and the number of sensors included in servers of the same model with different factory dates may be different. A server of a model corresponds to an initial mapping table, so the names of the sensors in the initial mapping table may not be exactly the same as the names of the actual sensors collected by each server of the model.

Referring to Table 1 and Table 2, it can be seen that for the servers of the first model and the second model, the table field names used in the corresponding initial mapping tables are the same, but the sensor names corresponding to the same table field names are different.

Step 103: Perform a data conversion step collected based on the IPMI protocol to remove data heterogeneity and generate a target mapping relationship table.

Because the data collected based on the IPMI protocol is heterogeneous, in order to facilitate the unified storage of data, before storing the data, the data structure must be transformed first. The specific data structure conversion process is as follows:

After parsing out the original running data of the server, a mapping relationship table (given in the form of a data hash table in this embodiment) between the actual collected sensor names and the corresponding collected values in the server is established according to the parsed data. For ease of understanding, this step is illustrated by taking Table 3 as an example corresponding to the data hash table B1 of a certain collected server of the first model, and Table 4 as an example of the data hash table B2 corresponding to a certain collected server of the second model sexual description.

Table 3 Data hash table B1

Sensor name collection value Name1 10 example Name2 20 example

Table 4 Data hash table B2

Sensor name collection value Name3 30 example Name4 40 example Name5 50

After obtaining the data hash table corresponding to the server, judge whether the set of sensor names included in Table 1 includes the set of sensor names included in Table 3, or whether all sensor names included in Table 3 exist in Table 1, if yes , it is said that Table 3 is completely covered by Table 1. If table 3 is completely covered by table 1, directly perform the merging of table 1 and table 3, and establish the target mapping table between the table field name and the collection value. For the server of the first model, as shown in table 5, It is the target mapping relationship table obtained by merging Table 1 and Table 3, that is, the target mapping relationship table of a certain server whose first model is collected.

Table 5 Target mapping relationship table of the first model server

table field name collection value Col1 10 example Col2 20 example

For the collected server of the first model, its initial mapping table and data hash table are completely covered, and the merging is relatively simple. It can be seen from Table 2 and Table 4 that, for a server of the second type collected, its corresponding initial mapping table and data hash table are not completely covered. In this case, it is necessary to establish a mapping relationship between sensor names not included in the initial mapping relationship table and the field names of the newly added table. In actual operation, because of the unpredictability of server upgrades, a set of table field names can be established. When encountering sensor names that are not included in the initial mapping table, unused tables can be obtained from the set of table field names. Field names to map to sensor names that are not included. For example, if Col3 is used to correspond to the collection value of Name5, then refer to Table 6 for the obtained target mapping relationship table corresponding to a server of the second model that is collected.

Table 6 Target mapping relationship table of the second model

table field name collection value Col1 30 example Col2 40 example Col3 50 example

The step 103 is described by taking one collected server as an example, and the principle is the same when processing other server data.

Step 104: Generate an SQL statement according to the target mapping relationship table to store the data collected based on the IPMI protocol into a database.

After the target mapping table is generated, the data collected based on the IPMI protocol is stored in the database by generating SQL statements. Wherein, the SQL statement corresponding to the data in the target mapping relationship table includes an insert statement and an update update statement. Specifically, the state data in the target mapping relationship table is saved to the database through the update statement, and the performance data is saved to the database through the insert statement.

Insert statement format:

insert into table_name(x1,x2,x3,x4,...)values(y1,y2,y3,y4,...)

Among them, xn and yn respectively correspond to the table field names and collection value pairs in the target mapping relationship table.

update statement format:

update table_name set x1=y1,x2=y2,x3=y3,...where...

Among them, xn and yn respectively correspond to the table field names and collection value pairs in the target mapping relationship table. When data is stored in the database, it is only necessary to traverse the target mapping relationship table, and the time complexity is O(N).

In the technical solution of this embodiment, each server in the cluster has an initial mapping relationship table between the corresponding sensor name and the table field name, and then establishes a target mapping relationship table according to the initial relationship mapping table and the collected sensor values . What is recorded in the target mapping relationship table is the relationship between the table field name and the value of the collected sensor. By comparing the initial mapping relationship table with the target mapping relationship table, it can be seen that only the sensor There is no sensor name for the mapping relationship between the corresponding value and the field name of the table. At the same time, the table field names used in the initial mapping table corresponding to each server in the present invention are the same, so far, the data recorded in the target mapping relationship table has no heterogeneity. If the table field name is regarded as the parameter name, the corresponding sensor value is the parameter value, which is a simple one-to-one correspondence. When storing, just create a table with the above table field name, and then the corresponding The collection value can be stored under the corresponding table field name. It's just that for different servers, the physical meaning of the parameter value may be different. The target mapping relationship table has removed the data heterogeneity caused by the different sensor names and numbers. At this time, the collected server operation data can be stored in the database by using a unified SQL statement.

Through the technical solution of the present invention, the data collected by different types of servers based on the IPMI protocol can be uniformly stored in one table. For maintenance personnel, when it is necessary to monitor the overall operating status of the cluster, they only need to read and compare data in the same table. , even if the scale of the cluster is large, the speed of looking up data in the same table for comparison is relatively fast, which reduces the difficulty of maintaining the cluster and improves the maintenance efficiency of the cluster.

In the described embodiment 1, the data storage based on the IPMI protocol collection is described, and the existing data analysis technology is adopted when analyzing the data, but when the existing data is analyzed, the operation data received in real time is directly placed in the queue for buffering , when the data needs to be parsed, read the data directly from the data queue. Because the data is directly cached in the queue, it will occupy more operating system resources, and the queue capacity is limited, so it is not suitable for the storage of highly concurrent data. In order to solve the storage of high-concurrency data, the present invention provides a data storage method embodiment 2 in a cluster system, which adopts a new data analysis method to provide a storage method for high-concurrency data. At the same time, it provides an efficient storage method for the data collected based on the IPMI protocol and the operation data collected based on the SNMP protocol (Simple Network Management Protocol, Simple Network Management Protocol).

Referring to Fig. 2, it is a flowchart of a data storage method embodiment 2 in a cluster system provided by the present invention. On the basis of embodiment 1, in this embodiment, the storage of data collected by the SNMP protocol is taken as an example to carry out detailed Note that although the new data analysis form is introduced in the context of data collected by the SNMP protocol, it is only for the purpose of explaining the analysis method, not to say that this analysis method is only applicable to the data collected by the SNMP protocol. For the implementation manner of the same steps as those in the foregoing embodiments, refer to the steps in the foregoing embodiments, and details will not be repeated in this embodiment. This embodiment can specifically include:

Step 201: Pre-store an initial mapping relationship table corresponding to each type of server in the database.

The initial mapping relationship table described here is mainly set for the data collected based on the IPMI protocol.

Step 202: Analyzing the collected server operation data.

In this embodiment, the Epoll interface is used to receive data, and the Epoll interface is an enhanced version of the multiplexing IO interface select/poll under Linux, which can significantly improve the system CPU utilization rate of the program in the case of only a small amount of activity in a large number of concurrent connections. Because it will reuse the file descriptor set to deliver the result without forcing the developer to re-prepare the file descriptor set to be listened to each time before waiting for the event; another reason is that when getting the event, it does not need to traverse the entire To listen to the descriptor set, just traverse the descriptor sets that are asynchronously awakened by the kernel IO event and added to the queue. The queue is a "first in first out" (FIFO—first in first out) linear table.

Based on the way Epoll receives data, the specific process of data parsing is as follows: first, obtain the readable file descriptor from the system at the current moment through Epoll, and put the obtained readable file descriptor into the readable file descriptor In the queue, read the readable file descriptor at the head of the queue. And according to the readable file descriptor in the queue head, the original running data of the server (the running data of the server before encapsulation) is read from the corresponding file. When obtaining the original operating data of the server according to the file descriptor, first read the preset bytes from the Socket, and the preset bytes are stored in the file represented by the readable file descriptor in the queue head The length of the encapsulated data; according to the length value, read the data of this length from the corresponding file in the file descriptor, and then interpret the original operating data of the server from the data of the length according to the format agreed by the communication parties. Every time data analysis is performed, the readable file descriptor corresponding to the head of the queue is read for data analysis. If the parsed data is based on the IPMI protocol, store it with reference to the data storage method described in Embodiment 1, and if the parsed data is based on the SNMP protocol, go to step 203.

Step 203: Determine the mapping relation table between the data collected based on the SNMP protocol and the parameter names corresponding to the data.

If the parsed data is data collected based on the SNMP protocol, it is first necessary to determine a mapping relationship table between the data collected based on the SNMP protocol and the parameter names corresponding to the data. After the mapping relationship table is determined, the data storage in step 204 can be entered.

Step 204: Store state data in a relational database.

For the collected state data, the value collected multiple times may be the same, so there is no need to store the state data collected each time separately, as long as the state data in the database is updated each time. In actual operation, the status data collected based on the SNMP protocol can be used to update the values of these status data in the database through SQL statements.

Step 205: Store performance data in the relational database.

Preferably, the status data can be filtered from the mapping relationship table, and the remaining performance data can be stored in the database through SQL statements, that is, the storage of collected data based on the SNMP protocol is completed. The SQL statement format used when the data collected based on the IPMI protocol is stored in the database is suitable for the storage of the data collected by the SNMP protocol. In actual operation, the status data and performance data can be stored separately in different tables, and the data collected based on the IPMI protocol and the data collected based on the SNMP protocol are stored in different tables separately.

It can be seen from the above that the data storage based on the SNMP protocol is much simpler than the data storage based on the IPMI protocol, because the data dimensions collected based on the SNMP protocol are the same for different types of servers, that is, the number of collected parameters and parameters The names are the same. At this time, as long as there is a mapping relationship between the parameter name and the table field name, the hash table merge method can be used to obtain the target hash table between the collection value and the table field name, and for different models of servers The mapping relationship between corresponding parameter names and table field names is the same.

In the technical solution of this embodiment, a data parsing method different from the prior art is used, and the original running data of the server is read through the file descriptor, because the running file descriptor does not occupy the resources of the operating system, but the application itself. Reduced impact on the operating system. In addition, file descriptors are placed in the queue. If there is high concurrency data, you only need to cache the descriptor corresponding to the data in the queue at this time, which will not occupy too many resources, and effectively solve the problem of high concurrency data. storage problem.

In the prior art, when the cluster is monitored, the collected data is generally stored using RRD (RoundRobin Database, ring database). The so-called RRD is like a ring marked with points around - these points are the positions stored according to time . Store data on a ring with no start and end. After a long time, the new data in the RRD will overwrite the original data, that is, the durability of the data is poor. Correspondingly, in this embodiment, a relational database is used to store data, which can maintain data persistence.

Although the data storage based on the IPMI protocol collection and the SNMP protocol collection is described separately through different embodiments, in practical applications, they exist at the same time, because for a server, at some point, it must be based on the SNMP protocol To collect data, it is also necessary to collect data based on the IPMI protocol. Now integrate the processes of Embodiment 1 and Embodiment 2, as shown in Figure 3, which is a complete implementation flow chart of data storage in the cluster, and the implementation of related steps can be referred to Implementation of relevant steps in Embodiment 1 and Embodiment 2.

Corresponding to the method embodiment, correspondingly, the present invention also provides a data storage device in a cluster system. Referring to FIG. 4 , it is a schematic structural diagram of Embodiment 1 of a data storage device in a cluster system provided by the present invention. Devices include:

A data parsing module 401, configured to parse the collected server operation data;

The initial mapping table acquisition module 402 is used to determine the initial mapping table corresponding to the server according to the parsed server identification ID when the parsed data is data collected based on the IPMI protocol, and the initial mapping table records the The mapping relationship between at least one sensor name and table field name in the server, the same table field name is used in the initial mapping table corresponding to different servers;

The first execution module 403 is configured to perform conversion of data collected based on the IPMI protocol, so as to generate a target mapping relationship table.

The data storage module 404 is configured to generate an SQL statement to store the data collected based on the IPMI protocol into a database according to the target mapping relationship table.

Wherein, preferably, as shown in FIG. 5 , which is a schematic structural diagram of the first execution module 403, the first execution module 403 includes:

Judging sub-module 501, used to judge whether the names of all collected sensors are recorded in the initial mapping relation table;

Generate sub-module 502, when the result of the judgment sub-module is yes, set up the target mapping relationship table between the value of the collected sensor and the table field name according to the initial mapping relationship table;

The table field name adding submodule 503 is used to add a new table field name in the initial mapping relationship table, and establish a mapping relationship between the new table field name and the unrecorded collected sensor name;

The triggering submodule is used to trigger the generating submodule.

Preferably, as shown in FIG. 6 , which is a schematic structural diagram of the initial mapping table acquisition module 402, the initial mapping table acquisition module 402 includes:

The server model identifier determination submodule 601 is used to determine the server model identifier in a database retrieval manner according to the parsed server identifier;

The initial mapping relationship table determination submodule 602 is configured to determine the initial mapping relationship table corresponding to the server according to the server model identifier.

The initial mapping relationship table is established in the form of a hash index table, and the initial mapping relationship table determination sub-module 603 is specifically configured to search the pre-stored initial mapping relationship table through a hash method according to the identification corresponding to the server model.

The functions realized by each module in the device correspond to the operation steps of the method in the method embodiment 1, and will not be repeated here.

A schematic structural diagram of Embodiment 2 of a data storage device in a cluster system provided by the present invention, as shown in FIG. In addition to the modules, the device may also include:

The second execution module 701 is used to store data collected based on the SNMP protocol when the parsed data is collected based on the SNMP protocol;

The second execution module 701 includes: a mapping relationship table establishment submodule 7011, which is used to establish a mapping relationship between the parsed data and the parameter name corresponding to the data when the parsed data is data collected based on the SNMP protocol surface;

The status data storage sub-module 7012 is configured to update the value of the status data in the database through an SQL statement according to the status data in the mapping relationship table;

The performance data storage sub-module 7013 is configured to store the performance data in the mapping relationship table into the database through SQL statements.

Preferably, referring to FIG. 8 , which is a schematic structural diagram of the data parsing module 401 in this embodiment, the data parsing module 401 includes:

The descriptor queue acquisition submodule 801 is used to acquire a queue composed of readable file descriptors according to the Epoll mode;

The descriptor reading submodule 802 is used to obtain the file descriptor indicated by the head of the queue from the readable file descriptor queue;

The raw data reading sub-module 803 is configured to obtain the server running raw data according to the file descriptor indicated by the queue head, specifically including: reading a preset number of bytes from the Socket to determine the file indicated by the queue head The length of the server running data described by the descriptor; according to the determined data length, read the data of the length value from the file described by the file descriptor; read the server’s data from the data of the length according to the preset data format Raw data.

The functions realized by each module in the device correspond to the operation steps of the method in the method embodiment 2, and will not be repeated here.

Referring to FIG. 9 , which is an implementation of the embodiment of the present invention in a practical application scenario, the main thread puts available file descriptors into the readable file descriptor queue (shown in A1 ) through Epoll. The worker thread pool contains a certain number of threads, which are responsible for taking out the file descriptor in the head of the queue from the readable file descriptor queue (shown in A2), reading data from it, parsing the data and storing the data (shown in A3), and completing Unified storage of IPMI data. In the design of the database, it is stored separately according to the timeliness of the collected data. At the same time, a table is designed for each table to store the latest value of each field, and the latest value of each data can be quickly obtained when accessing the database.

It should be noted that the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, but also includes none other elements specifically listed, or also include elements inherent in such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

As for the system embodiment, since it basically corresponds to the method embodiment, for related parts, please refer to the part description of the method embodiment. The system embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without creative effort.

The foregoing is only a specific embodiment of the present invention. It should be pointed out that for those of ordinary skill in the art, some improvements and modifications can also be made without departing from the principle of the present invention. It should be regarded as the protection scope of the present invention.

Claims (11)

1. A data storage method in a cluster system, characterized in that the method comprises: Analyze the collected server operation data; If the parsed data is based on the data collected by the intelligent platform management IPMI protocol, determine the corresponding initial mapping table of the server according to the parsed server identification ID, and the initial mapping table records at least one sensor name in the server The mapping relationship with the table field name, the same table field name is used in the initial mapping table corresponding to different servers; Execute the data conversion step collected based on the IPMI protocol to remove data heterogeneity, the described data conversion step collected based on the IPMI protocol includes: judging whether the names of all collected sensors analyzed are recorded in the initial mapping table, If so, then set up the target mapping relationship table between the value of the collected sensor and the table field name according to the initial mapping relationship table; if not, add a new table field name in the initial mapping relationship table, and set up the new The mapping relationship between the table field name and the unrecorded collected sensor name; perform the step of establishing the target mapping relationship table between the value of the collected sensor and the table field name according to the initial mapping relationship table; According to the target mapping relationship table, the data collected based on the IPMI protocol is stored in a database by using a unified SQL statement. 2. The method according to claim 1, wherein said determining the initial mapping table corresponding to the server according to the parsed server identification ID comprises: Determine the server model identifier by means of database retrieval according to the server identifier ID; An initial mapping relationship table corresponding to the server is determined according to the server model identifier. 3. The method according to claim 2, wherein the initial mapping table is established in the form of a hash index table, and determining the initial mapping table corresponding to the server according to the model of the server comprises: according to the server The identifier corresponding to the model is searched for the pre-stored initial mapping relationship table through a hash method. 4. The method according to claim 1, wherein said analyzing the collected server operation data comprises: Obtain a queue composed of readable file descriptors according to the Epoll method; Obtain the file descriptor indicated by the head of the queue from the readable file descriptor queue; Obtaining the original server operation data according to the file descriptor indicated by the queue head includes: reading a preset number of bytes from the Socket to determine the length of the server operation data described by the file descriptor indicated by the queue head; According to the determined data length, read the data of the length value from the file described by the file descriptor; read the original data of the server from the data of the length according to the preset data format. 5. method according to claim 1, is characterized in that, if the data that parses out is the data collected based on the Simple Network Management SNMP protocol, execute the storage step of the data collected based on the SNMP protocol; The storage step of the data collected based on the SNMP protocol includes: establishing a mapping relationship table between the data analyzed and the corresponding parameter name of the data analyzed; according to the state data in the mapping relationship table, through the SQL statement Updating the value of the state data in the database; storing the performance data in the mapping relationship table into the database through SQL statements. 6. The method according to any one of claims 1-5, wherein the database is a relational database. 7. A data storage device in a cluster system, characterized in that the device comprises: The data analysis module is used to analyze the collected server operation data; The initial mapping table acquisition module is used to determine the initial mapping table corresponding to the server according to the parsed server identification ID when the parsed data is data collected based on the IPMI protocol, and the initial mapping table records the server In the mapping relationship between at least one sensor name and table field name, the same table field name is used in the initial mapping table corresponding to different servers; The first execution module is used to perform the conversion of the data collected based on the IPMI protocol to remove the heterogeneity of the data, and the first execution module includes: a judging sub-module, used to judge the names of all collected sensors in the initial mapping Whether relational table has record; Generate submodule, when the result for described judging submodule is yes, set up target mapping relational table between the numerical value of collected sensor and table field name according to initial mapping relational table; Table field name new Adding module, when the result of the judging submodule is No, add a new table field name in the initial mapping relationship table, and establish a relationship between the new table field name and the unrecorded collected sensor name The mapping relationship; the triggering submodule is used to trigger the generating submodule; The data storage module is used to store the data collected based on the IPMI protocol into a database by using a unified SQL statement according to the target mapping relationship table. 8. The device according to claim 7, wherein the initial mapping table acquisition module comprises: The server model identification determination submodule is used to determine the server model identification in a database retrieval manner according to the parsed server identification ID; The initial mapping relationship table determination submodule is configured to determine the initial mapping relationship table corresponding to the server according to the server model identifier. 9. The device according to claim 8, wherein the initial mapping relationship table is established in the form of a hash index table, and then the initial mapping relationship table determines the submodule, which is specifically used to pass the corresponding identification according to the server model Search the pre-stored initial mapping relationship table in a hash manner. 10. The device according to claim 7, wherein the data parsing module comprises: The descriptor queue acquisition submodule is used to acquire a queue composed of readable file descriptors according to the Epoll method; The descriptor reading submodule is used to obtain the file descriptor indicated by the head of the queue from the readable file descriptor queue; The raw data reading sub-module is used to obtain the server running raw data according to the file descriptor indicated by the queue head, specifically including: reading a preset number of bytes from the Socket to determine the file description indicated by the queue head The length of the server running data described by the descriptor; according to the determined data length, read the data of the length value from the file described by the file descriptor; read the original data of the server from the data of the length according to the preset data format data. 11. The device according to claim 7, further comprising: The second execution module is used to perform storage of data collected based on the SNMP protocol when the parsed data is data collected based on the SNMP protocol; The second execution module includes: a mapping relationship table establishment submodule, which is used to establish a mapping relationship table between the analyzed data and the parameter name corresponding to the data when the data analyzed is based on the data collected by the SNMP protocol; The state data storage submodule is used to update the value of the state data in the database through SQL statements according to the state data in the mapping relationship table; the performance data storage submodule is used to store the performance data in the mapping relationship table The data is stored in the database through SQL statements.