JPH0362235A - Down monitoring processing system for composite system - Google Patents

Abstract

PURPOSE:To effectively confirm the occurrence of abnormality and also to effectively process the abnormality by providing a process part which performs the process of a subsystem of the same kind registered previously or the process related to the abnormality when the information of the abnormality is received from another cluster about a subsystem. CONSTITUTION:When a down monitoring mechanism 15A detects the abnormality of a subsystem S1 in a cluster 13A, an informing part 17 informs the down monitoring mechanism 15B of a cluster 13B of the abnormality via a proper communication means. Thus an exit schedule part 18 of the mechanism 15B retrieves a subsystem monitoring table 14B based on the name of the abnormal subsystem S1 and schedules a resources collection exit E1 of the subsystem S1 working in its own cluster. Thus it is possible to effectively recognize the occurrence of abnormality and to effectively process the abnormality.

Description

[Detailed Description of the Invention] [Summary] The present invention relates to a down monitoring processing method in a complex system in which a down monitoring mechanism is notified from a business or cluster that a certain business or cluster has fallen into an abnormal state.

In addition to integrating the down monitoring mechanism at the OS level,
The purpose is to efficiently execute UAgli when an abnormality occurs and processing for the abnormality.

Each cluster has a down monitoring mechanism, and each down monitoring mechanism has means for monitoring the occurrence of an abnormality in a subsystem within its own cluster, and means for monitoring when an abnormality occurs in a subsystem within its own cluster.

A means of notifying a down monitoring mechanism existing in other clusters that the subsystem has fallen into an abnormal state, and a means of notifying a subsystem of the same type registered in advance when an abnormality notification regarding a subsystem is received from another cluster. or means for calling a processing unit that performs processing related to the abnormality.

[Industrial application field]

The present invention relates to a down monitoring processing method in a complex system in which a down monitoring mechanism is notified that a certain business or cluster has fallen into an abnormal state from the business or cluster that has fallen into the abnormal state.

When constructing a complex system in which multiple computers are connected through communication channels, a mechanism is required that allows other normal computers to recognize an abnormality in one computer. This mechanism is called the "Down Monitoring Mechanism," and the computer that makes up the complex system is called the "Down Monitoring Mechanism."
It is called "cluster".

In addition, if subsystems that perform the same type of work are operated in multiple clusters, and an error occurs in a subsystem in one cluster, the subsystems in other clusters will recover the resources of the subsystem that has become abnormal. ,
If possible, it is necessary to take over the work.

[Conventional technology]

FIG. 8 shows an example of conventional down monitoring.

Down monitoring mechanism 15A, 1 in conventional complex system
5B is normal raster 13A, as shown in FIG.
13B mutually sends a so-called "I"mALIVE" survival notification and monitors abnormal clusters. If this survival notification does not arrive within a certain predetermined period.

When there is no response after making several survival inquiries, it is recognized that the other party is in an abnormal state.

Furthermore, even when subsystems of the same type that carry out a certain task mutually monitor each other for abnormalities, each subsystem is made to recognize the occurrence of an abnormality by sending periodic survival notifications. Ta.

[Problem to be solved by the invention]

In conventional down monitoring, survival notifications are periodically sent to each other at certain intervals to notify each other that the cluster is normal, so even if an error occurs, other clusters do not immediately notice the error. There was a problem in that the abnormality could not be recognized and detection of the abnormality was delayed by the interval of one notification.

In addition, when subsystems of the same type that perform a certain job monitor each other for abnormalities, the down monitoring mechanism can be
There is a problem in that the development burden increases because it has to be created for each M (subsystem).Furthermore, when multiple subsystems operate within one cluster.

Since a down monitoring mechanism is required for each subsystem, there is a problem in that CPU resources, memory resources, etc. are under pressure.

The present invention aims to solve the above-mentioned problems, to integrate down monitoring IQ at the OS level, and to make it possible to efficiently recognize the occurrence of an abnormality and perform processing for the abnormality.

[Means to solve the problem]

FIG. 1 shows an example of the configuration of the present invention.

In FIG. 1, 10 is a cluster-related external storage device that can be commonly accessed by each cluster; 11 is an inter-cluster communication area that stores communication data between clusters;
12 is the cluster status (in operation, stopped, down, etc.)
Represents the cluster monitoring table that manages the cluster.

13A and 13B are clusters consisting of processing units each having a CPU and memory; 14A and 14B are subsystem monitoring tables that manage the status of subsystems operating in the cluster (in operation, stopped, down, etc.); 15A , 15
B is a down monitoring mechanism that is integrated at the operating system (O3) level to achieve system consistency; 1
6 is a monitoring unit, 17 is a notification unit, 1B is an exit schedule unit, Sl-S3 is a subsystem that performs database management, TSS, and other various tasks, and El recovers resources held by the subsystem in which an abnormality has occurred. Represents a resource recovery exit that serves as a point of contact for post-failure processing.

In the present invention, for each cluster 13A, 13B.

It has down monitoring mechanisms 15A and 15B at the OS level.

Each down monitoring mechanism 15A, 15B includes a monitoring section 16, a notification section 17. The monitoring unit 16, which has each processing unit of the exit schedule unit 18, manages the subsystems 31 to 31 in its own cluster.
The 0 notification unit 17, which performs the process of monitoring the occurrence of an abnormality in S3,
When an abnormality occurs in any of the subsystems 51 to S3 within the own cluster, the fact that the subsystem has fallen into an abnormal state is notified via the intercluster communication area 11 or the like.

The exit scheduler 18, which performs a process of notifying a down monitoring mechanism existing in another cluster, refers to the subsystem monitoring table 14B etc. when receiving an abnormality notification regarding a subsystem from another cluster.

It performs a process of calling a previously registered subsystem of the same type or a processing unit that performs processing related to an abnormality, that is, a resource recovery exit El corresponding to the subsystem.

[Effect]

When a certain subsystem falls into a state where it cannot continue its operation, it notifies other clusters of the abnormality before performing subsystem down processing to recover the subsystem operating environment. The subsystem monitoring mechanism is down monitoring mechanism 15A, 15B.
As a result, it is integrated at the OS level to achieve system consistency.

Each subsystem 5l-53 sends a monitoring request in advance,
The down monitoring mechanism 15A, 15B is performed for the down monitoring mechanism 15A, 15B.

The status of the subsystem to be monitored is registered in the subsystem monitoring tables 14A and 14B.

When each subsystem 5L-53 detects an abnormality in its own subsystem, it self-reports the abnormal subsystem salt to the down monitoring mechanism 15A and the like. In addition, the down monitoring mechanism 15A and the like depend on the schedule of survival notification exits for each subsystem 31-33 by the monitoring unit 16.

Detect subsystem abnormalities.

For example, in the cluster 13A, the down monitoring mechanism 15
When A detects an abnormality in subsystem S1 1 Notification unit 1
7 notifies the down monitoring mechanism 15B of the cluster 13B of the abnormality using an appropriate communication means.The information to be notified includes the name of the cluster that has fallen into the abnormal state and the subsystem salt.

Down monitoring mechanism 1 in cluster 13B that received the notification
The exit scheduler 18 of the subsystem 5B searches the subsystem monitoring table 14B based on the subsystem salt of the subsystem Sl that has become abnormal, and schedules the resource recovery exit E1 of the subsystem Sl that is operating in its own Rask. This resource recovery exit El recovers resources such as the operating environment used by the subsystem S1 that has become abnormal in the cluster 13A, and takes over processing of transactions that were being processed as necessary.

〔Example〕

FIG. 2 is an example of a system to which the present invention is applied, FIG. 3 is an example of a control table used in an embodiment of the present invention, and FIG. 4 is a processing flow 9 at the time of initializing the down monitoring mechanism according to an embodiment of the present invention. ,
FIG. 5 shows the processing flow of the monitoring unit according to the embodiment of the present invention;
The figure is an explanatory diagram of down monitoring mechanism processing according to an embodiment of the present invention,
FIG. 7 shows an example of processing performed by the down monitoring mechanism according to the embodiment of the present invention when notifying the down state.

The present invention can be applied to a complex system as shown in FIG. 2, for example. The system storage device 21 is composed of a semiconductor storage device that has an intercluster communication function and can be shared between clusters, for example.

Note that it is also possible to replace the inter-cluster communication function in this example with a communication function such as conventional inter-processor communication, and replace the function as an external storage device that can be shared between clusters with a DASD such as a magnetic disk device. .

In the example shown in FIG. 2, the processing unit 23 constituting each cluster
has a CPU 24 and a memory 25 for local use, and accesses the system storage device 21 via a memory control unit (MCU) 22.

It is accessible.

Inter-cluster communication area 11 shown in FIG. Cluster monitoring table 1
2 is created in the system storage device 21.

The subsystem monitoring tables 14A and 14B shown in FIG. 1 are created in the memory 25 of each cluster.

The cluster monitoring table 12 is for monitoring the operational status of each cluster, and consists of a storage area for information as shown in FIG. 3(a), for example.

A cluster identifier is an identification name or identification number that uniquely determines a cluster within a complex system.

In the status display area, there is a display indicating that the cluster has started operating and has become a monitoring target, and a display indicating that the cluster is alive. The number of subsystems operating in the cluster is stored in the area for the number of operating subsystems. The resource recovery exit pointer information is the virtual space address of the resource recovery exit for recovering the resources of the cluster that has become abnormal, even if the cluster is different.

This is information to help you understand it correctly.

The subsystem monitoring table 14 is used to monitor the operating status of the subsystems for each cluster, and consists of a storage area for information as shown in FIG. 3(b), for example.

A subsystem identifier is information that uniquely identifies a subsystem. In the status display area, a display indicating that the subsystem has become a monitoring target, a display indicating whether the subsystem is alive, etc. are performed. The survival notification exit address is the address of the exit that the down monitoring mechanism calls in order to periodically issue a survival notification at certain intervals.

The resource recovery exit address is used to recover resources from subsystems of the same type running in other clusters.

This is the address of the resource recovery routine that the down monitoring mechanism calls when another cluster goes down or a subsystem in another cluster goes down.

The initialization process of the down monitoring mechanisms 15A and 15B shown in FIG. 1 is as shown in FIG. 4, for example.

■ Determine whether the system storage shared by each cluster needs to be initialized. If another cluster has already initialized, there is no need to initialize, so the first process
Skip.

■ Initialize the cluster monitoring table 12.

■ Determine whether or not the cluster monitoring table 12 indicates that the own cluster is alive. If there is an indication that the own cluster is alive, it means that an abnormality occurred in the own cluster previously and it is currently in the process of being restarted after going down. Therefore, the primary process (2) is executed. If there is no survival indication, the 9th process■
Skip.

■ For post-processing of the failure, call the resource recovery exit and return the previously used system storage resources.

■ Register information regarding the own cluster in the cluster monitoring table 12.

■ Display the existence of the own cluster in the cluster monitoring table 12.

(2) Give an opportunity to start up the monitoring unit 16, which operates periodically. Thereafter, the initialization process ends.

The activated monitoring unit 16 executes the processes ① to ② shown in FIG. 5.

■ Set the time interval for "I's ALIVE" communication to notify other clusters that the cluster is alive.

■ "l'w ALIV sent and received from other clusters"
Set the monitoring time interval for E' communication.

■ Set the monitoring time interval for scheduling the survival notification exit of subsystems within the local cluster. after that,
Start monitoring.

■ When it is time to monitor a subsystem, schedule and call the survival notification exit of the subsystem being monitored.

This survival notification is carried out at predetermined monitoring time intervals.Although not shown in the figure, this survival notification is also carried out at predetermined time intervals.The survival notification of the own cluster to other clusters and the monitoring of non-arrival of survival notifications from other clusters are also carried out at predetermined time intervals. The related processing may be similar to conventional processing.

The interface between each subsystem 31 and the like and the down monitoring mechanism 15 is as shown in FIG. 6, for example.

Each subsystem 31 etc. has the following information for the down monitoring mechanism 15:
Request subsystem monitoring using the macro interface. On the other hand, the down monitor l115 has the subsystem identifier, survival notification exit,
Register information necessary for monitoring resource recovery exits, etc. Thereafter, this subsystem will be monitored by the processing of the monitoring unit shown in FIG. 5.

The monitoring unit of the down monitoring mechanism 15 periodically schedules the survival notification exit registered in the subsystem monitoring table. This survival notification exit asserts survival using the macro interface if the own subsystem is operating normally.
The down monitoring mechanism 15 displays in the status display area of the subsystem monitoring table that the subsystem is alive. If there is no survival notification even after waiting for a certain period of time, it is assumed that an abnormality has occurred in this subsystem.

In addition, when the subsystem 31 etc. detects an abnormality by itself, the down monitoring mechanism 1 is notified by the macro interface.
5, self-report abnormalities.

It should be noted that various techniques for the subsystem to detect its own operational abnormality have been known in the past, and detailed explanations thereof will be omitted here.

When the down monitoring mechanism 15 detects that the subsystem 31 or the like has fallen into an abnormal state, it notifies the down monitoring mechanisms 15 of other clusters of the down status.

Note that depending on the type of subsystem monitoring request to the down monitoring mechanism 15, cluster suspension is scheduled after the down monitoring mechanism 15 of another cluster is notified.

When the subsystem S1 etc. terminates processing.

Using the macro interface, a request is made to the down monitoring mechanism 15 to withdraw from subsystem monitoring.

On the other hand, the down monitoring mechanism 15 deletes information from the subsystem monitoring table, and as necessary.

Delete the resource recovery routine (resource recovery exit) for the current raster from the cluster monitoring table.

When there is a cluster or subsystem down notification from another cluster, the down monitoring mechanism 15 performs the process shown in FIG. 7.

The down monitoring mechanism 15 refers to the subsystem monitoring table in the cluster that received the notification, schedules the resource recovery exit El of the down monitoring target subsystem registered in this table, and instructs the same type of subsystem to go down. We will notify you accordingly. by this.

The subsystem St can recover the resources of the abnormal subsystem and take over its operations as necessary.

In the case of the cluster stop type, inter-cluster shared resources (system!, resource'a) are collected prior to the resource collection exit schedule.

〔Effect of the invention〕

As explained above, according to the present invention, the following effects are achieved compared to the conventional monitoring system using only survival notifications.

(a) Abnormalities in other clusters can be accurately and instantaneously recognized without being affected by the system load condition, and by self-reporting from the location where the abnormality has occurred.

This is particularly useful in hot standby systems and the like because it enables faster system switching processing.

f) By integrating the down monitoring mechanisms, there is no need to monitor cluster abnormalities or certain business abnormalities with multiple down monitoring mechanisms, and it is possible to eliminate the problem of misrecognition between the down monitoring mechanisms.

[Brief explanation of drawings]

FIG. 1 shows a configuration example of the present invention. FIG. 2 shows an example of a system to which the present invention is applied. FIG. 3 is an example of a control table used in the embodiment of the present invention. FIG. 4 is a process flow at the time of initialization of a down monitoring mechanism according to an embodiment of the present invention. FIG. 5 is a processing flow of a monitoring unit according to an embodiment of the present invention. FIG. Processing explanatory diagram. FIG. 7 is an example of processing performed by the down monitoring mechanism according to the embodiment of the present invention when notifying the down state. FIG. 8 shows an example of conventional down monitoring. In the figure, 10 is an inter-cluster external storage device, 11 is an inter-cluster communication area, 12 is a cluster monitoring table, 13A, 13B are clusters, 14A, 14B are subsystem monitoring tables, 15A
, 15B is a down monitoring mechanism. 16 is a monitoring unit, 17 is a notification unit, 18 is an exit schedule unit, Sl to S3 are subsystems, and El is a resource recovery exit.

Claims (1)

[Claims] A plurality of clusters (13A, 13B) each having a computer
In a complex system consisting of, each cluster has a down monitoring mechanism (15A, 15B), and each down monitoring mechanism has a means (16) for monitoring the occurrence of an abnormality in a subsystem within its own cluster, and a means (16) for monitoring the occurrence of an abnormality in a subsystem within its own cluster. When an abnormality occurs in a subsystem, a means (1) of notifying a down monitoring mechanism in another cluster that the subsystem has fallen into an abnormal state.
7), and means (18) for calling a pre-registered subsystem of the same type or a processing unit that performs processing related to the abnormality when an abnormality notification regarding the subsystem is received from another cluster. A down monitoring processing method for a complex system.