CN118474030A

CN118474030A - Flow control method, device, equipment and medium of unified storage system

Info

Publication number: CN118474030A
Application number: CN202410910956.9A
Authority: CN
Inventors: 周玉兰; 刘粉粉
Original assignee: Suzhou Metabrain Intelligent Technology Co Ltd
Current assignee: Suzhou Metabrain Intelligent Technology Co Ltd
Priority date: 2024-07-09
Filing date: 2024-07-09
Publication date: 2024-08-09
Anticipated expiration: 2044-07-09
Also published as: CN118474030B

Abstract

The invention relates to the technical field of storage, and particularly discloses a flow control method, a device, equipment and a medium of a unified storage system, which are realized through a multi-controller of the unified storage system, wherein the controller determines a burst flow queue mechanism trigger threshold, a maximum burst flow threshold and burst flow use time of a monitored object so as to execute flow monitoring of the monitored object, so that flow resources are accumulated when the data input and output flow of the monitored object does not exceed the burst flow queue mechanism trigger threshold, the accumulated flow resources are utilized when the data input and output flow of the monitored object exceeds the burst flow queue mechanism trigger threshold, the monitored object can realize burst flow, the waiting time of a user in burst flow is reduced, meanwhile, the multi-controller is utilized to realize flow accumulation for the monitored object, high-reliability control of the flow of the whole system level is realized, and the processing capacity of the unified storage system on burst network flow is improved.

Description

Flow control method, device, equipment and medium of unified storage system

Technical Field

The present invention relates to the field of storage technologies, and in particular, to a method, an apparatus, a device, and a medium for controlling a flow of a unified storage system.

Background

The unified storage system (Unified Storage System) is a system that integrates different types of storage technologies, allowing users to access and manage all types of storage resources through a single interface.

Flow control is one of the optimization problems of storage systems, and is no exception to unified storage systems. How to control the network flow of the unified storage system brings better experience to the user, and especially controls the network flow aiming at burst is a technical problem which needs to be solved by the person skilled in the art.

Disclosure of Invention

The invention aims to provide a flow control method, a flow control device, flow control equipment and flow control media for a unified storage system, which are used for improving the processing capacity of the unified storage system on burst network flow.

In order to solve the above technical problems, the present invention provides a flow control method of a unified storage system, which is applied to a controller in a unified storage system of multiple controllers, and includes:

Determining a burst flow queue mechanism trigger threshold, a maximum burst flow threshold and burst flow use time of a monitoring object;

Performing flow monitoring on the monitoring object;

if the data input/output flow of the monitoring object does not exceed the corresponding burst flow queue mechanism triggering threshold, taking the corresponding maximum burst flow threshold as an upper limit to accumulate flow resources of the monitoring object;

If the data input/output flow of the monitoring object exceeds the corresponding burst flow queue mechanism triggering threshold, controlling the monitoring object to enter a burst flow processing flow so as to allow the monitoring object to use accumulated flow resources in the corresponding burst flow using time;

Wherein the monitoring object is a volume or a volume group comprising a plurality of volumes;

when the flow resources are accumulated for the monitoring object, each controller of the unified storage system accumulates the flow resources for the monitoring object according to a preset proportion; when the monitoring object enters the burst flow processing flow, the controller of the monitoring object determines the actual accumulated flow resources of the monitoring object according to the preset proportion and the total value of the accumulated flow resources of the monitoring object, and increases the data input and output flow resources for the monitoring object according to the actual accumulated flow resources of the monitoring object.

In one aspect, each controller accumulates flow resources for the monitored object according to a preset ratio, including:

Each controller accumulates flow resources for the monitoring object according to the maximum burst flow threshold of the monitoring object as the upper limit;

the controller of the monitoring object determines an actual accumulated flow resource of the monitoring object according to the preset proportion and the total value of the accumulated flow resources of the monitoring object, and the method comprises the following steps:

And dividing the total value of the accumulated flow resources of the monitoring object by the number of the controllers by the controller of the monitoring object to obtain the actual accumulated flow resources of the monitoring object.

On the other hand, each controller accumulates flow resources for the monitoring object according to a preset proportion, including:

Each controller accumulates flow resources for the monitoring object according to the upper limit of the value of the maximum burst flow threshold value divided by the number of the controllers of the monitoring object;

And the controller of the monitoring object is used for taking the total value of the accumulated flow resources of the monitoring object as the actual accumulated flow resources of the monitoring object.

On the other hand, if the monitoring object is the volume group, determining the trigger threshold, the maximum burst flow threshold and the burst flow usage time of the burst flow queue mechanism of the monitoring object includes:

Determining a burst flow queue mechanism trigger threshold, a maximum burst flow threshold and burst flow using time of the volume group, and equally dividing the burst flow queue mechanism trigger threshold and the maximum burst flow threshold of the volume group to volumes in the volume group respectively.

Determining the burst flow using time of the volume group, determining the burst flow queue mechanism triggering threshold and the maximum burst flow threshold of the volume group according to the number of volumes with data input and output in the volume group, and equally dividing the burst flow queue mechanism triggering threshold and the maximum burst flow threshold of the volume group to the volumes with data input and output respectively.

On the other hand, the accumulating the traffic resource with the corresponding maximum burst traffic threshold as an upper limit for the monitored object includes:

And determining the rate of accumulating the flow resources for the monitoring object according to the data input/output flow of the monitoring object, and accumulating the flow for the monitoring object by taking the corresponding maximum burst flow threshold as the upper limit and according to the rate of accumulating the flow resources for the monitoring object.

On the other hand, determining the rate of accumulating the flow resource for the monitoring object according to the data input/output flow of the monitoring object comprises the following steps:

Calculating the difference value between the trigger threshold value of the burst flow queue mechanism of the monitoring object and the data input/output flow of the monitoring object;

The greater the difference, the greater the rate at which flow resources are accumulated for the monitored object.

adding tokens for the token bucket of the monitoring object by taking the corresponding maximum burst flow threshold as the maximum token number of the token bucket corresponding to the monitoring object;

Controlling the monitoring object to enter a burst traffic processing flow so as to allow the monitoring object to use accumulated traffic resources in corresponding burst traffic use time, including:

and when the monitoring object is controlled to enter the burst flow processing flow, allowing the monitoring object to use data input/output flow resources corresponding to all tokens in the token bucket.

On the other hand, when the flow resource is accumulated for the monitoring object, each controller of the unified storage system accumulates the flow resource for the monitoring object according to a preset proportion, including:

when flow resources are accumulated for the monitoring object, each controller of the unified storage system puts tokens into the token bucket corresponding to the monitoring object according to a first preset token putting rate until each controller puts the number of tokens corresponding to the maximum burst flow threshold of the monitoring object;

the first preset token throwing rate is determined according to the token throwing rate corresponding to the data input and output rate of the monitoring object, and the first preset token throwing rate is larger than the token throwing rate corresponding to the data input and output rate of the monitoring object.

When the flow resources are accumulated for the monitoring object, each controller of the unified storage system puts tokens into the token bucket corresponding to the monitoring object according to a second preset token putting rate until the number of tokens in the token bucket of the monitoring object reaches the number of tokens corresponding to the maximum burst flow threshold of the monitoring object;

The second preset token release rate is determined according to the token release rate corresponding to the data input/output rate of the monitoring object divided by the number of the controllers, and is greater than the token release rate corresponding to the data input/output rate of the monitoring object divided by the number of the controllers.

On the other hand, retrieving a token from the token bucket of the monitoring object to allocate a corresponding data input/output traffic resource for the monitoring object, including:

dividing the number of tokens in the token bucket by the number of controllers to obtain the available token number of the monitoring object, and taking out the tokens from the available token number to provide the required data traffic resources for the monitoring object.

Taking the number of tokens in the token bucket as the available number of tokens of the monitoring object, and taking out the tokens from the available number of tokens to provide the required data flow resources for the monitoring object.

On the other hand, performing flow monitoring of the monitoring object includes:

performing flow monitoring on the monitored object based on a preset anti-shake range;

The preset anti-shake range includes at least one of counting nothing in preset time after the data input/output flow is started initially, taking average value in unit time as statistic value for the data input/output flow, marking as effective triggering when the data input/output flow triggering threshold reaches preset times, and marking as effective triggering the error range of the data input/output flow triggering threshold.

If the effective time condition of the monitoring object is met, executing flow monitoring of the monitoring object;

Wherein the validation time condition includes at least one of being within a single validation time and within an arrival cycle period.

In another aspect, the method further comprises:

When the controller for monitoring the monitoring object fails, adopting a standby controller corresponding to the failure controller in the mirror image pair divided by the unified storage system to take over the data flow and monitoring work of the failure controller, and recombining the mirror image pairs so that the number of the mirror image pairs is the same as that of the controllers which normally run;

And when the standby controller takes over the monitoring work of the fault controller, emptying the part corresponding to the fault controller in the accumulated flow resources of the monitored object monitored by the fault controller, and adjusting the preset proportion to be the proportion corresponding to the number of the controllers which normally run.

In order to solve the technical problem, the invention also provides a unified storage system which comprises a plurality of controllers, wherein the controllers are used for determining a burst flow queue mechanism trigger threshold value, a maximum burst flow threshold value and burst flow using time of a monitoring object; performing flow monitoring on the monitoring object; if the data input/output flow of the monitoring object does not exceed the corresponding burst flow queue mechanism threshold, taking the corresponding maximum burst flow threshold as an upper limit to accumulate flow resources of the monitoring object; if the data input/output flow of the monitoring object exceeds the corresponding burst flow queue mechanism triggering threshold, controlling the monitoring object to enter a burst flow processing flow so as to allow the monitoring object to use accumulated flow resources in the corresponding burst flow using time;

When the flow resources are accumulated for the monitoring object, each controller accumulates the flow resources for the monitoring object according to a preset proportion; when the monitoring object enters the burst flow processing flow, the controller of the monitoring object determines the actual accumulated flow resources of the monitoring object according to the preset proportion and the total value of the accumulated flow resources of the monitoring object, and increases the data input and output flow resources for the monitoring object according to the actual accumulated flow resources of the monitoring object.

In order to solve the technical problem, the present invention further provides a flow control device of a unified storage system, which is applied to a controller in a unified storage system of multiple controllers, and includes:

The analysis management unit is used for determining a burst flow queue mechanism trigger threshold value, a maximum burst flow threshold value and burst flow using time of the monitoring object; if the received performance statistics unit reports that the data input and output flow of the monitored object does not exceed the monitoring result of the trigger threshold of the corresponding burst flow queue mechanism, notifying a current limiting control unit to accumulate flow resources for the monitored object by taking the corresponding maximum burst flow threshold as an upper limit; if the performance statistics unit receives a monitoring result that the data input/output flow of the monitoring object exceeds the trigger threshold of the corresponding burst flow queue mechanism, notifying the current limiting control unit to control the monitoring object to enter a burst flow processing flow so as to allow the monitoring object to use accumulated flow resources in the corresponding burst flow using time;

The performance statistics unit is used for executing flow monitoring on the monitoring object and reporting a monitoring result to the analysis management unit;

the current limiting control unit is used for executing the control task notified by the analysis management unit;

In order to solve the above technical problem, the present invention further provides a flow control device of a unified storage system, including:

a memory for storing a computer program;

A processor for executing the computer program, which when executed by the processor, implements the steps of the flow control method of the unified storage system as described in any one of the above.

In order to solve the above technical problem, the present invention further provides a non-volatile storage medium, on which a computer program is stored, the computer program implementing the steps of the flow control method of the unified storage system according to any one of the above when being executed by a processor.

To solve the above technical problem, the present invention further provides a computer program product, which includes a computer program/instruction, where the computer program/instruction implements the steps of the flow control method of the unified storage system according to any one of the above when executed by a processor.

The flow control method of the unified storage system has the advantages that the flow control method is realized through the multiple controllers of the unified storage system, by introducing the burst flow queue mechanism, when a volume or a volume group is taken as a monitoring object, the controllers determine the burst flow queue mechanism triggering threshold value, the maximum burst flow threshold value and the burst flow using time of the monitoring object so as to execute flow monitoring of the monitoring object, so that flow resources are accumulated when the data input and output flow of the monitoring object does not exceed the burst flow queue mechanism triggering threshold value, the accumulated flow resources are utilized when the data input and output flow of the monitoring object exceeds the burst flow queue mechanism triggering threshold value, the monitoring object can realize burst flow, the waiting time when a user flows burst is reduced, meanwhile, the multiple controllers are utilized to realize flow accumulation for the monitoring object, so that the flow control can be seamlessly switched to other controllers when a certain controller fails, the high-reliability burst flow control of the whole system is realized, and the processing capacity of the unified storage system on the burst network flow is improved.

The invention also provides a flow control system, a device, equipment, a nonvolatile storage medium and a computer program product of the unified storage system, which have the beneficial effects and are not repeated here.

Drawings

For a clearer description of embodiments of the invention or of the prior art, the drawings that are used in the description of the embodiments or of the prior art will be briefly described, it being apparent that the drawings in the description below are only some embodiments of the invention, and that other drawings can be obtained from them without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a flow control method of a unified storage system according to an embodiment of the present invention;

FIG. 2 is a flow chart of flow control using a token bucket according to an embodiment of the present invention;

FIG. 3 is a flowchart of another flow control method of a unified storage system according to an embodiment of the present invention;

FIG. 4 is a flowchart of a specific implementation of S102 according to an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a flow control device of a unified storage system according to an embodiment of the present invention;

Fig. 6 is a schematic structural diagram of a flow control device of a unified storage system according to an embodiment of the present invention.

Detailed Description

The core of the invention is to provide a flow control method, a device, equipment and a medium of a unified storage system, which are used for improving the processing capacity of the unified storage system on burst network flow.

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

In order to facilitate understanding of the technical solution provided by the embodiments of the present invention, some key terms used in the embodiments of the present invention are explained first.

A unified storage system (Multi-Controller Storage or Multi-Cluster Service, MCS system for short) with multiple controllers can provide high availability, scalability, and fault tolerance. In MCS systems, each controller may be considered a node that works together to provide data services and manage storage resources. These nodes may be responsible for handling tasks such as data Input/Output (I/O) requests, data caching, and communication with other nodes.

Burst queuing (Burst) is a network traffic management technique used to control and optimize packet transmissions in a network. The bursty traffic queues may prevent server overload, and when bursty traffic arrives, the server may crash because too many requests cannot be handled without reasonable mechanism restrictions. The burst flow queue can improve user experience, and for a user, if a request is immediately refused, the request can be possibly discontented, and at the moment, the request can be temporarily stored through the burst flow queue, and the request can be processed one by one when the server has the capability of processing, so that the waiting time perceived by the user is reduced.

The bursty flow queuing mechanism is currently applied in various fields, such as a switch, a server and storage, is more applied in the switch industry, is mainly used for avoiding congestion, and needs to utilize congestion management and flow management when a network intermittently experiences congestion and a delay sensitive service requires higher quality service than a non-delay sensitive service, wherein the typical function of the flow management is to limit the flow emergency of a certain connection entering a certain network. When the message meets a certain condition, if the message flow of a certain connection is too large, the flow management can take different processing actions on the message, such as discarding the message or resetting the priority of the message.

Storage systems can be divided into distributed storage systems and unified storage systems. The embodiment of the invention is mainly aimed at realizing a burst flow queue mechanism in a unified storage system. In the prior art, when a burst traffic queue mechanism is applied to a storage system, control of burst traffic is mainly realized by a single controller, and after the controller fails, the burst traffic of the whole cluster layer can not be ensured to be still controlled.

In order to solve the problem, the embodiment of the invention provides a flow control scheme of a unified storage system, which optimizes the problem that the burst flow control of the current storage system only carries out volume limitation on a certain node, realizes the burst flow control on the whole cluster, and can coordinate the flow of other controllers after a certain controller fails so as to ensure the flow tuning of the whole system.

For ease of understanding, a system architecture to which the present invention is applicable will first be described. The embodiment of the invention provides a unified storage system, which comprises a plurality of controllers, wherein the controllers are used for determining a burst flow queue mechanism trigger threshold value, a maximum burst flow threshold value and burst flow using time of a monitored object; performing flow monitoring on the monitored object; if the data input/output flow of the monitoring object does not exceed the corresponding burst flow queue mechanism threshold, taking the corresponding maximum burst flow threshold as an upper limit to accumulate flow resources of the monitoring object; if the data input/output flow of the monitoring object exceeds the corresponding burst flow queue mechanism triggering threshold, the monitoring object is controlled to enter a burst flow processing flow so as to allow the monitoring object to use the accumulated flow resources in the corresponding burst flow using time.

Wherein the monitoring object is a volume or a volume group including a plurality of volumes.

When accumulating flow resources for the monitoring object, each controller accumulates the flow resources for the monitoring object according to a preset proportion; when the monitoring object enters the burst flow processing flow, the controller of the monitoring object determines the actual accumulated flow resources of the monitoring object according to the preset proportion and the total value of the accumulated flow resources of the monitoring object, and increases the data input and output flow resources for the monitoring object according to the actual accumulated flow resources of the monitoring object.

It should be noted that volumes and volume groups in embodiments of the present invention are concepts at the storage system level. Where a Volume (Volume) represents a logical partition on a storage device, it may be a hard disk drive, a Solid State Disk (SSD), an optical disk, or other type of storage medium. A volume may contain one or more file systems for organizing and storing data. In operating systems, volumes are typically assigned a drive number or mount point through which a user can access data on the volume. Volumes may be basic, dynamic (in Windows) or physical, logical (in Unix-like systems).

Volume Group (Volume Group) is typically associated with Logical Volume Management (LVM). In a system that supports LVM, such as Linux, a volume group is a component in logical volume management. A volume group is a collection of Physical Volumes (PVs) that may be part of a hard disk or the entire hard disk. Physical volumes are added to the volume group, which can then be used to create Logical Volumes (LVs). Logical volumes are logical partitions in a volume group, can be dynamically resized, and can span multiple physical volumes. The logical volume is transparent to the user and appears as a single storage space. In brief overview, a volume is a logical partition that stores data, and a volume group is a collection that contains multiple physical volumes for creating and managing logical volumes. This design provides greater flexibility, allowing for dynamic adjustment of the size of storage space and management of complex storage requirements.

In the embodiment of the invention, a plurality of volumes can be divided into one volume group for management according to the service correlation degree.

In the unified storage system, according to the situation of attributing to the controller, the attribution situation of the volume can be divided into the following types: a non-home volume, i.e. a volume is assigned to all controllers of the unified storage system, and there is a possibility of data input/output traffic to the volume on each controller; there is a home volume, i.e., the volume is attributed to a controller, only the data input and output traffic to the volume exists on that controller.

The invention provides a flow control scheme of a unified storage system based on a unified storage architecture of multiple controllers, which aims to ensure that volumes belonging to a certain controller or volumes not belonging to the same have the capacity of processing burst flow, and the volumes are combined into a volume group, so that the volume group also has the capacity of processing burst flow. And when the controller fails, another controller can take over the flow, so as to realize the burst flow control function.

The flow control scheme of the unified storage system is realized through the multi-controller of the unified storage system, when a volume or a volume group is taken as a monitoring object, the controller determines the burst flow queue mechanism triggering threshold value, the maximum burst flow threshold value and the burst flow using time of the monitoring object so as to execute flow monitoring of the monitoring object, so that flow resources are accumulated when the data input and output flow of the monitoring object does not exceed the burst flow queue mechanism triggering threshold value, the accumulated flow resources are utilized when the data input and output flow of the monitoring object exceeds the burst flow queue mechanism triggering threshold value, the monitoring object can realize burst flow, the waiting time of a user in burst flow is reduced, meanwhile, the multi-controller is utilized to realize flow accumulation for the monitoring object, so that the flow control can be seamlessly switched to other controllers when a certain controller fails, the high-reliability burst flow control of the whole system is realized, and the processing capacity of the unified storage system on burst network flow is improved.

Based on the above architecture, the flow control method of the unified storage system provided by the embodiment of the invention is described below with reference to the accompanying drawings.

Fig. 1 is a flowchart of a flow control method of a unified storage system according to an embodiment of the present invention.

As shown in fig. 1, a controller applied to a unified storage system of multiple controllers, a flow control method of a unified storage system provided by an embodiment of the present invention includes: s101: and determining a burst traffic queue mechanism trigger threshold, a maximum burst traffic threshold and burst traffic use time of the monitored object.

S102: and performing flow monitoring on the monitored object.

S103: if the data input/output flow of the monitoring object does not exceed the corresponding burst flow queue mechanism trigger threshold, taking the corresponding maximum burst flow threshold as an upper limit to accumulate flow resources of the monitoring object.

S104: if the data input/output flow of the monitoring object exceeds the corresponding burst flow queue mechanism triggering threshold, the monitoring object is controlled to enter a burst flow processing flow so as to allow the monitoring object to use the accumulated flow resources in the corresponding burst flow using time.

When accumulating flow resources for the monitored object, each controller of the unified storage system accumulates the flow resources for the monitored object according to a preset proportion; when the monitoring object enters the burst flow processing flow, the controller of the monitoring object determines the actual accumulated flow resources of the monitoring object according to the preset proportion and the total value of the accumulated flow resources of the monitoring object, and increases the data input and output flow resources for the monitoring object according to the actual accumulated flow resources of the monitoring object.

In the embodiment of the present invention, for S101, the unit of the burst traffic queue trigger threshold and the maximum burst traffic threshold may be represented by the number of read/write operations per second (Input/Output Operations Per Second, IOPS), or may be represented by the occupied bandwidth. The maximum burst flow threshold is the maximum flow resource which the monitoring object is allowed to use after entering the burst flow processing flow. The burst traffic queue trigger threshold is a threshold for triggering a control monitoring object to enter the burst traffic queue, and the threshold needs to be smaller than the maximum burst traffic threshold.

Since the burst traffic of the volume often appears temporarily, and in order to avoid excessive usage of traffic resources of other volumes, it is also necessary to set the burst traffic usage time for the monitoring object, that is, how long the monitoring object is allowed to occupy the traffic resources up to the maximum burst traffic threshold.

In the embodiment of the invention, the volumes may be divided according to user settings or service dependencies, for example, the volumes divided into the same volume group may be determined by the user, or the volumes of the same service may be divided into the same volume group. Even if the volume is a normal volume, a self-refined volume, a compressed volume and a repeated-deleted volume, the volume can be set according to the user requirement or the service requirement, and the volume still can keep the characteristic of compressed repeated-deleted.

In addition, because volumes in the unified storage system of multiple controllers have different attribution conditions, volumes belonging to a certain controller exist, and whether the volumes belong to the volumes (namely, all the controllers have data input and output to the volumes), volumes with different attribution conditions are further classified into different volume groups according to the attribution conditions of the volumes. I.e., the volumes in a volume group must all be non-home volumes or the volumes in a volume group must all be home volumes.

When the volume is the monitoring object, the threshold value related to the burst flow is set in volume units.

When the volume group is taken as the monitoring object, the set threshold related to the burst flow may be a threshold corresponding to the volume group, and further determining the threshold related to the burst flow of each volume in the volume group is required.

In some alternative implementations of embodiments of the invention, the volume threshold may be determined by dividing the volume group threshold evenly into volumes. If the monitored object is a volume group, determining the trigger threshold, the maximum burst flow threshold and the burst flow usage time of the burst flow queue mechanism of the monitored object may include: determining a burst flow queue mechanism trigger threshold, a maximum burst flow threshold and burst flow using time of the volume group, and equally dividing the burst flow queue mechanism trigger threshold and the maximum burst flow threshold of the volume group to volumes in the volume group respectively.

In practical applications, there may be a case where data input/output flows of each volume in the volume group are unbalanced, and even there may be a case where there is no data input/output operation on the volume. In other optional implementations of the embodiments of the present invention, if the monitoring object is a volume group, determining the burst traffic queue mechanism trigger threshold, the maximum burst traffic threshold, and the burst traffic usage time of the monitoring object may include: determining the burst flow using time of the volume group, determining the burst flow queue mechanism triggering threshold and the maximum burst flow threshold of the volume group according to the number of volumes with data input and output in the volume group, and equally dividing the burst flow queue mechanism triggering threshold and the maximum burst flow threshold of the volume group to the volumes with data input and output respectively.

That is, if a volume group has 5 volumes in total, but only 3 volumes have data input/output traffic, and the other 2 volumes do not exist, the volume group can be regarded as a volume group having 3 volumes to set a relevant threshold value and burst traffic control can be performed in such a manner that the volumes are equally divided or regarded as a whole according to the demand. Based on this, the relevant threshold value of the volume group can be dynamically adjusted according to the running condition of the volumes in the volume group.

For S102, monitoring of the monitoring object is performed according to the burst queue mechanism-related threshold value determined above.

For S103, when the data input/output traffic of the monitored object does not exceed the trigger threshold of the corresponding burst traffic queue mechanism, traffic resources are accumulated for the monitored object, so that more traffic resources can be obtained when the monitored object has burst traffic compared with the traffic resources in normal operation.

In some optional implementations of the embodiments of the present invention, accumulating traffic resources with a corresponding maximum burst traffic threshold as an upper limit for a monitored object may include: and determining the rate of accumulating flow resources for the monitored object according to the data input/output flow of the monitored object, and accumulating the flow for the monitored object by taking the corresponding maximum burst flow threshold as an upper limit and according to the rate of accumulating the flow resources for the monitored object.

That is, a value slightly larger than the data input output flow rate of the monitoring object may be adopted as the rate of accumulating the flow resources for the monitoring object, so that the accumulated flow resources of the monitoring object exceed the current data input output flow rate of the monitoring object.

Further, determining the rate of accumulating the flow resource for the monitored object according to the data input/output flow of the monitored object may include: calculating the difference value between the trigger threshold value of the burst flow queue mechanism of the monitoring object and the data input/output flow of the monitoring object; the larger the difference, the greater the rate at which traffic resources are accumulated for the monitored object. Even if it is said that, in the case where the burst traffic queue mechanism trigger threshold of the monitoring object is larger than the difference between the data input output traffic of the monitoring object, it may be set that the rate at which the monitoring object accumulates traffic resources exceeds the difference between the data input output traffic of the monitoring object, so that the traffic resources are accumulated for the monitoring object quickly at an early stage.

And S104, when the data input/output flow of the monitoring object exceeds the trigger threshold of the corresponding burst flow queue mechanism, controlling the monitoring object to enter a burst flow processing flow, and allowing the monitoring object to use the accumulated flow resources in the burst flow using time. Even if the data input/output flow of the monitoring object exceeds the trigger threshold of the corresponding burst flow queue mechanism, the accumulated flow resources do not reach the maximum burst flow threshold, and at this time, the flow resources which can be used by the burst flow of the monitoring object cannot reach the expectations.

In the embodiment of the invention, for the unified storage system of multiple controllers, in order to ensure high-reliability control of a burst queue mechanism of the whole system layer, each controller accumulates flow resources for each monitoring object together, and after the monitoring object enters the burst queue mechanism, the actual accumulated flow resources of the monitoring object are determined according to the flow resources accumulated by each controller for the monitoring object, so that after the controller currently executing the monitoring task of the monitoring object fails, the standby controller of the controller can seamlessly take over the monitoring task of the monitoring object.

In some optional implementations of the embodiments of the present invention, each controller accumulates flow resources for a monitored object according to a preset ratio, and may include: and each controller accumulates flow resources for the monitored object according to the maximum burst flow threshold of the monitored object as an upper limit. The controller of the monitoring object determining the actual accumulated flow resources of the monitoring object according to the preset proportion and the total value of the accumulated flow resources of the monitoring object may include: and the controller of the monitoring object divides the total value of the accumulated flow resources of the monitoring object by the number of the controllers to obtain the actual accumulated flow resources of the monitoring object.

Assuming that the unified storage system is a four-controller system, each controller accumulates flow resources for the monitored object, the total value of the accumulated flow resources is four times of the actual accumulated flow resources of the monitored object, and the total value of the accumulated flow resources needs to be divided by four for processing after the monitored object enters the burst queue mechanism.

In other optional implementations of the embodiments of the present invention, each controller accumulates flow resources for the monitored object according to a preset ratio, and may further include: and each controller accumulates flow resources for the monitored object according to the upper limit of the maximum burst flow threshold value divided by the number of the controllers of the monitored object. The controller of the monitoring object determining the actual accumulated flow resources of the monitoring object according to the preset proportion and the total value of the accumulated flow resources of the monitoring object may include: the controller of the monitoring object is used for taking the total value of the accumulated flow resources of the monitoring object as the actual accumulated flow resources of the monitoring object.

Even if the controllers accumulate the flow resources for the monitoring object in proportion, the total value of the accumulated flow resources is the actual accumulated flow resources of the monitoring object.

Based on the above embodiments, the embodiments of the present invention further describe various application scenarios when a volume group is used as a monitoring object.

In practical application, the following control policies can be classified according to the attribution situation and the operation situation of the volume:

If the volume is a non-home volume, that is, the volume belongs to all controllers in the unified storage system of the multiple controllers, each controller can have data input/output flow to the volume, when the flow on each controller exceeds the trigger threshold of the burst flow queue mechanism allocated by the controller and the accumulated flow resources are enough, the burst flow queue mechanism can be triggered, and meanwhile, the burst flow queue mechanism of the volume group can also be triggered, that is, each member of the volume group can reach the maximum burst flow threshold in a short time.

If all volumes in the volume group have data input and output traffic and the traffic of each volume is balanced, setting a volume group service quality-Burst traffic threshold (QoS-Burst), when the total traffic of the volume group bursts, allowing the total traffic of the volume group to exceed the maximum Burst traffic threshold within a certain time, but not exceeding the maximum Burst traffic threshold, and recovering to the Burst traffic queue mechanism triggering threshold or below after the maximum Burst traffic threshold is reached.

If all volumes in the volume group have data input and output flow and the flow of each volume is unbalanced, setting a volume group service quality-burst flow threshold, when the total flow of the volume group is burst, allowing the total flow of the volume group to exceed a maximum burst flow threshold within a certain time, but not exceeding the maximum burst flow threshold, and recovering to the burst flow queue mechanism triggering threshold or below until the set burst flow using time is reached.

If data input and output flow exists in part of the volumes in the volume group and the flow of each volume is balanced, a volume group service quality-burst flow threshold is set for all volumes or the data input and output flow exists according to a selected strategy, when the total flow of the volume group is burst, the total flow of the volume group is allowed to exceed the maximum burst flow threshold within a certain time, but not exceed the maximum burst flow threshold, the maximum burst flow use time is reached, and then the volume group service quality-burst flow threshold is restored to be equal to or lower than the burst flow queue mechanism trigger threshold.

If data input and output flow exists in part of the volumes in the volume group and the flow of each volume is unbalanced, setting a volume group service quality-burst flow threshold value for all volumes or the data input and output flow exists according to a selected strategy, allowing the total volume flow of the volume group to exceed a maximum burst flow threshold value within a certain time but not exceed the maximum burst flow threshold value, and recovering to the burst flow queue mechanism triggering threshold value or below until the set burst flow using time is reached.

It can be seen that there may be traffic imbalances from volume to volume in a volume group or even situations where some volumes do not have data input/output traffic. As described in the above embodiments of the present invention, the volume group correlation threshold may be determined based on the number of volumes actually running in the volume group.

In the above embodiments of the present invention, the volume group related threshold may be averaged over the volumes in the volume group. For example, the burst traffic queue mechanism trigger threshold set for a volume group is 10 ten thousand IOPS, the maximum burst traffic threshold is 11 ten thousand IOPS, the burst traffic usage time is 2 minutes, and assuming that there are five volumes in the volume group, the burst traffic queue mechanism trigger threshold of each volume is set to 2 ten thousand IOPS, and the maximum burst traffic threshold of each volume is set to 2.2 ten thousand IOPS.

And for the case of unbalanced flow between volumes in the volume group, one of the following three setting methods can be adopted according to the requirements.

In the first case, after determining the burst flow queue mechanism trigger threshold, the maximum burst flow threshold and the burst flow usage time of the volume group, if there is a first target volume in the volume group, where the data input/output flow is greater than other volumes, the average threshold may be set for other volumes according to the manner of equally dividing the relevant thresholds of the volume group, and a threshold greater than the average threshold may be set for the first target volume.

For example, a burst flow queue mechanism trigger threshold set for a volume group is 10 ten thousand IOPS, a maximum burst flow threshold is 11 ten thousand IOPS, a burst flow use time is 2 minutes, assuming that five volumes exist in the volume group, and data input/output flow of volume a is greater than that of other volumes, volume a is taken as a first target volume, the burst flow queue mechanism trigger threshold can be set for volume a to be 2.5 ten thousand IOPS, and the maximum burst flow threshold can be set for volume a to be 2.8 ten thousand IOPS; the thresholds of the other four volumes are all obtained by adopting equal division calculation, namely, the trigger threshold of the burst flow queue mechanism is 10/5=2ten thousand IOPS, and the maximum burst flow threshold is 11/5=2.2ten thousand IOPS.

Even though, in practical applications, the sum of the burst traffic related thresholds for each volume in the volume group is allowed to be unequal to the burst traffic related threshold of the volume group. And in particular, may be determined according to user settings or service requirements.

In the second case, after determining the burst flow queue mechanism trigger threshold, the maximum burst flow threshold and the burst flow usage time of the volume group, if there is a second target volume in the volume group, where the data input/output flow is smaller than that of other volumes, the average threshold may be set for the other volumes according to the manner of equally dividing the relevant thresholds of the volume group, and a threshold smaller than the average threshold may be set for the second target volume.

For example, the trigger threshold of the burst flow queue mechanism set for the volume group is 10 ten thousand IOPS, the maximum burst flow threshold is 11 ten thousand IOPS, the burst flow use time is 2 minutes, if five volumes exist in the volume group, the data input/output flow of the volume B is greater than that of other volumes, the volume B is taken as a second target volume, the trigger threshold of the burst flow queue mechanism can be set to 1.5 ten thousand IOPS for the volume B, and the maximum burst flow threshold can be set to 1.8 ten thousand IOPS for the volume B; the thresholds of the other four volumes are all obtained by adopting equal division calculation, namely, the trigger threshold of the burst flow queue mechanism is 10/5=2ten thousand IOPS, and the maximum burst flow threshold is 11/5=2.2ten thousand IOPS.

In the third case, after determining the burst flow queue mechanism trigger threshold, the maximum burst flow threshold and the burst flow usage time of the volume group, the data input/output flow ratio of each volume may be determined according to the historical data input/output flow distribution of each volume in the volume group, and the burst flow related threshold of the volume group may be divided into volumes according to the data input/output flow ratio.

For example, the burst flow queue mechanism trigger threshold set for the volume group is 10 ten thousand IOPS, the maximum burst flow threshold is 11 ten thousand IOPS, the burst flow use time is 2 minutes, if five volumes are in the volume group, the burst flow queue mechanism trigger threshold of each volume is set to 2 ten thousand IOPS when the volumes are divided according to the dividing thought, the maximum burst flow threshold of each volume is set to 2.2 ten thousand IOPS, but there may be a data input/output flow of volume a greater than other volumes, and a data input/output flow of volume B less than other volumes, the burst flow related threshold of volume a can be properly adjusted, the burst flow related threshold of volume B is reduced, and the sum of the burst flow related thresholds of the final volumes is equal to the burst flow related threshold of the volume group.

In each of the above examples, the burst traffic usage time of the volume in the volume group is equal to the burst traffic usage time of the volume group. In practical applications, there may be a scenario where the burst traffic related threshold of a volume is reached but not of a volume group, or a scenario where the burst traffic related threshold of a volume group is reached but not of a volume, since the data input output operation is actually performed on a volume, the flow control for each volume is thresholded according to the smaller of itself and the burst traffic related threshold of the volume into which it is divided, i.e. the smaller of the burst traffic related thresholds is used as an upper constraint. If the sum of the burst traffic related thresholds of the volumes in the volume group is greater than the burst traffic related threshold of the volume group, there may be a case where the maximum burst traffic threshold of the volume group is reached but the maximum burst traffic threshold of the volume cannot be reached. If the sum of the burst traffic related thresholds of the volumes in the volume group is smaller than the burst traffic related threshold of the volume group, the maximum burst traffic threshold of a part of the volumes may be reached but the maximum burst traffic threshold of the volume group may not be reached.

On the basis of the foregoing embodiment, the flow control method of a unified storage system provided by the embodiment of the present invention may further include: when the controller for monitoring the corresponding monitoring object fails, adopting a standby controller corresponding to the failure controller in the mirror image pair divided by the unified storage system to take over the data flow and monitoring work of the failure controller, and recombining the mirror image pairs so that the number of the mirror image pairs is the same as that of the controllers which normally run; when the standby controller takes over the monitoring work of the fault controller, the part corresponding to the fault controller in the accumulated flow resources of the monitored object monitored by the fault controller is emptied, and the preset proportion is adjusted to be the proportion corresponding to the number of the controllers in normal operation.

In implementations, each controller in a multi-controller system forms a pair of mirrors, and typically several controllers build several pairs of mirrors, each controller acting as a master controller in its corresponding one of the mirror pairs and as a spare controller in its corresponding other mirror pair. For example, if there are four controllers 1,2, 3,4 in the multi-controller system, four mirror pairs 14, 13, 23, 24 can be formed, corresponding to four domains (domains). When each controller works normally, each mirror pair works normally. If the controller 1 fails, the two mirror pairs 23 and 24 still work normally, and at this time, since there are three controllers 2,3 and 4 in the system, the domain management module of the unified storage system reorganizes the affected domain members and triggers the remapping of the service module on the relevant domain, so as to ensure that each domain is still two controllers, for example, can reorganize into 23, 24 and 34. And after the service module finishes the remapping, the service module indicates that the current domain reorganization is finished. And if a fault controller appears again, domain reorganization is performed again, and the remapping of the service module is started again until a controller remains finally. Only one controller remains for each domain within the system.

By applying the flow control method of the unified storage system provided by the embodiment of the invention, the burst flow control of all volumes can be executed as long as one controller which normally operates exists in the unified storage system.

Fig. 2 is a flow chart of flow control using a token bucket according to an embodiment of the present invention.

Based on the above embodiments, the embodiments of the present invention further describe a manner of performing burst flow control by using a token bucket algorithm.

In the embodiment of the invention, the data input and output and the bandwidth are dynamically regulated based on the result of continuous performance monitoring of the unified storage system of the multiple controllers, and a token bucket mechanism can be adopted to realize flow control. The accumulating the traffic resources with the corresponding maximum burst traffic threshold as the upper limit in S103 may include: and adding tokens for the token bucket of the monitored object by taking the corresponding maximum burst flow threshold as the maximum token number of the token bucket corresponding to the monitored object. In S104, controlling the monitoring object to enter the bursty traffic processing flow to allow the monitoring object to use the accumulated traffic resources in the corresponding bursty traffic usage time may include: when the control monitoring object enters the burst flow processing flow, the control monitoring object is allowed to use data input and output flow resources corresponding to all tokens in the token bucket.

Token flow control is a network flow management technique that uses "tokens" to control the rate of transmission of data packets in a network to avoid network congestion and to ensure quality of service (QoS). Token Bucket (Token Bucket) is a common flow control mechanism that allows a network device to add tokens to a Bucket at a rate. When a packet arrives, it needs to take a token from the token bucket to be transmitted. If there are no tokens in the bucket, the packet will be delayed or discarded. In a memory system, a token is typically used to represent a data input and output operation. In the embodiment of the invention, a mode that each data input/output operation of the volume needs to take a corresponding token can be adopted.

In the embodiment of the invention, as shown in fig. 2, when a data input/output request of a volume obtains a token, submitting the token to a lower layer data input/output stack, otherwise, enqueuing and waiting until the token is available; when the burst queue mechanism is implemented, when the token usage is smaller than the put amount, token accumulation (preparation for burst traffic) is performed, otherwise, when the token is accumulated to a certain amount (frequent triggering is avoided, the meaning of burst processing is lost), the accumulated token is used to allow the upper limit value to be exceeded in a short time.

In an embodiment of the present invention, the command line mkthrottle command (which is used to use to create and associate a new throttle object with an object (e.g., volume)) may be used to set the burst traffic queue mechanism trigger threshold, the maximum burst traffic threshold, and the burst traffic usage time. Wherein the maximum burst traffic threshold corresponds to a bucket capacity of the token bucket. And a switching zone bit can be additionally arranged for being determined by the current available flow resources, switching to a burst flow state when the accumulated flow resources are full and the consumption is larger than the speed of putting in, and switching to a burst flow queue mechanism triggering threshold state when the accumulated flow resources are consumed or the burst flow use time is reached.

In the embodiment of the invention, the total required token number of the monitored object in one burst flow is calculated according to the burst flow related threshold value set by the command line. In some optional implementations of the embodiments of the present invention, when accumulating the traffic resources for the monitored object, each controller of the unified storage system accumulates the traffic resources for the monitored object according to a preset proportion, may include: when accumulating flow resources for a monitored object, each controller of the unified storage system puts tokens into a token bucket corresponding to the monitored object according to a first preset token putting rate until each controller puts the number of tokens corresponding to the maximum burst flow threshold of the monitored object; the first preset token throwing rate is determined according to the token throwing rate corresponding to the data input and output rate of the monitored object, and the first preset token throwing rate is larger than the token throwing rate corresponding to the data input and output rate of the monitored object.

That is, when the traffic resources are accumulated for the monitored object, if the data input/output traffic of the monitored object does not exceed the burst traffic queue mechanism triggering threshold, the corresponding token number is generated by using the data input/output traffic of the monitored object, such as the IPOS, and put into the token bucket of the monitored object, for example, 1000 tokens are put in a second until the token number reaches the bucket capacity of the token bucket. If there are four controllers in the system, each controller needs to deliver the number up to the bucket capacity.

In practical application, if the number of tokens in the token bucket is larger than the bucket capacity, redundant tokens are discarded according to a funnel algorithm, if the token request rate of the monitored object is larger than the token put rate at the moment, accumulation is continued until the requirement is met, and if the token request rate of the monitored object is smaller than the token put rate, whether the token bucket is full is judged. If the IOPS or bandwidth of the monitoring object of one controller stops at the moment, the accumulated tokens are temporarily stored, and when the IOPS or bandwidth of the monitoring object of the controller is restarted, the tokens are accumulated, but when the related threshold value of the burst flow is changed, the originally accumulated tokens are all discarded.

In the embodiment of the present invention, the method for extracting the token from the token bucket of the monitored object to allocate the corresponding data input/output traffic resource for the monitored object may include: dividing the number of tokens in the token bucket by the number of controllers to obtain the number of available tokens of the monitoring object, and taking out the tokens from the number of available tokens to provide the required data flow resources for the monitoring object.

That is, when the monitoring object has burst traffic, tokens are acquired from the token bucket first, and at this time, the corresponding token number of each controller is rapidly reduced until 0. At this time, if the data input/output flow of the monitored object is smaller than the trigger threshold of the burst flow queue mechanism again, the token can be accumulated again, and the token is consumed again when the burst flow exists.

In the presence of bandwidth monitoring, the number of tokens required (current data input output traffic divided by 4K) is calculated for the data input output traffic of the monitored object.

In the unified storage system of multiple controllers, if only part of controllers have accumulated tokens to reach the bucket capacity for each monitoring object, the burst value of the IOPS and the bandwidth of the part of controllers can be triggered only at the moment even if the burst flow queue mechanism triggering threshold is met, and finally the burst value of the IOPS and the bandwidth can not reach the expected value; only when all controllers accumulate tokens for the monitored object that will reach the expected value (i.e., the maximum burst traffic threshold) at the time the burst traffic is encountered will the burst values of the IOPS and bandwidth reach the bucket capacity.

In other optional implementations of the embodiments of the present invention, when accumulating flow resources for a monitored object, each controller of the unified storage system accumulates flow resources for the monitored object according to a preset ratio, further includes: when accumulating flow resources for the monitored object, each controller of the unified storage system puts tokens into a token bucket corresponding to the monitored object according to a second preset token putting rate until the number of tokens in the token bucket of the monitored object reaches the number of tokens corresponding to the maximum burst flow threshold of the monitored object; the second preset token throwing rate is determined according to the token throwing rate corresponding to the data input and output rate of the monitoring object divided by the number of the controllers, and is larger than the token throwing rate corresponding to the data input and output rate of the monitoring object divided by the number of the controllers.

That is, when the controllers accumulate tokens for the monitored objects, namely, the token input rate corresponding to the data input/output rate of the monitored objects divided by the number of the controllers is used, the total number of the accumulated tokens of each controller is the bucket capacity of the token bucket divided by the number of the controllers and the corresponding token number is used, and when the controllers realize the control of the monitored objects, the total token number of the token bucket is obtained as accumulated flow resources. The step of extracting the token from the token bucket of the monitored object to allocate the corresponding data input/output traffic resource for the monitored object may include: taking the number of tokens in the token bucket as the available number of tokens of the monitoring object, and taking out the tokens from the available number of tokens to provide the required data traffic resources for the monitoring object.

As shown in fig. 2, in the token count calculation stage, the data input/output flow rate per second of the monitoring object is determined according to the set data input/output flow rate per second of the monitoring object.

When the set bandwidth exists, if the bandwidth is converted according to the 4K standard, the allowable 4K number per second is calculated according to the set value, and then the token putting rate (fixed x per second) is obtained by combining the set data input/output flow per second.

And in the upper flow control stage, determining the token throwing rate according to the current data input and output flow of the monitored object, for example, fixedly throwing x tokens per second. Keeping judging whether the token bucket is full, and discarding redundant tokens if the token bucket is full; if not, continuing to deliver the tokens to the token bucket according to the token delivery rate. Meanwhile, combining the set bandwidth, when bandwidth monitoring exists, calculating the required token number (the current data input/output flow divided by 4K) of the monitored object, if no bandwidth monitoring exists, skipping the step, and calculating the required token number of the monitored object only according to the data input/output flow of the monitored object. Judging whether a token is acquired, if so, submitting the token to a downstream layer data input/output stack; if no token is acquired, the data input/output request of the monitoring object is queued.

In the burst flow control stage, judging whether the token bucket is full, and if not, continuously accumulating tokens; and judging whether the token request rate is smaller than the token putting rate, if so, continuously judging whether the token bucket is full, if not, judging whether the accumulated amount of tokens is satisfied to trigger a burst flow queue mechanism, if so, consuming the accumulated tokens to exceed a burst flow queue mechanism triggering threshold by utilizing accumulated flow resources, and submitting the accumulated tokens to a lower layer data input and output stack by utilizing the tokens in the token bucket.

Fig. 3 is a flowchart of another flow control method of a unified storage system according to an embodiment of the present invention.

On the basis of the embodiment, the embodiment of the invention further describes a deployment method of the flow control method of the unified storage system.

In the embodiment of the present invention, as shown in fig. 3, the steps of the flow control method of the unified storage system provided in the embodiment of the present invention may be performed by a deployment command line unit, an analysis management unit, a performance statistics unit, and a current limit control unit.

The command line unit can create a trigger threshold value, a maximum burst flow threshold value and burst flow using time of a burst flow queue mechanism of each monitoring object through mkthrottle commands of the command line. Specifically, the analysis management unit performs average processing and saving on the relevant threshold value of the monitored object, and then notifies the performance statistics unit of the client through an interface call command (call_ geta). The analysis management unit is used for comparing the set value of the command line and the number of the current controllers with the data input/output flow to obtain the average flow limit of each controller or each volume in the volume group. Setting a mean value which is called by a command line for the first time, and calculating the mean value which is used as reference data in the subsequent system evaluation; and then notifying the client, starting the performance statistics unit and reporting the one-time performance value, after receiving the information reported by the client, acquiring the set quality of service (Qos) requirement value and the average value calculated previously in combination with the command line, and obtaining the actual current limit value of each monitoring object through system evaluation and comprehensive analysis and notifying the current limit control unit.

The performance statistics unit is used for triggering when related burst flow demands exist and acquiring performance values of all monitoring objects (volumes and volume groups) of all controllers; the average value calculation processing in unit time can be carried out on the data input and output flow of the monitoring object; in addition, a monitoring function can be set, so that when the performance of the monitoring object is changed or reaches a certain set threshold value, the performance information of the corresponding monitoring object is reported to an analysis management unit of the management end. After the relevant threshold value is set, the performance value of the monitoring object is changed, at the moment, the performance statistical information is reported to the management end once again, the feedback information is compared with the command line set value, whether the feedback information is within the error allowable range or not is judged, if the process is finished, the distribution value and the actual value of each monitoring object on each controller are not compared, and fine adjustment is carried out.

The performance statistics unit starts to acquire the performance value (data input/output flow) of each monitoring object, reports the effective value to the analysis management unit of the management end through an event (event) when the effective value is acquired, and starts performance monitoring of the monitoring object.

And the analysis management unit of the management end performs comprehensive analysis according to the reporting information, the command line set value, the average value and the like of the performance statistics unit to obtain a current limit value which is actually set by a certain monitoring object of a certain controller.

The analysis management unit initializes each field in the current limiting object according to the analyzed setting value, and calculates 'bucket capacity' and the like; when the performance is below the set "upper limit," a "quota" is started to be accumulated (i.e., traffic resources are accumulated); when the available "quota" value exceeds 80% of the "bucket capacity" and the consumption quota speed is greater than the offered quota speed, setting a "state switch flag bit" to valid (TRUE) indicating a switch to a bursty traffic state; when burst traffic exists, allowing the data input/output traffic of the monitored object to exceed the burst traffic queue mechanism triggering threshold value within a certain time, and maximally not exceeding the maximum burst traffic threshold value; the current limit when the monitoring object reaches the maximum burst flow threshold value uses the available quota (namely, the part smaller than the burst flow queue mechanism triggering threshold value) of the monitoring object, and synchronously consumes the accumulated quota; when the accumulated quota is less than or equal to 0 or reaches the set burst traffic use time, the state switching flag bit is set to be invalid (FALSE), which indicates that the switching to the burst traffic queue mechanism triggers the threshold state.

As shown in fig. 3, another method for controlling a flow of a unified storage system according to an embodiment of the present invention may include:

The analysis management unit sets a related threshold value of the burst flow queue mechanism, and the current limiting control unit distributes the monitoring object and fills each data value of the monitoring object.

Judging whether each volume or volume group in the system is provided with a burst queue mechanism or not; if yes, determining the monitoring object, entering the subsequent step, if not, skipping the related logic of the burst queue mechanism, and submitting the data input/output request downwards.

For a monitoring object, judging whether the available quota is larger than a threshold value and the consumed quota speed is larger than the provided quota speed, if so, switching to a state of maximum accumulated flow resource threshold value for operation, wherein the quota can be continuously accumulated according to the state when a burst flow queue mechanism triggers the threshold value, the maximum is not more than 'bucket capacity', and when a burst exists, the flow can exceed the burst flow queue mechanism triggering threshold value, but the maximum is not more than the maximum accumulated flow resource threshold value, and the accumulated quota is consumed at the moment; if not, triggering threshold state operation by using the burst flow queue mechanism, and starting to accumulate quota when the actual flow is smaller than the burst flow queue mechanism triggering threshold.

After the monitoring object enters the burst flow processing flow, judging that the available quota is less than or equal to 0 or reaches the set maximum burst flow time, if so, switching back to the burst flow queue mechanism triggering threshold state, returning to the burst flow queue mechanism triggering threshold state for operation, and starting to accumulate the quota when the actual flow is less than the burst flow queue mechanism triggering threshold.

On the basis of the above embodiment, in order to ensure that the service does not have excessive jitter due to the burst traffic mechanism, in the embodiment of the present invention, the performing, by S102, the traffic monitoring on the monitored object may include: performing flow monitoring on a monitored object based on a preset anti-shake range; the preset anti-shake range includes at least one of counting nothing in preset time after the data input/output flow is started initially, taking average value in unit time as a statistic value for the data input/output flow, marking as effective triggering when the data input/output flow triggering threshold reaches preset times, and marking as effective triggering an error range of the data input/output flow triggering threshold.

In the embodiment of the present invention, the anti-shake mechanism may be set from the following four angles: (1) Statistics are not counted in a preset time after the data input/output flow is started initially, for example, when the data input/output operation of the monitored object is started initially, that is, when the data input/output operation is changed from 0 to non-0, the data of the first two seconds can be obtained and not counted.

(2) The average value in unit time is used as a statistical value for the data input/output flow, for example, the average value can be obtained within 10 seconds of more than 3 seconds continuously, and the average value can be used as a reference.

(3) And when the data input/output flow triggering threshold reaches the preset times, the data input/output flow triggering threshold is marked as effective triggering, for example, the data input/output flow triggering threshold can be set to reach the relevant threshold for 3 times continuously, and the reporting flow or the control flow is allowed to be triggered.

(4) The error range of the data input/output flow trigger threshold is noted as a valid trigger, for example, an error range may be set within which the performance value is considered to meet expectations.

Fig. 4 is a flowchart of a specific implementation of S102 according to an embodiment of the present invention.

On the basis of the embodiment, the method can also adapt to the actual flow distribution time of the service corresponding to the monitoring object, and avoid the influence on the work of other services caused by the accumulation of flow resources for a certain monitoring object at all times. In the embodiment of the present invention, the step S102 of performing flow monitoring on the monitored object may include: if the effective time condition of the monitoring object is met, executing flow monitoring of the monitoring object; wherein the validation time condition includes at least one of being within a single validation time and within an arrival cycle period.

That is, the effective time may be set, and the trigger time of the burst traffic queue mechanism may be set by the user according to a specific service scenario, for example, a service is as early as 9: 00-11: 00 is busy, and the effective time can be set to 9:00.

As shown in FIG. 4, the mechanism for "time of effect" may include three modes, single, weekly, daily. A timer is set at initialization, and different timer cycle periods are designated according to different modes.

When the state is changed from a 'blank' mode to wait (Waiting) during initial setting, and when the state is modified halfway, the state is changed from the current state to wait; whether the time of initial setting or midway modification is not before the current system time, if the time is just equal to the current time, the system is started immediately, and the state is changed into an operation (Active); if the starting time is not reached, waiting is carried out, and the state is changed into waiting; in the non-null mode, the "immediate effect" parameter may not be set to "no".

Calculating the time difference between the current time and the set time of the system, if the time difference is more than 24 hours, circulating in days, when the time difference is more than 1 and less than 24 hours, circulating in hours, when the distance is less than 60 minutes, circulating in minutes, until the set time is reached, starting the set quality of service characteristic, and changing the state from waiting to running.

In the 'single' mode, if the effective time is 0, the timer is directly cancelled, which means that the timer is permanently effective from the set starting time; if the timer is not 0, the triggering period of the timer is changed according to the set duration, when the duration arrives, the characteristic is changed to be invalid, then the timer is cancelled, and the state is changed from running to stopping (Inactive).

Mode "weekly" and "daily": and changing the triggering period of the timer according to the set duration, changing the characteristic into invalid when the duration arrives, changing the state from running to waiting, and then entering the next cycle period.

S102 performs traffic monitoring on the monitored object, which may include:

step 1: the command line unit receives the command line parameters, and besides the characteristic value parameters of the burst flow queue mechanism, the command line unit also comprises parameters such as an operation mode, a starting time, an effective duration and the like.

Step 2: the command line unit communicates the parameter information to a client (CLIENT SIDE) management side of the hardware layer (HARDWARE LAYER) module.

Step 3: and starting a timer, and determining the interval duration of the timer according to the set mode and the starting time.

Step 4: cycling by day when the distance from the start time exceeds 24 hours; an hourly cycle of greater than 1 and less than 24 hours; cycling by minutes of less than 60 minutes.

Step 5: the characteristics are started after the arrival time.

Step 6: and (4) obtaining the end time according to the effective duration, wherein the timer running mode is as shown in the step (4).

Step 7: time-after-arrival stop feature.

Step 8: after the end, the timer is stopped in the single mode; the circulation mode enters the next period, and the steps 4,5, 6 and 7 are repeated.

Step 9: when the mode is set to null mode or the entire feature is deleted, the timer expires.

As shown in fig. 4, a specific implementation manner of S102 provided in the embodiment of the present invention may include:

Reading command line setting, judging whether the service quality characteristics exist, if so, recording each parameter value, and providing the parameter values for a mode management sub-module; if not, ending.

In the mode management sub-module, different branches are entered according to the set mode, if the mode is an empty mode, parameter values are emptied, and the timer is stopped and then the operation is finished; if the mode is a single time mode, a cycle mode or a daily cycle mode, a corresponding timer is entered.

The timer judges whether the distance starting time is more than 24 hours, if so, the timer is set to take a day as a period; if not, judging that the distance starting time is more than 1 hour. If the distance starting time is more than 1 hour, setting a timer to take the hour as a period; if the distance from the start time is not more than 1 hour, a timer is set to be periodic with minutes.

Judging whether the time is reached, and if the time is reached, judging whether the time is started. If the time is not reached, the step of setting the timer to be periodic in minutes is returned.

If the starting time is the starting time, starting the service quality characteristic, calculating the ending time, and returning to the step of judging whether the starting time is longer than 24 hours according to the single mode, the cycle mode or the daily cycle mode. If the current mode is the single mode, the step of judging whether the distance starting time is more than 24 hours is returned if the current mode is the cycle mode, and if the current mode is the daily cycle mode, the step of judging whether the distance starting time is more than 24 hours is returned.

It should be noted that, in the embodiments of the flow control method of the unified storage systems of the present invention, some of the steps or features may be omitted or not performed. The divided hardware or software functional modules are not the only implementation form for realizing the flow control method of the unified storage system provided by the embodiment of the invention.

Various embodiments of a flow control method of a unified storage system are detailed above, and on the basis of the embodiments, the invention also discloses a flow control device, equipment, a nonvolatile storage medium and a computer program product of the unified storage system corresponding to the method.

Fig. 5 is a schematic structural diagram of a flow control device of a unified storage system according to an embodiment of the present invention.

As shown in fig. 5, a controller applied to a unified storage system of multiple controllers, a flow control device of the unified storage system provided by an embodiment of the present invention includes:

An analysis management unit 501, configured to determine a burst traffic queue mechanism trigger threshold, a maximum burst traffic threshold, and a burst traffic use time of a monitoring object; if the received performance statistics unit 502 reports that the data input/output flow of the monitored object does not exceed the monitoring result of the trigger threshold of the corresponding burst flow queue mechanism, the current limiting control unit 503 is notified to accumulate flow resources for the monitored object by taking the corresponding maximum burst flow threshold as the upper limit; if the received performance statistics unit 502 reports that the data input/output flow of the monitored object exceeds the corresponding burst flow queue mechanism trigger threshold, notifying the current limiting control unit 503 to control the monitored object to enter a burst flow processing flow so as to allow the monitored object to use the accumulated flow resources in the corresponding burst flow using time;

a performance statistics unit 502, configured to perform flow monitoring on a monitored object, and report a monitoring result to an analysis management unit 501;

A current limit control unit 503 for executing the control task notified by the analysis management unit 501;

The flow control device of the unified storage system provided by the embodiment of the invention can further comprise:

The fault switching module is used for taking over the data flow and monitoring work of the fault controller by adopting a standby controller corresponding to the mirror image pair divided by the unified storage system when the controller for monitoring the monitoring object is faulty, and recombining the mirror image pairs so that the number of the mirror image pairs is the same as that of the controllers which normally run;

When the standby controller takes over the monitoring work of the fault controller, the part, corresponding to the fault controller, of the accumulated flow resources of the monitored object monitored by the fault controller is emptied, and the preset proportion is adjusted to be a proportion corresponding to the number of the controllers which normally run.

In the embodiments of the flow control device of the unified storage system according to the present invention, the division of the units is only one logical division, and other division modes may be adopted. The connection between the different units may be electrical, mechanical or other. Separate units may be located in the same physical location or distributed across multiple network nodes. The units may be implemented in hardware or in software functional units. The aim of the scheme of the embodiment of the invention can be realized by selecting part or all of the units provided by the embodiment of the invention according to actual needs and adopting a corresponding connection mode or an integration mode.

Since the embodiments of the apparatus portion and the embodiments of the method portion correspond to each other, the embodiments of the apparatus portion are referred to the description of the embodiments of the method portion, and are not repeated herein.

As shown in fig. 6, a flow control device of a unified storage system according to an embodiment of the present invention includes:

a memory 610 for storing a computer program 611;

A processor 620, configured to execute a computer program 611, where the computer program 611 when executed by the processor 620 implements the steps of the flow control method of the unified storage system provided in any of the embodiments described above.

Processor 620 may include one or more processing cores, such as a 3-core processor, an 8-core processor, etc., among others. The processor 620 may be implemented in at least one hardware form of digital signal Processing (DIGITAL SIGNAL Processing, DSP), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA), programmable logic array (Programmable Logic Array, PLA). Processor 620 may also include a main processor, which is a processor for processing data in an awake state, also referred to as a central processor (Central Processing Unit, CPU), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 620 may be integrated with an image processor (Graphics Processing Unit, GPU) for use in responsible for rendering and rendering of the content that is to be displayed by the display screen. In some embodiments, the processor 620 may also include an artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) processor for processing computing operations related to machine learning.

The memory 610 may include one or more non-volatile storage media, which may be non-transitory. Memory 610 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 610 is at least used for storing a computer program 611, where the computer program 611, when loaded and executed by the processor 620, can implement relevant steps in the flow control method of the unified storage system disclosed in any of the foregoing embodiments. In addition, the resources stored by the memory 610 may also include an operating system 612, data 613, and the like, and the storage manner may be transient storage or permanent storage. Operating system 612 may be Windows or another type of operating system. The data 613 may include, but is not limited to, data related to the above-described method.

In some embodiments, the flow control device of the unified memory system may further comprise a display 630, a power supply 640, a communication interface 650, an input-output interface 660, a sensor 670, and a communication bus 680.

Those skilled in the art will appreciate that the configuration shown in fig. 6 is not limiting of the flow control device of the unified storage system and may include more or fewer components than shown.

The flow control device of the unified storage system provided by the embodiment of the invention comprises a memory and a processor, wherein the processor can realize the steps of the flow control method of the unified storage system provided by the embodiment when executing the program stored in the memory.

An embodiment of the present invention provides a non-volatile storage medium having stored thereon a computer program which, when executed by a processor, can implement the steps of the flow control method of a unified storage system as provided in any one of the above embodiments.

The nonvolatile storage medium may include: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

For the introduction of the non-volatile storage medium provided by the embodiment of the present invention, please refer to the above method embodiment, and the effect achieved by the method is the same as the flow control method of the unified storage system provided by the embodiment of the present invention, which is not described herein.

Embodiments of the present invention provide a computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the flow control method of a unified storage system as provided in any of the embodiments above.

For the introduction of the computer program product provided by the embodiment of the present invention, please refer to the above method embodiment, and the effect achieved by the method is the same as the flow control method of the unified storage system provided by the embodiment of the present invention, and the description of the present invention is omitted here.

The flow control method, the device, the equipment and the medium of the unified storage system provided by the invention are described in detail. In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. The apparatus, device, non-volatile storage medium, and computer program product of the embodiments disclosed herein, as they correspond to the methods and systems of the embodiments disclosed herein, are described in the relatively simple text, where relevant to the description of the methods and systems sections. It should be noted that it will be apparent to those skilled in the art that the present invention may be modified and practiced without departing from the spirit of the present invention.

It should also be noted that in this specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

Claims

1. A flow control method for a unified storage system, the method being applied to a controller in a unified storage system of multiple controllers, comprising:

Performing flow monitoring on the monitoring object;

2. The flow control method of the unified storage system according to claim 1, wherein each of the controllers accumulates flow resources for the monitoring object according to a preset ratio, comprising:

3. The flow control method of the unified storage system according to claim 1, wherein each of the controllers accumulates flow resources for the monitoring object according to a preset ratio, comprising:

4. The method for controlling a flow of a unified storage system according to claim 1, wherein if the monitored object is the volume group, determining a burst flow queue mechanism trigger threshold, a maximum burst flow threshold, and a burst flow usage time of the monitored object comprises:

5. The method for controlling a flow of a unified storage system according to claim 1, wherein if the monitored object is the volume group, determining a burst flow queue mechanism trigger threshold, a maximum burst flow threshold, and a burst flow usage time of the monitored object comprises:

6. The method for controlling flow of a unified storage system according to claim 1, wherein accumulating the flow resources for the monitoring object with the corresponding maximum burst flow threshold as an upper limit comprises:

7. The flow control method of the unified storage system according to claim 6, wherein determining a rate at which flow resources are accumulated for the monitoring object according to the data input output flow of the monitoring object comprises:

8. The method for controlling flow of a unified storage system according to claim 1, wherein accumulating the flow resources for the monitoring object with the corresponding maximum burst flow threshold as an upper limit comprises:

9. The flow control method of the unified storage system according to claim 8, wherein each of the controllers of the unified storage system accumulates flow resources for the monitoring object according to a preset ratio when accumulating flow resources for the monitoring object, comprising:

10. The flow control method of the unified storage system according to claim 8, wherein each of the controllers of the unified storage system accumulates flow resources for the monitoring object according to a preset ratio when accumulating flow resources for the monitoring object, comprising:

11. The method of claim 8, wherein retrieving tokens from the token bucket of the monitored object to allocate corresponding data input and output traffic resources for the monitored object comprises:

12. The method of claim 9, wherein retrieving tokens from the token bucket of the monitored object to allocate corresponding data input and output traffic resources for the monitored object comprises:

13. The flow control method of the unified storage system according to claim 1, wherein performing flow monitoring of the monitoring object includes:

14. The flow control method of the unified storage system according to claim 1, wherein performing flow monitoring of the monitoring object includes:

15. The method for controlling flow of a unified storage system according to claim 1, further comprising:

16. The unified storage system is characterized by comprising a plurality of controllers, wherein the controllers are used for determining a burst flow queue mechanism trigger threshold value, a maximum burst flow threshold value and burst flow using time of a monitored object; performing flow monitoring on the monitoring object; if the data input/output flow of the monitoring object does not exceed the corresponding burst flow queue mechanism threshold, taking the corresponding maximum burst flow threshold as an upper limit to accumulate flow resources of the monitoring object; if the data input/output flow of the monitoring object exceeds the corresponding burst flow queue mechanism triggering threshold, controlling the monitoring object to enter a burst flow processing flow so as to allow the monitoring object to use accumulated flow resources in the corresponding burst flow using time;

17. A flow control device for a unified storage system, the flow control device being applied to a controller in a unified storage system of a plurality of controllers, comprising:

18. A flow control device for a unified storage system, comprising:

a memory for storing a computer program;

A processor for executing the computer program, which when executed by the processor performs the steps of the flow control method of the unified storage system of any one of claims 1 to 15.

19. A non-volatile storage medium having stored thereon a computer program, which when executed by a processor performs the steps of the flow control method of a unified storage system according to any of claims 1 to 15.

20. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the flow control method of a unified storage system according to any one of claims 1 to 15.