Nothing Special   »   [go: up one dir, main page]

WO2017140131A1 - 数据写入、读取方法及装置、云存储系统 - Google Patents

数据写入、读取方法及装置、云存储系统 Download PDF

Info

Publication number
WO2017140131A1
WO2017140131A1 PCT/CN2016/105094 CN2016105094W WO2017140131A1 WO 2017140131 A1 WO2017140131 A1 WO 2017140131A1 CN 2016105094 W CN2016105094 W CN 2016105094W WO 2017140131 A1 WO2017140131 A1 WO 2017140131A1
Authority
WO
WIPO (PCT)
Prior art keywords
storage server
data
management node
storage
index
Prior art date
Application number
PCT/CN2016/105094
Other languages
English (en)
French (fr)
Inventor
金平
林起芊
Original Assignee
杭州海康威视数字技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州海康威视数字技术股份有限公司 filed Critical 杭州海康威视数字技术股份有限公司
Priority to US15/999,307 priority Critical patent/US11586673B2/en
Priority to EP16890357.3A priority patent/EP3418877B1/en
Publication of WO2017140131A1 publication Critical patent/WO2017140131A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614—Improving the reliability of storage systems
    • G06F3/0617—Improving the reliability of storage systems in relation to availability
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614—Improving the reliability of storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/901—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1415—Saving, restoring, recovering or retrying at system level
    • G06F11/142—Reconfiguring to eliminate the error
    • G06F11/1425—Reconfiguring to eliminate the error by reconfiguration of node membership
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/3034—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a storage system, e.g. DASD based or network based
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/903—Querying
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G06F3/0605—Improving or facilitating administration, e.g. storage management by facilitating the interaction with a user or administrator
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G06F3/0635—Configuration or reconfiguration of storage systems by changing the path, e.g. traffic rerouting, path reconfiguration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • H04L67/1044—Group management mechanisms 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Definitions

  • the present application relates to the field of data processing technologies, and in particular, to a data writing and reading method and apparatus, and a cloud storage system.
  • Cloud storage is a new concept extended and developed in the concept of cloud computing. It is an emerging network storage technology that refers to functions such as cluster applications, network technologies or distributed file systems. A system in which a plurality of different types of storage devices work together through application software to jointly provide data storage and service access functions.
  • the cloud storage system is a distributed system with a master-slave architecture.
  • the master node that is, the management node, plays a vital role in the normal operation of the entire system. In the event of a failure of the management node, all services of the entire system cannot be performed normally. For example, when an access node in the cloud storage system receives a data write request, the access node needs to send the data write request to the management node, so that the management node allocates a corresponding storage server for the data write request. .
  • the management node when the management node fails, the management node will not respond to the data processing request sent by the access node, so the data processing will fail.
  • the architecture of the video cloud storage system shown in FIG. 1 is taken as an example to describe the corresponding processing flow when the available management node cannot be obtained.
  • the video cloud storage management node 130 When the available video cloud storage management node (CVM, Cloud Video Manage) 130 is not available, the video cloud storage management node 130 will not make a data processing request sent by the video cloud storage access node (CVA, Cloud Video Access) 120.
  • the video cloud storage access node 120 will not receive the video cloud storage storage server (CVS, Cloud Video Storage) 140 allocated by the video cloud storage management node 130 for the data processing request, and the data processing will fail.
  • CVS Cloud Video Storage
  • the main method is to increase the number of management nodes, and ensure that some management nodes are taken over by the normal management node when HA (High Availability) or cluster technology is used to ensure that some management nodes are faulty.
  • Service or add a standby management node to provide services in place of the fault management node.
  • adding a management node or adding a standby management node increases the project cost.
  • adding a management node or adding a standby management node can only ensure that the cloud storage system runs normally when some management nodes fail, and cannot guarantee that all management nodes are faulty. The cloud storage system runs normally if the management node cluster network is abnormal.
  • the purpose of the embodiment of the present application is to provide a data writing and reading method and device, and a cloud storage system, so that the cloud storage system can still operate normally when all management nodes are faulty or the management node cluster network is abnormal. Add extra cost.
  • the specific technical solutions are as follows:
  • the embodiment of the present application provides a data storage method based on a cloud storage system, where the cloud storage system includes at least: an access node, at least one management node, and at least one storage server, where the method includes:
  • the access node receives a data write request
  • the domain identifier information included in the data write request is identified, and the pre-stored index is queried according to the domain identifier information, where the index includes the resource pool and the domain identifier information.
  • each resource pool includes at least one storage server
  • the determining that one of the storage servers is a target storage server comprises:
  • the failure to obtain an available management node includes: each management node fails or the management node cluster network abnormality.
  • the process of constructing the index includes:
  • the method further includes:
  • Corresponding relationship between each resource pool and domain identifier information saved in the index is updated according to the set time interval.
  • the embodiment of the present application provides a data storage method based on a cloud storage system, where the cloud storage system includes: at least one management node and at least one storage server, which are applied to a query platform, and the method includes:
  • the pre-saved index is queried, wherein the index includes a correspondence between each storage server and its IP address;
  • the index further includes status information of whether each storage server is online, and the request for sending the attribute information of the data to each storage server according to the IP address of each storage server includes:
  • a request for acquiring attribute information of the data is transmitted to each of the corresponding storage servers in the online state according to the IP address of the storage server in the online state.
  • the failure to obtain an available management node includes: each management node fails or the management node cluster network abnormality.
  • the process of constructing the index includes:
  • the method further includes:
  • each storage server and the IP address saved in the index is updated according to the set time interval.
  • the embodiment of the present application provides a data storage device based on a cloud storage system, where the cloud storage system includes at least: an access node, at least one management node, and at least one storage server, and is applied to the access Node, the device includes:
  • a receiving module configured to receive a data write request
  • a processing module configured to identify domain identifier information included in the data write request, and query a pre-saved index according to the domain identifier information, where the index includes a resource pool, when the available management node is unavailable Correspondence with domain identification information;
  • An identification module configured to identify a target resource pool that matches the domain identifier information, where each resource pool includes at least one storage server;
  • a write module configured to determine, from the at least one storage server included in the target resource pool, one of the storage servers as a target storage server, and write the data to be written to the target storage server.
  • writing module is specifically configured to:
  • the failure to obtain an available management node includes: each management node fails or the management node cluster network abnormality.
  • the device further includes:
  • An execution module configured to send, to each management node, a request for obtaining a correspondence between each resource pool and domain identifier information, and receiving a correspondence between each resource pool and domain identifier information sent by each management node;
  • a saving module configured to save, in the index, a correspondence between each resource pool and domain identification information.
  • the device further includes:
  • an update module configured to update, according to the set time interval, a correspondence between each resource pool and the domain identifier information saved in the index.
  • the embodiment of the present application provides a data storage device based on a cloud storage system, where the cloud storage system includes at least one management node and at least one storage server, and is applied to the query platform, where the device includes:
  • a receiving module configured to receive a data read request
  • a query module configured to query a pre-saved index when the available management node cannot be obtained, where the index includes a correspondence between each storage server and its IP address;
  • a processing module configured to send, according to an IP address of each storage server, a request for acquiring attribute information of the data to each storage server, and receive corresponding data that is saved by the storage server and returned by the storage server Attribute information
  • a reading module configured to determine at least one target storage server according to the received attribute information of the data that is saved by each storage server and the attribute information of the data to be read included in the data read request And read the corresponding data from each target storage server.
  • the index further includes status information of whether each storage server is online, and the processing module includes:
  • the processing submodule is configured to send, according to the IP address of the storage server in an online state, a request for acquiring attribute information of the data to each corresponding storage server in an online state.
  • the failure to obtain an available management node includes: each management node fails or the management node cluster network abnormality.
  • the device further includes:
  • An execution module configured to send, to each management node, a request for obtaining a correspondence between each storage server and an IP address, and receive a correspondence between each storage server and an IP address sent by each management node;
  • a saving module configured to save, in the index, a correspondence between each storage server and an IP address.
  • the device further includes:
  • an update module configured to update, according to the set time interval, a correspondence between each storage server and an IP address saved in the index.
  • the embodiment of the present application provides a cloud storage system, where the cloud storage system includes the device disposed in the access node, and the at least one storage server, as described in the foregoing third aspect.
  • the application provides a storage medium, where the storage medium is used to store executable program code, and the executable program code is configured to execute a cloud storage system based on the present application at runtime. Data writing method.
  • the application provides a storage medium, where the storage medium is used to store executable program code, and the executable program code is configured to execute a cloud storage system based on the present application at runtime. Data reading method.
  • the application provides an application, wherein the application is configured to execute a cloud storage system-based data writing method described in the present application at runtime.
  • the application provides an application, wherein the application is configured to execute a cloud storage system-based data reading method described in the present application at runtime.
  • the application provides an access node device, including:
  • processor a memory, a communication interface, and a bus
  • the processor, the memory, and the communication interface are connected by the bus and complete communication with each other;
  • the memory stores executable program code
  • the processor runs a program corresponding to the executable program code by reading executable program code stored in the memory, for executing a cloud storage system-based data writing method described in the present application. .
  • the application provides an electronic device, including:
  • processor a memory, a communication interface, and a bus
  • the processor, the memory, and the communication interface are connected by the bus and complete communication with each other;
  • the memory stores executable program code
  • the processor runs a program corresponding to the executable program code by reading executable program code stored in the memory, for executing a cloud storage system-based data reading method described in the present application. .
  • the embodiment of the present application provides a data writing and reading method and device, and a cloud storage system, where the data writing method includes: the access node receives a data write request; when an available management node cannot be obtained, Identifying the domain identifier information included in the data write request, and querying the pre-stored index according to the domain identifier information, where the index includes a correspondence between the resource pool and the domain identifier information; and identifying and the domain identifier a target resource pool that matches information, wherein each resource pool includes at least one storage server; and from at least one storage server included in the target resource pool, one of the storage servers is determined to be a target storage server, and is to be written Data is written to the target storage server.
  • the embodiment of the present application can ensure that the cloud storage system can still operate normally when all management nodes are faulty or the management node cluster network is abnormal, and no additional cost is required.
  • Figure 1 is a schematic diagram of a cloud storage system unable to provide services when a management node cannot be obtained;
  • FIG. 2 is a schematic flow diagram of a cloud storage system processing a data request under normal conditions
  • FIG. 3 is a data writing method based on a cloud storage system according to an embodiment of the present disclosure
  • FIG. 4 is a method for constructing an index according to an embodiment of the present disclosure
  • FIG. 5 is a data reading method based on a cloud storage system according to an embodiment of the present invention.
  • FIG. 6 is another method for constructing an index according to this embodiment.
  • FIG. 7 is a data writing device based on a cloud storage system according to an embodiment of the present invention.
  • FIG. 8 is a data storage device based on a cloud storage system according to an embodiment of the present invention.
  • FIG. 3 is a data writing method of a cloud storage system according to the embodiment, which may include the following steps:
  • step 301 the access node receives a data write request.
  • the cloud storage system may include at least: an access node, at least one management node, and at least one storage server.
  • the access node may be configured to receive a data write request and send the data write request to the management node, such that the management node allocates a corresponding storage server for the data write request.
  • the access node may further store the request for the data write request according to the management node
  • the storage server processes the corresponding data in the data write request.
  • the access node may first receive a data write request and write the corresponding data.
  • the architecture of the video cloud storage system shown in FIG. 2 is taken as an example to describe a process of processing a data request by a cloud storage system under normal circumstances.
  • the electronic device 110 may send a data processing request to a video cloud storage access node (CVA) 120.
  • CVA video cloud storage access node
  • the access node 120 needs to send the data processing request.
  • a management node (CVM) 130 is provided to cause the management node 130 to allocate a corresponding storage server (CVS) 140 for the data processing request.
  • CVS storage server
  • the allocation result may be fed back to the access node 120, and the access node 120 may perform corresponding data processing, such as data read and write operations, based on the allocated storage server 140. .
  • step 302 when the available management node is unavailable, the domain identifier information included in the data write request is identified, and the pre-stored index is queried according to the domain identifier information, where the index includes the resource pool and the domain. The correspondence between the identification information.
  • a plurality of storage servers may be generally included.
  • resource pools may be established, where each resource pool may include at least one storage server.
  • Each resource pool can have corresponding domain identification information.
  • the data write request When the access node receives the data write request, the data write request usually includes the domain identifier information, that is, the domain identifier information of the resource pool in which the storage server to be written is to be written, so that the access node can Based on the domain identification information, a corresponding storage server is allocated for the data write request.
  • the access node may send a data write request to the management node, so that the management node allocates a corresponding storage server for the data write request.
  • the access node will not be able to obtain the storage server allocated for the data write request through the management node.
  • an index including a correspondence relationship between the resource pool and the domain identification information may be saved in advance.
  • the access node may write the domain identifier information in the request according to the identification data, and query the pre-prefix according to the domain identifier information.
  • Step 303 Identify a target resource pool that matches the domain identifier information, where each resource pool includes at least one storage server.
  • the access node may search for the correspondence between each resource pool and the domain identifier information saved in the index to identify the data write request.
  • the resource pool corresponding to the domain identifier information included in the domain, and the identified resource pool is used as the target resource pool to process the data write request based on the target resource pool.
  • Step 304 Determine, from the at least one storage server included in the target resource pool, that one of the storage servers is a target storage server, and write the data to be written to the target storage server.
  • the access node may further determine, from the at least one storage server included in the target resource pool, that one of the storage servers is the target storage server, To write the data to be written to the target storage server.
  • one of the at least one storage server included in the target resource pool may be selected as the target storage server.
  • the current load amount of each storage server may be first obtained, and then the current storage server identifies the current load. The storage server with the smallest load as the target storage server.
  • the cloud storage system is a video cloud storage system
  • a large amount of video data can be stored in the video cloud storage system.
  • the video capture system can write the collected real-time video data into the video cloud storage system.
  • the video data collected by the same video capture system is saved to a fixed storage server of the video cloud storage system, and the fixed storage server constitutes a resource pool, and the resource pool corresponds to a resource pool. Domain identification information. And when the read/write operation is configured, the address information of the video processing device of the fixed storage server is accessed.
  • the access node When the access node receives the video write request and cannot obtain the available management node, it may first identify the domain identifier information included in the video write request, and then according to the domain identifier information, and the resource pool and the domain identifier saved in the index. Corresponding relationship of the information, identifying a target resource pool that matches the domain identification information, and finally, determining, from the information of the at least one storage server included in the target resource pool, one of the storage servers as the target storage server, and writing the video data Enter the target storage server.
  • the current load amount of each storage server may be first obtained, and then the storage server with the smallest current load amount is identified, and As the target storage server.
  • the access node may issue a request for acquiring the current load amount to each storage server in the target resource pool, and acquire each storage according to the response message returned by each storage server. The current load of the server. Then, the storage server with the smallest current load can be used as the target storage server.
  • the access node may save the load corresponding to each storage server in the index during the process of building the index.
  • the access node can query the load of each storage server in the target resource pool saved in the index, and select the storage server with the smallest load as the target storage server.
  • the load corresponding to the target storage server saved in the index may be updated.
  • the method provided by the embodiment of the present application can identify that the data write request is included when receiving a data write request and failing to acquire an available management node by pre-storing an index including a correspondence relationship between the resource pool and the domain identifier information.
  • the domain identifier information and then, according to the domain identifier information, and the corresponding relationship between the resource pool and the domain identifier information saved in the index, identify a target resource pool that matches the domain identifier information, and finally from the target resource pool.
  • the at least one storage server included in the storage server determines that one of the storage servers is the target storage server to write data to the target storage server, and the cloud storage system can still be ensured when all the management nodes are faulty or the management node cluster network is abnormal. Write data normally without losing data.
  • FIG. 4 is a method for constructing an index according to the embodiment, which may include the following steps:
  • Step 401 Send a request for obtaining a correspondence between each resource pool and domain identifier information to each management node, and receive a correspondence between each resource pool and domain identifier information sent by each management node.
  • the management node plays a vital role in the normal operation of the entire system.
  • the management node needs to allocate a storage server for the data write request based on the attribute information of each storage server to write the corresponding data.
  • the attribute information of each storage server cannot be obtained. Therefore, the corresponding storage server cannot be allocated for the data write request, and data writing may fail.
  • an index including a correspondence relationship between each resource pool and domain identification information may be pre-built to be unavailable.
  • the attribute information of each storage server can still be obtained. Further, based on the attribute information of each storage server, a corresponding storage server is allocated for the data write request to ensure the normal operation of the cloud storage system.
  • the access node may send a request for obtaining a correspondence between each resource pool and domain identifier information to each management node, and receive a correspondence between each resource pool and domain identifier information sent by each management node.
  • Step 402 The corresponding relationship between each resource pool and domain identification information is saved in an index.
  • the corresponding relationship between each resource pool and the domain identifier information may be saved in the index, so that when the data write request is received and the management node cannot be obtained, according to the index
  • the corresponding relationship between each resource pool saved in the domain and the domain identification information determines a target resource pool corresponding to the data write request, and processes the data write request based on the target resource pool.
  • the correspondence between each resource pool and the domain identifier information may change. Therefore, in order to ensure the correctness of the correspondence between the resource pools and the domain identifier information saved in the index, the time interval may be set according to the set time interval ( For example, 2 hours, 12 hours, 24 hours, etc., the above steps 401 and 402 are performed to implement updating of the correspondence between each resource pool and domain identification information saved in the index.
  • the method provided in this embodiment pre-builds an index that includes a correspondence between each resource pool and domain identifier information.
  • the data write request may be determined according to the index.
  • At least one target resource pool, and the target storage server is determined based on the target resource pool to write the corresponding data, so that the cloud storage system can still operate normally when all the management nodes are faulty or the management node cluster network is abnormal.
  • FIG. 5 is a data storage method based on the cloud storage system, which is applied to the query platform, and may include the following steps:
  • Step 501 Receive a data read request.
  • the query platform when it performs data reading, it may first receive a data read request and read the corresponding data.
  • Step 502 Query a pre-saved index when the available management node cannot be obtained, where the index includes a correspondence between each storage server and its IP address.
  • the query platform may send a data read request to the management node, so that the management node allocates a corresponding storage server for the data read request.
  • the querying platform will not be able to obtain the storage server assigned to the data read request through the management node.
  • an index including a correspondence relationship of each storage server and its IP address can be stored in advance.
  • the query platform can query the index to obtain an IP address of each storage server.
  • Step 503 Send, according to the IP address of each storage server, a request for acquiring attribute information of the data to each storage server, and receive corresponding attribute information of the data that is saved by each storage server. .
  • the query platform may send a request for obtaining attribute information of the data to each storage server according to the IP address of each storage server, and receive the corresponding storage returned by each storage server.
  • the attribute information of the data may be sent to each storage server according to the IP address of each storage server, and receive the corresponding storage returned by each storage server.
  • the attribute information of the above data may include, for example, the key of data stored in each storage server. Words, such as when the data stored in the storage server is a road real-time monitoring video, the above attribute information may be road segment information, time information, and the like.
  • the query platform may receive attribute information of the data saved by each storage server corresponding to each storage server, to determine a storage server where the data to be read is located based on the attribute information, and read corresponding data from the corresponding storage server.
  • the storage server may be offline.
  • the query platform when the query platform sends the storage server the attribute information of the data it has saved, the storage server may not return a response message. Therefore, when there is a storage server in an offline state, the query platform may send resource information for obtaining the data it saves to each storage server, which may cause waste of resources.
  • status information of whether each storage server is online may be saved in an index of the query platform.
  • the query platform may first identify the storage server in the online state according to the state information of each storage server, and then send the corresponding storage server to the online storage server according to the IP address of the storage server in the online state. A request to get property information for a data.
  • Step 504 Determine, according to the received attribute information of the data that is saved by each storage server and the attribute information of the data to be read included in the data read request, at least one target storage server, and The corresponding data is read in each target storage server.
  • the query platform may be based on the attribute information of the data saved by each storage server and the attribute information of the data to be read included in the data read request. Identify at least one target storage server.
  • the data read request received by the query platform usually includes the attribute information of the data to be read. Therefore, after receiving the attribute information of the data saved by each storage server itself, the query platform can identify and include the data read request.
  • the storage server corresponding to the attribute information of the data to be read, and The identified storage server is used as the target storage server to read the corresponding data from the target storage server.
  • the cloud storage system is a video cloud storage system
  • a large amount of video data can be stored in the video cloud storage system, and the user can read the desired video data from the video cloud storage system.
  • the traffic monitoring video data is stored in the video cloud storage system
  • the user can read corresponding video data of the corresponding time segment or the corresponding road segment.
  • the data read request carries the attribute information of the video data to be read, and the attribute may be, for example, time period information and/or link information.
  • the query platform is provided with an interface that interacts with the platform, and the data read request sent by the platform can be received through the interface.
  • the query platform When the query platform receives the data read request and cannot obtain the available management node, the query saves the pre-stored index, where the index includes the correspondence between each storage server and its IP address, for example, reading 9:00-10:00. Video data. Then, the query platform may determine, according to the pre-built index, each storage server corresponding to the IP address, and send a request for acquiring attribute information of the video data to each storage server, and receive the corresponding self returned by each storage server. The saved video clip of the corresponding time. Finally, the querying platform may determine at least one target storage server according to the received video clips returned by each storage server and the attribute information of the video data to be read included in the video data read request. And sending the information of the target storage server to the query platform, querying the target storage server corresponding to the platform access, and reading the video data of the corresponding time period from the target storage server.
  • the method provided in this embodiment by pre-storing an index including a correspondence relationship between each storage server and its IP address, may receive a pre-stored index when obtaining a data read request and cannot obtain an available management node, and obtain each The storage server's IP address, and then, according to the IP address of each storage server, sends a request for obtaining the attribute information of the data saved by itself to each storage server, and receives the self-saved data sent by each corresponding storage server. Attribute information, and finally determining at least one target storage server from the target storage server according to the attribute information of the data stored by each storage server and the attribute information of the data to be read included in the data read request.
  • the corresponding data is read to ensure that the cloud storage system can still provide data reading services when all management nodes are faulty or the management node cluster network is abnormal.
  • FIG. 6 is another method for constructing an index according to the embodiment, which may include the following steps:
  • Step 601 Send a request for obtaining a correspondence between each storage server and an IP address to each management node, and receive a correspondence between each storage server and an IP address sent by each management node.
  • the management node plays a vital role in the normal operation of the entire system.
  • the management node needs to allocate a storage server for the data read request based on the attribute information of each storage server to perform corresponding data reading.
  • the attribute information of each storage server cannot be obtained. Therefore, the corresponding storage server cannot be allocated for the data read request, and data reading may fail.
  • the query platform may pre-build an index including a correspondence relationship between each storage server and an IP address, so as to be unavailable.
  • the attribute information of each storage server can still be obtained when the management node is available. Further, based on the attribute information of each storage server, a corresponding storage server is allocated for the data read request to ensure the normal operation of the cloud storage system.
  • the query platform may send a request for obtaining a correspondence between each storage server and an IP address to each management node, and receive a correspondence between each storage server and an IP address sent by each management node.
  • Step 602 Save the correspondence between each storage server and an IP address in an index.
  • the query platform may store the correspondence between each storage server and the IP address in the index, so as to receive the data read request and fail to obtain the management node, according to the index Corresponding relationship between each storage server and an IP address saved in the storage, sending a request for obtaining attribute information of the data to each storage server, and determining a target storage server corresponding to the data read request according to the response message returned by each storage server And based on the target storage server, read the corresponding data.
  • the query platform is ensured in order to ensure the correctness of the correspondence between each storage server and the IP address saved in the index.
  • the above steps 601 and 602 may be performed according to the set time interval (for example, 2 hours, 12 hours, 24 hours, etc.) to implement updating of the correspondence between each storage server and the IP address saved in the index.
  • the method provided in this embodiment pre-builds an index including a correspondence between each storage server and an IP address, and when receiving a data read request and failing to acquire an available management node, determining, according to the index, a data read request corresponding to at least A target storage server, based on the target storage server, reads the corresponding data, ensuring that the cloud storage system can still operate normally when all management nodes fail or the management node cluster network is abnormal.
  • FIG. 7 is a data storage device based on the cloud storage system according to the embodiment, which may include:
  • the receiving module 710 is configured to receive a data write request.
  • the processing module 720 is configured to identify the domain identifier information included in the data write request, and query the pre-stored index according to the domain identifier information, where the index includes the resource, when the available management node cannot be obtained. Correspondence between the pool and the domain identification information;
  • the identification module 730 is configured to identify a target resource pool that matches the domain identifier information, where each resource pool includes at least one storage server;
  • a writing module 740 configured to determine, from the at least one storage server included in the target resource pool, one of the storage servers as a target storage server, and write data included in the data write request to the target storage server.
  • the apparatus provided in this embodiment can identify the index included in the data write request when receiving the data write request and failing to acquire the available management node by pre-storing the index including the correspondence between the resource pool and the domain identifier information. And identifying the target resource pool that matches the domain identifier information, and finally from the target resource pool, according to the domain identifier information, and the corresponding relationship between the resource pool and the domain identifier information saved in the index,
  • the at least one storage server included, determining one of the storage servers as the target storage server to write data to the target storage server can ensure that the cloud storage system can still be normal when all the management nodes are faulty or the management node cluster network is abnormal. Writing data will not cause data loss.
  • writing module 740 is specifically configured to:
  • the failure to obtain an available management node includes: each management node fails or the management node cluster network abnormality.
  • the device further includes:
  • An execution module (not shown) is configured to send, to each management node, a request for obtaining a correspondence between each resource pool and domain identifier information, and receive each resource pool and domain identifier sent by each management node. Correspondence of information;
  • a saving module (not shown) is configured to save the correspondence between each resource pool and domain identification information in an index.
  • the device further includes:
  • an update module (not shown), configured to update, according to the set time interval, a correspondence between each resource pool and the domain identifier information saved in the index.
  • At least one target resource pool corresponding to the data write request may be determined according to the index. And determining the target storage server based on the target resource pool to write the corresponding data, ensuring that the cloud storage system can still operate normally when all the management nodes are faulty or the management node cluster network is abnormal.
  • FIG. 8 is a data storage device based on the cloud storage system, which is applied to the query platform, and may include:
  • the receiving module 810 is configured to receive a data read request.
  • the querying module 820 is configured to query a pre-saved index when the available management node cannot be obtained, where the index includes a correspondence between each storage server and its IP address;
  • the processing module 830 is configured to store the storage address according to the IP address of each storage server. Sending, by the server, a request for acquiring attribute information of the data, and receiving corresponding attribute information of the data saved by the storage server that is saved by the storage server;
  • the reading module 840 is configured to determine at least one target storage according to the received attribute information of the data that is saved by each storage server and the attribute information of the data to be read included in the data read request. Server and read the corresponding data from each target storage server.
  • the device provided in this embodiment can query the pre-stored index and obtain each index when the data read request is received and the available management node cannot be obtained by pre-storing the index including the correspondence relationship between each storage server and its IP address.
  • the storage server's IP address and then, according to the IP address of each storage server, sends a request for obtaining the attribute information of the data saved by itself to each storage server, and receives the self-saved data sent by each corresponding storage server.
  • Attribute information and finally determining at least one target storage server from the target storage server according to the attribute information of the data stored by each storage server and the attribute information of the data to be read included in the data read request.
  • the corresponding data is read to ensure that the cloud storage system can still provide data reading services when all management nodes are faulty or the management node cluster network is abnormal.
  • the index further includes status information of whether each storage server is online, and the processing module 830 includes:
  • Identifying a sub-module (not shown) for identifying a storage server in an online state according to status information of each storage server;
  • the processing submodule (not shown) is configured to send, according to the IP address of the storage server in an online state, a request for acquiring attribute information of the data to each corresponding storage server in an online state.
  • the failure to obtain an available management node includes: each management node fails or the management node cluster network abnormality.
  • the device further includes:
  • An execution module (not shown) is configured to send, to each management node, a request for obtaining a correspondence between each storage server and an IP address, and receive each storage server and an IP address sent by each management node.
  • a saving module (not shown) is configured to save the correspondence between each storage server and an IP address in an index.
  • the device further includes:
  • An update module (not shown) is configured to update a correspondence between each storage server and an IP address saved in the index according to a set time interval.
  • an index including a correspondence relationship between each storage server and an IP address
  • determining at least one target storage server corresponding to the data read request according to the index and Based on the target storage server, the corresponding data is read to ensure that the cloud storage system can still operate normally when all management nodes are faulty or the management node cluster network is abnormal.
  • the embodiment of the present application further provides a cloud storage system, where the cloud storage system may include: an apparatus disposed in an access node provided by the embodiment shown in FIG. 7, and at least one storage server.
  • the cloud storage system provided in this embodiment can ensure that the cloud storage system can still operate normally when all management nodes are faulty or the management node cluster network is abnormal, and no additional cost is required.
  • the present application further provides a storage medium, wherein the storage medium is used to store executable program code, and the executable program code is configured to execute a cloud storage system based on the present application at runtime.
  • Data writing method The data storage method of the cloud storage system is as follows: the cloud storage system at least includes: an access node, at least one management node, and at least one storage server, where the method includes:
  • the access node receives a data write request
  • the domain identifier information included in the data write request is identified, and the pre-stored index is queried according to the domain identifier information, where the index includes the resource pool and the domain identifier information.
  • each resource pool includes One less storage server
  • the domain included in the data write request may be identified. Identifying information, and then identifying, according to the domain identifier information, a correspondence between the resource pool and the domain identifier information saved in the index, a target resource pool that matches the domain identifier information, and finally including from the target resource pool.
  • the present application further provides a storage medium, wherein the storage medium is used to store executable program code, and the executable program code is configured to execute a cloud storage system based on the present application at runtime.
  • Data reading method The data storage method based on the cloud storage system of the present application, the cloud storage system includes at least: at least one management node and at least one storage server, which is applied to the query platform, and the method includes:
  • the pre-saved index is queried, wherein the index includes a correspondence between each storage server and its IP address;
  • the pre-stored index may be queried to obtain each Store the server's IP address and then according to each storage server IP address, sending a request for obtaining attribute information of data saved by itself to each storage server, and receiving attribute information of the data saved by each corresponding storage server, and finally sending according to each storage server received
  • the attribute information of the data saved by itself and the attribute information of the data to be read included in the data read request, and the at least one target storage server is determined to read the corresponding data from the target storage server, and can be guaranteed at all management nodes.
  • the cloud storage system can still provide data reading services when a fault occurs or the management node cluster network is abnormal.
  • the present application further provides an application, where the application is used to execute a cloud storage system-based data writing method described in the present application at runtime.
  • the data storage method of the cloud storage system is as follows: the cloud storage system at least includes: an access node, at least one management node, and at least one storage server, where the method includes:
  • the access node receives a data write request
  • the domain identifier information included in the data write request is identified, and the pre-stored index is queried according to the domain identifier information, where the index includes the resource pool and the domain identifier information.
  • each resource pool includes at least one storage server
  • the domain included in the data write request may be identified. Identifying information, and then identifying, according to the domain identifier information, a correspondence between the resource pool and the domain identifier information saved in the index, a target resource pool that matches the domain identifier information, and finally including from the target resource pool.
  • the application also provides an application, wherein the application is used to run A cloud storage system-based data reading method described in the present application is executed.
  • the data storage method based on the cloud storage system of the present application, the cloud storage system includes at least: at least one management node and at least one storage server, which is applied to the query platform, and the method includes:
  • the pre-saved index is queried, wherein the index includes a correspondence between each storage server and its IP address;
  • the pre-stored index may be queried to obtain each The IP address of the storage server is stored, and then, according to the IP address of each storage server, a request for obtaining attribute information of the data saved by itself is sent to each storage server, and the data saved by the corresponding storage server is received. Attribute information, and finally determining at least one target storage server from the target storage server according to the attribute information of the data stored by each storage server and the attribute information of the data to be read included in the data read request.
  • the cloud storage system can still provide the data reading service normally when all the management nodes are faulty or the management node cluster network is abnormal.
  • the application further provides an access node device, including:
  • processor a memory, a communication interface, and a bus
  • the processor, the memory, and the communication interface are connected by the bus and complete communication with each other;
  • the memory stores executable program code
  • the processor runs a program corresponding to the executable program code by reading executable program code stored in the memory, for executing a cloud storage system-based data writing method described in the present application.
  • the data storage method of the cloud storage system is as follows: the cloud storage system at least includes: an access node, at least one management node, and at least one storage server, where the method includes:
  • the access node receives a data write request
  • the domain identifier information included in the data write request is identified, and the pre-stored index is queried according to the domain identifier information, where the index includes the resource pool and the domain identifier information.
  • each resource pool includes at least one storage server
  • the domain included in the data write request may be identified. Identifying information, and then identifying, according to the domain identifier information, a correspondence between the resource pool and the domain identifier information saved in the index, a target resource pool that matches the domain identifier information, and finally including from the target resource pool.
  • an electronic device including:
  • processor a memory, a communication interface, and a bus
  • the processor, the memory, and the communication interface are connected by the bus and complete communication with each other;
  • the memory stores executable program code
  • the processor operates and reads by reading executable program code stored in the memory
  • the program corresponding to the program code is used to execute a cloud storage system-based data reading method described in the present application.
  • the data storage method based on the cloud storage system of the present application, the cloud storage system includes at least: at least one management node and at least one storage server, which is applied to the query platform, and the method includes:
  • the pre-saved index is queried, wherein the index includes a correspondence between each storage server and its IP address;
  • the pre-stored index may be queried to obtain each The IP address of the storage server is stored, and then, according to the IP address of each storage server, a request for obtaining attribute information of the data saved by itself is sent to each storage server, and the data saved by the corresponding storage server is received. Attribute information, and finally determining at least one target storage server from the target storage server according to the attribute information of the data stored by each storage server and the attribute information of the data to be read included in the data read request.
  • the cloud storage system can still provide the data reading service normally when all the management nodes are faulty or the management node cluster network is abnormal.
  • the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Databases & Information Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computing Systems (AREA)
  • Quality & Reliability (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Signal Processing (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种数据写入、读取方法及装置、云存储系统，所述数据写入方法包括：接入节点接收数据写入请求（301）；当无法获取到可用管理节点时，识别所述数据写入请求中包含的域标识信息，并根据所述域标识信息，查询预先保存的索引，其中，所述索引包含资源池与域标识信息的对应关系（302）；识别与所述域标识信息匹配的目标资源池，其中，每个资源池中包含至少一个存储服务器（303）；从所述目标资源池中包含的至少一个存储服务器中，确定其中一个存储服务器为目标存储服务器，并将待写入数据写入所述目标存储服务器（304）。能够保证在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常运行，且不需要增加额外的成本。

Description

数据写入、读取方法及装置、云存储系统
本申请要求于2016年2月17日提交中国专利局、申请号为201610088973.4发明名称为“数据写入、读取方法及装置、云存储系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及数据处理技术领域，特别是涉及一种数据写入、读取方法及装置、云存储系统。
背景技术
云存储是在云计算(cloud computing)概念上延伸和发展出来的一个新的概念，是一种新兴的网络存储技术，是指通过集群应用、网络技术或分布式文件系统等功能，将网络中各种不同类型的大量存储设备通过应用软件集合起来协同工作，共同对外提供数据存储和业务访问功能的一个系统。
云存储系统是一个主从架构的分布式系统，主节点也就是管理节点在整个系统的正常运行中起到至关重要的作用。管理节点一旦出现故障，整个系统的所有业务都不能正常进行。例如，当云存储系统中的接入节点接收到一个数据写入请求时，接入节点需要将该数据写入请求发送给管理节点，以使管理节点为该数据写入请求分配对应的存储服务器。
然而，当管理节点出现故障时，管理节点将不会对接入节点发送的数据处理请求作出响应，因此，该数据处理将会失败。
以图1所示的视频云存储系统的架构为例对无法获取到可用管理节点时相应处理流程进行说明。当无法获取到可用视频云存储管理节点(CVM，Cloud Video Manage)130时，视频云存储管理节点130将不会对视频云存储接入节点(CVA，Cloud Video Access)120发送的数据处理请求作出响应，因此，视频云存储接入节点120将不会收到视频云存储管理节点130为该数据处理请求分配的视频云存储存储服务器(CVS，Cloud Video Storage)140，该数据处理将会失败。对于与上述视频云存储系统相似的其他云存储系统也同样存在 上述问题。
因此，云存储系统在实际运行中，遇到管理节点出现故障导致云存储系统无法提供服务时，可能会造成数据的丢失或者数据无法读取，给用户造成重大损失。
现有技术中，为了保证云存储系统的稳定，主要方法是增加管理节点的数量，并通过HA(High Available，高可用性集群)或者集群技术保证部分管理节点故障时由正常的管理节点接管继续提供服务，或者添加备用管理节点来代替故障管理节点提供服务。但是，增加管理节点或者添加备用管理节点会增加项目成本，而且，增加管理节点或者添加备用管理节点也只能保证部分管理节点故障时云存储系统正常运行，而无法保证所有的管理节点都出现故障或管理节点集群网络异常的情况下云存储系统正常运行。
发明内容
本申请实施例的目的在于提供一种数据写入、读取方法及装置、云存储系统，以在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常运行，且不需要增加额外的成本。具体技术方案如下：
第一方面，本申请实施例提供了一种基于云存储系统的数据写入方法，所述云存储系统至少包括：接入节点、至少一个管理节点以及至少一个存储服务器，所述方法包括：
所述接入节点接收数据写入请求；
当无法获取到可用管理节点时，识别所述数据写入请求中包含的域标识信息，并根据所述域标识信息，查询预先保存的索引，其中，所述索引包含资源池与域标识信息的对应关系；
识别与所述域标识信息匹配的目标资源池，其中，每个资源池中包含至少一个存储服务器；
从所述目标资源池中包含的至少一个存储服务器中，确定其中一个存储服务器为目标存储服务器，并将待写入数据写入所述目标存储服务器。
进一步地，所述确定其中一个存储服务器为目标存储服务器，包括：
获取每个存储服务器的当前负载量；
识别当前负载量最小的存储服务器，将其作为目标存储服务器。
进一步地，所述无法获取到可用管理节点包括：每个管理节点均出现故障或管理节点集群网络异常。
进一步地，构建所述索引的过程包括：
向每个管理节点发送获取每个资源池与域标识信息的对应关系的请求，并接收所述每个管理节点发送的每个资源池与域标识信息的对应关系；
在索引中保存所述每个资源池与域标识信息的对应关系。
进一步地，所述方法还包括：
按照设定的时间间隔，更新所述索引中保存的每个资源池与域标识信息的对应关系。
第二方面，本申请实施例提供了一种基于云存储系统的数据读取方法，所述云存储系统至少包括：至少一个管理节点以及至少一个存储服务器，应用于查询平台，所述方法包括：
接收数据读取请求；
当无法获取到可用管理节点时，查询预先保存的索引，其中，所述索引包含每个存储服务器及其IP地址的对应关系；
根据所述每个存储服务器的IP地址，向所述每个存储服务器发送获取数据的属性信息的请求，并接收对应的所述每个存储服务器返回的其自身保存的数据的属性信息；
根据接收的所述每个存储服务器发送的其自身保存的数据的属性信息，以及所述数据读取请求中包含的待读取数据的属性信息，确定至少一个目标存储服务器，并从每个目标存储服务器中读取相应的数据。
进一步地，所述索引中还包括每个存储服务器是否在线的状态信息，所述根据所述每个存储服务器的IP地址，向所述每个存储服务器发送获取数据的属性信息的请求，包括：
根据每个存储服务器的状态信息，识别处于在线状态的存储服务器；
根据处于在线状态的存储服务器的IP地址，向对应的每个处于在线状态的存储服务器发送获取数据的属性信息的请求。
进一步地，所述无法获取到可用管理节点包括：每个管理节点均出现故障或管理节点集群网络异常。
进一步地，构建所述索引的过程包括：
向每个管理节点发送获取每个存储服务器和IP地址的对应关系的请求，并接收所述每个管理节点发送的每个存储服务器和IP地址的对应关系；
在索引中保存所述每个存储服务器和IP地址的对应关系。
进一步地，所述方法还包括：
按照设定的时间间隔，更新所述索引中保存的每个存储服务器和IP地址的对应关系。
第三方面，本申请实施例提供了一种基于云存储系统的数据写入装置，所述云存储系统至少包括：接入节点、至少一个管理节点以及至少一个存储服务器，应用于所述接入节点，所述装置包括：
接收模块，用于接收数据写入请求；
处理模块，用于当无法获取到可用管理节点时，识别所述数据写入请求中包含的域标识信息，并根据所述域标识信息，查询预先保存的索引，其中，所述索引包含资源池与域标识信息的对应关系；
识别模块，用于识别与所述域标识信息匹配的目标资源池，其中，每个资源池中包含至少一个存储服务器；
写入模块，用于从所述目标资源池中包含的至少一个存储服务器中，确定其中一个存储服务器为目标存储服务器，并将待写入数据写入所述目标存储服务器。
进一步地，所述写入模块，具体用于：
获取每个存储服务器的当前负载量；
识别当前负载量最小的存储服务器，将其作为目标存储服务器。
进一步地，所述无法获取到可用管理节点包括：每个管理节点均出现故障或管理节点集群网络异常。
进一步地，所述装置还包括：
执行模块，用于向每个管理节点发送获取每个资源池与域标识信息的对应关系的请求，并接收所述每个管理节点发送的每个资源池与域标识信息的对应关系；
保存模块，用于在索引中保存所述每个资源池与域标识信息的对应关系。
进一步地，所述装置还包括：
更新模块，用于按照设定的时间间隔，更新所述索引中保存的每个资源池与域标识信息的对应关系。
第四方面，本申请实施例提供了一种基于云存储系统的数据读取装置，所述云存储系统至少包括：至少一个管理节点以及至少一个存储服务器，应用于查询平台，所述装置包括：
接收模块，用于接收数据读取请求；
查询模块，用于当无法获取到可用管理节点时，查询预先保存的索引，其中，所述索引包含每个存储服务器及其IP地址的对应关系；
处理模块，用于根据所述每个存储服务器的IP地址，向所述每个存储服务器发送获取数据的属性信息的请求，并接收对应的所述每个存储服务器返回的其自身保存的数据的属性信息；
读取模块，用于根据接收的所述每个存储服务器发送的其自身保存的数据的属性信息，以及所述数据读取请求中包含的待读取数据的属性信息，确定至少一个目标存储服务器，并从每个目标存储服务器中读取相应的数据。
进一步地，所述索引中还包括每个存储服务器是否在线的状态信息，所述处理模块，包括：
识别子模块，用于根据每个存储服务器的状态信息，识别处于在线状态 的存储服务器；
处理子模块，用于根据处于在线状态的存储服务器的IP地址，向对应的每个处于在线状态的存储服务器发送获取数据的属性信息的请求。
进一步地，所述无法获取到可用管理节点包括：每个管理节点均出现故障或管理节点集群网络异常。
进一步地，所述装置还包括：
执行模块，用于向每个管理节点发送获取每个存储服务器和IP地址的对应关系的请求，并接收所述每个管理节点发送的每个存储服务器和IP地址的对应关系；
保存模块，用于在索引中保存所述每个存储服务器和IP地址的对应关系。
进一步地，所述装置还包括：
更新模块，用于按照设定的时间间隔，更新所述索引中保存的每个存储服务器和IP地址的对应关系。
第五方面，本申请实施例提供了一种云存储系统，所述云存储系统包括如上述第三方面所述的设置在接入节点中的装置，以及至少一个存储服务器。
第六方面，本申请提供了一种存储介质，其中，该存储介质用于存储可执行程序代码，所述可执行程序代码用于在运行时执行本申请所述的一种基于云存储系统的数据写入方法。
第七方面，本申请提供了一种存储介质，其中，该存储介质用于存储可执行程序代码，所述可执行程序代码用于在运行时执行本申请所述的一种基于云存储系统的数据读取方法。
第八方面，本申请提供了一种应用程序，其中，该应用程序用于在运行时执行本申请所述的一种基于云存储系统的数据写入方法。
第九方面，本申请提供了一种应用程序，其中，该应用程序用于在运行时执行本申请所述的一种基于云存储系统的数据读取方法。
第十方面，本申请提供了一种接入节点设备，包括：
处理器、存储器、通信接口和总线；
所述处理器、所述存储器和所述通信接口通过所述总线连接并完成相互间的通信；
所述存储器存储可执行程序代码；
所述处理器通过读取所述存储器中存储的可执行程序代码来运行与所述可执行程序代码对应的程序，以用于执行本申请所述的一种基于云存储系统的数据写入方法。
第十一方面，本申请提供了一种电子设备，包括：
处理器、存储器、通信接口和总线；
所述处理器、所述存储器和所述通信接口通过所述总线连接并完成相互间的通信；
所述存储器存储可执行程序代码；
所述处理器通过读取所述存储器中存储的可执行程序代码来运行与所述可执行程序代码对应的程序，以用于执行本申请所述的一种基于云存储系统的数据读取方法。
本申请实施例提供了一种数据写入、读取方法及装置、云存储系统，所述数据写入方法包括：所述接入节点接收数据写入请求；当无法获取到可用管理节点时，识别所述数据写入请求中包含的域标识信息，并根据所述域标识信息，查询预先保存的索引，其中，所述索引包含资源池与域标识信息的对应关系；识别与所述域标识信息匹配的目标资源池，其中，每个资源池中包含至少一个存储服务器；从所述目标资源池中包含的至少一个存储服务器中，确定其中一个存储服务器为目标存储服务器，并将待写入数据写入所述目标存储服务器。本申请实施例能够保证在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常运行，且不需要增加额外的成本。
附图说明
为了更清楚地说明本申请实施例和现有技术的技术方案，下面对实施例和现有技术中所需要使用的附图作简单地介绍，显而易见地，下面描述中的 附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为无法获取到管理节点时云存储系统无法提供服务的示意图；
图2为正常情况下云存储系统处理数据请求的示意性流程；
图3为本实施例提供的基于云存储系统的数据写入方法；
图4为本实施例提供的一种构建索引的方法；
图5为本实施例提供的基于云存储系统的数据读取方法；
图6为本实施例提供的另一种构建索引的方法；
图7为本实施例提供的基于云存储系统的数据写入装置；
图8为本实施例提供的基于云存储系统的数据读取装置。
具体实施方式
为了使本领域技术人员更好地理解本申请实施例中的技术方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。下面将参考附图并结合实施例来详细说明本申请。
图3为本实施例提供的基于云存储系统的数据写入方法，可以包括如下步骤：
步骤301，接入节点接收数据写入请求。
一般情况下，云存储系统至少可以包括：接入节点、至少一个管理节点以及至少一个存储服务器。接入节点可以用来接收数据写入请求，并将数据写入请求发送给管理节点，以使得管理节点为该数据写入请求分配对应的存储服务器。接入节点可以进一步地根据管理节点为该数据写入请求分配的存 储服务器，对该数据写入请求中相应的数据进行处理。
在本实施例中，云存储系统进行数据写入时，可以首先由接入节点接收数据写入请求，并对相应的数据进行写入。
以图2所示的视频云存储系统的架构为例进行说明，对正常情况下云存储系统处理数据请求的过程作简单介绍。如图2所示，电子设备110可以向视频云存储接入节点(CVA)120发送数据处理请求，当接入节点120接收到一个数据处理请求时，接入节点120需要将该数据处理请求发送给管理节点(CVM)130，以使管理节点130为该数据处理请求分配对应的存储服务器(CVS)140。管理节点130为该数据处理请求分配对应的存储服务器140后，可以将分配结果反馈给接入节点120，接入节点120可以基于分配的存储服务器140，进行相应的数据处理，例如数据读写操作。
步骤302，当无法获取到可用管理节点时，识别所述数据写入请求中包含的域标识信息，并根据所述域标识信息，查询预先保存的索引，其中，所述索引包含资源池与域标识信息的对应关系。
在云存储系统中，通常可以包含多个存储服务器，为了便于对多个存储服务器进行管理，可以建立资源池，其中，每个资源池可以包括至少一个存储服务器。每个资源池可以有对应的域标识信息。
当接入节点接收到数据写入请求时，数据写入请求中通常会包含域标识信息，即待写入数据将要写入的存储服务器所在的资源池的域标识信息，以使得接入节点可以基于该域标识信息，为数据写入请求分配对应的存储服务器。
正常情况下，接入节点接收到数据写入请求时，可以将数据写入请求发送给管理节点，以使得管理节点为该数据写入请求分配对应的存储服务器。然而，当无法获取到可用管理节点时，接入节点将不能通过管理节点获得为该数据写入请求分配的存储服务器。
因此，在本实施例中，可以预先保存包含资源池与域标识信息的对应关系的索引。当接收到数据写入请求且无法获取到可用管理节点时，接入节点可以根据识别数据写入请求中的域标识信息，并根据该域标识信息，查询预 先保存的包含资源池与域标识信息的对应关系的索引，以识别与该域标识信息匹配的目标资源池。
步骤303，识别与所述域标识信息匹配的目标资源池，其中，每个资源池中包含至少一个存储服务器。
在本申请实施例中，接入节点识别出数据写入请求中包含的域标识信息后，可以查找索引中保存的每个资源池与域标识信息的对应关系，以识别出与数据写入请求中包含的域标识信息对应的资源池，并将识别到的资源池作为目标资源池，以基于目标资源池，对该数据写入请求进行处理。
步骤304，从所述目标资源池中包含的至少一个存储服务器中，确定其中一个存储服务器为目标存储服务器，并将待写入数据写入所述目标存储服务器。
当识别出与数据写入请求中包含的域标识信息对应的目标资源池后，接入节点可以进一步地从目标资源池中包含的至少一个存储服务器中，确定其中一个存储服务器为目标存储服务器，以将待写入数据写入该目标存储服务器。
例如，可以在目标资源池中包含的至少一个存储服务器中，选择其中任一个存储服务器作为目标存储服务器。
可选地或附加地，为了保证每个存储服务器的负载均衡，在选择目标资源池中的目标存储服务器时，可以首先获取每个存储服务器的当前负载量，然后在各存储服务器中，识别当前负载量最小的存储服务器，将其作为目标存储服务器。
例如，当云存储系统为视频云存储系统时，在该视频云存储系统中可以存储大量的视频数据。具体地，视频采集系统可以将其采集的实时视频数据写入视频云存储系统中。
为了便于对视频云存储系统中的视频数据进行管理，同一视频采集系统采集的视频数据保存到视频云存储系统的固定的存储服务器中，该固定的存储服务器构成一个资源池，该资源池对应一个域标识信息。并配置读写操作时，访问该固定的存储服务器的视频处理设备的地址信息。
当接入节点接收到视频写入请求且无法获取到可用管理节点时，可以首先识别视频写入请求中包含的域标识信息，然后根据该域标识信息，以及索引中保存的资源池与域标识信息的对应关系，识别与该域标识信息匹配的目标资源池，最后，可以从目标资源池中包含的至少一个存储服务器的信息中，确定其中一个存储服务器为目标存储服务器，并将视频数据写入该目标存储服务器。
优选地，为了保证每个存储服务器的负载均衡，在选择用于存储视频数据的目标存储服务器时，可以首先获取每个存储服务器的当前负载量，然后识别当前负载量最小的存储服务器，将其作为目标存储服务器。
在一种实现方式中，在选择目标存储服务器时，接入节点可以向目标资源池中的每个存储服务器发出获取其当前负载量的请求，并根据各存储服务器返回的响应消息获取每个存储服务器的当前负载量。然后，可以将当前负载量最小的存储服务器作为目标存储服务器。
在另一种实现方式中，接入节点在构建索引的过程中，可以在该索引中保存每个存储服务器对应的负载量。在选择目标存储服务器时，接入节点可以查询索引中保存的目标资源池中的每个存储服务器的负载量，并选择负载量最小的存储服务器作为目标存储服务器。并且，可选地，接入节点将待写入数据写入目标存储服务器后，可以更新索引中保存的该目标存储服务器对应的负载量。
本申请实施例提供的方法，通过预先保存包含资源池与域标识信息的对应关系的索引，在接收到数据写入请求并且无法获取到可用管理节点时，可以识别所述数据写入请求中包含的域标识信息，然后根据所述域标识信息，以及所述索引中保存的资源池与域标识信息的对应关系，识别与所述域标识信息匹配的目标资源池，最后从所述目标资源池中包含的至少一个存储服务器中，确定其中一个存储服务器为目标存储服务器，以将数据写入该目标存储服务器，能够保证在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常写入数据，不会造成数据丢失。
基于本申请的上述实施例，图4为本实施例提供的一种构建索引的方法，可以包括如下步骤：
步骤401，向每个管理节点发送获取每个资源池与域标识信息的对应关系的请求，并接收所述每个管理节点发送的每个资源池与域标识信息的对应关系。
在云存储系统中，管理节点在整个系统的正常运行中起到至关重要的作用。当接收到数据写入请求时，管理节点需要基于各存储服务器的属性信息，为数据写入请求分配存储服务器，以写入相应的数据。当无法获取到管理节点时，由于无法获得各存储服务器的属性信息，因此，无法为数据写入请求分配对应的存储服务器，数据写入可能会失败。
因此，在本实施例中，为了保证在无法获取到可用管理节点时云存储系统仍能提供正常服务，可以预先构建包含每个资源池与域标识信息的对应关系的索引，以在无法获取到可用管理节点时仍能获得各存储服务器的属性信息，进一步地，基于各存储服务器的属性信息，为数据写入请求分配相应的存储服务器，确保云存储系统正常运行。
具体地，接入节点可以向每个管理节点发送获取每个资源池与域标识信息的对应关系的请求，并接收每个管理节点发送的每个资源池与域标识信息的对应关系。
步骤402，在索引中保存所述每个资源池与域标识信息的对应关系。
获取到每个资源池与域标识信息的对应关系后，可以在索引中保存每个资源池与域标识信息的对应关系，以在接收到数据写入请求且无法获取到管理节点时，根据索引中保存的每个资源池与域标识信息的对应关系，确定与数据写入请求对应的目标资源池，并基于目标资源池，对数据写入请求进行处理。
可选地，由于每个资源池与域标识信息的对应关系可能会变化，因此，为了保证索引中保存的各资源池与域标识信息的对应关系的正确性，可以按照设定的时间间隔(如2小时、12小时、24小时等)，进行上述步骤401和步骤402，实现对所述索引中保存的每个资源池与域标识信息的对应关系的更新。
本实施例提供的方法，预先构建包含每个资源池与域标识信息的对应关系的索引，在接收到数据写入请求并且无法获取到可用管理节点时，可以根据该索引确定数据写入请求对应的至少一个目标资源池，并基于目标资源池确定目标存储服务器，以对相应的数据进行写入，保证了在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常运行。
图5为本实施例提供的基于云存储系统的数据读取方法，应用于查询平台，可以包括如下步骤：
步骤501，接收数据读取请求。
在本实施例中，查询平台进行数据读取时，可以首先接收数据读取请求，并对相应的数据进行读取。
步骤502，当无法获取到可用管理节点时，查询预先保存的索引，其中，所述索引包含每个存储服务器及其IP地址的对应关系。
正常情况下，查询平台接收到数据读取请求时，可以将数据读取请求发送给管理节点，以使得管理节点为该数据读取请求分配对应的存储服务器。然而，当无法获取到可用管理节点时，查询平台将不能通过管理节点获得为该数据读取请求分配的存储服务器。
因此，在本实施例中，可以预先保存包含每个存储服务器及其IP地址的对应关系的索引。当接收到数据读取请求且无法获取到可用管理节点时，查询平台可以查询该索引，以获得各存储服务器的IP地址。
步骤503，根据所述每个存储服务器的IP地址，向所述每个存储服务器发送获取数据的属性信息的请求，并接收对应的所述每个存储服务器返回的其自身保存的数据的属性信息。
当查询平台获得每个存储服务器的IP地址后，可以根据每个存储服务器的IP地址，向每个存储服务器发送获取数据的属性信息的请求，并接收对应的每个存储服务器返回的其自身保存的数据的属性信息。
上述数据的属性信息例如可以包括每个存储服务器中保存的数据的关键 词，如当存储服务器中保存的数据为道路实时监控视频时，上述属性信息可以为路段信息、时间信息等。
查询平台可以接收对应的每个存储服务器发送的自身保存的数据的属性信息，以基于该属性信息，确定待读取数据所在的存储服务器，并从相应的存储服务器中读取相应的数据。
可以理解，在云存储系统的多个存储服务器中，针对每个存储服务器，其都可能会出现故障，因此该存储服务器可能出现离线状态。这种情况下，当查询平台向该存储服务器发送获取其保存的数据的属性信息时，该存储服务器可能不会返回响应消息。因此，当存在处于离线状态的存储服务器时，查询平台向每个存储服务器发送获取其保存的数据的属性信息可能会造成资源浪费。
因此，可选地，在本申请实施例中，可以在查询平台的索引中保存每个存储服务器是否在线的状态信息。这种情况下，查询平台可以首先根据每个存储服务器的状态信息，识别处于在线状态的存储服务器，然后根据处于在线状态的存储服务器的IP地址，向对应的每个处于在线状态的存储服务器发送获取数据的属性信息的请求。
通过在索引中保存每个存储服务器的状态信息，并根据各存储服务器的状态信息，只向处于在线状态的存储服务器发送获取数据的属性信息的请求，能够节省网络资源。
步骤504，根据接收的所述每个存储服务器发送的其自身保存的数据的属性信息，以及所述数据读取请求中包含的待读取数据的属性信息，确定至少一个目标存储服务器，并从每个目标存储服务器中读取相应的数据。
查询平台接收到每个存储服务器发送的其自身保存的数据的属性信息后，可以根据每个存储服务器自身保存的数据的属性信息，以及数据读取请求中包含的待读取数据的属性信息，确定至少一个目标存储服务器。
查询平台接收到的数据读取请求中通常会包含待读取数据的属性信息，因此，查询平台接收到每个存储服务器自身保存的数据的属性信息后，可以识别与数据读取请求中包含的待读取数据的属性信息对应的存储服务器，并 将识别到的存储服务器作为目标存储服务器，以从目标存储服务器中读取相应的数据。
例如，当云存储系统为视频云存储系统时，在该视频云存储系统中可以存储大量的视频数据，用户可以从该视频云存储系统中读取其想要的视频数据。具体地，当视频云存储系统中存储有交通监控视频数据时，用户可以读取对应时间段或对应路段的相应的视频数据。
当查询平台接收到视频处理设备通过平台发送的数据读取请求时，该数据读取请求中携带待读取的视频数据的属性信息，该属性例如可以为时间段信息和/或路段信息等。查询平台中设置有与平台交互的接口，可以通过该接口接收平台发送的数据读取请求。
查询平台接收到数据读取请求且无法获取到可用管理节点时，查询预先保存的索引，其中，索引包含每个存储服务器及其IP地址的对应关系，例如读取9:00-10:00的视频数据。然后，查询平台可以根据预先构建的索引，确定该IP地址对应的每个存储服务器，并向每个存储服务器发送获取视频数据的属性信息的请求，并接收对应的每个存储服务器返回的其自身保存的该对应时间的视频片段。最后，查询平台可以根据接收的每个存储服务器返回的其自身保存的视频片段，以及视频数据读取请求中包含的待读取视频数据的属性信息，确定至少一个目标存储服务器。并将该目标存储服务器的信息发送给查询平台，查询平台访问对应的目标存储服务器，从目标存储服务器中读取对应时间段的视频数据。
本实施例提供的方法，通过预先保存包含每个存储服务器及其IP地址的对应关系的索引，在接收到数据读取请求并且无法获取到可用管理节点时，可以查询预先保存的索引，获得每个存储服务器的IP地址，然后根据每个存储服务器的IP地址，向每个存储服务器发送获取自身保存的数据的属性信息的请求，并接收所述对应的每个存储服务器发送的自身保存的数据的属性信息，最后根据接收的每个存储服务器发送的自身保存的数据的属性信息，以及数据读取请求中包含的待读取数据的属性信息，确定至少一个目标存储服务器，以从目标存储服务器中读取相应的数据，能够保证在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常提供数据读取服务。
基于本申请的上述实施例，图6为本实施例提供的另一种构建索引的方法，可以包括如下步骤：
步骤601，向每个管理节点发送获取每个存储服务器和IP地址的对应关系的请求，并接收所述每个管理节点发送的每个存储服务器和IP地址的对应关系。
在云存储系统中，管理节点在整个系统的正常运行中起到至关重要的作用。当接收到数据读取请求时，管理节点需要基于各存储服务器的属性信息，为数据读取请求分配存储服务器，以进行相应的数据读取。当无法获取到管理节点时，由于无法获得各存储服务器的属性信息，因此，无法为数据读取请求分配对应的存储服务器，数据读取可能会失败。
因此，在本实施例中，为了保证在无法获取到可用管理节点时云存储系统仍能提供正常服务，查询平台可以预先构建包含每个存储服务器和IP地址的对应关系的索引，以在无法获取到可用管理节点时仍能获得各存储服务器的属性信息，进一步地，基于各存储服务器的属性信息，为数据读取请求分配相应的存储服务器，确保云存储系统正常运行。
具体地，查询平台可以向每个管理节点发送获取每个存储服务器和IP地址的对应关系的请求，并接收每个管理节点发送的每个存储服务器和IP地址的对应关系。
步骤602，在索引中保存所述每个存储服务器和IP地址的对应关系。
获取到每个存储服务器和IP地址的对应关系后，查询平台可以在索引中保存每个存储服务器和IP地址的对应关系，以在接收到数据读取请求且无法获取到管理节点时，根据索引中保存的每个存储服务器和IP地址的对应关系，向每个存储服务器发送获取数据的属性信息的请求，并根据每个存储服务器返回的响应消息，确定与数据读取请求对应的目标存储服务器，并基于目标存储服务器，读取相应的数据。
可选地，由于每个存储服务器和IP地址的对应关系可能会变化，因此，为了保证索引中保存的每个存储服务器和IP地址的对应关系的正确性，查询平台 可以按照设定的时间间隔(如2小时、12小时、24小时等)，进行上述步骤601和步骤602，实现对所述索引中保存的每个存储服务器和IP地址的对应关系的更新。
本实施例提供的方法，预先构建包含每个存储服务器和IP地址的对应关系的索引，在接收到数据读取请求并且无法获取到可用管理节点时，根据该索引确定数据读取请求对应的至少一个目标存储服务器，并基于目标存储服务器，读取相应的数据，保证了在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常运行。
图7为本实施例提供的基于云存储系统的数据写入装置，可以包括：
接收模块710，用于接收数据写入请求；
处理模块720，用于当无法获取到可用管理节点时，识别所述数据写入请求中包含的域标识信息，并根据所述域标识信息，查询预先保存的索引，其中，所述索引包含资源池与域标识信息的对应关系；
识别模块730，用于识别与所述域标识信息匹配的目标资源池，其中，每个资源池中包含至少一个存储服务器；
写入模块740，用于从所述目标资源池中包含的至少一个存储服务器中，确定其中一个存储服务器为目标存储服务器，并将所述数据写入请求中包含的数据写入所述目标存储服务器。
本实施例提供的装置，通过预先保存包含资源池与域标识信息的对应关系的索引，在接收到数据写入请求并且无法获取到可用管理节点时，可以识别所述数据写入请求中包含的域标识信息，然后根据所述域标识信息，以及所述索引中保存的资源池与域标识信息的对应关系，识别与所述域标识信息匹配的目标资源池，最后从所述目标资源池中包含的至少一个存储服务器中，确定其中一个存储服务器为目标存储服务器，以将数据写入该目标存储服务器，能够保证在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常写入数据，不会造成数据丢失。
进一步地，所述写入模块740，具体用于：
获取每个存储服务器的当前负载量；
识别当前负载量最小的存储服务器，将其作为目标存储服务器。
进一步地，所述无法获取到可用管理节点包括：每个管理节点均出现故障或管理节点集群网络异常。
进一步地，所述装置还包括：
执行模块(图中未示出)，用于向每个管理节点发送获取每个资源池与域标识信息的对应关系的请求，并接收所述每个管理节点发送的每个资源池与域标识信息的对应关系；
保存模块(图中未示出)，用于在索引中保存所述每个资源池与域标识信息的对应关系。
进一步地，所述装置还包括：
更新模块(图中未示出)，用于按照设定的时间间隔，更新所述索引中保存的每个资源池与域标识信息的对应关系。
通过预先构建包含每个资源池与域标识信息的对应关系的索引，在接收到数据写入请求并且无法获取到可用管理节点时，可以根据该索引确定数据写入请求对应的至少一个目标资源池，并基于目标资源池确定目标存储服务器，以对相应的数据进行写入，保证了在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常运行。
图8为本实施例提供的基于云存储系统的数据读取装置，应用于查询平台，可以包括：
接收模块810，用于接收数据读取请求；
查询模块820，用于当无法获取到可用管理节点时，查询预先保存的索引，其中，所述索引包含每个存储服务器及其IP地址的对应关系；
处理模块830，用于根据所述每个存储服务器的IP地址，向所述每个存储 服务器发送获取数据的属性信息的请求，并接收对应的所述每个存储服务器返回的其自身保存的数据的属性信息；
读取模块840，用于根据接收的所述每个存储服务器发送的其自身保存的数据的属性信息，以及所述数据读取请求中包含的待读取数据的属性信息，确定至少一个目标存储服务器，并从每个目标存储服务器中读取相应的数据。
本实施例提供的装置，通过预先保存包含每个存储服务器及其IP地址的对应关系的索引，在接收到数据读取请求并且无法获取到可用管理节点时，可以查询预先保存的索引，获得每个存储服务器的IP地址，然后根据每个存储服务器的IP地址，向每个存储服务器发送获取自身保存的数据的属性信息的请求，并接收所述对应的每个存储服务器发送的自身保存的数据的属性信息，最后根据接收的每个存储服务器发送的自身保存的数据的属性信息，以及数据读取请求中包含的待读取数据的属性信息，确定至少一个目标存储服务器，以从目标存储服务器中读取相应的数据，能够保证在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常提供数据读取服务。
进一步地，所述索引中还包括每个存储服务器是否在线的状态信息，所述处理模块830，包括：
识别子模块(图中未示出)，用于根据每个存储服务器的状态信息，识别处于在线状态的存储服务器；
处理子模块(图中未示出)，用于根据处于在线状态的存储服务器的IP地址，向对应的每个处于在线状态的存储服务器发送获取数据的属性信息的请求。
进一步地，所述无法获取到可用管理节点包括：每个管理节点均出现故障或管理节点集群网络异常。
进一步地，所述装置还包括：
执行模块(图中未示出)，用于向每个管理节点发送获取每个存储服务器和IP地址的对应关系的请求，并接收所述每个管理节点发送的每个存储服务器和IP地址的对应关系；
保存模块(图中未示出)，用于在索引中保存所述每个存储服务器和IP地址的对应关系。
进一步地，所述装置还包括：
更新模块(图中未示出)，用于按照设定的时间间隔，更新所述索引中保存的每个存储服务器和IP地址的对应关系。
通过预先构建包含每个存储服务器和IP地址的对应关系的索引，在接收到数据读取请求并且无法获取到可用管理节点时，根据该索引确定数据读取请求对应的至少一个目标存储服务器，并基于目标存储服务器，读取相应的数据，保证了在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常运行。
另一方面，本申请实施例还提供了一种云存储系统，所述云存储系统可以包括：如图7所示实施例提供的设置在接入节点中的装置，以及至少一个存储服务器。
本实施例提供的云存储系统，能够保证在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常运行，且不需要增加额外的成本。
相应的，本申请还提供了一种存储介质，其中，该存储介质用于存储可执行程序代码，所述可执行程序代码用于在运行时执行本申请所述的一种基于云存储系统的数据写入方法。其中，本申请所述的一种基于云存储系统的数据写入方法，所述云存储系统至少包括：接入节点、至少一个管理节点以及至少一个存储服务器，所述方法包括：
所述接入节点接收数据写入请求；
当无法获取到可用管理节点时，识别所述数据写入请求中包含的域标识信息，并根据所述域标识信息，查询预先保存的索引，其中，所述索引包含资源池与域标识信息的对应关系；
识别与所述域标识信息匹配的目标资源池，其中，每个资源池中包含至 少一个存储服务器；
从所述目标资源池中包含的至少一个存储服务器中，确定其中一个存储服务器为目标存储服务器，并将待写入数据写入所述目标存储服务器。
本申请实施例中，通过预先保存包含资源池与域标识信息的对应关系的索引，在接收到数据写入请求并且无法获取到可用管理节点时，可以识别所述数据写入请求中包含的域标识信息，然后根据所述域标识信息，以及所述索引中保存的资源池与域标识信息的对应关系，识别与所述域标识信息匹配的目标资源池，最后从所述目标资源池中包含的至少一个存储服务器中，确定其中一个存储服务器为目标存储服务器，以将数据写入该目标存储服务器，能够保证在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常写入数据，不会造成数据丢失。
相应的，本申请还提供了一种存储介质，其中，该存储介质用于存储可执行程序代码，所述可执行程序代码用于在运行时执行本申请所述的一种基于云存储系统的数据读取方法。其中，本申请所述的一种基于云存储系统的数据读取方法，所述云存储系统至少包括：至少一个管理节点以及至少一个存储服务器，应用于查询平台，所述方法包括：
接收数据读取请求；
当无法获取到可用管理节点时，查询预先保存的索引，其中，所述索引包含每个存储服务器及其IP地址的对应关系；
根据所述每个存储服务器的IP地址，向所述每个存储服务器发送获取数据的属性信息的请求，并接收对应的所述每个存储服务器返回的其自身保存的数据的属性信息；
根据接收的所述每个存储服务器发送的其自身保存的数据的属性信息，以及所述数据读取请求中包含的待读取数据的属性信息，确定至少一个目标存储服务器，并从每个目标存储服务器中读取相应的数据。
本申请实施例中，通过预先保存包含每个存储服务器及其IP地址的对应关系的索引，在接收到数据读取请求并且无法获取到可用管理节点时，可以查询预先保存的索引，获得每个存储服务器的IP地址，然后根据每个存储服务器 的IP地址，向每个存储服务器发送获取自身保存的数据的属性信息的请求，并接收所述对应的每个存储服务器发送的自身保存的数据的属性信息，最后根据接收的每个存储服务器发送的自身保存的数据的属性信息，以及数据读取请求中包含的待读取数据的属性信息，确定至少一个目标存储服务器，以从目标存储服务器中读取相应的数据，能够保证在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常提供数据读取服务。
相应的，本申请还提供了一种应用程序，其中，该应用程序用于在运行时执行本申请所述的一种基于云存储系统的数据写入方法。其中，本申请所述的一种基于云存储系统的数据写入方法，所述云存储系统至少包括：接入节点、至少一个管理节点以及至少一个存储服务器，所述方法包括：
所述接入节点接收数据写入请求；
当无法获取到可用管理节点时，识别所述数据写入请求中包含的域标识信息，并根据所述域标识信息，查询预先保存的索引，其中，所述索引包含资源池与域标识信息的对应关系；
识别与所述域标识信息匹配的目标资源池，其中，每个资源池中包含至少一个存储服务器；
从所述目标资源池中包含的至少一个存储服务器中，确定其中一个存储服务器为目标存储服务器，并将待写入数据写入所述目标存储服务器。
本申请实施例中，通过预先保存包含资源池与域标识信息的对应关系的索引，在接收到数据写入请求并且无法获取到可用管理节点时，可以识别所述数据写入请求中包含的域标识信息，然后根据所述域标识信息，以及所述索引中保存的资源池与域标识信息的对应关系，识别与所述域标识信息匹配的目标资源池，最后从所述目标资源池中包含的至少一个存储服务器中，确定其中一个存储服务器为目标存储服务器，以将数据写入该目标存储服务器，能够保证在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常写入数据，不会造成数据丢失。
相应的，本申请还提供了一种应用程序，其中，该应用程序用于在运行 时执行本申请所述的一种基于云存储系统的数据读取方法。其中，本申请所述的一种基于云存储系统的数据读取方法，所述云存储系统至少包括：至少一个管理节点以及至少一个存储服务器，应用于查询平台，所述方法包括：
接收数据读取请求；
当无法获取到可用管理节点时，查询预先保存的索引，其中，所述索引包含每个存储服务器及其IP地址的对应关系；
根据所述每个存储服务器的IP地址，向所述每个存储服务器发送获取数据的属性信息的请求，并接收对应的所述每个存储服务器返回的其自身保存的数据的属性信息；
根据接收的所述每个存储服务器发送的其自身保存的数据的属性信息，以及所述数据读取请求中包含的待读取数据的属性信息，确定至少一个目标存储服务器，并从每个目标存储服务器中读取相应的数据。
本申请实施例中，通过预先保存包含每个存储服务器及其IP地址的对应关系的索引，在接收到数据读取请求并且无法获取到可用管理节点时，可以查询预先保存的索引，获得每个存储服务器的IP地址，然后根据每个存储服务器的IP地址，向每个存储服务器发送获取自身保存的数据的属性信息的请求，并接收所述对应的每个存储服务器发送的自身保存的数据的属性信息，最后根据接收的每个存储服务器发送的自身保存的数据的属性信息，以及数据读取请求中包含的待读取数据的属性信息，确定至少一个目标存储服务器，以从目标存储服务器中读取相应的数据，能够保证在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常提供数据读取服务。
相应的，本申请还提供了一种接入节点设备，包括：
处理器、存储器、通信接口和总线；
所述处理器、所述存储器和所述通信接口通过所述总线连接并完成相互间的通信；
所述存储器存储可执行程序代码；
所述处理器通过读取所述存储器中存储的可执行程序代码来运行与所述可执行程序代码对应的程序，以用于执行本申请所述的一种基于云存储系统的数据写入方法。其中，本申请所述的一种基于云存储系统的数据写入方法，所述云存储系统至少包括：接入节点、至少一个管理节点以及至少一个存储服务器，所述方法包括：
所述接入节点接收数据写入请求；
当无法获取到可用管理节点时，识别所述数据写入请求中包含的域标识信息，并根据所述域标识信息，查询预先保存的索引，其中，所述索引包含资源池与域标识信息的对应关系；
识别与所述域标识信息匹配的目标资源池，其中，每个资源池中包含至少一个存储服务器；
从所述目标资源池中包含的至少一个存储服务器中，确定其中一个存储服务器为目标存储服务器，并将待写入数据写入所述目标存储服务器。
本申请实施例中，通过预先保存包含资源池与域标识信息的对应关系的索引，在接收到数据写入请求并且无法获取到可用管理节点时，可以识别所述数据写入请求中包含的域标识信息，然后根据所述域标识信息，以及所述索引中保存的资源池与域标识信息的对应关系，识别与所述域标识信息匹配的目标资源池，最后从所述目标资源池中包含的至少一个存储服务器中，确定其中一个存储服务器为目标存储服务器，以将数据写入该目标存储服务器，能够保证在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常写入数据，不会造成数据丢失。
相应的，本申请提供了一种电子设备，包括：
处理器、存储器、通信接口和总线；
所述处理器、所述存储器和所述通信接口通过所述总线连接并完成相互间的通信；
所述存储器存储可执行程序代码；
所述处理器通过读取所述存储器中存储的可执行程序代码来运行与所述 可执行程序代码对应的程序，以用于执行本申请所述的一种基于云存储系统的数据读取方法。其中，本申请所述的一种基于云存储系统的数据读取方法，所述云存储系统至少包括：至少一个管理节点以及至少一个存储服务器，应用于查询平台，所述方法包括：
接收数据读取请求；
当无法获取到可用管理节点时，查询预先保存的索引，其中，所述索引包含每个存储服务器及其IP地址的对应关系；
根据所述每个存储服务器的IP地址，向所述每个存储服务器发送获取数据的属性信息的请求，并接收对应的所述每个存储服务器返回的其自身保存的数据的属性信息；
根据接收的所述每个存储服务器发送的其自身保存的数据的属性信息，以及所述数据读取请求中包含的待读取数据的属性信息，确定至少一个目标存储服务器，并从每个目标存储服务器中读取相应的数据。
本申请实施例中，通过预先保存包含每个存储服务器及其IP地址的对应关系的索引，在接收到数据读取请求并且无法获取到可用管理节点时，可以查询预先保存的索引，获得每个存储服务器的IP地址，然后根据每个存储服务器的IP地址，向每个存储服务器发送获取自身保存的数据的属性信息的请求，并接收所述对应的每个存储服务器发送的自身保存的数据的属性信息，最后根据接收的每个存储服务器发送的自身保存的数据的属性信息，以及数据读取请求中包含的待读取数据的属性信息，确定至少一个目标存储服务器，以从目标存储服务器中读取相应的数据，能够保证在全部管理节点出现故障或管理节点集群网络异常时，云存储系统仍能正常提供数据读取服务。
对于装置/存储介质/应用程序/接入节点设备/电子设备实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示 这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
本说明书中的各个实施例均采用相关的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于装置实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
本领域普通技术人员可以理解实现上述方法实施方式中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，所述的程序可以存储于计算机可读取存储介质中，这里所称得的存储介质，如：ROM/RAM、磁碟、光盘等。
以上所述仅为本申请的较佳实施例而已，并非用于限定本申请的保护范围。凡在本申请的精神和原则之内所作的任何修改、等同替换、改进等，均包含在本申请的保护范围内。

Claims (27)

  1. 一种基于云存储系统的数据写入方法，所述云存储系统至少包括：接入节点、至少一个管理节点以及至少一个存储服务器，其特征在于，所述方法包括：
    所述接入节点接收数据写入请求；
    当无法获取到可用管理节点时，识别所述数据写入请求中包含的域标识信息，并根据所述域标识信息，查询预先保存的索引，其中，所述索引包含资源池与域标识信息的对应关系；
    识别与所述域标识信息匹配的目标资源池，其中，每个资源池中包含至少一个存储服务器；
    从所述目标资源池中包含的至少一个存储服务器中，确定其中一个存储服务器为目标存储服务器，并将待写入数据写入所述目标存储服务器。
  2. 根据权利要求1所述的方法，其特征在于，所述确定其中一个存储服务器为目标存储服务器，包括：
    获取每个存储服务器的当前负载量；
    识别当前负载量最小的存储服务器，将其作为目标存储服务器。
  3. 根据权利要求1所述的方法，其特征在于，所述无法获取到可用管理节点包括：每个管理节点均出现故障或管理节点集群网络异常。
  4. 根据权利要求1-3任一所述的方法，其特征在于，构建所述索引的过程包括：
    向每个管理节点发送获取每个资源池与域标识信息的对应关系的请求，并接收所述每个管理节点发送的每个资源池与域标识信息的对应关系；
    在索引中保存所述每个资源池与域标识信息的对应关系。
  5. 根据权利要求4所述的方法，其特征在于，还包括：
    按照设定的时间间隔，更新所述索引中保存的每个资源池与域标识信息的对应关系。
  6. 一种基于云存储系统的数据读取方法，所述云存储系统至少包括：至少一个管理节点以及至少一个存储服务器，其特征在于，应用于查询平台，所述方法包括：
    接收数据读取请求；
    当无法获取到可用管理节点时，查询预先保存的索引，其中，所述索引包含每个存储服务器及其IP地址的对应关系；
    根据所述每个存储服务器的IP地址，向所述每个存储服务器发送获取数据的属性信息的请求，并接收对应的所述每个存储服务器返回的其自身保存的数据的属性信息；
    根据接收的所述每个存储服务器发送的其自身保存的数据的属性信息，以及所述数据读取请求中包含的待读取数据的属性信息，确定至少一个目标存储服务器，并从每个目标存储服务器中读取相应的数据。
  7. 根据权利要求6所述的方法，其特征在于，所述索引中还包括每个存储服务器是否在线的状态信息，所述根据所述每个存储服务器的IP地址，向所述每个存储服务器发送获取数据的属性信息的请求，包括：
    根据每个存储服务器的状态信息，识别处于在线状态的存储服务器；
    根据处于在线状态的存储服务器的IP地址，向对应的每个处于在线状态的存储服务器发送获取数据的属性信息的请求。
  8. 根据权利要求6所述的方法，其特征在于，所述无法获取到可用管理节点包括：每个管理节点均出现故障或管理节点集群网络异常。
  9. 根据权利要求6-8任一所述的方法，其特征在于，构建所述索引的过程包括：
    向每个管理节点发送获取每个存储服务器和IP地址的对应关系的请求，并接收所述每个管理节点发送的每个存储服务器和IP地址的对应关系；
    在索引中保存所述每个存储服务器和IP地址的对应关系。
  10. 根据权利要求6-8任一所述的方法，其特征在于，所述方法还包括：
    按照设定的时间间隔，更新所述索引中保存的每个存储服务器和IP地址的对应关系。
  11. 一种基于云存储系统的数据写入装置，所述云存储系统至少包括：接入节点、至少一个管理节点以及至少一个存储服务器，其特征在于，应用于所述接入节点，所述装置包括：
    接收模块，用于接收数据写入请求；
    处理模块，用于当无法获取到可用管理节点时，识别所述数据写入请求中包含的域标识信息，并根据所述域标识信息，查询预先保存的索引，其中，所述索引包含资源池与域标识信息的对应关系；
    识别模块，用于识别与所述域标识信息匹配的目标资源池，其中，每个资源池中包含至少一个存储服务器；
    写入模块，用于从所述目标资源池中包含的至少一个存储服务器中，确定其中一个存储服务器为目标存储服务器，并将待写入数据写入所述目标存储服务器。
  12. 根据权利要求11所述的装置，其特征在于，所述写入模块，具体用于：
    获取每个存储服务器的当前负载量；
    识别当前负载量最小的存储服务器，将其作为目标存储服务器。
  13. 根据权利要求11所述的装置，其特征在于，所述无法获取到可用管理节点包括：每个管理节点均出现故障或管理节点集群网络异常。
  14. 根据权利要求11-13任一所述的装置，其特征在于，还包括：
    执行模块，用于向每个管理节点发送获取每个资源池与域标识信息的对应关系的请求，并接收所述每个管理节点发送的每个资源池与域标识信息的对应关系；
    保存模块，用于在索引中保存所述每个资源池与域标识信息的对应关系。
  15. 根据权利要求14所述的装置，其特征在于，还包括：
    更新模块，用于按照设定的时间间隔，更新所述索引中保存的每个资源池与域标识信息的对应关系。
  16. 一种基于云存储系统的数据读取装置，所述云存储系统至少包括：至少一个管理节点以及至少一个存储服务器，其特征在于，应用于查询平台，所述装置包括：
    接收模块，用于接收数据读取请求；
    查询模块，用于当无法获取到可用管理节点时，查询预先保存的索引，其中，所述索引包含每个存储服务器及其IP地址的对应关系；
    处理模块，用于根据所述每个存储服务器的IP地址，向所述每个存储服务器发送获取数据的属性信息的请求，并接收对应的所述每个存储服务器返回的其自身保存的数据的属性信息；
    读取模块，用于根据接收的所述每个存储服务器发送的其自身保存的数据的属性信息，以及所述数据读取请求中包含的待读取数据的属性信息，确定至少一个目标存储服务器，并从每个目标存储服务器中读取相应的数据。
  17. 根据权利要求16所述的装置，其特征在于，所述索引中还包括每个存储服务器是否在线的状态信息，所述处理模块，包括：
    识别子模块，用于根据每个存储服务器的状态信息，识别处于在线状态的存储服务器；
    处理子模块，用于根据处于在线状态的存储服务器的IP地址，向对应的每个处于在线状态的存储服务器发送获取数据的属性信息的请求。
  18. 根据权利要求16所述的装置，其特征在于，所述无法获取到可用管理节点包括：每个管理节点均出现故障或管理节点集群网络异常。
  19. 根据权利要求16-18任一所述的装置，其特征在于，所述装置还包括：
    执行模块，用于向每个管理节点发送获取每个存储服务器和IP地址的对应关系的请求，并接收所述每个管理节点发送的每个存储服务器和IP地址的对应关系；
    保存模块，用于在索引中保存所述每个存储服务器和IP地址的对应关系。
  20. 根据权利要求16-18任一所述的装置，其特征在于，所述装置还包括：
    更新模块，用于按照设定的时间间隔，更新所述索引中保存的每个存储服务器和IP地址的对应关系。
  21. 一种云存储系统，其特征在于，所述云存储系统包括如权利要求11-15任一项所述的设置在接入节点中的装置，以及至少一个存储服务器。
  22. 一种存储介质，其特征在于，所述存储介质用于存储可执行程序代码，所述可执行程序代码用于在运行时执行如权利要求1-5任一项所述的一种基于云存储系统的数据写入方法。
  23. 一种存储介质，其特征在于，所述存储介质用于存储可执行程序代码，所述可执行程序代码用于在运行时执行如权利要求6-10任一项所述的一种基于云存储系统的数据读取方法。
  24. 一种应用程序，其特征在于，所述应用程序用于在运行时执行如权利要求1-5任一项所述的一种基于云存储系统的数据写入方法。
  25. 一种应用程序，其特征在于，所述应用程序用于在运行时执行如权利要求6-10任一项所述的一种基于云存储系统的数据读取方法。
  26. 一种接入节点设备，包括：
    处理器、存储器、通信接口和总线；
    所述处理器、所述存储器和所述通信接口通过所述总线连接并完成相互间的通信；
    所述存储器存储可执行程序代码；
    所述处理器通过读取所述存储器中存储的可执行程序代码来运行与所述可执行程序代码对应的程序，以用于执行如权利要求1-5任一项所述的一种基于云存储系统的数据写入方法。
  27. 一种电子设备，包括：
    处理器、存储器、通信接口和总线；
    所述处理器、所述存储器和所述通信接口通过所述总线连接并完成相互间的通信；
    所述存储器存储可执行程序代码；
    所述处理器通过读取所述存储器中存储的可执行程序代码来运行与所述可执行程序代码对应的程序，以用于执行如权利要求6-10任一项所述的一种基于云存储系统的数据读取方法。
PCT/CN2016/105094 2016-02-17 2016-11-08 数据写入、读取方法及装置、云存储系统 WO2017140131A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US15/999,307 US11586673B2 (en) 2016-02-17 2016-11-08 Data writing and reading method and apparatus, and cloud storage system
EP16890357.3A EP3418877B1 (en) 2016-02-17 2016-11-08 Data writing and reading method and apparatus, and cloud storage system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610088973.4 2016-02-17
CN201610088973.4A CN107092437B (zh) 2016-02-17 2016-02-17 数据写入、读取方法及装置、云存储系统

Publications (1)

Publication Number Publication Date
WO2017140131A1 true WO2017140131A1 (zh) 2017-08-24

Family

ID=59624949

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/105094 WO2017140131A1 (zh) 2016-02-17 2016-11-08 数据写入、读取方法及装置、云存储系统

Country Status (4)

Country Link
US (1) US11586673B2 (zh)
EP (1) EP3418877B1 (zh)
CN (1) CN107092437B (zh)
WO (1) WO2017140131A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115242807A (zh) * 2022-06-30 2022-10-25 深圳震有科技股份有限公司 一种5g通信系统中的数据存取方法及相关设备
CN115517009A (zh) * 2020-05-19 2022-12-23 深圳市欢太科技有限公司 集群管理方法、集群管理装置、存储介质与电子设备
CN115905219A (zh) * 2022-11-04 2023-04-04 上海威固信息技术股份有限公司 多通道数据存云库用的数据存储驱动系统及其传输方法

Families Citing this family (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109587191B (zh) * 2017-09-29 2022-04-05 杭州海康威视系统技术有限公司 跨云域的数据传输方法、互联云中心、云域及互联云系统
CN109597567B (zh) * 2017-09-30 2022-03-08 网宿科技股份有限公司 一种数据处理方法和装置
CN107885803B (zh) * 2017-10-31 2020-05-01 中国地质大学（武汉） 一种大数据写入、读取速度耦合的方法、设备及存储设备
CN110069210B (zh) 2018-01-23 2021-09-28 杭州海康威视系统技术有限公司 一种存储系统、存储资源的分配方法及装置
CN110139166B (zh) * 2018-02-08 2021-07-09 杭州海康威视系统技术有限公司 一种视频数据传输方法、装置、系统及电子设备
CN110471613B (zh) * 2018-05-09 2021-01-12 杭州海康威视系统技术有限公司 存储数据的方法、读取数据的方法、装置及系统
CN110502333B (zh) * 2018-05-16 2022-09-02 杭州海康威视系统技术有限公司 一种访问请求的处理方法和云存储系统
CN110765125B (zh) * 2018-07-25 2022-09-20 杭州海康威视数字技术股份有限公司 一种存储数据的方法及装置
CN111221468B (zh) * 2018-11-26 2023-10-13 杭州海康威视系统技术有限公司 存储块数据删除方法、装置、电子设备及云存储系统
CN109492419B (zh) * 2018-11-27 2022-07-01 众安信息技术服务有限公司 用于获取区块链中的数据的方法、装置及存储介质
CN111352725B (zh) * 2018-12-24 2023-08-18 杭州海康威视系统技术有限公司 存储负载均衡的方法和装置
CN111367905B (zh) * 2018-12-26 2023-10-10 杭州海康威视系统技术有限公司 对象数据存储、数据索引构建方法、装置及接入服务器
CN109992531B (zh) * 2019-04-15 2020-11-10 成都四方伟业软件股份有限公司 数据存储方法及装置
CN112214466B (zh) * 2019-07-12 2024-05-14 海能达通信股份有限公司 分布式集群系统及数据写入方法、电子设备、存储装置
US11888933B2 (en) * 2019-08-06 2024-01-30 Xi'an Zhongxing New Software Co., Ltd. Cloud service processing method and device, cloud server, cloud service system and storage medium
CN110531934A (zh) * 2019-08-23 2019-12-03 北京浪潮数据技术有限公司 基于分布式系统的数据存储方法和装置
CN111404978B (zh) * 2019-09-06 2023-05-02 杭州海康威视系统技术有限公司 一种数据存储方法及云存储系统
CN112543213B (zh) * 2019-09-23 2022-06-03 杭州海康威视数字技术股份有限公司 一种数据处理方法及装置
CN111404980B (zh) * 2019-09-29 2023-04-18 杭州海康威视系统技术有限公司 一种数据存储方法及一种对象存储系统
CN111198662B (zh) * 2020-01-03 2023-07-14 腾讯云计算(长沙)有限责任公司 一种数据存储方法、装置和计算机可读存储介质
CN111258978B (zh) * 2020-01-17 2024-04-19 广东小天才科技有限公司 一种数据存储的方法
CN113806593B (zh) * 2020-06-17 2024-10-15 北京金风科创风电设备有限公司 风电场的通信异常检测方法、装置以及场站控制器
CN111913658B (zh) * 2020-07-13 2021-08-03 华中科技大学 一种提升云对象存储系统中负载性能的方法及系统
CN114077639B (zh) * 2020-08-20 2024-08-23 北京字节跳动网络技术有限公司 数据写入方法、装置、电子设备和存储介质
CN112000850B (zh) * 2020-08-26 2024-02-20 杭州海康威视系统技术有限公司 进行数据处理的方法、装置、系统及设备
CN113010376B (zh) * 2021-03-01 2023-07-21 北京聚云科技有限公司 一种对存储训练数据的云存储系统的监测方法及装置
CN113923095B (zh) * 2021-09-30 2024-06-14 郑州浪潮数据技术有限公司 一种集群消息转发方法、系统及存储介质
CN114928621A (zh) * 2022-02-15 2022-08-19 杭州瑞网广通信息技术有限公司 一种云存储系统的数据写入方法以及装置
CN114697353B (zh) * 2022-05-27 2022-09-06 邹平市供电有限公司 一种分布式存储集群电网数据存储控制方法
CN118568331A (zh) * 2024-08-01 2024-08-30 浙江大华技术股份有限公司 一种数据通讯方法和相关装置

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102035865A (zh) * 2009-09-30 2011-04-27 阿里巴巴集团控股有限公司 数据存储及数据寻址方法、系统和设备
CN104331330A (zh) * 2014-10-27 2015-02-04 华为技术有限公司 资源池生成方法以及装置
CN104901903A (zh) * 2015-04-07 2015-09-09 北京邮电大学 软件定义网络多域资源处理方法及系统

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101827121B (zh) * 2010-03-12 2013-01-30 成都市华为赛门铁克科技有限公司 在raid中创建文件的方法、服务端和系统
CN102714615B (zh) * 2010-11-29 2015-05-27 华为技术有限公司 节点故障处理方法、系统及相关设备
CN102594852B (zh) * 2011-01-04 2016-03-30 中国移动通信集团公司 数据访问方法、节点及系统
CN102137014B (zh) * 2011-03-11 2013-12-04 华为技术有限公司 资源管理方法、系统和资源管理器
US20140181116A1 (en) * 2011-10-11 2014-06-26 Tianjin Sursen Investment Co., Ltd. Method and device of cloud storage
CN102694860A (zh) * 2012-05-25 2012-09-26 北京邦诺存储科技有限公司 一种云存储的数据处理方法、设备及系统
US8804740B2 (en) * 2012-06-15 2014-08-12 Citrix Systems, Inc. Systems and methods for reassembly of packets distributed across a cluster
CN103942112B (zh) * 2013-01-22 2018-06-15 深圳市腾讯计算机系统有限公司 磁盘容错方法、装置及系统
US9405926B2 (en) * 2014-06-30 2016-08-02 Paul Lewis Systems and methods for jurisdiction independent data storage in a multi-vendor cloud environment
CN104679611B (zh) * 2015-03-05 2018-03-09 浙江宇视科技有限公司 数据资源复制方法以及装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102035865A (zh) * 2009-09-30 2011-04-27 阿里巴巴集团控股有限公司 数据存储及数据寻址方法、系统和设备
CN104331330A (zh) * 2014-10-27 2015-02-04 华为技术有限公司 资源池生成方法以及装置
CN104901903A (zh) * 2015-04-07 2015-09-09 北京邮电大学 软件定义网络多域资源处理方法及系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3418877A4 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115517009A (zh) * 2020-05-19 2022-12-23 深圳市欢太科技有限公司 集群管理方法、集群管理装置、存储介质与电子设备
CN115517009B (zh) * 2020-05-19 2023-10-13 深圳市欢太科技有限公司 集群管理方法、集群管理装置、存储介质与电子设备
CN115242807A (zh) * 2022-06-30 2022-10-25 深圳震有科技股份有限公司 一种5g通信系统中的数据存取方法及相关设备
CN115905219A (zh) * 2022-11-04 2023-04-04 上海威固信息技术股份有限公司 多通道数据存云库用的数据存储驱动系统及其传输方法
CN115905219B (zh) * 2022-11-04 2023-09-19 上海威固信息技术股份有限公司 多通道数据存云库用的数据存储驱动系统及其传输方法

Also Published As

Publication number Publication date
CN107092437B (zh) 2019-11-22
EP3418877B1 (en) 2023-04-05
CN107092437A (zh) 2017-08-25
US20190042659A1 (en) 2019-02-07
US11586673B2 (en) 2023-02-21
EP3418877A4 (en) 2019-03-13
EP3418877A1 (en) 2018-12-26

Similar Documents

Publication Publication Date Title
WO2017140131A1 (zh) 数据写入、读取方法及装置、云存储系统
US7490265B2 (en) Recovery segment identification in a computing infrastructure
KR102013005B1 (ko) 확장 가능한 환경에서의 파티션 관리 기법
US8239536B2 (en) System for generic service management in a distributed and dynamic resource environment, providing constant service access to users
CN108696581B (zh) 分布式信息的缓存方法、装置、计算机设备以及存储介质
US8713352B2 (en) Method, system and program for securing redundancy in parallel computing system
US11368407B2 (en) Failover management using availability groups
US10425502B2 (en) System and method for acquiring, processing and updating global information
US10673694B2 (en) Private network mirroring
US9390156B2 (en) Distributed directory environment using clustered LDAP servers
US20170031700A1 (en) Methods, systems, and computer readable storage devices for handling virtualization of a physical telephone number mapping service
US8990608B1 (en) Failover of applications between isolated user space instances on a single instance of an operating system
CN112380072A (zh) 多数据中心访问方法及系统
CN112162852A (zh) 一种多架构cpu节点的管理方法、装置及相关组件
CN107508700B (zh) 容灾方法、装置、设备及存储介质
CN109067564B (zh) 分布式处理方法、装置、计算机设备及存储介质
CN114371914A (zh) 容器ip地址配置方法、装置、存储介质及电子设备
CN116074195A (zh) 域名同步方法、业务容器、集群、计算节点及存储介质
US8595349B1 (en) Method and apparatus for passive process monitoring
CN109976944B (zh) 数据处理方法和系统，存储介质和电子设备
US10771539B2 (en) Systems and methods for cross-cluster service provision
CN114090541A (zh) 数据库业务处理方法和装置
US11599352B2 (en) Method of creating an intelligent upgrade flow for a heterogeneous data center
US20240028346A1 (en) Linking kubernetes resources with underlying cloud infrastructure
CN112965763B (zh) 一种业务处理系统、方法、装置和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16890357

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2016890357

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2016890357

Country of ref document: EP

Effective date: 20180917