CN113453038A - Effectiveness optimal collaborative cache management method under CDN-P2P hybrid architecture - Google Patents
Effectiveness optimal collaborative cache management method under CDN-P2P hybrid architecture Download PDFInfo
- Publication number
- CN113453038A CN113453038A CN202110709682.3A CN202110709682A CN113453038A CN 113453038 A CN113453038 A CN 113453038A CN 202110709682 A CN202110709682 A CN 202110709682A CN 113453038 A CN113453038 A CN 113453038A
- Authority
- CN
- China
- Prior art keywords
- node
- nodes
- super
- video
- resource
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
- H04N21/23106—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving caching operations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
- H04L67/5682—Policies or rules for updating, deleting or replacing the stored data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2181—Source of audio or video content, e.g. local disk arrays comprising remotely distributed storage units, e.g. when movies are replicated over a plurality of video servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
- H04N21/23113—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving housekeeping operations for stored content, e.g. prioritizing content for deletion because of storage space restrictions
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a method for managing an optimal collaborative cache under a CDN-P2P hybrid architecture, which comprises the steps of firstly constructing a super node group, supporting the combination of a P2P technology and a content distribution network, and providing services for other nodes by utilizing the characteristics of high expandability and low deployment cost of a P2P overlay network; secondly, constructing a corresponding mathematical model based on the global utility value according to the resource supply and demand relationship among ISPs domains, and optimizing the model by using a resource allocation greedy algorithm; and finally, designing a cooperative cache management method based on an optimal utility model, and placing the video resource with high popularity at the side close to the user through cooperative cache among nodes in the super node group and CDNs based on uploading bandwidth of the sending node and downloading bandwidth and disk capacity of the receiving node, so that the flow of a return link among ISPs is reduced, and the playing smoothness and the cache hit rate of the user are improved.
Description
Technical Field
The invention relates to the technical field of computer networks, peer-to-peer network technologies and streaming media, in particular to a method for managing an optimal collaborative cache under a CDN-P2P hybrid architecture.
Background
Network video is one of the main applications of the internet and one of the main traffic sources in the network, and the user scale is large and growing, and the large user scale generates huge bandwidth cost. In recent years, the HTTP adaptive Streaming media technology is abbreviated as DASH, which combines the advantages of the conventional Real Time Streaming Protocol (RTSP)/(Real Time Protocol, RTP) technology and the HTTP progressive download technology. The method has the characteristics of simple deployment, easy expansion and the like, and is widely applied. Representative products of commercial deployment include Smooth Streaming technology (SS) by microsoft, HLS (HTTP Live Streaming) by apple inc, HDS (HTTP Dynamic Streaming) by Adobe. The streaming media transmission has the greatest advantages that the whole media file cannot be downloaded before playing, only part of media content is cached, a user can play the streaming media without waiting for the completion of downloading of the whole streaming media resource, and the user can watch the streaming media resource only by waiting for a short start delay, so that the streaming media can be played simultaneously.
The P2P technology is combined with a Content Delivery Network (CDN) to build a hybrid CDN-P2P architecture. The CDNs nodes are organized in a P2P mode, and data sharing and caching are carried out among the CDNs nodes in a P2P connection mode, so that the pressure of a central server can be effectively relieved. However, CDN technology for video delivery is still a terminal/server architecture in nature, and when users are large, the bandwidth cost becomes expensive. The peer-to-peer network technology has been proven to be a technology for reducing bandwidth cost and improving system expansibility in the streaming media field, and if the peer-to-peer network technology is fused with the HTTP adaptive streaming media technology, the advantages of low cost, high expansibility and the like of the peer-to-peer network technology can be utilized, and the advantages of simple deployment, easy expansion and the like of the peer-to-peer network technology can also be utilized.
In the P2P streaming system, the streaming video segments required by the nodes come from nearby nodes or servers. When the adjacent nodes of one node have no corresponding streaming media segment resource, the request is sent to the server. With the popularization of vast intelligent terminal devices (smart phones, ipads, portable computers, and the like), many devices have large capacity and high bandwidth. If multiple types of terminals can adopt the same streaming media technology and a content distribution strategy of mutual cooperation, a cooperation caching scheme is adopted based on the equipment caching characteristic, and the transmission streaming media video sharing is facilitated. Many fixed terminal devices (desktop computers, televisions, etc.) currently exist with a disk buffer that stores and holds video for a significant period of time, which makes some popular videos even longer to serve. However, not every end node has the ability to serve other nodes, and therefore, selecting some well-performing nodes from the end devices for serving can offload a portion of the backhaul traffic. Considering that the service capability of a single terminal node is weak, how to organize the terminal nodes to be aggregated into a node group is a problem to be considered. Most of the traffic cost existing in the network is generated by transmitting data across ISPs, and how to construct a utility mathematics-based model and optimize the model is an important problem according to the supply-demand relationship of video resources among ISPs. Based on the utility optimal model, how to use the CDNs nodes and the terminal nodes to decide the leaving of the video to be cached is an important problem.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a method for managing an optimal collaborative cache under a CDN-P2P hybrid architecture.
The technical scheme for realizing the purpose of the invention is as follows:
a method for managing an optimal collaborative cache under a CDN-P2P hybrid architecture comprises the steps of firstly constructing a super node group; secondly, constructing a corresponding mathematical model based on a global optimization target according to the resource supply and demand relationship among ISPs domains, and optimizing the model by using a resource allocation greedy algorithm; finally, designing a collaborative cache management method based on the utility optimal model;
the super node cluster takes network time delay between super nodes as a distance between the two nodes, the super nodes establish the super node cluster based on a principle of proximity, the super nodes in the cluster are neighbor nodes, each super node maintains a group of neighbor super node information, a sharing area between the nodes is relatively transparent, the super nodes can acquire video resources from the sharing area of the neighbor nodes, and the specific steps of establishing the super node cluster are as follows:
1-1) starting;
1-2) calculating the Ability value Abiliity of the common node ii;
1-3) Ability to comply with Abiliity valuesiArranging the nodes in a descending order to obtain an ordered node list orderSuPeerList;
1-4) top list orderPeerListEach node is marked as a super node, and a mark symbol SPflag is 1, wherein Numsp represents the number of super nodes, N represents the total number of nodes, whereinRepresents a super node occupation ratio, and
1-5) Each super node k1Sending a detection packet to other super nodes to obtain a super node list orderPeerRttDist with transmission delay less than delta, and sequencing the list according to ascending time delay; delta represents the maximum acceptable time delay, and when the time delay is too large, a large amount of time is needed for periodic information exchange between nodes.
1-6) sequentially inquiring whether a super node k exists in the list orderPeerRttDistspWhether the size of the group is smaller than tau; if yes, the super node k1Adding kspThe group in which the gene belongs; otherwise, the super node k1Establishing a new group; tau represents the maximum node number of a super node group, if the number of super nodes in a group is too small, the benefit brought by the shared service is not very large, and if the number of nodes is too large, the nodes in the group are maintainedWith certain difficulties.
1-7) finishing;
the mathematical model is established according to the supply and demand relationship among ISPs domains, and is optimized through a resource allocation greedy algorithm to ensure that the effectiveness of ISP domain request resources is optimal, wherein the mathematical model expression is as follows:
Minimize:Utility= ∑i∈N,v∈Qvqi,v·tω (1)
in the above formula (1), QvIndicating the missing target resource, tωRepresenting a link cost;
the resource allocation greedy algorithm is to set a resource priority acquisition path, preferentially acquire resources from a super node group of a neighbor ISP domain, and then acquire resources from a CDN node, and specifically comprises the following steps:
2-1) starting;
2-2) initialization Utility value Utility, t1,t2,t3,t4(t1,t2,t3,t4∈tω);
2-3) obtaining target resource Q lacking in ISP domainv;
2-4) acquiring neighbor ISPs list information, and executing steps 2-5) -2-7) on each neighbor ISP domain;
2-5) judging whether Q exists in the super node groupvIf not, executing step 6), if yes, updating QvAnd Utility, judgment QvIf the result is empty, executing the step 2-8), otherwise, executing the step 2-6);
2-6) judging whether Q exists in CDN nodevIf not, executing the step 2-7), and if so, updating the QvAnd Utility, judgment QvIf the result is empty, executing the step 2-8) if the result is empty, and executing the step 2-7) if the result is not empty;
2-7) judging whether Q exists in neighbor nodes of the CDN nodevIf not, executing the step 2-8), and if so, updating the QvAnd Utility;
2-8) ending;
the cooperative cache management method comprises a cooperative cache mode of a super node in a super node group, a cooperative cache mode of the super node group and CDNs nodes and a cache replacement method;
the cooperative caching mode of the super nodes in the super node clusters is that each super node cluster is regarded as a whole, the nodes in the clusters can share resources, and the nodes in the clusters can store more video resources in limited capacity by caching different video resources, so that the diversity of the resources in the clusters is improved;
the method is characterized in that a coordinated cache mode of the super node group and the CDNs nodes is that target resources are divided into popular resources and secondary popular resources according to popularity degrees, the popular resources are stored in the super node group close to a user side preferentially, and the secondary popular resources are stored in the CDNs nodes secondly;
the cache replacement method specifically comprises the following steps:
3-1) starting;
3-2) judging whether the current node has enough bandwidth, if not, executing the step 3-5), otherwise, judging whether the disk capacity of the node is enough, if so, directly caching the resource, otherwise, executing the step 3-3) -the step 3-4);
3-3) calculating the Energy value Energy of the video resource in the diskvSorting the video resources in a descending order according to the energy values;
3-4) replacing the video resource to be cached with the video resource with the minimum energy value;
3-5) finishing.
In step 1-2), the Ability value AbiliityiThe evaluation index representing the service performance of the node is comprehensively determined by four factors of historical online time, bandwidth, storage capacity and exit frequency of the node. The historical online time, the bandwidth and the capacity of the node serve as positive attributes, and the historical exit frequency serves as negative attributes, but the positive attributes are not weighted as much as possible. The longer the historical online time of a node is, the longer the continuous service time may be, and the less the bandwidth and capacity of the node is, although the historical online time of the node is large, the node has no free bandwidth anda capacity resource service request node. The historical exit frequency of the node reflects the online stable state of the node, when the bandwidth and the capacity of the node are large, the higher the historical exit frequency is, the request node cannot obtain stable service guarantee, in addition, the historical online time and the historical exit frequency are mutually restricted, the larger the historical exit frequency is, and the larger the historical online time is, the node cannot be used as a service node, and the capacity value of the node is defined as follows:
Abilityi=ρ1OTime+ρ2Bandwidth+ρ3diskCapacity+ρ4exitFreq (2)
in the formula (2), OTime represents the historical online time of the node, Bandwidth represents the Bandwidth of the node, diskCapacity represents the disk capacity of the node, exitFreq represents the exit frequency of the node, and rho1、ρ2、ρ3And ρ4Coefficients representing historical online duration, bandwidth, disk capacity, and historical exit frequency attributes, respectively, and ρ1+ρ2+ρ3+ρ4=1。
The expression of the mathematical model meets the following constraint conditions:
constraint conditionsEnsuring that the available bandwidth of the node is not less than 0 in the sending period and bandwidthi,avaiIs the available bandwidth of node i, sjIs the length of video segment j, Γ is the maximum allowed transmission duration;
In step 2-3), resource QvFrom super node clusters and CDNs nodes, QvThe calculation formula of (2) is as follows:
in the above formula (3), G represents a neighbor ISP set; g _ sps represents a set of super node clusters within an ISP domain, g _ CDN represents a CDN node within an ISP domain, Xv,sE {0,1}, is a binary representation of the resource in the supernode, Xv,sWhen the value is 1, the resource v exists in the super node s, and otherwise, the resource v does not exist;is a resource binary representation of a neighboring super node, Yv,cIs a resource binary representation of the CDN node,is a resource binary representation of a neighboring CDN node.
In the step 2-7), the calculation formula of the Utility value Utility is as follows:
in the above formula (4), t1Representing the link cost, t, of acquiring resources from a super node s2Represents the link cost, t, of acquiring resources from s' neighbor nodes2=at1,t2>t1,(a=1,2,…,n),t3Represents the link cost, t, of acquiring resources from a CDN node4Represents the link cost, t, of acquiring resources from CDN neighbor nodes4=bt3,t4>t3And b is 1, 2, …, n, and a and b respectively represent the number of super node hops and the number of CDN hops.
In step 3-3), the Energy value EnergyvThe method comprehensively considers the popularity, the scarcity, the request frequency and the video size of the video, carries out weighted summation on the four factors, and obtains the Energy value Energy of the videovThe calculation formula is as follows:
Energyv=σ·Pv+β·Scarcityv+γ·Reqv,T+λ·Sizev (5)
the above formula (5)In, PvRepresenting the popularity of the video v, ScarcityvIndicating the degree of scarcity, Size, of the video vvRepresenting the size of the video, Reqv,TRepresents the number of times that the video v is requested in a period T, and sigma, beta, gamma and lambda represent the parameters of the popularity, the Scarcity, the request frequency and the video size of the video respectively, ScarcityvThe calculation formula of (a) is as follows:
num in the above formula (6)vRepresenting the number of copies of video v and TotalNum representing the number of copies of all videos.
The invention provides an optimal-utility collaborative cache management method under a CDN-P2P hybrid architecture, which comprises the steps of firstly, considering the limitation of service capability of a single node, constructing a super node group, supporting the combination of a P2P technology and content distribution, and utilizing the characteristics of high expandability and low deployment cost of a P2P overlay network to serve other nodes; secondly, constructing a corresponding mathematical model based on the global utility value according to the resource supply and demand relationship among ISPs domains, and optimizing the model by using a resource allocation greedy algorithm to achieve the optimal global utility; and finally, designing a cooperative cache management method based on an optimal utility model, preferentially placing the video resources with high popularity in the super node group close to the side of the common user and placing the video resources with low popularity in the CDNs nodes through the cooperative cache among the nodes in the super node group and the cooperative cache among the super node group and the CDNs nodes under the constraints of uploading bandwidth of the sending node and downloading bandwidth and disk capacity of the receiving node, thereby reducing the flow of a backhaul link between ISPs and improving the playing fluency and cache hit rate of the user.
Drawings
FIG. 1 is a CDN-P2P hybrid stream architecture diagram;
FIG. 2 is a flow chart of a process for constructing a cluster of supernodes;
FIG. 3 is a flow diagram of a resource allocation greedy algorithm;
FIG. 4 is a flow chart of a cache replacement method.
Detailed Description
The invention will be further elucidated with reference to the drawings and examples, without however being limited thereto.
Example (b):
fig. 1 is a CDN-P2P hybrid streaming architecture diagram according to the present invention, which is composed of a video source server, a tracking server, edge CDN nodes, super nodes (special user nodes), and general nodes (user nodes). The video source server firstly distributes the video to edge CDNs nodes in different ISPs, and then the video is distributed to common nodes by the edge CDNs nodes. The architecture comprises two layers of overlay networks, one layer is an overlay network organized by CDNs nodes, and the other layer is an overlay network formed by super nodes.
The super node group is composed of fixed terminal nodes (fixed nodes) with good performance, the fixed nodes are provided with a disk cache region which can be used for storing resources for a long time, and a sharing region which can be used for sharing the resources with the adjacent nodes and playing the cache region for playing the requested resources. Each super node is a request node and a service node, and stores a neighbor node information list and a neighbor node entry list, which respectively record an IP address, a port number, a URL address, an IP address, a URL address and a video entry.
When a super node leaves the network, it firstly sends the information of exiting the network to the tracking server of the ISP domain and the neighbor super nodes in the cluster, and the tracking server and the neighbor super nodes update the self list information.
When a super node suddenly leaves the network due to network disconnection or failure, the tracking server and the neighbor super nodes do not receive periodic broadcast information from the node, the node leaves the network due to abnormality, and at the moment, the tracking server and the neighbor super nodes delete abnormal node information.
Specifically, for the purpose of illustrating the embodiments of the present invention, in the embodiment, it is first assumed that there are one video source server and three tracking servers corresponding to three network service providers (ISPs) in the CDN-P2P hybrid architecture, and CDN nodes in each ISP domainThe number is 10, the CDN node cache size is 50, and the long transfer/download bandwidth is 20. Further, assuming that the number of common nodes in each ISP domain is 2000,setting 10 super node groups in each ISP domain, wherein the maximum number of nodes in the groups is 20, the cache size of the super nodes is 4, the uploading/downloading bandwidth is 1, the total number of resources is 1000, and the coefficient rho of the super nodes is set1=0.1,ρ2=ρ3=ρ4Setting t to simulate the difference of different link costs as 0.31=1,t3=4。
Further assuming that σ ═ γ ═ 0.4 and β ═ λ ═ 0.1, in the cache replacement phase, the ultimate goal is to store more popular resources, replacing the worthless video resources in the disk. For the resources in the cache with large size and small popularity and request times, the resources need to be replaced so as to avoid the waste of capacity; when the popularity is high, the number of requests is large, but the copy resources are insufficient, more copy caching is needed.
A method for managing an optimal collaborative cache under a CDN-P2P hybrid architecture comprises the steps of firstly constructing a super node group; secondly, constructing a corresponding mathematical model based on a global optimization target according to the resource supply and demand relationship among ISPs domains, and optimizing the model by using a resource allocation greedy algorithm; finally, designing a collaborative cache management method based on the utility optimal model;
the super node cluster takes network time delay between super nodes as a distance between the two nodes, the super nodes establish the super node cluster based on a principle of proximity, the super nodes in the cluster are neighbor nodes, each super node maintains a group of neighbor super node information, a shared area between the nodes is relatively transparent, the super nodes can acquire video resources from the shared area of the neighbor nodes, and as shown in fig. 2, the specific steps of establishing the super node cluster are as follows:
1-1) starting;
1-2) calculating the Ability value Abiliity of the common node ii;
1-3) Ability to comply with Abiliity valuesiArranging the nodes in a descending order to obtain an ordered node list orderSuPeerList;
1-4) top list orderPeerListEach node is marked as a super node, and a mark symbol SPflag is 1, wherein Numsp represents the number of super nodes, N represents the total number of nodes, whereinRepresents a super node occupation ratio, and
1-5) Each super node k1Sending a detection packet to other super nodes to obtain a super node list orderPeerRttDist with transmission delay less than delta, and sequencing the list according to ascending time delay; delta represents the maximum acceptable time delay, and when the time delay is too large, a large amount of time is needed for periodic information exchange between nodes.
1-6) sequentially inquiring whether a super node k exists in the list orderPeerRttDistspWhether the size of the group is smaller than tau; if yes, the super node k1Adding kspThe group in which the gene belongs; otherwise, the super node k1Establishing a new group; τ represents the maximum number of nodes in a super node cluster, and if the number of super nodes in a cluster is too small, the benefit brought by the shared service is not very large, and when the number of nodes is too large, the nodes in the cluster are difficult to maintain.
1-7) finishing;
in step 1-2), the Ability value AbilityiThe evaluation index representing the service performance of the node is comprehensively determined by four factors of historical online time, bandwidth, storage capacity and exit frequency of the node. The longer the historical online time is, the longer the node can have the service duration, while the smaller the bandwidth and capacity of the node, although the node is online for a long time, the available bandwidth and capacity resources cannot be well contributed, and the node retreatsThe exit frequency represents the stability of the node on line, the higher the exit frequency is, the node cannot provide a stable service, and the capability value of the node is defined as follows:
Abilityi=ρ1OTime+ρ2Bandwidth+ρ3diskCapacity+ρ4exitFreq
(1)
in formula (1), OTime represents the historical online time of the node, Bandwidth represents the Bandwidth of the node, diskCapacity represents the disk capacity of the node, exitFreq represents the exit frequency of the node, and ρ1、ρ2、ρ3And ρ4Coefficients representing historical online duration, bandwidth, disk capacity, and historical exit frequency attributes, respectively, and ρ1+ρ2+ρ3+ρ4=1。
As shown in fig. 3, the greedy resource allocation algorithm sets a resource priority path, preferentially obtains resources from a super node group in a neighboring ISP domain, and then obtains resources from a CDN node, and specifically includes the following steps:
2-1) starting;
2-2) initialization Utility value Utility, t1,t2,t3,t4(t1,t2,t3,t4∈tω);
2-3) obtaining target resource Q lacking in ISP domainv;
2-4) acquiring neighbor ISPs list information, and executing steps 2-5) -2-7) on each neighbor ISP domain;
2-5) judging whether Q exists in the super node groupvIf not, executing step 6), if yes, updating QvAnd Utility, judgment QvIf the result is empty, executing the step 2-8), otherwise, executing the step 2-6);
2-6) judging whether Q exists in CDN nodevIf not, executing the step 2-7), and if so, updating the QvAnd Utility, judgment QvIf the result is empty, executing the step 2-8) if the result is empty, and executing the step 2-7) if the result is not empty;
2-7) judging whether Q exists in neighbor nodes of the CDN nodevIf not, executing the step 2-8), and if so, updating the QvAnd Utility;
2-8) ending;
in step 2-3), resource QvFrom super node clusters and CDNs nodes, QvThe calculation formula of (2) is as follows:
in the above formula (3), G represents a neighbor ISP set; g _ sps represents a set of super node clusters within an ISP domain, g _ CDN represents a set of CDN nodes within an ISP domain; xv,sE {0,1}, is a binary representation of the resource in the supernode, Xv,sWhen the value is 1, the resource v exists in the super node s, and otherwise, the resource v does not exist;is a resource binary representation of a neighboring super node, Yv,cIs a resource binary representation of the CDN node,is a resource binary representation of a neighboring CDN node.
In the step 2-7), the calculation formula of the Utility value Utility is as follows:
in the above formula (4), t1Representing the link cost, t, of acquiring resources from a super node s2Represents the link cost, t, of acquiring resources from s' neighbor nodes2=at1,t2>t1,(a=1,2,…,n),t3Represents the link cost, t, of acquiring resources from a CDN node4Represents the link cost, t, of acquiring resources from CDN neighbor nodes4=bt3,t4>t3,(b1, 2, …, n), a and b respectively represent the number of super node hops and the number of inter-CDN hops.
The cooperative cache management method comprises a cooperative cache mode of a super node in a super node group, a cooperative cache mode of the super node group and CDNs nodes and a cache replacement method;
the cooperative caching mode of the super nodes in the super node clusters is that each super node cluster is regarded as a whole, the nodes in the clusters can share resources, and the nodes in the clusters can store more video resources in limited capacity by caching different video resources, so that the diversity of the resources in the clusters is improved;
the method is characterized in that a coordinated cache mode of the super node group and the CDNs nodes is that target resources are divided into popular resources and secondary popular resources according to popularity degrees, the popular resources are stored in the super node group close to a user side preferentially, and the secondary popular resources are stored in the CDNs nodes secondly;
as shown in fig. 4, the cache replacement method specifically includes the following steps:
3-1) starting;
3-2) judging whether the current node has enough bandwidth, if not, executing the step 3-5), otherwise, judging whether the disk capacity of the node is enough, if so, directly caching the resource, otherwise, executing the step 3-3) -the step 3-4);
3-3) calculating the Energy value Energy of the video resource in the diskvSorting the video resources in a descending order according to the energy values;
3-4) replacing the video resource to be cached with the video resource with the minimum energy value;
3-5) finishing.
In step 3-3), the Energy value EnergyvThe method comprehensively considers the popularity, the scarcity, the request frequency and the video size of the video, carries out weighted summation on the four factors, and obtains the Energy value Energy of the videovThe calculation formula is as follows:
Energyv=σ·Pv+β·Scarcityv+γ·Reqv,T+λ·Sizev (5)
in the above formula (5), PvRepresenting the popularity of the video v, ScarcityvIndicating the degree of scarcity, Size, of the video vvRepresenting the size of the video, Reqv,TRepresents the number of times that the video v is requested in a period T, and sigma, beta, gamma and lambda represent the parameters of the popularity, the Scarcity, the request frequency and the video size of the video respectively, ScarcityvThe calculation formula of (a) is as follows:
num in the above formula (6)vRepresenting the number of copies of video v and TotalNum representing the number of copies of all videos.
Claims (6)
1. A method for managing an optimal collaborative cache under a CDN-P2P hybrid architecture is characterized in that the method comprises the steps of firstly constructing a super node group; secondly, constructing a corresponding mathematical model based on the global utility value according to the resource supply and demand relationship among ISPs domains, and optimizing the model by using a resource allocation greedy algorithm; finally, designing a collaborative cache management method based on the utility optimal model;
the super node cluster takes network time delay between super nodes as a distance between the two nodes, the super nodes establish the super node cluster based on a principle of proximity, the super nodes in the cluster are neighbor nodes, each super node maintains a group of neighbor super node information, a sharing area between the nodes is relatively transparent, the super nodes can acquire video resources from the sharing area of the neighbor nodes, and the specific steps of establishing the super node cluster are as follows:
1-1) starting;
1-2) calculating the Ability value Abiliity of the common node ii;
1-3) Ability to comply with Abiliity valuesiArranging the nodes in a descending order to obtain an ordered node list orderSuPeerList;
1-4) top list orderPeerListEach node is marked as a super node, and a mark symbol SPflag is 1, wherein Numsp represents the number of super nodes, N represents the total number of nodes, whereinRepresents a super node occupation ratio, and
1-5) Each super node k1Sending a detection packet to other super nodes to obtain a super node list orderPeerRttDist with transmission delay less than delta, and sequencing the list according to ascending time delay; δ represents the maximum acceptable delay;
1-6) sequentially inquiring whether a super node k exists in the list orderPeerRttDistspWhether the size of the group is smaller than tau; if yes, the super node k1Adding kspThe group in which the gene belongs; otherwise, the super node k1Establishing a new group; wherein τ represents the maximum number of nodes in the super node cluster;
1-7) finishing;
the mathematical model is established according to the supply and demand relationship among ISPs domains, and is optimized through a resource allocation greedy algorithm to ensure that the effectiveness of ISP domain request resources is optimal, wherein the mathematical model expression is as follows:
in the above formula (1), QvIndicating the missing target resource, tωRepresenting a link cost;
the resource allocation greedy algorithm is to set a resource priority acquisition path, preferentially acquire resources from a super node group of a neighbor ISP domain, and then acquire resources from a CDN node, and specifically comprises the following steps:
2-1) starting;
2-2) initialization utility value Utility,t1,t2,t3,t4(t1,t2,t3,t4∈tω);
2-3) obtaining target resource Q lacking in ISP domainv;
2-4) acquiring neighbor ISPs list information, and executing steps 2-5) -2-7) on each neighbor ISP domain;
2-5) judging whether Q exists in the super node groupvIf not, executing step 6), if yes, updating QvAnd Utility, judgment QvIf the result is empty, executing the step 2-8), otherwise, executing the step 2-6);
2-6) judging whether Q exists in CDN nodevIf not, executing the step 2-7), and if so, updating the QvAnd Utility, judgment QvIf the result is empty, executing the step 2-8) if the result is empty, and executing the step 2-7) if the result is not empty;
2-7) judging whether Q exists in neighbor nodes of the CDN nodevIf not, executing the step 2-8), and if so, updating the QvAnd Utility;
2-8) ending;
the cooperative cache management method comprises a cooperative cache mode of a super node in a super node group, a cooperative cache mode of the super node group and CDNs nodes and a cache replacement method;
the cooperative caching mode of the super nodes in the super node clusters is that each super node cluster is regarded as a whole, the nodes in the clusters can share resources, and the nodes in the clusters can store more video resources in limited capacity by caching different video resources;
the method is characterized in that a coordinated cache mode of the super node group and the CDNs nodes is that target resources are divided into popular resources and secondary popular resources according to popularity degrees, the popular resources are stored in the super node group close to a user side preferentially, and the secondary popular resources are stored in the CDNs nodes secondly;
the cache replacement method specifically comprises the following steps:
3-1) starting;
3-2) judging whether the current node has enough bandwidth, if not, executing the step 3-5), otherwise, judging whether the disk capacity of the node is enough, if so, directly caching the resource, otherwise, executing the step 3-3) -the step 3-4);
3-3) calculating the Energy value Energy of the video resource in the diskvSorting the video resources in a descending order according to the energy values;
3-4) replacing the video resource to be cached with the video resource with the minimum energy value;
3-5) finishing.
2. The method of claim 1, wherein in step 1-2), the capability value Ability is obtained by performing the most effective cooperative cache management under the CDN-P2P hybrid architectureiThe evaluation index representing the service performance of the node is comprehensively determined by four factors of historical online time, bandwidth, storage capacity and exit frequency of the node, and the capacity value of the node is defined as follows:
Abilityi=ρ1OTime+ρ2Bandwidth+ρ3diskCapacity+ρ4exitFreq (2)
in the formula (2), OTime represents the historical online time of the node, Bandwidth represents the Bandwidth of the node, diskCapacity represents the disk capacity of the node, exitFreq represents the exit frequency of the node, and rho1、ρ2、ρ3And ρ4Coefficients representing historical online duration, bandwidth, disk capacity, and historical exit frequency attributes, respectively, and ρ1+ρ2+ρ3+ρ4=1。
3. The method of claim 1, wherein the mathematical model has an expression that satisfies the following constraints:
constraint 1:ensuring that the available bandwidth of the node is not less than 0 in the sending period and bandwidthi,avaiIs the available bandwidth of node i, sjIs the length of video segment j, Γ is the maximum allowed transmission duration;
4. The method of claim 1, wherein in step 2-3), resource Q is used as a resource for managing the most effective cooperative cache in the CDN-P2P hybrid architecturevFrom super node clusters and CDNs nodes, QvThe calculation formula of (2) is as follows:
in the above formula (3), G represents a neighbor ISP set; g _ sps represents a set of super node clusters within an ISP domain, g _ CDN represents a CDN node within an ISP domain, Xv,sE {0,1}, is a binary representation of the resource in the supernode, Xv,sWhen the value is 1, the resource v exists in the super node s, and otherwise, the resource v does not exist;is a resource binary representation of a neighboring super node, Yv,cIs a resource binary representation of the CDN node,is a resource binary representation of a neighboring CDN node.
5. The method for managing the optimal collaborative cache under the CDN-P2P hybrid architecture according to claim 1, wherein in step 2-7), a calculation formula of the Utility value Utility is:
in the above formula (4), t1Representing the link cost, t, of acquiring resources from a super node s2Represents the link cost, t, of acquiring resources from s' neighbor nodes2=at1,t2>t1,(a=1,2,…,n),t3Represents the link cost, t, of acquiring resources from a CDN node4Represents the link cost, t, of acquiring resources from CDN neighbor nodes4=bt3,t4>t3And b is 1, 2, …, n, and a and b respectively represent the number of hops between super nodes and the number of hops between CDNs.
6. The method of claim 1, wherein in step 3-3), the Energy value Energy is used in the method for managing the most effective collaborative cache in the CDN-P2P hybrid architecturevThe method comprehensively considers the popularity, the scarcity, the request frequency and the video size of the video, carries out weighted summation on the four factors, and obtains the Energy value Energy of the videovThe calculation formula is as follows:
Energyv=σ·Pv+β·Scarcityv+γ·Reqv,T+λ·Sizev (5)
in the above formula (5), PvRepresenting the popularity of the video v, ScarcityvIndicating the degree of scarcity, Size, of the video vvRepresenting the size of the video, Reqv,TRepresents the number of times that the video v is requested in a period T, and sigma, beta, gamma and lambda represent the parameters of the popularity, the Scarcity, the request frequency and the video size of the video respectively, ScarcityvThe calculation formula of (a) is as follows:
num in the above formula (6)vRepresenting the number of copies of video v and TotalNum representing the number of copies of all videos.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110709682.3A CN113453038B (en) | 2021-06-25 | 2021-06-25 | Effectiveness optimal collaborative cache management method under CDN-P2P hybrid architecture |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110709682.3A CN113453038B (en) | 2021-06-25 | 2021-06-25 | Effectiveness optimal collaborative cache management method under CDN-P2P hybrid architecture |
Publications (2)
Publication Number | Publication Date |
---|---|
CN113453038A true CN113453038A (en) | 2021-09-28 |
CN113453038B CN113453038B (en) | 2022-03-29 |
Family
ID=77812739
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110709682.3A Active CN113453038B (en) | 2021-06-25 | 2021-06-25 | Effectiveness optimal collaborative cache management method under CDN-P2P hybrid architecture |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN113453038B (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114095573A (en) * | 2021-10-19 | 2022-02-25 | 陕西悟空云信息技术有限公司 | Content copy placement method of CDN-P2P network based on edge cache |
CN114124939A (en) * | 2021-11-25 | 2022-03-01 | 北京奇艺世纪科技有限公司 | Pre-cache file processing method and device and file pre-cache system |
CN114124971A (en) * | 2021-10-19 | 2022-03-01 | 陕西悟空云信息技术有限公司 | Content copy placement method of CDN-P2P network based on edge cache |
CN116405563A (en) * | 2023-06-08 | 2023-07-07 | 湖南快乐阳光互动娱乐传媒有限公司 | Resource acquisition method and system based on point-to-point content distribution network |
CN113992653B (en) * | 2021-10-19 | 2023-09-15 | 西安邮电大学 | CDN-P2P network content downloading, pre-storing and replacing method based on edge cache |
WO2024212767A1 (en) * | 2023-04-12 | 2024-10-17 | 北京字跳网络技术有限公司 | Video pushing method and apparatus, device, and storage medium |
Citations (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101159580A (en) * | 2007-11-29 | 2008-04-09 | 中国电信股份有限公司 | Content P2P method and system in content distribution network |
JP2009042944A (en) * | 2007-08-07 | 2009-02-26 | Grid Solutions Inc | File distribution system |
US20090198833A1 (en) * | 2007-12-17 | 2009-08-06 | Alcatel-Lucent Via The Electronic Patent Assignment System (Epas). | Method for distributing content data packages originated by users of a super peer-to-peer network |
CN101667999A (en) * | 2008-09-04 | 2010-03-10 | 华为技术有限公司 | Method and system for transmitting peer-to-peer broadcast stream, data signature device and client |
KR20100042778A (en) * | 2008-10-17 | 2010-04-27 | 주식회사 아이토비 | Content delivery network based on multiple p2p protocol |
US20110099226A1 (en) * | 2009-03-17 | 2011-04-28 | Nec (China) Co., Ltd. | Method of requesting for location information of resources on network, user node and server for the same |
US20120054282A1 (en) * | 2010-08-27 | 2012-03-01 | Industrial Technology Research Institute | Architecture and method for hybrid peer to peer/client-server data transmission |
CN102547395A (en) * | 2011-12-31 | 2012-07-04 | 上海聚力传媒技术有限公司 | Method and device for determining video data source of network player |
US20130212226A1 (en) * | 2007-11-05 | 2013-08-15 | Limelight Networks, Inc. | Origin request with peer fulfillment |
CN103475719A (en) * | 2013-09-12 | 2013-12-25 | 北京科技大学 | Content distribution method for minimizing cross-domain flows in CDN-P2P fusion network |
CN103873282A (en) * | 2012-12-14 | 2014-06-18 | 中国移动通信集团福建有限公司 | Method and device for realizing P2P data traffic optimization |
CN107181734A (en) * | 2017-04-07 | 2017-09-19 | 南京邮电大学 | A kind of stream media buffer replacing method of the CDN P2P network architectures |
CN107483614A (en) * | 2017-08-31 | 2017-12-15 | 京东方科技集团股份有限公司 | Content scheduling method and communication network based on CDN Yu P2P networks |
CN110336843A (en) * | 2015-02-24 | 2019-10-15 | 深圳梨享计算有限公司 | A kind of content distribution method, central node and fringe node for crowdsourcing |
CN111200657A (en) * | 2020-01-03 | 2020-05-26 | 网宿科技股份有限公司 | Method for managing resource state information and resource downloading system |
CN111212114A (en) * | 2019-12-19 | 2020-05-29 | 网宿科技股份有限公司 | Method and device for downloading resource file |
CN111372100A (en) * | 2020-04-21 | 2020-07-03 | 白杨 | End-to-end content distribution network system and distribution method based on distributed election |
US20210044642A1 (en) * | 2019-07-31 | 2021-02-11 | Theta Labs, Inc. | Methods and systems for blockchain incentivized data streaming and delivery over a decentralized network |
CN112565796A (en) * | 2019-09-25 | 2021-03-26 | 北京硬核聚视科技有限公司 | Video content decentralized access method and system |
-
2021
- 2021-06-25 CN CN202110709682.3A patent/CN113453038B/en active Active
Patent Citations (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2009042944A (en) * | 2007-08-07 | 2009-02-26 | Grid Solutions Inc | File distribution system |
US20130212226A1 (en) * | 2007-11-05 | 2013-08-15 | Limelight Networks, Inc. | Origin request with peer fulfillment |
CN101159580A (en) * | 2007-11-29 | 2008-04-09 | 中国电信股份有限公司 | Content P2P method and system in content distribution network |
US20090198833A1 (en) * | 2007-12-17 | 2009-08-06 | Alcatel-Lucent Via The Electronic Patent Assignment System (Epas). | Method for distributing content data packages originated by users of a super peer-to-peer network |
CN101667999A (en) * | 2008-09-04 | 2010-03-10 | 华为技术有限公司 | Method and system for transmitting peer-to-peer broadcast stream, data signature device and client |
KR20100042778A (en) * | 2008-10-17 | 2010-04-27 | 주식회사 아이토비 | Content delivery network based on multiple p2p protocol |
US20110099226A1 (en) * | 2009-03-17 | 2011-04-28 | Nec (China) Co., Ltd. | Method of requesting for location information of resources on network, user node and server for the same |
US20120054282A1 (en) * | 2010-08-27 | 2012-03-01 | Industrial Technology Research Institute | Architecture and method for hybrid peer to peer/client-server data transmission |
CN102547395A (en) * | 2011-12-31 | 2012-07-04 | 上海聚力传媒技术有限公司 | Method and device for determining video data source of network player |
CN103873282A (en) * | 2012-12-14 | 2014-06-18 | 中国移动通信集团福建有限公司 | Method and device for realizing P2P data traffic optimization |
CN103475719A (en) * | 2013-09-12 | 2013-12-25 | 北京科技大学 | Content distribution method for minimizing cross-domain flows in CDN-P2P fusion network |
CN110336843A (en) * | 2015-02-24 | 2019-10-15 | 深圳梨享计算有限公司 | A kind of content distribution method, central node and fringe node for crowdsourcing |
CN107181734A (en) * | 2017-04-07 | 2017-09-19 | 南京邮电大学 | A kind of stream media buffer replacing method of the CDN P2P network architectures |
CN107483614A (en) * | 2017-08-31 | 2017-12-15 | 京东方科技集团股份有限公司 | Content scheduling method and communication network based on CDN Yu P2P networks |
US20210044642A1 (en) * | 2019-07-31 | 2021-02-11 | Theta Labs, Inc. | Methods and systems for blockchain incentivized data streaming and delivery over a decentralized network |
CN112565796A (en) * | 2019-09-25 | 2021-03-26 | 北京硬核聚视科技有限公司 | Video content decentralized access method and system |
CN111212114A (en) * | 2019-12-19 | 2020-05-29 | 网宿科技股份有限公司 | Method and device for downloading resource file |
CN111200657A (en) * | 2020-01-03 | 2020-05-26 | 网宿科技股份有限公司 | Method for managing resource state information and resource downloading system |
CN111372100A (en) * | 2020-04-21 | 2020-07-03 | 白杨 | End-to-end content distribution network system and distribution method based on distributed election |
Non-Patent Citations (3)
Title |
---|
ZHI HUI LU: "Scalable and Reliable Live Streaming Service through Coordinating CDN and P2P", 《2011 IEEE 17TH INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED SYSTEMS》 * |
姜小平: "CDN与P2P混合网络中Peer节点负载均衡研究", 《中国优秀硕士学位论文全文库》 * |
李成森: "一种基于节点信息的负载均衡算法", 《桂林电子科技大学学报》 * |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114095573A (en) * | 2021-10-19 | 2022-02-25 | 陕西悟空云信息技术有限公司 | Content copy placement method of CDN-P2P network based on edge cache |
CN114124971A (en) * | 2021-10-19 | 2022-03-01 | 陕西悟空云信息技术有限公司 | Content copy placement method of CDN-P2P network based on edge cache |
CN113992653B (en) * | 2021-10-19 | 2023-09-15 | 西安邮电大学 | CDN-P2P network content downloading, pre-storing and replacing method based on edge cache |
CN114124971B (en) * | 2021-10-19 | 2023-11-24 | 陕西悟空云信息技术有限公司 | Content copy placement method of CDN-P2P network based on edge cache |
CN114095573B (en) * | 2021-10-19 | 2023-11-28 | 陕西悟空云信息技术有限公司 | Content copy placement method of CDN-P2P network based on edge cache |
CN114124939A (en) * | 2021-11-25 | 2022-03-01 | 北京奇艺世纪科技有限公司 | Pre-cache file processing method and device and file pre-cache system |
CN114124939B (en) * | 2021-11-25 | 2024-03-26 | 北京奇艺世纪科技有限公司 | Method and device for processing pre-cached file and file pre-caching system |
WO2024212767A1 (en) * | 2023-04-12 | 2024-10-17 | 北京字跳网络技术有限公司 | Video pushing method and apparatus, device, and storage medium |
CN116405563A (en) * | 2023-06-08 | 2023-07-07 | 湖南快乐阳光互动娱乐传媒有限公司 | Resource acquisition method and system based on point-to-point content distribution network |
CN116405563B (en) * | 2023-06-08 | 2023-08-18 | 湖南快乐阳光互动娱乐传媒有限公司 | Resource acquisition method and system based on point-to-point content distribution network |
Also Published As
Publication number | Publication date |
---|---|
CN113453038B (en) | 2022-03-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN113453038B (en) | Effectiveness optimal collaborative cache management method under CDN-P2P hybrid architecture | |
JP5390413B2 (en) | Hierarchically clustered P2P streaming system | |
US7995473B2 (en) | Content delivery system for digital object | |
US20080040420A1 (en) | Content distribution network | |
Shen et al. | A DHT-aided chunk-driven overlay for scalable and efficient peer-to-peer live streaming | |
EP2252057B1 (en) | Method and system for storing and distributing electronic content | |
CN102158767B (en) | Scalable-coding-based peer to peer live media streaming system | |
Zhang et al. | A distributed multichannel demand-adaptive P2P VoD system with optimized caching and neighbor-selection | |
CN103179191B (en) | P2P network control device and P2P network managing and control system | |
Liang et al. | ipass: Incentivized peer-assisted system for asynchronous streaming | |
CN102821316A (en) | Improved video on demand (VOD) transmission method based on peer-to-peer computing core algorithm | |
Kim et al. | Efficient peer-to-peer overlay networks for mobile IPTV services | |
CN109194767A (en) | A kind of flow medium buffer dispatching method suitable for mixing network | |
Jiang et al. | A replica placement algorithm for hybrid CDN-P2P architecture | |
Muscat et al. | A Hybrid CDN-P2P Architecture for Live Video Streaming | |
Zhang et al. | Geo-edge: Geographical resource allocation on edge caches for video-on-demand streaming | |
CN110139126B (en) | Mobile video system resource sharing method based on user interaction behavior perception | |
CN114095573A (en) | Content copy placement method of CDN-P2P network based on edge cache | |
CN107800567B (en) | Method for establishing P2P streaming media network topology model of mixed mode | |
Liu et al. | Reducing Video Transmission Cost of the Cloud Service Provider with QoS-Guaranteed | |
Jia et al. | Modeling and optimization of bandwidth supply performance for cloud-assisted video systems under flash crowd | |
Tu et al. | An efficient data scheduling scheme for P2P storage-constrained IPTV system | |
Huang et al. | Joint optimization of content replication and server selection for video-on-demand | |
Lee et al. | A vEB-tree-based architecture for interactive video on demand services in peer-to-peer networks | |
Sivakumar et al. | Artificial Bee Colony Based Data Scheduling for Peer to Peer Network Video on Demand System |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |