CN110769000B - Dynamic compression prediction control method of continuous monitoring data in unstable network transmission - Google Patents
Dynamic compression prediction control method of continuous monitoring data in unstable network transmission Download PDFInfo
- Publication number
- CN110769000B CN110769000B CN201911056258.2A CN201911056258A CN110769000B CN 110769000 B CN110769000 B CN 110769000B CN 201911056258 A CN201911056258 A CN 201911056258A CN 110769000 B CN110769000 B CN 110769000B
- Authority
- CN
- China
- Prior art keywords
- time
- data
- compression ratio
- sequence
- transmission
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 101
- 238000007906 compression Methods 0.000 title claims abstract description 100
- 230000006835 compression Effects 0.000 title claims abstract description 100
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000012544 monitoring process Methods 0.000 title claims abstract description 10
- 238000005457 optimization Methods 0.000 claims abstract description 24
- 230000009365 direct transmission Effects 0.000 claims abstract description 14
- 230000006837 decompression Effects 0.000 claims abstract description 12
- 238000004364 calculation method Methods 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 claims description 27
- 230000008859 change Effects 0.000 claims description 15
- 238000005070 sampling Methods 0.000 claims description 10
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 230000035772 mutation Effects 0.000 claims description 6
- 238000012360 testing method Methods 0.000 claims description 6
- 238000009825 accumulation Methods 0.000 claims description 4
- 230000002068 genetic effect Effects 0.000 claims description 4
- 238000011156 evaluation Methods 0.000 claims description 3
- 238000000547 structure data Methods 0.000 claims description 3
- YHXISWVBGDMDLQ-UHFFFAOYSA-N moclobemide Chemical compound C1=CC(Cl)=CC=C1C(=O)NCCN1CCOCC1 YHXISWVBGDMDLQ-UHFFFAOYSA-N 0.000 claims description 2
- 150000001875 compounds Chemical class 0.000 claims 1
- 238000007689 inspection Methods 0.000 claims 1
- 238000013144 data compression Methods 0.000 abstract description 6
- 238000004590 computer program Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000001514 detection method Methods 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 235000006679 Mentha X verticillata Nutrition 0.000 description 2
- 235000002899 Mentha suaveolens Nutrition 0.000 description 2
- 235000001636 Mentha x rotundifolia Nutrition 0.000 description 2
- 238000013178 mathematical model Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/04—Protocols for data compression, e.g. ROHC
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
一种连续型监测数据在非稳定网络传输中的动态压缩预测控制方法,包括有S1数据采集步骤、S2建模计算压缩比步骤、S3压缩传输步骤和S4解压步骤。相比于基于无损数据压缩的传输优化方法而言,采用基于动态压缩比的传输耗时优化方法来降低传输耗时,大量节省了压缩、传输和解压的耗时;将数据分时间周期进行处理,通过预测网速的方式来计算压缩比,适用于非稳定网络环境;通过计算原始数据在预测网速环境下的直接传输耗时来设立约束条件,结合以耗时最少为目标的函数,计算预测网速压缩比和当前网速压缩比进行比较,确保最优压缩比的准确性。
A dynamic compression prediction control method of continuous monitoring data in unstable network transmission includes S1 data acquisition step, S2 modeling calculation compression ratio step, S3 compression transmission step and S4 decompression step. Compared with the transmission optimization method based on lossless data compression, the transmission time-consuming optimization method based on the dynamic compression ratio is adopted to reduce the transmission time-consuming, which greatly saves the time-consuming of compression, transmission and decompression; the data is processed in time periods. , calculate the compression ratio by predicting the network speed, which is suitable for the unstable network environment; establish constraints by calculating the direct transmission time of the original data in the predicted network speed environment, and combine the function with the least time-consuming as the goal to calculate The predicted network speed compression ratio is compared with the current network speed compression ratio to ensure the accuracy of the optimal compression ratio.
Description
技术领域technical field
本发明涉及数据压缩技术领域,特别是一种非稳定网络中的动态压缩预测控制方法。The invention relates to the technical field of data compression, in particular to a dynamic compression prediction control method in an unstable network.
背景技术Background technique
数据压缩是一种广泛应用、保证原始数据质量又减小数据规模的处理方法,然而压缩和解压缩本身需要耗费时间,与压缩比具有密切的关联关系。若采用数据压缩来提升网络传输效率,需要考虑如下问题:Data compression is a widely used processing method to ensure the quality of the original data and reduce the size of the data. However, compression and decompression are time-consuming and closely related to the compression ratio. If data compression is used to improve network transmission efficiency, the following issues need to be considered:
1、整个监测数据的传输任务包括从底层传感器采集到直到上载至目标对象并还原成原始数据的全过程,在这个过程中,仅仅根据当前网络环境及数据占比大小来选取最优压缩比并不可靠,在忽略了数据传输速率的波动频率对压缩控制决策的影响下,后续传输速率的变化会受当前压缩策略较大影响,而受制于连续型时间序列数据的持续特性,这个影响可能或导致传输效率不增反降。1. The transmission task of the entire monitoring data includes the whole process from the acquisition of the underlying sensor to uploading to the target object and restoring it to the original data. In this process, the optimal compression ratio is only selected according to the current network environment and the proportion of data. Unreliable, ignoring the influence of the fluctuation frequency of the data transmission rate on the compression control decision, the subsequent change of the transmission rate will be greatly affected by the current compression strategy, and subject to the continuous characteristics of continuous time series data, this influence may or As a result, the transmission efficiency does not increase but decreases.
2、若采用动态压缩比,需要充分考虑压缩比的更新频率,这个频率与传输速率的变化有关。由于数据一经发送,其内部结构及大小将不能再被改变,即待传输数据的占比大小在整个传输过程当中是确定的,若此时后续传输速率的频繁变化可能会使得此时的最优压缩比无法满足当前传输阶段的整体耗时优化需求,直接导致压缩动态控制调整与实际数据传输需求不对等,优化决策滞后乃至失效。2. If a dynamic compression ratio is used, the update frequency of the compression ratio needs to be fully considered, which is related to the change of the transmission rate. Once the data is sent, its internal structure and size can no longer be changed, that is, the proportion of the data to be transmitted is determined in the entire transmission process. If the subsequent transmission rate changes frequently at this time, it may make the optimal The compression ratio cannot meet the overall time-consuming optimization requirements of the current transmission stage, which directly leads to the asymmetry between the compression dynamic control adjustment and the actual data transmission requirements, and the optimization decision lags or even fails.
在CN103957582A中公开了一种名称为“无线传感器网络自适应压缩方法”的发明专利,该专利公开了根据数据类型、精度要求选择压缩算法、预测平均压缩比、预测执行压缩的平均时间、以能耗最优为目标建立模型、求解最佳压缩策略等技术手段。CN103957582A discloses an invention patent named "wireless sensor network adaptive compression method", which discloses selecting compression algorithms according to data types and precision requirements, predicting the average compression ratio, predicting the average time to perform compression, and The technical means such as establishing a model for the goal of optimal consumption and solving the optimal compression strategy.
该对比文件的目的是以最优能耗为目标建立数学模型并求解,并没未以时间最优为目标建模,因此并未公开由压缩、传输、解压组成的数学模型公式,约束条件设立也不相同;该对比文件在计算过程中,并未考虑到网络状况不稳定的情况,未对当前网络进行预测,无法适用于非稳定网络。The purpose of this comparison document is to establish a mathematical model and solve it with the goal of optimal energy consumption. It does not model with the goal of time optimization. Therefore, it does not disclose the mathematical model formula composed of compression, transmission and decompression, and the constraints are established. It is not the same; the comparison document does not take into account the unstable network conditions during the calculation process, and does not predict the current network, so it cannot be applied to unstable networks.
发明内容SUMMARY OF THE INVENTION
本发明的目的就是提供一种连续型监测数据在非稳定网络传输中的动态压缩预测控制方法,它将数据分周期进行压缩处理,通过下一个周期的网速预测,并以传输耗时最短为目标建模计算最优压缩比,大量节省了数据压缩、传输和解压的耗时。The purpose of the present invention is to provide a dynamic compression prediction control method for continuous monitoring data in unstable network transmission, which compresses the data in cycles, predicts the network speed of the next cycle, and takes the shortest transmission time as Target modeling calculates the optimal compression ratio, saving a lot of time in data compression, transmission and decompression.
本发明的目的是通过这样的技术方案实现的,它包括有S1数据采集步骤、S2建模计算压缩比步骤、S3压缩传输步骤和S4解压步骤:The purpose of the present invention is achieved through such a technical solution, which includes S1 data acquisition step, S2 modeling calculation compression ratio step, S3 compression transmission step and S4 decompression step:
所述S1数据采集步骤获取当前时间周期内待传输的连续型数据和网络速率历史样本;时间周期针对信息系统在一段时间内积累的数据包需要发送,其发送时间要小于数据积累周期 The S1 data collection step obtains the current time period Continuous data and network rate historical samples to be transmitted; time period For information systems over a period of time The accumulated data packets need to be sent, and the sending time should be less than the data accumulation period
所述S2建模计算压缩比步骤包括有:S2-1、根据网络速率历史样本计算下一个时间周期的预测网速;S2-2、根据预测网速以耗时最少为目标建模求解最优压缩比;The step of S2 modeling and calculating the compression ratio includes: S2-1, calculating the next time period according to the network rate historical sample The predicted network speed; S2-2, according to the predicted network speed, modeling and solving the optimal compression ratio with the least time consuming as the goal;
步骤S2-1中所述根据网络速率历史样本采用时间序列预测法计算下一个时间周期的预测网速,具体步骤如下:In step S2-1, the time series prediction method is used to calculate the next time period according to the historical samples of the network rate The specific steps are as follows:
S2-1、历史网络序列的平稳性检验:根据该网络序列的均值、方差和相关系数来判断该时间序列是否是平稳性时间序列;S2-1. Stationarity test of historical network series: judge whether the time series is a stationary time series according to the mean, variance and correlation coefficient of the network series;
S2-2、序列差分运算:如果该序列是非平稳时间序列,则对这些序列进行差分运算,直到序列满足平稳性为止;S2-2. Sequence difference operation: if the sequence is a non-stationary time series, perform a difference operation on these sequences until the sequence satisfies stationarity;
S2-3、序列的模型拟合:根据样本的相关系数ACF、偏相关系数PACF和周期性对差分后的序列进行求和自回归移动平均模型ARIMA(p,d,q)进行拟合得到时间序列函数d(t);S2-3. Model fitting of the sequence: According to the correlation coefficient ACF, partial correlation coefficient PACF and periodicity of the sample, the differenced sequence is summed and the autoregressive moving average model ARIMA(p, d, q) is fitted to obtain the time sequence function d(t);
S2-4、残差序列的检验:对时间序列函数d(t)的残差序列进行白噪声检验,若残差序列不满足白噪声序列,则返回步骤S2-3重新拟合模型,直到残差序列为白噪声为止;S2-4. Test of residual sequence: perform white noise test on the residual sequence of the time series function d(t), if the residual sequence does not satisfy the white noise sequence, return to step S2-3 to re-fit the model until the residual until the difference sequence is white noise;
S2-5、模型的优化:对时时间序列拟合模型函数d(t)采用最小信息准则进行模型优化从而得到更新后的模型函数d(t);S2-5. Model optimization: the time series fitting model function d(t) is optimized by using the minimum information criterion to obtain the updated model function d(t);
S2-6、序列的预测:用时间序列拟合函数d(t+l)来预测出下一时间段的网络速率。S2-6. Prediction of series: fitting function d(t+l) with time series to predict the next time period network speed.
进一步,步骤S2-2中所述据预测网速以耗时最少为目标建模求解最优压缩比的具体方法如下:Further, according to the predicted network speed described in step S2-2, the specific method for modeling and solving the optimal compression ratio with the least time consuming as the target is as follows:
S2-2-1、计算原始数据在预测网速环境下的直接传输耗时 S2-2-1. Calculate the direct transmission time of the original data in the predicted network speed environment
S2-2-2、建立以整体传输耗时为最优目标的优化模型,结合S2-2-1中计算出的直接传输耗时设立约束条件;S2-2-2, establish an optimization model with the overall transmission time as the optimal goal, and combine the direct transmission time calculated in S2-2-1 establish constraints;
S2-2-3、在当前网速环境下,求解压缩比和整体传输时耗t;S2-2-3. Under the current network speed environment, solve the compression ratio and the overall transmission time consumption t;
S2-2-4、在预测网速环境下,求解压缩比和整体传输时耗t;S2-2-4, in the prediction network speed environment, solve the compression ratio and the overall transmission time consumption t;
S2-2-5、对比S2-2-3和S2-2-4的和整体传输时耗t,选择较小整体传输时耗t对应的压缩比作为最优压缩比。S2-2-5, compare the total transmission time consumption t of S2-2-3 and S2-2-4, and select the compression ratio corresponding to the smaller overall transmission time consumption t as the optimal compression ratio.
进一步,步骤S2-2-1所述计算原始数据在预测网速环境下的直接传输耗时的具体方法为:Further, the specific method for calculating the time-consuming direct transmission of the original data in the predicted network speed environment described in step S2-2-1 is:
判断数据传输速率预测序列d(t)是否可积,若可积根据公式可以直接求出原始传输耗时 Determine whether the data transmission rate prediction sequence d(t) is integrable, if it is, according to the formula The original transmission time can be directly calculated
若不可积,由于连续数据在采样周期△t内速率的变化较小可以忽略不计,对d(t)的离散序列集进行分段积分,即求解出n,进而获取原始传输耗时 If it is not integrable, since the change of the rate of continuous data within the sampling period Δt is small and can be ignored, the discrete sequence set of d(t) is integrated in pieces, namely Solve n, and then obtain the original transmission time
式中,ti表示采样时刻,Di表示第i个采样周期内的速率,n表示采样时间点的个数。In the formula, t i represents the sampling time, D i represents the rate in the ith sampling period, and n represents the number of sampling time points.
进一步,步骤S2-2-2所述建立以整体传输耗时为最优目标的优化模型,结合S2-2-1中计算出的直接传输耗时设立约束条件的具体方法为:Further, as described in step S2-2-2, an optimization model with the overall transmission time consumption as the optimal target is established, combined with the direct transmission time consumption calculated in S2-2-1 The specific method of setting up constraints is as follows:
针对多次试验可得到单位原始数据包压缩时间t1与压缩比p的关系为t1=T1(p)和单位原始数据包解压时间t2与压缩比p的关系为t2=T2(p);构建目标函数For many experiments, it can be obtained that the relationship between the unit original data packet compression time t 1 and the compression ratio p is t 1 =T 1 (p) and the unit original data packet decompression time t 2 and the compression ratio p The relationship between the compression ratio p is t 2 =T 2 (p); build the objective function
mint=t1+t*+t2=QT1(p)+t*+QT2(p)mint=t 1 +t*+t 2 =QT 1 (p)+t*+QT 2 (p)
分别设定最大压缩比和最小压缩比为Pmax和Pmin,设立约束条件:Set the maximum compression ratio and the minimum compression ratio as P max and P min respectively, and set up constraints:
式中,为未压缩数据原始传输耗时,网速序列预测周期,t*为压缩后数据传输耗时,Q为待传输数据的规模大小。In the formula, Time consuming for raw transmission of uncompressed data, The prediction period of the network speed sequence, t * is the time taken for data transmission after compression, and Q is the size of the data to be transmitted.
进一步,步骤S2-2-3和步骤S2-2-4采用遗传算法来求解最优压缩比,具体方法如下:Further, step S2-2-3 and step S2-2-4 adopt genetic algorithm to solve the optimal compression ratio, and the specific method is as follows:
1、编码:根据各模态的压缩比范围为[Pmin,Pmax]采用长度为k的二进制编码,共有2k种不同编码,相邻的编码间隔为 1. Coding: According to the compression ratio range of each mode [P min , P max ], binary coding with length k is used, and there are 2 k different codes. The adjacent coding interval is
2、初始种群的生成:随机生成N个串结构数据作为初始种群开始进化,即产生N个以二进制为编码的初始压缩比p作为初始种群;2. Generation of the initial population: randomly generate N string structure data as the initial population to start evolution, that is, generate N initial compression ratios p encoded in binary as the initial population;
3、适应度评估:选择适应度函数为t=t1+t*+t2=QT1(p)+t*+QT2(p),并计算种群中每一个初始个体的适应度函数值;3. Fitness evaluation: select the fitness function as t=t 1 +t*+t 2 =QT 1 (p)+t*+QT 2 (p), and calculate the fitness function value of each initial individual in the population ;
4、自然选择:个体中适应度函数值t满足约束条件且小于种群平均值的个体保留下来作为适应性强的个体添加至新种群中;4. Natural selection: individuals whose fitness function value t satisfies the constraints and are smaller than the population average are retained as individuals with strong adaptability and added to the new population;
5、交叉和变异:交叉是将个体与个体之间的部分编码进行交换,变异是在种群中随机选择一个个体以很小的概率随机改变编码中的某个字符,交叉和变异的目的都是得到新的个体添加至新的种群中;5. Crossover and mutation: Crossover is to exchange part of the code between individuals and individuals. Mutation is to randomly select an individual in the population to randomly change a character in the code with a small probability. The purpose of crossover and mutation are both. Get new individuals to add to the new population;
6、是否停止进化:终止条件:进化次数是达到设定次数或者种群适应度的方差小于设定方差,若满足该终止条件,则停止进化,否则将更新后种群返回至步骤3;6. Whether to stop the evolution: Termination condition: the number of evolution is the set number or the variance of the population fitness is less than the set variance, if the termination condition is met, the evolution will be stopped, otherwise the updated population will return to step 3;
7、解码:将选择留下的最优个体通过编码转换成原始参数作为最优压缩比,同时得到最优适应度。7. Decoding: Convert the optimal individual left by selection into the original parameters through encoding as the optimal compression ratio, and obtain the optimal fitness at the same time.
进一步,时间周期大于系统数据发送时间。Further, the time period It is longer than the system data sending time.
由于采用了上述技术方案,本发明具有如下的优点:Owing to adopting the above-mentioned technical scheme, the present invention has the following advantages:
1、本发明相比于基于无损数据压缩的传输优化方法而言,采用基于动态压缩比的传输耗时优化方法来降低传输耗时,大量节省了压缩、传输和解压的耗时;1. Compared with the transmission optimization method based on lossless data compression, the present invention adopts the transmission time-consuming optimization method based on the dynamic compression ratio to reduce the transmission time-consuming, and greatly saves the time-consuming of compression, transmission and decompression;
2、本发明将数据分时间周期进行处理,通过预测网速的方式来计算压缩比,适用于非稳定网络环境;2. The present invention processes the data in time periods, and calculates the compression ratio by predicting the network speed, which is suitable for unstable network environments;
3、本发明通过计算原始数据在预测网速环境下的直接传输耗时来设立约束条件,结合以耗时最少为目标的函数,计算预测网速压缩比和当前网速压缩比进行比较,确保最优压缩比的准确性。3. The present invention sets up constraints by calculating the direct transmission time of the original data in the predicted network speed environment, and combines the function with the least time consuming as the goal to calculate the predicted network speed compression ratio and compare the current network speed compression ratio to ensure that the The accuracy of the optimal compression ratio.
本发明的其他优点、目标和特征在某种程度上将在随后的说明书中进行阐述,并且在某种程度上,基于对下文的考察研究对本领域技术人员而言将是显而易见的,或者可以从本发明的实践中得到教导。本发明的目标和其他优点可以通过下面的说明书和权利要求书来实现和获得。Other advantages, objects, and features of the present invention will be set forth in the description that follows, and will be apparent to those skilled in the art based on a study of the following, to the extent that is taught in the practice of the present invention. The objectives and other advantages of the present invention may be realized and attained by the following description and claims.
附图说明Description of drawings
本发明的附图说明如下。The accompanying drawings of the present invention are described below.
图1为本发明的流程示意图;Fig. 1 is the schematic flow chart of the present invention;
图2为电力系统的连续型运行检测数据传输的结构层级图;Fig. 2 is the structural level diagram of continuous operation detection data transmission of the power system;
图3为时间序列拟合预测流程图。Figure 3 is a flow chart of time series fitting and forecasting.
具体实施方式Detailed ways
下面结合附图和实施例对本发明作进一步说明。The present invention will be further described below with reference to the accompanying drawings and embodiments.
如图1、图2和图3所示,以电力系统的连续型运行检测数据在连续型监测数据在非稳定网络传输中的动态压缩预测控制方法为实施例进行说明,包括以下步骤:As shown in Figure 1, Figure 2 and Figure 3, the dynamic compression prediction control method of continuous operation detection data of the power system in the transmission of continuous monitoring data in an unstable network is described as an example, including the following steps:
S1:对电力系统中网络速率进行周期性采集,获取当前网络数据;S1: Periodically collect the network rate in the power system to obtain the current network data;
S2:根据当前数据传输环境,获取连续型数据当前直接传输耗时T;S2: According to the current data transmission environment, it takes T to obtain the current direct transmission of continuous data;
S3:预测当前时刻的下一个时间段内传输速率的变化趋势;S3: Predict the next time period at the current moment The change trend of internal transmission rate;
S4:根据未来传输速率的变化来获取未来直接传输耗时 S4: Obtain future direct transmission time based on changes in future transmission rates
S5:根据当前传输速率建立以整体传输耗时为最优目标的优化模型;S5: Establish an optimization model with the overall transmission time as the optimal goal according to the current transmission rate;
S6:将未来时间段内的网络速率作为决策变量来调整最优压缩比;S6: Use the network rate in the future time period as a decision variable to adjust the optimal compression ratio;
S7:获取最优压缩比,并对压缩后的连续型检测数据进行传输。S7: Obtain the optimal compression ratio, and transmit the compressed continuous detection data.
进一步,所述网络速率采集,是针对一定占比的数据包从服务端传输到客户端所消耗的时间,从而计算出当前的传输速率。Further, the network rate collection is based on the time taken for a certain proportion of data packets to be transmitted from the server to the client, so as to calculate the current transmission rate.
其具体步骤如下:The specific steps are as follows:
S11:在数据传输过程中,设置服务器以100毫秒的周期对客户端发送询问信号get_time,同时记录下询问信号的大小q,在每次发送询问信号前,服务器将当前发送时间节点作为回执起始节点,并对其进行缓存,回执起始节点记为start_time;S11: During the data transmission process, set the server to send the query signal get_time to the client at a cycle of 100 milliseconds, and record the size q of the query signal at the same time. Before sending the query signal each time, the server uses the current sending time node as the start of the receipt node, and cache it, and the start node of the receipt is recorded as start_time;
S12:客户端接收到get_time信号后立即记下当前时间节点,并将其记为end_time,最后将end_time返回给服务端用于请求回执,至此完成了一次数据传输速率序列采集;S12: After the client receives the get_time signal, it immediately records the current time node, and records it as end_time, and finally returns the end_time to the server for requesting a receipt, thus completing a data transmission rate sequence collection;
S13:通过计算end_time及start_time的差值,则可以得到回执信号的请求时间,结合回执信号大小q,则可以得到各采集周期下的数据传输速率为: S13: By calculating the difference between end_time and start_time, the request time of the receipt signal can be obtained. Combined with the size of the receipt signal q, the data transmission rate in each collection period can be obtained as:
直接传输耗时T,是根据当前数据传输速率D和数据占比Q,得出直接传输耗时 The direct transmission time T is calculated based on the current data transmission rate D and the data ratio Q to obtain the direct transmission time.
预测传输速率的变化趋势,是需要通过有效的时间序列预测算法来预测未来一段时间内传输速率的变化趋势。To predict the change trend of the transmission rate, it is necessary to predict the change trend of the transmission rate in a period of time in the future through an effective time series prediction algorithm.
具体步骤如下:Specific steps are as follows:
S31:定义预测周期,针对电力信息系统在一段时间内积累的数据包需要发送给控制中心,其发送时间要小于数据积累周期否则会导致数据发送混乱,因此可将预测周期为 S31: Define the forecast period, for the power information system in a period of time The accumulated data packets need to be sent to the control center, and the sending time should be less than the data accumulation period Otherwise, it will cause confusion in data transmission, so the prediction period can be set as
S32:依据电力系统中大量历史网络速率样本,采用时间序列预测的方法,来预测出下一时间段的网络速率预测序列 S32: According to a large number of historical network rate samples in the power system, the method of time series prediction is used to predict the next time period The network rate prediction sequence of
获取原始传输耗时,是结合传输速率预测的变化趋势来获取未压缩数据在非稳定网速下的传输耗时。Obtaining the original transmission time is a combination of the predicted change trend of the transmission rate to obtain the transmission time of the uncompressed data under the unstable network speed.
设d(t)表示本阶段的数据传输速率预测序列,Q代表原始监测数据占比大小,因此根据公式倘若d(t)是可积序列,则可以直接求出原始传输耗时倘若d(t)无法直接积分则可采用下述方法:Let d(t) represent the data transmission rate prediction sequence at this stage, and Q represents the proportion of the original monitoring data, so according to the formula If d(t) is an integrable sequence, the original transmission time can be directly calculated If d(t) cannot be directly integrated, the following method can be used:
根据实验分析,在采集周期内速率的变化较小可以忽略不计,基于此,可以对d(t)的离散序列集进行分段积分,即其中ti表示采样时刻;Di表示第i个采样周期内的速率,从而求解出n来,进而获取原始传输耗时 According to the experimental analysis, the change of the rate in the acquisition period is small and can be ignored. Based on this, the discrete sequence set of d(t) can be integrated in pieces, namely Among them, t i represents the sampling time; D i represents the rate in the ith sampling period, so as to solve n, and then obtain the original transmission time.
建立优化模型,是引入压缩算法后的数据传输时间将由压缩时间、编码后监测数据在传输过程中所消耗的时间以及解压时间三部分组成,即需要建立以整体传输耗时为最优目标的优化模型。To establish an optimization model, the data transmission time after the introduction of the compression algorithm will be composed of the compression time, the time consumed by the encoded monitoring data in the transmission process, and the decompression time, that is, it is necessary to establish an optimization with the overall transmission time as the optimal goal. Model.
具体步骤如下:Specific steps are as follows:
S51:针对多次试验可得到单位原始数据包压缩时间t1与压缩比p的关系为t1=T1(p);S51: For multiple trials, the relationship between the compression time t 1 of the unit original data packet and the compression ratio p can be obtained as t 1 =T 1 (p);
S52:同理也可得到单位原始数据包解压时间t2与压缩比p的关系为t2=T2(p);S52: Similarly, it can also be obtained that the relationship between the unit original data packet decompression time t 2 and the compression ratio p is t 2 =T 2 (p);
S53:针对非稳定网络环境下传输速率对传输耗时优化决策的影响,通过动态调整压缩比来保证优化决策在整个传输阶段的有效性。基于此,构建基于传输速率预测的最优传输耗时模型,目标函数为:S53: In view of the influence of the transmission rate on the transmission time-consuming optimization decision in an unstable network environment, the effectiveness of the optimization decision in the entire transmission stage is ensured by dynamically adjusting the compression ratio. Based on this, an optimal transmission time-consuming model based on transmission rate prediction is constructed. The objective function is:
mint=t1+t*+t2=QT1(p)+t*+QT2(p)mint=t 1 +t*+t 2 =QT 1 (p)+t*+QT 2 (p)
其中t*代表在当前网络传输速率D的情况下,数据经压缩编码后用于传输阶段所消耗的时间。Among them, t* represents the time consumed in the transmission phase after the data is compressed and encoded under the current network transmission rate D.
S54:将传输速率变化对当前优化结果的影响考虑在内,则有约束条件:S54: Considering the influence of the transmission rate change on the current optimization result, there are constraints:
求解最优压缩比,针对上述基于当前速率的传输耗时优化模型和约束条件,将压缩比作为决策变量来控制整体传输时间,根据优化目标从而求解出在数据传输速率动态变化的情况下的最优压缩比,以此来保证系统传输耗时最小。To solve the optimal compression ratio, according to the above-mentioned transmission time-consuming optimization model and constraints based on the current rate, the compression ratio is used as a decision variable to control the overall transmission time. Optimal compression ratio, in order to ensure that the system transmission time is minimized.
由于该模型是非线性模型,因此本发明采用遗传算法这种进化算法来求解这个非线性规划问题。Since the model is a nonlinear model, the present invention adopts an evolutionary algorithm such as a genetic algorithm to solve the nonlinear programming problem.
其求解的具体步骤如下:The specific steps for its solution are as follows:
S61:编码:根据各模态的压缩比范围为[2,100]采用长度为k的二进制编码,共有2k种不同编码,相邻的编码间隔为 S61: Coding: According to the compression ratio range of each mode is [2,100], binary coding with length k is used, there are 2 k different codes, and the adjacent coding interval is
S62:初始种群的生成:随机生成N个串结构数据作为初始种群开始进化,即产生N个以二进制为编码的初始压缩比p作为初始种群;S62: Generation of the initial population: randomly generate N string structure data as the initial population to start evolution, that is, generate N initial compression ratios p encoded in binary as the initial population;
S63:适应度评估:选择适应度函数为t=t1+t*+t2=QT1(p)+t*+QT2(p),并计算种群中每一个初始个体的适应度函数值;S63: Fitness evaluation: select the fitness function as t=t 1 +t*+t 2 =QT 1 (p)+t*+QT 2 (p), and calculate the fitness function value of each initial individual in the population ;
S64:自然选择:个体中适应度函数值t满足约束条件且小于种群平均值的个体保留下来作为适应性强的个体添加至新种群中;S64: natural selection: individuals whose fitness function value t satisfies the constraint condition and is smaller than the population average value among individuals are retained as individuals with strong adaptability and added to the new population;
S65:交叉和变异:交叉是将个体与个体之间的部分编码进行交换,变异是在种群中随机选择一个个体以很小的概率随机改变编码中的某个字符,交叉和变异的目的都是得到新的个体添加至新的种群中;S65: Crossover and mutation: Crossover is to exchange part of the code between individuals and individuals, and mutation is to randomly select an individual in the population to randomly change a character in the code with a small probability. Get new individuals to add to the new population;
S66:是否停止进化:终止条件:进化次数是达到设定次数或者种群适应度的方差小于设定方差,若满足该终止条件,则停止进化,否则将更新后种群返回至步骤S63;S66: Whether to stop the evolution: Termination condition: the number of evolutions reaches the set number or the variance of the population fitness is less than the set variance, if the termination condition is met, the evolution is stopped, otherwise the updated population is returned to step S63;
S67:解码:将选择留下的最优个体通过编码转换成原始参数作为最优压缩比,同时得到最优适应度。S67: Decoding: Convert the optimal individual left by selection into the original parameter as the optimal compression ratio through encoding, and obtain the optimal fitness at the same time.
进一步,所述调整最优压缩比,就是依据网络速率预测趋势d(t)来调整当前的最优压缩比,此时约束条件应该变为:Further, the adjustment of the optimal compression ratio is to adjust the current optimal compression ratio according to the network rate prediction trend d(t). At this time, the constraints should become:
此时t*代表在传输速率的变化影响当前压缩决策的情况下,数据经压缩编码后于传输阶段所消耗的时间。根据优化目标采用步骤S6中的遗传算法求解出在数据传输速率动态变化的情况下的最优压缩比和最优整体压缩时耗。At this time, t* represents the time spent in the transmission phase after the data is compressed and encoded when the change of the transmission rate affects the current compression decision. According to the optimization objective, the genetic algorithm in step S6 is used to obtain the optimal compression ratio and the optimal overall compression time when the data transmission rate changes dynamically.
比较两者的整体传输时耗t比,以较小的时耗对应的压缩比作为最优压缩比。Compare the overall transmission time consumption t ratio of the two, and take the compression ratio corresponding to the smaller time consumption as the optimal compression ratio.
最后获取最优压缩比之后,对待传输的电力系统数据包进行压缩并传输。。After the optimal compression ratio is finally obtained, the power system data packets to be transmitted are compressed and transmitted. .
本发明是基于预测传输速率调节压缩比的传输耗时优化方法,可以有效降低电力系统数据传输耗时。在系统任务启动后的设备调整周期内,数据传输速率波动频率较快,通过预测数据传输速率来改进当前压缩比的方案能够保障数据有效地进行传输,也就是说,在网络传输速率变化频率较大的情况下,通过预测分析其变化趋势来调整当前压缩决策,仍然可以保证系统监测数据的传输状态不受影响。在系统任务启动后的各传输阶段中,引入传输速率预测来改善当前优化决策可以使数据传输耗时控制在低于各系统的底层采集周期,即此时的传输状态可以保证监测数据的时效性、准确性,从而使上层客户端可以及时展现、反馈系统运行状态,并由决策层做进一步处理。The invention is a transmission time-consuming optimization method based on predicting the transmission rate and adjusting the compression ratio, which can effectively reduce the time-consuming of data transmission in the power system. During the device adjustment period after the system task is started, the data transmission rate fluctuates more frequently. The scheme of improving the current compression ratio by predicting the data transmission rate can ensure the effective transmission of data. That is to say, when the network transmission rate changes more frequently In large cases, adjusting the current compression decision by predicting and analyzing its change trend can still ensure that the transmission status of the system monitoring data is not affected. In each transmission stage after the system task is started, the introduction of transmission rate prediction to improve the current optimization decision can control the data transmission time to be lower than the underlying acquisition period of each system, that is, the transmission state at this time can ensure the timeliness of monitoring data , accuracy, so that the upper-level client can display and feedback the operating status of the system in time, and the decision-making level can do further processing.
本领域内的技术人员应明白,本申请的实施例可提供为方法、系统、或计算机程序产品。因此,本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且,本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器,使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中,使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品,该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上,使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理,从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.
最后应当说明的是:以上实施例仅用以说明本发明的技术方案而非对其限制,尽管参照上述实施例对本发明进行了详细的说明,所属领域的普通技术人员应当理解:依然可以对本发明的具体实施方式进行修改或者等同替换,而未脱离本发明精神和范围的任何修改或者等同替换,其均应涵盖在本发明的权利要求保护范围之内。Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: the present invention can still be Modifications or equivalent replacements are made to the specific embodiments of the present invention, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention shall be included within the protection scope of the claims of the present invention.
Claims (2)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911056258.2A CN110769000B (en) | 2019-10-31 | 2019-10-31 | Dynamic compression prediction control method of continuous monitoring data in unstable network transmission |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911056258.2A CN110769000B (en) | 2019-10-31 | 2019-10-31 | Dynamic compression prediction control method of continuous monitoring data in unstable network transmission |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110769000A CN110769000A (en) | 2020-02-07 |
CN110769000B true CN110769000B (en) | 2020-09-25 |
Family
ID=69335909
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201911056258.2A Active CN110769000B (en) | 2019-10-31 | 2019-10-31 | Dynamic compression prediction control method of continuous monitoring data in unstable network transmission |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110769000B (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111949110B (en) * | 2020-07-16 | 2022-10-25 | 北京邮电大学 | Processing method and device for minimizing energy consumption in mobile edge calculation |
EP3944029A1 (en) * | 2020-07-21 | 2022-01-26 | Siemens Aktiengesellschaft | Method and system for determining a compression rate for an ai model of an industrial task |
CN118317269B (en) * | 2024-04-22 | 2024-12-20 | 深圳市智璨科技研发有限公司 | Intelligent ring data synchronous transmission method based on wireless communication |
CN118573208B (en) * | 2024-08-01 | 2024-10-11 | 地球脉动(无锡)科技有限公司 | Real-time original data compression transmission method of detector based on Internet of things technology |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH08125902A (en) * | 1994-10-27 | 1996-05-17 | Fuji Film Micro Device Kk | Digital still picture camera |
CN102256183A (en) * | 2010-05-21 | 2011-11-23 | 上海捷骐数码科技有限公司 | Mobile-communication-network-based audio and video signal real-time transmission method |
CN103957582A (en) * | 2014-05-17 | 2014-07-30 | 浙江大学宁波理工学院 | Wireless sensor network self-adaptation compression method |
CN104102144A (en) * | 2014-06-20 | 2014-10-15 | 杭州电子科技大学 | Batch process predictive function control method based on genetic algorithm optimization |
CN104935926A (en) * | 2014-03-18 | 2015-09-23 | 英特尔公司 | Techniques for evaluating compressed motion video quality |
CN105491614A (en) * | 2016-01-22 | 2016-04-13 | 中国地质大学(武汉) | Wireless sensor network abnormal event detection method and system based on secondary mixed compression |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7813564B2 (en) * | 2007-03-30 | 2010-10-12 | Eastman Kodak Company | Method for controlling the amount of compressed data |
JP5930702B2 (en) * | 2011-12-22 | 2016-06-08 | キヤノン株式会社 | Focus adjustment device |
CN102984520A (en) * | 2012-12-04 | 2013-03-20 | 江南大学 | Video compression method and system |
-
2019
- 2019-10-31 CN CN201911056258.2A patent/CN110769000B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH08125902A (en) * | 1994-10-27 | 1996-05-17 | Fuji Film Micro Device Kk | Digital still picture camera |
CN102256183A (en) * | 2010-05-21 | 2011-11-23 | 上海捷骐数码科技有限公司 | Mobile-communication-network-based audio and video signal real-time transmission method |
CN104935926A (en) * | 2014-03-18 | 2015-09-23 | 英特尔公司 | Techniques for evaluating compressed motion video quality |
CN103957582A (en) * | 2014-05-17 | 2014-07-30 | 浙江大学宁波理工学院 | Wireless sensor network self-adaptation compression method |
CN104102144A (en) * | 2014-06-20 | 2014-10-15 | 杭州电子科技大学 | Batch process predictive function control method based on genetic algorithm optimization |
CN105491614A (en) * | 2016-01-22 | 2016-04-13 | 中国地质大学(武汉) | Wireless sensor network abnormal event detection method and system based on secondary mixed compression |
Non-Patent Citations (2)
Title |
---|
Trend-based Symbolic Aggregate Approximation for Time Series Representation;Ke Zhang et al;《IEEE》;20181231;全文 * |
基于时间相关性的差分消零数据压缩算法研究;范欢欢等;《电子测量技术》;20180630;全文 * |
Also Published As
Publication number | Publication date |
---|---|
CN110769000A (en) | 2020-02-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110769000B (en) | Dynamic compression prediction control method of continuous monitoring data in unstable network transmission | |
CN110769453B (en) | Dynamic Compression Control Method for Multimodal Monitoring Data in Unstable Network Environment | |
CN111198808A (en) | Method, device, storage medium and electronic equipment for predicting performance index | |
CN110121171B (en) | Trust prediction method based on exponential smoothing method and gray model | |
CN114500561B (en) | Power Internet of things network resource allocation decision-making method, system, equipment and medium | |
CN118740694B (en) | A security monitoring and management system based on big data Internet of Things | |
WO2024104401A1 (en) | Cloud network abnormality detection model training method based on reinforcement learning, and storage medium | |
CN117371622A (en) | Prediction method, system and equipment for long-term dynamic carbon emission factor in region | |
CN112905343A (en) | Resource scheduling system based on load characteristics in industrial cloud environment | |
CN112100024B (en) | Resource load abnormity detection method, device and equipment | |
CN104008426A (en) | Distributed computing environment performance predicting method based on integrated learning | |
Shayesteh et al. | Automated concept drift handling for fault prediction in edge clouds using reinforcement learning | |
US20250047515A1 (en) | Blockchain-based industrial internet of things data management and optimization system and method | |
CN117236571B (en) | Planning method and system based on Internet of things | |
CN117366810B (en) | Air conditioning system control method and device | |
CN117834625A (en) | A MAPPO computation offloading method based on attention mechanism for industrial Internet of Things | |
CN118677849A (en) | Electric power communication network resource load balancing method based on reinforcement learning | |
CN112288278A (en) | A device asset management method based on knowledge graph | |
CN117177279A (en) | Multi-user multi-task computing offloading method, device and medium including throughput prediction | |
CN115456168B (en) | Training method of reinforcement learning model, energy consumption determining method and device | |
CN112667394B (en) | Computer resource utilization rate optimization method | |
CN116149848A (en) | Load prediction method and device, electronic equipment and storage medium | |
Falis et al. | Practical day-ahead power prediction of solar energy-harvesting for IoT systems | |
CN118134023B (en) | Wind-solar output quantitative prediction method and prediction device for electric power system | |
CN118868423B (en) | Virtual power plant control terminal capable of realizing novel load monitoring function |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |