Nothing Special   »   [go: up one dir, main page]

CN116684668B - Self-adaptive video frame processing method and playing terminal - Google Patents

Self-adaptive video frame processing method and playing terminal Download PDF

Info

Publication number
CN116684668B
CN116684668B CN202310970874.9A CN202310970874A CN116684668B CN 116684668 B CN116684668 B CN 116684668B CN 202310970874 A CN202310970874 A CN 202310970874A CN 116684668 B CN116684668 B CN 116684668B
Authority
CN
China
Prior art keywords
frame
video
playing
parameters
frame operation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310970874.9A
Other languages
Chinese (zh)
Other versions
CN116684668A (en
Inventor
邓正秋
吕绍和
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hunan Malanshan Video Advanced Technology Research Institute Co ltd
Original Assignee
Hunan Malanshan Video Advanced Technology Research Institute Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hunan Malanshan Video Advanced Technology Research Institute Co ltd filed Critical Hunan Malanshan Video Advanced Technology Research Institute Co ltd
Priority to CN202310970874.9A priority Critical patent/CN116684668B/en
Publication of CN116684668A publication Critical patent/CN116684668A/en
Application granted granted Critical
Publication of CN116684668B publication Critical patent/CN116684668B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234381—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2181—Source of audio or video content, e.g. local disk arrays comprising remotely distributed storage units, e.g. when movies are replicated over a plurality of video servers
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The invention discloses a self-adaptive video frame processing method and a playing terminal, wherein the method comprises the following steps: the cloud server generates a self-adaptive frame operation module; when playing video, detecting real-time equipment parameters and video parameters of a playing terminal, closing frame operation permission in a playing period when frame operation permission is not allowed to be opened, and sending an adaptive frame operation module to the playing terminal in the playing period when frame operation permission is allowed to be opened so as to open frame operation permission through the adaptive frame operation module at the playing terminal; in the period that the self-adaptive frame operation module starts the frame operation authority, the playing terminal detects the continuous condition of the video frame to be played and judges whether to trigger a frame operation instruction; when the frame operation instruction is triggered, the frame operation is started through the self-adaptive frame operation module. The technical scheme provided by the invention is beneficial to adaptively processing the video frames according to the hardware condition of the playing terminal.

Description

Self-adaptive video frame processing method and playing terminal
Technical Field
The invention relates to the technical field of video processing, in particular to a self-adaptive video frame processing method and a playing terminal.
Background
Changing the frame rate of video is a relatively common requirement, mainly to provide differentiated services to users with different requirements; due to the limitation of storage space, some video recorded by the video cameras have poor video continuity due to relatively fewer frames per unit time, or lack of pictures connected with video frames due to video clipping causes insufficient video continuity. While users who are researching on video compression may pursue higher frame rates for better display effects or select lower frame rates for higher compression rates. The variation of the frame rate can be divided into two types: the low frame rate becomes the high frame rate; the high frame rate becomes the low frame rate. Although both are for different needs, the same implementation is adopted. The frame insertion improves the video playing effect, however, the frame insertion is not as much as possible, and the frame insertion operation has requirements on various aspects of hardware. Therefore, there is a need to propose a method for adaptively performing video frame processing according to the hardware condition of a playback terminal and a playback terminal.
Disclosure of Invention
The invention mainly aims to provide a self-adaptive video frame processing method and a playing terminal, which aim to adaptively process video frames according to hardware conditions of the playing terminal.
In order to achieve the above objective, in the adaptive video frame processing method provided by the present invention, a playing terminal is in communication connection with a cloud server; the method comprises the following steps:
the method comprises the steps that initial equipment parameters, video parameters and equipment parameter changes of a playing terminal in the process of historically playing videos are sent to a cloud server;
the cloud server calculates preferred frame rate sets under different initial equipment parameters and different video parameters according to initial equipment parameters, video parameters and equipment parameter changes in the process that the playing terminal plays each video in a historical mode, wherein the preferred frame rate sets are frame rate ranges for guaranteeing stable performance of the playing terminal;
the cloud server generates a self-adaptive frame operation module, wherein the self-adaptive frame operation module stores different initial equipment parameters and corresponding preferable frame rate sets under different video parameters so as to self-adaptively start frame operation permission according to the performance change of the playing terminal, and the frame operation comprises frame inserting operation or frame subtracting operation;
when playing video, detecting real-time equipment parameters and video parameters of a playing terminal, closing frame operation permission in a playing period when frame operation permission is not allowed to be opened, and sending an adaptive frame operation module to the playing terminal in the playing period when frame operation permission is allowed to be opened so as to open frame operation permission through the adaptive frame operation module at the playing terminal;
In the period that the self-adaptive frame operation module starts the frame operation authority, the playing terminal detects the continuous condition of the video frame to be played and judges whether to trigger a frame operation instruction;
when the frame operation instruction is triggered, the frame operation is started through the self-adaptive frame operation module.
Preferably, the cloud server calculates a preferred frame rate set under different initial device parameters and different video parameters according to initial device parameters, video parameters and device parameter changes in the process that the playing terminal plays each video in a history, where the preferred frame rate set is a frame rate range for ensuring stable performance of the playing terminal, and the method includes:
acquiring frame rate, resolution, code rate and playing duration from each video historically played by a playing terminal as video parameters;
respectively acquiring initial data of the computing capacity occupancy rate, the residual storage space, the endurance time and the heating temperature of the playing terminal in the historical playing process of the playing terminal as initial equipment parameters;
respectively acquiring the data changes of the computing capacity occupancy rate, the residual storage space, the endurance time and the heating temperature of the playing terminal in the historical playing process of the playing terminal as the equipment parameter changes;
Calculating the influence factor of each video parameter on the performance of each device when the video is played;
a preferred set of frame rates for different initial device parameters and different video parameters is calculated.
Preferably, the step of calculating a preferred set of frame rates for different initial device parameters and different video parameters comprises:
determining a first set of frame rates related to video parameters and computing power occupancy;
determining a second set of frame rates associated with the video parameters and the remaining storage space;
determining a third set of frame rates related to video parameters and endurance time;
determining a fourth set of frame rates related to the video parameters and the heat generation temperature;
a preferred set of frame rates is determined based on the first set of frame rates, the second set of frame rates, the third set of frame rates, and the fourth set of frame rates.
Preferably, the step of generating the adaptive frame operation module by the cloud server includes:
the cloud server calculates frame operation limiting conditions under different video parameters and initial equipment parameters, wherein the frame operation limiting conditions comprise a preferable frame rate set, an allowable frame inserting quantity and an allowable frame subtracting quantity;
the cloud server establishes mapping relations among different video parameters, initial equipment parameters and frame operation limiting conditions;
According to the mapping relation, the video parameters and the real-time equipment parameters of the playing terminal are used as input conditions of the self-adaptive frame operation module, and the frame operation limiting conditions are set as control parameters output by the self-adaptive frame operation module, so that the self-adaptive frame operation module can adaptively adjust the frame operation limiting conditions according to the difference of the video parameters and the real-time equipment parameters.
Preferably, when playing video, detecting real-time equipment parameters and video parameters of the playing terminal, closing frame operation permission in a playing period when frame operation permission is not allowed to be opened, and sending an adaptive frame operation module to the playing terminal in a playing period when frame operation permission is allowed to be opened, so as to open frame operation permission at the playing terminal through the adaptive frame operation module, wherein the method comprises the following steps:
when playing video, sending the detected real-time equipment parameters and video parameters of the playing terminal to a cloud server;
the cloud server determines corresponding conditions for allowing the frame operation permission to be opened and conditions for not allowing the frame operation permission to be opened;
the cloud server determines a first playing period of not allowing the frame operation permission to be opened according to the condition of not allowing the frame operation permission to be opened;
The cloud server determines a second playing period for allowing the frame operation permission to be opened according to the condition for allowing the frame operation permission to be opened;
transmitting the first playing period and the second playing period to a playing terminal;
the playing terminal closes the frame operation authority in a first playing period;
and the cloud server sends the adaptive frame operation module to the playing terminal in the second playing period so as to start the frame operation authority through the adaptive frame operation module at the playing terminal.
Preferably, when playing video, detecting real-time equipment parameters and video parameters of the playing terminal, closing frame operation permission in a playing period when frame operation permission is not allowed to be opened, and sending the adaptive frame operation module to the playing terminal in a playing period when frame operation permission is allowed to be opened, so that the frame operation permission is opened by the adaptive frame operation module at the playing terminal, and further comprising:
the cloud server calculates a first frame operation permission closing condition triggered by a single equipment parameter;
the cloud server calculates a second frame operation permission closing condition triggered by the combination of a plurality of equipment parameters;
the cloud server acquires the equipment parameters of the playing terminal to determine whether a first frame operation permission closing condition or a second frame operation permission closing condition is reached;
If yes, sending a frame closing operation permission instruction to the self-adaptive frame operation module so as to stop frame operation.
Preferably, the step of the playing terminal detecting the continuous condition of the video frame to be played and judging whether to trigger the frame operation instruction in the period of the frame operation permission opening of the adaptive frame operation module includes:
when the self-adaptive frame operation module is generated, the playing terminal detects the inter-frame image block motion relation of the decoded video frame to be played;
determining image block displacement vectors of adjacent frames according to the inter-frame image block motion relationship;
determining whether to trigger a frame operation instruction according to the image block displacement vectors of adjacent frames and real-time equipment parameters, wherein the frame operation instruction comprises a frame inserting instruction or a frame subtracting instruction;
if yes, triggering a frame operation instruction.
Preferably, the preferred set of frame rates is determined as follows:
a first set of frame rates related to video parameters and computing power occupancy is determined as follows:
；
；
；
；
wherein ,in order to not allow the playing period of opening the frame operation authority, the computing capability occupancy rate of the playing terminal under the common influence of the initial frame rate and other video parameters;In order to allow the playing period of opening the frame operation authority, the computing capacity occupancy rate of the playing terminal under the common influence of the frame rate to be adjusted and other video parameters; / >The initial computing capacity occupancy rate of the playing terminal;The upper limit of the occupancy rate of the preset computing capacity of the playing terminal;
for the duration of the play period for which the frame operation authority is not allowed to be opened,/for the duration of the play period for which the frame operation authority is not allowed to be opened>Broadcasting to allow opening of frame operation rightsThe duration of the discharge period;For the unit time influence factor of the initial frame rate in the video parameters to the computing power occupancy of the playing terminal,/for>For the unit time influence factor of the to-be-adjusted frame rate in the video parameters on the computing capability occupancy rate of the playing terminal,/for>For the unit time influence factor of the resolution in the video parameters on the computing power occupancy of the playback terminal,For the unit time influence factor of the code rate in the video parameters on the computing capability occupancy rate of the playing terminal,The method comprises the steps that a unit time influence factor of playing time length in video parameters on the computing capability occupancy rate of a playing terminal is used; j is the video parameter corresponding to j, and the value range of j is [2,4 ]]；
According to g 1 Determining a corresponding optional frame rate, and according toDetermining a corresponding alternative frame rate, the two alternative frame rates intersecting to determine a first set of frame rates G 1 ；
A second set of frame rates associated with the video parameters and the remaining storage space is determined as follows:
；
；
wherein ,For the preset lower limit of the remaining memory space of the playback terminal, < > for>For the remaining storage space of the playback terminal after the current video has been played back,/the playback terminal is provided with a memory for storing the video data>The initial remaining storage space for the playing terminal;For the unit time influence factor of the initial frame rate in the video parameters on the remaining memory space of the playback terminal, is>For the unit time influence factor of the frame rate to be adjusted in the video parameters on the remaining memory space of the playback terminal,/for the video parameters>For the resolution in the video parameters to the unit time influence factor of the remaining memory space of the playback terminal,For the unit time influence factor of the code rate in the video parameters to the remaining storage space of the playing terminal, +.>The unit time influence factor of the play time length in the video parameters on the residual storage space of the play terminal is used;
according toDetermining a corresponding optional frame rate to determine a second set of frame rates G 2 ；
A third set of frame rates related to video parameters and endurance is determined as follows:
；
；
wherein ,for the initial endurance of the playback terminal, +.>For the preset duration lower limit of the playing terminal, < +.>The method comprises the steps that the duration of a playing terminal after the current video is played is set;For the unit time influence factor of the initial frame rate in the video parameters to the duration of the playing terminal,/ >For the unit time influence factor of the frame rate to be adjusted in the video parameters on the duration of the playing terminal,/for the unit time>For the unit time influence factor of resolution in video parameters on the endurance of the playback terminal, +.>For the unit time influence factor of the code rate in the video parameters to the duration of the playing terminal, +.>The method comprises the steps that a unit time influence factor of play duration in video parameters on the duration of a play terminal is used;
according toDetermining a corresponding optional frame rate to determine a third set of frame rates G 3 ；
A fourth set of frame rates related to video parameters and heat generation temperature is determined as follows:
；
；
；
；
wherein ,in order to not allow the playing time period of opening the frame operation authority, the temperature of the playing terminal under the common influence of the initial frame rate and other video parameters;In order to allow the playing period of opening the frame operation authority, the temperature of the playing terminal under the common influence of the frame rate to be adjusted and other video parameters;For the initial temperature of the playback terminal, +.>The upper limit of the preset temperature of the playing terminal is set;
for the unit time influence factor of the initial frame rate in the video parameters on the temperature of the playback terminal, +.>For the unit time influence factor of the frame rate to be adjusted in the video parameters on the temperature of the playing terminal,For the unit time influence factor of the resolution in the video parameters on the temperature of the playback terminal, +.>For the unit time influence factor of the code rate in the video parameters to the temperature of the playing terminal,/for>The unit time influence factor of the playing time length in the video parameters on the temperature of the playing terminal is used;
according toDetermining a corresponding optional frame rate and according to +.>Determining a corresponding alternative frame rate, the two alternative frame rates intersecting to determine a fourth set of frame rates G 4 ；
Determining a preferred set of frame rates。
Preferably, the method further comprises:
the cloud server acquires the device parameter change increment of the playing terminal caused in the frame operation process;
and the cloud server adjusts the control condition of the self-adaptive frame operation module on the frame operation according to the initial equipment parameter of the playing terminal, the equipment parameter change and the equipment parameter change increment caused by the frame operation.
In addition, in order to achieve the above objective, the present invention further provides a playing terminal, which is configured to execute the method described in any one of the above, where the playing terminal is configured to be in communication connection with a cloud server.
According to the technical scheme, when the cloud server plays videos in daily life, initial equipment parameters, video parameters and equipment parameter changes of the playing terminal are detected to form historical playing data, the cloud server analyzes the detected historical playing data, when the videos are played, the influence of each video parameter on the change of each equipment parameter of the playing terminal can be determined when the values of each video parameter are different, and the optimal frame rate set under different initial equipment parameters and different video parameters is calculated. Therefore, the cloud server generates an adaptive frame operation module according to the analysis result of the historical playing data detected by the playing terminal, the adaptive frame operation module stores the corresponding preferred frame rate sets under different initial equipment parameters and different video parameters, and the adaptive frame operation module has the following functions: detecting video parameters and real-time equipment parameters of a playing terminal, taking the real-time equipment parameters as initial equipment parameters, and opening corresponding frame operation authorities under the condition that the performance of the playing terminal is stable (namely, under the condition that the equipment parameters are in a set range) according to the difference of the video parameters and the real-time equipment parameters of the video played by the playing terminal, so that the playing terminal can carry out video frame rate adjustment in the frame operation authorities (namely, the preferred frame rate set), and the video frame rate adjustment operation exceeding the frame operation authorities can be refused by the self-adaptive frame operation module. In the process of playing the video by the playing terminal, the cloud server detects real-time equipment parameters and video parameters of the playing terminal, determines a playing period in which frame operation permission is not allowed to be opened and a playing period in which frame operation permission is allowed to be opened according to implementation equipment parameters, and sends the self-adaptive frame operation module to the playing terminal as a virtual switch in the playing period in which frame operation permission is allowed to be opened so as to open the frame operation permission. Therefore, in the process of playing the video by the playing terminal, the adaptive frame operation module can adaptively determine the frame operation authority according to the actual condition of the playing terminal, adaptively start the frame operation or reject the frame operation, and the started frame operation accords with the preferred frame rate set because the frame operation is in the frame operation authority, so that the serious performance deficiency of the playing terminal can not be caused, and the aim of adaptively processing the video frame according to the hardware condition of the playing terminal is fulfilled.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to the structures shown in these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of an adaptive video frame processing method according to an embodiment of the present invention;
fig. 2 is a functional block diagram of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The description as it relates to "first", "second", etc. in the present invention is for descriptive purposes only and is not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise.
In addition, the technical solutions of the embodiments of the present invention may be combined with each other, but it is necessary to be based on the fact that those skilled in the art can implement the technical solutions, and when the technical solutions are contradictory or cannot be implemented, the combination of the technical solutions should be considered as not existing, and not falling within the scope of protection claimed by the present invention.
Referring to fig. 1 to fig. 2, in a first embodiment of the adaptive video frame processing method of the present invention, a playing terminal is in communication connection with a cloud server; the method comprises the following steps:
step S10, initial equipment parameters, video parameters and equipment parameter changes of a playing terminal in the process of historically playing each video are sent to a cloud server;
step S20, the cloud server calculates a preferred frame rate set under different initial equipment parameters and different video parameters according to initial equipment parameters, video parameters and equipment parameter changes in the process that the playing terminal plays each video in a historical mode, wherein the preferred frame rate set is a frame rate range for guaranteeing stable performance of the playing terminal;
step S30, the cloud server generates an adaptive frame operation module, wherein the adaptive frame operation module stores different initial equipment parameters and corresponding preferred frame rate sets under different video parameters so as to adaptively start frame operation permission according to the performance change of the playing terminal, and the frame operation comprises frame inserting operation or frame subtracting operation;
Step S40, when playing video, detecting real-time equipment parameters and video parameters of a playing terminal, closing frame operation permission in a playing period when frame operation permission is not allowed to be opened, and sending an adaptive frame operation module to the playing terminal in the playing period when frame operation permission is allowed to be opened so as to open frame operation permission through the adaptive frame operation module at the playing terminal; the real-time equipment parameters of the playing terminal are used as initial equipment parameters corresponding to the playing terminal at each moment;
step S50, in the period of opening frame operation authority of the self-adaptive frame operation module, the playing terminal detects the continuous condition of the video frame to be played and judges whether to trigger a frame operation instruction;
step S60, when the frame operation instruction triggers, the adaptive frame operation module starts the frame operation.
According to the technical scheme, when the cloud server plays videos in daily life, initial equipment parameters, video parameters and equipment parameter changes of the playing terminal are detected to form historical playing data, the cloud server analyzes the detected historical playing data, when the videos are played, the influence of each video parameter on the change of each equipment parameter of the playing terminal can be determined when the values of each video parameter are different, and the optimal frame rate set under different initial equipment parameters and different video parameters is calculated. Therefore, the cloud server generates an adaptive frame operation module according to the analysis result of the historical playing data detected by the playing terminal, the adaptive frame operation module stores the corresponding preferred frame rate sets under different initial equipment parameters and different video parameters, and the adaptive frame operation module has the following functions: detecting video parameters and real-time equipment parameters of a playing terminal, taking the real-time equipment parameters as initial equipment parameters, and opening corresponding frame operation authorities under the condition that the performance of the playing terminal is stable (namely, under the condition that the equipment parameters are in a set range) according to the difference of the video parameters and the real-time equipment parameters of the video played by the playing terminal, so that the playing terminal can carry out video frame rate adjustment in the frame operation authorities (namely, the preferred frame rate set), and the video frame rate adjustment operation exceeding the frame operation authorities can be refused by the self-adaptive frame operation module. In the process of playing the video by the playing terminal, the cloud server detects real-time equipment parameters and video parameters of the playing terminal, determines a playing period in which frame operation permission is not allowed to be opened and a playing period in which frame operation permission is allowed to be opened according to implementation equipment parameters, and sends the self-adaptive frame operation module to the playing terminal as a virtual switch in the playing period in which frame operation permission is allowed to be opened so as to open the frame operation permission. Therefore, in the process of playing the video by the playing terminal, the adaptive frame operation module can adaptively determine the frame operation authority according to the actual condition of the playing terminal, adaptively start the frame operation or reject the frame operation, and the started frame operation accords with the preferred frame rate set because the frame operation is in the frame operation authority, so that the serious performance deficiency of the playing terminal can not be caused, and the aim of adaptively processing the video frame according to the hardware condition of the playing terminal is fulfilled.
The adaptive frame operation module adaptively opens frame operation authority at the playing terminal, specifically, the following manner may be referred to:
for example, after the adaptive frame operation module sends the current video parameter of the playing terminal to the playing terminal, assuming that the current video parameter of the playing terminal is a, the initial equipment parameter is B (a and B are respectively an array, and each array includes a plurality of parameters), and the adaptive frame operation module determines, according to the video parameter a and the initial equipment parameter B, that the optimal frame rate set under the condition of ensuring the stable performance of the playing terminal (the performance stability refers to that phenomena such as downtime, serious clamping, outage, excessive temperature and the like are not caused by serious performance deficiency before the playing terminal plays the current video) is 20-30fps.
If the current frame rate of the video played by the playing terminal is 25fps and is between 20-30fps of the preferred frame rate set, the adaptive frame operation module allows the playing terminal to perform frame inserting operation or frame subtracting operation on the video played currently, so long as the frame rate of the video does not exceed 20-30fps after the frame inserting or frame subtracting operation. Therefore, when the frame inserting operation or the frame subtracting operation is triggered, the frame operation can be started through the self-adaptive frame operation module.
If the current frame rate of the video currently played by the playing terminal is 30fps and is at the upper limit value of 20-30fps of the preferred frame rate set, if the frame inserting instruction in the frame operating instruction triggers at this time, the frame inserting will cause the frame inserting to exceed the preferred frame rate set, so the adaptive frame operating module refuses to start the frame inserting instruction.
Specifically, steps S10 to S30 are history play data processing steps before formally starting the adaptive video frame processing method, and steps S40 to S60 are steps executed by formally adaptive video frame processing.
In a second embodiment of the present invention, based on the first embodiment of the present invention, the step S20 includes:
step S21, obtaining frame rate, resolution, code rate and playing duration from each video played by the playing terminal in history as video parameters;
step S22, respectively acquiring initial data of the computing capacity occupancy rate, the residual storage space, the endurance time and the heating temperature of the playing terminal in the history playing process of the playing terminal as initial equipment parameters;
step S23, respectively acquiring the data changes of the computing capacity occupancy rate, the residual storage space, the endurance time and the heating temperature of the playing terminal in the history playing process of the playing terminal as the device parameter changes;
step S24, calculating the influence factor of each video parameter on the performance of each device when the video is played;
step S25, calculating a preferred frame rate set under different initial device parameters and different video parameters.
Specific:
the frame rate is the number of frames per second, the larger the frame rate is, the smoother the video is played, and the smaller the frame rate is, the more the video is blocked.
Resolution refers to the number of pixels contained in a unit inch. Resolution affects image size, which is proportional to the image size: the higher the resolution, the larger the image; the lower the resolution, the smaller the image.
The code rate refers to the number of data bits transmitted per second. The higher the code rate is, the clearer the video is, the longer the network loading time is, the higher the requirement on the network bandwidth is, the larger the file is, and the occupied storage space is large. The code rate and video duration also directly affect the storage space.
The playing duration is the video length of the playing terminal, and has obvious influence on computing capacity, storage space, duration and temperature.
In addition to the duration of play, the frame rate, resolution, and code rate of the video are typically selected from several common data. Therefore, a plurality of groups of video parameters with the same values of three parameters and different values of the other parameters can be easily found. The device parameter changes brought by the video parameters of the groups are analyzed to determine the unit time influence factors of the video parameters with different values on each device parameter. Thus, with this method, it is possible to determine the unit time influence factor of each video parameter on each device parameter, respectively.
Specifically, the video parameters of the i-th video played by the play terminal in a historical way adopt an array C i The representation is:
；
wherein ,frame rate of i-th video historically played for play terminal,/for play terminal>Resolution of i-th video historically played for play terminal,/for play terminal>Code rate of i-th video historically played for play terminal,/->The playing duration of the ith video which is historically played by the playing terminal; i is more than or equal to 1 and less than or equal to I, wherein I is the number of videos historically played by a playing terminal acquired by a cloud server;
the initial equipment parameter of the play terminal for historically playing the ith video adopts an array D i0 The representation is:
；
wherein ,initial computing capability occupancy for historically playing the ith video for the playback terminal,/for the playback terminal>Original remaining memory space for historically playing the ith video for the play terminal,/the play terminal>For the play terminal to play the initial duration of the ith video in history,/the play terminal is provided with a display unit for displaying the initial duration of the ith video>The method comprises the steps of historically playing an initial temperature of an ith video for a playing terminal;
the device parameter change adopts an arrayThe representation is:
；
wherein ,calculating capability occupancy rate change amount of i-th video historically played for playing terminal, +.>The residual storage space variation of the ith video is historically played for the play terminal,/for the play terminal>For the play terminal to play the endurance time variable quantity of the ith video in history,/for the play terminal >The temperature variation of the ith video is historically played for the playing terminal;
the real-time equipment parameters of the playing terminal acquired by the playing terminal adopt an array E 0 The representation is:
；
wherein ,for the initial computing power occupancy of the playback terminal, < >>In order to play the initial remaining memory space of the terminal,for the initial endurance of the playback terminal, +.>The initial temperature of the playing terminal is set;
when the initial device parameter satisfies at least one of the following formulas (1) to (5), for a play period in which the frame operation authority is not allowed to be opened, the remaining play periods are play periods in which the frame operation authority is allowed to be opened:
；
；
；
；
；
wherein ,for the upper limit of the preset computing power occupancy of the playback terminal,/->For the preset lower limit of the remaining memory space of the playback terminal, < > for>For the preset duration lower limit of the playing terminal, < +.>The upper limit of the preset temperature of the playing terminal is set; f is a set constant, the value range of F is [3.5,5 ], F can be determined according to the real-time video parameters, the real-time video parameters are compared with the set video parameters, if the real-time video parameters exceed the set video parameters, F takes smaller values in the value range, and if the real-time video parameters do not exceed the set video parameters, F takes larger values in the value range;
In a third embodiment of the present invention, based on the second embodiment of the present invention, the step S25 includes:
step S251, a first frame rate set related to video parameters and computing power occupancy is determined;
step S252, determining a second frame rate set related to the video parameters and the remaining storage space;
step S253, determining a third set of frame rates related to the video parameters and the endurance time;
step S254, determining a fourth frame rate set related to the video parameters and the heating temperature;
step S255, determining a preferred frame rate set according to the first frame rate set, the second frame rate set, the third frame rate set, and the fourth frame rate set.
Step S25 is specifically performed with reference to the following manner:
1) A first set of frame rates related to video parameters and computing power occupancy is determined as follows:
；
；
；
；
wherein ,in order to not allow the playing period of opening the frame operation authority, the computing capability occupancy rate of the playing terminal under the common influence of the initial frame rate and other video parameters;In order to allow the playing period of opening the frame operation authority, the computing capacity occupancy rate of the playing terminal under the common influence of the frame rate to be adjusted and other video parameters;
for the duration of the play period for which the frame operation authority is not allowed to be opened,/for the duration of the play period for which the frame operation authority is not allowed to be opened >A duration of a play period for allowing the frame operation right to be opened;For the unit time influence factor of the initial frame rate in the video parameters to the computing power occupancy of the playing terminal,/for>For the unit time impact factor of the frame rate to be adjusted in the video parameters on the computing capability occupancy of the playback terminal,for the unit time influence factor of the resolution in the video parameters on the computing power occupancy of the playback terminal,For the unit time influence factor of the code rate in the video parameters on the computing capability occupancy rate of the playing terminal,The method comprises the steps that a unit time influence factor of playing time length in video parameters on the computing capability occupancy rate of a playing terminal is used; j is the video parameter corresponding to j, and the value range of j is [2,4 ]]；
；
；
According to g 1 Determining a corresponding optional frame rate, and according toDetermining a corresponding alternative frame rate, the two alternative frame rates intersecting to determine a first set of frame rates G 1 ；
2) A second set of frame rates associated with the video parameters and the remaining storage space is determined as follows:
；
；
；
wherein ,for the remaining storage space of the playback terminal after the current video has been played back,/the playback terminal is provided with a memory for storing the video data>For the unit time influence factor of the initial frame rate in the video parameters on the remaining memory space of the playback terminal, is >For the unit time influence factor of the frame rate to be adjusted in the video parameters on the remaining memory space of the playback terminal,/for the video parameters>For the resolution in the video parameters to the unit time influence factor of the remaining memory space of the playback terminal,For the unit time influence factor of the code rate in the video parameters to the remaining storage space of the playing terminal, +.>Unit time of play duration in video parameter to residual storage space of play terminalAn influence factor; />
According toDetermining a corresponding optional frame rate to determine a second set of frame rates G 2 ；
3) A third set of frame rates related to video parameters and endurance is determined as follows:
；
；
；
wherein ,the method comprises the steps that the duration of a playing terminal after the current video is played is set;For the unit time influence factor of the initial frame rate in the video parameters to the duration of the playing terminal,For the unit time influence factor of the frame rate to be adjusted in the video parameters on the duration of the playing terminal,/for the unit time>For the unit time influence factor of resolution in video parameters on the endurance of the playback terminal, +.>For the unit time influence factor of the code rate in the video parameters to the duration of the playing terminal, +.>The method comprises the steps that a unit time influence factor of play duration in video parameters on the duration of a play terminal is used;
According toDetermining a corresponding optional frame rate to determine a third set of frame rates G 3 ；
4) A fourth set of frame rates related to video parameters and heat generation temperature is determined as follows:
；
；
；
；
wherein ,in order to not allow the playing time period of opening the frame operation authority, the temperature of the playing terminal under the common influence of the initial frame rate and other video parameters;In order to allow the playing period of opening the frame operation authority, the temperature of the playing terminal under the common influence of the frame rate to be adjusted and other video parameters;
for the unit time influence factor of the initial frame rate in the video parameters on the temperature of the playback terminal, +.>In video parametersTo adjust the unit time influencing factor of the frame rate to the temperature of the playback terminal,/for the playback terminal>For the unit time influence factor of the resolution in the video parameters on the temperature of the playback terminal, +.>For the unit time influence factor of the code rate in the video parameters to the temperature of the playing terminal,/for>The unit time influence factor of the playing time length in the video parameters on the temperature of the playing terminal is used;
；/>
；
according toDetermining a corresponding optional frame rate and according to +.>Determining a corresponding alternative frame rate, the two alternative frame rates intersecting to determine a fourth set of frame rates G 4 ；
Determining a preferred set of frame rates 。
In a fourth embodiment of the present invention, based on the first to third embodiments of the present invention, the step S30 includes:
step S31, the cloud server calculates frame operation limiting conditions under different video parameters and initial equipment parameters, wherein the frame operation limiting conditions comprise a preferable frame rate set, an allowable frame inserting quantity and an allowable frame subtracting quantity;
step S32, the cloud server establishes mapping relations among different video parameters, initial equipment parameters and frame operation limiting conditions;
step S33, taking the video parameters and the real-time equipment parameters of the playing terminal as the input conditions of the self-adaptive frame operation module according to the mapping relation, and setting the frame operation limiting conditions as the control parameters output by the self-adaptive frame operation module, so that the self-adaptive frame operation module can adaptively adjust the frame operation limiting conditions according to the difference of the video parameters and the real-time equipment parameters.
Wherein the allowable number of inserting frames and the allowable number of subtracting frames are determined together with the initial frame rate according to the preferred frame rate set after the initial frame rate in the video parameters of the playback terminal is acquired.
Allowing the number of the inserted frames to be the difference value between the upper limit value of the preferred frame rate set and the current frame rate; the number of frames is allowed to be subtracted as the difference between the current frame rate and the lower limit of the preferred set of frame rates.
The self-adaptive frame operation module stores a mapping relation, can take video parameters and real-time equipment parameters of the playing terminal as input conditions, and outputs frame operation limiting conditions by searching the mapping relation. Thus, the frame operation of the playing terminal can be controlled according to the preferred frame rate set, the allowable frame inserting number and the allowable frame subtracting number in the frame operation limiting condition.
Further, the method further comprises:
in step S70, the cloud server determines the conditions of allowing and not allowing to open the frame operation permission under different video parameters and initial device parameters according to the performance change of the playing terminal under different device parameters and different video parameters.
When the initial equipment parameter satisfies at least one of formulas (1) to (5) of the second embodiment, a condition that the frame operation authority is not allowed to be opened is reached; if none of the formulas (1) to (5) of the second embodiment is satisfied, a condition is reached that allows the frame operation authority to be opened.
In a fifth embodiment of the present invention, based on the first to fourth embodiments of the present invention, the step S40 includes:
step S41, when playing video, sending the detected real-time equipment parameters and video parameters of the playing terminal to a cloud server;
Step S42, the cloud server determines corresponding conditions for allowing the frame operation permission to be opened and conditions for not allowing the frame operation permission to be opened;
step S43, the cloud server determines a first playing period of not allowing the frame operation permission to be opened according to the condition of not allowing the frame operation permission to be opened;
step S44, the cloud server determines a second playing period for allowing the frame operation permission to be opened according to the condition for allowing the frame operation permission to be opened;
step S45, the first playing period and the second playing period are sent to a playing terminal;
step S46, the playing terminal closes the frame operation authority in the first playing period;
in step S47, the cloud server sends the adaptive frame operation module to the playing terminal in the second playing period, so as to open the frame operation authority through the adaptive frame operation module at the playing terminal.
Specifically, when playing video, the frame operation authority is first closed by default, and the real-time equipment parameters of the playing terminal are acquired according to the set period, so that the change rule of the real-time equipment parameters along with time under the current video parameters can be acquired.
According to the real-time equipment parameters of the playing terminal and the change rule of the real-time equipment parameters along with time, a first playing period when the real-time equipment parameters meet the condition of not allowing the frame operation permission to be opened and a second playing period when the frame operation permission to be opened can be calculated.
According to a fifth embodiment of the present invention, in a sixth embodiment of the present invention, the step S40 further includes:
step S48, the cloud server calculates a first frame operation authority closing condition triggered by a single device parameter;
step S49, the cloud server calculates a second frame operation authority closing condition triggered by the combination of a plurality of device parameters;
step S410, the cloud server acquires the device parameters of the playing terminal to determine whether a first frame operation right closing condition or a second frame operation right closing condition is reached;
if yes, step S411 is executed to send a command for closing the frame operation permission to the adaptive frame operation module to stop the frame operation.
The cloud server detects the computing capability occupancy rate, the residual storage space, the endurance time and the heating temperature of the playing terminal in real time according to a set period, and when detecting that each equipment parameter meets at least one of formulas (1) to (5) of the second embodiment, sends a frame closing operation permission instruction to the adaptive frame operation module to stop frame operation, and at the moment, frame insertion and frame reduction are not allowed to be carried out so as to avoid serious performance degradation of the playing terminal caused by frame insertion and frame reduction.
According to the first to sixth embodiments of the present invention, in a seventh embodiment of the present invention, the step S50 includes:
Step S51, when the self-adaptive frame operation module generates, the playing terminal detects the inter-frame image block motion relation of the decoded video frame to be played;
step S52, determining image block displacement vectors of adjacent frames according to the inter-frame image block motion relationship;
step S53, determining whether to trigger a frame operation instruction according to the image block displacement vector of the adjacent frame and the real-time equipment parameter, wherein the frame operation instruction comprises a frame inserting instruction or a frame subtracting instruction;
if yes, go to step S54: and triggering a frame operation instruction.
Further, the step S60 in the present invention includes:
step S61, when the frame operation instruction is triggered, the self-adaptive frame operation module determines real-time frame operation limiting conditions according to real-time equipment parameters and video parameters of the playing terminal;
step S62, the self-adaptive frame operation module determines frame operation parameters according to the continuous condition of the video frames to be played and real-time frame operation limiting conditions;
in step S63, the adaptive frame operation module controls the frame operation according to the frame operation parameters, so as to adjust the frame rate of the played video through the frame operation.
According to the eighth embodiment of the present invention, based on the first to seventh embodiments of the present invention, the method further includes:
Step S80, the cloud server acquires the device parameter change increment of the playing terminal caused in the frame operation process;
step S90, the cloud server adjusts the control condition of the self-adaptive frame operation module on the frame operation according to the initial equipment parameter of the playing terminal, the equipment parameter change and the equipment parameter change increment caused by the frame operation.
During frame operations, including both the insert and subtract frame operations, additional device parameter delta may result, and the present invention primarily contemplates computing power occupancy delta. And correcting the calculation process of the first preferred frame rate set by adopting the increment of the change of the occupancy rate of the calculation capacity.
Specific:
the modified first set of frame rates is obtained as follows:
；
；
；
；
a unit time influence factor of frame operation in video parameters on the computing capability occupancy rate of the playing terminal;in order to not allow the playing period of opening the frame operation permission, the computing capability occupancy rate of the playing terminal under the common influence of the initial frame rate, other video parameters and the frame operation;In order to allow the playing period of opening the frame operation permission, the computing capacity occupancy rate of the playing terminal under the common influence of the frame rate to be adjusted, other video parameters and the frame operation;
；
；
g 1s G is g 1 Is used for the correction result of the (a),is->According to g 1s andCorrecting the first frame rate set +.>Obtaining a modified first frame rate set +.>。
Determining a modified preferred frame rate set。
In addition, in order to achieve the above objective, the present invention further provides a playing terminal for executing the method; the playing terminal is used for being in communication connection with the cloud server.
The foregoing description of the preferred embodiments of the present invention should not be construed as limiting the scope of the invention, but rather utilizing equivalent structural changes made in the present invention description and drawings or directly/indirectly applied to other related technical fields are included in the scope of the present invention.

Claims (7)

1. The self-adaptive video frame processing method is characterized in that a playing terminal is in communication connection with a cloud server; the method comprises the following steps:
the method comprises the steps that initial equipment parameters, video parameters and equipment parameter changes of a playing terminal in the process of historically playing videos are sent to a cloud server;
the cloud server calculates preferred frame rate sets under different initial equipment parameters and different video parameters according to initial equipment parameters, video parameters and equipment parameter changes in the process that the playing terminal plays each video in a historical mode, wherein the preferred frame rate sets are frame rate ranges for guaranteeing stable performance of the playing terminal;
The cloud server generates a self-adaptive frame operation module, wherein the self-adaptive frame operation module stores different initial equipment parameters and corresponding preferable frame rate sets under different video parameters so as to self-adaptively start frame operation permission according to the performance change of the playing terminal, and the frame operation comprises frame inserting operation or frame subtracting operation;
when playing video, detecting real-time equipment parameters and video parameters of a playing terminal, closing frame operation permission in a playing period when frame operation permission is not allowed to be opened, and sending an adaptive frame operation module to the playing terminal in the playing period when frame operation permission is allowed to be opened so as to open frame operation permission through the adaptive frame operation module at the playing terminal;
in the period that the self-adaptive frame operation module starts the frame operation authority, the playing terminal detects the continuous condition of the video frame to be played and judges whether to trigger a frame operation instruction;
when the frame operation instruction is triggered, starting frame operation through the self-adaptive frame operation module;
the cloud server calculates a preferred frame rate set under different initial equipment parameters and different video parameters according to initial equipment parameters, video parameters and equipment parameter changes in the process that the playing terminal plays each video in a historical mode, wherein the preferred frame rate set is a frame rate range for guaranteeing stable performance of the playing terminal, and the method comprises the following steps: acquiring frame rate, resolution, code rate and playing duration from each video historically played by a playing terminal as video parameters; respectively acquiring initial data of the computing capacity occupancy rate, the residual storage space, the endurance time and the heating temperature of the playing terminal in the historical playing process of the playing terminal as initial equipment parameters; respectively acquiring the data changes of the computing capacity occupancy rate, the residual storage space, the endurance time and the heating temperature of the playing terminal in the historical playing process of the playing terminal as the equipment parameter changes; calculating the influence factor of each video parameter on the performance of each device when the video is played; calculating a preferred frame rate set under different initial equipment parameters and different video parameters;
The step of calculating the preferred frame rate set for different initial device parameters and different video parameters comprises: determining a first set of frame rates related to video parameters and computing power occupancy; determining a second set of frame rates associated with the video parameters and the remaining storage space; determining a third set of frame rates related to video parameters and endurance time; determining a fourth set of frame rates related to the video parameters and the heat generation temperature; determining a preferred frame rate set based on the first, second, third, and fourth frame rate sets;
the preferred set of frame rates is determined as follows:
a first set of frame rates related to video parameters and computing power occupancy is determined as follows:
；
；
；
；
wherein ,in order to not allow the playing period of opening the frame operation authority, the computing capability occupancy rate of the playing terminal under the common influence of the initial frame rate and other video parameters;In order to allow the playing period of opening the frame operation authority, the computing capacity occupancy rate of the playing terminal under the common influence of the frame rate to be adjusted and other video parameters;The initial computing capacity occupancy rate of the playing terminal;The upper limit of the occupancy rate of the preset computing capacity of the playing terminal;
For the duration of the play period for which the frame operation authority is not allowed to be opened,/for the duration of the play period for which the frame operation authority is not allowed to be opened>A duration of a play period for allowing the frame operation right to be opened;As a unit time impact factor of the initial frame rate in the video parameters on the computational power occupancy of the playback terminal,for the unit time influence factor of the to-be-adjusted frame rate in the video parameters on the computing capability occupancy rate of the playing terminal,/for>For the unit time influence factor of the resolution in the video parameters on the computing power occupancy of the playback terminal,For the unit time influence factor of the code rate in the video parameters on the computing capability occupancy rate of the playing terminal,The method comprises the steps that a unit time influence factor of playing time length in video parameters on the computing capability occupancy rate of a playing terminal is used; j is the video parameter corresponding to j, and the value range of j is [2,4 ]]；
According to g 1 Determining a corresponding optional frame rate, and according toDetermining a corresponding alternative frame rate, the two alternative frame rates intersecting to determine a first set of frame rates G 1 ；
A second set of frame rates associated with the video parameters and the remaining storage space is determined as follows:
；
；
wherein ,for the preset lower limit of the remaining memory space of the playback terminal, < > for>For the remaining storage space of the playback terminal after the current video has been played back,/the playback terminal is provided with a memory for storing the video data >The initial remaining storage space for the playing terminal;For the unit time influence factor of the initial frame rate in the video parameters on the remaining memory space of the playback terminal, is>For the unit time influence factor of the frame rate to be adjusted in the video parameters on the remaining memory space of the playback terminal,/for the video parameters>For the resolution in the video parameters to the unit time influence factor of the remaining memory space of the playback terminal,For the unit time influence factor of the code rate in the video parameters to the remaining storage space of the playing terminal, +.>The unit time influence factor of the play time length in the video parameters on the residual storage space of the play terminal is used;
according toDetermining a corresponding optional frame rate to determine a second set of frame rates G 2 ；
A third set of frame rates related to video parameters and endurance is determined as follows:
；
；
wherein ,for the initial endurance of the playback terminal, +.>For the preset duration lower limit of the playing terminal, < +.>The method comprises the steps that the duration of a playing terminal after the current video is played is set;For the unit time influence factor of the initial frame rate in the video parameters to the duration of the playing terminal,For the unit time influence factor of the frame rate to be adjusted in the video parameters on the duration of the playing terminal,/for the unit time >For the unit time influence factor of resolution in video parameters on the endurance of the playback terminal, +.>For the unit time influence factor of the code rate in the video parameters to the duration of the playing terminal, +.>The method comprises the steps that a unit time influence factor of play duration in video parameters on the duration of a play terminal is used;
according toDetermining a corresponding optional frame rate to determine a firstThree frame rate set G 3 ；
A fourth set of frame rates related to video parameters and heat generation temperature is determined as follows:
；
；
；
；
wherein ,in order to not allow the playing time period of opening the frame operation authority, the temperature of the playing terminal under the common influence of the initial frame rate and other video parameters;In order to allow the playing period of opening the frame operation authority, the temperature of the playing terminal under the common influence of the frame rate to be adjusted and other video parameters;For the initial temperature of the playback terminal, +.>The upper limit of the preset temperature of the playing terminal is set;
for the unit time influence factor of the initial frame rate in the video parameters on the temperature of the playback terminal, +.>For the unit time influence factor of the frame rate to be adjusted in the video parameters on the temperature of the playback terminal,For the unit time influence factor of the resolution in the video parameters on the temperature of the playback terminal, +. >For the unit time influence factor of the code rate in the video parameters to the temperature of the playing terminal,/for>The unit time influence factor of the playing time length in the video parameters on the temperature of the playing terminal is used;
according toDetermining a corresponding optional frame rate and according to +.>Determining a corresponding alternative frame rate, the two alternative frame rates intersecting to determine a fourth set of frame rates G 4 ；
Determining a preferred set of frame rates G:。
2. the adaptive video frame processing method according to claim 1, wherein the step of generating the adaptive frame operation module by the cloud server includes:
the cloud server calculates frame operation limiting conditions under different video parameters and initial equipment parameters, wherein the frame operation limiting conditions comprise a preferable frame rate set, an allowable frame inserting quantity and an allowable frame subtracting quantity;
the cloud server establishes mapping relations among different video parameters, initial equipment parameters and frame operation limiting conditions;
according to the mapping relation, the video parameters and the real-time equipment parameters of the playing terminal are used as input conditions of the self-adaptive frame operation module, and the frame operation limiting conditions are set as control parameters output by the self-adaptive frame operation module, so that the self-adaptive frame operation module can adaptively adjust the frame operation limiting conditions according to the difference of the video parameters and the real-time equipment parameters.
3. The adaptive video frame processing method according to claim 1, wherein the step of detecting real-time device parameters and video parameters of the playback terminal when playing video, closing frame operation permission during a playback period in which frame operation permission is not allowed to be opened, and transmitting the adaptive frame operation module to the playback terminal during a playback period in which frame operation permission is allowed to be opened, so as to open frame operation permission at the playback terminal through the adaptive frame operation module, comprises:
when playing video, sending the detected real-time equipment parameters and video parameters of the playing terminal to a cloud server;
the cloud server determines corresponding conditions for allowing the frame operation permission to be opened and conditions for not allowing the frame operation permission to be opened;
the cloud server determines a first playing period of not allowing the frame operation permission to be opened according to the condition of not allowing the frame operation permission to be opened;
the cloud server determines a second playing period for allowing the frame operation permission to be opened according to the condition for allowing the frame operation permission to be opened;
transmitting the first playing period and the second playing period to a playing terminal;
the playing terminal closes the frame operation authority in a first playing period;
and the cloud server sends the adaptive frame operation module to the playing terminal in the second playing period so as to start the frame operation authority through the adaptive frame operation module at the playing terminal.
4. The adaptive video frame processing method according to claim 3, wherein the step of detecting real-time device parameters and video parameters of the playback terminal when playing video, closing frame operation permission during a playback period in which frame operation permission is not allowed to be opened, and transmitting the adaptive frame operation module to the playback terminal during a playback period in which frame operation permission is allowed to be opened, so as to open frame operation permission at the playback terminal through the adaptive frame operation module, further comprises:
the cloud server calculates a first frame operation permission closing condition triggered by a single equipment parameter;
the cloud server calculates a second frame operation permission closing condition triggered by the combination of a plurality of equipment parameters;
the cloud server acquires the equipment parameters of the playing terminal to determine whether a first frame operation permission closing condition or a second frame operation permission closing condition is reached;
if yes, sending a frame closing operation permission instruction to the self-adaptive frame operation module so as to stop frame operation.
5. The adaptive video frame processing method according to claim 1, wherein the step of the playback terminal detecting a continuation of a video frame to be played back and determining whether to trigger a frame operation instruction during a period when the adaptive frame operation module opens the frame operation permission includes:
When the self-adaptive frame operation module is generated, the playing terminal detects the inter-frame image block motion relation of the decoded video frame to be played;
determining image block displacement vectors of adjacent frames according to the inter-frame image block motion relationship;
determining whether to trigger a frame operation instruction according to the image block displacement vectors of adjacent frames and real-time equipment parameters, wherein the frame operation instruction comprises a frame inserting instruction or a frame subtracting instruction;
if yes, triggering a frame operation instruction.
6. The adaptive video frame processing method according to any one of claims 1 to 5, further comprising:
the cloud server acquires the device parameter change increment of the playing terminal caused in the frame operation process;
and the cloud server adjusts the control condition of the self-adaptive frame operation module on the frame operation according to the initial equipment parameter of the playing terminal, the equipment parameter change and the equipment parameter change increment caused by the frame operation.
7. A playing terminal, configured to perform the method according to any one of claims 1 to 6, where the playing terminal is configured to communicatively connect to a cloud server.
CN202310970874.9A 2023-08-03 2023-08-03 Self-adaptive video frame processing method and playing terminal Active CN116684668B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310970874.9A CN116684668B (en) 2023-08-03 2023-08-03 Self-adaptive video frame processing method and playing terminal

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310970874.9A CN116684668B (en) 2023-08-03 2023-08-03 Self-adaptive video frame processing method and playing terminal

Publications (2)

Publication Number Publication Date
CN116684668A CN116684668A (en) 2023-09-01
CN116684668B true CN116684668B (en) 2023-10-20

Family

ID=87787716

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310970874.9A Active CN116684668B (en) 2023-08-03 2023-08-03 Self-adaptive video frame processing method and playing terminal

Country Status (1)

Country Link
CN (1) CN116684668B (en)

Citations (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003047268A2 (en) * 2001-11-30 2003-06-05 Telefonaktiebolaget Lm Ericsson (Publ) Global motion compensation for video pictures
WO2013135141A1 (en) * 2012-03-12 2013-09-19 华为技术有限公司 System, terminal and method for dynamically adjusting video
EP2713609A1 (en) * 2012-09-28 2014-04-02 Stockholms Universitet Holding AB Dynamic delay handling in mobile live video production systems
CN104282286A (en) * 2014-10-30 2015-01-14 广东欧珀移动通信有限公司 Method and system for adjusting display refresh rate of smart mobile terminal according to application scene
WO2015017541A1 (en) * 2013-07-30 2015-02-05 Robert Bosch Gmbh Adaptive methods for wireless camera communication
CN106341700A (en) * 2016-09-05 2017-01-18 Tcl集团股份有限公司 Method and system for automatically adjusting video frame rate
WO2017113600A1 (en) * 2015-12-30 2017-07-06 深圳Tcl数字技术有限公司 Video playing method and device
WO2018000567A1 (en) * 2016-06-30 2018-01-04 宇龙计算机通信科技(深圳)有限公司 Frame rate adjustment method and device
CN108574874A (en) * 2017-03-10 2018-09-25 杭州海康威视数字技术股份有限公司 A kind of video playout speed method of adjustment and device
WO2018177165A1 (en) * 2017-03-30 2018-10-04 上海七牛信息技术有限公司 Method and system for optimizing quality network pushed stream
CN110012315A (en) * 2019-03-15 2019-07-12 北京奇艺世纪科技有限公司 A kind of video broadcasting method, device and electronic equipment
CN110401870A (en) * 2019-07-30 2019-11-01 努比亚技术有限公司 Method of adjustment, terminal and the computer readable storage medium of record screen frame per second
CN110446072A (en) * 2019-08-14 2019-11-12 咪咕视讯科技有限公司 Video stream switching method, electronic device and storage medium
CN110881136A (en) * 2019-11-14 2020-03-13 腾讯科技（深圳）有限公司 Video frame rate control method and device, computer equipment and storage medium
CN111345772A (en) * 2018-12-20 2020-06-30 重庆金山医疗器械有限公司 Method for adjusting image acquisition frame rate and capsule endoscope system
WO2021238473A1 (en) * 2020-05-29 2021-12-02 Oppo广东移动通信有限公司 Screen refresh rate adjusting method and apparatus, electronic device, and storage medium
WO2022111198A1 (en) * 2020-11-26 2022-06-02 展讯通信（上海）有限公司 Video processing method and apparatus, terminal device and storage medium
CN114584814A (en) * 2020-11-30 2022-06-03 华为技术有限公司 Method for adjusting power consumption of terminal, method for adjusting code rate and related equipment
CN114584835A (en) * 2022-02-10 2022-06-03 百果园技术(新加坡)有限公司 Video processing method, device, equipment and storage medium
CN115334335A (en) * 2022-07-13 2022-11-11 北京优酷科技有限公司 Video frame insertion method and device
CN115509764A (en) * 2022-11-09 2022-12-23 湖南马栏山视频先进技术研究院有限公司 Real-time rendering multi-GPU parallel scheduling method and device and memory
CN115713923A (en) * 2022-11-22 2023-02-24 北京小米移动软件有限公司 Terminal control method, device, medium and chip
WO2023065873A1 (en) * 2021-10-22 2023-04-27 荣耀终端有限公司 Frame rate adjustment method, terminal device, and frame rate adjustment system

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105872570A (en) * 2015-12-11 2016-08-17 乐视网信息技术（北京）股份有限公司 Method and apparatus for implementing multi-camera video synchronous playing
US20170171638A1 (en) * 2015-12-15 2017-06-15 Le Holdings (Beijing) Co., Ltd. Method and Terminal for Video Play
US10097746B2 (en) * 2016-07-01 2018-10-09 Tribune Broadcasting Company, Llc Method and system for dynamically adjusting a camera setting for a video stream

Patent Citations (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003047268A2 (en) * 2001-11-30 2003-06-05 Telefonaktiebolaget Lm Ericsson (Publ) Global motion compensation for video pictures
WO2013135141A1 (en) * 2012-03-12 2013-09-19 华为技术有限公司 System, terminal and method for dynamically adjusting video
EP2713609A1 (en) * 2012-09-28 2014-04-02 Stockholms Universitet Holding AB Dynamic delay handling in mobile live video production systems
WO2015017541A1 (en) * 2013-07-30 2015-02-05 Robert Bosch Gmbh Adaptive methods for wireless camera communication
CN104282286A (en) * 2014-10-30 2015-01-14 广东欧珀移动通信有限公司 Method and system for adjusting display refresh rate of smart mobile terminal according to application scene
WO2017113600A1 (en) * 2015-12-30 2017-07-06 深圳Tcl数字技术有限公司 Video playing method and device
WO2018000567A1 (en) * 2016-06-30 2018-01-04 宇龙计算机通信科技(深圳)有限公司 Frame rate adjustment method and device
CN106341700A (en) * 2016-09-05 2017-01-18 Tcl集团股份有限公司 Method and system for automatically adjusting video frame rate
CN108574874A (en) * 2017-03-10 2018-09-25 杭州海康威视数字技术股份有限公司 A kind of video playout speed method of adjustment and device
WO2018177165A1 (en) * 2017-03-30 2018-10-04 上海七牛信息技术有限公司 Method and system for optimizing quality network pushed stream
CN111345772A (en) * 2018-12-20 2020-06-30 重庆金山医疗器械有限公司 Method for adjusting image acquisition frame rate and capsule endoscope system
CN110012315A (en) * 2019-03-15 2019-07-12 北京奇艺世纪科技有限公司 A kind of video broadcasting method, device and electronic equipment
CN110401870A (en) * 2019-07-30 2019-11-01 努比亚技术有限公司 Method of adjustment, terminal and the computer readable storage medium of record screen frame per second
CN110446072A (en) * 2019-08-14 2019-11-12 咪咕视讯科技有限公司 Video stream switching method, electronic device and storage medium
CN110881136A (en) * 2019-11-14 2020-03-13 腾讯科技（深圳）有限公司 Video frame rate control method and device, computer equipment and storage medium
WO2021238473A1 (en) * 2020-05-29 2021-12-02 Oppo广东移动通信有限公司 Screen refresh rate adjusting method and apparatus, electronic device, and storage medium
WO2022111198A1 (en) * 2020-11-26 2022-06-02 展讯通信（上海）有限公司 Video processing method and apparatus, terminal device and storage medium
CN114584814A (en) * 2020-11-30 2022-06-03 华为技术有限公司 Method for adjusting power consumption of terminal, method for adjusting code rate and related equipment
WO2023065873A1 (en) * 2021-10-22 2023-04-27 荣耀终端有限公司 Frame rate adjustment method, terminal device, and frame rate adjustment system
CN114584835A (en) * 2022-02-10 2022-06-03 百果园技术(新加坡)有限公司 Video processing method, device, equipment and storage medium
CN115334335A (en) * 2022-07-13 2022-11-11 北京优酷科技有限公司 Video frame insertion method and device
CN115509764A (en) * 2022-11-09 2022-12-23 湖南马栏山视频先进技术研究院有限公司 Real-time rendering multi-GPU parallel scheduling method and device and memory
CN115713923A (en) * 2022-11-22 2023-02-24 北京小米移动软件有限公司 Terminal control method, device, medium and chip

Also Published As

Publication number Publication date
CN116684668A (en) 2023-09-01

Similar Documents

Publication Publication Date Title
CN103826121B (en) Scene switching detection based code rate control method in low-delay video coding
US20130304934A1 (en) Methods and systems for controlling quality of a media session
EP3637770A1 (en) Code rate adjustment method and device for encoding process
US8040947B2 (en) Bitrate control device for controlling bitrate of video data
JP4273207B2 (en) Method and apparatus for encoding or decoding digital video data
CN107846590A (en) The method and video encoder of a kind of Video coding
CN111491201B (en) Method for adjusting video code stream and video frame loss processing method
CN111614909B (en) Automatic exposure control method, equipment and storage medium
CN110708578A (en) Video code rate self-adaptive adjusting method
CN115242759B (en) Cloud game processing method, cloud game processing device, computer equipment and medium
EP1487210B1 (en) Image data processing method of mobile communication terminal
CN116684668B (en) Self-adaptive video frame processing method and playing terminal
JPH09130787A (en) Dynamic image coder
EP3664456A1 (en) Apparatus and method for playing streamed media
US7382401B2 (en) Data processing apparatus
RU2485711C2 (en) Method of controlling video bitrate, apparatus for controlling video bitrate, machine-readable recording medium on which video bitrate control program is recorded
CN101217615A (en) An image compression bit rate control method and device
Nguyen et al. WISH: User-centric bitrate adaptation for HTTP adaptive streaming on mobile devices
US6943833B1 (en) Electronic camera
CN116074513A (en) Video coding method applied to network camera, computer readable medium and electronic equipment
CN112019918B (en) Video playing method and device
CN115190325A (en) Frame loss control method, device, equipment, storage medium and program product
CN114630120A (en) Video compression method and circuit system based on self-adaptive compression rate
CN115037701B (en) Video processing method, device, server and medium
CN117998042B (en) Network conference optimization method and system and conference distribution and control integrated machine

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant