CN110149560B

CN110149560B - Player optimization method and system based on HLS protocol, storage medium and terminal

Info

Publication number: CN110149560B
Application number: CN201910487532.5A
Authority: CN
Inventors: 张超; 朱善宝; 肖友能
Original assignee: Otvcloud Internet Technology Shanghai Co ltd
Current assignee: Otvcloud Internet Technology Shanghai Co ltd
Priority date: 2019-06-05
Filing date: 2019-06-05
Publication date: 2021-11-16
Anticipated expiration: 2039-06-05
Also published as: CN110149560A

Abstract

The invention provides a player optimization method and system based on an HLS protocol, a storage medium and a terminal, and the method comprises the following steps: when the player starts playing, judging whether the current video stream based on the HLS protocol can be detected forwards; if yes, forward detection is carried out on the current video stream to obtain a transmission stream media fragment file with preset duration; downloading a transmission stream media fragment file of the current video stream based on the current code rate; and replacing the current code rate based on the bandwidth change, and downloading the transport stream media fragment file of the current video stream based on the replaced current code rate. The player optimization method and system based on the HLS protocol, the storage medium and the terminal can select the proper code rate to play based on the current bandwidth, thereby ensuring the fluency of video playing and greatly improving the user experience.

Description

Player optimization method and system based on HLS protocol, storage medium and terminal

Technical Field

The present invention relates to the technical field of players, and in particular, to a player optimization method and system, a storage medium, and a terminal based on an HLS protocol.

Background

With the rise of the internet, devices such as internet televisions and television set-top boxes have been used as an infrastructure to reach various households. Users watch video programs on the Internet through the television, so that much pleasure is brought to life. However, the network environment of the internet varies greatly, so that the stability is not as stable as that of a wire network. Therefore, almost every hardware manufacturer develops a player for its own hardware in order to guarantee the fluency of the video watched by the user. These players, however, vary widely in performance, and some are even directly modified based on open source code.

In the prior art, an open source player based on the IJK is widely used. Being an open source player, it accommodates a wide variety of streaming situations. However, since the player is a general-purpose open source player, there is no limitation on the usage scenario, and therefore, it is impossible to optimize and customize for a certain scenario or a certain stream, resulting in a small difference in performance among all players.

Currently, in order to cope with the difference of bandwidth, many players enable users to independently and manually select viewing quality such as fluency, high definition, super definition and the like when viewing videos, and then play videos with different code rates such as 360P, 480P and 720P according to the quality selected by the users. But this method is only applicable to situations where the bandwidth does not change. If the bandwidth fluctuates and the user selects a higher bitrate, the player will frequently be stuck.

Disclosure of Invention

In view of the above drawbacks of the prior art, an object of the present invention is to provide a method and a system for optimizing a player based on an HLS protocol, a storage medium, and a terminal, which can select a proper bitrate to play based on a current bandwidth, thereby ensuring fluency of video playing and greatly improving user experience.

To achieve the above and other related objects, the present invention provides a player optimization method based on HLS protocol, including the following steps: when the player starts playing, judging whether the current video stream based on the HLS protocol can be detected forwards; if yes, forward detection is carried out on the current video stream to obtain a transmission stream media fragment file with preset duration; downloading a transmission stream media fragment file of the current video stream based on the current code rate; and replacing the current code rate based on the bandwidth change, and downloading the transport stream media fragment file of the current video stream based on the replaced current code rate.

In an embodiment of the present invention, the transport stream media segment file of the current video stream is detected forward based on a naming rule of the transport stream media segment file.

In an embodiment of the present invention, when downloading the transport stream media fragment file of the current video stream, the index file of the current video stream is downloaded first, and then the corresponding transport stream media fragment file is downloaded based on the index file.

In an embodiment of the present invention, when the current code rate is changed based on a bandwidth change, a mode of increasing gradually is adopted to change from a low code rate to a high code rate.

In an embodiment of the present invention, when the current bitrate is changed based on a bandwidth change and a high bitrate is changed to a low bitrate, the high bitrate is changed to a lowest bitrate first, and then the lowest bitrate is changed to the low bitrate.

In an embodiment of the present invention, when the current bit rate is changed based on a bandwidth change, the current bit rate is changed after at least downloading a preset number of transport stream media slicing files corresponding to the current bit rate.

In an embodiment of the present invention, when the current code rate is changed based on bandwidth change, if the cache is insufficient, the current code rate is directly reduced to a code rate matched with the current bandwidth.

Correspondingly, the invention provides a player optimization system based on the HLS protocol, which comprises a forward detection module, a download module and a replacement module;

the forward detection module is used for judging whether the current video stream based on the HLS protocol can be subjected to forward detection when the player starts playing; if yes, forward detection is carried out on the current video stream to obtain a transmission stream media fragment file with preset duration;

the downloading module is used for downloading the transmission stream media fragment file of the current video stream based on the current code rate;

and the replacing module is used for replacing the current code rate based on the bandwidth change and downloading the transport stream media slicing file of the current video stream based on the replaced current code rate.

The present invention provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the above-described player optimization method based on the HLS protocol.

Finally, the present invention provides a terminal comprising: a processor and a memory;

the memory is used for storing a computer program;

the processor is configured to execute the computer program stored in the memory, so as to enable the terminal to execute the above-mentioned player optimization method based on the HLS protocol.

As described above, the player optimization method and system, the storage medium, and the terminal based on the HLS protocol according to the present invention have the following advantages:

(1) the most suitable code rate can be selected according to the current bandwidth for playing, and the phenomenon of pause during playing is avoided through code rate self-adaption;

(2) the forward detection technology is adopted for the live Stream, when the player starts playing, a Transport Stream (TS) media fragment file which is generated before the live Stream is detected, and the content of the generated TS media fragment file is downloaded and added into a cache, so that the playing jam condition of the player caused by the network jitter problem is resisted;

(3) through delayed playing, the HLS protocol-based player sacrifices certain real-time performance, fluency is realized, and watching experience of users of live television and television set top boxes is greatly improved.

Drawings

FIG. 1 is a flow chart illustrating a method for optimizing a player based on an HLS protocol according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a player optimization system based on HLS protocol according to an embodiment of the present invention;

fig. 3 is a schematic structural diagram of a terminal according to an embodiment of the invention.

Description of the element reference numerals

21 forward detection module

22 download module

23 replacement module

31 processor

32 memory

Detailed Description

The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.

It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.

The player optimization method and system based on the HLS protocol, the storage medium and the terminal can adaptively select the proper code rate to play based on the current bandwidth, and meanwhile, the smoothness of video playing is ensured by adopting the delay cache and preloading technology, the pause phenomenon in the video playing process is avoided, the watching experience of a user is greatly improved, and the method and system are suitable for application scenes such as smart televisions, television boxes and the like.

As shown in fig. 1, in an embodiment, the method for optimizing a player based on the HLS protocol of the present invention includes the following steps:

step S1, when the player starts playing, judging whether the current video stream based on HLS protocol can be detected forward; if yes, forward detection is carried out on the current video stream to obtain a transmission stream media fragment file with preset duration.

Specifically, hls (HTTP Live streaming) is an HTTP-based streaming media network transport protocol proposed by apple inc. It works by dividing the entire stream into small HTTP-based files to download, only a few at a time. When the media stream is playing, the client can choose to download the same resource from many different backup sources at different rates, allowing the streaming media session to adapt to different data rates, including an M3U8 index file, TS media slice file, and key encryption string file.

Under the HLS protocol, an example of an M3U8 index file for a live video stream is as follows:

#EXTM3U

#EXT-X-VERSION:3

#EXT-X-TARGETDURATION:2

#EXT-X-MEDIA-SEQUENCE:1711

#EXTINF:2.000000,

2300/20181030/20181030T190311.ts

#EXTINF:2.000000,

2300/20181030/20181030T190313.ts

#EXTINF:2.000000,

2300/20181030/20181030T190315.ts

#EXTINF:2.000000,

2300/20181030/20181030T190317.ts

#EXTINF:2.000000,

2300/20181030/20181030T190319.ts

under the HLS protocol, an example of an M3U8 index file for an on-demand video stream is as follows:

#EXTM3U

#EXT-X-VERSION:3

#EXT-X-TARGETDURATION:2

#EXT-X-MEDIA-SEQUENCE:1711

#EXTINF:2.000000,

2300/20181030/20181030T190311.ts

#EXTINF:2.000000,

2300/20181030/20181030T190313.ts

#EXTINF:2.000000,

2300/20181030/20181030T190315.ts

#EXTINF:2.000000,

2300/20181030/20181030T190317.ts

#EXTINF:2.000000,

2300/20181030/20181030T190319.ts

#ENDLIST

from the above, the biggest difference between the live video stream and the on-demand video stream is: the on-demand video stream will have a word of # ENDLIST at the last end of M3U8, and the rest will be identical.

Since the on-demand video stream is to store all TS media segment files in the M3U8 file, there is no unstable playing caused by network jitter. Therefore, the player optimization method based on the HLS protocol is mainly performed aiming at live video streaming. As can be seen from the M3U8 of the live video stream, each TS file is 2 seconds, and there is a delay of about 10 seconds in the M3U8 file (2 seconds by 5 pieces to 10 seconds).

Video streaming according to the HLS protocol typically has several different bit rates, such as smooth, high definition, super definition, and different M3U8 files for different bit rates. The code rate is the number of data bits transmitted per unit time during data transmission. The bandwidth is the transfer rate or the maximum number of bytes transferred per second. Different bandwidths correspond to different code rates. Therefore, the player needs to acquire the current corresponding M3U8 file in real time according to the current network bandwidth, so as to acquire the corresponding TS media fragment file.

When the player plays the video, the buffer amount of the TS media fragment file determines whether the video playing is blocked or not. In the live broadcasting process, the player acquires a few TS media fragment files as much as possible. This is because the name of the subsequent TS media segment file cannot be known during the live broadcast process. As can be seen from the contents of the M3U8 file, the file format and the file name of the M3U8 file have certain rules. Therefore, the file format and file name of the previous M3U8 file can be calculated. Although there is a possibility of error in the estimation, the file name estimated according to the format has at least 95% accuracy. Therefore, the player optimization based on the HLS protocol adopts a delay cache mode to preload the TS media fragment file detected forward. Specifically, when the player starts playing, it is determined whether the current video stream based on the HLS protocol can be probed forward. If forward detection can be performed, forward detection is performed on the current video stream based on the naming rule of the TS media segment file to obtain a transport stream media segment file with a preset duration, such as a forward detection 6s TS media segment file. For example, a TS file named 2300/20190306/20190306t113028.TS, if the interval time is 2 seconds, the name of the resulting TS media segment file can be presumed to be:

2300/20190306/20190306T113026.ts

2300/20190306/20181030T113024.ts

2300/20190306/20181030T113022.ts

…………..

and preloading the TS media fragment file detected forwards, and storing the TS media fragment file in a cache. When the network is not good, the normal playing can still be carried out within a certain time, thereby realizing the purpose of resisting the network jitter.

It should be noted that if the forward probing cannot be performed, no operation is performed.

And step S2, downloading the transport stream media slicing file of the current video stream based on the current code rate.

Specifically, the matched code rate is adaptively selected according to the current bandwidth, so that the transport stream media fragment file of the current video stream is downloaded. When downloading the transport stream media fragment file of the current video stream, firstly downloading the index file of the current video stream, and then downloading the corresponding transport stream media fragment file based on the index file.

And step S3, replacing the current code rate based on the bandwidth change, and downloading the transport stream media slicing file of the current video stream based on the replaced current code rate.

Specifically, if the current bandwidth changes, the current bitrate needs to be changed according to the change condition of the bandwidth, so as to ensure the smoothness of video playing. Wherein, the replacement code rate needs to follow the following rules:

(1) and when the current code rate is changed based on the bandwidth change, changing from a low code rate to a high code rate in a step-by-step increasing mode. That is, the code rate must be increased one by one during the rising process. The video stream is set to have 3 code rates, namely 700 code rates, 1300 code rates and 2300 code rates. For example, the code rate 700, the code rate 1300, and the code rate 2300 must be sequentially passed from the code rate 700 to the code rate 2300. Specifically, TS (a) _ B is used for representing the code rate, TS is represented as a TS media slicing file, a represents a serial number, and B represents the code rate of the TS media slicing file. The code rate change corresponding to the user downloading the TS file is as follows: TS (1) _700 code rate, TS (2) _700 code rate, TS (3) _1300 code rate, TS (4) _1300 code rate, TS (5) _2300 code rate, TS (6) _2300 code rate.

(2) And when the current code rate is changed based on the bandwidth change, at least downloading the transport stream media slicing files corresponding to the preset number of the current code rate and then changing the current code rate. That is, the current code rate is maintained as much as possible, and an inertia principle is adopted for the change of the code rate. The change can be performed only after at least a preset number of TS media fragment files appear. Preferably, the preset number is 2. For example, when the bandwidth only satisfies the user playing 1300 bit rate, the bit rate corresponding to the user downloading the TS media segment file changes as follows: TS (1) _700 code rate, TS (2) _700 code rate, TS (3) _1300 code rate, TS (4) _1300 code rate, TS (5) _1300 code rate, TS (6) _1300 code rate.

(3) And when the current code rate is changed based on bandwidth change, if the cache is insufficient, directly reducing the current code rate to the code rate matched with the current bandwidth. Specifically, when the download buffer is insufficient, the lazy principle will not work, but directly reduce the current code rate to the most suitable code rate. For example, when the user is performing 2300-bit rate smooth playing, the network can only perform 700-bit rate playing when jitter occurs, and the bit rate changes as follows when the cache is insufficient: TS (N) _2300 code rate, TS (N +1) _2300 code rate, TS (N +2) _700 code rate, TS (N +3) _700 code rate.

(4) And when the current code rate is changed based on bandwidth change and the high code rate is changed to the low code rate, firstly, changing the high code rate to the lowest code rate, and then, changing the lowest code rate to the low code rate. For example, when the user network has jitter, the code rate changes from being able to smoothly view the 2300 code rate to being able to smoothly view the 1300 code rate as follows: TS (N) _2300 code rate, TS (N +1) _2300 code rate, TS (N +2) _700 code rate, TS (N +3) _700 code rate, TS (N +4) _1300 code rate, TS (N +5) _1300 code rate. When the user network shakes, the code rate changes from the code rate of 2300 capable of being watched smoothly to the code rate of 1300 capable of being played smoothly, and then the code rate changes when the code rate of 2300 capable of being played smoothly are as follows: TS (N) _2300 code rate, TS (N +1) _2300 code rate, TS (N +2) _700 code rate, TS (N +3) _700 code rate, TS (N +4) _1300 code rate, TS (N +5) _1300 code rate.

As shown in fig. 2, in an embodiment, the player optimization system based on the HLS protocol of the present invention includes a forward probing module 21, a downloading module 22, and a replacing module 23.

The forward detection module 21 is configured to determine whether a current video stream based on the HLS protocol can be forward detected when the player starts playing; if yes, forward detection is carried out on the current video stream to obtain a transmission stream media fragment file with preset duration;

the downloading module 22 is connected to the forward detection module 21, and configured to download the transport stream media fragment file of the current video stream based on the current bit rate;

the replacing module 23 is connected to the downloading module 22, and configured to replace the current bitrate based on a bandwidth change, and download the transport stream media fragment file of the current video stream based on the replaced current bitrate.

The structures and principles of the forward detection module 21, the download module 22 and the replacement module 23 all correspond to the optimization method of the player based on the HLS protocol one to one, and therefore, the details are not repeated herein.

It should be noted that the division of the modules of the above apparatus is only a logical division, and the actual implementation may be wholly or partially integrated into one physical entity, or may be physically separated. And the modules can be realized in a form that all software is called by the processing element, or in a form that all the modules are realized in a form that all the modules are called by the processing element, or in a form that part of the modules are called by the hardware. For example: the x module can be a separately established processing element, and can also be integrated in a certain chip of the device. In addition, the x-module may be stored in the memory of the apparatus in the form of program codes, and may be called by a certain processing element of the apparatus to execute the functions of the x-module. Other modules are implemented similarly. All or part of the modules can be integrated together or can be independently realized. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software. These above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), one or more microprocessors (DSPs), one or more Field Programmable Gate Arrays (FPGAs), and the like. When a module is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. These modules may be integrated together and implemented in the form of a System-on-a-chip (SOC).

The storage medium of the present invention has stored thereon a computer program which, when executed by a processor, implements the above-described player optimization method based on the HLS protocol. The storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic disk, U-disk, memory card, or optical disk.

As shown in fig. 3, in an embodiment, the terminal of the present invention includes: a processor 31 and a memory 32.

The memory 32 is used for storing computer programs.

The memory 32 includes: various media that can store program codes, such as ROM, RAM, magnetic disk, U-disk, memory card, or optical disk.

The processor 31 is connected to the memory 32, and is configured to execute the computer program stored in the memory 32, so that the terminal executes the above-mentioned player optimization method based on the HLS protocol.

Preferably, the Processor 31 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.

In summary, the player optimization method and system, the storage medium and the terminal based on the HLS protocol of the present invention can select the most suitable code rate according to the current bandwidth for playing, and avoid the pause phenomenon during playing through code rate adaptation; the forward detection technology is adopted for the live stream, TS media fragment files generated before the live stream are detected when the player starts playing, and the generated TS media fragment file contents are downloaded and added into the cache, so that the playing jam condition of the player caused by the network jitter problem is resisted; through delayed playing, the HLS protocol-based player sacrifices certain real-time performance, fluency is realized, and watching experience of users of live television and television set top boxes is greatly improved. Therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.

The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.

Claims

1. A player optimization method based on HLS protocol is characterized by comprising the following steps:

when the player starts playing, judging whether the current video stream based on the HLS protocol can be detected forwards; if yes, forward detection is carried out on the current video stream to obtain a transmission stream media fragment file with preset duration;

downloading a transmission stream media fragment file of the current video stream based on the current code rate;

changing the current code rate based on the bandwidth change, and downloading the transport stream media fragment file of the current video stream based on the changed current code rate;

when the current code rate is changed based on bandwidth change and the high code rate is changed to the low code rate, firstly, the high code rate is changed to be the lowest code rate, and then, the lowest code rate is changed to be the low code rate;

and when the current code rate is changed based on the bandwidth change, at least downloading the transport stream media slicing files corresponding to the preset number of the current code rate and then changing the current code rate.

2. The method of claim 1, wherein the transport stream media slice file of the current video stream is probed forward based on a naming rule of the transport stream media slice file.

3. The method of claim 1, wherein when downloading the transport stream media slice file of the current video stream, the method first downloads the index file of the current video stream, and then downloads the corresponding transport stream media slice file based on the index file.

4. The optimization method of player based on HLS protocol according to claim 1, wherein when the current bitrate is changed based on a bandwidth change, it is changed from a low bitrate to a high bitrate in a stepwise increasing manner.

5. The optimization method of player based on HLS protocol according to claim 1, wherein when the current bitrate is changed based on a bandwidth change, if the cache is insufficient, the current bitrate is directly reduced to a bitrate matched with the current bandwidth.

6. The player optimization system based on the HLS protocol is characterized by comprising a forward detection module, a download module and a replacement module;

the replacing module is used for replacing the current code rate based on bandwidth change and downloading the transport stream media slicing file of the current video stream based on the replaced current code rate;

7. A storage medium on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out the method for player optimization based on the HLS protocol of any one of claims 1 to 5.

8. A terminal, comprising: a processor and a memory;

the memory is used for storing a computer program;

the processor is configured to execute the computer program stored in the memory to cause the terminal to execute the HLS protocol-based player optimization method according to any one of claims 1 to 5.