CN113556620A

CN113556620A - Media playing method, device and system

Info

Publication number: CN113556620A
Application number: CN202110820428.0A
Authority: CN
Inventors: 吴坚强; 陈小波; 谭嵩; 罗准; 张文兵; 冯斌
Original assignee: Hunan MgtvCom Interactive Entertainment Media Co Ltd
Current assignee: Hunan MgtvCom Interactive Entertainment Media Co Ltd
Priority date: 2021-07-20
Filing date: 2021-07-20
Publication date: 2021-10-26

Abstract

The application provides a media playing method, a device and a system, wherein the method comprises the following steps: the method comprises the steps that a server obtains attribute information corresponding to each data frame contained in sub-media data, key information corresponding to each key frame contained in the sub-media data is obtained according to the attribute information corresponding to each data frame contained in the sub-media data, the key information corresponding to each key frame contained in the sub-media data and index information of the sub-media data are distributed to a CDN, then a client side obtains the index information from the CDN and the key information corresponding to each key frame contained in the sub-media data, data are downloaded, decoded and played based on the index information, when fast forward operation or fast backward operation is detected, a timestamp matched with an operation time point is determined from the obtained key information, and the data are obtained, decoded and played from a position indicated by position information in the key information where the timestamp is located. According to the method and the device, the bandwidth of the CDN can be effectively saved, and the playing experience of the user is better.

Description

Media playing method, device and system

Technical Field

The present application relates to the field of video processing technologies, and in particular, to a method, an apparatus, and a system for media playing.

Background

In the existing media playing method, a client sequentially plays each piece of sub-media data obtained by segmenting target media data through a player, and when the client detects fast-forward operation or fast-backward operation of a user on the target media data, the client downloads the sub-media data from a first frame after positioning the sub-media data matched with an operation time point, and decodes and plays the data from the first frame after downloading the sub-media data.

Since the fast forward operation or fast backward operation of the user may not be at the first frame of the sub-media data, the existing media playing method can only download data from the first frame of the sub-media data matching the operation time point, which wastes bandwidth, and the existing media playing method can only decode and play data from the first frame of the sub-media data matching the operation time point, which results in poor user playing experience.

Disclosure of Invention

In view of this, the present application provides a media playing method, apparatus and system, so as to solve the problems of bandwidth waste and poor user experience caused by downloading, decoding and playing data from the first frame of sub-media data in the prior art, and the technical solution is as follows:

a media playing method is applied to a server and comprises the following steps:

acquiring attribute information corresponding to each data frame contained in sub-media data, wherein the sub-media data is obtained by segmenting target media data, and the attribute information corresponding to one data frame comprises a key frame identifier for indicating whether the data frame is a key frame, position information of the data frame and a timestamp of the data frame;

acquiring key information corresponding to each key frame contained in the sub-media data according to attribute information corresponding to each data frame contained in the sub-media data, wherein the key information corresponding to a key frame comprises a time stamp of the key frame and position information corresponding to the key frame, and the position information corresponding to the key frame is the position information of the key frame or the position information of the data frame of which the time stamp is positioned near the time stamp of the key frame;

and distributing key information corresponding to each key frame contained in the sub-media data and index information of the sub-media data to the CDN, so that when a client detects fast-forward operation or fast-backward operation of a user in the process of playing the target media data, a timestamp matched with an operation time point is determined from the key information corresponding to each key frame contained in the sub-media data, data is obtained from a position indicated by position information in the key information where the determined timestamp is located, and the obtained data is decoded and played.

Optionally, obtaining key information corresponding to each key frame included in the sub-media data according to attribute information corresponding to each data frame included in the sub-media data, respectively, includes:

determining each key frame contained in the sub-media data according to a key frame identifier which is used for indicating whether the data frame is a key frame in the attribute information respectively corresponding to each data frame contained in the sub-media data;

acquiring a time stamp of each key frame contained in the sub-media data from attribute information respectively corresponding to each key frame contained in the sub-media data;

and acquiring the position information corresponding to each key frame contained in the sub-media data according to the position information of the data frame and the time stamp of the data frame in the attribute information corresponding to each data frame contained in the sub-media data.

Optionally, the target media data is audio/video data;

acquiring the position information corresponding to each key frame contained in the sub-media data according to the position information of each data frame and the timestamp of each data frame in the attribute information corresponding to each data frame contained in the sub-media data, including:

if the sub-media data comprises audio frames and video frames, for each key frame contained in the sub-media data:

determining a timestamp which is positioned in front of the timestamp of the key frame and is closest to the timestamp of the key frame from attribute information corresponding to each audio frame contained in the sub-media data of the key frame as a target timestamp;

and determining the position information in the attribute information where the target timestamp is located as the position information corresponding to the key frame.

A media playing method is applied to a client and comprises the following steps:

acquiring index information of sub-media data obtained by segmenting target media data from a CDN (content distribution network), and key information corresponding to each key frame contained in the sub-media data, wherein the key information corresponding to a key frame comprises a time stamp of the key frame and position information corresponding to the key frame, and the position information corresponding to the key frame is the position information of the key frame or the position information of a data frame of which the time stamp is positioned near the time stamp of the key frame;

downloading data based on the index information of the sub-media data, and decoding and playing the downloaded data;

when the fast forward operation or the fast backward operation of a user is detected, determining a timestamp matched with an operation time point from key information respectively corresponding to each key frame contained in the sub-media data;

and starting to acquire data from the position indicated by the position information in the key information where the determined time stamp is positioned, and decoding and playing the acquired data.

Optionally, determining a timestamp matched with the operation time point from key information respectively corresponding to each key frame included in the sub-media data includes:

and determining a timestamp which is positioned before the operation time point and is closest to the operation time point from key information corresponding to each key frame contained in the sub-media data as a timestamp matched with the operation time point.

A media playing device applied to a server comprises: the system comprises an attribute information acquisition module, a key information acquisition module and an information distribution module;

the attribute information acquisition module is used for acquiring attribute information corresponding to each data frame contained in the sub-media data, wherein the sub-media data is obtained by segmenting target media data, and the attribute information corresponding to one data frame comprises a key frame identifier for indicating whether the data frame is a key frame, position information of the data frame and a timestamp of the data frame;

a key information obtaining module, configured to obtain, according to attribute information corresponding to each data frame included in the sub-media data, key information corresponding to each key frame included in the sub-media data, where the key information corresponding to a key frame includes a timestamp of the key frame and position information corresponding to the key frame, and the position information corresponding to the key frame is position information of the key frame or position information of a data frame whose timestamp is located near the timestamp of the key frame;

the information distribution module is used for distributing key information corresponding to each key frame contained in the sub-media data and index information of the sub-media data to the CDN, so that when a client detects a fast-forward operation or a fast-backward operation of a user in the process of playing the target media data, a timestamp matched with an operation time point is determined from the key information corresponding to each key frame contained in the sub-media data, data is obtained from a position indicated by position information in the key information where the determined timestamp is located, and the obtained data is decoded and played.

Optionally, the key information obtaining module includes: the system comprises a key frame determining module, a timestamp acquiring module and a position information acquiring module;

a key frame determining module, configured to determine each key frame included in the sub-media data according to a key frame identifier, which is used to indicate whether a data frame is a key frame, in attribute information respectively corresponding to each data frame included in the sub-media data;

the timestamp acquisition module is used for acquiring timestamps of all key frames contained in the sub-media data from the attribute information respectively corresponding to all key frames contained in the sub-media data;

and the position information acquisition module is used for acquiring the position information corresponding to each key frame contained in the sub-media data according to the position information of the data frame and the timestamp of the data frame in the attribute information corresponding to each data frame contained in the sub-media data.

Optionally, the target media data is audio/video data;

if the sub-media data includes an audio frame and a video frame, the position information obtaining module includes, for each key frame included in the sub-media data: a target timestamp determining module and a location information determining module;

a target timestamp determining module, configured to determine, from attribute information corresponding to each audio frame included in the sub-media data in which the key frame is located, a timestamp that is located before the timestamp of the key frame and is closest to the timestamp of the key frame, as a target timestamp;

and the position information determining module is used for determining the position information in the attribute information of the target timestamp as the position information corresponding to the key frame.

A media playing device is applied to a client and comprises: the system comprises an information acquisition module, a data downloading and playing module, a timestamp determination module and a data acquisition and playing module;

the information acquisition module is used for acquiring index information of sub-media data obtained by segmenting target media data from the CDN and key information corresponding to each key frame contained in the sub-media data, wherein the key information corresponding to a key frame comprises a time stamp of the key frame and position information corresponding to the key frame, and the position information corresponding to the key frame is the position information of the key frame or the position information of a data frame of which the time stamp is positioned near the time stamp of the key frame;

the data downloading and playing module is used for downloading data based on the index information of the sub-media data and decoding and playing the downloaded data;

the timestamp determining module is used for determining a timestamp matched with the operation time point from key information respectively corresponding to each key frame contained in the sub-media data when the fast forward operation or the fast backward operation of the user is detected;

and the data acquisition and playing module is used for acquiring data from the position indicated by the position information in the key information where the determined timestamp is positioned, and decoding and playing the acquired data.

A media playing system comprises a server and a client;

the server is used for acquiring attribute information corresponding to each data frame contained in the sub-media data, wherein the sub-media data is obtained by segmenting target media data, and the attribute information corresponding to one data frame comprises a key frame identifier for indicating whether the data frame is a key frame, position information of the data frame and a timestamp of the data frame; determining key information corresponding to each key frame contained in the sub-media data according to attribute information corresponding to each data frame contained in the sub-media data, wherein the key information corresponding to a key frame comprises a time stamp of the key frame and position information corresponding to the key frame, and the position information corresponding to the key frame is the position information of the key frame or the position information of the data frame of which the time stamp is positioned near the time stamp of the key frame; distributing key information corresponding to each key frame contained in the sub-media data and index information of the sub-media data to the CDN;

the client is used for acquiring the index information of the sub-media data from the CDN and the key information corresponding to each key frame contained in the sub-media data; downloading data based on the index information of the sub-media data, and decoding and playing the downloaded data; when the fast forward operation or the fast backward operation of a user is detected, determining a timestamp matched with an operation time point from key information respectively corresponding to each key frame contained in the sub-media data; and starting to acquire data from the position indicated by the position information in the key information where the determined time stamp is positioned, and decoding and playing the acquired data.

It can be known from the foregoing technical solutions that, in the media playing method applied to the server provided by the present application, first, attribute information corresponding to each data frame included in the sub-media data is obtained, then, key information corresponding to each key frame included in the sub-media data is obtained according to the attribute information corresponding to each data frame included in the sub-media data, then, the key information corresponding to each key frame included in the sub-media data and the index information of the sub-media data are distributed to the CDN, so that when the client detects a fast-forward operation or a fast-rewind operation of a user during playing of the target media data, a timestamp matching an operation time point is determined from the key information corresponding to each key frame included in the sub-media data, and data is obtained from a position indicated by position information in the determined key information where the timestamp is located, and decodes and plays the acquired data. According to the method and the system, the key information corresponding to each key frame contained in the sub-media data is distributed to the CDN, so that the client can start to acquire, decode and play data from the position indicated by the position information in the key information where the timestamp matched with the operation time point is located, and compared with the prior art that the client starts to acquire, decode and play data from the first frame of the sub-media data matched with the operation time point, the CDN bandwidth can be effectively saved, and the user playing experience is better. In addition, the online CDN framework does not need to be modified, the application layer is not perceived, and the original media playing system is not invasive.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

Fig. 1 is a schematic flowchart of a media playing method applied to a server according to an embodiment of the present application;

FIG. 2 is a schematic diagram of seek operation provided in the present application;

fig. 3 is a schematic flowchart of a media playing method applied to a client according to an embodiment of the present disclosure;

fig. 4 is a schematic structural diagram of a media playing device applied to a server according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of a media playing device applied to a client according to an embodiment of the present disclosure;

fig. 6 is a block diagram of a hardware structure of a media playing device applied to a server according to an embodiment of the present application;

fig. 7 is a block diagram of a hardware structure of a media playing device applied to a client according to an embodiment of the present disclosure.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In view of the problems in the prior art, the present inventors have conducted extensive research, and finally have proposed a media playing method applied to a server and a media playing method applied to a client, and correspondingly have proposed a media playing apparatus applied to a server, a media playing apparatus applied to a client, and a media playing system. The media playing method, the device and the system can be suitable for scenes needing fast forward playing of media data or fast backward playing of the media data, and when the player installed on the client is used for playing the media data, the fast forward (or fast backward) button is clicked or a progress bar corresponding to the media data is clicked to realize fast forward (or fast backward) playing. The server may be one server, a server cluster formed by a plurality of servers, or a cloud computing service center. In this embodiment, the server may be connected to the client through a network, where the network includes but is not limited to: a wide area network, a metropolitan area network, or a local area network.

First, a detailed description will be given of a media playing method applied to a server according to the following embodiments.

Referring to fig. 1, a schematic flow chart of a media playing method applied to a server according to an embodiment of the present application is shown, where the method may include:

step S10, obtaining attribute information corresponding to each data frame included in the sub media data.

The sub-media data is obtained by carrying out segmentation processing on the target media data according to a set protocol. It should be noted that the target media data in this embodiment may be video data (only including video frames), or may also be audio and video data (including audio frames and video frames).

Each data frame included in the sub-media data corresponds to attribute information, optionally, the attribute information corresponding to a data frame includes a key frame identifier for indicating whether the data frame is a key frame, location information of the data frame, and a timestamp of the data frame, and optionally, the attribute information corresponding to a data frame may further include a data type of the data frame.

The position information of the data frame may be an offset position of the data frame in the sub-media data (optionally, the position information of the first frame data frame included in the sub-media data is 0), which may be used to indicate that the data frame is in the second frame of the sub-media data, the timestamp of the data frame may be a PTS (presentation time stamp), and the attribute information of a data frame is defined as follows:

after obtaining the attribute information corresponding to each data frame included in the sub media data, the embodiments of the present application may sort the obtained attribute information from small to large according to the position information of the data frames, and store the sorted attribute information to the memory, where if the target media data is video data, the attribute information corresponding to the video frames included in the sub media data is sorted and stored from small to large according to the position information, and if the target media data is audio/video data, the attribute information corresponding to the video frames and the audio frames included in the sub media data is sorted and stored from small to large according to the position information.

The setting protocol may be an HLS (HTTP Live Streaming) protocol, or may be a DASH (Dynamic Adaptive Streaming over HTTP) protocol.

It should be noted that, the set protocol specification includes an index information, for example, the HLS protocol specification includes an m3u (8) index information, the index information includes duration information of each sub-media data and sequence number information used for indicating a playing order of each sub-media data, and the index information corresponding to the HLS protocol specification is, for example, as follows:

# EXTM3U// fixed string

# EXT-X-VERSION:3// VERSION number of target media data

10// the maximum integer of the duration of each sub-media data

9.009,// duration of first sub-media data

Ts// first child media data storage URI (Uniform resource identifier)

9.009,// duration of second sub-media data

Ts// second child media data storage URI

3.003,// duration of the third sub-media data

Ts// third child media data storage URI

# EXT-X-ENDLIST// target media data end if the string does not exist as a live stream

Step S11, obtaining key information corresponding to each key frame included in the sub media data according to the attribute information corresponding to each data frame included in the sub media data.

If the target media data or the sub-media data needs to be decoded based on a key frame (I frame), the key information corresponding to each key frame included in the sub-media data needs to be obtained according to the attribute information corresponding to each data frame included in the sub-media data, so that the client finds the key frame included in the sub-media data based on the key information and decodes the key frame based on the key frame.

The key information corresponding to the key frame may include a timestamp of the key frame and position information corresponding to the key frame, and the position information corresponding to the key frame may be the position information of the key frame or the position information of a data frame whose timestamp is located near the timestamp of the key frame.

Preferably, if the target media data is video data, the position information corresponding to the key frame is position information of the key frame, and if the target media data is audio-video data, the position information corresponding to the key frame is position information of an audio frame of which the timestamp is located near the timestamp of the key frame.

Optionally, if the target media data is audio-video data, the key information corresponding to the key frame may further include a time stamp of an audio frame whose time stamp is located near the time stamp of the key frame, and based on this, the definition of the key information is as follows:

it can be understood that, in order to obtain the key information corresponding to each key frame included in the sub-media data, each key frame included in the sub-media data needs to be determined first, and based on this, the specific implementation process of this step may include:

step S111, determining each key frame included in the sub media data according to the key frame identifier indicating whether the data frame is a key frame in the attribute information corresponding to each data frame included in the sub media data.

In this step, whether each data frame included in each piece of sub media data is a key frame or not may be determined according to a key frame identifier indicating whether the data frame is a key frame, so as to obtain each key frame included in each piece of sub media data. For example, if the target media data is divided into 4 pieces of sub-media data, where the 4 pieces of sub-media data are 1.ts, 2.ts, 3.ts, and 4.ts, respectively, then each key frame included in the 4 pieces of sub-media data obtained in this step can be referred to as shown in fig. 2, and the 4 pieces of sub-media data include 2 key frames (bold line segments in fig. 2).

Step S112, obtaining the timestamp of each key frame included in the sub media data from the attribute information corresponding to each key frame included in the sub media data.

Referring to the definition of the attribute information in step S10, after determining the key frames included in the sub-media data, the timestamps of the key frames included in the sub-media data may be determined according to the attribute information corresponding to the key frames.

Step S113, obtaining the position information corresponding to each key frame included in the sub media data according to the position information of the data frame and the timestamp of the data frame in the attribute information corresponding to each data frame included in the sub media data.

Optionally, if the target media data is video data, the step may determine, after determining the key frames included in the sub media data, position information of the key frames included in the sub media data according to the attribute information corresponding to the key frames.

Optionally, if the target media data is audio/video data, each sub-media data includes a video frame and an audio frame. For convenience of description, the following gives a process of acquiring the position information corresponding to a key frame by taking a key frame included in sub-media data including a video frame and an audio frame as an example, and the process of acquiring the position information of each key frame included in the sub-media data is the same as the process of acquiring the position information of the key frame. Optionally, the process of acquiring the position information corresponding to the key frame may include:

step S1131 determines, from the attribute information corresponding to each audio frame included in the sub-media data in which the key frame is located, a timestamp that is located before the timestamp of the key frame and is closest to the timestamp of the key frame, as a target timestamp.

As described above, if the target media data is audio-video data, the position information corresponding to the key frame may be position information of an audio frame whose timestamp is located near the timestamp of the key frame.

Optionally, in this embodiment of the present application, the position information corresponding to the key frame may be position information of an audio frame whose timestamp is located before the timestamp of the key frame, so that it can be ensured that the audio frame and the key frame are kept synchronized, and thus no matter where a fast forward (or fast backward) point is located in the sub media data, after the fast forward (or fast backward) operation, the corresponding audio can be heard while the video picture starts to be played.

In this case, the step may determine, as the target timestamp, a timestamp that is located before the timestamp of the key frame and is closest to the timestamp of the key frame from the attribute information corresponding to each audio frame included in the sub-media data in which the key frame is located, and may determine the position information corresponding to the key frame based on the target timestamp.

In order to improve the efficiency of searching for the target timestamp, optionally, in this step, a binary search method may be used to determine the target timestamp from the attribute information corresponding to each audio frame included in the sub-media data in which the key frame is located.

Step S1132, determining the position information in the attribute information where the target timestamp is located as the position information corresponding to the key frame.

In this step, the position information in the attribute information where the target timestamp is located is the position information of the audio frame whose timestamp is located near the timestamp of the key frame.

It should be noted that, after the key information corresponding to each key frame included in the sub-media data is obtained in this step, the key information may be sorted and stored from small to large according to the position information corresponding to the key frame. Optionally, if one piece of sub-media data includes multiple key frames, the key information corresponding to the multiple key frames may be stored in a SLICE structure.

Step S12, key information corresponding to each key frame contained in the sub media data and index information of the sub media data are distributed to the CDN, so that when the client detects a fast-forward operation or a fast-rewind operation of a user during playing of the target media data, a timestamp matching an operation time point is determined from the key information corresponding to each key frame contained in the sub media data, data acquisition is started from a position indicated by position information in the key information where the determined timestamp is located, and the acquired data is decoded and played.

Specifically, in this step, the index information of the sub-media data may be distributed to a CDN (Content Delivery Network), the key information corresponding to each key frame included in the sub-media data is organized according to a set format, the key information in the set format is obtained, and then the key information in the set format is distributed to the CDN. For example, the time stamp of the audio frame with the time stamp near the time stamp of the key frame, and the position information corresponding to the key frame in the key information corresponding to a key frame are separated by commas, and the key information corresponding to each key frame in each piece of sub-media data is separated by semicolons, so as to obtain the key information with a set format and distribute the key information to the CDN.

Optionally, in this step, when organizing the key information corresponding to each key frame included in the sub-media data according to the set format, the number of the sub-media data, the version number of the target media data, the pixel value corresponding to the width and the height of the target media data, the duration of each sub-media data, and other information may be organized according to the set format at the same time, and based on this, a data structure storage example of the key information of the set format is as follows:

VERSION:2

WIDTH:1920

HEIGHT:1016

TOTAL-NUM:7

D＝5.080000,NUM＝1,INFO＝"129420,133200,564"

D＝10.000000,NUM＝2,INFO＝"1026060,1033200,564；1471500,1483200,1367324"

D＝10.000000,NUM＝2,INFO＝"1926540,1933200,564；2370060,2383200,1205644"

D＝10.000000,NUM＝2,INFO＝"2827020,2833200,564；3272460,3283200,1432748"

D＝10.000000,NUM＝2,INFO＝"3727500,3733200,564；4172940,4183200,804452"

D＝10.000000,NUM＝2,INFO＝"4626060,4633200,564；5073420,5083200,1528252"

D＝2.080000,NUM＝1,INFO＝"15426060,15433200,564"

ENDLIST

the VERSION field is the VERSION number of the target media data, so that the subsequent effective expansion of the data is facilitated and the historical data is compatible; the TOTAL-NUM field is the number of the sub-media data, the WIDTH field and the HEIGHT field are respectively the pixel values corresponding to the WIDTH and the HEIGHT of the target media data, and the client can determine whether the obtained key information is matched with the index information based on the TOTAL-NUM field, the WIDTH field and the HEIGHT field; the above D is the duration of the sub-media data, NUM is the number of the key frames included in the sub-media data, and INFO is the key information corresponding to each key frame included in the sub-media data.

The media playing method applied to the server provided by the application firstly obtains the attribute information respectively corresponding to each data frame contained in the sub-media data, then, according to the attribute information corresponding to each data frame contained in the sub-media data, obtaining the key information corresponding to each key frame contained in the sub-media data, then, the key information corresponding to each key frame contained in the sub-media data and the index information of the sub-media data are distributed to the CDN, so that when the client detects a fast forward operation or a fast backward operation by the user during the playing of the target media data, and determining a time stamp matched with the operation time point from key information respectively corresponding to each key frame contained in the sub-media data, acquiring data from a position indicated by position information in the key information where the determined time stamp is located, and decoding and playing the acquired data. According to the method and the system, the key information corresponding to each key frame contained in the sub-media data is distributed to the CDN, so that the client can start to acquire, decode and play data from the position indicated by the position information in the key information where the timestamp matched with the operation time point is located, and compared with the prior art that the client starts to acquire, decode and play data from the first frame of the sub-media data matched with the operation time point, the CDN bandwidth can be effectively saved, and the user playing experience is better. In addition, the online CDN framework does not need to be modified, the application layer is not perceived, and the original media playing system is not invasive.

Corresponding to the above media playing method applied to the server, the present application also provides a media playing method applied to the client, and the following embodiments will describe the media playing method applied to the client in detail.

Referring to fig. 3, a schematic flow chart of a media playing method applied to a client according to an embodiment of the present application is shown, where the method may include:

step S30, obtaining, from the CDN, index information of the sub-media data obtained by segmenting the target media data, and key information corresponding to each key frame included in the sub-media data.

The key information in this step is the same as the key information in the media playing method applied to the server, and details can be introduced with reference to the foregoing embodiment, and are not repeated here.

It should be noted that, in this step, after the index information and the key information are obtained, it may be determined whether the obtained key information matches the index information, if the key information does not match the index information, the data needs to be downloaded, decoded and played according to a media playing method in the prior art to avoid a playing error, and if the key information matches the index information, the data is downloaded (or obtained) according to the following steps, and the data is decoded and played.

And step S31, downloading data based on the index information of the sub-media data, and decoding and playing the downloaded data.

When the user needs to play the target media data, each sub-media data can be downloaded based on the index information, and each downloaded sub-media data is decoded and played.

Step S32 is performed to determine a time stamp matching the operation time point from the key information corresponding to each key frame included in the sub media data when the fast forward operation or the fast backward operation by the user is detected.

For convenience of description, the fast forward operation or fast backward operation of the user is hereinafter referred to as a seek operation, the client may detect the seek operation of the user, and when the seek operation is detected, a timestamp matching a time point of the seek operation may be determined from key information corresponding to each key frame included in the sub-media data.

It can be understood that, if the client wants to play the data matched with the seek operation time point, it is necessary to acquire the key frame before the seek operation time point, so as to decode and play the data based on the key frame before the seek operation time point. For example, referring to the seek operation diagram shown in fig. 2, the timestamp obtained in this step and matching the seek operation time point refers to the timestamp of the second key frame included in the sub-media file 2. ts.

And step 33, starting to acquire data from the position indicated by the position information in the key information where the determined time stamp is located, and decoding and playing the acquired data.

As described above, the key information corresponding to a key frame includes the timestamp of the key frame and the location information corresponding to the key frame, and after the timestamp matching the seek operation time point is determined in the previous step, this step may start to acquire data from the location indicated by the location information in the key information where the determined timestamp is located.

Optionally, if the data to be acquired is not cached currently, the step may start downloading the data from the position indicated by the position information in the key information where the timestamp is determined, and preferably, in order to improve the downloading efficiency of the data, the step may start downloading the data from the position indicated by the position information in the key information where the timestamp is determined in a multithread downloading manner; if the data needing to be obtained is cached currently, the step does not need to download the data again, and the cached data is obtained directly from the position indicated by the position information in the determined key information where the timestamp is located.

Optionally, in order to enable the played data to be more accurately matched with the seek operation of the user, so as to improve the user playing experience, the step may also start to decode the acquired data at the background but not display the data, and display the data to the user after decoding to the seek point.

For example, referring to the seek operation diagram shown in fig. 2, in this step, data may be obtained from the position information corresponding to the second key frame included in the sub-media file 2.ts, and played after decoding to the seek point, and compared with the prior art, the actual CDN bandwidth flow that can be saved in the embodiment of the present application is the black area shown in fig. 2.

According to the embodiment of the application, index information of sub-media data obtained by segmenting target media data and key information corresponding to each key frame contained in the sub-media data are obtained from a CDN (content distribution network), then data are downloaded based on the index information of the sub-media data, the downloaded data are decoded and played, when fast-forward operation or fast-backward operation of a user is detected, a timestamp matched with an operation time point is determined from the key information corresponding to each key frame contained in the sub-media data, and finally, the data are obtained from a position indicated by position information in the determined key information where the timestamp is located, and the obtained data are decoded and played. This application carries out accurate matching through the time stamp with operating time point and key frame, obtain the time stamp with operating time point matching, position department that indicates through the positional information in the key information from this time stamp place begins to acquire data, can realize the accurate positioning to seek operation, thereby furthest reduces because of seek operation to the waste of CDN bandwidth, service provider's cost has been reduced, and this application is after obtaining data, adopt the mode of decoding to resending after the seek point and showing, realize the accurate broadcast of seek operation, user's broadcast experience is better. In addition, the online CDN framework does not need to be modified, the application layer is not perceived, and the original media playing system is not invasive.

The following describes the media playing device applied to the server and the media playing device applied to the client, and the media playing device applied to the server and the media playing device applied to the client described below may respectively correspond to the media playing method applied to the server and the media playing method applied to the client described above.

Referring to fig. 4, a schematic structural diagram of a media playing device applied to a server according to an embodiment of the present application is shown, and as shown in fig. 4, the media playing device applied to the server may include: an attribute information acquisition module 41, a key information acquisition module 42, and an information distribution module 43.

The attribute information acquiring module 41 is configured to acquire attribute information corresponding to each data frame included in the sub-media data, where the sub-media data is obtained by segmenting the target media data, and the attribute information corresponding to a data frame includes a key frame identifier used to indicate whether the data frame is a key frame, location information of the data frame, and a timestamp of the data frame.

The key information obtaining module 42 is configured to obtain, according to attribute information corresponding to each data frame included in the sub-media data, key information corresponding to each key frame included in the sub-media data, where the key information corresponding to a key frame includes a timestamp of the key frame and position information corresponding to the key frame, and the position information corresponding to the key frame is position information of the key frame or position information of a data frame whose timestamp is located near the timestamp of the key frame.

An information distribution module 43, configured to distribute key information corresponding to each key frame included in the sub-media data and index information of the sub-media data to the CDN, so that when the client detects a fast-forward operation or a fast-rewind operation of a user in a process of playing target media data, a timestamp matching an operation time point is determined from key information corresponding to each key frame included in the sub-media data, data is obtained from a position indicated by position information in the key information where the determined timestamp is located, and the obtained data is decoded and played.

The media playing device applied to the client side can distribute the key information corresponding to each key frame contained in the sub-media data to the CDN, so that the client side can start to acquire, decode and play data from the position indicated by the position information in the key information where the timestamp matched with the operation time point is located. In addition, the media playing device applied to the client does not need to modify an online CDN framework, has no perception on an application layer and has no invasion on an original media playing system.

In a possible implementation manner, the key information obtaining module 42 may include: the device comprises a key frame determining module, a timestamp acquiring module and a position information acquiring module.

And the key frame determining module is used for determining each key frame contained in the sub-media data according to the key frame identifier which is used for indicating whether the data frame is the key frame in the attribute information respectively corresponding to each data frame contained in the sub-media data.

And the timestamp acquisition module is used for acquiring timestamps of all key frames contained in the sub-media data from the attribute information respectively corresponding to all key frames contained in the sub-media data.

In a possible implementation manner, the target media data is audio/video data.

If the sub-media data includes an audio frame and a video frame, the position information obtaining module may include, for each key frame included in the sub-media data: a target timestamp determination module and a location information determination module.

And the target timestamp determining module is used for determining a timestamp which is positioned before the timestamp of the key frame and is closest to the timestamp of the key frame from the attribute information corresponding to each audio frame contained in the sub-media data of the key frame as a target timestamp.

Referring to fig. 5, a schematic structural diagram of the media playing device applied to the client according to the embodiment of the present application is shown, and as shown in fig. 5, the media playing device applied to the client may include: an information acquisition module 51, a data download and play module 52, a timestamp determination module 53, and a data acquisition and play module 54.

The information obtaining module 51 is configured to obtain, from the CDN, index information of sub-media data obtained by segmenting target media data, and key information corresponding to each key frame included in the sub-media data, where the key information corresponding to a key frame includes a timestamp of the key frame and position information corresponding to the key frame, and the position information corresponding to the key frame is position information of the key frame or position information of a data frame whose timestamp is located near the timestamp of the key frame.

And a data downloading and playing module 52, configured to download data based on the index information of the sub-media data, and decode and play the downloaded data.

The timestamp determining module 53 is configured to determine, when a fast forward operation or a fast backward operation of a user is detected, a timestamp that matches an operation time point from key information corresponding to each key frame included in the sub-media data.

And the data acquiring and playing module 54 is configured to acquire data from a position indicated by the position information in the key information where the determined timestamp is located, and decode and play the acquired data.

The application provides a media play device who is applied to customer end, through carrying out the accurate matching with the timestamp of operating time point and key frame, obtain the timestamp that matches with the operating time point, begin to acquire data through the position department that indicates from the positional information in the key information of this timestamp place, can realize the accurate positioning to seek operation, thereby furthest reduces because of the waste of seek operation to CDN bandwidth, service provider's cost has been reduced, and this application is after acquiring data, adopt the mode that sends the demonstration after decoding seek point, realize the accurate broadcast of seek operation, user playing experience is better. In addition, the online CDN framework does not need to be modified, the application layer is not perceived, and the original media playing system is not invasive.

In a possible implementation manner, the timestamp determining module 53 is specifically configured to determine, from the key information corresponding to each key frame included in the sub-media data, a timestamp that is located before the operation time point and is closest to the operation time point, as a timestamp that matches the operation time point.

The embodiment of the application also provides a media playing system, which comprises a server and a client.

The server is used for acquiring attribute information corresponding to each data frame contained in the sub-media data, the sub-media data is obtained by segmenting target media data, and the attribute information corresponding to one data frame comprises a key frame identifier for indicating whether the data frame is a key frame, position information of the data frame and a timestamp of the data frame; determining key information corresponding to each key frame contained in the sub-media data according to attribute information corresponding to each data frame contained in the sub-media data, wherein the key information corresponding to a key frame comprises a time stamp of the key frame and position information corresponding to the key frame, and the position information corresponding to the key frame is the position information of the key frame or the position information of the data frame of which the time stamp is positioned near the time stamp of the key frame; and distributing key information corresponding to each key frame contained in the sub-media data and the index information of the sub-media data to the CDN.

The media playing system provided in the embodiment of the present application may refer to the media playing method applied to the server and the media playing method applied to the client, which are described above, and may refer to the description in the foregoing embodiments in detail, which is not repeated herein.

The embodiment of the application also provides media playing equipment applied to the server. Alternatively, fig. 6 is a block diagram illustrating a hardware structure of a media playing device applied to a server, and referring to fig. 6, the hardware structure of the media playing device applied to the server may include: at least one processor 601, at least one communication interface 602, at least one memory 603, and at least one communication bus 604;

in the embodiment of the present application, the number of the processor 601, the communication interface 602, the memory 603, and the communication bus 604 is at least one, and the processor 601, the communication interface 602, and the memory 603 complete communication with each other through the communication bus 604;

the processor 601 may be a central processing unit CPU, or an application Specific Integrated circuit asic, or one or more Integrated circuits configured to implement embodiments of the present invention, or the like;

the memory 603 may include a high-speed RAM memory, and may further include a non-volatile memory (non-volatile memory), etc., such as at least one disk memory;

wherein the memory 603 stores a program and the processor 601 may call the program stored in the memory 803 for:

Alternatively, the detailed function and the extended function of the program may be as described above.

The embodiment of the application also provides the media playing equipment applied to the client. Alternatively, fig. 7 is a block diagram illustrating a hardware structure of a media playing device applied to a client, and referring to fig. 7, the hardware structure of the media playing device applied to the client may include: at least one processor 701, at least one communication interface 702, at least one memory 703 and at least one communication bus 704;

in the embodiment of the present application, the number of the processor 701, the communication interface 702, the memory 703 and the communication bus 704 is at least one, and the processor 701, the communication interface 702 and the memory 703 complete mutual communication through the communication bus 704;

the processor 701 may be a central processing unit CPU, or an application Specific Integrated circuit (asic), or one or more Integrated circuits configured to implement embodiments of the present invention, or the like;

the memory 703 may include a high-speed RAM memory, and may further include a non-volatile memory (non-volatile memory) or the like, such as at least one disk memory;

wherein the memory 703 stores a program, and the processor 701 may call the program stored in the memory 703, the program being configured to:

The embodiment of the present application also provides a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the media playing method applied to a server is implemented as described above.

The embodiment of the present application further provides a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the media playing method applied to the client is implemented as described above.

Finally, it is further noted that, herein, relational terms such as, for example, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A media playing method is applied to a server and comprises the following steps:

2. The method of claim 1, wherein the obtaining key information corresponding to each key frame included in the sub-media data according to the attribute information corresponding to each data frame included in the sub-media data comprises:

determining each key frame contained in the sub-media data according to a key frame identifier used for indicating whether the data frame is a key frame in the attribute information respectively corresponding to each data frame contained in the sub-media data;

acquiring the time stamp of each key frame contained in the sub-media data from the attribute information respectively corresponding to each key frame contained in the sub-media data;

3. The media playing method according to claim 2, wherein the target media data is audio/video data;

the acquiring, according to the position information of each data frame and the timestamp of each data frame in the attribute information corresponding to each data frame included in the sub-media data, the position information corresponding to each key frame included in the sub-media data includes:

4. A media playing method is applied to a client and comprises the following steps:

5. The media playing method according to claim 4, wherein the determining a timestamp matching an operation time point from key information corresponding to each key frame included in the sub-media data comprises:

and determining a timestamp which is positioned before the operation time point and is closest to the operation time point from key information respectively corresponding to each key frame contained in the sub-media data, and taking the timestamp as a timestamp matched with the operation time point.

6. A media playing device, applied to a server, includes: the system comprises an attribute information acquisition module, a key information acquisition module and an information distribution module;

the key information acquisition module is used for acquiring key information corresponding to each key frame contained in the sub-media data according to attribute information corresponding to each data frame contained in the sub-media data, wherein the key information corresponding to a key frame comprises a time stamp of the key frame and position information corresponding to the key frame, and the position information corresponding to the key frame is the position information of the key frame or the position information of the data frame of which the time stamp is positioned near the time stamp of the key frame;

the information distribution module is configured to distribute key information corresponding to each key frame included in the sub-media data and index information of the sub-media data to the CDN, so that when the client detects a fast-forward operation or a fast-rewind operation of a user during playing of the target media data, a timestamp matching an operation time point is determined from key information corresponding to each key frame included in the sub-media data, data is obtained from a position indicated by position information in the key information where the determined timestamp is located, and the obtained data is decoded and played.

7. The media playing device of claim 6, wherein the key information obtaining module comprises: the system comprises a key frame determining module, a timestamp acquiring module and a position information acquiring module;

the key frame determining module is configured to determine each key frame included in the sub-media data according to a key frame identifier, which is used to indicate whether a data frame is a key frame, in attribute information respectively corresponding to each data frame included in the sub-media data;

the timestamp acquiring module is configured to acquire timestamps of the key frames included in the sub-media data from attribute information respectively corresponding to the key frames included in the sub-media data;

the position information acquiring module is configured to acquire, according to the position information of the data frame and the timestamp of the data frame in the attribute information corresponding to each data frame included in the sub-media data, position information corresponding to each key frame included in the sub-media data.

8. The media playing device of claim 7, wherein the target media data is audio-video data;

the target timestamp determining module is used for determining a timestamp which is positioned before the timestamp of the key frame and is closest to the timestamp of the key frame from the attribute information respectively corresponding to each audio frame contained in the sub-media data of the key frame as a target timestamp;

and the position information determining module is used for determining the position information in the attribute information where the target timestamp is located as the position information corresponding to the key frame.

9. A media playing device, applied to a client, includes: the system comprises an information acquisition module, a data downloading and playing module, a timestamp determination module and a data acquisition and playing module;

the information acquisition module is used for acquiring index information of sub-media data obtained by segmenting target media data from the CDN and key information corresponding to each key frame contained in the sub-media data, the key information corresponding to a key frame comprises a time stamp of the key frame and position information corresponding to the key frame, and the position information corresponding to the key frame is the position information of the key frame or the position information of a data frame of which the time stamp is positioned near the time stamp of the key frame;

the timestamp determining module is configured to determine, when a fast forward operation or a fast reverse operation of a user is detected, a timestamp matching an operation time point from key information corresponding to each key frame included in the sub-media data;

10. A media playback system, comprising: a server and a client;

the client is configured to obtain, from the CDN, index information of the sub-media data and key information corresponding to each key frame included in the sub-media data; downloading data based on the index information of the sub-media data, and decoding and playing the downloaded data; when the fast forward operation or the fast backward operation of a user is detected, determining a timestamp matched with an operation time point from key information respectively corresponding to each key frame contained in the sub-media data; and starting to acquire data from the position indicated by the position information in the key information where the determined time stamp is positioned, and decoding and playing the acquired data.