CN112822518A

CN112822518A - Video playing method, device, system, electronic equipment and storage medium

Info

Publication number: CN112822518A
Application number: CN202110415627.3A
Authority: CN
Inventors: 孟赟; 杨涛; 俞鸣园; 曹亚曦; 王克彦
Original assignee: Zhejiang Huachuang Video Signal Technology Co Ltd
Current assignee: Zhejiang Huachuang Video Signal Technology Co Ltd
Priority date: 2021-04-19
Filing date: 2021-04-19
Publication date: 2021-05-18

Abstract

The invention provides a video playing method, a device, a system, electronic equipment and a storage medium, wherein the method is applied to a browser and comprises the following steps: acquiring a video address and a video key from a media server according to the video playing request; acquiring a data stream of a video to be played through the video address; decrypting the data stream frame by using the video key to obtain data to be played; and playing the data to be played in the playable format processed by the webpage of the browser. The video data in the embodiment of the application adopts a streaming frame-by-frame encryption and decryption mode, and compared with a fragmentation encryption mode, the method can effectively reduce file fragments, reduce playing time delay and improve playing effect.

Description

Video playing method, device, system, electronic equipment and storage medium

Technical Field

The present invention relates to the field of information technologies, and in particular, to a video playing method, apparatus, system, electronic device, and storage medium.

Background

In the related art, the Web (Web page) video playing technology mainly plays video based on browser plug-ins and plays video based on HTML5 (hypertext 5.0).

The browser plug-in has compatibility problems of different operating systems, and meanwhile, the browser plug-in has the authority of operating local data of the equipment, which brings security problems to user equipment. The Web playing video based on the modes such as no plug-in and the like has higher requirements on the security of video transmission, and simultaneously requires the data transmission rate, and particularly has higher playing requirements with low delay for the modes such as video live broadcast and the like.

Disclosure of Invention

The invention provides a video playing method, a video playing device, a video playing system, electronic equipment and a storage medium, which are used for at least solving the technical problems in the prior art.

The first aspect of the present invention provides a video playing method, where the method is applied to a browser, and includes:

acquiring a video address and a video key from a media server according to the video playing request;

acquiring a data stream of a video to be played through the video address;

decrypting the data stream frame by using the video key to obtain data to be played;

and playing the data to be played in the playable format processed by the webpage of the browser.

In some embodiments, the method further comprises:

and processing the decrypted data to be played into the data to be played in the playable format which can be directly identified by the page element of the browser.

In some embodiments, the processing the decrypted data to be played to the data to be played in the playable format that can be directly identified by the page element of the browser includes:

decapsulating the decrypted data to be played to obtain original audio data and original video data;

decoding the original audio data and the original video data through Webelementary (web-oriented binary coding format) to obtain data to be played in the playable format; or the like, or, alternatively,

and encapsulating the original audio data and the original video data into data to be played in a data format supported by MSE (Media Source Extensions) of the browser.

In some embodiments, the video to be played is a recorded video file; the playing of the web page in the browser is processed into data to be played in a playable format, and the data includes:

storing the data to be played processed into a playable format in a video buffer area;

and responding to the data to be played in the playable format in the video buffer area, and playing the data to be played in the playable format in the video buffer area.

In some embodiments, the method further comprises:

acquiring a video index file from the media server, wherein the video index file is used for determining a plurality of key frames in the video file; the key frame is used for determining the playing position of the video to be played.

In some embodiments, the obtaining, by the video address, a data stream of a video to be played includes:

in response to detecting that a preset instruction for controlling video playing or video time corresponding to the data to be played in the playable format in the video buffer area is smaller than the preset time, determining a key frame corresponding to the position to be played of the video according to the index file, wherein the position of the key frame is earlier than or equal to the position to be played;

and if the data stream of the video to be played is not downloaded, downloading the data stream of the preset duration with the position of the key frame as the initial position through the video address.

In some embodiments, the video to be played is a live video; the playing of the web page in the browser is processed into data to be played in a playable format, and the data includes:

storing the data to be played processed into a playable format in a video buffer area in real time; playing the data to be played in the playable format in the video buffer area in real time on a webpage of the browser;

or

And directly playing the data to be played in the playable format decoded by Webestablishment on the webpage of the browser.

In some embodiments, the method further comprises:

acquiring user information of a browser;

and responding to the browser to enter a video playing page, and initiating a video playing request to the media server according to the user information.

A second aspect of the present invention provides a video playing method, which is applied to a media server, and includes:

sending a video address and a video secret key according to the received video playing request;

and transmitting the data stream of the video to be played, which is encrypted frame by frame based on the video address.

In some embodiments, the video to be played is a recorded video file; the method further comprises the following steps:

transmitting a video index file for determining a plurality of key frames in the video file; the key frame is used for determining the playing position of the video to be played.

In some embodiments, the method further comprises:

encrypting a video to be played frame by frame;

storing the video to be played and the video index file which are encrypted frame by frame to a storage server;

the video address is a corresponding address in the storage server; the transmitting the data stream of the video to be played encrypted frame by frame based on the video address comprises the following steps:

and transmitting the data stream of the video to be played, which is encrypted frame by frame, to a browser through the storage server.

In some embodiments, the video to be played is a live video; the data stream for transmitting the frame-by-frame encrypted video to be played comprises:

encrypting the data stream of the live video frame by frame in real time;

and transmitting the encrypted data stream in real time.

A third aspect of the present invention provides a video playback apparatus, which is applied to a browser, including:

the first acquisition module is used for acquiring a video address and a video key from the media server according to the video playing request;

the second acquisition module is used for acquiring the data stream of the video to be played through the video address;

the decryption module is used for decrypting the data stream frame by using the video key to obtain data to be played;

and the playing module is used for playing the data to be played which is processed into a playable format on the webpage of the browser.

A fourth aspect of the present invention provides a video playback apparatus, which is applied to a media server, including:

the second sending module is used for sending the video address and the video secret key according to the received video playing request;

and the transmission module is used for transmitting the data stream of the video to be played, which is encrypted frame by frame based on the video address.

A fifth aspect of the present invention provides a video playback system, the system comprising:

the media server is used for encrypting the video to be played frame by frame;

and the browser is used for acquiring the video to be played from the media server and playing the video after decrypting the video frame by frame.

In some embodiments, the browser comprises:

the user interface layer is used for receiving a play control instruction of a user and playing the video to be played;

the logic control layer is used for acquiring the video to be played;

and the compiling processing layer is used for decrypting the video to be played and processing the video to be played into a format supported by the browser.

In some embodiments, the compilation processing layer comprises:

the decryption module is used for decrypting the video to be played frame by frame;

the decapsulation module is used for decapsulating the video to be played to obtain audio data and video data;

a decoding module or an encapsulation module; the decoding module is used for decoding the audio data and the video data after being unpacked into data to be played, which can be played by the user interface layer; and the encapsulation module is used for encapsulating the audio data and the video data after decapsulation into to-be-played data supported by MSE of the browser.

In some embodiments, the system further comprises:

and the storage server is used for storing the recorded video file and the index file corresponding to the video file, wherein the video file comprises the video to be played after the video is encrypted frame by the media server.

A sixth aspect of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus; a memory for storing a computer program; and a processor for implementing the steps of any of the above methods when executing the program stored in the memory.

A seventh aspect of the invention provides a computer-readable storage medium having stored therein a computer program which, when executed by a processor, implements the steps of any of the methods described above.

According to the technical scheme of the embodiment of the disclosure, the video data played online by the browser adopts a streaming frame-by-frame encryption and decryption mode, and compared with a fragment encryption mode, the method can effectively reduce file fragments, reduce playing time delay and improve playing effect.

Drawings

Fig. 1 shows a first flow chart of a video playing method;

FIG. 2 is a flowchart illustrating a video playing method according to a second embodiment;

FIG. 3 is a block diagram illustrating a video playback system;

FIG. 4 is a block diagram of a video playback system;

FIG. 5 is a block diagram of a video playback system;

fig. 6 shows a block diagram of a video playback system;

FIG. 7 is a flow chart illustrating a video playing method;

FIG. 8 is a fourth flowchart illustrating a video playing method;

FIG. 9 is a flowchart illustrating a fifth exemplary video playing method;

fig. 10 shows a sixth flowchart of a video playing method;

FIG. 11 is a block diagram of a video player;

fig. 12 shows a block diagram of a video playback device.

Detailed Description

In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

As shown in fig. 1, an embodiment of the present application provides a video playing method, where the method is applied to a browser, and includes:

step S101, acquiring a video address and a video key from a media server according to a video playing request;

step S102, acquiring a data stream of a video to be played through the video address;

step S103, decrypting the data stream frame by using the video key to obtain data to be played;

and step S104, playing the data to be played which is processed into a playable format on the webpage of the browser.

In an embodiment of the present application, a browser is used to browse a web page, i.e. a web browser. The browser can open a video playing page through a webpage link which is indicated by a user to enter, and the video playing page sends a video playing request to the media server; or sending a video playing request to the media server by receiving a video playing instruction of the user on the page.

Here, the video to be played may be an existing video that has been recorded, for example, a movie and television work, an advertisement video, a page start animation, a short video, and the like; or may be live video.

In the embodiment of the application, in order to improve the security performance of video data transmission, the video to be played is encrypted before transmission, and is decrypted by the browser according to the obtained video key after being transmitted to the browser.

In the embodiment of the present disclosure, the decrypted data to be played may also need to be unpacked, decoded, or repackaged, and the data to be played is processed into a playable format and then played by the browser webpage.

Here, the encryption mode of the video to be played is streaming frame-by-frame encryption, so the decryption process also needs to perform frame-by-frame decryption. Therefore, the browser can play the acquired part without acquiring all videos, and compared with a fragment encryption mode, the method can effectively reduce file fragments and reduce the playing delay of the videos, and is particularly suitable for scenes played in real time such as live broadcast.

In the embodiment of the present application, the data stream of the video to be played may be a national cryptographic algorithm based on the cryptographic industry standard of the national cryptographic authority, such as "sequence cryptographic algorithm of ancestor Chong" including SM4 block cryptographic algorithm. The video data are encrypted and decrypted by using the national encryption algorithm, so that domestic autonomous safety data transmission is realized, and the development prospect of a subsequent browser and an encryption standard is facilitated.

In some embodiments, the method further comprises:

The browser receives the data stream of the video to be played, and the data to be played is obtained after decryption, and the format of the data to be played at the moment may not conform to the format supported by the browser, so that the data cannot be played.

In an embodiment, a video playing plug-in may be used to identify and play the data to be played, however, not all browsers have the same plug-in, and the increasing video browsing demand is difficult to meet due to the problems of untimely update of the plug-in, etc.

Therefore, in the embodiment of the present application, the page element is used to identify and play the data to be played, which is processed into a playable format. Here, the page element may include a video element (video element) that supports video playback in the formats of Ogg, MPEG4, and WebM, etc. In addition, based on the MSE (mean Square error) extended webpage interface (Web API), more formatted media streams can be allowed to be directly played on a webpage. In another embodiment, the page element may also include a canvas element (canvas element) that enables the playing of the media file of the video by controlling the drawing of each pixel.

Here, processing the decrypted data to be played into a data format that can be identified by the page element may include performing operations such as decapsulation and decoding on the decrypted data to be played to obtain and play the original audio/video data that can be identified by the video element; or the decrypted data to be played is unpackaged and repackaged into a format which can be recognized by a browser, and the like. Therefore, the video data can be directly played through the browser without a video playing plug-in, so that the application range is wider.

decoding the original audio data and the original video data through Webelementary to obtain the data to be played in the playable format; or the like, or, alternatively,

And after the decrypted data to be played is unpacked, original audio data and original video data are obtained. And then the format which can be played by the browser can be obtained after processing. The processing manner of the original audio data and the original video data may include a manner of decoding through Webassembly. WebAssembly is a language which can be run in a browser naturally, and as a binary format, other static language codes such as C/C + +/Rust and the like can be conveniently and quickly run in the browser.

In this embodiment of the application, the data to be played in the playable format may include video data and audio data, where the audio data may be pcm-format data decoded by Webassembly and played by an audio player (e.g., AudioContext); the video data may be yuv format data decoded by Webassembly, and then rendered by a drawing protocol (e.g., WebGL) of a web page, thereby playing a video.

The data to be played in the playable format may also be in a data format supported by MSE, for example, fMP4 format. Here, the original audio data and the original video data after being decapsulated may be encapsulated to obtain data in fMP4 format, and then the data in fMP4 format may be decoded and played by MSE of video tag (video tag) of the browser.

storing the data to be played in the playable format in a video buffer area;

and responding to the data to be played in the playable format in the video buffer area, and playing the data to be played in the playable format. The video to be played in the embodiment of the present application may be a recorded video file, such as a movie, a television work, an advertisement video, a start picture of some pages, a short video, and the like.

In order to smoothly play the video, the data stream of the video to be played may be downloaded in the playing process, and the data to be played in the playable format supported by the browser may be obtained after decryption, decapsulation, and the like, and then may be cached in the video buffer. When the video buffer stores the data to be played in a playable format, the browser can play the data.

Here, the downloading and buffering may be performed in time length segments, for example, 10 or 20 seconds of the video to be played is downloaded, the data to be played processed into the playable format is buffered, and the data is played in the browser, and when the remaining time for playing is short, the video to be played in the next segment is continuously downloaded and the data to be played processed into the playable format is buffered.

If the time length of the data to be played in the playable format which is cached in the video buffer area and is not played is longer than the preset time length, the downloading and caching can be suspended, and the data to be played in the playable format in the video buffer area can be played. Therefore, on one hand, the delay of playing can be reduced, on the other hand, the communication burden of the browser can be reduced, the duration of occupied bandwidth is reduced, and the communication efficiency is improved.

In some embodiments, the method further comprises:

Considering that the recorded video has a complete file, the transmission and playing processes may not be performed according to the time sequence.

Here, the playing position in the video is identified by the key frame in the video.

In the embodiment of the present application, the key frame is an intra picture frame (I frame) in a video file, and the I frame is a first frame forming each GOP (Group of Pictures) in the process of compressing a video. I frames have all parameter information of pictures and thus can be decoded independently, while other frames attached to I frames, such as P frames (predictive coded frames) and B frames (bidirectionally predictive coded frames), can be decoded by combining the parameter information of the P or B frame itself with the parameter information of the I frame to which it is attached.

Therefore, the positions of the key frames in the video index file can be determined through the video index file, and the playing position or the downloading position of the video can be determined according to the key frames. That is to say, through the video index file, the skipping of video downloading and playing can be realized, and the data stream of the video to be played is continuously downloaded according to the position indicated by the user and is further processed and played.

in response to detecting that a preset instruction for controlling video playing or video time corresponding to data to be played in a playable format in the video buffer area is smaller than the preset time, determining a key frame corresponding to a position to be played of a video according to the index file, wherein the position of the key frame is earlier than or equal to the position to be played;

Here, the predetermined instruction may be an instruction to control a video playing position, for example, a fast forward instruction, a fast backward instruction, an instruction to specify a play start time, and a replay instruction, and the like. The position of the video playing time corresponding to the instructions is the position to be played, and the position can be any position in the video. Therefore, after the predetermined instruction is detected, the key frame corresponding to the position to be played may be determined according to the index file, where the position of the key frame may be the position to be played or may be earlier than the position to be played, but if the position is earlier than the position to be played, the frame at the position to be played may be a frame that belongs to the key frame and can be decoded based on the parameter information of the key frame.

In addition, in the process of normal playing of the video, that is, playing according to the time sequence, if the video duration corresponding to the data to be played in the playable format in the video buffer area is less than the predetermined duration, the subsequent video needs to be continuously downloaded and cached, that is, the video is continuously downloaded according to the key frame of the next video segment until the downloading of all the videos is completed.

In this way, the position to be played in the video can be determined through the video index file, and operations such as downloading, decrypting, caching, decoding and playing of the data stream are performed based on the position to be played. Therefore, the video index file can be used for realizing the accurate positioning of the video, so that the skip playing can be carried out at any position of the whole video file.

storing the data to be played processed into a playable format in a video buffer area in real time; playing data to be played in a playable format in the video buffer area in real time on a webpage of the browser;

or

In the embodiment of the application, the video to be played is a live video, and the recorded file does not exist, but the video to be played is directly transmitted to the browser and played in the recording process of the recording end. And thus cannot perform jumping, pausing, etc.

After the browser acquires the data stream of the video to be played, the data stream is decrypted frame by frame to obtain the data to be played. After the data to be played is processed, the data to be played in the playable format supported by the browser is obtained, and then the processed data to be played in the playable format can be stored in the video buffer area in real time and played synchronously. Illustratively, the original audio data and the original video data may be encapsulated into a format supported by the MSE of the browser, for example, fMP4 format, and then the encapsulated data is continuously added to the video buffer of the MSE, at which time the MSE of the browser video tag decodes and plays the data of the buffer. The method has higher playing performance and obtains better playing effect, for example, the method is more suitable for videos with information transmission properties such as art programs, real-time news and the like.

In another embodiment, the decrypted data to be played may be unpackaged into original audio data and original video data. The original audio data and the original video data can be directly decoded and played through Webassembly. The method can bring lower delay and is more suitable for live programs with higher requirements on timeliness, such as live sales, live lottery and the like.

Illustratively, data to be played, which is obtained by decrypting an acquired data stream, may be decapsulated to obtain original audio data and original video data, and the audio data is decoded into pcm data by webelementary and played by an audio playing function; and decoding the video data into yuv data, and rendering and playing the yuv data through a drawing protocol of a webpage of the browser and the like.

In some embodiments, the method further comprises:

acquiring user information of a browser;

In the embodiment of the application, before the browser plays the video, the browser may first acquire user information of the browser. The user information may include information such as a name, a password, a rating, and a right of the user, and thus, when the browser enters the video playing page, a video playing request may be initiated to the media server according to the user information. If the media server determines that the user is a valid user through the user information, for example, the user purchases a playing permission of a currently requested video, or the user meets a playing condition, and the like, the browser may acquire a data stream of the video to be played, which is transmitted by the media server; if the user information is illegal, the media server may return prompt information, and at this time, the browser cannot obtain the data stream of the video, so that the video cannot be played.

Therefore, the browser sends a video playing request to the media server through the user information, so that the data security can be effectively improved, and the rights and interests of a video provider are maintained.

As shown in fig. 2, an embodiment of the present application provides a video playing method, where the method is applied to a media server, and includes:

step S201, sending a video address and a video key according to a received video playing request;

and step S202, transmitting the data stream of the video to be played, which is encrypted frame by frame based on the video address.

In this embodiment of the application, in order to improve the security performance of video data transmission, the media server may provide the video address and the video key to a legitimate user, transmit the encrypted data stream through a request of the browser, and decrypt and play the encrypted data stream according to the obtained video key after transmitting the encrypted data stream to the browser.

Here, the video may be encrypted frame by frame in a data stream manner, and the data stream of the video to be played, which is encrypted frame by frame, is transmitted to the browser based on the video address.

The video address may be an address for storing the video in the media server, a corresponding address in a storage server for storing video data other than the media server, or a live address.

The method may further comprise: and generating a video index file based on the video file. The media server can perform compression, encoding, packaging and other processing during the processing of the video file, and therefore, the media server can generate a corresponding video index file based on the processing of the video file. In addition, the media server may also directly encrypt the acquired processed video file and transmit the encrypted video file to the browser, so that the media server may also analyze and process the video file and generate a corresponding video index file according to information such as key frames in the file.

In some embodiments, the method further comprises:

encrypting a video to be played frame by frame;

In the embodiment of the application, the media server can be used for encrypting and the like of the video file and then storing the video file to the storage server, so that processing and storage of a super-large file are facilitated, file fragments are reduced, and the processing speed is increased. When the browser requests playing, the media server can provide the corresponding address in the storage server to the browser, so that the browser can conveniently download the data of the video file.

encrypting the data stream of the live video frame by frame in real time;

and transmitting the encrypted data stream in real time.

For live video, a video recording end transmits a recorded video data stream to a media server in real time, and the media server encrypts the video data stream frame by frame in real time. And synchronously transmitting the encrypted data stream to the browser end requesting to play. The browser can acquire subsequent video content in real time while playing and continue to decrypt and play. Therefore, the playing speed of the data can be effectively improved, the delay is reduced, and the data security is ensured.

As shown in fig. 3, an embodiment of the present application provides a video playing system, where the system includes:

the media server 31 is used for encrypting the video to be played frame by frame;

and the browser 32 is configured to acquire the video to be played from the media server 31, and play the video after decrypting the video frame by frame.

The media server 31 mainly comprises a national secret encryption module which encrypts audio and video media data through a national secret SM4 algorithm; and the video recording module is responsible for sending the encrypted audio and video media data to a storage server or directly to a browser 32 of a video data request end.

The browser 32 is configured to obtain a corresponding video to be played, decrypt audio and video media data through the SM4 algorithm, and play the data.

In some embodiments, the browser 32 includes:

the logic control layer is used for acquiring the video to be played;

and the compiling processing layer is used for decrypting the video to be played and processing the video to be played into a format supported by the browser 32.

The user interface layer, namely the UI layer, can comprise a display module used for rendering videos and displaying interfaces related to video playing control; and the playing control module is used for playing the video, stopping the video, adjusting the volume, adjusting the video progress and the like.

The logic control layer can adopt a Js layer realized by Javascript, is mainly used for logic operation, and can comprise a downloading module, namely a module for obtaining a key and an index from a server and downloading a video according to the index fragment; a decoding playing module or playing rendering module, the decoding playing module is used for decoding and playing the media data in fMP4 format supported by the browser 32 in MSE; the play rendering module is used for playing audio supported by the browser 32, such as pcm format, and rendering video supported by the browser 32, such as yuv format.

The compiling processing layer can be realized by a Webassombly technology and an Emscript technology, and can also be called a Wasm layer. Emscript is a complete open source compiler tool chain of WebAssembly, and C/C + +/Rust or any other language using LLVM (Low Level Virtual Machine) can be compiled into wasm using LLVM, with special attention to speed, size and Web platform. The wasm is a binary instruction format of the stack-based virtual machine, the performance of the wasm in the program execution process is superior due to the fact that the wasm adopts binary coding, and compared with a text format, the binary coding format occupies a smaller storage space.

In some embodiments, the compilation processing layer comprises:

a decoding module or an encapsulation module; the decoding module is used for decoding the audio data and the video data after being unpacked into data to be played, which can be played by the user interface layer; and the encapsulating module is used for encapsulating the audio data and the video data after decapsulation into to-be-played data supported by the MSE of the browser 32.

The compiling processing layer mainly comprises a national password decrypting module, and the decrypting module is used for decrypting the downloaded media data segment through a national password SM4 algorithm to obtain data before encryption; and a decapsulating module, namely, for decapsulating the decrypted media data into audio data and video data, such as audio data in aac format and video data in h.264 format. And an encapsulation module, namely, a module for re-encapsulating the de-encapsulated audio data and video data into fMP4 format supported by the MSE of the browser 32.

In some embodiments, the system further comprises:

The media server 31 stores the encrypted video file in the storage server, and when the browser requests to play the video, the media server downloads the data stream based on the corresponding video address in the storage server, decrypts the data stream frame by frame, and plays the corresponding video data.

The embodiments of the present application also provide the following examples:

in some embodiments, Web video playback technology plays video primarily based on browser plug-ins and based on HTML 5.

The browser plug-in has compatibility problems of different operating systems, and meanwhile, the browser plug-in has the authority of operating local data of the equipment, which brings security problems to user equipment. With this current situation, more and more browsers have either not supported or strictly restricted plug-ins. Meanwhile, the Adobe Flash plug-in also stops updating in 2020. This means that the need for plug-in-less Web-playback of video is becoming more and more stringent.

Video on the Web currently does not have a standard for displaying video only on Web pages, most of which are displayed via plug-ins (such as Flash). However, not all browsers have the same plug-in. HTML5 specifies a standard method of containing video through video elements. This element currently supports three video formats Ogg, MPEG4, WebM. With the development of video services such as video on demand and live broadcasting, video is distributed from a server to a client via a streaming media transport protocol (e.g., MPEG-DASH and HLS), and media content is further included in a layer of transport protocol, so that video elements cannot be identified. Media Source Extensions, were introduced to help the browser identify and process other video formats. Media Source Extensions (MSE) is a new Web API supported by mainstream browsers such as Chrome, Safari, Edge, etc. MSE is a world Wide Web Consortium (W3C) standard that allows JavaScript to dynamically construct < video > and < audio > media streams. It defines objects that allow JavaScript to transfer media stream fragments to an HTMLMediaElement. By using MSE, the media stream can be dynamically modified without any plug-in. This allows the front-end JavaScript to do more things, perform transcoding, and processing or even transcoding in JavaScript.

In another embodiment, the canvas element of HTML5 uses JavaScript to draw images on web pages. The canvas is a rectangular area, each pixel of which can be controlled. canvases have a variety of drawing paths, rectangles, circles, characters, and methods of adding images.

Due to the openness of the Web, video hotlinking is a common phenomenon, that is, a video file can only be viewed under an authorized domain name, and cannot be normally viewed under an unauthorized domain name. It is necessary to encrypt the video file itself, which guarantees the security of the video transmission process, so that even if someone downloads the video, the video cannot be directly viewed through the player. Correspondingly, the requirement of a decryption scheme for Web-played video is urgent.

In 3 months 2012, the national crypto-authority has formally published 6 crypto-industry standards such as "Zu Chong's sequence crypto-algorithm" including SM4 block crypto-algorithm. Similar to the DES and AES algorithms, the SM4 algorithm is a block cipher algorithm. The packet length is 128 bits, and the key length is also 128 bits. The encryption algorithm and the key expansion algorithm both adopt 32-round nonlinear iteration structures, encryption operation is carried out by taking a word (32 bits) as a unit, and each iteration operation is a round of transformation function F. The SM4 algorithm encryption and decryption algorithms are identical in structure, except that round keys are used instead, where the decryption round keys are the reverse order of the encryption round keys.

HTTPS transmission encryption, wherein a mainstream general browser only supports an AES encryption algorithm; the SM4 and other cryptographic algorithms are used, and can be realized only by a user-defined browser and other modes, so that the operation complexity and the product popularization difficulty of an end user are increased, and the original purpose of a plug-in-free scheme is violated.

In the embodiment of the application, a webestablishment-based method and a webestablishment-based system for supporting SM4 national encryption and Web plug-in-free decryption playing of a video are provided, so as to overcome many defects in the related art, which mainly include the disadvantages of fragment storage, the problem of transmission process safety, the problem of low decryption efficiency, the problem of low decoding efficiency, and the like.

The embodiment of the present application provides a video playing system as shown in fig. 4, which mainly includes a media server 41, a storage server 42, and a Web browser 43, and is suitable for a scene of real-time streaming recording and playing. Wherein:

the media server 41 mainly comprises a national secret encryption module, namely, the module encrypts audio and video media data through a national secret SM4 algorithm; and the video recording module is responsible for sending the encrypted audio and video media data to the storage server.

The storage server 42 mainly includes a media storage module, i.e. for receiving and storing media data from the media server; and the media sending module is used for sending the media data to the client.

The Web browser 43 mainly includes a UI layer 43a, a Js layer 43b, and a Wasm layer 43 c.

The UI layer 43a mainly includes a display module, i.e., an interface for rendering video and displaying video playback control; and the playing control module is used for playing the video, stopping the video, adjusting the volume, adjusting the video progress and the like.

The Js layer 43b is mainly used for logical operations, and mainly includes a download module, that is, a module for acquiring a key and an index from a server and downloading a video according to the index fragment; and a decoding playing module, namely, a module for decoding and playing the media data in fMP4 format supported by the browser in MSE.

The Wasm layer 43c is mainly realized by a Webassombly technology and an Emscript technology. In 2019, W3C announced that Webassembly was formally a fourth language that can run natively in a browser, following HTML, CSS, and JavaScript. WebAssembly is a new binary format, which can conveniently and rapidly 'run' other static language codes such as C/C + +/Rust and the like in a browser, and this characteristic provides infinite possibilities for front-end intensive computing scenarios. Emscript is a complete open source compiler tool chain of WebAssembly, using LLVM, with special attention to speed, size and Web platform, can compile C/C + +/Rust or any other language using LLVM into wasm. The wasm is a binary instruction format of the stack-based virtual machine, the performance of the wasm in the program execution process is superior due to the fact that the wasm adopts binary coding, and compared with a text format, the binary coding format occupies a smaller storage space.

The Wasm layer 43c mainly comprises a national password decryption module, namely, the module is used for decrypting the downloaded media data segment by using the national password SM4 algorithm to obtain data before encryption; and a decapsulating module, namely, for decapsulating the decrypted media data into audio data and video data, such as audio data in aac format and video data in h.264 format. And the encapsulation module is used for re-encapsulating the de-encapsulated audio data and video data into fMP4 format supported by the MSE of the Web browser.

In another embodiment of the present application, a system is provided as shown in fig. 5, comprising a media server 51 and a Web browser 52. In contrast to the previous embodiment, the Web browser 52 and its modules are identical. The difference is that a storage server is not needed, and the method is more suitable for scenes of live videos.

In another embodiment, a system is provided as shown in FIG. 6, similar to the previous embodiments, including a media server 61 and a Web browser 62. The difference is that the Js layer and the Wasm layer directly decode the coded data after being unpacked, and the Js layer only plays and renders the decoded data without decoding. The scheme also does not need a storage server and is suitable for scenes of live video.

In an embodiment of the present application, a method for playing a video is further provided, as shown in fig. 7, where the method is applied to a media server and a storage server of the above system, and includes the following steps:

step S701, when the media server receives a recording and broadcasting task, a secret key is distributed to the recording and broadcasting;

s702, encrypting audio and video data by using a cryptographic algorithm, and adding a frame header to each frame of data; simultaneously generating an index file;

specifically, a) a frame header is required to be added for each frame of video or audio, and the frame header includes information such as frame length and frame type.

b) Encrypting the media using SM4 cryptographic algorithm; the method proposes an ofb mode, is characterized by high encryption and decryption speed, does not require byte alignment of original data according to a certain length, is easy to realize and is most suitable for streaming media.

c) And simultaneously generating an index file. The index file records the time and the relative file offset of each I frame in the video, and the mode can realize the full-file accurate jump playing control operation based on the I frame interval.

d) The streaming storage has the advantages that the streaming storage is in a super-large file form, the implementation is simple, the file fragments can be effectively avoided, and the streaming storage is more advantageous than the file fragment storage in the aspect of time delay when being used for a low-delay live broadcast scene.

Step S703, the media server stores the index file and the encrypted video file in a storage server via a protocol including, but not limited to, Network File System (NFS).

The independent storage server is a node which needs to be independent and has larger flow under the condition of large concurrency; or under the public cloud environment, the existing storage service is utilized, and lower bandwidth charge can be obtained; the method can be simplified under the conditions of private deployment and the like, the media server directly stores the files or bare equipment into a local disk, and the media server undertakes the service of providing video reading.

In an embodiment of the present application, a method for playing a video is further provided, as shown in fig. 8, where the method is applied to a Web browser of the above system, and includes the following steps:

step S801, a user inputs a user name and a password through a Web browser, logs in a media server management platform and acquires Token; and the user enters a video playing page to request for playing the video.

Step S802, the Web browser carries a login Token to send a viewing request to the media server, and acquires a download address (pointing to an HTTP port corresponding to a media sending module of the storage server), a secret key and an index file of the video file after the authentication is passed. By using the index file, the full-text accurate jump playing action based on the I frame interval can be performed.

Step S803, according to the actions of playing the next segment, jumping, etc., the index file is analyzed, and the initial position of the video to be played in the index is determined.

Step S804, according to the time and offset of the I frame in the index file, checking whether the video file is downloaded completely, if the video in the index file is not downloaded completely, downloading the video in a time-sliced manner according to the time length, where the length of the sliced manner may be 10 seconds or about 20 seconds, and if the video is downloaded completely, ending the process until the data in the buffer area of the MSE is played completely, and then playing the video completely.

And step S805, decrypting the media data frame by using the SM4-OFB cryptographic algorithm according to the acquired key and the frame length information in the custom frame header.

Step 806, decapsulate the decrypted media data into original audio data and video data. Such as audio data in aac format and video data in h.264 format.

Step S807, repackage the audio data and video data into fMP4 format supported by the Web browser MSE.

And step S808, continuously adding the data in the fMP4 format to the MSE buffer area, and decoding and playing the video by the MSE of the Web browser video tag.

Step S809, when the data addition is completed at fMP4, a callback event of the completion of the update is triggered, at this time, a timer is started to check the difference between the current position of the video playing and the length of the video buffer, for example, the difference is less than 2 seconds, the timer is stopped, and the step 3 is executed again, that is, the index file is continuously analyzed to download the video.

In another embodiment, a storage server is not required, and compared with the previous embodiment, this embodiment is more suitable for a live scene, as shown in fig. 9, the method for playing a video provided by the embodiment of the present application is applied to a media server, and includes the following steps:

step S901, when the media server receives a live task, a key is allocated for this live broadcast.

Step S902, the media server uses the national encryption algorithm to encrypt the audio and video data in a real-time streaming way, and adds a frame header to each frame of data.

Step S903, the media server transmits the encrypted media data to the Web browser in real time.

As shown in fig. 10, the method for playing video as described above is applied to a Web browser, and includes the following steps:

step S1001, a user inputs a user name and a password through a Web browser, and logs in a media server management platform to obtain Token; and the user enters a video playing page to request for playing the video.

Step S1002, the Web browser carries a login Token to send a viewing request to the media server, and after the authentication is passed, a live broadcast address (pointing to a Websocket port corresponding to a media sending module of the media server) and a secret key of the video file are obtained.

Step S1003, establishing a connection between the Web browser and the media server according to the live broadcast address, and starting to transmit the encrypted media data.

And step S1004, decrypting the media data frame by using SM4-OFB cryptographic algorithm according to the acquired key and the frame length information in the custom frame header.

Step S1005, decapsulating the decrypted media data into original audio data and video data. Such as audio data in aac format and video data in h.264 format.

Step S1006, judging whether the user selects the low-delay mode or the high-performance mode.

If the user selects the low-delay mode under the condition that the machine performance is excellent according to self judgment, the following steps are carried out:

and step S1007, directly decoding the original audio data and the video data into pcm data and yuv data in a Webelementary mode.

Step S1008, playing the decoded pcm data through the AudioContext; the yuv data is rendered by WebGL.

If the user selects the high performance mode, the following steps are performed:

step S1009 repackages the audio data and the video data into fMP4 format supported by the Web browser MSE.

And step S1010, continuously adding the data in the fMP4 format to an MSE buffer area, and decoding and playing the video by the MSE of the Web browser video label.

In the embodiment of the application, the accurate skip play of the full file based on the I frame interval in the recorded broadcast scene, the low delay and the high-performance optional play of the live broadcast scene are simultaneously realized in a way of encrypting and storing data in a streaming way. And, by using SM4 cryptographic algorithm to encrypt and decrypt data, domestic autonomous data transmission process security is realized. In addition, the embodiment of the application realizes the safety of decryption analysis process and the acceleration of decoding hardware under the condition that the Web has no plug-in playing through Webassembery technology and MSE technology.

Therefore, the embodiment of the application adopts a streaming media data encryption storage mode, and compared with a TS format video fragment file-based fragment encryption storage mode, the method can effectively avoid file fragments, and has more advantages in video playing delay in a real-time scene.

Compared with the index file based on the slicing M3U8 format, the method has the advantages of playing control operation such as full-file accurate skip based on the I frame interval.

In addition, in the embodiment of the application, the SM4-OFB cryptographic algorithm is adopted, and compared with encryption and decryption algorithms such as AES, MD5 and DES, domestic autonomous safe data transmission is achieved.

As shown in fig. 11, an embodiment of the present application provides a video playing apparatus 1100, which is applied to a browser, and includes:

a first obtaining module 1101, configured to obtain a video address and a video key from a media server according to a video playing request;

a second obtaining module 1102, configured to obtain, through the video address, a data stream of a video to be played;

a decryption module 1103, configured to decrypt the data stream frame by using the video key, so as to obtain data to be played;

a playing module 1104, configured to play the data to be played processed into a playable format on the web page of the browser.

In some embodiments, the apparatus further comprises:

and the processing module is used for processing the data to be played obtained by decryption into the data to be played in the playable format which can be directly identified by the page element of the browser.

In some embodiments, the processing module comprises: a decapsulation submodule and a decoding submodule, or a decapsulation submodule and an encapsulation submodule, wherein,

the decapsulation submodule is used for decapsulating the decrypted data to be played to obtain original audio data and original video data;

the decoding submodule is used for decoding the original audio data and the original video data through a web-oriented binary coding format Webelementary to obtain the data to be played in the playable format;

and the packaging submodule is used for packaging the original audio data and the original video data into data to be played in a data format supported by media source expansion MSE of the browser.

In some embodiments, the video to be played is a recorded video file; the playing module comprises:

the first storage submodule is used for storing the data to be played which is processed into a playable format in a video buffer area;

and the first playing submodule is used for responding to the situation that the data to be played in the playable format exists in the video buffer area, and playing the data to be played in the playable format in the video buffer area.

In some embodiments, the apparatus further comprises:

a third obtaining module, configured to obtain a video index file from the media server, where the video index file is used to determine a plurality of key frames in the video file; the key frame is used for determining the playing position of the video to be played.

In some embodiments, the second obtaining module includes:

the first determining submodule is used for determining a key frame corresponding to a position to be played of a video according to the index file in response to the fact that a preset instruction for controlling video playing or video time corresponding to data to be played in a playable format in the video buffer area is detected to be less than the preset time, wherein the position of the key frame is earlier than or equal to the position to be played;

and the downloading submodule is used for downloading the data stream of the key frame as the data stream of the starting position for the preset time length through the video address if the data stream of the video to be played is not downloaded.

In some embodiments, the video to be played is a live video;

the playing module comprises:

the second storage submodule is used for storing the data to be played in the playable format in a video buffer area in real time;

the second playing submodule is used for playing the data to be played in the playable format in the video buffer area in real time on a webpage of the browser;

or

And the third playing sub-module is used for directly playing the data to be played in the playable format after being decoded by Webestablishment on the webpage of the browser.

In some embodiments, the apparatus further comprises:

the fourth acquisition module is used for acquiring the user information of the browser;

and the first sending module is used for responding to the browser entering a video playing page and initiating a video playing request to the media server according to the user information.

As shown in fig. 12, an embodiment of the present application provides a video playing apparatus 1200, which is applied to a media server, and includes:

a second sending module 1201, configured to send a video address and a video key according to the received video playing request;

a transmission module 1202, configured to transmit a frame-by-frame encrypted data stream of the video to be played based on the video address.

In some embodiments, the video to be played is a recorded video file; the device further comprises:

a third sending module, configured to send a video index file for determining a plurality of key frames in the video file; the key frame is used for determining the playing position of the video to be played.

In some embodiments, the apparatus further comprises:

the first encryption module is used for encrypting the video to be played frame by frame;

the storage module is used for storing the video to be played and the video index file which are encrypted frame by frame to a storage server;

the video address is a corresponding address in the storage server; the transmission module is specifically configured to:

In some embodiments, the video to be played is a live video; the transmission module includes:

the encryption submodule is used for encrypting the data stream of the live video frame by frame in real time;

and the sending submodule is used for sending the encrypted data stream in real time.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

In addition to the methods, apparatus and devices described above, embodiments of the present application may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to the various embodiments of the present application described in the "exemplary methods" section of this specification above.

The computer program product may be written with program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present application may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform steps in a method according to various embodiments of the present application described in the "exemplary methods" section above of this specification.

The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The foregoing describes the general principles of the present application in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present application are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present application. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the foregoing disclosure is not intended to be exhaustive or to limit the disclosure to the precise details disclosed.

The block diagrams of devices, apparatuses, systems referred to in this application are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".

It should also be noted that in the devices, apparatuses, and methods of the present application, the components or steps may be decomposed and/or recombined. These decompositions and/or recombinations are to be considered as equivalents of the present application.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit embodiments of the application to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.

Claims

1. A video playing method is applied to a browser and comprises the following steps:

acquiring a data stream of a video to be played through the video address;

2. The method of claim 1, further comprising:

3. The method according to claim 2, wherein the processing the decrypted data to be played back into the data to be played back in the playable format that can be directly identified by the page element of the browser comprises:

decoding the original audio data and the original video data through a web-oriented binary coding format Webelementary to obtain data to be played in the playable format; or the like, or, alternatively,

and packaging the original audio data and the original video data into data to be played in a data format supported by media source extended MSE of the browser.

4. The method according to any one of claims 1 to 3, wherein the video to be played is a recorded video file; the playing of the web page in the browser is processed into data to be played in a playable format, and the data includes:

5. The method of claim 4, further comprising:

6. The method according to claim 5, wherein the obtaining the data stream of the video to be played through the video address comprises:

in response to detecting that a preset instruction for controlling video playing or video time corresponding to the data to be played in the playable format in the video buffer area is smaller than a preset residual playing time, determining a key frame corresponding to a position to be played of a video according to the index file, wherein the position of the key frame is earlier than or equal to the position to be played;

7. The method according to any one of claims 1 to 3, wherein the video to be played is a live video; the playing of the web page in the browser is processed into data to be played in a playable format, and the data includes:

or

8. The method of claim 1, further comprising:

acquiring user information of a browser;

9. A video playing method is applied to a media server and comprises the following steps:

10. The method according to claim 9, wherein the video to be played is a recorded video file; the method further comprises the following steps:

11. The method of claim 10, further comprising:

encrypting a video to be played frame by frame;

12. The method according to claim 9, wherein the video to be played is a live video; the data stream for transmitting the frame-by-frame encrypted video to be played comprises:

encrypting the data stream of the live video frame by frame in real time;

and transmitting the encrypted data stream in real time.

13. A video playing apparatus, wherein the apparatus is applied to a browser, and comprises:

14. A video playing apparatus, applied to a media server, comprising:

15. A video playback system, the system comprising:

the media server is used for encrypting the video to be played frame by frame;

16. The system of claim 15, wherein the browser comprises:

the logic control layer is used for acquiring the video to be played;

17. The system of claim 16, wherein the compilation processing layer comprises:

18. The system of claim 16, further comprising:

19. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus; a memory for storing a computer program; a processor for implementing the method steps of any one of claims 1-8 or 9-12 when executing a program stored in a memory.

20. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any of claims 1-8 or 9-12.