CN116797971A - Video stream identification method and device - Google Patents

Abstract

One or more embodiments of this specification provide a video stream identification method and device. The method includes: determining the optical flow change information corresponding to the target video stream, wherein the theoretical collection time period of the target video stream is a preset time period. And the theoretical collection device is the target user terminal. And, determine the posture change information of the target user terminal, wherein the posture change information is determined based on the sensor detection information of the target user terminal within a preset time period. Based on the comparison information between the determined optical flow change information and pose change information, the credibility identification result for the target video stream is determined.

Description

A video stream identification method and device

This application was submitted to the China Patent Office on December 31, 2019. The application number is 2019114033183. Invention

A divisional application of a Chinese patent application titled "A video stream identification method and device".

Technical field

This document relates to the field of Internet technology, and in particular to a video stream identification method and device.

Background technique

At present, with the advent of the Internet era and the rapid development of mobile Internet technology, the Internet has been widely used in people's daily study, work and life. Various daily affairs of people can be processed and presented through the Internet using user terminals (such as smartphones). Among them, users can install corresponding applications in their smartphones according to their actual needs, such as payment applications, financial management applications, instant messaging applications, shopping applications, etc.

Currently, if a user needs to complete a certain business under a certain application, the user needs to trigger the camera in the smartphone to view the target object (such as the user's face, bank card, ID card) through the upload control under the application. , bills, etc.) and upload the video stream information of the target object so that corresponding Internet services can be opened for users based on the video stream information. However, there may be malicious users using video stream injection attacks, that is, directly hooking the hardware/driver/API layer of the smartphone to replace the original video stream collected in real time with the pre-stored video stream information, so that the pre-stored video can be Flow information is used as the input information source to achieve the purpose of maliciously triggering target business execution.

For example, taking account login for payment applications based on face recognition as an example, a malicious user collects video information by obtaining the target user's face in advance, and then replaces the target user's face with the video frame when collecting the face image. The face collection video information is uploaded to the identity verification server. At this time, the identity verification server will authenticate the logged-in user based on the face collection video information, and then determine that the user identity verification has passed, so that the malicious user can complete the identity verification and enter the user The operation interface will provide an entrance for malicious users to perform illegal activities, and the purpose of ensuring account security through identity verification cannot be achieved.

Therefore, it is necessary to provide a technical solution that can quickly, accurately, and reliably identify video streams.

Contents of the invention

The purpose of one or more embodiments of this specification is to provide a video stream identification method. The video stream identification method includes:

Determine the optical flow change information corresponding to the target video stream, wherein the theoretical collection time period of the target video stream is a preset time period and the theoretical collection device is the target user terminal. And, determine the posture change information of the target user terminal, wherein the posture change information is determined based on the sensor detection information of the target user terminal within the preset time period. Based on the comparison information between the optical flow change information and the pose change information, a credibility identification result for the target video stream is determined.

The purpose of one or more embodiments of this specification is to provide a video stream identification device. The video stream identification device includes:

Determine the optical flow change information corresponding to the target video stream, wherein the theoretical collection time period of the target video stream is a preset time period and the theoretical collection device is the target user terminal. And, determine the posture change information of the target user terminal, wherein the posture change information is determined based on the sensor detection information of the target user terminal within the preset time period. Based on the comparison information between the optical flow change information and the pose change information, a credibility identification result for the target video stream is determined.

An object of one or more embodiments of this specification is to provide a video stream recognition device, including: a processor; and a memory arranged to store computer-executable instructions.

When executed, the computer-executable instructions cause the processor to determine the optical flow change information corresponding to the target video stream, wherein the theoretical collection time period of the target video stream is a preset time period and the theoretical collection device is the target user. terminal. And, determine the posture change information of the target user terminal, wherein the posture change information is determined based on the sensor detection information of the target user terminal within the preset time period. Based on the comparison information between the optical flow change information and the pose change information, a credibility identification result for the target video stream is determined.

An object of one or more embodiments of this specification is to provide a storage medium for storing computer-executable instructions. When the executable instructions are executed by the processor, the optical flow change information corresponding to the target video stream is determined, wherein the theoretical collection time period of the target video stream is a preset time period and the theoretical collection device is the target user terminal. And, determine the posture change information of the target user terminal, wherein the posture change information is determined based on the sensor detection information of the target user terminal within the preset time period. Based on the comparison information between the optical flow change information and the pose change information, a credibility identification result for the target video stream is determined.

Description of the drawings

In order to more clearly illustrate one or more embodiments of this specification or technical solutions in the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, in the following description The drawings are only some of the embodiments described in one or more of this specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a first flow diagram of a video stream identification method provided by one or more embodiments of this specification;

Figure 2 is a schematic diagram of the specific implementation principle of the video stream identification method provided by one or more embodiments of this specification;

Figure 3 is a second flow diagram of a video stream identification method provided by one or more embodiments of this specification;

Figure 4 is a schematic diagram of the specific implementation principle of determining optical flow change information in the video stream identification method provided by one or more embodiments of this specification;

Figure 5 is a third flow diagram of a video stream identification method provided by one or more embodiments of this specification;

Figure 6 is a schematic diagram of the specific implementation principle of determining pose change information in the video stream recognition method provided by one or more embodiments of this specification;

Figure 7 is a fourth schematic flowchart of a video stream identification method provided by one or more embodiments of this specification;

Figure 8 is a schematic diagram of the module composition of the video stream identification device provided by one or more embodiments of this specification;

Figure 9 is a schematic structural diagram of a video stream recognition device provided by one or more embodiments of this specification.

Detailed ways

In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be described below in conjunction with the drawings in one or more embodiments of this specification. To clearly and completely describe, it is obvious that the described embodiments are only one or more partial embodiments of this specification, but not all embodiments. Based on one or more embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts should fall within the protection scope of this document.

It should be noted that, without conflict, one or more embodiments and features in the embodiments in this specification can be combined with each other. One or more embodiments of this specification will be described in detail below in conjunction with the embodiments with reference to the accompanying drawings.

One or more embodiments of this specification provide a video stream identification method and device, which identifies the target by comparing and analyzing the optical flow change information determined based on the target video stream and the pose change information of the target user terminal. Whether the video stream is the video stream information captured and uploaded in real time against the target object, so as to quickly identify the situation of a malicious video stream attack that uses a pre-stored non-real-time captured video stream to replace the real-time captured video stream, so as to promptly attack the malicious video stream The non-real-time captured video stream injected by the attack is intercepted to improve the accuracy of subsequent business processing.

Figure 1 is a schematic flowchart of a first video stream identification method provided by one or more embodiments of this specification. The method in Figure 1 can be executed by a user terminal or by a server. The user terminal can be an intelligent Mobile terminals such as mobile phones can also be terminal devices such as Internet of Things devices. Specifically, the user terminal can be used to collect the video stream information of the target object and perform credibility identification on the video stream information. After the credibility identification passes When identifying whether to perform corresponding control operations based on the target video stream information, or uploading the video stream information to the server, so that the server continues to perform user authentication based on the video stream information; wherein, the server can be a background Server or cloud server. Specifically, the server is used to receive the video stream information uploaded by the user terminal, and perform credibility identification on the video stream information. When the credibility identification passes, the server performs the verification based on the video stream information. User authentication, and providing certain business services to the user when the authentication is passed.

Among them, regarding the process of credibility identification of the video stream information of the target object by the user terminal or the server, as shown in Figure 1, the above video stream identification method at least includes the following steps:

S102. Determine the optical flow change information corresponding to the target video stream, where the theoretical collection time period of the target video stream is a preset time period and the theoretical collection device is the target user terminal;

Specifically, after detecting the video stream collection request, the user terminal uses the camera device to collect the video stream information of the target object, and obtains the target video stream for user identity verification. The theoretical collection time period of the target video stream includes: The time period from the video stream collection start time to the collection end time;

Among them, considering that the above target video stream may be video stream information collected in real time by the camera device; it may also be that the video stream information collected in real time is replaced by a preset video stream attack method, and the injected video stream information that is not collected in real time, Therefore, before performing user identity verification based on the target video stream, the optical flow change information corresponding to the target video stream is first determined so that the optical flow change information can be compared with the pose change information of the target user terminal to identify the target video stream. Whether it is credible video stream information collected in real time by the target user terminal;

Among them, for the case where the target video stream is video stream information collected in real time, the theoretical collection time period and the actual collection time period of the target video stream are the same, and the theoretical collection device and the actual collection device are both target user terminals, that is, the target video The stream is collected by the target user terminal within a preset time period; and the target video stream is a situation where the video stream information collected in real time is replaced by the preset video stream attack method, and the non-real-time collected video stream information is injected , the theoretical collection time period of the target video stream is different from the actual collection time period, that is, the actual collection time of the target video stream is not the preset time period, and the actual collection device may not be the target user terminal, but pre-stored by the target user terminal. Used to replace real-time collected video stream information.

S104. Determine the pose change information of the target user terminal, where the pose change information is determined based on the sensor detection information of the target user terminal within a preset time period;

Wherein, the target user terminal includes: a camera device and at least one preset sensor; the preset sensor may be an IMU sensor, such as a gyroscope, or an acceleration sensor, and the sensor detection information may include: a three-axis attitude angle of the sensor, an angle At least one of velocity, and acceleration;

Specifically, while the target user terminal uses a camera device to collect the video stream of the target object, it also uses at least one preset sensor to collect sensor detection information, so as to determine the pose change information of the target user terminal based on the sensor detection information; and then The pose change information is used as a basis for comparison with the optical flow change information of the target video stream, thereby identifying the credibility of the target video stream.

S106: Determine the credibility identification result for the target video stream based on the determined comparison information between the optical flow change information and the pose change information.

Among them, for the case where the target video stream is video stream information collected in real time, the target video stream and the sensor detection information are collected synchronously when the target user terminal is in a certain jittering state, that is, when the target user terminal is not absolutely still. In this case, the target video stream and sensor detection information are collected at the same time. Therefore, there should be direction change consistency between the optical flow change direction naturally recorded in the target video stream and the sensor spatial motion direction naturally recorded in the sensor detection information. Therefore, based on The comparison information between the optical flow change information determined by the target video stream and the pose change information determined based on the sensor detection information should meet the preset change consistency conditions;

In the case where the target video stream is non-real-time collected video stream information, the target video stream and the sensor detection information are not collected synchronously in the same jitter state. Therefore, the direction of change of the optical flow naturally recorded in the target video stream is different from that of the sensor detection information. There is no direction change consistency between the sensor spatial motion directions naturally recorded in the sensor detection information. Therefore, the comparison between the optical flow change information determined based on the target video stream and the pose change information determined based on the sensor detection information The information will not meet the preset change consistency conditions;

It can be seen that by comparing the optical flow change information determined based on the target video stream and the pose change information determined based on the sensor detection information, it can be identified whether the target video stream is a trusted video stream collected by the target user terminal in real time. information;

In one or more embodiments of this specification, by comparing and analyzing the optical flow change information determined based on the target video stream and the pose change information of the target user terminal, whether the target video stream is real-time for the target object is identified. Captured and uploaded video stream information to quickly identify malicious video stream attacks that use pre-stored non-real-time captured video streams to replace real-time captured video streams, so that the non-real-time captured video streams injected into the malicious video stream attacks can be promptly identified Interception is performed to improve the accuracy of subsequent business processing.

Specifically, for the case where the user terminal performs credibility identification on the target video stream, the user terminal obtains the target video stream and the sensor detection information collected within the preset time period, and determines the target video based on the above steps S102 to S106. The credibility identification result of the stream; when the credibility identification result is that the target video stream is a trusted video stream collected in real time, identify whether to perform the corresponding control operation based on the target video stream information, or upload the target video stream to the server end, so that the server side performs user authentication based on the target video stream.

Correspondingly, in the case where the server side performs credibility identification on the target video stream, the user terminal obtains the target video stream and the sensor detection information collected within the preset time period, and uploads the target video stream and sensor detection information. to the server, so that the server determines the credibility identification result for the target video stream based on the above steps S102 to S106; and when the credibility identification result is that the target video stream is a credible video stream collected in real time, based on the Target video stream for user authentication.

In the specific implementation, as shown in Figure 2, taking the target object as the user ID and the target user terminal as a smartphone as an example, a schematic diagram of the specific implementation principle of the video stream recognition method is given, which specifically includes:

(1) After detecting the video stream collection request, that is, after detecting the user's trigger operation on the video stream upload control, the user terminal uses the camera device to collect the video stream information of the target object; wherein, the video stream information of the target object The actual collection time period is a preset time period, which includes: multiple designated time nodes;

(2) The user terminal uses at least one preset sensor to collect sensor detection information within the above preset time period; wherein the target user terminal includes: a camera device and at least one preset sensor;

(3) Obtain the target video stream to be identified; wherein the theoretical collection time period of the target video stream is a preset time period and the theoretical collection device is the target user terminal;

Specifically, if the theoretical collection time period and the actual collection time period of the target video stream are the same, and the theoretical collection equipment and the actual collection equipment are both target user terminals, then the target video stream is the target collected by the above-mentioned camera device. The video stream information of the object; otherwise, the target video stream is an untrusted video stream that has been maliciously replaced;

(4) Obtain sensor detection information collected within a preset time period; wherein the sensor detection information may include: at least one of the sensor's three-axis attitude angle, angular rate, and acceleration; the camera used by the target user terminal While collecting the video stream of the target object, the device also uses at least one preset sensor to collect sensor detection information;

(5) Based on the obtained multiple object image frames corresponding to multiple specified time nodes in the target video stream, determine the optical flow change information corresponding to the target video stream;

(6) Based on the obtained sensor detection information corresponding to each designated time node, determine the pose change information of the target user terminal;

(7) Based on the comparison information between the determined optical flow change information and pose change information, determine the credibility recognition result for the target video stream.

It should be noted that the above processes (3) to (7) can be executed by the user terminal, especially the information processing module in the user terminal, and can also be executed by the server.

Wherein, for the process of determining the optical flow change information based on the target video stream of the target object, the above-mentioned preset time period includes a plurality of designated time nodes; wherein, the plurality of designated time nodes can be performed on the preset time period according to a certain time interval. obtained by division;

Correspondingly, as shown in Figure 3, the above-mentioned S102 determines the optical flow change information corresponding to the target video stream, specifically including:

S1021. Obtain the target video stream to be identified, where the target video stream includes: multiple object image frames containing the target object corresponding to multiple specified time nodes respectively;

Among them, for the case where the server side performs credibility identification on the target video stream, the target video stream is uploaded to the server side by the target user terminal to trigger the server side to perform user identity verification based on the target video stream;

S1022: Determine the optical flow change information corresponding to the target video stream based on the object image frames at multiple specified time nodes.

Specifically, after receiving the target video stream uploaded by the target user terminal, the server does not directly perform user identity verification based on the target video stream, but first determines the corresponding optical flow change information based on the target video stream, and then based on the target video stream. The optical flow change information and the pose change information determined by the sensor detection information collected in the same preset time period are used to identify the credibility of the target video stream. The credibility identification result is that the target video stream is collected in real time. When receiving a video stream, user authentication is performed based on the target video stream.

Specifically, the above-mentioned S1022 determines the optical flow change information corresponding to the target video stream based on the object image frames at multiple specified time nodes, which specifically includes:

Step 1: Determine two object image frames corresponding to two adjacent specified time nodes as an image frame combination;

Step 2: For each image frame combination, use the preset optical flow method to identify the image optical flow information for the image frame combination, and obtain the corresponding optical flow spatial motion matrix;

Among them, the existing optical flow method can be used to analyze the optical flow direction of multiple object image frames in the target video stream to obtain optical flow change information; specifically, the optical flow method analyzes the optical flow direction of each pixel in each object image frame. The point is assigned a velocity vector, forming a motion vector field. Based on the displacement direction of each pixel in the object image frame corresponding to two adjacent specified time nodes, the velocity vector characteristics of each pixel are obtained. According to the velocity vector characteristics of each pixel, the corresponding optical flow spatial motion matrix is determined.

Step 3: Determine the optical flow change information corresponding to the target video stream according to the optical flow spatial motion matrix corresponding to each image frame combination.

Among them, the optical flow naturally recorded in the video stream information is generated by the movement of the camera device caused by the jitter of the user terminal; the optical flow can express changes in the image, and the optical flow reflects the jitter amplitude of the user terminal. At this time, if the sensor detection information is collected through the preset sensor in the user terminal at the same time, the spatial movement direction naturally recorded in the sensor detection information is also caused by the movement of the preset sensor caused by the jitter of the user terminal. The spatial movement direction It can also reflect the jitter amplitude of the user terminal. Since the jitter amplitude is synchronous and consistent, the optical flow direction change rules naturally recorded in the video stream information are directly related to the spatial motion direction change rules naturally recorded in the sensor detection information. The change pattern of the optical flow direction and the change pattern of the sensor's spatial motion direction should be consistent.

Further, consider that if the target object is relatively stationary, for example, the target object is a relevant document placed on a fixed plane, the optical flow naturally recorded in the video stream information at this time is the movement of the camera device caused by the jitter of the user terminal. produced. But in fact, there may be situations where the target object itself will move. For example, if the target object is the user's face, the optical flow naturally recorded in the video stream information is not only caused by the movement of the camera device caused by the jitter of the user terminal. It also includes the optical flow caused by the movement of the target itself. If the optical flow caused by the movement of the target itself is taken into account, it will affect the accuracy of the comparison information between the optical flow change information and the pose change information. degree, thereby reducing the accuracy of the credibility identification results for the target video stream;

Therefore, in order to further improve the accuracy of the credibility recognition results for the target video stream, the image area where the moving target is located can be eliminated from the object image frame, and the eliminated object image frame can be used as the basis for determining the optical flow change information, corresponding to In the above step two, for each image frame combination, use the preset optical flow method to identify the image optical flow information for the image frame combination, and obtain the corresponding optical flow spatial motion matrix, which specifically includes:

For each image frame combination, perform designated area elimination on the two object image frames in the image frame combination to obtain the two object image frames after elimination; wherein, the designated area is the image area containing the moving target;

The preset optical flow method is used to identify the image optical flow information of the two eliminated object image frames, and the corresponding optical flow spatial motion matrix is obtained.

Furthermore, considering that it is necessary to compare the optical flow change information of the target video stream with the pose change information of the target user terminal to identify whether the target video stream is the trusted video stream information collected by the target user terminal in real time, in order to improve the information The accuracy and referability of the comparison, therefore, it is necessary to determine the optical flow change information and pose change information in the same spatial coordinate system. Based on this, the above step three is to combine the corresponding optical flow spatial motion matrices according to each image frame. , determine the optical flow change information corresponding to the target video stream, specifically including:

For each optical flow spatial motion matrix, perform coordinate system transformation on the optical flow spatial motion matrix under the preset spatial coordinate system to obtain the transformed optical flow spatial motion matrix;

According to each transformed optical flow spatial motion matrix, the optical flow change information corresponding to the target video stream is determined.

In a specific embodiment, as shown in Figure 4, a schematic diagram of the specific implementation principle of determining optical flow change information in the video stream identification method is given, specifically as follows:

(1) Obtain the object image frames corresponding to multiple specified time nodes within the preset time period, for example, object image frame 1...object image frame i...object image frame n;

(2) Determine two object image frames corresponding to two adjacent designated time nodes as an image frame combination;

(3) For each image frame combination, use the preset optical flow method to identify the image optical flow information of the image frame combination to obtain the corresponding optical flow spatial motion matrix. For example, the currently recognized image frame combination includes: object image frame i and object image frame i+1, where i=1...n-1;

(4) For each optical flow spatial motion matrix, perform coordinate system transformation on the optical flow spatial motion matrix under the preset spatial coordinate system to obtain the transformed optical flow spatial motion matrix;

(5) According to each transformed optical flow spatial motion matrix, determine the optical flow change information corresponding to the target video stream.

Wherein, while the target user terminal uses a camera device to collect the video stream of the target object, it also collects sensor detection information through at least one preset sensor, so as to determine the pose change information of the target user terminal based on the sensor detection information. Specifically, , for the process of determining pose change information based on sensor detection information, the above-mentioned preset time period includes multiple designated time nodes; wherein, the multiple designated time nodes can be obtained by dividing the preset time period according to a certain time interval, That is, the multiple designated time nodes are the same as the designated time nodes selected to determine the optical flow change information;

Correspondingly, as shown in Figure 5, the above-mentioned S104 determines the pose change information of the target user terminal, specifically including:

S1041, obtain sensor detection information collected by at least one preset sensor of the target user terminal at each designated time node;

Among them, for the case where the server side performs credibility identification of the target video stream, the sensor detection information is uploaded to the server side by the target user terminal, and the actual collection time of the sensor detection information is the same as the theoretical collection time of the target video stream;

S1042: Determine the pose change information of the target user terminal based on the sensor detection information at multiple specified time nodes.

After obtaining the sensor detection information corresponding to each designated time node, based on the sensor detection information, the sensor spatial motion information of each designated time node is determined, and further, the pose change information of the target user terminal is determined.

Specifically, the above-mentioned S1042 determines the pose change information of the target user terminal based on the sensor detection information at multiple specified time nodes, specifically including:

Step 1: Determine the corresponding sensor spatial motion matrix based on the sensor detection information corresponding to two adjacent designated time nodes;

Specifically, the spatial state information of the preset sensors is identified based on the sensor detection information collected by the preset sensors; the corresponding sensor spatial motion matrix is determined based on the spatial state information corresponding to two adjacent designated time nodes;

Step 2: Determine the pose change information of the target user terminal according to the spatial motion matrix of each sensor.

Furthermore, considering that it is necessary to compare the optical flow change information of the target video stream with the pose change information of the target user terminal to identify whether the target video stream is the trusted video stream information collected by the target user terminal in real time, in order to improve the information Therefore, the optical flow change information and pose change information need to be determined in the same spatial coordinate system. Based on this, the above step 2 determines the position of the target user terminal according to the spatial motion matrix of each sensor. Posture change information, including:

For each sensor spatial motion matrix, perform coordinate system transformation on the sensor spatial motion matrix under the preset spatial coordinate system to obtain the transformed sensor spatial motion matrix;

According to each transformed sensor spatial motion matrix, the pose change information of the target user terminal is determined.

In a specific embodiment, as shown in Figure 6, a schematic diagram of the specific implementation principle of determining pose change information in the video stream recognition method is given, specifically as follows:

(1) Obtain the sensor detection information corresponding to multiple specified time nodes within the preset time period, for example, sensor detection information 1...sensor detection information i...sensor detection information n;

(2) Determine the corresponding sensor spatial motion matrix based on the sensor detection information corresponding to two adjacent specified time nodes. For example, the currently identified detection information combination includes: sensor detection information i and sensor detection information i+1, where, i=1...n-1;

(3) For each sensor spatial motion matrix, perform coordinate system transformation on the sensor spatial motion matrix under the preset spatial coordinate system to obtain the transformed sensor spatial motion matrix;

(4) Determine the pose change information of the target user terminal based on each transformed sensor spatial motion matrix.

Among them, considering that if the target video stream is video stream information collected in real time, there should be direction change consistency between the optical flow change direction naturally recorded in the target video stream and the sensor spatial motion direction naturally recorded in the sensor detection information. Therefore, , the credibility identification process for the target video stream, as shown in Figure 7, the above-mentioned S106, determine the credibility of the target video stream based on the comparison information between the determined optical flow change information and the pose change information. The identification results include:

S1061, compare the determined optical flow change information with the pose change information, and obtain the corresponding optical flow pose comparison result;

The above-mentioned optical flow pose comparison results may include: at least one of a relative difference comparison result between the optical flow change information and the pose change information, a change similarity comparison result, or a relative change trend comparison result.

S1062, determine whether the determined optical flow pose comparison result meets the preset change consistency condition;

Among them, the preset change consistency conditions include: the relative difference of the optical flow pose is less than the preset maximum difference threshold, the similarity of the optical flow pose change is greater than the preset minimum similarity threshold, or the relative change trend of the optical flow pose satisfies the preset Trends towards consistency conditions.

If the judgment result is yes, then S1063, determine that the target video stream is the trusted video stream information collected in real time;

Specifically, if it is determined that the optical flow pose comparison result satisfies the preset change consistency condition, it means that the theoretical collection time period and the actual collection time period of the target video stream are the same, and the theoretical collection device and the actual collection device are both targets. The user terminal, that is, the target video stream is collected by the target user terminal within a preset time period. Therefore, the target video stream has not been attacked by malicious users.

If the judgment result is no, then S1064, determine that the target video stream is untrusted video stream information collected in non-real time;

Specifically, if it is determined that the optical flow pose comparison result does not meet the preset change consistency condition, it means that the theoretical collection time period of the target video stream is different from the actual collection time period, that is, the actual collection time of the target video stream is not During the preset time period, the actual collection device may not be the target user terminal. Therefore, the target video stream may be a non-real-time collected video stream injected by a malicious user who replaces the real-time collected video stream information through the preset video stream attack method. information.

Wherein, the above-mentioned optical flow change information includes: multiple optical flow spatial motion matrices within a preset time period, and the above-mentioned pose change information includes: multiple sensor spatial motion matrices within a preset time period; preferably, the The optical flow spatial motion matrix and the sensor spatial motion matrix are obtained by coordinate system transformation in the same spatial coordinate system;

Correspondingly, in S1061 above, the determined optical flow change information is compared with the pose change information to obtain the corresponding optical flow pose comparison result, which specifically includes:

Compare multiple optical flow spatial motion matrices with multiple sensor spatial motion matrices to obtain corresponding optical flow pose comparison results.

Specifically, the optical flow spatial motion matrix and the sensor spatial motion matrix are in one-to-one correspondence. For the same two adjacent specified time nodes (that is, the same specified time node combination), they respectively correspond to an optical flow spatial motion matrix and an optical flow spatial motion matrix. Sensor spatial motion matrix. Therefore, for each specified time node combination, perform a difference operation between the corresponding optical flow spatial motion matrix and the sensor spatial motion matrix to obtain the relative difference in optical flow pose corresponding to the specified time node combination; or,

For each specified time node combination, perform matrix similarity calculation on the corresponding optical flow spatial motion matrix and sensor spatial motion matrix to obtain the optical flow pose change similarity (for example, matrix distance) corresponding to the specified time node combination; or ,

The first change trend is determined based on multiple optical flow spatial motion matrices, and the second change trend is determined based on multiple sensor spatial motion matrices. Based on the first change trend and the second change trend, a relative change trend of optical flow pose is determined.

Furthermore, in order to further improve the credibility recognition accuracy of the target video stream, the initial change information can be preprocessed before comparing the optical flow change information with the pose change information, and then the preprocessed optical flow information can be preprocessed. The change information is compared with the preprocessed pose change information. Based on this, multiple optical flow spatial motion matrices are compared with multiple sensor spatial motion matrices to obtain the corresponding optical flow pose comparison results. Specifically include:

Preprocess multiple optical flow spatial motion matrices and multiple sensor spatial motion matrices to obtain preprocessed multiple optical flow spatial motion matrices and preprocessed multiple sensor spatial motion matrices, where the preprocessing includes: At least one of smoothing filtering, de-noising, and comparison starting point alignment;

Compare the preprocessed multiple optical flow spatial motion matrices with the preprocessed multiple sensor spatial motion matrices to obtain the corresponding optical flow pose comparison results.

Among them, considering that there may be a certain time delay between the collection time period of video stream information and the collection time period of sensor detection information, therefore, by aligning the starting point of comparison between the optical flow change information and the pose change information; and also considering There may be some abnormal points in the determined optical flow change information and pose change information, so the abnormal points need to be eliminated; and the collected relevant information may be interfered by external noise, so the change information can be de-noised. ;Thereby further improving the credibility identification accuracy of the target video stream.

Specifically, for the determination process of the optical flow pose comparison result, it can be analyzed whether the difference between the optical flow spatial motion matrix and the sensor spatial motion matrix is less than a preset threshold, or the optical flow spatial motion matrix and the sensor spatial motion can be analyzed. Whether the relative change trends between the matrices are consistent. Based on this, the above-mentioned comparison of the preprocessed multiple optical flow spatial motion matrices and the preprocessed multiple sensor spatial motion matrices is performed to obtain the corresponding optical flow pose comparison. The results include:

For two adjacent specified time nodes within a preset time period, subtract the preprocessed optical flow spatial motion matrix corresponding to the two adjacent specified time nodes from the preprocessed sensor spatial motion matrix. , obtain the corresponding optical flow pose comparison results;

or,

Compare the first changing trend of the preprocessed multiple optical flow spatial motion matrices with the second changing trend of the preprocessed multiple sensor spatial motion matrices to obtain the corresponding optical flow pose comparison result;

The first change trend and the second change trend can be represented by waveform curves. The upper and lower amplitudes of the first waveform curve corresponding to the first change trend represent the changing direction of the image optical flow, and the upper and lower amplitudes of the second waveform curve corresponding to the second change trend represent The amplitude represents the spatial movement direction of the sensor. Since the changing direction of the image optical flow and the spatial movement direction of the sensor are both caused by the same jitter of the target user terminal, therefore, if the waveforms of the first waveform curve and the second waveform curve change in amplitude up and down If there is consistency, it is determined that the target video stream is the trusted video stream information collected by the target user terminal in real time; in addition, considering that there may be individual abnormal points or external noise, the first waveform curve and the second waveform curve can be first Perform smoothing filtering, and then compare whether the upper and lower amplitudes of the smoothed filtered first waveform curve and the second waveform curve are consistent. If the changes in the upper and lower amplitudes of the waveforms are consistent, it is determined that the target video stream is collected in real time by the target user terminal. Trusted video streaming information.

The video stream identification method in one or more embodiments of this specification determines the optical flow change information corresponding to the target video stream, wherein the theoretical collection time period of the target video stream is a preset time period and the theoretical collection device is the target user terminal. ; and, determine the pose change information of the target user terminal, where the pose change information is determined based on the sensor detection information of the target user terminal within a preset time period; based on the determined optical flow change information and pose change Compare the information between the information to determine the credibility identification result for the target video stream. By comparing and analyzing the optical flow change information determined based on the target video stream and the pose change information of the target user terminal, it is possible to identify whether the target video stream is video stream information that is captured and uploaded in real time for the target object, thereby achieving Quickly identify the situation of a malicious video stream attack that uses a pre-stored non-real-time captured video stream to replace the real-time captured video stream, so as to promptly intercept the non-real-time captured video stream injected by the malicious video stream attack and improve the accuracy of subsequent business processing. .

Corresponding to the video stream identification method described in Figures 1 to 7 above, based on the same technical concept, one or more embodiments of this specification also provide a video stream identification device. Figure 8 provides one or more embodiments of this specification. Schematic diagram of the module composition of a video stream identification device. The device is used to execute the video stream identification method described in Figures 1 to 7. As shown in Figure 8, the device includes:

The optical flow change information determination module 801 determines the optical flow change information corresponding to the target video stream, wherein the theoretical collection time period of the target video stream is a preset time period and the theoretical collection device is the target user terminal; and,

The pose change information determination module 802 determines the pose change information of the target user terminal, wherein the pose change information is determined based on the sensor detection information of the target user terminal within the preset time period. ;

The video stream credibility identification module 803 determines the credibility identification result for the target video stream based on the comparison information between the optical flow change information and the pose change information.

In one or more embodiments of this specification, by comparing and analyzing the optical flow change information determined based on the target video stream and the pose change information of the target user terminal, whether the target video stream is real-time for the target object is identified. Captured and uploaded video stream information to quickly identify malicious video stream attacks that use pre-stored non-real-time captured video streams to replace real-time captured video streams, so that the non-real-time captured video streams injected into the malicious video stream attacks can be promptly identified Interception is performed to improve the accuracy of subsequent business processing.

Optionally, the preset time period includes multiple designated time nodes; the optical flow change information determination module 801 is:

Obtain a target video stream to be identified, wherein the target video stream includes: a plurality of object image frames containing the target object corresponding to the plurality of designated time nodes respectively;

Optical flow change information corresponding to the target video stream is determined according to the object image frames at the plurality of designated time nodes.

Optionally, the optical flow change information determination module 801:

Determine the two object image frames corresponding to the two adjacent designated time nodes as an image frame combination;

For each of the image frame combinations, use a preset optical flow method to identify the image optical flow information for the image frame combination, and obtain the corresponding optical flow spatial motion matrix;

According to the optical flow spatial motion matrix corresponding to each of the image frame combinations, the optical flow change information corresponding to the target video stream is determined.

Optionally, the optical flow change information determination module 801:

For each of the optical flow spatial motion matrices, perform a coordinate system transformation on the optical flow spatial motion matrix under a preset spatial coordinate system to obtain a transformed optical flow spatial motion matrix;

According to each of the transformed optical flow spatial motion matrices, the optical flow change information corresponding to the target video stream is determined.

Optionally, the preset time period includes multiple designated time nodes; the pose change information determination module 802 is:

Obtain sensor detection information collected by at least one preset sensor of the target user terminal at each specified time node;

According to the sensor detection information at the plurality of designated time nodes, the pose change information of the target user terminal is determined.

Optionally, the pose change information determination module 802:

Determine the corresponding sensor spatial motion matrix according to the sensor detection information corresponding to the two adjacent designated time nodes;

According to the spatial motion matrix of each sensor, the pose change information of the target user terminal is determined.

Optionally, the pose change information determination module 802:

For each of the sensor spatial motion matrices, perform a coordinate system transformation on the sensor spatial motion matrix under a preset spatial coordinate system to obtain a transformed sensor spatial motion matrix;

According to each of the transformed sensor spatial motion matrices, the pose change information of the target user terminal is determined.

Optionally, the video stream credibility identification module 803:

Compare the optical flow change information with the pose change information to obtain the corresponding optical flow pose comparison result;

Determine whether the optical flow pose comparison result meets the preset change consistency condition;

If the judgment result is yes, it is determined that the target video stream is credible video stream information collected in real time;

If the judgment result is no, it is determined that the target video stream is untrusted video stream information collected in non-real time.

Optionally, the optical flow change information includes: multiple optical flow spatial motion matrices within the preset time period, and the pose change information includes: multiple sensor spatial motion matrices within the preset time period. ;

The video stream credibility identification module 803, which:

The plurality of optical flow spatial motion matrices are compared with the plurality of sensor spatial motion matrices to obtain corresponding optical flow pose comparison results.

Optionally, the video stream credibility identification module 803:

The plurality of optical flow spatial motion matrices and the plurality of sensor spatial motion matrices are preprocessed to obtain a plurality of preprocessed optical flow spatial motion matrices and a plurality of preprocessed sensor spatial motion matrices, wherein: The above-mentioned preprocessing includes: at least one of smoothing filtering, de-drying processing, and comparison starting point alignment;

The preprocessed optical flow spatial motion matrices are compared with the preprocessed sensor spatial motion matrices to obtain corresponding optical flow pose comparison results.

Optionally, the video stream credibility identification module 803:

For two adjacent designated time nodes within the preset time period, the preprocessed optical flow space motion matrix corresponding to the two adjacent designated time nodes is combined with the preprocessed sensor space The motion matrix is subtracted to obtain the corresponding optical flow pose comparison result;

or,

Compare the preprocessed first change trend of the plurality of optical flow spatial motion matrices with the preprocessed second change trend of the plurality of sensor spatial motion matrices to obtain a corresponding optical flow pose comparison. result.

The video stream identification device in one or more embodiments of this specification determines the optical flow change information corresponding to the target video stream, wherein the theoretical collection time period of the target video stream is a preset time period and the theoretical collection device is the target user terminal. ; and, determine the pose change information of the target user terminal, where the pose change information is determined based on the sensor detection information of the target user terminal within a preset time period; based on the determined optical flow change information and pose change Compare the information between the information to determine the credibility identification result for the target video stream. By comparing and analyzing the optical flow change information determined based on the target video stream and the pose change information of the target user terminal, it is possible to identify whether the target video stream is video stream information that is captured and uploaded in real time for the target object, thereby achieving Quickly identify the situation of a malicious video stream attack that uses a pre-stored non-real-time captured video stream to replace the real-time captured video stream, so as to promptly intercept the non-real-time captured video stream injected by the malicious video stream attack and improve the accuracy of subsequent business processing. .

It should be noted that the embodiments of the video stream identification device in this specification and the embodiments of the video stream identification method in this specification are based on the same inventive concept. Therefore, for the specific implementation of this embodiment, please refer to the corresponding video stream identification method mentioned above. Implementation, the repetitive parts will not be repeated.

Further, corresponding to the above-mentioned methods shown in Figures 1 to 7, based on the same technical concept, one or more embodiments of this specification also provide a video stream recognition device, which is used to perform the above-mentioned video stream recognition method. , as shown in Figure 9.

The video stream recognition device may vary greatly due to different configurations or performance, and may include one or more processors 901 and memory 902, and the memory 902 may store one or more storage applications or data. Among them, the memory 902 may be short-term storage or persistent storage. The application program stored in the memory 902 may include one or more modules (not shown), and each module may include a series of computer-executable instructions in the video stream recognition device. Furthermore, the processor 901 may be configured to communicate with the memory 902 and execute a series of computer-executable instructions in the memory 902 on the video stream recognition device. The video stream identification device may also include one or more power supplies 903, one or more wired or wireless network interfaces 904, one or more input and output interfaces 905, one or more keyboards 906, etc.

In a specific embodiment, the video stream recognition device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and Each module may include a series of computer-executable instructions in a video stream identification device, and the one or more programs configured to be executed by one or more processors may include computer-executable instructions for:

Determine the optical flow change information corresponding to the target video stream, wherein the theoretical collection time period of the target video stream is a preset time period and the theoretical collection device is the target user terminal; and,

Determine the pose change information of the target user terminal, wherein the pose change information is determined based on sensor detection information of the target user terminal within the preset time period;

Based on the comparison information between the optical flow change information and the pose change information, a credibility identification result for the target video stream is determined.

In one or more embodiments of this specification, by comparing and analyzing the optical flow change information determined based on the target video stream and the pose change information of the target user terminal, whether the target video stream is real-time for the target object is identified. Captured and uploaded video stream information to quickly identify malicious video stream attacks that use pre-stored non-real-time captured video streams to replace real-time captured video streams, so that the non-real-time captured video streams injected into the malicious video stream attacks can be promptly identified Interception is performed to improve the accuracy of subsequent business processing.

Optionally, when the computer-executable instructions are executed, the preset time period includes a plurality of designated time nodes;

Determining the optical flow change information corresponding to the target video stream includes:

Obtain a target video stream to be identified, wherein the target video stream includes: a plurality of object image frames containing the target object corresponding to the plurality of designated time nodes respectively;

Optical flow change information corresponding to the target video stream is determined according to the object image frames at the plurality of designated time nodes.

Optionally, when the computer-executable instructions are executed, the optical flow change information corresponding to the target video stream is determined based on the object image frames at the plurality of specified time nodes, including:

Determine the two object image frames corresponding to the two adjacent designated time nodes as an image frame combination;

For each of the image frame combinations, use a preset optical flow method to identify the image optical flow information for the image frame combination, and obtain the corresponding optical flow spatial motion matrix;

According to the optical flow spatial motion matrix corresponding to each of the image frame combinations, the optical flow change information corresponding to the target video stream is determined.

Optionally, when the computer-executable instructions are executed, the optical flow change information corresponding to the target video stream is determined based on the optical flow spatial motion matrix corresponding to each of the image frame combinations, including:

For each of the optical flow spatial motion matrices, perform a coordinate system transformation on the optical flow spatial motion matrix under a preset spatial coordinate system to obtain a transformed optical flow spatial motion matrix;

According to each of the transformed optical flow spatial motion matrices, the optical flow change information corresponding to the target video stream is determined.

Optionally, when the computer-executable instructions are executed, the preset time period includes a plurality of designated time nodes;

Determining the pose change information of the target user terminal includes:

Obtain sensor detection information collected by at least one preset sensor of the target user terminal at each specified time node;

According to the sensor detection information at the plurality of designated time nodes, the pose change information of the target user terminal is determined.

Optionally, when the computer-executable instructions are executed, determining the pose change information of the target user terminal based on the sensor detection information at the plurality of designated time nodes includes:

Determine the corresponding sensor spatial motion matrix according to the sensor detection information corresponding to the two adjacent designated time nodes;

According to the spatial motion matrix of each sensor, the pose change information of the target user terminal is determined.

Optionally, when the computer-executable instructions are executed, determining the pose change information of the target user terminal according to each of the sensor spatial motion matrices includes:

For each of the sensor spatial motion matrices, perform a coordinate system transformation on the sensor spatial motion matrix under a preset spatial coordinate system to obtain a transformed sensor spatial motion matrix;

According to each of the transformed sensor spatial motion matrices, the pose change information of the target user terminal is determined.

Optionally, when the computer-executable instructions are executed, the credibility identification result for the target video stream is determined based on the comparison information between the optical flow change information and the pose change information, include:

Compare the optical flow change information with the pose change information to obtain the corresponding optical flow pose comparison result;

Determine whether the optical flow pose comparison result meets the preset change consistency condition;

If the judgment result is yes, it is determined that the target video stream is credible video stream information collected in real time;

If the judgment result is no, it is determined that the target video stream is untrusted video stream information collected in non-real time.

Optionally, when the computer-executable instructions are executed, the optical flow change information includes: a plurality of optical flow spatial motion matrices within the preset time period, and the pose change information includes: the preset time Multiple sensor spatial motion matrices within the segment;

Comparing the optical flow change information with the pose change information to obtain a corresponding optical flow pose comparison result includes:

The plurality of optical flow spatial motion matrices are compared with the plurality of sensor spatial motion matrices to obtain corresponding optical flow pose comparison results.

Optionally, when the computer-executable instructions are executed, the plurality of optical flow spatial motion matrices are compared with the plurality of sensor spatial motion matrices to obtain corresponding optical flow pose comparison results, including :

The plurality of optical flow spatial motion matrices and the plurality of sensor spatial motion matrices are preprocessed to obtain a plurality of preprocessed optical flow spatial motion matrices and a plurality of preprocessed sensor spatial motion matrices, wherein: The above-mentioned preprocessing includes: at least one of smoothing filtering, de-drying processing, and comparison starting point alignment;

The preprocessed optical flow spatial motion matrices are compared with the preprocessed sensor spatial motion matrices to obtain corresponding optical flow pose comparison results.

Optionally, when the computer-executable instructions are executed, the preprocessed plurality of optical flow spatial motion matrices are compared with the preprocessed plurality of sensor spatial motion matrices to obtain corresponding light flow matrices. Flow pose comparison results include:

For two adjacent designated time nodes within the preset time period, the preprocessed optical flow space motion matrix corresponding to the two adjacent designated time nodes is combined with the preprocessed sensor space The motion matrix is subtracted to obtain the corresponding optical flow pose comparison result;

or,

Compare the preprocessed first change trend of the plurality of optical flow spatial motion matrices with the preprocessed second change trend of the plurality of sensor spatial motion matrices to obtain a corresponding optical flow pose comparison. result.

The video stream identification device in one or more embodiments of this specification determines the optical flow change information corresponding to the target video stream, wherein the theoretical collection time period of the target video stream is a preset time period and the theoretical collection device is the target user terminal. ; and, determine the pose change information of the target user terminal, where the pose change information is determined based on the sensor detection information of the target user terminal within a preset time period; based on the determined optical flow change information and pose change Compare the information between the information to determine the credibility identification result for the target video stream. By comparing and analyzing the optical flow change information determined based on the target video stream and the pose change information of the target user terminal, it is possible to identify whether the target video stream is video stream information that is captured and uploaded in real time for the target object, thereby achieving Quickly identify the situation of a malicious video stream attack that uses a pre-stored non-real-time captured video stream to replace the real-time captured video stream, so as to promptly intercept the non-real-time captured video stream injected by the malicious video stream attack and improve the accuracy of subsequent business processing. .

It should be noted that the embodiment of the video stream identification device in this specification and the embodiment of the video stream identification method in this specification are based on the same inventive concept. Therefore, for the specific implementation of this embodiment, please refer to the corresponding video stream identification method mentioned above. Implementation, the repetitive parts will not be repeated.

Further, corresponding to the methods shown in Figures 1 to 7 above, based on the same technical concept, one or more embodiments of this specification also provide a storage medium for storing computer executable instructions. A specific implementation For example, the storage medium can be a U disk, optical disk, hard disk, etc. When the computer executable instructions stored in the storage medium are executed by the processor, the following process can be achieved:

Determine the optical flow change information corresponding to the target video stream, wherein the theoretical collection time period of the target video stream is a preset time period and the theoretical collection device is the target user terminal; and,

Determine the pose change information of the target user terminal, wherein the pose change information is determined based on sensor detection information of the target user terminal within the preset time period;

Based on the comparison information between the optical flow change information and the pose change information, a credibility identification result for the target video stream is determined.

In one or more embodiments of this specification, by comparing and analyzing the optical flow change information determined based on the target video stream and the pose change information of the target user terminal, whether the target video stream is real-time for the target object is identified. Captured and uploaded video stream information to quickly identify malicious video stream attacks that use pre-stored non-real-time captured video streams to replace real-time captured video streams, so that the non-real-time captured video streams injected into the malicious video stream attacks can be promptly identified Interception is performed to improve the accuracy of subsequent business processing.

Optionally, when the computer-executable instructions stored in the storage medium are executed by the processor, the preset time period includes a plurality of designated time nodes;

Determining the optical flow change information corresponding to the target video stream includes:

Obtain a target video stream to be identified, wherein the target video stream includes: a plurality of object image frames containing the target object corresponding to the plurality of designated time nodes respectively;

Optical flow change information corresponding to the target video stream is determined according to the object image frames at the plurality of designated time nodes.

Optionally, when the computer executable instructions stored in the storage medium are executed by the processor, the optical flow change information corresponding to the target video stream is determined based on the object image frames at the multiple designated time nodes. ,include:

Determine the two object image frames corresponding to the two adjacent designated time nodes as an image frame combination;

For each of the image frame combinations, use a preset optical flow method to identify the image optical flow information for the image frame combination, and obtain the corresponding optical flow spatial motion matrix;

According to the optical flow spatial motion matrix corresponding to each of the image frame combinations, the optical flow change information corresponding to the target video stream is determined.

Optionally, when the computer-executable instructions stored in the storage medium are executed by the processor, the optical flow corresponding to the target video stream is determined based on the optical flow spatial motion matrix corresponding to each image frame combination. Change information, including:

For each of the optical flow spatial motion matrices, perform a coordinate system transformation on the optical flow spatial motion matrix under a preset spatial coordinate system to obtain a transformed optical flow spatial motion matrix;

According to each of the transformed optical flow spatial motion matrices, the optical flow change information corresponding to the target video stream is determined.

Optionally, when the computer-executable instructions stored in the storage medium are executed by the processor, the preset time period includes a plurality of designated time nodes;

Determining the pose change information of the target user terminal includes:

Obtain sensor detection information collected by at least one preset sensor of the target user terminal at each specified time node;

According to the sensor detection information at the plurality of designated time nodes, the pose change information of the target user terminal is determined.

Optionally, when the computer executable instructions stored in the storage medium are executed by the processor, the pose change information of the target user terminal is determined based on the sensor detection information at the multiple designated time nodes, include:

Determine the corresponding sensor spatial motion matrix according to the sensor detection information corresponding to the two adjacent designated time nodes;

According to the spatial motion matrix of each sensor, the pose change information of the target user terminal is determined.

Optionally, when the computer-executable instructions stored in the storage medium are executed by the processor, determining the pose change information of the target user terminal according to each of the sensor spatial motion matrices includes:

For each of the sensor spatial motion matrices, perform a coordinate system transformation on the sensor spatial motion matrix under a preset spatial coordinate system to obtain a transformed sensor spatial motion matrix;

According to each of the transformed sensor spatial motion matrices, the pose change information of the target user terminal is determined.

Optionally, when the computer-executable instructions stored in the storage medium are executed by the processor, the target video stream is determined based on the comparison information between the optical flow change information and the pose change information. The credibility identification results include:

Compare the optical flow change information with the pose change information to obtain the corresponding optical flow pose comparison result;

Determine whether the optical flow pose comparison result meets the preset change consistency condition;

If the judgment result is yes, it is determined that the target video stream is credible video stream information collected in real time;

If the judgment result is no, it is determined that the target video stream is untrusted video stream information collected in non-real time.

Optionally, when the computer executable instructions stored in the storage medium are executed by the processor, the optical flow change information includes: multiple optical flow spatial motion matrices within the preset time period, the pose change information It includes: multiple sensor spatial motion matrices within the preset time period;

Comparing the optical flow change information with the pose change information to obtain a corresponding optical flow pose comparison result includes:

The plurality of optical flow spatial motion matrices are compared with the plurality of sensor spatial motion matrices to obtain corresponding optical flow pose comparison results.

Optionally, when the computer-executable instructions stored in the storage medium are executed by the processor, the plurality of optical flow spatial motion matrices are compared with the plurality of sensor spatial motion matrices to obtain the corresponding optical flow. The pose comparison results include:

The plurality of optical flow spatial motion matrices and the plurality of sensor spatial motion matrices are preprocessed to obtain a plurality of preprocessed optical flow spatial motion matrices and a plurality of preprocessed sensor spatial motion matrices, wherein: The above-mentioned preprocessing includes: at least one of smoothing filtering, de-drying processing, and comparison starting point alignment;

The preprocessed optical flow spatial motion matrices are compared with the preprocessed sensor spatial motion matrices to obtain corresponding optical flow pose comparison results.

Optionally, when the computer-executable instructions stored in the storage medium are executed by the processor, the preprocessed plurality of optical flow spatial motion matrices and the preprocessed plurality of sensor spatial motion matrices are processed. Compare and obtain the corresponding optical flow pose comparison results, including:

For two adjacent designated time nodes within the preset time period, the preprocessed optical flow space motion matrix corresponding to the two adjacent designated time nodes is combined with the preprocessed sensor space The motion matrix is subtracted to obtain the corresponding optical flow pose comparison result;

or,

Compare the preprocessed first change trend of the plurality of optical flow spatial motion matrices with the preprocessed second change trend of the plurality of sensor spatial motion matrices to obtain a corresponding optical flow pose comparison. result.

When executed by the processor, the computer-executable instructions stored in the storage medium in one or more embodiments of this specification determine the optical flow change information corresponding to the target video stream, wherein the theoretical collection time period of the target video stream is a preset time period and the theoretical collection device is the target user terminal; and, determine the pose change information of the target user terminal, wherein the pose change information is determined based on the sensor detection information of the target user terminal within the preset time period; according to the determination The comparison information between the optical flow change information and pose change information is used to determine the credibility recognition result for the target video stream. By comparing and analyzing the optical flow change information determined based on the target video stream and the pose change information of the target user terminal, it is possible to identify whether the target video stream is video stream information that is captured and uploaded in real time for the target object, thereby achieving Quickly identify the situation of a malicious video stream attack that uses a pre-stored non-real-time captured video stream to replace the real-time captured video stream, so as to promptly intercept the non-real-time captured video stream injected by the malicious video stream attack and improve the accuracy of subsequent business processing. .

It should be noted that the embodiment about the storage medium in this specification and the embodiment about the video stream identification method in this specification are based on the same inventive concept. Therefore, for the specific implementation of this embodiment, please refer to the implementation of the corresponding video stream identification method mentioned above. The repetitive parts will not be repeated.

The foregoing describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desired results. Additionally, the processes depicted in the figures do not necessarily require the specific order shown, or sequential order, to achieve desirable results. Multitasking and parallel processing are also possible or may be advantageous in certain implementations.

In the 1990s, improvements in a technology could be clearly distinguished as hardware improvements (for example, improvements in circuit structures such as diodes, transistors, switches, etc.) or software improvements (improvements in method processes). However, with the development of technology, many improvements in today's method processes can be regarded as direct improvements in hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement of a method flow cannot be implemented using hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit, and its logic function is determined by the user programming the device. Designers can program themselves to "integrate" a digital system on a PLD, instead of asking chip manufacturers to design and produce dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented using "logic compiler" software, which is similar to the software compiler used in program development and writing. Before compiling, The original code must also be written in a specific programming language, which is called Hardware Description Language (HDL). There is not only one type of HDL, but many types, such as ABEL (Advanced Boolean Expression Language) , AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HD Cal, JHDL (Java Hardware Description Language), Lava, Lola, My HDL, PALASM, RHDL (Ruby Hardware Description Language), etc., are currently the most commonly used The most popular ones are VHDL (Very-High-SpeedIntegrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that by simply logically programming the method flow using the above-mentioned hardware description languages and programming it into the integrated circuit, the hardware circuit that implements the logical method flow can be easily obtained.

The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (eg, software or firmware) executable by the (micro)processor. , logic gates, switches, Application Specific Integrated Circuit (ASIC), programmable logic controllers and embedded microcontrollers. Examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the memory control logic. Those skilled in the art also know that in addition to implementing the controller in the form of pure computer-readable program code, the controller can be completely programmed with logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded logic by logically programming the method steps. Microcontroller, etc. to achieve the same function. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the means for implementing various functions can be considered as structures within hardware components as well as software modules implementing the methods.

The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or A combination of any of these devices.

For the convenience of description, when describing the above device, the functions are divided into various units and described separately. Of course, when implementing one or more aspects of this specification, the functions of each unit can be implemented in the same or multiple pieces of software and/or hardware.

It should be understood by those skilled in the art that one or more embodiments of this specification may be provided as a method, system, or computer program product. Thus, one or more of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware aspects. Furthermore, one or more of the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk memory, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein. form.

One or more aspects of this specification are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to one or more embodiments of the specification. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce a use A device for realizing the functions specified in one process or multiple processes of the flowchart and/or one block or multiple blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction means, the instructions The device implements the functions specified in a process or processes of the flowchart and/or a block or blocks of the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device, causing a series of operating steps to be performed on the computer or other programmable device to produce computer-implemented processing, thereby executing on the computer or other programmable device. Instructions provide steps for implementing the functions specified in a process or processes of a flowchart diagram and/or a block or blocks of a block diagram.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include non-permanent storage in computer-readable media, random access memory (RAM), and/or non-volatile memory in the form of read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

Computer-readable media includes both persistent and non-volatile, removable and non-removable media that can be implemented by any method or technology for storage of information. Information may be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), and read-only memory. (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, Magnetic tape cassettes, tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium can be used to store information that can be accessed by a computing device. As defined in this article, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.

It should also be noted that the terms "comprises," "comprises," or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements not only includes those elements, but also includes Other elements are not expressly listed or are inherent to the process, method, article or equipment. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article, or device that includes the stated element.

It will be appreciated by those skilled in the art that one or more embodiments of this specification may be provided as a method, system, or computer program product. Thus, one or more of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware aspects. Furthermore, one or more of the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk memory, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein. form.

One or more of this specification may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more of this specification may also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including storage devices.

Each embodiment in this specification is described in a progressive manner. The same and similar parts between the various embodiments can be referred to each other. Each embodiment focuses on its differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For relevant details, please refer to the partial description of the method embodiment.

The above descriptions are only one or more embodiments of this specification, and are not intended to limit one or more of this specification. Various modifications and variations may occur to one or more of this description to those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more of this specification shall be included in the scope of one or more of the claims of this specification.

Claims (24)

1. A video stream identification method, including: Determine the optical flow change information corresponding to the target video stream, wherein the optical flow change information is obtained by analyzing the optical flow direction based on multiple object image frames in the target video stream; and, Determine the pose change information of the target user terminal, wherein the pose change information is determined based on the sensor spatial motion information corresponding to the sensor detection information of the target user terminal at multiple designated time nodes within a preset time period. ; According to the comparison information between the optical flow change information and the pose change information, it is determined whether the target video stream is a video stream collected by the target user terminal in real time within the preset time period. 2. The method according to claim 1, wherein determining the optical flow change information corresponding to the target video stream includes: Obtain a target video stream to be identified, wherein the target video stream includes: a plurality of object image frames containing the target object corresponding to the plurality of designated time nodes respectively; Optical flow change information corresponding to the target video stream is determined according to the object image frames at the plurality of designated time nodes. 3. The method according to claim 2, wherein determining the optical flow change information corresponding to the target video stream according to the object image frames at the plurality of designated time nodes includes: Determine the two object image frames corresponding to the two adjacent designated time nodes as an image frame combination; For each of the image frame combinations, use a preset optical flow method to identify the image optical flow information for the image frame combination, and obtain the corresponding optical flow spatial motion matrix; According to the optical flow spatial motion matrix corresponding to each of the image frame combinations, the optical flow change information corresponding to the target video stream is determined. 4. The method according to claim 3, wherein determining the optical flow change information corresponding to the target video stream according to the optical flow spatial motion matrix corresponding to each image frame combination includes: For each of the optical flow spatial motion matrices, perform a coordinate system transformation on the optical flow spatial motion matrix under a preset spatial coordinate system to obtain a transformed optical flow spatial motion matrix; According to each of the transformed optical flow spatial motion matrices, the optical flow change information corresponding to the target video stream is determined. 5. The method according to claim 1, wherein determining the pose change information of the target user terminal includes: Obtain sensor detection information collected by at least one preset sensor of the target user terminal at each specified time node; According to the sensor detection information at the plurality of designated time nodes, the pose change information of the target user terminal is determined. 6. The method according to claim 5, wherein determining the pose change information of the target user terminal based on the sensor detection information at the plurality of designated time nodes includes: Determine the corresponding sensor spatial motion matrix according to the sensor detection information corresponding to the two adjacent designated time nodes; According to the spatial motion matrix of each sensor, the pose change information of the target user terminal is determined. 7. The method according to claim 6, wherein determining the pose change information of the target user terminal according to each of the sensor spatial motion matrices includes: For each of the sensor spatial motion matrices, perform a coordinate system transformation on the sensor spatial motion matrix under a preset spatial coordinate system to obtain a transformed sensor spatial motion matrix; According to each of the transformed sensor spatial motion matrices, the pose change information of the target user terminal is determined. 8. The method according to claim 1, wherein, based on the comparison information between the optical flow change information and the pose change information, it is determined whether the target video stream is the target user terminal in The video streams collected in real time within the preset time period include: Compare the optical flow change information with the pose change information to obtain the corresponding optical flow pose comparison result; Determine whether the optical flow pose comparison result meets the preset change consistency condition; If the judgment result is yes, it is determined that the target video stream is the trusted video stream information collected by the target user terminal in real time within the preset time period; If the judgment result is no, it is determined that the target video stream is untrusted video stream information collected by the target user terminal in non-real time within the preset time period. 9. The method according to claim 8, wherein the optical flow change information includes: a plurality of optical flow spatial motion matrices within the preset time period, and the pose change information includes: the preset time Multiple sensor spatial motion matrices within the segment; Comparing the optical flow change information with the pose change information to obtain a corresponding optical flow pose comparison result includes: The plurality of optical flow spatial motion matrices are compared with the plurality of sensor spatial motion matrices to obtain corresponding optical flow pose comparison results. 10. The method according to claim 9, wherein comparing the plurality of optical flow spatial motion matrices with the plurality of sensor spatial motion matrices to obtain corresponding optical flow pose comparison results includes: : The plurality of optical flow spatial motion matrices and the plurality of sensor spatial motion matrices are preprocessed to obtain a plurality of preprocessed optical flow spatial motion matrices and a plurality of preprocessed sensor spatial motion matrices, wherein: The above-mentioned preprocessing includes: at least one of smoothing filtering, de-drying processing, and comparison starting point alignment; The preprocessed optical flow spatial motion matrices are compared with the preprocessed sensor spatial motion matrices to obtain corresponding optical flow pose comparison results. 11. The method according to claim 10, wherein the preprocessed optical flow spatial motion matrices are compared with the preprocessed sensor spatial motion matrices to obtain corresponding optical flow spatial motion matrices. Flow pose comparison results include: For two adjacent designated time nodes within the preset time period, the preprocessed optical flow space motion matrix corresponding to the two adjacent designated time nodes is combined with the preprocessed sensor space The motion matrix is subtracted to obtain the corresponding optical flow pose comparison result; or, Compare the preprocessed first change trend of the plurality of optical flow spatial motion matrices with the preprocessed second change trend of the plurality of sensor spatial motion matrices to obtain a corresponding optical flow pose comparison. result. 12. A video stream identification device, comprising: The optical flow change information determination module determines the optical flow change information corresponding to the target video stream, wherein the optical flow change information is obtained by analyzing the optical flow direction based on multiple object image frames in the target video stream; and, A pose change information determination module determines the pose change information of the target user terminal, wherein the pose change information is based on sensor detection information of the target user terminal at multiple designated time nodes within a preset time period, respectively. The corresponding sensor spatial motion information is determined; A video stream credibility identification module, which determines whether the target video stream is the target user terminal in the preset time period based on the comparison information between the optical flow change information and the pose change information. Video stream captured in real time. 13. The device according to claim 12, wherein the optical flow change information determination module: Obtain a target video stream to be identified, wherein the target video stream includes: a plurality of object image frames containing the target object corresponding to the plurality of designated time nodes respectively; Optical flow change information corresponding to the target video stream is determined according to the object image frames at the plurality of designated time nodes. 14. The device according to claim 13, wherein the optical flow change information determination module: Determine the two object image frames corresponding to the two adjacent designated time nodes as an image frame combination; For each of the image frame combinations, use a preset optical flow method to identify the image optical flow information for the image frame combination, and obtain the corresponding optical flow spatial motion matrix; According to the optical flow spatial motion matrix corresponding to each of the image frame combinations, the optical flow change information corresponding to the target video stream is determined. 15. The device according to claim 14, wherein the optical flow change information determination module: For each of the optical flow spatial motion matrices, perform a coordinate system transformation on the optical flow spatial motion matrix under a preset spatial coordinate system to obtain a transformed optical flow spatial motion matrix; According to each of the transformed optical flow spatial motion matrices, the optical flow change information corresponding to the target video stream is determined. 16. The device according to claim 12, wherein the pose change information determination module: Obtain sensor detection information collected by at least one preset sensor of the target user terminal at each specified time node; According to the sensor detection information at the plurality of designated time nodes, the pose change information of the target user terminal is determined. 17. The device according to claim 16, wherein the pose change information determination module: Determine the corresponding sensor spatial motion matrix according to the sensor detection information corresponding to the two adjacent designated time nodes; According to the spatial motion matrix of each sensor, the pose change information of the target user terminal is determined. 18. The device according to claim 17, wherein the pose change information determination module: For each of the sensor spatial motion matrices, perform a coordinate system transformation on the sensor spatial motion matrix under a preset spatial coordinate system to obtain a transformed sensor spatial motion matrix; According to each of the transformed sensor spatial motion matrices, the pose change information of the target user terminal is determined. 19. The device according to claim 12, wherein the video stream credibility identification module: Compare the optical flow change information with the pose change information to obtain the corresponding optical flow pose comparison result; Determine whether the optical flow pose comparison result meets the preset change consistency condition; If the judgment result is yes, it is determined that the target video stream is the trusted video stream information collected by the target user terminal in real time within the preset time period; If the judgment result is no, it is determined that the target video stream is untrusted video stream information collected by the target user terminal in non-real time within the preset time period. 20. The device according to claim 19, wherein the optical flow change information includes: a plurality of optical flow spatial motion matrices within the preset time period, and the pose change information includes: the preset time Multiple sensor spatial motion matrices within the segment; The video stream credibility identification module, which: The plurality of optical flow spatial motion matrices are compared with the plurality of sensor spatial motion matrices to obtain corresponding optical flow pose comparison results. 21. The device according to claim 20, wherein the video stream credibility identification module: The plurality of optical flow spatial motion matrices and the plurality of sensor spatial motion matrices are preprocessed to obtain a plurality of preprocessed optical flow spatial motion matrices and a plurality of preprocessed sensor spatial motion matrices, wherein: The above-mentioned preprocessing includes: at least one of smoothing filtering, de-drying processing, and comparison starting point alignment; The preprocessed optical flow spatial motion matrices are compared with the preprocessed sensor spatial motion matrices to obtain corresponding optical flow pose comparison results. 22. The device according to claim 21, wherein the video stream credibility identification module: For two adjacent designated time nodes within the preset time period, the preprocessed optical flow space motion matrix corresponding to the two adjacent designated time nodes is combined with the preprocessed sensor space The motion matrix is subtracted to obtain the corresponding optical flow pose comparison result; or, Compare the preprocessed first change trend of the plurality of optical flow spatial motion matrices with the preprocessed second change trend of the plurality of sensor spatial motion matrices to obtain a corresponding optical flow pose comparison. result. 23. A video stream recognition device, including: processor; and Memory arranged to store computer-executable instructions which, when executed, cause the processor to: Determine the optical flow change information corresponding to the target video stream, wherein the optical flow change information is obtained by analyzing the optical flow direction based on multiple object image frames in the target video stream; and, Determine the pose change information of the target user terminal, wherein the pose change information is determined based on the sensor spatial motion information corresponding to the sensor detection information of the target user terminal at multiple designated time nodes within a preset time period. ; According to the comparison information between the optical flow change information and the pose change information, it is determined whether the target video stream is a video stream collected by the target user terminal in real time within the preset time period. 24. A storage medium for storing computer-executable instructions that, when executed by a processor, implement the following methods: Determine the optical flow change information corresponding to the target video stream, wherein the optical flow change information is obtained by analyzing the optical flow direction based on multiple object image frames in the target video stream; and, Determine the pose change information of the target user terminal, wherein the pose change information is determined based on the sensor spatial motion information corresponding to the sensor detection information of the target user terminal at multiple designated time nodes within a preset time period. ; According to the comparison information between the optical flow change information and the pose change information, it is determined whether the target video stream is a video stream collected by the target user terminal in real time within the preset time period.