CN118015712B - Taiji boxing scoring method, system, equipment and medium based on time window - Google Patents
Taiji boxing scoring method, system, equipment and medium based on time window Download PDFInfo
- Publication number
- CN118015712B CN118015712B CN202410424786.3A CN202410424786A CN118015712B CN 118015712 B CN118015712 B CN 118015712B CN 202410424786 A CN202410424786 A CN 202410424786A CN 118015712 B CN118015712 B CN 118015712B
- Authority
- CN
- China
- Prior art keywords
- model
- time window
- frame
- gesture
- gestures
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013077 scoring method Methods 0.000 title claims description 20
- 230000009471 action Effects 0.000 claims abstract description 56
- 239000011159 matrix material Substances 0.000 claims abstract description 51
- 238000000034 method Methods 0.000 claims abstract description 28
- 230000006870 function Effects 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 17
- 238000003860 storage Methods 0.000 claims description 12
- 238000011156 evaluation Methods 0.000 claims description 7
- 238000001514 detection method Methods 0.000 claims description 5
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 239000013598 vector Substances 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 8
- 238000012549 training Methods 0.000 description 8
- 238000002474 experimental method Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000010191 image analysis Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000009323 psychological health Effects 0.000 description 1
- 238000010223 real-time analysis Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
- G06Q50/205—Education administration or guidance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Tourism & Hospitality (AREA)
- Strategic Management (AREA)
- Educational Technology (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Educational Administration (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Human Resources & Organizations (AREA)
- General Business, Economics & Management (AREA)
- Economics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method, system equipment and medium for scoring a Taiji fist based on a time window, which comprises the steps of collecting Taiji fist action key points in a Taiji fist video, and defining the key points in each frame number in the video; taking the actions and gestures in the defined key points as a set of key data, constructing a mathematical matrix, and obtaining an identification model through the mathematical matrix; dividing actions and gestures in the mathematical matrix by using a time window according to the recognition model, dynamically adjusting the actions and gestures after dividing the time window by setting an adjusting coefficient, and outputting a model result; according to the method, actions and gestures in the mathematical matrix are divided by using the time window, and key frames are determined, so that accurate division of the actions is realized; by setting the adjustment coefficient, the motion and the gesture after the time window is divided are dynamically adjusted, and a model result is output, so that the robustness of the model is effectively improved, and the accuracy of the model in outputting the Taijiquan score is ensured.
Description
Technical Field
The invention relates to the technical field of Taiji fist scoring, in particular to a Taiji fist scoring method, a Taiji fist scoring system, taiji fist scoring equipment and a Taiji fist scoring medium based on a time window.
Background
Nowadays, the Taiji boxing is favored as a project in the sports choosing course of colleges and universities, and can influence the physical and psychological health of individuals because of the profound Chinese cultural tradition connotation.
It has been proposed to analyze and score the tai chi boxing action frame by frame in a video manner to evaluate the correctness of the action. In this way, training analysis using models is required, whereas deep learning model training in conventional methods requires a lot of computational resources and requires advanced definition of evaluation criteria and features, which limits flexibility in handling personalized and non-conventional tai chi fist style of performance.
Disclosure of Invention
This section is intended to outline some aspects of embodiments of the application and to briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section as well as in the description of the application and in the title of the application, which may not be used to limit the scope of the application.
The present invention has been made in view of the above-described problems occurring in the prior art. Therefore, the invention provides a Taiji boxing scoring method based on a time window, which utilizes the time window to divide actions and gestures in a mathematical matrix and determines key frames or characteristic points, thereby realizing accurate division of the actions; by setting the adjustment coefficient, the motion and the gesture after the time window is divided are dynamically adjusted, and a model result is output, so that the robustness of the model is effectively improved, and the accuracy of the model in outputting the Taijiquan score is ensured.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
In a first aspect, the present invention provides a taijiquan scoring method based on a time window, including:
Acquiring a Taiji boxing action key point in a Taiji boxing video, and defining the key point in each frame number in the video, wherein the key point comprises coordinate information and gesture marks;
Taking the coordinate information and the gesture mark in the defined key points as a set of key data, constructing a mathematical matrix, and obtaining an identification model through the mathematical matrix; four-dimensional information of each column corresponding to a key point;
Dividing actions and gestures in the mathematical matrix by utilizing a time window according to the recognition model, dynamically adjusting the actions and gestures divided by the time window by setting an adjusting coefficient, and outputting a model result;
and carrying out overall evaluation on the model according to the output result of the identification model to obtain a Taiji boxing scoring result.
As a preferred technical solution, collecting the key points of the taijiquan motion in the taijiquan video, and defining each frame number in the key points, including:
For each frame, a plurality of key points are contained, and each key point contains coordinate information and gesture marks;
setting N key points, wherein the information of each key point is expressed as four-dimensional vectors (x, y, t, g);
wherein x and y are coordinates of key points in the Taiji boxing video, t is a timestamp corresponding to the Taiji boxing video, and g is a gesture mark.
As a preferred technical solution, constructing a mathematical matrix by using the defined actions and gestures in the key points as a set of key data, and obtaining an identification model through the mathematical matrix, including:
Setting M frames to represent matrix as Constructing the mathematical matrix X, expressed as:
；
Wherein X represents a multidimensional matrix consisting of M frames and N key points, 、、、Gestures respectively representing the abscissa, the ordinate, the time and the mark of the 1 st key point of the 1 st frame;、、、 gestures respectively representing the abscissa, the ordinate, the time and the mark of the 2 nd key point of the 1 st frame; 、、、 Gestures respectively representing the abscissa, the ordinate, the time and the mark of the nth key point of the 1 st frame; 、、、 gestures respectively representing the abscissa, the ordinate, the time and the mark of the 1 st key point of the 2 nd frame; 、、、 Gestures respectively representing the abscissa, the ordinate, the time and the mark of the nth key point of the Mth frame;
inputting the X into a recognition model f to obtain:
；
Wherein, AndThe recognition results of the motion and the gesture, respectively.
As a preferable technical scheme, when the number of frames is input into the mathematical matrix X, it is necessary to delete consecutive frames whose number of key points detected does not reach a set threshold and image frames whose number of key points detected does not satisfy the set threshold, and adjust the timestamp t corresponding to the tai chi fist video.
As a preferred technical solution, according to the recognition model, dividing the actions and the gestures in the mathematical matrix by using a time window includes:
The time window sequence in the identification model is The gesture sequence is，
Wherein each ofRepresenting an ith time window in the video, eachRepresenting an mth gesture;
A start frame of the gesture sequence A start frame corresponding to the sequence of time windowsStarting from the next frame of the starting frames of the time window, wherein the current frame of the gesture sequence of each frame is the starting frame of the time window of the next frame until the current frame circulated to the gesture sequence is equal to the starting frame of the time window, so as to obtain the time window sequence。
As an preferable technical solution, dynamically adjusting the motion and gesture after the time window division by setting an adjustment coefficient, and outputting a model result, including:
setting double adjustment coefficients And；
By passing throughTime window sequenceAdjusting to obtain a change sequence corresponding to the gesture sequence；In the variation sequenceOn the basis of (1) adjusting the identification model to obtain:
；
Wherein, Indicating the passage at time point tThe adjusted time series value; the coefficients are adjusted for a sequence of time windows, For varying the sequenceDynamic coefficients of (2); the time period for which the model is identified is represented, A tai chi boxing score is output for the recognition model.
As a preferred technical scheme, the method further comprises:
Defining model loss function and introducing binary variable Judging the output result of the identification model; the loss function comprises an action loss function and a gesture loss function;
If the binary variable Greater thanMarking as 1, belonging to a trusted model; otherwise, the model is marked as 0, belongs to an untrusted model, returns to the position where the mathematical matrix is built, and reestablishes the model.
In a second aspect, the present invention provides a tai chi fist scoring system based on a time window, and a tai chi fist scoring method based on the time window, including:
The video acquisition module acquires the key points of the Taiji boxing action in the Taiji boxing video, and defines the key points in each frame number in the video, wherein the key points comprise coordinate information and gesture marks;
The model building module is used for building a mathematical matrix by taking the coordinate information and the gesture marks in the defined key points as a set of key data, and obtaining an identification model through the mathematical matrix; four-dimensional information of each column corresponding to a key point;
The model algorithm module divides actions and gestures in the mathematical matrix by utilizing a time window according to the recognition model, dynamically adjusts the actions and gestures divided by the time window by setting an adjustment coefficient, and outputs a model result;
And the model result scoring module is used for integrally evaluating the model according to the output result of the identification model to obtain a Taiji fist scoring result.
In a third aspect, the invention provides a computer device comprising a memory storing a computer program and a processor implementing the steps of the method when the processor executes the computer program.
In a fourth aspect, the present invention provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the method.
Compared with the prior art, the invention has the following advantages and beneficial effects:
According to the method, actions and gestures in the mathematical matrix are divided by using the time window, and key frames or characteristic points are determined, so that accurate division of the actions is realized; by setting the adjustment coefficient, the motion and the gesture after time window division are dynamically adjusted, and a model result is output, so that the robustness of the model is effectively improved, and the accuracy of the follow-up score of the model is ensured.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. Wherein:
FIG. 1 is a general flow chart of a Tai Jib scoring method based on a time window according to one embodiment of the invention;
fig. 2 is a system configuration diagram of a tai chi boxing scoring method based on a time window according to an embodiment of the present invention.
Detailed Description
So that the manner in which the above recited objects, features and advantages of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to the embodiments, some of which are illustrated in the appended drawings. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways other than those described herein, and persons skilled in the art will readily appreciate that the present invention is not limited to the specific embodiments disclosed below.
Further, reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic can be included in at least one implementation of the invention. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments.
While the embodiments of the present invention have been illustrated and described in detail in the drawings, the cross-sectional view of the device structure is not to scale in the general sense for ease of illustration, and the drawings are merely exemplary and should not be construed as limiting the scope of the invention. In addition, the three-dimensional dimensions of length, width and depth should be included in actual fabrication.
Also in the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper, lower, inner and outer", etc. are based on the orientation or positional relationship shown in the drawings, are merely for convenience of describing the present invention and simplifying the description, and do not indicate or imply that the apparatus or elements referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present invention. Furthermore, the terms "first, second, or third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
The terms "mounted, connected, and coupled" should be construed broadly in this disclosure unless otherwise specifically indicated and defined, such as: can be fixed connection, detachable connection or integral connection; it may also be a mechanical connection, an electrical connection, or a direct connection, or may be indirectly connected through an intermediate medium, or may be a communication between two elements. The specific meaning of the above terms in the present invention will be understood in specific cases by those of ordinary skill in the art.
Example 1
Referring to fig. 1, a tai chi boxing scoring method based on a time window is provided in a first embodiment of the present invention, including:
S1, acquiring the key points of the Taiji boxing action in the Taiji boxing video, and defining the key points in each frame number in the video.
Further, for each frame, a number of keypoints are contained, each of which contains coordinate information and gesture markers.
Further, N key points are set, and information of each key point is expressed as a four-dimensional vector (x, y, t, g).
Wherein x and y are coordinates of key points in the Taiji boxing video, t is a timestamp corresponding to the Taiji boxing video, and g is a gesture mark.
It should be noted that the extraction and definition of the keypoints helps to improve the accuracy of the tai chi boxing action.
S2, taking the actions and gestures in the defined key points as a set of key data, constructing a mathematical matrix, and obtaining an identification model through the mathematical matrix;
Further, taking the actions and gestures defined in the key points as a set of key data, constructing a mathematical matrix, and obtaining an identification model through the mathematical matrix, wherein the method comprises the following steps:
Further, M frames are set to represent a matrix as Each row corresponds to data of a time point, each column corresponds to four-dimensional information of a key point, and a mathematical matrix X is constructed and expressed as follows:
；
It will be appreciated that this three-dimensional data set is expanded into a two-dimensional matrix for ease of representation and processing in this embodiment. In this two-dimensional matrix, the information of all key points in the first frame is arranged in turn, then the second frame, and so on until the last frame. In particular the number of the elements, 、、、Gestures respectively representing an abscissa, an ordinate and a mark of a first key point of a first frame, which are continuously arranged in a two-dimensional matrix;、、、 corresponding information respectively representing second key points of the first frame is also continuously arranged in the matrix; and so on until 、、、Information representing the nth key point of the first frame. Next, key point information of the second frame、、、Starting to arrange until the information of the Nth key point of the last frame、、、. Thus, each row of matrix data contains coordinate information and gesture markers for all key points within a time point (i.e., a frame). In addition, the time for all keypoints within each frame is the same, i.e. associated with the frame itself. From this expanded two-dimensional matrix, the present embodiment can construct a recognition model.
Further, the input data X is passed to a shared feature extractor h, the output h (X) of which is used as an action recognition sub-networkAnd gesture recognition subnetworkIs input to the computer. Finally, the output of model f is:
；
Wherein each sub-network utilizes the feature h (X) provided by the shared feature extractor to generate the recognition result of the respective task, namely AndThe recognition results of the motion and the gesture, respectively.
It should be noted that, when the mathematical matrix X is constructed by inputting the number of frames, it is necessary to delete the continuous frames whose number of key points does not reach the set threshold and the image frames whose number of key points does not satisfy the set threshold, and adjust the time stamp t corresponding to the tai chi fist video.
Specifically, if the detection quantity of the key points is not satisfied, that is, the coordinates of x and y are absent, filling the corresponding absent points with 'NaN'.
S3, dividing actions and gestures in the mathematical matrix by using a time window according to the recognition model, dynamically adjusting the actions and gestures after the time window division by setting an adjustment coefficient, and outputting a model result;
further, the sequence of time windows in the recognition model is:
；
The gesture sequence is as follows:
；
wherein each of Representing an ith time window in the video, eachRepresenting the mth gesture.
Further, the start frame of the gesture sequenceStart frame corresponding to time window sequenceStarting from the next frame of the starting frame of the time window, wherein the current frame of the gesture sequence of each frame is the starting frame of the time window of the next frame until the current frame circulated to the gesture sequence is equal to the starting frame of the time window, thereby obtaining。
In particular, the method comprises the steps of,Expressed as:
；
It should be noted that it is also possible to provide, AndThe equal sign in (2) is an assignment operation.
It should be noted that the current frame of the gesture sequence is equal to the start frame of the time window, so that real-time analysis and feedback of the gesture can be realized, and application optimization of the algorithm can be realized.
Further, setting double adjustment coefficientsAnd；
Further, byTime window sequenceAdjusting to obtain a change sequence corresponding to the gesture sequence；
In particular, the method comprises the steps of,Expressed as:
；
In the variation sequence On the basis of (1) adjusting the identification model to obtain:
；
Wherein, Indicating the passage at time point tThe adjusted time series value; the coefficients are adjusted for a sequence of time windows, For varying the sequenceDynamic coefficients of (2); the time period for which the model is identified is represented, A tai chi boxing score is output for the recognition model.
It should be noted that, by setting the adjustment coefficient, the degree of flexibility of the model becomes high, which helps to improve the robustness of the model.
Further, defining model loss function and introducing binary variableJudging the model output result;
Further, it is provided with An output layer representing the action is provided,An output layer representing the gesture, the recognition model output being represented as:
；
specifically, the loss function is divided into an action loss function and a gesture loss function;
specifically, the action loss function Expressed as:
；
specifically, the gesture loss function Expressed as:
；
preferably, the action part is weighted Weights of gesture portionsObtaining the total loss valueExpressed as:
；
Further, the method comprises the steps of, AndThe value of (1) is True or False; Expressed as binary cross entropy.
It should be noted that the process of using the binary cross entropy as a loss function is to perform class processing in the case where the motion and gesture are not equal, and helps to explain the model output result.
Further, if the binary variableGreater thanMarking as 1, belonging to a trusted model; otherwise, the model is marked as 0, belongs to an untrusted model, returns to the position where the mathematical matrix is built, and reestablishes the model.
And S4, carrying out overall evaluation on the model according to the output result of the model to obtain a Taiji fist scoring result.
Based on the same ideas the time window-based Tai Jib scoring method in the above embodiment, the present invention also provides a time window-based Tai Jib scoring system, which can be used to execute the time window-based Tai Jib scoring method. For ease of illustration, only those portions of the exemplary embodiment of the tai chi fist scoring system are shown in the schematic block diagram based on the time window, and those skilled in the art will appreciate that the illustrated structure is not limiting of the apparatus and may include more or fewer components than illustrated, or may combine certain components, or a different arrangement of components.
Further, the embodiment also provides a taijiquan scoring system based on a time window, which comprises:
the video acquisition module acquires the key points of the Taiji boxing action in the Taiji boxing video and defines the key points in each frame number in the video;
the model building module is used for taking the actions and gestures in the defined key points as a set of key data, constructing a mathematical matrix and obtaining an identification model through the mathematical matrix;
The model algorithm module divides actions and gestures in the mathematical matrix by utilizing a time window according to the recognition model, dynamically adjusts the actions and gestures divided by the time window by setting an adjustment coefficient, and outputs a model result;
And the model result scoring module is used for integrally evaluating the model according to the output result of the identification model to obtain a Taiji fist scoring result.
The embodiment also provides a computer device, which is applicable to the case of the Taiji boxing scoring method based on a time window, and comprises the following steps:
A memory and a processor; the memory is used for storing computer executable instructions, and the processor is used for executing the computer executable instructions to realize the Taiji fist scoring method based on the time window as proposed by the embodiment.
The computer device may be a terminal comprising a processor, a memory, a communication interface, a display screen and input means connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
The present embodiment also provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the tai chi fist scoring method based on a time window as proposed in the above embodiments.
The storage medium according to the present embodiment belongs to the same inventive concept as the data storage method according to the above embodiment, and technical details not described in detail in the present embodiment can be seen in the above embodiment, and the present embodiment has the same advantageous effects as the above embodiment.
Example 2
Referring to tables 1 and 2, for a second embodiment of the present invention, there is provided a tai chi fist scoring method based on a time window, including: the method provided by the invention is used for verifying the accuracy of the Taiji boxing simulation in a simulation experiment mode;
Model-defined loss function Obtain the score of each action of the Taiji boxingEach simulation experiment included: 24 Taiji boxing actions, each of which has a scoring range of 1 to 100 points.
The experiments were divided into two independent groups: the first group was subjected to 10 experiments, labeled experiments 1 to 10, without the use of the present invention; in this group, the model discontinues the evaluation of subsequent actions when the score is below 5 points (excluding 5 points); refer to table 1;
Table 1 first set of action scores for 10 experiments
The second group carries out 10 experiments, namely experiments 11 to 20, and by adopting the method, the judgment of the follow-up action of the model in the whole scoring process is not interrupted by dividing a time window; refer to table 2;
Table 2 action score for the second set of 10 experiments
From tables 1 and 2, it can be found that more actions can be monitored using the method of the present invention in table 2, and the action evaluation scores, i.e., the accuracy of the tai chi boxing method, are higher than the first 10 sets of experimental data of table 1;
therefore, the method can bring the improvement of the accuracy of the Taiji boxing method and the continuous judgment of the actions, and improves the detection accuracy for the subsequent model training.
Example 3
Referring to table 3, for a third embodiment of the present invention, there is provided a tai chi fist scoring method based on a time window, including:
generating a dataset comprising 100 image frames; each image frame contains 10 students, each student independently randomly generates 15 to 25 Taijiquan action key points in the image frames, and the key points represent the number of human body key points detected by the model; in numbered order, the 100 image frames are divided into 10 packets for detection, each packet including 10 image frames;
in the key point generation of each student, the number of key points meeting the condition of 18 or more is defined as a passing target for the student, otherwise, the student is defined as a failing target; for each image frame, if the number of passing objects in the image frame is more than or equal to 7, the image frame is marked as a passing picture; otherwise, marking as a failed picture;
at the grouping level, if the number of pass pictures in one group is more than or equal to 7, the group is a pass group; otherwise, the failed packets are grouped;
Training the key points and the image frames in the conditions as models AndUntil the model outputs a result; refer to table 3;
TABLE 3 comparison of results from model training of key points and image frames
From table 3, the number of failed targets, pictures and packets is less than or equal to 1/2, which indicates that the time window and gesture sequence of the present invention can not generate large fluctuation when processing continuous image frames, thereby increasing stability and robustness for the model.
Example 4
Referring to fig. 2, a fourth embodiment of the present invention provides a tai chi fist scoring system based on a time window, including: a central server 100, an edge server 200, a camera 300, a microphone 400, a display device 500, a network 600;
The central server 100 includes a server that is configured to,
The image storage module 11: the video image storage device is used for storing video images of the Taiji boxing students;
Model training module 12: inputting video images of the Taiji boxing students for training, and outputting a trained model;
the edge server 200 includes a server that,
The data storage module 21: a four-dimensional vector for storing keypoints generated by video images;
Image analysis module 22: the video image of the tai chi boxing student collected by the camera 300 is analyzed by using the model output by the model training module 12;
The camera 300 includes a camera lens that is configured to receive a camera lens,
Video acquisition module 31: the video image acquisition device is used for acquiring video images of Taiji boxing students;
the microphone 400 may comprise a set of transducers,
The information notification module 41: sending out information notification according to the analysis result output by the image analysis module 22;
the display device 500 may comprise a display device,
The information display module 51: displaying the evaluation result and the improvement suggestion according to the analysis result output by the image analysis module 22;
the network 600 is used to connect the central server 100, the edge server 200, the camera 300, the microphone 400, the display device 500.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The scheme in the embodiment of the application can be realized by adopting various computer languages, such as object-oriented programming language Java, an transliteration script language JavaScript and the like.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (8)
1. A method for scoring a taiji fist based on a time window, comprising:
Acquiring a Taiji boxing action key point in a Taiji boxing video, and defining the key point in each frame number in the video, wherein the key point comprises coordinate information and gesture marks;
Taking the coordinate information and the gesture mark in the defined key points as a set of key data, constructing a mathematical matrix, and obtaining an identification model through the mathematical matrix; four-dimensional information of each column corresponding to a key point;
Dividing actions and gestures in the mathematical matrix by utilizing a time window according to the recognition model, dynamically adjusting the actions and gestures divided by the time window by setting an adjusting coefficient, and outputting a model result;
According to the output result of the identification model, carrying out overall evaluation on the model to obtain a Taiji boxing scoring result;
The method for constructing the mathematical matrix by using the coordinate information and the gesture mark in the defined key points as a set of key data comprises the following steps of:
When the frame number is input into the mathematical matrix X, deleting continuous frames with the number of key point detection not reaching a set threshold value and image frames with the number of key point detection not meeting the set threshold value, and simultaneously adjusting a time stamp t corresponding to the Taiji boxing video;
Dynamically adjusting the actions and gestures after the time window division by setting an adjustment coefficient, and outputting a model result, wherein the method comprises the following steps:
setting double adjustment coefficients And；
By passing throughTime window sequenceAdjusting to obtain a change sequence corresponding to the gesture sequence；In the variation sequenceOn the basis of (1) adjusting the identification model to obtain:
；
Wherein, Indicating the passage at time point tThe adjusted time series value; the coefficients are adjusted for a sequence of time windows, For varying the sequenceDynamic coefficients of (2); the time period for which the model is identified is represented, A score of taiji boxing outputted for the recognition model;
inputting the X into a recognition model f to obtain:
；
Wherein, AndThe recognition results of the motion and the gesture, respectively.
2. The method for scoring a tai chi punch based on a time window according to claim 1, wherein collecting tai chi punch action key points in tai chi punch video and defining each frame number in the key points comprises:
For each frame, a plurality of key points are contained, and each key point contains coordinate information and gesture marks;
setting N key points, wherein the information of each key point is expressed as four-dimensional vectors (x, y, t, g);
wherein x and y are coordinates of key points in the Taiji boxing video, t is a timestamp corresponding to the Taiji boxing video, and g is a gesture mark.
3. The time window based tai chi fist scoring method according to claim 2, wherein identifying the model includes:
Setting M frames to represent matrix as A mathematical matrix X is constructed, expressed as:
；
Wherein X represents a multidimensional matrix consisting of M frames and N key points, 、、、Gestures respectively representing the abscissa, the ordinate, the time and the mark of the 1 st key point of the 1 st frame;、、、 gestures respectively representing the abscissa, the ordinate, the time and the mark of the 2 nd key point of the 1 st frame; 、、、 Gestures respectively representing the abscissa, the ordinate, the time and the mark of the nth key point of the 1 st frame; 、、、 gestures respectively representing the abscissa, the ordinate, the time and the mark of the 1 st key point of the 2 nd frame; 、、、 gestures representing the abscissa, ordinate, time and mark of the nth key point of the mth frame, respectively.
4. A time window based tai chi fist scoring method according to claim 3, wherein dividing the actions and gestures in the mathematical matrix with time windows according to the recognition model comprises:
The time window sequence in the identification model is The gesture sequence is，
Wherein each ofRepresenting an ith time window in the video, eachRepresenting an mth gesture;
A start frame of the gesture sequence A start frame corresponding to the sequence of time windowsStarting from the next frame of the starting frames of the time window, wherein the current frame of the gesture sequence of each frame is the starting frame of the time window of the next frame until the current frame circulated to the gesture sequence is equal to the starting frame of the time window, so as to obtain the time window sequence。
5. The time window based tai chi fist scoring method according to claim 1, further comprising:
Defining model loss function and introducing binary variable Judging the output result of the identification model; the loss function comprises an action loss function and a gesture loss function;
If the binary variable Greater thanMarking as 1, belonging to a trusted model; otherwise, the model is marked as 0, belongs to an untrusted model, returns to the position where the mathematical matrix is built, and reestablishes the model.
6. A tai chi fist scoring system based on a time window, which is based on the tai chi fist scoring method based on the time window according to any one of claims 1 to 5, and is characterized by comprising:
The video acquisition module acquires the key points of the Taiji boxing action in the Taiji boxing video, and defines the key points in each frame number in the video, wherein the key points comprise coordinate information and gesture marks;
The model building module is used for building a mathematical matrix by taking the coordinate information and the gesture marks in the defined key points as a set of key data, and obtaining an identification model through the mathematical matrix; four-dimensional information of each column corresponding to a key point;
The model algorithm module divides actions and gestures in the mathematical matrix by utilizing a time window according to the recognition model, dynamically adjusts the actions and gestures divided by the time window by setting an adjustment coefficient, and outputs a model result;
And the model result scoring module is used for integrally evaluating the model according to the output result of the identification model to obtain a Taiji fist scoring result.
7. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1-5 when executing the computer program.
8. A computer readable storage medium having stored thereon a computer program, characterized in that the computer program when executed by a processor realizes the steps of the method according to any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410424786.3A CN118015712B (en) | 2024-04-10 | 2024-04-10 | Taiji boxing scoring method, system, equipment and medium based on time window |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410424786.3A CN118015712B (en) | 2024-04-10 | 2024-04-10 | Taiji boxing scoring method, system, equipment and medium based on time window |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118015712A CN118015712A (en) | 2024-05-10 |
| CN118015712B true CN118015712B (en) | 2024-07-12 |
Family
ID=90944946
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410424786.3A Active CN118015712B (en) | 2024-04-10 | 2024-04-10 | Taiji boxing scoring method, system, equipment and medium based on time window |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118015712B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113947811A (en) * | 2021-10-15 | 2022-01-18 | 湘潭大学 | Taijiquan action correction method and system based on generation of confrontation network |
| CN114870384A (en) * | 2022-04-30 | 2022-08-09 | 程亚红 | Taijiquan training method and system based on dynamic recognition |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6072494A (en) * | 1997-10-15 | 2000-06-06 | Electric Planet, Inc. | Method and apparatus for real-time gesture recognition |
-
2024
- 2024-04-10 CN CN202410424786.3A patent/CN118015712B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113947811A (en) * | 2021-10-15 | 2022-01-18 | 湘潭大学 | Taijiquan action correction method and system based on generation of confrontation network |
| CN114870384A (en) * | 2022-04-30 | 2022-08-09 | 程亚红 | Taijiquan training method and system based on dynamic recognition |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118015712A (en) | 2024-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021017606A1 (en) | Video processing method and apparatus, and electronic device and storage medium | |
| US10832069B2 (en) | Living body detection method, electronic device and computer readable medium | |
| WO2020156245A1 (en) | Action recognition method, apparatus and device and storage medium | |
| CN111582342B (en) | Image identification method, device, equipment and readable storage medium | |
| JP2022526548A (en) | Target detection methods, devices, electronic devices and computer readable storage media | |
| CN113870395A (en) | Animation video generation method, device, equipment and storage medium | |
| JP2023548921A (en) | Image line-of-sight correction method, device, electronic device, computer-readable storage medium, and computer program | |
| CN107368820B (en) | Refined gesture recognition method, device and equipment | |
| JP6810048B2 (en) | How to simulate and control virtual balls on mobile devices | |
| KR20150039252A (en) | Apparatus and method for providing application service by using action recognition | |
| CN112507806A (en) | Intelligent classroom information interaction method and device and electronic equipment | |
| CN107395594A (en) | A kind of automatic identifying method for pulling identifying code | |
| CN111652017A (en) | Dynamic gesture recognition method and system | |
| KR101642200B1 (en) | Apparatus and method for generating motion effects by analyzing motion of object | |
| CN112950640A (en) | Video portrait segmentation method and device, electronic equipment and storage medium | |
| CN110516749A (en) | Model training method, method for processing video frequency, device, medium and calculating equipment | |
| Liu et al. | Trampoline motion decomposition method based on deep learning image recognition | |
| CN113434722B (en) | Image classification method, device, equipment and computer readable storage medium | |
| CN117078976B (en) | Action scoring method, action scoring device, computer equipment and storage medium | |
| JP2022058315A (en) | Assist system, assist method and assist program | |
| CN106682669A (en) | Image processing method and mobile terminal | |
| CN118015712B (en) | Taiji boxing scoring method, system, equipment and medium based on time window | |
| CN110910478B (en) | GIF map generation method and device, electronic equipment and storage medium | |
| CN110415171B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN109299777B (en) | Data processing method and system based on artificial intelligence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |