CN112230779B - Operation response method, device, equipment and storage medium - Google Patents
Operation response method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112230779B CN112230779B CN202011208255.9A CN202011208255A CN112230779B CN 112230779 B CN112230779 B CN 112230779B CN 202011208255 A CN202011208255 A CN 202011208255A CN 112230779 B CN112230779 B CN 112230779B
- Authority
- CN
- China
- Prior art keywords
- target
- sensor data
- response
- acceleration
- hardware interrupt
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000004044 response Effects 0.000 title claims abstract description 87
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000013528 artificial neural network Methods 0.000 claims abstract description 16
- 238000013135 deep learning Methods 0.000 claims abstract description 11
- 230000001133 acceleration Effects 0.000 claims description 69
- 230000008859 change Effects 0.000 claims description 22
- 238000012549 training Methods 0.000 claims description 15
- 238000013527 convolutional neural network Methods 0.000 claims description 10
- 230000010355 oscillation Effects 0.000 claims description 10
- 230000007257 malfunction Effects 0.000 claims description 3
- 230000003068 static effect Effects 0.000 claims 2
- 238000005516 engineering process Methods 0.000 abstract description 6
- 230000003993 interaction Effects 0.000 abstract description 3
- 238000004891 communication Methods 0.000 description 15
- 238000010079 rubber tapping Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 9
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 238000004590 computer program Methods 0.000 description 5
- 230000005236 sound signal Effects 0.000 description 4
- 238000000605 extraction Methods 0.000 description 3
- 230000001105 regulatory effect Effects 0.000 description 3
- 230000003190 augmentative effect Effects 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 230000005484 gravity Effects 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000000712 assembly Effects 0.000 description 1
- 238000000429 assembly Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000000284 resting effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Human Computer Interaction (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The embodiment of the application discloses an operation response method, an operation response device, operation response equipment and a storage medium, and belongs to the technical field of man-machine interaction. The method comprises the following steps: responding to the reported hardware interrupt, and acquiring target sensor data; inputting the data of the target sensor into an operation recognition model to obtain an operation recognition result output by the operation recognition model; responding to the target operation in response to the operation identification result indicating that the target operation is not a misoperation. In the embodiment of the application, the sensor data is further identified by the deep learning neural network on the basis of threshold division, so that the accuracy of operation response is improved, misoperation is avoided, the accuracy of response operation is not required to be improved by adopting a mode of improving the threshold in the related technology, the identification range of target operation is enlarged on the basis of ensuring the accuracy, and normal operation is prevented from being identified as misoperation.
Description
Technical Field
The embodiment of the application relates to the technical field of man-machine interaction, in particular to an operation response method, an operation response device, operation response equipment and a storage medium.
Background
In order to improve convenience and attractiveness of wearable devices such as bluetooth headphones, augmented reality (Augmented Reality, AR) glasses and smart watches, the size of the wearable devices is smaller and smaller, and independent control devices or function keys are not arranged generally, so that developers need to set corresponding function operations of the wearable devices to simple triggering operations, and users can directly control the wearable devices such as the control devices or the keys.
In the related art, operations that can be received by the wearable device mainly include touch operations and knocking operations, for the knocking operations, an acceleration sensor is generally arranged in the wearable device and used for detecting the vibration of the wearable device caused by the knocking operations, meanwhile, in order to improve the recognition rate of the knocking operations, the wearable device is also provided with a knocking threshold for preventing the response to misoperation such as movement and touch, and when the device vibrates, if the slope corresponding to the displacement of the device exceeds a threshold value, the wearable device confirms that the knocking operations are received, otherwise, the wearable device does not respond.
However, in the related art, if the threshold is high, the operation recognition rate is reduced, that is, normal knocking operation can be shielded, and if the threshold is low, a response to the false touch operation of the user is easy to occur, and the accuracy rate and the recognition rate cannot be both achieved.
Disclosure of Invention
The embodiment of the application provides an operation response method, an operation response device, operation response equipment and a storage medium. The technical scheme is as follows:
in one aspect, an embodiment of the present application provides an operation response method, where the method includes:
responding to reported hardware interruption, acquiring target sensor data, wherein the target sensor data are sensor data acquired by a sensor in the wearable equipment before the hardware interruption, and reporting the hardware interruption when the sensor identifies a target operation based on a threshold condition;
inputting the target sensor data into an operation recognition model to obtain an operation recognition result output by the operation recognition model, wherein the operation recognition model is a neural network obtained based on deep learning training, and the operation recognition result is used for indicating whether the target operation is misoperation or not;
responding to the target operation in response to the operation identification result indicating that the target operation is not a misoperation.
In one aspect, an embodiment of the present application provides an operation response device, including:
the acquisition module is used for responding to the reported hardware interrupt, acquiring target sensor data, wherein the target sensor data are sensor data acquired by a sensor in the wearable device before the hardware interrupt, and the hardware interrupt is reported when the sensor identifies a target operation based on a threshold condition;
The identification module is used for inputting the target sensor data into an operation identification model to obtain an operation identification result output by the operation identification model, the operation identification model is a neural network obtained based on deep learning training, and the operation identification result is used for indicating whether the target operation is misoperation;
and the first response module is used for responding to the target operation in response to the operation identification result indicating that the target operation is not misoperation.
In another aspect, embodiments of the present application provide a wearable device including a processor and a memory, where at least one instruction, at least one program, a code set, or an instruction set is stored in the memory, where the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the operation response method as described in the above aspect.
In another aspect, embodiments of the present application provide a computer readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which are loaded and executed by a processor to implement the operation response method as described in the above aspect.
According to one aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the wearable device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the wearable device to perform the operational response methods provided in the various alternative implementations of the above aspects.
The beneficial effects of the technical scheme provided by the embodiment of the application at least comprise:
in the embodiment of the application, the operation recognition model is used for recognizing the target sensor data before the hardware interrupt, and judging whether the operation causing the hardware interrupt is misoperation, so that the target operation is responded when the target operation is determined not to belong to misoperation, the sensor data is further recognized by using the deep learning neural network on the basis of threshold division, the accuracy of operation response is improved, the response to misoperation is avoided, the accuracy of response operation is not required to be improved in a mode of improving the threshold in the related art, the recognition range of the target operation is enlarged on the basis of ensuring the accuracy, and the normal operation is prevented from being recognized as misoperation.
Drawings
FIG. 1 is a schematic diagram of an earphone provided in an exemplary embodiment of the present application;
FIG. 2 is a flow chart of a method of operation response provided by an exemplary embodiment of the present application;
FIG. 3 is a flow chart of a method of operation response provided by another exemplary embodiment of the present application;
FIG. 4 is a schematic diagram of a slope change corresponding to a tapping operation according to one exemplary embodiment of the present application;
FIG. 5 is a schematic diagram of a network architecture of an operational recognition model provided in an exemplary embodiment of the present application;
FIG. 6 is a flow chart of a method of operation response provided by another exemplary embodiment of the present application;
FIG. 7 is a block diagram of an operation response device provided in an exemplary embodiment of the present application;
fig. 8 is a block diagram of a wearable device provided in an exemplary embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
References herein to "a plurality" means two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.
Related art generally uses hardware to identify user operation, for example, for tapping operation, an acceleration sensor is disposed in the wearable device, and is used for collecting motion conditions of the wearable device, and when sensor data corresponding to target operation is detected, hardware interrupt is reported, so that the processor executes instructions corresponding to the target operation. In order to improve accuracy of identifying target operation and avoid responding to misoperation, a threshold value is arranged in the wearable device, and when sensor data reaches the threshold value, the wearable device can identify the sensor data.
However, in the related art, if the threshold is set too high, the target operation recognition rate may be reduced, that is, normal operation may be shielded, and if the threshold is set too low, a response to the false touch operation of the user may be easily generated, and the accuracy rate and the reporting rate may not be both achieved.
In order to solve the problem that the accuracy rate of operation identification and the operation reporting rate cannot be improved simultaneously in the related art, the embodiment of the application provides an operation identification method, after target sensor data meeting a threshold condition is obtained, the target sensor data is identified by using an operation identification model, whether the target operation is misoperation is further judged, and the accuracy rate of operation response can be improved without adjusting a threshold value; and because the accuracy rate of the operation recognition model for recognizing the target operation is higher, the threshold value can be properly adjusted down, the reporting rate of the target operation is improved, and partial normal operation is prevented from being shielded.
The following embodiments are described taking as an example an operation response method for a wearable device such as an earphone, a smart watch, AR glasses, etc. As shown in fig. 1, a schematic diagram of a wireless headset 100 is shown. The wireless earphone 100 is provided with sensors in the bodies 101 and 102 for collecting sensor data, and the operation performed by a user on the earphone can cause the sensor data to change regularly, for example, the wireless earphone 100 is shocked by knocking operation, the acceleration collected by the acceleration sensor is correspondingly changed, the earphone 100 reports hardware interruption when the sensor data meets a threshold condition, the operation recognition model is utilized for further recognizing the target sensor data, and whether the target operation is misoperation or not is judged, so that the response to the target operation is performed when the target operation does not belong to misoperation.
Fig. 2 shows a flowchart of an operation response method provided in an exemplary embodiment of the present application. The embodiment is described by taking the method for wearable equipment as an example, and the method comprises the following steps:
in step 201, in response to the reported hardware interrupt, target sensor data is acquired, where the target sensor data is sensor data acquired by a sensor in the wearable device before the hardware interrupt, and the hardware interrupt is reported when the sensor identifies a target operation based on a threshold condition.
The interruption refers to the process that when some unexpected situations occur and the host intervention is needed, the device can automatically stop the running program and transfer to the program for processing the new situation, including software interruption and hardware interruption. And wherein the hardware interrupt is an asynchronous signal indicating that attention or change to the currently executing program is required, generated by an external device (e.g., network card, hard disk, keyboard, etc.) coupled to the system. In the field of man-machine interaction, a hardware interrupt is generally used for processing a trigger operation, when a device receives the trigger operation, the hardware interrupt is reported to a processor, and a central processing unit (Central Processing Unit, CPU) responds to the hardware interrupt and controls the device to execute an instruction corresponding to the trigger operation.
In a possible implementation manner, the wearable device (such as an earphone, a smart watch, etc.) judges whether a touch operation is received through the sensor data, a threshold condition of hardware interrupt is preset in the wearable device, when the sensor data meets the threshold condition, the wearable device triggers the hardware interrupt and reports the hardware interrupt to the processor, and after receiving the hardware interrupt, the processor acquires the target sensor data and further identifies the target sensor data.
The target sensor data are sensor data acquired by a sensor in the wearable device before hardware interruption. For example, the wearable device detects a touch operation, a press operation, or the like through a pressure sensor; the shaking operation, the knocking operation, and the like are detected by the gravity sensor. After the target sensor data is acquired, the wearable device further identifies whether the target sensor data is sensor data generated by normal operation.
Step 202, inputting the data of the target sensor into an operation recognition model to obtain an operation recognition result output by the operation recognition model, wherein the operation recognition model is a neural network obtained based on deep learning training, and the operation recognition result is used for indicating whether the target operation is misoperation.
When normal operation and misoperation are divided only by means of threshold conditions, if the threshold is too high for shielding misoperation, normal operation is possibly shielded, and operation recognition rate is low; if the threshold value is too low to increase the operation recognition rate, the wearable device may respond to partial misoperation, and the accuracy of the operation response is not high. Therefore, in the embodiment of the application, after the target sensor data is acquired, the wearable device inputs the target sensor data into the operation recognition model, and the machine learning is utilized to recognize the target sensor data, so that the operation recognition rate and the accuracy of operation response are improved.
Optionally, the wearable device is loaded with an operation recognition model, and the wearable device directly inputs the acquired target sensor data into the operation recognition model of the wearable device; or, an operation recognition model is loaded in a terminal (such as a terminal of a smart phone, a tablet computer, a notebook computer and the like) connected with the wearable device, the wearable device sends the target sensor data to the connected terminal, and the terminal uses the operation recognition model to recognize the target sensor data and then feeds back an operation recognition result to the wearable device. The embodiments of the present application are not limited in this regard.
Illustratively, the operation recognition model is a neural network based on deep learning training, such as convolutional neural network (Convolutional Neural Networks, CNN), cyclic neural network (Recurrent Neural Network, RNN), support vector machine (Support Vector Machines, SVM), and the like.
Step 203, responding to the target operation in response to the operation identification result indicating that the target operation is not a misoperation.
And the operation identification result output by the operation identification model indicates whether the target operation corresponding to the target sensor data is misoperation or not, and if the target operation is not misoperation, the response is carried out on the target operation.
In one possible embodiment, the operation recognition model is used to recognize whether the data of the at least one sensor belongs to sensor data resulting from a malfunction. For example, the operation recognition model recognizes target sensor data corresponding to the gravity sensor, and judges whether the target operation is a knocking operation or not; the operation recognition model recognizes target sensor data corresponding to the pressure sensor, and judges whether the target operation is a pressing operation or not.
In summary, in the embodiment of the present application, the operation recognition model is used to recognize the target sensor data before the hardware interrupt, and determine whether the operation that causes the hardware interrupt is an incorrect operation, so that when it is determined that the target operation does not belong to the incorrect operation, the response is performed on the target operation, and the sensor data is further recognized by using the deep learning neural network on the basis of using the threshold value division, so that the accuracy of the operation response is improved, the response to the incorrect operation is avoided, and the accuracy of the response operation is not required to be improved in a manner of improving the threshold value in the related art, so that the recognition range of the target operation is enlarged on the basis of ensuring the accuracy, and the normal operation is prevented from being recognized as the incorrect operation.
Fig. 3 shows a flowchart of an operation response method provided in another exemplary embodiment of the present application. The embodiment is described by taking the method for wearable equipment as an example, and the method comprises the following steps:
in step 301, in response to the reported hardware interrupt, n groups of sensor data continuously collected before the hardware interrupt is reported are determined as target sensor data, where n is a positive integer greater than or equal to 2.
The sensor is used for acquiring sensor data according to a preset sampling frequency.
The collected sensor data can trigger and report the hardware interrupt after meeting the preset condition, so that the sensor data collected before the hardware interrupt reporting is the sensor data generated by the target operation, and the wearable device needs to identify the sensor data collected before the hardware interrupt reporting. In addition, sensor data change caused by target operation, sensor data change caused by misoperation, but the change processes caused by the sensor data change caused by the target operation and the sensor data change caused by misoperation are different, so that the wearable device needs to identify the change process of the sensor data and judge whether the change corresponding to the sensor data is caused by normal operation or not, at least 2 groups of sensor data need to be identified, namely, the wearable device determines n groups of sensor data continuously collected before hardware interrupt reporting as the target sensor data.
In one possible implementation, the sensor is an acceleration sensor, and step 301 includes the steps of:
step 301a, determining an acceleration slope of the x-axis, an acceleration slope of the y-axis, and an acceleration slope of the z-axis at the same acquisition time as a set of sensor data, where the acceleration slopes are used to characterize a change condition of the acceleration.
In one possible implementation, the target operation is an operation that is in direct contact with the wearable device and can cause the wearable device to vibrate, and the wearable device collects acceleration change conditions in the x-axis, y-axis and z-axis directions, namely acceleration slopes, through the acceleration sensor, so as to determine whether the wearable device generates vibration caused by normal operation. The wearable device thus determines the acceleration slope of the x-axis, the acceleration slope of the y-axis, and the acceleration slope of the z-axis at the same acquisition time as a set of sensor data.
In step 301b, n groups of sensor data corresponding to the first n acquisition moments are determined as target sensor data.
In order to judge whether the target operation causing the hardware interrupt is an misoperation, the wearable device inputs the acceleration slope before the hardware interrupt into an operation identification model to identify, namely, the acceleration slope of the x axis, the acceleration slope of the y axis and the acceleration slope of the z axis at the same acquisition time are taken as one group of sensor data, and n groups of sensor data corresponding to n acquisition times before the hardware interrupt is reported are determined as target sensor data.
In one possible implementation, the target operation is a tapping operation, and the hardware interrupt is reported when the acceleration slope of any one of the x-axis, the y-axis, and the z-axis is greater than a slope threshold, and the acceleration slope oscillates during the oscillation period, and the acceleration slope of each axis is less than the slope threshold during the rest period.
Schematically, as shown in fig. 4, a change rule of acceleration under a certain coordinate axis corresponding to a tapping operation is shown. The value of the acceleration is 0 when resting, the wearable device immediately generates vibration after receiving the knocking operation, the acceleration gradually increases along the knocking direction and can exceed an acceleration threshold value, the acceleration value starts to decrease at a certain moment and increases reversely, and then gradually decreases to 0 to restore to rest.
An oscillation period and a rest period are preset in the wearable equipment, when the acceleration acquired by the acceleration sensor is larger than an acceleration threshold value, continuous n groups of acceleration acquired in the subsequent oscillation period and the rest period are calculated to obtain the slope of the acceleration, and if the slope of the acceleration in the oscillation period oscillates and changes, and the slope of the acceleration of each axis in the rest period is smaller than the slope threshold value, hardware interruption is reported.
Illustratively, sensor data when the wearable device receives a tap operation is shown in table 1, and sensor data when the wearable device receives an error operation (e.g., a touch operation, a movement operation, etc.) is shown in table 2
TABLE 1
| X-axis | Y-axis | Z-axis |
| 0 | 1 | 1 |
| 0 | -1 | 0 |
| -1 | 1 | 0 |
| 0 | -2 | 1 |
| -1 | -1 | -1 |
| -1 | -1 | 0 |
| -2 | 2 | 0 |
| 0 | 0 | 0 |
| 0 | -1 | -1 |
| 0 | 2 | -1 |
| 2 | 1 | 0 |
| 1 | 0 | 1 |
| 1 | 0 | 0 |
TABLE 2
It can be seen that a large gap exists between the acceleration data generated by the knocking operation and the misoperation, the vibration amplitude generated by the knocking operation is large, the acceleration value is large, and the vibration amplitude generated by the misoperation is small, and the acceleration value is also small. According to the method and the device, the acceleration change condition of the oscillation of the wearable device is analyzed by using the operation identification model, and whether the target operation is misoperation is judged.
Because the target operation is identified by using the neural network, the accuracy of the identification is higher, so that the knocking operation in the larger acceleration range can be identified, that is, the knocking operation with the smaller knocking force, which causes the oscillation amplitude to be close to the misoperation oscillation amplitude, can still be identified by the wearable device, and in a possible implementation manner, before step 301, the embodiment of the application further includes the following steps:
and the slope threshold is downwards regulated, wherein the reporting rate of the hardware interrupt after the slope threshold is downwards regulated is higher than that before the slope threshold is downwards regulated.
The wearable device (such as an earphone) is provided with a plurality of slope thresholds when leaving the factory, and a developer needs to select one of the slope thresholds as a threshold condition according to the corresponding actual operation of the wearable device. In the related art, in order to improve accuracy of operation response and to shield misoperation as much as possible, a developer needs to select a higher slope threshold as a judgment condition of a knocking operation, and this way can lead to that a part of knocking operation with smaller knocking force is shielded, and for a user with smaller knocking force in use habit, a situation that the knocking operation is performed and the wearable device does not respond may often occur in the process of using the wearable device.
According to the embodiment of the application, the neural network is utilized to identify the target operation, so that the slope threshold can be adjusted downwards, the knocking operation with smaller knocking strength can still be identified, and the reporting rate of the knocking operation is improved, namely, the reporting rate of the hardware interrupt after the slope threshold is adjusted downwards is higher than that before the slope threshold is adjusted downwards.
Optionally, the developer selects a minimum slope threshold value in the factory setting of the wearable device as a threshold condition; or, the developer sets a lower slope threshold according to the actual requirement, so that the recognition range of the knocking operation is larger.
And 302, inputting n groups of sensor data into the operation recognition model according to the sequence of the acquisition time to obtain an operation recognition result output by the operation recognition model.
The operation recognition model needs to recognize the change condition of the sensor data so as to judge whether knocking operation is received, and therefore the wearable device inputs n groups of sensor data into the operation recognition model according to the sequence of the acquisition moments, so that the operation recognition model can recognize the vibration process of the wearable device based on the association relation of acceleration between adjacent acquisition moments, and whether target operation is misoperation is judged.
In one possible implementation, as shown in fig. 5, the operation recognition model is a CNN model, and the operation recognition model includes an input layer, a convolution layer, a separable convolution layer, and a full connection layer. Step 302 includes the steps of:
in step 302a, input target sensing data is acquired through an input layer.
The input layer of the CNN is used for inputting data to be identified, and preprocessing operations, such as normalization operations, are performed on the input data. Illustratively, as shown in fig. 5, the operation recognition model receives 13 sets of sensor data at a time, the input data of which is a feature vector of 3×13, and one set of sensor data is a feature vector.
In step 302b, feature extraction is performed on the target sensor data through the convolution layer and the separable convolution layer.
The convolution layer and the separable convolution layer perform local perception on data input, and a certain number of convolution cores are utilized to perform feature extraction and feature mapping on the input data.
Illustratively, as shown in fig. 5, the convolutional layer of CNN includes 8 convolutional kernels of 2×2, and the separable convolutional layer includes 1 layer of 16 convolutional kernels of 2×2.
And step 302c, classifying the characteristics output by the separable convolution layer through the full connection layer, and outputting an operation identification result.
The full-connection layer is usually arranged at the tail part of the CNN and is used for re-fitting the local features extracted from the previous layers, reducing the loss of feature information, classifying input data according to the features obtained by fitting, and obtaining the identification result.
Illustratively, as shown in fig. 5, the full-connection layer of CNN includes 2 layers of convolution kernels of 1×1×16.
Optionally, the operation recognition model in the embodiment of the present application may further include other neural network structures, such as a hidden layer, a pooling layer, an excitation layer, or other neural network models may also be used, which is not limited in the embodiment of the present application.
Step 303, responding to the target operation in response to the operation identification result indicating that the target operation is not a false operation.
For a specific embodiment of step 303, reference may be made to step 203, which is not described herein.
Step 304, responding to the operation identification result to indicate that the target operation is a misoperation, and not responding to the hardware interrupt.
If the operation identification result indicates that the target operation is misoperation, it is determined that the user does not perform preset operation on the wearable device, the wearable device does not respond to the hardware interrupt, and the instruction before the hardware interrupt is continuously executed.
In a possible implementation manner, a plurality of operations are set in the wearable device, and are respectively used for triggering different instructions, and the wearable device needs to respond according to the instruction corresponding to the target operation identified in the preset duration. For example, in the tapping operation on the earphone, a single tapping operation is used for triggering the starting and the stopping of playing of music, a double tapping operation is used for starting a voice call function, the time interval between two taps in the double tapping operation is not more than 0.5s, after the single tapping operation is identified, the earphone identifies whether the tapping operation is received again within 0.5s, and an instruction to be executed is determined according to the identification result.
In the embodiment of the application, the multiple groups of sensor data continuously collected by the sensor before the hardware interrupt reporting are identified, and the change rule of the sensor data is identified by using the operation identification model, so that whether the target operation is misoperation is judged, and the accuracy of responding to the target operation is improved; and the slope threshold is properly lowered, the range of target operation identification is enlarged, the reporting rate of target operation is improved, and the target operation with similar sensor data to misoperation is prevented from being shielded.
The above embodiment shows a process of identifying a target operation by using an operation identification model, where the operation identification model loaded in the wearable device is a neural network model that is trained in advance, and before the operation identification model identifies the target operation, the operation identification model needs to be trained, so that the accuracy of the operation identification result reaches an expected value. In one possible implementation, the operation recognition model is trained according to positive sample data and negative sample data, the positive sample data comprises sensor data collected by a sensor when a target operation is received, and the negative sample data comprises sensor data collected by the sensor when the target operation corresponds to misoperation.
The developer collects sensor data under multiple target operations and sensor data under different types of misoperation in advance, for example, for a knocking operation, the sensor data of the knocking operation corresponding to different knocking forces and knocking drop points are collected in advance to serve as positive sample data, and sensor data collected by a sensor during touch operation, movement operation of the wearable device and movement of the user wearing the wearable device are collected to serve as negative sample data. The developer adds the target operation or misoperation labels for each group of positive sample data and negative sample data, and trains the operation recognition model for a plurality of times by using the computer equipment until the model converges, for example, the training times reach the preset times, or the accuracy of the operation recognition result output by the operation recognition model reaches the preset value, which is not limited in the embodiment of the application.
In a possible implementation manner, after step 203 described above and shown in fig. 6, the operation response method provided in the embodiment of the present application further includes step 204, on the basis of fig. 2:
in response to receiving the error response indication, the target sensor data is determined as negative sample data, which is used to update the training operation recognition model, step 204.
The error response indication is triggered and generated by a user, if the wearable device performs a response to the target operation after the user performs other operations, or the wearable device automatically performs a response to the target operation when the user does not perform the target operation, the user can trigger the error response indication, and after receiving the error response indication, the wearable device determines the target sensor data as negative sample data and returns to a task before the hardware interrupt reporting, or prompts the user to perform correct operation again.
Optionally, the wearable device receives the operation corresponding to the instruction executed before the hardware interrupt is reported again within a preset time period after responding to the target operation, or determines that an error response instruction is received when receiving the instruction for returning to the previous operation; alternatively, the wearable device determines that the error response indication is received upon receiving a triggering operation of the error response indication by the user, e.g., a triggering operation of the error response control.
When the wearable device receives the error response indication, the target sensor data is determined to be negative sample data, and in one possible implementation manner, after the wearable device acquires a certain amount of negative sample data, the target sensor data when the error response indication is not received is taken as positive sample data, and updating training is performed on the operation identification model. Optionally, the wearable device directly performs update training on the loaded operation identification model, or uploads collected positive sample data and negative sample data to the cloud server, the cloud server performs update training on the current operation identification model based on the received sample data, and feeds back various parameters of the operation identification model after update training to the corresponding wearable device, and the wearable device updates the operation identification model after receiving the model parameters sent by the cloud server, so that the accuracy of the wearable device in responding to target operation is improved.
In the embodiment of the application, when the error response indication is received, the target sensor data are determined to be the negative sample data, and the obtained negative sample data are utilized to update and train the operation recognition model, so that the classification result of the operation recognition model on the target operation and the misoperation is closer to the actual operation of the user, the wearable device performs personalized update on the operation recognition model according to the use condition of the user, and the applicability of the operation recognition model on different users is improved.
Fig. 7 is a block diagram of an operation response device according to an exemplary embodiment of the present application, the device including:
an obtaining module 701, configured to respond to a reported hardware interrupt, obtain target sensor data, where the target sensor data is sensor data collected by a sensor in the wearable device before the hardware interrupt, and report when the hardware interrupt is identified by the sensor based on a threshold condition;
the recognition module 702 is configured to input the target sensor data into an operation recognition model, and obtain an operation recognition result output by the operation recognition model, where the operation recognition model is a neural network obtained based on deep learning training, and the operation recognition result is used to indicate whether the target operation is a misoperation;
The first response module 703 is configured to respond to the target operation in response to the operation identification result indicating that the target operation is not a false operation.
Optionally, the sensor is configured to collect sensor data according to a preset sampling frequency;
the acquiring module 701 includes:
the determining unit is used for determining n groups of sensor data continuously collected before the hardware interrupt reporting as the target sensor data, wherein n is a positive integer greater than or equal to 2;
the identification module 702 includes:
the input unit is used for inputting n groups of sensor data into the operation recognition model according to the sequence of the acquisition time to obtain the operation recognition result output by the operation recognition model.
Optionally, the sensor is an acceleration sensor;
the determining unit is further configured to:
determining acceleration slope of an x-axis, acceleration slope of a y-axis and acceleration slope of a z-axis at the same acquisition time as a set of sensor data, wherein the acceleration slope is used for representing the change condition of acceleration;
and determining n groups of sensor data corresponding to the first n acquisition moments of the hardware interrupt report as the target sensor data.
Optionally, the target operation is a tapping operation, and the hardware interrupt is reported when the acceleration slope of any one of the x-axis, the y-axis and the z-axis is greater than a slope threshold, the acceleration slope oscillates in an oscillation period, and the acceleration slope of each axis is less than the slope threshold in a rest period.
Optionally, the apparatus further includes:
and the adjusting module is used for downwards adjusting the slope threshold, wherein the reporting rate of the hardware interrupt after the slope threshold is downwards adjusted is higher than that before the slope threshold is downwards adjusted.
Optionally, the operation recognition model is a convolutional neural network model, and the operation recognition model comprises an input layer, a convolutional layer, a separable convolutional layer and a full-connection layer;
the identification module 702 includes:
the acquisition unit is used for acquiring the input target sensing data through the input layer;
a feature extraction unit for extracting features of the target sensor data through the convolution layer and the separable convolution layer;
and the classifying unit is used for classifying the characteristics output by the separable convolution layer through the full connection layer and outputting the operation identification result.
Optionally, the operation recognition model is obtained through training according to positive sample data and negative sample data, the positive sample data comprises sensor data collected by the sensor when the target operation is received, and the negative sample data comprises sensor data collected by the sensor when the misoperation corresponding to the target operation is received.
Optionally, the apparatus further includes:
a determination module for determining the target sensor data as the negative sample data for updating training the operation recognition model in response to receiving an error response indication.
Optionally, the apparatus further includes:
and the second response module is used for responding to the operation identification result to indicate that the target operation is misoperation and not responding to the hardware interrupt.
In summary, in the embodiment of the present application, the operation recognition model is used to recognize the target sensor data before the hardware interrupt, and determine whether the operation that causes the hardware interrupt is an incorrect operation, so that when it is determined that the target operation does not belong to the incorrect operation, the response is performed on the target operation, and the sensor data is further recognized by using the deep learning neural network on the basis of using the threshold value division, so that the accuracy of the operation response is improved, the response to the incorrect operation is avoided, and the accuracy of the response operation is not required to be improved in a manner of improving the threshold value in the related art, so that the recognition range of the target operation is enlarged on the basis of ensuring the accuracy, and the normal operation is prevented from being recognized as the incorrect operation.
As shown in fig. 8, embodiments of the present application provide a wearable device 800, which wearable device 800 may include one or more of the following components: a processor 801, a memory 802, a power supply component 803, an audio component 804, an Input/Output (I/O) interface 805, a sensor component 806, and a communication component 807.
The processor 801 generally controls overall operation of the wearable device, such as operations associated with telephone calls, data communications, audio playback, data recording, and operation identification. Processor 801 may include one or more processing cores. The processor 801 utilizes various interfaces and lines to connect various portions of the overall wearable device 800, perform various functions of the wearable device 800 and process data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 802, and invoking data stored in the memory 802. Alternatively, the processor 801 may be implemented in hardware in at least one of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 801 may integrate one or a combination of several of a CPU and a modem, etc. The CPU mainly processes an operating system, a user interface, an application program and the like; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 801 and may be implemented solely by a single communication chip.
The memory 802 is configured to store various types of data to support operation at the wearable device. Examples of such data include instructions, models, contact data, phonebook data, messages, audio, etc. for any application or method operating on the wearable device. The Memory 802 may include a random access Memory (Random Access Memory, RAM) or a Read-Only Memory (ROM). Optionally, the memory 802 includes a non-transitory computer readable medium (non-transitory computer-readable storage medium). Memory 802 may be used to store instructions, programs, code, sets of codes, or instruction sets. The memory 802 may include a stored program area and a stored data area, where the stored program area may store instructions for implementing an operating system, which may be an Android (Android) system (including a system developed based on an Android system), an IOS system developed by apple corporation (including a system developed based on an IOS system depth), or other systems, instructions for implementing at least one function (such as a touch function, a sound playing function, etc.), instructions for implementing the various method embodiments described above, and so forth. The storage data area may also store data collected by the wearable device 800 in use (e.g., phonebook, audio data, sensor data), and so forth.
The power component 803 provides power to the various components of the wearable device 800. The power components 803 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the wearable device 800.
The audio component 804 is configured to output and/or input audio signals. For example, the audio component 804 includes a Microphone (MIC) configured to receive external audio signals when the wearable device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 802 or transmitted via the communication component 807. In some embodiments, the audio component 804 also includes a speaker for outputting audio signals.
The I/O interface 805 provides an interface between the processor 801 and peripheral interface modules, which may be click wheels, buttons, touch panels, and the like. These buttons may include, but are not limited to: volume button, start button and lock button.
The sensor assembly 806 includes one or more sensors for providing status assessment of various aspects of the wearable device 800. For example, the sensor assembly 806 may detect an on/off state of the wearable device 800, a relative positioning of the assemblies, the sensor assembly 806 may also detect a change in position of the wearable device 800 or the wearable device 800, the presence or absence of user contact with the wearable device 800, a change in orientation or acceleration/deceleration of the wearable device 800, and a temperature change of the wearable device 800. The sensor assembly 806 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. In some embodiments, the sensor assembly 806 can also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor. For example, the wearable device 800 determines the operation type of the control operation by the pressure sensor, and determines whether or not the tapping operation is received by the acceleration sensor.
The communication component 807 is configured to facilitate wired or wireless communication between the wearable device 800 and other devices (e.g., control devices). The wearable device 800 may access a wireless network based on a communication standard. In one exemplary embodiment, the communication component 807 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 807 further includes a near field communication (Near Field Communication, NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on radio frequency identification (Radio Frequency Identification, RFID) technology, infrared data association (Infrared Data Association, irDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies. The wearable device 800 synchronously receives information sent by the control device, such as audio files played by the control device, through the communication component 807.
In addition, those skilled in the art will appreciate that the structure of the wearable device 800 illustrated in the above figures does not constitute a limitation of the wearable device 800, and the device may include more or less components than illustrated, or may combine certain components, or may be arranged in different components.
Embodiments of the present application also provide a computer readable storage medium storing at least one instruction that is loaded and executed by a processor to implement the operation response method described in the above embodiments.
According to one aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the wearable device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the wearable device to perform the operational response methods provided in the various alternative implementations of the above aspects.
Those skilled in the art will appreciate that in one or more of the examples described above, the functions described in the embodiments of the present application may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, these functions may be stored on or transmitted over as one or more instructions or code on a computer-readable storage medium. Computer-readable storage media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
The foregoing description of the preferred embodiments is merely exemplary in nature and is in no way intended to limit the invention, since it is intended that all modifications, equivalents, improvements, etc. that fall within the spirit and scope of the invention.
Claims (9)
1. A method of operational response, the method comprising:
determining acceleration slopes of an x axis, an acceleration slope of a y axis and an acceleration slope of a z axis at the same acquisition time as a set of sensor data in response to the reported hardware interrupt, wherein the acceleration slopes are used for representing the change condition of acceleration, the hardware interrupt is reported when a target operation is identified by a sensor based on a threshold condition, and the hardware interrupt is reported when the acceleration slope of any one of the x axis, the y axis and the z axis is larger than a slope threshold value and the acceleration slope of each axis is oscillated and changed in an oscillation period and the acceleration slope of each axis is smaller than the slope threshold value in a static period;
determining n groups of sensor data corresponding to n acquisition moments before the hardware interrupt report as target sensor data, wherein the target sensor data are the sensor data acquired by the sensors in the wearable equipment before the hardware interrupt, and n is a positive integer greater than or equal to 2;
Inputting n groups of sensor data into an operation recognition model according to the sequence of the acquisition time to obtain an operation recognition result output by the operation recognition model, wherein the operation recognition model is a neural network obtained based on deep learning training, and the operation recognition result is used for indicating whether the target operation is misoperation or not;
responding to the target operation in response to the operation identification result indicating that the target operation is not a misoperation.
2. The method of claim 1, wherein prior to acquiring the target sensor data in response to the reported hardware interrupt, the method further comprises:
and the slope threshold is adjusted downwards, wherein the reporting rate of the hardware interrupt after the slope threshold is adjusted downwards is higher than the reporting rate of the hardware interrupt before the slope threshold is adjusted downwards.
3. The method of claim 1 or 2, wherein the operational recognition model is a convolutional neural network model, the operational recognition model comprising an input layer, a convolutional layer, a separable convolutional layer, and a fully-connected layer;
the step of inputting the target sensor data into an operation recognition model to obtain an operation recognition result output by the operation recognition model comprises the following steps:
Acquiring input target sensing data through the input layer;
extracting features of the target sensor data through the convolution layer and the separable convolution layer;
and carrying out feature classification on the features output by the separable convolution layer through the full connection layer, and outputting the operation identification result.
4. The method according to claim 1 or 2, wherein the operation recognition model is trained from positive sample data including sensor data collected by the sensor when the target operation is received and negative sample data including sensor data collected by the sensor when the target operation corresponds to a malfunction.
5. The method of claim 4, wherein the responding to the operation recognition result is used to indicate that the target operation is not a malfunction, and wherein after responding to the target operation, the method further comprises:
in response to receiving the error response indication, the target sensor data is determined to be the negative sample data, which is used to update train the operational identification model.
6. The method according to claim 1 or 2, wherein after the inputting of the target sensor data into the operation recognition model and obtaining the operation recognition result output by the operation recognition model, the method further comprises:
And responding to the operation identification result to indicate that the target operation is misoperation, and not responding to the hardware interrupt.
7. An operation response device, the device comprising:
the identification module is used for responding to the reported hardware interrupt, determining the acceleration slope of the x axis, the acceleration slope of the y axis and the acceleration slope of the z axis at the same acquisition time as a group of sensor data, wherein the acceleration slope is used for representing the change condition of acceleration, the hardware interrupt is reported when a target operation is identified by a sensor based on a threshold condition, and the hardware interrupt is reported when the acceleration slope of any one of the x axis, the y axis and the z axis is larger than a slope threshold value and the acceleration slope is oscillated and changed in an oscillation period and the acceleration slope of each axis is smaller than the slope threshold value in a static period;
the identification module is further configured to determine n groups of sensor data corresponding to n acquisition moments before the hardware interrupt is reported as target sensor data, where the target sensor data is the sensor data acquired by the sensor in the wearable device before the hardware interrupt, and n is a positive integer greater than or equal to 2;
The identification module is further used for inputting n groups of sensor data into an operation identification model according to the sequence of the acquisition time to obtain an operation identification result output by the operation identification model, wherein the operation identification model is a neural network obtained based on deep learning training, and the operation identification result is used for indicating whether the target operation is misoperation or not;
and the first response module is used for responding to the target operation in response to the operation identification result indicating that the target operation is not misoperation.
8. A wearable device, characterized in that it comprises a processor and a memory, in which at least one instruction, at least one program, code set or instruction set is stored, which is loaded and executed by the processor to implement the operation response method according to any of claims 1 to 6.
9. A computer readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by a processor to implement the operational response method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011208255.9A CN112230779B (en) | 2020-11-03 | 2020-11-03 | Operation response method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011208255.9A CN112230779B (en) | 2020-11-03 | 2020-11-03 | Operation response method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112230779A CN112230779A (en) | 2021-01-15 |
| CN112230779B true CN112230779B (en) | 2024-04-05 |
Family
ID=74122481
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011208255.9A Active CN112230779B (en) | 2020-11-03 | 2020-11-03 | Operation response method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112230779B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113177471A (en) * | 2021-04-28 | 2021-07-27 | Oppo广东移动通信有限公司 | Motion detection method, motion detection device, electronic device, and storage medium |
| CN114449405B (en) * | 2022-04-08 | 2022-10-21 | 中测智联(深圳)科技有限公司 | Wireless earphone motion false touch prevention method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2214093A1 (en) * | 2009-01-30 | 2010-08-04 | Research In Motion Limited | Method for tap detection and for interacting with a handheld electronic device, and a handheld electronic device configured therefor |
| CN103218062A (en) * | 2013-04-24 | 2013-07-24 | 伍斌 | Man-machine interaction method and equipment based on acceleration sensor and motion recognition |
| CN108563387A (en) * | 2018-04-13 | 2018-09-21 | Oppo广东移动通信有限公司 | Display control method and device, terminal, computer readable storage medium |
| CN108733427A (en) * | 2018-03-13 | 2018-11-02 | 广东欧珀移动通信有限公司 | Configuration method, device, terminal and the storage medium of input module |
| CN109885371A (en) * | 2019-02-25 | 2019-06-14 | 努比亚技术有限公司 | False-touch prevention exchange method, mobile terminal and computer readable storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8442797B2 (en) * | 2009-03-30 | 2013-05-14 | Kionix, Inc. | Directional tap detection algorithm using an accelerometer |
| KR20190104101A (en) * | 2019-08-19 | 2019-09-06 | 엘지전자 주식회사 | Method, device, and system for determining a false touch on a touch screen of an electronic device |
-
2020
- 2020-11-03 CN CN202011208255.9A patent/CN112230779B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2214093A1 (en) * | 2009-01-30 | 2010-08-04 | Research In Motion Limited | Method for tap detection and for interacting with a handheld electronic device, and a handheld electronic device configured therefor |
| CN103218062A (en) * | 2013-04-24 | 2013-07-24 | 伍斌 | Man-machine interaction method and equipment based on acceleration sensor and motion recognition |
| CN108733427A (en) * | 2018-03-13 | 2018-11-02 | 广东欧珀移动通信有限公司 | Configuration method, device, terminal and the storage medium of input module |
| CN108563387A (en) * | 2018-04-13 | 2018-09-21 | Oppo广东移动通信有限公司 | Display control method and device, terminal, computer readable storage medium |
| CN109885371A (en) * | 2019-02-25 | 2019-06-14 | 努比亚技术有限公司 | False-touch prevention exchange method, mobile terminal and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112230779A (en) | 2021-01-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108681498B (en) | CPU occupancy rate monitoring method and device and mobile terminal | |
| CN107610698A (en) | A kind of method for realizing Voice command, robot and computer-readable recording medium | |
| CN109947650B (en) | Script step processing method, device and system | |
| KR20150012577A (en) | Apparatus Method for controlling voice input in electronic device supporting voice recognition function | |
| CN111491249B (en) | Method and device for detecting in-ear state of earphone, earphone and storage medium | |
| CN107613146B (en) | Volume adjustment method and device and mobile terminal | |
| KR20160148067A (en) | Method for controlling alarm clock of electronic device and electronic device | |
| CN108391008B (en) | Message reminding method and mobile terminal | |
| CN112230779B (en) | Operation response method, device, equipment and storage medium | |
| CN107870674B (en) | Program starting method and mobile terminal | |
| CN109862430B (en) | Multimedia playing method and terminal equipment | |
| KR20200094732A (en) | Method and system for classifying time series data | |
| CN111158487A (en) | Man-machine interaction method for interacting with intelligent terminal by using wireless earphone | |
| CN110830368A (en) | Instant messaging message sending method and electronic equipment | |
| CN108429855B (en) | Message sending control method, terminal and computer readable storage medium | |
| CN110335629B (en) | Pitch recognition method and device of audio file and storage medium | |
| CN112423176A (en) | Earphone noise reduction method and device, storage medium and noise reduction earphone | |
| CN113744736B (en) | Command word recognition method and device, electronic equipment and storage medium | |
| CN113778255B (en) | Touch recognition method and device | |
| CN108628534B (en) | Character display method and mobile terminal | |
| CN113225624B (en) | Method and device for determining time consumption of voice recognition | |
| CN107819946B (en) | A kind of method, device and mobile terminal of speech recognition | |
| CN111897916B (en) | Voice instruction recognition method, device, terminal equipment and storage medium | |
| CN106971715A (en) | A kind of speech recognition equipment applied to robot | |
| CN109032482B (en) | Split screen control method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |