CN107102737A - Contactless palm motion track method for tracing based on linear frequency modulation audio signal - Google Patents
Contactless palm motion track method for tracing based on linear frequency modulation audio signal Download PDFInfo
- Publication number
- CN107102737A CN107102737A CN201710299788.4A CN201710299788A CN107102737A CN 107102737 A CN107102737 A CN 107102737A CN 201710299788 A CN201710299788 A CN 201710299788A CN 107102737 A CN107102737 A CN 107102737A
- Authority
- CN
- China
- Prior art keywords
- palm
- signal
- audio
- frequency modulation
- linear frequency
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 230000005236 sound signal Effects 0.000 title claims abstract description 32
- 238000012937 correction Methods 0.000 claims abstract description 9
- 238000001914 filtration Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 5
- 238000009432 framing Methods 0.000 claims description 4
- 238000005070 sampling Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 2
- 230000000630 rising effect Effects 0.000 claims description 2
- 230000009977 dual effect Effects 0.000 claims 1
- 230000003993 interaction Effects 0.000 abstract description 14
- 230000008569 process Effects 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 12
- 230000000694 effects Effects 0.000 description 10
- 238000001514 detection method Methods 0.000 description 9
- 238000002592 echocardiography Methods 0.000 description 5
- 230000002452 interceptive effect Effects 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 3
- 238000009499 grossing Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 206010052428 Wound Diseases 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 238000001266 bandaging Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000013401 experimental design Methods 0.000 description 1
- 230000007274 generation of a signal involved in cell-cell signaling Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000010183 spectrum analysis Methods 0.000 description 1
- 239000002023 wood Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
- G01S15/02—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems using reflection of acoustic waves
- G01S15/06—Systems determining the position data of a target
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
- H04M1/7243—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages
- H04M1/72433—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages for voice messaging, e.g. dictaphones
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Acoustics & Sound (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Signal Processing (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
Contactless palm motion track method for tracing based on linear frequency modulation audio signal, implementation process is：Step one, the double-channel audio frequency signal being pre-designed is generated；Step 2: carrying out acoustic ranging；Step 3: carrying out palm position estimation；Step 4: being compensated by Doppler shift, trajectory error correction carries out track estimation, improves the trajectory track degree of accuracy.Palm trajectory track precision of the present invention reaches the Centimeter Level order of magnitude, without special equipment, realizes contactless interaction.
Description
Technical Field
The invention relates to an audio ranging and positioning method for mobile computing, in particular to a non-contact palm movement track tracking method based on linear frequency modulation audio signals, and belongs to the field of computer human-computer interaction.
Background
In the era of internet of things, more and more intelligent devices or applications in life need interaction of users. These smart devices or applications include smart televisions, smart homes, VR/AR applications, and some motion sensing games. However, as the complexity of the interactive content in these applications increases, the requirements of flexibility and usability of the interactive mode cannot be met:
(1) for smart televisions and smart homes, most of the existing interaction methods adopt a key-type special remote controller for interaction, and usually, a user needs to click a plurality of buttons to complete one operation, which is not an easy-to-use and quick interaction method;
(2) for VR/AR applications, a pair of hand-held handles are typically provided to sense and recognize user gestures. The handle is special equipment, and the problem that the handle cannot be applied to other interactive scenes exists;
(3) for a motion-sensing game based on image processing, an external facility assumed to be expensive near a display is generally required, and stable power supply to the facility is required. The interactive mode has the problems of high cost and poor mobility.
In addition, there are many situations in life and work where it is not appropriate to interact with a direct hand-held remote control or handle. For example, in public environments (hospitals, game rooms, etc.), a remote controller or a handle that a user needs to directly contact has a sanitary problem due to contact of a plurality of people. Furthermore, it is not suitable for direct contact with the interaction medium when the user's hands are not clean (wet, greasy, etc.) or injured (bandaging, wounds on the surface). Therefore, it is an object of the present patent to devise a non-contact palm trajectory tracking method that is easy to use, does not require additional equipment to be deployed. Due to the wide popularization of mobile terminals such as smart phones and watches, the method is realized based on the existing commercial smart terminals (mobile phones).
In conclusion, the existing interaction method has the problems of poor usability, weak universality and high cost. The method is based on an audio ranging and based on an intelligent terminal design accurate palm movement track tracking scheme.
Disclosure of Invention
The purpose of the invention is: based on commercial smart phone, utilize chirp audio signal to realize non-contact formula palm movement track and track. Compared with the traditional gesture recognition (recognizing the approaching, the departing and the sliding of the palm), the palm movement trajectory tracking is a more flexible and rich interaction method.
The core idea of the invention is as follows: based on the intelligent equipment supporting stereo playing (provided with two homogeneous speakers respectively positioned at the receiver and the microphone of the mobile phone), the invention respectively measures the path lengths from the speakers to the palm of a user and then to the microphone by using a linear frequency modulation audio signal through forming an audio signal receiving and transmitting array by the two speakers and the microphone on the mobile phone, thereby completing the continuous position tracking of the palm of the user. Meanwhile, the invention combines the Doppler shift compensation and the track error correction method, thereby reducing the positioning error caused by palm mobility and improving the track tracking accuracy.
The invention is realized by the following technical scheme:
a system supported by the non-contact palm movement track tracking method based on the linear frequency modulation audio signal, which is called a non-contact palm tracking system for short, comprises an intelligent mobile terminal supporting stereo playing, a positioning object and an external server;
wherein the positioning object refers to a palm;
the function of the computing module can be completed by the intelligent handheld device or an external server; if the intelligent handheld device is used for completing the operation, the external server can be omitted;
the intelligent mobile terminal can be wearable equipment such as a smart phone and a smart watch, a notebook, a microcomputer type intelligent terminal or a non-intelligent terminal with processing capacity;
the intelligent handheld equipment comprises a pair of homogeneous loudspeakers and a microphone;
a non-contact palm movement track tracking method based on linear frequency modulation audio signals mainly comprises the following four steps:
step one, generating a two-channel audio signal: the signal structure has the following characteristics: 1. the signal is a double channel, and two loudspeakers on the mobile terminal respectively play a sub-channel in a stereo playing mode; 2. each channel is modulated with a linear frequency modulation signal added with a Hamming window; 3. the frequency range of the linear frequency modulation signal avoids the frequency range of daily noise, and is supposed to be 16kHz-23kHz, so that the frequency domain bandwidth of the audio frequency is maximized under the condition of ensuring that the interference to users is not obvious; 4. in order to avoid collisions between the chirp signals in the two sub-channels, the modulation is performed in a time division multiplexed manner. Specifically, in a time domain, two linear frequency modulation signals have no overlapping part; 5. after the linear frequency modulation signal, reserving a blank area for resisting multipath noise in the environment; 6. the two sub-channels are modulated using a frequency Up chirp signal (Up-chirp) and a frequency Down chirp signal (Down-chirp), respectively.
Step two, audio ranging is carried out:
the intelligent terminal periodically plays pre-designed dual-channel chirp signals, and the two loudspeakers respectively play one channel of the dual-channel chirp signals. At the same time, the microphone will record the sound continuously, recording the binaural signal and its echo reflected by the palm. By processing the recorded audio, the signal propagation path length is calculated.
Step three, palm position estimation is carried out:
and estimating the palm position according to the audio stream output in the step two. Inside the audio stream is a time sequence regarding the position of the palm. To obtain the position of the palm at each time, the audio stream is first framed such that none of the frames contain the original beacon signal and its echo signal in one audio range. And then denoising the frame by using band-pass filtering, and finding out original audio data and echo data by using a matched filtering method so as to obtain the flight time of the beacon sound. Then, the position of the palm is calculated based on the geometric shutdown between the speaker microphone and the palm on the mobile phone.
To estimate the palm position, a three-dimensional coordinate system is established. The speaker (right channel) below the handset is the origin and the coordinates are Sr (0,0, 0). Theta represents the palm to handset azimuth. The sample indexes of the original beacon signal and the echo signal are respectively I0、I1、I2、I3. It can be derived that from the loudspeaker SlAnd SrPropagation path of emitted chirp signalAndlength of (d):
wherein,andfor loudspeakers SlAnd SrDistance to microphone, fsFor audio sampling rate, c is sound propagation speed.
The distance can be calculated by the placement position of the speaker and the microphone on the mobile phone.
The coordinates of the palm can thus be derived by:
and fourthly, performing track estimation through Doppler shift compensation and track error correction, and improving the track tracking accuracy. Firstly, Doppler shift compensation is carried out, Doppler shift generated by the palm relative to the radial velocity of the mobile phone causes errors of audio path measurement, frequency spectrum analysis is carried out on signal echoes recorded each time, frequency shift of the echoes is obtained, only when the shift is larger than a certain threshold value, the results after matched filtering are compensated, and the relation between the echo frequency shift and the matched filtering errors caused by the Doppler shift is a function relation fitted in advance.
And secondly, performing track correction based on a rough punishment smoothing method, further improving the accuracy of track tracking and reducing interest points in the path. Reference is made to 1993Nonparametric regression and generated linear algorithms, a rough reliability approach, CRC Press.
Advantageous effects
Compared with the existing interaction method, the non-contact palm movement track tracking method based on the linear frequency modulation audio signal has the following beneficial effects:
1. centimeter level accuracy: the palm trajectory tracking precision of the invention reaches centimeter-level magnitude, specifically, in terms of error accumulation error, the precision of 3cm on average is reached under the probability of 76%, and the precision of 2cm is reached under the probability of 48%;
2. no special equipment is needed: by utilizing common electronic equipment (mobile terminals such as mobile phones and smart watches) in living and working environments and tracking the palm tracks of users, a more flexible and universal interpersonal interaction mode is realized;
3. non-contact interaction: the invention does not require the user to hold the mobile intelligent terminal in the interaction process, thereby achieving the purpose of non-contact interaction.
4. The user is not sensitive: the present invention uses a chirp signal (LFS) having a frequency range of 16kHz to 23kHz because the sound in this range is in a frequency band insensitive to the human ear.
Drawings
Fig. 1 is a flowchart of non-contact palm tracking based on chirp audio signals according to one aspect of the present invention.
Fig. 2 is a schematic structural diagram of a two-channel acoustic beacon for performing audio ranging according to the present invention.
Fig. 3 is a scene diagram of non-contact palm tracking based on a mobile phone according to the present invention.
Fig. 4 is a schematic diagram of a coordinate system for tracking a palm trajectory of a user according to the present invention.
Fig. 5 is a diagram of processing the acquired original audio data and echo data in embodiment 1 of the present invention;
wherein, (a) is a frame after framing, the frame includes the double-channel beacon and the echo reflected by the palm; (b) the signal is the signal after the environmental noise is eliminated by a band-pass filter; (c) an output schematic diagram of envelope detection after matched filtering with a left channel signal in the dual-channel beacon; (d) and (3) carrying out matched filtering on the envelope detection signal and a right channel signal in the dual-channel beacon to obtain an output schematic diagram of envelope detection.
Fig. 6 is an experimental scenario of the present invention.
FIG. 7 is an experimental design drawing of example 2 of the present invention;
FIG. 8 is a diagram showing the results of an experiment of positioning accuracy in embodiment 2 of the present invention;
wherein (a) the plot is the mean distance error of the positioning; (b) the standard deviation of the positioning error is plotted;
FIG. 9 is a diagram illustrating the track following effect of embodiment 2 of the present invention;
wherein, (a) the effect and error map of the three linear tracks; (b) effect and error plot of three arc trajectories (c) effect and error plot of triangular trajectory; (d) cumulative profile of trajectory tracking errors.
Detailed Description
The following detailed description of embodiments of the present invention will be made with reference to the accompanying drawings and examples.
This section will describe in detail the non-contact palm movement trajectory tracking method based on the chirp audio signal with reference to the above drawings.
The flow of this example is shown in FIG. 1.
As can be seen from FIG. 1, the method of the present invention includes three parts, namely, initialization, audio ranging, position estimation, and trajectory estimation; wherein the initialization in turn comprises signal generation; sensing comprises signal identification and framing; the position estimation comprises denoising, multipath elimination, signal positioning and coordinate calculation; the trajectory estimation includes doppler compensation and trajectory correction.
Step one, generating a two-channel audio signal: the distance between the palm and a loudspeaker and a microphone on the mobile device is measured by using the audio signal, so that the palm is positioned. The intelligent mobile terminal needs to call a dual-channel loudspeaker to send a pre-designed dual-channel sound signal. Therefore, designing and generating a two-channel sound signal is the first step of the present invention. The invention relates to a dual-channel audio signal based on a linear frequency modulation signal, which can be used for ranging, synchronizing the left and right sound channels of a loudspeaker on a mobile terminal and tracking the palm of a user in a non-contact track mode. Considering the frequency response of the microphone and the loudspeaker and the purpose that the audio signal should be as far as possible free from interference to the user, the frequency range of the chirp signal in the two-channel audio signal is correspondingly set to a range insensitive to the user: 16kHz to 23 kHz;
the two-channel signal designed in this embodiment is shown in fig. 2, where the horizontal axis represents time and the vertical axis represents signal amplitude, the signals indicated as Right and Left control the Right and Left channels independently, and the Tc portion is a chirp signal for ranging. In order to improve the easy recognition degree of the echoes of the left and right sound channels, the invention sets the left and right sound channels as linear frequency modulation signals with descending frequency and ascending frequency respectively. Td is the time interval to avoid left and right channel collisions, and the present invention sets Tc and Td to 1ms each. Te is the minimum interval of front and back two-channel audio signals in front and back time domains, and aims to reduce the interference of multipath effect to echo. The invention sets the duration of Te to 18 ms.
Step two, audio ranging is carried out:
to track the trajectory of the user's palm, we continue to locate the palm. The palm positioning scenario is shown in fig. 3. In one positioning process, a left channel loudspeaker and a right channel loudspeaker and a microphone on the mobile phone form a transceiver array to position the palm.
And the operating system of the mobile terminal controls the loudspeaker to analyze and transmit the two-channel audio signal generated in the first step in a stereo mode. Thus, in FIG. 3, SlAnd SrTransmitting chirp signals C in left and right channels of a two-channel beacon signal separatelylAnd Cr. At this time, ClAnd CrDirectly received by a microphone, which is called as an original beacon, and reflected by the palm, ClAnd CrThe echoes of (b) are received by the microphone (echo beacon). By detecting the time difference between the original beacon and the echo beacon, the length of the beacon propagation path (which is emitted from the speaker and recorded by the microphone after being reflected by the palm) is calculated. As shown in fig. 3, for SlAnd SrRespectively calculating the propagation paths of the signals as SlHM and SrAnd HM (maximum value). A transceiver array incorporating a plurality of speakers and microphones can position the palm. Therefore, palm tracking is a continuous positioning process, so that the speaker will periodically play the two-channel beacon signal generated in the previous step, and the microphone will record at the same time. Thus, the system is inThe first thing obtained in the trajectory tracking process is the audio stream recorded by the microphone. The audio stream contains a sequence of positions of the palm. Separating out the position of the palm at each moment, we first frame the audio stream. The specific method can be completed through a sliding window, and the length of the sliding window can be determined according to the sensing period. The overlapping portion length of the sliding window is set to 10% of the window length. Thus, the recorded sound is divided into several small segments containing the entire two-channel chirp audio data and its echo data. Fig. 5(a) is a diagram of a frame after framing, with time on the horizontal axis and signal amplitude on the vertical axis, and the curve in the diagram represents the original audio recorded by the microphone and its echo. The coordinate system for palm trajectory tracking is shown in fig. 4.
Estimating the palm position according to the third step: for each audio frame (fig. 5(a)), we do the following:
denoising: the purpose of this operation is to eliminate noise in the environment that affects signal detection. The specific method is to use a band-pass filter to filter the audio frame, wherein the cut-off frequency range of the filtering is the set range of the chirp signal frequency in the two-channel beacon: 16kHz to 23 kHz. Fig. 5(b) shows the signal after being processed by the band-pass filter, where the horizontal axis represents time and the vertical axis represents signal amplitude.
Eliminating the multipath effect: multipath effects in indoor environments generally prevent the detection of signal echoes, and therefore, the multipath effects are eliminated. The invention is preset, and the palm only moves within the range of 1m above the mobile phone. Therefore, we crop the audio frame to preserve sound within 1m of sound propagation after the original beacon.
Signal detection: the purpose of this operation is to accurately identify the sample index of the chirp signal and its echo in the audio frame. We use matched filtering of the left and right channel chirp signals in a two channel beacon, respectively. Furthermore, to filter out some outliers that interfere with echo detection. We output the envelope of the matched filtering results separately. Fig. 5(c) and 5(d) show the matched filtered envelopes of the rising and falling chirps, respectively, in two channels, time on the horizontal axis and signal amplitude on the vertical axis. At this time, as shown in fig. 5(c) and 5(d), we can identify the positions of the original signal and the echo signal in the audio frame by the peak detection method.
And (3) position estimation: to estimate the palm position, we establish a three-dimensional coordinate system. As shown in fig. 4, the speaker (right channel) below the handset is the origin with coordinates Sr (0,0, 0). Theta represents the palm to handset azimuth. We define the sample indexes of the original beacon signal and the echo signal in FIG. 5(c) and FIG. 5(d) as I0、I1、I2、I3. Then we conclude that from the loudspeaker SlAnd SrPropagation path of emitted chirp signalAndlength of (d):
wherein,andfor loudspeakers SlAnd SrDistance to microphone, fsFor audio sampling rate, c is sound propagation speed.
The distance can be calculated by the placement position of the speaker and the microphone on the mobile phone.
Therefore, we can derive the coordinates of the palm by the following method:
and estimating the track through Doppler shift compensation and track error correction according to the fourth step:
doppler frequency shift compensation: the doppler frequency shift caused by the radial moving speed of the palm relative to the mobile phone will increase the error in signal detection, thereby increasing the palm positioning error, and therefore, doppler frequency shift compensation is required.
And (3) correcting the track: to cull the sharp points in the track, the track is smoothed using a coarse penalty smoothing method.
To verify the beneficial effects of the present invention, a real experiment was performed on example 1.
The embodiment is implemented by implementing the method of the invention based on a mobile terminal,
the mobile terminal (a smart phone with a stereo speaker) periodically sends a two-channel linear frequency modulation signal, continuously records the signal, and locally processes the signal at the mobile terminal to finally obtain a moving track of a palm. In a specific experimental stage, the invention performs a palm trajectory tracking test based on Nexus 6P.
The experiment was carried out indoors with an indoor area of 4m x 5 m. As shown in fig. 7, the mobile phone is fixed on the tripod, the palm of the user is a distance away from the mobile phone, the mobile phone is placed at the coordinate (0,0) point, and the circle point is the position reference point. The palm of the user is a wood board with the size of 6cm x 20cm, and can be regarded as a substitute for the palm. In order to accurately control the real position, we draw some reference points on the ground, as shown in fig. 8, the horizontal axis and the vertical axis are the horizontal coordinate of the palm, the vertical axis is a table of the vertical axis of the palm, which also includes the polar coordinate information of the palm, and the circle is used as the reference point of each known position. As a positioning test, we place the user's palm at each reference point.
The present invention measures the performance of the system from the mean error and standard deviation of the positioning and trajectory tracking. For the palm location, the graphs (a) and (b) of fig. 8 show the average error and standard deviation of the palm location, the horizontal direction of the table represents the angle of the palm, the vertical direction represents the distance from the palm to the mobile phone, and the data in the table represents the average error and standard deviation of the location at the corresponding angle and distance, respectively. The data table shows that the positioning of the palm presents a trend of increasing average error and standard deviation from the origin to the periphery.
We tested the accuracy of palm trajectory tracking by three trajectory modes: straight lines, arcs, and triangles. Fig. 9(a), 9(b) and 9(c) are illustrative diagrams of the tracking effect in the three modes, respectively, and there are three types of data in the diagrams: the real trajectory, the original palm position, and the palm trajectory after trajectory correction. As can be seen from the above figures, the positioning error of the mobile phone in the linear mode is large because the radial moving speed of the palm relative to the mobile phone is large in this scenario. Under the triangular track mode, the rough punishment smoothing method cannot well process the situation that the track has a sharp turn. Fig. 9(d) is a probability density map of the trajectory estimation of the present embodiment. The palm trajectory tracking method provided by the invention has the advantages that the probability of 76% reaches 3cm, and the probability of 48% reaches 2 cm.
In summary, the non-contact palm moving trajectory tracking method based on chirp audio signals proposed by the present invention is based on a smart phone equipped with two speakers (supporting stereo playing), and the left and right channels of a stereo sound box respectively emit chirp signals, a microphone records the signals after the signals are reflected by the palm of the user, the time of flight of the audio signals between the speaker and the microphone is obtained by detecting the peak point of the recorded audio signals, and then the distance from the palm of the user to the mobile phone and the azimuth angle to the mobile phone in the triangular relationship are derived by using the sound velocity and the sampling rate, in addition, the audio signals are used, the frequency range is 12kHz to 20kHz, which belongs to a frequency band insensitive to the human ear, so that when the palm of the user is accurately positioned, the interference to the palm of the user is minimal, the palm of the user is not required to carry the mobile phone during all positioning processes, thereby achieving the purpose of zero interference of the portable equipment.
The above detailed description is intended to illustrate the objects, aspects and advantages of the present invention, and it should be understood that the above detailed description is only exemplary of the present invention, and is not intended to limit the scope of the present invention, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (3)
1. A non-contact palm movement track tracking method based on linear frequency modulation audio signals is characterized by comprising the following steps: based on the intelligent equipment supporting stereo playing, an audio signal receiving and transmitting array is formed by two loudspeakers and a microphone on a mobile phone, the path length from the loudspeakers to the user palm and then to the microphone is measured by linear frequency modulation audio signals, the continuous position tracking of the user palm is completed, the positioning error caused by palm mobility is reduced by combining a Doppler shift compensation method and a track error correction method, and the accuracy of track tracking is improved.
2. The non-contact palm movement trajectory tracking method based on chirp audio signals as claimed in claim 1, further characterized by:
step one, generating a two-channel audio signal; the signal structure has the following characteristics: 1. the signal is a dual channel, and two loudspeakers on the mobile terminal respectively play a sub-channel in a stereo playing mode; 2. each channel is modulated with a linear frequency modulation signal added with a Hamming window; 3. the frequency range of the linear frequency modulation signal avoids the frequency range of daily noise and is supposed to be 16kHz-23 kHz; 4. in order to avoid collision between the linear frequency modulation signals in the two sub-channels, the modulation is carried out in a time division multiplexing mode; specifically, in a time domain, two linear frequency modulation signals have no overlapping part; 5. after the linear frequency modulation signal, reserving a blank area for resisting multipath noise in the environment; 6. using the frequency rising linear frequency modulation signal and the frequency falling linear frequency modulation signal to respectively modulate the two sub-channels;
step two, audio ranging is carried out: the intelligent terminal periodically plays pre-designed dual-channel linear frequency modulation signals, and the two loudspeakers respectively play one channel of the dual-channel linear frequency modulation signals; meanwhile, the microphone records the sound continuously, records the binaural signal and the echo reflected by the palm, and calculates the signal propagation path length by processing the recorded audio;
step three, palm position estimation is carried out:
firstly, framing the audio stream to enable no frame to contain a beacon original signal and an echo signal thereof in primary audio ranging, then denoising the frame by using band-pass filtering, finding out original audio data and echo data by using a matched filtering method, and further obtaining the flight time of beacon sound; then, calculating the position of the palm based on geometric shutdown between a loudspeaker microphone and the palm on the mobile phone;
and fourthly, estimating the track through Doppler shift compensation and track error correction.
3. The non-contact palm movement trajectory tracking method based on chirp audio signals as set forth in claim 2, further characterized by:
in the position calculation operation in step three, the sample indexes of the original beacon signal and the echo signal are I respectively0、I1、I2、I3(ii) a Then the slave loudspeaker S is based on the established coordinate systemlAnd SrPropagation path of emitted chirp signalAndlength of (d):
wherein,andfor loudspeakers SlAnd SrDistance to microphone, fsIs the audio sampling rate, c is the sound propagation speed;
the distance can be calculated by the placing position between the loudspeaker and the microphone on the mobile phone;
the coordinates of the palm can thus be derived by:
。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710299788.4A CN107102737A (en) | 2017-04-28 | 2017-04-28 | Contactless palm motion track method for tracing based on linear frequency modulation audio signal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710299788.4A CN107102737A (en) | 2017-04-28 | 2017-04-28 | Contactless palm motion track method for tracing based on linear frequency modulation audio signal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107102737A true CN107102737A (en) | 2017-08-29 |
Family
ID=59657342
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710299788.4A Pending CN107102737A (en) | 2017-04-28 | 2017-04-28 | Contactless palm motion track method for tracing based on linear frequency modulation audio signal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107102737A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020147099A1 (en) * | 2019-01-18 | 2020-07-23 | 深圳大学 | Underwater manipulation system for intelligent electronic device, and manipulation method therefor |
| CN112180377A (en) * | 2020-09-22 | 2021-01-05 | 湖南大学 | Non-contact human-computer interaction positioning method, tracking method, terminal and readable storage medium |
| CN113419557A (en) * | 2021-06-17 | 2021-09-21 | 哈尔滨工业大学 | Audio synthesis method for unmanned aerial vehicle |
| CN113612881A (en) * | 2021-07-08 | 2021-11-05 | 北京小唱科技有限公司 | Loudspeaking method and device based on single mobile terminal and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102449500A (en) * | 2010-02-18 | 2012-05-09 | 松下电器产业株式会社 | Ultrasonic measurement method and ultrasonic measurement device |
| CN106296854A (en) * | 2016-08-12 | 2017-01-04 | 上海电机学院 | A kind of classroom based on microphone array roll calling system |
| CN106339081A (en) * | 2016-08-15 | 2017-01-18 | 北京理工大学 | Commercial equipment-based equipment carrying-free palm-positioning human-computer interaction method |
| EP3131311A1 (en) * | 2015-08-14 | 2017-02-15 | Nokia Technologies Oy | Monitoring |
-
2017
- 2017-04-28 CN CN201710299788.4A patent/CN107102737A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102449500A (en) * | 2010-02-18 | 2012-05-09 | 松下电器产业株式会社 | Ultrasonic measurement method and ultrasonic measurement device |
| EP3131311A1 (en) * | 2015-08-14 | 2017-02-15 | Nokia Technologies Oy | Monitoring |
| CN106296854A (en) * | 2016-08-12 | 2017-01-04 | 上海电机学院 | A kind of classroom based on microphone array roll calling system |
| CN106339081A (en) * | 2016-08-15 | 2017-01-18 | 北京理工大学 | Commercial equipment-based equipment carrying-free palm-positioning human-computer interaction method |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020147099A1 (en) * | 2019-01-18 | 2020-07-23 | 深圳大学 | Underwater manipulation system for intelligent electronic device, and manipulation method therefor |
| CN112180377A (en) * | 2020-09-22 | 2021-01-05 | 湖南大学 | Non-contact human-computer interaction positioning method, tracking method, terminal and readable storage medium |
| CN112180377B (en) * | 2020-09-22 | 2023-07-14 | 湖南大学 | Non-contact type man-machine interaction positioning method, tracking method, terminal and readable storage medium |
| CN113419557A (en) * | 2021-06-17 | 2021-09-21 | 哈尔滨工业大学 | Audio synthesis method for unmanned aerial vehicle |
| CN113419557B (en) * | 2021-06-17 | 2022-07-19 | 哈尔滨工业大学 | Audio synthesis method for unmanned aerial vehicle |
| CN113612881A (en) * | 2021-07-08 | 2021-11-05 | 北京小唱科技有限公司 | Loudspeaking method and device based on single mobile terminal and storage medium |
| CN113612881B (en) * | 2021-07-08 | 2022-11-15 | 北京小唱科技有限公司 | Loudspeaking method and device based on single mobile terminal and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chen et al. | EchoTrack: Acoustic device-free hand tracking on smart phones | |
| CN107102737A (en) | Contactless palm motion track method for tracing based on linear frequency modulation audio signal | |
| EP2930530B1 (en) | System and method for object position estimation based on ultrasonic reflected signals | |
| Lin et al. | Rebooting ultrasonic positioning systems for ultrasound-incapable smart devices | |
| KR101520021B1 (en) | System and method for positioning | |
| Cao et al. | Earphonetrack: involving earphones into the ecosystem of acoustic motion tracking | |
| US9632586B2 (en) | Audio driver user interface | |
| CN105718064A (en) | Gesture recognition system and method based on ultrasonic waves | |
| Li et al. | Lasense: Pushing the limits of fine-grained activity sensing using acoustic signals | |
| US20150023138A1 (en) | Ultrasonic Positioning System and Method Using the Same | |
| CN110493690A (en) | A kind of sound collection method and device | |
| CN106339081B (en) | It is a kind of that Portable device palm positioning man-machine interaction method is exempted from based on business machine | |
| CN107004426A (en) | The method and mobile terminal of the sound of admission video recording object | |
| CN114566171A (en) | Voice awakening method and electronic equipment | |
| Chen et al. | A three-dimensional pen-like ultrasonic positioning system based on quasi-spherical PVDF ultrasonic transmitter | |
| CN103257338B (en) | Indoor positioning method and system | |
| Huang et al. | Ubiquitous WiFi and Acoustic Sensing: Principles, Technologies, and Applications | |
| CN105277936A (en) | Range finding system based on mobile phone and method thereof | |
| Jiang et al. | Ctrack: acoustic device-free and collaborative hands motion tracking on smartphones | |
| CN112180377B (en) | Non-contact type man-machine interaction positioning method, tracking method, terminal and readable storage medium | |
| US8798923B2 (en) | Non-echo ultrasonic doppler for corrected inertial navigation | |
| CN112099654A (en) | Intelligent blackboard positioning system based on double ultrasonic modules and method thereof | |
| Liu et al. | Toward device-free micro-gesture tracking via accurate acoustic doppler-shift detection | |
| WO2020071927A1 (en) | Absorption rate detection | |
| Jia et al. | Robust Acoustic TOA Estimation based on Multipath Extraction in Frequency Domain |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20170829 |
|
| WD01 | Invention patent application deemed withdrawn after publication |