WO2007091199A2

WO2007091199A2 - Assessment of attention span or lapse thereof

Info

Publication number: WO2007091199A2
Application number: PCT/IB2007/050355
Authority: WO
Inventors: Martin Ouwerkerk; Frank Pasveer; Joyce Westerink; Therese Overbeek
Original assignee: Koninklijke Philips Electronics, N.V.; U.S. Philips Corporation
Priority date: 2006-02-09
Filing date: 2007-02-02
Publication date: 2007-08-16
Also published as: CN101378696A; WO2007091199A3; EP1983894A2

Abstract

A device and method for monitoring and/or assessing attention changes or lapses of a subject are described. The inventive device and method analyze the regularity of the period of a subject's cardiac cycle to detect a lapse or change of attention. When a subject is concentrating on a particular task or item the heart cycle exhibits strong regularity. But when attention to the task or item is interrupted or the subject's attention declines, the regular periodicity of the heart cycle declines. In a preferred implementation the heart cycle is detected by monitoring the subject's ECG waveform and performing cepstrum processing of the ECG data to determine a measure of R-wave regularity.

Description

ASSESSMENT OF ATTENTION SPAN OR LAPSE THEREOF

This invention relates to analysis of a person's physiological state and, in particular, to the assessment of a person's state of attention to a task or subject.

There are many tasks performed by people which require rigorous concentration on the task at hand. One such task is flying an airplane. A pilot's attention should be focused on the plane's instrument panel and on the airspace in front of the plane. A loss of concentration or the drifting off of the pilot into a reverie could have potentially injurious results. Similarly, a long distance truck driver needs to be focused on other traffic and be on the lookout for potential road hazards. A trucker who falls asleep at the wheel can similarly create a danger for himself and others. A transplant surgeon reattaching blood vessels of a transplanted organ likewise needs to direct his full attention on the surgical procedure. Accordingly it is desirable to be able to monitor a person' s attention to the task at hand in situations such as these.

Numerous efforts have been directed toward sensing a lapse in attention. One straightforward approach is to optically observe a subject's eyelids. When the eyelids are detected to be closed longer than the time of a normal blink of the eyes, an alarm is sounded, alerting the subject to the condition and hopefully directing the attention of the subject back to the task at hand. This approach has application to long-haul truck drivers as described above, where falling asleep at the wheel would be hazardous. However, while this approach can detect a person falling asleep, it cannot detect a drift in attention

004954 such as daydreaming, where the driver's attention to the highway is gone while the driver is still wide awake .

Other psychophysiological variables related to attention to task have also been studied. In the study described in "The influence of task demand and learning on the psychophysiological response" by Stephen H. Fairclough et al., published in the International Journal of Psychophysiology vol. 56 at pp 171-84 (2005), the authors monitored the EEG, ECG, EOG and respiration of thirty subjects over a learning period of 64 minutes. A multiple regression analysis revealed that specific psychophysiological variables predicted learning at different stages on the learning curve. The performance of two groups of children on a selective attention task was described in "Cortical and Autonomic Correlates of Visual Selective Attention in Introverted and Extraverted Children" by Monika Althaus et al . , published in the Journal of Psychophysiology vol. 19(1) at pp 35-49 (2005) . EEG and cardiac activity were continuously recorded during the tasks. Decreases in specific spectra of heart rate variability were found to correlate with the degree of extraversion and task performance and with the children's temperament. The results of these studies, however, were derived only after the collection and post-collection analysis of vast amounts of physiological data. It would be desirable to be able to produce indications of attention change in real time and with minimal analytical complexity.

In accordance with the principles of the present invention, a method is described which senses a subject's changes and/or lapses in attention or interest by analysis of only a single physiological

004954 characteristic, the subject's heart rate. In a preferred implementation the heart rate is acquired by ECG data acquisition. The ECG data is processed to produce a measure of the short-term regularity of the heart rate. In a preferred implementation this is done by sampling the ECG R-wave data and performing a Cepstrum analysis of the samples. A five to eight second interval with a high degree of heart rate regularity correlates with a continuous level of attention or interest whereas significant short-term heart rate variability correlates with lapse of attention. The inventive method has applicability to the monitoring or detection of change and/or lapse of attention or interest in virtually any human activity.

In the drawings :

* FIGURE 1 illustrates an ECG waveform used to explain the principles of the present invention.

FIGURE 2 illustrates an event detection probability curve for the event timing of FIGURE 1.

FIGURE 3 illustrates in block diagram form an attention change/lapse detection device constructed in accordance with the principles of the present invention. FIGURE 4 is a flowchart of a method of the present invention.

FIGURES 5a, 5b, 6a and 6b are typical waveforms of an example of the present invention.

FIGURE 7 illustrates typical waveforms for an example of the present invention for periods of attention lapse.

FIGURE 8 illustrates typical waveforms for an example of the present invention for periods of constant attention. Referring first to FIGURE 1, an ECG waveform of

004954 a subject being monitored by an implementation of the present invention is shown. The present inventors have observed that when an individual is directing his or her attention continuously on a given subject, that individual's short-term heart rate is steady and constant. The present inventors have further observed that, when the individual's attention to the subject is interrupted by an external event or even the drifting of the individual's attention to a different subject, there is a change in the short- term heart beat interval. It is hypothesized that these changes result from changes in the relative involvement of the sympathetic and the parasympathetic nervous systems on the heart. The sympathetic nervous system is known to increase heart rate, whereas the parasympathetic nervous system decreases heart rate. When a person is startled, for instance, it is often said that the person's "heart skipped a beat." This is not a mere figure of speech; it is a clinically verifiable fact. The individual's heart rate does change with a lengthening or shortening of the heart beat interval and the heart can indeed actually skip a heart beat. The present inventors have applied these observations in their invention of the following method and system which are able to monitor and detect changes in a person's attention or lapses of attention to a task or activity.

FIGURE 1 illustrates the application of these principles to the present invention. The ECG waveform 10 is seen to have R-waves 12 recurring at the regular intervals ti, t₂, t_$ ... of a heart beat. The ECG waveform 10 is sampled as indicated by the sampling time arrows in row 14. As will be explained below, these samples can be processed to deduce the

004954 regularity or irregularity of the heart beat. In this example the regular recurrence of the heart beat continues until an event transpires at the time of the event arrow E. This causes a disruption of the regular recurrence of the heart beat, in this example, a skipping of the next regularly expected R- wave 16 which should occur at time t_n. The heart beat then resumes two heart cycles later with an R-wave at time t_m+2. This missing heart beat cannot be detected following the R-wave occurring at time t_m until a sample is obtained at or after the expected time t_n of the occurrence of the next R-wave and the expected R- wave is found to be missing. The sample taken at the time of the sampling arrow at or just before time t_n is the first time at which the missing R-wave 16 could be detected. As more and more samples are taken following the time t_n the probability that the R-wave 16 will be detected as missing increase. This probability is illustrated by the probability curve 18 in FIGURE 2, where it is shown that there is virtually no possibility of detecting the R-wave missing at heart beat HB_n following the heart beat HB_m until the anticipated time of missing heart beat HB_n, or later. Following that time the probability that the absence of the expected heart beat will be detected rapidly increases, approaching near certainty by the time of the next actual heart beat HB_m+2 at time t_m+2 as the probability curve 18 indicates. It is therefore seen that the sampling of an ECG waveform can lead to the detection of a missing heart beat. The same technique can also lead to the detection of a heart beat which does not occur at the same regular interval as the preceding heart beats, as described below. It is this understanding that underlies the following examples of the present

004954 invention in which a lapse or change of attention can be quickly detected in real time.

FIGURE 3 illustrates an attention change/lapse detection device constructed in accordance with the principles of the present invention. An electrode 20 is provided with a conductive layer 22 which receives a person's ECG signal. The electrode 20 is coupled to an amplifier 24 which amplifies the ECG signal, which is then digitally sampled by an analog to digital converter 26. The digital samples of the ECG signal are stored in a memory 28. The ECG signal samples are coupled to a processor 30, in this example illustrated as a digital signal processor, which processes groups of samples by executing an algorithm which determines the short-term regularity of the heart cycle of the ECG signal, as described more fully below. An output signal indicative of attention change or lapse is coupled to an output device 32 which produces a visual or audible or tactile indication of the determined change or lapse of attention. For instance, the output device could produce a continual visual display such as a lighted indicator for so long as the subject using the device was continuously concentrating on a subject. If the subject's attention begins to drift or changes to another subject, an audible indicator could sound or a vibration initiated if such change is to be brought to the attention of the subject or other individual. The active elements of the device are controlled by a controller 40 which can control variables affecting the process such as sampling time, algorithm processing variables, display parameters, and the like. The controller 40 is in turn controlled by a user through a user interface 42 by which the user can control the process as by determining time

004954 constants and/or other process variables.

FIGURE 4 is an example of a method of the present invention. A subject's ECG signal is received in step 52 and sampled in step 54. From a sequence of signal samples a window, or group, of samples is selected for processing in step 56. Each group of samples is then processed to determine the regularity of the R to R interval which, in a preferred embodiment, is done by cepstrum processing. Cepstrum analysis is a mathematical homomorphic transformation introduced in a 1963 paper by Bogert, Healy and Tukey. It is useful for determining periodicities in the autospectrum, the averaged magnitude of multiple instantaneous spectra. The cepstrum can be seen as providing information about the rate of change in different spectral bands. Cepstrum processing produces the inverse Fourier transform of the logarithm of the power spectrum of a signal. In signal processing the cepstrum is commonly viewed as the result of taking the Fourier transform of the decibel (logarithmic) spectrum as if it were a signal. Cepstrum processing has been applied in a variety of area including audio processing, speech processing, geophysics, radar, medical imaging, and others. In speech processing the cepstrum has been used to separate the words and pitch of a voice signal from the transfer function which contains the voice quality. In geophysics the cepstrum has been used to characterize the seismic echoes resulting from earthquakes and explosions. A device called the Heart Tuner which performs cepstrum analysis of an ECG waveform has been developed by Mr. Dan Winter to analyze and display a person' s emotions. A person connected to the Heart Tuner watches the graphical displays produced and observes

004954 the harmonic content of his or her ECG waveform. As the person's emotions become positive, the first cepstrum coherence peak is observed to rise. A person is taught to be empathetic by thinking shareable, positive thoughts to increase the cepstrum coherence peak of the Heart Tuner. Cepstrum processing has also been used to analyze radar signal returns .

In the method of FIGURE 4 cepstrum processing is used as illustrated in steps 62-66 to detect changes in the periodicity of the heart cycle as indicated by the R- to R-wave spacing. While the same information could be obtained by peak detection of the R-waves and measurement and comparison of the R-wave spacings, cepstrum processing is used in the illustrated method because of its robustness, its ready adaptation to a sampled data signal, and its sensitivity to subtle changes in heart rate.

In step 62 the Fourier transform is taken of the window of samples selected in step 56. A logio is taken of this result in step 64. In step 66 an inverse Fourier transform is taken of the log result. This produces a series of values on a time axis exhibiting peaks corresponding to recurrent intervals of the sampled ECG waveform over the sampling interval of the window of step 56. The plots of FIGURES 5 and 6 graphically illustrate the results of this cepstrum processing. FIGURE 5a illustrates the spectrum 82 of an ECG signal produced by taking a Fourier transform of the samples of the ECG waveform of a sequence of heart beats (step 62), in which the abscissa is in frequency (Hertz) . In this example it is seen that the energy of the signal is distributed almost randomly over a wide range of frequencies. After a log₁₀ and an inverse Fourier transform are

004954 taken of this result (steps 64 and 66) by cepstrum processing, the result is a graph 84 on a time axis (Hertz^"1, or seconds) as shown in FIGURE 5b. The peaks of this graph 84 are identified in step 72 by any of a number of standard or sophisticated peak detection techniques. In the example of FIGURE 5b several peaks may be observed, including those identified at 86 and 88. The dominant peak for a heart rate is expected at the heart beat interval, such as the R- to R-wave interval. When analysis of these peaks 86 and 88 is performed to look for a heart rate interval (step 74), the peak 86 at 0.65 sec. is recognized as a heart beat of 91 beats per minute (bpm) . The peak 88 at approximately 1.3 sec. is a subharmonic of the fundamental heart beat rate of 91 bpm. It is seen that in this example the amplitude of the heart rate peak 86 is relatively low. In accordance with the principles of the present invention a relatively low heart rate peak indicates a low level of concentration by the subject because the heart is not beating at a consistently constant rate. This result is communicated to the output device 32 in the device of FIGURE 3 after cepstrum processing by the DSP 30 produces such a result.

FIGURE 6a illustrates a Fourier spectrum 92 of a different sequence of ECG signal samples. In this example the spectral energy is seen to coalesce around a series of well defined peaks. After a log₁₀ and inverse Fourier transform are taken of this data, the time graph 94 of FIGURE 6b results which is seen to exhibit two sharply defined peaks 96 and 98. The dominant peak at approximately 0.73 sec. is identified as the peak produced by a fundamental heart rate of 82 bpm and the peak 98 is a subharmonic

004954 peak at about 1.46 sec. The amplitude of the peak 96 is seen to be of a much higher magnitude than the corresponding peak of FIGURE 5b, indicating a very consistent heart rate. In accordance with the present invention this high magnitude peak 96 is indicative of a high level of attention by the subject producing the ECG waveform of this data. The data of FIGURES 5 and 6 were taken from the same subject, showing that at the time the window data for FIGURES 6a and 6b was acquired the subject was focusing attention on a specific subject or task, and at the time the window data for FIGURES 5a and 5b was acquired the subject's attention had shifted or the subject exhibited a lapse of attention to the prior subject, or task.

In an embodiment of the present invention the short-term regularity of the heart rate is analyzed, preferably by cepstrum processing, to identify changes or lapses in attention of an individual focusing on a subject or task. Examples of this are shown in FIGURES 7 and 8 in which trends in the fundamental cepstrum peak are analyzed to determine not only changes or lapses in attention, but trends in the level of attention. The rate-of-change of the cepstrum plot over time can be used as a criterion for a warning of a change or lapse of attention, for example. The plot 102 at the top of FIGURE 7 illustrates a subject's ECG waveform. In this example the ECG waveform is digitized (sampled) at a 500 Hz rate. After eight seconds of samples have been acquired a window of data is taken of these initial 4000 samples. This window of data then undergoes cepstrum processing as described above. This process is repeated by sliding the window every one-half second to take a different window of 4000

004954 samples for cepstrum processing. The five graphs 104 in FIGURE 7 illustrate the results of Fourier transform processing of five consecutive overlapping windows of 4000 samples. In this example the energy is seen to be fairly evenly distributed over the displayed spectrum. The five results of cepstrum processing of these data windows are shown at 106. These results are seen to exhibit a barely discernible peak at 72 bpm. The cepstrum peaks of the five graphs 106 are averaged and plotted as a data point on the heavy line plot 108 at the bottom of FIGURE 7. This plot is extended by a new cepstrum peak average calculated each time a new data window is cepstrum processed for a different set of five cepstrum peaks. Also plotted at a thin line 110 is a plot of the most recently calculated cepstrum peak. The plots of FIGURE 7 are thus seen to show trends of the cepstrum processing with both a long and a short time constant. These lines 108 and 110 provide an indication of whether a subject's concentration is increasing, decreasing, or staying constant. In FIGURE 7 it is seen that there is no discernible trend to the subject's degree of attention, as the plots 108 and 110 are fairly low, indicating a change or lapse of attention, and move up and down almost randomly.

FIGURE 8 shows the results of the same processing of different data. The subject's ECG waveform 202 is seen to appear substantially the same as the ECG waveform 102 of FIGURE 7. However the Fourier spectra 204 of the four most recent sample windows processed shows regularly recurring spectral peaks. The result of cepstrum processing of these spectra yield a series of high, sharply defined fundamental cepstrum peaks 207 at 78 bpm in the

004954 cepstrum time plots 206. These high cepstrum peaks manifest themselves in the most recent trend lines 208 and 210 at the bottom of the drawing which are seen to strongly trend upward in the most recent time interval at the right side of the trend line plots. The output device 32 would therefore be indicating a constant level of attention in its output signal or display.

Numerous applications of a method and device of the present invention will readily occur to those skilled in the art. The heart rate signal can be acquired in many ways, such as electrodes attached directly to a person's skin, or in a person's clothing or seat or armrest. An earclip or finger sensor could also be used to acquire the heart signal. Various data window sizes may be employed, such as windows ranging from 5 to 11 seconds and preferably in the range of 5.5 to 8 seconds. For persons with long attention spans, longer windows may be desirable. Short-term heart beat regularity generally can be determined in eight to ten heart beats with the techniques described above, which will take longer to acquire for persons at rest than individuals engaging in more physical activity. Sampling rates other than 500 Hz may be used. The output signal can be indicative of the magnitude of the instantaneous cepstrum peak or of the magnitude of the average of a plurality of cepstrum peaks, or of the trend (increasing or decreasing) of the cepstrum signal. Various time constants can be used to produce the longer term averages . The cepstrum peak can be compared to a threshold level or a plurality of peak values can be integrated or differentiated to produce longer term indications of the level of concentration. Techniques other than

004954 cepstrum processing can be used such as analyzing the variance in a person's median heart rate. As FIGURES 1 and 2 show, changes in attention or a lapse in attention can be detected rapidly and in real time, usually in two seconds or less. An embodiment of the present invention has applicability not only in the activities described at the outset of this patent, where safety is an important concern, but also for activities as diverse as evaluating the quality of entertainment. For instance, capacitive ECG sensors in chairs at a movie screening or sporting event can be used to unobtrusively monitor the interest of viewers in the movie or event. A similar embodiment can be used to rate television programs, with the viewer' s interest level monitored in real time or time-stamped and recorded for later comparison with the time of viewing. Other applications will readily occur to the reader, such as a monitor of a person's stress level, which monitors the level of a person's concentration over a long period of time and warns of too little relaxation (too much stress) if the record shows too high a level of attention to tasks over too long a period of time. An embodiment of the present invention can be used to teach or monitor students with learning disabilities to help develop a focus on learning activities.

004954

Claims

WHAT IS CLAIMED IS:

1. A device for monitoring changes in the level of interest of a subject comprising: a heart signal sensor; a circuit which samples the heart signal to produce heart signal samples; a processor responsive to groups of heart signal samples which acts to produce an output signal indicative of the short-term regularity of the heart beat which produced the heart signal; and an output device responsive to the processor output signal which produces a visual or audible output indicative of a change in the level of interest of the subject.

2. The monitoring device of Claim 1, wherein the heart signal sensor acts to sense an ECG waveform.

3. The monitoring device of Claim 2, wherein the circuit which samples acts to produce ECG signal samples .

4. The monitoring device of Claim 3, wherein the processor comprises a digital signal processor.

5. The monitoring device of Claim 4, wherein the output device produces a visual or audible output indicative of changes of the level of interest.

6. The monitoring device of Claim 4, further comprising a memory coupled to the circuit which samples and the digital signal processor which acts to store ECG signal samples.

004954

7. The monitoring device of Claim 1, wherein the processor acts to produce an output signal by cepstrum processing of a group of heart signal samples .

8. The monitoring device of Claim 7, wherein the processor produces an output signal indicative of a cepstrum peak at the heart rate of the subject.

9. The monitoring device of Claim 7, wherein the processor produces an output signal indicative of the trend of the cepstrum peaks of a plurality of groups of heart signal samples.

10. The monitoring device of Claim 9, wherein the output device produces a visual or audible output indicative of a change in the level of interest within two seconds of a change in the subject's level of interest.

11. A method for detecting a change in the attention of a subject comprising: detecting a heart rate signal of the subject; sampling the heart rate signal; selecting groups of heart rate signal samples; cepstrum processing the groups of heart rate signal samples to produce cepstrum data; identifying peaks in the cepstrum data; and detecting a change in attention from the cepstrum data peaks.

12. The method of Claim 11, wherein detecting a change in attention further comprises comparing cepstrum data peaks to a threshold.

004954

13. The method of Claim 11, wherein detecting a change in attention further comprises integrating cepstrum data peaks.

14. The method of Claim 11, wherein detecting a change in attention further comprises differentiating cepstrum data peaks.

15. The method of Claim 11, further comprising actuating an output device in response to detection of a change in attention.

16. A method for detecting a change of attention of a subject comprising: acquiring an ECG signal of the subject; analyzing the regularity of the heart beats of the ECG signal; and detecting a change of attention by a decline in the regularity of the heart beats .

17. The method of Claim 16, wherein analyzing further comprises analyzing the ECG signal by cepstrum processing.

18. The method of Claim 17, wherein the cepstrum processing acts to produce a succession of cepstrum peaks at the heart beat rate; wherein detecting further comprises detecting a change of attention by identifying a change in the level of the cepstrum peaks.

19. The method of Claim 17, wherein the cepstrum processing acts to produce a succession of cepstrum peaks at the heart beat rate; wherein detecting further comprises detecting a

004954 lapse of attention by identifying a decline in the level of the cepstrum peaks.

20. The method of Claim 17, wherein the cepstrum processing acts to produce a succession of cepstrum peaks at the heart beat rate; wherein detecting further comprises detecting a high level of concentration by identifying a high level of the cepstrum peaks.

004954