WO2017167630A1

WO2017167630A1 - System and method for detecting a breathing pattern

Info

Publication number: WO2017167630A1
Application number: PCT/EP2017/056900
Authority: WO
Inventors: Albertus Cornelis Den Brinker; Caifeng Shan; Radu Serban Jasinschi; Ronaldus Maria Aarts
Original assignee: Koninklijke Philips N.V.
Priority date: 2016-03-31
Filing date: 2017-03-23
Publication date: 2017-10-05

Abstract

A system for detecting a breathing pattern of a user, comprising an airflow sensor for continuously obtaining an airflow signal of the user; a processing unit for determining the breathing pattern of the user based on the obtained airflow signal of the user; memory configured for storing a predefined COPD severity rule database; an evaluation unit for evaluating the COPD severity of the user by comparing the breathing pattern of the user with the predefined COPD severity rule database to determine the COPD severity of the user, wherein the system comprises a head-mountable wearable device including the airflow sensor; the head-mountable wearable device further comprising a first wireless communication module; and a remote device including the processing unit, the memory, the evaluation unit, and a second wireless communication module for communicating with the first wireless communication module. A method for detecting a breathing pattern and a computer program product implementing the method are also disclosed.

Description

System and method for detecting a breathing pattern

FIELD OF THE INVENTION

The present invention relates to a system for detecting a breathing pattern of a user, comprising an airflow sensor for continuously obtaining an airflow signal of the user; and a processing unit for determining the breathing pattern of the user based on the obtained airflow signal of the user.

The present invention further relates to a method for detecting a breathing pattern of a user, comprising continuously obtaining an airflow signal of the user by an airflow sensor; determining the breathing pattern of the user based on the obtained airflow signal of the user.

The present invention yet further relates to a computer program product comprising a computer-readable data carrier carrying computer program code for, when executed on the above system, causing a processor of the system to implement the above method. BACKGROUND OF THE INVENTION

Chronic Obstructive Pulmonary Disease (COPD) is a progressive and irreversible disease that affects the lung function but also has significant extra-pulmonary effects. COPD is a major disease. The more severe COPD patients have frequent acute exacerbations (AE-COPD: a sudden worsening of the symptoms) that these may require hospitalization. A better management of patients at home, including prediction of acute exacerbations with sufficient lead time, will reduce hospitalizations, morbidity, mortality, and improve quality of life. Therefore, methods for cost-effective management of these patients are desired.

It is known that prediction of AE-COPD by subjective approaches (questionnaires) is far from the set goal of a cost-effective solution. This is due to the limited information that can be gathered by this type of measurement (i.e., leading to poor specificity/sensitivity balance in the prediction) and by the limited lead time that is obtained. The current approaches using actigraphy are limited in success where an issue is that the sensors are considered uncomfortable to wear, and therefore the use of these systems is limited due to the low adherence.

The current cough monitoring systems are twofold. There are unobtrusive stationary systems as well as mobile systems. Stationary systems have obvious disadvantages in terms of sensitivity. Mobile systems have similar problems as accelerometers in the sense that the adherence to correct usage is limited. Blood oxygenation may be another relevant marker for the determining the state of the COPD patient.

SUMMARY OF THE INVENTION

The present invention seeks to provide a system for improved detection of a breathing pattern of a person.

The present invention further seeks to provide a method for improved detection of the breathing pattern of a person.

The present invention yet further seeks to provide a computer program product implementing the method for improved detection of the breathing pattern of a person when executed on a system processor.

According to an aspect, there is provided a head-mountable wearable device for detecting a breathing pattern of a user, comprising an airflow sensor for continuously obtaining an airflow signal of the user; a processing unit for determining the breathing pattern of the user based on the obtained airflow signal of the user.

In an embodiment, the processing unit in the head-mountable is configured to determine a normal breathing pattern of the user, for instance, breathing pattern during sport activity or resting; an abnormal breathing pattern of the user, for instance, breathing pattern under a lung disease such as coughing or Chronic Obstructive Pulmonary Disease (COPD), or breathing pattern under cardiovascular condition such as heart arrhythmia.

In an embodiment, the head-mountable wearable device further comprises memory configured for storing a predefined COPD severity rule database; an evaluation unit for evaluating the COPD severity of the user by comparing the breathing pattern of the user with the predefined COPD severity rule database to determine the COPD severity of the user.

The invention is based on the insight that upon continuously detecting, for instance, how often the patient feels "breathless", how often the patient coughing, and further how active the patient is (normally due to less oxygen intake, the patient easily feels tired and becomes inactive), the COPD severity of a person can be therefore determined. In addition, users with COPD have difficulties breathing in and out, primarily due to the narrowing of their airways, which is called airflow obstruction. The COPD patients have typical symptoms such as increasing breathlessness when active and a persistent cough with phlegm. Prior art solution is based on a spot measurement, such as Forced Expiratory Volume (FEV) measurement, which is the greatest volume of air that can be breathed out by the patient in the first second of breath. Such measurement is on a discrete measurement basis and requires a high demanding exercise to the user. In contrast, by using the airflow sensor to

continuously measuring the airflow signal of a person to determine the breathing pattern of the person, and by comparing the breathing pattern of the user with the predefined COPD severity rule database, the COPD severity of the person can be determined. This may increase the accuracy of the COPD severity measurement because more than one type of signals sensed from the user are available compared with the FEV measurement. In addition, this allows the patient to observe quantitatively his/her patterns on breathing and coughing, and to increase awareness about the progression of the COPD disease, e.g. by coupling the collected objective breathing pattern data to a diary (annotated by the patient) and creating overviews on a daily, weekly or monthly basis.

In an embodiment, the head-mountable wearable device comprises an audio sensor configured for continuously obtaining an audio signal of the user.

In an embodiment, the processing unit is configured for determining the breathing pattern of the user based on a correlation of the obtained airflow signal of the user and the audio signal of the user. Advantageously, the following indicators of the respiratory condition can be determined based on the correlation of the obtained airflow signal of the user and the audio signal of the user: minute ventilation (VE); respiration frequency (fk), Breath by breath respiratory time (Ττοτ), Inspiratory time (Ti), Expiratory time (TE), Inspiration to expiration time ratio (TI/TE), and Fractional inspiration time (TI/TTOT).

Advantageously, by modelling the correlation between the sound measurement and airflow measurement of the user, the determination of the COPD severity can be derived even when one of the measurement results is inferior, e.g. due to the noise, or missing. Therefore, the determination result of the COPD severity can be more robust.

In an embodiment, the head-mountable wearable device further comprises an accelerometer for continuously obtaining a motion signal of the user for identifying a user activity.

In an embodiment, the processing unit is configured for determining the breathing of the user based on a correlation of the obtained airflow signal of the user and the motion signal of the user. This may allow the head-mountable wearable device to detect under what conditions the user may have a specific breathing pattern, e.g. feel breathless. Advantageously, by knowing the context of air flow measurements, the head-mountable wearable device may provide more accurate assessment with respect to the determination of the COPD severity of the user.

In an embodiment, the processing unit is configured for determining the breathing pattern of the user based on a correlation of the obtained airflow signal of the user, the audio signal of the user and the motion signal of the user.

In an embodiment, the memory is configured for storing information indicating a historical breathing pattern of the user.

In an embodiment, wherein the processing unit is configured for determining the breathing pattern of the user based on a correlation of the obtained airflow signal of the user, the audio signal of the user, the motion signal of the user, and the stored historical breathing patterns of the user. Advantageously, by taking the historical breathing patterns of the user into account, the head-mountable wearable device may precisely detect the short- term events, e.g. infection, of the user from the measurements, for example, by short-term (sudden) increases in airflow limitations, and therefore provide more accurate assessment with respect to the determination of the COPD severity of the user. This short-term events detection can be improved by coupling with other measurements such coughing measured from the microphone and/or decreased activity measured from the accelerometer.

In an embodiment, the evaluation unit for evaluating the COPD severity of the user is configured for applying a learned model associated with evaluation result to the determined breathing pattern of the user to determine whether the evaluation result is applicable to the user's current COPD severity or not.

In an embodiment, the evaluation unit for evaluating the COPD severity of the user is further configured for receiving feedback indicating the correctness of the evaluation result to the user's current COPD severity; updating the COPD severity rule database based on the received feedback and the determined breathing pattern of the user; retraining the learned model based on the updated COPD severity rule database using a machine learning process. Advantageously, this may allow an automatically update of the COPD severity rule database to be adapted to a specific user such that the precision of the determination of the COPD severity can be improved.

According to another aspect, there is provided a system for detecting a breathing pattern of a user, comprising an airflow sensor for continuously obtaining an airflow signal of the user; a processing unit for determining the breathing pattern of the user based on the obtained airflow signal of the user memory configured for storing a predefined COPD severity rule database; an evaluation unit for evaluating the COPD severity of the user by comparing the breathing pattern of the user with the predefined COPD severity rule database to determine the COPD severity of the user, wherein the system comprises a head- mountable wearable device including the airflow sensor; the head-mountable wearable device further comprising a first wireless communication module; and a remote device including the processing unit, the memory, the evaluation unit, and a second wireless communication module for communicating with the first wireless communication module. Advantageously, this allows processing and analyzing the breathing pattern in a remote server, thus reduces the power consumption in the head-mountable wearable device.

Alternatively, the first wireless communication module may automatically initialize a communication with the remote device, such as promoting to start a conversation with the remote device or to transmitting the measured data to the remote device.

Advantageously, this may allow sharing the monitored breathing pattern of the user via the internal or clouding, such that the relatives or friends of the user may also observe the breathing patterns or coughing of the user. In addition, this may allow triggering a conversation with the friends or relatives of the user when the measured breathing pattern is above a certain threshold.

Alternatively, the first wireless communication module may automatically transmit the measured breathing pattern to a remote device, such as the server in the hospital such that the breathing pattern is analyzed and presented in such a way that the progression of the disease, e.g. long term effects, and insights are gained. Advantageously, this may allow the hospital and insurance company to objectively assess the condition of the patient, e.g. after they are dismissed from the hospital. Alternatively, the second wireless communication module may send information back to the first wireless communication module.

Advantageously, this may also allow the medical professionals to send warnings or alerts to the user.

According to another aspect, there is provided a computer-implemented method for detecting a breath pattern of a user, comprising continuously obtaining an airflow signal of the user by an airflow sensor; determining the breathing pattern of the user based on the obtained airflow signal of the user; storing a predefined COPD severity rule database; and evaluating a COPD severity of the user by comparing the breathing pattern of the user with the predefined COPD severity rule database. According to another aspect, there is provided a computer program comprising program code means for causing a computer to carry out the steps of the method as mentioned above when said computer program is carried out on the computer. BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention are described in more detail and by way of non- limiting examples with reference to the accompanying drawings, wherein

Fig. 1 schematically depicts a head-mountable wearable device according to an embodiment;

Fig. 2 schematically depicts an aspect of the head-mountable wearable device of FIG. 1;

Fig. 3 schematically depicts a system having a head-mountable wearable device and a remote system according to an embodiment;

Fig. 4A and Fig. 4B further schematically depicts an aspect of the head- mountable wearable device of Fig. 1;

Fig. 5 shows an example of a COPD severity rule database according to an embodiment;

Fig. 6 is a flow chart of an algorithm implemented by a system according to an embodiment.

DETAILED DESCRIPTION OF THE EMBODIMENTS

It should be understood that the figures are merely schematic and are not drawn to scale. It should also be understood that the same reference numerals are used throughout the figures to indicate the same or similar parts.

In the context of the present application, a head-mountable wearable device is a device that can be worn of the head of its user and provides the user with computing functionality. The head-mountable wearable device may be configured to perform specific computing tasks as specified in a software application (app) that may be retrieved from the Internet or another computer-readable medium. Non-limiting examples of such head- mountable wearable devices include smart headgear, e.g. eyeglasses, goggles, a helmet, a hat, a visor, a headband, or any other device that can be supported on or from the wearer's head, and so on.

In the context of the present application, an airflow sensor or flow sensor means any sensor capable of providing a breathing volume measurement of a user of the head-mountable wearable device. The airflow sensor may be integrated in the head- mountable wearable device, closed to the nose or may be separated from the head-mountable wearable device, e.g. inserted in nostrils, such that the airflow sensor is communicatively coupled via a wired or wireless connection to the head-mountable wearable device. In this way, the air flow of the user may be continuously monitored.

In the context of the present application, an audio sensor means any sensor capable of deriving an audio or acoustic signal of a user of the head-mountable wearable device. The audio sensor may be one or more microphones integrated in the head-mountable wearable device to sense the sound associated with coughing and breathing generated by the patient. Compared to a stationary setup, i.e. using the microphones installed in the room, the microphone(s) integrated in the head-mountable wearable device may allow continuous monitoring the audio signal of the user no matter where the user is e.g. outdoors, or in a different patient room, and furthermore, better sound quality for more reliable coughing detection. By inserting a microphone close to the nose, e.g. on the nose pad of the head- mountable wearable device, it is also possible to monitor the breathing patterns using the microphone. The microphone can also be used to determine the coughing rate/frequency. In addition, the device can also recognize different types of coughing by analyzing the characteristics of sound, for example, dry or wet coughing, cough with wheezing, etc.

Alternatively, if the head-mountable wearable device is not worn by the user, e.g. when the user falls asleep during the night, the head-mountable wearable device may be located in a place close to the user such that the audio signal of the user can still be monitored, e.g.

monitor the coughing of the user via the microphone. In this way, the microphone in the head-mountable wearable device can be used as a stationary microphone setup.

In the context of the present application, an activity sensor means any sensor capable of obtaining a motion signal of the user of the head-mountable wearable device for determining a user activity, such as coughing. The activity sensor may be an accelerometer for continuously obtaining a motion signal of the user for identifying a user activity. The accelerometer may be integrated in the head-mountable wearable device for measuring the patient's head and/or eye (lid) movement during coughing. By combining the motion signal and the audio data, the activity of the user, such as coughing, can be more accurately assessed. In addition, the user having COPD disease may often feel breathless and exhausted, so they tend to be less active than average healthy user. The motion of the user sensed by the accelerometer may also provide information about how active the user is in a given period. The activity sensor can also be a frontal-looking camera integrated in the head-mountable wearable device to measure the head movement of the user, which can be used to determine the coughing rate/frequency. By analyzing the images acquired by the camera during the coughing, the shaking of head can be derived. Alternatively or additionally, the eyelid movements can be detected due to the fact that eyelids typically close during coughing.

In the context of the present application, an oximetry sensor means any sensor capable of measuring oxygen saturation (SPO2) in the blood of a user of the head-mountable wearable device. The Oximetry sensor can be either integrated in the head-mountable wearable device, or as an accessory of the head-mountable wearable device, which can be attached to the ears of the user to measure the patient's oxygen level and/or the heartbeat. Alternatively, the head-mountable wearable device may be connected to an oxygen box for measuring the amount of oxygen consumption of the user.

Fig. 1 schematically depicts an embodiment of a head-mountable wearable device 100 for detecting a breathing pattern of a user. Fig. 2 schematically depicts a block diagram of an embodiment of the head-mountable wearable device 100, further highlighting the functionality of the head-mountable wearable device 100 in terms of functional blocks, at least some of which may be optional functionality. By way of non-limiting example, the head mountable computing device 100 is depicted as smart glasses, but it should be understood that the head-mountable wearable device 100 may take any suitable shape as previously explained.

The head-mountable wearable device 100 comprises a microphone 112 for deriving an audio or acoustic signal of a user of the head-mountable wearable device 100; a frontal- looking camera 114 for measuring the head movement of the user so as to determine a user activity; an airflow sensor 116 for measuring breathing volume of the user the head- mountable wearable device 100; an accelerometer 118 for continuously obtaining a motion signal of the user of the head-mountable wearable device 100 for identifying a user activity; and an oximetry sensor 120 for measuring oxygen saturation in the blood of the user of the head-mountable wearable device 100. Any suitable types of microphone 112, camera 114, airflow sensor 116, accelerometer 118 and oximetry sensor 120 may be integrated in the head-mountable wearable device 100 for this purpose. The microphone 112 and airflow sensor 116 may be mounted in a frame 125 of the head-mountable wearable device 100 in a location that may be close to the nose of the user when wearing the head-mountable wearable device 100.

The head-mountable wearable device 100 may comprise at least one transparent or see-through display module 106, under control of a discrete display controller (not shown). Alternatively, the display controller may be implemented by a processing unit 110 of the head-mountable wearable device 100, as shown in Fig. 2. The processing unit 110 may further determine the breathing pattern of the user based on the obtained one or more signals of the user sensed by the one or more sensors 112, 114, 116, 118, 120. The breathing pattern determined by the processing unit 110 may comprises a normal breathing pattern of the user, for instance, breathing pattern during sport activity or resting; or an abnormal breathing pattern of the user, for instance, breathing pattern under a lung disease such as coughing or Chronic Obstructive Pulmonary Disease (COPD), or breathing pattern under cardiovascular condition such as heart arrhythmia.

When present, the at least one display module 106 is typically arranged to cover the field of view of the wearer when the head-mountable wearable device 100 is worn by the wearer such that a wearer of the head-mountable wearable device 100 may observe the field of view through an image displayed on the at least one the display module 106. In an embodiment, the head-mountable wearable device 100 comprises a pair of transparent display modules 106 including a first display module that can be observed by the right eye of the wearer and a second display module that can be observed by the left eye of the wearer. Alternatively, the at least one display module 106 may be a single display module covering both eyes of the wearer.

The functioning of at least part of the head-mountable wearable device 100 may be controlled by a processing unit 110 that executes instructions, i.e. computer program code, stored in a non-transitory computer readable medium, such as data storage or memory 102. Thus, the processing unit 110 in combination with processor-readable instructions stored in data storage 102 may function as a controller of the head-mountable wearable device 100. In addition to instructions that may be executed by the processing unit 110, data storage 102 may store a predefined COPD severity rule database 124. In addition, data storage 102 may further store information indicating a historical breathing pattern of the user.

In an embodiment, the head-mountable wearable device 100 may be adapted to wirelessly communicate with a remote device 200 as shown in Fig. 2 via a first wireless communication module 104. To this end, the head-mountable wearable device 100 may include a first wireless communication module 104 for wirelessly communicating with a remote target such as the remote device 200. Any suitable wireless communication protocol may be used for any of the wireless communication between the head-mountable wearable device 100 and the remote device 200, e.g., an infrared link, Zigbee, Bluetooth, a wireless local area network protocol such as in accordance with the IEEE 802.11 standards, a 2G, 3G or 4G telecommunication protocol, and so on. The remote device 200 may for instance be controlled to provide the user of the head-mountable wearable device 100 with feedback information such as warning signals and/or medical instructions, as will be further explained below.

The head-mountable wearable device 100 may additionally or alternatively include an input sensor (not shown in Fig.1), e.g. a button or the like for facilitating the wearer of the head-mountable wearable device 100 to select the user instruction from a list of options. Such list of options for instance may be displayed on the at least one transparent display module 106 of the head-mountable wearable device 100, when present.

The head-mountable wearable device 100 may further include a user interface

108 for receiving input from the user. User interface 108 may include, for example, a touchpad, a keypad, buttons, a microphone, and/or other input devices. The processing unit 110 may control at least some of the functioning of head-mountable wearable device 100 based on input received through user interface 108. In some embodiments, the at least one further sensor 118 may define or form part of the user interface 108.

The head-mountable wearable device 100 may further include an evaluation unit 122 for evaluating the COPD severity of the user by comparing the breathing pattern of the user with the predefined COPD severity rule database 124 to determine the COPD severity of the user. Note that the evaluation unit 122 may be implemented as part of the processing unit 110.

Although Fig. 2 shows various components of head-mountable wearable device 100, i.e., the data storage 102, the first wireless communication module 104, the user interface 108, the processing unit 110, the microphone 112, the frontal-looking camera 114, the airflow sensor 116, the accelerometer 118, and the oximetry sensor 120, as being separate from the at least one display module 106, one or more of these components may be mounted on or integrated into the at least one display module 106 or the frame 125. For example, the frontal- looking camera 114 may be mounted on a see-through display module 106, the user interface 108 could be provided as a touchpad on a see-through display module 106, the processing unit 110 and the data storage 102 may make up a computing system in a see- through display module 106, and the other components of head-mountable wearable device 100 could be similarly integrated into a see-through display module 106. As already mentioned above, the microphone 112 and airflow sensor 116 are preferably mounted in the frame 125 of the head-mountable wearable device 100 in a location that may be close to the nose of the user when wearing the head-mountable wearable device 100. Alternatively, the head-mountable wearable device 100 may be provided in the form of separate devices that can be worn on any part of the body or carried by the wearer, apart from at least the one display module 106 and the airflow sensor 116, which typically will be mounted on the head and the nose, respectively. The separate devices that make up head-mountable wearable device 100 may be communicatively coupled together in either a wired or wireless fashion.

The remote device 200 may comprises a second wireless communication module 202 for communicating with the first wireless communication module 104 of the head-mountable wearable device 100.

It should be understood that the one or more components of the head- mountable wearable device 100 may be located in the remote device 200.

Fig 3 schematically depicts a system 300 for detecting a breathing pattern of a user having a head-mountable wearable device 100 and a remote device 200. The head- mountable wearable device 100 may include an airflow sensor 116 for continuously obtaining an airflow signal of the user; a microphone 112 for deriving an audio or acoustic signal of a user of the head-mountable wearable device 100; a frontal- looking camera 114 for measuring the head movement of the user so as to determine a user activity; an accelerometer 118 for continuously obtaining a motion signal of the user of the head-mountable wearable device 100 for identifying a user activity; an oximetry sensor 120 for measuring oxygen saturation in the blood of the user of the head-mountable wearable device 100; and a first wireless communication module 104 for wirelessly communicating with the remote device 200. The remote device 200 may include a processing unit 110 for determining the breathing pattern of the user based on the obtained airflow signal of the user, memory 102 for storing a predefined COPD severity rule database 124, an evaluation unit 122 for evaluating the COPD severity of the user by comparing the breathing pattern of the user with the predefined COPD severity rule database 124 to determine the COPD severity of the user, and a second wireless communication module 202 for communicating with the first wireless communication module 104.

Fig. 4A and Fig. 4B further schematically depicts an aspect of the head- mountable wearable device of Fig. 1. Fig. 4A shows a first method, also called OR method, of determining a breathing pattern of a user of the head-mountable wearable device 100 based on the signals obtained one or more sensors. In Fig. 4A, a set of sensors, e.g., video cameras 114, microphones 112, accelerometer/GPS 118, oximetry sensors 120, nasal air flow sensor 116, are shown. Signals sensed by each sensor 112, 114, 116, 118, 120 are processed independently in each processing unit 110. Each processing unit 110 has the capacity of processing for the associated sensor 112, 114, 116, 118, 120. Accordingly, an output 402 of each processing unit 110, e.g., alerts to the user and/or doctor/nurse, or the breathing pattern determined by each processing unit 110, may be generated independently for each sensor 112, 114, 116, 118, 120. Afterwards, the outputs 402 of each processing unit 1 10 may be transmitted to an integration processing unit 404 to generate an integrated output 406. The integrated output 406 of the integration processing unit 404 based on a correlation of the outputs 402 of one or more processing units 110 may be transmitted to the head-mountable wearable device 100 and/or to the remote device 200 to determine the breathing pattern of the user, or to memory 102 of the head-mountable wearable device 100 for storage. It should be understood that any combination of the outputs 402 of the sensors 112, 114, 116, 118, 120 are possible. Accordingly, the integrated output 406 may be generated based on the correlation of the outputs 402 received from any one or more processing units 110. For instance, the integration processing unit 404 may determine the breathing pattern of the user based on a correlation of the obtained airflow signal of the user sensed by the airflow sensor 116 and the audio signal of the user sensed by the microphone 112. Alternatively, the integration processing unit 404 may determine the breathing pattern of the user based on a correlation of the obtained airflow signal of the user sensed by the airflow sensor 116 and the motion signal of the user sensed by the accelerometer/GPS 118. Alternatively, the integration processing unit 404 may determine the breathing pattern of the user based on a correlation of the obtained airflow signal of the user sensed by the airflow sensor 116, the audio signal of the user sensed by the microphone 112 and the motion signal of the user sensed by the accelerometer/GPS 118. Note that the integration processing unit 404 may also determine the breathing pattern of the user based on a correlation of any one or more of the obtained signals sensed by the abovementioned sensors 112, 114, 116, 118, 120 and the historical breathing patterns of the user stored in the memory 102.

Fig. 4B shows a second method, also called AND method, of determining a breathing pattern of a user of the head-mountable wearable device 100 based on the signals obtained one or more sensors. In Fig. 4B, a set of sensors, e.g., video cameras 114, microphones 112, accelerometer/GPS 118, oximetry sensors 120, nasal air flow sensor 116, are shown. Signals sensed by each sensor 112, 114, 116, 118, 120 are transmitted to a centralized processing unit 110 that combines all received signals to generate a single output 412. The single output 412 may be generated based on a correlation of the signals sensed by each sensor 112, 114, 116, 118, 120. The single output 412 may be transmitted to the head- mountable wearable device 100 and/or to the remote device 200 to determine the breathing pattern of the user, or to memory 102 of the head-mountable wearable device 100 for storage. For instance, the single output 412 may be generated based on the correlation of the obtained airflow signal of the user sensed by the airflow sensor 116 and the audio signal of the user sensed by the microphone 112. Alternatively, the single output 412 may be generated based on the correlation of the obtained airflow signal of the user sensed by the airflow sensor 116 and the motion signal of the user sensed by the accelerometer/GPS 118. Alternatively, the single output 412 may be generated based on the correlation of the obtained airflow signal of the user sensed by the airflow sensor 116, the audio signal of the user sensed by the microphone 112 and the motion signal of the user sensed by the accelerometer/GPS 118.

Note that the centralized processing unit 110 may also determine the breathing pattern of the user based on a correlation of any one or more of the obtained signals sensed by the abovementioned sensors 112, 114, 116, 118, 120 and the historical breathing patterns of the user stored in the memory 102.

Fig. 5 shows an example of a COPD severity rule database 124. The COPD severity rule database 124 may be stored in memory 102 either located in the head-mountable wearable device 100 or located in the remote device 200. The COPD severity rule database 124 may include audio information 504, motion information 506, oximetry information 508, airflow information 510, historical information 512, and other information 514. Based on the specific value 520 of one or more of the above mentioned information, a breathing pattern value 530 is predefined.

In an embodiment, the evaluation unit 122 for evaluating the COPD severity of the user may apply a learned model associated with evaluation result to the determined breathing pattern of the user to determine whether the evaluation result is applicable to the user's current COPD severity or not. The evaluation unit 122 for evaluating the COPD severity of the user may receive feedback, such as manual input, from the user indicating the correctness of the evaluation result to the user's current COPD severity. For instance, the user may indicate that the evaluation result is indeed identical to his/her current breathing pattern, such as heavy coughing or light coughing. If the user indicates that the evaluation result is not identical to her/current breathing pattern, e.g. the evaluation result is a false positive, the evaluation unit 122 may update the COPD severity rule database 124 based on the received feedback and the determined breathing pattern of the user. Afterwards, the evaluation unit 122 may retrain the learned model based on the updated COPD severity rule database 124 using a machine learning process to verify whether the updated COPD severity rule database 124 is correct or not. The machine learning process may be a logistic regression algorithm. In some embodiment, the learned model may be adapted to the particular user by receiving feedback as to the accuracy of each given breathing pattern.

The processing unit 110 may implement a method 600 for detecting a breathing pattern as depicted in the flow chart of Fig. 6. The method 600 commences in step 601, e.g. initializing an airflow sensor. After that, the method progresses to step 603, i.e. continuously obtaining an airflow signal of the user by an airflow sensor, after which the method 600 progresses to step 605 in which the breathing pattern of the user is determined based on the obtained airflow signal of the user. In step 607, a predefined COPD severity rule database is stored in memory, after which a COPD severity of the user is evaluated by comparing the breathing pattern of the user with the predefined COPD severity rule database in step 609. The method 600 terminates in step 611.

It should be understood that the embodiment of the method 600 as shown in the flowchart of Fig. 6 is a non- limiting example of this embodiment, and that many alterations of this method may be made without departing from the present invention. For example, in step 601, other sensors such as the accelerometer or microphone may also be initiated. In step 603, the audio signal and/or the motion signal of the user may be derived in addition to the airflow signal. In step 605, the breathing pattern of the user may be determined based on a correlation of the obtained airflow signal, the motion signal and/or the audio signal, etc.

Aspects of the present invention may be embodied as a head-mountable wearable device, method or computer program product. Aspects of the present invention may take the form of a computer program product embodied in one or more computer-readable medium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Such a system, apparatus or device may be accessible over any suitable network connection; for instance, the system, apparatus or device may be accessible over a network for retrieval of the computer readable program code over the network. Such a network may for instance be the Internet, a mobile communications network or the like. More specific examples (a non- exhaustive list) of the computer readable storage medium may include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out the methods of the present invention by execution on the processing unit 110 may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the processing unit 110 as a stand-alone software package, e.g. an app, or may be executed partly on the processing unit 110 and partly on a remote server. In the latter scenario, the remote server may be connected to the wearable computing device 100 through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer, e.g. through the Internet using an Internet Service Provider.

Aspects of the present invention are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions to be executed in whole or in part on the processing unit 110 of the computing device 100, such that the instructions create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable medium that can direct the computing device 100 to function in a particular manner.

The computer program instructions may be loaded onto the processing unit 110 to cause a series of operational steps to be performed on the processing unit 110, to produce a computer-implemented process such that the instructions which execute on the processing unit 110 provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. The computer program product may form part of the head-mountable wearable device 100, e.g. may be installed on the head-mountable wearable device 100.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements. In the device claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

CLAIMS:

1. A head-mountable wearable device for detecting a breathing pattern of a user, comprising:

an airflow sensor for continuously obtaining an airflow signal of the user; a processing unit for determining the breathing pattern of the user based on the obtained airflow signal of the user,

an audio sensor configured for continuously obtaining an audio signal of the user,

wherein the processing unit is configured for determining the breathing pattern of the user based on a correlation of the obtained airflow signal of the user and the audio signal of the user.

2. The head-mountable wearable device of claim 1, wherein the processing unit is configured to determine:

a normal breathing pattern of the user, for instance, breathing pattern during sport activity or resting;

an abnormal breathing pattern of the user, for instance, breathing pattern under a lung disease such as coughing or Chronic Obstructive Pulmonary Disease (COPD), or breathing pattern under cardiovascular condition such as heart arrhythmia.

3. The head-mountable wearable device of claim 1, further comprises:

memory configured for storing a predefined COPD severity rule database; an evaluation unit for evaluating the COPD severity of the user by comparing the breathing pattern of the user with the predefined COPD severity rule database to determine the COPD severity of the user.

4. The head-mountable wearable device of any of preceding claims 1-3, further comprising an accelerometer for continuously obtaining a motion signal of the user for identifying a user activity.

5. The head-mountable wearable device of claim 4, wherein the processing unit is configured for determining the breathing pattern of the user based on a correlation of the obtained airflow signal of the user and the motion signal of the user.

6. The head-mountable wearable device of claim 5, wherein the processing unit is configured for determining the breathing pattern of the user based on a correlation of the obtained airflow signal of the user, the audio signal of the user and the motion signal of the user.

7. The head-mountable wearable device of claim 1, wherein the memory is configured for storing information indicating a historical breathing pattern of the user.

8. The head-mountable wearable device of claim 7, wherein the processing unit is configured for determining the breathing pattern of the user based on a correlation of the obtained airflow signal of the user, the audio signal of the user, the motion signal of the user, and the stored historical breathing patterns of the user.

9. The head-mountable wearable device of claim 3, wherein the evaluation unit for evaluating the COPD severity of the user is configured for applying a learned model associated with evaluation result to the determined breathing pattern of the user to determine whether the evaluation result is applicable to the user's current COPD severity or not.

10. The head-mountable wearable device of claim 9, wherein the evaluation unit for evaluating the COPD severity of the user is further configured for:

receiving feedback indicating the correctness of the evaluation result to the user's current COPD severity;

updating the COPD severity rule database based on the received feedback and the determined breathing pattern of the user;

- retraining the learned model based on the updated COPD severity rule database using a machine learning process.

11. A system for detecting a breathing pattern of a user, comprising:

an airflow sensor for continuously obtaining an airflow signal of the user; a processing unit for determining the breathing pattern of the user based on the obtained airflow signal of the user;

memory configured for storing a predefined COPD severity rule database; an evaluation unit for evaluating the COPD severity of the user by comparing the breathing pattern of the user with the predefined COPD severity rule database to determine the COPD severity of the user;

wherein the system comprises a head-mountable wearable device including the airflow sensor; the head-mountable wearable device further comprising a first wireless

communication module;

and a remote device including the processing unit, the memory, the evaluation unit, and a second wireless communication module for communicating with the first wireless communication module.

12. A computer-implemented method for detecting a breath pattern of a user, comprising:

continuously obtaining an airflow signal of the user by an airflow sensor; continuously obtaining an audio signal of the user by an audio sensor- determining the breathing pattern of the user based on a correlation of the obtained airflow signal of the user and the audio signal of the user;

- storing a predefined COPD severity rule database;

evaluating a COPD severity of the user by comparing the breathing pattern of the user with the predefined COPD severity rule database.

13. Computer program comprising program code means for causing a computer to carry out the steps of the method as claimed in claims 8-12 when said computer program is carried out on the computer.