Inferring Human Activity in Mobile Devices by Computing Multiple Contexts
<p>Structure of the Naïve Bayes classifier used for classifying significant activities.</p> "> Figure 2
<p>An empirical probability distribution of the observable local time given the activity of “having a lunch”.</p> "> Figure 3
<p>Diagram of the procedure for inferring significant activities. The probability model can be either from user input (empirical model) or from training process using labeled activities.</p> "> Figure 4
<p>The trajectories of the indoor positioning solution with a Samsung Galaxy Note 3 smartphone and the ground truth generated by the NovAtel SPAN-IGM-S1 system.</p> "> Figure 5
<p>Horizontal positioning error statistics. (<b>a</b>) illustrates the horizontal positioning error in two different zoom levels and (<b>b</b>) shows the histogram, cumulative distribution curve and other related statistics.</p> "> Figure 6
<p>The significant locations used in the experiment. The number tags of these locations are defined in <a href="#sensors-15-21219-t003" class="html-table">Table 3</a>.</p> "> Figure 7
<p>Poses of the mobile devices during data logging.</p> ">
Abstract
:1. Introduction
- (1)
- A spatial context, which is a geofence that can be a node associated with a circle, or a polygon;
- (2)
- A temporal context, which can be the local time, a time difference between two geographical locations, or a timespan;
- (3)
- A set of spatiotemporal contexts, which are location-dwelling lengths;
- (4)
- A set of user contexts, which theoretically form a combination of user mobility contexts (e.g., static, walking, running, and driving), user environmental contexts (e.g., lighting condition, noise level, and weather conditions), user psychological contexts (e.g., levels of fatigue, excitement, and nervousness) and user social contexts (e.g., calling, texting/chatting, and using Apps). Within the scope of this study, only user mobility contexts are considered. However, the proposed framework is flexible for adopting other user contexts.
- (1)
- A more comprehensive literature review is added;
- (2)
- A new section describing our ubiquitous positioning solution is added. A dedicated experiment is conducted to assess the positioning performance in a typical indoor environment;
- (3)
- The activity inference model has been massively upgraded by adapting multiple spatiotemporal contexts and user contexts into the contextual tuple;
- (4)
- The live experiment has been enhanced with a week-long dataset including 710,437 labeled activities collected by different individuals.
2. Related Works
3. Research Methods
- (1)
- Raw data collection including measurements obtained from smartphone sensors and radio receivers;
- (2)
- Real-time computation of the
- temporal context by recording the local time,
- spatial context by locating the user,
- spatiotemporal contexts by counting the dwelling length at each significant location, and
- user contexts by determining various user states including user motion states, user environmental states, user psychological states, and user social states;
- (3)
- Inference of the significant activity.
3.1. Determination of the Spatial Contexts
3.2. Determination of the Spatiotemporal Contexts
Observable | Description |
---|---|
1 | Dwelling length between 0 and 5 s |
2 | Dwelling length between 6 and 15 s |
3 | Dwelling length between 16 and 60 s |
3.3. Determination of User Contexts
- (1)
- User mobility contexts such as motion patters;
- (2)
- User psychological contexts such as levels of fatigue, excitement, nervousness, and depression;
- (3)
- User environmental contexts such as ambient noise level, light intensity, temperature, and weather conditions;
- (4)
- User social contexts such as calling, messaging/chatting, and using applications.
Category | User Context | Observable Set of Each User context |
---|---|---|
Mobility | Motion pattern | static, slow walking, walking, fast moving |
Environment | Light intensity | low, normal, high |
Noise level | low, normal, high | |
Temperature | freeze, low, comfortable, high | |
Weather | sunny, cloudy, raining, hazardous weather | |
Psychology | Level of fatigue | low, medium, high |
Level of excitement | low, medium, high | |
Level of nervousness | low, medium, high | |
Level of depression | low, medium, high | |
Social | Social contexts | calling, texting/chatting, using App |
3.4. Activity Inference
- The probability distributions of the significant activities including an undefined activity: A = [p(A1), p(A2), ..., P(Ana)], with ,
- The observation probability matrix B for all p(xi|Ak), where i = 1,…,d, and k =1,...,na. B is a d × na dimensional matrix of probability density functions (PDFs). Each PDF is a histogram with various bins depending on the size of the sample set of the corresponding observable, e.g. the size of the sample set of a user mobility context [static, slow walking, walking, fast moving] is four. For each PDF, we have , where Pj is the probability of the jth element in the sample set, and nb is the size of the sample set.
- (1)
- A computationally efficient approach that can be implemented in smartphones;
- (2)
- Flexible in supporting incremental learning;
- (3)
- Insensitive to irrelevant features.
4. Experiments and Data Analysis
4.1. Evaluation of the Indoor Positioning Accuracy
4.2. Assessment of the Activity Inference
4.2.1. Significant Locations
Location-ID | Description |
---|---|
1 | Office |
2 | Meeting room |
3 | Kitchen |
4 | Coffee Break Area |
5 | Library |
6 | Classroom |
7 | Bus Stop |
8 | Undefined Location |
4.2.2. Significant Activities
Activity-ID | Description | Probability |
---|---|---|
1 | Working | 0.3333 |
2 | Having a meeting | 0.0208 |
3 | Having a lunch | 0.0417 |
4 | Taking a coffee break | 0.0208 |
5 | Visiting library | 0.0208 |
6 | Taking a class | 0.0573 |
7 | Waiting for bus | 0.0053 |
8 | Other activities (undefined activities) | 0.5000 |
Total | 1.0000 |
4.2.3. Data Logging
- (1)
- Recording the local time tag;
- (2)
- Locating the mobile users ubiquitously indoors/outdoors using GPS, built-in sensors, and WiFi signals;
- (3)
- Determining the location-dwelling length for each significant location;
- (4)
- Determining the user mobility context using the built-in accelerometer;
- (5)
- Logging the time series of the contextual tuples and labeling the real activity for each second.
4.3. Data Analysis
ID | Description |
---|---|
1 | static, speed <= 0.1 m/s |
2 | slow walking, 0.1 m/s < speed <= 0.7 m/s (less than one step per second) |
3 | walking, 0.7 m/s < speed <= 1.4 m/s (1–2 steps per second) |
4 | fast moving, speed > 1.4 m/s (more then 2 steps per second, or driving) |
Solution# | Supervised/Unsupervised | Multi-context/Spatial-Context-Only | Probability Model |
---|---|---|---|
1 | Unsupervised a | Multi-context | Empirical |
2 | Supervised a | Multi-context b | Trained with multiple contexts |
3 | Supervised | Spatial-context-only b | Trained with spatial context only |
4.3.1. Comparison between the Supervised and Unsupervised Solutions
4.3.2. Comparison between the Multi-Context and Spatial-Context-Only Solutions
Participant | Solutions | Number of Labeled Activities | |
---|---|---|---|
Unsupervised (Solution 1) | Supervised (Solution 2) | ||
1 | 66.5% | 88.9% | 237,085 |
2 | 50.3% | 87.9% | 211,834 |
3 | 57.2% | 89.6% | 261,517 |
Mean | 58.0% | 88.8% | 236,812 |
Participant | Solutions | Number of Labeled Activities | |
---|---|---|---|
Spatial-Context-Only (Solution 3) | Multi-Context (Solution 2) | ||
1 | 64.0% | 88.9% | 237,085 |
2 | 65.7% | 87.9% | 211,834 |
3 | 55.4% | 89.6% | 261,517 |
Mean | 61.7% | 88.8% | 236,812 |
5. Conclusions and Future Works
Acknowledgments
Author Contributions
Conflicts of Interest
References
- Google Now. Available online: http://www.google.com/landing/now/ (accessed on 26 June 2015).
- Dey, A.K.; Abowd, G.D. Towards a better understanding of context and context-awareness. In Proceedings of the Workshop on the What, Who, Where, When and How of Context Awareness, The Hague, The Netherlands, 3 April 2000; pp. 19–22.
- Lara, O.D.; Labrador, M.A. A Survey on Human Activity Recognition Using Wearable Sensors. IEEE Commun. Surv. Tutor. 2013, 15, 1192–1209. [Google Scholar] [CrossRef]
- Chen, R.; Chu, T.; Xu, W.; Li, X.; Liu, J.; Chen, Y.; Chen, L.; Hyyppa, J.; Tang, J. Development of a contextual thinking engine in mobile devices. In Proceedings of the IEEE UPINLBS 2014, Corpus Christi, TX, USA, 2–5 November 2014; pp. 90–96.
- Chen, R.; Guinness, R. Geospatial Computing in Mobile Device; Artech House Publishers: Norwood, MA, USA, 2014. [Google Scholar]
- Chen, R. Ubiquitous Positioning and Mobile Location-Based Services in Smart Phones; IGI Global: Hershey, PA, USA, 2012. [Google Scholar]
- Zandbergen, P.A. Accuracy of iPhone Locations: A Comparison of Assisted GPS, WiFi and Cellular Positioning. Trans. GIS 2009, 13, 5–25. [Google Scholar] [CrossRef]
- Chen, R.; Pei, L.; Chen, Y. A smart phone based PDR solution for indoor navigation. In Proceedings of the 24th International Technical Meeting of the Satellite Division of the Institute of Navigation, Portland, OR, USA, 20–23 September 2011; pp. 1404–1408.
- Chen, W.; Chen, R.; Chen, X. Comparison of EMG-based and Accelerometer-based Speed Estimation Methods in Pedestrian Dead Reckoning. J. Navig. 2011, 64, 265–280. [Google Scholar] [CrossRef]
- Liu, J.; Chen, R.; Pei, L.; Guinness, R.; Kuusniemi, H. A Hybrid Smartphone Indoor Positioning Solution for Mobile LBS. Sensors 2012, 12, 17208–17233. [Google Scholar] [CrossRef] [PubMed]
- Bhuiyan, M.Z.H.; Kuusniemi, H.; Chen, L.; Pei, L.; Ruotsalainen, L.; Guinness, R.; Chen, R. Performance Evaluation of Multi-Sensor Fusion Models in Indoor Navigation. Eur. J. Navig. 2013, 11, 20–28. [Google Scholar]
- Zhao, S.; Yao, Z.; Yin, J.; Lu, M. Application of extended and unscented Kalman filtering for non-linear positioning in nearby region of navigation sources. In Proceedings of the 2015 International Technical Meeting of the Institute of Navigation, Dana Point, CA, USA, 26–28 January 2015; pp. 640–650.
- Pei, L.; Chen, R.; Liu, J.; Kuusniemi, H.; Tenhunen, T.; Chen, Y. Using Inquiry-based Bluetooth RSSI Probability Distributions for Indoor Positioning. J. Glob. Position. Syst. 2010, 9, 122–130. [Google Scholar]
- Bahl, P.; Padmanabhan, V.N. Radar: An in-building RF-based user location and tracking system. In Proceedings of the IEEE International Conference on Computer Communications, Tel Aviv, Israel, 26–30 March 2000; Volume 2, pp. 775–784.
- Holder, L.B.; Cook, D.J. Automated Activity-aware Prompting for Activity Initiation. Gerontechnology 2013, 11, 534–544. [Google Scholar] [CrossRef] [PubMed]
- Marszalek, M.; Laptev, I.; Schmid, C. Actions in context. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 2929–2936.
- Ravi, N.; Dandekar, N.; Mysore, P.; Littman, M.L. Activity recognition from accelerometer data. In Proceedings of the 17th Conference on Innovative Applications of Artificial Intelligence, Pittsburgh, PA, USA, 9–13 July 2005; pp. 1541–1546.
- Susi, M.; Renaudin, V.; Lachapelle, G. Motion Mode Recognition and Step Detection Algorithms for Mobile Phone Users. Sensors 2013, 13, 1539–1562. [Google Scholar] [CrossRef] [PubMed]
- Chen, Y.; Chen, R.; Pei, L.; Kroeger, T.; Chen, W.; Kuusniemi, H.; Liu, J. Knowledge-Based Error Detection and Correction Method of a Multi-Sensor Multi-Network Positioning Platform for Pedestrian Indoor Navigation. In Proceedings of the IEEE/ION PLANS 2010 Conference, Palm Springs, CA, USA, 4–6 May 2010; pp. 873–879.
- Pei, L.; Guinness, R.; Chen, R.; Liu, J.; Kuusniemi, H.; Chen, Y.; Chen, L.; Kaistinen, J. Human Behavior Cognition Using Smartphone Sensors. Sensors 2013, 13, 1402–1424. [Google Scholar] [CrossRef] [PubMed]
- Reddy, S.; Mun, M.; Burke, J.; Estrin, D.; Hansen, M.; Srivastava, M. Using Mobile Phones to Determine Transportation Modes. ACM Trans. Sens. Netw. 2010, 6, 1–27. [Google Scholar] [CrossRef]
- Bancroft, J.; Garrett, D.; Lachapelle, G. Activity and environment classification using foot mounted navigation sensors. In Proceedings of the International Conference on Indoor Positioning and Indoor Navigation, Sydney, Australia, 13–15 November 2012.
- Jin, G.H.; Lee, S.B.; Lee, T.S. Context Awareness of Human Motion States Using Accelerometer. J. Med. Syst. 2008, 32, 93–100. [Google Scholar] [CrossRef] [PubMed]
- Liu, J.; Zhu, L.; Wang, Y.; Liang, X.; Hyyppä, J.; Chu, T.; Liu, K.; Chen, R. Reciprocal Estimation of Pedestrian Location and Motion State toward a Smartphone Geo-Context Computing Solution. Micromachines 2015, 6, 699–717. [Google Scholar] [CrossRef]
- Guinness, R.E. Beyond Where to How: A Machine Learning Approach for Sensing Mobility Contexts Using Smartphone Sensors. Sensors 2015, 15, 9962–9985. [Google Scholar] [CrossRef] [PubMed]
- Wan, E.A.; van der Merwe, R. The unscented Kalman filter for nonlinear estimation. In Proceedings of the Adaptive Systems for Signal Processing, Communications, and Control Symposium, Lake Louise, AB, Canada, 1–4 October 2000; pp. 153–158.
- SPAN-IGM-S1. Available online: http://www.novatel.com/assets/Documents/Papers/SPAN-IGM-S1-PS.pdf (accessed on 26 June 2015).
- Hall, M.; Frank, E.; Holmes, G.; Pfahringer, B.; Reutemann, P.; Witten, I.H. The WEKA Data Mining Software: An Update. ACM SIGKDD Explor. Newsl. 2009, 11, 10–18. [Google Scholar] [CrossRef]
© 2015 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Chen, R.; Chu, T.; Liu, K.; Liu, J.; Chen, Y. Inferring Human Activity in Mobile Devices by Computing Multiple Contexts. Sensors 2015, 15, 21219-21238. https://doi.org/10.3390/s150921219
Chen R, Chu T, Liu K, Liu J, Chen Y. Inferring Human Activity in Mobile Devices by Computing Multiple Contexts. Sensors. 2015; 15(9):21219-21238. https://doi.org/10.3390/s150921219
Chicago/Turabian StyleChen, Ruizhi, Tianxing Chu, Keqiang Liu, Jingbin Liu, and Yuwei Chen. 2015. "Inferring Human Activity in Mobile Devices by Computing Multiple Contexts" Sensors 15, no. 9: 21219-21238. https://doi.org/10.3390/s150921219
APA StyleChen, R., Chu, T., Liu, K., Liu, J., & Chen, Y. (2015). Inferring Human Activity in Mobile Devices by Computing Multiple Contexts. Sensors, 15(9), 21219-21238. https://doi.org/10.3390/s150921219