Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Automatic Detection of Visual Search for the Elderly using Eye and Head Tracking Data

  • Technical Contribution
  • Published:
KI - Künstliche Intelligenz Aims and scope Submit manuscript

Abstract

With increasing age we often find ourselves in situations where we search for certain items, such as keys or wallets, but cannot remember where we left them before. Since finding these objects usually results in a lengthy and frustrating process, we propose an approach for the automatic detection of visual search for older adults to identify the point in time when the users need assistance. In order to collect the necessary sensor data for the recognition of visual search, we develop a completely mobile eye and head tracking device specifically tailored to the requirements of older adults. Using this device, we conduct a user study with 30 participants aged between 65 and 80 years (\(avg = 71.7,\) 50% female) to collect training and test data. During the study, each participant is asked to perform several activities including the visual search for objects in a real-world setting. We use the recorded data to train a support vector machine (SVM) classifier and achieve a recognition rate of 97.55% with the leave-one-user-out evaluation method. The results indicate the feasibility of an approach towards the automatic detection of visual search in the wild.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. http://eyeinfo.itu.dk.

References

  1. Altun K, Barshan B (2010) Human activity recognition using inertial/magnetic sensor units. In: Proc. Human-behavior understanding. Lecture Notes in Computer Science, vol 6219. Springer, pp 38–51

  2. Bao L, Intille SS (2004) Activity recognition from user-annotated acceleration data. In: Proc. Pervasive computing. Lecture Notes in Computer Science, vol 3001. Springer, pp 1–17

  3. Beatty J (1982) Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychol Bull 91(2):276

    Article  Google Scholar 

  4. Bixler R, D’Mello S (2014) Toward fully automated person-independent detection of mind wandering. In: Proc. User modeling, adaptation, and personalization. Lecture Notes in Computer Science, vol 8538. Springer, pp 37–48

  5. Bulling A, Ward JA, Gellersen H, Troster G (2011) Eye movement analysis for activity recognition using electrooculography. IEEE Trans Pattern Anal Mach intell 33(4):741–753

    Article  Google Scholar 

  6. Buswell GT (1935) How people look at pictures: a study of the psychology and perception in art. University of Chicago Press, Chicago

    Google Scholar 

  7. Castelhano MS, Mack ML, Henderson JM (2009) Viewing task influences eye movement control during active scene perception. J Vis 9(3):6.115

    Article  Google Scholar 

  8. Chen YP, Yang JY, Liou SN, Lee GY, Wang JS (2008) Online classifier construction algorithm for human activity detection using a tri-axial accelerometer. Appl Math Comput 205(2):849–860

    MathSciNet  Google Scholar 

  9. Coco MI, Keller F (2014) Classification of visual and linguistic tasks using eye-movement features. J Vis 14(3):11

    Article  Google Scholar 

  10. Damian I, Baur T, André E (2016) Measuring the impact of behavioural feedback loops on social interactions. In: Proc. ICMI. ACM, pp 201–208

  11. Damian I, Dietz M, Gaibler F, André E (2016) Social signal processing for dummies. In: Proc. ICMI. ACM, pp 394–395

  12. Dietz M, Garf ME, Damian I, André E (2016) Exploring eye-tracking-driven sonification for the visually impaired. In: Proc. AH. ACM, pp. 5:1–5:8

  13. Dietz M, Schork D, André E (2016) Exploring eye-tracking-based detection of visual search for elderly people. In: Proc. IE. IEEE, pp. 151–154

  14. Findlay JM, Gilchrist ID (1998) Eye guidance and visual search. In: Eye guidance in reading and scene perception. Elsevier, pp 295–312

  15. Greene MR, Liu T, Wolfe JM (2012) Reconsidering yarbus: a failure to predict observers’ task from eye movement patterns. Vis Res 62:1–8

    Article  Google Scholar 

  16. Haji-Abolhassani A, Clark JJ (2013) A computational model for task inference in visual search. J Vis 13(3):29

    Article  Google Scholar 

  17. Hanai Y, Nishimura J, Kuroda T (2009) Haar-like filtering for human activity recognition using 3d accelerometer. In: 13th Digital Signal Processing Workshop. IEEE, pp 675–678

  18. He Z, Jin L (2009) Activity recognition from acceleration data based on discrete consine transform and svm. In: Proc. SMC. IEEE, pp 5041–5044

  19. Henderson JM, Shinkareva SV, Wang J, Luke SG, Olejarczyk J (2013) Predicting cognitive state from eye movements. PloS One 8(5):e64,937

    Article  Google Scholar 

  20. Huynh T, Schiele B (2005) Analyzing features for activity recognition. In: Proc. Smart objects and ambient intelligence. ACM, pp 159–163

  21. Kächele M, Werner P, Al-Hamadi A, Palm G, Walter S, Schwenker F (2015) Bio-visual fusion for person-independent recognition of pain intensity. In: Proc. Multiple classifier systems. Lecture Notes in Computer Science, vol 9132. Springer, pp 220–230

  22. Khan AM, Lee YK, Lee SY (2010) Accelerometer’s position free human activity recognition using a hierarchical recognition model. In: Proc. Healthcom. IEEE, pp 296–301

  23. Khan AM, Lee YK, Lee SY, Kim TS (2010) A triaxial accelerometer-based physical-activity recognition via augmented-signal features and a hierarchical recognizer. IEEE T-ITB 14(5):1166–1172

    Google Scholar 

  24. Kittler J, Hatef M, Duin RPW, Matas J (1998) On combining classifiers. IEEE Trans Pattern Anal Mach Intell 20(3):226–239

    Article  Google Scholar 

  25. Knauer U, Seiffert U (2013) A comparison of late fusion methods for object detection. In: Proc. ICIP, pp 3297–3301

  26. Lara OD, Labrador MA (2013) A survey on human activity recognition using wearable sensors. IEEE Commun Surv Tutor 15(3):1192–1209

    Article  Google Scholar 

  27. Lingenfelser F, Wagner J, André E (2011) A systematic discussion of fusion techniques for multi-modal affect recognition tasks. In: Proc. ICMI. ACM, pp 19–26

  28. Lingenfelser F, Wagner J, Vogt T, Kim J, André E (2010) Age and gender classification from speech using decision level fusion and ensemble based techniques. In: Proc. INTERSPEECH, pp 2798–2801

  29. Lu H, Pan W, Lane ND, Choudhury T, Campbell AT (2009) Soundsense: scalable sound sensing for people-centric applications on mobile phones. In: Proc. MobiSys. ACM, pp 165–178

  30. Mills M, Hollingworth A, van der Stigchel S, Hoffman L, Dodd MD (2011) Examining the influence of task set on eye movements and fixations. J Vis 11(8):17

    Article  Google Scholar 

  31. Partala T, Surakka V (2003) Pupil size variation as an indication of affective processing. Int J Hum-Comput Stud 59(1):185–198

    Article  Google Scholar 

  32. Rahman SA, Merck C, Huang Y, Kleinberg S (2015) Unintrusive eating recognition using google glass. In: Proc. PervasiveHealth. ICST, pp 108–111

  33. Ravi N, Dandekar N, Mysore P, Littman ML (2005) Activity recognition from accelerometer data. In: Proc. Innovative applications of artificial intelligence, vol 3. AAAI Press, pp 1541–1546

  34. Sadasivan S, Greenstein JS, Gramopadhye AK, Duchowski AT (2005) Use of eye movements as feedforward training for a synthetic aircraft inspection task. In: Proc. CHI. ACM, pp 141–149

  35. Salthouse TA, Babcock RL (1991) Decomposing adult age differences in working memory. Dev Psychol 27(5):763

    Article  Google Scholar 

  36. Snoek CGM, Worring M, Smeulders AWM (2005) Early versus late fusion in semantic video analysis. In: Proc. MULTIMEDIA. ACM, pp 399–402

  37. Torralba A, Oliva A, Castelhano MS, Henderson JM (2006) Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. Psychol Rev 113(4):766–786

    Article  Google Scholar 

  38. Treisman AM, Gelade G (1980) A feature-integration theory of attention. Cognit Psychol 12(1):97–136

    Article  Google Scholar 

  39. Verghese P (2001) Visual search and attention: A signal detection theory approach. Neuron 31(4):523–535

    Article  Google Scholar 

  40. Wagner J, Lingenfelser F, Baur T, Damian I, Kistler F, André E (2013) The social signal interpretation (ssi) framework: multimodal signal processing and recognition in real-time. In: Proc. MM. ACM, pp 831–834

  41. Webb AR (2002) Statistical pattern recognition. Wiley, New York

    Book  MATH  Google Scholar 

  42. Weyerer S, Bickel H (2007) Epidemiologie psychischer Erkrankungen im höheren Lebensalter. Kohlhammer

  43. Yarbus AL (1967) Eye movements during perception of complex objects. Springer, New York

    Book  Google Scholar 

  44. Yatani K, Truong KN (2012) Bodyscope: a wearable acoustic sensor for activity recognition. In: Proc. UbiComp. ACM, pp 341–350

Download references

Acknowledgements

This work was partially funded by the German Federal Ministry of Education and Research (BMBF) under Project Glassistant (FKZ 16SV7270).

Author information

Authors and Affiliations

  1. Human Centered Multimedia, Augsburg University, Universitätsstraße 6a, 86159, Augsburg, Germany

    Michael Dietz, Daniel Schork, Ionut Damian & Elisabeth André

  2. Geriatrics Research Group, Charité-Universitätsmedizin Berlin, Reinickendorfer Straße 61, 13347, Berlin, Germany

    Anika Steinert & Marten Haesner

Authors
  1. Michael Dietz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel Schork
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ionut Damian
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anika Steinert
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marten Haesner
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Elisabeth André
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Dietz.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dietz, M., Schork, D., Damian, I. et al. Automatic Detection of Visual Search for the Elderly using Eye and Head Tracking Data. Künstl Intell 31, 339–348 (2017). https://doi.org/10.1007/s13218-017-0502-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13218-017-0502-z

Keywords

Search

Navigation