Abstract
A personal visual lifelog can be considered to be a human memory augmentation tool and in recent years we have noticed an increased interest in the topic of lifelogging both in academic research and from industry practitioners. In this preliminary work, we explore the concept of event segmentation of visual lifelog data. Lifelog data, by its nature is continual and streams of multimodal data can easy run into thousands of wearable camera images per day, along with a significant number of other sensor sources. In this paper, we present two new approaches to event segmentation and compare them against pre-existing approaches in a user experiment with ten users. We show that our approaches based on visual concepts occurrence and image categorization perform better than the pre-existing approaches. We finalize the paper with a suggestion for next steps for the research community.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Given a 16 h day, with one image per minute and 30 events identified per day (all reasonable assumptions), then an evaluation with a 16 min boundary would tend not to penalize random segmentation algorithms.
References
Microsoft’s Computer Vision API. https://docs.microsoft.com/en-us/azure/cognitive-services/computer-vision/category-taxonomy
Bell, G., Gemmell, J.: A digital life. Sci. Am. 296, 58–65 (2007)
Bolanos, M., Mestre, R., Talavera, E., Nieto, X.G., Radeva, P.: Visual summary of egocentric photostreams by representative keyframes. In: IEEE First International Workshop on Wearable and Ego-Vision Systems for Augmented Experience (WEsAX), 29 June–3 July 2015, Turin, Italy (2015)
Bush, V.: As we may think. Atlantic Mon. 176(1), 101–108 (1945)
Byrne, D., Lavelle, B., Doherty, A.R., Jones, G.J., Smeaton, A.F.: Using bluetooth and GPS metadata to measure event similarity in SenseCam images. In: 5th International Conference on Intelligent Multimedia and Ambient Intelligence, July 2007
Chen, Y., Jones, G.J., Ganguly, D.: Segmenting and summarizing general events in a long-term lifelog. In: The 2nd Workshop Information Access for Personal Media Archives (IAPMA) at ECIR 2011, April 2011
Chu, H.Y., Trujillo, R.G.: New views on R. Buckminster Fuller, pp. 6–23 (2009)
Doherty, A.R., Smeaton, A.F.: Automatically segmenting lifelog data into events. In: 9th International Workshop on Image Analysis for Multimedia Interactive Services, 30 June 2008
Doherty, A.R., Smeaton, A.F., Lee, K., Ellis, D.P.: Multimodal segmentation of lifelog data. In: Large Scale Semantic Access to Content (Text, Image, Video, and Sound), pp. 21–38, June 2007
Fang, H., Gupta, S., Iandola, F., Srivastava, R., Deng, L., Dollar, P., Gao, J., He, X., Mitchell, M., Platt, J., Zitnick, L., Zweig, G.: From captions to visual concepts and back. IEEE Institute of Electrical and Electronics Engineers, June 2015
Gargi, U.: Modeling and clustering of photo capture streams. In: Proceedings of the 5th ACM SIGMM International Workshop on Multimedia Information Retrieval, pp. 47–54, November 2003
Gemmell, J., Bell, G., Lueder, R.: MyLifeBits: a personal database for everything. Commun. ACM 49(1), 88–95 (2006)
Gurrin, C., Byrne, D., O’Connor, N., Jones, G.J., Smeaton, A.F.: Architecture and challenges of maintaining a large-scale, context-aware human digital memory. In: VIE 2008 - The 5th IET Visual Information Engineering 2008 Conference, July 2018
Gurrin, C., Joho, H., Hopfgartner, F., Zhou, L., Albatal, R.: NTCIR Lifelog: The First Test Collection for Lifelog Research (2016)
Gurrin, C., Smeaton, A.F., Doherty, A.R.: LifeLogging: personal big data. Found. Trends Inf. Retrieval 8(1), 1–125 (2014)
Hinbarji, Z., Albatal, R., O’Connor, N., Gurrin, C.: LoggerMan, a comprehensive logging and visualization tool to capture computer usage. In: Tian, Q., Sebe, N., Qi, G.-J., Huet, B., Hong, R., Liu, X. (eds.) MMM 2016. LNCS, vol. 9517, pp. 342–347. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-27674-8_31
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093 (2014)
Li, Z., Wei, Z., Jia, W., Sun, M.: Daily life event segmentation for lifestyle evaluation based on multi-sensor data recorded by a wearable device. In: Conference on Proceedings of IEEE Engineering in Medicine and Biology Society, 30 October 2013
Zacks, M.J., Braver, S.T., Sheridan, A.M., Donaldson, I.D., Snyder, Z.A., Ollinger, M.J., Buckner, L.R., Raichle, E.M.: Human brain activity time-locked to perceptual event boundaries. Nat. Neurosci. 4(6), 651–655 (2001)
Acknowledgment
This publication has emanated from research conducted with the financial support of Science Foundation Ireland (SFI) under grant number SFI/12/RC/2289.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Gupta, R., Gurrin, C. (2018). Approaches for Event Segmentation of Visual Lifelog Data. In: Schoeffmann, K., et al. MultiMedia Modeling. MMM 2018. Lecture Notes in Computer Science(), vol 10704. Springer, Cham. https://doi.org/10.1007/978-3-319-73603-7_47
Download citation
DOI: https://doi.org/10.1007/978-3-319-73603-7_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-73602-0
Online ISBN: 978-3-319-73603-7
eBook Packages: Computer ScienceComputer Science (R0)