Abstract
This paper presents an overview of the ImageCLEF 2019 lab, organized as part of the Conference and Labs of the Evaluation Forum - CLEF Labs 2019. ImageCLEF is an ongoing evaluation initiative (started in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2019, the 17th edition of ImageCLEF runs four main tasks: (i) a medical task that groups three previous tasks (caption analysis, tuberculosis prediction, and medical visual question answering) with new data, (ii) a lifelog task (videos, images and other sources) about daily activities understanding, retrieval and summarization, (iii) a new security task addressing the problems of automatically identifying forged content and retrieve hidden information, and (iv) a new coral task about segmenting and labeling collections of coral images for 3D modeling. The strong participation, with 235 research groups registering, and 63 submitting over 359 runs, shows an important interest in this benchmark campaign.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
There was a limit of maximum 10 run submissions per team. The table includes only the valid runs that were graded (total# 80 out of 90 submissions).
- 9.
- 10.
Note that the question category-wise accuracy scores are normalized (each divided by a factor of 4) so that the summation is equal to the overall accuracy.
- 11.
- 12.
Last.FM Music Tracker and Recommender - https://www.last.fm/.
- 13.
Fitbit Fitness Tracker (FitBit Versa) - https://www.fitbit.com.
- 14.
Freestyle Libre wearable glucose monitor - https://www.freestylelibre.ie/.
- 15.
Moves App for Android and iOS - http://www.moves-app.com/.
References
Abdallah, F.B., Feki, G., Ammar, A.B., Amar, C.B.: Big data for lifelog moments retrieval improvement. In: CLEF2019 Working Notes. CEUR Workshop Proceedings, 09–12 September 2019, Lugano, Switzerland. CEUR-WS.org (2019). http://ceur-ws.org
Abacha, A.B., Hasan, S.A., Datla, V.V., Liu, J., Demner-Fushman, D., Müller, H.: VQA-Med: overview of the medical visual question answering task at ImageCLEF 2019. In: CLEF2019 Working Notes. CEUR Workshop Proceedings, 09–12 September 2019, Lugano, Switzerland. CEUR-WS.org (2019). http://ceur-ws.org
Birkeland, C.: Global status of coral reefs: in combination, disturbances and stressors become ratchets. In: World Seas: An Environmental Evaluation, pp. 35–56. Elsevier (2019)
Bodenreider, O.: The unified medical language system (UMLS): integrating biomedical terminology. Nucleic Acids Res. 32(Database–Issue), 267–270 (2004). https://doi.org/10.1093/nar/gkh061
Brander, L.M., Rehdanz, K., Tol, R.S., Van Beukering, P.J.: The economic impact of ocean acidification on coral reefs. Climate Change Econ. 3(01), 1250002 (2012)
Bullimore, R.D., Foster, N.L., Howell, K.L.: Coral-characterized benthic assemblages of the deep northeast atlantic: defining “coral gardens" to support future habitat mapping efforts. ICES J. Mar. Sci. 70(3), 511–522 (2013)
Chamberlain, J., Campello, A., Wright, J.P., Clift, L.G., Clark, A., de Herrera, A.G.S.: Overview of ImageCLEFcoral 2019 task. In: CLEF2019 Working Notes. CEUR Workshop Proceedings. CEUR-WS.org (2019)
Clough, P., Müller, H., Sanderson, M.: The CLEF 2004 cross-language image retrieval track. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 597–613. Springer, Heidelberg (2005). https://doi.org/10.1007/11519645_59
Clough, P., Sanderson, M.: The CLEF 2003 cross language image retrieval track. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 581–593. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30222-3_56
Dang-Nguyen, D.T., Piras, L., Riegler, M., Tran, M.T., Zhou, L., Lux, M., Le, T.K., Ninh, V.T., Gurrin, C.: Overview of ImageCLEFlifelog 2019: solve my life puzzle and lifelog moment retrieval. In: CLEF2019 Working Notes. CEUR Workshop Proceedings, 09–12 September 2019, Lugano, Switzerland. CEUR-WS.org (2019)
Dao, M.S., Vo, A.K., Phan, T.D., Zettsu, K.: BIDAL@imageCLEFlifelog2019: the role of content and context of daily activities in insights from lifelogs. In: CLEF2019 Working Notes. CEUR Workshop Proceedings, 09–12 September 2019, Lugano, Switzerland. CEUR-WS.org (2019). http://ceur-ws.org
Cid, Y.D., Jimenez-del-Toro, O., Depeursinge, A., Müller, H.: Efficient and fully automatic segmentation of the lungs in CT volumes. In: Goksel, O., Jimenez-del-Toro, O., Foncubierta-Rodriguez, A., Müller, H. (eds.) Proceedings of the VISCERAL Challenge at ISBI, pp. 31–35. No. 1390 in CEUR Workshop Proceedings, April 2015
Cid, Y.D., Kalinovsky, A., Liauchuk, V., Kovalev, V., Müller, H.: Overview of ImageCLEFtuberculosis 2017 - predicting tuberculosis type and drug resistances. In: CLEF2017 Working Notes. CEUR Workshop Proceedings, 11–14 September 2017, Dublin, Ireland. CEUR-WS.org (2017). http://ceur-ws.org
Cid, Y.D., Liauchuk, V., Klimuk, D., Tarasau, A., Kovalev, V., Müller, H.: Overview of ImageCLEFtuberculosis 2019 - automatic CT-based report generation and tuberculosis severity assessment. In: CLEF2019 Working Notes. CEUR Workshop Proceedings, 09–12 September 2019, Lugano, Switzerland. CEUR-WS.org (2019). http://ceur-ws.org
Cid, Y.D., Liauchuk, V., Kovalev, V., Müller, H.: Overview of ImageCLEFtuberculosis 2018 - detecting multi-drug resistance, classifying tuberculosis type, and assessing severity score. In: CLEF2018 Working Notes. CEUR Workshop Proceedings, 10–14 September 2018, Avignon, France. CEUR-WS.org (2019). http://ceur-ws.org
Dogariu, M., Ionescu, B.: Multimedia lab @ ImageCLEF 2019 lifelog moment retrieval task. In: CLEF2019 Working Notes. CEUR Workshop Proceedings, 09–12 September 2019, Lugano, Switzerland. CEUR-WS.org (2019). http://ceur-ws.org
Gilbert, A., et al.: Overview of the ImageCLEF 2015 scalable image annotation, localization and sentence generation task. In: CLEF Working Notes (2015)
Gilbert, A., et al.: Overview of the ImageCLEF 2016 scalable concept image annotation task. In: CLEF Working Notes, pp. 254–278 (2016)
Gonzalez-Rivero, M., et al.: The catlin seaview survey-kilometre-scale seascape assessment, and monitoring of coral reef ecosystems. Aquat. Conserv.: Mar. Freshw. Ecosyst. 24, 184–198 (2014)
Hasan, S.A., Ling, Y., Farri, O., Liu, J., Lungren, M., Müller, H.: Overview of the ImageCLEF 2018 medical domain visual question answering task. In: CLEF2018 Working Notes. CEUR Workshop Proceedings, 10–14 September 2018, Avignon, France. CEUR-WS.org (2018). http://ceur-ws.org
de Herrera, A.G.S., Eickhoff, C., Andrearczyk, V., Müller, H.: Overview of the ImageCLEF 2018 caption prediction tasks. In: CLEF2018 Working Notes. CEUR Workshop Proceedings, 10–14 September 2018, Avignon, France. CEUR-WS.org (2018). http://ceur-ws.org
de Herrera, A.G.S., Schaer, R., Bromuri, S., Müller, H.: Overview of the ImageCLEF 2016 medical task. In: Working Notes of CLEF 2016 (Cross Language Evaluation Forum), September 2016
Hoang, T.H., Tran, M.K., Nguyen, V.T., Tran, M.T.: Solving life puzzle with visual context-based clustering and habit reference. In: CLEF2019 Working Notes. CEUR Workshop Proceedings, 09–12-2019, Lugano, Switzerland. CEUR-WS.org (2019). http://ceur-ws.org
Ionescu, B., et al.: Overview of ImageCLEF 2018: challenges, datasets and evaluation. In: Bellot, P., et al. (eds.) CLEF 2018. LNCS, vol. 11018, pp. 309–334. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-98932-7_28
Kalpathy-Cramer, J., de Herrera, A.G.S., Demner-Fushman, D., Antani, S., Bedrick, S., Müller, H.: Evaluating performance of biomedical image retrieval systems: overview of the medical image retrieval task at ImageCLEF 2004–2014. Comput. Med. Imaging Graphics 39, 55–61 (2015)
Karampidis, K., Vasillopoulos, N., Cuevas Rodríguez, C., del Blanco, C.R., Kavallieratou, E., Garcia, N.: Overview of the ImageCLEFsecurity 2019 task. In: CLEF2019 Working Notes. CEUR Workshop Proceedings, 09–12 September 2019, Lugano, Switzerland. CEUR-WS.org (2019)
Kougia, V., Pavlopoulos, J., Androusopoulos, I.: AUEB NLP group at ImageCLEFmed Caption 2019. In: CLEF2019 Working Notes. CEUR Workshop Proceedings, 09–12 September 2019, Lugano, Switzerland. CEUR-WS.org (2019). http://ceur-ws.org
Le, N.K., Nguyen, D.H., Nguyen, V.T., Tran, M.T.: Lifelog moment retrieval with advanced semantic extraction and flexible moment visualization for exploration. In: CLEF2019 Working Notes. CEUR Workshop Proceedings, 9–12 September 2019, Lugano, Switzerland. CEUR-WS.org (2019). http://ceur-ws.org
Müller, H., Clough, P., Deselaers, T., Caputo, B. (eds.): ImageCLEF - Experimental Evaluation in Visual Information Retrieval. The Springer International Series On Information Retrieval, vol. 32. Springer, Berlin (2010). https://doi.org/10.1007/978-3-642-15181-1
Ninh, V.T., et al.: LIFER 2.0: discover personal lifelog insight by interactive lifelog retrieval system. In: CLEF2019 Working Notes. CEUR Workshop Proceedings, 09–12 September 2019, Lugano, Switzerland. CEUR-WS.org (2019). http://ceur-ws.org
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 311–318. Association for Computational Linguistics (2002)
Pelka, O., Friedrich, C.M., de Herrera, A.G.S., Müller, H.: Overview of the ImageCLEFmed 2019 concept prediction task. In: CLEF2019 Working Notes. CEUR Workshop Proceedings, 09–12 September 2019, Lugano, Switzerland. CEUR-WS.org (2019). http://ceur-ws.org
Pelka, O., Koitka, S., Rückert, J., Nensa, F., Friedrich, C.M.: Radiology Objects in COntext (ROCO): a multimodal image dataset. In: Stoyanov, D., et al. (eds.) LABELS/CVII/STENT 2018. LNCS, vol. 11043, pp. 180–189. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01364-6_20
Pelka, O., Nensa, F., Friedrich, C.M.: Adopting semantic information of grayscale radiographs for image classification and retrieval. In: Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2018) - Volume 2: BIOIMAGING, 19–21 January 2018, Funchal, Madeira, Portugal, pp. 179–187 (2018). https://doi.org/10.5220/0006732301790187
Ribeiro, R., Neves, A.J.R., Oliveira, J.L.: UAPTBioinformatics working notes at ImageCLEF 2019 lifelog moment retrieval (LMRT) task. In: CLEF2019 Working Notes. CEUR Workshop Proceedings, 09–12 September 2019, Lugano, Switzerland. CEUR-WS.org (2019). http://ceur-ws.org
Roberts, R.J.: PubMed central: the GenBank of the published literature. Proc. Nat. Acad. Sci. U.S.A. 98(2), 381–382 (2001). https://doi.org/10.1073/pnas.98.2.381
Speers, A.E., Besedin, E.Y., Palardy, J.E., Moore, C.: Impacts of climate change and ocean acidification on coral reef fisheries: an integrated ecological-economic model. Ecol. Econ. 128, 33–43 (2016)
Stokes, M., Deane, G.: Automated processing of coral reef benthic images. Limnol. Oceanogr. Methods 7, 157–168 (2009)
Taubert, S., Kahl, S.: Automated lifelog moment retrieval based on image segmentation and similarity scores. In: CLEF2019 Working Notes. CEUR Workshop Proceedings, 09–12 September 2019, Lugano, Switzerland. CEUR-WS.org (2019). http://ceur-ws.org
Tournadre, M., Dupont, G., Pauwels, V., Cheikh, B., Lmami, M., Ginsca, A.L.: A multimedia modular approach to lifelog moment retrieval. In: CLEF2019 Working Notes. CEUR Workshop Proceedings, 09–12 September 2019, Lugano, Switzerland. CEUR-WS.org (2019). http://ceur-ws.org
Tsikrika, T., de Herrera, A.G.S., Müller, H.: Assessing the scholarly impact of ImageCLEF. In: Forner, P., Gonzalo, J., Kekäläinen, J., Lalmas, M., de Rijke, M. (eds.) CLEF 2011. LNCS, vol. 6941, pp. 95–106. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23708-9_12
Tsikrika, T., Larsen, B., Müller, H., Endrullis, S., Rahm, E.: The scholarly impact of CLEF (2000–2009). In: Forner, P., Müller, H., Paredes, R., Rosso, P., Stein, B. (eds.) CLEF 2013. LNCS, vol. 8138, pp. 1–12. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40802-1_1
Villegas, M., Paredes, R.: Overview of the ImageCLEF 2012 scalable web image annotation task. In: CLEF Working Notes (2012)
Villegas, M., Paredes, R.: Overview of the ImageCLEF 2014 scalable concept image annotation task. In: CLEF Working Notes, pp. 308–328. Citeseer (2014)
Villegas, M., Paredes, R., Thomee, B.: Overview of the ImageCLEF 2013 scalable concept image annotation subtask. In: CLEF Working Notes (2012)
World Health Organization, et al.: Global tuberculosis report 2016 (2016)
Xu, Y., Mo, T., Feng, Q., Zhong, P., Lai, M., Chang, E.I.: Deep learning of feature representation with multiple instance learning for medical image analysis. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2014, 4–9 May 2014, Florence, Italy, pp. 1626–1630 (2014). https://doi.org/10.1109/ICASSP.2014.6853873
Zhou, P., Bai, C., Xia, J.: ZJUTCVR team at ImageCLEFlifelog2019 lifelog moment retrieval task. In: CLEF2019 Working Notes. CEUR Workshop Proceedings, 09–12 September 2019, Lugano, Switzerland. CEUR-WS.org (2019) http://ceur-ws.org
Acknowledgements
The work of Bogdan Ionescu was partially supported by the Ministry of Innovation and Research, UEFISCDI, project SPIA-VA, agreement 2SOL/2017, grant PNIII-P2-2.1-SOL-2016-02-0002.
The data collection of the ImageCLEFcoral task was funded by an IAA grant with support from Professor David Smith and Operation Wallacea. The work of Antonio Campello was supported by Innovate UK, Knowledge Transfer Partnership project KTP010993, and hosted at Filament Consultancy Group Limited.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Ionescu, B. et al. (2019). ImageCLEF 2019: Multimedia Retrieval in Medicine, Lifelogging, Security and Nature. In: Crestani, F., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2019. Lecture Notes in Computer Science(), vol 11696. Springer, Cham. https://doi.org/10.1007/978-3-030-28577-7_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-28577-7_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-28576-0
Online ISBN: 978-3-030-28577-7
eBook Packages: Computer ScienceComputer Science (R0)