Abstract
We present a method for classifying tasks in fetal ultrasound scans using the eye-tracking data of sonographers. The visual attention of a sonographer captured by eye-tracking data over time is defined by a scanpath. In routine fetal ultrasound, the captured standard imaging planes are visually inconsistent due to fetal position, movements, and sonographer scanning experience. To address this challenge, we propose a scale and position invariant task classification method using normalised visual scanpaths. We describe a normalisation method that uses bounding boxes to provide the gaze with a reference to the position and scale of the imaging plane and use the normalised scanpath sequences to train machine learning models for discriminating between ultrasound tasks. We compare the proposed method to existing work considering raw eye-tracking data. The best performing model achieves the F1-score of 84% and outperforms existing models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ahmidi, N., Hager, G.D., Ishii, L., Fichtinger, G., Gallia, G.L., Ishii, M.: Surgical task and skill classification from eye tracking and tool motion in minimally invasive surgery. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010. LNCS, vol. 6363, pp. 295–302. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15711-0_37
Cai, Y., et al.: Spatio-temporal visual attention modelling of standard biometry plane-finding navigation. Med. Image Anal. 65 (2020). https://doi.org/10.1016/j.media.2020.101762
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: EMNLP 2014–2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, pp. 1724–1734 (2014). https://doi.org/10.3115/v1/d14-1179
Droste, R., Cai, Y., Sharma, H., Chatelain, P., Papageorghiou, A.T., Noble, J.A.: Towards capturing sonographic experience: cognition-inspired ultrasound video saliency prediction. In: Zheng, Y., Williams, B.M., Chen, K. (eds.) MIUA 2019. CCIS, vol. 1065, pp. 174–186. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-39343-4_15
Droste, R., Chatelain, P., Drukker, L., Sharma, H., Papageorghiou, A.T., Noble, J.A.: Discovering salient anatomical landmarks by predicting human gaze. In: Proceedings - International Symposium on Biomedical Imaging 2020-April, pp. 1711–1714 (2020). https://doi.org/10.1109/ISBI45749.2020.9098505
Drukker, L., et al.: Transforming obstetric ultrasound into data science using eye tracking, voice recording, transducer motion and ultrasound video. Sci. Rep. 11(1), 14109 (2021). https://doi.org/10.1038/s41598-021-92829-1
Ebeid, I.A., Bhattacharya, N., Gwizdka, J., Sarkar, A.: Analyzing gaze transition behavior using Bayesian mixed effects Markov models. In: Eye Tracking Research and Applications Symposium (ETRA) (2019). https://doi.org/10.1145/3314111.3319839
Fuhl, W., Castner, N., Kübler, T., Lotz, A., Rosenstiel, W., Kasneci, E.: Ferns for area of interest free scanpath classification. In: Eye Tracking Research and Applications Symposium (ETRA) (2019). https://doi.org/10.1145/3314111.3319826
Hild, J., Kühnle, C., Voit, M., Beyerer, J.: Predicting observer’s task from eye movement patterns during motion image analysis. In: Eye Tracking Research and Applications Symposium (ETRA) (2018). https://doi.org/10.1145/3204493.3204575
Lee, Y.H., Wei, C.P., Cheng, T.H., Yang, C.T.: Nearest-neighbor-based approach to time-series classification. Decis. Support Syst. 53(1), 207–217 (2012). https://doi.org/10.1016/j.dss.2011.12.014. https://www.sciencedirect.com/science/article/pii/S0167923612000097
Li, L., et al.: Massively parallel hyperparameter tuning. CoRR abs/1810.0 (2018). http://arxiv.org/abs/1810.05934
Liaw, R., Liang, E., Nishihara, R., Moritz, P., Gonzalez, J.E., Stoica, I.: Tune: a research platform for distributed model selection and training (2018). https://arxiv.org/abs/1807.05118
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection (2018). https://arxiv.org/abs/1708.02002
Openvinotoolkit: openvinotoolkit/cvat. https://github.com/openvinotoolkit/cvat
Public Health England (PHE): NHS Fetal Anomaly Screening Programme Handbook, August 2018. https://www.gov.uk/government/publications/fetal-anomaly-screening-programme-handbook/20-week-screening-scan
Sharma, H., Droste, R., Chatelain, P., Drukker, L., Papageorghiou, A.T., Noble, J.A.: Spatio-temporal partitioning and description of full-length routine fetal anomaly ultrasound scans. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 987–990 (2019). https://doi.org/10.1109/ISBI.2019.8759149
Sharma, H., Drukker, L., Chatelain, P., Droste, R., Papageorghiou, A.T., Noble, J.A.: Knowledge representation and learning of operator clinical workflow from full-length routine fetal ultrasound scan videos. Med. Image Anal. 69, 101973 (2021). https://doi.org/10.1016/j.media.2021.101973. http://www.sciencedirect.com/science/article/pii/S1361841521000190
Sharma, H., Drukker, L., Papageorghiou, A.T., Noble, J.A.: Multi-modal learning from video, eye tracking, and pupillometry for operator skill characterization in clinical fetal ultrasound. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 1646–1649 (2021). https://doi.org/10.1109/ISBI48211.2021.9433863
Thorndike, R.L.: Who belongs in the family? Psychometrika 18(4), 267–276 (1953). https://doi.org/10.1007/bf02289263
Yamak, P.T., Yujian, L., Gadosey, P.K.: A comparison between ARIMA, LSTM, and GRU for time series forecasting. In: ACM International Conference Proceeding Series, pp. 49–55 (2019). https://doi.org/10.1145/3377713.3377722
Acknowledgements
We acknowledge the ERC (Project PULSE: ERC-ADG-2015 694581) and the NIHR Oxford Biomedical Research Centre.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Teng, C., Sharma, H., Drukker, L., Papageorghiou, A.T., Noble, J.A. (2021). Towards Scale and Position Invariant Task Classification Using Normalised Visual Scanpaths in Clinical Fetal Ultrasound. In: Noble, J.A., Aylward, S., Grimwood, A., Min, Z., Lee, SL., Hu, Y. (eds) Simplifying Medical Ultrasound. ASMUS 2021. Lecture Notes in Computer Science(), vol 12967. Springer, Cham. https://doi.org/10.1007/978-3-030-87583-1_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-87583-1_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87582-4
Online ISBN: 978-3-030-87583-1
eBook Packages: Computer ScienceComputer Science (R0)