Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Motion-aware ensemble of three-mode trackers for unmanned aerial vehicles

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

To tackle problems arising from unexpected camera motions in unmanned aerial vehicles (UAVs), we propose a three-mode ensemble tracker where each mode specializes in distinctive situations. The proposed ensemble tracker is composed of appearance-based tracking mode, homography-based tracking mode, and momentum-based tracking mode. The appearance-based tracking mode tracks a moving object well when the UAV is nearly stopped, whereas the homography-based tracking mode shows good tracking performance under smooth UAV or object motion. The momentum-based tracking mode copes with large or abrupt motion of either the UAV or the object. We evaluate the proposed tracking scheme on a widely-used UAV123 benchmark dataset. The proposed motion-aware ensemble shows a 5.3% improvement in average precision compared to the baseline correlation filter tracker, which effectively employs deep features while achieving a tracking speed of at least 80fps in our experimental settings. In addition, the proposed method outperforms existing real-time correlation filter trackers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Bhat, G., Johnander, J., Danelljan, M., Shahbaz Khan, F., Felsberg, M.: Unveiling the power of deep tracking. In: The European Conference on Computer Vision (ECCV) (2018)

  2. Bokeno, E.T., Bort, T.M., Burns, S.S., Rucidlo, M., Wei, W., Wires, D.L., et al.: Package delivery by means of an automated multi-copter UAS/UAV dispatched from a conventional delivery vehicle (2016). US Patent App. 14/989,870

  3. Bolme, D.S., Beveridge, J.R., Draper, B.A., Lui, Y.M.: Visual object tracking using adaptive correlation filters. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2544–2550 (2010)

  4. Bouttefroy, P.L.M., Bouzerdoum, A., Phung, S.L., Beghdadi, A.: Vehicle tracking by non-drifting mean-shift using projective Kalman filter. In: IEEE Conference on Intelligent Transportation Systems (ITSC), pp. 61–66 (2008)

  5. Caballero, F., Merino, L., Ferruz, J., Ollero, A.: Homography based kalman filter for mosaic building, applications to UAV position estimation. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 2004–2009 (2007)

  6. Cao, X., Lan, J., Yan, P., Li, X.: Vehicle detection and tracking in airborne videos by multi-motion layer analysis. Mach. Vis. Appl. 23, 921–935 (2012)

    Article  Google Scholar 

  7. Choi, J., Chang, H.J., Fischer, T., Yun, S., Lee, K., Jeong, J., Demiris, Y., Young Choi, J.: Context-aware deep feature compression for high-speed visual tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 479–488 (2018)

  8. Crammer, K., Dekel, O., Keshet, J., Shalev-Shwartz, S., Singer, Y.: Online passive-aggressive algorithms. J. Mach. Learn. Res. 7, 551–585 (2006)

    MathSciNet  MATH  Google Scholar 

  9. Danelljan, M., Bhat, G., Khan, F.S., Felsberg, M., et al.: ECO: efficient convolution operators for tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, p. 3 (2017)

  10. Danelljan, M., Häger, G., Khan, F., Felsberg, M.: Accurate scale estimation for robust visual tracking. In: British Machine Vision Conference (BMVC) (2014)

  11. Danelljan, M., Hager, G., Shahbaz Khan, F., Felsberg, M.: Learning spatially regularized correlation filters for visual tracking. In: IEEE International Conference on Computer Vision (ICCV), pp. 4310–4318 (2015)

  12. Danelljan, M., Robinson, A., Khan, F.S., Felsberg, M.: Beyond correlation filters: learning continuous convolution operators for visual tracking. In: European Conference on Computer Vision (ECCV), pp. 472–488 (2016)

  13. Dollár, P.: Piotr’s Computer Vision Matlab Toolbox (PMT). https://github.com/pdollar/toolbox. Accessed 25 Feb 2019

  14. Dollár, P., Appel, R., Belongie, S., Perona, P.: Fast feature pyramids for object detection. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 36(8), 1532–1545 (2014)

    Article  Google Scholar 

  15. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. http://www.pascal-network.org/challenges/VOC/ voc2012/workshop/index.html. Accessed 25 Feb 2019

  16. Fan, H., Ling, H.: Sanet: Structure-aware network for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 42–49 (2017)

  17. Greiner, H., Walker, J., Norman, C.O., Bohorquez, F., Zaparovanny, A.: Unmanned delivery (2017). US Patent App. 14/581,027

  18. Han, B., Sim, J., Adam, H.: Branchout: regularization for online ensemble tracking with convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3356–3365 (2017)

  19. He, Z., Fan, Y., Zhuang, J., Dong, Y., Bai, H.: Correlation filters with weighted convolution responses. In: ICCV Workshops, pp. 1992–2000 (2017)

  20. Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: Exploiting the circulant structure of tracking-by-detection with kernels. In: European Conference on Computer Vision (ECCV), pp. 702–715 (2012)

  21. Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intel. (TPAMI) 37(3), 583–596 (2015)

    Article  Google Scholar 

  22. Hong, Z., Chen, Z., Wang, C., Mei, X., Prokhorov, D., Tao, D.: Multi-store tracker (muster): A cognitive psychology inspired approach to object tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 749–758 (2015)

  23. Huang, Z., Fu, C., Li, Y., Lin, F., Lu, P.: Learning aberrance repressed correlation filters for real-time UAV tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2891–2900 (2019)

  24. Jia, X., Lu, H., Yang, M.H.: Visual tracking via adaptive structural local sparse appearance model. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1822–1829 (2012)

  25. Kalman, R.E.: A new approach to linear filtering and prediction problems. J. Basic Eng. 82(1), 35–45 (1960)

    Article  MathSciNet  Google Scholar 

  26. Kiani Galoogahi, H., Fagg, A., Lucey, S.: Learning background-aware correlation filters for visual tracking. In: IEEE International Conference on Computer Vision (ICCV), pp. 1135–1143 (2017)

  27. Koster, K.L.: Delivery platform for unmanned aerial vehicles (2015). US Patent App. 14/560,821

  28. Li, D., Wen, G., Kuai, Y., Porikli, F.: Beyond feature integration: a coarse-to-fine framework for cascade correlation tracking. Mach. Vis. Appl. 30, 519–528 (2019)

    Article  Google Scholar 

  29. Li, F., Tian, C., Zuo, W., Zhang, L., Yang, M.H.: Learning spatial-temporal regularized correlation filters for visual tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4904–4913 (2018)

  30. Li, S., Yeung, D.Y.: Visual object tracking for unmanned aerial vehicles: a benchmark and new motion models. In: AAAI (2017)

  31. Li, Y., Zhu, J.: A scale adaptive kernel correlation filter tracker with feature integration. In: European Conference on Computer Vision (ECCV), pp. 254–265. Springer (2014)

  32. Lusk, P.C., Beard, R.W.: Visual multiple target tracking from a descending aerial platform. In: 2018 Annual American Control Conference (ACC), pp. 5088–5093 (2018)

  33. Mueller, M., Sharma, G., Smith, N., Ghanem, B.: Persistent aerial tracking system for UAVs. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1562–1569 (2016)

  34. Mueller, M., Smith, N., Ghanem, B.: A benchmark and simulator for UAV tracking. In: European Conference on Computer Vision (ECCV), pp. 445–461 (2016)

  35. Nam, H., Han, B.: Learning multi-domain convolutional neural networks for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4293–4302 (2016)

  36. Naseer, T., Sturm, J., Cremers, D.: Followme: Person following and gesture recognition with a quadrocopter. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 624–630 (2013)

  37. Patrick, W.G.: Request apparatus for delivery of medical support implement by UAV (2016). US Patent 9,307,383

  38. Rodríguez-Canosa, G.R., Thomas, S., Del Cerro, J., Barrientos, A., MacDonald, B.: A real-time method to detect and track moving objects (DATMO) from unmanned aerial vehicles (UAVs) using a single camera. Remote Sens. 4(4), 1090–1111 (2012)

    Article  Google Scholar 

  39. Ross, D.A., Lim, J., Lin, R.S., Yang, M.H.: Incremental learning for robust visual tracking. Int. J. Comput. Vis. (IJCV) 77(1–3), 125–141 (2008)

    Article  Google Scholar 

  40. Sapkota, K.R., Roelofsen, S., Rozantsev, A., Lepetit, V., Gillet, D., Fua, P., Martinoli, A.: Vision-based unmanned aerial vehicle detection and tracking for sense and avoid systems. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1556–1561 (2016)

  41. Shi, J., Tomasi, C.: Good features to track. Technical Report, Cornell University (1993)

  42. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014). arXiv:1409.1556

  43. Sun, Z., Wang, Y., Laganière, R.: Hard negative mining for correlation filters in visual tracking. Mach. Vis. Appl. 30, 487–506 (2019)

    Article  Google Scholar 

  44. Teng, Z., Xing, J., Wang, Q., Lang, C., Feng, S., Jin, Y.: Robust object tracking based on temporal and spatial deep networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1144–1153 (2017)

  45. Teutsch, M., Krüger, W.: Detection, segmentation, and tracking of moving objects in UAV videos. In: IEEE International Conference on Advanced Video and Signal-Based Surveillance (AVSS), pp. 313–318. IEEE (2012)

  46. Vedaldi, A., Lenc, K.: Matconvnet—convolutional neural networks for matlab. In: ACM MM (2015)

  47. Wang, Y., Shi, W., Wu, S.: Robust UAV-based tracking using hybrid classifiers. Mach. Vis. Appl. 30, 125–137 (2019)

    Article  Google Scholar 

  48. Wu, Y., Lim, J., Yang, M.H.: Online object tracking: a benchmark. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2411–2418 (2013)

  49. Wu, Y., Lim, J., Yang, M.H.: Object tracking benchmark. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 37(9), 1834–1848 (2015)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by Next-Generation ICD program through NRF funded by Ministry of S&ICT [2017M3C4A7077582] and ICT R&D Program MSIP/IITP [2017-0-00306, Outdoor Surveillance Robots], and BK21 4th program.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, ASRI, Seoul National University, 599 Gwanak-ro, Gwanak-gu, Seoul, Korea

    Kyuewang Lee & Jin Young Choi

  2. School of Computer Science, University of Birmingham, Birmingham, B15 2TT, UK

    Hyung Jin Chang & Aleš Leonardis

  3. Department of Imaging Science and Arts, Chung-Ang University, 84 Heukseok-ro, Dongjak-gu, Seoul, Korea

    Jongwon Choi

  4. NAVER AI LAB, Green Factory, 6 Buljeong-ro, Bundang-gu, Seongnam-si, Gyeonggi-do, Korea

    Byeongho Heo

Authors
  1. Kyuewang Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hyung Jin Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jongwon Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Byeongho Heo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Aleš Leonardis
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jin Young Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jin Young Choi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, K., Chang, H.J., Choi, J. et al. Motion-aware ensemble of three-mode trackers for unmanned aerial vehicles. Machine Vision and Applications 32, 54 (2021). https://doi.org/10.1007/s00138-021-01181-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-021-01181-x

Keywords

Search

Navigation