research-article

LSOTB-TIR: A Large-Scale High-Diversity Thermal Infrared Object Tracking Benchmark

Authors:

Feng ZhengAuthors Info & Claims

MM '20: Proceedings of the 28th ACM International Conference on Multimedia

Pages 3847 - 3856

https://doi.org/10.1145/3394171.3413922

Published: 12 October 2020 Publication History

Abstract

In this paper, we present a Large-Scale and high-diversity general Thermal InfraRed (TIR) Object Tracking Benchmark, called LSOTB-TIR, which consists of an evaluation dataset and a training dataset with a total of 1,400 TIR sequences and more than 600K frames. We annotate the bounding box of objects in every frame of all sequences and generate over 730K bounding boxes in total. To the best of our knowledge, LSOTB-TIR is the largest and most diverse TIR object tracking benchmark to date. To evaluate a tracker on different attributes, we define 4 scenario attributes and 12 challenge attributes in the evaluation dataset. By releasing LSOTB-TIR, we encourage the community to develop deep learning based TIR trackers and evaluate them fairly and comprehensively. We evaluate and analyze more than 30 trackers on LSOTB-TIR to provide a series of baselines, and the results show that deep trackers achieve promising performance. Furthermore, we re-train several representative deep trackers on LSOTB-TIR, and their results demonstrate that the proposed training dataset significantly improves the performance of deep TIR trackers. Codes and dataset are available at https://github.com/QiaoLiuHit/LSOTB-TIR.

Supplementary Material

MP4 File (3394171.3413922.mp4)

This video gives a compact introduction to the paper 'LSOTB-TIR: a large-scale high-diversity thermal infrared object tracking benchmark'.

Download
9.04 MB

References

[1]

B. Babenko, Ming Hsuan Yang, and S. Belongie. 2009. Visual tracking with online Multiple Instance Learning. In IEEE Conference on Computer Vision and Pattern Recognition.

[2]

Chenglong Bao, Yi Wu, Haibin Ling, and Hui Ji. 2012. Real time robust L1 tracker using accelerated proximal gradient approach. In IEEE Conference on Computer Vision and Pattern Recognition.

[3]

A. Berg, J. Ahlberg, and M. Felsberg. 2015. A thermal Object Tracking benchmark. In IEEE International Conference on Advanced Video and Signal Based Surveillance.

[4]

Amanda Berg, Jorgen Ahlberg, and Michael Felsberg. 2016. Channel coded distribution field tracking for thermal infrared imagery. In IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[5]

Luca Bertinetto, Jack Valmadre, Stuart Golodetz, Ondrej Miksik, and Philip HS Torr. 2016a. Staple: Complementary learners for real-time tracking. In IEEE Conference on Computer Vision and Pattern Recognition.

[6]

Luca Bertinetto, Jack Valmadre, Joao F Henriques, Andrea Vedaldi, and Philip HS Torr. 2016b. Fully-Convolutional Siamese Networks for Object Tracking. In European Conference on Computer Vision Workshops.

[7]

David S Bolme, J Ross Beveridge, Bruce A Draper, and Yui Man Lui. 2010. Visual object tracking using adaptive correlation filters. In IEEE Conference on Computer Vision and Pattern Recognition.

[8]

Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, and Michael Felsberg. 2019. Atom: Accurate tracking by overlap maximization. In IEEE Conference on Computer Vision and Pattern Recognition.

[9]

Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, and Michael Felsberg. 2017. ECO: efficient convolution operators for tracking. In IEEE Conference on Computer Vision and Pattern Recognition.

[10]

Martin Danelljan, Gustav H"ager, Fahad Khan, and Michael Felsberg. 2014. Accurate scale estimation for robust visual tracking. In British Machine Vision Conference.

[11]

Martin Danelljan, Gustav Hager, Fahad Shahbaz Khan, and Michael Felsberg. 2015. Learning spatially regularized correlation filters for visual tracking. In IEEE International Conference on Computer Vision.

Digital Library

[12]

James W Davis and Vinay Sharma. 2007. Background-subtraction using contour-based fusion of thermal and visible imagery. Computer Vision and Image Understanding, Vol. 106, 2 (2007), 162--182.

Digital Library

[13]

Huseyin Seckin Demir and Omer Faruk Adil. 2018. Part-Based Co-Difference Object Tracking Algorithm for Infrared Videos. In International Conference on Image Processing.

[14]

Xingping Dong and Jianbing Shen. 2018. Triplet Loss in Siamese Network for Object Tracking. In European Conference on Computer Vision.

[15]

Heng Fan, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Hexin Bai, Yong Xu, Chunyuan Liao, and Haibin Ling. 2019. Lasot: A high-quality benchmark for large-scale single object tracking. In IEEE Conference on Computer Vision and Pattern Recognition.

[16]

Michael Felsberg, Amanda Berg, Gustav Hager, Jorgen Ahlberg, et al. 2015. The thermal infrared visual object tracking VOT-TIR2015 challenge results. In IEEE International Conference on Computer Vision Workshops.

Digital Library

[17]

Michael Felsberg, Matej Kristan, Jivr i Matas, Alevs Leonardis, et al. 2016. The Thermal Infrared Visual Object Tracking VOT-TIR2016 Challenge Results. In European Conference on Computer Vision Workshops.

[18]

Rikke Gade and Thomas B Moeslund. 2014. Thermal cameras and applications: a survey. Machine vision and applications, Vol. 25, 1 (2014), 245--262.

[19]

Jin Gao, Haibin Ling, Weiming Hu, and Junliang Xing. 2014. Transfer learning based visual tracking with gaussian processes regression. In European Conference on Computer Vision.

[20]

Peng Gao, Yipeng Ma, Ke Song, Chao Li, Fei Wang, and Liyi Xiao. 2018. Large margin structured convolution operator for thermal infrared object tracking. In International Conference on Pattern Recognition.

[21]

Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. 2098. ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations.

[22]

Georgia Gkioxari and Jitendra Malik. 2015. Finding action tubes. In IEEE Conference on Computer Vision and Pattern Recognition.

[23]

Erhan Gundogdu, Aykut Koc, Berkan Solmaz, et al. 2016. Evaluation of feature channels for correlation-filter-based visual object tracking in infrared spectrum. In IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[24]

Erhan Gundogdu, Huseyin Ozkan, H. Seckin Demir, et al. 2015. Comparison of infrared and visible imagery for object tracking: Toward trackers with superior IR performance. In IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[25]

Qing Guo, Wei Feng, Ce Zhou, et al. 2017. Learning dynamic siamese network for visual object tracking. In IEEE International Conference on Computer Vision.

[26]

Sam Hare, Stuart Golodetz, Amir Saffari, Vibhav Vineet, et al. 2015. Struck: Structured output tracking with kernels. IEEE Transactions of Pattern Analysis and Machine Intelligence, Vol. 38, 10 (2015), 2096--2109.

Digital Library

[27]

Jo ao F Henriques, Rui Caseiro, Pedro Martins, and Jorge Batista. 2015. High-speed tracking with kernelized correlation filters. IEEE Transactions of Pattern Analysis and Machine Intelligence, Vol. 37, 3 (2015), 583--596.

Digital Library

[28]

Hamed Kiani Galoogahi, Ashton Fagg, and Simon Lucey. 2017. Learning background-aware correlation filters for visual tracking. In IEEE International Conference on Computer Vision.

[29]

M. Kristan, A. Leonardis, J. Matas, M. Felsberg, et al. 2017. The Visual Object Tracking VOT2017 Challenge Results. In IEEE International Conference on Computer Vision Workshops.

[30]

Bo Li, Wei Wu, Qiang Wang, Fangyi Zhang, Junliang Xing, and Junjie Yan. 2019 e. Siamrpn+: Evolution of siamese visual tracking with very deep networks. In IEEE Conference on Computer Vision and Pattern Recognition.

[31]

Chenglong Li, Xinyan Liang, Yijuan Lu, Nan Zhao, and Jin Tang. 2019 a. RGB-T object tracking: benchmark and baseline. Pattern Recognition, Vol. 96 (2019), 106977.

Digital Library

[32]

Feng Li, Cheng Tian, Wangmeng Zuo, Lei Zhang, and Ming-Hsuan Yang. 2018. Learning spatial-temporal regularized correlation filters for visual tracking. In IEEE Conference on Computer Vision and Pattern Recognition.

[33]

Meihui Li, Lingbing Peng, Yingpin Chen, et al. 2019 d. Mask Sparse Representation Based on Semantic Features for Thermal Infrared Target Tracking. Remote Sensing, Vol. 11, 17 (2019), 1967.

[34]

Xin Li, Qiao Liu, Nana Fan, et al. 2019 b. Hierarchical spatial-aware Siamese network for thermal infrared object tracking. Knowledge-Based Systems, Vol. 166 (2019), 71--81.

[35]

Xin Li, Chao Ma, Baoyuan Wu, Zhenyu He, and Ming-Hsuan Yang. 2019 c. Target-Aware Deep Tracking. In IEEE Conference on Computer Vision and Pattern Recognition.

[36]

Yang Li, Jianke Zhu, and Steven CH Hoi. 2015. Reliable patch trackers: Robust visual tracking by exploiting reliable patches. In IEEE Conference on Computer Vision and Pattern Recognition.

[37]

Qiao Liu, Zhenyu He, Xin Li, and Yuan Zheng. 2019. PTB-TIR: A Thermal Infrared Pedestrian Tracking Benchmark. IEEE Transaction on Multimedia, Vol. 22, 3 (2019), 666--675.

[38]

Qiao Liu, Xin Li, Zhenyu He, Nana Fan, Di Yuan, and Hongpeng Wang. 2020. Learning Deep Multi-Level Similarity for Thermal Infrared Object Tracking. IEEE Transaction on Multimedia (2020). https://doi.org/10.1109/TMM.2020.3008028

[39]

Qiao Liu, Xiaohuan Lu, Zhenyu He, et al. 2017. Deep convolutional neural networks for thermal infrared object tracking. Knowledge-Based Systems, Vol. 134 (2017), 189--198.

[40]

Chao Ma, Jia-Bin Huang, Xiaokang Yang, and Ming-Hsuan Yang. 2015a. Hierarchical convolutional features for visual tracking. In IEEE International Conference on Computer Vision.

Digital Library

[41]

Chao Ma, Xiaokang Yang, Chongyang Zhang, and Ming Hsuan Yang. 2015b. Long-term correlation tracking. In IEEE Conference on Computer Vision and Pattern Recognition.

[42]

Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research, Vol. 9, Nov (2008), 2579--2605.

[43]

Matthias Muller, Adel Bibi, Silvio Giancola, Salman Alsubaihi, and Bernard Ghanem. 2018. Trackingnet: A large-scale dataset and benchmark for object tracking in the wild. In European Conference on Computer Vision.

Digital Library

[44]

Hyeonseob Nam and Bohyung Han. 2016. Learning multi-domain convolutional neural networks for visual tracking. In IEEE Conference on Computer Vision and Pattern Recognition.

[45]

Jan Portmann, Simon Lynen, Margarita Chli, and Roland Siegwart. 2014. People detection and tracking from aerial thermal views. In IEEE Robotics and Automation Society.

[46]

Yuankai Qi, Shengping Zhang, Lei Qin, et al. 2016. Hedged Deep Tracking. In IEEE Conference on Computer Vision and Pattern Recognition.

[47]

Olga Russakovsky, Jia Deng, Hao Su, et al. 2015. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision, Vol. 115, 3 (2015), 211--252.

Digital Library

[48]

Laura Sevilla-Lara and Erik Learned-Miller. 2012. Distribution fields for tracking. In IEEE Conference on Computer Vision and Pattern Recognition.

[49]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

[50]

Yibing Song, Chao Ma, Lijun Gong, et al. 2017. Crest: Convolutional residual learning for visual tracking. In IEEE International Conference on Computer Vision.

[51]

Yibing Song, Chao Ma, Xiaohe Wu, Lijun Gong, et al. 2018. Vital: Visual tracking via adversarial learning. In IEEE Conference on Computer Vision and Pattern Recognition.

[52]

Jack Valmadre, Luca Bertinetto, Jo ao Henriques, Andrea Vedaldi, and Philip HS Torr. 2017. End-to-end representation learning for correlation filter based tracking. In IEEE Conference on Computer Vision and Pattern Recognition.

[53]

Vijay Venkataraman, Guoliang Fan, Joseph P Havlicek, et al. 2012. Adaptive kalman filtering for histogram-based appearance learning in infrared imagery. IEEE Transactions on Image Processing, Vol. 21, 11 (2012), 4622--4635.

Digital Library

[54]

Ning Wang, Yibing Song, Chao Ma, Wengang Zhou, Wei Liu, and Houqiang Li. 2019 a. Unsupervised Deep Tracking. In IEEE Conference on Computer Vision and Pattern Recognition.

[55]

Ning Wang, Wengang Zhou, Qi Tian, Richang Hong, Meng Wang, and Houqiang Li. 2018. Multi-cue correlation filters for robust visual tracking. In IEEE Conference on Computer Vision and Pattern Recognition.

[56]

Qiang Wang, Li Zhang, Luca Bertinetto, Weiming Hu, and Philip HS Torr. 2019 b. Fast online object tracking and segmentation: A unifying approach. In IEEE Conference on Computer Vision and Pattern Recognition.

[57]

Yi Wu, Jongwoo Lim, and Ming-Hsuan Yang. 2013. Online object tracking: A benchmark. In IEEE Conference on Computer Vision and Pattern Recognition.

Digital Library

[58]

Zheng Wu, Nathan Fuller, Diane Theriault, and Margrit Betke. 2014. A thermal infrared video benchmark for visual analysis. In IEEE Conference on Computer Vision and Pattern Recognition Workshops.

Digital Library

[59]

Jia Xu, Huchuan Lu, and Ming Hsuan Yang. 2012. Visual tracking via adaptive structural local sparse appearance model. In IEEE Conference on Computer Vision and Pattern Recognition.

[60]

Xianguo Yu, Qifeng Yu, Yang Shang, and Hongliang Zhang. 2017. Dense structural learning for infrared object tracking at 200+ frames per second. Pattern Recognition Letter, Vol. 100 (2017), 152--159.

Digital Library

[61]

Lichao Zhang, Abel Gonzalez-Garcia, Joost van de Weijer, Martin Danelljan, and Fahad Shahbaz Khan. 2019. Synthetic data generation for end-to-end thermal infrared tracking. IEEE Transactions on Image Processing, Vol. 28, 4 (2019), 1837--1850.

Digital Library

Cited By

Li QTan KYuan DLiu Q(2025)Progressive Domain Adaptation for Thermal Infrared TrackingElectronics10.3390/electronics1401016214:1(162)Online publication date: 2-Jan-2025
https://doi.org/10.3390/electronics14010162
Wang QJin PWu YZhou LShen T(2025)Infrared Image Enhancement: A ReviewIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing10.1109/JSTARS.2024.352341818(3281-3299)Online publication date: 2025
https://doi.org/10.1109/JSTARS.2024.3523418
Rizk MBayad I(2025)Bringing Intelligence to SAR Missions: A Comprehensive Dataset and Evaluation of YOLO for Human Detection in TIR ImagesIEEE Access10.1109/ACCESS.2025.352948413(17208-17235)Online publication date: 2025
https://doi.org/10.1109/ACCESS.2025.3529484
Show More Cited By

Index Terms

LSOTB-TIR: A Large-Scale High-Diversity Thermal Infrared Object Tracking Benchmark
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Tracking
      2. Computer vision representations
        Image representations

Recommendations

TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild
Computer Vision – ECCV 2018
Abstract
Despite the numerous developments in object tracking, further improvement of current tracking algorithms is limited by small and mostly saturated datasets. As a matter of fact, data-hungry trackers based on deep-learning currently rely on object ...
LaSOT: A High-quality Large-scale Single Object Tracking Benchmark
Abstract
Despite great recent advances in visual tracking, its further development, including both algorithm design and evaluation, is limited due to lack of dedicated large-scale benchmarks. To address this problem, we present LaSOT, a high-quality Large-...
NT-VOT211: A Large-Scale Benchmark for Night-Time Visual Object Tracking
Computer Vision – ACCV 2024
Abstract
Many current visual object tracking benchmarks such as OTB100, NfS, UAV123, LaSOT, and GOT-10K, predominantly contain day-time scenarios while the challenges posed by the night-time has been less investigated. It is primarily because of the lack ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '20: Proceedings of the 28th ACM International Conference on Multimedia

October 2020

4889 pages

ISBN:9781450379885

DOI:10.1145/3394171

General Chairs:
Chang Wen Chen
Chinese University of Hong Kong, Shenzhen, China
,
Rita Cucchiara
UNIMORE, Italy
,
Xian-Sheng Hua
Alibaba Group, China
,
Program Chairs:
Guo-Jun Qi
Futurewei Technologies, USA
,
Elisa Ricci
UNITN & Fondazione Bruno Kessler, Italy
,
Zhengyou Zhang
Tencent, China
,
Roger Zimmermann
National University of Singapore, Singapore

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Shenzhen Research Council
National Natural Science Foundation of China

Conference

MM '20

Sponsor:

SIGMM

MM '20: The 28th ACM International Conference on Multimedia

October 12 - 16, 2020

WA, Seattle, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

68
Total Citations
View Citations
348
Total Downloads

Downloads (Last 12 months)80
Downloads (Last 6 weeks)5

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li QTan KYuan DLiu Q(2025)Progressive Domain Adaptation for Thermal Infrared TrackingElectronics10.3390/electronics1401016214:1(162)Online publication date: 2-Jan-2025
https://doi.org/10.3390/electronics14010162
Wang QJin PWu YZhou LShen T(2025)Infrared Image Enhancement: A ReviewIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing10.1109/JSTARS.2024.352341818(3281-3299)Online publication date: 2025
https://doi.org/10.1109/JSTARS.2024.3523418
Rizk MBayad I(2025)Bringing Intelligence to SAR Missions: A Comprehensive Dataset and Evaluation of YOLO for Human Detection in TIR ImagesIEEE Access10.1109/ACCESS.2025.352948413(17208-17235)Online publication date: 2025
https://doi.org/10.1109/ACCESS.2025.3529484
Zhang LWang LWu YChen MZheng DCao LZeng BCai Y(2025)UniRTL: A universal RGBT and low-light benchmark for object trackingPattern Recognition10.1016/j.patcog.2024.110984158(110984)Online publication date: Feb-2025
https://doi.org/10.1016/j.patcog.2024.110984
Ji QZhang PChen KZhang LHou C(2025)Scene-aware classifier and re-detector for thermal infrared trackingJournal of Visual Communication and Image Representation10.1016/j.jvcir.2024.104319105:COnline publication date: 11-Feb-2025
https://dl.acm.org/doi/10.1016/j.jvcir.2024.104319
Wang HXu TTang ZWu XKittler J(2025)Multi-modal adapter for RGB-T trackingInformation Fusion10.1016/j.inffus.2025.102940118(102940)Online publication date: Jun-2025
https://doi.org/10.1016/j.inffus.2025.102940
DAI Yuhang 代LIU Qiao 刘YUAN Di 袁FAN Nana 范LIU Yunpeng 刘(2024)低秩自适应微调的一阶段红外目标跟踪Infrared and Laser Engineering10.3788/IRLA2024019953:8(20240199)Online publication date: 2024
https://doi.org/10.3788/IRLA20240199
Wang ZHu YYang JZhou GLiu FLiu Y(2024)A Contrastive-Augmented Memory Network for Anti-UAV Tracking in TIR VideosRemote Sensing10.3390/rs1624477516:24(4775)Online publication date: 21-Dec-2024
https://doi.org/10.3390/rs16244775
Shu XHuang FQiu ZZhang XYuan D(2024)Learning Unsupervised Cross-Domain Model for TIR Target TrackingMathematics10.3390/math1218288212:18(2882)Online publication date: 15-Sep-2024
https://doi.org/10.3390/math12182882
Wang BLi QMao QWang JChen CShangguan AZhang H(2024)A Survey on Vision-Based Anti Unmanned Aerial Vehicles MethodsDrones10.3390/drones80905188:9(518)Online publication date: 23-Sep-2024
https://doi.org/10.3390/drones8090518
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten