Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3281548.3281556acmconferencesArticle/Chapter ViewAbstractPublication PagesgisConference Proceedingsconference-collections
research-article

Vision-based UAVs Aerial Image Localization: A Survey

Published: 06 November 2018 Publication History

Abstract

Unmanned aerial vehicles (UAVs) have been increasingly used in earth observation, public safety, military and civilian applications due to its portability, high mobility and flexibility. In some GPS-denied environments, accurate drone position cannot be obtained due to occlusion, multi-path interference and other factors. While understanding and localization the content of the images is vital for earth observation, map revision, multi-source image fusion, disaster relief, smart city and other applications. The progress of computer vision and convolutional neural networks(CNNs) in image processing provide a promising solution to locate UAVs aerial image and mapping to the large-scale reference image. Firstly, key localization techniques based on image retrieval-----image description, image matching and position mapping are summarized considering the characteristics of UAVs aerial images. And then, image localization based on extracting deep semantic features and image localization based on classification method by subdividing areas are recommended. Throughout this paper, we will have an insight into the prospect of the UAVs image localization and the challenges to be faced.

References

[1]
Adrien Angeli, Stéphane Doncieux, Jean-Arcady Meyer, and David Filliat. 2008. Real-time visual loop-closure detection. In International Conference on Robotics and Automation. 1842--1847.
[2]
Roberto Arroyo, Pablo F Alcantarilla, Luis M Bergasa, and Eduardo Romera. 2016. Fusion and binarization of CNN features for robust topological localization across seasons. In Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on. IEEE, 4656--4663.
[3]
H Badino, D Huber, and T Kanade. 2012. Real-time topometric localization. In IEEE International Conference on Robotics and Automation. 1635--1642.
[4]
Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. 2006. SURF: speeded up robust features. In European Conference on Computer Vision. 404--417.
[5]
Sean Bell, C. Lawrence Zitnick, Kavita Bala, and Ross Girshick. 2015. Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks. (2015), 2874--2883.
[6]
N Carlevaris-Bianco and R. M Eustice. 2014. Learning visual feature descriptors for dynamic lighting conditions. In Ieee/rsj International Conference on Intelligent Robots and Systems. 2769--2776.
[7]
Bindita Chaudhuri, Begüm Demir, Lorenzo Bruzzone, and Subhasis Chaudhuri. 2016. Region-based retrieval of remote sensing images using an unsupervised graph-theoretic approach. IEEE Geoscience and Remote Sensing Letters 13, 7 (2016), 987--991.
[8]
Liang Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. 2015. Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs. Computer Science 4 (2015), 357--361.
[9]
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. 2018. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence 40, 4 (2018), 834--848.
[10]
Gianpaolo Conte and Patrick Doherty. 2009. Vision-based unmanned aerial vehicle navigation using geo-referenced information. EURASIP Journal on Advances in Signal Processing 2009 (2009), 10.
[11]
David J. Crandall, Lars Backstrom, Daniel Huttenlocher, and Jon Kleinberg. 2009. Mapping the world's photos. In International Conference on World Wide Web. 761--770.
[12]
M Cummins. 2008. FAB-MAP: Probabilistic localization and mapping in the space of appearance. Int.j.robot.res 27, 6 (2008), 647--665.
[13]
Mark Cummins and Paul M. Newman. 2011. Appearance-only SLAM at large scale with FAB-MAP 2.0. International Journal of Robotics Research 30, 9 (2011), 1100--1123.
[14]
A. J. Davison, I. D. Reid, N. D. Molton, and O Stasse. 2007. MonoSLAM: real-time single camera SLAM. IEEE Trans Pattern Anal Mach Intell 29, 6 (2007), 1052--1067.
[15]
Qiang Dong and Qinghua Zou. 2018. Visual UAV detection method with online feature classification. In IEEE Information Technology, Networking, Electronic and Automation Control Conference.
[16]
Peijun Du, Yunhao Chen, Tang Hong, and Fang Tao. 2005. Study on content-based remote sensing image retrieval. In IEEE International Geoscience and Remote Sensing Symposium. 4 pp.
[17]
Ethan Eade and Tom Drummond. 2009. Edge landmarks in monocular SLAM. Butterworth-Heinemann. 588--596 pages.
[18]
Baojie Fan, Yingkui Du, Linlin Zhu, and Yandong Tang. 2010. The Registration of UAV Down-Looking Aerial Images to Satellite Images with Image Entropy and Edges. Springer Berlin Heidelberg. 609--617 pages.
[19]
David Filliat. 2007. A visual bag of words method for interactive qualitative localization and mapping. In Robotics and Automation, 2007 IEEE International Conference on. IEEE, 3921--3926.
[20]
Dorian Galvez-Lopez and Juan D Tardos. 2011. Real-time loop detection with bags of binary words. In Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on. IEEE, 51--58.
[21]
Dorian Gálvez-López and Juan D Tardos. 2012. Bags of binary words for fast place recognition in image sequences. IEEE Transactions on Robotics 28, 5 (2012), 1188--1197.
[22]
Emilio Garcia-Fidalgo and Alberto Ortiz. 2017. Hierarchical Place Recognition for Topological Mapping. IEEE Transactions on Robotics PP, 99 (2017), 1--14.
[23]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672--2680.
[24]
R. M Haralick, K Shanmugam, and Its'Hak Dinstein. 1973. Textural Features for Image Classification. Systems Man & Cybernetics IEEE Transactions on smc-3, 6 (1973), 610--621.
[25]
C Harris. 1988. A combined corner and edge detector. In Proc. of Fourth Alvey Vision Conference. 147--151.
[26]
James Hays and Alexei A Efros. 2008. IM2GPS: estimating geographic information from a single image. In Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on. IEEE, 1--8.
[27]
James Hays and Alexei A Efros. 2015. Large-scale image geolocalization. In Multimodal Location Estimation of Videos and Images. Springer, 41--62.
[28]
J. He, Y. Li, X. Li, and M. Tang. 2012. Registration method for unmanned aerial vehicle images based on point feature and edge feature. Journal of Southwest Jiaotong University 47, 6 (2012), 955--961.
[29]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.
[30]
Steven C. H. Hoi, Wei Liu, and Shih Fu Chang. 2008. Semi-supervised distance metric learning for Collaborative Image Retrieval. In Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on. 1--7.
[31]
Yi Hou. 2017. Visual Place Recognition with Deep Convolutional Neural Networks for Mobile Robots. Ph.D. Dissertation. Electronics Science and Technology Graduate School of National University of Defense Technology.
[32]
Shih-Ming Huang, Ching-Chun Huang, and Cheng-Chuan Chou. 2012. Image registration among UAV image sequence and Google satellite image under quality mismatch. In ITS Telecommunications (ITST), 2012 12th International Conference on. IEEE, 311--315.
[33]
E Kalogerakis. 2009. Image Sequence Geolocation with Human Travel Priors. Proc Iccv 30, 2 (2009), 253--260.
[34]
Yan Ke and R. Sukthankar. 2004. PCA-SIFT: a more distinctive representation for local image descriptors. In Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on. II.
[35]
G. K. Kuchimanchi, V. V. Phoha, K. S. Balagani, and S. R. Gaddam. 2004. Dimension reduction using feature extraction methods for real-time misuse detection systems. In Information Assurance Workshop, 2004. Proceedings From the Fifth IEEE Smc. 195--202.
[36]
Hanjiang Lai, Yan Pan, Ye Liu, and Shuicheng Yan. 2015. Simultaneous feature learning and hash coding with deep neural networks. (2015), 3270--3278.
[37]
Thomas Lemaire, Cyrille Berger, Il Kyun Jung, and Simon Lacroix. 2007. Vision-Based SLAM: Stereo and Monocular Approaches. International Journal of Computer Vision 74, 3 (2007), 343--364.
[38]
Stefan Leutenegger, Margarita Chli, and Roland Y. Siegwart. 2012. BRISK: Binary Robust invariant scalable keypoints. In IEEE International Conference on Computer Vision. 2548--2555.
[39]
Wu-Jun Li, Sheng Wang, and Wang-Cheng Kang. 2015. Feature learning based deep supervised hashing with pairwise labels. arXiv preprint arXiv:1511.03855 (2015).
[40]
Xiaodan Liang, Xiaohui Shen, Donglai Xiang, Jiashi Feng, Liang Lin, and Shuicheng Yan. 2016. Semantic Object Parsing with Local-Global Long Short-Term Memory. In Computer Vision and Pattern Recognition. 3185--3193.
[41]
L. I. Lichun, Yang Gui, Yang Shang, and Kunpeng Wang. 2009. Horizon Detection Based on Edge and Region Feature for Images Captured by UAV. Journal of Projectiles Rockets Missiles & Guidance 29, 4 (2009), 281--284.
[42]
Kevin Lin, Huei-Fang Yang, Jen-Hao Hsiao, and Chu-Song Chen. 2015. Deep learning of binary hash codes for fast image retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 27--35.
[43]
Tsung Yi Lin, Serge Belongie, and James Hays. 2013. Cross-View Image Geolocalization. In Computer Vision and Pattern Recognition. 891--898.
[44]
David G. Lowe. 2004. Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision 60, 2 (2004), 91--110.
[45]
Stephanie Lowry, Niko Sünderhauf, Paul Newman, John J Leonard, David Cox, Peter Corke, and Michael J Milford. 2016. Visual place recognition: A survey. IEEE Transactions on Robotics 32, 1 (2016), 1--19.
[46]
S. M Lowry, M. J Milford, and G. F Wyeth. 2014. Transforming morning to afternoon using linear regression techniques. In IEEE International Conference on Robotics and Automation. 3950--3955.
[47]
Jonathan Mamou, Yosi Mass, Michal Shmueli-Scheuer, and Benjamin Sznajder. 2009. A unified inverted index for an efficient image and text retrieval. 814--815.
[48]
D Marmanis, J. D Wegner, S Galliani, K Schindler, M Datcu, and U Stilla. 2016. Semantic Segmentation of Aerial Images with AN Ensemble of Cnns. III-3 (2016), 473--480.
[49]
M Modiri, A Salehabadi, M Mohebbi, AM Hashemi, and M Masumi. 2015. Classification of urban feature from unmanned aerial vehicle images using GASVM integration and multi-scale segmentation. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences 40, 1 (2015), 479.
[50]
H. P Moravec. 1977. Towards automatic visual obstacle avoidance. Proc of Ijcai 584 (1977).
[51]
Ahmed Nassar, Karim Amer, Reda ElHakim, and Mohamed ElHelw. 2018. A Deep CNN-Based Framework For Enhanced Aerial Imagery Registration with Applications to UAV Geolocalization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 1513--1523.
[52]
Peer Neubert, Niko Sünderhauf, and Peter Protzel. 2015. Superpixel-based appearance change prediction for long-term navigation across seasons. Robotics and Autonomous Systems 69 (2015), 15--27.
[53]
Bien Van Nguyen, Duy Pham, Thanh Duc Ngo, Duy Dinh Le, and Duc Anh Duong. 2014. Integrating Spatial Information into Inverted Index for Large-Scale Image Retrieval. IEEE (2014), 102--105.
[54]
Aude Oliva and Antonio Torralba. 2001. Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope. Kluwer Academic Publishers. 145--175 pages.
[55]
Matti Pietikäinen, Timo Ojala, and Zelin Xu. 2000. Rotation-invariant texture classification using feature distributions. Pattern Recognition 33, 1 (2000), 43--52.
[56]
S. Rady, A. A. Kandil, and E. Badreddin. 2011. A hybrid localization approach for UAV in GPS denied areas. In Ieee/sice International Symposium on System Integration. 1269--1274.
[57]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. 234--241.
[58]
Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski. 2012. ORB: An efficient alternative to SIFT or SURF. In International Conference on Computer Vision. 2564--2571.
[59]
WANG Rui and ZHU Zheng-dan. 2015. SIFT matching with color invariant characteristics and global context. Optics and Precision Engineering 23, 1 (2015), 295--301.
[60]
G. Schindler, M. Brown, and R. Szeliski. 2007. City-Scale Location Recognition. Cvpr (2007), 1--7.
[61]
Grant J. Scott, Matthew N. Klaric, Curt H. Davis, and Chi Ren Shyu. 2011. Entropy-Balanced Bitmap Tree for Shape-Based Object Retrieval From Large-Scale Satellite Imagery Databases. IEEE Transactions on Geoscience & Remote Sensing 49, 5 (2011), 1603--1616.
[62]
M. Shan, F. Wang, F. Lin, and Z. Gao. 2015. Google map aided visual navigation for UAVs in GPS-denied environment. In IEEE International Conference on Robotics and Biomimetics. 114--119.
[63]
Sunderhauf and Protzel. 2011. BRIEF-Gist - closing the loop by simple means. (2011), 1234--1241.
[64]
Niko Sünderhauf, Sareh Shirazi, Adam Jacobson, Feras Dayoub, Edward Pepperell, Ben Upcroft, and Michael Milford. 2015. Place recognition with convnet landmarks: Viewpoint-robust, condition-robust, training-free. Proceedings of Robotics: Science and Systems XII (2015).
[65]
T. Taisho, L. Enfu, T. Kanji, and S. Naotoshi. 2015. Mining visual experience for fast cross-view UAV localization. In Ieee/sice International Symposium on System Integration. 375--380.
[66]
Yicong Tian, Chen Chen, and Mubarak Shah. 2017. Cross-View Image Matching for Geo-Localization in Urban Environments. (2017), 1998--2006.
[67]
Rahul Rama Varior, Bing Shuai, Jiwen Lu, Dong Xu, and Gang Wang. 2016. A Siamese Long Short-Term Memory Architecture for Human Re-identification. In European Conference on Computer Vision. 135--153.
[68]
Florian Walch, Caner Hazirbas, Laura Leal-Taixe, Torsten Sattler, Sebastian Hilsen-beck, and Daniel Cremers. 2017. Image-based localization using lstms for structured feature correlation. In Int. Conf. Comput. Vis.(ICCV). 627--637.
[69]
Jia Wang, Wen-jann Yang, and Raj Acharya. 1997. Color clustering techniques for color-content-based image retrieval from image databases. In icmcs. IEEE, 442.
[70]
Yanan Wei, Zulin Wang, and Mai Xu. 2017. Road Structure Refined CNN for Road Extraction in Aerial Image. IEEE Geoscience & Remote Sensing Letters 14, 5 (2017), 709--713.
[71]
Tobias Weyand, Ilya Kostrikov, and James Philbin. 2016. PlaNet-Photo Geolocation with Convolutional Neural Networks. Springer International Publishing. 37--55 pages.
[72]
Scott Workman, Richard Souvenir, and Nathan Jacobs. 2016. Wide-Area Image Geolocalization with Aerial Reference Imagery. In IEEE International Conference on Computer Vision. 3961--3969.
[73]
Gui Song Xia, Xin Yi Tong, Fan Hu, Yanfei Zhong, Mihai Datcu, and Liangpei Zhang. 2017. Exploiting Deep Features for Remote Sensing Image Retrieval: A Systematic Investigation. (2017).
[74]
Xiongwu Xiao, Bingxuan Guo, Deren Li, Linhui Li, Nan Yang, Jianchen Liu, Peng Zhang, and Zhe Peng. 2016. Multi-View Stereo Matching Based on Self-Adaptive Patch and Image Grouping for Multiple Unmanned Aerial Vehicle Imagery. Remote Sensing 8, 2 (2016), 89.
[75]
W. Xie, Y. Peng, and J. Xiao. 2014. Cross-view feature learning for scalable social image analysis. (2014).
[76]
Fu Zhong Yin and Li Min Sun. 2010. The Research of Map Publishing Platform Development Based on the Tile Pyramid Technology. Geomatics & Spatial Information Technology (2010).
[77]
Hanwang Zhang, Zheng Jun Zha, Yang Yang, Shuicheng Yan, Yue Gao, and Tat Seng Chua. 2014. Attribute-Augmented Semantic Hierarchy:Towards a Unified Framework for Content-Based Image Retrieval. Acm Transactions on Multimedia Computing Communications & Applications 11, 1s (2014), 1--21.
[78]
Zhengxin Zhang, Qingjie Liu, and Yunhong Wang. 2017. Road Extraction by Deep Residual U-Net. IEEE Geoscience & Remote Sensing Letters PP, 99 (2017), 1--5.
[79]
Shuangming Zhao, Guorong Yu, and Yunfan Cui. 2018. New UAV image registration method based on geometric constrained belief propagation. Multimedia Tools and Applications 7 (2018), 1--21.
[80]
Wenzhi Zhao and Shihong Du. 2016. Spectral--spatial feature extraction for hyperspectral image classification: A dimension reduction and deep learning approach. IEEE Transactions on Geoscience and Remote Sensing 54, 8 (2016), 4544--4554.
[81]
Lichen Zhou, Chuang Zhang, and Ming Wu. 2018. D-LinkNet: LinkNet with Pre-trained Encoder and Dilated Convolution for High Resolution Satellite Imagery Road Extraction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 182--186.
[82]
Xiangyu Zhuo, Tobias Koch, Franz Kurz, Friedrich Fraundorfer, and Peter Reinartz. 2017. Automatic UAV Image Geo-Registration by Matching UAV Images to Georeferenced Image Data. Remote Sensing 9, 4 (2017).

Cited By

View all
  • (2024)A Review of Electric UAV Visual Detection and Navigation Technologies for Emergency Rescue MissionsSustainability10.3390/su1605210516:5(2105)Online publication date: 3-Mar-2024
  • (2024)Expediting the Convergence of Global Localization of UAVs through Forward-Facing Camera ObservationDrones10.3390/drones80703358:7(335)Online publication date: 19-Jul-2024
  • (2024)CurriculumLoc: Enhancing Cross-Domain Geolocalization Through Multistage RefinementIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.338019162(1-14)Online publication date: 2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
GeoAI '18: Proceedings of the 2nd ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery
November 2018
68 pages
ISBN:9781450360364
DOI:10.1145/3281548
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Deep Learning
  2. Image Description
  3. Semantic
  4. UAVs Aerial Image
  5. Vision-based Image Localization

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

SIGSPATIAL '18
Sponsor:

Acceptance Rates

Overall Acceptance Rate 17 of 25 submissions, 68%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)103
  • Downloads (Last 6 weeks)8
Reflects downloads up to 18 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)A Review of Electric UAV Visual Detection and Navigation Technologies for Emergency Rescue MissionsSustainability10.3390/su1605210516:5(2105)Online publication date: 3-Mar-2024
  • (2024)Expediting the Convergence of Global Localization of UAVs through Forward-Facing Camera ObservationDrones10.3390/drones80703358:7(335)Online publication date: 19-Jul-2024
  • (2024)CurriculumLoc: Enhancing Cross-Domain Geolocalization Through Multistage RefinementIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.338019162(1-14)Online publication date: 2024
  • (2024)A Novel UAV Visual Navigation Method Using Online Customizable Image ReferenceIEEE Geoscience and Remote Sensing Letters10.1109/LGRS.2024.346546421(1-5)Online publication date: 2024
  • (2024)UAVD4L: A Large-Scale Dataset for UAV 6-DoF Localization2024 International Conference on 3D Vision (3DV)10.1109/3DV62453.2024.00156(1574-1583)Online publication date: 18-Mar-2024
  • (2023)F3-Net: Multiview Scene Matching for Drone-Based Geo-LocalizationIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2023.327825761(1-11)Online publication date: 2023
  • (2023)Machine Learning-Aided Operations and Communications of Unmanned Aerial Vehicles: A Contemporary SurveyIEEE Communications Surveys & Tutorials10.1109/COMST.2023.331222126:1(496-533)Online publication date: 11-Sep-2023
  • (2023)Exploration and Sterilization Coronavirus in Air based on Unmanned Aerial Vehicles2023 International Conference on Artificial Intelligence Science and Applications in Industry and Society (CAISAIS)10.1109/CAISAIS59399.2023.10270442(1-6)Online publication date: 3-Sep-2023
  • (2023)Engineering Challenges for AI-Supported Computer Vision in Small Uncrewed Aerial Systems2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)10.1109/CAIN58948.2023.00033(158-170)Online publication date: May-2023
  • (2023)LSVLRobotics and Autonomous Systems10.1016/j.robot.2023.104497168:COnline publication date: 1-Oct-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media