research-article

FPGA-based Binocular Image Feature Extraction and Matching System

Authors:

Peng GaoAuthors Info & Claims

ICMSSP '19: Proceedings of the 2019 4th International Conference on Multimedia Systems and Signal Processing

Pages 182 - 187

https://doi.org/10.1145/3330393.3330429

Published: 10 May 2019 Publication History

Abstract

Image feature extraction and matching is a fundamental but computation intensive task in machine vision. This paper proposes a novel FPGA-based embedded system to accelerate feature extraction and matching. It implements SURF feature point detection and BRIEF feature descriptor construction and matching. For binocular stereo vision, feature matching includes both tracking matching and stereo matching, which simultaneously provide feature point correspondences and parallax information. Our system is evaluated on a ZYNQ XC7Z045 FPGA. The result demonstrates that it can process binocular video data at a high frame rate (640 x 480 @ 162fps). Moreover, an extensive test proves our system has robustness for image compression, blurring and illumination.

References

[1]

Lowe, D. G. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60, 2 (2004), 91--110.

Digital Library

[2]

Bay, H., Ess, A., Tuytelaars, T. and Van Gool, L. Speeded-Up Robust Features. Computer Vision & Image Understanding, 110, 3 (2008), 404--417.

Digital Library

[3]

Calonder, M., Lepetit, V., Strecha, C. and Fua, P. BRIEF: Binary Robust Independent Elementary Features. Springer Berlin Heidelberg, City, 2010.

[4]

Harris, C. G. and Stephens, M. A combined corner and edge detector. Citeseer, City, 1988.

[5]

Rosten, E. and Drummond, T. Machine learning for high-speed corner detection. Springer, City, 2006.

Digital Library

[6]

Rublee, E., Rabaud, V., Konolige, K. and Bradski, G. ORB: An efficient alternative to SIFT or SURF (2011).

[7]

Lentaris, G., Stamoulias, I., Soudris, D. and Lourakis, M. HW/SW codesign and FPGA acceleration of visual odometry algorithms for rover navigation on Mars. IEEE Transactions on Circuits and Systems for Video Technology, 26, 8 (2016), 1563--1577.

Digital Library

[8]

Mistry, P., Gregg, C., Rubin, N., Kaeli, D. and Hazelwood, K. Analyzing program flow within a many-kernel OpenCL application. City, 2011.

[9]

Chariot, A. and Keriven, R. GPU-boosted online image matching. City, 2008.

[10]

Wilson, C., Zicari, P., Craciun, S., Gauvin, P., Carlisle, E., George, A. and Lam, H. A power-efficient real-time architecture for SURF feature extraction. IEEE, City, 2014.

[11]

Čížek, P. and Faigl, J. Real-Time FPGA-Based Detection of Speeded-Up Robust Features Using Separable Convolution. IEEE Transactions on Industrial Informatics, 14, 3 (2018), 1155--1163.

[12]

Jia, X., Guo, K., Wang, W., Wang, Y. and Yang, H. SRI-SURF: A better SURF powered by scaled-RAM interpolator on FPGA. IEEE, City, 2016.

[13]

Liu, L., Zhang, W., Deng, C., Yin, S., Cai, S. and Wei, S. SURFEX: A 57fps 1080P resolution 220mW silicon implementation for simplified speeded-up robust feature with 65nm process, 2013.

[14]

Fularz, M., Kraft, M., Schmidt, A. and Kasiński, A. A High-Performance FPGA-Based Image Feature Detector and Matcher Based on the FAST and BRIEF Algorithms. International Journal of Advanced Robotic Systems, 12, 10 (2015), 141.

[15]

Vancea, C. and Nedevschi, S. LUT-based Image Rectification Module Implemented in FPGA. City, 2007.

[16]

Ehsan, S., Clark, A., Rehman, N. and McDonald-Maier, K. Integral images: efficient algorithms for their computation and storage in resource-constrained embedded vision systems. Sensors, 15, 7 (2015), 16804--16830.

[17]

Mikolajczyk, K. and Schmid, C. A performance evaluation of local descriptors. IEEE transactions on pattern analysis and machine intelligence, 27, 10 (2005), 1615--1630.

Digital Library

[18]

Chum, O. and Matas, J. Optimal Randomized RANSAC. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30, 8 (2008), 1472--1482.

Digital Library

[19]

Scharstein, D. and Szeliski, R. High-accuracy stereo depth maps using structured light. IEEE, City, 2003.

[20]

Bonato, V., Marques, E. and Constantinides, G. A. A parallel hardware architecture for scale and rotation invariant feature detection. IEEE transactions on circuits and systems for video technology, 18, 12 (2008), 1703--1712.

Digital Library

[21]

Fowers, S. G., Lee, D., Ventura, D. A. and Archibald, J. K. The Nature-Inspired BASIS Feature Descriptor for UAV Imagery and Its Hardware Implementation. IEEE Transactions on Circuits and Systems for Video Technology, 23, 5 (2013), 756--768.

Digital Library

[22]

Heo, H., Lee, J., Lee, K. and Lee, C. FPGA based implementation of FAST and BRIEF algorithm for object recognition. City, 2013.

[23]

Wang, J., Zhong, S., Yan, L. and Cao, Z. An embedded system-on-chip architecture for real-time visual detection and matching. IEEE transactions on Circuits and Systems for Video Technology, 24, 3 (2014), 525--538.

Digital Library

Cited By

Wang SGong ZZhang ZWang SZhao H(2023)Design of binocular vision system based on ZynqThird International Conference on Signal Image Processing and Communication (ICSIPC 2023)10.1117/12.3005211(112)Online publication date: 20-Oct-2023
https://doi.org/10.1117/12.3005211
Liu DZhou GZhang DZhou XLi C(2020)Ground Control Point Automatic Extraction for Spaceborne Georeferencing Based on FPGAIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing10.1109/JSTARS.2020.299883813(3350-3366)Online publication date: 2020
https://doi.org/10.1109/JSTARS.2020.2998838
Lepecq MDarouich M(2020)A Stream Hardware Architecture for Keypoint Matching Based on a Speculative Approach2020 IEEE International Symposium on Circuits and Systems (ISCAS)10.1109/ISCAS45731.2020.9180928(1-5)Online publication date: Oct-2020
https://doi.org/10.1109/ISCAS45731.2020.9180928
Show More Cited By

Index Terms

FPGA-based Binocular Image Feature Extraction and Matching System
1. Hardware
  1. Integrated circuits
    1. Reconfigurable logic and FPGAs
      1. Hardware accelerators

Recommendations

Dense Feature Matching Core for FPGA-based Smart Cameras
ICDSC 2017: Proceedings of the 11th International Conference on Distributed Smart Cameras

Smart cameras are image/video acquisition devices that integrate image processing algorithms close to the image sensor, so they can deliver high-level information to a host computer or high-level decision process. In this context, a central issue is the ...
Binocular structured light stereo matching approach for dense facial disparity map
AI'11: Proceedings of the 24th international conference on Advances in Artificial Intelligence

Binocular stereo vision technology shows a particular interesting for face recognition, in which the accurate stereo matching is the key issue for obtaining dense disparity map used for exploiting 3D shape information of object. This paper proposed a ...
Evaluating performance of feature extraction methods for practical 3D imaging systems
IVCNZ '12: Proceedings of the 27th Conference on Image and Vision Computing New Zealand

Smart cameras are extensively used for multi-view capture and 3D rendering applications. To achieve high quality, such applications are required to estimate accurate position and orientation of the cameras (called as camera calibration-pose estimation). ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICMSSP '19: Proceedings of the 2019 4th International Conference on Multimedia Systems and Signal Processing

May 2019

213 pages

ISBN:9781450371711

DOI:10.1145/3330393

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Shenzhen University: Shenzhen University
Sun Yat-Sen University

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICMSSP 2019

ICMSSP 2019: 2019 4th International Conference on Multimedia Systems and Signal Processing

May 10 - 12, 2019

Guangzhou, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
122
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)3

Reflects downloads up to 21 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang SGong ZZhang ZWang SZhao H(2023)Design of binocular vision system based on ZynqThird International Conference on Signal Image Processing and Communication (ICSIPC 2023)10.1117/12.3005211(112)Online publication date: 20-Oct-2023
https://doi.org/10.1117/12.3005211
Liu DZhou GZhang DZhou XLi C(2020)Ground Control Point Automatic Extraction for Spaceborne Georeferencing Based on FPGAIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing10.1109/JSTARS.2020.299883813(3350-3366)Online publication date: 2020
https://doi.org/10.1109/JSTARS.2020.2998838
Lepecq MDarouich M(2020)A Stream Hardware Architecture for Keypoint Matching Based on a Speculative Approach2020 IEEE International Symposium on Circuits and Systems (ISCAS)10.1109/ISCAS45731.2020.9180928(1-5)Online publication date: Oct-2020
https://doi.org/10.1109/ISCAS45731.2020.9180928
Chakraborty ABanerjee A(2020)A Novel VLSI Architecture of CORDIC Based Image Registration2020 Sixth International Conference on Bio Signals, Images, and Instrumentation (ICBSII)10.1109/ICBSII49132.2020.9167539(1-6)Online publication date: Feb-2020
https://doi.org/10.1109/ICBSII49132.2020.9167539
Zhao ZWang FNi Q(2019)An FPGA-based Hardware Accelerator of RANSAC Algorithm for Matching of Images Feature Points2019 IEEE 13th International Conference on ASIC (ASICON)10.1109/ASICON47005.2019.8983656(1-4)Online publication date: Oct-2019
https://doi.org/10.1109/ASICON47005.2019.8983656

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents