Abstract
To capture more details and enhance image’s contrast in a low-light-level (LLL) environment and improve the performance of the LLL night vision system, a method of extending the dynamic range for the LLL night vision system based on guided filter and adaptive detail enhancement is proposed. Moreover, a complete high dynamic range (HDR) intensified complementary metal oxide semiconductor (ICMOS) camera based on this method is implemented. Unlike conventional technologies to create HDR images, the proposed camera based on an HDR CMOS can better process images’ edge information while avoiding artifacts’ appearance and solve the storage problem in a multi-images fusion so that real-time image is realized processing. Based on this, the ICMOS camera can capture details in a high contrast environment with an extended dynamic range of up to 80 dB. As the core processing unit of this camera, a Xilinx Spartan 6 series FPGA has completed the work of CMOS driving, image detail enhancement, image storage control, video compression output. Finally, this camera can realize an output video with a resolution of 1280 × 1024 at the rate of 60fps, and experimental results show that this camera has a good performance under a 10−4 lx environment.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abolbashari, M., Magalhaes, F., Araujo, F.M.M., et al.: High dynamic range compressive imaging: a programmable imaging system. Opt. Eng. 51(7), 071407 (2012). https://doi.org/10.1117/1.OE.51.7.071407
Ahmad, A., Amira, A., Nichoil, P., et al.: FPGA-based IP cores implementation for face recognition using dynamic partial reconfiguration. J. Real-Time Image Process. 8(3), 327–340 (2013). https://doi.org/10.1007/s11554-011-0221-x
Bae, S.M., Paris, S., Durand, F.E.: Two-scale tone management for photographic look. ACM Trans. Gr. 25(3), 637–645 (2006). https://doi.org/10.1145/1141911.1141935
Branchitta, F., Diani, M., Corsini, G., et al.: New technique for the visualization of high dynamic range infrared images. Opt. Eng. 48(9), 096401 (2009). https://doi.org/10.1117/1.3216575
Cadik, M., Wimmer, M., Neumann, L., et al.: Evaluation of HDR tone mapping methods using essential perceptual attributes. Computer Gr.-UK 32(3), 330–349 (2008). https://doi.org/10.1016/j.cag.2008.07.007
Duan, J., Bressan, M., Dance, C., et al.: Tone-mapping high dynamic range images by novel histogram adjustment. Pattern Recogn. 43(5), 1847–1862 (2010). https://doi.org/10.1016/j.patcog.2009.12.006
Durand, F., Dorsey, J.: Fast bilateral filtering for the display of high-dynamic-range images. ACM Trans. Gr. 21(3), 257–266 (2002)
Farbman, Z., Fattal, R., Lischinski, D., et al.: Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans. Gr. 27(3), 67 (2008). https://doi.org/10.1145/1360612.1360666
Hassan, F., Carletta, J.E.: An FPGA-based architecture for a local tone-mapping operator. J. Real-Time Image Process. 2(4), 293–308 (2007). https://doi.org/10.1007/s11554-007-0056-7
Karaduzovic-Hadziabdic, K., Hasi-Telalovi, J.H., Mantiuk, R.K.: Assessment of multi-exposure HDR image deghosting methods. Computer Gr.-UK 63, 1–17 (2017). https://doi.org/10.1016/j.cag.2017.01.002
Katsaggelos, A.K., Biemond, J., Schafer, R.W., et al.: A regularized iterative image restoration algorithm. IEEE Trans. Signal Process. 39(4), 914–929 (1991). https://doi.org/10.1109/78.80914
Lapray, P.J., Heyrman, B., Ginhac, D.: Hardware-based smart camera for recovering high dynamic range video from multiple exposures. Opt. Eng. 53(10), 102110 (2014). https://doi.org/10.1117/1.OE.53.10.102110
Lapray, P.J., Heyrman, B., Ginhac, D.: HDR-ARtiSt: an adaptive real-time smart camera for high dynamic range imaging. J. Real-Time Image Process. 12(4), 747–762 (2016). https://doi.org/10.1007/s11554-013-0393-7
Lee, S.H., Woo, H., Kang, M.G.: Global illumination invariant object detection with level set based bimodal segmentation. IEEE Trans. Circuits Syst. Video Technol. 20(4), 616–620 (2010). https://doi.org/10.1109/TCSVT.2010.2041824
Mendis, S.K., Kemeny, S.E., Gee, R.C., et al.: CMOS active pixel image sensors for highly integrated imaging systems. IEEE J. Solid-State Circuits 32(2), 187–197 (1997). https://doi.org/10.1109/4.551910
Nosko, S., Musil, M., Zemcik, P., et al.: Color HDR video processing architecture for smart camera. J. Real-Time Image Process. 17(3), 555–566 (2020). https://doi.org/10.1007/s11554-018-0810-z
Popovic, V., Seyid, K., Pignat, E., et al.: Multi-camera platform for panoramic real-time HDR video construction and rendering. J. Real-Time Image Process. 12(4), 697–708 (2016). https://doi.org/10.1007/s11554-014-0444-8
Tang, X.D., Qian, Y.S., Kong, X.Y., et al.: A high-dynamic range CMOS camera based on dual-gain channels. J. Real-Time Image Process. 17(3), 703–712 (2020). https://doi.org/10.1007/s11554-019-00877-8
Tsai, C.Y., Huang, C.H.: Real-time implementation of an adaptive simultaneous dynamic range compression and local contrast enhancement algorithm on a GPU. J. Real-Time Image Process. 16(2), 321–337 (2019). https://doi.org/10.1007/s11554-015-0532-4
Winnemöller, H., Olsen, S.C., Gooch, B.: Real-Time video abstraction. ACM Trans. Gr. 25(3), 1221–1226 (2006). https://doi.org/10.1145/1141911.1142018
Zhou, Z.Q., Dong, M.J., Xie, X.Z., et al.: Fusion of infrared and visible images for night-vision context enhancement. Appl. Opt. 55(23), 6480–6490 (2016). https://doi.org/10.1364/AO.55.006480
Zuo, C., Chen, Q., Liu, N., et al.: Display and detail enhancement for high-dynamic-range infrared images. Opt. Eng. 50(12), 127401 (2011). https://doi.org/10.1117/1.3659698
Jacquot, B.C., Johnson-Williams, N.: Multiple-samples-method enabling high dynamic range imaging for high frame rate CMOS image sensor by FPGA and co-processor. 8th Conference of Optics and Photonics for Information Processing. 9216: 921609(2014). doi: https://doi.org/10.1117/12.2062284
Kuang, J.T., Liu, C.M., Johnson, G.M.: Evaluation of HDR image rendering algorithms using real-world scenes. 30th International Congress of Imaging Science. 256- (2006).
Liu, C., Freeman, W.T., Szeliski, R., et al.: Noise estimation from a single image. 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1(0), 901–908(2006).
Mann, S., Lo, R.C.H., Ovtcharov, K., et al.: Realtime HDR (High Dynamic Range) video for eyetap wearable computers, FPGA-based seeing aids, and glasseyes (EyeTaps). 2012 25th IEEE Canadian Conference on Electrical and Computer Engineering. (2012). doi: https://doi.org/10.1109/CCECE.2012.6335012
Pham, C.C., Ha, S.V.U., Jeon, J.W.: Adaptive guided image filtering for sharpness enhancement and noise reduction. 5th Pacific-Rim Symposium on Video and Image Technology. 7087 323– 334(2011).
Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. 6th International Conference on Computer Vision. 839–846 (1998). doi: https://doi.org/10.1109/ICCV.1998.710815
Zhao, H., Shi, B.X., Fernandez-Cull, C., et al.: Unbounded high dynamic range photography using a modulo camera. 2015 IEEE International Conference on Computational Photography. 137: 46 (2015).
Funding
This work was supported by the Shandong Provincial Natural Science Foundation (Grant No. ZR2019MF010), the Science and Technology Program for Higher Education of Shandong Province (Grant No. J18KA320), and the National Defense Basic Research Program of China (Grant No. JCKY2018208B016).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflicts of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lang, YZ., Qian, YS., Wang, HG. et al. A real-time high dynamic range intensified complementary metal oxide semiconductor camera based on FPGA. Opt Quant Electron 54, 304 (2022). https://doi.org/10.1007/s11082-022-03679-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11082-022-03679-8