Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Low-latency histogram equalization for infrared image sequences: a hardware implementation

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

This work describes a hardware implementation of the contrast-limited adaptive histogram equalization algorithm (CLAHE). The intended application is the processing of image sequences from high-dynamic-range infrared cameras. The variant of histogram equalization implemented is the one most commonly used today. It involves dividing the image into tiles, computing a transformation function on each of them, and interpolating between them. The contrast-limiting is modified to facilitate the hardware implementation, and it is shown that the error introduced by this modification is negligible. The latency of the design is minimized by performing its successive steps simultaneously on the same frame and by exploiting the vertical blank pause between frames. The resource usage of the histogram equalization module and how it depends on its parameters has been determined by synthesis. The design has been synthesized and tested on a Xilinx FPGA. The implementation supports substituting other dynamic range reduction modules for the histogram equalization component by partial dynamic reconfiguration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. http://sipi.usc.edu/database/index.html.

  2. http://www.imagecompression.info/test_images/.

  3. The normalization factor is derived from the pixel count excluding bad pixels but including the secondary excess, which slightly biases the transformation function towards smaller pixel values. This could be corrected by deferring normalization until the transformation functions are used, when the secondary excess is known.

  4. Available from http://www.cvg.rdg.ac.uk/datasets/index.html.

  5. VHDL is one of the major hardware description languages. It stands for “VHSIC hardware description language”, where VHSIC stands for “very high speed integrated circuit”.

References

  1. Abramowitz, M., Stegun, I.A.: Handbook of Mathematical Functions. Dover Publication, New York (1964)

    MATH  Google Scholar 

  2. Alpha Data (2004) ADM-XP data sheet. Alpha Data Parallel Systems Ltd. http://www.alpha-data.co.uk/archive/adm-xp.pdf

  3. Alsuwailem, A.M., Aishebeili, S.A.: A new approach for real-time histogram equalization using FPGA. In: Proceedings of 2005 International Symposium on Intelligent Signal Processing and Communication Systems, pp 397–400 (2005)

  4. Braun, L., Göhringer, D., Perschke, T., Schatz, V., Hübner, M., Becker, J.: (2009) Adaptive real-time image processing exploiting two dimensional reconfigurable architecture. J. Real-Time Image Process. 4(2), 109–125. doi:10.1007/s11554-008-0095-8

    Article  Google Scholar 

  5. Claus, C., Müller, F.H., Zeppenfeld, J., Stechele, W.: (2007) A new framework to accelerate Virtex-II Pro dynamic partial self-reconfiguration. In: 21st IEEE International Parallel and Distributed Processing Symposium, Reconfigurable Architectures Workshop, pp 1–7

  6. Claus, C., Zhang, B., Stechele, W., Braun, L., Hübner, M., Becker, J.: (2008) A multi-platform controller allowing for maximum dynamic partial reconfiguration throughput. In: 2008 International Conference on Field Programmable Logic and Applications, pp 535–538

  7. Dongsheng, G., Nansheng, Y., Defu, P., Min, H., Xiaoyan, S., Ruolan, Z.: (2001) DSP+FPGA-based real-time histogram equalization system of infrared image. In: Society of Photo-Optical Instrumentation Engineers (SPIE) Conference 2001, vol. 4602, pp. 160–165. doi:10.1117/12.445721

  8. Ericksen, J.P., Pizer, S.M., Austin, J.D.: MAHEM: A multiprocessor engine for fast contrast-limited adaptive histogram equalization. In: SPIE, vol. 1233 Medical Imaging IV: Image Processing, pp 322–333 (1990)

  9. Ferguson, P.D., Arslan, T., Erdogan, A.T., Parmley, A.: Evaluation of contrast limited adaptive histogram equalization (clahe) enhancement on a FPGA. In: SoCC, pp 119–122 (2008)

  10. Ferryman, J.M.: (ed.) Tenth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS 2007), IEEE Computer Society. http://www.cvg.rdg.ac.uk/PETS2007/ (2007)

  11. Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 2nd edn. Prentice-Hall, New Jersey (2002)

  12. Gradshteyn, I.S., Ryzhik, I.M.: (2000) Table of integrals, series and products, 6th edn. Academic Press, New York

  13. Hübner, M., Becker, T., Becker, J.: Real-time lut-based network topologies for dynamic and partial FPGA self-reconfiguration. In: 17th Symposium on Integrated Circuits and Systems Design (SBCCI04), pp. 28–32 (2004)

  14. Hummel, R.A.: (1975) Histogram modification techniques. Comput. Graphics Image Process. 4 , 209–224

    Article  MathSciNet  Google Scholar 

  15. Hummel, R.A.: Image enhancement by histogram transformation.Comput. Graphics Image Process. 6, 184–195 (1977)

    Article  Google Scholar 

  16. Jeon, I.J., Choi, B.M., Rhee, P.K.: Evolutionary reconfigurable architecture for robust face recognition. In: International Parallel and Distributed Processing Symposium, IEEE Computer Society, Los Alamitos, CA, USA, p. 192a. doi:10.1109/IPDPS.2003.1213356 (2003)

  17. Ketcham, D.J., Lowe, R.W., Weber, J.W.: Image enhancement techniques for cockpit displays. Technical report, Hughes Aircraft (1974)

  18. Ketcham, D.J., Lowe, R.W., Weber, J.W.: Real-time image enhancement techniques. In: Seminar on Image Processing, Hughes Aircraft, pp. 1–6 (1976)

  19. Kurak, C.W.: Adaptive histogram equalization: a parallel implementation. In: Proceedings of the Fourth Annual IEEE Symposium on Computer-Based Medical Systems, pp. 192–199 (1991)

  20. Li, X., Ni, G., Cui, Y., Pu, T., Zhong, Y.: Real-time image histogram equalization using FPGA. In: Zhou, L., Li, C.S. (eds) Society of Photo-Optical Instrumentation Engineers (SPIE) Conference 1998, vol. 3561, pp. 293–299 (1998)

  21. Lysaght, P., Blodget, B., Mason, J., Young, J., Bridgford, B.: Enhanced architectures, design methodologies and CAD tools for dynamic reconfiguration of Xilinx FPGAs. In: 2006 International Conference on Field Programmable Logic and Applications, pp. 12–17 (2006)

  22. NEMA (2009) Digital Imaging and Communications in Medicine (DICOM) Standard, Part PS 3.14. ftp://medical.nema.org/medical/dicom/2009/09_14pu.pdf

  23. Pizer, S.M.: Intensity mappings for the display of medical images. In: Functional Mapping of Organ Systems and Other Computer Topics, Society of Nuclear Medicine (1981)

  24. Pizer, S.M., Amburn, E.P., Austin, J.D., Cromartie, R., Geselowitz, A., Greer, T., ter Haar Romeny, B., Zimmerman, J.B., Zuiderveld, K.: Adaptive histogram equalization and its variations. Comput. Vis. Graphics Image Process. 39, 355–368 (1987)

    Google Scholar 

  25. Reza, A.M.: Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement. J. VLSI Signal Process. 38, 35–44 (2004)

    Article  Google Scholar 

  26. Salcic, Z., Sivaswamy, J.: IMECO: a reconfigurable FPGA-based image enhancement co-processor framework. Real-Time Imaging 5(6), 385–395 (1999). doi:10.1006/rtim.1998.0134

    Article  Google Scholar 

  27. Xilinx (2009) Spartan-6 Family Overview. Xilinx Inc. http://www.xilinx.com/support/documentation/data_sheets/ds160.pdf

Download references

Acknowledgments

The author thanks Diana Göhringer for vital tutoring in the use of the Xilinx Partial Reconfiguration design flow. Furthermore, he thanks Diana Göhringer and Thomas Perschke for proofreading the manuscript, and Lars Braun for helping overcome our FPGA board’s resistance to partial bit streams.

Author information

Authors and Affiliations

  1. Fraunhofer Institut für Optronik, Systemtechnik und Bildauswertung, Gutleuthausstraße 1, 76275, Ettlingen, Germany

    Volker Schatz

Authors
  1. Volker Schatz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Volker Schatz.

Appendix

Appendix

This appendix presents an overall functional description of the CLAHE implementation and shows how the components described in the main part of the paper work together. Readers intending to reproduce the design should also consult [25], where an implementation of the unmodified CLAHE algorithm is described in great detail.

Figure 11 displays a functional schematic of the operations performed while pixel values arrive at the module. Thick lines represent numerical quantities, while thin lines are boolean signals. Figure 10 defines the utilized components. As pixel values arrive, they are added to the global histogram necessary for the range determination of the preprocessing step. The windowing transformation based on the range from the previous frame is applied to the values. The range check component determines which pixel values are within the valid range. Its output prevents bad pixels from being processed by the CLAHE.

Fig. 10
figure 10

Key to the component symbols used in Figs. 11 and 12. The windowing component maps the valid input data range to the complete output data range. The range check component outputs a logical 1 if the input value is in the valid range

Full size image
Fig. 11
figure 11

Operation of the CLAHE circuit while pixel data arrive during the frame, see Fig. 5 for the tile histogram generation circuit

Full size image

In the right part of the figure, the pixel value is added to the appropriate tile histogram. This task is performed by the circuit shown in Fig. 5 the operation of which is qualified by the data valid and in-range signals. Simultaneously, it is transformed using transformation functions computed from the tile histograms of the previous frame. The transformation results are then interpolated to produce the result pixel value. See below for how the right tiles are selected.

The dotted boxes in Fig. 11 represent submodules within which other operations are performed during the inter-frame gap. These operations are shown in Fig. 12. The pixel value counters in that figure start from zero and count up to the maximum once during each vertical blank interval. The circuit on the left accumulates the global histogram and keeps updating the range registers until the corresponding boundary of the valid range has been passed. The quantities BadPixLow and BadPixHigh are the numbers of pixels at the bottom and the top of the histogram to discard as bad pixels of the camera.

Fig. 12
figure 12

Determination of the valid pixel range and generation of the tile transformation functions during the vertical blank gap. The dotted boxes represent the same submodules as in Fig. 11. A copy of the circuit on the right exists for each tile

Full size image

The submodule on the right which contains the tile histograms and transformation functions operates only on pixels which are within the range of the preprocessing step. L is the clip limit. The registers for the number of in-range pixels, the excess and the number of saturated bins are provided by the tile histogram generation circuit, see Fig. 5. The amount to be redistributed per bin is computed, it is added to the bin value, and that value is clipped again. The result is accumulated, normalized by division by the pixel count and written to the transformation LUT.

Selecting the right tiles for tile histogram generation and interpolation requires auxiliary quantities that are derived from the pixel coordinates. These are more concisely expressed as formulas than block diagrams. The horizontal tile indices are derived from the X coordinate as follows:

$$ \begin{aligned}&\hbox{HistTileX} = X / \hbox{TileWidth} \\ &\hbox{InterTileLeft} = (X - \hbox{TileWidth} / 2) / \hbox{TileWidth} \\ &\hbox{InterTileRight} = \hbox{InterTileLeft} + 1 \\ &\hbox{InterTileXFrac} = (X / \hbox{TileWidth})\;\hbox{mod}\; 1.0 - 0.5 \end{aligned} $$
(11)

HistTileX is the horizontal index of the tile histogram to which a pixel value is added. InterTileLeft and InterTileRight are the column indices of the tile transformation functions to use for interpolation. All divisions in the first two formulas are truncating integer divisions. InterTileXFrac is the fractional horizontal interpolation coefficient. Vertical tile indices and the vertical interpolation coefficient are computed analogously from the Y coordinate. Pixels in the image margins or corners have to bypass one or both of the interpolation steps, see Fig. 1 in Sect. 2.2.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Schatz, V. Low-latency histogram equalization for infrared image sequences: a hardware implementation. J Real-Time Image Proc 8, 193–206 (2013). https://doi.org/10.1007/s11554-011-0204-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-011-0204-y

Keywords

Search

Navigation