Analysis and Evaluation of

Sampled
Imaging
Systems

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 Tutorial Texts Series
 Nanotechnology: A Crash Course, Raúl J. Martín-Palma, Akhlesh Lakhtakia, Vol. TT86
 Direct-Detection LADAR Systems, Richard D. Richmond, Stephen C. Cain, Vol. TT85
 Optical Design: Applying the Fundamentals, Max J. Riedl, Vol. TT84
 Infrared Optics and Zoom Lenses, Second Edition, Allen Mann, Vol. TT83
 Optical Engineering Fundamentals, Second Edition, Bruce H. Walker, Vol. TT82
 Fundamentals of Polarimetric Remote Sensing, John Schott, Vol. TT81
 The Design of Plastic Optical Systems, Michael P. Schaub, Vol. TT80
 Radiation Thermometry: Fundamentals and Applications in the Petrochemical Industry, Peter Saunders,
Vol. TT78
 Matrix Methods for Optical Layout, Gerhard Kloos, Vol. TT77
 Fundamentals of Infrared Detector Materials, Michael A. Kinch, Vol. TT76
 Practical Applications of Infrared Thermal Sensing and Imaging Equipment, Third Edition, Herbert
Kaplan, Vol. TT75
 Bioluminescence for Food and Environmental Microbiological Safety, Lubov Y. Brovko, Vol. TT74
 Introduction to Image Stabilization, Scott W. Teare, Sergio R. Restaino, Vol. TT73
 Logic-based Nonlinear Image Processing, Stephen Marshall, Vol. TT72
 The Physics and Engineering of Solid State Lasers, Yehoshua Kalisky, Vol. TT71
 Thermal Infrared Characterization of Ground Targets and Backgrounds, Second Edition, Pieter A. Jacobs,
Vol. TT70
 Introduction to Confocal Fluorescence Microscopy, Michiel Müller, Vol. TT69
 Artificial Neural Networks: An Introduction, Kevin L. Priddy and Paul E. Keller, Vol. TT68
 Basics of Code Division Multiple Access (CDMA), Raghuveer Rao and Sohail Dianat, Vol. TT67
 Optical Imaging in Projection Microlithography, Alfred Kwok-Kit Wong, Vol. TT66
 Metrics for High-Quality Specular Surfaces, Lionel R. Baker, Vol. TT65
 Field Mathematics for Electromagnetics, Photonics, and Materials Science, Bernard Maxum, Vol. TT64
 High-Fidelity Medical Imaging Displays, Aldo Badano, Michael J. Flynn, and Jerzy Kanicki, Vol. TT63
 Diffractive Optics–Design, Fabrication, and Test, Donald C. O’Shea, Thomas J. Suleski, Alan D.
Kathman, and Dennis W. Prather, Vol. TT62
 Fourier-Transform Spectroscopy Instrumentation Engineering, Vidi Saptari, Vol. TT61
 The Power- and Energy-Handling Capability of Optical Materials, Components, and Systems, Roger M.
Wood, Vol. TT60
 Hands-on Morphological Image Processing, Edward R. Dougherty, Roberto A. Lotufo, Vol. TT59
 Integrated Optomechanical Analysis, Keith B. Doyle, Victor L. Genberg, Gregory J. Michels, Vol. TT58
 Thin-Film Design: Modulated Thickness and Other Stopband Design Methods, Bruce Perilloux, Vol. TT57
 Optische Grundlagen für Infrarotsysteme, Max J. Riedl, Vol. TT56
 An Engineering Introduction to Biotechnology, J. Patrick Fitch, Vol. TT55
 Image Performance in CRT Displays, Kenneth Compton, Vol. TT54
 Introduction to Laser Diode-Pumped Solid State Lasers, Richard Scheps, Vol. TT53
 Modulation Transfer Function in Optical and Electro-Optical Systems, Glenn D. Boreman, Vol. TT52
 Uncooled Thermal Imaging Arrays, Systems, and Applications, Paul W. Kruse, Vol. TT51
 Fundamentals of Antennas, Christos G. Christodoulou and Parveen Wahid, Vol. TT50
 Basics of Spectroscopy, David W. Ball, Vol. TT49
 Optical Design Fundamentals for Infrared Systems, Second Edition, Max J. Riedl, Vol. TT48
 Resolution Enhancement Techniques in Optical Lithography, Alfred Kwok-Kit Wong, Vol. TT47
 Copper Interconnect Technology, Christoph Steinbrüchel and Barry L. Chin, Vol. TT46
 Optical Design for Visual Systems, Bruce H. Walker, Vol. TT45
 Fundamentals of Contamination Control, Alan C. Tribble, Vol. TT44
 Evolutionary Computation: Principles and Practice for Signal Processing, David Fogel, Vol. TT43
 Infrared Optics and Zoom Lenses, Allen Mann, Vol. TT42

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 Analysis and Evaluation of

Sampled
Imaging
Systems

Richard H. Vollmerhausen
Donald A. Reago, Jr.
Ronald G. Driggers

Tutorial Texts in Optical Engineering

Volume TT87

Bellingham, Washington USA

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 Library of Congress Cataloging-in-Publication Data

Vollmerhausen, Richard H.
Analysis and evaluation of sampled imaging systems / Richard H. Vollmerhausen,
Donald Reago, and Ronald G. Driggers.
p. cm. -- (Tutorial texts in optical engineering ; v. TT87)
Includes bibliographical references and index.
ISBN 978-0-8194-8077-4 (alk. paper)
1. Imaging systems--Image quality. 2. Image processing--Statistical methods.
3. Fourier analysis. 4. Sampling (Statistics) I. Reago, Donald. II. Driggers, Ronald
G. III. Title.
TK8315.V6495 2010
621.36'7--dc22
2009053629

Published by

SPIE
P.O. Box 10
Bellingham, Washington 98227-0010 USA
Phone: +1 360.676.3290
Fax: +1 360.647.1445
Email: books@spie.org
Web: http://spie.org

Copyright © 2010 Society of Photo-Optical Instrumentation Engineers (SPIE)

All rights reserved. No part of this publication may be reproduced or distributed in any
form or by any means without written permission of the publisher.

The content of this book and CD-ROM reflects the work and thought of the author(s).
Every effort has been made to publish reliable and accurate information herein, but the
publisher is not responsible for the validity of the information or for any outcomes
resulting from reliance thereon.

The authors make no warranties that the software programs contained on the enclosed
CD-ROM are free of error, or are consistent with any standard of merchantability, or will
meet the requirements for any particular application. CD-ROM Copyright © 2010 RHV
Consulting, LLC, 760 Jacktown Rd., Lexington, VA 24450.

Printed in the United States of America.

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 Introduction to the Series
Since its inception in 1989, the Tutorial Texts (TT) series has grown to more than
8 titles covering many diverse fields of science and engineering. The initial idea
for the series was to make material presented in SPIE short courses available to
those who could not attend and to provide a reference text for those who could.
Thus, many of the texts in this series are generated by augmenting course notes
with descriptive text that further illuminates the subject. In this way, the TT
becomes an excellent stand-alone reference that finds a much wider audience
than only short course attendees.
Tutorial Texts have grown in popularity and in the scope of material covered
since 1989. They no longer necessarily stem from short courses; rather, they are
often generated by experts in the field. They are popular because they provide a
ready reference to those wishing to learn about emerging technologies or the
latest information within their field. The topics within the series have grown from
the initial areas of geometrical optics, optical detectors, and image processing to
include the emerging fields of nanotechnology, biomedical optics, fiber optics,
and laser technologies. Authors contributing to the TT series are instructed to
provide introductory material so that those new to the field may use the book as a
starting point to get a basic grasp of the material. It is hoped that some readers
may develop sufficient interest to take a short course by the author or pursue
further research in more advanced books to delve deeper into the subject.
The books in this series are distinguished from other technical monographs
and textbooks in the way in which the material is presented. In keeping with the
tutorial nature of the series, there is an emphasis on the use of graphical and
illustrative material to better elucidate basic and advanced concepts. There is also
heavy use of tabular reference data and numerous examples to further explain the
concepts presented. The publishing time for the books is kept to a minimum so
that the books will be as timely and up-to-date as possible. Furthermore, these
introductory books are competitively priced compared to more traditional books
on the same subject.
When a proposal for a text is received, each proposal is evaluated to
determine the relevance of the proposed topic. This initial reviewing process has
been very helpful to authors in identifying, early in the writing process, the need
for additional material or other changes in approach that would serve to
strengthen the text. Once a manuscript is completed, it is peer reviewed to ensure
that chapters communicate accurately the essential ingredients of the science and
technologies under discussion.
It is my goal to maintain the style and quality of books in the series and to
further expand the topic areas to include new emerging fields as they become of
interest to our reading audience.

James A. Harrington
Rutgers University

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 Contents
Preface .................................................................................................... xi
Acronyms and Abbreviations ............................................................. xiii

Part I Analysis of Sampled Imaging Systems

Chapter 1 The Sampling Process ....................................................... 3

1.1 Description of a Sampled Imager ....................................................3
1.2 Description of the Sampling Process ...............................................6
1.3 Linearity and Shift Invariance .........................................................9
1.4 Signal Reconstruction ....................................................................13
1.5 Three Ways of Viewing the Sampling Process .............................15
1.5.1 The displayed image as the sum of its parts .....................16
1.5.2 The display as a filter of the image samples .....................18
1.5.3 The display as a filter of the sampled image ....................20
1.6 The Sampling Theorem .................................................................24
1.6.1 Theory ..............................................................................24
1.6.2 Example ............................................................................26
1.6.3 Discussion ........................................................................29
Bibliography ............................................................................................29

Chapter 2 Fourier Integral Representation of an Optical Image .... 31

2.1 Linear Shift-Invariant Optical Systems .........................................31
2.2 Equivalence of Spatial and Frequency Domain Filters .................34
2.3 Reducing LSI Imager Analysis to One Dimension .......................36
2.4 Perspectives on One-Dimensional Analysis ..................................42
2.5 Imager Modulation Transfer Functions .........................................45
2.5.1 Imager components ..........................................................45
2.5.2 Line-of-sight jitter ............................................................47
2.5.2.1 Reduction of line-of-sight jitter by eye
tracking.............................................................50
2.5.2.2 Effect of temporal sampling on line-of-sight
jitter ..................................................................52
2.5.3 Electronic stabilization .....................................................53
2.5.4 Motion blur .......................................................................55
vii

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 viii Contents

2.5.5 Field replication ................................................................56

2.5.6 Analog electronic filters ...................................................56
2.5.7 Display MTF ....................................................................57
Bibliography ............................................................................................59

Chapter 3 Sampled Imager Response Function .............................. 61

3.1 Fourier Transform of a Sampled Image.........................................63
3.2 The Sampled Imager Response Function ......................................67
3.3 Examples of Sampled Imager Response Functions .......................69
3.3.1 Example 1: The pictures of Lena in Chapter 1 .................69
3.3.2 Example 2: Effect of changing sample rate ......................71
3.3.3 Example 3: Midwave thermal imager ..............................79
3.3.4 Example 4: Two-dimensional SIR example .....................81
Bibliography ............................................................................................85

Chapter 4 Sampled Imager Optimization ......................................... 87

4.1 Interpolation Implementation ........................................................89
Bibliography ............................................................................................97

Chapter 5 Interlace and Dither .......................................................... 99

5.1 Sampling Improvement with Static Scene ...................................102
5.2 Resolution and Sensitivity ...........................................................107
5.3 Effect of Scene-to-Sensor Motion ...............................................110
Bibliography ..........................................................................................112

Part II Evaluating the Performance of Electro-Optical

Imagers

Chapter 6 Quantifying Visual Task Performance .......................... 115

6.1 Specifying and Evaluating Field Performance ............................118
6.2 Factors That Influence Target Identification ...............................120
6.3 Measuring Target Signatures .......................................................121
6.4 Experimental Procedure ..............................................................122
6.5 Field Test Procedure ....................................................................125
6.6 Test Sets Other Than Tactical Vehicles.......................................127
6.7 Field Testing Using Bar Targets ..................................................129

Chapter 7 Evaluating Imager Resolution ....................................... 131

7.1 Imager Evaluation Procedure ......................................................132
7.2 Modeling Gain, Level, and the User Interface ............................134
7.3 Observer Vision ...........................................................................138
7.4 Predicting Probability of Identification .......................................143

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 Contents ix

7.4.1 Comparing experimental data to model predictions .......145

7.5 Test Sets Other Than Tactical Vehicles.......................................146
Bibliography ..........................................................................................147

Chapter 8 Quantifying the Effect of Aliasing on Visual

Task Performance........................................................... 149
8.1 Model Treatment of Spatial Noise...............................................150
8.2 Treatment of Temporal Noise in Detectivity Versus
Photon-Counting Models.............................................................152
8.3 Relating Target and Imager Coordinate Systems ........................155
8.4 Spatial Scaling of Aliasing Noise ................................................161
Bibliography ..........................................................................................163

Chapter 9 Thermal Imager Topics .................................................. 165

9.1 Effective Blackbody Temperature ...............................................168
9.2 Signal and Noise in the Detectivity Model ..................................172
9.3 Thermal Imager Contrast Threshold Function ............................175
9.4 Adding Aliasing Noise ................................................................177
9.5 Predicting Range Performance ....................................................179
9.6 Modeling Contrast Enhancement and Boost ...............................180
9.7 Minimum Resolvable Temperature .............................................182
9.7.1 Predicting minimum resolvable temperature ..................183
9.7.2 Predicting sampled imager minimum resolvable
temperature .....................................................................187
9.7.3 Improving the minimum resolvable temperature
procedure ........................................................................190
Bibliography ..........................................................................................191

Chapter 10 Imagers of Reflected Light............................................. 193

10.1 Calculating Target Set Contrast ...................................................193
10.2 System Contrast Threshold Function ..........................................196
10.2.1 Interlace ..........................................................................200
10.2.2 Snapshot and frame integration ......................................201
10.3 Predicting Range Performance ....................................................202
Bibliography ..........................................................................................203

Part III Applications

Chapter 11 Computer Programs and Application Data .................. 207

11.1 Optics Modulation Transfer Function .........................................208
11.1.1 Thermal imagers .............................................................208
11.1.2 Imagers of reflected light................................................211
11.2 Display Modulation Transfer Function .......................................213

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 x Contents

11.2.1 Cathode ray tubes ...........................................................213

11.2.2 Liquid crystal displays ....................................................216
11.2.3 Display interface format .................................................218
11.3 Atmospheric Transmission and Turbulence ................................218
11.3.1 Atmosphere in the reflective model................................219
11.3.2 Atmosphere in the thermal model ..................................221
11.3.3 Atmospheric turbulence..................................................223
11.4 Detector Calculations ..................................................................223
11.4.1 Detector noise .................................................................223
11.4.2 Detector modulation transfer function ............................226
11.5 Computer Program Description ...................................................227
11.6 Imager Analysis Using the Programs ..........................................235
11.6.1 Imager resolution ............................................................235
11.6.2 System contrast threshold function.................................236
11.6.3 Range plots .....................................................................237
References .............................................................................................240
Bibliography ..........................................................................................240

Chapter 12 Infrared Focal Plane Arrays ........................................... 241

12.1 Photon Detector Infrared Focal Plane Arrays..............................241
12.1.1 Photon detector basic principles ....................................241
12.1.2 Readout integrated circuit...............................................246
12.1.3 Photon detector dark current ..........................................247
12.2 IR FPA Performance Characterization ........................................251
12.2.1 Responsivity and detectivity background-limit
performance ....................................................................252
12.2.2 Flux-based signal-to-noise ratio .....................................255
12.3 Commonly Available Photon Detector FPAs ..............................258
12.3.1 Indium antimonide (InSb) detectors ...............................258
12.3.2 Quantum well infrared photoconductor (QWIP)
detectors..........................................................................261
12.3.3 Mercury cadmium telluride (HgCdTe) detectors ...........264
12.4 Uncooled Detectors .....................................................................269
12.4.1 Introduction ....................................................................269
12.4.2 Signal-to-noise ratio and performance limits .................271
12.4.3 Typical uncooled detectors .............................................273
References .............................................................................................274

Appendix Observer Vision Model ................................................... 277

A.1 Contrast Threshold Function .......................................................277
A.2 Engineering Model of the Eye .....................................................278
References .............................................................................................282

Index ..................................................................................................... 285

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 Preface

This tutorial is written for the design engineer or system analyst interested in
quantifying the performance of electro-optical imagers. Advancing technology in
detector arrays, flat panel displays, and digital image processing provide new
opportunities to expand imaging applications and enhance system performance.
Technical managers and design engineers are faced with evaluating the cost,
weight, and performance of an ever-expanding selection of technology options.
This book provides the theory and procedures for performance assessment.
This text supersedes Analysis of Sampled Imaging Systems, which was
published by SPIE Press in 2000. Part I updates the earlier work. Part II discusses
performance evaluation of electro-optical imagers. Part III provides computer
programs and up-to-date information on detector arrays, optics, and display
options. This book provides the theory, procedures, and information needed to
evaluate and compare the performance of available imaging technologies.
Our prior work Analysis of Sampled Imaging Systems focused on the
mathematical formulism needed to analyze sampled imagery. That book
described the sampled imager response (SIR) function. SIR quantified sampled
imager aliasing as well as the system transfer response. Fourier transform theory
was used to describe and quantify sampling artifacts such as display raster,
blocky images, and the loss or alteration of image detail due to aliasing.
However, the metrics provided by the earlier book were “rules of thumb.”
Sampled imager design rules were based on experience and experimentation. No
theory existed to quantify the effect of aliasing on visual task performance. The
earlier work provided guidance on how to optimize sampled imagers by
minimizing aliasing. Analysis of Sampled Imaging Systems did not provide a
procedure to quantify the impact of aliasing on performance.
In the intervening years since the first book, we have discovered that the
effect of aliasing on targeting performance is predictable by treating aliasing as
noise. This book presents a resolution metric that predicts the effect of imager
blur, noise, and sampling on the probability of correctly identifying targets. This
new publication includes quantitative procedures for evaluating target acqui-
sition performance.
Part I of this book includes all of the pertinent material from Analysis of
Sampled Imaging Systems. The first five chapters remain substantially the same
as the previous work. These chapters introduce sampling concepts and describe
the differences between shift-variant and shift-invariant systems. Chapter 2 on
Fourier optics is extensively rewritten. The errors associated with assuming
xi

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 xii Preface

separability in Cartesian coordinates are discussed, and examples are provided.

The blurs associated with vibration and electronic stabilization are described. In
Chapter 3, additional examples are added to better describe the SIR function. The
focus of Chapters 1 through 5 remains the same, however. These chapters
provide the mathematical tools needed to analyze sampled imagers.
Part II of this new book includes Chapters 6 through 10. This new material
describes electro-optical imager evaluation. Chapter 6 describes target identifi-
cation experiments. These experiments quantify visual task performance. Chapter
7 describes a resolution metric that predicts the probability of identifying targets.
Chapter 7 also discusses the relationship between imager resolution and field
performance. Chapter 8 explains aliasing as noise theory. For some years, we
have known how to predict the effect of noise on target acquisition. Aliasing as
noise theory predicts the effect of aliasing on target acquisition. Chapters 9 and
10 provide details on analyzing thermal imagers and imagers of reflected light,
respectively.
Part III provides computer programs that implement the theory. These
programs calculate the resolution of thermal and reflected light imagers. The
programs are used to evaluate expected target acquisition performance and to
compare imagers and assess the benefit or penalty of design changes.
Information is also provided to help make realistic assessments of imager
performance. The book discusses optical performance and provides the
characteristics of typical, good, and ideal lens systems. The book also contains
information on a variety of display formats and interfaces, as well as detailed
information on available focal plane arrays (FPAs). The information is presented
in written form and is also coupled to the computer programs.
Particular emphasis is placed on theory and practice for the wide variety of
available infrared FPAs. Technologies represented include InSb, HgCdTe,
QWIP, and uncooled thermal arrays. Information is provided on the quantum
efficiency, blur, crosstalk, and noise characteristics of each technology. The
detector and array dimensions of available FPAs are provided. The availability of
current information on optics, display, and FPA subassemblies allows the model
user to make quick and realistic performance assessments of electro-optical
imager designs.

Richard H. Vollmerhausen
Donald A. Reago, Jr.
Ronald G. Driggers
February 2010

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 Acronyms and Abbreviations
2afc two alternative forced choice
A ampere
Å angstrom
AAN aliasing as noise
AR antireflection
A/W units (amperes/watt) for spectral current responsivity
BDI buffered direction injection
BLIP background-limited photoconductor
CCD charge-coupled device
CMOS complementary metal-oxide semiconductor
CRT cathode ray tube
CSF contrast sensitivity function
CTF contrast threshold function
CTIA capacitive transimpedance amplifier
DC direct current
DI direct injection
DLHJ double-layer heterojunction
erf error function
E field electric field
E-stab electronic stabilization
ezoom electron zoom
FF fill factor
FOV field of view
FPA focal plane array
FPN fixed pattern noise
FWHM full width at half maximum
HDTV high-definition television
HgCdTe mercury cadmium telluride
HUD head-up display
IFOV instantaneous field of view
InSb indium antimonide
IR infrared
LACE local-area contrast enhancement
LCD liquid crystal display
LOS line of sight
LSI linear and shift invariant

xiii

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 Acronyms and Abbreviations xiv

LWIR long-wave infrared

MHz megahertz
MKS meter-kilogram-second
MRT minimum resolvable temperature
MTF modulation transfer function
MWIR midwave infrared
NEP noise equivalent power
NETD noise equivalent temperature difference
NIR near infrared
NUC nonuniformity correction
OTF optical transfer function
Pf picofarad
PID probability of correct identification
psf point spread function
PSQ square of the Pearson’s coefficient
PV photovoltaic
QE quantum efficiency
QWIP quantum well infrared photoconductor
RC resistor-capacitor
rms root mean square
ROIC readout integrated circuit
RSS square root of the sum of squares
SGR sky-to-ground ratio
SIR sampled imager response
SIT system intensity transfer
SMAG system magnification
S/N signal-to-noise ratio
SOM specific object model
SRH Shockley-Read-Hall
SSD signal spectral density
str steradian
SWIR short-wave infrared
TCR temperature coefficient of resistance
tgt target
TOD triangle orientation discrimination
TTP targeting task performance
UAV unmanned aerial vehicle
VFOV vertical field of view
W watt
WFOV wide field of view

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 Chapter 1
The Sampling Process
This chapter provides an introduction to several topics. First, the physical pro-
cesses involved in sampling and displaying an image are discussed. In a staring
sensor, the image is blurred by the optics, and then the detector array both blurs
and samples the image. The detector samples are used to construct the displayed
picture. The physical processes that occur in a sampled imager are described.
When analyzing sampled systems, it is convenient to conceptually separate
the preblur, sampling, and postblur (display blur) attributes of the system. These
three steps in the generic sampling process are described, and the importance of
each step discussed.
Next, the system properties known as linearity and shift invariance are
described. Systems that are both linear and shift invariant can be characterized by
a transfer function. Circuits and optical systems, for example, are characterized
by their modulation transfer functions (MTFs). This chapter describes the
transfer function concept using an electrical low-pass filter as an example. For
linear shift-invariant systems, the transfer function provides a quantitative way of
characterizing system behavior.
Sampled systems are linear, so Fourier transform theory can be used to
analyze sampled systems. However, sampled systems are not shift invariant. It is
the lack of the shift-invariance property that differentiates sampled systems from
other systems and makes sampled systems more difficult to analyze. The lack of
shift invariance in a sampled system is illustrated. Also, the reasons that a
sampled system cannot be assigned a transfer function are explained.
At the end of this chapter, three different mathematical ways of representing
the sampling processes are presented. These three derivations correspond to three
different physical views of the sampling process. The physical basis of the
mathematical techniques used in Chapter 3 is described, and the role of the
display in a sampled system is put into perspective. For simplicity, many of the
examples in this chapter are one-dimensional, but the discussion and conclusions
apply to two-dimensional imagery.

1.1 Description of a Sampled Imager

Figure 1.1 shows a camera imaging a scene and an observer viewing the sampled
image on a display. The components of the camera are shown in Fig. 1.2. A lens

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 4 Chapter 1

images the scene onto a staring detector focal plane array (FPA). Diffraction and
aberrations blur the image. In the figure, the image on the FPA is upside-down as
well as blurred because the optical system inverts the image.
The detector array consists of a number of rows and columns of individual
detectors, as shown in the inset in Fig. 1.2. Photodetection occurs over the active
area of these individual detectors. The incoming light generates photo-electrons.
Since each detector integrates photo-signal over a finite active area, the detector
itself also blurs the image. Individual points in the optical image are summed
together if they fall on the same detector, and this summing of adjacent points
causes a blur.
The detector photocurrent is integrated (in a capacitor, charge well, or by
some other mechanism) for a period of time. Periodically (generally, every
sixtieth of a second in the U.S. or every fiftieth of a second in Europe), the
resulting signal charge is read out, and the integrator is reset. The amount of
charge from each detector depends directly on the intensity of light falling on that
detector. The charge output from each detector represents a sample of the lens-
and detector-blurred scene intensity. Note that, in the example shown in Fig. 1.2,
the active detector area does not cover the entire FPA. This array does not have a
100% fill factor.

scene
camera

lay
disp
Figure 1.1 Observer viewing the display of a sampled image.

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 The Sampling Process 5

Figure 1.2 Components of a camera. The lens images a scene onto the detector array.
Optical diffraction and aberrations in the lens blur the image. The detector further blurs the
image by integrating signal over the active detector area. Each detector provides one
sample of the blurred scene.

The two images in Fig. 1.3 summarize the action of the camera. Concept-
ually, the optical and detector blurs are lumped together and called the presample
blur. The image with optical and detector presample blur applied is shown in the
left-hand image in Fig. 1.3. The detectors then convert the light intensity at
specific locations in the blurred image to electrical signals. The electrical signals
represent image samples. In the right-hand image, the white dots indicate the
locations where the blurred image is sampled by the detector array.
A display device is used to reconstruct the image from the detector
(electrical) samples. The display device consists of an array of individual display
pixels. A “display pixel” is an individual light-emitting area on the display
surface. In the simplest case, the number and location of pixels in the display
correspond to the number and location of detectors in the FPA. The brightness of
each display pixel is proportional to the photo-signal from the corresponding
detector.
Figure 1.4 shows the display. An individual display pixel is shown in the
upper left-hand corner of the image. These pixels happen to be square. In this
example, there is one display pixel for each sensor sample shown in Fig. 1.3. The
intensity of each pixel shown in Fig. 1.4 is proportional to the photo-intensity at
the corresponding sample location in Fig. 1.3.

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 6 Chapter 1

Figure 1.3 Left-hand image is blurred by the optics and the detector. Right-hand image
shows the location of detector samples as white dots.

Figure 1.4 Display of a sampled image. An individual pixel is shown in the upper left-hand
corner. Each pixel on the display is illuminated in proportion to the amplitude of the
sample from the corresponding location in Fig. 1.3.

1.2 Description of the Sampling Process

Sampled imaging systems are characterized by a three-step process. First, the
original image of Lena shown in Fig. 1.5(a) is degraded by a presample blur. The
blur is caused by the combined effects of optical diffraction and aberrations, the
finite size and shape of the detector, camera vibration, motion smear, and other
effects. The presample blurred image is shown in Fig. 1.5(b). Next, the blurred
image is sampled. That is, some electronic mechanism is used to find the
amplitude of the blurred image at discrete points. In this example, the blurred
image in Fig. 1.5(b) is sampled at the points shown as white dots in Fig. 1.5(c).
The third step is reconstruction of the image. Each sensor sample controls the
intensity of a display pixel (or a group of display pixels), as shown in Fig. 1.5(d).
The shape and size (intensity distribution) of the display pixels determine the
postsample blur. The shape and size of an individual display pixel are shown in
the upper left-hand corner of Fig. 1.5(d). The postsample blur (reconstruction
blur) also includes any postsampling electronic filtering and eye blur.

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 The Sampling Process 7

(a) (b)

(c) (d)

Figure 1.5 (a) Original picture. (b) Presample blur applied to original picture. (c) Picture
showing locations of samples. (d) Picture showing reconstructed image with a single
display pixel shown in the upper left-hand corner.

All three stages of the sampling process are necessary: preblur of the image,
sampling, and postblur or reconstruction. Figure 1.6(a) shows the result of
sampling the original image without prefiltering [that is, Fig. 1.5(a) is sampled
rather than Fig. 1.5(b)]. In this case, aliasing hurts the final, displayed image.
Figure 1.5(d) looks more like Fig. 1.5(a) than Fig. 1.6(a) does.
Figure 1.6(b) shows the image samples displayed as points rather than as the
large pixels used in Fig. 1.5(d). In Fig. 1.6(b), the image is not blurred by the
display pixel, and the image cannot be integrated by the eye. To get a good
image, display reconstruction using a postsample blur is necessary.
As an illustration, move Fig. 1.6(b) close to the eye so that only points are
seen. Now, slowly move the figure away from the eye. Lena begins to appear as
the figure moves away because eye blur acts as a reconstruction filter.
Rules can be established for determining the optimum relationship between
preblur, sample spacing, and postblur. A well-sampled imaging system is one in
which the spacing (in milliradians or millimeters) between image samples is
small compared to the width of the presample blur. In this case, sampling
artifacts are not apparent in the image.

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 8 Chapter 1

(a) (b)

Figure 1.6 The picture in (a) shows the original image in Fig. 1.5(a) sampled without pre-
blur and then reconstructed in a similar manner to Fig. 1.5(d). Figure 1.5(d) looks more
like Fig. 1.5(a) than Fig. 1.6(a) does because some presample blurring is necessary. The
picture in (b) is constructed with pixels that are spatially much smaller than the pixel pitch.
The pixels in (b) do not blur the image, and the picture is very hard for the eye to integrate.
Postblur of the image samples is also necessary.

An undersampled imaging system is one in which the sample spacing is a

large fraction of the presample blur. Depending on scene content, the artifacts
caused by undersampling can limit the performance of the imaging system. Poor
sampling can corrupt the image by generating localized disturbances or artifacts.
The corruption results in shifting object points, lines, and edges. Poor sampling
can also modify the apparent width of an object or make a small object or detail
disappear. That is, a fence post imaged by an undersampled sensor can be seen as
thicker, thinner, or slightly misplaced.
An optimum postblur depends on the sample spacing and other factors. If the
display pixel is large compared to the sample spacing, then the image will be
blurred. If the display pixel is small compared to the sample spacing, then the
shape of the pixel itself might be visible. Display pixels that are individually
visible because of their size or intensity distribution will add spurious content to
the image. The spurious response due to poor image reconstruction can seriously
degrade sensor system utility.
Visible raster or a “pixelated” display makes it difficult for the observer to
visually integrate the underlying sensor imagery. This is certainly true for the
picture in Fig. 1.6(b), for example. However, the picture shown in Fig. 1.5(d) is
also degraded by the sharply demarcated display pixels. The picture of Lena in
Fig. 1.7 is generated using the same samples as were used to generate the picture
in Fig. 1.5(d). The picture in Fig. 1.7 is improved because the display pixel shape
provides a better match to the original image between sample points.

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms

 The Sampling Process 9

Figure 1.7 Picture of Lena reconstructed using the same samples as were used in Fig.
1.5(d). This picture is improved because the display pixel shape provides a better match
to the original image between sample points.

A well-designed system is the result of a trade-off between sample rate or

spacing, presample blur, and display reconstruction. Too much preblur can limit
performance by degrading resolution while wasting detector count. Too little
preblur can cause significant aliased content that limits imager performance. A
display blur that is too small can introduce spurious content such as visible raster
and pixelation effects that can ruin the displayed image. A display blur that is too
large will limit performance by degrading resolution. Achieving good perform-
ance with a sampled imager requires trade-offs between the pre- and postsample
blurs and the sample spacing.

1.3 Linearity and Shift Invariance

Systems that are both linear and shift invariant (LSI systems) can be analyzed in
a very special way. A transfer function (or system response function) can be
defined for an LSI system. The transfer function completely describes system
behavior. For example, most circuits have a transfer response function that
describes their electrical behavior. A well-corrected optical telescope has an
optical transfer function that characterizes the image. LSI analysis is so common,
and the ability to find a single function that completely describes a system is so
typical, that it is ingrained in the modern engineering psyche. This section
describes LSI theory and explains why it does not apply to sampled systems.
Systems are linear if superposition holds. That is, if
input A yields  output A,
and
input B yields  output B,
then
input A + input B yield  output A + output B.

Downloaded From: http://ebooks.spiedigitallibrary.org/ on 01/30/2013 Terms of Use: http://spiedl.org/terms