Fourier Transform and Video

Dereje Teferi(PhD)
dereje.teferi@aau.edu.et

1
Fourier Transform
 The Fourier transform, named after Joseph Fourier, is
a mathematical transformation employed to transform
signals between time (or spatial) domain and
frequency domain.
 Fourier Transform has many applications in signal
processing, physic,s and engineering.
 The new function is known as the Fourier
transform and/or the frequency spectrum of the
function f.
 The Fourier transform is also a reversible operation.
Thus, given the function F(the Fourier Transform), one
can determine the original function, f.
2
Fourier transform…
 Any periodic function can be expressed as the sum of a
series of sines and cosines (of varying amplitudes)
 Ex: Square waves

3
Fourier transform…
 Example Sawtooth wave

4
Fourier transform…
Fourier series states that

5
Fourier Transform…

6
Complex numbers
 Any number that can be expressed in the form
a+bi, where a and b are real numbers and i is
imaginary unit is called a complex number. Here
i  1
 Two complex numbers are equal if their real parts
are equal and their imaginary parts are equal. Ex
If a + bi = c + di, then a = c and b = d
 When adding or subtracting complex
numbers, combine like terms.
 Ex (8-3i)+(2+5i)=(8+2)+(-3i+5i)
=10+2i

7
Fourier Transform…
 Fourier Transform maps a time series
(eg audio samples) into the series of
frequencies (their amplitudes and
phases)
 Inverse Fourier Transform maps the
series of frequencies (their amplitudes
and phases) back into the
corresponding time series.
 Discrete Fourier transform is needed to
compute the frequency spectrum of a
function that is sampled,

8
Discrete Fourier Transform

9
Discrete Cosine Transform(DCT)
 When the input data contains only real numbers
from an even function, the sin component of the
DFT is 0, and the DFT becomes a Discrete Cosine
Transform (DCT)
 Because

 There are several variants of DCT

10
DCT/DFT for multimedia

11
12
Digital Video
 We have already looked at audio and images as two possible
media types. Here, we will be looking at a third media type,
video to understand some fundamentals governing this
media.
 We will first look at the types of video followed by the
differences between analog an digital video.
 Since, digital video is what we use in multimedia we will
understand the concepts of sub-sampling used to achieve a
digital representation that is in tune with human
perceptive abilities with respect to color and luminosity.
 We will briefly discuss some video standards such as HDTV
and lastly elaborate on video formats used on computer
displays.
13
Types of Video Signals
Video signals come in three possible types, component video,
composite video and separated video.
 Component video - each primary is sent as a separate signal.
 The primaries can either be RGB or a luminance-chrominance
transformation of them (e.g., YIQ, YUV).
 Best color reproduction
 Requires more bandwidth and good synchronization of the three
components
 Composite video -- color (chrominance) and luminance
signals are mixed into a single carrier wave.
 Some interference between the two signals is inevitable.
 S-Video (Separated video, e.g., in S-VHS) -- a compromise
between component analog video and the composite video.
It uses two lines, one for luminance and another for
composite chrominance signal.
14
Analog Video
Analog video is represented as a continuous (time varying) signal; Digital
video is represented as a sequence of digital images
NTSC Video PAL (SECAM) Video
• 525 scan lines per frame, 30 fps • 625 scan lines per frame, 25
(33.37 msec/frame). frames per second (40
• Interlaced, each frame is divided msec/frame)
into 2 fields, 262.5 lines/field • Interlaced, each frame is divided
• 20 lines reserved for control into 2 fields, 312.5 lines/field
information at the beginning of • Color representation:
each field • Uses YUV color model
• So a maximum of 485 lines of
visible data
• Laserdisc and S-VHS have actual
resolution of ~420 lines
• Ordinary TV -- ~320 lines
• Each line takes 63.5 microseconds
to scan.
• Color representation:
• Uses YIQ color model.

15
Interlacing vs Progressive scan
 Interlaced scan refers to a methods for "painting" a video image on
an electronic display screen by scanning or displaying each line of
pixels.
 This technique uses two fields to create a frame. One field contains
all the odd lines in the image, the other contains all the even lines of
the image.
 A PAL based television display, for example, scans 50 fields every
second (25 odd and 25 even). The two sets of 25 fields work together
to create a full frame every 1/25th of a second, resulting in a display
of 25 frames per second.
 Such scan of every second line is called interlacing.
 With progressive scan, an image is captured, transmitted and
displayed in a path similar to text on a page: line by line, from top to
bottom
16
Interlacing: example

17
Frame Rate and Interlacing
 Persistence of vision: The human eye retains an image for a fraction of a
second after it views the image. This property is essential to all visual
display technologies.
 The basic idea is quite simple, single still frames are presented at a high
enough rate so that persistence of vision integrates these still frames
into motion.
 Motion pictures originally set the frame rate at 16 frames per second.
This was rapidly found to be unacceptable and the frame rate was
increased to 24 frames per second.
 In Europe, this was changed to 25 frames per second, as the European
power line frequency is 50 Hz.
 When NTSC television standards were introduced, the frame rate was
set at 30 Hz (1/2 the 60 Hz line frequency).
 Movies filmed at 24 frames per second are simply converted to 30
frames per second on television broadcasting.

18
 Frame Rate and Interlacing
 For some reason, the brighter the still image
presented to the viewer, the shorter the persistence
of vision. So, bright pictures require more frequent
repetition.
 If the space between pictures is longer than the
period of persistence of vision -- then the image
flickers.
 Large bright theater projectors avoid this problem by
placing rotating shutters in front of the image in
order to increase the repetition rate by a factor of 2
(to 48) or three (to 72) without changing the actual
images.
19
Frame Rate and Interlacing ..
 Unfortunately, there is no easy way to "put a
shutter" in front of a television broadcast!
 Therefore, to arrange for two "flashes" per frame,
the flashes are created by interlacing.
 With interlacing, the number of "flashes" per
frame is two, and the field rate is double the
frame rate.
 Thus, NTSC systems have a field rate of 59.94 Hz
and PAL/SECAM systems a field rate of 50 Hz.

20
Digital Video
 Video is a sequence of images viewed at a certain
fixed rate (attached to audio) to give us a sense of
motion
 Both video and animation give a sense of motion
 It is viewed as continuous motion due to our ability
of interpreting images
 Advantages over analog:
 Direct random access --> good for nonlinear video editing
 No problem for repeated recording
 Almost all digital video uses component video

21
 Digital video and the human eye
 Persistence of vision: This is a
biological phenomenon that retains an
image for about 25ms on the retina
before another image is processed

 Phi phenomenon: is the optical illusion

of perceiving continuous motion
between separate objects viewed
rapidly in succession.
22
 Digital Video …
 The human eye responds more precisely to brightness
information than it does to color, chroma subsampling
(decimating) takes advantage of this.
 In a 4:4:4 scheme, each 8×8 matrix of RGB pixels converts to
three YCrCb 8×8 matrices: one for luminance (Y) and one for
each of the two chrominance bands (Cr and Cb).
 A 4:2:2 scheme also creates one 8×8 luminance matrix but
decimates every two horizontal pixels to create each
chrominance-matrix entry. Thus reducing the amount of data
to 2/3rds of a 4:4:4 scheme.
 Ratios of 4:2:0 decimate chrominance both horizontally and
vertically, resulting in four Y, one Cr, and one Cb 8×8 matrix
for every four 8×8 pixel-matrix sources. This conversion
creates half the data required in a 4:4:4 chroma ratio.
23
Chroma Subsampling(…contd.)
o 4:1:1 and 4:2:0 are used in JPEG and
MPEG
o 256-level gray-scale JPEG images
8x8 : 8x8 : 8x8 aren't usually much smaller than their
24-bit color counterparts, because
4:4:4 most JPEG implementations
aggressively subsample the color
information. Color data therefore
represents a small percentage of the
total file size.

8x8 : 4x4 : 4x4

8x8 : 8x4 : 8x4 4:2:2 8x8 : 8x2 : 8x2 4:1:1
8x8 : 8x2 : 0x0 4:2:0

24
Digital Video…
 Due to the above two phenomena of our vision
system, a discrete sequence of individual pictures
can be perceived as a continuous sequence
 Temporal aspect of Illumination—To represent
visual reality, two conditions must be met
 the rate of repetition of the images must be high
enough to guarantee smooth motion from frame to
frame
 the rate must be high enough so that the persistence
of vision extends over the interval between flashes
25
Frame rate
System Frame rate Remark
PAL(Phase 25 fps A picture consists of 625 lines and
Alternating Line) frame rate is 25Hz

SECAM(SEquential 25 fps A picture consists of 625 lines and

Couleur Avec frame rate is 25Hz
Memoire)

NTSC(National 30 fps A picture consists of 525 lines and

Television Systems frame rate is
Committee) approximately 30Hz

HDTV 1080x720
up to
1920x1080
pixels.

26
HDTV
Name Lines Aspect Opt. View P/I Freq.
Ratio dist MHz
HDTV 1050 16:9 2.5H P 8
USA, ana
HDTV 1250 16:9 2.4 P 9
Eur, ana
HDTV 1125 16:9 3.3 I 20
NHK
NTSC© 525 4:3 7 I 4.2
NTSC 525 4:3 5 P 4.2
PAL© 625 4:3 6 I 5.5
PAL 625 4:3 4.3 P 5.5
SECAM© 625 4:3 6 I 6
SECAM 625 4:3 4.3 P 6

27
28