Spatial image processing for the enhancement and restoration of film,

photography and print

Carinna Parraman Alessandro Rizzi
Centre for Fine Print Research, Department of Information Technology,
University of the West of England, Bristol, University of Milano,
BS3 2JT UK Via Bramante, 65-26013 Crema (CR) Italy

direction. Previous preliminary tests of this approach on

Abstract:- This paper presents a review of research display and prints are reported in [7-10]
undertaken into algorithms for colour enhancement methods
based on the behaviour of the human vision system and the II. SCENARIOS AND PROBLEMS
applied research in film, photography and print. These colour
enhancements methods perform automatic adjustments that are The following examples demonstrate the range of
based on the content of the image without the need of any
scenarios and problems that have been considered and
statistic or a-priori knowledge about it. We have found that it
can have a range of benefits and implications for enhancement
addressed.
of new images and restoration of deteriorated images. This
paper presents an overview of series of research projects that A). Film restoration
have been undertaken since 2005 and which are still being
explored as new algorithms are being introduced and refined. Wet-process film materials are the result of a
The paper will discuss the strengths and limits of this approach. chemically unstable process, subject to fading over time.
This fading is irreversible and in several cases
photochemical restoration of faded prints is problematic
I. INTRODUCTION and not always possible. In these cases, digital colour
restoration can address the problem. Faded film images
Film, photography and print are important for capturing are dull, have poor saturation and an overall colour cast.
and recording a huge range of activities for art, This is due to the bleaching of one or two chromatic
entertainment, advertising and documentation. However, layers of the film. Since it is necessary to deal with lost
these materials – dyes, inks, papers, film-stock, digital chromatic information, restoring the colour of faded
data, are also subject to change and deterioration and it movies is more complex than a simple colour balance.
has become increasingly important to preserve our recent Digital techniques allows easily multiple
photographic and printed cultural heritage. In order to restoring solutions, but still the restoration process can be
obtain a good quality image or a restored image, the a very delicate and long process, based mainly on the
process of adjusting images can be a time consuming and professional personal skills of the restoring technicians.
a costly procedure. In this paper, we consider the use of The restoration market demands for automatic solutions,
unsupervised colour enhancement methods as final but the big diversity of all the possible film conditions
automatic processing or pre-processing for printing. makes every automatism difficult to implement.
We tested some classic widely used algorithms such as
Histogram Equalisation and Auto Levels and compared
them to alternative approaches that utilise Spatial Colour B). Image enhancement for the printing of digital
Algorithms (SCA). The term Spatial Colour Algorithm photographic images
refers to a family of algorithms that re-compute
wavelength/energy arrays into calculated colour User objectives for digital capture have more far
appearance arrays, or preferred color enhancement arrays, reaching objectives: documentation, collection of images
according to the spatial distribution of pixel values in the for recording, uploading to Internet, dissemination of
scene [1]. Their basic idea is to mimic the behaviour of images, prints and artworks through the Internet,
the human vision system. recording of events and preparing images in preparation
Among these algorithms, we present the for printing eg. wedding or holiday albums, the process of
following: Retinex [2-4], RSR [5], ACE [6]. SCAs are selecting images therefore reflects these requirements.
generally parameter dependent [1]. We do not intend to
consider fine tuning for each of the techniques, rather to C). Limited devices or non standard external
consider an average parameter set for each one and then conditions
test if this approach can go in the user preference
 As more spontaneous images are captured by III. DESCRIPTION OF COLOUR
phone, events may be captured in an environment that is ENHANCEMENT METHODS
under poor or artificial lighting conditions. The intention
is to consider a range of photographic conditions and In this paper we work with two categories of image
reasons for image capture, such as artefacts that are enhancement methods. The first are classic global
poorly lit (possibly due to museum or conservation enhancement methods that are included in photo editing
issues), conditions where flash photography is prohibited software which enables the user to adjust brightness,
or unsuitable, scenes taken in artificial light conditions, saturation, contrast by hand and in digital imaging
de-saturated scenes, atmospheric influences, photography software such as Photoshop: histogram equalisation, auto
of flat artworks, objects taken against a white background levels and auto-colour. This global approach maps every
and underwater photographs are sampled. pixel using the same value.
The second is a spatial image enhancement
D). Image enhancement for drawings, artworks and approach (SCA), which is inspired by human visual
digitally generated images system characteristics and share both global and local
colour correction characteristics. This method makes a
As artists, designers and archivists require pixel-by-pixel comparison to re-compute the pixels based
scanning and photography of artworks and artefacts, these on the context of the scene. Considered examples are
might be used for digital archives, for the Internet, Retinex, RSR, ACE. The main characteristics of these
promotional materials. Some might be flat scanned, such methods are described here in more detail.
as maps or small prints and photographs, and some might
be very large that it is difficult to obtain a uniform lit A). Histogram equalisation and curves
surface. These artworks may be textured or collaged,
digitally generated or manipulated artworks, black and In commercial software applications, such as
white prints, commercial CMYK prints, newspaper Photoshop, there is a range of tools for the fine-tuning of
images, scanned artwork, colour photographs, black and an image. Therefore, improving the dynamic range of an
white photographs, (see Figure 1). image can be undertaken using the histogram in Levels
by changing the input and output levels; or through
Curves by adjusting any point on a 0-255 tonal scale; or
by assigning target values to the highlight and shadow
pixels using either the Levels of Curves.
Photoshop automatic tools process the image on a
global level, therefore making an assessment of an image
by building a histogram, and applying corrections to the
whole image. As demonstrated in figure 1, by stretching
the black and white points, a brighter image is obtained.
In the lower image, the histogram shows peaks in the
black and white and nothing in between. However, using
auto levels to an image that has high contrast no
improvement is made to the mid tones (Figure 2)

Fig.1. Examples of artworks

E). Image restoration of faded fine art prints

For artists and conservators who need to restore the print

to its original condition would benefit from the assistance
of colour restoration tools to reproduce colours that are as
close to the original as possible. This can be undertaken
for instance using a model of dye or pigment fading. The
problem in these cases is to solve how to estimate the
amount of fading. In many cases, an original is not
always at hand for comparison, which presents a problem
as to the original colour. In this paper we suggest the use
of techniques that aim to restore the colour appearance
and not the colour itself.
 the midtones and clipping the white and black pixels.
This can be undertaken in Curves on a more precise level
by assigning up to 14 points on a curve, or by using the
auto levels. Because each channel is adjusted
individually, the algorithm may remove or introduce
colour casts.

C). Retinex

Based on the human colour perception system, the

term is a composite of ‘retina’ and ‘cortex’. The Retinex
algorithm, mimicking human vision, computes image
colour appearance, and is described by Edwin Land in
1971 [2]. It performs global and local filtering according
to the way it scans the input image. Several
implementations have been developed so far [4].

Fig.3. Image sampled using Auto colour

command in Photoshop (top left) and Retinex
(top right), hand correction using curves
(bottom left), ACE Slope=5 (bottom right)

D). RSR
Fig.2. Levels are useful for correcting images
that are slightly imbalanced (top), unlike figure Random Spray Retinex, (RSR) [5] is one of the
3, which benefits from spatial tuning implementations of the original Retinex model. In this
parameters implementation, locality is performed by random sprays
of a target point rather than paths.
A). Auto levels
E). ACE
Auto Levels defines the lightest and darkest pixels
in each colour channel as white and black and then ACE [6] is an algorithm for unsupervised
stretches the in- between pixel values proportionately. enhancement of digital images. The implementation of
Because Auto Levels adjusts each colour channel ACE follows the classic SCA scheme: the first stage re-
individually, it may remove or introduce colour casts. computes each pixel according to the content and the
relative position of the values in the image while the
B). Auto colour second stage map the computed pixel values into the
Takes an average distribution from the darkest, available device range.
midtone to the lightest areas of an image, by neutralising
 the frames of the same shot [11]. Figure 4 shows the
IV). Test setup and results application of ACE on a black and white film in order to
restore its original dynamic range. The example is from
We present some results for each one of the described test Tom Tight et Dum Dum (1903) by Georges Méliès. In
study. Some of the results have been previously presented figure 4 the relative histograms are presented.
in [7-10]. We concentrate in the last section, presenting
new preliminary results and comments. The parameters
used for these tests were:

Table 1. ACE Parameters

Parameter Parameter range(s)
Subset Sub-sampled at a sub-factor of LL2
selection
Fig. 4. Application of ACE on a black and white
Chromatic SL5
film.
comparison SL20 a) Original frame from “Tom Tight et Dum Dum”.
Sign b) Frame filtered with SLOPE=20.
Distance 0.01

Table 2. Retinex Parameters

Parameter Parameter range(s)
Sub sampling 4
factor
a)
Number of 20
random paths
Type of path Brownian

Table 3. RSR Parameters

Parameter Parameter range(s) b)
Number of 5 Fig. 5. Relative histograms of frames in Fig. 3
spray per
pixel B). Image enhancement for the printing of digital
Number of 500 photographic images
point per
spray An investigation was undertaken to assess whether an
Type of spray Random automatic solution could be obtained that could enhance
out-of-balance images to obtain images that were more
natural and pleasing. The project [10] sampled 13 images,
A). Film restoration and project [9] from an initial set of thirty images, ten
were chosen for printing, which was due to the other
To implement a standard tuning procedure, we images showing similar characteristics. All images were
extracted a set of still images (key frames) that subjected to a range of spatial and non-spatial parameters.
summarise the video content in a rapid and compact way. The same parameters were set for each test set. The
Different methods can be used to select key frames. In images are captured on a flat bed scanner, digital camera,
general these methods assume that the video has already or are digitally generated on computer. All are
been segmented into shots by a shot detection algorithm, constrained to the same file size of 2MB, and saved as
and extract the key-frames from within each shot 24bit RGB images.
detected. In order to assess the automatic methods for
After the extraction of key frames, these images correcting images, a series of high quality printed sheets
are used as a set for the parameter tuning of ACE, the of each image were made, which contained all of the
chosen algorithm for colour correction. By default the spatial and global sample variations. A questionnaire was
key frames are used to set the colour correction method devised to gauge preferences of the artist, technical expert
parameters, which are then applied to the whole shot. Due and the general user, with the intention for extending the
to the robustness of the colour correction method the application as means for improving workflow and colour
setting used for the key frames is used successfully for all management systems prior to printing.
C). Limited capture devices or non standard lighting
conditions

Fig. 6. Example of a printed sheet ‘Church’, 8

sampled images are placed in a random order
D

Fig. 8. Detail of ‘Angel’ from printed test sheet,

showing the application of ACE at varying
parameters: A) The original image B) ACE
SLOPE=5 C) ACE SLOPE=20 D) ACE SIGN

Figures 6 and 7 illustrate two images taken

under different lighting conditions. In figure 6 the sample
image Church, where flash was disallowed, the only light
source is the illuminated figure and the surrounding
environment is in darkness. The second image Angel
(figure 7) is a painting on a wood panel, which is
displayed in a museum where flash photography is not
permitted and therefore the original photographs of both
Church and Angel appeared de-saturated and dull.
Gold is a colour that is particularly difficult to
reproduce and print, and so the balance or brightness of
gold in relation to the rest of the image might be
considered as: too bright, overpowering, too dull, not
convincing as gold. The red might also be an aspect that
is subject to comparison: the red tunic of the figure, the
red line along the top of the painting; the red of the carpet
Fig. 7. Example of a printed sheet ‘Angel’, 8 around the altar, the red lights on either side of the altar.
sampled images are placed in a random order Does the red in both the images stand out as dull,
balanced or too bright? Lastly, large flat areas of
chromatic dominance, which contains little information As described in projects [9] and [10] (section
could be problematic: for example, situations where art 4.2) sampling included: scans of woven textiles, collage,
works are photographed against a white background or newspaper prints, drawings, etchings, as well as digitally
where the light source is uneven, might results in pooling generated images, black and white images that had been
or halos of light. This is particularly obvious in figure 7. converted from colour. These were subjected to the same
range of spatial and non-spatial parameters.
D). Image enhancement for drawings, artworks and In the scan of Drawing (figure 9), the most
digitally generated artworks. preferred enhancement method by users was Random
Spray Retinex (RSR), which increased the whiteness of
Fig. 9. Samples of ‘Drawing’ showing the A) the paper and the brightness of the coloured marks. If
compared to the auto-levels whilst the image is brighter
than the original the green pen marks appear darker and
more contrast.

E). Image restoration of faded fine art prints

For this research project, we had access to a

collection of both exposed an unexposed fine art prints.
The exposed prints are works that have been on display,
under glass and in the same north-east facing position
A since 1991. The unexposed prints are from the same
edition, which have been stored in a print archive at the
Centre for Fine Print Research. It was very useful to gain
access to a collection of works that had been subjected to
the same conditions over the same period of time, to be
able to gauge how fine art materials such as paper, ink
and process, fared over time. Furthermore it was equally
significant to be able to compare the faded prints to a set
that had not been exposed to light, heat and humidity.
The following example demonstrate the extend of fading
of an early inkjet print.

C
Fig. 10. Example of a 3-colour inkjet fine art print
made in 1991. All the channels have faded
except for the cyan. The faded sample can bee
seen in the bottom left-hand corner. Through the
original image, B) the application of RSR C) application of ACE slope=5, 20 and Sign, Yellow
Autolevels. and Black have been recovered.
 These prints demonstrate a range of traditional dynamic range images, such as those taken under low
(etching, stone litho) and more contemporary print lighting or poor atmospheric conditions, more contrast
methods of that time (screenprint, 4-colour-offset litho, and detail could be discerned (figure 6 and 7).
laser-print, inkjet). Due to the use of fugitive inks some of Significantly, it has been demonstrated that the spatial
the contemporary printing methods (4-colour-offset litho, image enhancement methods are able to achieve
laser-print, inkjet) the magenta, yellow colours have surprising results in regenerating an image from a single
considerably faded resulting in two cases in which just channel to three (figure 10).
the black and blue layers remain. It was thought to be
impossible to restore these as no actual colour residue References
remained on the print.
All images were subjected to the same [1] A. Rizzi, J.J. McCann, “On the behavior of
parameters and were printed on the same printer onto fine spatial models of color”, IS&T/SPIE Electronic
art paper, using the same settings. For this experiment, Imaging 2007, S. Jose (California – USA), 28
only inkjet printing was used to emulate all the traditional January – 1 February 2007.
print processes. Users are asked to identify their preferred [2] E. Land, J.J. McCann: ``Lightness and Retinex
print in relation to lightness, tonal range, colour range, Theory'', J. Opt. Soc. Am. {\bf 61}, 1-11 (1971).
quality of detail and overall subjective preference. [3] D. Marini, A. Rizzi: ``A computational approach
There were some surprising results for 4-colour to color adaptation effects'', Im. and Vis. Comp.
images even where only one of the four colours remained. 18 (13), 1005-1014 (2000).
Through the application of the spatial algorithm of ACE, [4] E. Provenzi, A. Rizzi, L. De Carli, D. Marini,
some traces of the yellow and black channel were “Mathematical definition and analysis of the
recovered as demonstrated in figure 10. Also as Retinex algorithm” Journal of Optical Society of
exampled in other monochrome images, contrast and America A, Vol. 22, December (2005).
dynamic range is improved (figure 11) [5] E. Provenzi, M. Fierro, A. Rizzi, L. De Carli, D.
Gadia, D. Marini, “Random Spray Retinex: a
new Retinex implementation to investigate the
local properties of the model” IEEE
Transactions on Image Processing, Vol. 16,
Issue 1, pp. 162-171, January 2007.
[6] A. Rizzi, C. Gatta, D. Marini, “A New
Algorithm for Unsupervised Global and Local
Color Correction”, Pattern Recognition Letters,
Vol 24 (11), pp. 1663-1677, July 2003.
[7] A. Rizzi, C. Gatta, F. Taraschi, M. Zanardini, M.
Fig. 11. Example of monochrome fine art print Abbiati, “Un algoritmo per la valutazione
using ACE slope=5. percettiva delle interfacce visuali”, DDD
Disegno e Design Digitale, Vol. 7 (2), Apr/Jun
2003, ed. Poli-Design (Politecnico di Milano).
VI). Conclusion [8] A. Rizzi, C. Gatta, M. Maggiore, E. Agnelli, D.
Ferrari, D. Negri, “Automatic Lightness and
As demonstrated in the figures included in this
Color Adjustment of Visual Interfaces”, HCI-
paper, the use of spatial enhancement methods such as
Italy 2003, Torino (Italy), October 2003.
ACE, Retinex an RSR can assist in improving the balance
[9] C. Parraman, A. Rizzi, “User Preferences in
an detail of both film, photographs and fine art prints. In
Color Enhancement Unsupervised Methods for
general it was found that ACE fails with images that have
Printing”, IS&T/SPIE Electronic Imaging 2007,
a particular chromatic dominance, in this case, colour is
S.Josè (California – USA), 28 January –1
significantly changed and can result in a better image but
February 2007.
with an extreme colour shift. Fine-tuning of the
[10] C. Parraman, A. Rizzi, “Searching User
parameters in order to obtain these results requires some
Preferences in Printing: A Proposal for an
knowledge of the strengths and weaknesses of each of the
Automatic Solution”, Printing Technology
algorithms. Where an increase in brightness of contrast
SpB06, St Peterburg (Russia), June 2006.
was required for photographic images, in many cases
[11] A. Rizzi, C. Gatta, C. Slanzi, G. Ciocca, R.
global image enhancements were the most suitable and all
Schettini, Unsupervised Color Film Restoration
that was required.
Using Adaptive Color Equalization, Lecture
It was generally found that for the majority of
Notes in Computer Science, Volume 3736, Dec
black and white images the spatial correction methods
2006, Pages 1 – 12.
were strongly preferred (figure 4 and 11). Also for low
[12] A. Rizzi, Perceptual colour film restoration,
Preservation and Conservation Issues Related to
Digital Printing and Digital Photography,
Institute of Physics, London, 24th - 25th April
2006.