Reflectance Transformation Imaging (RTI)

Contents:  What is it?  How does it work?  Examples  En Español

What is it?

RTI is a computational photographic method that captures a subject’s surface shape and color and enables the interactive re-lighting of the subject from any direction. RTI also permits the mathematical enhancement of the subject’s surface shape and color attributes. The enhancement functions of RTI reveal surface information that is not disclosed under direct empirical examination of the physical object. Today’s RTI software and related methodologies were constructed by a team of international developers.

The Sennedjem Lintel from the Phoebe A. Hearst Museum of Anthropology at the University of California, Berkeley.
RTI representation showing color information (bottom portion) and “specular enhancement” mode showing surface shape and enhanced reflectance (top portion).

RTI images are created from information derived from multiple digital photographs of a subject shot from a stationary camera position. In each photograph, light is projected from a different known, or knowable, direction. This process produces a series of images of the same subject with varying highlights and shadows. Lighting information from the images is mathematically synthesized to generate a mathematical model of the surface, enabling a user to re-light the RTI image interactively and examine its surface on a screen.

Each RTI resembles a single, two-dimensional (2D) photographic image. Unlike a typical photograph, reflectance information is derived from the three-dimensional (3D) shape of the image subject and encoded in the image per pixel, so that the synthesized RTI image “knows” how light will reflect off the subject. When the RTI is opened in RTI viewing software, each constituent pixel is able to reflect the software's interactive “virtual” light from any position selected by the user. This changing interplay of light and shadow in the image discloses fine details of the subject's 3D surface form.

RTI was invented by Tom Malzbender and Dan Gelb, who were research scientists at Hewlett-Packard Labs. A landmark paper describing these first tools and methods, named Polynomial Texture Mapping (PTM), was published in 2001. Learn more about PTM on Tom Malzbender's Google Docs site.

Since then, RTI research, application development, and evaluation in practical-use environments remains very active. Many new RTI tools, methods, and uses have emerged. Go to our Downloads area to access the most recent versions of the software and user guides. Participate in the CHIForums to learn about new projects and new research.

How does it work?

Mathematically, the direction that is perpendicular to the surface at any given location is represented by a vector (direction) called a normal (Figure 1). Technically it is a vector that is perpendicular to the tangent plane at any point on the surface, as real surfaces are in 3D and this graphic is only 2-dimensional.

Figure 1

Light bounces off of surfaces such that the incident angle of the light and the reflected angle of the light are equal angles to the surface normal. Since the camera is in a fixed position, and we know where the light is coming from in each image, and because we sample from a variety of light positions, RTI software can calculate the surface normal per pixel in the image.

Figure 2

The mathematical description of the normal is saved per pixel, along with the RGB (red-green-blue) color information of a regular photograph. This ability to record efficiently the color and true 3D shape information is the source of RTI's documentary power.

Figure 3 shows the reflection information captured in the RTI.

Figure 3

Examples

There are three video examples embedded below, each showing a different example of how RTI can be applied in the study of cultural heritage objects.

See also our collection of more examples of RTI, where you can read about the Smithsonian Institution's Squeeze Project online, with nearly 400 interactive examples of RTI (in an earlier form of RTI called Polynomial Texture Mapping, PTM).

Video: “RTI Example: Papyrus Fragment”

Ancient papyrus fragment from the Bancroft Library (UC Berkeley).

RTI example: papyrus fragment from Cultural Heritage Imaging on Vimeo.

Video: “RTI Example: Marble Stele”

Inscribed and recarved marble stele from the Tauric Preserve of Chersonesos, Ukraine. Imaged in July 2008.

RTI example: marble stele from Cultural Heritage Imaging on Vimeo.

Video: “RTI Example: Illuminated Manuscript”

An illuminated manuscript page study that demonstrates how RTI can be used to reveal hidden artifacts, in this case a letter that was erased on the page. Courtesy of and in collaboration with the Bancroft Library, UC Berkeley, California.

RTI example : illuminated manuscript from Cultural Heritage Imaging on Vimeo.