Exploring RGB-D Cameras for 3D Reconstruction of Cultural Heritage: A New Approach Applied to Brazilian Baroque Sculptures

RGB-D cameras have a great potential to solve several problems arising during the digitization of objects, such as cultural heritage. Three-dimensional (3D) digital preservation is usually performed with the use of high-end 3D scanners, as the 3D points generated by this type of equipment are in average millimeter up to sub-millimeter accurate. The downside of 3D scanners, in addition to the high cost, is the infrastructure requirements. It requires its own source of energy, a large workspace with tripods, special training to calibrate and operate the equipment, and high acquisition time, potentially taking several minutes for capturing a single image. An alternative is the use of low-cost depth cameras that are easy to operate and only require connection to a laptop and a source of energy. There are several recent studies showing the potential of RGB-D sensors. However, they often exhibit errors when applied to a full 360 degrees 3D reconstruction setup, known as the loop closure problem. This kind of error accumulation is intensified by the lower accuracy and large volume of data generated by RGB-D cameras. This article proposes a complete methodology for 3D reconstruction based on RGB-D sensors. To mitigate the loop closure effect, a pairwise alignment method was developed. The proposed approach expands the connectivity graph connections in a pairwise alignment system, by automatically discovering new pairs of meshes with overlapping regions. Then the alignment is more evenly distributed over the aligned pairs, avoiding the loop closure problem of full 3D reconstructions. The experiments were performed on a collection of 30 artworks made by the Baroque artist Antonio Francisco Lisboa, known as Aleijadinho, as part of the Aleijadinho Digital project conducted in partnership with IPHAN (Brazilian National Institute for Cultural and Artistic Heritage) and United Nations Educational, Scientific and Cultural Organization (UNESCO). Experimental results show 3D models that are favorably compared to state-of-the-art methods available in the literature using RGD-D sensors. The main contributions of this work are: a new method for 3D alignment dedicated to attenuate the RGB-D camera loop closure problem; the development and disclosure of a complete, practical solution for 3D reconstruction of artworks; and the construction of 3D digital models of an important and challenging collection of Brazilian cultural heritage, made accessible by a virtual museum.