

Overview 

We are working on the digital restoration of missing parts in damaged paintings. This is part of an ongoing project on the virtual restoration of the 3600 years old wall paintings excavated in the prehistoric Aegean settlement in Akrotiri, Thera, Greece. As part of this work we have acquired hundreds of photographs of several wall paintings preserved at the Akrotiri Thera excavation site, which we have used to create ultra high resolution mosaics of each composition. For filling in the missing parts we have developed digital image inpainting techniques, relying on either physical diffusion processes or statistical waveletdomain image models. 

Theran Wall Paintings 

Our interest in image inpainting is motivated by the problem of digital restoration of the ancient wallpaintings discovered in the prehistoric settlement of Akrotiri, Thera, some representative examples of which are shown in Figure 2. The Theran wall paintings are famous for their thematic gamut, their artistic value, and the abundant and diversified information they yield about the Aegean world more than 3500 years ago. Restoring these unique 17th century B.C. wallpaintings is a particularly challenging and time consuming process, requiring the painstaking effort of several specially trained conservators working under expert archaeological guidance. The restoration process comprises, among others, gathering the plaster fragments from the archaeological site, cleaning and restoring each fragment in the lab, and reassembling each composition from its constituent fragments. The main goal of our work is to virtually restore the reassembled wall paintings by filling in the missing parts.


Creating and Viewing High Resolution Mosaics of the Wall Paintings 

For digitally restoring the artwork we need to capture it as closely as possible in highquality digital photographs. A significant difficulty with the Theran wall paintings is that they are very large, each composition typically covering an area of several square meters. Their big size makes it impossible to photograph them in sufficient detail in a single snapshot using conventional equipment. Our approach has been to divide each composition into several small overlapping patches and photograph each of them in sufficient detail by placing the camera at a quite close distance. Using interest point detection and description methods we have been able to semiautomatically align all photographs corresponding to a particular wall painting as shown in Figure 3. Similar alignment techniques are very popular for creating panoramic images of scenery. However in our case several factors differ, for example, the geometry of the painting surface is flat and the camera position is nonconstant. The image alignment step is followed by image blending, yielding the final ultra high resolution mosaic which captures the whole composition in sufficient detail.
The resulting mosaics are typically hundreds of Mpixel large and their file size makes them difficult to manipulate and share. To make it easier to use them we have set up a system based on IIPImage which allows viewing coarse scale instances or zooming into details of the mosaics through a web interface, as shown in Figure 4. The viewer is accessible here (requires authentication).


Filling In the Gaps: Virtual Restoration with Image Inpainting Techniques 

We have developed digital image inpainting techniques for filling in the missing parts of the wall paintings. We have explored two broad classes of techniques, the one based on physical diffusion processes and the other based on statistical waveletdomain image modeling. In diffusionbased techniques for image inpainting one tries to propagate structure and color from the known image areas into the missing parts. To ensure good edge continuation of image edges the diffusion process is designed to be of anisotropic nature and formulated as a nonlinear anisotropic diffusion partial differential equation (PDE). For efficiently solving numerically the resulting PDE we employ multigrid techniques, which are closely related to the algorithms we have previously developed for solving similar PDEs arising in the context of image segmentation (see here). An indicative inpainting result of this technique is shown in Figure 1. We are also working on another class of techniques which considers image inpainting as a missing data inverse problem. For solving it in a Bayesian framework one needs to adopt an appropriate probabilistic prior model to regularize the inversion process. Multiscale image descompositions such as the wavelet transform present several advantages as an image representation on which one can build sufficiently accurate yet tractable prior image models. In our research we have combined an overcomplete complexwavelet image representation, which ensures good shift invariance and directional selectivity with a discretestate/continuousobservation hidden Markov tree model for the wavelet coefficients, which captures key statistical properties of natural image wavelet responses, such as heavytailed histograms and persistence of large wavelet coefficients across scales. We have explored alternative deterministic and Markov chain Monte Carlo algorithms for image inpainting under this multiscale image model. An inpainting example using our technique is shown in Figure 5. We are currently working into extending our transform domain inpainting model in two main directions: (a) using multiple dictionaries for compactly representing heterogeneous image layers (e.g. texture) and (b) integrating into our probabilistic model constraints for the phase of the complex wavelet transforms.


People 

CVSP group people involved: 

Publications 



Acknowledgments 

Our work is part of an ongoing project on the virtual restoration of Theran wall paintings sponsored by the General Secretariat of Research and Technology in Greece under grant PENED2003ED865. We thank the other project participants and particularly F. Georma, Dr. A. Vlachopoulos, and Prof. C. Doumas of Akrotiri Excavation, Thera, for their invaluable help with the wall paintings. 