Machine Vision and Human perception: The case study of fine art reproduction

Organisation: TU Delft

When it comes to visual perception, the human brain exhibits amazing capabilities. On the other hand, and despite impressive progress, machines still have difficulties making sense of even simple scenes.

An interesting approach to study this apparent paradox is the special case of fine art reproduction: in such applications, a computer vision system is used in order to acquire some of the visual properties of the target that can be fed into a reproduction device with the aim of producing a trustful copy of the original. The difference between the original and the reproduction can be quantified knowing the properties of the acquisition/reproduction device but, more interesting, can be assessed qualitatively by a human observer comparing it's perception of the copy with it's perception of the original.

In this presentation, we will focus on three main visual properties of objects and compare how they are captured by machines and perceived by humans. We will start our journey with colors, describe how digital camera perceive and render colors compared to the way this task is performed by the human visual system. With the case study of digital restoration of ancient paintings, we will see how colors are represented and can be manipulated in machines and how they are perceived by humans. The second aspect we will study is relief. In the case of a painting, a first distinction must be made between the apparent relief of the objects in the painting and the relief of the painting's surface itself. After a quick survey of methods to perceive depth that are available to humans and machines, we will describe the specific approach that is used to capture relief on the surface of a painting. Finally we will discuss the reaction of a human observer when placed in front a reproduction of both surface and color of a painting.

This evaluation will show that other aspects, more difficult to capture directly, also matter to faithfully reproduce the visual perception of an object. One of the most important aspects is glossiness. The third part of our presentation will therefore describe the physics of glossiness and some means to capture it as well as the difficulties reproducing it accurately... from a perceptual perspective.