Estimating spectral information of Reflection, Cameras and Illumination

Abstract

Appearance of objects in captured images and videos not only relies on objects themselves, but also significantly depends on imaging sensors and illuminations. Therefore, investigating how light emitted from light source interacts with objects and sensors has been an important task for a variety of applications in computer vision field. It is well known that light spans in a wide wavelength range, thus this interaction should be analyzed in spectral domain. However, it is hard to carry out the analysis because dominating equipments usually provide RGB 3 values only which are far from enough for spectral information estimation in the visible wavelength range. To deal with this problem, we present a framework for estimating spectral information of objects, cameras and illumination in this thesis. First, we show a system to recover spectral reflectance of objects with high temporal resolution. Spectral reflectance offers intrinsic characteristics of objects that are independent of illuminations and sensors. This direct representation about the objects is useful for solving many computer vision problems. However, existing methods for spectral reflectance recovery are limited either by their low temporal resolution or requirement for special hardware. To remove these limitations, we present a novel system for spectral reflectance recovery by exploiting the unique color-forming mechanism of Digital Light Processing (DLP) projectors. DLP projectors use color wheels which are composed of several color segments and rotate quickly to produce desired light. We make effective use of this mechanism and show that a DLP projector can be used as a light source with spectrally distinct illuminations when we capture scenes'appearance under the projector's irradiation by a high-speed camera. Our imaging system is built on easily available devices and capable of conducting spectral measurements at 100Hz. Based on the measurements obtained by our system, spectral reflectance of the scene is recovered using a linear approximation of surface reflectance. We carefully evaluate the accuracy of our system and demonstrate its effectiveness by spectral relighting of dynamic scenes with fast-moving objects. Then, we use fluorescence to estimate camera spectral sensitivity under unknown illuminations. Camera spectral sensitivity is an indispensable factor for various color-based computer vision tasks. Though several methods have been proposed to estimate it, their applications are all severely restricted by the requirement for a known illumination spectrum. In this thesis, we present a single-image estimation method using fluorescence with no requirement for a known illumination spectrum. Under different illuminations, the spectral distributions of fluorescence emitted from the same material remain unchanged up to a certain scale. Thus, a camera's response to the fluorescence would have the same chromaticity. Making use of this chromaticity invariance, the camera spectral sensitivity can be estimated under an arbitrary illumination whose spectrum is unknown. Through extensive experiments, we proved that our method is accurate under different illuminations. Finally, based on the estimated camera spectral sensitivity, we show how to recover spectra of daylights with high accuracy. Making use the estimated camera spectral sensitivities and daylight spectra, color correction problems can be solved. Results show that images after the correction match the target images well.