High-resolution Multiband Imaging for Accurate Color Reproduction

2.1 Principle of color estimation using multispectral data

As shown in Fig. 1, an object’s surface reflects light from an illumination source. Let the illumination spectrum and spectral reflectance be E(λ) and f(λ), respectively. The observed spectrum I(λ) can then be represented as

Fig. 1. Observation model.

where λ is the wavelength. Let us consider the situation where the reflected light is captured by an N-band sensor. Let the sensor’s spectral sensitivity be S=[S₁(λ), S₂(λ), …, S_N(λ)]^t and the matrix whose diagonal elements represent the spectral power distribution of the illumination be W. Equation (1) can then be rewritten in a vector representation as

By using the Wiener estimation method, we can estimate the spectral reflectance from the camera signal, c=SWf=Hf, as

where M is the Wiener estimation matrix obtained from H, and R is a priori knowledge about the spectral reflectance of objects [8].

2.2 Two-shot-type six-band image capturing system

The principle of two-shot-type six-band image capturing [9] is shown in Fig. 2. The system consists of a commercially available 35-mm digital camera and a customized interference filter. The filter is mounted in front of the camera lens and cuts off the left (i.e., short-wavelength) sides of both the blue and red peaks in the camera’s original spectral sensitivity. It also cuts off the green peak’s right (i.e., long-wavelength) side. First, an image is taken with the filter in front of the lens. Then, another image is taken without the filter by manually sliding the filter horizontally. Although the filter can be moved smoothly, there is a displacement error of a few pixels between the two images, which can be corrected by software [10]. After the correction process, the two three-band images are combined into a six-band image.

Fig. 2. Principle of two-shot-type six-band image capturing.

2.3 High-pixel-number image integrated by image stitching

In the conventional system [9], the number of pixels is determined by the image sensor. When an object 2 m × 2 m in size is captured with a 100-megapixel image sensor, the image resolution is 0.2 mm per pixel. To achieve image resolution of 0.01 mm per pixel with this sensor, the object should be divided into several parts, and an image of each part should be captured. The obtained images then need to be integrated into a high-pixel-number image.

There are two ways to integrate divided images. The general approach is to move the camera or object. Although this method is very simple and requires no special equipment, the effects of lens aberration, such as distortion and shading, have to be corrected before image integration. The second approach is to move the image sensor instead of the camera or object. This requires a good-quality lens with a large enough focused image area on the sensor plane. In addition, the camera needs a function for shifting the sensor horizontally and vertically. One advantage of this method is that the effects of lens aberration can be ignored in image integration. A practical way to implement this method is to combine a large-format camera (e.g., a 4 × 5 inch format camera) and a digital camera back. We used both methods in the experiments described below.

3.1 Image-capturing system

In estimating spectral reflectance by using multiband images, one should use full-spectrum light that does not have any emission lines. As the illumination source in the experiments, we used SOLAX^TM (SERIC Ltd., Japan), which has a spectral power distribution close to that of natural sunlight. To avoid damaging the objects with ultraviolet (UV) and infrared (IR) light, we mounted UV- and IR-cut filters in front of the lamp. The spectral power distribution of this light is shown in Fig. 3.

Fig. 3. Spectral distribution of illumination.

The camera system used in our experiments is shown in Fig. 4. We used a 4 × 5 inch format camera (TOYO-VIEW 45GII) to which we attached a scanning digital back (DF-S2, Pioneer) as an image sensor. The sensor is a line charge-coupled device (CCD) sensor with 10,600 pixels. Scanned images have 10,600 × 17,460 pixels (185 megapixels). It takes about 15 minutes to make one image scan (depending on the brightness of the illumination light), so it takes almost 30 minutes to obtain a six-band image. Note that we replaced the IR-cut filter with a customized one that cuts wavelengths in the 800-nm range. The original IR-cut filter cuts wavelengths in the 680-nm range, which degraded the signal-to-noise (S/N) ratio of the near-IR channel of the six-band image capturing system used in the experiments. The spectral transmittance of the customized interference filter mounted in front of the lens and the spectral sensitivity of the six-band image capturing system are shown in Fig. 5. Regarding the color reproduction accuracy of this camera system, the average color difference between a real object and its image displayed on a liquid crystal display was ΔE_ab=1.7 when the target was the Macbeth ColorChecker^TM. The color gamuts of the six-band and three-band capturing systems (under sunlight) on the u’-v’ chromaticity diagram are shown in Fig. 6. The color gamut of Adobe RGB is also shown on the diagram as a reference. These results show that the six-band capturing system can record more deeply saturated color, especially red and blue, than an imaging system based on the Adobe RGB color space.

Fig. 4. Two-shot six-band camera system.

Fig. 5. Spectral transmittance of the filter and spectral sensitivity of the six-band camera.

Fig. 6. Comparion of color gamuts.

3.2 Captured images

To confirm the ability of the six- and three-band camera system to record saturated color, we captured a photograph of a butterfly using it. Figure 7 shows the resultant images after color reproduction using six- and three-band images captured by the camera system described in section 3.1. From these images, it is clear that the saturated blue color can be reproduced better when the six-band images are used. Figure 8 shows the definition of the butterfly image in Fig. 7 and that of a wood-block print image. The pixel numbers of the two photographs are 10,000 × 10,000 pixels for the butterfly and 28,000 × 14,000 pixels for the wood-block print. The image resolution is 0.01 mm per pixel.

Fig. 7. Resultant images from three-band image and six-band image.

Fig. 8. Definition of the resultant images.

An image that was captured outdoors under sunlight is shown in Fig. 9. The object is a ceramic wall 2.5 m high and 50 m wide. The pixel number of the synthesized image is 200,000 × 10,000 pixels (i.e., 2 gigapixels). The image resolution is 0.1 mm per pixel.

Fig. 9. Resultant image captured outdoors under sunlight using two-shot six-band camera system.