Bayer Filter Array

[News Feeds]

Bayer Filter Array
Thomas Sapiano and Pro Photo Wiki Contributors



Introduction

The majority of conventional image sensor technologies aren't capable of measuring colour by themselves. As such, additional components are required in order to design cameras that can generate colour images. The most popular method of achieving this goal is the Bayer colour filter array.
[edit]
How it Works

Fig 1 - Bayer Colour Filter Arrays are used to capture colour data on image sensors that are otherwise not sensitive to colour. Adjacent pixels capture only one of the three available colours, and software is later used to interpolate the missing information back into the image.


Bayer filter arrays place colour filters directly above the image sensor. The filter contains an alternating pattern composed of one red, one blue and two green patches - each one covering a single photosite (see figure 1). This results in each pixel of the resulting image containing information for only one colour. This information is read out of the imager and digitized for further processing.
Fig 2 - Light entering the camera travels through the Bayer CFA prior to exposing the image on the monochrome sensor. This raw filtered data is then run through a demosaicing filter in order to regenerate a colour image file.


In order to generate a useable image from this raw data, some reconstruction is required. The process used to recover this information is called demosaicing, and is performed when the camera generates a finished image file. This process interpolates the missing colour information from adjacenet pixels in order to generate a full colour image.
When the camera is set to capture images in a RAW format, the demosaicing process is defered to the conversion process. RAW files contain the unprocessed information read directly from the sensor. When they are loaded on a computer, the conversion software will perform the demosaicing process to generate an image file the user can see.
As conventional computers typically have significantly more processing resources than the embedded systems within cameras, this software typically uses more sophisticated algorithms to complete this process. As such, the results from RAW conversion software can produce additional detail when compared with image files generated in-camera.
[edit]
Aliasing

Fig 3 - Without an Anti-Aliasing Filter, fine details projected through a Bayer filter can produce artifacts in the final image.


As only a single colour is captured at each photosite, simple Bayer imagers can be vulnerable to a number of issues that can be generated by fine patterns in the scene being captured. As objects can illuminate the sensor in such a way as they don't hit all three colours, errors can be made when reassembling the image (see figure 3). These aliasing artifacts typically manifest themselves as rainbow patterns within high-detail regions of the image.
To help remedy this problem, most image sensors with Bayer filter arrays are also equipped with an Anti-Aliasing Filter. This filter softens the image being projected on the sensor in order to ensure that any point of light in the scene will be sampled by at least three pixels. Unfortunately, this can also result in some loss of detail so manufacturers have to tweak the strength of this filter to obtain the optimal result.
[edit]
Variants

Fig 4 - The various different colour filter patterns used in Bayer cameras.


While the vast majority of cameras using Bayer filters use the traditional RGB pattern, a number of other variants have been used. Each of these patterns has its advantages and disadvantages, so manufacturers have selected these alternate options in order to extract additional functionality.
[edit]
CMY Filters

Kodak first used the CMY filter pattern in their DCS 620x and 720x models. The CMY dyes used to generate these filters absorb less light, so sensors based on this pattern generally have higher sensitivity than similarly equipped RGB cameras. This allowed these cameras to provide sensitivity levels that even current DSLRs have not been able to match.
Unfortunately, cameras using this filter pattern have also had problems with colour fidelity. Some saturated colours would produce colour shifts in the final image, making it difficult to correct for these problems in post processing.
This filter pattern has since been used on a number of high resolution consumer cameras to counteract the negative effects of small pixel sizes. As modern technology has been successful in battling noise via other measures, these filters have become relatively rare in the current crop of cameras.
[edit]
Sony RGBE Filter

In 2003 Sony patented a new process that replaced the second green element in the Bayer pattern with a new 'Emerald' filter. As cameras using this technology are sampling four discrete colours, this potentially provides a wider colour gamut.
[edit]
See Also

• Anti-Aliasing Filter - A filter that intentionally softens the image in order to help reduce the aliasing artifacts in the final image.
• Foveon X3 image sensor - A sensor design that uses three layers of photo sites to sense red, green, and blue components of the image, in a manner similar to colour film. In the Foveon X3 sensor all three colours are captured at each pixel location. This is claimed to increase resolution and reduce aliasing problems. Currently only a handful of cameras use the Foveon sensor. The first camera maker to use the Foveon sensor was Sigma in the SD9 and SD10 cameras.

[edit]
External Links

• US Patent 3,971,065 - Kodak's original patent covering the original invention of the Bayer pattern described above.

[maxbox51]

I wonder if a "GYCW" filter pattern would improve on the Bayer pattern AND the new Kodak pattern? Here's my reasoning:

1. A GYCW pattern contains pixels transmitting 1/3 (green G), 2/3 (cyan C, yellow Y), and all (white W) of the incident photons to the sensor, or 2/3 on average across the full sensor. This is twice as sensitive to light as the RGBG Bayer pattern, just like the YCYW Bayer variant and the new Kodak patterns.

2. All pixels receive green light, providing at least partial spatial resolution in the green channel at a finer scale than either the RGBG Bayer pattern or the new Kodak patterns.

3. Color information is available in both low and bright areas of the filter array: when the W pixel saturates, RGB can be determined from the other 3, less sensitive pixels (G, R = Y-G, B = C-G); in low light areas, the three more sensitive pixels provide color information (R = W-C, B = W-Y, G = C+Y-W). Obviously there are better interpolation algorithms, but this indicates how the array is more robust to both high and low light levels than the YCYW Bayer pattern, while retaining (or through the W pixel, improving on) its light sensitivity.

4. A GYCW pattern provides an increase in resolution in the green channel at all light levels. In the Bayer pattern, out of every 4 pixels 2 are sensitive only to green, 1 only to red, and 1 only to blue. The Bayer pattern therefore provides information about green every 4/2 = 2 pixels (lower is better), and information about red or blue once every 4/1 = 4 pixels. For the new Kodak patterns, out of every 8 pixels 4 are sensitive to 3 colors, 2 are sensitive only to green, 1 is sensitive to only to red, and 1 is sensitive only to blue. These patterns therefore provide a green resolution of 8/(4(1/3)+2(1)) = 2.4 pixels and red and blue resolutions of 8/(4(1/3)+1(1)) = 3.4 pixels. In a GYCW pattern, out of every 4 pixels 1 is sensitive to 3 colors, 1 is sensitive only to green, and 2 are sensitive to green and either red or blue. This provides a green resolution (computed as above) of 1.7 pixels and red and blue resolutions of 4.8 pixels. Since we are apparently most sensitive to resolution in green, this may represent an advantage.

That analysis assumes that all pixels are providing information. When the most sensitive white pixels saturate (but others do not), the Bayer pattern is unaffected, and the new Kodak patterns degrade to a green resolution of 4 pixels and red and blue resolutions of 8 pixels. The GYCW pattern degrades to a green resolution of 2 pixels and red and blue resolutions of 8 pixels. The GYCW pattern therefore should provide significantly better color resolution in highlights than the new Kodak patterns.

It is harder to analyze low light situations because the pixels don't become useless when they contain 0 information, but degrade into the sensor noise. When the Bayer pattern provides no color information, the new Kodak patterns also provide no color information, but the white pixels will provide luminance information at 2 pixel resolution. The GYCW pattern, however, is able to provide some color information (most robustly in green) as well as luminance information at a resolution somewhere between 1.333 and 4 pixel resolution; taking the average of these extremes yields an estimate of 2.6 pixels, worse than the new Kodak patterns but better than the Bayer pattern. This analysis also does not take into account the fact that low and high light situations are detectable in the GYCW pattern using a set of pixels (G for low light, W for high) independent from the pixels used to estimate color under those circumstances. This independence should also lead to better color estimates in these extreme conditions.

Do you think a GYCW pattern is worth a try?