Change Signals Provide Clues to Perception Mechanisms––
Visual Illusions Reveal the Brain’s Information Selection Strategy

1. Conscious and unconscious visual perception

When our eyes are open, we see the visual scene before us. However, what we see is not simply a matter of the sensors in our retina reacting to light and of fragmented signals being conveyed to the brain. What enters our consciousness is the result of various kinds of information processing to remove unimportant details. In a sense, the brain is the ultimate image compression system.

The first stage of this subconscious information processing, which is no more than the acquisition of the light signal and video coding and analysis, has been gradually clarified through several decades of research. Some of the results are being applied in current image technology [1]. However, the next stage, which is to integrate the results of the first stage into a single conscious perception, remains a mystery. Although considerable information reduction is believed to occur in that stage as well, we do not know the criteria for selecting the information to appear in our conscious experience. Some people contend that the criteria are provided by attention, but it may be that the criteria are contained in the image itself [2], [3].

Recent research based on two newly discovered illusions [4], [5] is leading us to realize that the temporal change information in dynamic image inputs is a strong criterion for determining what is in our consciousness at a given moment. Below, we describe those illusions, which can be experienced on the webpage of our laboratories [6], and describe the role played by change signals in conscious perception.

2. Change is a necessary condition for conscious perception

Retinal images are full of change. Information about changes is believed to play a role in our perception of change and movement in the outside world, but that is not all. We think that change information is essential to our conscious experience of seeing something.

If you are presented with a pattern of high-contrast light and dark regions against a plain background, you would normally be able to see that pattern clearly. However, if the same pattern is presented after you have been watching a flickering moving image for about 20 seconds, you will not see the pattern at all. The effect is the same for any high-contrast image. Simply watching a flickering stimulus for a while causes people with normal visual acuity to be afflicted by a kind of blindness for a short time (Fig. 1) [4].

Fig. 1. Adaptation-induced blindness.

You are probably aware that exposure to a strong stimulus over a long time decreases sensitivity to light and color to some degree. That effect is called adaptation. However, we know that the inability to see the pattern in abovementioned illusion is not entirely due to adaptation in the early visual system of the eyes and brain because the pattern’s shape and color are analyzed accurately by subconscious processes (Fig. 2). There is another illusion in which a vertical stripe pattern surrounded by a stripe pattern that is slightly inclined from the vertical is seen as being inclined in the opposite direction. We found that this direction contrast effect occurs even if the surrounding pattern cannot be consciously seen because of adaptation. If the surrounding pattern is not seen because the visual sensors are not responding to it, then the contrast effect should not occur. This adaptation-induced blindness is unlikely to be the effect of ordinary adaptation. Although the stimulus is correctly analyzed at the initial stage, the result of that analysis does not enter consciousness.

Fig. 2. Tilt-induction effect caused by invisible surrounds.

Why does that happen? Adaptation-induced blindness does not occur after adaptation to a still image or when the target pattern is presented suddenly. On the basis of that fact, we propose the hypothesis that visual information that is analyzed in the initial stage does not enter into consciousness if there is no signal indicating a change (Fig. 3).

Fig. 3. Gate model of conscious perception.

Somewhere in the brain, there is a mechanism that acts as a gateway to determine whether or not subconsciously processed visual information should be moved into consciousness. If we consider that a change signal is necessary to open that gate, then we have an easy explanation for the illusion. Under normal circumstances, even a slow stimulus will provide a change signal that opens the gate, so the pattern enters our consciousness. If our sensitivity to change is decreased to a degree by adaptation to a dynamic stimulus, however, then a pattern that is presented slowly will not generate a change signal that is sufficient to open the gate and we will not be conscious of that pattern, even though the shape and colors of the pattern are analyzed subconsciously. We believe that the change information included in the stimulus image serves as the criterion for whether or not the stimulus itself is consciously seen as well as providing the basis for recognizing change and movement in the video.

Although this gate circuit is extremely simple, it is highly compatible with explanations of various phenomena having to do with can be seen and cannot be seen that have been reported in the past [7]. What is particularly interesting is the visual extinction effect, in which a stimulus is completely extinguished from part of the visual field when there is damage to the parietal lobe: a part of the brain involved in processing position and timing [8]. Some research has confirmed that stimuli that cannot be seen are processed subconsciously [9]. That phenomenon is exactly like adaptation-induced blindness. It is likely that the gate mechanism is located in the parietal area and controls whether or not the information processed in the occipital and temporal areas is presented to the consciousness. We are currently trying to elucidate the physiological foundation of this mechanism through multiple approaches.

3. Change signals determine the content of perception

There are other illusions that demonstrate the important role of signals of transient change (Fig. 4) [5]. Take the case of a large surface that smoothly changes color from green to red over a period of about one second. When this is observed normally, the color change is clearly perceived. However, if a square is presented suddenly during the color change, the color change is seen to stop completely for about 0.2 seconds, but only in the area bounded by the contour. Because the color transition is seen normally in the area outside the contour, a difference in color between the inside and outside of the contour is seen. The same effect occurs with various other features such as pattern shape and size as well as color. For example, if a figure is shown in a rotating pattern of stripes, the rotation appears to stop in a certain direction. The abrupt appearance of the contour is strong change information that results in a fixed perception of color or texture that is actually changing.

Fig. 4. Temporal freezing illusion.

Unlike a video camera, the brain does not represent the input images as a sequence of discrete images. Instead, pattern and color information is represented with varying timing. This freezing illusion suggests the possibility that a strong change signal acts as a synchronization signal, integrating dispersed information into a coherent whole and bringing about the perception of events. With reference to the gate mechanism introduced for the adaptation-induced blindness phenomenon (Fig. 3), shape and color information within a specific time can be considered to be integrated and presented to the consciousness as perceptually simultaneous events each time the gate opens. The 0.2 seconds required for the stable occurrence of the freezing illusion may be the range of that perceptual simultaneity.

4. Visual representation based on saliency

In our daily lives, changes that appear suddenly in our visual scene stand out exceptionally and almost automatically draw our attention. Things that do not change or change slowly, on the other hand, do not stand out and are easily overlooked if we do not intentionally direct attention to them. In research on attention, this tendency to stand out is called saliency. How to calculate saliency in still images or video has been the subject of much research [10].

The two illusions described in this article show that saliency does not simply direct our attention to one of many things that we can already see but is an element in determining the conscious sensory experience of seeing. Whether or not perception occurs and the content of that perception vary greatly with the strength and timing of the change signals that give rise to saliency.

The results of the research reported here provide a hint concerning the representation of visual information beyond the initial stage of vision. In the brain, it is likely that information about the color, patterns, and movement that constitute visual content is represented loosely, both spatially and temporally. That information is then integrated with reference to a saliency signal that is extracted by a separate path to create a visual world that is continuous in time and space. Conversely, if the position and timing of saliency could be determined accurately, it might be possible to convey video color, pattern, and movement data using a much lower frame rate than is currently used.

References

[1]	B. A. Wandell, “Foundations of Vision,” Sinauer Associates, 1995.
[2]	V. A. Lamme, “Why Visual Attention and Awareness are Different,” Trends in Cognitive Sciences, Vol. 7, No. 1, pp. 12–18, 2003.
[3]	C. Koch and N. Tsuchiya, “Attention and Consciousness: Two Distinct Brain Processes,” Trends in Cognitive Sciences, Vol. 11, No. 1, pp. 16–22, 2007.
[4]	I. Motoyoshi and S. Hayakawa, “Adaptation-induced Blindness to Sluggish Stimuli,” Journal of Vision, Vol. 10, No. 2, pp. 16.1–16.8, 2010.
[5]	I. Motoyoshi, “Temporal Freezing of Visual Features,” Current Biology, Vol. 17, No. 11, pp. R404–R406, 2007.
[6]	http://www.brl.ntt.co.jp/people/imotoyoshi/index-e.htm
[7]	C. Y. Kim and R. Blake, “Psychophysical Magic: Rendering the Visible ‘Invisible’,” Trends in Cognitive Sciences, Vol. 9, No. 8, pp. 381–388, 2005.
[8]	A. R. Luria, “Disorders of ‘Simultaneous Perception’ in a Case of Bilateral Occipito-parietal Brain Injury,” Brain, Vol. 82, pp. 437–449, 1959.
[9]	J. B. Mattingley, G. Davis, and J. Driver, “Preattentive Filling-in of Visual Surfaces in Parietal Extinction,” Science, Vol. 275, No. 5300, pp. 671–674, 1997.
[10]	L. Itti and C. Koch, “Computational Modelling of Visual Attention,” Nature Reviews Neuroscience, Vol. 2, No. 3, pp. 194–203, 2001.

↑ TOP