Exploring the Mechanism of Human Perception: A Challenge Befitting NTT

Explaining vision through psychophysics

—Dr. Nishida, please tell us about your main research theme.

I research the mechanism behind the way that people look at things. When you think about vision, what first comes to mind are words like eye and lens. The eye plays the role of a camera (input device) while neural circuits in the retina and brain do the processing. This scheme can extract useful information from images captured by the retina. The visual system extending from the eye to the brain consists of a component that plays the role of a camera and a component that acts much like a processor in a digital camera. Thus, in simple terms, my research aims to understand these two functions and the underlying mechanism, which I probe using psychophysical methods.

Psychophysics research has a long history. The question of how the human brain processes and interprets visual information captured by the retina has been examined by quantitatively analyzing judgments made by subjects on the basis of human perception.

The method of showing something to people and asking them how it looks has been used for about a hundred years. At one time, this was done using simple stimuli such as a picture drawn on a piece of paper. Recently, however, it has become possible to present images that had previously been difficult to prepare—such as artificial materials created with computer graphics—and to elicit responses about the material quality, such as “It appears shiny.”

As a result of this research, it has gradually become clear that human visual perception is achieved through close coordination among cortical modules that process different kinds of information such as an object’s movement, shape, and color.

—It seems that your research can be described as an attempt to determine and understand what people feel when they see something.

This might sound a bit vague, but you can call it research that investigates how we see an object itself and its shape, texture, movement, etc. (Figs. 1 and 2).

Fig. 1. Mutual interaction between movement and shape.

Fig. 2. Example of texture research.

Let me compare human vision with machine vision. We would like machines to be capable of recognizing complex objects such as human faces. We human observers can immediately recognize our friends simply by observing their faces. However, trying to get a machine to recognize people in this way is difficult. Up to now, there has been no popular alternative to inconvenient methods such as passwords for identifying people.

For a human being, recognizing who you are currently talking to is an everyday occurrence, but asking a machine to do the same thing is far from easy. Moreover, when people look at something, they can sense it to be glittering, feel it to be beautiful, or judge it to have an interesting shape, for example. Needless to say, it is difficult to have a machine judge an object in the same way.

Thus, in a human being, there are a number of visual processing stages from sensing an input with the eye to having an emotional response to the input in the brain. If we don’t understand the mechanism of, say, how we judge the quality and material of something that we are looking at, such as a metal or plastic object, we will not be able to understand how any subsequent emotion arises from them. Strictly speaking, we do not even know whether an emotional response is due to a conscious process of judging a characteristic like material or whether it is made through a separate, unconscious process. To fully understand the mechanism of human perception, we need to answer profound questions such as what is the role of conscious awareness.

—How would you go about explaining that function?

I would, for example, have a subject look at a motion picture and ask him or her to make judgments that would help me determine how human beings process motion-related information. That is, instead of measuring brain activity by some device, I would want to find out what the subject is seeing—I would listen to subjective information that only that person could access. Although devising what to present to a subject is, in a sense, a classic technique, we still make use of it in our cutting-edge research.

There has been much research recently on brain-machine interfaces, as in methods of estimating what is going on in the mind by capturing images of the brain through techniques such as magnetic resonance imaging (MRI). However, it is still at an elementary stage of research—rather than reading the mind, it has just begun to be possible to infer certain judgments in a piecemeal manner.

On the other hand, asking a subject directly, while a classic technique, provides a much richer understanding of that person’s state of mind. It is not easy to understand a person’s mind state from only brain activity. Asking what a person senses and feels is currently the best way to access that person’s state of mind or emotional state, and I don’t believe that this will change in the future.

Up to now, it has been believed that different types of information such as that about movement, position, shape, and color have been processed separately within the brain. For example, it has been thought that information received about movement has not been used in recognizing the shape or color of that moving object. In accordance with this view, the brain region responsible for motion-related processing is different from the region responsible for processing shape or some other characteristic. However, I have uncovered a number of pieces of evidence for closely coordinated processing among these regions through mutually linked pathways, and I have shown that there are misunderstandings in the conventional view of this processing.

It all began by asking: “Why can human beings do things that machines cannot?”

—Dr. Nishida, how did you become involved in this research?

To begin with, I was a very argumentative student, and I thought that the act of seeing was somewhat mysterious. In high school, I was fascinated with the idea that the way in which something right in front of my eyes appeared to exist was simply the way in which my brain recognized it.

Planning for college, I thought that I would want to enter a field of study focusing on humans, and though initially unsure whether to pursue philosophy or psychology, I eventually entered the Faculty of Letters and majored in psychology. I also aspired to psychophysics with which I could objectively analyze somewhat vague and subjective areas such as mind and spirit by a scientific approach.

I found it interesting that cortical mechanisms could be clarified by simply analyzing what we perceive when looking at, for example, an optical illusion. Moreover, when I moved on to graduate school, I encountered a number of attractive research topics and somehow my path came to be set. I believe it all started with my desire to understand why human beings have capabilities that machines do not.

—In this research field, when do you feel that you have achieved something and obtained a good response?

In research fields such as optical fibers and quantum computers, it should be easy to define results that are clearly recognizable and there should be many opportunities to receive concrete feedback. But in research targeting human beings, how to go about understanding the obtained results is a problem in itself, which makes it difficult to reach conclusions. But it is exactly for this reason that one can become addicted to such research and find it hard to extract oneself. It’s like taking up the challenge of playing a difficult game that never ends. There is consequently still a question as to what standpoint this research should be conducted from, and I continue to reflect on this, though it makes me feel like a perpetual student.

However, science, being what it is, is a win-lose proposition—it’s important to do good work, that is, compelling work, that can win the admiration of people. As I described earlier, research on material perception has expanded in recent years, and I believe that our results have contributed to this expansion. At the same time, I have been working in this field for many years with many questions still unanswered, and this can be depressing and it sometimes prompts me to question myself.

With regard to material perception, a pioneering paper that I submitted in 1998 helped me to recognize the value of my work, and from around 2000 on, my research began to draw attention from those around me. It was also reassuring to receive feedback indicating that my work had played a part in the growth of this research field.

I have also been recognized as a specialist in fields other than material perception such as visual motion and time perception, and I feel that I have been able to have an impact on those fields (Fig. 3). But as there are many things that are still not understood, I would like to make more original proposals in the future, and to this end, I plan to put my nose to the grindstone as a researcher with a worldview.

Fig. 3. Example of research on judging time.

I worry every day about not obtaining convincing answers in my research, but this worry seems to dissipate when I hold discussions with young researchers in NTT. It is also fortunate that NTT does not demand short-term output. My aim is to exercise my ideas freely in order to come up with something novel that represents a leap from the present, and I can do this because I am not expected to set detailed targets.

—What with obtaining no answers and forever seeking answers, your work is really an extraordinary endeavor. How do you keep motivated?

My research to date has not been very goal-oriented. Of course, there are various ideas about how my individually obtained results might be applied. My approach, however, is to feel my way along and break down barriers a little at a time in my quest for a breakthrough (imagine the videogame Breakout). What barrier to choose I leave to intuition. But once I began, I want to break it down completely even if it means doing it one piece at a time.

I am also conscious of dropping an idea that I have become overly attached to and looking at the problem from a different viewpoint. Nevertheless, it’s not that I’m doing anything special to unleash new ideas. Call me obstinate or non-conformist, but this is just the way that I enjoy thinking.

I could not have reached where I am today without the many things that I learned from overseas pioneering researchers in their journal papers. I have recently had opportunities to talk with some of them at academic conferences, and they really enjoyed hearing this. Perhaps they felt that I was continuing in their footsteps and carrying on their ideas. They made comments about their expectations of me and about our common approaches, which I was very glad to hear.

Being a little different from others is fine. Take joy in different ways of thinking!

—What advice would you give to young researchers?

Each and every researcher holds a completely different worldview, so there will always be some misunderstandings between them. Still, enjoying such differences can be a meaningful thing.

Although it is a characteristic of Japanese to try to reach mutual understanding with others, I believe that researchers are different—they are naturally argumentative. In research, it’s important to disagree. I would like young researchers to enjoy the fact that people can have different viewpoints and ways of thinking.

I also believe that it is not good to overly study other people’s work or listen excessively to what other people say. You will become blind to your differences from other people unless you can listen while having your own hypotheses and way of thinking. Isn’t it true that each person is born with something special? That, I feel, is the key to understanding researchers. Of course, there are some occupations that call for an ability to gather up and disseminate information correctly, but in the case of researchers, I think that they must first be able to develop a different perspective from other people instead of trying to achieve a balance with those around them. There’s no need to be a good boy or good girl. I never thought that being different from other people was a bad thing; indeed, I purposely set out to be a little different.

Researchers are often asked to write about and explain their research in an easy-to-understand manner. However, for researchers who have unique perspectives on things, explaining their findings in a way that anyone can understand on the basis of common sense or accepted wisdom would appear to be difficult. In today’s society, though, that excuse is becoming more difficult to fall back on. The situation is such that people without good social and communication skills may fail to win approval or praise. Perhaps researchers will now be asked to have good writing skills and the social awareness of screenwriters and movie directors! I truly feel that this is necessary for people who specialize in certain fields like myself in brain science. I would therefore like young researchers to cultivate a sense of balance so that their writings will attract the interest of the general reader while still being grounded firmly in science.

—Dr. Nishida, what objectives have you set for yourself looking forward?

I am still looking for answers, so it would be great if I could put myself again in a situation in which I could devote myself to round-the-clock research. I want to go forward, but not in any particular direction. I want to open up new areas that have yet to be fully explored, and I want to expand my endeavors freely without being confined to a particular format. I believe that new things are born when one engages and clashes with new people. I want to continuously challenge myself by examining my current situation and asking myself whether that’s good enough.