Toward Life-enriching Technologies

Probing the human mind via an integrated approach

—Dr. Kashino, could you first give us some background to your research?

Since entering NTT, I have been exclusively researching human perception with a focus on human auditory mechanisms. While hearing a sound may appear to be easy, it is actually a very complex process. If you visit the Illusion Forum website [1] that we have created, it’s plain to see, or rather, it’s plain to hear, that this is so. In hearing, a variety of peculiar things can occur; for example, you might hear nonexistent sounds or hear the same sound in different ways. Such phenomena are manifestations of brain functions that enable humans to adapt to their surroundings. In my research, I have been working to elucidate these clever neural mechanisms.

In this kind of research, I have found it essential to integrate three approaches. The first is to quantify the features of human perception, as in the case of illusions such as those that I just mentioned. The second is to study the workings of the human brain and nervous system that serve as the hardware for achieving those features. And the third is to comprehend the principles of information processing that make those features possible in the first place. In other words, we can begin to explain the mechanisms of hearing through the three fields of psychology, neuroscience, and information science. Let me make an analogy here. If I were to research the flight of birds, I could not simply analyze wing strokes and the structures of wings and muscles: I would also need to understand the principles of flight, that is, of aerodynamics. Only then would I be able to apply the knowledge gained to aircraft.

—What is your current research theme?

Well, my research is still centered on hearing, but I am currently working to broaden the scope of my studies. It occurred to me that this integrated approach that I’ve developed could perhaps be used to scientifically explain heretofore elusive problems like preferences and feelings. Let me give you three specific examples of this expanded research in our group.

First, in relation to hearing, we’re researching the neural mechanism that enables one to selectively listen to only what one intends to hear in a mixture of many sounds. (Fig. 1). This cocktail party effect is often said to be an outstanding feature of human hearing, but it tends to be deficient in some groups of people such as the elderly and hearing impaired. A hearing aid often just amplifies noise as well, resulting in no substantial improvement in hearing. In our research on testing hearing ability, we have found that some people who show no apparent abnormalities in an ordinary otological test may sometimes have difficulty hearing a sound in the presence of competing sounds. We have been investigating a new hearing test that can identify such people and we are close to finding the cause of this phenomenon.

Fig. 1. Uncovering neural information-processing principles from an illusion.

Cases in which a hearing problem can hinder an individual’s full participation in society despite ample skills or motivation are expected to multiply as we enter the aging society. The creation of a barrier-free society is a key objective of information and communications technology (ICT), and I think that we can help give many people a greater sense of purpose in their lives if we can develop a hearing aid that lets them hear only what they prefer to hear.

Second, we’re researching ways of quantitatively predicting preferences in relation to music, design, etc. For example, a person may respond favorably to certain kinds of music saying something like “that’s good” or “I like that music.” In such a case, the music is probably somewhat familiar and not completely new to the user. In other words, music that is all too common for the listener can be boring while something that is too novel or original may turn the listener off. So how new should something be to be the most attractive?

The ability to derive this in theoretical terms would no doubt bring big changes to the way that products are designed and advertised. Traditionally, design and advertising have depended on the experiences and hunches of those in charge, but a systematic approach could make the process more efficient. All this might sound too good to be true, but I believe that such predictions should be possible to some extent by applying the principles of neural information processing. What we need here is some kind of proof, and to this end, we are conducting various types of experiments.

Third, we’re researching subliminal signals and body language (Fig. 2). In interpersonal communication, information that cannot be directly conveyed by words plays an important role. In particular, the atmosphere between two people is often created by slight changes in facial expression or tone of voice that are usually unnoticeable to either party. This type of unconscious exchange of implicit information between two people helps to establish smooth communication. We are now working to explain the actual mechanisms of nonverbal communication through experiments that measure slight body movements, physiological changes, and neural activity during interpersonal communication.

Fig. 2. Clarifying subliminal signals based on IIPI.

Such implicit information was probably discarded as unnecessary in past technologies because, in conventional communication systems, it was sufficient to assess whether a phrase like “thank you” was actually heard as “thank you”. As a consequence, information that could convey whether this “thank you” was actually an expression of gratitude or a sarcastic remark was simply ignored. However, if such elements of communication can be clarified, it should be possible to achieve a form of communication with a high level of presence such as in a videoconference system connecting remote locations via the network.

Toward a career in hearing research from an attraction to temporal flow

—What moved you to become a researcher in the field of hearing?

Most of my instructors and seniors in my university research laboratory were researching topics related to vision. For example, they were studying the response of goldfish retina cells to light, the neural activity of monkeys, and human visual illusions. Moreover, my advisor was researching mathematical models of visual pattern recognition. I learned a lot about methodology from my involvement in these studies. However, I had developed an interest in sounds through listening to a wide variety of music and building electronic gadgets from discarded parts starting in my elementary school days, and what I really wanted to research was the perception of sound, or in other words, hearing. In Japan, there were not many researchers in this area. I could therefore pursue this interest to my heart’s desire, which suited me perfectly.

In addition, since my childhood, I’ve been interested in how one captures things that change and the nature of impermanence. Living things move and change, and this change occurs continuously together with a temporal flow. I found this flow to be very appealing.

At the same time, I thought that sound, which cannot exist without incessant change, is closer in essence to flow than vision. This is also one factor that led me to specialize in hearing.

—What kind of researcher do you think you are?

I believe that my motivating force is a philosophical awareness of problems, but in my case, pure speculation by itself has its limits. Come to think of it, I’ve enjoyed doing experiments since my childhood. What I like about experiments is that they do not always turn out as expected. Of course, if we establish a certain hypothesis, perform an experiment to test it, and obtain a result that meets our expectations, then it’s time to celebrate! But this could also be boring in the sense of a pre-established harmony as discussed in philosophy. An unexpected result, on the other hand, can broaden our scope of thought and be much more interesting. If we give lots of thought to an unexpected result, we may find it to be the most natural outcome after all.

At the same time, I also enjoy theory, and it gives me great pleasure to be able to provide a uniform explanation for perplexing things or unrelated things through simple principles. In short, a continuous cycle of theory and experiment is essential to my integrated approach.

Baseball and work: unconditioned nature

—I’ve heard that pitching—as in baseball—is a hobby for you.

Yes, that’s right, but not in the sense of playing sandlot baseball just for fun. What I pursue is simply the act of throwing with great devotion. I do this every day if possible. But please don’t ask me why I do it. Perhaps I find the act of throwing one ball after another interesting like the cycle of hypothesis and experiment. At any rate, one throw out of maybe 30 or 40 surprises even me. But if I get all excited and try to do it again, I end up making a throw that’s not nearly as good. This is very strange.

A ball is thrown not by the power of one’s arm but rather by accelerating the tips of one’s fingers to the utmost through power generated by the movement of one’s weight and the twisting of one’s lower body. An arm that is simply shaking looks exactly like a flexible whip. However, when you consciously attempt to throw a fast ball, excessive power will enter your arm and shoulders, and at that instant, the power that would normally be conveyed by the lower part of your body and by your trunk will fall behind. It so happens that when I fall into some kind of rhythm and stop worrying about my next pitch, I get this sensation of being driven only by gravity and inertia and I end up making some incredible pitches while feeling quite relaxed.

Perhaps this is an example of unconditioned nature as described in the Zhuangzi. In some way, you can perform your best when you lose the self and do not fight the providence of nature (when you go with the flow). If, by plain luck, you were to throw a great ball, it could then be said that you already had it in you to do so, but the problem now is whether you can invoke that ability at will. In other words, the problem is how to coordinate with good timing the roughly 400 skeletal muscles in the human body without mutually cancelling out the flow of power through them. It is said that the great baseball player Ichiro, though having a thin figure, can outperform other major-league players far superior in strength by optimizing muscle coordination.

For this reason, I do not go out of my way to build up my muscles, but look for ways of improving my coordination. I feel as if I’m advancing little by little, and I don’t think it’s altogether unrealistic to aim for a pitching speed of 130 km/h at 50 years old.

The importance of natural coordination should also hold true in work. The more one thinks about having to go all out, the more one’s natural pace will fall apart without any gain in performance. This also applies to management in an organization. If superiors in their excitement to excel assign useless work, subordinates are bound to become exhausted. Whether it be an individual or an organization, I don’t think that intrinsic potential can be tapped in this way. It must be said, however, that working in a natural way is really difficult.

Future outlook for ICT

—What do you think is the future of ICT?

The research that I conduct aims to elucidate this providence of nature. In other words, I am involved in scientific research, but at the same time, I am deeply concerned with the future of ICT. As a basic belief, I feel that ICT is not simply a means of making life more convenient—it must also serve to enrich the lives of human beings. And to this end, ICT must help people achieve their intrinsic, natural state of being. Accordingly, basic research on clarifying the providence of nature in relation to human beings should, in the end, lead to a form of ICT that works to enrich people’s lives.

It would be natural to think that ICT could be used to good effect by establishing a barrier-free society and to assist the sensory and motor functions of the elderly. But on a long-term basis, what if such measures turned out to actually weaken brain functions? My answer to this is to make use of nature and isolate and enhance intrinsic neural and physical functions. Ideally, there would be a set of tools that can do just that.

While it is easy to measure the efficiency and physical performance of just about any product, it is difficult to measure the extent to which such a product actually enriches a person’s life. It is exactly this point that should become a matter of importance in the years to come. The provision of top specifications and high quality is a matter of conscience for certain types of enterprises, but as a natural precondition for this, I believe that products should be designed taking into account feelings and perception so that possessing a product brings some delight into a person’s life. For this reason, feelings and perception should be subjects of discussion in scientific and engineering fields.

Today, Japan provides the fastest and most inexpensive broadband environment in the world. The optical-fiber service area covers 90% of all households, but no more than 30% are actually making use of these services. Why are the remaining 60% not using broadband? No doubt price is one factor, but could another factor be not actually being able to feel the value of broadband? Do these households perhaps feel that their current way of using the web, email, and weblogs (blogs) is more than sufficient? If a household were to upgrade to broadband or introduce ICT, to what extent would the lives of its family members be enriched? Customers don’t think in these terms, and neither, I suspect, do service providers. Here, remote medicine, to give one example, could be used not simply as a means of transmitting digital medical records but also to encourage interest in broadband and ICT. So there’s still hope. I feel strongly that disseminating such a concrete image of broadband services should be the responsibility of us basic researchers.

To young researchers

—Dr. Kashino, please leave us with a message for young researchers.

It is often said that originality is essential for researchers, but I feel that the meaning of this is sometimes misinterpreted. In particular, in today’s competitive society, we are made to feel like a failure if we don’t do something different from other people, and we start out by thinking “How can I do something unique?” However, great baseball players like Nomo and Ichiro were not great just because they had different forms of play. Each worked ceaselessly to optimize his individual performance and simply came to adopt that form as a result. While appearing to be unique at first glance, their forms of play owe a lot to the providence of nature.

I think this also holds true in research. The etymology of the word original is origin, and the origin of ICT is to find out what is needed to enrich people’s lives. If we return to this origin and listen carefully to the voice of nature, I think that we should be able to perform great work.

Reference

[1]	Illusion forum (in Japanese). http://www.brl.ntt.co.jp/IllusionForum/

↑ TOP