Communication Research Focused on Tactile Quality and Reality

1. Introduction

The presentation of tactile information has been used in mobile terminals and gaming devices in recent years. This has spurred research and development of information-presentation technologies based on tactile illusions. Illusion is a phenomenon that results when there is a discrepancy between what our brain perceives and what actually exists. It forms the basis for presenting information using limited resources and for creating a sensation that has not been perceived previously. Some examples of presenting information via tactile illusions are shown in Fig. 1. As they indicate, understanding the principle of how humans sense their environment is the key to developing new technologies for tactile information presentation and communication.

Fig. 1. Excerpt from the exhibit on information presentation devices using tactile illusions entitled "Touchable Illusions: tools for understanding human tactile perception" (by Kitagawa, Ho, and Watanabe) in NTT Communication Science Laboratories Open House 2011.

Humans perceive and differentiate various tactile qualities such as roughness, hardness, comfort, and fit from all aspects of daily life (clothing, furniture, one’s own skin or another person’s skin, etc.). In the same way as we feel comfort or discomfort with what we wear, we feel comfort or discomfort even in the simple act of pushing a button, depending on the button’s material and how it feels when pushed. Toys based on only the sensation of touching a button have recently been created. As such, humans use tactile sensation not only for operating objects, but also for perceiving differences in tactile qualities and for feeling comfort, affection, and other emotions.

Likewise, we confirm the existence of objects by touching them. By touching something with our hands, we make sure that the object actually exists, and by touching a person’s body, we ascertain the presence of the other person. Thus, the sense of touch is considered to be the most important sensation that humans use to confirm the reality of things around them. In this article, I discuss the unique features of touch, namely, tactile quality and reality, and introduce research and projects being conducted in NTT Communication Science Laboratories on these topics.

2. Tactile quality and onomatopoeia

First, I will explain about the qualities of tactile sensation and give an overview of our research on its classification. Analyzing the words for expressing sensations is one of the methods used in psychological analysis and sensation classification. For example, the colors red, green, and blue are the most commonly used categories of visual sensation. Studying the relationships among these color categories is one way to understand how humans classify visual sensations. For example, the color hue circle shows that green is opposite to red, and it provides intuitive understanding of the relationships among the colors that humans perceive. The relationships among tactile sensation categories, however, are not as clearly generalized as color category ones. Although categories pertaining to the material composition of objects, such as metal, cloth, or paper, exist, there are currently no generally used categories for the sensations derived from touching objects.

To analyze tactile sensation categories, we have focused, in particular, on onomatopoeia (general term for mimetic words formed by imitation of a sound or synesthetic association between a sound and sensation) used to express tactile sensations. Onomatopoeia is a convenient way to express sensations in everyday life and is also widely used in Japanese manga and literary works. Compared with other languages, Japanese is known to have a large number of onomatopoeic words for tactile sensations. Thus, we chose tactile onomatopoeic words as tactile sensation categories and analyzed the impressions that they connote. In our experiment, we chose forty-two Japanese onomatopoeic words for tactile sensations and asked participants to numerically rate their perception of how the onomatopoeic words connote size, friction, and viscosity. For example, the word zara-zara may connote size at a level of 3, friction at 5, and viscosity at 2 for a certain participant, indicating how he or she perceives different onomatopoeic words in terms of these impression criteria. A two-dimensional distribution map of onomatopoeic words constructed from the responses of 20 participants is shown in Fig. 2 (For details of the analysis, refer to [3]).

Fig. 2. Two-dimensional distribution map of onomatopoeic words for tactile sensation.

The distribution map shows a spatial diagram of how tactile sensations are categorized by Japanese people. In this diagram, onomatopoeic words that express closely related sensations are also located close to each other on the map. Mapping the onomatopoeic words spatially enables visualization of categories and grouping axes of tactile sensations. On the distribution map, words such as jyari-jyari and jyori-jyori, which express roughness, are grouped in the upper left, while words such as tsuru-tsuru and sube-sube, which express smoothness, are grouped in the lower right. The word kori-kori, which expresses hardness, is located in the lower left, while words such as gunya-gunya and necho-necho, which express softness, are grouped in the upper right. And words such as nuru-nuru and nyuru-nyuru, which express wetness, are grouped in the middle right, while words such as gasa-gasa and kasa-kasa, which express dryness, are grouped in the middle left. These groupings of onomatopoeic words, which also indicate their phonemic similarities, represent the major categories of tactile sensations. The distribution map also revealed that the sensations of smoothness/roughness, hardness/softness, and wetness/dryness serve as the basic classification criteria for tactile sensations.

3. Visualizing tactile material relationships

Next, we used the distribution map to visualize the relationships among tactile materials in accordance with the spatial distribution of tactile onomatopoeic words on the map. We used the distribution map in a workshop aimed at enabling participants to recognize how they perceive objects through touch [4]. Workshop participants were asked to touch ten different kinds of materials and determine their locations on the map on the basis of how they perceived those materials. Then, the participants were asked to choose a material whose tactile quality they liked and another that they did not like and to draw an arrow from the latter to the former. As shown in Fig. 3, the arrows on the map show the directions of personal preferences. The onomatopoeic words were superimposed on the pictures of the materials for easier viewing. On this map, most of the arrows pointed from the hard and dry sensations in the upper left to the smooth sensations in the lower right (preference for smoothness) and to the soft and wet sensations in the upper right (preference for softness). The onomatopoeic distribution map enabled a systematic discussion of personal preferences.

Fig. 3. Arrows showing personal preferences.

This research is based on the diversity of words (onomatopoeia) used to express tactile sensation in the Japanese language and is aimed at establishing the principles for evaluating the quality of tactile sensations. Although, as mentioned earlier, the theories for effectively determining visual relationships and auditory relationships, such as color and musical scales, are already more or less well established, through this distribution map, we hope to enrich communication through touch by providing a useful and effective format for conveying the quality of tactile sensations.

4. Perceiving reality through the sense of touch

In this section, I discuss how the sense of touch provides us with a way to confirm the reality of objects. Visual and auditory modalities deal with the sensations of objects that are far from the body, while tactile sensation is derived from the act of directly touching an object. Therefore, tactile sensation provides a means of confirming the existence of the object as well as knowing what kind of object it is. We have attempted to extend this ability of confirming the existence of objects by touching them to objects that cannot be touched. We believe that touching invisible or conceptual objects (called haptization) will allow a deeper understanding of these objects through their physical reality.

5. Workshop on touching heartbeats

We developed a simple device, shown in Fig. 4(a), for haptizing the heartbeat, a very basic phenomenon of life, in a workshop called Heartbeat Picnic aimed at reaffirming people’s lives [5], [6]. The participants were asked to hold a stethoscope in one hand and a vibration speaker (referred to as the heart box) in the other, as shown in Fig. 4(b). When the stethoscope was placed on the person’s chest, his or her own heartbeat was output as both sounds and vibrations from the heart box, which enabled participants to not only hear their own heartbeats but also feel them as vibrations.

Since the workshop was held outdoors, participants were free to move about as if they were having a picnic, and they were able to feel the changes in their heartbeat with their own hands. Moreover, as shown in Fig. 4(c), by exchanging their heart boxes with those of other participants, they were able to feel the differences between their own heartbeats and those of other people. The experience of touching heartbeats, which is usually impossible, provided each participant with the opportunity to appreciate the importance of his or her own life as well as those of others.

Fig. 4. (a) Equipment used in the workshop: stethoscope, vibration speaker (heart box), signal processing circuit, and batteries. (b) Equipment in use. (c) Exchange of heart boxes among participants.

Life appreciation workshops are usually held in a natural environment where participants can directly experience the abundant life around them. In the highly digitized cities in which we live, however, there are almost no opportunities for us to appreciate our own lives. Although there are sometimes opportunities for us to face life-and-death situations through accidents, such as when we go to hospital, these are extraordinary and limited experiences. Everyday life in this modern digital world provides us with little opportunity to appreciate the dignity of life. This workshop, however, is aimed at enabling people to experience the reality of life without having to escape from their modern everyday environment, by evoking the imagination through the sense of touch.

We received various responses from the workshop participants regarding their experience of touching their own heartbeats. Some said that they felt a sense of endearment with their own heart boxes and a sense of affinity and kindness upon feeling other people’s heart boxes. One participant commented that the only other time she had felt another person’s heartbeat was when she had a baby in her womb. These responses indicate that the experience of touching the heartbeat, even if only simulated and artificial, provides an opportunity to appreciate the vitality of one’s own life and the vitality of other people’s lives.

In our modern world, we live surrounded by information in the form of digital code representing phenomena, but such representations are not the real phenomena themselves. Our digitized world enables us to use symbols to simultaneously convey phenomena to many people. Symbolic representations of phenomena, however, dilute the actual sensation of the environment in which phenomena occur and weaken our connection to them. We believe that using the sense of touch, as in the workshop, enables us to have a physical and realistic connection with the wealth of information about various phenomena that surround us every day. As the information era advances, touch-based communication technologies will provide the means for us to more realistically understand ourselves and people around us.

6. Future research prospects

In this article, I introduced research activities based on the unique features of tactile sensation, namely, tactile quality and reality. Using these experiences as a springboard, we plan to continue our efforts to elucidate human sensory mechanisms and carry out practical initiatives for exploring the role of tactile communication in society and in our daily lives.

References

[1]	H.-N. Ho, J. Watanabe, H. Ando, and M. Kashino, “Mechanisms Underlying Referral of Thermal Sensations to Sites of Tactile Stimulation,” The Journal of Neuroscience, Vo. 31, No. 1, pp. 208–213, 2011.
[2]	J. Watanabe, Y. Fukuzawa, H. Kajimoto, and H. Ando, “Presentation of Feeling When Pierced Using Apparent Movement Passing thorough the Body,” Transaction of Information Processing Society of Japan, Vol. 49, No. 10, pp. 3542–3545, 2008 (in Japanese).
[3]	T. Hayakawa, S. Matsui, and J. Watanabe, “Classification Method of Tactile Textures Using Onomatopoeias,” Transaction of Virtual Reality Society of Japan, Vol. 15, No. 3, pp. 487–490, 2010 (in Japanese).
[4]	Tactile texture workshop. http://www.junji.org/texture/ (in Japanese, but includes videos).
[5]	J. Watanabe, Y. Kawaguchi, K. Sakakura, and H. Ando, ““Heartbeat Picnic”: Workshop for Touching Heartbeats,” Trans. of Virtual Reality Society of Japan, Vol. 16, No. 3, pp. 303–306, 2011 (in Japanese).
[6]	Heartbeat Picnic. http://www.junji.org/heartbeatpicnic/

↑ TOP