Towards Understanding Human Skin Sensations

1. Tactile × perceptual psychology

The hand is a tool and sensor. The manual dexterity of humans has contributed to the remarkable progress of our species. To skillfully use tools, it helps to be able to measure and process sensory information quickly and correctly. We (myself and other researchers at NTT Communication Science Laboratories) are interested in this processing of information through the skin. Tactile references often appear in metaphors that allude to a deep understanding of things. For example, someone with the “common touch” understands how ordinary people think, someone with the “magic touch” is exceptionally skilled, and someone with the ability to retrieve information instantly is said to have it “at their fingertips.” However, surprisingly little research has been done on the information-processing aspects of how tactile sensations actually work, and the mechanisms with which they are processed are not well understood. Since tactile sensations originate from the boundary between oneself and the outside world, it has also been technically difficult to conduct studies such as directly measuring or artificially reproducing phenomena that take place when touching a surface. With recent advances in technology, progress is gradually being made in the clarification of tactile sensing, which is catching up with research on visual and auditory sensing.

Perceptual psychology is a field of study in which humans are regarded as systems and which seeks to identify their internal processes. It provides a means of determining what types of perception a human observer obtains as output when provided with input stimuli such as images or sounds. For example, people can perceive changes in brightness when a light is slowly switched on and off, but as the switching-speed increases, the flickering becomes less visible. At higher speeds (like the frequency of fluorescent lights), this flickering becomes imperceptible. By making small changes to input stimuli in this manner, we can examine the ranges of various physical quantities that affect our perception.

This article introduces two illusions that offer inroads into reasoning about the mechanisms and characteristics of tactile perception. These illusions are phenomena in which, for example, identical physical inputs are perceived as being different or different physical inputs are perceived as being the same. These illusions allow us to investigate what type of information the brain is sensitive and not sensitive to. They are also useful for considering the means in which information can be presented more effectively.

2. The illusion of mixed frequency from two vibrations

When you move your fingers across the surface of an object, it causes vibrations in the skin of your fingertips. These vibrations allow you to distinguish between different textures, such as rough and smooth surfaces. Our perception of these surfaces is based on vibration frequencies as encoded by tactile sensors, i.e., mechanoreceptors, in the skin. The skin includes different types of sensors in much the same manner that the retina includes different sensors that are sensitive to different colors. Each sensor has a different shape and distribution. Some are small and concentrated at shallow parts of the skin surface, while others are large and scattered deep within the skin. As a result, each type has its own idiosyncrasies. For example, some sensors respond to slow deformations, while others respond to high-frequency vibrations such as received vibrations (Fig. 1). How are frequencies encoded using the signals of these different sensors?

Fig. 1. Response characteristics of mechanoreceptors and the nerves.

The difference in frequency sensitivities across sensors is observed not only in tactile sensing but also in visual and auditory sensing. For example, when we see red light (which has a long wavelength) and green light (which has a slightly shorter wavelength) at the same time, we perceive yellow light (which has a wavelength between those of red and green). It is not possible to perceive the red and green lights separately. This is described by the well-known theory of the three primary colors. However, when we hear the musical notes C and E played at the same time, we recognize them as two separate notes forming a major third, instead of hearing the note D that lies between them. Which of these processes is exhibited by the tactile system?

We conducted a series of experiments to ascertain whether the tactile system detects frequency in a manner analogous to the three primary colors by testing whether an intermediate frequency is perceived when multiple vibrations with different frequencies are presented [1]. If the skin is simply subjected to a synthetic vibration obtained by adding two frequencies together, this vibration may be physically attenuated by the time it reaches the sensor due to the elastic nature of the skin. We therefore simulated a situation of seeing red and green at the same time by applying two vibrations of different frequencies to two different fingers (the index and middle fingers) at the same time. In these experiments, we applied synthetic stimuli consisting of two sinusoidal vibrations at two different frequencies on two different fingers of each participant and compared the perceived frequency with comparative stimuli consisting of two sinusoidal vibrations of the same frequency applied to the two fingers (Fig. 2). (See [1] for details about this method.) We found that the perceived frequency of the synthetic stimuli was an intermediate frequency between the two presented frequencies of the synthetic stimuli. Based on this result, it appears that the perception of vibrations is similar to the perception of the three primary colors of light. We also found that this phenomenon occurred not only when the synthetic stimuli were applied to two fingers on the same hand but also when applied to fingers on the left hand and fingers on the right hand. Even when vibrations with completely different frequencies were applied to clearly different parts, i.e., fingers of the left and right hands, the participants seemed to perceive as if the two vibrations had the same frequency.

Fig. 2. The illusion of mixed frequency from two vibrations.

However, we also found that the synthetic stimuli do not mix completely. When touching synthetic stimuli with two fingers, it is possible to feel differences (e.g. in vibration strength) between the two fingers. Also, the synthetic stimuli feel noisier than simple sinusoidal vibration at a matched frequency. Unlike in vision, there is no guarantee of perceptual equivalence between synthetic stimuli and matched vibrations in touch.

In summary, when you experience multiple vibrations at different frequencies via the skin, it is possible to perceive multiple vibrations, which shows that the encoding of tactile vibrations differs from that of visual colors. It also differs from the encoding of auditory sounds because the tactile system cannot encode the two frequencies separately. It seems that tactile-frequency information is processed through a strategy that lies somewhere between the strategies of sight and hearing.

3. The illusion of textures that look different but feel similar

Everyday objects have a variety of surface irregularities and textures. Since it had been technically difficult to artificially design tangible textures exactly as intended, tactile texture research has been conducted either by creating a simple experimental stimulus or using existing materials without alteration. To bridge the gap between simple experimental stimuli and complex real-world textures, we use laser cutters and three-dimensional (3D) printers to create surface textures with complex spatial patterns [2].

For texture design, we used images as height maps (Fig. 3(a)). Lighter areas correspond to bumps and darker areas correspond to hollows. Using images like these, we can easily adjust statistics, such as the spatial frequency and bandwidth, of surface irregularities. In this experiment, each image was 3D printed as a 4 × 4 cm sample piece with a maximum engraving depth of 2 mm to create a set of surface textures. The five images shown in Fig. 3(a) have central frequencies represented numerically (similar to grit numbers on sandpaper) and are modulated with relatively simple statistical quantities. In the 3D-printed sample stimuli based on these images, the central frequency increased from left to right, resulting in surfaces ranging from bumpy to grainy. With these stimuli, texture discrimination is quite easy. Humans are very sensitive to central frequency modulation. Therefore, how well can human observers discriminate between 3D-printed stimuli based on completely different images? Five natural scene images obtained from an image database are shown in Fig. 3(b). Although the brightness values of these images were matched to have the same average and variance, there are differences in the spatial arrangement of black and white that are clearly visible when viewed with the eye. However, it was not easy to distinguish between the corresponding 3D-printed textures by touching with the hands. It was particularly impossible to distinguish between the stone, coral, and leaf textures shown in Fig. 3(b) (the images outlined in dark blue), even though they looked completely different. This illusion shows that the tactile and visual systems use different mechanisms to process spatial textures.

How does information processing differ between the eyes and hands? To answer this, we analyzed the original images used as templates for the 3D-printed surfaces. A characteristic of natural images is that the amplitude decreases at higher spatial frequencies. We calculated the Fourier transform of each of the five images shown in Fig. 3(b) and compared the amplitude and phase spectrum of each transform (amplitude indicates the intensity of the light and shade and phase conveys information about the distribution of light and shade). Although the amplitude spectra were very similar for all five images, their phase spectra were completely different. The visual system processes both the amplitude and (some of) the phase characteristics of an image after Fourier decomposition. It seems that the tactile system mostly processes the amplitude characteristics without using much of the phase characteristics (Fig. 3(c)). This finding can be exploited to freely create visually dissimilar textures that feel similar when touched. The calculation of phase is computationally more expensive than the calculation of amplitude. Our tentative understanding is that the tactile system does not expend many resources on processing phase information and may instead have evolved to be sensitive to small differences in amplitude characteristics.

Fig. 3. The illusion of textures that look different but feel the same.

As mentioned above, we are using perceptual psychology methods to study tactile sensory mechanisms. Originally, most research on the human senses focused on vision and audition, and few considered the tactile sense. However, the number of tactile studies has recently increased due to advances in the tools that can be used for conducting research. Although the five senses involve many similar processes, the fact that these sensory organs evolved in different forms suggests that they may be processing information in different ways. In other words, visual information is processed by visual rules, auditory information by auditory rules, and tactile information by tactile rules. We hope that by accumulating an understanding of these rules of perception, it will be possible to gain an understanding of the tactile sense while at the same time increasing the possibility of multimodal information presentation.

References

[1]	S. Kuroki, J. Watanabe, and S. Nishida, “Integration of Vibrotactile Frequency Information beyond the Mechanoreceptor Channel and Somatotopy,” Sci. Rep., Vol. 7, 2758, 2017.
[2]	S. Kuroki, M. Sawayama, and S. Nishida, “Haptic Metameric Textures,” bioRxiv, 2019. doi: https://doi.org/10.1101/653550.