Feature Articles: ICT R&D Initiatives for a Smart World that Harmonizes People and Environments

Vol. 18, No. 3, pp. 59–64, Mar. 2020. https://doi.org/10.53829/ntr202003fa9

Sports Social View—Method of Inclusive Sports Appreciation Extracting and Sharing the Essence of Sports through the Sense of Touch

Akiko Hayashi, Junji Watanabe, and Kentaro Shimizu

Abstract

NTT Service Evolution Laboratories is collaborating with the Tokyo Institute of Technology to enable people with various physicalities, including those with disabilities, to enjoy sports together. This article introduces a new method of enjoying sports that not only uses the audiovisual senses but also the sense of touch.

Keywords: tactile information, inclusive design, translation of sports

1. For inclusive sports viewing

What can be done so that people with various physicalities, including people with disabilities, can enjoy watching sports together? Understanding diversity has become increasingly important as reflected in the vision “diversity and harmony” of the international sports event to be held in Tokyo in 2020. NTT Service Evolution Laboratories is collaborating with Associate Professor Asa Ito of the Tokyo Institute of Technology on research into sports viewing for the visually impaired. We started with the aim to support people with disabilities to enjoy sports, but as the joint research progressed, rather than support, we arrived at the awareness of accepting differences and co-creating new experiences.

2. The limitations of describing things with words

The most common way for the visually impaired to enjoy sporting events is through audio commentary. There are people who enjoy audio commentary of baseball or soccer. However, some visually impaired people who once were able to see before have said, “We were able to enjoy exciting scenes as we saw them,” but after their sight deteriorated they described their sports enjoyment as only a contextual overview of the flow of matches through audio commentary. In other words, they became unable to enjoy the raw skills of the players. There are also limitations to audio commentary for some competitions. Although competitions, in which offense and defense take turns, such as baseball, can be understood through audio commentary, it is difficult to convey the fine details, such as the force of movements and rhythm, in competitions such as tennis with its dizzying rallies or judo in which offense and defense are deployed at the same time.

A sense of unity is also an important factor affecting satisfaction when watching sports [1]. This means getting excited at the same time as a spectator nearby. However, it is difficult for the visually impaired to feel a sense of unity with the surrounding spectators. Therefore, current audio commentary is problematic in that it cannot express movements in sports very well, and the visually impaired are left out of the surrounding excitement.

3. Physical translation of sports

To enable sharing of sport-viewing experiences with the visually impaired, we developed a method of viewing sports that physically re-expresses (translates) what is happening instead of using language. With this method, we attempted to express the force and rhythm of movements during a tennis match and judo match, which are difficult to verbalize.

3.1 Tennis match

The translation of the tennis match involved a translator (a sighted person) and visually impaired person (with total late-onset blindness) sitting face-to-face with the translator tapping a circular board laid across both their knees (Fig. 1) to reflect the position and strength of the ball as it was hit while the translator watched the tennis match. During a rally, the board was struck rhythmically on the visually impaired person’s left and right, and he moved his head left and right as if he were following the movement of the ball to watch the rally. When a ball was hit and had a long flight, the visually impaired person could be heard uttering “this is a big one,” indicating that he was enjoying the spatial nature of the rally. The translator also conveyed not only the physical action of hitting serves but also the rhythm of the players before serves by tapping the board. Although such rhythm does not appear as actual movement, the translation of tension and rhythm helps the visually impaired understand the situation more deeply. This can be consistent with the notion that the processes of generating a sense of being deeply moved (“Kandoh” in Japanese) are tension and release [2].

Fig. 1. Translation of tennis match.

3.2 Judo

The translation of the judo match involved two translators (sighted persons) and a visually impaired person, and used a towel to express force interactions. Two translators held both ends of a towel, and the visually impaired person (with total late-onset blindness) grabbed the middle of it (Fig. 2). Translators played the roles of the judo players by pulling the towel up and down and left and right to express movements, pulling against each other, falling backwards to destabilize the opponent’s center of gravity, etc. The visually impaired person described his impression of it as like “another judo match was being held on the towel. It felt more judo-like that watching it on television.” The translations enabled some kind of transformation or interpretation to occur, which gave rise to other events while imitating the competition. This is because the translators were able to reflect the anxiety, tension, surprise, and joy that they felt in their translations. In other words, the visually impaired persons were able to experience another’s events through the translators’ interpretations, rather than just simply being conveyed visual information that had been converted into haptic information. This could be what led to the impression that “another judo match was being held on the towel” described above.

Fig. 2. Translation of judo match.

This research project was inspired by “museum social view” in the field of art appreciation. In art appreciation, people with visual disabilities and sighted people interact through dialogue in front of art works to deepen their understanding of them. Although art appreciation and sports watching may seem different, both are similar in that they entail deep understanding of the subject through the physicality and interpretation of others. Thus, we call our project “Sports Social View.”

4. Essence and description in translation

Having translated sports for several competitions, we realized that there are two factors in the expression of a translation. One is the sense of the essence of movements that express the rhythm and impact of a tennis rally or the destabilizing of the center of gravity in judo. The other factor is description. This is information that conveys the state of a competition such as the position of the ball and winning and losing. The translation is generally structured to put the essence on the description element. In tennis, the description element is expressed by tapping positions to represent the ball being hit in a rally, while the intensity and way of tapping express the impact of the player hitting the ball as the essence. In tennis or badminton, although the description elements that express the space in rallies are similar, the essences of the ball bouncing in tennis and the shuttlecock cutting through the air in badminton are different.

If we review sports from the perspective of essence and description elements, we can reclassify them from different perspectives. Hence, we are currently working on further deepening translations by asking researchers specializing in sports and sports players about the essence of competitions. Activities of this project have been published on the “Invisible Sports Picture Book” [3] website, which currently introduces various sports such as rugby, table tennis, and sailing. In the case of table tennis, although the speed and force of hitting seems important from the viewpoint of an amateur, we found “reading the rotation of the ball” is rather important, so rotation is expressed by hitting a disk (Fig. 3). Soccer and rugby are also competitions that develop around the ball in a similar wide-open space, but what is different is that soccer gets its attraction from the development of tactics, whereas rugby is highlighted by a battle of players trying to steal the ball from each other as they collide and the forcefulness of that activity. Thus, pursuing the essences of sports that involve fighting for balls entails significant differences. Even if someone is a sighted person, one could say that he/she could not “see” the sport. Thus, watching sports with translated expressions of essences such as those above may be effective not only for the visually impaired but also for sighted people.

Fig. 3. Translation of table tennis (Photo by K. Nishida).

5. Technology for sharing inclusive sports viewing experiences

Through this kind of sports translation, we wanted to share the experience of sports translation with even more people across time and space as the principle that the visually impaired and sighted can enjoy watching sports together became more or less established. We used tactile recording and presentation technologies to accomplish this aim. Focusing on the sports translation of tennis in particular, we produced exhibitions of “tennis match experienced in darkness” and “tennis match felt through the palm” through which one can experience sports translation anywhere and anytime via recording and playback of vibrations.

5.1 Tennis match experienced in darkness

“Tennis match experienced in darkness” involves a 1800 x 900 mm table that mimics a tennis court. Sports translations are carried out on the table, then participants experience a tennis match by touching the table (Fig. 4). The translator taps the table in sync with the tennis rally, and microphones are attached to the underside of the table at four locations to record the vibrations. Vibration speakers are attached at the same positions as the microphones, and the recorded vibrations are replayed with the video. Participants in this exhibit place their palms on the table and feel the vibration of the table being tapped. The experience consists of the above table and a large screen in a small room, and a tennis match is viewed under the following conditions. (1) Watching tennis with all information; video, ambient sound, and vibrations (sighted-people situation); (2) enjoying tennis with the video subtracted and only ambient sound and vibrations (vision subtracted); and (3) enjoying tennis with video and sound subtracted, with only vibrations (vision and hearing subtracted). This makes it possible to create different physical situations and enjoy sports through physical translation.

Fig. 4. Tennis match experienced in darkness.

In interviews after the experience, the visually impaired described their understanding of the match as “it was the first time I could tell where the ball hit in a tennis match” and described their feeling of unity as “I felt like I was watching with someone.” Some of the sighted people described the essence of tennis as having their “senses sharpened and feeling more the rhythm of rallies,” which indicates that physical translations are also an effective way for sighted people to enjoy sports.

5.2 Tennis match felt through the palm

“Tennis match felt through the palm” involves a 450 x 300 mm table that mimics a tennis court. In the same way as “Tennis match experienced in darkness,” tennis video and sound are replayed with a smartphone and headphones, and tapping vibration is replayed with vibrating speakers attached to the underside of a table at four locations (Fig. 5). Participants watch a tennis video on their smartphone screens while listening to the sound through headphones and feel the player hitting the ball by placing their palms on the table. The smartphone provides video and audio playback as well as power. This is a miniature version of “Tennis match experienced in darkness” that enables sports translation in stadiums and public viewing sites where an external power source is not available. As described above, exploring the spectator experience with the visually impaired also made it possible to create new ways for sighted people to enjoy sports, thus opening up the possibility of an inclusive spectator experience.

Fig. 5. Tennis match felt through the palm.

6. Future developments

Our project is based on the idea of translating the essence of sports and sharing it with the visually impaired. Although there are people with many types of physical conditions, changing the modalities of sensation might enable many people to enjoy sports viewing together. In the near future, we would also like to apply this method to actual situations such as in stadiums and public viewing spaces.

References

[1]	D. Oshimi and M. Harada, “Heart-capturing (Kandoh) Scene in Sports,” Japanese Journal of Sport Management, Vol. 2, No. 2, pp. 163–178, 2010.
[2]	A. Tokaji, “Mechanisms for Evoking Emotional Responses of ‘Kandoh (the State of Being Emotionally Moved)’,” Cognitive Studies, Vol. 8, No. 4, pp. 360–368, 2001.
[3]	Invisible Sports Picture Book (in Japanese), https://mienaisports.com/

	Akiko Hayashi Researcher, NTT Service Evolution Laboratories. She received a master’s degree from Graduate School of Frontier Biosciences, Osaka University, and became a researcher at NTT Communication Science Laboratories in 2009. Her research is focused on user experience design and cognitive science.
	Junji Watanabe Senior Distinguished Researcher, NTT Communication Science Laboratories; NTT Service Evolution Laboratories. He received a Ph.D. in information science and technology from the University of Tokyo in 2005. From 2005 to 2009, he was a PRESTO researcher in the project called Foundation of Technology Supporting the Creation of Digital Media Content at the Japan Science & Technology Agency (JST). He became a researcher at NTT Communication Science Laboratories in 2013 and concurrently at NTT Service Evolution Laboratories in 2016. His research is focused on cognitive science and communication devices with applied human perception. His interests are visual and haptic perception and communications.
	Kentaro Shimizu Senior Research Engineer, Supervisor, NTT Service Evolution Laboratories. He became a researcher at NTT in 1999 and received a Ph.D. in medical science from the Tokyo Women’s Medical University in 2009. He studies medical image processing and artificial intelligence, and his research interests are image processing and communications.

↑ TOP