Feature Articles: Olympic and Paralympic Games Tokyo 2020 and NTT R&D—Technologies for Viewing Tokyo 2020 Games
Marathon × Ultra-low-latency Communication Technology
NTT provided ultra-low-latency communication technology to the Tokyo 2020 Real-time Remote-cheering Project, which was implemented by the Tokyo Organising Committee of the Olympic and Paralympic Games for the marathons of the Olympic and Paralympic Games Tokyo 2020 held in Sapporo on August 7 and 8, 2021. This project proposed a new means of watching sports by connecting the marathon course in Sapporo with the cheering venue in Tokyo in real time. By delivering the cheers of spectators to the athletes from the remote location in Tokyo, it was possible to create a sense of unity between the athletes and spectators, as well as a sense of realism similar to roadside cheering while maintaining safety and security. Our efforts concerning this project and the technologies that we used are introduced in this article.
Keywords: ultra-low-latency communication technology, remote cheering, IOWN
1. Overview of the project
During the novel coronavirus (COVID-19) pandemic, it has become increasingly difficult to cheer on athletes directly at competition venues. By using new communication technologies, the Tokyo 2020 Real-time Remote-cheering Project aimed to create a world in which athletes at a competition venue and spectators in a remote area can share the excitement, emotion, and sense of unity at the venue (Fig. 1).
As a Gold Partner of the Olympic and Paralympic Games Tokyo 2020, NTT has been contributing to the creation of the most-innovative Games in history by providing cutting-edge communication technology. In light of the situation concerning the COVID-19 pandemic, we adopted a new mission to propose a new form of sports viewing using communication services and make it a springboard for the future and decided to provide the technology for this project.
Regarding remote spectator experience, delay time is the largest challenge in transmitting cheering from remote venues to the competition venue. For marathons in particular, even a small delay can have a significant effect on the ability to transmit cheering to athletes running at 5 m/s. The total delay in watching a sporting event remotely had been several seconds each way, which includes not only the propagation delay of light but also media-processing delay such as transmission-processing delay and compression delay for video information; as a result, it has been impossible to transmit cheering to players, athletes, etc. without a delay. NTT undertook the Tokyo 2020 Real-time Remote-cheering Project with the goal of solving these problems by using its ultra-low-latency communication technology to minimize the latency of the transmission process to about 100 ms each way and ensure the transmission of cheering to the athletes.
2. Device configuration of the project
The overall device configuration of the project is shown in Fig. 2. We aimed to create a space in which remote spectators could feel as if they were cheering along the roadside as the runners were running by. Therefore, we installed light-emitting diode (LED) displays (approximately 50 m wide and 2 m high) at both the Sapporo marathon course (in front of Sapporo Sosei Square) and Tokyo remote-viewing venue. The displays enabled the runners and spectators to see each other in actual size. The total width of the LED displays at both sites was approximately 50 m, but the location and spacing of the displays varied in accordance with the conditions at the installation site (Figs. 3(a) and (b)). Eight cameras were set up in front of the LED displays at both sites to capture images of spectators and runners (Fig. 3(c)). The 4K images from each of the two sites were transmitted to the other site via a 20-Gbit/s high-speed broadband line via a media-processing device using ultra-low-latency media-processing technology and transmission device using disaggregated configuration for video transport. These two technologies constitute our ultra-low-latency communication technology.
3. Ultra-low-latency communication technology
The Innovative Optical and Wireless Network (IOWN) is an innovative concept proposed by NTT as a next-generation communication platform. IOWN will enable the creation of a network and information-processing infrastructure that includes devices capable of providing high-speed, high-capacity communications and vast computing resources by using innovative technologies centered on optical technology. One of the components of IOWN, the All-Photonics Network (APN), will achieve ultra-low-power consumption, ultra-high capacity, and ultra-low latency by introducing photonics-based technology into everything from the network to devices. The ultra-low-latency communication technology used in this project uses two APN elemental technologies: disaggregated configuration for video transport and ultra-low-latency media-processing technology (Fig. 4).
3.1 Disaggregated configuration for video transport
Disaggregated configuration for video transport enables flexible changes in configuration, addition of functions, and cost reduction by separating network functions that were previously provided in combination and configuring them so that each function can be controlled by standardized interfaces. We developed a plug-in unit (video PIU) in which a direct serial digital interface (SDI)-signal acquisition function is added to a white-box switch with an optical transponder for long-distance transmission. The new video PIU enables direct transmission of uncompressed video and audio in SMPTE ST 2110* format over optical long-distance transmission lines. As a result, the delay between the video input on the transmitter side and video output on the receiver side was reduced to about 1 ms, and the one-way delay (including the distance delay between Tokyo and Sapporo) was reduced to about 20 ms. We also implemented a set of functions for video routing in the network operating system, which is the control software for the white-box switch, to achieve integrated operation of network and video devices (Fig. 5).
3.2 Ultra-low-latency media-processing technology
Ultra-low-latency media-processing technology captures SDI video signals output from a camera on a subframe or line basis and executes video processing such as geometric transformation and composition at the video-signal level. Implementing each video process at the video-signal level instead of at the frame level makes it possible to reduce the frame latency required by conventional video processing and achieve low latency (Fig. 6).
In this project, when transmitting video from multiple cameras, the SDI video signals from each camera were made into subframes in line units, and the received subframes were immediately separated and displayed on multiple displays in a manner that achieved even lower latency (Fig. 7).
4. Results of the project
On the days of the men’s and women’s marathons, after a demonstration of low latency was conducted, the remote spectators cheered for the runners. Via the 50-m-wide display, the Tokyo venue provided a sense of realism to the spectators, namely, the feeling that the athletes were running right in front of them, and the spectators could show their support by clapping and waving flags. For past remote-cheering experiences, due to delays, the timing of the cheering was not synchronized with the athletes, so the spectators found it difficult to feel a sense of unity while cheering for the athletes. For the Sapporo marathon course, however, the ultra-low-latency communication technology enabled the spectators to see their cheering reaching the runners as the runners passed by. As a result, the spectators were able to feel the same sense of unity even in the Tokyo venue, which made them even more enthusiastic about cheering (Figs. 8(a) and (b)).
Through this cheering experience, the spectators at the Tokyo venue commented, “I could experience the same sense of realism and speed as cheering at the roadside,” “I felt that I could cheer from a position closer to the athletes than possible at the roadside, which is usually crowded,” and “Above all, it was lovely to see the cheering reach the athletes.” In other words, the spectators seemed to be very satisfied with the sense of unity by having their cheering reach the marathon venue, an outcome that had been impossible with the remote cheering. The significance of this project was also confirmed by the fact that some of the marathon runners showed interest in the real-time cheering from the remote area via the display, and that result was a world’s first. As expected, the delay time for the transmission process was kept to 100 ms each way. Unfortunately, typhoon Mirinae was approaching Japan, and the men’s marathon was held in rainy weather on August 8th; nevertheless, after ensuring safety of the athletes and spectators, we were able to transmit cheering from Tokyo to Sapporo in spite of the rain (Figs. 8(c) and (d)).
5. Future developments
In light of the results of this project, NTT aims to create a new style of watching sports competitions that allows people who are unable to cheer at a competition venue to experience the sense of presence and unity, which could only be experienced at the venue, in a safe and secure manner. Regarding the Tokyo 2020 Real-time Remote-cheering Project, the cheering was transmitted from a special venue in Tokyo. We will accelerate our research and development so that our ultra-low-latency communication technology can be widely used and the same real-time remote cheering will be possible from homes.
NTT is an Olympic and Paralympic Games Tokyo 2020 Gold Partner (Telecommunication Services).