You need Adobe Reader 7.0 or later in order to read PDF files on this site.
If Adobe Reader is not installed on your computer, click the button below and go to the download site.

Regular Articles

Vol. 23, No. 2, pp. 19–25, Feb. 2025. https://doi.org/10.53829/ntr202502ra1

Field Verification of Automated Driving Using Cooperative Infrastructure Platform

Taichi Kawano, Nobuhiro Azuma, Takehiro Fujinaga,
Takuya Tojo, Takeshi Kuwahara, Motoharu Sasaki,
Mitsuki Nakamura, Kenichi Kawamura, Shinpei Yasukawa, Mitsuru Toda, and Haruki Nishikawa

Abstract

This article introduces a field verification of automated driving using the Cooperative Infrastructure Platform being developed by NTT laboratories. The Cooperative Infrastructure Platform aims to ensure stable communication in remote monitoring and improve the safety of automated driving. This platform is also being used as the communication function of the remote control system for automated driving developed by NTT DOCOMO, and field tests are being conducted in collaboration with NTT DOCOMO.

Keywords: IOWN, cyber-physical systems, remote monitoring of automated driving

1. Issues with wireless communications in remote control systems for automated driving

In April 2023, the Road Traffic Law was amended to lift the ban on Level 4 automated driving (fully automated driving that does not require a driver under specific conditions) in Japan, which allows driverless automated driving on public roads. Automated driving is expected to bring various benefits to society, including resolving the driver shortage, ensuring transportation for the elderly and disabled, reducing traffic accidents, and improving the efficiency of logistics.

To safely operate Level 4 automated driving, however, several conditions are mandatory. One of the most important of these is remote monitoring via video by an operator. This is an essential element for enabling operators to monitor the vehicle’s operating status in real time and respond to any problems as necessary. Video monitoring requires data transmission via mobile communications, and the major challenge is to achieve high-capacity communications with a high level of reliability.

One of the technical issues that needs to be addressed is that the communication environment is constantly changing due to the vehicle’s movement. The quality of communication is affected by factors, such as the attenuation of radio waves by obstructions such as high-rise buildings and overhead structures, and congestion in the network caused by an increase in the number of users in the area. As the effects of these factors fluctuate due to the movement of vehicles, the quality of communication also fluctuates. When communication becomes unstable, there is a risk that the video may be interrupted or delayed; thus, the operator may not be able to respond to the vehicle at the appropriate time.

To address these issues, NTT laboratories are developing the Cooperative Infrastructure Platform to achieve stable mobile communications. This platform aims to ensure stable data transmission by linking terminals, networks, and the cloud and enable flexible responses to changes in the communication environment. This will enable operators to grasp the status of vehicles in real time and support safe and efficient automated driving.

2. Cooperative Infrastructure Platform

The Cooperative Infrastructure Platform is a fundamental technology aimed at accommodating mission-critical cyber-physical system (CPS) services [1]. In this article, we introduce the elemental technologies that contribute to the high reliability of communication between vehicles and the cloud, which is particularly required for remote monitoring of automated driving.

CPS services require extremely high reliability and real-time performance because any delay in control could have fatal consequences. However, when the surrounding environment is constantly changing, such as when driving a vehicle, it is difficult to maintain high reliability and real-time performance because the quality of communication fluctuates greatly due to various external factors.

One possible solution is to measure the communication quality on the vehicle side and switch the communication line when quality degradation is detected, but if quality degradation occurs suddenly, it may not be possible to detect it in time and switch the line, and communication may be cut off.

In contrast, the Cooperative Infrastructure Platform continuously predicts communication quality and avoids using lines where quality degradation is predicted in advance, achieving highly reliable communication even in environments where sudden quality degradation occurs.

As shown in Fig. 1, the Cooperative Infrastructure Platform is composed of multiple elemental technologies deployed on the vehicle and cloud sides, i.e., the (1) Quality Index Map (QIM) and (2) Cradio^® (multi-radio proactive control technology), both of which predict communication quality; (3) Multi-path QUIC gateway (MPQUIC GW), which allocates multiple lines on a packet-by-packet basis based on the predicted values, and the end-to-end overlay network (E2E overlay NW), which allocates multiple lines on a flow-by-flow basis; (4) location information collection and prediction function and (5) wireless quality collection function, both of which collect the information necessary for prediction; and (6) cooperative control GW, which is responsible for the distribution of control information. These technologies work in close coordination.

Fig. 1. Architecture of Cooperative Infrastructure Platform.

The QIM is a technology that learns communication quality on the basis of actual service communication (such as monitoring video) on a specific date and time and area basis then uses the learning results to estimate communication quality. The QIM achieves high-precision quality estimation that incorporates the latest changes in the surrounding environment without the need for probe packets for learning and without putting pressure on the available bandwidth.

Cradio^® is a group of wireless technologies for providing a natural communication environment that does not make users aware of the type of wireless network used by adapting to the ever-changing user requirements and radio wave conditions and is composed of three groups of technologies: understanding, prediction, and control. For the Cooperative Infrastructure Platform, we use Cradio^® prediction technology to predict the quality of wireless communication in a future driving location using information collected from various wireless communication devices and machine learning.

The MPQUIC GW is a device that transparently transfers packets using the stream and datagram frames of a multipath extension to the QUIC protocol [2], the latest technology currently being standardized, and provides various network control functions for highly reliable, real-time packet transmission, including packet-based line allocation and aggregation based on predicted and actual communication quality values.

The Cooperative Infrastructure Platform thus achieves highly reliable, real-time communication even in environments where rapid quality degradation occurs through advanced coordination such as high-precision quality prediction and advanced network control based on quality prediction.

3. Application of the Cooperative Infrastructure Platform to remote control systems

A remote control system for automated driving is a system that ensures the safe and efficient operation of automated driving vehicles even when the driver is not present. This system monitors the status of the vehicle in real time and responds remotely as necessary, enabling rapid response even in the event of an abnormality.

3.1 Remote control operations

Remote control operations are carried out for crew duties other than vehicle operation. There are two main aspects to these operations.

1) Passenger safety and support: It is important to ensure passenger safety and provide support as necessary. A remote control system is used to respond to accidents, deal with suspicious persons on board, check seating, and respond to passenger inquiries. This ensures that the safety of the vehicle is maintained even when the driver is not present, and that passengers feel secure.

2) Operation management: Operation management tasks, such as vehicle allocation, punctuality management, and fare checking, are also carried out remotely. These tasks are managed appropriately to ensure that the automated vehicles run smoothly, and support is provided to prevent delays in operation.

In some cases, remote assistance (SAE J3016) is used to provide support to the automated driving system from an operator. At construction sites, for example, the operator provides “guidance” on the driving path, etc., on the basis of requests from the automated driving system.

While the automated driving system installed in the autonomous vehicle is responsible for the dynamic driving operations of accelerating, stopping, and turning, remote monitoring enables safe operation of driverless autonomous vehicles.

3.2 Remote control system

NTT DOCOMO is developing a remote control system that supports Level 4 automated driving (Fig. 2). This system is made up of an autonomous vehicle, control server on multi-access edge computing (MEC) and the cloud, and a control client at a remote control center. It also has various processing functions on the remote control server to support remote control operations such as transmitting video, audio, and telemetry between the vehicle and remote control room (Fig. 3).

1) Autonomous vehicle: The vehicle is equipped with cameras, microphones, and sensors, and the data from them are sent to the control server in real time. The vehicle is also equipped with edge artificial intelligence (AI) functions for detecting abnormalities, and if an abnormality is detected, the client at the remote control center is immediately notified via the control server. This system enables a quick response in the event of an abnormality. The vehicle runs autonomously, but if it requires assistance, it can also receive instructions remotely.

2) Control server: The control server relays and processes the video, audio, and telemetry data sent from the autonomous vehicle. It has functions for storing and analyzing telemetry data and for visualizing the data, so it can determine the situation in real time. This enables the vehicle to be operated while automatically checking the safety of the interior, and in the event of an abnormality, prompt analysis and alert transmission can be carried out, enabling a quick response.

3) Control client: The operators at the remote control center constantly monitor the status of the vehicles via the control client. Based on the video and audio received in real time, if an abnormality or emergency situation occurs, it is possible to remotely stop the vehicle or communicate with the passengers via audio. This makes it possible to operate safely even when not on site.

Fig. 2. NTT DOCOMO’s remote control system.

Fig. 3. Application of Cooperative Infrastructure Platform to remote control.

3.3 Communication requirements for remote control systems

Reliability and real-time performance are extremely important for communication requirements for remote control systems. The following requirements must be met.

1) Real-time performance: Video, audio, and telemetry data must be transmitted in real time without delay. For example, vehicle location information and status data must be updated almost immediately, and a quick response is required, particularly in emergency situations.

2) Reliability: Video, audio, and telemetry data must be transmitted stably without interruption. There must be no loss or delay of data, and a system that immediately notifies the user of any abnormalities is required. A system that can quickly restore communication in the event of a disruption is also necessary.

3.4 Application of the Cooperative Infrastructure Platform

NTT DOCOMO’s remote control system for automated driving experimentally uses the Cooperative Infrastructure Platform for communication between vehicles and the control server to meet communication requirements (Fig. 3). Using this platform ensures stable communication between vehicles and the control server without interruption. This enables real-time data transmission and ensures the safe and efficient operation of automated vehicles through an automated driving remote control system with stable transmission of vehicle status, location information, video, and audio data.

4. Automated driving field verification

The remote control system using the Cooperative Infrastructure Platform was used in the automated driving field verification in Hiratsuka City, Kanagawa Prefecture, from January 22 to February 2, 2024. The verification test was conducted with the aim of achieving Level 4 automated driving, and NTT DOCOMO provided the remote control system. Cameras were installed inside the bus, and the images captured with these cameras were sent to a remote monitoring room via the Cooperative Infrastructure Platform. By monitoring the continuous and stable images, the remote monitoring room can check the status of the vehicle and ensure safety even when there is no driver. The vehicle used in the verification test and the equipment installed in the vehicle are shown in Fig. 4.

Fig. 4. Autonomous driving bus.

Without the Cooperative Infrastructure Platform, there may be long periods of disruption depending on the location, time of day, and other circumstances. In the remote control system with the Cooperative Infrastructure Platform, we confirmed that stable video can be transmitted without long periods of disruption by proactively and reactively selecting lines with good network quality.

We plan to develop the technologies of the Cooperative Infrastructure Platform based on the experience and issues identified in the field verification.

5. Future developments

This article introduced a field verification of automated driving using the Cooperative Infrastructure Platform being developed by NTT laboratories. The platform flexibly responds to fluctuations in communication quality and achieves stable data transmission in remote monitoring of automated vehicles. In collaboration with NTT DOCOMO, we used this platform in a field test in February 2024 and confirmed that the platform improved the safety and reliability of automated vehicles.

We will continue to conduct further verification tests and improve the platform to achieve remote control services for Level 4 automated driving. We will also aim to develop the platform for other use cases that require highly reliable wireless communication, such as drones and smart factories, as well as develop new elemental technologies.

References

[1]	N. Azuma, K. Ono, T. Tsubaki, T. Kawano, T. Tojo, and T. Kuwahara, “Cooperative Infrastructure Platform to Accommodate Mission-critical CPS Services,” NTT Technical Review, Vol. 22, No. 12, pp. 40–46, Dec. 2024. https://doi.org/10.53829/ntr202412fa4
[2]	Y. Liu, Y. Ma, Q. De Coninck, O. Bonaventure, C. Huitema, and M. Kuehlewind, “Multipath Extension for QUIC,” IETF Internet-Draft: draft-ietf-quic-multipath-04, Mar. 2023.

	Taichi Kawano Senior Research Engineer. NTT Network Service Systems Laboratories. He received a B.E. and M.E. in engineering from the University of Tsukuba, Ibaraki, in 2006 and 2008. He joined NTT laboratories in 2008, where he engaged in research on quality of experience (QoE) for video services. In 2011, he was awarded the Young Investigators’ Award by the Institute of Electronics, Information and Communication Engineers (IEICE). In 2015, he moved to NTT Communications, where he focused on technology validation for software-defined networking (SDN) and network function virtualization (NFV). He returned to NTT laboratories in 2018, where he has been involved in research and development (R&D) related to the visualization, analysis, and control of network services based on quality and QoE. He is currently managing research on enhancing the reliability of mobile communications for the remote control of autonomous driving systems.
	Nobuhiro Azuma Senior Research Engineer, NTT Network Service Systems Laboratories. He received a B.E. and M.E. in engineering from Niigata University in 2009 and 2011. In 2011, he joined NTT Network Service Systems Laboratories, where he studied the design of fixed-mobile converged architecture and core technologies to accommodate machine-to machine/Internet-of-Things services. He was with NTT DOCOMO from 2016 to 2020, where he was engaged in the development of mobile application service systems. He is currently engaged in R&D of a network service platform to accommodate mission-critical services.
	Takehiro Fujinaga Senior Research Engineer. NTT Network Service Systems Laboratories. He received an M.E. in informatics from Kyoto University in 2006. He joined NTT WEST in 2006, where he was engaged in the development and network design of the Next Generation Network (NGN). In 2020, he moved to NTT laboratories, where he is currently engaged in R&D of a network service platform for mission-critical services such as autonomous driving.
	Takuya Tojo Senior Research Engineer, Group Leader, Network Service Platform Research Group, Core Network Technology Research Project, NTT Network Service Systems Laboratories. He received a B.E., M.E., and Ph.D. from Tokyo Denki University in 2002, 2004, and 2007. In 2004, he joined NTT Service Integration Laboratories, where he was engaged in research on network architecture, quality of service, and transport protocols for the NGN. He also worked on designing the Internet protocol (IP) backbone network at NTT DOCOMO. Since 2015, he has been engaged in IP and optical network architecture, multilayer SDN control, and network slicing. He is currently engaged in R&D of a network service platform for mission-critical services such as autonomous driving.
	Takeshi Kuwahara Executive Research Engineer, Head of Core Network Technology Research Project, NTT Network Service Systems Laboratories. He received a B.E. and M.E. from Waseda University, Tokyo, in 1995 and 1997. Since joining NTT in 1997, he has been engaged in the R&D of asynchronous transfer mode networks, IP virtual private networks, cloud systems, network security, and edge computing technologies. He was in charge of telecom carrier relations regarding network R&D while based in Washington, D.C., USA and led the establishment of ATII (APAC Telecom Innovation Initiative) while in his former position. He has been working on establishing various technologies for networking and computing in IOWN (Innovative Optical and Wireless Network).
	Motoharu Sasaki Distinguished Researcher, Wireless Access Systems Project, NTT Access Network Service Systems Laboratories. He received a B.E. in engineering and M.E. and Ph.D. in information science and electrical engineering from Kyushu University, Fukuoka, in 2007, 2009, and 2015. In 2009, he joined NTT Access Network Service Systems Laboratories, where he has been engaged in research on propagation modeling and proactive prediction methods for various wireless communication systems.
	Mitsuki Nakamura Research Engineer, Wireless Access Systems Project, NTT Access Network Service Systems Laboratories. He received a B.E. and M.E. from Keio University, Tokyo, in 2012 and 2014. He joined NTT Access Network Service Systems Laboratories in 2014. From 2020 to 2023, He was a research engineer at NTT DOCOMO. He has been engaged in the research of radio propagation characteristics and predictive quality of service for wireless access systems.
	Kenichi Kawamura Senior Research Engineer, Group Leader, Wireless Access Systems Project, NTT Access Network Service Systems Laboratories. He received a B.E. in electrical engineering and M.E. in informatics from Kyoto University in 1999 and 2001. Since joining NTT in 2001, he has been engaged in the R&D of wireless LAN systems, mobile routers, and network architecture for wireless access systems. He is currently engaged in R&D of quality control technologies for wireless access systems.
	Shinpei Yasukawa Assistant Manager, Mobility Technology Development, X-Tech Development Department, NTT DOCOMO, INC. He received an M.E. from Doshisha University, Kyoto, in 2009. He joined NTT DOCOMO in 2009 and has been engaged in R&D of uplink multiple-input multiple-output in LTE (Long-Term Evolution). Since 2013, he has been engaged in 3GPP standardization including the areas of machine-type communications and vehicle-to-everything communication. He is currently working on autonomous driving and its communication.
	Mitsuru Toda Service and Product Developer, Mobility Tech Group, X-Tech Development Department, NTT DOCOMO, INC. He received an M.E. from the Graduate School of Science and Technology, Tokyo University of Science, in 2021. He joined NTT DOCOMO in 2021, where he engaged in project management and requirements definition tasks. Since 2024, he has been working on studying various communication methods for remote control systems in autonomous driving.
	Haruki Nishikawa Service and Product Developer, Mobility Tech Group, X-Tech Development Department, NTT DOCOMO, INC. He received a B.Sc. in computer science from University of Canterbury in 2008 and an M.S. in information technology from the Kyoto College of Graduate Studies for Informatics in 2010. In 2023, he joined NTT DOCOMO, where he has been working on developing remote control systems for autonomous driving.

↑ TOP