Feature Articles: Digital Transformation of Urban Areas Focusing on Mobility

Vol. 19, No. 1, pp. 30–37, Jan. 2021. https://doi.org/10.53829/ntr202101fa5

Technical Developments and Verification of Connected Cars

Atsushi Isomura, Kazuya Matsuo, Yuki Yokohata,
Yoshikazu Hirabayashi, and Atsushi Koizumi

Abstract

Toyota Motor Corporation and NTT have agreed to collaborate on the technical development and verification of connected cars by combining Toyota’s automobile-related technology and NTT’s information and communication technology (ICT)-related technology. They have begun a demonstration experiment to check the current degree of completion of the required technical development. This article presents an overview of the ICT platform for connected cars and describes technical issues, technologies used to address these issues, and the demonstration experiment.

Keywords: connected cars, IoT, big data

1. Development of the ICT platform for connected cars

After the collaboration agreement was concluded [1], Toyota Motor Corporation and the NTT Group have been developing an information and communication technology (ICT) platform for connected cars by taking future automated driving into consideration. To achieve safe and secure automated driving, various data sent from vehicles must be collected and used in a cloud. The ICT platform is being developed as a means to achieve this.

2. ICT platform for connected cars

A connected car is a car that also functions as an ICT terminal. Since a vehicle holds many items of information, it is possible to understand the vehicle and its surroundings from the data it sends. If we are to solve social problems, such as traffic accidents and traffic jams, and upgrade mobility services, it is essential to use data collected from connected cars via a network.

Many key technologies are required to support connected cars: technology to collect and store a large volume of vehicle data sent by numerous vehicles; technologies to analyze the collected large volume of data in real time and reproduce the surroundings on a cloud; artificial intelligence (AI) algorithms to determine the behavior of vehicles; technology to collect and distribute a large volume of data; 5G (fifth-generation mobile communication system); edge computing; and communication technology to collect and distribute a large volume of data reliably and securely using Internet of Things (IoT) networks.

Specifically, a large volume of data from electronic control units (ECUs) installed in vehicles on roads and sensor data, including images from cameras, need be collected and stored in a cloud reliably and efficiently. From the collected data, the status of vehicles and their surroundings need be reproduced in cyberspace in real time. These processes make it possible to generate maps to be used in next-generation traffic systems and allow various services and automated driving. The ICT platform for connected cars will support the reproduction of the status of vehicles and their surroundings in cyberspace (Fig. 1).

Fig. 1. ICT platform for connected cars.

3. Demonstration experiment

The development of the ICT platform for connected cars is being undertaken through short cycles of studying individual key technologies and a demonstration experiment in which these technologies are combined and verified on an end-to-end connection, including terminals, a network, and datacenter. In the demonstration experiment, data from multiple experiment vehicles running on roads are collected via a network and stored in a cloud. The collected data are analyzed to confirm whether technical issues identified for use cases of connected cars and automated driving have been solved. In addition, vehicle data equivalent to millions of vehicles are generated to evaluate the feasibility of communication and datacenter technologies.

Specific uses cases being studied are: (1) generation of a high-precision map, (2) detection and notification of obstacles, and (3) traffic-jam detection in each lane. The extent to which the technical requirements of each use case are satisfied is being examined.

(1) Generation of a high-precision map

Whether video data captured with vehicle-mounted cameras can be sent to the cloud and whether information about vehicle positions and about the surroundings can be reproduced in cyberspace are being examined (Fig. 2). In particular, we are developing a technology for collecting video data and location information from vehicles and estimating the positions of road signs, road marks, and other structures from the collected video data captured with vehicle-mounted monocular cameras with a high degree of precision. Currently, changes in road surfaces are examined by driving a dedicated measurement vehicle over the road to be examined. This method is costly and cannot produce examination data in real time. Instead, our technology can be used to produce a high-precision map from image data captured with an ordinary vehicle-mounted camera.

Fig. 2. Generation of a high-precision map.

(2) Detection and notification of obstacles

We are examining whether data sent by a vehicle can be shared with other vehicles in real time (Fig. 3). Specifically, when a vehicle-mounted monocular camera captures image data of an obstacle, the platform receives its data from the vehicle and estimates the position of the obstacle. The information about the obstacle is registered on a dynamic map. The platform quickly searches for vehicles that can be affected by the obstacle. Finally, the information about the obstacle is transmitted to these vehicles. We are developing a technology that accelerates the processing of these steps in less than 10 seconds.

Fig. 3. Detection and notification of obstacles.

(3) Traffic-jam detection in each lane

We are verifying whether the level of congestion in neighboring lanes can be reproduced from video data captured with a vehicle-mounted monocular camera. In particular, we are developing a technology for identifying the lane in which the vehicle runs and accurately estimating the level of congestion in neighboring lanes from the video data.

Technical targets have been defined for the key technologies being used in the demonstration experiment by assuming several use cases. The targets are the ability to connect tens of millions of vehicles to the system and perform the above steps within just a few seconds. This demonstration experiment began in 2018. It has been gradually expanded and will continue until the end of fiscal year 2020. We have already confirmed that 5 million vehicles can be connected to the system and that all the required steps can be performed in less than 20 seconds.

4. Main technologies that support the ICT platform

The ICT platform for connected cars involves a number of technical challenges. The main technologies being developed to meet these challenges are described below (Fig. 4).

Fig. 4. Main technologies supporting the ICT platform.

(1) Object-position-estimation technology

This technology estimates the positions of display objects (road signs, signboards, etc.) in the road surroundings from vehicle status data and video data captured with vehicle-mounted cameras. It can even process data from low-end dashcams. It improves positional accuracy by selecting high-quality data from a large volume of data and using statistical methods. Accurate recognition of three-dimensional objects, such as road signs and traffic lights, on roads and high-precision estimation of the positions of these objects make it possible to map the objects on a map. This technology is expected to be used for generating maps for automated driving and automatically updating them.

(2) Algorithm for selective collection of vehicle data

Data collected from connected cars are both voluminous and varied, ranging from vehicle data to video and sensor data. Since these data are collected consecutively from a large number of vehicles, the volume of collected data is enormous. This imposes a huge load on the network, and data processing capacity is stretched to the limit. To solve these problems, we developed a technology for selecting data collected from vehicles based on meta-information, such as location information, time information, sensor type, resolution, observational range, and observation impairment caused by the presence of surrounding vehicles. This technology makes it possible to efficiently collect data with minimum duplication of observational ranges and adjust the volume of collected data in accordance with the load on communication links and the analysis platform.

(3) Real-time spatio-temporal data-management technology

This technology enables a large volume of data to be stored instantaneously and searches a specified range (time and space) of data at high speed. It is a spatio-temporal database that manages time and location information generated by moving objects, such as vehicles and smartphones, as one-dimensional data using a unique method and enables the searching for those moving objects within a specified time and rectangular space range. It also searches within a space with a complex shape, such as a road or parking lot. Therefore, it enables the storing of sensor information generated by a large number of moving objects in real space, and time and space information associated with each item of sensor information, as well as searching for this information within a specified time and space range.

(4) Vertically distributed computing technology

This technology processes applications at high speed by offloading computation tasks done at datacenter servers to edge servers. Since the processing capacity of the system server is limited, it is sometimes not possible for the system to respond to all devices instantly. In reality, the required response time varies depending on the status of each vehicle (vehicle speed, moving direction, time of day, etc.). For this reason, vertically distributed computing technology dynamically distributes applications between edge servers and datacenter servers based on the status of vehicles. This technology can use limited server resources effectively while ensuring that the system responds instantly to those vehicles that require a quick response. This technology enables the system to serve a large number of vehicles and to collect and respond to a large volume of data. Thus, limited distributed computing resources can be used efficiently.

(5) Traffic-jam detection in each lane

This technology detects traffic jams in each lane from vehicle-status data and video data captured with vehicle-mounted cameras. It detects a queue of vehicles from the relative locations of vehicles moving along the surrounding lanes. It can also recognize the cause of a queue by identifying the first and last vehicles and combining this information with the surrounding facility information. Currently, information about traffic jams by road is provided by the Vehicle Information and Communication System (VICS) and smartphone map apps. When the above technology has been developed and in wide use, more valuable traffic-jam information will become available. As with the object-position-estimation technology described above, this technology can detect a traffic jam using video data captured with even low-end dashcams.

5. Conclusion

It is said that the automobile industry is facing a once-in-a-hundred-year transformation. The environment surrounding automobiles has been changing dramatically over the last few years. One of these changes is the spread of connected cars, resulting in the emergence of various services for drivers, fellow travelers, automakers, insurance companies, and service garages.

When it will become possible to process a large volume of data on vehicles quickly and at low cost, information that cannot be captured using current sensors will become available, and faster and more reliable services than today will be provided. These developments will not only enhance convenience and efficiency but also provide safety and security to the coming age of automated driving.

Reference

[1]	Press release issued by NTT on Mar. 27, 2017 (in Japanese), https://www.ntt.co.jp/news2017/1703/170327a.html

	Atsushi Isomura Researcher, IoT Framework SE Project, NTT Software Innovation Center. He received a B.S. and M.S. in information science and technology from Aichi Prefectural University in 2014 and 2016. He joined NTT Software Innovation Center in 2016 and is working on Axispot^TM, a distributed spatio-temporal database management technology. In 2019, he participated in Redisconf19 as a presenter and introduced the core technologies of Axispot in San Francisco. He is a member of the Institute of Electronics, Information and Communication Engineers (IEICE) of Japan. He received the IEICE Tokai Section Student Award and the best poster award in 12th International Conference on Ubiquitous Healthcare in 2015.
	Kazuya Matsuo Researcher, NTT Media Intelligence Laboratories, NTT Digital Twin Computing Research Center. He received a Ph.D. in information science from Osaka University in 2017, and joined NTT in the same year. While at university (2012–2017), he studied the collection of data in mobile sensor networks. He studied modeling of vertically distributed applications in NTT until 2019. From 2018, he has been involved in collaborative work between Toyota and NTT. Since 2020, he has also been working on a Digital Twin Computing project. He received a Young Researcher Award from IEICE in 2018. He is a member of the Information Processing Society of Japan.
	Yuki Yokohata Senior Researcher, NTT Service Evolution Laboratories, NTT Smart Data Science Center. She received an M.S. in earth and planetary science from Nagoya University in 2004 and joined NTT in the same year. She studied ubiquitous services in NTT Network Service Systems Laboratories. From 2008 to 2015, she was involved in the development of public wireless local area network services at NTT Communications. Since 2016, she has been active in the development of smart city applications and analysis of data collected from vehicles for driving support in NTT Network Innovation Laboratories.
	Yoshikazu Hirabayashi Director, R&D Produce Group, NTT Research and Development Planning Department. He received a B.E. from Doshisha University, Kyoto, in 1997. After conducting telecommunication infrastructure research and development at NTT WEST, he was transferred to NTT, where he has been engaged in the research and development of connected cars.
	Atsushi Koizumi Senior Manager, R&D Produce Group, NTT Research and Development Planning Department. He received a B.A. in commerce from Waseda University, Tokyo, in 1995. After working on system development in the public sector at NTT DATA, he was seconded to the NTT DATA Institute of Management Consulting Inc. After returning to NTT DATA, he worked on sales reform and marketing in the Corporate Planning Department then engaged in business planning in the organization dedicated to business process outsourcing and robotic process automation. Currently, he is at NTT and engaged in the research and development of connected cars.

Young researcher’s view

Research and Development of Digital Twin Computing Technology to Support Future Transport Systems

Masaru Takagi
Researcher, Unit-2, NTT Digital Twin Computing Research Center

My research team is developing technology for building a digital twin city using data collected from connected cars. Collection of LiDAR (light detection and ranging) and video data from connected cars allows continuous scanning of an entire city. A digital twin of a city can be built in virtual space by analyzing these data. Such a digital twin can be used for many purposes. In transport, it can be used to create a road-surface image map for autonomous driving cars and enable vehicles to share information about obstacles on the road. It can also be used in other applications such as detecting queues at restaurants located along a road, checking the damage after a disaster, and detecting anomalies in utility poles and cables.

The challenge is the sheer amount of sensing data. A single car equipped with multiple cameras and LiDAR sensors generates 10 MB of data every second. If 10 million cars are running throughout the country, the total amount of data generated per second reaches 100 TB. This is a huge amount, 10 times the amount of data carried by communications networks in Japan. It is extremely difficult to collect all such data due to the capacity limits of communication networks, computing resources, and storage systems. This situation can be mitigated because the urgency of collecting sensing data varies. Some need to be collected immediately for analysis, while other data can be collected after a delay. Some need to be collected only once a day, and some have no utility value whatsoever.

Therefore, we developed a technology that prioritizes data collection based on meta-information, such as location information, time information, sensor type, resolution, observational range, and observation impairment caused by the presence of surrounding vehicles. Although the amount of meta-information is huge, the high-speed spatiotemporal data management technology Axispot^TM makes it possible to search meta-information at a practical processing speed. Our technology makes it possible to collect data efficiently with minimum duplication of observational ranges and adjust the volume of data to collect in accordance with the load on communication links and analysis platform.

In a demonstration experiment conducted in FY2019, this technology was implemented on an analysis platform, and vehicles running on public roads were connected to the platform. The function to selectively collect data based on the priority of the data was verified. In a demonstration experiment currently being conducted (FY2020), we are examining a function to dynamically switch the servers executing analysis on the basis of data priorities and processing load.

We are also making an ambitious effort to develop technology capable of predicting impending traffic status, to within just a few minutes to several hours from the present, with a high degree of precision. This technology is based on a multi-agent, microscopic traffic-flow simulator, which can reproduce the movement of each vehicle. We set the initial conditions by analyzing collected sensing data. We are aiming to improve reproduction precision by adjusting important parameters such as the stop-go indications of traffic signals. In the event of an unexpected incident, such as a traffic accident, this technology enables rapid assessment of the accident’s impact on traffic flow, thereby helping to prevent secondary damage or traffic jams.

I believe that, as connected cars become widespread, there will be an increasing demand for the ability to use spatiotemporal data intuitively in real time. By fully exploiting virtual reality technology, I will develop an intuitive user interface that displays digital twins in an elegant and well-organized manner so that anyone will be able to easily use spatiotemporal data. I will promote data usage in a wide range of applications through this effort and will do my utmost to develop technologies that will close the digital divide.

■Profile

Masaru Takagi received a B.E. in engineering and an M.E. and Ph.D. in information science and technology from the University of Tokyo in 2013, 2015, and 2018. He joined NTT Network Innovation Laboratories in 2018 and is engaged in a collaboration project with Toyota Motor Corporation. He developed the technology to efficiently collect the sensor data from connected cars using the edge computing platform (ICT platform). He is currently studying multi-agent mobility simulation as part of Digital Twin Computing technology. He received the Best Paper Nominee Award from the Association for Computing Machinery UbiComp 2014. He is a member of the Institute of Electronics, Information and Communication Engineers and Information Processing Society of Japan. He is interested in mobile terminals and owns more than 100 Android devices.

↑ TOP