Feature Articles: Toward a New Form of Coexistence between the Real World and Cyberworld
Research and Development of Digital Twin Computing for Creating a Digitalized World
The Digital Twin Computing Research Project aims to make Digital Twin Computing, a key component of the Innovative Optical and Wireless Network (IOWN) initiative promoted by NTT, a reality. We established four grand challenges to achieve this objective and have been conducting research and development to meet each one. This article outlines each grand challenge and associated technologies and describes horizontal activities across these challenges.
Keywords: Digital Twin Computing, grand challenges, IOWN
1. Grand challenges for achieving Digital Twin Computing
Digital Twin Computing (DTC) is a key component of the Innovative Optical and Wireless Network (IOWN) initiative promoted by NTT . It aims to make large-scale and high-accuracy predictions and trials of the future beyond the limits of conventional information and communication technologies. It also aims to achieve advanced forms of communication having new value by combining various types of high-accuracy information related to things, people, and society in the real world. The idea is to create a smart society by solving a variety of social issues affecting the world and creating innovative services.
We established “Mind-to-Mind Communications,” “Another Me,” “Exploring Engine for Future Society,” and “Inducing Inclusive Equilibrium Solutions for the Earth and Its Social and Economic Systems” as four grand challenges for achieving DTC as announced in an NTT press release in November 2020 . We have since begun specific studies on each of these grand challenges. In this article, we outline these challenges and the technologies being developed to meet them and introduce future objectives. We also describe activities designed to accelerate the meeting of these four challenges.
2. Mind-to-Mind Communications
“Mind-to-Mind Communications” is a research and development (R&D) objective to achieve a new way of communicating that will enable people to directly understand how other people perceive and feel things. This new form of communication will transcend not only language and cultural differences but also individual characteristics such as one’s experiences and sensitivities.
To achieve this objective, we are undertaking the R&D centered about three key technologies: mind-to-mind modeling, mind-to-mind conversion and presentation, and state transformation and environment creation. First, mind-to-mind modeling technology uses sensor information consisting of biological signals, images, speech, etc. to infer expressions in mind-to-mind communication, that is, in ways of perceiving and feeling things. Next, mind-to-mind conversion and presentation technology converts and presents expressions in mind-to-mind communication that differ from one individual to another in a form suitable to one’s communication partner. Finally, state-transformation and environment-creation technology promotes a change in how one perceives and feels things by intervening not only in terms of the information exchanged in communication but also in regard to a person’s inner world, such as receptivity, and the communication site.
Achieving this R&D objective will lead to a new stage of communication that will solve the problems of discommunication (failure to communicate one’s intentions) and miscommunication (mutual misunderstanding) prevalent in past ways of communicating. This will enable a person to realistically and directly understand how another person mentally perceives and feels things beyond differences in language, culture, experiences, values, sense perceptions, and sensitivities. By improving psychological safety and promoting mutual understanding, Mind-to-Mind Communications creates an inclusive society in which people having diverse characteristics engage in activities together while exploiting those characteristics, thereby inspiring and helping each other to grow (Fig. 1).
3. Another Me
The loss of various opportunities in one’s life due, for example, to the difficulty of balancing childcare and caregiving with work or the inability to participate in society despite an interest or desire to do so, is becoming a social problem. To dramatically increase the opportunities for a person to become more active and grow while expanding and merging the range of activities from the real world to the virtual world, we are taking up the challenge of achieving “Another Me” as another version of oneself in digital form who can participate actively as oneself in a way that transcends the limitations of the real world and who can share the results of such activities as experiences (Fig. 2). Our main initiatives in meeting this challenge are summarized below.
(1) Transcend the limitations of the real world: To transcend the limitations of time, space, and personal ability that reduce one°«s opportunities in life, we seek to establish autonomous agent technology that enables Another Me to automatically perform activities in place of the user and telepresence/body-augmentation technology to accommodate remotely located users or solve ability-related issues of the physically challenged, elderly, etc.
(2) Foster the user’s sense of self-agency: To make the user feel as if the activities of Another Me are those done by oneself and experience a feeling of accomplishment and growth, we seek to establish experience-transfer technology to achieve psychological teleportation and feedback. So that the actions of Another Me can be recognized by the user and others as those of the user himself/herself, we will also establish technology for creating and nurturing a human digital twin who can develop a personal image that changes from day to day and exhibit individual characteristics such as a sense of values and personality.
(3) Social acceptance research: We will investigate philosophical, ethical, and legal issues on the basis of how we view ourselves toward widespread adoption of Another Me in society.
This approach will accelerate self-realization by providing a variety of opportunities in one°«s life and improving one°«s state of mind, health, and motivation while enabling the experiences of Another Me to grow as those of one°«s own through many growth opportunities.
4. Exploring Engine for Future Society
Due to the increasing complexity of the social structure and an increase in unpredictable events such as natural disasters, it is becoming difficult to observe the effects of individual behavior and cooperation on society and the natural environment and how those effects come back to affect the individual. Under these circumstances, it would be desirable to search for ways that society and lifestyles can coexist with nature. To this end, we are researching and developing an °»Exploring Engine for Future Society.°… This engine will divide the society that people are active in into the three layers of environment, city, and people and express these layers as digital twins. It will then use these digital twins to simulate the transformation of this multilayered society.
To make this engine a reality, we are researching and developing technology for creating in real time digital twins of the real world obtainable through sensing and for modeling abstract information such as public opinion and individual consciousness. We intend to use basic technologies such as previously developed temporal-spatial data management technology. We are also researching and developing basic simulation techniques to combine the above with city and environment simulations. These technologies will make it possible to drive society°«s digital twins and present a variety of future scenarios and candidate measures that should be taken now by calculating backward from an ideal image of society.
Visualizing various forms that future society can take in this manner should make it possible to bolster various types of social solidarity by having not only government and corporate management but also local residents and individual citizens view the same future images. Additionally, we aim for a world in which certain types of behavior and cooperation can be proactively searched for and spontaneous actions can be encouraged so that a society desired by the individual can be achieved (Fig. 3).
5. Inducing Inclusive Equilibrium Solutions for the Earth and Its Social and Economic Systems
In view of climate change and social crises now occurring on a global scale, discussions have begun on reconstructing social and economic systems that can maintain an autonomous equilibrium on the Earth. This is a problem with a high degree of uncertainty, so it is important to prepare multiple options and scenarios and study and execute comprehensive approaches. With this in mind, we have taken up the challenge of presenting options with regard to the transformation of energy, the economic system, and human society. The aim is to determine changes and mutual effects (chain reactions) among the many different systems making up the Earth, such as the environment, ecosystem, energy, economic systems, and human society, and preserve an autonomous equilibrium that can maintain a balanced state in which no system will exceed a critical point and collapse.
First, to meet this challenge using as themes the energy supply corresponding to changes in climate and water circulation along with economic systems and human society, we will undertake the R&D of (1) large-scale multimodal technology for massive coupling of model groups of many and varied phenomena and simulation of inter-model chain reactions, (2) modeling technology for designing on a global scale while taking into account inter-system coupling between the environment and economic systems, and (3) techniques for determining key indices of autonomous equilibrium and local solutions of those indices and techniques for evaluating index-value groups for each scenario.
In this way, we will be able to link many and varied large-scale models and conduct inclusive circulation simulations on a global scale of diverse and complex phenomena occurring in the real world throughout the Earth while presenting multiple options for a social transformation (Fig. 4).
6. Horizontal activities for the grand challenges
Technology development for meeting the above grand challenges covers a wide range of fields, so we are engaged in unified development activities across these grand challenges. We introduce three of these activities: (1) creation of a DTC reference model, (2) collaboration with partners, and (3) standardization activities.
(1) Creation of a DTC reference model
In creating a DTC reference model that compiles technical achievements across multiple projects, we have documented digital-twin concepts common to DTC while taking a comprehensive view across the four grand challenges and released the document both inside and outside NTT laboratories . This document includes conceptual structures now being studied at the NTT Digital Twin Computing Research Center with respect to the definition of digital twins and configuration of human digital twins. The aim of releasing this information is to promote discussion with partner companies having a common vision and make joint explorations of even more essential DTC concepts. Going forward, we will add more research results as they become available from the NTT Digital Twin Computing Research Center to provide an even more complete reference document.
(2) Collaboration with partners
To accelerate the R&D of the above grand challenges through collaboration with partners, we will search for outside partners to accelerate DTC R&D and work on creating a collaboration scheme. Although necessary technology areas differ from one grand challenge to another, we will pursue a broad range of partnering, including tie-ups with universities, other research institutions, and startup companies targeting a wide range of fields from artificial intelligence technologies for understanding human emotions and behavior to digital-twin technologies for smart cities. The field of digital twins is relatively new, but at the same time, it is connected to a wide range of technology areas such as virtual reality (extended reality, etc.) and digital humans; therefore, many startup companies with deeply interesting technologies are appearing and growing. It is our belief that establishing collaborative partnerships with such companies will accelerate R&D and enable the early implementation of novel technologies.
(3) Standardization activities
Finally, we will engage in standardization activities to standardize and spread DTC on a global scale. As part of this effort, we will promote activities for visualizing DTC and research results at global forums. Specifically, we are participating in the Digital Twin Consortium and the IOWN Global Forum while making contributions and presentations in relation to DTC and grand challenges. Through these activities, we hope to increase NTT°«s presence in the digital twin field and promote standardization activities for digital-twin technologies together with global partners we can collaborate with in making DTC a reality.