Research on Symbiotic Intelligence for Achieving Symbiosis among People and between People and Machines

1. Introduction

As the real and cyber worlds rapidly converge, the manner in which people work and live is changing dramatically. Meetings and collaborative work that used to be done face-to-face in real spaces, such as classes at schools and social gatherings, are now done remotely, and social networking services (SNSs) are becoming the mainstream for chatting and gossip. Sporting events and theatrical performances, which used to require a direct trip to a stadium or theater, can now be richly experienced in cyberspace. The novel-coronavirus (COVID-19) pandemic has sped up the merging of the two (real and cyberspace) worlds.

We believe that by using the power of information and communication technologies, all people, and even people and machines, will be able to coexist more prosperously than ever before, and that this coexistence will enable people to experience even greater well-being than before. For example, we will be able to gently guide people and groups toward a better future and shared emotions or work together with machines that have a similar inner self to that of people through more natural and complex interactions. With this vision of the future in mind, we are researching and developing technologies that will contribute to (i) modeling and simulating human behavior, especially the behavior (thoughts, actions, etc.) of groups, and (ii) predicting and optimizing future social activities in which people as well as people and machines collaborate and live together. Specifically, this research and development (R&D) is based on the following three themes.

Hereafter, our efforts concerning these three themes are introduced.

2. Group psychological and behavioral modeling

We are conducting R&D on the modeling of human decision-making and behavior. People make various daily decisions, such as what to eat for dinner, what to buy, whether to exercise, where to go for pleasure, and what to post on SNSs, that dictate their social activities. To predict the future of social activities and engender ever greater well-being, modeling and computer simulation of human decision-making and behavior are indispensable technologies. For example, if we could model behavior and simulate how that behavior changes with interventions such as messages and incentives, we could create apps and services that enable users to naturally make healthier, safer, or more enjoyable choices. We are analyzing human behavior, modeling and simulating that behavior, and deriving measures to change behavior on the basis of the models (Fig. 1).

Fig. 1. Research initiatives on group psychology and behavior modeling.

Application of various mathematical-modeling and big-data-based optimization techniques, including deep learning, is a promising approach to model and simulate behavior. However, it is not easy to collect a large amount of data on the behavior of a wide variety of people. We believe that to elucidate the mechanisms of human decision-making and behavior, a more in-depth approach than simply applying current mathematical models is necessary.

Human behavior has been studied in the fields of behavioral economics and social psychology, and a great deal of knowledge has been accumulated. At NTT Human Informatics Laboratories, by constructing new mathematical models that incorporate this knowledge and verifying hypotheses, we aim to clarify the mechanism of human decision-making and behavior and implement efficient and highly accurate computer simulation.

3. Emergence and social brain

We are researching and developing knowledge processing that (i) accurately understands and converts the textual content into knowledge and (ii) provides the information required for collaboration with people in a natural exchange. Language modeling is a fundamental technology for accurately understanding and converting textual content into knowledge. Regarding Japanese, a language model is a representation of “the essence of the Japanese language” in a form that is easy for a computer to handle. Language modeling has been dominated by methods for creating deep-learning models through large-scale pre-training from huge amounts of text data.

We are also constructing large-scale pre-training language models for Japanese and conducting R&D to further improve their performance. Through these efforts, we aim to build a foundation for linguistic communication to enable collaboration among people as well as between people and machines while contributing to various language-processing-related services of the NTT Group.

One of the most-promising use cases for collaboration between people and machines in regard to textual content is automatic document summarization. We are developing a technology that enables highly accurate summarization using large-scale pre-training language models. This technology can, for example, generate a summary that includes keywords specified by the user and allows the length of the summary to be adjusted in accordance with the application. We believe that if we can further develop this technology to automatically generate reports concerning tasks such as examining a large amount of literature (medical care and intellectual property), it would generate great value (Fig. 2).

Fig. 2. Collaboration between people and machines (automatic generation of reports).

Collaboration would be more natural if people could talk to machines in the same language as they talk to other people. Technologies for interacting with users in natural language are called dialogue systems and can be broadly classified as task-oriented (which perform specific tasks) and systems that conduct so-called “chat.” We are researching and developing chat-oriented dialogue systems; in particular, we are developing technology to add character to such systems so that users feel familiar with them.

4. Emotional perception control

We are researching and developing technology for improving the quality of symbiosis as a group by (i) understanding the characteristics of people from perception to emotion and (ii) intervening in communication between and among people in accordance with these characteristics. Specifically, we are developing the following two technologies.

Regarding emotional perception control, we are developing emotion-estimation technology for estimating the emotions of individuals and groups (groups, crowds, etc.) by using biometric data, images, sounds, content, etc. and emotion-control technology for changing such emotions to a desirable state in accordance with the estimation results (Fig. 3).

Fig. 3. Overview of technology for controlling emotional perception.

Regarding collaborative-work support, which focuses on psychological safety as an important factor in improving the quality of collaborative work in groups, the aim is to make it easier for people with diverse characteristics to work together by enhancing their psychological safety.

For this purpose, we are developing a technology that can improve psychological safety, namely, reduce communication discrepancies, by understanding the communication characteristics of individuals and converting verbal and non-verbal information in accordance with those characteristics (Fig. 4).

Fig. 4. Overview of technology for supporting collaborative work.