To view PDF files

You need Adobe Reader 7.0 or later in order to read PDF files on this site.
If Adobe Reader is not installed on your computer, click the button below and go to the download site.

Feature Articles: Toward a New Form of Coexistence between the Real World and Cyberworld

doi: 10.53829/ntr202112fa3

Research on Symbiotic Intelligence for Achieving Symbiosis among People and between People and Machines

Tetsuya Kinebuchi, Taichi Asami, Sen Yoshida,
and Ryuji Yamamoto

Abstract

With the rapidly progressing fusion of the real and cyber worlds, the manner of symbiosis among people and between people and machines is changing significantly. Information and communication technologies will make it possible to gently guide the behavior of people and groups toward a better future and shared emotions and to enable people and machines to work together through more-natural and complex interactions. An overview of the research being conducted by the Symbiotic Intelligence Research Project toward this future vision is given in this article.

Keywords: behavioral modeling, knowledge processing, emotional perception control

PDF PDF

1. Introduction

As the real and cyber worlds rapidly converge, the manner in which people work and live is changing dramatically. Meetings and collaborative work that used to be done face-to-face in real spaces, such as classes at schools and social gatherings, are now done remotely, and social networking services (SNSs) are becoming the mainstream for chatting and gossip. Sporting events and theatrical performances, which used to require a direct trip to a stadium or theater, can now be richly experienced in cyberspace. The novel-coronavirus (COVID-19) pandemic has sped up the merging of the two (real and cyberspace) worlds.

We believe that by using the power of information and communication technologies, all people, and even people and machines, will be able to coexist more prosperously than ever before, and that this coexistence will enable people to experience even greater well-being than before. For example, we will be able to gently guide people and groups toward a better future and shared emotions or work together with machines that have a similar inner self to that of people through more natural and complex interactions. With this vision of the future in mind, we are researching and developing technologies that will contribute to (i) modeling and simulating human behavior, especially the behavior (thoughts, actions, etc.) of groups, and (ii) predicting and optimizing future social activities in which people as well as people and machines collaborate and live together. Specifically, this research and development (R&D) is based on the following three themes.

1) Group psychological and behavioral modeling: Modeling of psychological, judgmental, and social behaviors of people and their groups and simulation and optimization of those behaviors on the basis of the models.

2) Emergence and social brain: Knowledge processing that (i) accurately understands and converts the textual content into knowledge and (ii) provides the information required for collaboration with people in a natural exchange.

3) Emotional perception control: Explanation and sensing of an internal model of human emotional psychology (enthusiasm, sense of unity, etc.), and collaborative-work support to resolve communication discrepancies by empathizing with individual interpersonal characteristics.

Hereafter, our efforts concerning these three themes are introduced.

2. Group psychological and behavioral modeling

We are conducting R&D on the modeling of human decision-making and behavior. People make various daily decisions, such as what to eat for dinner, what to buy, whether to exercise, where to go for pleasure, and what to post on SNSs, that dictate their social activities. To predict the future of social activities and engender ever greater well-being, modeling and computer simulation of human decision-making and behavior are indispensable technologies. For example, if we could model behavior and simulate how that behavior changes with interventions such as messages and incentives, we could create apps and services that enable users to naturally make healthier, safer, or more enjoyable choices. We are analyzing human behavior, modeling and simulating that behavior, and deriving measures to change behavior on the basis of the models (Fig. 1).


Fig. 1. Research initiatives on group psychology and behavior modeling.

Application of various mathematical-modeling and big-data-based optimization techniques, including deep learning, is a promising approach to model and simulate behavior. However, it is not easy to collect a large amount of data on the behavior of a wide variety of people. We believe that to elucidate the mechanisms of human decision-making and behavior, a more in-depth approach than simply applying current mathematical models is necessary.

Human behavior has been studied in the fields of behavioral economics and social psychology, and a great deal of knowledge has been accumulated. At NTT Human Informatics Laboratories, by constructing new mathematical models that incorporate this knowledge and verifying hypotheses, we aim to clarify the mechanism of human decision-making and behavior and implement efficient and highly accurate computer simulation.

3. Emergence and social brain

We are researching and developing knowledge processing that (i) accurately understands and converts the textual content into knowledge and (ii) provides the information required for collaboration with people in a natural exchange. Language modeling is a fundamental technology for accurately understanding and converting textual content into knowledge. Regarding Japanese, a language model is a representation of “the essence of the Japanese language” in a form that is easy for a computer to handle. Language modeling has been dominated by methods for creating deep-learning models through large-scale pre-training from huge amounts of text data.

We are also constructing large-scale pre-training language models for Japanese and conducting R&D to further improve their performance. Through these efforts, we aim to build a foundation for linguistic communication to enable collaboration among people as well as between people and machines while contributing to various language-processing-related services of the NTT Group.

One of the most-promising use cases for collaboration between people and machines in regard to textual content is automatic document summarization. We are developing a technology that enables highly accurate summarization using large-scale pre-training language models. This technology can, for example, generate a summary that includes keywords specified by the user and allows the length of the summary to be adjusted in accordance with the application. We believe that if we can further develop this technology to automatically generate reports concerning tasks such as examining a large amount of literature (medical care and intellectual property), it would generate great value (Fig. 2).


Fig. 2. Collaboration between people and machines (automatic generation of reports).

Collaboration would be more natural if people could talk to machines in the same language as they talk to other people. Technologies for interacting with users in natural language are called dialogue systems and can be broadly classified as task-oriented (which perform specific tasks) and systems that conduct so-called “chat.” We are researching and developing chat-oriented dialogue systems; in particular, we are developing technology to add character to such systems so that users feel familiar with them.

4. Emotional perception control

We are researching and developing technology for improving the quality of symbiosis as a group by (i) understanding the characteristics of people from perception to emotion and (ii) intervening in communication between and among people in accordance with these characteristics. Specifically, we are developing the following two technologies.

1) Emotional-perception-control technology: Technology that expands communication (e.g., fostering a sense of unity and enthusiasm through sharing of emotions) by understanding the characteristics of human emotional expressions, estimating emotional characteristics through sensing and data analysis, and controlling emotions with perceptual stimuli tailored to emotional characteristics.

2) Collaborative-work-support technology: Technology for improving the quality of collaborative work within a group by understanding the communication characteristics of people and eliminating communication discrepancies by converting information tailored to those characteristics in a manner that induces behaviors that facilitate communication.

Regarding emotional perception control, we are developing emotion-estimation technology for estimating the emotions of individuals and groups (groups, crowds, etc.) by using biometric data, images, sounds, content, etc. and emotion-control technology for changing such emotions to a desirable state in accordance with the estimation results (Fig. 3).


Fig. 3. Overview of technology for controlling emotional perception.

Regarding collaborative-work support, which focuses on psychological safety as an important factor in improving the quality of collaborative work in groups, the aim is to make it easier for people with diverse characteristics to work together by enhancing their psychological safety.

For this purpose, we are developing a technology that can improve psychological safety, namely, reduce communication discrepancies, by understanding the communication characteristics of individuals and converting verbal and non-verbal information in accordance with those characteristics (Fig. 4).


Fig. 4. Overview of technology for supporting collaborative work.

Tetsuya Kinebuchi
Senior Manager, Symbiotic Intelligence Laboratory, NTT Human Informatics Laboratories.
He received an M.S. in physics from Tohoku University, Miyagi, in 1997 and joined NTT Corporation the same year. His research interests include media processing and artificial intelligence (AI). He is a member of the Institute of Electronics, Information and Communication Engineers (IEICE) of Japan.
Taichi Asami
Senior Research Engineer, Symbiotic Intelligence Laboratory, NTT Human Informatics Laboratories.
He received a B.E., M.E., and Ph.D. in engineering from Tokyo Institute of Technology in 2004, 2006, and 2019. He joined NTT Corporation in 2006. His research interests include media processing, AI, and behavioral modeling. He is a member of the Institute of Electrical and Electronics Engineers (IEEE), IEICE, the Information Processing Society of Japan (IPSJ), and the Acoustical Society of Japan (ASJ).
Sen Yoshida
Senior Research Engineer, Supervisor of Symbiotic Intelligence Laboratory, NTT Human Informatics Laboratories.
He received a B.E. and M. Info. Sci. from Tohoku University, Miyagi, in 1993 and 1995. He joined NTT Corporation in 1995. His research interests include natural language processing and AI. He is a member of IPSJ and the Japanese Society for Artificial Intelligence (JSAI).
Ryuji Yamamoto
Senior Research Engineer, Supervisor of Symbiotic Intelligence Laboratory, NTT Human Informatics Laboratories.
He received an M.E. from Kyushu Institute of Technology, Fukuoka, in 1998. He joined NTT Corporation in 1998. His research interests include media processing and human-computer interaction.

↑ TOP