Event Report: NTT Communication Science Laboratories Open House 2021

1. Overview

NTT Communication Science Laboratories (NTT CS Labs) celebrated its 30th anniversary this year. NTT CS Labs has aimed to establish cutting-edge technologies that enable heart-to-heart communication between people and people and between people and computers. We are thus working on a fundamental theory that approaches the essence of humans and information science, as well as on innovative technologies that will transform society. NTT CS Labs’ Open House is held annually to introduce the results of our basic research and innovative leading-edge research with many hands-on intuitive exhibits to those who are engaged in research, development, business, and education.

Open House 2021 was held virtually due to measures to prevent the further spread of COVID-19. The latest research results were published with recorded lecture videos on the Open House 2021 web page at noon on June 3rd [1]. The content attracted many views within a month not only from NTT Group employees but also from businesses, universities, and research institutes. The event content is still available.

This article summarizes the event’s research talks and exhibits.

2. Keynote speech

Dr. Takeshi Yamada, vice president and head of NTT CS Labs, presented a speech entitled “No matter how far apart, my heart will always be with you – Exploring the essence of communication that will create a spiritually rich society –,” in which he looked back upon the history of NTT and establishment of NTT CS Labs then introduced present and future cutting-edge basic research and technologies (Photo 1).

For the past 30 years since its founding on July 4, 1991, NTT CS Labs has been developing innovative technologies such as media processing and machine learning that approach and even surpass human capabilities and seeking out fundamental principles in fields such as cognitive neuroscience and brain science that afford a deeper understanding of humans. After introducing the latest research results of NTT CS Labs from the three perspectives of “how to communicate accurately and efficiently” (information), “how to communicate by what means” (media), and “what is being indicated” (meaning), Dr. Yamada declared that NTT CS Labs will boldly and persistently tackle new challenges for the next 30 years to enable communication that can be conveyed to the heart through research that only NTT CS Labs can conduct.

Photo 1. Keynote speech (Dr. Takeshi Yamada). “No matter how far apart, my heart will always be with you – Exploring the essence of communication that will create a spiritually rich society –.”

3. Research talks

The following three research talks highlighted recent significant research results and high-profile research themes. Each talk introduced some of the latest research results and provided background and an overview of the research. After each talk, we received many questions from participants in real time for the question-and-answer (Q&A) session, confirming that the viewers had a high interest in our research.

(1) “The coming ages when AI becomes our conversation partners – Cutting edge of conversational systems with large-scale neural network models –,” Dr. Hiroaki Sugiyama, Innovative Communication Laboratory

Dr. Hiroaki Sugiyama explained chatting dialogue systems for satisfying people’s desire for dialogue through natural chatting with people. NTT CS Labs has been researching various dialogue systems for a long time, including research on giving systems personalities and improving the smoothness of dialogue by organically linking multiple dialogue robots. With the recent rapid development in deep learning, English social dialogue systems using large-scale deep learning have been proposed. He introduced details of NTT CS Labs’ latest deep learning-based Japanese social dialogue system and its achievements and remaining issues (Photo 2).

Photo 2. Research talk (Dr. Hiroaki Sugiyama). “The coming ages when AI becomes our conversation partners – Cutting edge of conversational systems with large-scale neural network models –.”

(2) “Looking more, acting better – New concept of eye-hand coordination for skilled action –,” Dr. Naotoshi Abekawa, Human and Information Science Laboratory

Dr. Naotoshi Abekawa explained skillful motor control using the mechanism of coordinated eye and arm motion as an example. One key issue in developing user-friendly information and communication technology is to understand “behavior” or “action,” that is, to understand the essence of “motion.” Although we seem to control motions easily, motor controls are achieved by sophisticated brain mechanisms, including the control of eye movements to obtain target information and the generation of limb movements. In his talk, he focused on the question, “Why is the eye important for skilled motor actions?”, introduced explanations from the literature, and proposed an interpretation based on his findings (Photo 3).

Photo 3. Research talk (Dr. Naotoshi Abekawa). “Looking more, acting better – New concept of eye-hand coordination for skilled action –.”

(3) “Developing AI that pays attention to who you want to listen to – Deep learning based selective hearing with SpeakerBeam –,” Dr. Marc Delcroix, Media Information Laboratory

Dr. Marc Delcroix discussed approaches to achieve computational selective hearing. Selective hearing refers to the ability to focus on listening to a desired speaker, even in a noisy environment such as a cocktail party. He first introduced SpeakerBeam, which is a deep-learning-based method he and his co-researchers proposed to extract speech of a desired target speaker in a mixture of several speakers by using a few seconds of pre-recorded audio data of the target speaker. He then presented recent research on (1) extension to multi-modal processing, where he and his co-researchers use video of the lip movement of the target speaker in addition to audio pre-recording, (2) integration with automatic speech recognition, and (3) the generalization to the extraction of arbitrary sounds (Photo 4).

Photo 4. Research talk (Dr. Marc Delcroix). “Developing AI that pays attention to who you want to listen to – Deep learning based selective hearing with SpeakerBeam –.”

4. Research exhibition

The Open House featured 29 exhibits displaying NTT CS Labs’ latest research results. We categorized them into four areas: Science of Machine Learning, Science of Communication and Computation, Science of Media Information, and Science of Humans. Each exhibit prepared videos explaining the latest results and were published on the event web page (Photo 5). Several provided online demonstrations or demo videos to make them closer to direct demonstrations. We also introduced a Q&A system in which page visitors can freely post questions and comments and CS Labs’ researcher can answer them. There were more than 130 public questions, and some expressed the enthusiasm of the visitors, such as long comments and professional questions.

The following list, taken from the Open House website, summarizes the research exhibits in each category.

Photo 5. Exhibition web page.

5. Special lecture

We asked Professor Asa Ito, professor of Tokyo Institute of Technology, to give a special lecture entitled “Bodies in the remote era,” have a talk with NTT Fellow Makio Kashino, and review the research exhibitions. In the lecture, based on her research on haptic communication with the visually impaired, she talked about the problems of remote communication and the potential of the body (haptics). As an example of how trust was created through haptic communication, she shared an episode in which she simulated the experience of a visually impaired person being accompanied by a runner, and the rope connected to the runner quickly transmitted subtle information such as emotions and sensations, resulting in a surprising level of trust. She mentioned, however, that remote communication tends to be a one-way transmission of information, making it difficult to build a relationship of trust. She also introduced her research on watching sports with the visually impaired, remotely operated alter-ego robot communication, and memory of sensation and haptic perception. She mentioned that in haptic remote communication, we can sometimes feel the existence of a person because we cannot touch their actual body, and that she has sensed the possibility for a new form of communication.

6. Concluding remarks

Open House 2021 was held as an event to present our latest results on a website. The lecture videos were viewed more than 6000 times in June by various segments of users. Participants provided many valuable opinions that have encouraged us to pursue further research activities through interactive means of communication with CS Labs researchers such as the Q&A system and Q&A session during the lectures. In closing, we would like to offer our sincere thanks to all the participants of this online event.

Reference

[1]	Website of NTT Communication Science Laboratories Open House 2021, http://www.kecl.ntt.co.jp/openhouse/2021/index_en.html