Report on the NTT R&D FORUM 2023 — IOWN ACCELERATION

2.1 Keynote Speech 1: Innovating a Sustainable Future for People and Planet
NTT R&D Initiatives

NTT President and CEO Akira Shimada opened his speech by introducing NTT’s new R&D theme of “Innovation for a sustainable future for people and planet,” with the major goals of resolving issues faced by society and enhancing the well-being of people in society. He then went on to describe IOWN and tsuzumi, NTT’s LLM, as key technologies for achieving these goals. He then discussed how IOWN and tsuzumi can solve difficult issues faced by societies around the world, such as labor shortages, environment and energy issues, aging populations and increasing medical costs. He introduced some concrete examples of how NTT technologies have solved issues to improve well-being in society and a vision for the future that NTT and its partners have created together (Photo 1).

Photo 1. Keynote Speech 1: Akira Shimada.

He introduced “IOWN1.0” as the All-Photonics Network (APN) that was launched commercially in March 2023 and described plans for “IOWN2.0” and beyond, with a roadmap to advance optical technologies from the network into the computing environment. He also drew attention to a pavilion planned for Expo 2025 in Osaka, which will enable visitors to experience services using IOWN technologies.

He also discussed efforts making steady progress toward the commercial release of tsuzumi that is causing a stir in the field of generative artificial intelligence (AI) and has been attracting attention around the world. He then stressed the great potential of having these two technologies, to solve social issues and make great contributions to realizing well-being in society.

He concluded the speech with a message of “innovating a sustainable future for people and planet,” stating that NTT will continue working and creating innovation for people and the planet.

2.2 Keynote Speech 2: IOWN ACCELERATION —Imagination and Creation—

NTT Executive Vice President, Head of Research and Development Market Strategy Sachiko Oonishi spoke about IOWN, with concrete examples of commercial application for full implementation in society. With the theme and title, “IOWN ACCELERATION —Imagination and Creation—,” she presented a new vision for the future created through the accelerating plan and implementation of IOWN, from both “product-out” and “market-in” R&D perspectives (Photo 2).

Photo 2. Keynote Speech 2: Sachiko Oonishi.

In her opening she said, “I hope that you can now feel this idea (of IOWN) coming to life,” and presented a roadmap to telecommunications breakthroughs for IOWN2.0, brought by progress in elemental technologies in various fields such as opto-electric integration. With key points in the speech, she also identified historical milestones developed by NTT in the form of a quiz, giving background on how NTT was able to achieve those breakthroughs.

She also introduced solutions to issues that NTT has promoted in a wide range of fields; touching on areas considered basic to human life such as clothing, food, and shelter and presenting a new “market-in” R&D perspective that NTT is adopting, which focuses on technologies and solutions needed to improve people’s well-being in society.

Finally, she described NTT’s goal of creating value for the social well-being of people and the planet, toward an exciting future realized by NTT.

2.3 Keynote Speech 3: LLM+×IOWN —The advancement of IOWN, the launch of NTT’s LLM, and their synergy—

NTT Senior Vice President and Head of Research and Development Planning Shingo Kinoshita spoke about progress regarding NTT’s LLM tsuzumi, which was announced in November 2023, and also IOWN and the synergies between these two technologies. Using “LLM+×IOWN” as a theme and providing much technical explanation, he described the far-reaching potential of combining these technologies in various ways (Photo 3).

Photo 3. Keynote Speech 3: Shingo Kinoshita.

LLM tsuzumi, which is bringing new perspectives to the LLM field, has four major features: (1) It is lightweight and has low power consumption, (2) It has strong linguistic capability (especially Japanese), (3) It is flexible for customization, and (4) It is multi-modal. These were each introduced in detail, including the underlying technologies that make them possible, and showing the resulting high-performance.

Regarding synergies with IOWN, he described practical experiments related to the dramatic developments in LLMs. This included experiments that overcame physical distance, combining computing resources to perform LLM training and inference efficiently, using graphics processing units (GPUs) in a datacenter roughly 100 km away to process training data held locally, and treating datacenters that are far apart as though they were a single datacenter. He also described plans to expand experiments in the field globally in the future.

He also announced a joint R&D initiative with Sakana AI, a Tokyo-based startup that is attracting attention globally. He suggested that cooperation between these companies in the field of AI, for which they both have great affinity, will have great influence on how generative AI should advance and how architectures are considered in the future.

Finally, he also reflected on the “fountain of knowledge” mindset inherited from NTT R&D. He described three elements comprising this mindset, which are to promote R&D, to implement research results practically, and to implement them in society together with partner enterprises.

3.1 Special Session 1: IOWN opens up a data-driven era

Hiroaki Nagashii, VP, Strategic Client Group, Oracle Corporation Japan, and Yoshisato Fukatsu, Enterprise Architect, Industry, Value and Architecture, Oracle Corporation Japan, were invited to give a lecture on the roles IOWN and Oracle play in realizing a data-driven society and their vision for that society (Photos 4, 5).

Photo 4. Special Session 1: Hiroaki Nagashii.

Photo 5. Special Session 1: Yoshisato Fukatsu.

Mr. Nagashii introduced his vision for the future, based on the vision of the founder of Oracle Corporation. He expressed his desire for Oracle to contribute to realizing a data-driven society and how that desire is reflected in their efforts and achievements utilizing data in various domains, including fields that require critical elements such as renewable energy.

He also indicated they have high expectations for the paradigm shift in implementing a data-driven society due to the IOWN features of high-security, high-capacity, and low latency. Mr. Fukatsu then introduced some concrete “IOWN × Oracle” solutions from the perspectives of potential and feasibility. He described concepts for future solutions created utilizing the high-capacity, low-latency communications infrastructure implemented by NTT with IOWN, together with the best data-management technology in the world from Oracle Corp. He expressed particularly high expectations for NTT regarding the high security technology in IOWN.

Finally, Mr. Nagashii talked about his strong desire to create a society in which “no one is left behind” in the future, after further collaboration between NTT and Oracle results in dramatic progress in technology. It was a captivating speech regarding co-creation by NTT and Oracle in the future.

3.2 Special Session 2: Does AGI dream of living with Humans? —R&D of LLM “tsuzumi”—

In the first part of Special Session 2, NTT Human Informatics Laboratories’ Senior Distinguished Researcher Kyosuke Nishida presented a detailed introduction to R&D on NTT’s LLM tsuzumi, and its features (Photo 6).

Photo 6. Special Session 2: Kyosuke Nishida.

Mr. Nishida, who drove R&D on tsuzumi, included a conversation with tsuzumi as the first section of his speech, showing the astonishing achievement and high performance of tsuzumi from the beginning.

He then presented his research vision of creating an artificial general intelligence (AGI) thinking engine that can naturally coexist with people and support their well-being in any environment. In the speech he demonstrated a conversation with tsuzumi using concrete questions. LLM tsuzumi even answered questions that are unavoidable when discussing AI, such as “How can AI be trained to follow rules?”

The session gave an impression of the high performance of tsuzumi through the responses and sense of the issues it presented, toward a world in which the tsuzumi AI and people can coexist.

3.3 Special Session 2: Natured Inspired Intelligence and a New Paradigm for LLM

For the second half of Special Session 2, we invited David Ha, Co-Founder CEO, Sakana AI, and Llion Jones, Co-Founder CTO, Sakana AI (Photo 7).

Photo 7. Special Session 2: David Ha (left) and Llion Jones (right).

David Ha and Llion Jones spoke about future directions for language models, focusing on LLMs. David Ha surprised the audience by indicating that, although he is among a minority in the current LLM community, he thinks that “even if the extremely large models become even larger, they will not be possible to handle the complexity of the real world.” He went on to say that based on ideas from nature, “human intelligence is largely a collective intelligence,” and from that perspective, rather than having a single, gigantic LLM, LLMs must evolve to cooperate and adapt dynamically to conditions, so that multiple LLMs with various features cooperate to produce intelligence.

He also talked about his expectations for the potential of Japan itself. He believes that Japan can make its presence known globally in AI, and spoke passionately about this as one of the reasons Sakana AI established its base in Japan.

Llion Jones, also from Sakana AI, then described “character-level modeling” in detail as one of the strengths of tsuzumi. He explained that current LLMs, which mainly focus on English, use both character-level and word-level models, but he emphasized that they would need to transition to character-level models. He also pointed out the surprising fact that there is redundancy in AI learning between different languages, so there is still potential yet to be identified in AI.

In their speeches, both speakers showed the affinity of both NTT and Sakana AI for the forms and ideal architectures of LLMs in the future, so the session raised expectations for the commercialization and later development of tsuzumi, with its particular strengths in the Japanese language.

3.4 Special Session 3: Lithium Niobate Photonics In The Era of AI

In this session, Bob Byer, the William R. Kenan, Jr. Professor Emeritus of Applied Physics, Stanford University, and Timothy McKenna, Principal Scientist, Physics & Informatics Laboratories, NTT Research, expanded on the topic, including episodes involving breakthroughs from their long history and tradition (Photo 8).

Photo 8. Special Session 3: Timothy McKenna (left), Bob Byer (right).

This session introduced the history and background of photonics using lithium niobate (LN), which is one of the most widely used functional optical materials. Bob Byer, an LN pioneer, described the “1st-generation” history in the 1960s and 1970s, and Timothy McKenna, who led an NTT Research team in recent years driving technical innovation and commercialization, described the “2nd-generation” since the 1980s, and the following “3rd-generation.”

A surprising revelation was that during his research Mr. Byer once dropped and damaged a photonic crystal, but this later led to a technical breakthrough. His answer to what to do with the damaged crystal is related to recent rapid advances in fabrication technologies. Now that fabrication of LN devices with nanometer precision has been achieved, LN is expected to play an important role in implementing high-speed, efficient communication in datacenters. The importance of thin-film LN (TFLN) photonics is also predicted to increase even more due to the rapidly increasing size and bandwidth requirements of AI clusters, an expanding photonics market, and the expanding scale and requirements of datacenters. Incredibly, the size of the TFLN market has exceeded two billion dollars in 2023, just five years since it appeared as a market. Overall, the session was very entertaining from two individuals who have been instrumental in the field of LN.

Finally, Timothy McKenna and Bob Byer expressed their hopes to make further great contributions to solving hardware issues, unlocking the fascination and hidden potential of TFLN, and building excitement for a bright future.

4.1 MIRAI Seminar 1: How the Technology Innovation Brings “Space within Your Reach”

President & CEO of Axelspace Corporation Yuya Nakamura spoke about technical innovation in satellite development and use, the importance of observations from space, and the relationship between humans and space in the future (Photo 9).

Photo 9. MIRAI Seminar 1: Yuya Nakamura.

Space business is seen as a rapidly maturing industry around the world, with a market expected to exceed one trillion dollars by 2040. Mr. Nakamura spoke about how “observations from space” are an important element resulting from use of satellites. He emphasized that satellite imagery is used for more than just smart agriculture; that conditions on earth and in space observed from satellites are important data for understanding the world accurately and making improvements in all fields, and can also be used as evidence.

He also introduced a new, small satellite business that provides one-stop satellite use in a short period of time, incorporating setting objectives, development, manufacturing, launch and operations, and he discussed how this initiative expands the potential for satellite use in fields and industries more removed from conventional space business and “brings democracy to space.”

Mr. Nakamura also expressed strong expectations for the impact of optical communications being advanced by NTT’s IOWN initiative, as a game changer for space business. Space Compass Corp. and Axel Space Corp. are already collaborating with NTT and SKY Perfect JSAT Corporation to expand optical communications into space. Mr. Nakamura concluded his speech by emphasizing that he will contribute to the next-generation communications infrastructure to be built by Space Compass and further expand cooperation.

4.2 MIRAI Seminar 2: Future Health with Digital Technology —Autonomic Nervous System and Intestinal Environment

Hiroyuki Kobayashi, Professor, Department of Research and Development for Elderly Physical Function, Juntendo University Graduate School of Medicine, was invited and gave an eye-opening speech on solutions to various issues in medicine and new solutions available through advances in technology (Photo 10).

Photo 10. MIRAI Seminar 2: Hiroyuki Kobayashi.

Prof. Kobayashi spoke about how we assume that data science basically “makes the invisible visible” by luck, but actually these facts are established by science. He described to a surprised audience how this does not just apply to illness and infection, but that sports are also explained by science using a golf example. From the perspective of “reproducibility,” which is an important element of medicine and sports science, the level of R&D and technology that NTT is advancing is highly regarded, such as the “motor-skill transfer technology” which brings potential to the world of medicine for applications such as remote surgery. He gave the impression that we have reached an age where even surgical skills can be transferred from experienced doctors to trainees.

He then expanded his topic to “What is health?”, talking about how our intestinal environment is an important element for living a healthy lifestyle, and why yoghurt has a positive effect on this environment. He explained the importance of the intestinal environment for a healthy lifestyle, adding a humorous metaphor of yogurt being a “transfer student” in that yogurt stimulates its environment like a transfer student stimulates unfamiliar students.

To end the seminar, he explained how various obstacles to a healthy lifestyle can lead to a negative spiral in behavior and thinking, while also taking questions from the audience.

4.3 MIRAI Seminar 3: The Vision of a Future Society Beyond DX (Digital Transformation)

Hiroaki Miyata, Professor and Chair, Department of Health Policy Management, School of Medicine, Keio University, spoke mainly on the theme of what kind of future can be created in a world focused on digital transformation (DX) and after the IOWN concept is implemented (Photo 11).

Photo 11. MIRAI Seminar 3: Hiroaki Miyata.

Prof. Miyata reflected on how work-life has changed greatly as application of IT has progressed and information gathering with technologies like search engines has spread, and the large impact that generative AI, which is now poised to permeate society, is having on a wide range of fields. He talked about how generative AI is organizing issues, but what people need is to look at the future and to position technology accordingly. He said that a perspective of realizing our ideals rather than optimizing our current work with technologies like generative AI and DX is important and has not been achieved yet.

He also emphasized the value in the ability to “connect”—overcoming physical distances and recognize and sympathize with each other—by consolidating communication infrastructure and expanding social networking services, and expressed hope that such connections will be strengthened by NTT’s IOWN initiatives.

On the other hand, he also spoke of the “Great reset” being advocated by the World Economic Forum (WEF) and further changes coming to values and the positioning of the economy. From that important aspect, he closed his speech by indicating the importance of an approach that emphasizes “maximum well-being with maximum diversity,” rather than the traditional values of “maximizing happiness for the many,” and “creating a future living together” with IOWN-based data sharing.

5.1 IOWN Pick Up

The spotlight was placed on NTT’s LLM tsuzumi, which was announced in November 2023. It brings together R&D results that NTT has been cultivating in natural language processing and other technologies, and has demonstrated a range of communication possibilities with language and beyond.

(1) The “tsuzumi” Large Language Model

The ‘“tsuzumi” Large Language Model’ exhibit presented to the world NTT’s long-awaited tsuzumi technology, which has been attracting much attention. It described the technology and introduced its main features as “lightweight and proficient in Japanese,” and “flexible customization.” Worth noting was that tsuzumi is lightweight enough to operate on a single GPU in a local environment. By increasing the quality and quantity of language-training data, it has achieved the highest performance in Japan while also greatly reducing power consumption and operating costs. The exhibit also showed that tsuzumi was able to derive correct responses by having smaller, specialized LLMs cooperate rather than having a single large LLM, as with earlier LLMs. Another major feature is that tsuzumi has a generalized LLM as a base, and “adapters” can be applied to it, enabling flexible customization for a customer’s requirements or characteristics of an industry. We can anticipate these features being utilized in a wide range of applications (Fig. 1).

Fig. 1. The “tsuzumi” Large Language Model.

(2) “tsuzumi” for understanding graphic documents

LLM tsuzumi has also been expanded to communication beyond language. In “tsuzumi for understanding graphic documents,” tsuzumi demonstrated abilities as though the LLM could actually “see.” In contrast to earlier LLMs based on text data, tsuzumi is able to understand a wide range of graphical and hand-written materials and derive correct responses. A world in which LLM’s can communicate with people using language, even in non-digital environments and conditions, is within sight (Fig. 2).

Fig. 2. “tsuzumi” for understanding graphic documents.

(3) “tsuzumi” with hearing and speaking abilities

LLM tsuzumi has also been expanded to utilize communication with sound. In “tsuzumi with hearing and speaking abilities,” it is able to obtain information by “hearing” voices and other sounds (Fig. 3). By capturing non-linguistic information that is not represented in text, such as tone, physical condition, or emotion, it demonstrated the ability to communicate more intimately with people. It introduced how tsuzumi is not limited to vision and has expanded to communication using the five senses and to other means beyond language as shown in other exhibits as well.

Fig. 3. “tsuzumi” with hearing and speaking abilities.

5.2 IOWN Now

In “IOWN Now,” the current state of IOWN was exhibited, as it advances from networks into computing. NTT research results and elemental technologies that are starting to be used in various fields were described, showing the major impacts they are having on society.

(1) Photonic and electronic device technology for All-Photonics Networks

“Photonic and electronic device technology for All-Photonics Networks,” presented various advanced devices that support APN and IOWN 2.0. Various NTT Laboratories technologies were gathered, such as multi-core fiber, wavelength modulation devices, 1-Tbps-class devices, and ultra-wide-band optical switches, and these research results are behind implementation of high capacity, low latency and low-power in APN (Fig. 4).

Fig. 4. Photonic and electronic device technology for All-Photonics Networks.

(2) Photonics-Electronics Convergence devices

The “Photonics-Electronics Convergence devices” exhibit presented R&D utilizing optics in even smaller components of networks, introducing compact photo-electric devices that can be placed in large-scale integrated (LSI) circuits. As IOWN progresses from networks to computing, this technology will be used to apply mechanisms for converting electrical signals to optical signals connecting progressively smaller components, from between boards to between semiconductor packages and then within semiconductor packages. It will contribute greatly to progress with IOWN high-capacity, low-latency and low power on the roadmap for the future (Fig. 5).

Fig. 5. Photonics-Electronics Convergence devices.

(3) IOWN APN for remote control and grasp of the on-site environment

The “IOWN APN for remote control and grasp of the on-site environment” exhibit presented and dynamically demonstrated the potential for the IOWN features of high capacity, low latency, and low power to be utilized to solve social issues in a more tangible way. Fine remote operation was made possible by using the APN to transmit high-definition video with low latency from a site, and to provide real-time control feedback from a human operator. It demonstrated the usefulness of APN in contributing to addressing various issues in the construction industry, such as personnel shortages and ensuring safe work environments for technicians (Fig. 6).

Fig. 6. IOWN APN for remote control and grasp of the on-site environment.

5.3 IOWN Evolution

“IOWN Evolution” introduced everyone to a future vision of what is close to becoming reality, opened up through advances in IOWN occurring daily. It showed NTT R&D to create solutions for various social issues and to realize well-being through application of IOWN.

(1) On-demand APN and In-network video processing

The “On-demand APN and In-network video processing” exhibit displayed implementation of a near-real-time broadcasting and distribution service and interactive video experience. This redefines video distribution utilizing two technology elements: a technology providing on-demand optical paths and a technology that can process video in its transmitted form. A video processing demo was presented at the venue using news and video introducing cultural properties (Fig. 7), and showed how the camera video was delivered to the viewer without loss of time or information, but also that the viewer was now able to select which of the cameras’ video to view. The fact that it can be controlled from the communications side using a Super White Box rather than a specialized broadcaster device also made a strong impression.

Fig. 7. On-demand APN and In-network video processing.

(2) Environment monitoring with optical fiber

IOWN is also being developed to utilize existing equipment, creating new value. The “Environment monitoring with optical fiber” exhibit introduced technology that uses optical cable installed underground as a sensor to analyze vibration. It demonstrated the world’s first example of making decisions from a distant location regarding snow removal in a heavy snowfall area. It showed the prospects for using optical fiber installed in many areas around the world for more than just communication and to contribute to solutions for various regional issues by using it as infrastructure for environment maintenance (Fig. 8).

Fig. 8. Environment monitoring with optical fiber.

(3) Hyperspectral analysis technology for different views of reality

To understand the world beyond the human perception, the “Hyperspectral analysis technology for different views of reality” exhibit introduced a technology that changes coloring and makes predictions from images captured with a hyperspectral camera. The exhibit showed the potential for giving additional value to earlier images and video by visualizing information that cannot be captured by the human eye. The live demo also showed the case of using it in the entertainment field, by shining light over a wide area on the clothing at the 2023 Paris Collection to change the appearance of color in the clothing (Fig. 9). The neighboring exhibit, “Hyperspectral imaging with a metalens and AI,” displayed an example of visualizing the sweet parts of fruit by overlaying visual information on sugar content over images of the fruit (Fig. 10), and introduced many application examples. By accurately visualizing various colors captured in hyperspectral images, various visual representations and communication of information should be possible using color.

Fig. 9. Hyperspectral analysis technology for different views of reality.

Fig. 10. Hyperspectral imaging with a metalens and AI.

(4) XR Sports space generation to reproduce the real world’s experience

“XR Sports space generation to reproduce the real world’s experience” exhibited technologies to perceive virtual space with all five senses, and not just vision. It presented “XR Sports Space” technology, which is an ultra-realistic meta-based space that provides a realistic virtual sports experience using time-change, sound, vibration, and other information (Fig. 11). Also presented was a dynamic demonstration that attracted much attention, introducing the feeling of participating in a bicycle race using technology that reproduces video, audio, and tactile sensation just like actually being there (Fig. 12). This brings us closer to a world in which viewers can participate in an international tournament being held in a far-away place and “race” with the contestants in real time.

Fig. 11. XR Sports space generation to reproduce the real world’s experience.

Fig. 12. Bicycle race demo exhibit using space reproduction technology.

5.4 IOWN Future

Exhibits in “IOWN Future” introduced initiatives related to improving well-being for people in society and developing new solutions to large scale social issues at a global level with NTT technologies.

(1) Artificial photosynthesis

Achieving carbon neutrality has become a global issue and “Artificial Photosynthesis” exhibits some R&D focusing on the workings of plants as they grow. It exhibited a technology being developed to reduce CO₂ using “high-quality semiconductor photocatalysts,” one of NTT’s strengths, by adding a protective layer to suppress deterioration (Fig. 13). This technology has enabled NTT to achieve world-class performance, with “continuous operation of more than several hundred hours.” The exhibit introduced conditions producing real photosynthesis and generated much excitement and anticipation for implementation (Fig. 14).

Fig. 13. Artificial photosynthesis.

Fig. 14. Artificial photosynthesis presentation demo.

(2) IOWN and future car

There is a range of initiatives and R&D being conducted around the world on automobiles and some of this R&D is looking to the future. The “IOWN and future car” exhibit featured applications of IOWN optical transmission technology, opto-electric integration technology, and computing. These will be essential in realizing “future cars” as they are being discussed at the IOWN Global Forum, to solve social issues and also to increase well-being. Of course they will realize a safer, more-enjoyable in-car space while fully utilizing information from cameras and sensors in all directions to ensure safe and secure driving, but they will also be equipped with more-advanced features using generative AI and low-energy computing, so that vehicles will evolve from being just a means of transport into a “lifestyle support partner” for a comfortable and sustainable future. This exhibit also attracted much attention (Fig. 15).

Fig. 15. IOWN and future car.