Feature Articles: R&D and Commercialization of NTT’s Large Language Model “tsuzumi”

Vol. 22, No. 8, pp. 31–37, Aug. 2024. https://doi.org/10.53829/ntr202408fa3

Commercialization of NTT’s LLM “tsuzumi”

Naoyuki Shibuya, Ichiro Kudo, Masahiro Kaido,
and Shunichiro Yamamoto

Abstract

NTT Human Informatics Laboratories has been researching and developing NTT’s large language models called “tsuzumi,” and on March 25, 2024, NTT held a press conference to announce the launch of tsuzumi commercial services. This article introduces the solution menu, partner program, members forums, and other features of tsuzumi as presented at the press conference.

Keywords: LLM, tsuzumi, commercial use

PDF

1. Introduction

NTT’s large language models (LLM) “tsuzumi,” which is now being researched and developed at NTT Human Informatics Laboratories, has been receiving good reviews since its announcement in November 2023 with more than 500 inquiries being received from a variety of companies and local governments. A press conference was later held on March 25, 2024 announcing the launch of commercial services. This article introduces the tsuzumi usage domain and usage applications, solution menu, partner program, members forums, and other features of tsuzumi as presented at the press conference.

2. Usage domain and usage applications

Among the inquiries that have been received about tsuzumi from a variety of companies and local governments, two-thirds have expressed an interest in training an LLM on in-house data and using a customized LLM. In other words, they would like to construct an LLM on their own premises (on-premise development) so that it can be trained in a closed and secure environment. Inquiries on introducing an LLM have also been received from a wide range of industries, many of which are from sectors such as manufacturing, local governments, and finance that handle highly confidential data.

We expect tsuzumi to be used in various types of applications. These include improving the customer experience (CX) at customer contact points, such as call centers, and improving the employee experience (EX) by streamlining in-house tasks such as automatically creating/summarizing the minutes of a meeting and creating a question and answer (Q&A) list from a business manual, as well as automating information technology (IT) operations and developing software.

3. Solution menu

Against the above background, tsuzumi commercial services that combine three usage environments and three types of solutions were announced on March 25, 2024.

3.1 Usage environments

Since tsuzumi is a lightweight LLM model, it can be used on premise at the customer’s office without having to construct a large-scale hardware environment or via a private cloud operated by an NTT Group datacenter. A customer may also select a public cloud as desired tailored to the way that tsuzumi is to be used. Using tsuzumi in an on-premise environment makes it possible to train the LLM safely on in-house data without having to send that data outside the company (Fig. 1).


Fig. 1. Usage environments.

3.2 CX solutions

CX solutions aim to improve CX at all types of customer contact points such as contact centers, brick and mortar stores, and e-commerce sites. We introduce two specific CX solutions.

The first is operator support for contact centers (Fig. 2). After a customer has been helped at a contact center, there is a need for after-call work such as summarizing the help given and registering necessary information in the system. This CX solution makes it possible to automate the summarizing of a completed call, entering of knowledge for a frequently asked question (FAQ) page, etc., thus reducing the time taken up by after-call work. It can also help improve the quality of customer service by finding candidate answers for a variety of customer inquiries from knowledge data stored by tsuzumi and giving the operator advice in real time, etc.


Fig. 2. Operator support at contact centers.

Another CX solution is virtual concierge (Fig. 3). A virtual concierge is a digital human that provides customer service. A digital human expresses human-like behavior that combines the appearances of real persons or an appearance based on a model tailored to the usage scenario and by applying motion generative artificial intelligence (GenAI). Combining technologies such as advanced automated replies, speech synthesis, and facial-expression analysis through tsuzumi in this way can provide personalized attention during a consultation with a customer while observing the customer’s information, facial expressions, etc. Thus, a virtual concierge shows promise for use in customer service at a variety of company counters and online contact points.


Fig. 3. Virtual concierge.

3.3 EX solutions

EX solutions are for automating and increasing the efficiency of common analog tasks in a variety of industries. These tasks include creating and summarizing the minutes of a meeting, searching through business manuals, creating a Q&A list, searching for products, supporting the preparation of reports, supporting the creation of application forms, checking internal rules and regulations, and conducting a legal check.

EX solutions provide employee support on the basis of industry knowledge and the customer’s proprietary business knowledge by providing an employee-friendly user interface combined with a specialized LLM trained on industry knowledge and in-house documents. They help improve job satisfaction, productivity, and EX at the target client. For example, an EX solution for local governments would enable personnel at an administrative counter to easily search for answers even for a wide range of inquiries without making residents wait. This would be accomplished by using a specialized LLM trained on government tasks in conjunction with the business processes carried out at that administrative counter.

Since tsuzumi can be directly constructed within a network storing highly confidential data inside a local government, it can be trained securely on diverse types of data unique to government services without having to send that data outside while generating answers tailored to the processes of an administrative counter.

3.4 IT-operations support solutions

To enable a variety of work styles, such as remote work and “workcations,” in a secure manner, the volume of work in IT operations, in-house information-system departments, and at help desks has recently been increasing. At the same time, the work is becoming more advanced and complicated, and the shortage of human resources is becoming increasingly serious.

IT-operations support solutions support the streamlining and automation of IT operations within a company and the strengthening of business continuity combined with the unique expertise of the NTT Group in providing IT services. They can be used, for example, in security measures. The threats of cyber attacks, such as malware, are increasing yearly and the attacks are becoming increasingly sophisticated while the number of security personnel that can respond to these threats is decreasing.

The NTT Group has had a number of achievements in the field of security such as defending against cyber attacks directed at international events. In fact, by using these solutions that detect security threats and respond automatically using NTT proprietary knowledge, 95% of these achievements involved cases of automatically responding without human intervention by automatically detecting access-log records that suggest cyber attacks. The remaining 5% involved access-log records of new, previously unknown cyber attacks. These records had to be analyzed and dealt with from various angles by personnel specialized in security, which took time to carry out.

By having tsuzumi learn the cyber-defense expertise of the NTT Group and the customer’s unique network environment then using it as an LLM specialized in security operations, tsuzumi can give advice on methods for dealing with new attacks that had been escaping from the filters of existing AI automated solutions. These IT-operations support solutions can both mitigate risks and reduce the labor of specialized personal by reducing the range of the role played by people in security measures (Fig. 4).


Fig. 4. Example of security response.

3.5 Total support for introducing tsuzumi

For a company introducing tsuzumi solutions, the NTT Group can provide total support for a series of processes from initial studies to development, installation, and operation.

With GenAI, the desired response accuracy differs in accordance with the training data used, number of tunings executed, and target use case. The NTT Group supports agile development in accordance with the circumstances by revising training data and varying the tuning method while testing. It also supports customized maintenance toward further improvements in CX and EX after tsuzumi has been installed and put into use.

Before customized development of an LLM targeted for installation, an AI risk evaluation can be conducted using expertise from the AI governance initiative undertaken by the NTT Group from 2019. Customers can therefore use GenAI including tsuzumi in a safe and secure manner.

4. Partner program

Recruitment of three types of partners for different objectives began in May 2024 (Fig. 5).


Fig. 5. Partner program.

The first type is a solution partner targeting companies having application services. This type of partner will be able to use some of tsuzumi’s application programming interfaces (APIs) free of charge to make it easy to study how tsuzumi can be incorporated behind the scenes in solution services. By incorporating tsuzumi functions in existing customer relationship management (CRM) services and business application services provided by a partner company, and by enabling startup companies that are studying new services using GenAI to try out tsuzumi, we hope to increase the added value of new services and existing solution services.

The second type is a model partner targeting companies having data specific to certain industries. The aim is to train tsuzumi on industry/business-specific data, promote the joint development of specific models for solving industrial problems, and enable joint marketing and expansion of sales.

The third type is an integration partner targeting companies such as system integrators engaged in system integration from consulting to construction to improve the digital transformation (DX) and CX of their clients. By becoming an integration partner, a company can obtain the right to use “industry/business-specific models” developed, for example, by a model partner (second type of partner), and by sharing expertise, such as use cases, partner companies can jointly deliver solutions using tsuzumi to even more customers.

By establishing partner forums, we will create opportunities for these three types of partners to develop skills in using tsuzumi and share knowledge with each other with the aim of creating new value together. The development of tsuzumi usage skills will not be limited to urban environments—we will also develop programs for supporting re-skilling and DX in non-urban regions by matching urban resources and non-urban work. DOCOMO gacco, Inc., which provides one of the largest online video-training services in Japan, plans to launch a program promoting improvement in DX skills including the use of tsuzumi in the summer of 2024 while providing specific support for the Noto Peninsula, which was struck by a strong earthquake in January 2024. In short, by participating in this program, participants can become directly involved in supporting the Noto Peninsula while developing tsuzumi usage skills.

5. Members forums

Members forums targeting customers who are using tsuzumi will also be held (Fig. 6). They will take place in both digital space and real space and enable customers who are using tsuzumi to share use cases and knowledge and ask questions such as “What kind of business are you using tsuzumi for?” “What kind of company data are you training tsuzumi on?” and “What kind of effects have been obtained?” Targeting customers who are using tsuzumi, forums in digital space will enable use cases and other types of information to be shared via a membership website that only registered members can enter. There are also plans for programs to be held at NTT Group co-creation spaces where case studies can be presented to other members in person and members can network with each other.


Fig. 6. Members forums.

6. Global expansion

The tsuzumi LLM is already attracting interest not only from customers in Japan but also from global customers. To promote the use of GenAI from Japan on a global basis, we are preparing a GenAI global platform on which customers will be able to use tsuzumi safely and securely. Specifically, we are incorporating tsuzumi in solutions that are being provided globally by NTT DATA and providing these solutions to customers via datacenters that are being rolled out globally. Our aim is to provide a multilingual environment in which global customers in a variety of industries can use tsuzumi in a safe and secure manner.

Naoyuki Shibuya
Director, Global Marketing Strategy Group, Market Planning and Analysis Department, Research and Development Market Strategy Division, NTT Corporation.
He received a B.A. in economics from Keio University, Tokyo in 2007 and MBA from GLOBIS University, Tokyo in 2020. He joined NTT DOCOMO in 2007.
Ichiro Kudo
Director, Global Marketing Strategy Group, Market Planning and Analysis Department, Research and Development Market Strategy Division, NTT Corporation.
He received a B.E. in electrical engineering in 1998 and M.E. in informatics in 2000 from Kyoto University. He joined Business Communications Headquarters of NTT WEST in 2000. He joined NTT in 2014.
Masahiro Kaido
Senior Manager, Global Marketing Strategy Group, Market Planning and Analysis Department, Research and Development Market Strategy Division, NTT Corporation.
He received a B.A. in economics from Aoyama Gakuin University, Tokyo in 2000 and joined NTT WEST the same year.
Shunichiro Yamamoto
Senior Manager, Global Marketing Strategy Group, Market Planning and Analysis Department, Research and Development Market Strategy Division, NTT Corporation.
He received a B.E. in information systems engineering from Osaka University in 1997 and M.E. in information science from Nara Institute of Science and Technology in 1999. He joined NTT in 1999.

↑ TOP