Commercialization of NTT’s LLM “tsuzumi”

3.1 Usage environments

Since tsuzumi is a lightweight LLM model, it can be used on premise at the customer’s office without having to construct a large-scale hardware environment or via a private cloud operated by an NTT Group datacenter. A customer may also select a public cloud as desired tailored to the way that tsuzumi is to be used. Using tsuzumi in an on-premise environment makes it possible to train the LLM safely on in-house data without having to send that data outside the company (Fig. 1).

Fig. 1. Usage environments.

3.2 CX solutions

CX solutions aim to improve CX at all types of customer contact points such as contact centers, brick and mortar stores, and e-commerce sites. We introduce two specific CX solutions.

The first is operator support for contact centers (Fig. 2). After a customer has been helped at a contact center, there is a need for after-call work such as summarizing the help given and registering necessary information in the system. This CX solution makes it possible to automate the summarizing of a completed call, entering of knowledge for a frequently asked question (FAQ) page, etc., thus reducing the time taken up by after-call work. It can also help improve the quality of customer service by finding candidate answers for a variety of customer inquiries from knowledge data stored by tsuzumi and giving the operator advice in real time, etc.

Fig. 2. Operator support at contact centers.

Another CX solution is virtual concierge (Fig. 3). A virtual concierge is a digital human that provides customer service. A digital human expresses human-like behavior that combines the appearances of real persons or an appearance based on a model tailored to the usage scenario and by applying motion generative artificial intelligence (GenAI). Combining technologies such as advanced automated replies, speech synthesis, and facial-expression analysis through tsuzumi in this way can provide personalized attention during a consultation with a customer while observing the customer’s information, facial expressions, etc. Thus, a virtual concierge shows promise for use in customer service at a variety of company counters and online contact points.

Fig. 3. Virtual concierge.

3.3 EX solutions

EX solutions are for automating and increasing the efficiency of common analog tasks in a variety of industries. These tasks include creating and summarizing the minutes of a meeting, searching through business manuals, creating a Q&A list, searching for products, supporting the preparation of reports, supporting the creation of application forms, checking internal rules and regulations, and conducting a legal check.

EX solutions provide employee support on the basis of industry knowledge and the customer’s proprietary business knowledge by providing an employee-friendly user interface combined with a specialized LLM trained on industry knowledge and in-house documents. They help improve job satisfaction, productivity, and EX at the target client. For example, an EX solution for local governments would enable personnel at an administrative counter to easily search for answers even for a wide range of inquiries without making residents wait. This would be accomplished by using a specialized LLM trained on government tasks in conjunction with the business processes carried out at that administrative counter.

Since tsuzumi can be directly constructed within a network storing highly confidential data inside a local government, it can be trained securely on diverse types of data unique to government services without having to send that data outside while generating answers tailored to the processes of an administrative counter.

3.4 IT-operations support solutions

To enable a variety of work styles, such as remote work and “workcations,” in a secure manner, the volume of work in IT operations, in-house information-system departments, and at help desks has recently been increasing. At the same time, the work is becoming more advanced and complicated, and the shortage of human resources is becoming increasingly serious.

IT-operations support solutions support the streamlining and automation of IT operations within a company and the strengthening of business continuity combined with the unique expertise of the NTT Group in providing IT services. They can be used, for example, in security measures. The threats of cyber attacks, such as malware, are increasing yearly and the attacks are becoming increasingly sophisticated while the number of security personnel that can respond to these threats is decreasing.

The NTT Group has had a number of achievements in the field of security such as defending against cyber attacks directed at international events. In fact, by using these solutions that detect security threats and respond automatically using NTT proprietary knowledge, 95% of these achievements involved cases of automatically responding without human intervention by automatically detecting access-log records that suggest cyber attacks. The remaining 5% involved access-log records of new, previously unknown cyber attacks. These records had to be analyzed and dealt with from various angles by personnel specialized in security, which took time to carry out.

By having tsuzumi learn the cyber-defense expertise of the NTT Group and the customer’s unique network environment then using it as an LLM specialized in security operations, tsuzumi can give advice on methods for dealing with new attacks that had been escaping from the filters of existing AI automated solutions. These IT-operations support solutions can both mitigate risks and reduce the labor of specialized personal by reducing the range of the role played by people in security measures (Fig. 4).

Fig. 4. Example of security response.

3.5 Total support for introducing tsuzumi

For a company introducing tsuzumi solutions, the NTT Group can provide total support for a series of processes from initial studies to development, installation, and operation.

With GenAI, the desired response accuracy differs in accordance with the training data used, number of tunings executed, and target use case. The NTT Group supports agile development in accordance with the circumstances by revising training data and varying the tuning method while testing. It also supports customized maintenance toward further improvements in CX and EX after tsuzumi has been installed and put into use.

Before customized development of an LLM targeted for installation, an AI risk evaluation can be conducted using expertise from the AI governance initiative undertaken by the NTT Group from 2019. Customers can therefore use GenAI including tsuzumi in a safe and secure manner.