2020 Town—Developing MACHINAKA Service, a Device Integration Service that Utilizes Artificial Intelligence Technology

1. Introduction

Currently, most MACHINAKA (in the city) services are provided with human labor. However, with the progress of artificial intelligence (AI) technology, the need for human intervention is gradually decreasing. Amazon.com, Inc., a well-known example of a logistics company in the US, has already automated part of its operations using AI and robots, from management of its logistic services to product delivery [1]. The increased use of robots in place of humans has also been seen in the customer service industry recently with the growth of AI [2, 3, 4]. The communication robot “Pepper” of SoftBank Corporation is a typical example of a robot replacing humans in a MACHINAKA service industry. It is an example of leveraging communication robots that can interact intelligently with humans through words and actions using AI. The use of communication robots is expected to lead to a reduction in the cost of human labor as well as greater effectiveness of advertising presented using such robots.

As mentioned, we are starting to see communication robots interacting with people and replacing humans in the customer service field. However, it can be said that most of these interactions are still very limited and are centered on routine one-to-one dialogues with customers, for example, to explain goods and services at a reception counter or provide guidance at facilities. This limited use is presumably because the main feature of communication robots is interacting with people using voice and gestures.

At the NTT laboratories, however, we believe that it is possible to expand the scope of applications for communication robots beyond one-to-one customer service situations by combining communication robots with a number of other devices and systems to constitute a device integration service. We describe here the advances that can be achieved when a device integration service centered around communication robots is realized.

As previously mentioned, the customer service industry is increasingly using communication robots to provide guidance or explain facilities or services using voice. This is occurring more and more frequently in the hospitality industry such as at hotels and reception centers, as well as in the banking and retail industry to provide information to customers. However, combining communication robots with images or videos shown on a display will greatly increase the clarity of information that can be conveyed. Also, connecting communication robots with health monitoring devices such as blood pressure monitors opens up the possibility of providing health advice to users [5].

It is also possible for moderators of recreational activities to use a communication robot as a support device by connecting it with a karaoke system. Furthermore, the use of communication robots in cooperation with various devices placed inside a facility or outdoors within a target area makes it possible to not only provide information and guidance to customers, but also to improve the convenience of getting around in the city and to increase the number of visitors and facilitate visitor movements as well.

In this way, we believe that we can assist peoples’ understanding and encourage their behavior and thus stimulate numerous activities within the city by linking various devices to communication robots. In the following sections, we introduce R-env^®, the key technology to make all this happen.

2. R-env, technology supporting device and service integration

For more than 40 years, NTT has carried out research and development (R&D) on speech and audio processing technologies and natural language processing technology. NTT is also a world leader in image processing technology, knowledge processing technology, and various other media processing technologies. Efforts are underway to achieve a natural dialogue between communication robots and humans by making use of these technologies. To expand the role of communication robots and increase their existential value, we started R&D on R-env in 2015. R-env enables robots to be easily connected with other external devices and systems (Fig. 1).

Fig. 1. Overview of R-env^®, a platform for expanding the human potential through integration of devices and robots.

R-env was developed as a part of ongoing research on interaction technology aimed at expanding the possibilities of humans. This technology enables easy connection and integration of various devices such as communication robots. By linking these devices, this technology will make it possible for communication robots to accurately understand the situation in which a person is placed, provide guidance to the person in accordance with the situation, and prompt the person to adopt a specific behavior or level of awareness.

With R-env, anyone can develop a device integration service using graphical user interface (GUI) screens on a browser. This is done by creating a state transition diagram, or in other words, by combining actions specified for each device connected to R-env, and the condition for the transition to the next action. R-env can also be used to easily create a complex device integration service by reusing and operating multiple state transition diagrams in parallel. Also, services developed on the GUI can be instructed to continue on with their operation even after closing the browser by keeping it on hold in its execution state. Furthermore, when a new device is added, it can be easily recognized, registered, and prepared for use in the R-env service by simply transmitting the predefined JSON (JavaScript Object Notation) format using WebSocket before use.

3. Device integration service using R-env in MACHINAKA (trial case)

We have been conducting trials with NTT Group companies and their external partners to extract the needs and identify any technical problems of device integration services that utilize R-env [5, 6]. In particular, at the end of July 2016, we initiated a joint demonstration experiment in collaboration with five major NTT operating companies. The purpose of the experiment is to evaluate the effectiveness and acceptability of device integration services in a variety of fields and industries in the city (Fig. 2) [7]. In the following section, we introduce a trial case staged in the Shinjuku area.

Fig. 2. Examples of device integration services by NTT Group companies.

4. Trial in Shinjuku

The number of tourists visiting Japan from abroad has increased rapidly. This is evident from the increase in the government’s target number of foreign tourists to Japan, which was raised from 20 million per year to 40 million per year by 2020 at a tourism related meeting chaired by Prime Minister Shinzo Abe on March 30, 2016 [8].

The driving force behind this is the dramatic increase in the number of tourists from China, which doubled in 2015 compared to the previous year. Tourists from China now account for the largest number of foreign tourists to Japan and greatly exceed the number from Taiwan, the second-largest group [9]. Their primary purpose for visiting Japan is shopping, as symbolized by the expression baku-gai, or shopping spree.

Popular areas for Chinese tourists, in addition to well-known tourist destinations, are Ginza and Shinjuku, where there are many shops [10]. Places such as Shinjuku Golden Gai and the Kabukicho neighborhood are also favored by foreign tourists in Shinjuku [11]. Therefore, from the viewpoint of regional development, services in Shinjuku are required not only for local residents but also for people coming from outside Shinjuku, including tourists.

We carried out two joint demonstration experiments utilizing a device integration service targeting the Shinjuku area. The first one was a joint experiment conducted by three groups: Takashimaya Company, Ltd., which is engaged in the retail business, NTT Communications, and the NTT laboratories. In the experiment, we used a device integration service to inform and guide customers to event venues using multiple robots placed in a commercial facility (department store) from August 3 to August 14, 2016. In particular, robots were placed in the following locations based on the flow of pedestrian traffic.

At each location, a device integration service composed of an external sensor linked to a communication robot provided smooth induction to the venue by arousing customer interest in the event through dialogue with customers and by providing floor guidance of the department store (Fig. 3).

Fig. 3. Service image in Shinjuku Takashimaya.

The second case is an effort targeting the broad outdoors and is not limited to the indoor space of a facility. It is a joint trial being conducted by the Shinjuku branch of the Tokyo Chamber of Commerce and Industry, the Shinjuku Tourism Promotion Association, NTT EAST, and the NTT laboratories. The trial provides a device integration service consisting of robots and digital signage at multiple locations within a radius of approximately 1 km from Shinjuku Station. In particular, communication robots installed in shops and in tourist facilities in the area provide access information about the respective shops and facilities by utilizing the public wireless local area network service Shinjuku Free Wi-Fi provided by Shinjuku ward.

Communication robots are also playing an important role to increase the number of visitors to the area by providing attractive services such as a robot stamp rally that utilizes multiple robots in different places that cooperate with one another. We are in the process of validating how these services help to increase the number of visitors and facilitate visitor movements (Fig. 4).

Fig. 4. Service image in Shinjuku area.

5. Future development

We are promoting trials to apply device integration services to a wide variety of fields and industries in addition to the cases introduced here; some examples of where these services may be beneficial include long-term care facilities and tourist facilities. Furthermore, in parallel with the joint demonstration experiment, we are also working to expand the application of R-env by broadening cooperation and promoting joint events with hardware manufacturers and service developers [12].

We are also working to spread and improve the device integration service by expanding four initiatives:

We aim to create new business opportunities using robots by enhancing and promoting the value of our MACHINAKA service using device integration services, and by improving the performance of the associated element technology based on the verification results obtained from the joint demonstration experiment.

References

	Takayoshi Mochizuki Senior Research Engineer, Networked Robot & Gadget Project, NTT Service Evolution Laboratories. He received an M.A. in human sciences from Waseda University, Tokyo, in 2001. He joined NTT in 2001 and moved to NTT Service Evolution Laboratories in 2014. His research interests include human-robot interaction and human-machine interfaces. He is currently engaged in R&D on non-verbal interactions that encourage new behavior and awareness of people by utilizing robots and other devices and AI.
	Manabu Norota Senior Manager, Supply Chain & Service Industry Group, NTT Communications Corporation. He received a B.A. in economics from Fukushima University in 1996. He joined NTT Communications in 1996. He is currently engaged in sales promotion.
	Kousuke Suga Associate Manager, New Business Development Headquarters, NTT EAST Corporation. He received a B.A. from Rikkyo University, Tokyo, in 1996. He joined NTT EAST the same year. He is currently engaged in developing new services for elderly people to promote their use of the Internet.