Co-Innovation Leading to the Future

1.1 Big data: Predictive

NTT’s R&D focuses on predictive big data. This is not simply a vague prediction of what will happen in the future. The aim is to predict not just what will happen, but when and where something will happen (Fig. 2). The key technology for this is spatio-temporal multidimensional data analysis. A feature of this technology is that it models time and space on multidimensional axes and uses this model to predict the time of occurrence. Since one of the challenges in handling big data is how to protect personal information, we have devised a way to analyze aggregate data without first decomposing the data. By combining the spatio-temporal multidimensional data analysis with Jubatus, which is a real-time machine learning engine, and Grapon, a kind of graph analysis technology, we are making it possible to analyze flows of people, things, and information in order to predict what will happen and when and where it will happen. For example, we plan to apply this capability to control the flow of people or vehicular traffic in society, and to control communication traffic and predict failures in the communication infrastructure.

Fig. 2. Big data: Predictive.

What motivated us to work on this issue was the Great East Japan Earthquake that struck in 2011. In a bid to escape from the tsunami, many people leapt into their cars, and most of them headed in the same direction. This caused traffic congestion and resulted in the deaths of many people. What struck home then was the need to guide people in a totally, rather than individually, optimized way. We will explore the application of our analysis technologies in this area.

1.2 Security: By design

We have focused on four security areas: data security with an emphasis on cryptographic theory, network security centering on malware analysis, proactive security to detect signs of threats before they materialize, and reactive security to respond to threats in real time. In addition to continuing these efforts, we will start to focus on security quality management. The intent is to incorporate preventive measures at the planning and development stages of networks and systems (Fig. 3).

Fig. 3. Security by design.

Let us consider as an example the case of applying security quality management to network design. The security task begins with identifying what to protect in the entire network. Then we analyze threats, assess risks, study countermeasures, and incorporate the results into the network and system design. Incorporating security measures at the design stage has been shown to reduce security costs (and risks) to one-hundredth that if measures were to be taken during the operation stage.

1.3 Cloud: Multilateral

The NTT Group considers global clouds one of the pillars of its growth and has been developing cloud technology that enables users to access clouds safely at an affordable price, and to migrate their tasks from existing computers on their own premises to clouds and to do so flexibly and rapidly in a way suited to their requirements. Going forward, we will develop new technology that enables users to engage in multilateral collaboration (Fig. 4). If enterprises are to create a wide range of marketable services and products in the years to come, it is extremely important for them to develop businesses that involve collaboration or integration of multiple industries instead of confining their initiatives to their own industry. NTT’s R&D will enable a diverse range of enterprises to create new products and services by analyzing data in collaboration with them rather than these enterprises simply entrusting their data to some providers. We intend to achieve this by developing a mechanism that enables multiple customers to share applications and data within a datacenter or across datacenters of the NTT Group, and by combining the big data analysis with video development technologies owned by the NTT Group or external enablers.

Fig. 4. Cloud: Multilateral.

1.4 Network: Dependable and elastic

It is about time that we conceive a new network. There are two reasons for this, one of which is technical. With the dramatic increase in the performance of off-the-shelf servers, it is now possible to shift from using only carrier-specific devices to using inexpensive commodity devices when building networks and systems. The second reason is related to the carrier’s business. We need more dependable and elastic networks and systems. Having witnessed a dramatic change in the telecommunications business environment over the last decade, we consider that our networks and systems should be flexible enough to be able to adapt to a changing business environment down the road. In addition, if we are to collaborate with service providers as the value partner that customers continue to select, we need to respond rapidly, flexibly, and economically to their diverse range of requirements. We have named our activities to create such a network NetroSphere, a word combining “Netro,” which means learning from the history of the Internet and telecommunications to develop the future network, and “Sphere,” which is derived from “atmosphere.” We plan to create a space that can encompass the needs of customers and adapt to changes in our business environment by integrating our accumulated network technologies and expertise and using commonly available parts.

NetroSphere consists of three elements: the transport, the network functions for controlling the transport, and the operations of the entire network. Its key concept is separation and combination (Fig. 5). Although virtualization technology has strengthened separation of hardware and software, the focus of NTT’s research is on separation at a smaller component level.

Fig. 5. NetroSphere activities.

In the transport element, we will clearly separate optical and electrical parts to achieve all-optical transport. By minimizing electrical processing, we will reduce the number of devices used in networks, which in turn will reduce the number of devices subject to maintenance. In addition, we will implement network functionality in software. This will dramatically enhance our ability to use off-the-shelf hardware. We will advance to this stage rapidly by collaborating with other global carriers instead of attempting to undertake this project on our own.

It is also necessary to break network functions up into applications that can be treated as components and to provide capabilities common to multiple applications as a platform. The idea is to simplify applications. This will facilitate the introduction or modification of services. In addition, by making the platform a distributed one, we can achieve high reliability without incurring additional costs.

Our plan for operations is to provide an API (application programming interface) that has the capacity to operate the aforementioned components effectively and to respond rapidly to diverse requests from a variety of players. In addition, we will develop a proactive maintenance approach by which machine learning is used to detect signs of faults and quality degradation before they materialize.

1.5 Media and knowledge processing: Semantic and real

Our overall approach to communication is to consider the meaning of the particular communication from the communicator’s perspective, and to provide a high sensation of reality that far surpasses the experience of conventional viewing and listening. This will require video processing, speech and acoustic processing, and language processing. It is important to convey a sensation of reality in a way that matches human sensitivity instead of simply aiming to reproduce reality (Fig. 6). This means that we need to delve deeply into the very core of communication on the basis of communication science, taking into account how humans see and hear the things around them.

Fig. 6. Media/knowledge processing: Semantic and real.

Let me introduce some of the highlights of our research outcomes in the area of communication science. The first is what we call HenGenTo (transforming light) (Fig. 7). Just by projecting a special light, you can make a still picture appear animated. Our understanding is that when someone perceives an object, the brain takes in information about the object’s shape, color, and movement separately, and then combines these separate pieces of information. When a light that is related to movement based on this principle is projected onto an object, the still object looks as if it were animated. Another example is a robot that NTT used in its project called Can a robot enter the University of Tokyo? In this project, we gave the robot the task of sitting the English test, a challenge that completely stumps most computers. In a mock exam given in September 2014, our robot raised its deviation value from 41.0 to 50.5, exceeding the average human score in the exam for the first time.

Fig. 7. Light projection technology: HenGenTo.

We consider that R&D in the area of physics and its application to devices are also important for the advancement of ICT. One of our main accomplishments in this area last year was an all-optical RAM (random access memory) chip containing more than 100 bits. We will continue to pour our efforts into the pursuit of leading-edge R&D.

2.1 Resolving issues through open innovation

Whether we are talking about 2020 or not, NTT cannot resolve issues by itself. It is essential to undertake innovation with a variety of partners in the areas of both business and R&D.

For the last two years, we have been promoting open innovation, which can take two forms: working with homogeneous partners and working with heterogeneous partners (Fig. 8). The objective of the former is to dramatically raise performance in a specific area or to expedite development. The objective of the latter is to create new value that can be achieved only when we work with customers or partners in fields completely different from our own. We plan to increase collaboration with heterogeneous partners in our open innovation activities.

Fig. 8. Objectives of open innovation.

2.2 Specific activities

Some of our specific activities toward 2020 are introduced below.

(1) Providing stable and high-quality network services

Network traffic is not only increasing at a prodigious rate but also changing in ways we have never experienced before. Traffic can concentrate in a specific location in the network—just as foot traffic can in certain areas—or it can continue to move, as happens with the front-runners in a marathon race. NTT can contribute to managing such traffic by providing technology that makes it possible to handle high volumes of traffic efficiently with a small amount of equipment, which eliminates the need for a massive investment in lots of equipment. A case in point is proactive network control technology. Instead of observing a change in traffic linearly, this technology generates geographically distributed traffic forecasts by analyzing user behavior and traffic-generating events and controls the network accordingly. This network control makes it possible to handle high traffic volumes with minimal equipment (Fig. 9).

Fig. 9. Proactive network control technology.

An approach from a different perspective is control of the traffic itself. This approach has been adopted in collaboration with Dwango, a video delivery service provider. The objective is to enhance customers’ quality of experience by varying the bit rate when sending video signals. A high bit rate is used in a good communication environment, but it is lowered to a bit rate that is just good enough to avoid video interruption in an adverse communication environment. We conducted a small-scale evaluation experiment and found that interruptions to viewing were reduced dramatically from 33% to 2%, and the volume of transmitted data decreased by more than 10%.

(2) Network security measures

Cyber-attacks are expected to continue to increase in number. In view of the fact that all industries are connected to the Internet, this is not simply a problem in the field of communications; rather, it concerns all social infrastructures including those for finance, electric power, and transportation. As a communications carrier, NTT will of course strengthen the resilience of its networks to such attacks but will also need to intensify collaboration with other industries.

Let me introduce the concept of resilient security, which is a representative technology designed to thwart cyber-attacks. This technology automatically detects attacks based on communication patterns collected from the network and semi-automatically implements recovery actions appropriate for the specific type of attack (Fig. 10). This is effective for slow DoS (denial of service) and reflection DDoS (distributed DoS) attacks that have become problematic recently. We are refining the technologies required to counter these threats by monitoring and analyzing the latest attack patterns.

Fig. 10. Resilient security technology.

We have also taken the initiative in cultivating cybersecurity experts. Considering the wide range of cybersecurity-related tasks undertaken by NTT Group companies, we have classified human resources into three categories: those capable of undertaking cybersecurity management and providing consultation, those capable of undertaking cybersecurity operations, and those capable of developing systems with due attention to cybersecurity. Each category is divided into three levels of proficiency.

We also support cybersecurity education at universities. As the first initiative, we have provided funding for a class designed for graduate and undergraduate students at Waseda University. NTT’s malware analysts will give lectures on basic technologies and oversee practical exercises. An online course, gacco, which is operated by NTT DOCOMO, will offer the Ultra-Basic Course on Information Security starting in May 2015.

We intend to undertake these kinds of initiatives in different industries. We will work with other industries to provide multifaceted education that incorporates knowledge specific to each industry.

(3) Providing new experiences and creating a welcoming and hospitable atmosphere

We will work on creating a hospitable atmosphere by providing new and meaningful experiences that reflect a welcoming attitude to not only spectators and foreign tourists but also all people throughout Japan and around the world. In line with the strength of its technical development capabilities, the R&D arm of NTT will collaborate with a wide range of partners to convert technologies into business opportunities.

Let me introduce angle-free object search as an example of research in the field of video search, (Fig. 11). This technology is extremely robust against the angle from which a 3D (three-dimensional) object is photographed and can associate a captured image with reference images even if the number of relevant reference images is small. A potential application of this technology is searching for information displayed on signs. A reference image is linked in advance to a multilingual image (English, Chinese, etc.), information about the location of the sign, and information about the content of the sign. When a visitor from overseas directs the camera of his/her smartphone toward a sign at a station, the information on the sign is displayed in multiple languages. This is not a translation. Rather, the sign is linked to information that was prepared in advance. This arrangement makes it possible to provide information that is taken for granted by Japanese people but is invaluable for foreign visitors. We can create a truly hospitable atmosphere for guests from abroad by providing such easily overlooked information. This technology is undergoing a feasibility test near Tokyo Station. In particular, because it is very easy to get lost in underground areas, it will be useful if visitors can ascertain their current location just by directing the camera of their smartphones toward a sign or a pattern on the ceiling.

Fig. 11. Angle-free object detection.

In the field of high-definition video delivery, NTT collaborated with NHK to hold 8K super-high-definition public viewings of the 2014 FIFA World Cup in Brazil. This project used MPEG (Moving Picture Experts Group) media transport (MMT), which can deliver a number of HEVC (high-efficiency video coding) videos that are fully synchronized with each other. This enables you to view two fully synchronized videos. For example, while you are watching a soccer game on a public screen, you can use your tablet as a second screen and view the game from a different angle.

In the field of acoustics, we developed noise cancelation technology, which can extract a specific voice even in extremely noisy environments where the noise level exceeds 100 dB. This technology uses two general microphones. It extracts a specific voice based on differences in the voice level between the sounds from the two microphones. We have confirmed that the technology can reduce the noise level by a factor of 10,000. Even under an elevated railroad track, one of the noisiest places conceivable, speech recognition experiments using this technology recorded a success rate in excess of 90%. We have been collaborating since last year with Mitsubishi Heavy Industries to test the effectiveness of this technology in a factory. We have confirmed that the speech recognition works well, and conversations can be conducted normally even in this noisy environment.

Another achievement that evolved from noise cancellation technology is target microphone technology, which extracts a target sound clearly using a sharp rise in the sound as a clue in addition to determining the spatial noise distribution for each direction (Fig. 12). This technology will make it possible, for example, to extract the sound of a soccer player kicking the ball or the sound of bodies colliding in a sumo bout, thereby providing sound with a high sensation of reality to viewers.

Fig. 12. Target microphone.

Technology for interactive viewing of omnidirectional audio and video has been developed in collaboration with Dwango. It combines the above acoustic technologies with a 360-degree camera to enable users to experience the sensation of being on the field with the players. As users move their viewing direction, they can hear a stereo sound that is in sync with their movement.

Immersive telepresence is another challenging research project (Fig. 13). Information that represents the space and environment of a particular location is sent to a remote location. This information consists of object information, layer information (which represents the depth of an object), illumination information, and acoustic space information. The space and environment of the original location is reconstructed at the remote location from these pieces of information, enabling users there to feel as if they were at the actual location. We have named this project “Kirari!” and we plan to implement it for public viewing.

Fig. 13. Immersive telepresence.

In the field of materials, a number of partners have collaborated to create a B2B2X (business-to-business-to-X) service. NTT and Toray worked together to develop a sensing fabric called hitoe. Goldwin produced a shirt that incorporates hitoe. Panasonic developed a receiver/transmitter that picks up bioelectric signals from hitoe and transmits them. NTT DOCOMO collaborated with Runtastic to produce an app that uses these bioelectric signals. While hitoe is now used to measure the user’s heart rate and check the workload of the current sports activity, it can also be used to measure other signals such as myoelectric and electrocardiographic signals. We consider that by teaming up with other partners, we can expand the applications of hitoe to a wide range of fields such as athlete training and health management. In these areas, hitoe can be used to check muscle condition and even mental states and eventually provide more generalized medical support.

In the field of human information science, we aim to apply bioinformation measurement technology in sports training. This technology measures myoelectric states and represents the time pattern of an exercise by means of synthetic sounds. When it is applied to the training of a baseball pitcher, for example, it is possible to perceive intuitively through sound whether the pitcher bends his/her arm sufficiently or is overly cautious in throwing the ball. This audibilization is expected to motivate and raise the awareness of the player.