Approach to Network Science—Solving Complex Network Problems through an Interdisciplinary Approach

2.1 Space-related theory

Space-related theory deals with spatial information against the background of academic demands and social trends. Up to now, traffic theory has targeted stochastic behavior along a time axis, but in space-related theory, the focus is on a theory that targets stochastic behavior along a space axis. Integral geometry forms some of the foundation for this theory; given that the area and perimeter length of certain objects are known, the probability that their shapes overlap can be calculated. This principle can be used, for example, to calculate the probability that spatially and stochastically distributed objects overlap [3]. Application examples of this theory include the analysis of measurements taken by sensors and the design and control of networks that are robust to natural disasters. Constructing an actual physical network involves designing a network highly resistant to disaster by calculating the probability of being hit by a disaster based on a hazard map showing the likelihood of such an event (an earthquake, for example). This theory can also be applied to network control in a virtual network in which the logical configuration of the network can be controlled independently of its physical configuration. For example, the ever-changing probability of being hit by a disaster can be calculated based on forecasts of typhoons, torrential rains, or other extreme weather conditions, so that network servers can be switched to an arrangement with a low probability of being affected by such a disaster.

At NTT Network Technology Laboratories, we are also studying base-station design using homology. The use of conventional mathematical programming to calculate an optimal arrangement of sector antennas can result in an explosive number of combinations that hinders problem solving. Homology-based modeling, however, can drastically reduce computational complexity in such cases, enabling a solution to be found [4].

2.2 Machine learning and data mining

The combined approach of machine learning and data mining enables the discovery of features and regularities in large volumes of data. For our purposes, the target of analysis is traffic collected from the network and data related to service quality. This type of data is not limited to numerical data. It can include, for example, atypical messages in logs generated by equipment and devices. Specifically, we use machine learning and data mining to analyze large volumes of data and discover the relationship between two types of numerical data (regression), to discover groups of data having similar properties from large volumes of data (clustering), and to sort data into predetermined categories based on training data (classification). We are also using machine learning and data mining to achieve advanced network operations [5].

2.3 Information theory

Information theory includes a technique called compressed sensing [6]. Because the elements of an observation vector can be expressed as the product of an unknown vector and a known matrix, compressed sensing can be used to infer the unknown vector. Specifically, if the structure of the known matrix is sparse—that is, if many of the elements of the matrix are zero—the unknown vector can be inferred. Compressed sensing is a technique originally developed in fields such as image processing and signal processing, but its application to network operations is progressing. Here, assuming that end-to-end quality on a variety of network paths can be measured, we consider a situation in which quality deterioration has been detected on several paths. Compressed sensing can then be used to determine which interval in the network is the source of this deterioration. In mobile communications, compressed sensing can be applied to techniques for isolating locations of quality deterioration [7].

2.4 Model predictive control

Model predictive control can be used in cases where predicting the response to a network control operation such as changing transmission paths is difficult. This technique was originally developed for controlling plant processes. Model predictive control iteratively solves an optimization control problem while shifting forward an N-step interval at every step of the process. It applies only the first operation step of the optimization input sequence to the control target (Fig. 3). An optimization control problem that takes N forward steps into account makes predictions N steps into the future and solves an optimization control problem with control conditions attached at every input step at that time. Model predictive control theory has so far been applied to traffic engineering, where it was found to be effective [8]. It is now being studied for use in resolving the problem of mutual interference in control operations between telecommunications operators and Hyper Giants.

Fig. 3. Model predictive control.

3.1 Network data analysis

In network data analysis, we mainly use machine learning and data mining to formulate effective countermeasures to faults in network services. We study in particular each step making up a fault countermeasure, namely, fault detection, root cause analysis, and service restoration. First, for the fault detection step, we are developing technologies for visualizing service status, supporting visual monitoring, detecting silent faults, performing predictive detection, and other tasks. For the root cause analysis step, we are developing technologies for identifying the root cause and location of a fault and for determining the impact of a fault on services. Finally, for the service restoration step, we are devel­oping technologies for formalizing and automating restoration tasks.

In developing these technologies, we apply machine learning and data mining, information theory (compressed sensing etc.), and optimization theory to diverse types of data related to network operations such as syslog^*5 data, Twitter^*6 data, numerical data, response histories, and workflow data [9]. For example, network devices generate syslog data in great quantities, but the messages that are output are in formats that differ from one device to another and from vendor to vendor. As a result, using this information in on-site maintenance operations can be difficult. However, machine learning and data mining technology can be used to discover regularities in such a huge quantity of syslog messages such as messages that tend to be issued simultaneously, and it can also be used to visualize network operations [10]. Additionally, this technology can be used to detect and comprehend service failures by analyzing Twitter data. The idea here is to train the system beforehand using previous tweets posted at the time of service failures and to then identify tweets related to service failures during real-time monitoring of Twitter feeds. These failure-related tweets can then be extracted and used to clarify the nature of a service failure [11].

The network of the future will have an increasingly complicated and diverse configuration, and network operation will be all the more complex. Furthermore, if we include the cloud when talking about the network, we can expect even greater complexity. In light of this complexity, the detection of faults and failures, troubleshooting and factor clarification after detection, and failure recovery after troubleshooting will become increasingly difficult. Moreover, in addition to simply determining whether a service interruption has occurred, it will also be necessary to deal effectively with service flaws that are difficult to quantitatively evaluate such as deterioration in the customer’s QoE. Such service flaws are sometimes called obscure faults whose reproducibility is low, and solving them may require a relatively long time as a result. The introduction of SDN, NFV, and other virtualization technologies is expected to make problem response even more complicated, so we can expect network data analysis to become increasingly important going forward.

3.2 Proactive network control

Proactive network control is a network control technique that optimizes the network configuration based on traffic predictions [12]. To make traffic predictions, it is important to clarify the latent mechanisms involved in generating traffic.

The ways in which users use the network are also diversifying, so it is also important to understand for what purposes users use the network in order to make reliable traffic predictions. Additionally, there is a need to be aware of mobile traffic arising from the increasing popularity of smartphones and tablets. Here, it is important to understand how users move through both cyberspace and physical space. We are developing technologies to solve the above problems by making use of space-related theory and machine learning and data mining.

Controlling traffic based on traffic predictions requires that the error arising in those predictions be minimized. However, when performing network control as in changing transmission paths, it is becoming increasingly difficult to predict the response to such a control operation. This difficulty is reflected by the mutual interference that can occur between content control by a Hyper Giant and network control by a telecommunications operator. A Hyper Giant may change the servers it uses to deliver content based on measurements of network quality. This can have the effect of changing traffic flow on the telecommunications operator’s network and generating congestion. The telecommunications operator may then change network transmission paths to deal with that congestion, but from the Hyper Giant’s point of view, this may mean a change in communications quality. It is in this way that mutual interference arises between content control by a Hyper Giant and network control by a telecommunications operator. It has consequently become difficult for a telecommunications operator to predict the result of its own control operation to change transmission paths. Model predictive control is considered to be an effective technique under such conditions.

3.3 QoE-centric operation

In the operation and management of network services, there is a growing need to consider the user’s QoE in addition to simply determining whether a service interruption has occurred [13]. Although QoE is a subjective matter involving such characteristics as beautiful (e.g., for images) and natural (for sound), quantifying such subjective characteristics should enable QoE to be inferred based on indicators measured on the network side. The QoE so obtained could then be used in the operation and management of network services.

Until recently, the main approach to quantifying QoE has been to conduct experiments with multiple subjects seated in evaluation booths. However, it is now becoming important to ascertain user QoE in the field though the use of crowdsourcing with smartphones and other novel means. Furthermore, to facilitate estimation of QoE, we can model the correspondence between data that can be directly measured within the network such as throughput and delay and QoE related to video, voice, and data-communications services. As for video, we are researching and developing HTTP/TCP (Hypertext Transfer Protocol/Transmission Control Protocol)-based progressive-download video services that have recently become popular as well as 4K/8K high-definition video services. Next, in terms of voice, we are researching and developing mobile VoIP (voice over Internet protocol) applications such as LINE^*7 and Skype^*8 and IP-based voice services in a mobile environment such as voice over LTE (VoLTE)^*9. Finally, in the area of data communications, we are targeting web browsing in our research and development (R&D) efforts.

As described above, QoE can be understood on the basis of data measured by the network and used in the operation and management of network services. At the same time, such data can be provided to service providers to assist them in adding value to their services. A quality-related application programming interface (quality API) is one means of providing such data, and we are conducting field tests in collaboration with service providers using such a quality API [14].

3.4 Disaster-free Networks

We are researching and developing Disaster-free Networks applying space-related theory [15]. A Disaster-free Network is a network whose design and control methods are robust to natural disasters and other calamities. Space-related theory can be used to calculate the probability that areas in which natural disasters may occur overlap with the network. In short, when deploying network facilities, the probability that those facilities will be affected by a disaster can be calculated based on a hazard map that indicates the likelihood of earthquakes occurring. The results of those calculations can then be used to design a network highly resistant to disasters. Space-related theory can also be applied to network control in a virtual network focusing on the fact that the logical configuration of the network can be controlled independently of its physical configuration. For example, the ever-changing probability of being hit by a disaster can be calculated based on forecasts of typhoons, torrential rains, and other weather events so that the current arrangement of network servers can be switched to one with a low probability of being affected by such a disaster.