To view PDF files

You need Adobe Reader 7.0 or later in order to read PDF files on this site.
If Adobe Reader is not installed on your computer, click the button below and go to the download site.

Feature Articles: New Generation Network Platform and Attractive Network Services

Smart CPE R&D for Highly Intelligent Enterprise Networks

Katsuhiro Sebayashi, Osamu Kamatani,
and Osamu Akashi


Enterprise users’ information and communication technology (ICT) demands vary widely from industry to industry and also depend on business conditions and company size. In this article, we present the status of our research and development of an enterprise business solution called Highly Intelligent Customer Premises Equipment, or Smart CPE, which can offer smart traffic control and advanced network management functions for existing wide area network services such as broadband Ethernet and virtual private network services. This solution makes it possible to construct optimal customer ICT environments and to meet individual needs.

Keywords: enterprise network, service customization, customer premises equipment


1. Introduction

The information and communication technology (ICT) demands of enterprise users vary widely from industry to industry and depend on business conditions and the size of the company. In addition, enterprise users must promptly respond to the changes in the business environment. Small- and medium-sized enterprise clients without dedicated network operators have particularly high expectations of carriers with respect to the construction and operation of their ICT infrastructure. However, when only existing network services and dedicated, off-the-shelf hardware virtual private network (VPN) routers are used, it can be difficult to quickly respond to many different kinds of requests. Users might request, for example, a service with a level of quality and price point that lies between a best-effort service and a guaranteed quality of service, or they may state that they want to run backup wide area network (WAN) services at all times. Consequently, a way of quickly and flexibly customizing services to meet individual customer needs is required.

NTT Network Innovation Laboratories has been researching and developing one promising solution called Highly Intelligent Customer Premises Equipment, or Smart CPE, to meet such requirements.

2. Conceptual overview

A conceptual diagram of Smart CPE is shown in Fig. 1. Smart CPE serves as a replacement for a VPN router. Like a conventional VPN router, it simultaneously connects to multiple WAN services. Unlike existing VPN routers, however, Smart CPE runs carrier services that can be customized according to individual customer needs.

Fig. 1. Conceptual diagram of Smart CPE.

Smart CPE’s network concierge feature is one way in which it customizes original services. Ordinary VPN routers control traffic primarily by switching between active and backup circuits or simply assigning traffic to a fixed number of circuits. Smart CPE, on the other hand, assesses policies and usage along with the state of a client carrier’s WAN services and equipment. Through a combination of optimal traffic controls and WAN service choices, Smart CPE then intelligently controls traffic for individual network flows. To customers, multiple WAN services thus appear to be a single WAN service that is suited to any situation.

Smart CPE’s remote maintenance feature represents another way in which it customizes services. By working together with a carrier’s operational systems and maintenance departments, Smart CPE can quickly diagnose and repair errors without the need to dispatch an engineer to the site.

Our solution also provides a customer portal for enterprise users, network carriers, and partner companies of our Hikari Collaboration Model, which involves wholesaling fiber access service, in order to customize their services and check on the visualized service status.

3. Implementation approaches

One approach for implementing the aforementioned concepts is illustrated in Fig. 2. To implement services that can be quickly and flexibly customized to meet customers’ needs, we use general-purpose Linux computers for Smart CPE rather than dedicated hardware because they offer ease of development as well as remarkable improvements in cost-performance ratios. When we receive a customer request, we can then customize Smart CPE with application software implementations of the appropriate services that would ordinarily be provided by a customer’s servers and networking equipment. We implement services in application software using the existing software resources available from the Linux community.

Fig. 2. Approach for a Smart CPE implementation.

Furthermore, we provide customer portals and element management system and network management system (EMS/NMS) web services for configuring and controlling Smart CPE services.

4. Architecture

Smart CPE uses a three-layer architecture, as shown in Fig. 3. The bottom layer has a traffic control component for packet forwarding and a packet monitoring component for high-resolution traffic measurement. Above this is the platform layer, which provides an abstraction layer for all the functionality below it. At the top is the application layer, in which service customization features are implemented. The platform layer currently provides a northbound application programming interface (API) with traffic control and packet monitoring features for the application layer, but later it will also provide a southbound API that allows the traffic control component to be replaced with OpenFlow and bare-metal switches without changing the platform or application layers. Both the northbound and southbound APIs will be REST (representational state transfer)/JSON (JavaScript Object Notation) APIs, which are widely used by web services, to make it easier to develop applications that provide prompt services and to then connect them to EMS/NMS.

Fig. 3. Smart CPE Architecture.

5. Prototype development

We developed a prototype to verify the effectiveness of our Smart CPE concept and architecture, and we are currently testing it by connecting three offices via more than two WAN services including the Master’s One Wireless VPN and FLET’S VPN WIDE (Fig. 4). We are using two types of hardware: desktop computers and 1U rack servers.

Fig. 4. Illustrated usage in enterprise networks.

The Linux kernel has the same capabilities as a VPN router. It supports VLANs (virtual local area networks), PPoE (point-to-point over Ethernet), GRE (Generic Routing Encapsulation) tunnels, and other virtual interfaces, and also has NAT (network address translation), firewall, and QoS (quality of service) features. Our prototype’s traffic control component manages individual traffic flows using the Linux kernel’s standard packet forwarding features to the greatest extent possible. Furthermore, our prototype’s packet monitoring component uses a 10-Gigabit network monitoring system (PRESTA 10G) [1] developed by NTT Network Innovation Laboratories. This implements active monitoring of one-way latency and packet loss between Smart CPE installations as well as passive monitoring with the same millisecond resolution as a network analyzer. This functionality is all available through the northbound API, which we use in our prototype to implement the following applications for our network concierge and on-site maintenance features. Every application can be configured and controlled through a customer portal GUI (graphical user interface) (Fig. 3).

• Traffic assignment

This application specifies a WAN service to use for a per flow basis.

• Network switching

This application configures upper bounds on the permissible one-way latency and packet loss rate for any network flow. When a WAN service metric exceeds these thresholds, its network flows are switched over to different circuits. This application can also save packet dumps for a fixed period of time around such a switch for use in future troubleshooting.

• SLA (service-level agreement) measurement

This application measures one-way latency and packet loss rates between Smart CPE installations for each WAN service. The data appear in visualized graphs on customer portals and can also be exported in a comma-separated values (CSV) format.

• Traffic visualization

This application measures traffic volume for any network flow—whether configured by protocol, application (e.g., web, VoIP (voice over Internet protocol), video streaming), or server—and visually plots them in real time with millisecond resolution on customer portals. This allows network administrators to notice sudden spikes (bursts) in network traffic, which lead to packet loss and cannot be discerned through ordinary SNMP (Simple Network Management Protocol) or NetFlow monitoring. The data can also be exported in CSV format.

• Network topology maps

This application displays on customer portals a map of Smart CPE connections along with the real-time traffic volume between Smart CPE installations for each WAN service. This application also provides Smart CPE uptime monitoring and can show remote Smart CPE customer portals.

We exhibited this prototype at the NTT R&D Forum 2015 (Fig. 5). With multiple network flows using FLET’S VPN WIDE, we demonstrated how we could switch one particular video flow over to the Master’s One Wireless VPN in response to variations in one-way latency. We also demonstrated this same functionality at the NTT R&D Messe 2015 using a commercial web conferencing system (MeetingPlaza) and IP-PBX (Internet protocol-based private branch exchange) (Crossway) installed on our Smart CPE.

Fig. 5. Exhibit at NTT R&D Forum 2015.

6. Use cases

We plan to continue developing Smart CPE applications that allow enterprise network system integrator engineers at our Hikari Collaboration partner companies and in the enterprise sales departments of business firms to customize WAN services for small- and medium-sized enterprise users.

We specifically expect to encounter use cases like the one illustrated in Fig. 6. Under ordinary conditions, our Smart CPE’s network concierge feature uses the appropriate WAN service to access particular servers or for individual application flows given a customer’s operational policies and conditions. When there is a network failure or WAN services experience network congestion, the concierge feature switches network flows over to WAN services that are functioning properly, prioritizing flows that are most important for the customer’s continued business operations. We believe that this can contribute to making our customers’ businesses more efficient. Furthermore, because Smart CPE’s remote maintenance feature can always check a customer’s network status, we believe that it can lead to proposals for bandwidth and WAN service adjustments that match changes in the customer’s network usage. Through the initial connections to customer networks without dispatching engineers and by speeding up failure response times, adding and removing virtual servers, controlling traffic in conjunction with network failures, and integrating with cloud services, we will strive to provide customized ICT environments to our customers.

Fig. 6. Usage for emergency response.

As shown in Fig. 6, we are also considering how we can apply our Smart CPE technology to the movable ICT units [2] developed by NTT Network Innovation Laboratories. When a disaster occurs, we could use our Smart CPE for multiple types of WAN connections (e.g., fiber-optic cables, LTE (Long Term Evolution), and satellite) to main offices both inside and outside the affected areas, as well as for wireless LAN connections provided by each of the emergency response organizations both at the disaster site and at shelters for disaster victims. This would allow the emergency response organizations to share information using a suitable WAN given its current throughput, the importance of the information being shared, and the characteristics of the media (e.g., text, audio, images, or video) being shared. We thus hope to make Smart CPE capable of sharing information efficiently when communication resources are limited.

7. Future development

We have so far developed a prototype with Smart CPE’s basic functionality. We think that our most important task in bringing Smart CPE to market is to continue to develop applications while collecting feedback from actual customers, partner companies, and other businesses; this includes careful investigation of our expected use cases. Ultimately, we believe that we will also need to build application development environments that can be tailored to our partner companies and other businesses. In our continued research and development efforts in which we take Smart CPE from field trials to a finished product, we plan to exchange ideas related to the development of services and scenarios with wider business uses.


[1] S. Kuwabara, “Monitoring Technology for Programmable Highly Functional Networks,” NTT Technical Review, Vol. 10, No. 8, 2012.
[2] T. Sakano, S. Kotabe, and T. Komukai, “Overview of Movable and Deployable ICT Resource Unit Architecture,” NTT Technical Review, Vol. 13, No. 5, 2015.
Katsuhiro Sebayashi
Senior Researcher, NTT Network Innovation Laboratories.
He received a B.E. in electrical engineering from Tokyo Denki University and joined NTT in 1991. His research interests include monitoring and control of network systems. He is a member of the Institute of Electronics, Information and Communication Engineers and the Institute of Electrical and Electronics Engineers Communication Society.
Osamu Kamatani
Senior Research Engineer, Media Innovation Laboratory, NTT Network Innovation Laboratories.
He received a B.E., M.E., and Ph.D. in electrical engineering from the University of Tokyo in 1988, 1990, and 1993. He joined NTT Transmission Systems Laboratories in 1993 and worked on high-speed optical time division multiplexed transmission systems, optical networking architectures, and high-speed interactive data transfer applications. He moved to NTT Service Integration Laboratories in 2004 and worked on the NGN (Next Generation Network) architecture and transport control protocols and also took part in relevant standardization activities in ITU-T and ETSI TISPAN. He is currently researching an internode cross-layer cooperation network architecture that can offer information-value-centric data processing.
Osamu Akashi
Senior Researcher, NTT Network Innovation Laboratories.
He received an M.S. in information science and a Ph.D. in mathematical and computing sciences from Tokyo Institute of Technology in 1989 and 2001. He joined NTT in 1989. His research interests include distributed systems, multi-agent systems, and network architectures.