Feature Articles: New Generation Network Platform and Attractive Network Services
Smart CPE R&D for Highly Intelligent Enterprise Networks
Enterprise users’ information and communication technology (ICT) demands vary widely from industry to industry and also depend on business conditions and company size. In this article, we present the status of our research and development of an enterprise business solution called Highly Intelligent Customer Premises Equipment, or Smart CPE, which can offer smart traffic control and advanced network management functions for existing wide area network services such as broadband Ethernet and virtual private network services. This solution makes it possible to construct optimal customer ICT environments and to meet individual needs.
Keywords: enterprise network, service customization, customer premises equipment
The information and communication technology (ICT) demands of enterprise users vary widely from industry to industry and depend on business conditions and the size of the company. In addition, enterprise users must promptly respond to the changes in the business environment. Small- and medium-sized enterprise clients without dedicated network operators have particularly high expectations of carriers with respect to the construction and operation of their ICT infrastructure. However, when only existing network services and dedicated, off-the-shelf hardware virtual private network (VPN) routers are used, it can be difficult to quickly respond to many different kinds of requests. Users might request, for example, a service with a level of quality and price point that lies between a best-effort service and a guaranteed quality of service, or they may state that they want to run backup wide area network (WAN) services at all times. Consequently, a way of quickly and flexibly customizing services to meet individual customer needs is required.
NTT Network Innovation Laboratories has been researching and developing one promising solution called Highly Intelligent Customer Premises Equipment, or Smart CPE, to meet such requirements.
2. Conceptual overview
A conceptual diagram of Smart CPE is shown in Fig. 1. Smart CPE serves as a replacement for a VPN router. Like a conventional VPN router, it simultaneously connects to multiple WAN services. Unlike existing VPN routers, however, Smart CPE runs carrier services that can be customized according to individual customer needs.
Smart CPE’s network concierge feature is one way in which it customizes original services. Ordinary VPN routers control traffic primarily by switching between active and backup circuits or simply assigning traffic to a fixed number of circuits. Smart CPE, on the other hand, assesses policies and usage along with the state of a client carrier’s WAN services and equipment. Through a combination of optimal traffic controls and WAN service choices, Smart CPE then intelligently controls traffic for individual network flows. To customers, multiple WAN services thus appear to be a single WAN service that is suited to any situation.
Smart CPE’s remote maintenance feature represents another way in which it customizes services. By working together with a carrier’s operational systems and maintenance departments, Smart CPE can quickly diagnose and repair errors without the need to dispatch an engineer to the site.
Our solution also provides a customer portal for enterprise users, network carriers, and partner companies of our Hikari Collaboration Model, which involves wholesaling fiber access service, in order to customize their services and check on the visualized service status.
3. Implementation approaches
One approach for implementing the aforementioned concepts is illustrated in Fig. 2. To implement services that can be quickly and flexibly customized to meet customers’ needs, we use general-purpose Linux computers for Smart CPE rather than dedicated hardware because they offer ease of development as well as remarkable improvements in cost-performance ratios. When we receive a customer request, we can then customize Smart CPE with application software implementations of the appropriate services that would ordinarily be provided by a customer’s servers and networking equipment. We implement services in application software using the existing software resources available from the Linux community.
Furthermore, we provide customer portals and element management system and network management system (EMS/NMS) web services for configuring and controlling Smart CPE services.
5. Prototype development
We developed a prototype to verify the effectiveness of our Smart CPE concept and architecture, and we are currently testing it by connecting three offices via more than two WAN services including the Master’s One Wireless VPN and FLET’S VPN WIDE (Fig. 4). We are using two types of hardware: desktop computers and 1U rack servers.
The Linux kernel has the same capabilities as a VPN router. It supports VLANs (virtual local area networks), PPoE (point-to-point over Ethernet), GRE (Generic Routing Encapsulation) tunnels, and other virtual interfaces, and also has NAT (network address translation), firewall, and QoS (quality of service) features. Our prototype’s traffic control component manages individual traffic flows using the Linux kernel’s standard packet forwarding features to the greatest extent possible. Furthermore, our prototype’s packet monitoring component uses a 10-Gigabit network monitoring system (PRESTA 10G)  developed by NTT Network Innovation Laboratories. This implements active monitoring of one-way latency and packet loss between Smart CPE installations as well as passive monitoring with the same millisecond resolution as a network analyzer. This functionality is all available through the northbound API, which we use in our prototype to implement the following applications for our network concierge and on-site maintenance features. Every application can be configured and controlled through a customer portal GUI (graphical user interface) (Fig. 3).
• Traffic assignment
This application specifies a WAN service to use for a per flow basis.
• Network switching
This application configures upper bounds on the permissible one-way latency and packet loss rate for any network flow. When a WAN service metric exceeds these thresholds, its network flows are switched over to different circuits. This application can also save packet dumps for a fixed period of time around such a switch for use in future troubleshooting.
• SLA (service-level agreement) measurement
This application measures one-way latency and packet loss rates between Smart CPE installations for each WAN service. The data appear in visualized graphs on customer portals and can also be exported in a comma-separated values (CSV) format.
• Traffic visualization
This application measures traffic volume for any network flow—whether configured by protocol, application (e.g., web, VoIP (voice over Internet protocol), video streaming), or server—and visually plots them in real time with millisecond resolution on customer portals. This allows network administrators to notice sudden spikes (bursts) in network traffic, which lead to packet loss and cannot be discerned through ordinary SNMP (Simple Network Management Protocol) or NetFlow monitoring. The data can also be exported in CSV format.
• Network topology maps
This application displays on customer portals a map of Smart CPE connections along with the real-time traffic volume between Smart CPE installations for each WAN service. This application also provides Smart CPE uptime monitoring and can show remote Smart CPE customer portals.
We exhibited this prototype at the NTT R&D Forum 2015 (Fig. 5). With multiple network flows using FLET’S VPN WIDE, we demonstrated how we could switch one particular video flow over to the Master’s One Wireless VPN in response to variations in one-way latency. We also demonstrated this same functionality at the NTT R&D Messe 2015 using a commercial web conferencing system (MeetingPlaza) and IP-PBX (Internet protocol-based private branch exchange) (Crossway) installed on our Smart CPE.
6. Use cases
We plan to continue developing Smart CPE applications that allow enterprise network system integrator engineers at our Hikari Collaboration partner companies and in the enterprise sales departments of business firms to customize WAN services for small- and medium-sized enterprise users.
We specifically expect to encounter use cases like the one illustrated in Fig. 6. Under ordinary conditions, our Smart CPE’s network concierge feature uses the appropriate WAN service to access particular servers or for individual application flows given a customer’s operational policies and conditions. When there is a network failure or WAN services experience network congestion, the concierge feature switches network flows over to WAN services that are functioning properly, prioritizing flows that are most important for the customer’s continued business operations. We believe that this can contribute to making our customers’ businesses more efficient. Furthermore, because Smart CPE’s remote maintenance feature can always check a customer’s network status, we believe that it can lead to proposals for bandwidth and WAN service adjustments that match changes in the customer’s network usage. Through the initial connections to customer networks without dispatching engineers and by speeding up failure response times, adding and removing virtual servers, controlling traffic in conjunction with network failures, and integrating with cloud services, we will strive to provide customized ICT environments to our customers.
As shown in Fig. 6, we are also considering how we can apply our Smart CPE technology to the movable ICT units  developed by NTT Network Innovation Laboratories. When a disaster occurs, we could use our Smart CPE for multiple types of WAN connections (e.g., fiber-optic cables, LTE (Long Term Evolution), and satellite) to main offices both inside and outside the affected areas, as well as for wireless LAN connections provided by each of the emergency response organizations both at the disaster site and at shelters for disaster victims. This would allow the emergency response organizations to share information using a suitable WAN given its current throughput, the importance of the information being shared, and the characteristics of the media (e.g., text, audio, images, or video) being shared. We thus hope to make Smart CPE capable of sharing information efficiently when communication resources are limited.
7. Future development
We have so far developed a prototype with Smart CPE’s basic functionality. We think that our most important task in bringing Smart CPE to market is to continue to develop applications while collecting feedback from actual customers, partner companies, and other businesses; this includes careful investigation of our expected use cases. Ultimately, we believe that we will also need to build application development environments that can be tailored to our partner companies and other businesses. In our continued research and development efforts in which we take Smart CPE from field trials to a finished product, we plan to exchange ideas related to the development of services and scenarios with wider business uses.