You need Adobe Reader 7.0 or later in order to read PDF files on this site.
If Adobe Reader is not installed on your computer, click the button below and go to the download site.

Regular Articles

Vol. 14, No. 10, pp. 56–62, Oct. 2016. https://doi.org/10.53829/ntr201610ra2

Ethernet Private Line System with World’s Highest Quality (Bandwidth Guarantee, High Availability, and Low Delay)

Masumi Sakamoto, Katsuhiro Araya, Gen-ichi Nishio,
Masafumi Ando, Masaki Shinkai, Takaaki Hisashima,
Kaoru Arai, Hidetoshi Onda, Hidenori Iwashita,
Masaya Ogawa, Osamu Kurokawa, and Katsutoshi Koda

Abstract

Virtual private networks and broadband Ethernet are increasingly being used as corporate wide area networks. Consequently, demand is growing for higher availability and lower delay in addition to the bandwidth guarantee and carrier-grade maintainability and operability of conventional digital private line systems. We describe here an Ethernet private line system that achieves the world’s highest level of quality to meet that demand.

Keywords: Ethernet private line service, transport network, hitless switching

1. Introduction

The use of Ethernet for constructing local area networks has increased recently, and along with this, virtual private networks (VPNs) and broadband Ethernet, which can provide high-speed lines at reasonable cost, are being increasingly deployed as corporate wide area networks. However, there is strong demand for higher availability and lower delay in addition to the existing demand for the bandwidth guarantee and carrier-grade operation, administration, and maintenance (OAM) of conventional digital private line systems. This demand is particularly high for service-critical lines and emergency lines. In the transfer of video data to be used for broadcasting or other important purposes, for example, even the loss of a single key frame may cause a few seconds of traffic interruption in video streaming, so there is a need for redundancy technology that prevents interruption of communication due to hardware failures. A delay in file transfer by TCP/IP (Transmission Control Protocol/Internet protocol) greatly affects throughput. Lower delay is also required for e-commerce and other use scenarios that are sensitive to delay. Furthermore, it is also necessary to ensure stable communication, even under peak traffic conditions such as during disasters and other emergencies, and the system should enable line quality to be maintained at the same level as conventional digital private line systems.

In conventional digital private line systems, synchronous digital hierarchy (SDH) technologies based on time-division multiplexing have been adopted to provide high-quality service at the full fixed-line bandwidth contracted by the user. With SDH, carrier-grade functions for line and path management, maintenance and operation, and redundancy have been implemented.

NTT Network Service Systems Laboratories has developed an Ethernet private line system that can provide broadband services with high availability and low delay while maintaining bandwidth guarantee and carrier-grade maintenance and operation functions by applying packet transport technology (Fig. 1). This system implements low-delay hitless switching, an end-to-end maintenance monitoring function, and wire-rate transmission to achieve an Ethernet system with a quality level that exceeds that of conventional digital private line systems (Fig. 2). The Ethernet private line system comprises a relay node, subscriber accommodation nodes, digital service units (DSUs), and an operation system. We describe here the main technical features of the Ethernet private line system.

Fig. 1. Positioning of private line systems.

Fig. 2. Overview of Ethernet private line system and key technical points.

2. Technical issues for achieving high quality in an Ethernet system

There are three additional technical requirements for achieving the high quality level of conventional digital private line systems using Ethernet (Fig. 2).

2.1 High availability (hitless switching) and low delay

In systems using statistical multiplexing such as Ethernet, the delay tends to be longer compared to fixed-rate transmission systems since all of the frames need to be momentarily buffered for processing in the node before transmission.

Also, hitless switching technology can be applied to achieve high availability in conventional SDH-based digital private line systems. Hitless switching uses a redundancy approach in which the same frame is copied onto two redundant paths. Frames are checked at the receiving side, and the valid frame is selected and transmitted so that there is no disconnection even in the event of a failure on a transmission path. This kind of hitless switching is different from packet transmission in that the signal is continuously transmitted with fixed time slots, so the frames in short paths are buffered to prevent disconnection due to differences in path length (variation in delay) caused by path switching. The frames remain in the buffer until the arrival of the long-path frame, and control is performed so that the selection is made after the long-path frame arrives. Accordingly, it is not possible to achieve both high availability and low delay at the same time because the output signal is always delayed by the amount of delay in the long path. Thus, implementation of redundancy that provides both high availability and low delay is a problem that needs to be solved in order to achieve high quality in an Ethernet private line system.

2.2 Carrier-grade maintenance and operation functions

Ethernet did not originally provide maintenance and operation functions, but such functions have been specified as Ethernet has come into use in carrier networks. However, these functions were inadequate for such high-quality networks because maintenance of high-quality networks requires testing functions that are capable of failure detection, quality measurement, and rapid isolation of failures in node sections, including end-to-end. Specifically, fine-grained monitoring and determination of failure locations between DSUs and in each node section by OAM are needed to quickly identify degraded points/locations. In particular, in-service loopback testing at various points on a path is an essential function for failure isolation. A quality measurement function is also necessary for maintaining quality with respect to bandwidth and delay.

2.3 Wire-rate transmission

Although it is now possible to use OAM functions to implement maintenance and monitoring functions such as failure detection, testing, and quality measurement, the monitoring is constant, so periodic circulation of OAM frames on the main signal path is necessary. That is accompanied by bandwidth constraints on the main signal, making it difficult to guarantee the equivalent of the physical line bandwidth (wire rate). It will be necessary to achieve wire-rate transmission, which is the upper limit of the physical interface, and reliably provide the contracted bandwidth, while ensuring carrier-grade maintenance and operation by OAM.

3. Three features of Ethernet private line systems

Here, we describe three technical features of Ethernet private line systems.

3.1 Low-delay hitless switching technology

Ethernet is unaffected by variations due to differences in path length because it uses frames instead of a continuous signal. We applied this feature to address the problem of reducing delay in conventional SDH-based hitless switching and to achieve both high availability and low delay in this system.

In the low-delay hitless switching applied in our Ethernet private line system shown in Fig. 3, frames are sent over two paths, and the first to arrive is selected. At that time, the frame is transmitted without waiting for the other frame to arrive over the long path, so a large decrease in delay relative to the SDH hitless switching mechanism is possible. This decrease is on the order of milliseconds and corresponds to the difference in the long path delay.

Fig. 3. Low-delay hitless switching.

In this system, the packet sequence may end up being reversed when a packet is lost on the short path. In view of the possible packet loss during the transmission, the system performs sequence control of packets in order to prevent a reversal in the packet sequence.

The priority processing for first-arriving frames used in this system greatly reduces the delay. Moreover, we implemented the world’s first hitless switching technology, which can be applied during planned maintenance and failures.

3.2 Carrier-grade maintenance and operation functions

(1) Measuring end-to-end quality

All of the nodes in the system, including the DSUs, are equipped with OAM functions for determining the locations of degraded quality in each node section, including measurement of end-to-end quality. These functions enable failure detection via continuity checks in all segments within the network and in all node sections. Also, the amount of delay can be periodically recorded as traffic data by conducting proactive delay measurement tests.

(2) Rapid failure isolation

To ensure the continuity of sequence numbers when OAM frames and the main signal frames are mixed by using a low-delay hitless switching mechanism, it is necessary to perform OAM loopback tests (Fig. 4). However, when looping back at nodes other than path termination points, jumps in the sequence numbers are misinterpreted as lost packets by the hitless switching processing unit.

Fig. 4. Virtual path testing.

To deal with this problem, the system implements a test that is equivalent to a loopback at any point by combining loopback tests between node units and carrying out monitoring within the node units. In this way, maintenance personnel can simply use the operation system to select lines for testing without having to be aware of each section test, thus implementing virtual path testing in which points of failure can be automatically identified by tests run on monitored sections. Furthermore, providing testing functions by OAM both enables in-service testing regardless of user bandwidth use conditions and provides an operating system function for visual display of locations where alarms were issued by nodes, thus supporting smooth communication with on-site maintenance personnel.

3.3 Wire-rate transmission technology

The Ethernet private line system applies synchronous Ethernet technology and wire-rate transmission technology that uses interframe gap (IFG) reduction for a bandwidth guarantee in order to implement carrier-grade maintenance and operation functions such as end-to-end OAM failure detection and delay measurement. Because Ethernet is an asynchronous protocol, deviations in clock synchronization between node units are absorbed by IFG reduction. In this system, however, we solve that problem with synchronous Ethernet (also referred as Sync-E), which transmits a clock signal between nodes via the Ethernet physical layer as specified by the Telecommunication Standardization Sector of the International Telecommunication Union (ITU-T) Recommendation G.8261 [1]. Doing so enables use of the available bandwidth by not using IFG reduction for transmitting OAM frames or downloading firmware. Compression of IFG according to bandwidth enables end-to-end failure detection and delay measurement, and it also enables remote in-service firmware updating to a DSU isolated from user traffic.

4. Future development

We have described an Ethernet private line system that was developed by NTT Network Service Systems Laboratories. In the future, we will continue to work on meeting customer needs for even higher bandwidth and service quality.

Reference

[1]	ITU-T Recommendation G.8261, https://www.itu.int/rec/T-REC-G.8261

	Masumi Sakamoto Researcher, Transport Network Innovation Project, NTT Network Service Systems Laboratories. She received a B.E. in precision machining engineering from Tokyo Metropolitan University in 2007. She joined NTT EAST in 2007, where she was involved in maintaining IP networks. She has been in her current position since August 2013 and is developing an Ethernet private line system.
	Katsuhiro Araya Senior Research Engineer, Transport Network Innovation Project, NTT Network Service Systems Laboratories. He received a B.S. in physics from Nihon University, Tokyo, in 1995 and an M.S. in physics from Tokyo Institute of Technology in 1997. He joined NTT in 1998. Since joining NTT Network Service Systems Laboratories in 2015, he has been developing an Ethernet private line system.
	Gen-ichi Nishio Senior Research Engineer, Transport Network Innovation Project, NTT Network Service Systems Laboratories. He received a B.E and M.E. in electronics from Tokyo Institute of Technology in 1990 and 1992. He joined NTT in 1992 and began working on the development of subscriber systems at NTT Transmission Systems Laboratories. He has been in his current position since 2016 and is engaged in developing an Ethernet private line system. He is a member of the Institute of Electronics, Information and Communication Engineers (IEICE).
	Masafumi Ando Researcher, Transport Network Innovation Project, NTT Network Service Systems Laboratories. He received a B.S. in physics from Tokyo University of Science in 2005 and an M.S. in physics from Tokyo Institute of Technology in 2007. He has been in his current position since July 2014 and is developing an Ethernet private line system. He is a member of the Physical Society of Japan.
	Masaki Shinkai Research Engineer, Transport Network Innovation Project, NTT Network Service Systems Laboratories. He received a B.E. and M.E. in engineering from the University of Electro-Communications, Tokyo, in 1996 and 1998. He joined NTT in 1998 and was involved in maintaining a dedicated service network at NTT and NTT EAST. He has been in his current position since July 2014 and is developing a packet transport multiplexing (PTM) adapter for dedicated services and the Ethernet private line system. He is a member of IEICE.
	Takaaki Hisashima Research Engineer, Transport Network Innovation Project, NTT Network Service Systems Laboratories. He received a B.S. and M.S. in chemical sciences from Osaka Prefecture University in 2006 and 2008. He joined NTT WEST in 2008, where he worked on the development of a security system at a research and development (R&D) center at NTT WEST. He has been in his current position since July 2014 and is engaged in developing network systems. He is a member of IEICE.
	Kaoru Arai Researcher, Transport Network Innovation Project, NTT Network Service Systems Laboratories. He received a B.S. and M.S. in applied physics from Tokyo University of Science in 2010 and 2012. Since joining NTT Network Service Systems Laboratories in 2012, he has been engaged in R&D of packet network systems such as clock supply systems. He is a member of IEICE.
	Hidetoshi Onda Senior Research Engineer, Transport Network Innovation Project, NTT Network Service Systems Laboratories. He received a B.S. and M.S. in science from Shizuoka University in 1995 and 1997. He joined NTT in 1997 and was involved in maintenance and planning for dedicated service networks at NTT and NTT EAST. He has been in his current position since July 2014 and is developing a PTM adapter for dedicated services and an Ethernet private line system.
	Hidenori Iwashita Research Engineer, Transport Network Innovation Project, NTT Network Service Systems Laboratories. He received a B.S. and M.S. in nuclear engineering from Hokkaido University in 2006 and 2008. Since 2008, he has been a researcher at NTT Network Service Systems Laboratories. He is involved in researching and developing a packet transport multiplexer, PTM cross-connect (PTM-XC), PTM adapter for dedicated services, and an Ethernet private line system. He is a member of IEICE.
	Masaya Ogawa Research Engineer, Transport Network Innovation Project, NTT Network Service Systems Laboratories. He received a B.E. and M.E. in electronics from Keio University, Kanagawa, in 2003 and 2005. He joined NTT EAST in 2005, where he was involved in maintaining transport networks. He has been in his current position since July 2009 and is developing a packet transport multiplexer, PTM-XC, PTM adapter for dedicated services, and an Ethernet private line system. He is a member of IEICE and the Japan Society of Applied Physics.
	Osamu Kurokawa Senior Research Engineer, Transport Network Innovation Project, NTT Network Service Systems Laboratories. He has been in his current position since August 2014 and is engaged in R&D on a PTM adapter for dedicated services, an Ethernet private line system, and a clock supply module.
	Katsutoshi Koda Director, Transport Network Innovation Project, NTT Network Service Systems Laboratories. He received a B.S. and M.S. in mechanical sciences from Tokyo University of Science in 1987 and 1989. He joined NTT in 1989 and began working on the development of transport systems at the Network System Development Center. He has been in his current position since July 2014. He is managing the development project for the Ethernet private line system. He also manages the R&D of future optical networking and transport network management technology.

↑ TOP