Ethernet Private Line System with World’s Highest Quality (Bandwidth Guarantee, High Availability, and Low Delay)
Virtual private networks and broadband Ethernet are increasingly being used as corporate wide area networks. Consequently, demand is growing for higher availability and lower delay in addition to the bandwidth guarantee and carrier-grade maintainability and operability of conventional digital private line systems. We describe here an Ethernet private line system that achieves the world’s highest level of quality to meet that demand.
Keywords: Ethernet private line service, transport network, hitless switching
The use of Ethernet for constructing local area networks has increased recently, and along with this, virtual private networks (VPNs) and broadband Ethernet, which can provide high-speed lines at reasonable cost, are being increasingly deployed as corporate wide area networks. However, there is strong demand for higher availability and lower delay in addition to the existing demand for the bandwidth guarantee and carrier-grade operation, administration, and maintenance (OAM) of conventional digital private line systems. This demand is particularly high for service-critical lines and emergency lines. In the transfer of video data to be used for broadcasting or other important purposes, for example, even the loss of a single key frame may cause a few seconds of traffic interruption in video streaming, so there is a need for redundancy technology that prevents interruption of communication due to hardware failures. A delay in file transfer by TCP/IP (Transmission Control Protocol/Internet protocol) greatly affects throughput. Lower delay is also required for e-commerce and other use scenarios that are sensitive to delay. Furthermore, it is also necessary to ensure stable communication, even under peak traffic conditions such as during disasters and other emergencies, and the system should enable line quality to be maintained at the same level as conventional digital private line systems.
In conventional digital private line systems, synchronous digital hierarchy (SDH) technologies based on time-division multiplexing have been adopted to provide high-quality service at the full fixed-line bandwidth contracted by the user. With SDH, carrier-grade functions for line and path management, maintenance and operation, and redundancy have been implemented.
NTT Network Service Systems Laboratories has developed an Ethernet private line system that can provide broadband services with high availability and low delay while maintaining bandwidth guarantee and carrier-grade maintenance and operation functions by applying packet transport technology (Fig. 1). This system implements low-delay hitless switching, an end-to-end maintenance monitoring function, and wire-rate transmission to achieve an Ethernet system with a quality level that exceeds that of conventional digital private line systems (Fig. 2). The Ethernet private line system comprises a relay node, subscriber accommodation nodes, digital service units (DSUs), and an operation system. We describe here the main technical features of the Ethernet private line system.
2. Technical issues for achieving high quality in an Ethernet system
There are three additional technical requirements for achieving the high quality level of conventional digital private line systems using Ethernet (Fig. 2).
2.1 High availability (hitless switching) and low delay
In systems using statistical multiplexing such as Ethernet, the delay tends to be longer compared to fixed-rate transmission systems since all of the frames need to be momentarily buffered for processing in the node before transmission.
Also, hitless switching technology can be applied to achieve high availability in conventional SDH-based digital private line systems. Hitless switching uses a redundancy approach in which the same frame is copied onto two redundant paths. Frames are checked at the receiving side, and the valid frame is selected and transmitted so that there is no disconnection even in the event of a failure on a transmission path. This kind of hitless switching is different from packet transmission in that the signal is continuously transmitted with fixed time slots, so the frames in short paths are buffered to prevent disconnection due to differences in path length (variation in delay) caused by path switching. The frames remain in the buffer until the arrival of the long-path frame, and control is performed so that the selection is made after the long-path frame arrives. Accordingly, it is not possible to achieve both high availability and low delay at the same time because the output signal is always delayed by the amount of delay in the long path. Thus, implementation of redundancy that provides both high availability and low delay is a problem that needs to be solved in order to achieve high quality in an Ethernet private line system.
2.2 Carrier-grade maintenance and operation functions
Ethernet did not originally provide maintenance and operation functions, but such functions have been specified as Ethernet has come into use in carrier networks. However, these functions were inadequate for such high-quality networks because maintenance of high-quality networks requires testing functions that are capable of failure detection, quality measurement, and rapid isolation of failures in node sections, including end-to-end. Specifically, fine-grained monitoring and determination of failure locations between DSUs and in each node section by OAM are needed to quickly identify degraded points/locations. In particular, in-service loopback testing at various points on a path is an essential function for failure isolation. A quality measurement function is also necessary for maintaining quality with respect to bandwidth and delay.
2.3 Wire-rate transmission
Although it is now possible to use OAM functions to implement maintenance and monitoring functions such as failure detection, testing, and quality measurement, the monitoring is constant, so periodic circulation of OAM frames on the main signal path is necessary. That is accompanied by bandwidth constraints on the main signal, making it difficult to guarantee the equivalent of the physical line bandwidth (wire rate). It will be necessary to achieve wire-rate transmission, which is the upper limit of the physical interface, and reliably provide the contracted bandwidth, while ensuring carrier-grade maintenance and operation by OAM.
3. Three features of Ethernet private line systems
Here, we describe three technical features of Ethernet private line systems.
3.1 Low-delay hitless switching technology
Ethernet is unaffected by variations due to differences in path length because it uses frames instead of a continuous signal. We applied this feature to address the problem of reducing delay in conventional SDH-based hitless switching and to achieve both high availability and low delay in this system.
In the low-delay hitless switching applied in our Ethernet private line system shown in Fig. 3, frames are sent over two paths, and the first to arrive is selected. At that time, the frame is transmitted without waiting for the other frame to arrive over the long path, so a large decrease in delay relative to the SDH hitless switching mechanism is possible. This decrease is on the order of milliseconds and corresponds to the difference in the long path delay.
In this system, the packet sequence may end up being reversed when a packet is lost on the short path. In view of the possible packet loss during the transmission, the system performs sequence control of packets in order to prevent a reversal in the packet sequence.
The priority processing for first-arriving frames used in this system greatly reduces the delay. Moreover, we implemented the world’s first hitless switching technology, which can be applied during planned maintenance and failures.
3.2 Carrier-grade maintenance and operation functions
(1) Measuring end-to-end quality
All of the nodes in the system, including the DSUs, are equipped with OAM functions for determining the locations of degraded quality in each node section, including measurement of end-to-end quality. These functions enable failure detection via continuity checks in all segments within the network and in all node sections. Also, the amount of delay can be periodically recorded as traffic data by conducting proactive delay measurement tests.
(2) Rapid failure isolation
To ensure the continuity of sequence numbers when OAM frames and the main signal frames are mixed by using a low-delay hitless switching mechanism, it is necessary to perform OAM loopback tests (Fig. 4). However, when looping back at nodes other than path termination points, jumps in the sequence numbers are misinterpreted as lost packets by the hitless switching processing unit.
To deal with this problem, the system implements a test that is equivalent to a loopback at any point by combining loopback tests between node units and carrying out monitoring within the node units. In this way, maintenance personnel can simply use the operation system to select lines for testing without having to be aware of each section test, thus implementing virtual path testing in which points of failure can be automatically identified by tests run on monitored sections. Furthermore, providing testing functions by OAM both enables in-service testing regardless of user bandwidth use conditions and provides an operating system function for visual display of locations where alarms were issued by nodes, thus supporting smooth communication with on-site maintenance personnel.
3.3 Wire-rate transmission technology
The Ethernet private line system applies synchronous Ethernet technology and wire-rate transmission technology that uses interframe gap (IFG) reduction for a bandwidth guarantee in order to implement carrier-grade maintenance and operation functions such as end-to-end OAM failure detection and delay measurement. Because Ethernet is an asynchronous protocol, deviations in clock synchronization between node units are absorbed by IFG reduction. In this system, however, we solve that problem with synchronous Ethernet (also referred as Sync-E), which transmits a clock signal between nodes via the Ethernet physical layer as specified by the Telecommunication Standardization Sector of the International Telecommunication Union (ITU-T) Recommendation G.8261 . Doing so enables use of the available bandwidth by not using IFG reduction for transmitting OAM frames or downloading firmware. Compression of IFG according to bandwidth enables end-to-end failure detection and delay measurement, and it also enables remote in-service firmware updating to a DSU isolated from user traffic.
4. Future development
We have described an Ethernet private line system that was developed by NTT Network Service Systems Laboratories. In the future, we will continue to work on meeting customer needs for even higher bandwidth and service quality.