Feature Articles: Toward Open and Intelligent Wireless Networks
Vol. 20, No. 11, pp. 52–63, Nov. 2022. https://doi.org/10.53829/ntr202211fa7
Initiatives toward Virtualized RAN
There is growing demand for high processing performance in wireless base-station equipment as bit rates and capacity continue their upward trend in Long-Term Evolution and fifth-generation mobile communications system networks. To meet this demand, NTT DOCOMO has been using purpose-built hardware and software. There has also been remarkable technical innovation in cloud computing for web services, as reflected by ongoing improvements in hardware performance and increasing decoupling of hardware and software provided via virtualization and cloud-computing technologies. These innovative technologies should lead to superior wireless base-station design and solutions so that a more disaggregated and efficient radio access network (RAN) can be achieved. The development and commercialization of such equipment is moving forward as RAN virtualization is becoming predominant. This article describes NTT DOCOMO initiatives toward RAN virtualization technology.
Keywords: vRAN, virtualization, Open RAN
NTT DOCOMO began providing services under the fifth-generation mobile communications system (5G) in March 2020 and has been expanding the 5G coverage area ever since. To roll out 5G while making maximum use of existing Long-Term Evolution (LTE) base-station equipment, NTT DOCOMO has developed the bare minimum amount of hardware together with software to run on that hardware and added both to existing LTE base-station equipment to achieve 5G services.
There has also been tremendous technical innovation in the information technology (IT) field. Thus, there have been rapid gains in the performance of general-purpose hardware, in the effective use of hardware resources by decoupling hardware and software using virtualization technology, and in the omnipresent use of common cloud technology by leveraging standard and de-facto cloudification and orchestration platforms. To achieve high-performance data encryption, artificial intelligence (AI), and machine-learning technologies, specialized hardware accelerators for various types of computing are now being developed and marketed.
General-purpose hardware as used in the IT field and virtualization technology based on hardware accelerators customized for wireless processing are starting to be applied to wireless base stations in a network transformation toward the virtualization of a radio access network (RAN), or virtualized RAN (vRAN) for short. The use of general-purpose hardware in virtualization and cloudification and the effective use of vRAN technology are expected to reduce infrastructure investment. With the idea of maximizing the benefits of vRAN, in February 2021 NTT DOCOMO jointly launched with other partners having various vRAN-related technologies an initiative for collaborative co-creation solutions called the 5G Open RAN Ecosystem (OREC) . Since NTT DOCOMO had already introduced virtualization technology for its core network, such development and operational experience is becoming a key asset and input for the introduction of vRAN.
In this article, we describe the technical issues that must be addressed to achieve vRAN and NTT DOCOMO’s approach to addressing those issues. We also describe the importance of standardization and initiatives toward standardization of vRAN.
2. RAN virtualization technology
2.1 Network functions virtualization technology
Network functions virtualization (NFV)*1 relies on the offering of a virtualization layer on top of general-purpose hardware based on IT virtualization and orchestration technology*2. The virtualization layer exposes virtual resources that can be composed and used to run applications, including network function software applications. NFV transforms the way network operators can deliver communication services; such communication services had conventionally been provided by dedicated hardware and software optimized for satisfying the high-reliability and high-performance requirements of systems operated by telecom operators. NFV supports the decoupling of hardware and software, which in turn enables the rapid deployment and provision of new network functions simply by deploying and/or updating the software that implements the network function behavior. The use of open source software and agile development techniques cultivated in the IT field can also provide additional benefits such as shortening the time-to-service to service launch.
NTT DOCOMO has been researching and developing NFV technology and promoting its standardization by the European Telecommunications Standards Institute (ETSI) NFV Industry Specification Group since the first half of the 2010s. In parallel, NTT DOCOMO has been introducing virtualization technology into the core network of the commercial network since FY2015. By the end of FY2020, the virtualization rate of core network equipment for LTE and beyond exceeded 50%, and the 5G core network deployed in FY2021 became fully virtualized . NTT DOCOMO has reaped the following four benefits through NFV in the core network .
(1) Improved economic performance of network function equipment
There had been equipment and telecom software for each equipment vendor and network function, and it had been necessary to perform maintenance for each piece of equipment and each package of telecom software. However, NFV technology enabled NTT DOCOMO to run telecom software from multiple vendors on a unified virtualization platform, which improved cost performance, unified and simplified operations and maintenance, and enabled the use and sharing of low-cost general-purpose hardware.
(2) Quick deployment of new services
NFV shortened the time-to-service by eliminating the preparation and installation of dedicated hardware when introducing a new service.
(3) Improved communication service connectivity during times of congestion
In the event of congestion, e.g., caused by a natural disaster or a sudden concentration of traffic, NFV enabled the network-functions capacity to be automatically increased in a short period, thereby improving connectivity for consumer communication services.
(4) Improved reliability of communication services
The use of low-cost hardware enabled the construction of redundant hardware configurations on which telecom software could be automatically loaded to maintain normal operations upon detecting hardware failures. This enabled the automatic recovery of network functions in a short time (auto-healing), and such a capability eliminated the need to dispatch personnel for immediate maintenance work on site while achieving high maintainability and high reliability of consumer communication services.
2.2 Expected effects of introducing vRAN
Recent advances in IT virtualization technology, general-purpose hardware, and hardware accelerators have broadened the domains in which virtualization can be applied. It has even become possible to apply virtualization to baseband*3 processing on the radio layer, where high-level service requests such as high performance and sensitive real-time requirements are even more demanding than those of the core network. With this in mind, a number of domestic and overseas operators have begun initiatives toward the deployment and commercialization of vRAN using virtualization technology. The expected effects of introducing vRAN are summarized as follows (Fig. 1).
(1) Optimal combination of RAN solutions by decoupling hardware and software
(2) Simple and intelligent RAN operation and maintenance through virtualization and automation
(3) Infrastructure sharing and unified operations from the edge to core network
2.3 Issues in meeting vRAN requirements through NFV technology
To maximize the benefits of vRAN described above, it is necessary to resolve the technical issues that arise from specific RAN requirements in addition to those that arose in the virtualization of the core network.
Issues 1: Issues in achieving an optimal combination of solutions by decoupling hardware and software
(1) Support of high-performance and real-time processing executed using general-purpose hardware and software
In conventional base-station equipment, computationally intensive processing on the radio layer requiring real-time characteristics became feasible through dedicated hardware such as an optimized field-programmable gate array and application-specific integrated circuit devices and dedicated software. Instead, in vRAN, conventional base-station-equipment architecture featuring such tightly coupled hardware and software must be achieved using general-purpose hardware and software running on that hardware. It is necessary to reduce fluctuations in the time sensitivity of processes on the radio layer that affect base-station performance and achieve low latency. To this end, optimized general-purpose hardware must be selected, high-performance and time-sensitive software must be implemented to meet such requirements, and the necessary execution environment fulfilling all these requirements must in turn be supported by the virtualization platform. For functions and components having particularly severe processing requirements, studies must also be conducted on optimizing software processing even further or on offloading some of the radio-layer processing to a hardware accelerator. Finally, high-performance and high real-time sensitive processing must be achieved while dealing with the problem of tightly coupled hardware and software that has been (until now) typically assumed for fulfilling extreme performance requirements.
(2) Construction of a mechanism supporting the installation of base-station equipment
In virtualizing the core network, it has been possible to consolidate and install relevant core network functions at few centralized sites (datacenters). However, in the case of RAN, many more compact and distributed base-station sites (e.g., placed in a room within a building or outdoors) are necessary to cover the required radio coverage. Therefore, required performance, installation space, and power limitations greatly differ depending on the involved network functions to deploy at a target location. General-purpose hardware also needs to be able to withstand severe conditions such as high dust environments and extreme temperatures. In short, a mechanism is needed that can support scalable and flexible installation formats under such a variety of installation environments while ensuring an efficient decoupling of software and hardware.
Issues 2: Issues in achieving simple and intelligent RAN operation and maintenance through virtualization and automation
(1) Management of virtual resources provided to RAN applications
In conventional base-station equipment, base-station functions are coupled to specific hardware, so hardware and software can be managed as a set and maintenance can be conducted on that set. In vRAN, in which virtual resources are provided from multiple hardware devices, the RAN application management system dynamically requests the virtual resources needed and runs RAN applications on those virtual resources. In the latter, software and virtual resources are managed independently. To maintain base-station functions under these conditions, there is a need to associate software, virtual resources (logical), and hardware (physical). There is also a need to handle new hardware components, such as hardware accelerators, in a unified manner together with other virtual resources.
(2) Support of maintenance and operation use cases
With conventional base-station equipment, operation methods such as hardware configuration, installation, and maintenance differ among vendors. With virtualization, if these different operation methods were to be simply implemented without making any changes, and if an attempt would be made to support them all, the volume of development and amount of testing would certainly increase and not be bearable by the network operator. Therefore, there is a need to both promote the integration of more automation into the operations support systems and standardize solutions that clearly identify the functional splits and boundaries of systems to integrate. This is a necessary step to ensure unifying operational and maintenance procedures for all systems involved in vRAN deployment.
(3) Reduction in on-site work and manual operations
An entire workflow must be designed to achieve automation in processes ranging from the construction of base stations to be distributed nationwide and creation of virtualization platforms. It is particularly important to minimize on-site work and expand the scope of automation by expanding the scope of software control when migrating to vRAN. For this reason, it is essential that construction work include the kitting*5 of general-purpose equipment, provision of spare parts to be standardized to reduce on-site work, and various types of interfaces to be set up to operate and manage this equipment remotely.
Issues 3: Issues in achieving infrastructure sharing and unified operations from edge to core network
(1) Support of diverse applications by using a unified virtualization platform
To support the deployment of new network functions in the future, a vRAN virtualization platform will have to support the deployment and execution of base-station functions, core-network functions, and higher-level layer services. The execution-environment capability of the platform will require both supporting virtual machine (VM)-based network function deployments as well as container*6-based network function deployments, the use of which has increased. To make operations more efficient, there will also be a need for a virtualization environment that can support application requirements in each network domain.
(2) Achieving long-term operation of virtualization platforms distributed nationwide
With vRAN commercialization, we can expect many small-scale virtualization sites to be distributed nationwide, so how to go about maintaining and managing those platforms is an important issue. It will be necessary to upgrade and maintain a virtualization platform periodically to cope with the introduction of new general-purpose hardware and new hardware accelerators, need for security updates, evolution of applications, etc. All version upgrades of virtualization platforms must also be carried out without interrupting any communication services. Taking into account recent development trends in container platforms, it will also be necessary to make short-term upgrades including those to the platforms, but it is unrealistic to expect an operator to carry out large-scale upgrades and development, etc. continuously. A mechanism that can execute long-term and stable platform operations would therefore be desirable.
3. Addressing open issues: approaches
At NTT DOCOMO, to address the issues described above and achieving its vRAN goals, two approaches will be adopted: vRAN standardization activity in the O-RAN ALLIANCE*7 and ETSI NFV and interoperability testing in a multi-vendor environment through OREC.
For Issues 1, there will be a need to study, in cooperation with RAN-application vendors and general-purpose hardware vendors, the optimal implementation of applications and optimization of their performance. NTT DOCOMO will promote these activities within OREC, which are described later in this article.
For Issues 2 and 3, recognizing the prime importance of standardizing interfaces and information models*8 between vRAN and associated systems and in achieving unified operation and maintenance of vRAN, NTT DOCOMO will promote the standardization of vRAN. Standardization will enable various types of products provided by multiple vendors to be combined. If the support period for a particular product in a product configuration is coming to an end, a different product conforming to the same standard on the market can be used to minimize the impact on other products in that configuration and on surrounding systems. To avoid fragmentation of specifications related to NFV with the same objective among standardization organizations within the industry and make maximum reuse of the NFV-related specifications that NTT DOCOMO has been actively pushing for standardization, further coordination among standardization organizations will be promoted.
Standardization activities related to the virtualization and orchestration support for vRAN are progressing in the O-RAN Work Group 6 (WG6). The standardization items that NTT DOCOMO must take up on the basis of the overall architecture of O-RAN (Fig. 2) are summarized below.
4. Latest developments in O-RAN WG6 standardization
At O-RAN, standardization related to virtualization and orchestration are mainly being conducted in the Cloudification & Orchestration WG (O-RAN WG6). O-RAN WG6 specifications and reports consist of General Aspects and Principles (GAP) defining overall concepts; Orchestration Use Cases (ORCH-UC), which defines use-case and information flow between the specified functions; and other interface specifications that specify data models and protocols. Details on each of the above specifications and reports are given as follows.
In WG6, function names such as O-Cloud are defined and the relationships among those functions are described. Two types of GAP documents are delivered: O2GAP, specific to concepts related to the O2 interface, and AAL-GAP, specific to concepts about the Acceleration Abstraction Layer (AAL)*14.
For O-Cloud, Network Function (NF), and Near-RT RIC application (xApp), typical use cases related to provisioning*15, fault management, and performance management are specified.
(3) Interface specifications
As specified by O-RAN WG6, O2 is the reference point between SMO and O-Cloud and AAL is the abstraction layer for the provision of hardware accelerators by O-Cloud to NF Deployment. The interface specifications cover the reference point and abstraction layer on various further detailed aspects. For instance, the O2ims interface document specifies the interfaces exposed by the O-Cloud IMS on the O2; the AAL forward error correction (FEC) specification and AAL High-PHY detail an AAL profile for FEC and AAL profile for the hardware acceleration of High-PHY functions, respectively. The specification of the protocol and data models for the O2dms interface exposed by the O-Cloud DMS on the O2 is in progress.
The constituent elements of vRAN are NF Deployment that specifies the virtual resources of vRAN applications, O-Cloud that serves as the virtualization platform providing virtual resources to NF Deployment, and SMO that operates and manages NF Deployments and O-Cloud (Fig. 2).
The SMO functional block consists of Federated O-Cloud Orchestration and Management (FOCOM) and Network Function Orchestration (NFO), which operate, manage, and orchestrate O-Cloud and NF Deployment as logical functions, respectively. FOCOM is capable of inventory management and alarm management for O-Cloud, while NFO is capable of lifecycle management, alarm management, and performance management of NF Deployments on O-Cloud.
O-Cloud consists mainly of the following logical functions: the O-Cloud Node, which provides virtual resources to NF Deployment; DMS, responsible for the management of NF Deployments; IMS, which manages O-Cloud Node and other O-Cloud infrastructure components including DMS instances; and HW (Hardware) Accelerator Manager that manages hardware accelerators. The O-Cloud Node, in turn, consists of computing resources, network resources, and storage resources, where computing resources include AAL functions (described later). Possible solutions of DMS are assumed to be, for instance, Kubernetes® or OpenStack®. As previously introduced, DMS exposes its services and connects to SMO via the O2dms interface, while IMS connects to the SMO over the O2ims interface. IMS mainly provides inventory management, alarm management, and performance management for the O-Cloud Node and deployment of DMS instances.
4.3 Hardware accelerator
One of the key specifications O-RAN WG6 is focusing on is the support of hardware accelerators, which are typically required to implement vRAN. In this context, AAL includes a hardware accelerator as physical equipment, AAL profile for achieving various types of processing using the hardware accelerator, and AAL interface (AALI) between AAL profile and vRAN application. The HW Accelerator Manager manages AAL profiles supported by hardware accelerations that are part of O-Cloud.
Two major acceleration types for offloading radio processing to the hardware accelerator are look-aside acceleration and inline acceleration. The former offloads parts of radio layer 1 and particularly coding/decoding having high processing load, while the latter offloads all of radio layer 1. For either type, specifications are progressing to facilitate virtualization and orchestration.
Going forward, the plan at O-RAN WG6 is to complete O2dms interface specifications, AAL use-case specifications, and a vRAN application package and to deliver specifications with clear functional splits and function definitions, which are key for network operators who consider fully interoperable multi-vendor vRAN deployment.
5. NTT DOCOMO’s standardization activities
NTT DOCOMO has been actively promoting the standardization of core network virtualization at ETSI NFV. To maximize the reuse of existing ETSI NFV specifications and avoid fragmentation of NFV standards and specifications with the same objective within the industry, NTT DOCOMO has been studying and comparing O-RAN specifications and ETSI NFV specifications, as shown in Fig. 3. Specifically, O-RAN WG6 is specifying the O2dms interface while ETSI NFV is analyzing any gaps between O-RAN specifications and ETSI NFV specifications on the basis of ETSI GR NFV-IFA 046.
The goal is to standardize an integrated and unified NFV platform able to cover from RAN to the core network, as shown in Fig. 4. For the distributed equipment making up a base station, requirements and features that are unique to each functional block need to be considered, such as the limited length and constrained delay in the fronthaul between the base station and radio unit (RU)*16. NTT DOCOMO will therefore continue to study optimal deployment scenarios taking into account such requirements and features. NTT DOCOMO considers the following points to be important in promoting O-RAN and ETSI-NFV standardization.
5.1 Positioning of ETSI-NFV-defined NFV Orchestrator in SMO
In O-RAN specifications, SMO becomes a huge functional block that includes the functionality specified by the 3rd Generation Partnership Project Service & Systems Aspects WG5 (3GPP SA5) regarding an operations support system (OSS) and Element Manager (EM)/Network Function Management Function (NFMF) as well as the NFV Orchestrator (NFVO) (Fig. 3) functionality defined by ETSI NFV. Therefore, to achieve an SMO supporting a multi-vendor environment, dividing up the SMO framework into more granular functional blocks in the standardization process is considered essential. Since the OSS functional blocks differ between operators in terms of existing assets, operational workflow, and external equipment to be connected to, current discussions are pointing out the difficulty of handling all the relevant functionality only by SMO; as a result, there exists the possibility that typical OSS functions will not be included in SMO solutions.
While O1 termination is assumed to be EM/NFMF, as currently defined by 3GPP SA5, the deployment scenario has been to provide NF and EM as a single set from a vendor, but simplifying this process by deploying general-purpose EM (Generic EM) is desirable. In terms of virtualization, SMO would then take on the role of a Generic EM functional block that terminates O1 and a NFVO functional block that terminates O2.
5.2 Relationship between O-Cloud and Container Infrastructure Service Manager/Virtual Infrastructure Manager of ETSI NFV
The O2dms interface is currently being specified in O-RAN, and the finalization of detailed specifications is forthcoming. In terms of ETSI NFV specifications, two platforms having a deep relationship with DMS are being specified: Container Infrastructure Service (CIS) and CIS Manager (CISM) as a container platform and Virtual Infrastructure Manager (VIM) and NFV Infrastructure (NFVI) as a VM platform. The management entities (i.e., CISM and VIM) are considered to correspond to DMS (Fig. 3) functionality. For O2dms, studies are now being conducted on the use of ETSI NFV specifications such as ETSI GS NFV-SOL 018 , which specifies the interfaces produced by the CISM; ETSI GS NFV-SOL 014 , which specifies the data model of virtualized resource descriptors that can be used over the interfaces produced by the VIM; and ETSI GS NFV-SOL 003 , which specifies the interfaces produced by the Virtual Network Function Manager (VNFM) regarding VNF management, and that can be leveraged as an abstraction above the other aforementioned interfaces. These studies will determine whether VNFM functionality will belong to SMO or O-Cloud. The ETSI NFV will study the entire functionality of IMS and extend the ETSI NFV specifications so that NFV can support IMS or a related function.
6. Activities at OREC
Since there are many implementation-dependent functions not specified in standards and optional functions that are specified in standards, there is a need to align options and interfaces to combine a variety of products and conduct testing to develop vRAN solutions that can provide 5G services on a commercial level. In addition to promoting standardization at O-RAN and ETSI NFV, NTT DOCOMO is focusing its attention on the following points at OREC.
(1) Creation of guidelines for operational workflow and network design toward the provisioning of vRAN solutions
Considering that a variety of products can be combined from diverse vRAN application vendors, hardware accelerator vendors, and O-Cloud vendors, NTT DOCOMO is creating guidelines on achieving multi-vendor interoperability to cover various types of operational workflow and network designs. While a redundant configuration for the network and management nodes is essential in a cloud environment, it may not be possible to achieve suitable redundancy at a base-station site due to the installation environment, which would mean many cases in which the network disconnects or only a management node goes down. Due to such issues, the need arises for a variety of options, so studies in design guidelines are also proceeding from this viewpoint. The architecture of specific open-network initiatives targeted by NTT DOCOMO is shown in Fig. 5. With a focus on VNFM, NTT DOCOMO will work to maximize the use of ETSI NFV specifications by making various interfaces open with the end goal to achieve interoperability between vRAN applications and virtualization platforms delivered by multiple vendors.
(2) Provisioning of testing environments for end solutions and E2E function/performance testing
To use a variety of new technologies in vRAN, operational testing in terms of both functionality and maintenance is essential. To facilitate the use of hardware accelerators in new ways, performance testing will also be necessary. Testing of optimal hardware configurations at small-footprint base-station sites will likewise be necessary. NTT DOCOMO is now working on providing these testing environments to cover E2E testing from terminals to core networks.
Three combinations of vRAN applications, virtualization platforms, general-purpose servers, and hardware accelerators are being tested at the OREC Shared Open Lab. Verification of communications in a 5G stand-alone configuration has been completed, for look-aside hardware accelerators, the commercialization of which is in progress, as well as for inline hardware accelerators. NTT DOCOMO plans to accelerate these testing activities toward commercial-level quality testing.
This article provided an overview of NFV, described the effects of introducing virtualization in RANs, and the issues surrounding implementation, and presented NTT DOCOMO’s approach to addressing these issues. The article also described the current status of standardization at O-RAN, NTT DOCOMO’s standardization activities, and initiatives at OREC.
NFV has already produced positive effects in the deployment of NTT DOCOMO’s core network. To adapt such a vision and assets to make it extensible to also cover the virtualization of RAN, NTT DOCOMO is working and promoting standardization and OREC activities as two key approaches. Both standardization and OREC are evolving and aiming for a world that can achieve optimal combinations of solutions applicable to a variety of future network requirements. As a means to this goal, NTT DOCOMO will promote vRAN development together with its partners and conduct testing at the OREC Shared Open Lab. It will also contribute to standardization by expanding use cases and specifying interfaces so that functions that are key to operators can be provided and integrated easily, yet guaranteeing stability of the system for long-term telco support.
All company names or names of products, software, and services appearing in this article are trademarks or registered trademarks of their respective owners.