To view PDF files

You need Adobe Reader 7.0 or later in order to read PDF files on this site.
If Adobe Reader is not installed on your computer, click the button below and go to the download site.

Feature Articles: Technologies for Managing Smart Energy

Efforts Toward International Standardization for Green Datacenters

Toshihiro Hayashi and Tetsuya Tominaga


In this article, we describe international standardization activities for datacenter energy management systems, especially those of ITU-T (International Telecommunication Union, Telecommunication Standardization Sector). International standardization activities have a significant impact on research and development results for smart energy management at datacenters and telecommunication buildings being put into practical use and used widely in the future.

NTT Energy and Environment Systems Laboratories
Musashino-shi, 180-8585 Japan

1. Introduction

Demand for datacenters has been increasing each year with the expansion of housing services and hosting services as well as the advent of cloud computing. In datacenters, the amount of information and communications technology (ICT) equipment has increased and the rack density has become higher with the result that power consumption has continued to grow. This trend is observed not only in Japan but also worldwide. Therefore, it is fair to say that energy-saving and greening efforts at datacenters are urgent issues for achieving a sustainable low-carbon society.

Under these circumstances, NTT is researching and developing smart energy management at datacenters and telecommunication buildings, i.e., a datacenter energy management system (DEMS). It is intended to save energy in datacenters and telecommunication buildings by using air conditioning control and power supply control in collaboration with ICT equipment allocation control. More details are given in the Feature Article “Energy Management of Telecommunication Buildings and Datacenters” [1].

Anticipating a future with DEMS-related technologies being used widely, we have also been putting effort to international standardization activities based on activities at the International Telecommunication Union, Telecommunication Standardization Sector (ITU-T). In this article, we introduce the trend of ITU-T standardization related to datacenters and describe the topics on which we are working for standardization toward the spread of DEMSs.

2. Standardization efforts at ITU-T

As shown in Fig. 1, Study Group 5 (SG5) is working on theme of environment and climate change in ITU-T. Working Party 3 (WP3/5) on “ICT and climate change”, one of three working parties under SG5, has been studying the environmental impact of ICT use [2]. As of October 2011, WP3/5 consists of 7 questions (Qs) (Q17/5 may merge with Q20/5, but this is still under approval). Several work items are set for Q17/5, as listed in Table 1 and are being studied for the preparation of recommendations and handbooks.

Fig. 1. Structure of ITU-T Study Group 5 (as of December 2011).

Table 1. List of work items under Q 17/5 (as of December 2011).

Among these items, the ones currently being studied are energy efficiency measurement and metrics for telecommunication equipment and telecommunication infrastructure. Telecommunication equipment covers broadband equipment on both the network and user sides, and telecommunication infrastructure mainly refers to datacenters. To implement their energy saving and greening, it is necessary to quantify the environmental load of equipment and infrastructure in order to develop a plan for environmental load reduction. Therefore, establishing common metrics for energy efficiency and their measuring methods is considered to be essential. Recommendations for metrics and their measuring methods will be prepared first and then target values for the recommended metrics will be discussed.

In parallel with the abovementioned activity, best practices for green datacenters are being documented (L.DC; numbered as L.1300). L.DC is based on the “EU Code of Conduct for Data Centres [3]” published by the Joint Research Centre of the European Commission and has been prepared for the purpose of identifying and introducing measures for improving datacenter energy efficiency. Practices that contribute to reductions in datacenter power consumption are collected and categorized such as utilization, management, and planning of datacenters; ICT equipment and services; cooling; power supply equipment; buildings; and monitoring. L.DC was approved as L.1300 in December 2011.

3. Standardization topics

NTT has been involved in standardization activities on three topics in ITU-T SG5 WP3/5 for spreading DEMS-related technologies: (1) plates and ducts for correcting airflow directions for rack-mounted equipment, (2) a minimum set of internal information for datacenter equipment, and (3) a datacenter evaluation method. These are described in more detail in sections 3.1–3.3.

3.1 Plates and ducts for correcting airflow directions for rack-mounted equipment

In order to gain the full energy-saving effect that a DEMS should provide by its integrated control of ICT equipment and air conditioners, the airflows within a datacenter should be designed to prevent the cold air supplied by air conditioners mixing with hot air exhausted from ICT equipment, and a datacenter environment based on such an airflow design should be created. One common datacenter environment has cold aisles and hot aisles. Since most servers have an air supply opening at the front and an exhaust opening at the rear, racks are placed with their front sides facing each other and cold air is supplied to form a cold aisle and with their rear sides facing each other to form hot aisles.

However, some network equipment has air supply and exhaust openings in different positions such as air supply at the bottom and exhaust on the top, or both air supply and exhaust at the sides. If ICT equipment with different airflow directions is mounted thoughtlessly on the racks, local hot spots are created because it takes in hot air exhausted from the adjacent equipment, leading to an increase in cooling load and operating power of the air conditioners.

To improve the efficiency of datacenter air conditioning, we have been researching airflow control using plates and ducts, as illustrated in Fig. 2, so that each kind of ICT equipment can take and exhaust air efficiently. We submitted this result as a best practice for L.DC, and we proposed specification requirements that plates and ducts should be equipped as standard with rack-mounted equipment that needs air­flow direction control. Our proposal was included in the approved L.1300.

Fig. 2. Airflow direction control for rack-mounted equipment by using plates (ducts).

3.2 Minimum set of internal information for datacenter equipment

One of the most important aspects of the DEMS is monitoring of the status of facilities and equipment within a datacenter. The DEMS cooperatively controls various kinds of datacenter equipment on the basis of the monitoring information, which enables the load balance of multiple air conditioners to be adjusted and thus improves their energy efficiencies.

However, efforts to promote a multivendor environment with equipment of various kinds have resulted in the internal information that can be acquired from such equipment not being standardized, which leaves difficulties for integrated control at present. The internal information such as power consumption and temperature of ICT equipment is defined by the private management information base (MIB) of each vendor, but no unified format is available. Moreover, some items may not be measured for certain equipment. Therefore, we proposed studying the minimum required internal information data set, i.e., the minimum set for monitoring and integrated control associated with energy saving for datacenter equipment. It is intended to unify the list of items so that necessary internal information for monitoring and control can be acquired without any dependence on vendors. This proposal was agreed to be included in the approved L.1300.

Other related organizations and their ongoing activities are listed in Table 2: European Telecommunications Standards Institute (ETSI) is studying a data set related to the monitoring and control of air conditioning and power supply [4], Ecma International is studying a data set for integrated management and operation of smart datacenters [5], and Internet Engineering Task Force Energy Management is studying an MIB for energy management of equipment within or connected to a telecommunication network [6]. We are promoting the preparation of recommendations in cooperation with these organizations.

Table 2. Organizations and data sets under study.

3.3 Datacenter evaluation method

Power usage effectiveness (PUE) and datacenter infrastructure efficiency (DCiE) are commonly used as metrics representing a datacenter’s energy saving capability [7]. PUE is obtained by dividing the total power consumption of ICT and facility equipment in a datacenter by the ICT equipment’s power consumption. A datacenter is said to be most efficient if its total power consumption equals that of its ICT equipment, which gives PUE = 1.0. DciE, on the other hand, is the inverse of PUE and is shown as a percentage. Both metrics are recommended by The Green Grid, an industry organization for promoting energy saving at datacenters.

While PUE and DCiE are useful in evaluating the energy efficiency of facility equipment, they are not fully sufficient on their own in evaluating the overall energy saving capability of a datacenter because there could be a case, for example, in which the evaluated value may improve just because there has been an increase in the power consumption of ICT equipment. We need to seek not only better PUE and DCiE values, but also further reductions in the overall power consumption as well.

Moreover, even if ICT equipment tasks are allocated suitably by the DEMS and the power consumption per service becomes smaller through the energy-saving operation of ICT equipment, the PUE and DCiE values could become worse without significant reduction in the power consumption of air conditioners. In order to evaluate the effect of the DEMS quantitatively, we need to take account of the productivity generated by the datacenter.

Furthermore, datacenters have redundant structures as well as backup & restore functions to ensure system reliability corresponding to service-level demand, as shown in Fig. 3, but such differentiating factors are not reflected in PUE and DCiE. Although system reliability improves as the equipment structure is made redundant, the energy efficiency could decrease owing to a decrease in the operating rate of redundant equipment.

Fig. 3. Service level and redundancy in equipment configurations at datacenters.

For these reasons, we proposed a datacenter evaluation method that has an appropriate balance by considering additional parameters representing productivity (capabilities of servers, network equipment, and storage, etc.) and reliability (availability, redundancy of equipment configuration, etc.)

Datacenter productivity metrics are also being studied by The Green Grid, and one issue under discussion is how to define the useful output generated by a datacenter. We will continue to work on recommendations while ensuring consistency with the efforts of The Green Grid.

4. Future development

Monitoring and cooperative control of a datacenter by a DEMS require the control of airflow directions, unification of data sets, and establishment of an evaluation method. We intend to promote international standardization activities based on the activities at ITU-T while working collaboratively with related organizations such as ETSI, Ecma International, IETF Eman, and The Green Grid.


[1] K. Matsuda, T. Tominaga, and M. Yanagi, “Energy Management of Telecommunication Buildings and Datacenters,” NTT Technical Review, Vol. 10, No. 1, 2012.
[2] ITU-T SG5.
[3] Joint Research Centre of the European Commission.
[4] ETSI.
[5] Ecma International.
[6] IETF Eman.
[7] The Green Grid, White Paper #22, “Usage and Public Reporting Guidelines for The Green Grid’s Infrastructure Metrics (PUE/DCiE)”.
Toshihiro Hayashi
Energy Optimization Technology Group, Energy System Project, NTT Energy and Environment Systems Laboratories.
He received the B.E. and M.E. degrees in geosystem engineering from the University of Tokyo in 2004 and 2006, respectively. He joined NTT Energy and Environment Systems Laboratories in 2006. His research interests are energy control technologies, optimization techniques, and environmental impact assessment of ICT services.
Tetsuya Tominaga
Senior Manager, Energy Optimization Technology Group, Energy System Project, NTT Energy and Environment Systems Laboratories.
He received the B.E. and M.E. degrees in mechanical engineering from Doshisha University, Kyoto, in 1989 and 1991, respectively. Since joining NTT Telecommunication Networks Laboratories in 1991, he has been researching and developing electromagnetic environment systems, telecommunication equipment protection against lightning surges, and security technology related to electromagnetic waves. He also has a lot of experience in standardization related to these research topics; e.g., he is the Rapporteur of ITU-T SG5 Question 15 and a member of IEC SC77C and of CIGRE C4.206.