To view PDF files

You need Adobe Reader 7.0 or later in order to read PDF files on this site.
If Adobe Reader is not installed on your computer, click the button below and go to the download site.

Feature Articles: QoE Estimation Technologies

Toward QoE-centric Operation of Telecommunication Services

Akira Takahashi and Takanori Hayashi


The rapid spread in the use of smartphones and cloud-based services has accelerated the speed of evolution of telecommunications application software, resulting in a great diversity of such software. While the quality of experience (QoE) of telecommunications services has been directly related to network performance in the past, the current situation is not that simple. This is because the QoE of telecommunications applications is affected not only by network performance, but also by server and terminal performance to a great extent. In this article we propose a framework for improving user QoE, and we introduce recent research and development activities underway at NTT intended to achieve this goal.


1. Quality of experience (QoE) and network performance

All of the factors affecting the end-to-end quality of conventional public switched telephone network (PSTN) services are under the control of telecommunications carriers except for telephone terminals. The characteristics of terminals are clearly defined in standards and in domestic laws, for example, the Regulations for Telecommunications Facilities for Telecommunications Business. Therefore, if the PSTN network is confirmed to be operating normally, it is very rare for an end user to encounter a serious problem. This implies that QoE ≈ (is approximately equal to) network performance holds in the PSTN world.

When the Internet era arrived, the variation in terminals drastically increased. At that time, terminals consisted mainly of personal computers (PCs). However, in the 1990s, the primary factor affecting the QoE of web-browsing was network speed, referred to as throughput, since network capacity was limited. The relationship QoE network performance was still maintained.

With the arrival of the broadband era in the late 1990s, the behavior of telecommunications application software had become increasingly complex. This resulted in a change in the dominant factors of QoE; that is, the server and terminal performance seriously affects QoE. For example, the video quality of Internet protocol television (IPTV) is dependent on terminal technologies such as forward error correction. In cloud-based applications, the server mainly transmits executable programs, and the terminal executes them in order to create and render the content. This means that PC performance is more important than network speed. Moreover, the use of wireless local area networks and power-line communication in home networks has brought new quality degradation factors that cannot be monitored or controlled by network carriers.

The above-mentioned examples reveal that the relationship QoE network performance no longer holds. What makes the situation more complicated is that even network operation has become difficult in an end-to-end sense due to the multi-provider environment (Fig. 1).

Fig. 1. Factors affecting QoE of IPTV.

If we take all of these emerging factors into account, we can conclude that the first step towards maximizing user QoE is to quantify and visualize it.

2. QoE-centric operation

As a telecommunications carrier, NTT has made every effort to maintain and improve network performance through daily network operations. However, due to changes in the balance of quality factors, there are quite a few problems that cannot be found and solved through network operation only. How can a network carrier solve the problems in the following examples?

2.1 Example 1

Bob likes watching movies. He was interested in the video on demand menu of an IPTV service. He decided to subscribe to the IPTV service and to a Next Generation Network (NGN) service as well. He placed the set-top box (STB) in his living room, although the home gateway was placed in the pantry, where the former PSTN line was installed. It was difficult to lay Ethernet cable from there to the living room, so he bought a Wi-Fi router, on whose box was the claim “300 Mbit/s Wi-Fi! No degradation even for HD (high definition) videos!”

In the beginning, he enjoyed watching movies via the NGN. However, he began experiencing serious video degradation that continued to increase, and the video sometimes froze. He did not notice—which was not surprising—that the Wi-Fi router and the digital cordless telephone that he had recently bought used the same frequency range and were interfering with each other.

He was too angry about the quality to think of calling the customer support center. Rather, he remembered the flyer about the cable TV that the neighborhood self-governing body had recommended, and he immediately called the cable company. Then, he cancelled the IPTV service and at the same time cancelled the NGN service because the cable company salesman had suggested that it would be much cheaper to use the cable network for his telephone and Internet service.

The cable company laid the cable directly to the STB in the living room. Thus, Bob did not use the Wi-Fi router. Naturally, he experienced no degradation and therefore believed that cable TV was much better than IPTV. Since then, Bob talks about his experience to his friends whenever he has a chance and stresses the superiority of cable TV over IPTV.

2.2 Example 2

A corporate customer (Company A), was planning to replace the business applications they were using. At the same time, they were considering the possibility of using the cloud environment provided by a telecom operator (Operator B) in order to reduce costs. However, they were worried about quality degradation such as response latency in using the application software. Therefore, they asked a sales engineer of Operator B about this and were told that Operator B had carried out a complete performance evaluation and found no problems.

After a one-month trial by the telecommunications service division of Company A, they decided to adopt the proposal by Operator B since they had found no performance problems. They then extended the trial to all their divisions.

A couple of weeks later, the telecommunications service division started receiving many complaints from several locations about the very slow response of the application software. The person in charge of the telecommunications service division in Company A did not experience any degradation, so he asked Operator B to check the transmission performance of the virtual private network (VPN) provided by Operator B. Operator B enhanced the bandwidth of the VPN, and the performance improved in some locations. However, most users still experienced serious delays in using the application software.

Finally, because of this problem, Company A decided not to accept Operator B’s proposal when the trial was finished. Nobody knew that many users in Company A had installed a certain kind of software in the background, and this software wasted CPU (central processing unit) power, resulting in poor rendering performance for browser-based applications.

Although the above examples are fictitious, they could actually occur. The issue here is that it is becoming increasingly difficult to understand user dissatisfaction based only on conventional network management. We often take a questionnaire approach to determine QoE problems. However, this takes a lot of time to solve the problems. Thus, we have been studying methodologies for estimating user QoE based on parameters that can be observed by service/network providers. These technologies will contribute to the operation of telecommunications services based on user QoE, which we call QoE-centric operation.

3. QoE estimation technologies

These Feature Articles introduce QoE estimation technologies with which we can estimate user QoE based only on observable parameters such as packet-header information. Conventionally, such technologies were assumed to be applied to off-line quality measurement, and therefore, the media-signal-based approach was taken to analyze speech waveforms or pixel data of videos. This approach requires an algorithm capable of handling a very heavy and complex computational load. However, to enable online real-time monitoring of QoE, passive measurement and a light-weight computational load are necessary for estimating QoE.

The first article in this issue, which is entitled “Monitoring the Quality of IPTV Services” [1], introduces a QoE estimation methodology for UDP (user datagram protocol)-based IPTV. The validity of this technology was thoroughly evaluated by ITU-T (International Telecommunication Union, Telecommunication Standardization Sector) and was standardized as ITU-T Recommendation P.1201.1 in 2012.

The second article, entitled “Playback State Estimation of Progressive Download-based Video Services” [2], introduces a similar technology for TCP (transmission control protocol)-based progressive-download applications such as BeeTV, which is provided by NTT DOCOMO, and YouTube. The method developed at NTT intelligently exploits the mechanism of TCP and achieves QoE monitoring at any point between a server and terminal.

The third article, entitled “Performance Estimation Techniques for Browser-based Applications” [3], describes techniques that enable QoE monitoring of browser-based applications, which are widely used in cloud-based environments. These techniques make it possible to estimate the waiting time for complex content that uses recent web technologies such as Ajax and Flash.

Finally, in the fourth article, “QoE Assessment Methodologies for 3D Video Services” [4], we introduce techniques to evaluate the quality of 3D video and also describe their objective estimation methodology. In evaluating 3D video, it is important to quantify the degree of fatigue, for example, in addition to the conventional picture clarity.

4. Expanding horizons of service-quality research

Up to this point, we have emphasized the importance of evaluating/estimating QoE. It is clear that evaluation/estimation is the first step toward realizing QoE-centric operation, for which we also need to develop methodologies for mapping network, server, and terminal performance onto QoE. In addition, it is important to develop appropriate customer relations management policies.

The concept of QoE-centric operation is a framework for providing and maintaining good telecommunications quality. We also propose a framework in which network providers improve user QoE in collaboration with service providers and end users. This is not the conventional one-way quality given by network providers, but a mutual effort with service providers and end users (Fig. 2). We call this Co-creation Quality.

Fig. 2. Concept of Co-creation Quality.

Obtaining information on the current quality and possible alternatives is essential in order to encourage users to take action to improve their QoE. For example, users of public transportation such as trains expect the trains to run exactly on schedule. By contrast, people who drive their own cars need information on traffic jams and weather forecasts to help them decide their route and schedule. Similarly, we believe network providers should provide more information on quality to users and/or service providers. To do this, technologies for monitoring, predicting, and visualizing information on quality are necessary.

Such information could be used to improve customer support by service providers. For example, if a service provider can obtain the current QoE information of their end users from a network provider simply by requesting it with a user ID, they can recommend an alternative to the end user when quality was low before receiving a complaint from the user (Fig. 3). It may also be possible to optimize the behavior of application software so that it adjusts to current and future quality conditions of networks. We call this kind of network function a Quality API (application programming interface) and consider it part of the network API.

Fig. 3. Quality API.

The technologies introduced in this article are fundamental to achieving QoE-centric operation, Co-creation Quality, and Quality APIs in the future. NTT R&D (research and development) will continue such studies in the future.


[1] K. Yamagishi, N. Egi, and T. Tominaga, “Monitoring the Quality of IPTV Services,” NTT Technical Review, Vol. 11, No. 5, 2013.
[2] H. Honda, H. Yamamoto, S. Nakamura, and A. Takahashi, “Playback State Estimation of Progressive Download-based Video Services,” NTT Technical Review, Vol. 11, No. 5, 2013.
[3] H. Yamamoto, S. Nakamura, H. Honda, and A. Takahashi, “Performance Estimation Techniques for Browser-based Applications,” NTT Technical Review, Vol. 11, No. 5, 2013.
[4] K. Yamagishi, T. Kawano, and K. Kawashima, “QoE Assessment Methodologies for 3D Video Services,” NTT Technical Review, Vol. 11, No. 5, 2013.
Akira Takahashi
Manager of the IP Service Network Engineering Group, Communication Traffic & Service Quality Project, NTT Network Technology Laboratories.
He received the B.S. degree in mathematics from Hokkaido University in 1988, the M.S. degree in electrical engineering from California Institute of Technology, USA, in 1993, and the Ph.D. degree in engineering from the University of Tsukuba, Ibaraki, in 2007. He joined NTT in 1988 and has been engaged in the quality assessment of audio and visual communications. He was a co-Rapporteur of ITU-T Question 13/12 on Multimedia QoE and its assessment during the 2004–2008 Study Period. He is a Vice-Chairman of ITU-T Study Group 12 (SG12) for the 2009–2012 and 2013–2016 Study Periods. He is a Vice-Chairman of the Technical Committee of Communication Quality in the Institute of Electronics, Information and Communication Engineers (IEICE). He received the Telecommunication Technology Committee Award in Japan in 2004 and the ITU-AJ Award in Japan in 2005. He also received the Best Tutorial Paper Award from IEICE in Japan in 2006 and the Telecommunications Advancement Foundation Award in Japan in 2007 and 2008.
Takanori Hayashi
Manager of the Service Assessment Group, Communication Traffic & Service Quality Project, NTT Network Technology Laboratories.
He received the B.E., M.E., and Ph.D. degrees in engineering from the University of Tsukuba, Ibaraki, in 1988, 1990, and 2007, respectively. He joined NTT in 1990 and has been engaged in the quality assessment of multimedia telecommunication and network performance measurement methods. He received the Telecommunication Advancement Foundation Award in Japan in 2008.