Monitoring the Quality of IPTV Services

1. Introduction

Internet protocol television (IPTV) services are now widely provided. The quality of experience (QoE) of IPTV services is affected by the audiovisual content, encoding and decoding techniques, network performance, and display technology. It is therefore important to ensure that end-users receive high-quality IPTV content. The ideal way to do that would be for the service provider to identify the quality degradation factors, monitor QoE by taking the main quality degradation factors into account, and finally, to conduct a thorough investigation to determine the cause of quality degradation and then address the problems.

2. Activities of ITU-T SG12 in quality monitoring

An IPTV processing chain is shown in Fig. 1. This processing chain consists of video acquisition and editing, encoding, network transmission, decoding, and display, as mentioned in section 1. All of these elements can affect end-users’ QoE. ITU-T (International Telecommunication Union, Telecommunication Standardization Sector) Recommendation G.1081 [1] defines performance monitoring points for IPTV services that will enable the service provider and/or network operator to monitor the performance of the entire IPTV service delivery process. There are five points in the processing chain where performance quality is monitored: the source media and metadata are monitored at point 1; encoded and packetized source media are monitored at point 2; packet transmission characteristics are monitored at point 3; received packets are monitored at point 4 to determine whether they can adequately provide the required QoE at the client terminal, and the displayed media are monitored at the client terminal at point 5. If all the information obtained at these monitoring points is integrated, the locations where quality has degraded can be determined.

Fig. 1. IPTV processing chain.

ITU-T Recommendation G.1081 does not define how objective quality assessment models should be applied at each performance monitoring point. Monitoring the QoE at the head end is important because quality degradation at points 1 and 2 influences the QoE of all users. In these cases, full reference (FR) media-layer models are suitable for monitoring quality. At points 3 and 4, it is preferable to analyze packets using a method with low computational load because the analysis needs to be implemented in terminals such as mobile terminals, home gateways, and set-top boxes (STBs). In these cases, packet-layer models are suitable for estimating QoE from IP packet header information. At point 5, it is essential to monitor the QoE by using the no-reference (NR) media-layer model, which estimates QoE from media signals received at the client terminal. Because FR media-layer models [2]–[4] that can be applied to head-end QoE monitoring have already been standardized, NTT has focused on developing packet-layer models that can be used for end-user QoE monitoring using packet header information [5]. Application areas can be categorized into lower resolution (LR: quarter common intermediate format (QCIF), quarter video graphics array (QVGA), and half VGA (HVGA)) and higher resolution (HR: standard definition (SD) and high definition (HD)) areas. The LR application area can be used for mobile IPTV services, and the HR application area can be used for STB based IPTV services. In 2006, ITU-T study group 12 (SG12) also launched a project that aims to develop quality monitoring models for LR and HR application areas. In October 2012, ITU-T standardized two Recommendations; ITU-T Recommendation P.1201.1 [6] can be used for the LR application area, and P.1201.2 [7] can be used for the HR application area. This article introduces ITU-T Recommendation P.1201.1 as it is most relevant because of the recent rapid growth of Mobile IPTV services. The P.1201.1 model was developed by integrating the NTT and Huawei models and was verified in an ITU-T performance evaluation contest.

3. P.1201.1 model

A block diagram of the P.1201.1 model is shown in Fig. 2. This model consists of three modules: parameter-extraction, parameter-calculation, and quality-estimation modules as follows.

1) Parameter-extraction module

This module extracts a Real-time Transport Protocol (RTP) timestamp, sequence number, and marker bit from RTP headers and the rebuffering starting time and length from the client terminal.

2) Parameter-calculation module

This module calculates audio- and video-related quality parameters (e.g., coding bit rate and packet loss) using parameters extracted by the parameter-extraction module.

3) Quality-estimation module

This module estimates audio, video, and audiovisual quality using quality parameters calculated by the parameter-calculation module. Additionally, each module has a sub-module, as shown in Fig. 2.

Fig. 2. Block diagram of P.1201.1 model [5].

This article summarizes the quality degradation factors processed by the quality-estimation modules. The details of the parameter extraction and calculation modules are not explained here, as they are beyond the scope of this article.

4. Performance of P.1201.1 model

ITU-T SG12 verified the validity of the P.1201.1 model by conducting numerous subjective tests using audio, video, and audiovisual sequences that were generated by varying the codec type, coding bit rate, packet-loss pattern, packet-loss concealment, and rebuffering pattern. The audio, video, and audiovisual quality estimation modules were verified by using the root mean square error (RMSE) and Pearson’s correlation (PC). Table 1 (see Appendix of ITU-T Recommendation P.1201 [8]) lists RMSE and PC values. Additionally, ITU-T verified that the P.1201.1 model reached a sufficient level of quality-estimation accuracy because the RMSE was small and PC was high, as listed in Table 1.

Table 1. Performance of P.1201.1 model [7].

5. Application scenario

In-service quality monitoring is an important application of the P.1201.1 model. The P.1201.1 model can be applied as followed:

6. Conclusion

This article introduced a quality estimation model we developed for IPTV services and the P.1201.1 model. The P.1201.1 model has reached a sufficient level of quality-estimation accuracy and can be applied to monitor the quality of mobile IPTV. As described previously, the P.1201.1 model cannot be used for transmission control protocol (TCP)-based progressive video streaming because it was developed for realtime IPTV services. Although there are different types of video streaming, quality factors such as coding and rebuffering in realtime video streaming are the same as those of TCP-based video streaming. Therefore, it may be possible to extend the model and apply it to TCP-based video streaming if the parameter-extraction module can process TCP headers. Although the application area of the P.1201.1 model is limited to lower resolution, it is desirable to extend the model to the higher resolution application area because the newer mobile terminals support higher resolution. ITU-T SG12 is planning to extend the P.1201.1 model because of these technological developments.

In addition to promoting the P.1201.1 model, to which NTT contributed, our group is involved in globally competitive research and development of a quality-monitoring technique for a new service.

References

[1]	ITU-T Recommendation G.1081, “Performance monitoring points for IPTV,” Oct. 2008.
[2]	ITU-T Recommendation J.341, “Objective Perceptual Multimedia Video Quality Measurement of HDTV for Digital Cable Television in the Presence of a Full Reference,” Jan. 2011.
[3]	ITU-T Recommendation J.144, “Objective Perceptual Video Quality Measurement Techniques for Digital Cable Television in the Presence of a Full Reference,” Mar. 2004.
[4]	ITU-T Recommendation J.247, “Objective Perceptual Multimedia Video Quality Measurement in the Presence of a Full Reference,” Aug. 2008.
[5]	K. Yamagishi, K. Ushiki, T. Hayashi, and A. Takahashi, “Packet-layer Model for End-user QoE Management,” NTT Technical Review, Vol. 7, No. 4, 2009. https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr200904sf6.html
[6]	ITU-T Recommendation P.1201.1, “Parametric non-intrusive assessment of audiovisual media streaming quality––lower resolution application area,” Oct. 2012.
[7]	ITU-T Recommendation P.1201.2, “Parametric non-intrusive assessment of audiovisual media streaming quality––higher resolution application area,” Oct. 2012.
[8]	ITU-T Recommendation P.1201, “Parametric non-intrusive assessment of audiovisual media streaming quality,” Oct. 2012.