To view PDF files

You need Adobe Reader 7.0 or later in order to read PDF files on this site.
If Adobe Reader is not installed on your computer, click the button below and go to the download site.

Feature Articles: Video Technology for 4K/8K Services with Ultrahigh Sense of Presence

Research and Development Policies for Ultra-high-presence Video Technology toward 4K/8K Services

Yoshiko Sugaya, Hiroshi Fujii, Atsushi Sato,
Hiroaki Matsuda, Shinichiro Chaki,
and Hirohito Inagaki

Abstract

Efforts toward achieving 4K/8K broadcast services have been ramping up in countries throughout the world, and in Japan, broadcasters, telecommunications carriers, and equipment manufacturers have teamed up to establish the Next Generation Television & Broadcasting Promotion Forum (NexTV-F) to make 4K/8K communications and broadcasting a reality. As a founding company of NexTV-F, NTT is becoming a world pioneer in the development and promotion of 4K/8K services through a variety of research projects and technical developments. This article introduces these technical developments and the ultra-high-presence services targeted by NTT.

Keywords: 4K/8K, ultra-high-presence services, ultra-high-definition video

PDF PDF

1. Introduction

The transition to digital terrestrial broadcasting was completed in Japan in the summer of 2012, and as 2013 got underway, the term 4K/8K suddenly became popular as a reflection of the desire to achieve a level of resolution and picture quality far beyond that of high-definition (HD) images currently in use in terrestrial television (TV) broadcasts. In addition, business possibilities using 4K/8K ultra-high-definition video began to expand amid expectations that Tokyo’s successful bid for the 2020 Summer Olympics would increase the demand for high-presence video and accelerate the spread of 4K TV to homes. The NTT laboratories have long been active in the research and development (R&D) of technologies for 4K/8K communications and broadcasting, and today, their efforts toward providing full-scale 4K/8K services are gathering momentum.

In July 2012, NTT Network Innovation Laboratories conducted a joint experiment with NHK on the delivery of Super Hi-Vision 8K video of major sporting events for public viewing [1]. This video was transmitted between London and Japan over a shared Internet protocol (IP) network making use of high-speed error correction technology based on low-density generator matrix (LDGM) code. Meanwhile, NTT Media Intelligence Laboratories, as a long-time promoter of H.265/MPEG-H, also known as HEVC (High Efficiency Video Coding) as an international standard, announced in July 2013 the development of an HEVC-compliant software encoding engine with the world’s best video compression performance. Then, in August of the same year, NTT Advanced Technology began domestic and overseas sales of the world’s first HEVC software codec development kit, called HEVC-1000-SDK [2]. Studies and trials of 4K video delivery have also been quite active within the NTT Group, which seeks to get an early jump on the commercialization of 4K/8K and the development of peripheral business ventures. For example, NTT WEST held the world’s first 4K video Internet-delivery trial to a set-top box (STB) at IMC (Interop Media Convergence) TOKYO 2013 in June 2013. In this trial, HEVC-compressed 4K video content was streamed to an NTT WEST STB over the Internet, thereby demonstrating the feasibility of 4K TV for home use [3]. The delivery of 4K video was also demonstrated at NTT GROUP COLLECTION 2013 held in October 2013.

In addition to the above, NTT Plala has announced plans to hold a 4K trial with the HIKARI TV video service in the fourth quarter of fiscal year (FY) 2013 and to conduct a trial delivery of a 4K video-on-demand (VOD) service in the first quarter of FY2014 ahead of actual IP broadcasts. The company is also collaborating with the Shitamachi Bobsleigh Network Project Promotion Committee on the production of 4K-video content for this project.

2. Overseas trends in 4K/8K broadcasting

Efforts toward achieving 4K/8K broadcasting are not limited to Japan; there has been quite a lot of activity throughout the world since 2012. In France, the 4EVER project got underway in 2012 as a three-year project to advance R&D in the field of HEVC and ultra-high-definition TV (UHDTV). As part of this project, relay broadcasts of the French Open tennis tournament in 4K UHD format were conducted for public viewing using 4K displays in June 2013. In Korea too, 4K trial broadcasts were held in October 2012 by five leading cable broadcasters.

3. Trends toward 4K/8K broadcasting at NexTV-F

In Japan, the Next Generation Television & Broadcasting Promotion Forum (NexTV-F) was founded by NTT, NHK, Sky Perfect JSAT, and Sony in June 2013 with the aim of achieving early deployment of next-generation broadcast services involving 4K/8K broadcasting and smart TVs [4]. A total of 21 Japanese companies consisting of broadcasters, telecommunications carriers, equipment manufacturers, and advertising agencies made up NexTV-F at the time of its founding. In more detail, the objective of NexTV-F is to upgrade broadcast services toward 4K/8K broadcasting through a variety of activities ranging from the study of transmit/receive provisions and specifications and the preparation of broadcast systems, to content production, encoder development, testing, and holding of trial broadcasts. Plans are being made to broadcast various types of sports events as milestones using as a guide the roadmap prepared at the end of May 2013 by the Study Group on Upgrading of Broadcasting Services at the Ministry of Internal Affairs and Communications. For example, NexTV-F is planning to construct an environment for holding trial broadcasts in 2014, the year of the Brazilian World Cup, so that viewers interested in next-generation broadcasting can experience 4K broadcasts. Similarly, in 2016, the year of the Rio de Janeiro Summer Olympics, NexTV-F is planning to construct an environment for holding 8K trial broadcasts in addition to launching 4K broadcast services, and in 2020, the year of the Tokyo Summer Olympics, it is planning to launch 8K broadcast services. As one of the founding companies of NexTV-F, NTT has been promoting studies on specifications for real-time compression encoding in 4K broadcasts and collaborating with Japanese manufacturers in developing hardware-encoding large-scale integrated circuits (LSIs) compliant with HEVC next-generation encoding technology.

4. R&D policies toward ultra-high-presence services

NTT has a future vision of ultra-high-presence services and therefore aims to create an enriching and rewarding user environment by providing a user experience that combines high-definition video, high-presence audio, and diverse human sensations. Various elements will be needed to achieve this, including HD video, free viewpoint video, high-presence audio (ambient sound, free listening point), and communication of the five human senses. Among these, the NTT Group sees the development of services using HD video as the starting point in providing ultra-high-presence services and becoming the world leader in this field (Fig. 1).


Fig. 1. Future vision of ultra-high-presence services.

With a view to 2020 and beyond, NTT laboratories are researching and developing technologies toward the creation of HD-video business in the three areas described below (Fig. 2).


Fig. 2. Concept of HD video services.

(1) Smart TV services combining communications and broadcasting

The market surrounding high-definition video linking 4K broadcasting and Smart TV is expected to expand rapidly both inside and outside Japan. The aim here is to provide HD video services in a lifestyle-friendly format for a wide range of people from mass users to enterprise users.

Our first step will be to simultaneously achieve 4K delivery and 4K retransmission services over optical fiber. This will be achieved by holding technical trials led by the NTT Group in accordance with the 4K/8K broadcasting roadmap established by NexTV-F as well as by developing the world’s first HEVC hardware encoder and developing video transmission technologies. Furthermore, we will conduct studies on the application of 4K technologies to smart TV and the creation of HD smart TV content. In Japan, the smart TV market is expected to reach about 3 million units (2,080,000 TVs and 860,000 STBs)—or about 5% of all TVs in Japan—by 2017 [5], while in the 4K TV world market, the number of units shipped is expected to increase dramatically from 500,000 units in 2013 to 7,250,000 units in 2016 [7]. In such a business environment, we can expect the expansion of high-level, compelling services such as FLET’S TV and HIKARI TV based on 4K HD video to go hand in hand with getting users to enjoy the great advantages that optical circuits can offer.

At the same time, the market for live video delivery services as an enterprise application in Japan is predicted to grow rapidly from 640 million yen in FY2010 to 744.2 billion yen in FY2020 [7], and with the coming of the Tokyo Olympics, the expected growth in live video delivery services for public viewing and other mass-user applications is expected to stimulate the market in Japan for HD video.

(2) Video content transmission services

4K/8K broadcasting must be made to be economically feasible by compressing the bandwidth of video content. In this regard, we are developing an HEVC hardware encoder for achieving low-delay, high-quality video transmission [8]. Using this encoder in video transmission equipment developed by the NTT Group will make it possible to provide HD video transmission services for transmitting and receiving large volumes of HD video data in real time. In this way, we aim to increase the number of network users making use of video relays and other facilities and thereby expand the market.

(3) HD video usage services

Needless to say, we can expect to see various applications using 4K/8K HD video in the medical-care, education, and content-production industries, and the expansion of the HD solutions business can be anticipated. In particular, it is our desire at NTT to create novel services using HD video such as video transmission for remote medical care, remote classes, and efficient, cloud-based video production. In this way, we hope to improve medical care as well as the social environment and welfare of the people through the use of high-presence video.

5. Issues and technologies surrounding the provision of 4K/8K services

Achieving a low transmission bandwidth by reducing network load and shortening the delay in real-time transmission are two issues that must be addressed if a variety of 4K/8K services are to be provided. There is also a need for technologies that can facilitate the provision of high-level, compelling services that include the transmission of HD video. NTT laboratories are making a concerted effort to resolve these issues by conducting R&D in a variety of fields (Fig. 3).


Fig. 3. R&D fields at NTT laboratories toward the provision of 4K/8K services.

5.1 Activities at NTT Media Intelligence Laboratories

NTT Media Intelligence Laboratories is involved in a wide range of 4K/8K activities. These include the standardization of HEVC next-generation encoding technology as a key component in lowering transmission bandwidth by reducing network load, the development of a software encoding engine and encoding LSI, and R&D of high-presence-video viewing technology. The HEVC standard achieves a compression rate that is generally twice that of the existing H.264/MPEG-4 AVC (Advanced Video Coding) (also called H.264) video compression standard. This means that it can keep the transmission bandwidth within 30 Mbit/s even for 4K video and can enable 4K-based communications on the existing NGN (Next-Generation Network).

To provide video delivery services such as 4K VOD (video on demand), it must be possible to set optimal compression parameters depending on the type of content being delivered. This can be achieved by applying a software encoding engine using HEVC [9].

However, high-presence-video transmission services for live video relays, public viewing, and other real-time applications require real-time encoding and transmission, which can be achieved through the use of real-time video transmission equipment using an HEVC LSI.

There will also be a need for innovative technologies such as interactive panoramic video so that high-presence-video services can help make ever-changing individual lifestyles even more enjoyable and interesting [10].

5.2 Activities at NTT Network Innovation Laboratories

NTT Network Innovation Laboratories is engaged in the R&D of high-reliability IP transmission technology to enable the short-term (temporary) use of global, multi-domain networks for transmitting not only 4K video but also large-capacity 8K video content. It aims, in particular, to achieve highly reliable, end-to-end transmission from the IP network (backbone/access circuits) to the reception of IP packets in the user’s environment via low-cost network switches and receiving terminals. To this end, NTT Network Innovation Laboratories is participating in discussions at MPEG toward the development of the new MMT (MPEG Media Transport) standard while also proposing and promoting the standardization of LDGM code as a robust error correction technology in the application layer so that users need not rely solely on high-quality leased lines [11].

5.3 Activities at NTT Network Technology Laboratories

NTT Network Technology Laboratories is researching and developing techniques for evaluating video quality. These techniques are needed since the user’s quality of experience in relation to video display size, viewing position, and compression rate is thought to differ from that of conventional HD video. Studies are being conducted with the aim of establishing reliable techniques for assessing the quality of video in 4K broadcasting and video transmission services [12].

6. Future activities

A variety of issues must still be addressed in the NTT Group ranging from video production to transmission and viewing given the launch of 4K broadcasting in 2016 and 8K broadcasting in 2020. Solving these issues will, of course, involve the development of key technologies at NTT laboratories, but it will also require R&D activities focused on open innovation, as well as coordination with the NTT business companies and the establishment of alliances with other companies.

If we look at the timeline for future plans, market trends, and business strategies with a view to 2020, we can say that now is the time for the technology born of R&D in 4K/8K high-presence video to break out of its shell, emerge in the form of services, and spread its wings on a global scale. The NTT laboratories will continue to research and develop advanced technologies in a mutually supportive role with the NTT Group as the latter works to become a world leader in the provision of ultra-high-presence services.

References

[1] Press release published by NTT on February 22, 2011 (in Japanese).
http://www.ntt.co.jp/news2011/1102/110222a.html
[2] Press release published by NTT on August 8, 2013.
http://www.ntt.co.jp/news2013/1308e/130808a.html
[3] Press release published by NTT WEST on June 11, 2013 (in Japanese).
http://www.ntt-west.co.jp/news/1306/130611a.html
[4] Handout issued by NexTV-F on June 17, 2013.
http://www.nextv-f.jp/en/pdf/press20130617_2.pdf
[5] Information material issued by Nomura Research Institute, Ltd. on November 30, 2012 (in Japanese).
http://www.nri.com/jp/event/mediaforum/2012/pdf/forum184.pdf
[6] NPD DisplaySearch: “Quarterly TV Design and Features Report,” 2013.
http://www.displaysearch.com/cps/rde/xchg/displaysearch/hs.xsl/quarterly_
tv_design_features_report.asp
[7] Research summary released by Yano Research Institute Ltd.
http://www.yanoresearch.com/press/pdf/775.pdf
[8] T. Oonishi, T. Sano, K. Yokohari, K. So, M. Ikeda, A. Sagata, H. Iwasaki, and A. Shimizu, “HEVC Hardware Encoder Technology,” NTT Technical Review, Vol. 12, No. 5, 2014.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr201405fa3.html
[9] K. Iso, M. Kitahara, R. Tanida, T. Mitasaki, N. Ono, and A. Shimizu, “World’s Highest-performance HEVC Software Coding Engine,” NTT Technical Review, Vol. 12, No. 5, 2014.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr201405fa4.html
[10] Y. Tanaka and D. Ochi, “Interactive Distribution Technologies for 4K Live Video,” NTT Technical Review, Vol. 12, No. 5, 2014.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr201405fa5.html
[11] T. Nakachi, T. Yamaguchi, Y. Tonomura, and T. Fujii, “Next-generation Media Transport MMT for 4K/8K Video Transmission,” NTT Technical Review, Vol. 12, No. 5, 2014.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr201405fa6.html
[12] K. Kawashima and J. Okamoto, “QoE Assessment Methodologies for 4K Video Services, ” NTT Technical Review, Vol. 12, No. 5, 2014.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr201405fa7.html
Yoshiko Sugaya
Researcher, Visual Media Project, NTT Media Intelligence Laboratories.
She received the B.E. and M.E. degrees in mechanical engineering from Sophia University, Tokyo, in 2005 and 2007, respectively. She joined NTT Cyber Space Laboratories in 2007 and began conducting research on computer vision and projector-camera systems. She was subsequently engaged in product planning and development of telecommunication devices. Since 2013 she has been working on the development of high-definition visual communication service systems.
Hiroshi Fujii
Senior Research Engineer, Visual Media Project, NTT Media Intelligence Laboratories.
He received the B.E., M.E., and Ph.D. degrees in computer science from Kyoto University in 1989, 1991, and 2000, respectively. He joined NTT Information and Communication Processing Research Laboratory in 1991 and studied video distribution systems including multiple accesses to video data, content protection, video data storage, and digital watermarking. Then he developed digital rights management systems in a development project of the copyright management platform. Since 2006, he has been studying video encoding including SVC, HEVC, and evaluation of subjective video quality. He is a member of the Information Processing Society of Japan.
Atsushi Sato
Senior Research Engineer, Supervisor, Promotion Project 1, NTT Media Intelligence Laboratories.
He received the B.S. degree in mechanical sciences and engineering from Tokyo Institute of Technology in 1987 and the M.E. degree in information processing from Tokyo Institute of Technology in 1989. In 1989, he joined NTT Human Interface Laboratories, where he mainly engaged in R&D of computer vision and medical image processing systems and community ICT service systems. Recently, he has been developing high definition visual communication service systems.
Hiroaki Matsuda
Senior Research Engineer, Supervisor, Promotion Project 1, NTT Media Intelligence Laboratories.
He received the B.S. degree in communications engineering and the M.E. degree in information engineering from Tohoku University in 1987 and 1989, respectively. After joining NTT LSI Interface Laboratories in 1989, he engaged in R&D of VLSI design methodology for video coding and video codec systems, and content sharing platform systems. Recently, he has been working on the development of high definition visual communication service systems.
Shinichiro Chaki
Executive Research Engineer, Promotion Project 1, NTT Media Intelligence Laboratories.
He received the B.E. degree in electric and electrical engineering from Sophia University, Tokyo, in 1986 and the M.E. degree in electrical communication engineering from Waseda University, Tokyo, in 1988. He joined NTT Switching System Laboratories in 1988 and studied broadband NW architecture and switching systems based on ATM (Asynchronous Transfer Mode) technologies. From 1997 he was involved in developing an ATM PON system in NTT Access Network Service Systems Laboratories. He then engaged in developing network architecture and router systems for the FLET’s premium network in NTT Service Integration Laboratories and FLET’s Next in NTT Network Service Systems Laboratories. At NTT Media Intelligence Laboratories, he was developing systems based on speech, language, and image processing technologies. In March 2014, he moved to NTT Advanced Technology Corporation. He is a member of the Institute of Electrical and Electronics Engineers (IEEE) and the Institute of Electronics, Information and Communication Engineers (IEICE).
Hirohito Inagaki
Vice President, Head of NTT Media Intelligence Laboratories.
He received the B.E. and M.E. degrees in electrical engineering from Keio University, Kanagawa, in 1984 and 1986, respectively. Since joining NTT Electrical Communication Laboratories in 1986, he has been engaged in R&D of natural language processing and its applications. From 1994 through 1997, he was on transfer to NTT Intelligent Technology Co., Ltd., where he developed information security systems and multimedia systems. He moved to NTT Cyber Space Laboratories (now NTT Media Intelligence Laboratories) in 2006. His research interest is R&D of audio, video, and language interfaces and their applications. He is a member of IEICE.

↑ TOP