You need Adobe Reader 7.0 or later in order to read PDF files on this site.
If Adobe Reader is not installed on your computer, click the button below and go to the download site.

Feature Articles: Latest Trends of Multimedia Broadcasting Technologies for Mobile Terminals

Metadata Technologies in Multimedia Broadcasting for Mobile Terminals

Shinji Fukatsu^†, Kiyoshi Tanaka, Kyotaro Horiguchi, and Motoyuki Horii

Abstract

In this article, we explain metadata technologies with a focus on the use of metadata in multimedia broadcasting, which is a new type of broadcasting media service for mobile terminals that uses the new technical standard for broadcasting, ISDB-Tmm (integrated services digital broadcasting, terrestrial mobile multimedia), and offers both realtime and storage-based broadcasting services.

†	NTT Cyber Solutions Laboratories Yokosuka-shi, 239-0847 Japan

1. Introduction

Multimedia broadcasting is a broadcasting media service that provides live broadcast contents with high image and sound quality and storage-based broadcasting service in which contents are stored automatically and can be viewed anytime, anywhere. In particular, the storage-based service offers a wide variety of contents (images, documents, games, applications, files, and coupons in addition to video and audio programs). It enables that the integrated use of those two types of services and seamless browsing of their contents.

Our project has been researching and developing metadata technologies for content navigation, i.e., browsing, searching, retrieving, purchasing, and viewing content. In this article, we outline metadata, metadata delivery, and metadata operation in multimedia broadcasting and describe a metadata system that we have developed.

2. Metadata in multimedia broadcasting

Metadata in multimedia broadcasting consists of (1) EPG/ECG metadata, which contains information related to an electronic program guide (EPG) and electronic content guide (ECG) used for navigating content and (2) transmission control metadata, which contains information related to the reception, storage, and content complementation^*1 of storage-based broadcast content. Metadata information elements and metadata usage are outlined in Fig. 1 and explained below.

Fig. 1. Metadata in multimedia broadcasting.

*1	Content complementation: missing data is sent by the communication function and stored in order to complement the transmission by broadcasting and ensure that all of the content is received.

2.1 EPG/ECG metadata

The metadata schemas in multimedia broadcasting extend those in ARIB STD-B38 [1] and in TV-Anytime Phase 1 [2] and Phase 2 [3] to make them applicable to multimedia broadcasting service (ARIB: Association of Radio Industries and Businesses; STD: Standard). They consist of the nine metadata information elements described below. Among them, the coupon information element is newly introduced.

(1) Program information element

It describes general information about content such as Title, Synopsis, Genre, CreditList, and TitleImage. It is uniquely identified by a Content Reference Identifier (CRID).

(2) Group information element

It describes information about a group (series, package, etc.) of multiple program-information elements or multiple group-information elements. It is also uniquely identified by a CRID.

(3) Program location information element

This element has been extended to describe realtime and storage-based broadcasting services. The OnDemandService element describes information about storage-based broadcasting content, such as the channel (serviceIdRef) in the OnDemandService element and the broadcast period (StartOfAvailability, EndOfAvailability) and the uniform resource identifier (URI) of transmission control metadata (ProgramURL) in the OnDemandProgram element. The BroadcastEvent element describes information about realtime broadcasting content, such as the channel, broadcast period, and output restrictions.

(4) Service information element

It describes information about the service (channel) such as service name, service synopsis, and logo.

(5) Purchase information element

It indicates a purchase item, which is the charging unit for an individual content item or group, and describes information about the purchase item such as sales period, price, sales pattern (PayPerView, PayPerMonth, etc.), and URI for purchasing (PricingServerURL).

(6) License information element

The License is uniquely determined by a content (Program) and product (PurchaseIdRef) set and supports a variety of charging formats. It describes the number of times the content can be viewed or used and the period during which the content can be viewed or used, etc. It also describes license-issuing conditions and preview information (number of times, ending time, etc.) of realtime broadcasting content.

(7) Segment information element

It describes information related to a scene in video and audio content such as the scene’s start and end points, name, and thumbnail URI. It can also be applied to a chapter playback function.

(8) Segment group information element

It describes information related to a group (chapter list etc.) that combines multiple segment information elements.

(9) Coupon information element

A coupon service will be provided in multimedia broadcasting in accordance with market trends. The coupon information is newly introduced. It describes information about a coupon such as coupon name, amount of discount, and period of validity. It also describes the reference relationship with the product targeted by the coupon as well as the reference relationship with the content or purchase item from which the coupon is being provided.

2.2 Transmission control metadata

Transmission control metadata extends the metadata in the Third Generation Partnership Project (3GPP) [4] to make it applicable to multimedia broadcasting service. It consists of three information elements.

(1) User service description

It contains the two information elements described below and describes the CRID as a key for establishing the correspondence between EPG/ECG metadata and transmission control metadata. Furthermore, the CRID indicates the logical storage location of received content within the device. For example, if a device uses an SD (Secure Digital) memory card, the content is stored at /private/tmm/aaa/ (in the case of crid://aaa). In this way, the user can use the stored content even if he or she moves the SD memory card to another device.

(2) Session description

It describes information related to the reception of content such as a FLUTE (file delivery over unidirectional transport) session ID (a=flute-tsi), the destination address for broadcasting the content, and the content size. It also describes the file names of all transmitted objects, the transmission order of those objects, and the transmission period (a=transmission schedule), etc.

(3) Associated delivery procedure description

It describes information for content complementation such as the server URI and period for content complementation, which is a threshold for performing content complementation. It also describes the server URI and schedule for submitting a reception report, etc.

3. Metadata transmission and delivery

In multimedia broadcasting, metadata is provided by both broadcasting and communication. When it is transmitted by broadcasting, the bandwidth required for transmitting it and the frequency of its reception must be considered. Two methods have been proposed: one method transmits small amounts of metadata often (method A), while the other transmits large amounts of metadata periodically (method B).

3.1 Metadata transmission by broadcasting

(1) Metadata transmission method A

This method transmits the metadata for content that will be delivered within a few hours (e.g., 48 hours) from now. The metadata transmitted by this method is updated over time (e.g., at one-hour intervals) and the same metadata is periodically retransmitted within the time frame, as shown in Fig. 2(a). This approach provides metadata for generating an EPG/ECG for the next few hours within a short time and for updating the EPG/ECG immediately when a program is changed.

(2) Metadata transmission method B

This method groups together the metadata for content that will be delivered within a fixed period (e.g., 2–7 days from the present) and transmits it as storage-based broadcast content. The metadata transmitted by this method is updated on a regular schedule (e.g., every day from 01:00 to 01:10), as shown in Fig. 2(b). This approach provides the large amount of metadata for generating an EPG/ECG for a certain fixed period at one time and minimizes the bandwidth required for transmitting it.

Fig. 2. Metadata transmission by broadcasting.

3.2 Metadata delivery by communications

In this approach, the device sends a search request by either HTTP GET^*2 or POST^*3 to the metadata system operated by the service provider and receives metadata or a list of metadata (a list of CRIDs) corresponding to the search request. There are several query patterns such as searching by content title, keywords, period, and cast and some sorting functions such as sorting search result by content title, release date, duration, etc.

*2	GET: A method for requesting a page from a server.
*3	POST: A method used when transferring form data to a server.

4. Metadata usage

This section introduces the use of metadata for multimedia broadcasting with a focus on elements and attributes that have been added or extended for this purpose.

4.1 Content attributes

Multimedia broadcasting can handle a wide variety of content such as documents and applications in addition to video and audio programs. For this reason, a Content Properties element for describing the content attributes of the wide variety of content has been added. This element describes, for example, the type of content (audio, video, image, document, metadata set, etc.), file format (html, mp4, jpg, pdf, zip, container), and file size. Here, file size can be used to calculate the amount of memory needed to a store the content when it is ordered.

4.2 Device attributes

Many types of devices will be used in multimedia broadcasting and the specifications of mobile phones from different mobile carriers differ widely. This means that the type of content that can be handled will depend on the device. For this reason, a list describing the types of devices that can handle content (usable by all devices, usable by only devices of mobile carrier A, etc.) has been defined. Device attributes are described in the Genre element of metadata.

4.3 Version

The functions and features of devices (screen size, number of colors, supported audio/video codecs, etc.) are diverse and content specifications (profile and level of video, version of markup language, etc.) are also diverse. Moreover, new functions and specifications will be added.

To cope with this situation, version information is broken down by the functions that devices have and described in the keyword element of metadata. Here, the device compares the version information described in the metadata with its own version information and judges whether it can use the content corresponding to the metadata (it is judged to be usable if the device version is greater than or equal to the content version).

4.4 Event Information Table and metadata linkage

In multimedia broadcasting, both the EPG and ECG are generated using metadata. Moreover, to enable immediate EPG/ECG updating in response to program changes (extensions, interruptions, etc.), an Event Information Table (EIT) of the present, preceding, and subsequent programs is operated. The EIT is linked to EPG/ECG metadata by including the EIT’s service ID and event ID in the EPG/ECG metadata’s CRID.

4.5 Advanced expression

It is now common for mobile email and web pages to embed links, images, and pictograms. For this reason, EPG and ECG are being given the same advanced expression by encapsulating <A> or <img> tags in a CDATA section and they describe the CDATA section in the title or synopsis element of metadata.

5. Metadata system

The configuration of a metadata system for registering, managing, and delivering metadata by communications for use in multimedia broadcasting is shown in Fig. 3.

The metadata registration and delivery server receives configuration information from the service provider and generates EPG/ECG metadata, which conforms to the multimedia broadcasting schemas and registers that metadata in the metadata database. In the registration process, the EPG/ECG metadata is broken down into elements and attributes and registered in attribute tables in order to deal with a wide variety of searches. The metadata is also converted to binary format MPEG-7 (BiM) to minimize the data size and registered in a document table. The server is equipped with a function for receiving search requests from devices and returning search results.

The in-memory cache server manages a memory cache of search results and a memory cache for search requests set up beforehand on the operator side. In this process, reconstructed search requests are used as keys and the body of the search results are used as values.

If identical requests are received, the use of these caches can speed up the search response time (8.4 times faster on average than using the database) and reduce the processing load during simultaneous searches from many devices.

Fig. 3. Metadata system.

6. Future activities

The launching of commercial services for multimedia broadcasting requires the establishment of standards and technical reports, the development of a variety of systems, and trial operations. NTT Cyber Solutions Laboratories aims to contribute to the realization of multimedia broadcasting services and the linking of these services with other types of services and technologies.

References

[1]	ARIB Standard, “Coding, Transmission and Storage Specification for Broadcasting System Based on Home Servers,” ARIB STD-B38, 2010 (in Japanese).
[2]	ETSI, “Broadcast and On-line Services: Search, Select, and Rightful Use of Content on Personal Storage Systems (“TV-Anytime Phase 1”; Part 3: Metadata; Sub-part 1: Metadata Schemas,” ETSI TS 102 822-3-1V1.5.1, 2009.
[3]	ETSI, “Broadcast and On-line Services: Services: Search, Select, and Rightful Use of Content on Personal Storage System (“TV-Anytime”); Part 3: Metadata; Sub-part 3: Phase 2: Extended Metadata Schema,” ETSI TS 102822-3-3 V1.3.1, 2009.
[4]	GSM, “3rd Generation Partnership Project; Technical Specification Group Services and System Aspects; Multimedia Broadcast/Multicast Service (MBMS); Protocol and Codecs,” 3GPP TS 26.346, 2008.

	Shinji Fukatsu Senior Research Engineer, Promotion Project 2, NTT Cyber Solutions Laboratories. He received the B.E., M.E., and Ph.D. degrees in electrical, information, and energy engineering from Osaka University in 1997, 1999, and 2002, respectively. He joined NTT Cyber Solutions Laboratories in 2002 and has been studying human interface technologies for the web. During 2005–2009, he worked at NTT Plala Inc. and engaged in service planning and system development of Internet protocol television (IPTV) services. He is currently studying metadata technologies and developing metadata systems. He is a member of the Institute of Electronics, Information and Communication Engineers (IEICE).
	Kiyoshi Tanaka Senior Research Engineer, Supervisor, Human Interaction Project, NTT Cyber Solutions Laboratories He received the B.E., M.E., and Ph.D. degrees in communication engineering from Osaka University in 1992, 1994, and 2005, respectively. Since joining NTT in 1994, he has been engaged in research on video-on-demand systems and interactive video systems and services, which currently mean IPTV and its services, first at NTT Human Interface Laboratories and then at NTT Cyber Solutions Laboratories. His current activities include standardization of IPTV and multimedia broadcasting for mobile devices. He is a member of IEICE, the Human Interface Society in Japan, and the organizing committee of the Broadcast Communication and Computing Group of the Information Processing Society of Japan.
	Kyotaro Horiguchi Manager, Infrastructure Software Technology Unit, NTT Open Source Software Center. He received the B.E. and M.E. degrees in engineering from Waseda University, Tokyo, in 1990 and 1992, respectively. After joining NTT in 1992, he was mainly engaged in R&D of various types of database management systems. Recently, he has been engaged in developing a series of TV-Anytime-based program metadata systems, which have been commercially deployed.
	Motoyuki Horii Senior Research Engineer, Supervisor, Promotion Project 2, NTT Cyber Solutions Laboratories. He received the B.S. and M.S. degrees in computer science from Keio University, Kanagawa, in 1986 and 1988, respectively. Since joining NTT in 1988, he has mainly been engaged in R&D of natural language processing, agent communication, and IPTV systems. He is currently developing multimedia broadcasting systems and introducing them to NTT subsidiary companies.

↑ TOP