 |
|||
|
|
|||
|
Feature Articles: New Value Creation through Collaboration with B2B2X Partners Vol. 17, No. 1, pp. 20–27, Jan. 2019. https://doi.org/10.53829/ntr201901fa5 A Kabuki and Information Communication Technology Collaboration: Kabuki × ICTAbstractAt NTT, we have been promoting various initiatives concerning the application of information and communication technology (ICT) to kabuki, the traditional Japanese performing art. Specifically, we have been conducting joint experiments with Shochiku Co., Ltd. since 2016 aimed at providing new ways to appreciate kabuki by using ICT. We have also been providing technology for Cho Kabuki supported by NTT, which has been performed since 2016 at the Niconico Chokaigi festival held by Dwango Co., Ltd. With the success of these achievements, NTT and Shochiku are planning to conduct commercial performances at the Minamiza Theatre in Kyoto starting in 2019. In this article, we introduce past initiatives concerning kabuki × ICT and outline future prospects. Keywords: kabuki, ICT, collaboration  1. Initiatives involving joint experimentsIn 2016, NTT and Shochiku Co., Ltd. initiated a joint experiment aiming to create a new way to appreciate kabuki using information and communication technology (ICT). The view at NTT was that it would be beneficial to collaborate with people engaged in high culture in order to provide deep excitement and new experiences through ICT looking ahead to 2020. At the same time, Shochiku was looking into creating new content to further disseminate the charm of kabuki while passing on traditional kabuki to the rapidly increasing number of inbound customers to Japan. NTT and Shochiku agreed to start collaborating in experiments, four of which have been done so far. 1.1 Kabuki in Las VegasAt the venue of KABUKI LION SHI-SHI-O: The Adventures of the Mythical Lion, which was presented by Shochiku in Las Vegas, USA, in May 2016, we used ICT to encourage non-Japanese people to experience the charm of kabuki. At the same time, to provide kabuki-viewing experiences to many people beyond the barriers of time and distance, we conducted a variety of experiments at not only the Las Vegas performance venue but also at a remote location, namely, Haneda Airport in Japan. 1.1.1 Remote live transmission to Haneda AirportFootage captured by nine 4K cameras installed at the kabuki performance venue in Las Vegas was encoded using high-compression HEVC (High Efficiency Video Coding) developed by NTT and then synchronized in real time using MMT (MPEG* Media Transport) technology to flexibly synchronize multiple streams of video and audio. Accordingly, we achieved the world’s first live international transmission of 4K multiscreen videos. At Haneda Airport, by synchronously presenting 4K multiscreen videos as an all-around view, namely, a front-stage view (three screens), an elevated-walkway (through the audience to the stage) view (one screen on the left side and three screens at the rear), and a ceiling view (two screens above), it was possible to reproduce the performance of kabuki actors across the board and provide an audience experience like being in a seat surrounded by the actual stage and elevated walkway (Photo 1).
Moreover, we used object extraction technology, which is an element of our immersive telecommunication technology called Kirari!, to enable the kabuki actor Ichikawa Somegoro VII (now Matsumoto Koshiro X) to give a remote stage greeting. In other words, we created an experience in which it appeared that Mr. Ichikawa was actually holding a press conference at the Haneda Airport venue in person, when he was actually in Las Vegas (Photo 2).
1.1.2 Interactive exhibition experiment outside the theaterIn front of the theater in Las Vegas, we exhibited Henshin Kabuki (transformation kabuki), namely, an interactive-experience exhibition that combines advanced technologies of NTT with the motif of kabuki’s unique makeup method known as kumadori. The mysterious world combining culture with technology that Japan boasts of has gained such a favorable reputation that people who wanted to try this interactive experience—who were mainly non-Japanese—lined up to give it a try (Photo 3).
With Henshin Kabuki, an image of the kumadori mask selected by the person trying the experience is first captured. The kumadori mask image is automatically recognized with high precision regardless of its direction and inclination by NTT’s angle-free object-search technology. A kumadori pattern is then superimposed on the face of the person and displayed as augmented reality. Furthermore, a demonstration was carried out of the concept of amplified experience—by which projection mapping with dynamic presentations such as pause and expressions specific to kabuki is performed on a giant three-dimensional (3D) facial object, and important elements in the production are extracted and made to stand out beyond reality—along with the key-point extraction and emphasis technology necessary for realizing that concept. In addition, we changed the faces of 50 kumadori masks hanging on a wall surface, which typically do not move, so that they laugh, get angry, and form a variety of expressions by using NTT’s Hen Gen Tou (deformation lamps) light projection technology. We also applied Hen Gen Tou to offer new stage effects that give movement to background images conventionally used in kabuki. Additionally, we used interactive distribution technology for omnidirectional video in an experiment to transmit 4K omnidirectional video at low bandwidth while maintaining the quality of experience when viewing on site. In this manner, we have made efforts to expand the performance of kabuki both inside and outside the venue. 1.2 Kabuki Virtual TheatreTo expand the theatrical opportunities for kabuki fans in remote regions and to widely convey the appeal of kabuki, we set up Kabuki Virtual Theatres at the Kumamoto Prefectural Office in collaboration with NTT, NTT WEST, Kumamoto Prefecture, and Shochiku in March 2017 and at Paruse Iizaka, Fukushima City, in collaboration with NTT, NTT EAST, Fukushima Prefecture, and Shochiku in March 2018 as events to promote reconstruction efforts after the Kumamoto Earthquake and Great East Japan Earthquake, respectively. 1.2.1 Kabuki Virtual TheatreUtilizing Kirari! technology made it possible to present performance images of KABUKI LION SHI-SHI-O as a new screening work full of realism that gave viewers the virtual sensation that the kabuki actors were performing right in front of them. In addition, in Fukushima, we experimentally demonstrated establishing an economical and versatile image-projection system. Visitors to the Kabuki Virtual Theatre really appreciated the wonder of kabuki and the latest technology as well as the increased opportunities to view kabuki remotely in rural areas (Photo 4).
1.2.2 Henshin KabukiAn interactive-experience exhibition called Henshin Kabuki—which became popular in Las Vegas—was packaged in movable containers that allowed people in many different areas to experience the charm of kabuki. By utilizing the mobility of the containers, we were able to visit disaster areas in Kumamoto Prefecture and Fukushima Prefecture (Photo 5). A wide range of people, from young children to the elderly, visited the Henshin Kabuki container and responded very positively to the experience.
1.2.3 Kabuki ShoutIn Fukushima Prefecture, we also exhibited Kabuki Shout—which lets people enjoy an experience like becoming a master of omukou (in which the kabuki audience shouts out to the actors)—by using NTT’s acoustic processing technology (Photo 6).
An omukou shout is yelled toward the picture of an actor displayed on the big screen, and speech recognition is performed (even in noisy spaces) using NTT’s intelligent-microphone technology. Real-time wave-field synthesis technology is then used to propagate the sound in three dimensions synchronously with the movement of the voice. In addition, when the omukou shout hits the actor’s picture, the picture responds with various reactions depending on factors such as timing and kind of voice. Highlighting in this way the feature of omukou shouts by applying the latest technology enables people to become familiar with kabuki in a new form. 1.3 Real and virtual collaboration dance Miyako Musubi Yume no RenjishiIn November 2017, as the third part of the joint experiment, we implemented a special project called Miyako Musubi Yume no Renjishi within the Hajimemashite Kabuki (Nice to meet you kabuki) event being held by Kyoto City, with the aim of providing young people with opportunities to get to know traditional industries by experiencing the culture of Japan. This experiment utilized object extraction technology and ultra-realistic media synchronization technology in the Kirari! set of technologies as well as dance performances by Nakamura Hashinosuke IV, Nakamura Fukunosuke III, and Nakamura Utanosuke IV, who performed at Miyagawacho Kaburenjo Theatre. The performances were transmitted in real time to the Pontocho Kaburenjo Theatre (1.5 km away) in such a manner as to create a virtual collaboration with Nakamura Shikan VIII performing at the Pontocho Kaburenjo Theatre (Photo 7). Thanks to the utilization of ICT in this way, we received comments such as “Amazing” and “Very impressed” from many young people after they watched the four artists performing in a perfectly synchronized manner as if they were all performing at the same venue.
2. Efforts concerning Cho Kabuki supported by NTTCho Kabuki was first performed in 2016 at the Niconico Chokaigi event held by Dwango Co., Ltd., and three works have been produced through 2018. NTT and Dwango have been collaborating in business efforts related to video and social services since 2013, and NTT has presented a research and development (R&D) booth at Niconico Chokaigi since 2014. We have also been providing extra sponsorship and technologies for Cho Kabuki since Niconico Chokaigi 2016. NTT is taking up the challenge to create completely new kabuki productions using the latest technologies (including a technique called shadow cloning using Kirari!) and to broaden the ways of enjoying kabuki. We have advanced the technologies and stage effects each year in efforts to expand the younger generation of kabuki fans. 2.1 Shadow cloning using object extraction technologyWe have been implementing a technique called shadow cloning with Cho Kabuki every year since its inception. This technique is used to create productions that were not possible in the past by extracting video images of kabuki actors in real time using object extraction technology and stereoscopically projecting the extracted video images to different places. In 2016, shadow cloning was showcased for the first time in the play Hana Kurabe Senbonzakura to project multiple extracted videos of Nakamura Shido II, and we received a great response. In 2017 in the play Kuruwa Kotoba Awase Kagami, it was possible to create the impression that Nakamura Shido and Sawamura Kuniya were fighting impressively in a virtual action scene between their respective avatars by simultaneously operating multiple object extraction systems. In 2018, in the play Tsumoru Omoi Hana no Kaomise—Mata Kuru Haru Cho Kabuki no Nigiwai, the use of object extraction technology with improved extraction accuracy by machine learning enabled robust object extraction even in scenes where there was a change in the background that had never been done before. As a result, an action sequence with Nakamura Shido and the vocaloid Hatsune Miku performing a climax scene was successfully produced (Photo 8).
Shadow cloning continues to evolve year after year as a key technology underpinning a familiar stage effect of Cho Kabuki, and it has enabled us to provide new theater experiences every year. 2.2 New stage effects using double-sided transparent multilayer aerial-image display deviceThe paper-craft box for “Kirari! For Mobile” (multilayer aerial-image display technology) makes it possible to easily view 3D images in combination with smartphones. In the play Hana Kurabe Senbonzakura, this technology was used to enable viewers to enjoy Hatsune Miku performing in a box on the palm of the hand and to experience the flavor of Cho Kabuki. In 2018, we extended this multilayer aerial-image display technology to develop a double-sided transparent multilayer aerial-image display device that enables three layers of aerial images to be viewed simultaneously from the front and back by controlling the optical path length for multiple display devices. In the play Tsunoru Omoi Hanano Kaomise—Mata Kuru Haru Cho Kabuki no Nigiwai, this technology was used to create a new effect that had previously been impossible; namely, Hatsune Miku appears as if on a float, the background of which is expressed in multiple layers, so it seems like she is parading around the stage (Photo 9).
In addition to the above technologies, we used acoustic technologies, namely, virtual speaker technology and wave-field synthesis technology to create a powerful sound effect with high presence. Through these efforts, we received shouts from the audience and viewers of “Denwa-ya!” (telecom carrier) in the manner of an omukou shout, indicating that NTT has also succeeded in raising its presence among young customers. 3. Commercial performances of kabuki × ICTThrough the seven initiatives described so far, we have accumulated experience in the possibilities of kabuki × ICT. NTT and Shochiku will continue working to advance our efforts to utilize ICT in real kabuki performances, thereby raising our ability to attract customers, improve our business potential, and create new entertainment business opportunities. After the reopening of the Minamiza Theatre in Kyoto in November 2018, NTT and Shochiku will spend the period from 2019 to 2021 co-producing performances such as kabuki that utilize ICT at the Minamiza Theatre and sequentially expand them to other theaters. As a first step, we are planning to co-produce a Cho Kabuki performance with Dwango at the Minamiza Theatre in August 2019. Through the above efforts, as kabuki × ICT rises as a new genre, we will strive to expand our business by integrating kabuki and other forms of entertainment with ICT and to pursue the B2B2X(business-to-business-to-X) model through NTT Group companies. In addition, with an eye towards the imminent event in 2020, we will reflect the findings obtained during our efforts described in this report in the entertainment field as a whole. |
|||
