To view PDF files

You need Adobe Reader 7.0 or later in order to read PDF files on this site.
If Adobe Reader is not installed on your computer, click the button below and go to the download site.

Feature Articles: R&D on Devices Using Life-assist Technologies

Development of Applications for a Wearable Electrode Embedded in Inner Shirt

Takayuki Ogasawara, Kazuyoshi Ono,
Nobuaki Matsuura, Masumi Yamaguchi,
Junji Watanabe, and Shingo Tsukada

Abstract

NTT has developed a conductive fabric called hitoe that enables continuous measurement of the biological signals of the person wearing it. Heartbeat variations and electrocardiogram signals detected through hitoe are transmitted wirelessly by a compact dedicated device to a smartphone or tablet, where they can be readily checked using an application. Such technology is expected to lead to the creation of new services in fields such as sports training, health enhancement, security and safety, medical care support, and entertainment. In this article, we introduce some examples of approaches to application development.

Keywords: wearable, electrocardiogram, smartphone

PDF PDF

1. Introduction

NTT has collaborated with the textile manufacturer Toray Industries, Inc., to develop a conductive fabric called hitoe that enables continuous measurement of the biological signals of the person wearing it [1]. Hitoe is a woven fabric that was created by coating fiber surfaces with an electrically conductive polymer material (PEDOT-PSS: poly(3,4-ethylenedioxythiophene):poly(styrenesulfonate)) [2]. Integrating this fabric with an inner shirt makes it possible to easily record heartbeat variations and electrocardiogram signals over the long term in a variety of daily life scenarios (Fig. 1). These physiological signals are measured and then transmitted wirelessly by a compact dedicated device to a smartphone or tablet, where they can be readily checked by using the appropriate application.


Fig. 1. Future applications of wearable system using hitoe fabric.

Heartbeat variations are naturally of interest in the field of healthcare, but they can also be used as health indicators in sports or dietary activities [3]. It has previously been reported that hitoe can stably measure vital data during sports activities, for example, in sports with intense movements such as badminton [4]. In addition, variations in heartbeat are thought to be related to the sympathetic nervous system and are known to act as indicators of emotional stress [5]. By focusing on the development of both physical and emotional aspects, we can expect to see new services created in the fields of sports training, health enhancement, sleep management, medical care support, security/safety, and entertainment. To cover such a variety of services, it is essential to implement highly applicable systems in which smartphones and tablets are linked to a cloud service. In this article, we introduce three initiatives from the entertainment field and the security/safety field as applications of hitoe.

2. Applications in the entertainment field

Hitoe was exhibited at the Niconico Super Conference 3, an event attracting 100,000 fans of the Niconico website, on April 26–27, 2014, at Makuhari Messe, Chiba, Japan [6]. We asked various guests to wear hitoe shirts during the conference, and we held five events including talk shows and games. We describe two events here: the First hitoe Cup Heartbeat Shogi Tournament and the Analog Game Workshop at Heartbeat University.

2.1 First hitoe Cup Heartbeat Shogi Tournament

Heartbeat Shogi is a name we coined for this event. It is a shogi (Japanese chess) TV program in which the heart rates of the players wearing hitoe shirts were measured and displayed to visitors on monitors and to an online audience over the Internet. We were looking for a new form of entertainment in which viewers could not only enjoy the battles taking place on the game boards and the body language of the players but could also understand the physiological changes the players were experiencing that we usually have no means of knowing about from their outside appearance. We set up a tournament and invited three professional players to take part in it (Fig. 2). The players wore hitoe inner-shirts during the matches.


Fig. 2. Heartbeat Shogi.

The heart rates of the players during the final game are shown in Fig. 2(b). The heart rate is plotted along the vertical axis and the shogi move number along the horizontal axis. The heart rate of Shingo Ito, 5-dan (on the right in Fig. 2(a)), was within the 70- to 80-bpm level throughout the entire game. The heart rate of Naohiro Ishida, 4-dan (on the left in Fig. 2(a)), on the other hand, tended to rise from the middle game and even reached the 160-bpm level in the endgame. Such information was delivered to the audience in real time by integrating the wearable technology with an Internet image relay.

If we focus on more subtle heartbeat variations, we see that the heart rate of Ishida peaked at 7-2 rook in the middle game (Fig. 2(b)). This occurred at a game position where he took Ito’s bishop. Ishida’s heart rate continued to rise even in the endgame until Ito resigned. We think that this gives a glimpse into the correspondence between heartbeat variations and game position and can become a method of conveying a further sense of excitement to the audience.

Naohiro Ishida won the tournament and was given the title of First hitoe Heartbeat Shogi Meijin by Hiroo Unoura, President & CEO (Chief Executive Officer) of NTT. A better understanding of the relationship between heartbeat variations and emotional state was gained from the post-game discussion. Ishida, 4-dan, thought back and said that “The endgame was so tense, I could feel my own heartbeat. Even though I was playing aggressively and was in the lead, one false move and I would have lost the game.” We can infer from this that the anticipation of making the most of his chances in the game caused him inner tension. Ito, 5-dan, said that “By the endgame, I thought I had already lost, so I was completely relaxed.” This perspective might be one reason why his heartbeat variations were stable through the game. With shogi, decisive moments and relative superiority according to the game position can be seen on the board. However, the players’ own evaluations of the board positions and their level of tension can usually only be determined by their expressions or actions. We expect that the display of heart rates as an element that mirrors changes in the tension and emotional state of the players during games such as shogi will add new interest for viewers.

2.2 Analog game workshop at Heartbeat University

Heartbeat University is an educational program led by instructors with interests in particular areas. The first instructor we invited to be part of this program was Jun Kusaba, the originator of Game Market, Japan’s largest analog game experience and sales event and also the executive of the Gaming History Society. He gave a lecture and led workshops in an analog game. During the workshops, participants who were acting as students played an analog game under the coaching of Mr. Kusaba. The students wore hitoe shirts, and their heart rates were measured (Fig. 3(a)). We used the card game called Divinare in the workshop sessions (Fig. 3(b)). There are cards in four colors (red, blue, yellow, and green), and the game involves correctly predicting how many cards of each color were dealt. In this game, the luck and hunches of the players feature highly, in contrast to shogi, and even a beginner has a chance of winning. It was therefore expected that the heartbeat variations would be different from those of the shogi players. Each student was given three points if the predicted number of cards matched the correct number of cards, one point if the number differed by one, and minus one point otherwise.

They played four rounds and competed for the total number of points. An example of a student’s heartbeat variations is shown in Fig. 3(c). A rise in heart rate can be seen near the end of each round. This was assumed to be related to the way that the difference in his predicted number of cards and the correct number of cards was revealed at the end of each round; the previously unknown correct number of cards is gradually revealed as the game progresses. One student replied in a questionnaire given after the game was finished, that after the second round, “I was in fine form so I thought I might win if I carried on” and after the fourth round, “I thought that I would definitely hold on to my lead and receive a prize.” This shows that it is possible for such anticipation and agitation to be reflected in the heart rate.


Fig. 3. Heartbeat University.

However, variations in heart rate such as those of this student were not the only interesting example. There was another student whose heart rate remained about 30 bpm higher than usual from beginning to end, irrespective of developments during the game. To integrate biosignal measurements effectively into entertainment, it is important that repeatability between large numbers of people is confirmed and also that variations in the biosignals are presented in correspondence with the event in a way that the audience can understand. It is essential to explore the concept of creating events based on an understanding of the mechanisms of the body, then to search for methods of applying biological information from various viewpoints.

3. Application to security/safety

It is now possible to buy and sell an enormous variety of goods over the Internet, and the demand for distribution and delivery of goods is therefore very high. This has increased the importance of understanding the physical condition and stresses of vehicle drivers. Focusing on heartbeat variations and perspiration as stress markers during driving is thought to be effective for this [7]. We had people wear hitoe shirts and then measured their heartbeat variations while they participated in driving tests under real-life conditions. An example of an electrocardiogram waveform taken while a subject was driving is shown in Fig. 4(a). The subject’s R waves, which are features of the electrocardiogram waveform, can clearly be seen. Photos taken when the driver was traveling along a road through a pass in a mountain region and along a suburban road with good visibility are shown in Figs. 4(b) and 4(c). The road through the mountain pass was a narrow one-lane road with a sequence of curves such as that shown in the photo. In contrast, the road with good visibility was a two-lane road that was fairly straight.

We can visualize heartbeat variations from an electrocardiogram waveform by representing them in Poincaré plots, as shown in Figs. 4(d) and 4(e). A Poincaré plot is a graph in which the n-th RR interval (the interval between two consecutive R waves; see Fig. 4(a)) from the start of measurement is plotted on the horizontal axis, and the next (n+1)-th RR interval is plotted on the vertical axis. When the driver is in a stressful situation, the fluctuations in heartbeat variations are small, and the plotted points clump together [8]. Conversely, when the driver is relaxed, the plots scatter. A comparison of the plots of Figs. 4(d) and 4(e) shows that the plot for the mountain pass road is clumped closer together than that for the good-visibility road, suggesting the driver was under more stress on the mountain road. A questionnaire was later given to the driver, who said that on the mountain pass road, “I was driving cautiously since I didn’t know when an oncoming car would appear from a blind corner,” which verified that the driver had a certain sense of tension. This was a fundamental evaluation, but in the future, we plan to estimate driver fatigue, impaired physical condition, or drowsiness, which will contribute to improved security and safety while driving and also effective employee management support.


Fig. 4. Heart rate of driver while driving.

4. Future developments

In this article, we introduced three applications of hitoe. The ways in which these applications are implemented and combined with other technology will differ with the intended use, but it is important that each application has a high level of usability. We should provide a system that is easy to use, even for people who are not experienced with smartphones, and a full range of functions corresponding to individual needs. Apart from the methods described here, we are also promoting links with the GPS (global positioning system) functions of smartphones, various sensors, SNSs (social networking services), and digital games with business partners.

References

[1] NTT press release published on January 30, 2014 (in Japanese).
http://www.ntt.co.jp/news2014/1401/140130a.html
[2] S. Tsukada, N. Kasai, R. Kawano, K. Takagahara, K. Fujii, and K. Sumitomo, “Electrocardiogram Monitoring Simply by Wearing a Shirt––For Medical, Healthcare, Sports, and Entertainment, ” NTT Technical Review, Vol. 12, No. 4, 2014.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr201404fa4.html
[3] “Heart Rate Training,” R. Benson, D. Connolly, Human Kinetics, 2011.
[4] K. Takagahara, K. Ono, N. Oda, and T. Teshigawara, “‘hitoe’—A Wearable Sensor Developed through Cross-industrial Collaboration,” NTT Technical Review, Vol. 12, No. 9, 2014.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr201409ra1.html
[5] G. G. Berntson and J. T. Cacioppo, “Heart Rate Variability: Stress and Psychiatric Conditions,” In: M. Malik, A. J. Camm, editors. Dynamic Electrocardiography. New York: Blackwell Futura, 2004. pp. 57–64.
[6] S. Usui, U. Tamai, and S. Kinoshita, “Report on NTT Super Future Laboratory Exibit at Niconico Super Conference 3,” NTT Technical Journal, Vol. 26, No. 7, pp.59–62 (in Japanese).
http://www.ntt.co.jp/journal/1407/files/jn201407059.pdf
[7] J. A. Healey and R. W. Picard, “Detecting Stress during Real-world Driving Tasks Using Physiological Sensors,” IEEE Transactions on Intelligent Transportation Systems, Vol. 6, No. 2, pp. 156–166, 2005.
[8] Y. Matsumoto, N. Mori, R. Mitajiri, and Z. Jiang, “Study of Mental Stress Evaluation Based on Analysis of Heart Rate Variability - Considerations of Evaluation by Pattern Recognition,” Journal of Life Support Engineering, Vol. 24, No. 2, pp. 62–69, 2012.
Takayuki Ogasawara
Research Engineer, Healthcare Device Development Project, NTT Device Innovation Center.
He received the B.E. and M.E. in applied physics from Hokkaido University in 2006 and 2009, respectively. He also completed the Developmental Brain Science Master’s Program at the Research and Education Center for Brain Science, Hokkaido University, in 2009. He joined NTT Microsystem Integration Laboratories in 2009. His current interests include biosensors and wireless communications. He is a member of IEEE (Institute of Electrical and Electronics Engineers) and the Institute of Electronics, Information, and Communication Engineers (IEICE).
Kazuyoshi Ono
Research Engineer, Healthcare Device Development Project, NTT Device Innovation Center.
He received the B.S. in physics from Tokyo University of Science, Chiba, in 2004 and the M.E. in advanced applied electronics from Tokyo Institute of Technology, Kanagawa, in 2006. He joined NTT in 2006. He is now with NTT Device Innovation Center, where he is researching and developing MEMS (microelectromechanical systems) devices. He is a member of the Japan Society of Applied Physics (JSAP) and IEICE.
Nobuaki Matsuura
Senior Research Engineer, Healthcare Device Development Project, NTT Device Innovation Center.
He received the B.S. in applied physics and M.E. in energy science from Tokyo Institute of Technology in 1990 and 1992, respectively. He joined NTT in 1992. Since then, he has been conducting research on optical interconnection technology. He is a member of IEICE.
Masumi Yamaguchi
Manager, Research Planning Department, NTT Science and Core Technology Laboratory Group.
He received the B.S. in physics from Kanazawa University in 1998, and the M.S. and Ph.D. in physics from Kyoto University in 2000 and 2003. He joined NTT Basic Research Laboratories in 2003 as a research associate. He became a permanent researcher in 2005. His research has included transport and optical properties of low-dimensional electron systems. He is a member of the Physical Society of Japan, JSAP, and IEICE.
Junji Watanabe
Senior Research Scientist, NTT Communication Science Laboratories.
He received the B.E. and M.E. in mathematical engineering and information physics from the University of Tokyo in 2000 and 2002, respectively. He received the Ph.D. in information science and technology from the University of Tokyo in 2005. From 2005 to 2009, he was a PRESTO researcher in the project called Foundation of Technology Supporting the Creation of Digital Media Content at the Japan Science & Technology Agency (JST). From 2009 to 2011, he was a Research Fellow at the Japan Society for the Promotion of Science. He became a research specialist at NTT Communication Science Laboratories in 2011, and a research scientist in 2013. He studies cognitive science and communication devices with applied human perception. His fields of interests are visual and haptic perception and communications.
Shingo Tsukada
Senior Distinguished Researcher, Supervisor, Materials Science Laboratory, NTT Basic Research Laboratories.
He received the M.D. Diploma and License from Toyama University in 1990, and the Ph.D. in medicine from the University of Tsukuba, Ibaraki, in 2003. He joined NTT Basic Research Laboratories as a research specialist in 2010 and has been in his present position since 2013. He is engaged in work on human-machine interfaces, including wearable electrodes and their systems, both in vivo and in vitro, especially in the neuroscience field. He is a member of the Japan Neuroscience Society, the Physiological Society of Japan, the Japanese Circulation Society, and the Society for Neuroscience.

↑ TOP