Feature Articles: Platform Technologies for Services that Utilize Lifelogs
Prospects for Using Lifelogs in the Medical Field
In this article, we introduce studies being conducted in cooperation with Kyoto University Hospital in relation to new medical and healthcare services that make use of various lifelogs in every-day life as well as medical information obtained within medical facilities.
In the medical field, it is necessary to gather as much personal information as possible about patients in order to achieve high-quality diagnosis and treatment. Until now, the personal information used in diagnosis and treatment has basically been gathered and used only within medical facilities. It consists of clinical records, test results, medical images, and other such information. However, the medical information that can be gathered within a medical facility is very limited. It would seem that there is a large amount of data that would be useful for medical purposes within the voluminous and varied data gathered in daily life, most of which is spent outside medical facilities, but such lifelog data has gone unused in most cases.
2. Use of lifelogs for medical diagnosis and treatment
Since June 2009, we have been conducting various studies in cooperation with the Kyoto University Hospital to assess the possibilities for using such lifelog data, obtained outside medical facilities, for medical purposes. In this article, we introduce two such studies. The first aimed to evaluate how vital data obtained outside medical facilities (weight, blood pressure, number of steps walked, etc.) can be useful in medicine. The second concerned the development of technologies that can automatically gather a variety of lifelog data without placing undue burden on the user.
2.1. Experiments linking medical information and lifelogs
In these experiments, the Maiko-Net Web service, which is operated by Kyoto Association for Cooperative Medicine (a non-profit organization) with participation from the Kyoto University Hospital, was used as a platform for accumulating medical information within a medical facility. Maiko-Net is a regional medical infrastructure service linked to medical facilities in Kyoto prefecture, with functions for sharing medical information among medical facilities and allowing the public (patients receiving treatment at a medical facility) to view some of this information. For example, a Maiko-Net user receiving treatment at Kyoto University Hospital can view the results of all past blood tests (cholesterol levels, etc.) at any time. For these experiments, Maiko-Net was linked to “goo Karada Log” (karada is the Japanese word for body), which is a lifelog service on the goo Web portal oriented to the health field provided by NTT Resonant Inc. This created a mechanism allowing healthcare practitioners and the general public to access medical information gathered in medical facilities and lifelog information gathered outside medical facilities in an easy and secure manner (Fig. 1).
In mid-December 2009, an announcement was issued to Maiko-Net users receiving treatment at Kyoto University Hospital in order to recruit participants for experiments. Over several months beginning at the end of December, the recruited participants recorded lifelogs on goo Karada Log using the new mechanism. They were recommended to record a set of lifelog data consisting of about 30 items such as blood pressure. These items were selected as being correlated to lifestyle diseases, such as diabetes and high blood pressure, which occur frequently in the Maiko-Net membership, on the basis of advice from doctors at Kyoto University Hospital. Participant login information for Maiko-Net was also linked to goo Karada Log for the convenience of participants, so that logging into the Maiko-Net personal page also provided seamless login to the goo Karada Log personal page.
At the end of February 2009, we conducted a survey of participants and we had a meeting with doctors at Kyoto University Hospital to evaluate the potential for using lifelogs for medical treatment. The following is a summary of the responses to the participant survey.
(1) Approximately half of the participants were already recording health information daily before these experiments began (although for most of them this consisted of handwritten records). Moreover, most of the health information they were recording was covered by the lifelog items recommended for these experiments.
(2) In the experiments, almost 20% of the participants recorded health information every day using the constructed system.
(3) Almost all of the participants wanted healthcare practitioners to view the lifelog data.
The main comments received in the meeting with medical doctors were as follows.
(1) They were not particularly concerned that the lifelog information gathered outside medical facilities might be too inaccurate.
(2) Measurements such as body weight and blood pressure fluctuate depending on when they are taken, so lifelog data gathered over several days rather than the single measurements taken during out-patient treatment allows trends to be understood, and this is important.
(3) In addition to numerical lifelog data, free comments such as “felt dizzy” are important.
(4) It would be better if average, abnormal, and target values for each lifelog item could be displayed to patients.
Using the above results as a starting point, we will continue to study ways of making lifelog recording easier for patients and also study methods of linking lifelog services to medical data systems in medical facilities, so that patient lifelogs can be viewed and used easily by medical practitioners.
2.2. Gait visualization technology
In rehabilitation and the treatment of lifestyle diseases, it is important to record conditions and activity related to walking. Medical facilities have large on-site analytical equipment that can record gait in detail, and this has been useful for treatment. However, information recorded in a medical facility in this way is extremely limited in time and space, so a way of recording walking behavior under normal conditions is desirable for rehabilitation and other types of treatment. With this in mind, we have developed technology for visualizing lifelog data related to walking in ordinary life. It records when, where, and in what manner a person walks.
In developing this technology, one of our goals is to minimize the load on the user by only requiring the user to carry the data-collection terminal. In gathering continuous lifelog data, it is very important not to require much effort from the user. Considering the proliferation of mobile phones with sophisticated functions such as GPS (global positioning system) and accelerometers and the fact that it is fairly common for people to carry a mobile phone, we decided to use a mobile phone with these functions as the data collection terminal.
This technology has two main features. The first applies signal processing to the acceleration signals to distinguish periods of walking and identify details of the gait. In daily life, we perform various actions, such as resting or riding a train or bus, in addition to walking. To record a representation of this in daily life, the system constantly records acceleration signals, and walking periods are identified by finding segments within these signals that have characteristics that indicate walking. The system can also analyze and display the walking segments in further detail, for aspects like the pace of walking and shifts in balance. This provides a clearer view of what type of walking is occurring.
The second feature creates an easy-to-understand graphical visualization combining time and location information. Walking is greatly influenced by the surrounding environment, so location information can be very meaningful. This feature clearly shows when and where the subject was walking. By including location information, the system records the range of user activity as well as the type of walking.
A schematic of this system is shown in Fig. 2. The user carries a mobile phone with the data-collection application installed at all times. The software collects GPS and accelerometer data periodically and automatically sends it to the server, which accumulates it as a lifelog. The lifelog can be viewed on the server through the Internet by the patient himself, medical staff, or other authorized persons.
Visualizations of the movement log collected while the same person walked over the same course under two different conditions are shown in Fig. 3:
(1) Normal walking (Fig. 3(a)).
(2) Walking with constraints attached to the right knee and ankle that limited their movement range (Fig. 3(b)).
The walking course began in the lower right of the map and was approximately 3 km in length. The total change in elevation was about 100 m, with particularly steep inclines toward the end of the course. The marks along the course indicate the person’s walking pace, with the fastest pace shown in red and the color changing to green and blue as the pace decreased. In the normal walking case, the pace was initially fast after the start and slowed as walking continued. In contrast, in the constrained walking case, the pace was not fast after the start and became slower much earlier, as indicated by the change to blue markers.
An example of a detailed analysis of walking at the same location in each case (the middle points in Fig. 3) is shown in Fig. 4. In this example, the smoothness of walking was analyzed by taking the fast Fourier transform of the acceleration signals to compute their frequency components. The graphs show that in the normal walking case there were almost no frequency components other than the walking pace, while the constrained case contained various other frequency components. This result shows that wearing the constraints on the right leg was a hindrance to smooth walking.
Data can be gathered automatically in this way without placing a burden on the user, and a visualization of normal walking behavior can be created. Compared with conventional measurements of walking made in a medical facility, these measurements cover a much greater extent in both time and space, so this normal activity data can be used to understand patient rehabilitation and recovery conditions. It should also be useful for preventive care and capturing symptoms such as weakening legs or declining activity levels in elderly subjects.
3. Future developments
We intend to develop these lifelog-gathering experiments further by increasing the number of subjects in vital-data experiments and building mechanisms that make it easier to maintain patient motivation for collecting lifelog data. We will also continue to study ways of linking medical information systems at medical facilities to these lifelog services. We plan to conduct experiments on the gait visualization technology in the fields of preventive care and rehabilitation in cooperation with Kyoto University Hospital to study its effectiveness in these areas.