Feature Articles: Platform Technologies for Services that Utilize Lifelogs
Restaurant Recommendation Service Using Lifelogs
This article introduces a service that presents information about restaurants according to time, place, and occasion by using user preferences and behavior history. To make this service a reality, we have developed a variety of technologies, including technology for automatically obtaining user preferences from a record of terminal operations, GPS-based technology for automatically determining who, if anyone, is accompanying the user, and technology for automatically predicting the user’s destination from his or her behavior history (GPS: global positioning system). We describe the construction of a restaurant recommendation system for smartphones equipped with these lifelog processing technologies and present the results of field experiments targeting general users.
1. Existing restaurant recommendation services
Services that search for and recommend restaurants are quite popular on websites designed for access from desktop computers and mobile devices. A typical function of these services is to present a list of restaurants that match information input by the user such as desired district and type of cuisine. Some services can even display restaurants near the user’s present location provided that the user’s terminal is equipped with a GPS (global positioning system) function.
2. Advanced restaurant recommendations using lifelogs
NTT Cyber Solutions Laboratories aims to create advanced restaurant recommendations that suit the user’s condition by using personal lifelogs. Specific examples of target services are given below.
(1) Determining the user’s food preferences and recommending restaurants starting with those most likely to satisfy the user (Fig. 1)
(2) Determining whether the user is alone or with other people and recommending restaurants that should satisfy everyone (Fig. 1)
(3) Inferring where the user may go next and recommending restaurants near that location ahead of time in addition to restaurants near the present location (Fig. 2)
In developing these services, we targeted users having GPS-equipped smartphones.
3. System configuration
The configuration of the entire system is shown in Fig. 3. To begin with, we selected an Android terminal as the type of handset to be carried around by the user owing to its ease of operation, ease of viewing, and ease of development for a restaurant-recommendation service. In addition to listing restaurant information, this type of terminal can also maintain logs of GPS coordinates, azimuth data, and acceleration values and can keep a history of terminal operations.
A terminal operation history consists of, for example, detailed screens describing restaurants that are acquired whenever the user views them. This data is sent to the user profile server, which processes it as a log of terminal operations to gauge user preferences. At the same time, GPS data is gathered regularly from every few seconds to every few minutes and sent to the Lifelog Management System (LLMS), which uses the terminal operation history log to perform the following types of processing: determine means of transport, extract the present location, determine if the user is alone or with friends, extract movement patterns, predict the destination, and determine whether today is a routine or non-routine day.
The user profile server also has a function for using the lifelogs generated by the above processing to search for restaurants that suit the user’s condition and send a list of those restaurants to the terminal.
4. Lifelog generation functions
The lifelogs generated by this system are summarized below.
(1) User preferences
This system uses a conceptual structure of the target domain to determine user preferences. In the example in Fig. 4, tonkatsu (pork cutlet) is categorized as a type of meat dish, a type of Japanese food, and a type of fried food. Let’s say that the user has at one time selected tonkatsu. It would be impossible to determine which type of Japanese food, meat dishes, or fried food the user likes best if this is the only food-selection history to go on. In time, however, as more food selections are added to the user’s history, it should be possible to say, for example, that the user prefers Japanese food over meat dishes and fried food. Here, to prevent the system from becoming stuck on certain preferences or to counter erroneous learning, a mechanism that forgets learned preferences to some extent has been incorporated.
The learning of preferences in this system is performed by time period. In this way, preferences that correspond to particular times of the day, such as “Japanese-food/ramen for lunch” and “Japanese pub in the evening”, can be learned (ramen: a noodle soup). By learning preferences, we can generate a preference model for the user. And this system can combine the preferences of multiple users.
(2) Alone or with friends
Determining whether the user is alone or in the company of friends on the basis of their GPS data can be difficult if the accuracy of longitude and latitude data is poor owing to, for example, the presence of a high-rise building in the vicinity. To deal with this problem, we define a new index using reliability information accompanying GPS data and, as a result, achieve a high rate of correct results compared with determining the presence of friends using only longitude and latitude data (Fig. 5).
(3) Feature movement patterns
Characteristic behavior patterns for each user can be uncovered by obtaining GPS data over a relatively long period (about one week) (Fig. 6). This system first extracts a history of where the user stays (addresses) based on a log of GPS data and then uses sequential pattern mining and threshold processing to extract feature movement patterns.
(4) Destination prediction
The next place (destination) that a user will be going can be inferred by matching up movement patterns and present location and time (Fig. 6). In these experiments, we achieved a function for recommending restaurants not only near the user’s present location but also near the user’s destination.
(5) Routine/non-routine day
When predicting destination, it is clear that high accuracy cannot be obtained if the user is currently at an irregular location. For this reason, we have incorporated processing for automatically determining whether the day on which the user’s destination is being predicted is a routine day (regular pattern) or non-routine day. This processing makes use of the user’s history of visited places and a support vector machine (SVM) (Fig. 7).
5. Field experiments
To test this system, we conducted field experiments targeting general users in August and September 2009 (1st trial) and February 2010 (2nd trial) together with NTT Communications and NTT Resonant. We passed out Android terminals to 50 subjects in the 1st trial and about a dozen subjects in the 2nd trial. In both trials, the subjects belonged to nine categories (students, sales workers, food connoisseurs, etc.) and they used this recommendation service for about a month.
Comments received by a questionnaire-based survey and interviews conducted after the experiments revealed that more than 90% of users wanted to continue using the service and that more than 60% felt that the service is superior and more useful than existing restaurant recommendation services. On the other hand, it was also said that the results of user preference determination and destination prediction could be more accurate. In particular, we found that a user’s preferences could be swayed by mood, current situation, and other factors and that there was a need to determine the user’s state in more detail when recommending restaurants.
6. Future plans
Using the results of the abovementioned two field experiments, we plan to continue our research and development efforts with the aim of achieving an engine that can obtain a deeper understanding of the user by improving the accuracy of lifelog generation techniques and increasing the variety of lifelogs.