Buru-Navi3 Gives You a Feeling of Being Pulled

1. Introduction

What if a compass (or compass application in a mobile phone) actually pulled your hand in a certain direction instead of just showing the direction? The concept of force stimulation can be likened to the example of a parent leading a child by the hand, and it has the potential to be more intuitive and expressive than cutaneous stimulation, which refers to stimulation of nerves via skin contact. Unfortunately, the conventional force display systems are not suitable for mobile devices since the generation of low-frequency force requires a fixed mechanical ground, which mobile devices lack. Some mobile torque feedback devices that do not require grounding have also been proposed, but they produce neither a constant nor translational force; that is, they can generate only short-term rotational force.

We have been researching a way to create a force sensation in mobile devices and have developed a force display device called Buru-Navi, which generates a sensation of being pulled or pushed by exploiting the characteristics of human perception [1, 2]. Our objective is not to create a physical tugging force, but to create an illusory sensation of being pulled or pushed. We can do this by placing some kind of mass into a box and moving the mass back and forth with different acceleration patterns for the two directions. This asymmetric motion generates a brief and strong force in a desired direction (e.g., forward) and a weaker one over a longer period of time in the reverse direction (e.g., backward). Note, however, that the average magnitudes of the two forces are identical. Since the magnitude of the longer and weaker force is much smaller than the other force, people who hold the box feel as if they are being pulled in the desired direction rather than feeling only a discrete vibrating sensation that is common in conventional mobile devices today.

2. Miniaturization of force device

Over the past several years, we have been refining a method to create a sensory illusion of being pulled and have developed various prototypes to create such a sensation [3, 4]. Our previous prototypes were able to create a clear force illusion, but they were based on mechanical linkages, so they were generally too large and heavy to embed in mobile equipment. Thus, we have selected a linearly vibrating actuator^*1, which does not require any mechanism to convert from rotational motion to translational motion, and used it in a thumb-sized prototype (Figs. 1 and 2). However, generally speaking, the power-to-weight of the actuator drastically decreases as the size of the actuator decreases, leading to an insufficient force illusion.

Fig. 1. Comparison of Buru-Navi3 devices.

Fig. 2. Timeline of Buru-Navi prototypes.

Accordingly, we have redesigned and optimized an asymmetric oscillation pattern by working to achieve the best combination of the amplitude of the small actuator and the sensitivity of the human fingerpad. Specifically, we have used a higher frequency range than that used before; the motion patterns of the actuator for forward and backward directions are shown in Fig. 3.

Fig. 3. Motion patterns of asymmetric oscillation.

There are four tactile mechanoreceptors in the human fingerpad; of these, the Pacinian corpuscle is most sensitive to high temporal frequency vibrations (100–300 Hz) in the normal direction [5], but seems to have nearly the same sensitivity in sliding directions tangential to the skin [6, 7]. In contrast, SA1 and RA1 fibers, whose signals come mainly from Merkel and Meissner corpuscles, respectively, are sensitive to lower frequency (< 100 Hz) vibrations, and some of them can clearly code—or in other words, detect—the sliding or tangential force direction [6]. Thus, asymmetrically oscillating stimuli that contain the frequency components that stimulate these receptors will induce a clearer force sensation of being pulled. As a result, we have succeeded in creating a sensation of being pulled with a thumb-sized force display, and the sensation is just as clear and strong as it was with our previous prototypes.

3. Comparison of clarity of force illusion

It is of no value to us if the effect of force illusion decreases when the size and weight of the device are decreased. To determine an effective profile of asymmetric acceleration to induce a clear force sensation, we conducted an experiment to compare the clarity of perceived force sensation between actuators and asymmetric driving patterns of an input signal in order to find the best combination of these for a miniaturized force display.

We selected two kinds of linear actuators; each actuator was covered with a cylinder that was 40 mm in diameter and 17 mm thick and made of ABS (acrylonitrile butadiene styrene) resin. A piece of sandpaper (#1000 grit) was pasted on its surface to control the surface roughness. Two driving patterns of 40 Hz (pulse alternation 7:18 ms) and 125 Hz (pulse alternation 2:6 ms) were used as optimal inputs to each actuator, which gave us four stimulus conditions (two actuators × two driving patterns). In the experiment, each participant was asked which stimulus condition gave a stronger sensation of being pulled/pushed between two of the four conditions. Each participant made six paired comparisons^*2.

All participants reported that they felt a strong force sensation of being pulled/pushed with a specific condition. There were no significant differences between the tendencies of the participants’ evaluations (χ²(40) = 25.5, p = 0.96, not significant). Although a paired comparison provides ordinal data, the data can be converted into an interval scale by using the order statistics (Thurstone’s method). The values of the clarity of the force sensation in the interval scale are shown in Fig. 4. In this scale, the four conditions were ordered along a continuum to represent the degree of difference of the clarity of perceived force. The figure shows that an acceleration pattern created by the combination of a specific actuator (actuator B) and a specific driving pattern was highly likely to be judged to create a clear force sensation. This suggests that a clear force sensation can be created if we select the appropriate asymmetric acceleration pattern of oscillation.

Fig. 4. Results of paired comparison.

4. Applications

Applications using Buru-Navi3 include a pedestrian navigation system and a new haptic experience in gaming. At NTT Communication Science Laboratories Open House 2014, we presented an angling game application and a virtual dog walking application. In the angling game, users can feel a virtual fish nibbling and pulling the hook. A fishing line with a small weight at its tip was attached to Buru-Navi3 (Fig. 5(a)). When the weight approaches the mouth of the virtual fish, the fish bites it. When the fish is caught, Buru-Navi3 generates the sensation of the fish pulling the hook. Many participants seemed to enjoy the novel sensation produced by the device, and some reported to be “addicted” to it. In the dog walking application (Fig. 5(b)), the participants felt as if a virtual dog that walks around was pulling on the leash. We used the 2-DoF^*3 version of Buru-Navi3 in combination with a motion tracking system. The amplitude and direction of the force sensation were altered dynamically according to the positions of the user’s hand and the virtual dog.

Fig. 5. Applications of Buru-Navi3.

In addition, we recently presented a pedestrian navigation system at the ACM SIGGRAPH^*4 2014 Emerging Technologies conference [8]. We have implemented a tracking system based on fiducial markers since marker-based tracking provides repeatable, robust, and reliable performance. We hung some polystyrene boards containing original printed fiducial markers on the ceiling. Users held a web camera so it was facing the ceiling and thus able to capture the markers. The user’s position and orientation were estimated from each marker’s unique ID and its distorted form in the captured image. A photo of the pedestrian navigation system using Buru-Navi3 is shown in Fig. 6.

Fig. 6. Pedestrian navigation system using Buru-Navi3.

It is worth mentioning that in the former study, we conducted an experiment to examine whether Buru-Navi2, the previous prototype, allowed people with visual impairments to walk safely along a predefined route at their usual walking pace without any previous training. We found that by delivering simple navigational information, Buru-Navi2 enabled more than 90% of participants with visual impairment to walk along a predefined route through a maze without any prior training [4]. This finding implies that the 2-DoF version of Buru-Navi3 will also be capable of leading people along a route—even people who might ordinarily have difficulty finding their way—just as well as Buru-Navi2 did since the fundamentals underlying these systems are essentially identical.

5. Conclusion and future work

Buru-Navi3 is the result of our efforts to investigate the mechanism underlying human perception and motion. In the development process, we have overcome issues such as a trade-off between miniaturization of the device and the effect of illusory sensations. Miniaturized devices are advantageous in that they can be used with mobile and wearable appliances. Such technology will open the door to providing rich haptic experiences in mobile applications, which is a relatively new field.

The characteristics of human perception have been considered deeply in developing conventional video and audio systems. In the future, various kinds of sensory information—not only for perception but also for action—will be considered in order to develop sophisticated interactive systems. Thus, we will continue to focus not only on understanding the mechanisms of sensorimotor processing in the brain, but also on finding prerequisites for developing interactive and natural user-friendly interfaces.

References

	Tomohiro Amemiya Research Scientist, Human Information Science Laboratory, NTT Communication Science Laboratories. He received the B.S. and M.S. in mechano-informatics from the University of Tokyo in 2002 and 2004, respectively, and the Ph.D. in information science and technology from Osaka University in 2008. He joined NTT in 2004 and has been conducting research on human-computer interfaces that apply sensory illusions. He is also an honorary research associate at the Institute of Cognitive Neuroscience, University College London, UK, for the year 2014–2015. He has received several academic awards including the Outstanding Paper Award (Virtual Reality Society Japan, 2005 and 2014), Best Presentation Award (Human Interface Society, 2005 and 2007), Best Paper Award in Human Communications Group (IEICE, 2009), “Grand Prix” du Jury at the Laval Virtual International Awards (2007), Best Paper Award (IEEE VSMM 2012), and Best Demonstration Award (Eurohaptics 2014). He is a member of the Institute of Electrical and Electronics Engineers (IEEE).
	Shinya Takamuku Researcher, Human Information Science Laboratory, NTT Communication Science Laboratories. He received the B.E., M.E., and Ph.D. in engineering from Osaka University in 2003, 2005, and 2008, respectively. He joined NTT in 2009 and has been engaged in research on the computational basis of human motor awareness. He is a member of the Society for Neuroscience, the Japan Neuroscience Society, and The Robotics Society of Japan.
	Sho Ito Researcher, Human Information Science Laboratory, NTT Communication Science Laboratories. He received the B.E. and M.E. in mechanical engineering from Kyoto University in 2008 and 2010, respectively. He joined NTT in 2010 and has been researching human sensorimotor mechanisms. He is a member of the Society for Neuroscience, the Japan Neuroscience Society, and The Robotics Society of Japan.
	Hiroaki Gomi Distinguished Senior Research Scientist, Group Leader of Sensory and Motor Research Group, NTT Communication Science Laboratories. He received the B.E., M.E., and Ph.D. in mechanical engineering from Waseda University, Tokyo, in 1986, 1988, and 1994, respectively. He conducted research on computational biological motor control at ATR (Advanced Telecommunication Research Labs., Kyoto) from 1989 to 1994, in two JST-CREST projects from 1996 to 2003, and in a JST-ERATO project from 2005 to 2010. He was an adjunct professor at Tokyo Institute of Technology from 2000 to 2004. His current research interests include human sensorimotor control mechanisms and interactions between perception and action.