The Prospect of New Remote-controlled Robot Technology Derived from the Brain’s Control of Movements

Fundamental principle of basic research: Fluctuations in muscle activation timing determine the precision of movements

—Would you explain the background and details of your technology to quantify hand and foot dexterity?

This research was inspired by a casual but thought-provoking observation. One day, while brushing my teeth, I wondered, “Why can’t I brush my teeth with my left hand?” At the time I was working remotely from home due to the COVID-19 pandemic, so I devised an experiment that could be conducted at home to test this question, and that experiment became the starting point of this research.

In the experiment, I used my smartphone’s accelerometer to measure acceleration of the right (dominant) and left (non-dominant) hands while making circular movements, then analyzed the acceleration data. I knew from the beginning that my right hand was more dexterous, but the fundamental reason for this difference between hands was unclear. Although several hypotheses have been proposed to explain such differences in dexterity, many aspects remain unresolved. Given this, I hypothesized that the variability of hand movements may be determined by fluctuations in the timing of motor commands sent by the brain to the muscles.

In elderly individuals, for example, movements become uncoordinated. This inability to move as intended may be due to fluctuations in the timing of motor commands. When an elderly person takes a step, if the timing of the foot movement is delayed while the body is already leaning forward, a loss of balance can occur followed by a fall. A similar phenomenon is seen in patients with cerebellar ataxia, which may likewise be the result from fluctuations in the timing of the brain’s motor commands.

Even for a simple action such as picking up an apple from a table, the brain issues motor commands to multiple muscles in the arm, wrist, and fingers [1] (Fig. 1). These commands depend on many factors, including the magnitude of the required force and range of motion. Variations in the timings of these commands can affect the precision of the movement.

Fig. 1. Quantifying movement variability due to fluctuations in the timing of the brain’s motor commands using a smartphone.

—What type of technological research are you working on specifically?

I first developed a smartphone app that rapidly quantifies movement variability. Through learning, humans acquire the ability to move as intended by processing complex sensory information. Unlike robots, however, humans inherently exhibit variability in their movements no matter how well-trained the movement is. Movement variability has long been a topic of interest in neuroscience. To understand the brain’s information-processing mechanisms that enable coordinated movements of the hands and feet, it was necessary to examine how the control of movements changes with growth and training. I therefore focused on movement variability, a key component of dexterity, and developed an app that conveniently and quickly visualizes movement variability using a simple, daily action: moving a smartphone in a circular motion for about 15 seconds. Using this app, I have collected a large amount of movement variability data. In a joint experiment with the sporting-goods company Mizuno Corporation, data were collected from 1675 participants ranging in age from 4 to 88 years. We have continued the study and now have data from over 2000 people.

The right side of Fig. 1 shows examples of the measured acceleration trajectories of both hands and feet of a right-handed participant. Comparing the left and right trajectories reveals that the movements of the right limbs (hand and foot) show less variability than the left ones. The algorithm I developed expresses the difference between three-dimensional acceleration trajectories of two consecutive cycles of periodic motion as a distance metric. The average of these distances during the entire 15-second measurement is defined as the acceleration variability. My analysis revealed that the acceleration variability in both the dominant and non-dominant limbs decreases during growth, stabilizes in the teens, and increases with ageing [2] (see Fig. 2). It also showed that the acceleration variability is consistently smaller in the dominant limbs than in the non-dominant ones.

Fig. 2. Measurement data concerning acceleration variability of limb movement.

To verify whether the difference in the dexterity between the dominant and non-dominant are affected by training, I then compared the difference in the acceleration variability (left minus right hand) for right-handed participants, left-handed participants, and those who were left-handed but were forced to use their right hand from an early age (referred to as “forced-handers”) (Fig. 3(a)). The data indicate that right-handed participants exhibited greater acceleration variability in their left hand, while left-handed participants exhibited greater acceleration variability in their right hand (Fig. 3(a)). Interestingly, forced-handed participants exhibited less acceleration variability in both hands. The results indicate how forced use of the right hand decreases acceleration variability in the right hand without increasing it in the left hand, suggesting that the left hand’s dexterity is not lost through forced correction. Furthermore, the difference in the acceleration variability of the feet was different between left-handers and forced-handers (Fig. 3(b)). Thus, forced use of the right hand led to stronger right-handedness and, surprisingly, right-footedness.

Fig. 3. Difference in acceleration variability (left minus right) between the hands (a) and between the feet (b) of right-handers, left-handers, and left-handed participants who were forced to use their right hand (forced-handers).

As the next step in this research, I am collaborating with a company that operates rehabilitation facilities to use this app for collecting data from patients with movement disorders. For example, I plan to collect movement variability data of patients whose motor skills have declined due to a stroke and compare their pre- and post-rehabilitation movement variability to assess recovery. Rehabilitation typically involves repetitive movements, and once patients regain a certain level of function, they tend to lose motivation. By visualizing recovery using this app, I hope to provide motivation for patients to continue their rehabilitation.

This app also enables users to measure their dexterity and compare it with the average dexterity for their age group using reference data such as that shown in Fig. 2. It could also be used by top athletes to evaluate the stability and proficiency of their form while refining technique.

—Would you tell us about any difficulties you faced in your research and the challenges ahead?

My research is based on a new hypothesis that differs from conventional thinking in neuroscience. Because of this, I have faced some difficulty gaining understanding from others in the field. However, this hypothesis has led to unprecedented discoveries that expand the possibilities for research in motor control and deepen our understanding of how the brain controls movement. The remotely controlled robotic arm that my team is developing is one such outcome derived from our new understanding of the brain’s control of movements. My research focuses not on the robot hardware but on the control algorithm that drives it. This groundbreaking algorithm enables a remotely operated robot to follow a user’s movement with high accuracy despite being compliant, something that is difficult to achieve with conventional systems. Compliance is important as robots with high compliance are safer to operate and enables humans to physically touch and interact with the remote robot. Non-compliant or stiff robots traditionally used in factories are dangerous and must be separated from humans to prevent harm. A remote robot with high tracking accuracy and compliance has the potential to pave the way for advances in remote medical and nursing care.

With conventional teleoperation methods (tele-impedance), robots can accurately track the user’s movements when communication delay is negligible. However, when controlled remotely over a network where delays inevitably occur, achieving highly accurate tracking of the user’s movement while remaining compliant remains a challenge. Prioritizing accuracy sacrifices compliance, while prioritizing compliance reduces accuracy. To overcome this trade-off, I developed a control algorithm called the “motor intention transmission,” which infers the user’s intended movement from their current position and forces the remote robot to move accordingly. This enables both high tracking accuracy and compliance [3] (Fig. 4). That said, I still believe that the capabilities of this remote robot are not yet sufficient. The next step will be to incorporate force or tactile feedback to this approach.

Fig. 4. Comparison of two algorithms to remotely control a robot arm.

Another major challenge lies in practical application. The leader and follower robot arms must be the same size, which means identical hardware must be used across different sites, an impractical requirement. It is unrealistic to install the same model of robotic equipment in every facility. Overcoming this constraint will require further experiments and advances that enable scaling and miniaturization of the system for commercial use.

My research presents many potential applications and development paths, so another challenge is deciding where to focus within limited time and resources. Fortunately, my previous experience as an assistant professor at a university has helped me in mentoring young researchers. Working together as a group, we can pursue these possibilities collectively.

Pioneering the future by contributing to better medical and nursing care

—Would you tell us about your research prospects and your involvement with NTT business?

Although my specialty is basic research on understanding the brain’s control of movement, I have two major goals for practical application. The first is the smartphone app that quantifies hand and foot dexterity. I am currently conducting a joint experiment at a rehabilitation facility of another company, and we expect to obtain results within fiscal year 2025. For example, correlating existing rehabilitation assessments with the dexterity of the hands and feet from the app may yield valuable insights. Rehabilitation requires continuous effort over long periods, so tracking progress quantitatively will be crucial.

The second goal is to bring our new remote-controlled robot technology into real-world use. By inferring the human user’s movement intention to achieve both high tracking accuracy and compliance, the motor intention transmission algorithm could greatly advance telemedicine, remote rehabilitation, and caregiving. For instance, after a serious accident requiring emergency surgery, a highly specialized doctor could operate remotely on the patient at a nearby clinic instead of waiting for helicopter transport. This technology would be invaluable when every second counts. It could also be used in hazardous environments inaccessible to humans, such as nuclear reactor cores or outer space.

The remote-controlled robot that I am developing depends heavily on NTT’s Innovative Optical and Wireless Network (IOWN) platform, which provides stable, low-latency communication. In teleoperation, when the distance between the leader and follower robots increases, communication delays naturally occur. For stable control, the delay must remain consistent on the order of tens of milliseconds. If the delay fluctuates, it can make the robot’s movements appear unnatural. Stable, low-latency communication through IOWN will allow smooth, predictable control without frequent reprogramming. Once this is achieved, the potential applications of remote-controlled robot technology will expand dramatically.

—Would you tell us how you joined the company and about your place of work, NTT Communication Science Laboratories?

While I was studying at Imperial College London, a senior distinguished researcher from NTT visited my laboratory. At that time, I was researching the mechanism of haptic interaction between people. Since our research shared certain similarities, we had an in-depth discussion, during which I learned about NTT’s basic research. After completing my Ph.D. in 2016, I returned to Japan and began working as a specially appointed assistant professor at the Tokyo Institute of Technology (now Tokyo University of Science). I met many NTT researchers at conferences and other events, became friends with them, and was inspired by the research environment. With encouragement, I joined NTT in 2020. Looking back now, I feel that meeting those researchers was a major turning point in my life.

At NTT Communication Science Laboratories, we conduct a wide range of basic research guided by the philosophy to “create the novel concepts of knowledge and information communication needed for making the human-friendly humanoid computer a reality, such as information processing and media processing related to human knowledge and emotions”. Our mission is to explore the unknown and open new possibilities for the future. My workplace brings together basic researchers from diverse fields in an environment that values free thinking. Although I mainly work remotely, I frequently visit companies and universities to collect data. I am fortunate to work in an environment that offers both academic freedom and collaboration with outstanding researchers from various backgrounds.

—What message would you like to share with researchers, students, and business partners?

Research ideas often emerge from free thinking. In my case, I sometimes find inspiration from hobbies such as watching television or reading books. I try to stay curious and re-examine problems from new perspectives that may lead to new ideas.

Ideas also arise through communication with people from diverse fields. Conversations across disciplines can spark entirely new approaches. For example, my smartphone app was originally conceived for medical and rehabilitation purposes, but through such discussions, I realized it could also be applied to sports training to help top athletes improve their performance.

Finally, I’d like to emphasize to students pursuing research careers that proficiency in English is vital for success. Communicating in English is essential for presenting research, attending international conferences, and reading papers. It’s not just about perfect grammar; what matters most is conveying your ideas clearly. If possible, I encourage you to work or study abroad, as I did, to gain practical experience using English in real contexts. My overseas experience has enabled me to collaborate smoothly with researchers from around the world and achieve results that would have been impossible alone. Moving forward, I hope to continue building partnerships and promoting research that contributes to society.

References

[1]	A. Takagi and H. Gomi, “A Model of Motor Timing Volatility and Its Effect on Force Variability,” International Joint Conference on Neural Networks (IJCNN) 2024, Yokohama, Japan, June/July 2024. https://doi.org/10.1109/IJCNN60899.2024.10650849
[2]	A. Takagi, N. Tabuchi, W. Ishido, C. Kamimukai, and H. Gomi, “A Novel Assessment Reveals Motor Variability as a Sensitive Marker of Neurological Development, Decline and Plasticity,” IEEE Trans. Neural. Syst. Rehabil. Eng., Vol. 33, pp. 2597–2605, 2025. https://doi.org/10.1109/TNSRE.2025.3583687
[3]	A. Takagi, Y. Li, and E. Burdet, “Flexible Assimilation of Human’s Target for Versatile Human-robot Physical Interaction,” IEEE Trans. Haptics, Vol. 14, No. 2, pp. 421–431, 2020. https://doi.org/10.1109/TOH.2020.3039725