Looking More, Acting Better

1. Eye-hand coordination for human motor behavior

During daily activities, such as driving a car and playing sports, it is essential to properly control the movements of the eyes as well as those of the limbs. Have you ever had a tennis lesson in which you were told to “keep your eye on the ball”? Have you ever heard top athletes talk about their eyes? As these questions show, the brain has a mechanism for coordinating the movement of the eyes and hands. In this article, I review conventional theories of eye-hand coordination from the literature then propose our interpretation that the spatial relationship between eye and hand movements is inherently linked with the learning and execution of new skilled reaching movements.

2. Conventional view of eye-hand coordination

As an example of movement, let us consider reaching for a cup. First, we look at the cup to confirm its position, and at the same time, visually acquire necessary information such as the size of the cup, amount of water, and how easy it is to grasp. The visual resolution of primates, including humans, is the highest at the center of the eye (fovea) and decreases towards the periphery. To obtain more accurate visual information for the upcoming arm movement, the brain moves the eyes first, leading to the appropriate output of the arm movement.

The mechanism of eye-hand coordination has been extensively studied, including behavioral and neurophysiological experiments using human and monkeys. For example, it has been established that the eyes move before the hand, and that their temporal relationship is maintained within a certain range. Furthermore, the location to which the gaze is directed is directly related to the endpoint of reaching movement. This spatiotemporal coordination of eye and hand movements suggests that the eye and hand control systems in the brain exchange information to produce spatiotemporally coordinated motor output. The full extent of the brain mechanisms underlying eye-hand coordination is still under investigation, but it has been reported that various neural networks, including the cerebral cortex, cerebellum, and brainstem, are involved [1]. It is worth noting that theories in the previous studies have generally emphasized the importance of reaching behavior when looking at a target (foveal reach) compared with that when looking elsewhere (peripheral reach).

3. Questioning the conventional theory

While previous studies have mainly involved tasks in which participants performed reaching movements on a stationary target, target objects can move in complex and unpredictable ways in actual environments. For example, in tennis and baseball, we need to make decisions and produce complex movements (motor skills) within a limited time of a few hundred milliseconds. Does the conventional view based on the superiority of foveal reach still apply in such complex situations? Furthermore, how important is the eye-hand coordination in the process of acquiring and executing novel motor skills? These questions are the starting point of our research.

At NTT Communication Science Laboratories, we have conducted a series of studies on experimental tasks such as reaching for a target with unpredictable movements [2] and measuring the batting of professional baseball players [3]. We observed coordination patterns of eye-hand movements even for tasks requiring complex motor skills, but the results of the detailed analysis of spatiotemporal coordination patterns were not necessarily consistent with the conventional theory on the basis of the dominance of foveal reach. For example, in the results of the batting experiments with professional baseball players, rapid eye movements (saccades) toward the ball were frequently observed just 100 ms before the bat hits the ball. It takes more than 100 ms to swing a bat, so when the eyes move, the hand has already started to move. Given the processing time in the brain from vision to motor output, even if the ball is detected at the center of the eye at the moment of hitting, that visual information cannot be used for this batting. Nevertheless, why do top players frequently move their eyes just before hitting?

4. Eye-hand coordination directly related to motor learning of reaching movements

Considering the results of previous studies and common concepts, we hypothesized that the spatial relationship between the eyes and hand, regardless of foveal vision or peripheral vision, is an important factor in the acquisition and execution of motor skills for reaching [4]. To test this hypothesis, we conducted a series of experiments (Experiments 1 and 2) to evaluate the relationship between motor learning and eye-hand coordination.

First, we describe an experimental method for quantifying motor learning. Participants move a stylus pen on a digitizing tablet to put a visual cursor into the target, which is displayed on a computer monitor (Fig. 1(a)). During this reaching task, a visuomotor rotation of about 30° is introduced between the actual hand motion and visible cursor motion (Fig. 1(b)). Even if participants correctly move their hand toward the target, the visual cursor deviates from the target, resulting in a reaching error. With repetition involving hundreds of trials, hand movement gradually changes, decreasing the reaching error. This change in motor outputs can be evaluated as motor learning.

Fig. 1. Apparatus and paradigm for motor-learning experiment.

We used a learning paradigm to investigate how foveal and peripheral reach are related to motor learning in Experiment 1 (Fig. 2(a)). Under Condition 1, participants learned the visuomotor rotation with foveal reach (Fig. 2(b), Condition 1). We then compared the degree of motor-memory retrieval between foveal reach, the same coordination as during learning, and peripheral reach, different coordination from learning. The results indicate that the retrieval of motor memory was lower for peripheral reach than for foveal reach (Fig. 2(c), Condition 1). Under Condition 2, participants learned with peripheral reach (Fig. 2(b), Condition 2). We found that, unlike the results for Condition 1, the retrieval of motor memory was lower for foveal reach than for peripheral reach (Fig. 2(c), Condition 2). These results cannot be explained by the conventional theory that foveal reach is necessarily superior to peripheral reach under any condition. Instead, our results indicate that the spatial relationship between the eyes and hand used during learning needs to be maintained after learning to perform the learned reaching movements efficiently.

These results can be explained if we assume that foveal and peripheral reach are processed in different areas (or representations) of the brain (Fig. 2(d)). In this interpretation, the results of motor learning, or motor memory, would be associated with the representation of the eye-hand coordination used during learning. Therefore, if we use a different spatial eye-hand relationship between during and after learning, we are not able to have full access to motor memory. If this interpretation is correct, it may be possible to acquire different motor skills simultaneously by making good use of distinct representations related to eye-hand coordination. To test this possibility, we conducted Experiment 2.

Simultaneous acquisition of different motor skills is equivalent, for example, to practicing the forehand and backhand shots in tennis at the same time. For beginners and intermediates players, it would be easy to imagine how inefficient and difficult it would be to learn to randomly switch between forehand and backhand shots on every attempt. The difficulty of simultaneous motor learning has been confirmed through experiments, and this is thought to be due to the fact that different motor memories are overwritten by each other across trials.

In Experiment 2, we introduced visuomotor rotations with clockwise and counterclockwise directions as different motor skills and randomly presented two rotational directions across trials. Eye-hand coordination, foveal reach or peripheral reach, was also randomly selected across trials. The experimental task was designed so that clockwise rotation was presented in foveal-reach trials, and counterclockwise rotation was presented in peripheral-reach trials. This means that the direction of visuomotor rotation was uniquely specified with the type of eye-hand coordination. This task design is based on the interpretation that different motor memories can be stored in different brain representations associated with foveal and peripheral reach, allowing simultaneous learning with less interference. As shown in Fig. 2(e), the experimental results support our hypothesis, showing that reaching errors gradually decreased in both clockwise and counterclockwise rotations (i.e., both foveal- and peripheral-reach trials). This indicates that different motor skills can be acquired simultaneously. In addition to the decrease in reaching errors, we found a clear aftereffect^*, suggesting that in the post-learning phase, the brain accessed the appropriate motor memory in accordance with the eye-hand coordination (foveal reach or peripheral reach) that participants performed in that trial.

Fig. 2. Experimental tasks and results.

5. Future prospects

Our research results revealed that the spatial relationship between gaze and reaching target, specifically foveal and peripheral reach, is inherently related to the process of motor learning. This suggests that, for the acquisition and execution of novel skills for reaching movements, it is more important to maintain a constant eye-hand coordination than to always look at the reach target.

These findings are expected to be applied to new sports-training methods and rehabilitation programs that focus on the importance of the eyes. For example, in the early stages of training, it may be useful to increase efficiency and speed up the learning process (blue area in Fig. 3). In this case, fixing the gaze on a single location would be an effective strategy as it can accelerate learning by using a single brain representation associated with a specific eye-hand coordination. In the latter half of the learning process, however, robustness of learning, such as resistance to forgetting, is required rather than speed (red area in Fig. 3). In this case, moving the gaze in various directions during learning would be an effective strategy to enable robust learning by using the multiple brain representations associated with different eye-hand coordination.

Fig. 3. An example of a novel training method that involves different gaze states to enhance sports training or rehabilitation.

Gaze information is closely related not only to the movement of the hand but also to that of many other body parts such as the legs, head, and trunk. By deeply elucidating these brain mechanisms of motor coordination, we aim to essentially understand human motor control and propose ideas of potential applications for information and communication technology, such as designing interfaces that elicit natural human behavior and effective communication between humans and robots.

 

This study was partially supported by Grants-in-Aid for Scientific Research (JP16H06566) from the Japan Society for the Promotion of Science to H.G.

References

[1]	J.-R. Rizzo, M. Hosseini, E. A. Wong, W. E. Mackey, J. K. Fung, E. Ahdoot, J. C. Rucker, P. Raghavan, M. S. Landy, and T. E. Hudson, “The Intersection between Ocular and Manual Motor Control: Eye-hand Coordination in Acquired Brain Injury,” Front. Neurol., Vol. 8, p. 227, 2017.
[2]	N. Abekawa, T. Inui, and H. Gomi, “Eye-hand Coordination in On-line Visuomotor Adjustments,” Neuroreport, Vol. 25, No. 7, pp. 441–445, 2014.
[3]	Y. Kishita, H. Ueda, and M. Kashino, “Eye and Head Movements of Elite Baseball Players in Real Batting,” Front. Sports Act. Living, Vol. 2, p. 3, 2020.
[4]	N. Abekawa, S. Ito, and H. Gomi, “Different Learning and Generalization for Reaching Movements in Foveal and Peripheral Vision,” Proc. of Adv. Mot. Learn. Mot. Control, 2019.