Feature Articles: Toward a New Form of Coexistence between the Real World and Cyberworld
Cybernetics Technologies for Symbiosis between People and Machines
With the development of computing technology and artificial intelligence, an environment in which we can use advanced information and communication services in our daily lives is rapidly being established. By installing mechanisms that recognize and understand in a more human-oriented manner than conventional technologies, we aim to create a society in which people and machines can coexist in harmony. In this article, we introduce our research themes for achieving this aim.
Keywords: symbiosis between people and machines, motion support, embodied knowledge
1. Toward symbiosis between people and machines
As the computational efficiency of processers has increased significantly, the accuracy of image processing and recognition has improved; technology that handles cross-modal information, such as various sensors, has been refined; and automated driving of cars, which until now has only been depicted in science fiction, is gradually being achieved. The improvement in voice recognition, character recognition, and context understanding makes it possible for automatic-response engines, such as chatbots, to support counter services (contact persons), and people now carry around mobile devices (such as smartphones) with high-performance processors. These factors have led to rapid establishment of an environment in which we can use advanced information and communication technology (ICT) services in our daily lives. One of the technologies that contributes significantly to improving the accuracy of recognition and understanding is artificial intelligence (AI), which is constructed by machine learning using large amounts of real data.
Current mainstream AI technology is akin to modeling logical and objective recognition and understanding that works like the human left brain based on our experience and memory, and it has enriched our lives mainly by targeting things that can be expressed in words and those that can be symbolized such as images and sensors data.
We aim to create a society that enables people and machines to coexist in harmony (Fig. 1) by updating conventional AI technology with the following two steps:
(1) Elucidating the processing and action processes that work like the human right brain for things that cannot be expressed in symbols (such as sensations and actions) and for those that are subjective and cannot be uniquely expressed (such as judgments that differ from person to person)
(2) Installing the elucidated mechanisms that provide more human-oriented recognition and understanding into conventional AI technology
To achieve this aim, we introduce the following three technologies from our research themes:
2. Motion support
Cybernetics, proposed by Norbert Wiener in 1948, is a unified approach to communication and control concerning living organisms and machines. Paying renewed attention to this concept, we are developing technologies to support the motion of people.
Human motion is executed when motor instructions from the brain are transmitted to the muscles, which then contract on receiving the instructions. The brain then perceives and recognizes the result as a stimulus through sensory organs and repeats planning a new motion and sending its appropriate instruction. We are attempting to support this communication and control in the human body by sensing and feeding back biological signals such as electroencephalography (EEG) and electromyography (EMG).
More specifically, we aim to create a society in which people can move (live) better by reproducing (transcribing) their current motor skills temporarily and permanently on the basis of the motions of others (skilled people) and their former selves. For example, we are considering (i) training that uses electrical muscle stimulation (EMS) to reproduce the muscle motions of skilled people executing various skills and sports and (ii) rehabilitation based on records of one’s motions in daily activities.
We are working on enabling the following four functions, which we consider necessary for reproducing (transcribing) human motor skills by using biological signals (see Fig. 2).
We had been working on bio-sensing technology (using, for example, EMG) and feedback technology using tactile and EMS to the human body. Mechanisms of human motion, however, are complex, and various efforts are underway in the fields of physiology, brain science, and biomedical engineering to support and enhance motion, so an interdisciplinary approach is necessary. Looking to the future and using our accumulated technological strengths, we are planning to acquire new knowledge about the mechanisms of motion through creation of technology and interdisciplinary collaboration with internal and external experts. For a society in which people and machines coexist at a higher level, we will elucidate human processing and action processes related to motion and expand human capabilities.
3. Embodied knowledge
In human life and social activities, knowledge is the basis for improving and refining the quality and efficiency of people’s behavior and thinking. It is therefore important for people living in the present to acquire knowledge that has been accumulated over a long period and apply it in various situations in their daily lives.
The first thing that comes to mind when people think of acquiring knowledge is studying at school or learning a subject. Such already systematized and described knowledge is called explicit knowledge, and everyone has experienced the widely practiced methods of acquiring and demonstrating such knowledge. However, knowledge does not mean only the explicit type. Another type is the knowledge that people learn and express that is difficult to systematize or describe, and it can only be acquired through experience rather than through mental understanding. This type is what we call embodied knowledge.
Regarding learning to play a musical instrument, people generally refer to instructional books that systematically describe how to handle the instrument, its functions, and how to move the fingers. Therefore, if you read all of this instructional material and understand it, will you be able to play an instrument? Of course, the answer is “no.” In other words, only by moving your body (fingers, in this case) and practicing with that understanding in mind will you be able to play. This reality indicates the existence of embodied knowledge, which is the knowledge rooted in a physical activity, as opposed to explicit knowledge expressed in textbooks (Fig. 3).
Embodied knowledge is largely based on the senses and subjectivity of people, and it is far from being supported by ICT for acquiring and demonstrating in the manner of explicit knowledge, which is objective and easily verbalized. However, with the recent evolution of technology, it has become possible to sense and process information at high speed in a manner closer to the sensations received by the people involved. Accordingly, we are conducting research on embodied knowledge from the perspective of neo-cybernetics, which captures the mechanism of actions from the starting point of the person’s cognition. Taking windsurfing as a subject, we are clarifying the mechanism of acquiring and demonstrating embodied knowledge in sports and making it possible to support this mechanism. With this in mind, we are studying an analysis method to find the latent pattern between the physical activity of the athlete, behavior of the natural environment and tools, and performance of the athlete. Moreover, using the data acquired from actual competitors, we are also studying how to communicate the discovered pattern to other people and how to improve the value (acquisition and demonstration of embodied knowledge or changes in interest in the sport and the natural environment) provided to the recipients of the communication (Fig. 4).
Although sports is typical in the study of embodied knowledge, the aforementioned skills, which are knowledge acquired and demonstrated in strong connection with physical actions, such as playing a musical instrument, drawing a picture, or making a good presentation, exist in a variety of everyday situations. Accordingly, we are aiming to create a society in which anyone can acquire and apply embodied knowledge, which has been confined within certain people, through ICT based on the perspective of the person who wants to acquire that knowledge while simultaneously studying areas other than sports.
4. Zero latency
Physical work by remotely operated robots is promising for solving the labor shortage in the service industry, such as nursing and caregiving, and for ensuring safety in dangerous work, such as at medical institutions and disaster sites, during a pandemic. However, such remote operation always involves a delay between the operator and robot, and that delay is known to have adverse effects such as work errors, reduced work efficiency, and operator fatigue. It can thus be said that while the sophistication of remotely operated robots has increased their practicality, the effects of delays are becoming impossible to ignore. The delay is caused by not only the communication but also the operation of the camera, display servo, etc., and the control of these systems. Even if a low-latency system is used and remotely controlled over a very short distance, a round-trip delay (namely, the sum of the delay between the incidence of light at the camera and screen display to the operator and the delay between the operator’s operation and actuator’s operation) of 100 ms or more occurs.
We therefore aim to develop a remote-operation system that can fully demonstrate the capabilities of the operator by eliminating the negative effects of delays through four technologies: prediction of the operator’s motions, prediction of physical changes in the work area, correction of the robot’s motions, and correction of the operator’s perception (Fig. 5).
One example of predicting the operator’s motion and correcting the robot’s motion is predicting the motion of the operator several hundred milliseconds in the future and moving the robot before the operator performs the motion in accordance with the prediction. Therefore, the effect of delay is reduced (Fig. 6). However, it is possible that the robot’s motion may be wrong (namely, unintended by the operator), so we are currently working on improving prediction accuracy by estimating the operator’s intentions on the basis of the content of the work and motion of objects.
As an example of predicting physical changes in the work area and correcting the robot’s motion, we simulated a robot’s physical structure to determine how it moves according to the operator’s motions. Presenting the simulation results to the operator through augmented reality (AR) and other means allows the operator to recognize the results of the robot’s motions before the robot actually moves, which reduces the effects of delays (Fig. 7). Since the operator will be working on the basis of AR-based predictive information, in contrast to direct work, we are currently studying how to present information to support the operator more effectively during various tasks.
5. Future developments
We are currently refining these technologies on an individual basis. In the future, we will unify these technologies to provide services especially for domains that require flexible and fuzzy human-oriented support.