Recreating whole-body functions in the human metaverse will aid in disease research and treatment
—Please tell us about your collaborative research at the University of Osaka’s Premium Research Institute for Human Metaverse Medicine (PRIMe).
I feel that with the advent of artificial intelligence (AI), which encompasses large language models (LLMs) as represented by ChatGPT, the research environment has changed dramatically. The impact of these changes extends beyond information and communications to other fields and is having a major impact on my specialty, i.e., the medical field.
As the digitization of various data, including test results and images, progresses in clinical settings, the use of AI is becoming more widespread. At the same time, research into genes, molecules, and cells, as exemplified by research into induced pluripotent stem (iPS) cells, is progressing rapidly, and movements toward incorporating AI into this research are gaining pace. I feel that these changes are accelerating more than we had anticipated. Looking ahead, we at NTT Basic Research Laboratories believe that it is necessary to advance basic research in closer collaboration with university medical schools and university hospitals.
However, in the medical field, there is still a large gap between genetic and cellular data and actual pathology concerning adults, including the elderly. It is thus necessary to accelerate basic research to fill this gap, and as an international hub to promote this research, PRIMe was newly established at Osaka University with support from the Japan Society for the Promotion of Science. I have been involved in this project from the planning stage (Fig. 1).
Fig. 1. Collaborative research with the University of Osaka’s Premium Research Institute for Human Metaverse Medicine (PRIMe).
In this project, iPS cells are created from human cells, and from those cells, extremely small organs, such as miniature hearts and livers (measuring about 1 mm in size) are generated. In other words, we are now in an era in which organs can be created in a test tube by collecting cells from the skin or blood of a donor in a manner that does not damage the organs inside the body.
Of course, creating miniature organs in this manner poses technical challenges. One challenge is that, currently, we can only generate tissues at the maturity level of a fetus or baby; that is, we are unable to create mature adult tissues. Blood vessels play an important role in the process of tissue growth and maturation. In many cases, tissues cannot function properly unless they grow to a certain size as a three-dimensional (3D) tissue with blood vessels; however, it is extremely difficult to create 3D blood vessels. Even so, it is now becoming possible to reproduce genetic mutations and abnormalities caused by drugs or the environment even in miniature organs. Our current research aims to reproduce cellular and genetic diseases by using such small organs, which we call “organoids.”
As I mentioned above, organoids are primarily capable of reproducing congenital disorders that manifest abnormalities in early childhood, and they have yet to reach the stage in which they can recreate more common and complex adult diseases. We are addressing this gap by recreating organoids in digital space using AI and simulation. The diverse and highly international team, including researchers specializing in iPS cells, bioinformatics, and mathematical models, including AI, is advancing multifaceted research in which many researchers from NTT laboratories are also involved. Through this interdisciplinary collaboration, we are conducting future-oriented research that aims to recreate individual organs in digital twins and ultimately recreate entire-body functions in the human metaverse as avatars.
Applying waveform-analysis technology developed for information and communications to medicine
—Please tell us about the “tensor cardiography” that you are currently developing.
Japan is becoming a super-aging society, in which the number of heart-failure patients is increasing in the manner of a so-called “heart-failure pandemic.” It is therefore necessary to establish technology for the early detection and treatment of heart disease. The heart is also my area of expertise, and I am developing a technology called tensor cardiography. By mathematically analyzing electrocardiogram (ECG) waveforms, this technology is proving to be a groundbreaking analytical method for visualizing early-stage lesions that were previously invisible.
With a history going back over 100 years, electrocardiography is used worldwide; however, under the current level of medical care, a wide variation in normal ECG values exists, so it is difficult to detect all cardiac abnormalities by ECG testing alone. Depending on the type of heart disease, it is often impossible to diagnose the disease until it has significantly progressed.
Analysis of ECGs using machine learning is widely used in the heart-disease field but requires prior training using large amounts of data. Some heart diseases are rare, and collecting large amounts of data on them is difficult. The lead time from heart attack to cardiac arrest is also short, so it is necessary either to respond immediately after detecting an abnormality or detect abnormalities in advance. High-speed heart monitoring using machine learning requires considerable processing power. The difficulty in explaining the medical basis of the diagnosis also remains an issue.
We are conducting research on the basis of the argument that a tensor cardiogram (TCG) system could effectively address the above-mentioned issues. The method of roughly classifying ECGs as normal or abnormal by measuring the amplitude, time, and width and examining the characteristics of the overall waveform has remained fundamentally unchanged from ECG invention. In fact, experts in information and communications and mathematical analysis were surprised to hear that the state of ECG analysis has remained basically unchanged for the past 100 years. Since joining NTT, I’ve spoken with many researchers about various physical models and become aware that ECG analysis technology is lagging, which prompted me to start this research.
Under those circumstances, we attempted to model the waveform of an ECG with a Gaussian function. We noticed that when the Gaussian function is expressed as a cumulative distribution, it adopts a unique shape that rises smoothly from bottom to top, and we wondered if we could use such a Gaussian function to model how electrical excitation is transmitted in the heart.
The heart is characterized by the fact that electrical excitation of the heart begins inside the heart and travels to the outside, while recovery from the excitation occurs in the opposite direction, i.e., from the outside to the inside. Once the myocardium is excited, it maintains that state for a while and begins to recover after a certain time. The duration of this excitation inside of the heart differs from that outside of the heart, and this difference affects the shape of the waveform.
We devised a model that uses Gaussian functions to represent the two electrical activities (action potentials) in the heart, excitation and recovery. It is stated in classic textbooks that the difference in the action potentials of the inner and outer myocardium closely matches the waveform of the ECG, and this matching is called the “dipole model” of the ECG. In fact, when we determined the difference between two cumulative distribution functions on the basis of our data, we found that this waveform matched an actual ECG waveform, and that match directly links this analysis method to established medical theory. It is now possible to express ECGs with simple shapes with a combination of only four cumulative distribution functions. In clinical practice, it has become possible to express them by combining approximately ten cumulative distribution functions (Fig. 2). Although this combination is quite practical, it has not yet been considered by researchers worldwide, so we have applied for a patent for this analysis method as an analytical model.
Fig. 2. Analysis method using a tensor cardiogram (TCG) system.
As our research progresses, it has recently become clear that tensor cardiography can have a much greater effect than we expected. For example, when a myocardial infarction occurs, blood flow to the heart is insufficient, and traditionally, an elevation or depression of the baseline level of a waveform called the “ST segment” has been used as an evaluation indicator on an ECG. However, an ST segment can only be detected once the heart has fallen into a state of severe ischemia, that is, when blood flow in the heart has almost ceased.
On the contrary, our new indicators (standard deviation of cathode & anode of T wave) parameterize features such as the ECG waveform’s mean value and variance with high precision that makes it possible to more sensitively capture minute changes on the time axis and demonstrate the possibility of promptly detecting changes in the state of cardiac blood flow (σTn, σTp, Fig. 3).
Fig. 3. Comparison of tensor cardiography and conventional electrocardiography conducted during coronary catheterization.
Ventricular fibrillation, which can cause sudden death, is a condition in which the heart muscles contract irregularly, in a manner like spasms; however, prior to the seizure, an “unstable period” appears normal at first glance. With a normal ECG, it is almost impossible to visually detect abnormalities during this unstable period. However, when we analyzed ECG waveforms by using Gaussian functions and their cumulative distribution functions, we confirmed that the order of recovery from excitation of the myocardium (namely, outside to inside) was reversed. We also revealed that just before a seizure, the reversal process and the distribution of recovery from excitation become significantly disrupted (Fig. 4). If we can detect these disruptions with precision, it may be possible to detect the signs of ventricular fibrillation before it occurs and take appropriate measures.
Fig. 4. Analysis of cardiac abnormalities in early repolarization syndrome by tensor cardiography.
One of the current challenges that we face is that our research is based on information and communications concepts rather than medical ones, so it is difficult for medical researchers to understand. When I submitted a paper to a journal on digital health, it was not accepted for about a year because they couldn’t find reviewers. In that sense, I am experiencing the challenges of being a pioneer. When I explain our research to experts in the fields of AI and information and communications, however, they often understand and say, “That approach seems normal to me,” so I feel somewhat reassured in that regard.
Although medical textbooks contain information about action potentials during myocardial infarction, that information comes from the results of past experiments on animals. Although other studies that have provided supporting evidence, the information has not been confirmed from the perspective of human hearts. When we analyzed patients’ ECGs with our TCG system, we found that the ECG patterns agree with those described in medical textbooks, so we hope that our system will eventually be accepted by the medical community. With that end in mind, we will continue to steadily accumulate experimental results and publish our findings.
Advancing original technologies such as hitoe™ for microscopic biological observation at the cellular and genetic levels
—Please tell us about your future research prospects.
Currently, biological information that we measure is a macro quantity such as ECG waveform data, that is, information as the bulk of each individual cell that composes the heart. However, by observing the body at the micro level of cells, molecules, or genes, we believe we will be able to elucidate diseases and mental and physical disorders from a more-cellular and -genetic perspective.
It has recently become possible to create small organs and other tissues at the chip level and apply them to experiments on drug discovery, for example. Growing organoids on these chips has become a recent trend, and by micro-scaling our conductive fabric called hitoe™, which we jointly developed with Toray Industries as “clothing for ECG testing,” we have been able to create electrodes for iPS cells. In line with this trend, we are working on microfabrication technology in parallel, and we are collaborating with researchers who have been investigating iPS cells since the basic-research stage.
Researchers worldwide are paying attention to this kind of micro-level cardiac modeling and digital twin, and since everyone is competing to exploit these technologies, for our research to stand out, we need to develop differentiated technology. We have two unique technologies, tensor cardiography and highly biocompatible electrodes in the form of a fabric called hitoe™. We aim to apply these technologies to the microscopic world of molecules and genes and the macroscopic world of ECGs in a manner that links abnormalities at different scales and obtains highly original data.
I want to continue generating and processing—at the macro level—multimodal, multiscale information obtained at the micro level and, in the future, be able to adapt these technologies to the analysis of big data from ECGs measured throughout Japan.
Envisioning a future in which medicine and ICT are deeply intertwined
—Please tell us what motivated you—despite being a doctor—to join NTT, an information and communications technology (ICT) company.
I was originally a clinical surgeon, but I later shifted to basic research on nerve regeneration, and that shift led to an opportunity to study abroad at the University of California, San Diego (UCSD), where I researched genetic regenerative medicine for spinal-cord injury. At the time, iPS cells had not yet been developed, and I was researching on differentiating embryonic stem cells into neurons and transplanting them. UCSD has a very active industry-academia collaboration, and when useful technology or patents are available, I was amazed at how quickly they move to put them into practice.
University professors were also actively working on social implementation of their research in mind, and seeing this situation, I began to wonder whether it would be a good idea for me to continue working at a university after returning to Japan. In fact, I did return to university, but the feelings I had from my time studying abroad were still strong, and I was impressed by NTT’s thriving cutting-edge basic research and its proactive approach to social implementation of technologies such as optical communications, so I decided to join the company.
When I first joined NTT Basic Research Laboratories, I researched brain-machine interfaces that communicate directly with the brain, because I thought this research might be an effective strategy for treating spinal-cord injuries. In this field, Neuralink, a company co-founded by Elon Musk and others, was active in the United States, but at the time, NTT Basic Research Laboratories was the only institute active in Japan.
I didn’t just want to research brain-machine interfaces, I also wanted to pursue a wide range of research into applying Japan’s cutting-edge technologies to the medical field. That research included, for example, cutting-edge materials and devices, as well as the digital field, which has developed rapidly, such as information and communications and AI technologies. I thought that in medicine, too, the time would come when these cutting-edge technologies would suddenly evolve classical simulations and diagnostic methods.
Some of the engineers that I met in Silicon Valley and San Diego during my studies there had been dispatched from various Japanese companies, including Kyocera and Sony, and many, especially those in San Diego, where Qualcomm is headquartered, were international students specializing in communications. Through interactions with those engineers, I came to understand the fascination of information and communications, and after returning to Japan, I was able to get involved in a variety of interesting projects at NTT, such as linking ICT to the idea of tensor cardiography that I talked about earlier and developing hitoe™.
Draw a big-picture vision, take action and build a team, and value the data you collected
—What is your message to future generations of researchers?
International competition in the field of basic research has intensified, and research on LLMs and AI has increased in scale. In this environment, there are many opportunities to ask yourself what exactly you can do as a researcher. In fact, I have been struggling with this question right up to the present, and I’d like to share four key thoughts on how we should act in an era of rapid change, including an explosive increase in the volume of information.
My first thought is to have a broad prediction or vision, even if it’s just a vague idea, of what the future holds for the research you are conducting. When I was studying in the United States, I met many people and began to imagine that in the near future, the fields of medicine and biology would become deeply intertwined with the field of information and communications, and we would enter an era in which AI and other technologies would be used for the former fields. I encourage you to read the latest trends in your fields of expertise and make broad predictions and visions for the future.
My second thought is to act on those predictions. For example, the PRIMe project at Osaka University started when I responded to a call from another party. In other words, I think it’s important to make friends both inside and outside the company and take action through those friendships, and I hope that young people in particular will take such action with eagerness.
For my third thought, since research is a world in which you must produce achievements and results amidst international competition, sometimes you will be subject to strict evaluations, and it is easy to fall into a defensive mindset. However, I urge you to be constantly free-spirited in your ideas, such as when creating concepts, and to take bold action without fear of failure. At the same time, I think it would be good if you could balance these two mindsets and achieve the results that are required.
For my final thought, research related to machine learning, such as that based on LLMs, has been widely reported in scientific journals and international conferences, and the content of cutting-edge presentations and volume of information can be overwhelming. However, while LLMs provide a vast amount of knowledge and information, it is not the LLM that collects each piece of information or set of experimental data and conducts research through trial and error. LLMs only learn a great deal from information that has been verbalized and made public. In contrast, the strength of researchers like us is that we can learn from the actual measurement data obtained by either conducting experiments or by painstakingly gathering data ourselves, and we can combine those data—which have not yet been symbolized or verbalized—with our experiences.
Although I’ve had many twists and turns and struggles, what saved me in the end was the data and experimental results that I collected directly. When I looked closely at my data, I often found that it revealed truths. I feel that my data serve as my guidepost while teaching me things that I hadn’t noticed before. I hope that researchers will use AI only as a supplementary tool while placing importance on steadily and diligently collecting and analyzing data themselves.
