New Technology for Measurement and Analysis of Biological Sounds and Electrocardiographic Signals—Toward Early Detection of Heart Disease and Rehabilitation by Using Personal Heart Modeling

1.1 Telestethoscope

Doctors and nurses use auscultation to determine the presence or absence of abnormalities and the degree of urgency. Skilled medical practitioners can visualize in their minds the generation and transmission of biological sounds such as breathing, movement of heart valves, and blood flow. Inspired by such auscultation by medical professionals, we are conducting research and development on a wearable device called the telestethoscope (see Fig. 1(a)).

The telestethoscope is used for acquiring biological sounds (acoustic signals) from multi-channel microphones and electrocardiograms (ECGs). The acquired acoustic signals and ECG can be sent to a remote terminal via a network. Sounds from various points on the wearer’s body can be heard by touching the screen of that terminal. An overview of the telestethoscope is shown in Fig. 1.

The telestethoscope will allow medical personnel to listen to acoustic signals of patients remotely without having to come into contact with patients; therefore, it will be useful for examining patients with infectious diseases and assessing the urgency of patient care from a distance (e.g., when the patient is at home or in a rural area) (Fig. 1(b)). Since it is possible to record and share biological sounds, it is also possible to repeatedly auscultate offline or chronologically study the status by comparing current and past sounds. In addition to the above use by medical professionals, we are also researching its use by the general public for self-healthcare, as described below.

1.2 Adding descriptive text to heart sounds by using the telestethoscope

It is not easy for non-specialists to understand the information about living organisms by simply listening to biological sounds. However, signal processing and machine-learning technology makes it possible to convert biological sounds into a form that is easy for many people to understand. Therefore, we are investigating converting (translating) biological sounds directly into simple wording (Fig. 1(c)).

The most-straightforward means of explaining biological sounds would be to classify them as “normal” or “abnormal.” However, if a person’s sounds are classified as “abnormal,” a more-detailed explanation may be needed. For example, it would be useful to know what abnormalities are present and whether the person in question should go to the hospital immediately. To address this issue, we previously proposed a method called conditional sequence-to-sequence caption generation, which allows the user to specify a numerical value called specificity that represents the level of detail of the description to be generated [1]. For example, if the heart sounds and a large number as a desired level of detail are input into the system, a detailed explanation can be output.

1.3 Estimating health status via video images from heart sounds acquired from the telestethoscope

Another means of converting biological sounds into an easy-to-understand form is to generate moving images that represent biological functions in real time (Fig. 1(d)). We have attempted to reconstruct the movement of the heart from the sound of the heart. A normal heart moves in a cyclic fashion in which the transitions occur in the following order: atrial systole, isovolumic systole, ejection phase, isovolumic diastole, and filling phase. We therefore aim to estimate the transitional state of the heart from measuring heart sounds then estimate and reconstruct the movement of the heart in three dimensions (3D). Since the heart pumps blood around the body, a specific physical relationship between its pressure and volume is maintained through the state transitions. Focusing on this physical relationship, we devised a method of constructing a 3D moving image. This method uses the physical relationship to improve the accuracy of cardiac motion estimation by restraining it in accordance with the state transition, which is estimated from the heart sounds [2].

Accurately estimating and predicting the physical and psychological state of an individual from easily observable biometric information, such as sound, will necessitate many other innovations. We will continue to research and develop signal-processing and machine-learning technologies for measuring biological functions and dynamics from multiple perspectives, such as sound information and electrocardiographic information, as described below.

Fig. 1. System overview of telestethoscope. (a) Telestethoscope wearable device, (b) remote auscultation, (c) automatic sound-description generation, and (d) visualizing estimated 3D cardiac motion in accordance with the estimated heart states.

2.1 3D ECG induction suitable for long-term recording in wearable device

ECGs are analyzed on the basis of their potential difference (scalar quantity) and shape (pattern), which are obtained from multiple bioelectrodes placed at specified points (standard 12 leads) on the surfaces of the extremities and chest. As a method of recording electrocardiographic potential more three-dimensionally, a vector ECG (which uses a three-axis Cartesian-coordinate system) involves using electrodes on the chest, head, and lower extremities and a resistance-correction circuit. Since both standard-12-lead and vector ECGs involve using electrodes on the extremities, they are easily affected by body movements, so they are basically recorded while the patient is in a resting state. However, electrodes attached to the chest are less affected by body motion and can produce relatively large cardiac potentials, so they are used for Holter ECG monitors (portable devices for cardiac monitoring), exercise-load, and sports ECGs.

For long-term stable recording of a 3D ECG, we devised a wearable devise for measuring ECG with its reference point set in the apex region of the heart (i.e., on the left anterior wall, where the heart is closest to the thorax) and with counter poles in three linearly independent directions (Fig. 2(a)) [4]. By bipolar induction^*1 generated by placing two horizontal leads on opposite sides of the thorax and vertical leads on the upper right precordium in line with the electromotive-force base axis^*2, 3D electrocardiographic potential can be stably recorded (Fig. 2(b)). The electrodes and wires are integrated into an elastic belt that can be easily attached by simply tightening the shoulder and waist parts of the belt. We have also developed a non-invasive cardiac polygraph with hemodynamics for simultaneously measuring cardiac output and deep vascular pulse waves (Fig. 2(c)).

2.2 New method for analyzing ECGs: tensor ECG

Abnormalities in ECGs are shown as not only a standardized abnormal waveform (pattern) but also as a slight distortion of the shape or change in potential. A method of quantitatively evaluating such atypical ECG abnormalities has not been developed. We are currently designing individual criteria for specific diseases and special patterns to identify them. Regarding the collective action potential of the myocardium, the electromotive force of the cardiac potential has three plateau phases with continuous depolarization, whereas the cardiac potential at the body surface does not usually show such a plateau. We use the molecular-biology and physiological constraints of the membrane potential of each period (from phases 0 to 4) of the action potential to estimate the collective action potential of the myocardium from the electrocardiographic potential on the body surface (i.e., an inverse problem) (Fig. 2(d)). From the conversion of electrocardiographic potential to action potential, atypical distortions are magnified and made clearly visible (Fig. 2(e)). We are examining whether the parameters (tensor[s]) obtained during the conversion to action potential of the myocardium can be used as indicators for quantifying and uniformly evaluating complex abnormal patterns and minute distortions in ECGs. We anticipate that tensor ECG for multi-mode data measurement and analysis will be useful in diagnosing arrhythmias associated with heart failure, ischemic heart disease, and sudden cardiac death.

Fig. 2. Overview of tensor ECG: a new induction and analysis method.