Wireless-link-quality Prediction and Device-position Estimation Based on Relationship between Wireless-communication-link and Physical-space Information

1. Background and overview

Advanced wireless-communication systems, such as Wi-Fi, Long-Term Evolution (LTE), and the 5th-generation mobile communication system (5G), enable a wide variety of devices to connect to a network; thus, the number of connected devices and their traffic are exploding. The large number of connected devices will be a foundation of Society 5.0 [1], where a huge amount of physical-space information is accumulated in cyberspace. Physical-space information will be analyzed through artificial intelligence (AI), and the process is expected to bring new value to industry and society in ways not previously possible. Innovative Optical and Wireless Network (IOWN) [2] will accelerate the creation of big data of physical-space information and make them accessible from everywhere.

The advancement of AI technologies will also accelerate the use of physical-space information in the IOWN/6G era [3]. The accuracy of vision-based recognition using deep learning has been improving yearly [4], and the autonomous operation of robots and vehicles have also significantly advanced [5]. By using advanced AI technologies, physical-space information is expected to provide new value for wireless-communication systems.

Wireless communications is also being enhanced due to the demand of greater wireless link capacities, latencies, and reliabilities [6]. Wireless-communication systems, such as LTE, 5G, and Wi-Fi [7, 8], support a high-speed data rate and an increasing number of user terminals. The radio frequency of such systems is expanding to super high frequency (SHF) (3–30 GHz) and extremely high frequency (EHF) (30 GHz–3 THz) bands to obtain larger capacity. Since the link quality in the higher radio frequency is more subject to microwave propagation such as object shielding [7], the relationship between physical-space and wireless-communication-link information is becoming stronger.

NTT Network Innovation Laboratories is studying the relationship between physical-space and wireless-communication-link information and establishing the core technologies for new services in Society 5.0. In this article, wireless-link-quality prediction and device-position-estimation technologies are introduced to show the effectiveness of using the relationship between physical-space and wireless-communication-link information.

2. Physical-space and wireless-communication-link information

Figure 1 shows the relationship between physical-space and wireless-communication-link information. In Society 5.0, a huge number of devices will connect to a network using wireless-communication links, and cameras/sensors will be located everywhere and be equipped with mobility devices. Cameras/sensors obtain images of the physical space, and the information is updated via wireless or wired connections. The position and status information of wireless devices are also expected to be collected as physical-space information. Wireless signals will propagate through almost all the physical space, and the channel-state information of the wireless links and their link qualities are the result of the propagation through the physical space. Therefore, there are solid relationships between wireless-communication-link and physical-space information.

Fig. 1. Image of Society 5.0 and the relationship between wireless-communication-link and physical-space information.

NTT Network Innovation Laboratories has focused on the relationship between wireless-communication-link and physical-space information and developed wireless-link-quality-prediction and device-position-estimation technologies. Wireless-link-quality-prediction technology uses camera images and provides future throughput variation by using the detected movement of the surrounding objects. Device-position-estimation technology provides wireless-device position by monitoring the pilot signals between the wireless device and a single base station. Both technologies are based on supervised learning. We evaluated the performances of these technologies through experiments in actual outdoor and indoor environments, respectively.

3. Wireless-link-quality prediction using camera images

Wireless-link quality is strongly affected by the objects blocking the line of sight between a base station and user terminals as the radio frequency becomes high such as over the SHF band (> 3 GHz). Wireless-link-quality prediction for the millimeter wave (60 GHz) was proposed [9], and the authors used a depth camera to detect signal blocking. We developed prediction technologies for the SHF band by using high-definition (HD) cameras and confirmed that the long-term prediction corresponding to second-order future was achieved using physical-space information [10]. It is expected that long-term prediction will provide a sufficient lead time to counter the negative change in link quality. Our wireless-link-quality-prediction technology provides future throughput variation between a fixed base station and user terminal by detecting the movements of the objects surrounding the user terminal. We conducted outdoor experiments to confirm the effectiveness of this technology.

Figure 2 shows our developed user terminal testbed and the flow of wireless-link-quality prediction using camera images. The testbed communicates with the base station in the 5.6-GHz Wi-Fi channel. Cars, trucks, buses, and pedestrians passed through this site, and the testbed monitored the surrounding objects using two HD cameras (1920 × 1080 pixel). In wireless-link-quality prediction, the object-detection algorithm of M2Det [11] is used to obtain the bounding boxes of the moving objects from the camera images. It was confirmed that better detection algorithms provide more accurate wireless-link-quality prediction [10]. Object-detection algorithms output the bounding boxes and their classes. By using the bounding boxes, the throughput variation at one second in the future was predicted. The throughput-variation-prediction block uses random forest regression. Throughput variation is defined as the ratio of the future throughput to the median of the throughputs in the past 30 seconds. Therefore, throughput variation becomes 1.0 when there is no such variation.

Fig. 2. The wireless-link-quality-prediction testbed in the experimental environment. The camera images are used as the input of the object-detection algorithms, and the throughput-variation-prediction block provides future throughput variation by using the detected bounding boxes and their classes.

The measured dataset totaled 3490 seconds of data, containing 288 cars, 20 buses/trucks, and 36 people-transit events. The dataset was divided into 10 parts, and 9 were used for training to generate the throughput-prediction model. Wireless-link-quality prediction was conducted using the remaining dataset. The estimated and measured throughput variations are shown in Fig. 3. The vehicle-passing events and person-transit events denote the time when the vehicles or persons are passing around the user terminal. We can see that throughput degradation was predicted using camera images. Figure 4 shows the 90th percentile of the absolute errors of three prediction algorithms against the lead time. We compared the prediction performance of the wireless-link-quality-prediction technology using bounding boxes, past throughputs, and both past throughputs and bounding boxes. We found that the prediction performance using only the bounding boxes was the best when the lead time was greater than 1 second. These results indicate that the contribution of past throughputs becomes negligible for long-term prediction, and physical-space information is valid when the lead time is greater than 1 second.

Fig. 3. Predicted and measured throughput variation for person-transit and vehicle-passing events.

Fig. 4. 90% of the absolute error of throughput variation against lead time.

4. Device-position estimation

The relationship between wireless-communication-link and physical-space information can be used to find the physical-space information using the wireless-communication system information. Wireless sensing technologies have gained much attention [12] for the 6G era. Although the Global Positioning System (GPS) is the most well-known position estimation technology, it works only for outdoor environments and without tall buildings around the device. To obtain accurate position information in an indoor environment, wireless-communication systems are expected to play an important role [13]. We developed our device-position-estimation technology, which uses angle of arrival (AoA) estimation and deep learning using Wi-Fi feedback packets with a single access point [14].

This technology uses a monitoring terminal that detects the feedback signal in the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards, as shown in Fig. 5. The monitoring terminal obtains the information of the channel state information (CSI) between a base station and user terminal and the received signal strength indicator (RSSI) of the feedback signal, which corresponds to the link between the monitoring terminal and user terminal. The monitoring terminal calculates the correlation matrix R then outputs R as well as the signal-to-noise ratio (SNR) and RSSI into the position-estimation block. Since the elements of R contain the information of the phase differences in the arrival waves, the AoA estimation uses R [15]. The position of the user terminal is predicted using the input from the monitoring terminal and a deep neural network (DNN), which consists of gated recurrent units and three fully connected layers.

Fig. 5. Device-position estimation using Wi-Fi CSI feedback.

To confirm the device-position-estimation accuracy, we conducted indoor experiments using mobility robots. Figure 6 shows the developed user terminal and the indoor experimental environment. The user terminal ran in a figure eight, as shown with the black lines. The base station and monitoring terminal were at the locations shown in Fig. 6. The base station and user terminal use the 20-MHz channel in 5.6 GHz of IEEE 802.11ac. The number of transmit and receive antennas were 4 and 2, respectively. The CSI feedback signal of the user terminal was obtained every 100 ms, and 80 samples of the feedback signals were used as input of the DNN-based position estimation. We trained the position-estimation model using a training dataset containing 13.9 hours of the position and CSI feedback signal and evaluated the position-estimation error using a test dataset containing 1.0 hour of the CSI feedback signals.

Fig. 6. The user terminal and movement in an indoor experiment.

The cumulative distribution function (CDF) of the position-estimation errors are shown in Fig. 7. The position-estimation errors of the DNN with all the input features from the monitoring terminal, i.e., R, SNR, and RSSI; DNN with R; DNN with SNR and RSSI; and random forest regression with all these input features were compared. The DNN with all the features performed best and the median and 0.9 outage of the prediction errors were only 5.1 and 13.1 cm, respectively. The medians of the DNN with R and DNN with SNR and RSSI were 8.2 and 17.1 cm, respectively. The R, which is a parameter for AoA estimation, highly contributed to the accurate position estimation. We also found that the median estimation errors of the random forest regression with all features was 44.9 cm, which was 8.8 times that of the DNN with all features.

Fig. 7. CDFs of position-estimation errors.

5. Conclusion and future perspective

We presented wireless-link-quality-prediction and device-position-estimation technologies and discussed their effectiveness using a dataset measured in actual outdoor and indoor environments. Wireless-link-quality prediction uses the bounding box information detected using camera images and predicts precise long-term throughput variation. Device-position-estimation technology estimates the user device position on the basis of the correlation matrix, SNR, and RSSI obtained from the received Wi-Fi feedback packets. Both technologies are based on the relationship between wireless-communication-link and physical-space information. We believe that this relationship is one of the keys to creating new value in Society 5.0 and will continue to develop the core technologies of link-quality prediction and position estimation.

References

[1]	Cabinet Office, Government of Japan, “Society 5.0,” https://www8.cao.go.jp/cstp/english/society5_0/index.html
[2]	Website of NTT R&D, “IOWN,” https://www.rd.ntt/e/iown/
[3]	NTT DOCOMO, “DOCOMO 6G White Paper,” https://www.docomo.ne.jp/english/corporate/technology/whitepaper_6g/
[4]	L. Liu, W. Ouyang, X. Wang, P. Fieguth, J. Chen, X. Liu, and M. Pietikäinen, “Deep Learning for Generic Object Detection: A Survey,” Int. J. Comput. Vis., Vol. 128, pp. 261–318, Oct. 2019.
[5]	S. H. Alsamhi, O. Ma, and M. S. Ansari, “Convergence of Machine Learning and Robotics Communication in Collaborative Assembly: Mobility, Connectivity and Future Perspectives,” J. Intell. Robot. Syst., Vol. 98, pp. 541–566, 2020.
[6]	NGMN Alliance, “Perspectives on Vertical Industries and Implications for 5G,” Sept. 2016.
[7]	A. Osseiran, J. F. Monserrat, and P. Marsch (eds.), “5G Mobile and Wireless Communications Technology,” Cambridge University Press, 2016.
[8]	B. Bellalta, “IEEE 802.11ax: High-efficiency WLANs,” IEEE Wirel. Commun., Vol. 23, No. 1, pp. 38–46, Feb. 2016.
[9]	Y. Oguma, R. Arai, T. Nishio, K. Yamamoto, and M. Morikura, “Proactive Base Station Selection Based on Human Blockage Prediction Using RGB-D Cameras for mmWave Communications,” Proc. of the IEEE Global Communications Conference (GLOBECOM), San Diego, USA, Dec. 2015.
[10]	R. Kudo, K. Takahashi, T. Inoue, and K. Mizuno, “Using Vision-based Object Detection for Link Quality Prediction in 5.6-GHz Channel,” J. Wireless Com. Network., 207, Oct. 2020.
[11]	Q. Zhao, T. Sheng, Y. Wang, Z. Tang, Y. Chen, L. Cai, and H. Ling, “M2det: A Single-shot Object Detector Based on Multi-level Feature Pyramid Network,” Proc. of the 33rd AAAI Conference on Artificial Intelligence, pp. 9259–9266, Honolulu, USA, Jan./Feb. 2019.
[12]	C. J. Vaca-Rubio, P. Ramirez-Espinosa, K. Kansanen, Z. Tan, E. de Carvalho, and P. Popovski, “Assessing Wireless Sensing Potential with Large Intelligent Surfaces,” IEEE Open Journal of the Communications Society, Vol. 2, pp. 934–947, 2021.
[13]	A. Tahat, G. Kaddoum, S. Yousefi, S. Valaee, and F. Gagnon, “A Look at the Recent Wireless Positioning Techniques with a Focus on Algorithms for Moving Receivers,” IEEE Access, Vol. 4, pp. 6652–6680, 2016.
[14]	R. Kudo, K Takahashi, T. Murakami, and T Ogawa, “Deep Learning Based Position Estimation Method Using WLAN CSI Feedback,” IEICE Tech. Rep., Vol. 121, No. 173, CQ2021-45, pp. 40–45, Sept. 2021.
[15]	D. N. Simkins, “Multichannel Angle of Arrival Estimation,” Stanford University, 1981.