A New Training System Using VR Will Rev Up the Baseball Industry Like Never Before

1. Baseball-simulation system using virtual reality

The baseball simulation system that we provide uses virtual-reality (VR) technology to highly accurately reproduce an actual baseball ground and the pitching of actual pitchers in a 360-degree virtual space by matching the trajectory and speed of the ball. This system was developed using the research results from NTT Media Intelligence Laboratories and NTT Communication Science Laboratories (Sports Brain Science Project) and commercialized after several verification tests.

The key feature of this system is its high reproducibility. It is technically possible to reproduce pitching images with computer graphics (CG). An actual batter standing in the batter’s box focuses on the stance or even the interval between the pitcher’s movements and uses that knowledge to decide how to swing the bat or determine the type of pitch that will be thrown. The system therefore makes full use of actual throwing images to precisely reproduce positioning and timing when the pitcher releases the ball. By referring to actual pitching data, the system realistically reproduces the trajectory and speed of the pitch that the pitcher threw in a specific inning of a specific game. The same reproduction applies to the baseball ground in the background. By incorporating data taken on-site using a 360-degree camera, the system virtualizes a situation in which the player (batter) can receive the pitch as if he or she were standing on home plate. In an actual baseball ground, the atmosphere in the stadium and the appearance of the pitching differ between night and day games. This system can reproduce such environments in the VR space. By changing the background according to the purpose, it is possible to experience a more realistic space (Figs. 1 and 2).

Fig. 1. The baseball-simulation system.

Fig. 2. Reproduction of baseball ground with the baseball-simulation system.

2. Sports × VR

Although it may seem difficult to imagine using VR in the sports industry, VR can be advantageous for sports. Video analysis and the utilization of data are gradually becoming widespread in all sports disciplines. However, in many cases, such data are two-dimensional information, so they are understood using one’s intellect. VR makes it possible to experience things through the body in a highly immersive three-dimensional space, which may lead to more effective training. It is also possible to practice repeatedly with different ball types and speeds that are difficult to reproduce in the real world. Since the necessary equipment is compact, full-scale training is possible regardless of time, place, or weather.

3. Expansion to Japanese and US professional teams

The system is currently being offered for the purpose of training (improving performance of athletes) and entertainment (engaging with fans). There are two use cases regarding training. The first case is imagery training before a game. Reproducing, in VR space, the balls thrown by the starting pitcher of the opposing team in the previous game enables the batter to understand the pitching tendency of the pitcher and create a more concrete swing image. As mentioned above, more and more professional baseball teams are checking game videos and pitching data of opponents on tablets and other devices before a game. It can be said that being able to experience such data with the body, as opposed to understanding it with the mind only, is very effective just before a game. Although this system is basically for batters, we have had many requests from pitchers. Pitchers only have a limited number of ways to evaluate their pitching in the real world; however, with VR, it is possible to actually stand at the home plate and see what their pitching looks like from the batter’s perspective.

The other use case is visualization of a player’s performance. By monitoring the movement of the player (batter) via sensors while watching the pitch in VR space, it is possible to visualize the movement and timing of the swing. In addition to the head-mounted display (HMD) worn on the batter’s head, sensors are attached to the lower back and bat to monitor the swing movement in three axial directions (X, Y, and Z) (Fig. 3). For example, by comparing the pitching timing in the VR space with the movement of the swing, it is possible to determine, for example, delay and hesitation in the swing with objective data (Fig. 4). In this manner, it becomes possible to visualize—with objective data—things that are recognized as good or bad, which was previously difficult to verbalize, and athletes and coaches can better embody the training athletes need to overcome their weaknesses. In addition, by monitoring using VR for observing fixed points, it will be possible to determine the effects of training and certain changes.

Fig. 3. Swing movement is measured in three directions by using sensors.

Fig. 4. Sample results.

The system was initially introduced through verification testing to the Rakuten Eagles, a Japanese professional baseball team, in 2017. At that time, the purpose was to conduct imagery training to beat the pitcher of the opposing team before the game. However, we received feedback stating that “As well as watching the pitching, I want to check my swing movement while watching,” accordingly, we developed a sensing function for visualizing the batter’s swing movement while watching the pitching in VR space and added it to the system. We then focused on introducing the system to Major League Baseball (MLB) in the United States in consideration of MLB’s market size and environmental conditions.

The basis of the system for entertainment is the same as that for training, and the aim is to reproduce a ball thrown by an actual pitcher in an actual game with high accuracy. In the real world, it is unlikely a person will ever experience standing in the batter’s box in a real baseball stadium and receiving a ball thrown by a professional pitcher in a real game; however, this is possible in VR space. Being neither a batting cage nor a video game, we believe the system will provide a new means to enjoy baseball.

We launched a service for fans of Rakuten Eagles from the 2018 season onwards and a similar service for fans of Hiroshima Toyo Carp in 2019. The system is attracting attention as an effective tool for maintaining the relationship between teams and their fans in circumstances in which it is difficult to directly contact with fans during off-season or due to the current COVID-19 pandemic (Figs. 5 and 6).

Fig. 5. Top screen of entertainment version.

Fig. 6. Illustration of a play in the entertainment version.

4. Differences between Japanese and US sports markets

If baseball alone is looked at, we see that the market size in the United States is more than five times larger than in Japan, and the number of teams in MLB is 30 (that in Nippon Professional Baseball (NPB) is 12). Although that fact alone makes MLB valuable as a potential market, environmental conditions have a major impact on introduction of the system. Specialized staff, such as data analysts, are employed in all MLB teams. On top of that, if the system stands a chance of making a team stronger, even if it is a new tool that has never been available in the industry, MLB teams will actively try and find ways to use the system with the mindset of first giving it a try. As a result of their efforts, we were also able to compile effective use cases, acquire knowledge, and identify issues. On the basis of the usage record of the system in the United States, we are considering deploying the system to other teams such as more NPB teams.

5. Future developments: bringing change to the baseball industry

On the basis of our efforts thus far, we are currently redefining the system’s effectiveness, simplifying it to expand it to more NPB teams, and building and verifying a new service model. The VR equipment is not manufactured under the assumption that it will be used by sportspeople, so we want to optimize it as the manufacturer improves the VR equipment.

The introduction of the system to amateur baseball teams is also one of our goals. Compared to professional teams, amateur teams have a limited training environment and time. The types and speeds of pitches that can be experienced in actual games are also limited. Training using our system makes it possible to repeatedly practice for ball types and speeds that are difficult to train for in the real world. In the actual verification tests, we heard the comment that players who trained at a ball speed of 150 km/h in VR space began to feel that the ball speeds of 130 km/h and 140 km/h were slow in real practice, and it became easier for them to adjust their timing.

The ultimate goal of this project is to establish a platform that can visualize and manage player performance. The evaluation of a player’s performance has traditionally been highly opaque. An evaluation index for a batter is the batting average—the number of hits divided by the number of at-bats—which, however, is a comparison under different conditions such as opponent pitcher, time, and weather. In contrast, the reaction data for pitching in VR space is measured under constant conditions, including ball type and speed, so that data can be used to compare players. By managing the performance data of players from amateurs to NPB and MLB players in the form requested by each user, it is possible to support the development of young players within the team and the formulation of game strategy. It will also be possible to use the system to provide new reference data for scouting and drafting players, which used to be largely dependent on individuals. With these possibilities in mind, we would like to continue discussions with the individuals concerned.

This year’s National High School Baseball Championship, which is held in the spring and summer every year in Japan, was canceled due to the COVID-19 pandemic. Thus, it is necessary to establish a new format for the draft that provides amateur players a path to becoming professional players. As can be seen from the introduction of a limited number of pitched balls per pitcher per week at the National High School Baseball Championship, attention is also being paid to the viewpoint of a player’s career length. We thus believe that visualizing the player performance and making training more efficient will become even more important. We will contribute to the further development of the baseball industry by using new technologies and ideas regarding VR.