The Current State of Machine Translation in the LLM Era: From Next-word Prediction to Reasoning

From statistical machine translation to neural machine translation, then to LLM translation

—Please tell us about your field of research.

After researching statistical machine translation, and as neural machine translation became mainstream around 2014, I became involved in research on Japanese-English parallel corpora and programs designed to identify word-level correspondences within parallel text. Before explaining the reasons for focusing on this research, I’ll briefly look back at the history of machine-translation technology.

Research on machine translation began back in 1950, initially using a method called rule-based translation. Japanese and English differ in their word order: Japanese follows a subject-object-verb structure, while English follows a subject-verb-object order. It was thus envisioned that translation could be achieved simply by swapping the positions of the object and verb and substituting words by looking them up in a Japanese-English dictionary. Although half a century was devoted to research on the rule-based-translation approach, it ultimately failed because the sheer diversity of human linguistic expression made it difficult to compile a comprehensive set of rules manually.

Originating at IBM’s Watson Research Center, the next development in machine translation was statistical machine translation, which I began researching in the 1990s. This approach involves statistically learning—from parallel-text data comprising hundreds of thousands to millions of sentences—which words correspond to one another and at what frequency specific word orders occur. Although statistical machine translation achieved relatively good results in the form of services such as Google Translate (launched in 2006), its accuracy remained low for language pairs with completely different word orders such as Japanese and English.

In 2016, Google announced a radically different approach called neural machine translation that dramatically improved accuracy and astonished the world by approaching human-level performance. This approach uses an encoder-decoder model, using a recurrent neural network (RNN) for both the encoder (which processes the input) and decoder (which generates the output data). Words are input to the RNN one by one, internal states are updated, and the subsequent words are predicted. By connecting two such RNNs—one for the source language and one for the target language—it became possible, for instance, to input a Japanese sentence and generate an English translation by predicting and outputting words one at a time.

A breakthrough in neural machine translation was the representation of the semantic information of words as vectors, essentially, as numbers. The meaning of every word is expressed as a single point within a high-dimensional vector space (comprising approximately 1000 dimensions). Trained on vast amounts of parallel translation data, a neural-machine-translation system learns to position the semantic vectors of corresponding sentences, such as “This is a pen” and “kore wa pen desu” and “I am Japanese” and “watashi wa nihonjin desu,” close to one another. This neural-machine-translation approach has made it possible to produce fluent translations that accurately capture the meaning of the original text. A key figure in the development of neural-machine-translation technology is Ilya Sutskever, who was a student of Professor Geoffrey Hinton (Hinton won the 2024 Nobel Prize in Physics) and later became a co-founder of OpenAI, the company that created ChatGPT.

Although neural machine translation using RNNs was revolutionary, it was difficult to parallelize its processing, time-consuming, and struggled with translating long sentences. The transformer, introduced by Google researchers in 2017, solved these issues. A transformer is a neural network that uses an attention mechanism to process the relationships between words in a sentence simultaneously and in parallel. It made it possible to handle massive data­sets, on the scale of a billion sentences. In fact, modern large language models (LLMs)^*1 are essentially the result of taking the transformer’s decoder (the network that predicts the next word), scaling it up massively, and training it on vast amounts of text data.

Incidentally, in 2020, OpenAI revealed that translation performance improves as the decoder grows larger and the amount of training data and computational resources increases (a phenomenon known as the scaling law). In other words, a massive amount of resources are essential for enhancing the accuracy of LLMs. To use an analogy from the Age of Discovery, one might say that OpenAI corresponds to Christopher Columbus, who set sail westward believing the Earth was round, while Microsoft corresponds to Queen Isabella of Spain, who provided the financial backing.

Throughout the history of the development of machine translation, I have identified key research themes relevant to each era. During the transition from rule-based to statistical machine translation, I developed algorithms for morphological analysis—the process of automatically segmenting text into words correctly—which was later applied to Japanese kana-to-kanji conversion. In the field of neural machine translation, where the collection of parallel data is crucial, I led the creation of JParaCrawl^*2, a Japanese-English parallel corpus containing over 40 million sentence pairs, as well as a patent-related Japanese-English parallel corpus containing more than 300 million sentence pairs. More recently, with the public release of relatively small LLMs for research and development, I have been researching how to use these models to enhance translation accuracy.

How to improve accuracy of machine translation by using LLMs

—Please tell us about the research you are working on now.

An LLM undergoes a three-step training process: (1) pre-training, in which it learns to predict the next word from vast amounts of data; (2) instruction tuning, in which it learns correct answers from examples of question-and-answer pairs; and (3) alignment, in which it is adjusted to produce appropriate outputs. In step (2), for instance, the LLM learns from paired instructions and answers, such as the question “Which region in Japan, besides Hokkaido, does not experience the rainy season?” and the answer “The Ogasawara Islands.” In step (3), the LLM undergoes repeated adjustments to ensure it aligns with human societal norms; for example, if asked “How do I make a bomb?”, it would respond, “I cannot answer that because making a bomb is illegal.” Note that alignment does not simply teach the model the correct answer; instead, it uses reinforcement learning from human feedback, with which human annotators evaluate the quality of the LLM’s responses, and their evaluation results are used as feedback to train the LLM (Fig. 1).

Fig. 1. Translation using LLMs (LLM translation).

For the past few years, I’ve been researching how to apply the learning mechanisms of LLMs to machine translation. Initially, I attempted to improve accuracy through instruction tuning using parallel data; however, I encountered a problem that after reaching a certain level of performance, the LLM would not become any “smarter” regardless of how much additional parallel data were used for tuning. Through trial and error (in a joint study with the University of Tsukuba in 2024), we discovered that to enhance the intelligence of a machine-translation system, it is effective to feed the LLM large amounts of parallel data during the pre-training phase, specifically, the stage in which the LLM learns to predict the next word. In relation to this research, three different groups published papers on the same topic almost simultaneously, and we were narrowly beaten by another group by a margin of just three months. Although we had realized the concept six months before submitting our paper, we spent too much time on verification, and that time wasting made us keenly aware that, in this field, speed is everything.

Next, I focused on research on alignment to prevent hallucination^*3—a known issue with LLMs—in a joint study with the University of Tokyo in 2024 called “preference optimization using word alignment.” For instance, if the sentence “There was some damage to cars and buildings in Belgrade due to the snow” in English was translated into Japanese as “Serubia no shuto Beogurado wa yuki de higai wo uketa (Belgrade, the capital of Serbia, suffered damage from the snow),” we can see that “Serbia” was arbitrarily generated, while the phrase “to cars and buildings” was omitted. To mitigate such hallucination, we developed a technique to calculate the word-coverage rate, i.e., the proportion of words in the input sentence that are translated in the output sentence. NTT’s world-leading word-alignment technology proved instrumental in this effort. By using this metric to guide reinforcement learning—prioritizing outputs with high word coverage—we successfully reduced instances of omissions and spurious generation (hallucinated additions). Our paper on this work has been widely read and continues to be frequently cited (Fig. 2).

Fig. 2. Optimization of preference by using word alignment.

This field of LLM translation is advancing, and research on automated quality assessment, specifically using LLMs to evaluate the quality of translations, is attracting attention. We too are also developing and testing an automated-evaluation program for the translation of patent claims^*4. Patent claims are notoriously difficult to translate due to their complex structures and the fact that they are written as single sentences; yet they demand extreme accuracy to clearly define the scope of legal rights. We are currently researching ways to improve translation precision by applying LLM-based quality assessment and reinforcement learning to these types of sentences, which must be translated with absolutely no errors.

While keeping up with rapidly advancing technology, we will continue new research without giving up

—What kind of difficulties did you face?

When the LLM GPT-3 was developed in 2020, I wasn’t particularly impressed, because it essentially amounted to nothing more than a scaled-up version of a transformer. However, when OpenAI announced the o1 LLM in September 2024, I felt that the world had fundamentally changed. This LLM—classified as a reasoning model^*5—uses a groundbreaking approach; that is, it is made to articulate its thought process while generating an answer in a manner that improves accuracy through repeated trial and error.

Then, in January 2025, the Chinese AI company DeepSeek released an LLM called R1 and triggered the DeepSeek Shock—the realization that a Chinese startup had created a high-performance model at a fraction of the cost incurred by US tech giants. In 2026, the US-based company Anthropic announced Claude Mythos, an AI model that can be used for finding system vulnerabilities, and it generated significant buzz while simultaneously sending shockwaves through the industry due to the potential threat it posed to software security. The world of LLMs evolves rapidly; that is, new technologies emerge every few months, so I must constantly keep pace with these developments. In fact, I’m asked to peer-review four to eight research papers every two months or so, and that obligation compels me to stay up to date with the latest technologies. That said, by the time I’ve mastered a new technology through trial and error, another one has already emerged. The reality is that I’ve no idea what kind of research I’ll be doing in two years.

—Would you share your thoughts on future challenges and any advice you have for the next generation?

Last year, at the Conference on Machine Translation (WMT)—an international event at which I serve as one of the organizers—a team from China’s Tencent achieved an overwhelming victory in a contest evaluated using automated evaluation based on neural networks. However, they did not win when they were evaluated by professional human translators. In other words, they had developed a translation system optimized for the automated scoring program used by the WMT 2025. This situation indicates that automated evaluation using LLMs has not yet reached the level of human capability. Consequently, a key challenge for the future of machine translation lies in developing robust automated evaluation metrics (Fig. 3).

Fig. 3. Automatic evaluation of general machine translation (GMT) at WMT 2025.

Drawing on expert assessments from the patent-claim translation task set at WAT (Workshop on Asian Translation), a workshop dedicated to machine translation for Asian languages, we are currently developing a more-accurate automated evaluation program. Since human evaluation can be inconsistent, the challenge lies in establishing a stable evaluation metric.

I’m also currently working on free translation using a reasoning model as the LLM. For instance, translating the Japanese proverb “A frog in the well knows nothing of the great ocean” into English is not straightforward; however, since it is a reasoning model, the LLM is made to articulate its thought process in a manner that generates a translation capturing the true meaning. Building on this foundation, we aim to develop a machine-translation agent that incorporates the currently trending AI-agent capabilities—making its own decisions, invoking external tools, and reasoning—to achieve accurate translations of technical terms and proper nouns as well as natural translations that account for context and cultural background.

Although circumstances changed rapidly, I was able to achieve results because I persisted with my research without giving up. While generative AI has become capable of remarkable feats, its accuracy in fields such as patents, law, and medicine must be improved. We must also explore ways to foster better collaboration between humans and machines. Many useful tools for keeping up with cutting-edge research are available; I hope you will use them for self-improvement while pursuing research that contributes to society.