Abstract
NAVER Cloud presents HyperCLOVA X THINK, the first large language model (LLM) specifically engineered for advanced reasoning within the context of Korean language and culture. By integrating a massive 6-trillion-token bilingual corpus with innovative architectural choices and reinforcement learning from verifiable rewards (RLVR), THINK achieves state-of-the-art results on regional benchmarks. Notably, the model demonstrates high performance and stability while requiring significantly less training compute than comparable global frontier models.
Introduction
The recent emergence of reasoning-centric models, such as OpenAI’s o1 and DeepSeek-R1, has shifted the focus of LLM development toward complex logical processing. However, these models remain predominantly English-centric, often lacking the linguistic nuance and cultural grounding required for specific regions. This gap highlights the need for "Sovereign AI"—models tailored to the unique requirements of individual nations.
For the Korean market, developing such a model faces two primary hurdles: the scarcity of high-quality, complex reasoning data in Korean and the immense computational costs associated with training large-scale models. HyperCLOVA X THINK addresses these challenges through strategic data curation and architectural innovations, proving that regional excellence does not require "brute force" scaling.
Methodology
HyperCLOVA X THINK incorporates several technical advancements to optimize the balance between reasoning capability and computational efficiency.
Architectural Efficiency
The model employs a "shallower-but-wider" Transformer layout. By increasing width while reducing depth, the architecture minimizes the quadratic overhead typically associated with self-attention mechanisms without sacrificing representational power.
Stability-Oriented Scaling
To ensure stable training at scale, the authors utilize Peri-Layer Normalization (Peri-LN) and Maximal Update Parametrization ($μ$P). These techniques allow for hyperparameter tuning on small-scale proxy models, which are then directly transferable to the full-scale production model. This approach eliminates the need for expensive grid searches during large-scale training.
Training Curriculum and RLVR
The training process follows a three-stage curriculum: foundational knowledge acquisition (6T tokens), domain-specialized expertise (1T tokens), and long-context alignment (up to 128k tokens). During the final stage, the model internalizes Chain-of-Thought (CoT) reasoning.
A critical component of the post-training phase is Reinforcement Learning from Verifiable Rewards (RLVR). Unlike traditional RLHF, which relies on subjective human preferences, RLVR uses objective metrics—such as code compilation success or mathematical correctness—to provide ground-truth feedback. Using a modified version of Group Relative Policy Optimization (GRPO), the model is encouraged to explore diverse and accurate reasoning paths.
Data and Evaluation
The model was trained on a 6-trillion-token mixture of Korean and English data. To address the shortage of complex Korean reasoning examples, the team developed an in-house pipeline to generate high-fidelity synthetic data across domains like law, education, and history.
Evaluation was conducted using a comprehensive suite of Korea-centric benchmarks, including KMMLU (general aptitude), CSAT (college entrance exams), and KoBigBench (multi-step reasoning).
Key Insights and Results
Performance Benchmarks
THINK consistently outperformed similarly sized models, such as Qwen3-32B and EXAONE-Deep-32B. It achieved a score of 69.7 on KMMLU and 83.2 on the CSAT General Aptitude test. These results underscore the effectiveness of a regionally tailored corpus in offsetting the scale advantages of massive multilingual models.
Multimodal Reasoning
The vision-augmented variant of THINK demonstrated impressive cross-modal reasoning transfer. On the KCSAT STEM benchmark, it achieved 46.4% accuracy, surpassing GPT-4.1 (40.3%) and nearing the performance of OpenAI-o1 (50.9%). This suggests that logic learned during text-based pre-training can effectively generalize to visual problem-solving.
Reasoning Control
To address the "overthinking" issue common in reasoning models, THINK introduces Reasoning Length Controllability (LC). A penalty-based reward system allows users to specify a reasoning budget, making the model adaptable for various business applications where concise answers may be preferred over detailed CoT.
Conclusion and Future Work
HyperCLOVA X THINK represents a significant milestone for Sovereign AI in Korea. While the model achieves exceptional performance, the authors acknowledge ongoing challenges in ensuring 100% safety and mitigating the slight performance trade-offs when integrating multimodal capabilities.
Future work will focus on extending native reasoning to include audio modalities and releasing a distilled, open-source version of the model to support the broader research community.
Summary updated