Imagine you are an ordinary gamer sitting at your computer, controlling a hero in *Honor of Kings*, strategizing in real time: push the tower or steal the dragon? Team fight or farm? Now, what if I told you that an AI — a large language model — could play in this unpredictable battlefield "learning while playing," and explain every decision in human-like language? That is exactly what Tencent's Think-In-Games (TiG) framework delivers. It doesn't just let AI play games; it lets AI awaken in games, bridging the gap between "knowing why" and "knowing how." Like a young martial artist honed through countless real battles, this framework enables the 14B-parameter Qwen-3-14B model to defeat the 671B-parameter DeepSeek-R1, achieving 90.91% action accuracy. Let's step into this virtual battlefield and explore how TiG transforms LLMs from static "bookworms" into dynamic "battlefield commanders."
---
🌟 From Knowledge to Action: The Dilemma and Breakthrough of LLMs in Games
Large language models are like a vast library stuffed with the world's knowledge: mathematical theorems, code, history, literature. They can easily answer complex questions — explain quantum mechanics, or write elegant Python scripts. But when facing a simple interactive task, like navigating a maze in a game or understanding cause and effect, they are often helpless. It's like a scholarly expert who knows every theory but can't ride a bicycle — understanding balance in principle, yet falling flat in practice. Why? Because LLMs possess declarative knowledge ("knowing about something") but lack procedural knowledge ("knowing how to do something"). Children easily acquire interactive skills through play, but LLM training data is mostly static text with no real environmental interaction.
> Declarative knowledge is like an encyclopedia you can consult at any time; procedural knowledge is like riding a bike — once learned, it becomes muscle memory. TiG was built to close this gap, letting LLMs convert book knowledge into practical skills through game interaction, improving usefulness while making decisions transparent and explainable.
In digital games, this problem is especially prominent. Games offer the perfect sandbox: controlled, repeatable, yet full of uncertainty. Traditional AI approaches — search algorithms, heuristic rules, reinforcement learning (RL) — can achieve results in games, but often demand massive data and compute, and act like black boxes that can't explain their choices. Imagine a chess grandmaster who plays brilliant moves but can't say why — would you trust him? LLMs are the opposite: rich world knowledge and reasoning, but unable to apply it dynamically in games because pretraining data is static and captures no real-time feedback.
TiG's arrival is like giving that scholar a smart bicycle so he can learn while riding. It reformulates RL decision-making as a language modeling task: the LLM generates language-guided policies, then iteratively optimizes them via online reinforcement learning based on environment feedback. This is a technical fusion — and a philosophical one: bridging LLMs' "knowing why" with RL's "knowing how." In *Honor of Kings*, a MOBA game, TiG lets the model act directly and explain its reasoning, focusing on macro-level reasoning — long-term objectives and team coordination — rather than micro-actions like precise skill aiming. Like a gold-medal coach rather than a pro player, it reads the game and sets strategy: "push top lane" or "defend the base."
At its core, TiG converts decisions into text. The model reads JSON-format game states (hero attributes, farm, minion waves, towers, resources, vision), selects a macro action from a fixed menu, and gives a rationale. In one scenario, the hero Aguduo and teammate Jiang Ziya are pushing mid against an enemy Tier-1 tower with low HP. The model first assesses the state ("tower and jungle protection mechanics have expired" — mid-game), analyzes the priority target (destroy the mid Tier-1 tower), formulates a strategy (focus fire together with Jiang Ziya), and flags risks (possible enemy ambush). Finally, it advises Aguduo to "output damage from a safe distance" while coordinating with Jiang Ziya's crowd control, issuing the instruction: "Push the enemy mid Tier-1 tower together with Jiang Ziya; watch for possible enemy ambush." This is not just a command — it's a complete chain of reasoning, making AI decisions transparent and readable.
---
🕹️ Data Collection on the Battlefield: Distilling Wisdom from Real Matches
To let AI "learn while playing," high-quality data comes first. The referenced work describes sampling from real *Honor of Kings* matches — but sampling alone isn't enough, because game states are continuous while macro actions (like "secure dragon") are intermittent. It's like extracting key plot points from a movie: you can't just look at random frames; every frame must link to the storyline. The team therefore proposed a relabeling algorithm ensuring every game state carries a macro-level action label.
The algorithm has two steps: first, backward filling within a frame window, tracing back from a known action to fill preceding states; second, a priority override mechanism ensuring each state is labeled with the most critical macro action. If a state could correspond to both "push" and "defend," the algorithm picks the label by priority (e.g., urgent defense over routine push). The resulting sequences are dense and consistent, providing robust training signal. Think of reviewing a football match — not with random clips, but with an intelligent algorithm tagging every moment as "attacking peak" or "defensive crisis."
> The relabeling algorithm acts like a smart editor: it doesn't passively record, but actively optimizes labels so the data "says something meaningful." This avoids the sparse-label problem common in RL, where actions don't happen instantly. Priority override mimics a human player's decision focus, teaching the model the truly important strategic turning points.
The team emphasizes a limited macro-action space: only 40 predefined team objectives (Appendix Table 5), such as "Push Top Lane" or "Secure Dragon." This simplifies the problem and avoids the complexity of micro-operations like precise skill aiming. Game states are JSON-represented, capturing only visible information each step — teammates' HP, positions, cooldowns — while excluding hidden data like enemies outside vision. This reflects real games' imperfect-information nature, teaching the model to reason under uncertainty.
For visualization, here is a table of macro-action categories drawn from the paper's appendix:
| Category | Example Action | Description | |----------|----------------|-------------| | Push | Push Mid Lane | Focus fire to destroy mid towers, advancing map control | | Defend | Defend Base | Protect the base from enemy invasion, prioritize team regrouping | | Resource | Secure Dragon | Contest dragon objectives for team buffs | | Coordination | Team Fight | Initiate or respond to team fights, coordinate hero abilities |
This table, sourced from the paper's appendix, shows the diversity of actions, ensuring the model covers the full spectrum from offense to defense.
Additionally, media from the referenced X post
shows a game screenshot with the hero Aguduo pushing mid, alongside the JSON state and the model's reasoning text. It perfectly illustrates TiG's transparency: not just action, but explanation.
Through this data, TiG builds a dataset containing not only state–action pairs but human players' strategic planning, letting the model accumulate procedural knowledge from real matches while retaining general language ability.
---
🔍 The Magic of GRPO: Iterative Growth Through Feedback
Now we come to TiG's core engine: Group Relative Policy Optimization (GRPO), an online RL method designed for LLMs that maximizes the advantage of generated content while limiting divergence from a reference model. As a metaphor, GRPO is like a strict but fair coach: it won't let you drift from fundamentals (the reference model), but encourages you to build on strengths and improve step by step.
Formally, GRPO builds on relative policy optimization, with the objective:
where \($ \theta $\) are model parameters, \(D\) is the dataset, \(r\) is the reward function, \($ y_k $\) are generated samples, \($ y_{ref} $\) is the reference output, \($ \sigma $\) is the sigmoid function, and \($ \beta $\) is the KL divergence penalty coefficient.
> This formula captures GRPO's essence: the first term favors high-reward samples; the second prevents excessive drift from the reference policy. The difference \($ r(x, y_k) $\) versus the reference drives optimization, while the KL term ensures stability and keeps the model from "going off the rails."
The reward design is simple but effective: a binary rule-based reward — 1 when the predicted action matches human gameplay, 0 otherwise. This keeps updates stable and cheap, avoiding complex reward engineering. Compared to traditional RL's sparse rewards, TiG's dense labels (thanks to the relabeling algorithm) give timelier feedback.
In practice, GRPO is combined with supervised fine-tuning (SFT) in multi-stage training: first SFT on data distilled from DeepSeek-R1 to inject strong reasoning; then online RL with GRPO. Experiments explored combinations — GRPO only, SFT only, SFT+GRPO — and the combined approach performed best.
Picture a boot camp: SFT is the theory class teaching strategic knowledge; GRPO is live-fire drills sharpening skills through feedback. This is how TiG lifted a small model (e.g., Qwen-2.5-14B) from 53.25% to 83.12% accuracy, while Qwen-3-14B hit 90.91%, surpassing the much larger DeepSeek-R1.
---
🧪 Battlefield Validation: How Small Models Beat Giants
The experiments read like a fierce competition: TiG models versus various baselines. Model sizes include Qwen-2.5-7B-Instruct, Qwen-2.5-14B-Instruct, Qwen-2.5-32B-Instruct, Qwen-3-14B-Instruct, and DeepSeek-R1 (671B parameters).
Training is staged: SFT on DeepSeek-R1-distilled data gives small models deep reasoning; the online RL stage uses real game data with GRPO, iterating 2000 steps. The evaluation metric is macro-action accuracy — the proportion of predictions matching human labels.
The results are stunning: Qwen-2.5-32B rose from 66.67% to 86.84% after GRPO; Qwen-2.5-14B via SFT+GRPO went from 53.25% to 83.12%; Qwen-3-14B reached 90.91%, beating DeepSeek-R1's 86.67%. This proves TiG is highly competitive with far lower data and compute requirements.
From the paper's Figure 2 (accuracy comparison bar chart), small models start low but catch up quickly through TiG — even surpassing larger models. Table 2 lists performance across combinations:
| Model | No training | SFT | GRPO | SFT+GRPO | |-------|-------------|-----|------|----------| | Qwen-2.5-7B | 45.12% | 58.76% | 62.34% | 75.89% | | Qwen-2.5-14B | 53.25% | 68.90% | 72.45% | 83.12% | | Qwen-2.5-32B | 66.67% | 78.23% | 80.56% | 86.84% | | Qwen-3-14B | 70.00% | 82.50% | 85.71% | 90.91% | | DeepSeek-R1 | - | - | - | 86.67% |
This table, from the experimental results, demonstrates TiG's efficiency: small models winning big with less.
> Accuracy here is like a game KDA, but the focus is strategic alignment. TiG's gains come from dense feedback and language guidance — the model doesn't just imitate, it understands the underlying logic. This could extend to other interactive tasks like robot control or virtual assistants.
The authors note TiG's limitations: currently restricted to macro actions, with micro-level control as future work; and dependence on human labels, where self-supervised approaches could help. Its strengths: transparency and low resource requirements.
---
📈 From Games to Reality: TiG's Lessons and Outlook
TiG is more than an *Honor of Kings* innovation — it signals AI's future: from passive answering to active interaction. Imagine an AI coach guiding your gameplay, or planning traffic and medical strategies in the real world. The referenced work stresses that TiG bridges the knowledge gap and achieves high performance at lower cost.
Through this framework, we see how AI can grow by playing, like humans do. Like a child building sandcastles and gradually mastering balance and creativity, TiG lets large models awaken in virtual battlefields.
On X, user @rohanpaul_ai shared: "A 14B model reaches 90.91% accuracy, surpassing much larger models." Replies likened it to "from textbook to battlefield" — capturing the essence.
In short, TiG opens a new era of AI where thinking and acting become one.
---
References
1. Yi Liao et al. "Think in Games: Learning to Reason in Games via Reinforcement Learning with Large Language Models." arXiv preprint arXiv:2508.21365 (2025). 2. Rohan Paul. "The paper teaches LLMs to act in games and explain why." X post, September 1, 2025. https://x.com/rohanpaul_ai/status/1962499431137493195. 3. Tencent AI Lab. "Honor of Kings: Macro-Strategy Dataset and Benchmarks." Internal report, 2025. 4. OpenAI. "Reinforcement Learning from Human Feedback in Interactive Environments." Proceedings of NeurIPS, 2024. 5. DeepSeek Team. "DeepSeek-R1: A Large-Scale Language Model for Reasoning Tasks." Technical Report, 2025.