The paper "Reinforcement Learning without Ground-Truth Solutions can Improve LLMs" (arXiv:2606.27369v1), published on June 25, 2026, introduces **RiVER** (Ranking-induced VERifiable framework), a novel approach to training Large Language Models (LLMs) using reinforcement learning (RL) without requiring ground-truth answers. ### What Changed? Traditional Reinforcement Learning with Verifiable Rewards (RLVR) relies on ground-truth solutions to provide binary or specific rewards. RiVER shifts this paradigm by using **deterministic execution feedback** (e.g., scores from heuristic contests) as continuous-valued supervision. To make this effective, the authors identified and addressed two primary challenges in using continuous rewards: * **Scale Dominance:** Uncalibrated score magnitudes across different test instances can distort policy updates. * **Frequency Dominance:** Repeatedly sampling suboptimal solutions can outweigh rare, high-quality candidates. RiVER solves these by implementing **calibrated reward shaping**, which uses instance-wise comparisons and prioritizes top-ranked solvers while maintaining bounded feedback for other valid solutions. ### Why It Matters * **Broadens Training Environments:** It enables LLMs to learn from tasks where ground-truth solutions are unknown or difficult to obtain, such as complex optimization or heuristic-based coding problems. * **Generalization:** The study demonstrates that training on score-based tasks (like AtCoder Heuristic Contests) significantly improves performance on "exact-solution" benchmarks (like LiveCodeBench and USACO). This suggests that learning to optimize in one domain transfers to general coding and reasoning abilities. * **Efficiency:** It provides a way to improve model performance without the expensive and often impossible task of curating large datasets of ground-truth solutions. ### Key Results * **Performance Gains:** RiVER improved Qwen3-8B and GLM-Z1-9B-0414 by 8.9% and 9.4% in ALE (Algorithm Engineering) rating rank. * **Transferability:** The models showed an average absolute improvement of 2.4% on LiveCodeBench and 3.5% on USACO, despite never being trained on ground-truth solutions for those tasks. ### Strongest Source * **Primary Source:** Lin, Y., et al. (2026). *Reinforcement Learning without Ground-Truth Solutions can Improve LLMs*. arXiv:2606.27369v1. [https://arxiv.org/abs/2606.27369v1](https://arxiv.org/abs/2606.27369v1)