The Race Is Decided Early! A Kernel Law Predicts Lock-In in Majority-Vote Test-Time RL
Hwang et al.
In test-time reinforcement learning rewarded by majority vote, each question locks into one answer early. The authors derive the probability that the runner-up wins as a normal tail probability of the initial logit gap, corrected through the kernel of the shared parameters.
Source published · In the Weekly Review of 15 November 2026
Filed under Reasoning and inference, Probability
Test-time reinforcement learning trains a language model on unlabelled questions by rewarding the samples that agree with its most frequent answer. During training each question locks into one answer, on hard questions often a wrong one, and until now which answer won was known only afterwards.
The authors show the outcome is decided early, in a short race driven by sampling noise between answers that start close together. With a separate logit for each answer, the runner-up wins with a normal tail probability of the initial gap, scaled by an explicit noise constant. In a network the logits share parameters, so updates to trailing answers also move the gap between the two leaders; a closed form computes the shifted probability from a kernel probe and the starting logits. In LoRA experiments on three models from two families, predictions made before training, with the kernel frozen at its starting value, forecast the locked-in answer better than a logistic model refitted on the outcomes.