How Does Chain-of-Thought Supervision Teach Models to Reason? A Theoretical Analysis

Models trained on finite teacher chains of thought may need to keep reasoning beyond what they saw, and longer reasoning can degrade performance. The authors study this with an in-context weight prediction model trained on finite gradient descent trajectories for linear regression.

High-dimensional asymptotics show that success and failure of long-horizon reasoning undergo phase transitions depending on context length and the total training budget. They distinguish error accumulation from error amplification, and find that with short contexts a larger budget can worsen long-horizon error before reasoning stabilises. Trained linear and multi-head softmax attention show the same qualitative behaviour.