Frequency-Uniform Log-Regret Reinforcement Learning for Diffusions

High-frequency control problems are usually discretised in time, but refining the step \(\delta\) raises the discrete horizon \(H = T/\delta\), and standard finite-horizon regret bounds degrade even when the physical horizon \(T\) is fixed. The paper studies episodic value-based reinforcement learning for Euler discretisations of controlled diffusions, measuring regret against the optimal policy of the Euler model.

Under stability and regularity conditions on value-function gradients, Bellman residual derivatives and fitted critics, the authors prove a regret bound of \(O(\log K)\) after \(K\) episodes, uniformly in \(\delta\). For drift linear in unknown parameters and an unknown constant diffusion covariance, they verify these conditions for a constrained model-based fitted-\(Q\) algorithm and obtain high-probability regret of \(O(\log K \log\log K)\), again uniform in \(\delta\). Continuous-time control problems of this kind are common in portfolio and execution models.