Not stated in source 2025 Needs review

Learning to Plan with Reinforcement Learning

Abstract

This thesis investigates the domain of learning to plan within reinforcement learning. Traditional planning algorithms, such as Monte Carlo Tree Search, are typically handcrafted and lack the flexibility inherent in biological planning systems. To address this limitation, the Thinker algorithm is proposed. A central theme of this work is the shift in focus from designing specialized planning algorithms to augmenting the environment. By treating planning as an explicit action within an augmented environment, the agent is enabled to learn to plan using standard model-free reinforcement learning. Additionally, this thesis examines the interpretability of planning behaviour of RL agents trained by different algorithms. Finally, this thesis extends the Thinker algorithm to Large Language Models trained on mathematical reasoning tasks, demonstrating that similarly structured augmented environments significantly reduce computational costs while maintaining competitive performance on reasoning tasks.

Keywords

Not available

Specialisations

Similar theses