Reinforcement Learning Across Timescales
الملخص
This project addresses a fundamental problem faced by many reinforcement learning agents. Commonly used reinforcement learning agents can be seen to have deteriorating performances at increasing frequencies, as they are unable to correctly learn the ordering of expected returns for actions that are applied. We call this the disappearing reinforcements problem. Moreover, truly multi-task reinforcement learning is only possible when agents are able to operate across frequencies, as different platforms operate at different frequencies. Most algorithms from control theory working on similar tasks, on the other hand, show improved performances when their operating frequencies are increased. This suggests that addressing disappearing reinforcements should enable reinforcement learning agents to have improved performance and generalization ability across timescales and tasks. In this project, we show that disappearing reinforcements is an effect seen independent of the function approximator used in reinforcement learning, and is instead of a more fundamental nature. We explore both theoretically and empirically the relationship between agents and their performances at increasing frequencies. We show that two specific types of agents from literature address the problem, and we benchmark the agents' performances with novel benchmarking measures inspired from control theory. Finally, we create a novel agent we call the dueling advantage learner, by combining both approaches from the state-of-the-art. We then benchmark the different agents across frequencies and tasks, and our agent is seen to outperform each of the individual approaches on the majority of the tasks.
الكلمات المفتاحية
غير متوفر
التخصصات
رسائل وأطاريح مشابهة
- Deep Reinforcement Learning for Intrusion Detection and Response in Software-Defined Networks Trung Phan Van · Chemnitz University of Technology · 2026
- Reinforcement learning at the edge of chaos Rory Young · The University of Glasgow · 2026
- Safe Exploration in Reinforcement Learning Huy Nguyen · Aalto University · 2026
- Curiosity-based Reinforcement Learning GIANLUCA MANNONI · University of Pisa · 2026
- Achieving Robust Autonomous Driving through Multi-modality Deep Reinforcement Learning Kritsanaphong Kittithammo · 2026
- Diversity in Reinforcement Learning Arno Feiden · University of Bonn · 2026
