Investigating Overestimation Bias in Reinforcement Learning
الملخص
Overestimation bias is an inherent property of reinforcement learning algorithms that approximate maximum expected values by maximizing uncertain estimates. Since overestimation bias was identified in the literature, it has been generally considered to have a negative effect on reinforcement learning algorithms. In this thesis we investigate overestimation bias by examining Q-learning and conclude that overestimation bias may have either a negative or positive effect on reinforcement learning algorithms depending on the reinforcement learning problem. Based on this conclusion, we propose a new variant of Q-learning, called Variation-resistant Q-learning, to control and utilize estimation bias for better performance. We present the tabular version of Variation-resistant Q-learning, prove a convergence theorem for the algorithm in the tabular case, and extend the algorithm to a function approximation solution method. Additionally, we present empirical results from three different experiments, in which we compared the performance of Variation-resistant Q-learning, Q-learning, and Double Q-learning. The empirical results verify that Variation-resistant Q-learning can control and utilize estimation bias for better performance in the experimental tasks.
الكلمات المفتاحية
غير متوفر
التخصصات
رسائل وأطاريح مشابهة
- Natural Language Generation with Reinforcement Learning Thomas Scialom · 2022
- Absraction for Efficient Reinforcement Learning Alexander Telfar · 2020
- Securing Federated Learning: A Comprehensive Defence Against Privacy Attacks Moyeen, M A · Concordia University · 2025
- Learning to Plan with Reinforcement Learning Ming Hay Chung · University of Cambridge · 2025
- Explaining reinforcement learning Léo Saulières · 2024
- Artificial Intelligence in Modern Firewalls : Role of Artificial Intelligence in Network Security Wajeh Ahmad · Tampere University · 2024
