Causal Reinforcement Learning: Empowering Agents with Causality
Date:
Reinforcement learning has made significant progress in solving sequential decision problems under uncertainty. However, reinforcement learning agents generally lack a fundamental understanding of the world and must therefore learn from scratch through numeroustrial-anderror interactions. They may also face challenges in providing explanations for their decisions and generalizing the acquired knowledge. Causality presents a promising approach to address these issues by formalizing knowledge in a systematic manner and leveraging invariance for effective knowledge transfer. This tutorial aims to comprehensively review the emerging field of causal reinforcement learning. We will introduce the basic concepts of causality and reinforcement learning and demonstrate how causality can enhance traditional reinforcement learning algorithms. The tutorial will categorize and systematically review existing causal reinforcement learning approaches based on their target problems and methodologies. We will also outline open issues and future research directions to foster the continuous development and application of causal reinforcement learning in real-world scenarios. We believe that this tutorial will contribute significantly to the data mining community, offering a unique perspective on integrating causality into reinforcement learning and providing participants with valuable knowledge to explore this emerging field.