Webex Meeting information:
Webex Meeting Link: Click here at 10 AM to join the Guest Speaker seminar
Meeting number (access code): 798 229 676
Meeting password: QmRJmjwD543
Speaker: Thinh T. Doan, Georgia Institute of Technology
Title: Reinforcement Learning and Distributed Learning for Data-Driven Decision Making
Abstract: The rapid development of low-cost sensors, smart devices, communication networks, and learning algorithms has enabled data-driven decision making in large-scale multi-agent systems. Prominent examples include robotic networks and autonomous systems. The key challenge in these systems is in handling the vast quantities of information shared between the agents to find an optimal policy that maximizes the agents’ performance. Among potential approaches, reinforcement learning and distributed learning, which are not only amenable to low-cost implementation but can also be implemented in real time, have been recognized as important methods to address this challenge. Reinforcement learning and distributed learning share a common mathematical core: stochastic approximation. The focus of this talk is to discuss recent advances in these two areas from the perspective of stochastic approximation. First, I will discuss the policy evaluation problem, one of the most fundamental problems in multi-agent reinforcement learning. In this problem, a set of agents operate in an environment under a fixed control policy, working together to discover their cumulative reward associated with each environmental state. For solving this problem, I propose a distributed variant of the popular temporal difference learning algorithm. My main contribution is to provide the first finite-time bounds for the performance of this algorithm that depends on the connectivity of the agents and the intrinsic properties of the Markov process driving the agents’ decisions. Next, I will discuss my other contribution in improving the sample complexity of reinforcement learning algorithms. In particular, I will talk about my work in studying accelerated reinforcement learning algorithms using momentum techniques. I show that these techniques can significantly improve the performance of the classic REINFORCE algorithm. In addition, I provide a number of numerical simulations in a number of complicated control problems to support for my theoretical results. Finally, I conclude my talk with some discussion about my research vision in the context of data-driven decision making in multi-agent systems.
Bio: Thinh T. Doan is a TRIAD postdoc fellow at the Georgia Institute of Technology, jointly between the School of Electrical and Computer Engineering (ECE) and the School of Industrial and Systems Engineering. He was born in Vietnam, where he got his Bachelor’s degree in Automatic Control at Hanoi University of Science and Technology in 2008. He obtained his Master’s and Ph.D. degrees both in ECE from the University of Oklahoma in 2013 and the University of Illinois at Urbana-Champaign (UIUC) in 2018, respectively. At Illinois, he was the recipient of the Harriett & Robert Perry Fellowship in ECE in 2016 and 2017. His research interests lie at the intersection of control theory, optimization, distributed algorithms, and applied probability, with the main applications in machine learning, reinforcement learning, and multi-agent systems.