| Zeit: | 27. Juni 2023 |
|---|---|
| Download als iCal: |
|
Dr. Romain Postoyan
The Research Center for Automatic Control (CRAN)
University of Lorraine
Nancy, France
Tuesday 2023-06-27 4 p.m.
IST Seminar Room 2.255 - Pfaffenwaldring 9 - Campus Stuttgart-Vaihingen
Abstract
Dynamic programming (DP) plays a major role in various fields including reinforcement learning, operations research, computer sciences and of course control engineering. DP allows to solve general optimal control problems in terms of dynamical systems and cost functions. In this talk, we will focus on plants described by deterministic nonlinear discrete-time systems whose inputs are generated by a DP algorithm. Our goal is to establish sufficient conditions under which robust stability properties can be established for the closed-loop system. Interestingly, we will see that stability can then be exploited to analyse the near-optimality properties of the algorithm used to generate the control inputs, thereby improving existing near-optimality bounds. We will mainly focus on policy iteration (PI) and we will see that PI actually needs to be modified to be well-posed, in the sense that the optimization problems solved at each iteration are guaranteed to admit a solution. Afterwards, we will briefly see how similar results can be derived for other algorithms (value iteration, optimistic planning). We believe that these results would benefit the burgeoning literature on reinforcement learning, where the algorithms are typically assumed to be well-posed without a clear method for verifying it and where robust stability is essential for safe operation of the system.
Biographical Information
Romain Postoyan received the ``Ingénieur'' degree in Electrical and Control Engineering from ENSEEIHT (France) in 2005. He obtained the M.Sc. by Research in Control Theory & Application from Coventry University (United Kingdom) in 2006 and the Ph.D. in Control Theory from Université Paris-Sud (France) in 2009. In 2010, he was a research assistant at the University of Melbourne (Australia). Since 2011, he is a CNRS researcher at CRAN (France). He serves/served as an associate editor for the journals: IEEE Transactions on Automatic Control, Automatica, IEEE Control Systems Letters and IMA Journal of Mathematical Control and Information. His fields of interest include: Lyapunov stability, networked control systems, event-triggered control, hybrid systems, dynamic programming, nonlinear estimation, lithium-ion batteries.