Dual Ensemble Kalman Filter for Stochastic Optimal Control

Publication Links

• DOI    • arXiV    • Abstract    • Citation

Abstract

In this paper, stochastic optimal control problems in continuous time and space are considered. In recent years, such problems have received renewed attention from the lens of reinforcement learning (RL) which is also one of our motivation. The main contribution is a simulation-based algorithm – dual ensemble Kalman filter (EnKF) – to numerically approximate the solution of these problems. The paper extends our previous work where the dual EnKF was applied in deterministic settings of the problem. The theoretical results and algorithms are illustrated with numerical experiments.

Citation

@INPROCEEDINGS{10886131,
  author={Joshi, Anant A. and Taghvaei, Amirhossein and Mehta, Prashant G. and Meyn, Sean P.},
  booktitle={2024 IEEE 63rd Conference on Decision and Control (CDC)}, 
  title={Dual Ensemble Kalman Filter for Stochastic Optimal Control}, 
  year={2024},
  volume={},
  number={},
  pages={1917-1922},
  keywords={Optimal control;Reinforcement learning;Aerospace electronics;Approximation algorithms;Kalman filters;Lenses},
  abstract = {In this paper, stochastic optimal control problems in continuous time 
and space are considered. In recent years, such problems have received 
renewed attention from the lens of reinforcement learning (RL) which is 
also one of our motivation. The main contribution is a simulation-based 
algorithm – dual ensemble Kalman filter (EnKF) – to numerically 
approximate the solution of these problems. The paper extends our 
previous work where the dual EnKF was applied in deterministic settings 
of the problem. The theoretical results and algorithms are illustrated 
with numerical experiments.}
  doi={10.1109/CDC56724.2024.10886131}
}