Hailemichael, Habtamu (Clemson University), Ayalew, Beshah (Cemson University), Kerbel, Lindsey (Clemson University), Ivanco, Andrej (Allison Transmission), Loiselle, Keith (Allison Transmission, Inc)

Safety Filtering for Reinforcement Learning-Based Adaptive Cruise Control

Scheduled for presentation during the Invited Session "MODELLING AND CONTROL METHODS FOR ADVANCED VEHICLE CONTROL" (MoBT5), Monday, August 29, 2022, 15:30−15:50, Pfhal Hall 202

10th IFAC International Symposium on Advances in Automotive Control, August 28-31, 2022, Columbus, Ohio, USA

This information is tentative and subject to change. Compiled on April 25, 2024

Abstract

Reinforcement learning (RL)-based adaptive cruise control systems (ACC) that learn and adapt to road, traffic and vehicle conditions are attractive for enhancing vehicle energy efficiency and traffic flow. However, the application of RL in safety critical systems such as ACC requires strong safety guarantees which are difficult to achieve with learning agents that have a fundamental need to explore. In this paper, we derive control barrier functions as safety filters that allow an RL-based ACC controller to explore freely within a collision safe set. Specifically, we derive control barrier functions for high relative degree nonlinear systems to take into account inertia effects relevant for commercial vehicles. We also outline an algorithm for accommodating actuation saturation with these barrier functions. While any RL algorithm can be used as the performance ACC controller together with these filters, we implement the Maximum A Posteriori Policy Optimization (MPO) algorithm with a hybrid action space that learns fuel optimal gear selection and torque control policies. The safety filtering RL approach is contrasted with a reward shaping RL approach that only learns to avoid collisions after sufficient training. Evaluations on different drive cycles demonstrate significant improvements in fuel economy with the proposed approach compared to baseline ACC algorithms.