Wu, Haishan (BAIC Foton Automotive Co., Ltd), Cao, Bin (YanShan University), Wang, Xiang-Yu (State Key Laboratory of Automotive Safety and Energy, Tsinghua), Zhang, Yahui (Yanshan University), Li, Liang (Tsinghua University)

Option-Based Hierarchical Reinforcement Learning for Energy Management of Connected HEVs

Scheduled for presentation during the Invited session "Learning-based Optimization and Control for Intelligent Electrified Vehicles" (ThB1), Thursday, October 31, 2024, 13:30−13:50, Room T1

7th IFAC Conference on Engine and Powertrain Control, Simulation and Modeling, Oct 30 - Nov 1, 2024, Dalian, China

This information is tentative and subject to change. Compiled on May 16, 2025

Abstract

The energy management strategies are much important for hybrid electric vehicles to achieve better fuel economy. And the discrete-continuous hybrid controls and the physical characteristics of HEVs pose challenges to energy management strategies based on the deep reinforcement learning. This paper proposed an option-based hierarchical reinforcement learning energy management strategy for the series-parallel connected HEVs, which completes the operational modes selection and power distribution strategies sequentially according to the physical characteristics of HEVs. By combining with the option-critic architecture and abstracting the operational modes to the options, the upper-level policy makes the operational modes keep multiple steps which is not fixed and is dependent on the termination function, and the lower-level policy implements power distribution for each step. This paper shows the fuel economy of the option-based strategy is 101.77% of the adaptive equivalent consumption minimization strategies(AECMS). And comparing the results of two strategies, the proposed strategy minimizes less fuel consumption than AECMS and its effectiveness is validated.