Abstract: Aiming at the energy management problem of plug-in hybrid electric vehicles, a strategy based on the Q-learning algorithm in reinforcement learning is proposed. Take battery state of charge and required power as state variables, take auxiliary power unit output power as control variable, and use Temporal-Difference Learning algorithm to update the action-state value in real time. Comparing the results with the global optimal algorithm pontryagin minimum principle, the total price of the Q-learning strategy is only 1.57 yuan more expensive under the Chinese type city bus circle of nearly 100 kilometers, which shows the effectiveness of the Q-learning strategy. From the perspective of application, this strategy is conducive to improving the overall economic level of vehicle.

Key words: Plug-in hybrid electric vehicles; Reinforcement learning; Q-learning; Energy management strategy

摘要： 针对插电式混合动力汽车的能量管理问题，提出一种基于强化学习中 Q-learning 算法 的策略。以电池荷电状态和需求功率为状态变量，以动力辅助单元输出功率为控制变量，并 采用时序差分算法实时更新动作-状态值。将结果与全局最优算法庞特里亚金极小值原理对 比，在近百公里的中国典型城市客车工况下，Q-learning 策略的总价格仅贵出 1.57 元，表明 了基于 Q-learning 策略的有效性。从应用角度出发，该策略有利于提升车辆整体经济水平。

关键词: 插电式混合动力汽车；强化学习；Q-learning；能量管理策略