Abstract: Reinforcement learning (RL) has good application prospects in online optimization of controllers. However, in practical applications, there are serious security risks. To solve this safety hazard, an incremental Q-learning (IQ) algorithm is proposed and applied to online optimization of motor speed synchronization control. IQ divides a round of optimization process in classical Q-learning into multiple continuous optimization processes. Due to the very small allowable change interval limit in each round of optimization, the intelligent agent can safely and stably achieve global optimization. The simulation results show that IQ effectively avoids performance degradation and falls into local optima under strict stop criteria, and is superior to absolute Q-learning (AQ) in terms of optimality and safety.

Key words: Reinforcement learning; Incremental Q-learning; Synchronous control; Online optimization

摘要： 强化学习（RL）在控制器的在线优化中具有很好的应用前景。然而，在实际应用中， 却存在严重的安全隐患。为解决这一安全隐患，提出了一种增量 Q 学习（IQ）算法，将其应 用于电机转速同步控制的在线优化。IQ 将经典 Q 学习中的一轮优化过程划分为多个连续地优 化过程。由于在每轮优化中，将允许的更改间隔限制得非常小，因此智能体能够安全、稳定 地达到全局最优。仿真结果表明，IQ 有效地避免了性能衰退和在严苛的停止准则下陷入局部 最优的问题，在最优性、安全性方面优于绝对 Q 学习（AQ）。

关键词: 强化学习；增量 Q 学习；同步控制；在线优化