Predictive control of wind turbine yaw system model based on reinforcement learning

桑申刚; 李桂朋; 王向伟; 刘毅; 王森; 申向荣

doi:10.19666/j.rlfd.202501006

English

您当前的位置：

首页 >

文章列表页 >

Predictive control of wind turbine yaw system model based on reinforcement learning

Thermal Power Generation Vol. 54, Issue 9, Pages: 86-94(2025)

更新时间：2026-08-11

- Predictive control of wind turbine yaw system model based on reinforcement learning
- 作者机构：
  
  1. 华能新能源股份有限公司河北分公司
  2. 华北电力大学自动化系
- 作者简介：
- 基金信息：
- DOI：10.19666/j.rlfd.202501006
  CLC： TM315;TP181;TP273
- Published：2025
- 稿件说明：
移动端阅览
[1]桑申刚,李桂朋,王向伟,等.基于强化学习的风电机组偏航系统模型预测控制[J],2025,54(09):86-94.
[1]桑申刚,李桂朋,王向伟,等.基于强化学习的风电机组偏航系统模型预测控制[J],2025,54(09):86-94. DOI： 10.19666/j.rlfd.202501006.

DOI：10.19666/j.rlfd.202501006.

摘要

提升风电机组偏航系统在多工况场景下的动态性能至关重要。基于此，提出了一种基于强化学习的风电机组偏航系统模型预测控制策略，通过双延迟深度确定性策略梯度(TD3)算法实现多目标参数动态优化。首先，针对功率损失率和偏航执行器使用率2个相互矛盾的控制目标，建立偏航系统的多步模型预测控制器(yaw model predictive control

YMPC)；其次，针对偏航系统的优化目标及风况特性设计双延迟深度确定性策略梯度(TD3)智能体的输入状态、动作和奖励机制，从而使用TD3智能体整定YMPC的权重系数和控制步长；最后，采用中国北方风电场典型日数据验证了该方法的有效性。结果表明，与固定控制参数的YMPC相比，所提出的策略显著提升了偏航系统的综合性能。

Abstract

It is crucial to improve the dynamic performance of the yaw system of wind turbines in multiple operating scenarios. Therefore

a predictive control strategy for wind turbine yaw system model based on reinforcement learning is proposed

which achieves multi-objective parameter dynamic optimization through the dual-delay depth deterministic policy gradient(TD3) algorithm. Firstly

a multi-step model predictive controller for the yaw system(YMPC) is established to address the conflicting control objectives of power loss rate and yaw actuator utilization rate. Secondly

based on the optimization objectives and wind conditions of the yaw system

a dual-delay depth deterministic strategy gradient(TD3) intelligent agent is designed to determine the input state

action

and reward mechanism of the YMPC. The TD3 intelligent agent is then used to tune the weight coefficients and control step size of the YMPC. Finally

the effectiveness of this method was validated using typical daily data from wind farms in northern China. The results indicate that the proposed strategy significantly improves the overall performance of the yaw system compared with the YMPC with fixed control parameters.

关键词

Keywords

references

肖玥,封焯文,余子鹏,等.基于风向预测的新型偏航控制方法[J].太阳能学报, 2021, 42(2):144-149.XIAO Yue, FENG Zhuowen, YU Zipeng, et al. A novel yaw control method for wind turbines based on predicted wind directions[J]. Acta Energiae Solaris Sinica, 2021,42(2):144-149.

PéREZ P M J, MáRQUEZ G P F, TOBIAS A, et al.Wind turbine reliability analysis[J]. Renewable and Sustainable Energy Reviews, 2013, 23:463-472.

沈小军,杜万里.大型风力发电机偏航系统控制策略研究现状及展望[J].电工技术学报, 2015, 30(10):196-203.SHEN Xiaojun, DU Wanli. Expectation and review of control strategy of large wind turbine yaw system[J].Transactions of China Electrotechnical Society, 2015,30(10):196-203.

曹弘,孙建平.风力发电机偏航系统控制策略研究[J].电力设备管理, 2019(2):81-83.CAO Hong, SUN Jianping. Expectation and review of control strategy of large wind turbine yaw system[J].Electric Power Equipment Management, 2019(2):81-83.

谭芝,陈众,汤敏,等.风力发电机组偏航优化控制方法[J].电力学报, 2014, 29(1):66-69.TAN Zhi, CHEN Zhong, TANG Min, et al. Wind turbine yaw optimal control method[J]. Journal of Electric Power, 2014, 29(1):66-69.

叶锋,林周泉,李宁波,等.一种基于CNN-LSTM风向预测的风电机组偏航控制方法[J].电气时代,2022(9):43-46.YE Feng, LIN Zhouquan, LI Ningbo, et al. A wind turbine yaw control method based on CNN-LSTM wind direction prediction[J]. Electric Age, 2022(9):43-46.

CHEN W, LIU H, LIN Y, et al. LSTM-NN yaw control of wind turbines based on upstream wind information[J].Energies, 2020, 13(6):1482.

RUIYANG H, HONGXING Y, LIN L. Optimal yaw strategy and fatigue analysis of wind turbines under the combined effects of wake and yaw control[J]. Applied Energy, 2023, 337:223.

SONG D, FAN X, YANG J, et al. Power extraction efficiency optimization of horizontal-axis wind turbines through optimizing control parameters of yaw control systems using an intelligent method[J]. Applied Energy,2018, 224:267-279.

ZHANG S, HUANG L, SONG D, et al. Model predictive yaw control using fuzzy-deduced weighting factor for large-scale wind turbines[J]. Energy Engineering, 2021,118(2):237-250.

SONG D, LI Z, DENG X, et al. Deep optimization of model predictive control performance for wind turbine yaw system based on intelligent fuzzy deduction[J].Expert Systems with Applications, 2023, 221:119705.

吴西涛,魏超,翟建坤,等.考虑横摆稳定性的无人车轨迹跟踪控制优化研究[J].机械工程学报, 2022,58(6):130-142.WU Xitao, WEI Chao, ZHAI Jiankun, et al. Study on the optimization of autonomous vehicle on path-following considering yaw stability[J]. Chinese Journal of Mechanical Engineering, 2022, 58(6):130-142.

高阳,陈世福,陆鑫.强化学习研究综述[J].自动化学报, 2004, 30(1):86-100.GAO Yang, CHEN Shifu, LU Xin. Research on reinforcement learning technology:a review[J]. Journal of Automatica Sinica, 2004, 30(1):86-100.

柯恺宸,金士博,高博扬,等.基于强化学习的机器人多接触交互任务控制[J].动力学与控制学报, 2023,21(12):53-69.KE Kaichen, JIN Shibo, GAO Boyang, et al. A survey of robot intelligent control method in contact-rich tasks[J].Journal of Dynamics and Control, 2023, 21(12):53-69.

陈帅.基于强化学习的微电网能量管理与调度[D].北京:北京科技大学, 2024:1.CHEN Shuai. Microgrid energy management and dispatching based on reinforcement learning[D]. Beijing:University of Science and Technology Beijing, 2024:1.

王宇.基于强化学习的风电机组转矩控制优化策略研究[D].沈阳:沈阳工业大学, 2024:1.WANG Yu. Research on optimization strategy of wind turbine torque control based on reinforcement learning[D]. Shenyang:Shenyang University of Technology, 2024:1.

张勇,刘春,褚梦珂,等.计及尾流的改进深度确定性策略梯度风电场功率优化控制策略[J].电力系统及其自动化学报, 2025, 37(2):68-77.ZHANG Yong, LIU Chun, CHU Mengke, et al.Improved deep deterministic policy gradient for wind farm power optimal control strategies accounting for wake effect[J]. Proceedings of the CSU-EPSA, 2025,37(2):68-77.

NABEEL A, LASHEEN A, ELSHAFEI L A, et al.Fuzzy-based collective pitch control for wind turbine via deep reinforcement learning[J]. ISA Transactions, 2024,148:307-325.

黄益绍,王博,李晨艳,等.自适应MPC智能车轨迹跟踪控制[J].中国测试, 2025, 51(2):169-175.HUANG Yishao, WANG Bo, LI Chenyan, et al.Intelligent vehicle trajectory tracking control based on adaptive MPC[J]. China Measurement&Test, 2025,51(2):169-175.

胡兵,海玲,程丽娟,等.小型风力发电机偏航控制系统实验研究[J].微电机, 2020, 53(6):64-67.HU Bing, HAI Ling, CHENG Lijuan, et al. Experimental study on yaw control system of small wind turbine[J].Micromotor, 2020, 53(6):64-67.

SONG D, LI Z, WANG L, et al. Energy capture efficiency enhancement of wind turbines via stochastic model predictive yaw control based on intelligent scenarios generation[J]. Applied Energy, 2022, 312:118773.

SONG D R, LI Q A, CAI Z L, et al. Model predictive control using multi-step prediction model for electrical yaw system of horizontal-axis wind turbines[J]. IEEE Transactions on Sustainable Energy, 2019, 10(4):2084-2093.

顾雪平,刘彤,李少岩,等.基于改进双延迟深度确定性策略梯度算法的电网有功安全校正控制[J].电工技术学报, 2023, 38(8):2162-2177.GU Xueping, LIU Tong, LI Shaoyan, et al. Active power correction control of power grid based on improved twin delayed deep deterministic policy gradient algorithm[J].Transactions of China Electrotechnical Society, 2023,38(8):2162-2177.

李志军,徐博,张家安,等.基于TD3可变长度时间窗口最优加权的短期负荷预测策略[J].电力建设, 2024,45(6):140-148.LI Zhijun, XU Bo, ZHANG Jiaan, et al. Short-term load optimal weighted forecasting strategy based on TD3variable length time window[J]. Electric Power Construction, 2024, 45(6):140-148.

管超骏,雷正玲,霍海波,等.基于强化学习的固体氧化物燃料电池输出电压自抗扰控制研究[J].发电技术,2024, 45(6):1163-1172.GUAN Chaojun, LEI Zhengling, HUO Haibo, et al.Study on active disturbance rejection control of output voltage of solid oxide fuel cell based on reinforcement learning[J]. Power Generation Technology, 2024, 45(6):1163-1172.

陈佐明,詹捷宇.基于TD3算法的自动协商策略[J].计算机系统应用, 2023, 32(3):15-24.CHEN Zuoming, ZHAN Jieyu. Automatic negotiation strategy based on TD3 algorithm[J]. Computer Systems&Applications, 2023, 32(3):15-24.

周娴玮,包明豪,叶鑫,等.带Q网络过滤的两阶段TD3深度强化学习方法[J].计算机技术与发展, 2023,33(10):101-108.ZHOU Xianwei, BAO Minghao, YE Xin, et al.Two-stage TD3 deep reinforcement learning algorithm with Q network filtration[J]. Computer Technology and Development, 2023, 33(10):101-108.

喻磊,谈竹奎,王扬,等.基于深度强化学习的电动汽车有序充电优化方法[J].南方电网技术, 2024, 18(12):148-155.YU Lei, TAN Zhukui, WANG Yang, et al. Deep reinforcement learning based optimization method for ordered charging of electric vehicle[J]. Southern Power System Technology, 2024, 18(12):148-155.

Views

534

下载量

CSCD

Alert me when the article has been cited

提交

Publicity Resources

Pipe-blowing technology with combination of pressure stabilization and pressure reduction for 1000 MW unit dual tangential boiler

Application of online blending and combustion optimization system based on real-time coal quality detection

Study on slurry ORP parameters model and oxidation control in wet desulphurization systems

Error-based active disturbance rejection control and optimization of coordinated control system for ultra-supercritical unit

基于LHS-MC长时段随机场景的多类型电解槽制氢系统容量配置优化

Related Author

YANG Yi

CHEN Xun

LI Xiongfei

WANG An

HE Honghao

YANG Jianfeng

LYU Dangzhen

JIANG Sennian

Related Institution

Hunan Key Laboratory of Efficient and Clean Thermal Power Generation Technology State Grid Hunan Electric Power Co., Ltd. Electric Power Science Research Institute

Hunan Xiangdian Experimental Research Institute Co., Ltd

Xi'an Thermal Power Research Institute Co., Ltd

Huaneng Yushe Power Generation Co., Ltd

Huadian Xiangyang Power Generation Co., Ltd

⁰