摘要
随着地面用户对数据需求的大幅增加,以及卫星多波束间资源分配的需求,高效稳定的动态资源分配技术成为制约当前卫星通信行业的关键。与传统的算法相比,基于人工智能的动态算法将多维资源分配和复杂时空限制条件引入多波束卫星通信中,通过建立的多波束卫星通信模型,并基于确定性策略,提出了一种双延迟-确定策略梯度(TD3)算法。该算法构建了价值网络用于评估策略,构建了策略网络用于对动作策略进行更新,同时还采用了延迟更新和添加噪声平滑目标策略。仿真实验表明,该算法与其他强化学习算法和传统算法相比吞吐量提高了5.2%,此外,通过设置隐藏层的神经元数量,给出了神经网络最稳定的隐藏层参数。
With the increasing demand of ground users for data and the demand for resource allocation between satellite multi beams,efficient and stable dynamic resource allocation technology has become the key to restrict the current satellite communica⁃tion industry.Compared with traditional algorithms,the dynamic algorithm based on artificial intelligence introduces multi-dimen⁃sional resource allocation and complex space-time constraints into multi beam satellite communication.Through the established multi beam satellite communication model,a twin delayed deep deterministic policy gradient(TD3)algorithm based on determinis⁃tic policy is proposed.In this algorithm,a value network is constructed to evaluate strategies,a strategy network is constructed to up⁃date action strategies,and a delay update strategy and a noise smoothing target strategy are also adopted.Simulation results show that the throughput of this algorithm is improved by 5.2%compared with other reinforcement learning algorithms and traditional algo⁃rithms.In addition,by setting the number of neurons in the hidden layer,the most stable hidden layer parameters of the network model are found.
作者
兰松
李晖
徐永杰
彭号杰
LAN Song;LI Hui;XU Yongjie;PENG Haojie(College of Electronics and Information Engineering,Nanjing University of Information Science and Technology,Nanjing 210044;College of Electronics and Information Engineering,Wuxi University,Wuxi 214015)
出处
《计算机与数字工程》
2025年第1期21-25,62,共6页
Computer & Digital Engineering
基金
国家自然科学基金项目(编号:61661018)
江苏省基础研究计划青年基金项目(编号:BK20210064)资助。
关键词
功率分配
多波束卫星
深度强化学习
确定策略
power distribution
multi beam satellite
deep reinforcement learning
determine strategy
