摘要
为了应对智能反射面(RIS)辅助的无人机(UAV)在物联网数据收集过程中能量高效利用与信息收集时效性之间的均衡问题,该文提出一种基于深度强化学习的数据收集优化策略。针对无人机在数据采集过程中的飞行能耗、通信复杂性及采集信息时效性(AoI)约束,设计了一种基于双深度Q网络(DDQN)的联合优化方案,涵盖无人机轨迹规划、物联网设备调度以及智能反射面相位调整。该方案有效缓解了传统Q学习方法中Q值过估计的问题,使无人机能够根据实时环境动态调整飞行轨迹和通信策略,从而在提升数据传输效率的同时降低能量消耗。仿真结果表明,与传统方法相比,所提方案能够显著提高数据收集效率。此外,通过合理分配能量与通信资源,所提方案能够动态适应不同通信环境参数变化,确保系统在能耗与AoI之间达到最佳均衡。
Objective This study aims to develop and implement an optimization framework that addresses the critical balance between energy consumption and information freshness in Unmanned Aerial Vehicle(UAV)-assisted Internet of Things(IoT)data collection systems,enhanced by Reconfigurable Intelligent Surfaces(RIS).In complex urban environments,traditional line-of-sight communication between UAVs and ground-based IoT devices is often obstructed by buildings and infrastructure,hindering comprehensive coverage and efficient data collection.While RIS technology offers promising solutions by dynamically adjusting signal reflection directions,optimizing communication signal coverage,and enhancing quality,it introduces additional complexity in system design and resource allocation,requiring sophisticated adaptive optimization techniques.The integration of RIS enables stable communication connections across various UAV flight heights and angles,mitigating disruptions caused by obstacles or signal interference,thus improving data collection efficiency and reliability.However,this integration must account for multiple factors,including UAV energy consumption,communication complexity,and Age of Information(AoI)constraints.These approaches must adapt to the dynamic nature of UAV operations and fluctuating communication conditions,ensuring optimal performance in terms of energy efficiency and data freshness.The research also addresses several key challenges,including real-time adaptation to environmental changes,optimal scheduling of IoT device interactions,dynamic adjustment of RIS phase configurations,efficient trajectory planning,and the maintenance of data freshness under various system constraints.The proposed framework establishes a robust foundation for next-generation IoT data collection systems that can adapt to diverse operational conditions while maintaining high performance standards.This is achieved through the implementation of advanced deep reinforcement learning techniques,specifically designed to manage the complex interplay between UAV mobility,RIS configuration,and IoT device scheduling,ensuring efficient and timely data collection while optimizing system resources.Methods A comprehensive data collection optimization strategy is proposed,based on deep reinforcement learning principles,specifically designed to address the complex challenges in UAV-assisted IoT data collection systems enhanced by RIS technology.The methodology employs a Double Deep Q-Network(DDQN)architecture,integrating UAV trajectory planning,IoT device scheduling,and RIS phase adjustment within a three-dimensional grid-based movement space.The system incorporates a channel model that accounts for both direct and RIS-assisted communication paths,including a probabilistic path loss model for direct links and Rician fading for RIS-assisted links.The optimization problem is formulated as a Markov Decision Process(MDP),where the state space includes the UAV position,previous movement information,and average AoI,while the action space involves 3D movement decisions and IoT device scheduling.The reward function is designed to balance multiple performance metrics,including system AoI,UAV flight energy consumption,data collection energy,data upload energy,and penalties for boundary violations.The DDQN implementation utilizes two Q-networks—the current and target networks—separating action selection from action evaluation,effectively addressing the issue of Q-value overestimation.The training process incorporates experience replay for sample storage and periodic updates to the target network to enhance learning stability.Additionally,the RIS phase shift optimization is derived through geometric relationships,considering both direct and RIS-assisted communication paths.This comprehensive approach enables the joint optimization of UAV trajectory,IoT device scheduling,and RIS phase adjustment,while ensuring energy efficiency and timely data collection in complex communication environments.Results and Discussions The proposed method enables the UAV to dynamically adjust its flight trajectory and communication strategy based on real-time environmental conditions,enhancing data transmission efficiency while reducing energy consumption.Extensive simulation experiments comprehensively evaluate the performance of the DDQN-based optimization framework.Convergence analysis demonstrates that the method achieves faster and more stable convergence compared to traditional DQN approaches.The average reward steadily increases and stabilizes after approximately 200 episodes,while baseline methods exhibit slower convergence and higher performance variance(Fig.3).The optimized UAV trajectory visualization shows that the method effectively guides the UAV to collect data efficiently from all IoT devices while avoiding unnecessary detours.The trajectory strikes a balance between visiting high-priority devices(those with higher AoI)and maintaining energy-efficient flight paths,clearly illustrating the effectiveness of the joint optimization of movement and device scheduling decisions(Fig.4).Energy consumption analysis reveals that the proposed method achieves superior energy efficiency,with a 15%reduction in total energy consumption while maintaining comparable data collection performance.This improvement results from the intelligent integration of RIS-assisted communication and optimal trajectory planning,which reduces the need for energy-intensive maneuvers and prolonged hovering periods(Fig.5)(Fig.6).The AoI performance evaluation further confirms the method’s effectiveness in maintaining data freshness.The average AoI across all IoT devices remains consistently lower than in baseline methods,with a 20%improvement in worst-case AoI values.This demonstrates the method’s ability to balance the trade-off between visiting different devices and maintaining acceptable AoI levels,even under challenging network conditions.The framework’s adaptive nature is evident in its capacity to prioritize devices with critical AoI values while maintaining overall system efficiency,showing robust performance across varying network densities and device distributions(Fig.5)(Fig.6).Conclusions The proposed deep reinforcement learning-based optimization policy effectively addresses the complex challenges in UAV-assisted IoT data collection systems enhanced by RIS technology,demonstrating significant improvements in both energy efficiency and information freshness.The integration of advanced learning techniques with RIS-assisted communication provides a robust and adaptive solution for practical deployment in urban IoT environments.The comprehensive evaluation framework and detailed performance analysis offer valuable insights for system designers and practitioners.The superior performance in terms of convergence speed,trajectory optimization,energy efficiency,and AoI management confirms the effectiveness of the proposed approach.Future research will focus on extending the framework to multi-UAV coordination scenarios,exploring the impact of dynamic environmental changes,and developing more sophisticated reward mechanisms to address additional operational constraints,such as security and airspace restrictions.The promising results also indicate potential applications in emergency response systems,smart city infrastructure,and environmental monitoring networks.
作者
张涛
张迁
朱颖雯
代陈
ZHANG Tao;ZHANG Qian;ZHU Yingwen;DAI Chen(School of Information Technology,Jiangsu Open University,Nanjing 210000,China;College of Computer Science and Technology,Nanjing University of Aeronautics and Astronautics,Nanjing 210016,China;School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing 210023,China)
出处
《电子与信息学报》
北大核心
2025年第2期427-438,共12页
Journal of Electronics & Information Technology
基金
国家自然科学基金(62402232)
江苏省高等学校自然科学研究项目(23KJB520024)。
关键词
无人机辅助通信
时效性
深度强化学习
智能反射面
UAV-assisted communication
Age of Information(AoI)
Deep reinforcement learning
Reconfigurable Intelligent Surfaces(RIS)
