摘要
针对PolSARpro软件原有高分三号数据转换模块无法适应分布式环境及数据转换效率相对较低的问题,提出一种基于MPI、MapReduce和OpenMP并按照该软件的数据格式要求进行分布式负载均衡的并行转换算法。该算法外层采用MPI按任务分布式计算,中间层采用MapReduce按景并行处理,内层采用OpenMP按极化方式并行计算,并采用“Z”字形数据负载均衡策略,将高分三号不同成像模式下的多极化数据快速精确定标及格式转换,以抗数据偏斜。实际数据的测试结果表明,该算法的整体性能提升了约50%,验证了算法的可行性、高效性和正确性。
To address the inadaptability of the original GF-3 conversion module of the PolSARpro to distributed environments and its low data conversion efficiency,this paper proposes a distributed load balancing parallel conversion algorithm for handling data skew according to the data format requirements of its software based on MPI,MapReduce and OpenMP.Its external layer is based on MPI by task,intermediate layer based on MapReduce by scene,internal layer based on OpenMP by polarization,and load balancing strategy based on the"Z"letter by data size,which can quickly and accurately calibrate and convert GF-3 multi-polarization data into bin format.Tested in real data,the overall performance is improved by about 50%,which verifies the feasibility,efficiency and correctness of the algorithm.
作者
邱祥峰
QIU Xiangfeng(Xiamen KINGTOP Information Technology Co.,Ltd.,Xiamen 361008,China)
出处
《厦门理工学院学报》
2024年第5期33-39,共7页
Journal of Xiamen University of Technology
基金
厦门市科技计划项目“遥感影像智能解译关键技术研究”(3502Z20231038)。
