期刊文献+

Hadoop下资源匹配最大集作业调度算法 被引量:1

Resource matching maximum set job scheduling algorithm under Hadoop
在线阅读 下载PDF
导出
摘要 针对目前层级队列作业调度算法中资源占比高的作业执行效率低的问题,提出一种资源匹配最大集算法。该算法分析作业特征,引入完成度、等待时间、优先级、重调度次数为紧迫值因子,优先考虑资源占比高或等待时间长的作业,以改善作业公平性;采用双队列结构在可用资源总量内优先选择高紧迫值作业,在不同资源占比作业集比较中选择作业数最大集,以实现调度平衡。在与最大最小公平(Max-min fairness)算法的实例对比中发现,该算法可降低作业集平均等待时间、提高资源利用率。实验对比结果表明,该算法可将不同资源占比的单一类型作业集执行时间缩短18.73%,其中资源占比高的作业执行时间缩短27.26%;在混合型作业集中对应的执行时间可分别缩短22.36%与30.28%。所提算法能有效减少资源占比高作业的等待,提高作业整体执行效率。 Concerning the problem that jobs of high proportion of resources execute inefficiently in job scheduling algorithms of the present hierarchical queues structure, the resource matching maximum set algorithm was proposed. The proposed algorithm analysed job characteristics, introduced the percentage of completion, waiting time, priority and rescheduling times as urgent value factors. Jobs with high proportion of resources or long waiting time were preferentially considered to improve jobs fairness. Under the condition of limited amount of available resources, the double queues was applied to preferentially select jobs with high urgent values, select the maximum job set from job sets with different proportion of resources in order to achieve scheduling balance. Compared with the Max-min fairness algorithm, it is shown that the proposed algorithm can decrease average waiting time and improve resource utilization. The experimental results show that by using the proposed algorithm, the running time of the same type job set which consisted of jobs of different proportion of resources is reduced by 18. 73%, and the running time of jobs of high proportion of resources is reduced by 27. 26%; the corresponding percentages of reduction of the running time of the mixed-type job set are 22. 36% and 30. 28%. The results indicate that the proposed algorithm can effectively reduce the waiting time of jobs of high proportion of resources and improve the overall jobs execution efficiency.
出处 《计算机应用》 CSCD 北大核心 2015年第12期3383-3386,3446,共5页 journal of Computer Applications
基金 国家自然科学基金资助项目(61202136) 江苏省科技项目(BY2013095-3-11) 江苏省高校自然科学研究项目(13KJD520007) 南京晓庄学院科研项目(2012NXY14,2013NXY99)
关键词 HADOOP 层级队列 作业调度 最大集 最大最小公平算法 Hadoop hierarchical queue job scheduling maximum set Max-min fairness
  • 相关文献

参考文献4

二级参考文献47

  • 1刘芳,杨海潮.参数可调的克隆多播路由算法[J].软件学报,2005,16(1):145-150. 被引量:16
  • 2GONG Maoguo,DU Haifeng,JIAO Licheng.Optimal approximation of linear systems by artificial immune response[J].Science in China(Series F),2006,49(1):63-79. 被引量:21
  • 3TILAK S, ABU-GHAZALEH N B, HEINZELMAN W. A taxonomy of tireless micro-sensor network models [ J]. Mobile Computing andCommunications Review, 2002, 6(2): 28-36.
  • 4LI QING, ZHU QINGXIN, WANG MINGWEN. Design of a distrib- uted energy efficient clustering algorithm for heterogeneous wireless sensor networks[ J]. Computer Communications, 2006, 29 (12) : 2230- 2237.
  • 5PARUL S, A JAY K S. Energy etlicient scheme for clustering proweol prolonging the lifetime of heterogeneous wireless sensor networks[J]. Intematlona] Journal of Computer Applications, 2010, 6(2) : 30 -36.
  • 6HEINZELMAN W B, CHANDRAKAXAN A P, BALAKRISHNAN H. An application specific protocol architecture for wireless mi- crosensor networks[ J]. IEEE Transactions on Wireless Communica- tions, 2002, 1(4) :660 -670.
  • 7PARUL S, AJAY K S. Energy efficient scheme for clustering proto- col prolonging the lifetime of heterogeneous wireless sensor networks[ J]. International Journal of Computer Applications, 2010, 6(2) : 30 - 36.
  • 8A1-KARAKI J N, KAMAL A E. Routing techniques in wireles sen- sor network: a survey [ J]. IEEE Wireless Communications, 2004, 11 (6) : 6 -28.
  • 9HEINZELMAN W R, CHANDRAKASAN A, BALAKRISHNAN H. Energy-eflqcient communication protocol for wireless microsensor net- work[ C]// Proceedings of the 33rd Annual Hawaii International Conference on System Sciences. Washington, DC: IEEE Computer Society, 2000, 2:1 - 10.
  • 10SMARAGDAKIS G, MATrA I, BESTAVROS A. SEP: A stable e- lection protocol for clustered heterogeneous ,~reless sensor networks [ C] // Proceedings of 2nd International Workshop on Sensor and Ac- tor Network Protocol and Applications. Washington, DC: IEEE Com- puter Society, 2004:1 - 11.

共引文献44

同被引文献5

引证文献1

二级引证文献2

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部