IJSRSET calls volunteers interested to contribute towards the scientific development in the field of Science, Engineering and Technology

Home > IJSRSET1624155                                                     


A Time Sharing Scheduler with Multiple Priority Based Queues for Improving Scheduling In Hadoop

Authors(3):

Priya Chak, Jayesh Mevada, Bakul Panchal
  • Abstract
  • Authors
  • Keywords
  • References
  • Details
Today, In the Era of Big data, it is in need of high levels of scalability and efficiently processing is main issue. So there is lot of challenges to handling data like how to store, retrieve and to process data efficiently. Hadoop is a distributed software platform for processing big data on a large cluster, which implements basic mechanism of Google’s MapReduce. The MapReduce job-scheduling algorithm is one of the core technologies of Hadoop. The default job scheduler of Hadoop is FIFO, which will start the job in the order as it is submitted, and this causes the job to be started later when it is submitted later. This paper uses the Time Sharing with increased time slot algorithm to solve this problem. With this scheduler, the job which is submitted late, will get quick response and started without long delay.

Priya Chak, Jayesh Mevada, Bakul Panchal

FIFO, FAIR, Hadoop , MapReduce , Scheduling, SLS, Time Sharing.

  1. Jiayin Wang, Yi Yao, Ying Mao, Bo Sheng, Ningfang Mi, “Fair and Efficient Slot Configuration and Scheduling for Hadoop Clusters” , 978-1-4799-5063-8/14 © 2014. IEEE DOI 10.1109/CLOUD.2014.
  2. Fair scheduler [online] https://hadoop.apache.org/docs/r1.2.1/fair_scheduler.html
  3. Capacity scheduler [online] https://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
  4. Divya M, Annappa B, “Workload Characteristics and Resource Aware Hadoop Scheduler “ 978- 1-4799-8349-0/15 ©2015 IEEE.   
  5. Yi Yao, Jianzhe Tai, Bo Sheng, and Ningfang Mi,Member, IEEE, “LsPS: A Job Size-Based Scheduler for Efficient Task Assignments in Hadoop “ , 2168-7161 © 2014 IEEE.
  6. Thanga selvi, R.Aruna, “Longest Approximate Time to end Scheduling Algorithm in HADOOP Environment”  ISSN (ONLINE): 2454-9762,ISSN (PRINT): 2454-9762. 2016.
  7. Bin Ye, Xiaoshe Dong, Pengfei Zheng, Zhengdong Zhu*, Qiang Liu, Zhe Wang“A delay scheduling algorithm based on history time in heterogeneous environments “978-0-7695-5058- 9/13© 2013 IEEE.
  8. Dazhao Chang, Jio Rao, Changjun jiang and Xiaobo Zhou, “Resource and deadline-aware job scheduling in Dynamic Hadoop Clusters “ , 1530-2075/15 @ 2015 IEEE, DOI  1109/IPDPS.2015.
  9. Garima Sharma, Dr. Anita Ganpati “Performance evaluation of fair and capacity scheduling in Hadoop YARN” , 978-1-4673-7910-6/15/$31.00 ©2015 IEEE
  10. Seyed Reza Pakize, “ A Comprehensive View of Hadoop MapReduce Scheduling Algorithms “, ISSN 2308-9830.
  11. Aprigio Bezerra_†, Porfidio Hern´andez_, Antonio Espinosa_ and Juan Carlos Moure_Escola d’Enginyeria “Job Scheduling in Hadoop with Shared Input Policy and RAMDISK “,978-1- 4799-5548-0/14/$31.00 ©2014 IEEE
  12. Shen Li ∗, Shaohan Hu ∗, Shiguang Wang ∗, Lu Su †, Tarek Abdelzaher ∗, Indranil Gupta∗, Richard Pace, “WOHA: Deadline-Aware Map-Reduce Workflow Scheduling Framework over Hadoop Clusters” 1063-6927/14 $31.00 © 2014 IEEE,DOI 10.1109/ICDCS.2014.
  13. Rakesh Verma “ Survey on MapReduce and Scheduling Algorithms in Hadoop” paper ID: SUB151194. International Journal of Science and Research (IJSR),ISSN (Online): 2319-7064.
  14. A.U.Patil1, Mr T.I Bagban2, Mr.A.P.Pande, “Recent Job Scheduling Algorithms in Hadoop Cluster Environments “ , ISSN : 2278-1021, 2011.
  15. Peng Qin, Bin Dai, Benxiong Huang, and Guan Xu, “ Bandwidth-Aware Scheduling With SDN in    Hadoop:A New Trend for Big Data “ , 1932-8184 © 2015 IEEE.
  16. Deveeshree Nayak, Venkata Swamy Martha, David Threm,Srini Ramaswamy,Summer Prince and Gunter Fahrnberger, “Adaptive Scheduling in the Cloud – SLA for Hadoop Job Scheduling “ , 2015
  17. Jisha S Manjaly, Varghese S Chooralil, “TaskTracker Aware Scheduling for Hadoop MapReduce” in Third International Conference on Advances in Computing and Communications, 2013.
  18. Prajesh P Anchalia,” Improved MapReduce k-Means Clustering Algorithm with Combiner”, 978-1-4799-4923-6/14 © 2014 IEEE,DOI 10.1109/UKSim.2014.
  19. Akram Roshdi, Mahboubeh shamsi, " Review: Big data on Cloud computing" ISSN: 2319, july
  20. https://hadoop.apache.org/docs/r2.4.1/hadoop-sls/SchedulerLoadSimulator.html.

Publication Details

Published in : Volume 2 | Issue 4 | July-August - 2016
Date of Publication Print ISSN Online ISSN
2016-08-30 2395-1990 2394-4099
Page(s) Manuscript Number   Publisher
731-734 IJSRSET1624155   Technoscience Academy

Cite This Article

Priya Chak, Jayesh Mevada, Bakul Panchal, "A Time Sharing Scheduler with Multiple Priority Based Queues for Improving Scheduling In Hadoop ", International Journal of Scientific Research in Science, Engineering and Technology(IJSRSET), Print ISSN : 2395-1990, Online ISSN : 2394-4099, Volume 2, Issue 4, pp.731-734, July-August-2016.
URL : http://ijsrset.com/IJSRSET1624155.php

IJSRSET Xplore

Subscribe

Conferences

National Conference on Advances in Mechanical Engineering 2017(NCAME 2017)

National Conference on Emerging Trends in Civil Engineering 2017( NCETCE 2017)