A Time Sharing Scheduler with Multiple Priority Based Queues for Improving Scheduling in Hadoop Cluster - Cloud Environment

Authors(3) :-Bakul Panchal, Priya Chak, Jayesh Mevada

Today, In the Era of Big data, it is in need of high levels of scalability and efficiently processing is main issue. So there is lot of challenges to handling data like how to store, retrieve and to process data efficiently. Hadoop is a distributed software platform for processing big data on a large cluster, which implements basic mechanism of Google’s MapReduce. The MapReduce job-scheduling algorithm is one of the core technologies of Hadoop. The default job scheduler of Hadoop is FIFO, which will start the job in the order as it is submitted, and this causes the job to be started later when it is submitted later. This paper uses the Time Sharing with increased time slot algorithm to solve this problem. With this scheduler, the job which is submitted late, will get quick response and started without long delay.

Authors and Affiliations

Bakul Panchal
Information Technology, L. D. College of Engineering, Ahmedabad, Gujarat, India
Priya Chak
Computer Engineering, Merchant Engineering College, Basna, Gujarat, India
Jayesh Mevada
Computer Engineering, Merchant Engineering College, Basna, Gujarat, India

Cloud Computing, FIFO, FAIR, Hadoop , MapReduce , Scheduling, SLS, Time Sharing.

  1. Jiayin Wang, Yi Yao, Ying Mao, Bo Sheng, Ningfang Mi, “Fair and Efficient Slot Configuration and Scheduling for Hadoop Clusters” , 978-1-4799-5063-8/14 © 2014. IEEE DOI 10.1109/CLOUD.2014.
  2. Fair scheduler [online] https://hadoop.apache.org/docs/r1.2.1/fair_scheduler.html
  3. Capacity scheduler [online] https://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
  4. Divya M, Annappa B, “Workload Characteristics and Resource Aware Hadoop Scheduler “ 978- 1-4799-8349-0/15 ©2015 IEEE.   
  5. Yi Yao, Jianzhe Tai, Bo Sheng, and Ningfang Mi,Member, IEEE, “LsPS: A Job Size-Based Scheduler for Efficient Task Assignments in Hadoop “ , 2168-7161 © 2014 IEEE.
  6. Thanga selvi, R.Aruna, “Longest Approximate Time to end Scheduling Algorithm in HADOOP Environment”  ISSN (ONLINE): 2454-9762,ISSN (PRINT): 2454-9762. 2016.
  7. Bin Ye, Xiaoshe Dong, Pengfei Zheng, Zhengdong Zhu*, Qiang Liu, Zhe Wang“A delay scheduling algorithm based on history time in heterogeneous environments “978-0-7695-5058- 9/13© 2013 IEEE.
  8. Dazhao Chang, Jio Rao, Changjun jiang and Xiaobo Zhou, “Resource and deadline-aware job scheduling in Dynamic Hadoop Clusters “ , 1530-2075/15 @ 2015 IEEE, DOI  1109/IPDPS.2015.
  9. Garima Sharma, Dr. Anita Ganpati “Performance evaluation of fair and capacity scheduling in Hadoop YARN” , 978-1-4673-7910-6/15/$31.00 ©2015 IEEE
  10. Seyed Reza Pakize, “ A Comprehensive View of Hadoop MapReduce Scheduling Algorithms “, ISSN 2308-9830.
  11. Aprigio Bezerra_†, Porfidio Hern´andez_, Antonio Espinosa_ and Juan Carlos Moure_Escola d’Enginyeria “Job Scheduling in Hadoop with Shared Input Policy and RAMDISK “,978-1- 4799-5548-0/14/$31.00 ©2014 IEEE
  12. Shen Li ∗, Shaohan Hu ∗, Shiguang Wang ∗, Lu Su †, Tarek Abdelzaher ∗, Indranil Gupta∗, Richard Pace, “WOHA: Deadline-Aware Map-Reduce Workflow Scheduling Framework over Hadoop Clusters” 1063-6927/14 $31.00 © 2014 IEEE,DOI 10.1109/ICDCS.2014.
  13. Rakesh Verma “ Survey on MapReduce and Scheduling Algorithms in Hadoop” paper ID: SUB151194. International Journal of Science and Research (IJSR),ISSN (Online): 2319-7064.
  14. A.U.Patil1, Mr T.I Bagban2, Mr.A.P.Pande, “Recent Job Scheduling Algorithms in Hadoop Cluster Environments “ , ISSN : 2278-1021, 2011.
  15. Peng Qin, Bin Dai, Benxiong Huang, and Guan Xu, “ Bandwidth-Aware Scheduling With SDN in    Hadoop:A New Trend for Big Data “ , 1932-8184 © 2015 IEEE.
  16. Deveeshree Nayak, Venkata Swamy Martha, David Threm,Srini Ramaswamy,Summer Prince and Gunter Fahrnberger, “Adaptive Scheduling in the Cloud – SLA for Hadoop Job Scheduling “ , 2015
  17. Jisha S Manjaly, Varghese S Chooralil, “TaskTracker Aware Scheduling for Hadoop MapReduce” in Third International Conference on Advances in Computing and Communications, 2013.
  18. Prajesh P Anchalia,” Improved MapReduce k-Means Clustering Algorithm with Combiner”, 978-1-4799-4923-6/14 © 2014 IEEE,DOI 10.1109/UKSim.2014.
  19. Akram Roshdi, Mahboubeh shamsi, " Review: Big data on Cloud computing" ISSN: 2319, july
  20. https://hadoop.apache.org/docs/r2.4.1/hadoop-sls/SchedulerLoadSimulator.html.

Publication Details

Published in : Volume 2 | Issue 4 | July-August 2016
Date of Publication : 2016-08-30
License:  This work is licensed under a Creative Commons Attribution 4.0 International License.
Page(s) : 731-734
Manuscript Number : IJSRSET1624155
Publisher : Technoscience Academy

Print ISSN : 2395-1990, Online ISSN : 2394-4099

Cite This Article :

Bakul Panchal, Priya Chak, Jayesh Mevada, " A Time Sharing Scheduler with Multiple Priority Based Queues for Improving Scheduling in Hadoop Cluster - Cloud Environment, International Journal of Scientific Research in Science, Engineering and Technology(IJSRSET), Print ISSN : 2395-1990, Online ISSN : 2394-4099, Volume 2, Issue 4, pp.731-734, July-August-2016. Citation Detection and Elimination     |     
Journal URL : https://ijsrset.com/IJSRSET1624155

Article Preview