Today, In the Era of Big data, it is in need of high levels of scalability and efficiently processing is main issue. So there is lot of challenges to handling data like how to store, retrieve and to process data efficiently. Hadoop is a distributed software platform for processing big data on a large cluster, which implements basic mechanism of Google’s MapReduce. The MapReduce job-scheduling algorithm is one of the core technologies of Hadoop. The default job scheduler of Hadoop is FIFO, which will start the job in the order as it is submitted, and this causes the job to be started later when it is submitted later. This paper uses the Time Sharing with increased time slot algorithm to solve this problem. With this scheduler, the job which is submitted late, will get quick response and started without long delay.
Priya Chak, Jayesh Mevada, Bakul Panchal
FIFO, FAIR, Hadoop , MapReduce , Scheduling, SLS, Time Sharing.
- Jiayin Wang, Yi Yao, Ying Mao, Bo Sheng, Ningfang Mi, “Fair and Efficient Slot Configuration and Scheduling for Hadoop Clusters” , 978-1-4799-5063-8/14 © 2014. IEEE DOI 10.1109/CLOUD.2014.
- Fair scheduler [online] https://hadoop.apache.org/docs/r1.2.1/fair_scheduler.html
- Capacity scheduler [online] https://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
- Divya M, Annappa B, “Workload Characteristics and Resource Aware Hadoop Scheduler “ 978- 1-4799-8349-0/15 ©2015 IEEE.
- Yi Yao, Jianzhe Tai, Bo Sheng, and Ningfang Mi,Member, IEEE, “LsPS: A Job Size-Based Scheduler for Efficient Task Assignments in Hadoop “ , 2168-7161 © 2014 IEEE.
- Thanga selvi, R.Aruna, “Longest Approximate Time to end Scheduling Algorithm in HADOOP Environment” ISSN (ONLINE): 2454-9762,ISSN (PRINT): 2454-9762. 2016.
- Bin Ye, Xiaoshe Dong, Pengfei Zheng, Zhengdong Zhu*, Qiang Liu, Zhe Wang“A delay scheduling algorithm based on history time in heterogeneous environments “978-0-7695-5058- 9/13© 2013 IEEE.
- Dazhao Chang, Jio Rao, Changjun jiang and Xiaobo Zhou, “Resource and deadline-aware job scheduling in Dynamic Hadoop Clusters “ , 1530-2075/15 @ 2015 IEEE, DOI 1109/IPDPS.2015.
- Garima Sharma, Dr. Anita Ganpati “Performance evaluation of fair and capacity scheduling in Hadoop YARN” , 978-1-4673-7910-6/15/$31.00 ©2015 IEEE
- Seyed Reza Pakize, “ A Comprehensive View of Hadoop MapReduce Scheduling Algorithms “, ISSN 2308-9830.
- Aprigio Bezerra_†, Porfidio Hern´andez_, Antonio Espinosa_ and Juan Carlos Moure_Escola d’Enginyeria “Job Scheduling in Hadoop with Shared Input Policy and RAMDISK “,978-1- 4799-5548-0/14/$31.00 ©2014 IEEE
- Shen Li ∗, Shaohan Hu ∗, Shiguang Wang ∗, Lu Su †, Tarek Abdelzaher ∗, Indranil Gupta∗, Richard Pace, “WOHA: Deadline-Aware Map-Reduce Workflow Scheduling Framework over Hadoop Clusters” 1063-6927/14 $31.00 © 2014 IEEE,DOI 10.1109/ICDCS.2014.
- Rakesh Verma “ Survey on MapReduce and Scheduling Algorithms in Hadoop” paper ID: SUB151194. International Journal of Science and Research (IJSR),ISSN (Online): 2319-7064.
- A.U.Patil1, Mr T.I Bagban2, Mr.A.P.Pande, “Recent Job Scheduling Algorithms in Hadoop Cluster Environments “ , ISSN : 2278-1021, 2011.
- Peng Qin, Bin Dai, Benxiong Huang, and Guan Xu, “ Bandwidth-Aware Scheduling With SDN in Hadoop:A New Trend for Big Data “ , 1932-8184 © 2015 IEEE.
- Deveeshree Nayak, Venkata Swamy Martha, David Threm,Srini Ramaswamy,Summer Prince and Gunter Fahrnberger, “Adaptive Scheduling in the Cloud – SLA for Hadoop Job Scheduling “ , 2015
- Jisha S Manjaly, Varghese S Chooralil, “TaskTracker Aware Scheduling for Hadoop MapReduce” in Third International Conference on Advances in Computing and Communications, 2013.
- Prajesh P Anchalia,” Improved MapReduce k-Means Clustering Algorithm with Combiner”, 978-1-4799-4923-6/14 © 2014 IEEE,DOI 10.1109/UKSim.2014.
- Akram Roshdi, Mahboubeh shamsi, " Review: Big data on Cloud computing" ISSN: 2319, july
|Published in :
||Volume 2 | Issue 4 | July-August - 2016
|Date of Publication
Cite This Article
Priya Chak, Jayesh Mevada, Bakul Panchal, "A Time Sharing Scheduler with Multiple Priority Based Queues for Improving Scheduling In Hadoop ", International Journal of Scientific Research in Science, Engineering and Technology(IJSRSET), Print ISSN : 2395-1990, Online ISSN : 2394-4099, Volume 2, Issue 4, pp.731-734, July-August-2016.
URL : http://ijsrset.com/IJSRSET1624155.php