Journal of Applied Information Science

Open Access Open Access  Restricted Access Subscription Access
Open Access Open Access Open Access  Restricted Access Restricted Access Subscription Access

Phase Level Scheduler for MapReduce using Grained Resource


Affiliations
1 Seshachala Degree & P.G. College, Puttur, Andhra Pradesh, India
     

   Subscribe/Renew Journal


MapReduce is one of the important concepts of Hadoop that is used for data handling used by big companies today such as Google and Facebook. Here we divide each job into the map and reduce phases and try to complete the execution of the assigned task in a parallel form. In this paper, we suggest that it would be more efficient if we make the scheduler to work at the phase-level instead of the task-level. The reason is that the task demands a lot of requirements during its lifetime. For this very purpose, we introduce the concept called PRISM, which is a phase and information-aware scheduler for MapReduce and in this concept, we divide the tasks into unequal parts called as phases and apply phase-level scheduling to these phases and achieve efficient resource usage.

Keywords

Job Progress Monitor, MapReduce, Phase-Based Scheduler.
Subscription Login to verify subscription
User
Notifications
Font Size


  • Hadoop Map Reduce Distribution, 2015. [Online]. Available: http://hadoop.apache.org

  • Hadoop Fair Scheduler, 2015. [Online]. Available: http://hadoopapache.org/docs/r0.20.2/fair_scheduler.html

  • Hadoop Distributed File System, 2015. [Online]. Available: hadoop.apache.org/docs/hdfs/current/

  • The Next Generation of Apache Hadoop MapReduce, 2015. [Online]. Available: http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html

  • R. Boutaba, L. Cheng, and Q. Zhang, “On cloud computational models and the heterogeneity challenge,” J. Internet Serv. Appl., vol. 3, no. 1, pp. 1-10, 2012.

  • T. Condie, N. Conway, P. Alvaro, J. Hellerstein, K. Elmeleegy, and R. Sears, “MapReduce online,” In Proc. USENIX Symp. Netw. Syst. Des. Implementation, 2010, p. 21.

  • J. Dean, and S. Ghemawat, “MapReduce: Simplified data processing on large clusters,” Commun. ACM, vol. 51, no. 1, pp. 107-113, 2008.

  • A. Ghodsi, M. Zaharia, B. Hindman, A. Konwinski, S. Shenker, and I. Stoica, “Dominant resource fairness: Fair allocation of multiple resource types,” In Proc. USENIX Symp. Netw. Syst. Des. Implementation, 2011, pp. 323-336.

  • H. Herodotou, H. Lim, G. Luo, N. Borisov, L. Dong, F. Cetin, and S. Babu, “Starfish: A self-tuning system for big data analytics,” In Proc. Conf. Innovative Data Syst. Res., 2011, pp. 261-272.

  • M. Isard, V. Prabhakaran, J. Currey, U. Wieder, and K. Talwar, “Quincy: Fair scheduling for distributed computing clusters,” In Proc. ACMSIGOPS Symp. Oper. Syst. Principles, 2009, pp. 261-276.

  • C. Joe-Wong, S. Sen, T. Lan, and M. Chiang, “Multi-resource allocation: Flexible tradeoffs in a unifying framework,” In Proc. IEEE Int. Conf. Comput. Commun., 2012, pp. 1206-1214.

  • J. Polo, C. Castillo, D. Carrera, Y. Becerra, I. Whalley, M. Steinder, J. Torres, and E. Ayguade, “Resource-aware adaptive scheduling for MapReduce clusters,” In Proc. ACM/IFIP/USENIX Int. Conf. Middleware, 2011, pp. 187-207.

  • A. Rasmussen, M. Conley, R. Kapoor, V. T. Lam, G. Porter, and A. Vahdat, “ThemisMR: An I/O-Efficient MapReduce,” In Proc. ACM Symp. Cloud Compute, 2012, p. 13.

  • A. Verma, L. Cherkasova, and R. Campbell, “Resource provisioning framework for MapReduce jobs with performance goals,” In Proc. ACM/IFIP/USENIX Int. Conf. Middleware, 2011, pp. 165-186.

  • D. Xie, N. Ding, Y. Hu, and R. Kompella, “The only constant is change: Incorporating time-varying network reservations in data centers,” In Proc. ACM SIGCOMM, 2012, pp. 199-210.


Abstract Views: 460

PDF Views: 0




  • Phase Level Scheduler for MapReduce using Grained Resource

Abstract Views: 460  |  PDF Views: 0

Authors

Bandla Manjula
Seshachala Degree & P.G. College, Puttur, Andhra Pradesh, India

Abstract


MapReduce is one of the important concepts of Hadoop that is used for data handling used by big companies today such as Google and Facebook. Here we divide each job into the map and reduce phases and try to complete the execution of the assigned task in a parallel form. In this paper, we suggest that it would be more efficient if we make the scheduler to work at the phase-level instead of the task-level. The reason is that the task demands a lot of requirements during its lifetime. For this very purpose, we introduce the concept called PRISM, which is a phase and information-aware scheduler for MapReduce and in this concept, we divide the tasks into unequal parts called as phases and apply phase-level scheduling to these phases and achieve efficient resource usage.

Keywords


Job Progress Monitor, MapReduce, Phase-Based Scheduler.

References