International Journal of Advanced Networking and Applications

Open Access Open Access  Restricted Access Subscription Access

Specifying CPU Requirements for HPC Applications via ML Techniques


Affiliations
1 School of C&IT, REVA University, Bengaluru, India
2 School of CSA, REVA University, Bengaluru, India
 

Resource distribution in data centers is difficult for service providers because of the structures of usage and condition setup decisions. Customers encounter issues to anticipate the amount of CPU and memory required for job execution, and henceforth are not ready to assess when work yield shall be accessible to plan for next analyses. Systems that utilize cluster scheduler structures to gauge job execution time exists in the literature. Notwithstanding, we have seen that such methods are not appropriate for anticipating CPU utilization. In this paper, we assist customers to figure out their applications CPU usage utilizing machine learning (ML) techniques. We analyze how scheduler can be utilized to predict CPU utilization through ML techniques, and its evaluation on two frameworks containing an enormous number of user jobs.

Keywords

HPC, CPU Prediction, Machine Learning.
User
Notifications
Font Size

  • C. B. Lee, A. Snavely, "On the user–scheduler dialogue: studies of user-provided runtime estimates and utility functions", Journal of High Performance Computing Applications, 20 (4), 495-506, 2017.

  • C. B. Lee, Y. Schwartzman, J. Hardy, A. Snavely, "Are user runtime estimates inherently inaccurate?", Proceedings of the International Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), Springer, 253-263, 2014.

  • D. Tsafrir, Y. Etsion, D. G. Feitelson, "Backfilling using system-generated predictions rather than user runtime estimates", IEEE Transactions on Parallel and Distributed Systems 18 (6), 789-803, 2017.

  • S.-H. Chiang, A. Arpaci-Dusseau, M. K. Vernon, "The impact of more accurate requested runtimes on production job scheduling performance", Proceedings of the International Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), Springer, 103-127, 2012.

  • D. Tsafrir, D. G. Feitelson, "The dynamics of backfilling: solving the mystery of why increased inaccuracy may help", Proceedings of IEEE International Symposium on the Workload Characterization, IEEE, 131141, 2016.

  • D. Zotkin, P. J. Keleher, "Job-length estimation and performance in backfilling schedulers", Proceedings of the International Symposium on High Performance Distributed Computing (HPDC), IEEE, 236-243, 2017.

  • M. Hovestadt, O. Kao, A. Keller, A. Streit, "Scheduling in HPC resource management systems: Queuing vs. planning", Proceedings of the International Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), Springer, 1-20, 2013.

  • R. L. Cunha, E. R. Rodrigues, L. P. Tizzei, M. A. Netto, "Job placement advisor based on turnaround predictions for HPC hybrid clouds", Future Generation Computer Systems 67, 35-46, 2017.

  • A. Coates, A. Y. Ng, "The importance of encoding versus training with sparse coding and vector quantization", Proceedings of the 28th International Conference on Machine Learning (ICML-11), 921-928, 2017.

  • C. Cortes, V. Vapnik, "Support-vector networks", Journal of Machine Learning, 20 (3), 273-297, 2005.


Abstract Views: 284

PDF Views: 0




  • Specifying CPU Requirements for HPC Applications via ML Techniques

Abstract Views: 284  |  PDF Views: 0

Authors

Priyanka Bharti
School of C&IT, REVA University, Bengaluru, India
Rajeev Ranjan
School of CSA, REVA University, Bengaluru, India

Abstract


Resource distribution in data centers is difficult for service providers because of the structures of usage and condition setup decisions. Customers encounter issues to anticipate the amount of CPU and memory required for job execution, and henceforth are not ready to assess when work yield shall be accessible to plan for next analyses. Systems that utilize cluster scheduler structures to gauge job execution time exists in the literature. Notwithstanding, we have seen that such methods are not appropriate for anticipating CPU utilization. In this paper, we assist customers to figure out their applications CPU usage utilizing machine learning (ML) techniques. We analyze how scheduler can be utilized to predict CPU utilization through ML techniques, and its evaluation on two frameworks containing an enormous number of user jobs.

Keywords


HPC, CPU Prediction, Machine Learning.

References