In this research, we investigated two approaches to detect job anoma-
lies and/or contention for large scale computing efforts:
1. Preemptive job scheduling using binomial classification long short-term
memory networks
2. Forecasting intra-node computing loads from the active jobs and addi- tional job(s)
For approach 1, we achieved a 14% improvement in computational resources utilization and an overall classification accuracy of 85% on real tasks executed in a High Energy Physics computing workflow. For this paper, we present the preliminary results used in second approach.
Revised: January 15, 2020 |
Published: September 17, 2019
Citation
Schram M., N.R. Tallent, R.D. Friese, A. Singh, and I. Altintas. 2019.Application of Deep Learning on Integrating Prediction, Provenance, and Optimization. In Proceedings of the 23rd International Conference on Computing in High Energy and Nuclear Physics (CHEP 2018), EPJ Web of Conferences, 214, Article No. 06007.PNNL-SA-147454.doi:10.1051/epjconf/201921406007