A Hierarchical Deep Reinforcement Learning Strategy for Self-Adaptive CPU Scheduling in Multi-tenant Database Systems

Nathan Cole

doi:10.54097/9wp79v18

Authors

Nathan Cole

DOI:

https://doi.org/10.54097/9wp79v18

Keywords:

Hierarchical Reinforcement Learning, Multi-tenant Databases, CPU Scheduling, Deep Q-Networks, SLA Compliance, Adaptive Resource Management, Tenant Isolation, Database Performance

Abstract

Multi-tenant database systems face significant challenges in CPU resource allocation due to competing performance requirements across diverse tenant workloads with varying priorities, Service Level Agreements (SLAs), and resource consumption patterns. Traditional static scheduling approaches fail to adapt to dynamic tenant demands and changing system conditions, leading to SLA violations, resource wastage, and degraded overall system performance. This study proposes a Hierarchical Deep Reinforcement Learning (HDRL) strategy for self-adaptive CPU scheduling in multi-tenant database environments. The framework employs a two-tier architecture where a global scheduler manages tenant prioritization and resource allocation policies, while local schedulers optimize CPU utilization within individual tenant contexts. Deep Q-Networks (DQNs) and Proximal Policy Optimization (PPO) algorithms enable dynamic adaptation to evolving tenant workloads and system resource availability. Experimental evaluation using multi-tenant workload benchmarks demonstrates that the proposed strategy achieves 41% improvement in SLA compliance rates while reducing average query response time by 35% across all tenant categories. The hierarchical approach successfully balances resource fairness with performance optimization, resulting in 27% better resource utilization efficiency compared to conventional scheduling methods.

Downloads

Download data is not yet available.

References

[1] Narasayya, V., & Chaudhuri, S. (2021). Cloud data services: Workloads, architectures and multi-tenancy. Foundations and Trends® in Databases, 10(1), 1-107.

[2] Cao, J., Zheng, W., Ge, Y., & Wang, J. (2025). DriftShield: Autonomous Fraud Detection via Actor-Critic Reinforcement Learning with Dynamic Feature Reweighting. IEEE Open Journal of the Computer Society.

[3] Odun-Ayo, I., Udemezue, B., & Kilanko, A. (2019). Cloud service level agreements and resource management. Adv. Sci. Technol. Eng. Syst., 4(2), 228-236.

[4] Hu, X., Guo, L., Wang, J., & Liu, Y. (2025). Computational fluid dynamics and machine learning integration for evaluating solar thermal collector efficiency-Based parameter analysis. Scientific Reports, 15(1), 24528.

[5] Batista, C., Morais, F., Cavalcante, E., Batista, T., Proença, B., & Rodrigues Cavalcante, W. B. (2024). Managing asynchronous workloads in a multi-tenant microservice enterprise environment. Software: Practice and Experience, 54(2), 334-359.

[6] Ji, E., Wang, Y., Xing, S., & Jin, J. (2025). Hierarchical Reinforcement Learning for Energy-Efficient API Traffic Optimization in Large-Scale Advertising Systems. IEEE Access

[7] Shethiya, A. S. (2024). Ensuring Optimal Performance in Secure Multi-Tenant Cloud Deployments. Spectrum of Research, 4(2).

[8] Zheng, W., & Liu, W. (2025). Symmetry-Aware Transformers for Asymmetric Causal Discovery in Financial Time Series. Symmetry.

[9] Robinson, E., & Anderson, J. (2025). Comparative Study of Adaptive Indexing Techniques for Performance Improvement in Dynamic Workloads. Journal of Innovation in Governance and Business Practices, 1(1), 32-58.

[10] Cao, W., Mai, N., & Liu, W. (2025). Adaptive Knowledge Assessment via Symmetric Hierarchical Bayesian Neural Networks with Graph Symmetry-Aware Concept Dependencies. Symmetry.

[11] Ionescu, S. A., Diaconita, V., & Radu, A. O. (2025). Engineering Sustainable Data Architectures for Modern Financial Institutions. Electronics, 14(8), 1650.

[12] Jiang, B., Wu, B., Cao, J., & Tan, Y. (2025). Interpretable Fair Value Hierarchy Classification via Hybrid Transformer-GNN Architecture. IEEE Access.

[13] Pushpalatha, M., & Thiyagarajan, P. (2025). A Comprehensive Review Of Machine Learning Approaches For Dynamic Resource Allocation In Multi-Tenant Cloud Environments. International Journal of Environmental Sciences, 11(11s), 828-846.

[14] Zheng, W., Tan, Y., Jiang, B., & Wang, J. (2025). Integrating Machine Learning into Financial Forensics for Smarter Fraud Prevention. Technology and Investment, 16(3), 79-90.

[15] Vilà, I., Pérez-Romero, J., Sallent, O., & Umbert, A. (2021). A multi-agent reinforcement learning approach for capacity sharing in multi-tenant scenarios. IEEE Transactions on vehicular Technology, 70(9), 9450-9465.

[16] Wang, M., Zhang, X., Yang, Y., & Wang, J. (2025). Explainable Machine Learning in Risk Management: Balancing Accuracy and Interpretability. Journal of Financial Risk Management, 14(3), 185-198.

[17] Blanco, F. G., Russo, E., Palesi, M., Patti, D., Ascia, G., & Catania, V. (2024, June). A deep reinforcement learning based online scheduling policy for deep neural network multi-tenant multi-accelerator systems. In Proceedings of the 61st ACM/IEEE Design Automation Conference (pp. 1-6).

[18] Battula, M. (2024, April). A systematic review on a multi-tenant database management system in cloud computing. In 2024 international conference on cognitive robotics and intelligent systems (ICC-ROBINS) (pp. 890-897). IEEE.

[19] Chinesta Llobregat, S. (2024). Design of a Data Analysis Platform as a Multitenant Service in the Cloud: An Approach towards Scalability and Adaptability.

[20] Hilman, M. H., Rodriguez, M. A., & Buyya, R. (2020). Multiple workflows scheduling in multi-tenant distributed systems: A taxonomy and future directions. ACM Computing Surveys (CSUR), 53(1), 1-39.

[21] Alatawi, M. N. (2025). Optimizing Multitenancy: Adaptive Resource Allocation in Serverless Cloud Environments Using Reinforcement Learning. Electronics, 14(15), 3004.

[22] Masdari, M., & Khoshnevis, A. (2020). A survey and classification of the workload forecasting methods in cloud computing. Cluster Computing, 23(4), 2399-2424.

[23] Dantuluri, V. N. R. (2025). AI-Powered Query Optimization in Multitenant Database Systems. Journal of Computer Science and Technology Studies, 7(4), 802-813.

[24] Hattab, N., & Belalem, G. (2023). Modular models for systems based on multi-tenant services: A multi-level petri-net-based approach. Journal of King Saud University-Computer and Information Sciences, 35(8), 101671.

[25] Hussain, F., Hassan, S. A., Hussain, R., & Hossain, E. (2020). Machine learning for resource management in cellular and IoT networks: Potentials, current solutions, and open challenges. IEEE communications surveys & tutorials, 22(2), 1251-1275.

[26] Meer, I. A., Besser, K. L., Ozger, M., Schupke, D., Poor, H. V., & Cavdar, C. (2024). Hierarchical Multi-Agent DRL Based Dynamic Cluster Reconfiguration for UAV Mobility Management. arXiv preprint arXiv:2412.16167.

[27] Wang, M., Fu, W., He, X., Hao, S., & Wu, X. (2020). A survey on large-scale machine learning. IEEE Transactions on Knowledge and Data Engineering, 34(6), 2574-2594.

[28] Gil, Y., Garijo, D., Khider, D., Knoblock, C. A., Ratnakar, V., Osorio, M., ... & Shu, L. (2021). Artificial intelligence for modeling complex systems: taming the complexity of expert models to improve decision making. ACM Transactions on Interactive Intelligent Systems, 11(2), 1-49.

[29] Xing, S., Wang, Y., & Liu, W. (2025). Self-Adapting CPU Scheduling for Mixed Database Workloads via Hierarchical Deep Reinforcement Learning. Symmetry, 17(7), 1109.

[30] Ghafouri, N., Vardakas, J. S., Ramantas, K., & Verikoukis, C. (2024). A multi-level deep rl-based network slicing and resource management for o-ran-based 6g cell-free networks. IEEE Transactions on Vehicular Technology, 73(11), 17472-17484.

[31] Ramegowda, A., Agarkhed, J., & Patil, S. R. (2020). Adaptive task scheduling method in multi-tenant cloud computing. International Journal of Information Technology, 12(4), 1093-1102.