Stability-Aware Proactive Autoscaling Using a Double Deep Q-Network in Cloud Computing Environments

A DDQN-based proactive autoscaling approach with Resource Removal Strategy for stable and adaptive cloud scaling

Bablu Kumar; Anshul Verma; Rajkumar Buyya· 2026· DOI 10.48550/arXiv.2609.14894

The core problem

Dynamic workloads and latency-sensitive applications demand efficient autoscaling in cloud computing environments. However, most existing autoscaling approaches rely on reactive mechanisms based on static thresholds, which lead to delayed responses and scaling oscillations under workload uncertainty. These limitations motivate the need for proactive, intelligent autoscaling solutions that can anticipate workload changes and make stable scaling decisions. To address these challenges, the authors propose a double deep Q-Network-based proactive autoscaling approach (DDQN-Proactive) combined with a Resource Removal Strategy (RRS). The proposed DDQN+RRS enhances decision-making by decoupling action selection from value evaluation, enabling more stable and adaptive scaling. This digest presents the key aspects of the methodology, results, and implications of the proposed approach.

Experimental results demonstrate that the proposed DDQN+RRS method outperforms both reactive and existing proactive approaches. Key performance metrics are summarized as follows: - **SLA violation rate**: DDQN+RRS achieves a lower SLA violation rate of **11.81%**, compared to higher rates in baseline methods. - **CPU utilization**: The approach achieves a higher CPU utilization of **52.23%**, indicating efficient resource usage. - **Scaling stability**: DDQN+RRS exhibits improved scaling stability with fewer scaling events (**2,488**) and reduced pod restarts (**1,246**). - **Oscillation reduction**: The method significantly reduces oscillations over time, particularly during the 0-60 s interval. While reactive methods show substantial fluctuations in pod allocation, DDQN+RRS achieves the most stable and smooth scaling, especially during the 15-30 s, 40-45 s, and 55-60 s intervals. These results highlight the effectiveness of the DDQN+RRS approach in maintaining stability while meeting performance requirements.

Dynamic workloads and latency-sensitive applications demand efficient autoscaling in cloud computing environments. However, most existing autoscaling approaches rely on reactive mechanisms based on static thresholds, which lead to delayed responses and scaling oscillations under workload uncertainty. These limitations motivate the need for proactive, intelligent autoscaling solutions that can anticipate workload changes and make stable scaling decisions. To address these challenges, the authors propose a double deep Q-Network-based proactive autoscaling approach (DDQN-Proactive) combined with a Resource Removal Strategy (RRS). The proposed DDQN+RRS enhances decision-making by decoupling action selection from value evaluation, enabling more stable and adaptive scaling. This digest presents the key aspects of the methodology, results, and implications of the proposed approach.

The proposed approach, DDQN-Proactive, leverages a double deep Q-Network (DDQN) to make proactive scaling decisions. DDQN is a reinforcement learning technique that addresses the overestimation bias of standard DQN by decoupling action selection from value evaluation. This is achieved by using two separate networks: an online network for action selection and a target network for value evaluation. The update rule for the target value is given by:

The superior performance of DDQN+RRS can be attributed to its proactive nature and the stability introduced by the double Q-learning mechanism and the Resource Removal Strategy. By decoupling action selection from value evaluation, DDQN mitigates the overestimation bias, leading to more accurate value estimates and better policy decisions. The RRS further smooths the scaling actions by preventing rapid resource removal, which is a common cause of oscillations in reactive autoscaling. The reduction in SLA violations and pod restarts indicates improved reliability and user experience. The higher CPU utilization suggests that resources are being used more efficiently, which can lead to cost savings. The significant reduction in oscillations, especially during critical intervals, demonstrates the method's ability to handle workload uncertainty gracefully. Overall, the proposed approach offers a promising solution for autoscaling in cloud environments, balancing performance, stability, and efficiency. Future work may explore integration with other cloud management systems and adaptation to different workload patterns.

CS practitioners and researchers

Opening member content…