ETCInfer: An Energy-efficient Thermal-aware Cooling-joint Scheduler for LLM Inference in AI Datacenters

A physics-informed, learning-based controller that jointly optimizes CRAC setpoints, GPU frequency, and micro-batch size to cut energy and thermal throttling while preserving latency SLOs

Rui Lu; Rui Ge; Huanghuang Liang; Xiaobo Zhou; Dan Wang· 2026· DOI 10.48550/arXiv.2609.15230

The core problem

Large language model (LLM) inference in AI datacenters couples two traditionally separate control loops: GPU serving and facility cooling. Raising the ambient temperature setpoint reduces cooling energy and carbon footprint, but it also shrinks thermal headroom, induces GPU throttling, and can lead to Service-Level-Objective (SLO) violations. This paper studies joint cooling-computing control for LLM inference, aiming to minimize per-job GPU-plus-cooling energy while satisfying thermal safety and latency SLO constraints. The authors identify a fundamental trade-off: aggressive cooling wastes energy, while insufficient cooling degrades performance and violates latency targets. Existing schedulers typically treat cooling and computing as independent systems, missing opportunities for cross-layer optimization. ETCInfer addresses this gap by co-designing a pre-job Computer Room Air Conditioner (CRAC) setpoint selection with per-GPU frequency and micro-batch size adaptation during execution.

Evaluation across real-trace simulation and validation experiments shows that ETCInfer reduces total job energy by up to 33.1%, thermal throttle exposure by up to 92.9%, and keeps SLO violation rates below 0.7% even at ambient temperatures up to . These results demonstrate that joint cooling–computing control can achieve substantial energy savings without sacrificing latency guarantees.The experiments compare ETCInfer against baseline schedulers that either fix the CRAC setpoint or adapt only GPU parameters. The energy reduction comes from both lower cooling power (due to higher setpoints) and reduced GPU throttling (due to proactive frequency and micro-batch adjustments). The thermal throttle exposure metric, which measures the fraction of time GPUs operate above safe temperatures, drops dramatically, indicating improved thermal safety.

Large language model (LLM) inference in AI datacenters couples two traditionally separate control loops: GPU serving and facility cooling. Raising the ambient temperature setpoint reduces cooling energy and carbon footprint, but it also shrinks thermal headroom, induces GPU throttling, and can lead to Service-Level-Objective (SLO) violations. This paper studies joint cooling-computing control for LLM inference, aiming to minimize per-job GPU-plus-cooling energy while satisfying thermal safety and latency SLO constraints.

The authors identify a fundamental trade-off: aggressive cooling wastes energy, while insufficient cooling degrades performance and violates latency targets. Existing schedulers typically treat cooling and computing as independent systems, missing opportunities for cross-layer optimization. ETCInfer addresses this gap by co-designing a pre-job Computer Room Air Conditioner (CRAC) setpoint selection with per-GPU frequency and micro-batch size adaptation during execution.

The key insight of ETCInfer is that cooling and computing must be controlled jointly, not independently. By using physics-informed models to estimate hidden thermal states and time-to-throttle, the scheduler can make proactive decisions that balance energy, temperature, and latency. The POMDP formulation captures the partial observability of thermal dynamics, and the learning-based ETCAdapter provides a practical solution. The results suggest that AI datacenters can safely raise ambient temperatures well above conventional limits (up to 48°C) if the inference scheduler is thermal-aware. This has significant implications for reducing carbon footprint and operational costs. However, the approach relies on accurate telemetry and calibration; future work could explore robustness to sensor noise and model drift. Additionally, the current implementation focuses on a single job type; extending to heterogeneous workloads and multi-tenant settings remains an open challenge.

CS practitioners and researchers

Opening member content…