Risk-Aware Online Conformal State Probing

AI-based autonomous agents, often hosted at data centers, must acquire state information from robots or edge devices to issue informed control decisions. Managing uncertainty about the state is particularly consequential in safety-critical settings, where average-case guarantees are insufficient. The paper studies a sequential decision-maker process that jointly decides which actions to take and when to probe, given access to an arbitrary state prediction model. The authors propose online conformal state probing (OCSP), an action and probing policy that certifies worst-case reliability levels without relying on distributional assumptions. OCSP is designed to provably control the missed query error (MQE), defined as the fraction of instances where probing would have been beneficial, while minimizing the probing rate. A key advantage is that OCSP can be applied to existing pre-trained value-based control policies without requiring retraining or fine-tuning. The work is validated through numerical simulations to verify theoretical guarantees and assess performance trade-offs as a function of the calibration of the state predictor.

The authors validate OCSP through numerical simulations. They verify that the theoretical guarantees hold: the MQE remains below the target risk level $\alpha$ across different settings, and the probing rate is significantly lower than naive probing strategies. Performance trade-offs are assessed as a function of the calibration of the state predictor. When the state predictor is well-calibrated, OCSP achieves a lower probing rate for the same MQE guarantee. Conversely, with a poorly calibrated predictor, OCSP automatically increases probing to maintain reliability. The simulations also demonstrate that OCSP can be applied to existing pre-trained value-based control policies without retraining or fine-tuning, preserving their performance while adding safety guarantees. The results confirm that OCSP provides a practical and provably reliable solution for risk-aware state probing in safety-critical sequential decision-making.