Illusion of Depth: Revealing Hidden Stereo Vision Vulnerabilities in Depth Estimation

How simple repeating patterns can fool stereo cameras and deep depth models, displacing obstacles by up to 20 meters and triggering emergency braking in autonomous systems.

Sri Hrushikesh Varma Bhupathiraju; Tetsu Ishizue; Nicholas U. Costagliola; Ozora Sako; Kentaro Yoshioka; Takeshi Sugawara; Sara Rampazzi· 2026· DOI 10.48550/arXiv.2609.16336

The core problem

Stereo cameras are widely deployed in autonomous systems—self-driving cars, drones, and robots—because they offer cost-effective, precise depth estimation compared to LiDAR. However, this work reveals an intrinsic vulnerability rooted in the pixel sampling and calibration processes of stereo cameras. Attackers can manipulate the estimated depth of real obstacles using simple repeating patterns, without sophisticated adversarial machine learning. The vulnerability also affects deep learning-based depth estimation models. The authors evaluate the attack on two classical stereo matching algorithms (BM and SGBM), three deep learning models (PSMNet, MoCha-Stereo, and UniMatch), a stereo-LiDAR fusion model (SGM-DDC), and two commercial stereo cameras (ZED2 and Intel RealSense D435). For example, on the ZED2 camera, an attacker can displace obstacles up to 20 meters farther or 12 meters closer. In real-world driving tests, a brief 0.5-second attack can trigger emergency braking in a popular autonomous driving framework. The attack is also feasible at driving speeds up to 40 km/h in CARLA. State-of-the-art defenses are ineffective, and the authors propose a novel strategy using similarity

Innovation

The attack achieves fine-grained control over estimated depth. On the ZED2 camera, obstacles can be displaced up to 20 meters farther or 12 meters closer. Similar effects are observed on the Intel RealSense D435. The attack is effective on both classical and deep learning models: BM, SGBM, PSMNet, MoCha-Stereo, and UniMatch. In a real-world driving setting, a 0.5-second attack triggers emergency braking in a popular autonomous driving framework. In CARLA simulations, the attack remains feasible at driving speeds up to 40 km/h. The stereo-LiDAR fusion model SGM-DDC is also vulnerable, indicating that fusion does not inherently mitigate the issue. State-of-the-art defenses fail to detect or prevent the attack. The proposed similarity-score-based defense shows promise in dynamically detecting and suppressing depth discrepancies, but further validation is needed.

Stereo cameras are widely deployed in autonomous systems—self-driving cars, drones, and robots—because they offer cost-effective, precise depth estimation compared to LiDAR. However, this work reveals an intrinsic vulnerability rooted in the pixel sampling and calibration processes of stereo cameras. Attackers can manipulate the estimated depth of real obstacles using simple repeating patterns, without sophisticated adversarial machine learning. The vulnerability also affects deep learning-based depth estimation models. The authors evaluate the attack on two classical stereo matching algorithms (BM and SGBM), three deep learning models (PSMNet, MoCha-Stereo, and UniMatch), a stereo-LiDAR fusion model (SGM-DDC), and two commercial stereo cameras (ZED2 and Intel RealSense D435). For example, on the ZED2 camera, an attacker can displace obstacles up to 20 meters farther or 12 meters closer. In real-world driving tests, a brief 0.5-second attack can trigger emergency braking in a popular autonomous driving framework. The attack is also feasible at driving speeds up to 40 km/h in CARLA. State-of-the-art defenses are ineffective, and the authors propose a novel strategy using similarity scores to dynamically detect and suppress depth discrepancies.

The attack exploits the stereo matching pipeline, which relies on pixel sampling and calibration to compute disparity and depth. The authors design repeating patterns that induce systematic errors in matching costs. For classical algorithms like Block Matching (BM) and Semi-Global Block Matching (SGBM), the patterns cause consistent mismatches. For deep learning models (PSMNet, MoCha-Stereo, UniMatch), the vulnerability persists due to their reliance on similar sampling and feature matching. The attack is evaluated on commercial stereo cameras ZED2 and Intel RealSense D435, and on a stereo-LiDAR fusion model SGM-DDC. The authors also test in a driving simulator (CARLA) and in real-world driving scenarios. The core mechanism can be described by the disparity-depth relation:

Why it matters

The vulnerability stems from fundamental properties of stereo vision: pixel sampling and calibration. Repeating patterns can alias with the sampling grid, causing consistent matching errors. This is not limited to classical algorithms; deep learning models inherit the same weakness because they process similarly sampled data. The attack is stealthy, requiring only simple patterns, not adversarial ML. The implications for autonomous systems are severe: an attacker could cause phantom obstacles or hide real ones, leading to dangerous maneuvers. The ineffectiveness of current defenses highlights a critical gap. The proposed defense using similarity scores is a step forward, but it may not cover all attack variants. The authors call for rethinking stereo vision robustness, from hardware to algorithms. Future work should explore defenses that are robust to patterned attacks and validate them in diverse real-world conditions.

Who should read this

CS practitioners and researchers

Opening member content…