3D Primitives are a Spatial Language for VLMs

Vision-language models (VLMs) exhibit a striking paradox: they can generate executable code that reconstructs a 3D scene from geometric primitives with correct object counts, classes, and approximate positions, yet the same models fail at simpler spatial questions on the same image. The authors argue that 3D geometric primitives (cubes, spheres, cylinders, expressed in executable code) serve as a powerful intermediate representation for spatial understanding. They exploit this insight through three contributions: a benchmark (SpatialBabel), a training-free inference strategy (Code-CoT), and a self-supervised fine-tuning recipe (S³-FT). The work positions geometric primitives in code as both a diagnostic tool and a transferable spatial vocabulary for VLMs.

Training on primitive images alone, S³-FT improves Qwen3-VL-8B by $+4.6$ to $+8.6\%$ on SpatialBabel-Primitive-QA, $+9.7\%$ on CV-Bench-2D, and $+17\%$ on HallusionBench; the recipe transfers across model families. Code-CoT yields up to $+6.4\%$ on SpatialBabel-QA-Score and $+5.0\%$ on CV-Bench-3D for models with strong coding capabilities. SpatialBabel shows that object-detection $F_1$ can vary by up to $5.7\times$ across scene-code languages for a single model.