Ilmu Komputer & AI editorial
Schwarz: Solver-Aware Agentic Program Verification
The core problem
Agentic verification systems can often generate source-level specifications that look plausible, but plausibility is not enough: the verifier must still turn those specifications into SMT obligations that the solver can prove. When this step fails, current LLM-driven loops usually expose only a coarse verifier error, timeout, or unknown solver result. The model cannot tell whether the specification is wrong, a helper lemma is missing, the proof context contains irrelevant facts, or the obligation needs a different theory view.
This paper presents **Schwarz**, an agentic verification harness that makes SMT-backed proof failure local, checkable, and repairable. Schwarz turns failed verification into obligation-local repair tasks: program-point snapshots expose checked facts at a boundary, local lemmas let the agent propose missing proof steps, and theory-aware solver policies guide the agent toward solver-friendly formulations for numeric, quantified, memory, and floating-point obligations. The authors implement Schwarz for C and Rust/Verus and evaluate it on 1,475 tasks. On 475 benchmarks from recent agentic verification tools, Schwarz solves 95.2% of the tasks. On 1,000 tasks from
Innovation
Schwarz is evaluated on 1,475 tasks. The results are reported in two main groups.
First, on 475 benchmarks from recent agentic verification tools, Schwarz solves **95.2%** of the tasks. This group is directly comparable to prior agentic verification systems, so the result indicates that solver-aware repair substantially improves the success rate on tasks that already target LLM-driven verification.
Second, on 1,000 tasks from the SV-COMP 2026 ReachSafety track, Schwarz solves **91.5%** of the tasks. These tasks average **1,427 LOC**, making them a scalability test rather than a small-benchmark exercise. The comparison baseline is **CPAchecker**, which solves **60.1%** of the same tasks. The gap of 31.4 percentage points is the headline empirical result: a solver-aware agentic harness can outperform a mature, non-agentic verifier on a standard verification benchmark track.
The paper also reports ablations and a comparison with a pure-agent baseline. These show that solver-aware repair is effective and scalable. In other words, the gains are not merely from using an LLM agent; they come from the specific mechanisms that make failures local and solver-aware. The ablation design iso
Why it matters
The central insight of Schwarz is that the bottleneck in agentic verification is not specification generation alone, but the *translation* of specifications into SMT obligations that the solver can actually prove. When that translation fails, a coarse verifier error gives the LLM almost no signal. The model cannot distinguish among several very different root causes: a wrong specification, a missing helper lemma, irrelevant facts in the proof context, or an obligation that needs a different theory view. Schwarz addresses this by making the failure local, checkable, and repairable.
The three mechanisms map directly onto those root causes. Program-point snapshots address the proof-context problem by exposing the checked facts at a boundary. Local lemmas address the missing-proof-step problem by letting the agent propose a scoped intermediate fact. Theory-aware solver policies address the formulation problem by guiding the agent toward solver-friendly encodings for numeric, quantified, memory, and floating-point obligations. This decomposition is what turns an opaque `unknown` result into an actionable repair task.
The empirical results support the design. The 95.2% solve rate on 475 agentic verification benchmarks shows that the approach works on tasks already designed for LLM-driven verification. The 91.5% solve rate on 1,000 SV-COMP 2026 ReachSafety tasks, averaging 1,427 LOC, shows that it scales beyond small examples. The comparison with CPAchecker at 60.1% is particularly notable because CPAchecker is a mature, non-agentic verifier; the result suggests that solver-aware agentic repair can outperform traditional verification workflows on this track. The ablations and pure-agent baseline further indicate that the gain comes from solver-aware repair rather than from the mere presence of an LLM agent.
Several limitations and open questions remain. The evaluation is reported for C and Rust/Verus, so generalization to other languages and verification backends is not established. The theory-aware policies cover numeric, quantified, memory, and floating-point obligations, but other theory combinations may require additional policies. The paper does not report per-task timing or solver resource usage in the provided abstract, so the cost of the repair loop relative to CPAchecker is not quantified here. Finally, the SV-COMP 2026 ReachSafety track is a specific benchmark family; performance on other tracks and on industrial-scale codebases remains to be tested.
For practice, Schwarz suggests a design principle for agentic verification tools: expose the solver interface to the agent in a structured, obligation-local form. Instead of asking the LLM to fix a global verifier error, give it snapshots, local lemma slots, and theory-aware policies. This makes the agent's task checkable and keeps the proof search bounded. For future work, the natural extensions are broader language support, richer theory policies, and integration with interactive proof assistants where local lemmas can be reused across obligations.
Who should read this
Opening member content…