From Interpretation to Compilation: Compilation-Based Execution of Semantic Operators [Vision]

Semantic operator systems extend traditional data processing with natural-language interfaces, enabling operations such as semantic filtering, mapping, and joining. In these systems, users express intent in natural language, and the system applies that intent across a dataset. Existing systems commonly execute these operators through **interpretation-based execution**: for each row, record, or candidate pair, an LLM is invoked to interpret the semantic intent and produce an output. Although this approach is expressive and flexible, it places expensive LLM calls inside the data-processing loop, causing high latency, monetary cost, and limited scalability. The authors argue that this per-item invocation pattern is a fundamental bottleneck. They propose a paradigm shift: instead of using an LLM as a runtime interpreter for every data item, invoke it once during compilation to translate a semantic operator specification into deterministic executable code. This compiled physical operator approximates the behavior of the original LLM-based operator and runs locally over the dataset without per-row or per-pair LLM calls. The paper frames this as a vision at the intersection of database query processing, program synthesis, and LLM-powered data systems.

The proposed methodology is **compilation-based execution of semantic operators**. The key idea is to separate the semantic understanding phase from the data processing phase. During compilation, an LLM is invoked once to translate a semantic operator specification (e.g., a natural-language predicate or transformation) into deterministic executable code. This generated code serves as a compiled physical operator that approximates the original LLM-based operator's behavior. The authors instantiate this approach for three core semantic operators: **semantic filter**, **semantic map**, and **semantic join**. For each, they design a compilation strategy that produces code capable of running locally over the dataset without per-row or per-pair LLM calls. They compare this against LLM-based interpreted execution and integrate it into an existing semantic operator system. The compilation process can be conceptualized as a function

, where is the space of semantic operator specifications and is the space of executable programs. The goal is to find a program such that for a given dataset $D$, the output of $p$ approximates the output of the original LLM-based operator $O_{\text{LLM}}$: . The approximation quality is traded off against execution efficiency. The overall architecture is illustrated below.