Not Your Usual Type(s): Data contracts as types across languages and engines

A production-tested design for treating data contracts as first-class types in a composable, multi-language lakehouse

Aldrin Montana; Colin Marc; Luca Bigon; Jacopo Tagliabue· 2026· DOI 10.48550/arXiv.2607.13339

The core problem

Composable data systems promise to let developers combine languages, engines, and catalogs without sacrificing a coherent user experience. In practice, however, pipeline-node boundaries remain weakly specified: transformations exchange tables through schemas that are often checked late, enforced unevenly across languages, and disconnected from the semantics business users care about. The authors—Aldrin Montana, Colin Marc, Luca Bigon, and Jacopo Tagliabue—argue that this gap is a primary source of production failures in multi-language lakehouses. Their work is grounded in over a year of operating millions of jobs in Bauplan, a composable data platform. From this operational experience, they distill design principles for a new SDK that treats data contracts as types. The central idea is that users, whether humans or agents, annotate input and output tables with schema objects that encode column types, constraints, documentation, and lineage. Bauplan then interprets these annotations at different points in the execution lifecycle. This approach aims to make pipeline-node boundaries explicit and enforceable across languages and engines, while keeping the semantics that business users

Innovation

The primary result is a set of design principles and an SDK that treats data contracts as types for a composable, multi-language lakehouse. The authors report that this design addresses common production failures observed over a year of operating millions of jobs. Specifically, by encoding column types, constraints, documentation, and lineage into schema objects, the SDK makes pipeline-node boundaries explicit and enforceable. The interpretation of annotations at different points in the execution lifecycle allows for earlier and more even enforcement across languages and engines. The ''everything-as-code'' philosophy is shown to enable both deterministic and non-deterministic reasoning over data flows. While the abstract does not provide quantitative metrics such as error reduction percentages or job failure rates, it claims that the design addresses common production failures. The results are therefore qualitative and architectural: a validated approach to specifying and enforcing data contracts across heterogeneous data systems. The paper also implies that the same contract can be used by both human developers and agents, suggesting interoperability between manual and automated w

Composable data systems promise to let developers combine languages, engines, and catalogs without sacrificing a coherent user experience. In practice, however, pipeline-node boundaries remain weakly specified: transformations exchange tables through schemas that are often checked late, enforced unevenly across languages, and disconnected from the semantics business users care about. The authors—Aldrin Montana, Colin Marc, Luca Bigon, and Jacopo Tagliabue—argue that this gap is a primary source of production failures in multi-language lakehouses. Their work is grounded in over a year of operating millions of jobs in Bauplan, a composable data platform. From this operational experience, they distill design principles for a new SDK that treats data contracts as types. The central idea is that users, whether humans or agents, annotate input and output tables with schema objects that encode column types, constraints, documentation, and lineage. Bauplan then interprets these annotations at different points in the execution lifecycle. This approach aims to make pipeline-node boundaries explicit and enforceable across languages and engines, while keeping the semantics that business users care about connected to the technical schema. The paper positions this as a step toward an ''everything-as-code'' philosophy that enables both deterministic and non-deterministic reasoning over data flows.

The methodology is rooted in a year-long operational study of millions of jobs running on Bauplan. Rather than proposing a purely theoretical type system, the authors extract design principles from observed production failures and the shortcomings of existing schema-checking approaches. They identify that transformations exchange tables through schemas that are often checked late, enforced unevenly across languages, and disconnected from business semantics. To address this, they design an SDK where data contracts are represented as types. Users annotate input and output tables with schema objects. These schema objects encode four categories of information: column types, constraints, documentation, and lineage. The SDK is intended for a composable, multi-language lakehouse, meaning the same contract must be interpretable across different languages and execution engines. Bauplan interprets these annotations at different points in the execution lifecycle, allowing checks to occur earlier and more consistently than late-stage schema validation. The authors also adopt an ''everything-as-code'' philosophy, which means that contracts, transformations, and lineage are expressed as code artifacts. This enables both deterministic reasoning (e.g., static type checking, constraint verification) and non-deterministic reasoning (e.g., agent-driven exploration or LLM-assisted development) over data flows. The paper does not present a formal type-theoretic calculus; instead, it reports on the design principles and their implementation in the Bauplan SDK, validated by production experience.

Why it matters

The discussion centers on why treating data contracts as types is a meaningful shift for composable data systems. The authors argue that weakly specified pipeline-node boundaries are a root cause of production failures. Late schema checks allow invalid data to propagate, and uneven enforcement across languages creates inconsistencies that are hard to debug. By contrast, encoding contracts as types with column types, constraints, documentation, and lineage brings several benefits. First, it makes the contract a first-class artifact that can be versioned, reviewed, and tested like code. Second, it allows the same contract to be interpreted by different engines and languages, supporting the composability promise without sacrificing coherence. Third, it connects technical schemas to business semantics through documentation and lineage, so that business users and agents can reason about data flows. The ''everything-as-code'' philosophy is presented as an enabler of both deterministic and non-deterministic reasoning. Deterministic reasoning includes static checks and constraint validation; non-deterministic reasoning includes agent-driven or LLM-assisted workflows that can inspect contracts and suggest changes. The authors position this as a practical response to the complexity of multi-language lakehouses, where no single engine or language dominates. The limitations are not explicitly discussed in the abstract, but the approach is grounded in one platform (Bauplan) and may require adaptation for other systems. The paper contributes a set of design principles rather than a formal specification, leaving room for future work on standardization and cross-platform interoperability.

Who should read this

CS practitioners and researchers

Opening member content…