TRL-Bench: Standardizing Cross-Paradigm Representation-Level Evaluation of Tabular Encoders

A multi-granular benchmark for comparing tabular encoders at the representation level across column, row, and table granularities

Wei Pang; Xiangru Jian; Hehan Li; Zhixuan Yu; Alex Xue; Jinyang Li; Zhengyuan Dong; Xinjian Zhao; Hao Xu; Chao Zhang; Reynold Cheng; M. Tamer Özsu; Tianshu Yu· 2026· DOI 10.48550/arXiv.2606.09323

The core problem

Tabular encoders are typically evaluated inside task-specific end-to-end pipelines, which makes direct comparison across different training paradigms difficult even when models operate on similar tabular signals. This fragmentation hinders progress because reported performance conflates encoder quality with downstream head design, task formulation, and data preprocessing. TRL-Bench addresses this by introducing a multi-granular tabular representation learning (TRL) benchmark that standardizes cross-paradigm representation-level evaluation. Each encoder exports row-, column-, or table embeddings through its supported wrapper, and shared lightweight heads probe these representations across three suites: TRL-CTbench (column/table), TRL-Rbench (row), and TRL-DLTE (compositional Data-Lake Table Enrichment spanning all three granularities). The benchmark provides curated assets and task reformulations, including 50 OpenML tables with 123 verified targets, 16 row-pair linkage rewrites, and a 47,772-table DLTE lake derived from 1,379 parent tables. By fixing downstream conditions, TRL-Bench enables fair comparison of reusable signal in exported tabular representations.

Innovation

Across 20 models and 16 tasks, TRL-Bench demonstrates that once downstream conditions are standardized, encoder quality is capability-specific rather than captured by a single leaderboard. In TRL-CTbench, generic text encoders often lead on tasks with strong surface-text signal, while tabular specialists win where their pretraining objective aligns with the task. For example, on column/table tasks with textual features, text-based models such as BERT achieve higher scores, whereas tabular models like TabNet excel on tasks with numerical or categorical signals aligned with their pretraining. In TRL-Rbench, within-table prediction and cross-table linkage favor different training regimes. Atomic linkage performance correlates strongly with the row-matching stage of DLTE pipelines, with correlation coefficients exceeding 0.8 in some settings. In TRL-DLTE, the strongest pipelines combine capability-matched specialists rather than reuse a single encoder. Top end-to-end quality depends on non-additive compositional fit rather than per-stage marginal rank alone, indicating that pipeline design must consider interactions between stages. The benchmark also reveals that no single encoder domi

Tabular encoders are typically evaluated inside task-specific end-to-end pipelines, which makes direct comparison across different training paradigms difficult even when models operate on similar tabular signals. This fragmentation hinders progress because reported performance conflates encoder quality with downstream head design, task formulation, and data preprocessing. TRL-Bench addresses this by introducing a multi-granular tabular representation learning (TRL) benchmark that standardizes cross-paradigm representation-level evaluation. Each encoder exports row-, column-, or table embeddings through its supported wrapper, and shared lightweight heads probe these representations across three suites: TRL-CTbench (column/table), TRL-Rbench (row), and TRL-DLTE (compositional Data-Lake Table Enrichment spanning all three granularities). The benchmark provides curated assets and task reformulations, including 50 OpenML tables with 123 verified targets, 16 row-pair linkage rewrites, and a 47,772-table DLTE lake derived from 1,379 parent tables. By fixing downstream conditions, TRL-Bench enables fair comparison of reusable signal in exported tabular representations.

TRL-Bench defines a standardized evaluation protocol where each encoder produces embeddings at one or more granularities: row, column, or table. These embeddings are extracted via model-specific wrappers that conform to a common interface. Shared lightweight heads—such as logistic regression or shallow MLPs—are then trained on these fixed representations to probe their utility across tasks. The benchmark comprises three suites:

Why it matters

The findings from TRL-Bench have several implications for tabular representation learning. First, the lack of a universal winner suggests that practitioners should select encoders based on the specific task and granularity, rather than relying on a single leaderboard. Second, the strong correlation between atomic linkage performance and DLTE row-matching stage implies that improvements in row-level representations can propagate to complex compositional pipelines. Third, the non-additive compositional fit in DLTE highlights the importance of joint optimization or careful pipeline design, as per-stage marginal gains do not necessarily translate to end-to-end improvements. The benchmark also provides a common protocol for measuring reusable signal in exported tabular representations under shared downstream conditions, which can accelerate research by enabling fair comparisons. Limitations include the focus on lightweight heads, which may not capture the full potential of encoders when fine-tuned end-to-end. Future work could extend TRL-Bench to include more diverse tasks, larger data lakes, and additional granularities. Overall, TRL-Bench establishes a foundation for systematic evaluation of tabular encoders and encourages the development of capability-specific models.

Who should read this

CS practitioners and researchers

Opening member content…