Ilmu Komputer & AI editorial
Open AccessOA2026
QUARTET: Quad-branch cross-Attention and Random-walk Traces for Enhancing Transformers on Relational Graphs
A graph transformer architecture that combines causal random-walk sampling with quad-branch cross-attention to improve relational deep learning on multi-table databases.
Kyaw Hpone Myint; Nan Jiang; Xiang Li; Zhe Wu; Alexandre G. R. Day; Pranab Mohanty; Giri Iyengar· 2026· DOI 10.48550/arXiv.2609.26855
The core problem
Relational Deep Learning (RDL) models multi-table databases as heterogeneous temporal graphs, enabling end-to-end learning over relational structure. Graph transformers have achieved state-of-the-art performance on benchmarks such as RelBench. However, the leading model, RelGT, has two key limitations: its random local sampler yields loosely connected subgraphs that hinder message passing, and its global attention module relies on a single, seed-feature-based memory that ignores broader macro-level dynamics. This work introduces QUARTET, an expressive graph transformer architecture designed to overcome these limitations by applying full self-attention on local subgraphs while enriching global context through cross-attention branches.
Innovation
Across the RelBench v1 classification tasks, QUARTET consistently matches or outperforms the current state-of-the-art graph transformer baselines (HGT and RelGT). The authors report that QUARTET achieves superior performance on multiple tasks, demonstrating the effectiveness of the proposed components. Ablation studies confirm that the CRW sampler significantly enriches local neighborhood quality, while the global branches provide essential, task-specific predictive gains. Specifically, removing the CRW sampler leads to a drop in performance, and removing any of the four global branches also results in decreased accuracy, highlighting the importance of each component.
Relational Deep Learning (RDL) models multi-table databases as heterogeneous temporal graphs, enabling end-to-end learning over relational structure. Graph transformers have achieved state-of-the-art performance on benchmarks such as RelBench. However, the leading model, RelGT, has two key limitations: its random local sampler yields loosely connected subgraphs that hinder message passing, and its global attention module relies on a single, seed-feature-based memory that ignores broader macro-level dynamics. This work introduces QUARTET, an expressive graph transformer architecture designed to overcome these limitations by applying full self-attention on local subgraphs while enriching global context through cross-attention branches.
QUARTET comprises two main components: a Causal Random Walk (CRW) sampler and a quad-branch cross-attention module.
Why it matters
The results indicate that both the local sampling strategy and the global context integration are crucial for strong performance on relational graphs. The CRW sampler addresses the issue of loosely connected subgraphs by producing densely connected local neighborhoods that facilitate message passing. The quad-branch cross-attention module overcomes the limitation of a single seed-feature-based memory by incorporating multiple perspectives, allowing the model to capture macro-level dynamics. The ablation studies provide clear evidence for the contribution of each branch. However, the paper does not discuss potential limitations such as computational complexity or scalability to very large graphs. Future work could explore extending QUARTET to other relational learning tasks and investigating more efficient implementations.
Who should read this
CS practitioners and researchers
Opening member content…