Gaussian Relational Graph Transformer

A structure-semantic collaborative sampling and Gaussian attention approach for long-range relational graph learning

Zezhong Ding; Jin Li; Xugang Wang; Xike Xie· 2026· DOI 10.48550/arXiv.2605.15575

The core problem

Relational graph learning models relational databases as graphs and has demonstrated superior performance on a wide range of relational predictive tasks. However, existing methods struggle to capture long-range dependencies due to information decay in their message-passing mechanisms. Recent relational graph transformers remain limited in jointly modeling structural, semantic, and temporal information. To address these challenges, the authors propose GelGT, a Gaussian relational graph transformer that explicitly tackles these limitations. The key innovations are a structure-semantic collaborative sampling strategy and a Gaussian graph attention mechanism with a learnable Gaussian bias.

Extensive experiments on various real-world datasets demonstrate that GelGT achieves state-of-the-art downstream task performance. Notably, it yields up to a 13.8% improvement in predictive performance compared to existing methods. The results validate the effectiveness of the proposed sampling and attention mechanisms in capturing long-range dependencies and jointly modeling structural, semantic, and temporal information.

Relational graph learning models relational databases as graphs and has demonstrated superior performance on a wide range of relational predictive tasks. However, existing methods struggle to capture long-range dependencies due to information decay in their message-passing mechanisms. Recent relational graph transformers remain limited in jointly modeling structural, semantic, and temporal information. To address these challenges, the authors propose GelGT, a Gaussian relational graph transformer that explicitly tackles these limitations. The key innovations are a structure-semantic collaborative sampling strategy and a Gaussian graph attention mechanism with a learnable Gaussian bias.

GelGT introduces two main components:

The authors attribute GelGT's success to its ability to overcome information decay through the collaborative sampling that retains structural connectivity while filtering semantic noise. The Gaussian attention mechanism with a learnable bias effectively encodes temporal dependencies, allowing the model to dynamically adjust the influence of neighboring nodes based on temporal proximity. This addresses the limitation of prior relational graph transformers that struggle to jointly model structural, semantic, and temporal information. The up to 13.8% improvement highlights the practical impact of these innovations. Future work may explore scaling GelGT to even larger graphs and incorporating additional relational semantics.

CS practitioners and researchers

Opening member content…