Ilmu Komputer & AI editorial
Open AccessOA2025
Retrieval-Augmented Graph Reasoning with Large Language Models for Explainable Incident Diagnosis
A novel framework combining RAG, knowledge graphs, and LLMs achieves 92.3% precision in fault localization while providing transparent reasoning paths.
Yu’xiang He· Frontiers in Artificial Intelligence Research· 2025· DOI 10.71465/fair531
The core problem
Modern distributed systems have grown increasingly complex, making incident diagnosis and root cause analysis significant challenges. Traditional diagnostic approaches often lack explainability and struggle with the dynamic nature of system failures. Pure machine learning methods, while powerful, suffer from limited interpretability and contextual understanding, which hinders engineer trust and slows remediation workflows. This paper addresses these limitations by proposing a novel framework that integrates Retrieval-Augmented Generation (RAG) with graph-based reasoning and Large Language Models (LLMs) to enable explainable incident diagnosis. The core idea is to leverage knowledge graphs to capture causal relationships among system components, employ retrieval mechanisms to access relevant historical incident data, and utilize LLMs to generate human-interpretable explanations for diagnosed incidents. The framework aims to combine the structured reasoning capabilities of graph neural networks with the semantic understanding and generation abilities of large language models, thereby advancing the state-of-the-art in intelligent operations and system reliability engineering.
Innovation
The framework was evaluated on real-world incident datasets from large-scale distributed systems. The key performance metric is fault localization accuracy, measured as precision. The proposed method achieved 92.3% precision, significantly outperforming baseline approaches such as traditional rule-based systems and pure machine learning models. Additionally, the framework provides transparent reasoning paths that enhance engineer trust and accelerate remediation workflows. In a user study with site reliability engineers, the explanations generated by the LLM were rated as highly interpretable and actionable. The retrieval mechanism improved diagnostic accuracy by 15% compared to using the LLM alone, demonstrating the value of incorporating historical incident data. The graph-based reasoning contributed to a 10% reduction in false positives by capturing causal relationships that are often missed by correlation-based methods.
Modern distributed systems have grown increasingly complex, making incident diagnosis and root cause analysis significant challenges. Traditional diagnostic approaches often lack explainability and struggle with the dynamic nature of system failures. Pure machine learning methods, while powerful, suffer from limited interpretability and contextual understanding, which hinders engineer trust and slows remediation workflows. This paper addresses these limitations by proposing a novel framework that integrates Retrieval-Augmented Generation (RAG) with graph-based reasoning and Large Language Models (LLMs) to enable explainable incident diagnosis. The core idea is to leverage knowledge graphs to capture causal relationships among system components, employ retrieval mechanisms to access relevant historical incident data, and utilize LLMs to generate human-interpretable explanations for diagnosed incidents. The framework aims to combine the structured reasoning capabilities of graph neural networks with the semantic understanding and generation abilities of large language models, thereby advancing the state-of-the-art in intelligent operations and system reliability engineering.
The proposed framework consists of three main components: a knowledge graph, a retrieval module, and an LLM-based explanation generator. The knowledge graph encodes system components (e.g., services, hosts, network links) as nodes and their causal dependencies as edges. Formally, let where is the set of components and represents causal relationships. Each edge has a weight indicating the strength of the causal influence from component to component . The retrieval module uses the current incident's symptoms (e.g., logs, metrics) to query a historical incident database. It retrieves the top- similar past incidents based on a similarity function where is the query and is a historical incident. The retrieved incidents are then used to augment the context for the LLM. The LLM generates an explanation by conditioning on both the retrieved incidents and the relevant subgraph from the knowledge graph. The reasoning process can be summarized as:
Why it matters
The results demonstrate that integrating RAG with graph reasoning and LLMs effectively addresses the limitations of existing incident diagnosis approaches. The knowledge graph provides structured causal information that guides the LLM's reasoning, while the retrieval module supplies relevant historical context. This combination enables both high accuracy and explainability. The framework's ability to generate human-interpretable explanations is particularly valuable in operational settings where engineers need to understand and trust automated diagnoses. However, the approach relies on the quality and completeness of the knowledge graph and historical incident database. In dynamic environments where system topologies change frequently, maintaining an up-to-date knowledge graph is challenging. Future work could explore automated knowledge graph construction and continual learning to adapt to evolving systems. Additionally, the computational cost of LLM inference may be a concern for real-time diagnosis, though this can be mitigated with model distillation or caching. Overall, the proposed framework represents a significant step towards explainable and intelligent incident diagnosis in complex distributed systems.
Who should read this
CS practitioners and researchers
Opening member content…