Combining Exploratory Analysis and Automated Analysis for Anomaly Detection in Real-Time Data Streams

A prototype platform for real-time BGP anomaly detection that integrates automated threat indicators with interactive visualization to support security analysts.

Ahmed Shah; Ibrahim Abualhaol; Mahmoud Gad; Michael Weiss· 2026· DOI 10.48550/arXiv.2609.21222

The core problem

Security analysts face a deluge of real-time security information, often leading to alert fatigue and missed threats. The authors identify a core challenge: existing security analysis systems struggle to present multiple sources of information at varying levels of abstraction while remaining intuitive. This work examines the complementary nature of exploratory analysis (human-driven visualization) and automated analysis (machine-driven detection) in the context of Border Gateway Protocol (BGP) security. BGP is critical to Internet infrastructure but is highly vulnerable to hijacking, spam propagation, and denial-of-service attacks. Some attack scenarios are so elaborate that full automation is infeasible. The paper's contributions are twofold: (i) a prototype platform for computing indicators and threat alerts in real time and visualizing alert context, and (ii) a discussion of how exploratory and automated analysis interact. The target audience includes students, security researchers, and developers of real-time security monitoring systems.

The paper reports the development of a working prototype platform that computes indicators and threat alerts in real time and visualizes alert context. While the abstract does not provide quantitative performance metrics, the prototype demonstrates the feasibility of combining automated and exploratory analysis for BGP anomaly detection. The system is capable of monitoring live BGP streams and generating alerts for potential security threats. The authors highlight that the prototype enables analysts to explore alerts interactively, potentially reducing the risk of missing important events and links between them. The results are primarily qualitative, focusing on the design and interaction of the two analysis modes rather than on detection accuracy or speed.

Security analysts face a deluge of real-time security information, often leading to alert fatigue and missed threats. The authors identify a core challenge: existing security analysis systems struggle to present multiple sources of information at varying levels of abstraction while remaining intuitive. This work examines the complementary nature of exploratory analysis (human-driven visualization) and automated analysis (machine-driven detection) in the context of Border Gateway Protocol (BGP) security. BGP is critical to Internet infrastructure but is highly vulnerable to hijacking, spam propagation, and denial-of-service attacks. Some attack scenarios are so elaborate that full automation is infeasible. The paper's contributions are twofold: (i) a prototype platform for computing indicators and threat alerts in real time and visualizing alert context, and (ii) a discussion of how exploratory and automated analysis interact. The target audience includes students, security researchers, and developers of real-time security monitoring systems.

The authors developed a prototype system that ingests real-time BGP traffic and computes threat indicators and alerts. The architecture combines automated analysis modules with an interactive visualization interface. Automated analysis applies algorithms to detect anomalies, such as unexpected route changes or prefix hijacks, while exploratory analysis allows analysts to investigate alerts in context. The system is designed to handle streaming data and present multiple levels of abstraction. The paper describes the prototype's components and their interactions, though specific algorithms and implementation details are not fully enumerated in the abstract. The methodology emphasizes the integration of human-in-the-loop exploration with machine-driven detection to address complex attack scenarios that cannot be fully automated.

The authors discuss the complementary nature of exploratory and automated analysis. Automated analysis can process large volumes of data and flag potential anomalies, but it may miss elaborate attacks that require human judgment. Exploratory analysis allows analysts to investigate alerts, form hypotheses, and uncover subtle patterns, but it is time-consuming and may not scale. The paper argues that integrating both approaches creates a more robust security monitoring system. The prototype illustrates how visualization can provide context for automated alerts, enabling analysts to make informed decisions. The discussion also addresses the challenge of presenting multiple levels of abstraction without overwhelming the user. The authors conclude that the interaction between automated and exploratory analysis is essential for defending against complex BGP attacks, and they suggest that their findings are relevant to other real-time security monitoring domains.

CS practitioners and researchers

Opening member content…