ACE: An AI-Orchestrated Intrusion Detection and Compliance Mapping Engine

Engineering notes

Engineering notes will be added by the aipentium editorial team.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为大语言模型、生成式AI、ChatGPT相关技术、计算机视觉、深度学习等高价值论文补充中文说明。

Original abstract

The Automated Compliance Engine (ACE) is an AI-orchestrated intrusion detection and compliance mapping engine that is designed and implemented to bridge the gap between low-level network security events and high-level regulatory requirements. ACE combines Suricata IDS (Intrusion Detection System) with a five-stage LangGraph pipeline to automatically parse network alerts, leverage a Retrieval-Augmented Generation (RAG) architecture to map threats to NIST SP 800-53 Rev. 5 security controls, and deploy generated rules to a target system via SSH. The system is built on a two-virtual-machine VirtualBox environment, that includes a Kali Linux orchestrator and a Windows IDS host. It shows that AI-assisted automation can replace the manual work analysts usually do to move from detection to compliance mapping and countermeasure deployment. ACE is tested in eight simulated attack scenarios, including reconnaissance, web application attacks, DNS abuse, and denial-of-service attacks. The system found 100% alerts and zero false positives across three tests of legitimate traffic. For all the eight alerts processed, three resulted in deployable indicator-specific auto rules (37.5% rule-generation rate). All six rules generated are confirmed to have been deployed. The average confidence score for mapping NIST SP 800-53 across all eight alerts is 0.87. DeepEval LLM (Large Language Model) quality assessment (Challita, B., & Parrend, P., 2025) provides scores of 1.0 for Faithfulness and Answer Relevancy. The Hallucination score of 0.67 shows a known LLM-as-judge methodological issue, not an implementation flaw in the pipeline. This is supported by independent benchmarking of RAG evaluation frameworks under abstention policies (Siagian, L., 2025). The average time the pipeline took to run was about 173 seconds. SID (Security Identifier) based deduplication ensures that no unnecessary rules are used in pipeline runs, making the system idempotent and well-suited for long-running production environments.

Links and sources

Comments