Unique Log Parsing Framework for Enhanced Anomaly Detection in Network Security: Lauki Log Parser

Abstract

The increasing complexity of information security demands effective strategies to protect data across various domains. Traditional system log analysis, relying on unstructured logs, employs data mining and machine learning techniques to detect network threats. However, existing methods often struggle with logs of diverse formats and structures, resulting in missed anomalies and vulnerabilities. This paper introduces LaukiLogParser, a novel real-time log parsing framework designed to address these challenges by processing both structured and unstructured logs from multiple formats, including JSON, Syslog, and CEF. By incorporating unique parsing equations, the proposed parser enhances the identification of network threats, insider threats, and system vulnerabilities. Through comprehensive testing on publicly available datasets, LaukiLogParser demonstrated a significant 15% increase in anomaly detection accuracy compared to traditional parsers, along with improved F1-scores, precision, and recall. The parser’s ability to handle a variety of log formats provides unmatched flexibility in real-time environments, making it highly effective for modern network security systems. The paper compares LaukiLogParser with existing parsers, such as LogParser-LLM, OpenLog, and LogPPT, showcasing its superiority in accuracy, scalability, and adaptability. The results highlight the limitations of current parsers, while LaukiLogParser’s novel approach offers a robust solution for enhancing anomaly detection and improving real-time security monitoring.