Machine learning for anomaly detection applies statistical and computational models to identify data points, patterns, or behaviors that deviate significantly from established norms within a security context. Unlike signature-based systems that match known threats, anomaly detection identifies the unknown by learning what normal looks like and flagging everything that deviates.
Unsupervised algorithms including isolation forests, autoencoders, and one-class SVMs learn representations of normal system behavior from historical data without requiring labeled examples of attacks. When new observations fall outside the learned distribution, they generate anomaly scores indicating deviation severity. Time-series models like LSTM networks capture temporal patterns in network flow data, authentication events, and system metrics, detecting anomalies in sequence and timing rather than individual events. Graph-based anomaly detection maps entity relationships and identifies unusual connection patterns, lateral movement, and communication with rare external destinations. Ensemble methods combine multiple anomaly detectors to reduce false positives while maintaining sensitivity to genuine threats.
Anomaly detection addresses a fundamental limitation of rule-based security: the inability to detect truly novel attacks. Zero-day exploits, insider threats using legitimate credentials, and advanced persistent threats that mimic normal operations all evade signature-based detection. Anomaly detection provides coverage against these unknown threats. The challenge lies in threshold calibration -- too sensitive and analysts drown in false positives, too conservative and real threats slip through. Effective deployment requires continuous feedback loops where analyst investigations refine model performance.
CDA approaches anomaly detection through the Security Posture and Hygiene domain, recognizing that effective anomaly detection depends on accurate baselines of normal operations. Our missions build the foundational monitoring infrastructure, data pipeline quality, and analyst workflows necessary before deploying ML models, ensuring organizations avoid the common trap of deploying sophisticated AI on unreliable data.