Threat Hunting
Threat hunting is the proactive, analyst-driven search for threats that have evaded automated detection systems.
# Threat Hunting
Definition
Threat hunting is the proactive, analyst-driven search for threats that have evaded automated detection systems. Unlike SOC alert triage (which responds to alerts generated by detection rules), threat hunting begins with the assumption that the environment is already compromised and the existing SIEM rules, EDR alerts, and automated detections have not caught it.
The premise is statistically justified. The Mandiant M-Trends 2024 report measured the global median dwell time (the period between initial compromise and detection) at 10 days. Some intrusions persist for months. The most sophisticated threat actors (state-sponsored APT groups, advanced ransomware operators) design their operations specifically to evade automated detection: they use living-off-the-land techniques (legitimate system tools rather than malware), they move slowly (staying under threshold-based alert triggers), and they compromise legitimate credentials (bypassing authentication controls). These attacks are invisible to automated detection because they do not match predefined alert patterns.
Threat hunting fills this gap. The hunter does not wait for an alert. The hunter forms a hypothesis ("if APT29 targeted our environment, they would likely use this technique"), searches for evidence of that technique in historical telemetry, and either confirms the hypothesis (threat detected) or refines the detection rules to catch the pattern in future automated detection.
How It Works
The Hypothesis-Driven Approach
Effective threat hunting follows a structured methodology, not ad hoc "looking around" in logs.
Hypothesis formation. The hunt begins with a testable hypothesis derived from threat intelligence, incident trends, or environmental knowledge. Hypotheses take the form: "Given that [threat context], if [adversary technique] occurred in our environment, we would expect to see [observable evidence] in [data source]."
Examples of well-formed hypotheses:
"APT29 has been targeting organizations in our sector using SAML token forging (T1606.002). If this technique was used against our Azure AD tenant, we would expect to see anomalous SAML token activity in the Azure AD sign-in logs: tokens with unusual lifetimes, tokens issued by unexpected identity providers, or tokens used from IP addresses inconsistent with the normal user location."
"Ransomware operators commonly stage data for exfiltration before encrypting (T1074). If staging activity occurred in our environment, we would expect to see large volumes of file copy operations to a single directory on a non-standard file server, detected through Windows Security Event Log file audit events or EDR file operation telemetry."
"An insider threat actor with legitimate access would not trigger authentication alerts but would exhibit anomalous data access patterns (T1213). If insider data collection occurred, we would expect to see a user accessing significantly more SharePoint sites or file shares than their historical baseline, detected through Microsoft 365 audit logs or DLP telemetry."
Data collection and analysis. The hunter queries the relevant data sources (SIEM, EDR, cloud audit logs, network telemetry) to search for the evidence specified in the hypothesis. This requires proficiency with query languages (SPL, KQL, Lucene, SQL) and deep understanding of the data available in each source.
Analysis is iterative. Initial queries may return large result sets that require refinement. The hunter filters noise, identifies anomalies, and investigates outliers. Each query result either supports the hypothesis, refutes it, or suggests a modified hypothesis that requires a different query.
Outcome. Every hunt produces one of three outcomes:
Threat confirmed: evidence of adversary activity is identified. The hunt transitions to an incident response workflow. The confirmed threat is documented, and new detection rules are written to catch the technique in future automated detection.
Hypothesis refined: no evidence of the specific technique is found, but the analysis reveals gaps in detection coverage or data collection that need to be addressed. New detection rules are written, or additional log sources are connected.
Hypothesis disproven: no evidence is found, and the analysis confirms that the data sources and detection rules are adequate for the hypothesized technique. The hunt is documented, and the next hypothesis is tested.
All three outcomes produce value. A hunt that confirms a threat prevents a breach. A hunt that reveals detection gaps improves future automated detection. A hunt that disproves a hypothesis validates existing coverage. No hunt is wasted.
Data Requirements
Threat hunting requires deep, historical telemetry. Hunters search through weeks or months of historical data because advanced threats may have been present for extended periods. The quality of hunting is directly proportional to the quality and depth of available data.
Essential data sources for hunting:
Endpoint telemetry (EDR): process creation, file modifications, registry changes, network connections, loaded modules. This is the richest data source for hunting because most adversary activity manifests on endpoints.
Authentication and identity logs: Active Directory, Azure AD/Entra ID, Okta, Google Workspace. Authentication anomalies (unusual login times, impossible travel, failed authentication patterns, new device enrollments) are primary hunting indicators.
Network metadata: DNS query logs, proxy logs, NetFlow/IPFIX, firewall logs. Network data reveals command-and-control communication patterns, lateral movement, and data exfiltration that endpoint telemetry may not capture.
Cloud audit logs: AWS CloudTrail, Azure Activity Log, GCP Audit Logs. Cloud environment activity is often the target of modern threat actors (cloud credential theft, cloud resource abuse, cloud data exfiltration).
Email telemetry: inbound email metadata, attachment analysis results, URL click tracking. Phishing is the most common initial access vector, and email data often contains the earliest indicators.
Data retention requirements: Hunting requires 90 days of historical data at minimum. 180 days is recommended. 365 days provides the ability to detect the most patient adversaries. Shorter retention windows limit the hunter's ability to find threats that have been present longer than the retention period.
Hunting Frameworks
Several frameworks structure the hunting process:
MITRE ATT&CK. The primary hunting taxonomy. Hunters select ATT&CK techniques as the basis for hypotheses and structure hunts around specific technique IDs. The ATT&CK matrix provides a systematic way to cover the entire adversary playbook over successive hunt cycles.
The TaHiTI model (Targeted Hunting Integrating Threat Intelligence). A structured methodology for integrating threat intelligence into hunting operations. TaHiTI defines the process from intelligence intake through hypothesis formation, data analysis, and output production.
Sqrrl's Hunting Loop. A four-step iterative model: create hypothesis, investigate via tools and techniques, uncover new patterns, and inform and enrich automated analytics. Despite Sqrrl's acquisition by AWS, the hunting loop remains widely referenced as an operational model.
Hunting Maturity
Not every organization is ready for threat hunting. The SANS Hunting Maturity Model defines five levels:
Level 0 (Initial). No hunting capability. The organization relies entirely on automated detection. Data is collected but not proactively searched.
Level 1 (Minimal). Some ad hoc hunting occurs, typically in response to a specific threat advisory. No formal process. Hunting is reactive, not systematic.
Level 2 (Procedural). Hunting follows documented procedures based on threat intelligence. Hunts are planned and tracked. The organization has dedicated hunting time (even if not dedicated hunters).
Level 3 (Innovative). The organization creates new hunt hypotheses based on original analysis, not just external threat reports. Hunting produces novel detection rules that improve automated coverage.
Level 4 (Leading). Hunting is fully integrated with threat intelligence, detection engineering, and incident response. Hunt findings automatically feed detection rule development. The organization contributes to the broader threat intelligence community.
Most organizations are at Level 0 or 1. Reaching Level 2 requires dedicated time, appropriate data sources, and analyst skill. CDA's TID-H03 mission (Threat Hunting Program, 24 hours) builds the Level 2 foundation and provides the methodology to progress toward Level 3.
Why It Matters
Detection Rules Have Blind Spots
Every detection rule is written to match a known pattern. An adversary who uses a technique that does not match any existing rule operates undetected. Living-off-the-land techniques (PowerShell, WMI, PsExec, native OS tools) are particularly difficult to detect with rules because the tools are legitimate and widely used in normal operations. The difference between a system administrator running PowerShell and an attacker running PowerShell is intent and context, which automated rules struggle to distinguish.
Hunting addresses this blind spot by applying human analytical capability to the data. A hunter examining PowerShell execution patterns can identify anomalies that rules miss: PowerShell encoded commands from a user who never uses PowerShell, PowerShell network connections to newly registered domains, PowerShell executing at unusual hours from a machine that is normally idle. These contextual judgments are the hunter's comparative advantage over automated detection.
Dwell Time Reduction
Every day an attacker operates undetected, they expand access, exfiltrate data, prepare for encryption, and increase the eventual damage. Hunting directly reduces dwell time by searching for threats that automated detection has not found. An organization that hunts weekly has a maximum theoretical dwell time of one week for threats within the hunting scope. An organization that relies exclusively on automated detection has a maximum theoretical dwell time limited only by the attacker's patience.
Detection Engineering Fuel
Hunting and detection engineering form a virtuous cycle. Hunting discovers threats that rules missed. The discoveries are translated into new detection rules. New rules improve automated coverage. Improved coverage reduces the scope of threats that require hunting. The next hunting cycle focuses on the remaining gaps. Each cycle tightens the detection net.
This is why CDA positions hunting (TID-H03) and detection engineering (TID-H01) as companion missions in the C-HARDEN campaign. Neither is complete without the other. Rules without hunting become stale. Hunting without rules cannot scale.
CDA Perspective
Threat hunting is a core TID (Threat Intelligence and Defense) operation in the Planetary Defense Model. TID is the atmosphere: the detection layer that filters threats before they reach the surface. Automated detection (SIEM rules, EDR alerts) is the weather station network. Threat hunting is the scout team that goes beyond the stations to find what the sensors missed.
CDA's Predictive Defense Intelligence (PDI) methodology makes hunting a continuous, intelligence-driven operation rather than an occasional activity. "See the threat before it sees you." The speculatores did not wait for barbarians to attack the wall. They went into enemy territory to find the threat before it arrived. CDA's threat hunters operate the same way: they search the environment for adversary presence that the automated sensors have not detected.
The Roman parallel extends further. The frumentarii (Rome's covert intelligence agents) gathered information by operating within the population under non-official cover. They did not announce their presence. They observed, analyzed, and reported. Threat hunters operate in the data the same way: silently searching through historical telemetry, forming and testing hypotheses, and reporting findings that improve the organization's defensive posture.
Three TOP missions connect directly to threat hunting:
- TID-H03 (Threat Hunting Program): Establish the recurring threat hunting program. Define the methodology, hypothesis sources, data requirements, hunt cadence, and output workflow (how findings become detection rules). 24 estimated hours.
- TID-R01 (Threat Landscape Assessment): Provide the threat intelligence that generates hunting hypotheses. Which actors target this industry? Which techniques do they use? Which of those techniques are we not detecting? 20 estimated hours.
- TID-D02 (Purple Team Exercise): Validate hunting and detection capabilities. Red team executes ATT&CK-mapped techniques. Blue team (including hunters) attempts to detect them. Results map directly to detection coverage gaps that inform future hunts. 40 estimated hours.
CDA approaches hunting differently from generic MDR providers in one way: every hunt is mapped to the client's specific threat profile through TID-R01. Generic hunting programs cycle through ATT&CK techniques sequentially. CDA's hunts prioritize the techniques that the client's specific adversaries (based on industry, geography, and technology stack) are most likely to use. A healthcare client's hunting program prioritizes techniques used by FIN12 and state-sponsored actors targeting health data. A defense contractor's program prioritizes techniques used by APT29, APT41, and Volt Typhoon. The hunting is targeted because the threats are targeted.
Key Takeaways
- Threat hunting is the proactive, analyst-driven search for threats that automated detection has not caught. It starts with the assumption that the environment is already compromised.
- Hunting follows a hypothesis-driven methodology: form a testable hypothesis from threat intelligence, search for evidence in historical telemetry, and produce one of three outcomes (threat confirmed, detection gap identified, or coverage validated).
- Hunting requires deep historical telemetry (90 days minimum, 365 recommended), proficient analysts, and structured methodology. Not every organization is ready for it.
- Hunting and detection engineering form a virtuous cycle: hunting finds what rules miss, findings become new rules, better rules reduce the scope that requires hunting.
- CDA's hunting program is targeted to the client's specific threat profile, not generic ATT&CK technique cycling.
Related Articles
- Threat Intelligence and Defense (TID): The Atmosphere
- SIEM Architecture
- MITRE ATT&CK Framework
- Endpoint Detection and Response (EDR)
- Incident Response Lifecycle
- State-Sponsored Cyber Threats: A Global Overview
Sources
- Mandiant (Google Cloud). "M-Trends 2024: Special Report." Mandiant, April 2024. (Median dwell time statistics.)
- MITRE Corporation. "ATT&CK Framework." attack.mitre.org, updated continuously.
- SANS Institute. "SANS Threat Hunting Maturity Model." SANS, 2024.
- Ducau, Freddy, et al. "TaHiTI: Targeted Hunting Integrating Threat Intelligence." FI-ISAC, 2018.
- Bianco, David. "The Pyramid of Pain: Intelligence-Driven Incident Response." detect-respond.blogspot.com, 2014.
Word count: 1,918
Related CDA Missions
CDA Theater missions that address topics covered in this article.
Written by Evan Morgan
Found an issue? Help improve this article.