TL;DR
POLAR is an LLM-powered framework designed to automate end-to-end cyber threat prioritization. By integrating unstructured threat intelligence with authoritative vulnerability databases, it disentangles complex incidents, scores static CVSS severity, forecasts dynamic 30-to-90-day exploitation likelihoods, and prescribes an actionable mitigation roadmap.
What Problem Does This Paper Address?
Security operations centers (SOCs) are drowning in "threat clutter"—over 11,000 new vulnerabilities were reported in 2024 alone. Security teams face cognitive overload trying to determine which flaws demand immediate remediation versus which can wait.
Traditional prioritization methods fall short:
- Rule-based systems (e.g., Snort, Suricata, basic SIEM rules) rely on rigid signatures that fail against rapidly evolving or "living-off-the-land" attacks.
- Conventional machine learning classifiers treat vulnerability indicators in isolation, struggling with distribution shifts and missing the broader narrative context (e.g., whether a public proof-of-concept has just dropped on GitHub).
- Standardized rubrics like CVSS (severity) and EPSS (exploitability) provide metrics but are not integrated, leaving analysts to manually stitch together raw advisories, forum chatter, and vendor patches.
Methods
POLAR decomposes threat assessment into a four-stage sequential pipeline:
[Raw CTI Advisories & Logs]
│
▼
1. CTI Triage --> Disentangles multi-threat text; enriches with NVD/ATT&CK
│
▼
2. Static Analysis --> Extracts evidence for 8 CVSS metrics; computes CVSS score
│
▼
3. Exploitation Analysis --> Evaluates temporal event chains to predict 30/90-day exploit risk
│
▼
4. Mitigation Planning --> Generates prioritized patches, workarounds, & playbooks
- CTI Triage: Ingests raw, messy threat reports. An LLM disentangles multi-vulnerability reports into isolated threat instances ($\tau_k$) and maps them to external databases (NVD, MITRE ATT&CK, CISA KEV) to attach standardized metadata.
- Static Analysis: Rather than asking the LLM to guess a final severity score, POLAR uses the model to extract evidence spans and classify each of the eight individual CVSS v3.1 base metrics (e.g., Attack Vector, Privileges Required). It then feeds these categorical outputs into the official deterministic CVSS scoring algorithm.
- Exploitation Analysis: Builds a chronological narrative of temporal threat events (disclosure $\rightarrow$ PoC exploit release $\rightarrow$ in-the-wild observations). The LLM reasons over these inter-event time gaps to output a calibrated probability ($p_k \in [0, 1]$) of exploitation over the subsequent 30 or 90 days.
- Mitigation Recommendation: Calculates an overall composite risk score ($\text{Risk}_k = s_k \cdot p_k \cdot \alpha_{\text{exp}} \cdot \beta_{\text{crit}}$) incorporating asset exposure and criticality. It retrieves concrete patches and workarounds, using tie-breaking logic (e.g., patch maturity, operational complexity) to deliver an execution-ready schedule.
Datasets
The authors compiled a benchmark of 12.8K raw threat incidents sourced from vendor advisories (Microsoft, Cisco, Oracle, Adobe), security blogs (Mandiant), exploit databases (Exploit-DB, GitHub PoCs), and the CISA KEV catalog.
Evaluation ground truth was mapped to:
- CVSS v3.1 Base Metrics & Vectors: 12.8K official vectors from the National Vulnerability Database (NVD).
- Exploitation Likelihood: Historical FIRST.org EPSS scores (12.8K entries) snapshot at fixed evaluation windows, along with 6.8K Exploit-DB references.
- Mitigation Registries: 9.9K NVD/vendor patch releases, 5.2K ATT&CK mitigation mappings, and 11.4K CISA KEV remediation timelines.
Key Insights and Ideas
- Structured workflows beat model scale: General-purpose LLMs (e.g., GPT-5, Claude Opus 4.1) reason well but hallucinate without structured guardrails. Conversely, cybersecurity-specialized small LLMs have domain facts but struggle with complex reasoning. POLAR’s staged pipeline brings the best of both worlds, driving the largest gains on general-purpose backbones.
- Deterministic calculators avoid LLM math errors: LLMs are notorious for drifting when asked to output continuous numeric scores. By restricting the LLM to extracting categorical metric vectors (e.g.,
AV:N,PR:L) and delegating score calculation to the official CVSS formula, error rates drop significantly. - Narratives capture abrupt shifts: While traditional statistical models struggle when exploit code suddenly drops, POLAR’s narrative-based reasoning detects the qualitative significance of a public PoC release or criminal forum chatter, enabling it to catch sudden spikes in exploitation activity.
- More history is not always better: Counterintuitively, extending the historical exploitation window beyond a year hurt forecasting performance. Outdated exploitation campaigns introduce noise from obsolete product versions, confirming that operational context should prioritize recent quarterly or semi-annual activity.
Key Results
The authors evaluated POLAR across six general-purpose frontier LLMs (Gemini-2.5-Pro, GPT-o4-mini, GPT-5, LLaMA-3-70B, Claude Opus 4.1, Qwen3-30B) and three domain models (Foundation-Sec-8B, Lily-Cybersecurity-7B, ZySec-7B):
- Triage & Extraction: On entangled incidents, POLAR boosted general-purpose LLM F1-scores by 0.24 to 0.34.
- Static Severity: Improved metric classification accuracy by up to ~20% on nuanced metrics like Attack Vector and Attack Complexity, consistently lowering CVSS RMSE across all backbones.
- Exploitation Forecasting: For abrupt changes in exploitation, POLAR reduced prediction RMSE dramatically (e.g., dropping GPT-5 RMSE by $27.12 \times 10^{-3}$) and increased trend direction accuracy (DirAcc up to +0.15).
- Mitigation Prioritization: Outperformed baselines in ranking urgency, achieving an NDCG@5 of up to 0.873 and Kendall’s $\tau$ of 0.783 with GPT-5.
Limitations and Future Work
- Predictive Lag on Zero-Days: While POLAR adapts well once early signals appear, it still underestimates the first few days of sudden "zero-day shocks" where no precursor signals exist.
- Hallucination in Sparse Data: When advisories are exceptionally vague, LLMs occasionally hallucinate plausible-looking but non-existent CVEs or over-speculate on ATT&CK techniques (though simple automated regex/database cross-checks easily catch these).
- Vendor Formatting Heterogeneity: Performance gains were smaller on unstructured vendor advisories when workarounds were buried in atypical formats.
- Future Work: The authors propose incorporating real-time vendor feed parsers, developing probabilistic ranking to reflect uncertainty, and applying multi-objective optimization to balance patch downtime costs directly against risk scores.
Summary updated