Polar: Automating Cyber Threat Prioritization through LLM-Powered Assessment

Luoxi Tang, Yuqiao Meng, Ankita Patra, Weicheng Ma, Muchao Ye, Zhaohan XiSubmission to ICLR 2026 · Withdrawn

TL;DR

POLAR is an LLM-powered framework designed to automate end-to-end cyber threat prioritization. By integrating unstructured threat intelligence with authoritative vulnerability databases, it disentangles complex incidents, scores static CVSS severity, forecasts dynamic 30-to-90-day exploitation likelihoods, and prescribes an actionable mitigation roadmap.

What Problem Does This Paper Address?

Security operations centers (SOCs) are drowning in "threat clutter"—over 11,000 new vulnerabilities were reported in 2024 alone. Security teams face cognitive overload trying to determine which flaws demand immediate remediation versus which can wait.

Traditional prioritization methods fall short:

  • Rule-based systems (e.g., Snort, Suricata, basic SIEM rules) rely on rigid signatures that fail against rapidly evolving or "living-off-the-land" attacks.
  • Conventional machine learning classifiers treat vulnerability indicators in isolation, struggling with distribution shifts and missing the broader narrative context (e.g., whether a public proof-of-concept has just dropped on GitHub).
  • Standardized rubrics like CVSS (severity) and EPSS (exploitability) provide metrics but are not integrated, leaving analysts to manually stitch together raw advisories, forum chatter, and vendor patches.

Methods

POLAR decomposes threat assessment into a four-stage sequential pipeline:

[Raw CTI Advisories & Logs]
            │
            ▼
    1. CTI Triage           --> Disentangles multi-threat text; enriches with NVD/ATT&CK
            │
            ▼
    2. Static Analysis      --> Extracts evidence for 8 CVSS metrics; computes CVSS score
            │
            ▼
 3. Exploitation Analysis   --> Evaluates temporal event chains to predict 30/90-day exploit risk
            │
            ▼
4. Mitigation Planning      --> Generates prioritized patches, workarounds, & playbooks
  1. CTI Triage: Ingests raw, messy threat reports. An LLM disentangles multi-vulnerability reports into isolated threat instances ($\tau_k$) and maps them to external databases (NVD, MITRE ATT&CK, CISA KEV) to attach standardized metadata.
  2. Static Analysis: Rather than asking the LLM to guess a final severity score, POLAR uses the model to extract evidence spans and classify each of the eight individual CVSS v3.1 base metrics (e.g., Attack Vector, Privileges Required). It then feeds these categorical outputs into the official deterministic CVSS scoring algorithm.
  3. Exploitation Analysis: Builds a chronological narrative of temporal threat events (disclosure $\rightarrow$ PoC exploit release $\rightarrow$ in-the-wild observations). The LLM reasons over these inter-event time gaps to output a calibrated probability ($p_k \in [0, 1]$) of exploitation over the subsequent 30 or 90 days.
  4. Mitigation Recommendation: Calculates an overall composite risk score ($\text{Risk}_k = s_k \cdot p_k \cdot \alpha_{\text{exp}} \cdot \beta_{\text{crit}}$) incorporating asset exposure and criticality. It retrieves concrete patches and workarounds, using tie-breaking logic (e.g., patch maturity, operational complexity) to deliver an execution-ready schedule.

Datasets

The authors compiled a benchmark of 12.8K raw threat incidents sourced from vendor advisories (Microsoft, Cisco, Oracle, Adobe), security blogs (Mandiant), exploit databases (Exploit-DB, GitHub PoCs), and the CISA KEV catalog.

Evaluation ground truth was mapped to:

  • CVSS v3.1 Base Metrics & Vectors: 12.8K official vectors from the National Vulnerability Database (NVD).
  • Exploitation Likelihood: Historical FIRST.org EPSS scores (12.8K entries) snapshot at fixed evaluation windows, along with 6.8K Exploit-DB references.
  • Mitigation Registries: 9.9K NVD/vendor patch releases, 5.2K ATT&CK mitigation mappings, and 11.4K CISA KEV remediation timelines.

Key Insights and Ideas

  • Structured workflows beat model scale: General-purpose LLMs (e.g., GPT-5, Claude Opus 4.1) reason well but hallucinate without structured guardrails. Conversely, cybersecurity-specialized small LLMs have domain facts but struggle with complex reasoning. POLAR’s staged pipeline brings the best of both worlds, driving the largest gains on general-purpose backbones.
  • Deterministic calculators avoid LLM math errors: LLMs are notorious for drifting when asked to output continuous numeric scores. By restricting the LLM to extracting categorical metric vectors (e.g., AV:N, PR:L) and delegating score calculation to the official CVSS formula, error rates drop significantly.
  • Narratives capture abrupt shifts: While traditional statistical models struggle when exploit code suddenly drops, POLAR’s narrative-based reasoning detects the qualitative significance of a public PoC release or criminal forum chatter, enabling it to catch sudden spikes in exploitation activity.
  • More history is not always better: Counterintuitively, extending the historical exploitation window beyond a year hurt forecasting performance. Outdated exploitation campaigns introduce noise from obsolete product versions, confirming that operational context should prioritize recent quarterly or semi-annual activity.

Key Results

The authors evaluated POLAR across six general-purpose frontier LLMs (Gemini-2.5-Pro, GPT-o4-mini, GPT-5, LLaMA-3-70B, Claude Opus 4.1, Qwen3-30B) and three domain models (Foundation-Sec-8B, Lily-Cybersecurity-7B, ZySec-7B):

  • Triage & Extraction: On entangled incidents, POLAR boosted general-purpose LLM F1-scores by 0.24 to 0.34.
  • Static Severity: Improved metric classification accuracy by up to ~20% on nuanced metrics like Attack Vector and Attack Complexity, consistently lowering CVSS RMSE across all backbones.
  • Exploitation Forecasting: For abrupt changes in exploitation, POLAR reduced prediction RMSE dramatically (e.g., dropping GPT-5 RMSE by $27.12 \times 10^{-3}$) and increased trend direction accuracy (DirAcc up to +0.15).
  • Mitigation Prioritization: Outperformed baselines in ranking urgency, achieving an NDCG@5 of up to 0.873 and Kendall’s $\tau$ of 0.783 with GPT-5.

Limitations and Future Work

  • Predictive Lag on Zero-Days: While POLAR adapts well once early signals appear, it still underestimates the first few days of sudden "zero-day shocks" where no precursor signals exist.
  • Hallucination in Sparse Data: When advisories are exceptionally vague, LLMs occasionally hallucinate plausible-looking but non-existent CVEs or over-speculate on ATT&CK techniques (though simple automated regex/database cross-checks easily catch these).
  • Vendor Formatting Heterogeneity: Performance gains were smaller on unstructured vendor advisories when workarounds were buried in atypical formats.
  • Future Work: The authors propose incorporating real-time vendor feed parsers, developing probabilistic ranking to reflect uncertainty, and applying multi-objective optimization to balance patch downtime costs directly against risk scores.

Summary updated

Abstract

The rapid expansion of the cyber threat landscape, with over 11,000 new vulnerabilities reported in 2024 alone, has intensified the need for effective threat prioritization. Existing approaches, from rule-based systems to machine learning models, struggle with scalability, distribution shift, and context-independent scoring, often mis-ranking threats in dynamic exploitation environments. In this work, we present POLAR, an LLM-based framework that automates cyber threat prioritization across four sequential stages: Triage, Static Analysis, Exploitation Analysis, and Mitigation Recommendation. POLAR leverages LLM reasoning to transform unstructured threat intelligence into structured severity metrics, forecast exploitation likelihood using temporal narratives, and generate prioritized mitigation strategies. Through extensive evaluations, we highlight that POLAR not only improves prioritization accuracy for various cyber threats in the wild but also provides instructive outputs that assist analyst decision-making, which bridges the gap between automated threat hunting and real-world security practices.
BibTeX
@misc{tang2025polar,
  author = {Luoxi Tang and Yuqiao Meng and Ankita Patra and Weicheng Ma and Muchao Ye and Zhaohan Xi},
  title = {{Polar: Automating Cyber Threat Prioritization through LLM-Powered Assessment}},
  year = {2025},
  eprint = {2510.01552},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.01552},
}
Figure