TL;DR
Researchers have developed TRIAGE-CPG, a framework that allows Large Language Models (LLMs) to triage malware by analyzing only a tiny fraction (~4%) of a binary's code. The study reveals that different LLMs naturally adopt distinct triage "personalities"—some prioritize cautious detection while others favor broad escalation—proving that model choice is a critical policy decision in security workflows.
What Problem Does This Paper Address?
Threat hunters are drowning in unknown binaries. To manage the load, they use a "triage" process: quickly clearing benign files, flagging obvious malware, and escalating the "suspicious" middle ground for deep human analysis.
While dynamic analysis (running the file in a sandbox) is the standard, it's slow, expensive, and easily evaded by clever malware. Static analysis (inspecting code without running it) is faster but faces a massive scaling problem: a single Windows program can contain thousands of functions. Feeding all that code into an LLM is prohibitively expensive and often exceeds the model's "memory" (context window). The challenge is: How can an LLM effectively triage a binary when it can only afford to look at a handful of functions?
Methods
The authors introduce TRIAGE-CPG, a pipeline designed to treat malware triage as a "budgeted evidence acquisition" problem.
The pipeline operates in three main stages:
- Seed Analysis: An LLM-free heuristic called the Evidence Priority Score (EPS) ranks all functions in a binary based on structural prominence and interaction with sensitive APIs. The top $K$ functions are analyzed first to initialize an "evidence dossier."
- Evidence Exploration: An LLM agent acts as a detective. It reviews the initial dossier and uses a fixed "budget" of additional function reads to follow leads through the program's call graph (who calls whom).
- Verdict Reporting: A final LLM agent reviews the frozen dossier and issues a ternary verdict: Benign, Suspicious (escalate), or Malicious.
The researchers evaluated eight different models, including closed-source leaders like Gemini 3.1 Pro and open-weight models like GLM 5.2 and Qwen 3.6, all under the same strict budget and zero-shot protocol.
Datasets
The study used a dataset of 1,000 Windows PE binaries sampled from VirusTotal, balanced between 500 benign and 500 malicious samples.
- Consensus Labels: Benign files had zero detections, while malware had at least ten.
- Scale: The binaries had a median of 300 functions, but some reached nearly 50,000, highlighting the necessity of the budgeted approach.
Key Insights and Ideas
- The Asymmetry of Triage: The authors argue that triage errors aren't equal. Clearing malware as benign is a catastrophic failure, whereas escalating a benign file just costs a bit more analyst time. This motivates the use of the "Suspicious" verdict to capture uncertainty.
- Model "Personalities": Models don't behave the same way even with the same data. Some models (like Gemini 3.1 Pro) are high-specificity, producing small, malware-rich escalation queues. Others (like Gemma 4 26B) are recall-oriented, flagging or escalating almost everything to avoid missing a single threat.
- Exploration as a Filter: Interestingly, the agentic exploration stage (Stage 2) was better at correcting false alarms (improving specificity) than at finding malware that the initial seeding missed. If the initial seeds didn't look suspicious, the agent rarely found a reason to keep digging.
- Metadata as a Prior: Including PE metadata (headers, section entropy) acted as a "benign prior," helping models reduce false positives by providing context that the code alone might lack.
Key Results
The study found that LLMs can be remarkably effective even with a very tight budget. On average, models analyzed only about 40 functions out of a mean of 1,000 per binary.
- Top Performer: Gemini 3.1 Pro achieved a Triage Balanced Accuracy (tBA) of 0.855. It cleared 94% of benign samples while catching or escalating 77% of malware.
- Open-Weight Success: GLM 5.2 was the strongest open-weight model, trailing the top Gemini model by only about 5 points in accuracy.
- Efficiency: The highest-scoring models were often the most "frugal," stopping their exploration early once they had enough evidence, rather than exhausting their full budget.
Limitations and Future Work
- The "Wrapper" Problem: Static analysis still struggles with "wrapped" binaries (packers, protectors, and installers). These accounted for 61% of the escalated malware, as the malicious payload is hidden behind layers of delivery code that static analysis can't easily pierce.
- Base Rate Fallacy: The evaluation used a 50/50 split of malware and benign files. In a real-world environment where benign files vastly outnumber malware, the precision of the escalation queues would likely be much lower.
- Future Directions: The authors suggest that future work should integrate "unpack-and-re-extract" actions for the agent to handle wrapped files and benchmark the LLM pipeline against traditional, non-LLM static classifiers.
Summary updated