MalGTA: Large language Model-based Guided Malware Tactical Analysis

Xu‐Guang Guo, Jingfeng Xue, Zeyang Liu, Weijie Han, Jingjing HuResearch Square 2025 · DOI

Abstract

In High-Performance Computing (HPC) environments, a comprehensive understanding of cybersecurity threats and their underlying attack strategies is essential. However, current research predominantly focuses on maliciousness determination, typically emphasizing the code's operational behaviors rather than the attack strategies employed. The advancements in multimedia computing, particularly Large Language Models (LLMs), have paved the way for innovative solutions to the aforementioned bottleneck. This work proposes MalGTA (Guided Malware Tactical Analysis), an LLM-based system that automates ATT&CK (Adversarial Tactics, Techniques, and Common Knowledge)-aligned malware tactical analysis through Cuckoo Sandbox-driven dynamic profiling. Specifically, we construct a multi-source knowledge base integrated with Retrieval-Augmented Generation (RAG), which mitigates hallucinations in LLMs through context-sensitive threat intelligence retrieval. In addition, we propose a query optimization strategy to address challenges related to input information overload and attention dispersion in LLMs, enabling context-aware data refinement from Cuckoo reports. Experimental results show the effectiveness of the method.

Introduction

In the modern cybersecurity landscape, simply identifying a file as "malicious" is no longer sufficient for robust defense. Security researchers need to understand the underlying tactics—the "how" and "why" of an attack—to implement effective countermeasures. While dynamic analysis tools like Cuckoo Sandbox generate exhaustive logs of API calls and system changes, these reports are often massive, creating significant hurdles for automated analysis.

Large Language Models (LLMs) offer a potential solution, but they face two primary challenges in this domain: Input Overload, where raw JSON reports exceed context windows and cause models to miss critical details, and Hallucinations, where models misinterpret technical functions due to a lack of specialized domain knowledge. MalGTA addresses these issues through a structured framework combining query optimization and Retrieval-Augmented Generation (RAG).

The MalGTA Framework

MalGTA utilizes a four-stage pipeline to translate complex behavioral logs into high-level tactical insights aligned with the MITRE ATT&CK framework.

The four-stage MalGTA framework: Knowledge Base Construction, Query Optimization, Retrieval Augmentation, and Analysis Generation.
  1. Knowledge Base Construction: A specialized vector database is built from three sources: Microsoft API documentation, Cyber Threat Intelligence (CTI) reports, and the MITRE ATT&CK framework.
  2. Query Optimization: Raw sandbox reports are distilled into structured "behavior chains" that link events chronologically and semantically. An LLM then rewrites these chains into concise natural language queries.
  3. Retrieval-Augmented Generation (RAG): The system retrieves the top-5 most relevant knowledge chunks from the database to provide technical context.
  4. Analysis Generation: The LLM synthesizes the behavior chain and retrieved expert knowledge to produce a final report identifying specific attack techniques.

Data Processing and Feature Extraction

A core component of MalGTA's success is its ability to handle the diverse data types found in dynamic analysis reports. These reports contain layers of information ranging from network interactions to memory activities.

Summary of the various components extracted from dynamic analysis reports for tactical processing.

By extracting key features such as API calls, file operations, and registry modifications, MalGTA constructs a "Behavior Chain." This optimization step is the system's primary performance driver; moving from raw reports to optimized chains improved tactical analysis accuracy from 0.316 to 0.860 in experimental trials.

Experimental Evaluation and Results

The system was tested against a dataset of 850 samples, including 770 traditional malware samples from VirusShare and 80 Advanced Persistent Threat (APT) samples. Using the Phi3-14B model, MalGTA achieved an overall accuracy of 0.860 and an F1 score of 0.841.

Key findings from the evaluation include:

  • Multi-Source Synergy: Using a combined corpus of API, CTI, and ATT&CK data significantly outperformed any single source.
  • The Retrieval "Sweet Spot": Retrieving exactly five knowledge chunks (K=5) provided the best balance between context and noise.
  • The APT Challenge: The system performed better on traditional malware (0.872 accuracy) than on APTs (0.804 accuracy), as the latter often use anti-sandbox techniques to hide their behavior.

Conclusion

MalGTA demonstrates that LLMs can effectively automate malware tactical analysis when supported by structured data refinement and specialized knowledge retrieval. By bridging the gap between low-level system logs and high-level attack strategies, MalGTA provides a scalable tool for modern threat intelligence. Future work will focus on improving retrieval re-ranking and developing methods to bypass malware's anti-analysis triggers.

Summary updated

Abstract

<title>Abstract</title> In High-Performance Computing (HPC) environments, a comprehensive understanding of cybersecurity threats and their underlying attack strategies is essential. However, current research predominantly focuses on maliciousness determination, typically emphasizing the code's operational behaviors rather than the attack strategies employed. The advancements in multimedia computing, particularly Large Language Models (LLMs), have paved the way for innovative solutions to the aforementioned bottleneck. This work proposes MalGTA (Guided Malware Tactical Analysis), an LLM-based system that automates ATT\&amp;CK (Adversarial Tactics, Techniques, and Common Knowledge)-aligned malware tactical analysis through Cuckoo Sandbox-driven dynamic profiling. Specifically, we construct a multi-source knowledge base integrated with Retrieval-Augmented Generation (RAG), which mitigates hallucinations in LLMs through context-sensitive threat intelligence retrieval. In addition, we propose a query optimization strategy to address challenges related to input information overload and attention dispersion in LLMs, enabling context-aware data refinement from Cuckoo reports. Finally, this study conducts dynamic analysis on classical VirusShare and Advanced Persistent Threat (APT) samples and constructs an evaluation dataset based on the authoritative malware analysis platform HybridAnalysis. Experimental results show the effectiveness of the method.
BibTeX
@inproceedings{guo2025malgta,
  author = {Xu‐Guang Guo and Jingfeng Xue and Zeyang Liu and Weijie Han and Jingjing Hu},
  title = {{MalGTA: Large language Model-based Guided Malware Tactical Analysis}},
  booktitle = {Research Square},
  year = {2025},
  doi = {10.21203/rs.3.rs-6226366/v1},
}
Figure