TL;DR
ProcReverser is a multi-agent LLM system that automates the tedious process of black-box microarchitectural reverse engineering. By orchestrating a team of specialized AI agents to design, execute, and analyze hardware experiments, it successfully identifies security-critical processor properties (like branch predictor behavior and speculation windows) with over 80% accuracy on real-world RISC-V hardware.
What Problem Does This Paper Address?
Modern processors are incredibly complex "black boxes." Even when they share the same Instruction Set Architecture (ISA)—like RISC-V—different physical implementations handle speculative execution, branch prediction, and memory management in unique ways.
These implementation details are the "ground zero" for vulnerabilities like Spectre. To defend a system, security researchers must first understand these hidden hardware behaviors. Traditionally, this requires a "human-in-the-loop" approach: an expert must manually design experiments, write low-level test code, debug on specific hardware interfaces, and interpret noisy timing data. This process is slow, expensive, and must be repeated for every new chip or configuration. ProcReverser aims to automate this entire expert workflow.
Methods
The authors developed a multi-agent framework powered by Large Language Models (specifically Zhipu AI's GLM-5.3-Flash) to turn high-level microarchitectural questions into evidence-backed reports.
The Multi-Agent Workflow
As shown in Figure 1, the system divides the labor among five specialized roles:
- Designer: Formulates the experimental plan, hypotheses, and quantitative decision criteria.
- Reviewer: Sanity-checks the plan for feasibility on the target platform.
- Generator: Implements the plan by writing the actual C/Assembly test programs.
- Diagnostician: Analyzes execution logs and compiler errors to suggest code repairs.
- Analyst: Interprets the final measurements to produce the property report.
Discriminative Revalidation (DR)
A key innovation is Discriminative Revalidation. LLMs are prone to "hallucinating" conclusions from insufficient data. DR acts as a mandatory automated gatekeeper. It checks if the raw measurements actually distinguish between competing hypotheses before the Analyst sees them. If the data is ambiguous, DR triggers a targeted retest, forcing the system to gather better evidence.
Datasets & Benchmarks
The authors constructed a suite of 26 security-relevant property identification tasks. These tasks cover:
- Speculative Execution: Measuring speculation windows and barrier effectiveness.
- Branch Prediction: Identifying Return Address Stack (RAS) capacity and Branch Target Buffer (BTB) behavior.
- Cache & Memory: Characterizing cache capacity, prefetching, and memory dependence predictors.
The system was evaluated on three distinct platforms:
- XuanTie C910: A commercial RISC-V SoC.
- SiFive P550: A high-performance commercial RISC-V core.
- gem5 RISC-V O3: A configurable simulator used to test the system's ability to adapt to hidden configuration changes.
Key Insights and Ideas
- Role Specialization is Crucial: The "Five Roles" configuration vastly outperformed a single-agent baseline. Distributing tasks allowed for better error recovery and more rigorous planning, jumping from ~18% to ~81% success on the C910.
- The "Aha" Moment of DR: In a case study on hardware prefetching, the system initially misinterpreted a timing gap. The DR mechanism caught the lack of specific "probe" samples, forced a code repair, and eventually led the system to the correct conclusion that no prefetching was active under those specific patterns.
- Adaptability to "Hidden" Hardware: By testing on gem5 with varying internal settings hidden from the agents, the authors proved the system doesn't just "memorize" chip manuals—it actually infers properties from behavior. For example, it correctly identified RAS depths of 8 and 16 entries simply by observing return costs (see Figure 5a).
Key Results
ProcReverser demonstrated high reliability across different hardware:
- XuanTie C910: Achieved a Pass@1 score of 81.3% and a Pass@5 of 93.8%.
- gem5 O3 Simulator: Achieved a Pass@1 of 83.0% and a perfect Pass@5 of 100.0%.
- SiFive P550: Successfully characterized 23 out of 26 tasks, including identifying bidirectional hardware prefetching and measuring the specific cycle overhead of different memory barriers.
Figures 3 and 4 visualize these successes, showing that while some complex predictor tasks (Group R) occasionally require multiple attempts, the system is remarkably consistent across most categories.
Limitations and Future Work
- ISA Scope: The current evaluation is limited to RISC-V. While the logic is ISA-agnostic, future work is needed to validate it on x86 and Arm.
- Signal Strength: If a hardware property has a very weak observable signal (e.g., extremely low timing variance), the system may still struggle to reach a definitive conclusion.
- Inference Cost: The multi-agent "chatter" and iterative repair cycles consume significant LLM tokens, making it more expensive than a simple script (though still cheaper than human expert time).
- Future Directions: The authors plan to explore more complex agent collaboration strategies and expand the task suite to cover even more obscure microarchitectural side channels.
Summary updated