Chameleon: Taming Dynamic Operator Sequences for Memory-Intensive LLM Training

Zibo Wang, Yuhang Zhou, Zhibin Wang, Shipeng Li, Xinjing Huang, Chendong Cai, Bingxu Mu, Yuqing Sun, Zhiheng Hu, Bin She, Shu You, Guanghuan Fang, Rong Gu, Wanchun Dou, Guihai Chen, Chen TianarXiv 2025 · DOI · arXiv

Abstract

Training Large Language Models (LLMs) often exceeds the high-bandwidth memory (HBM) limits of modern accelerators. While memory swapping to host DRAM is a viable solution, existing techniques are primarily designed for static computation graphs. In dynamic "Eager Mode" frameworks like PyTorch, changing operator sequences can invalidate swap policies, leading to crashes or out-of-memory (OOM) errors. We present Chameleon, a swap-based memory optimization system that dynamically adapts to operator sequence changes. Chameleon enables training models up to 4x larger than physical memory capacity with minimal overhead and improved stability.

Introduction

The demand for memory in training state-of-the-art LLMs frequently outpaces the growth of GPU/NPU HBM. Swapping—offloading inactive tensors to CPU RAM—is a common strategy to mitigate this. However, traditional swap methods assume a "Graph Mode" execution where the sequence of operations is fixed. Modern development favors "Eager Mode" (e.g., PyTorch), where dynamic features like conditional branching and loss scaling cause operator sequences to fluctuate. These fluctuations break static swap schedules, causing systems to swap the wrong data or miss timing windows, resulting in OOM errors. Chameleon addresses these challenges by redesigning the swap workflow for dynamic environments.

System Architecture and Methods

Chameleon's architecture is built to handle the unpredictability of Eager Mode without sacrificing performance. It consists of three integrated components:

Overview of the Chameleon system architecture, showing the interaction between the Online Profiler, Policy Generator, and Executor within the training framework.
  1. Lightweight Online Profiler: To minimize overhead, Chameleon uses a dual-mode profiler. A "Lightweight Mode" continuously monitors operator sequences using integer hashing. When a sequence change is detected, it triggers a "Detailed Mode" to collect fresh memory usage data and generate an updated swap policy.
  2. Policy Generator: Instead of micro-profiling every operator, Chameleon groups operators into logical layers, leveraging the repetitive structure of Transformer models. It uses a global simulator to schedule swaps, ensuring they overlap with computation to avoid hardware stalls.
  3. Executor: Chameleon employs multi-feature fuzzy matching to identify tensors across iterations, as PyTorch does not provide persistent tensor IDs. Additionally, it replaces CPU-heavy cross-stream synchronization with an event-based, device-level mechanism, eliminating host-bound bottlenecks during large-scale training.

Key Insights

Grouping for Stability

A core insight of Chameleon is that exact per-operator timing is unnecessary for effective scheduling. By grouping operators into logical layers, the system averages out execution time variance. This approach significantly reduces profiling complexity while maintaining scheduling accuracy.

Analysis of grouping operators: as the number of groups decreases, the coefficient of variation (CV) and error in execution time estimation drop significantly.

Efficient Synchronization

Standard PyTorch synchronization for multi-stream memory management (e.g., recordStream) relies on the CPU polling the device. At scale, this polling becomes a major bottleneck. Chameleon's shift to intra-device synchronization allows the compute and swap streams to coordinate without constant host intervention.

Key Results

Chameleon demonstrates significant improvements over baseline PyTorch and existing swap techniques:

  • Reduced Overhead: The detailed profiling mode reduces overhead by 84.25% compared to native PyTorch tools, while the lightweight monitor adds less than 1%.
  • Massive Memory Scaling: Chameleon supports training with 4x larger batch sizes and sequence lengths compared to native PyTorch.
  • Performance Gains: The system outperforms activation recomputation by an average of 16-19%, with peak gains reaching nearly 39%.
  • Stability: In long-duration tests with dynamic scaling, Chameleon successfully completed 5,000 steps where baseline methods crashed early due to sequence changes.

Conclusion

Chameleon provides a robust solution for memory-intensive LLM training in dynamic execution environments. By combining lightweight profiling, operator grouping, and efficient device-level synchronization, it tames the chaos of Eager Mode, allowing researchers to train larger models on existing hardware infrastructure.

Summary updated

BibTeX
@misc{wang2025chameleon,
  author = {Zibo Wang and Yuhang Zhou and Zhibin Wang and Shipeng Li and Xinjing Huang and Chendong Cai and Bingxu Mu and Yuqing Sun and Zhiheng Hu and Bin She and Shu You and Guanghuan Fang and Rong Gu and Wanchun Dou and Guihai Chen and Chen Tian},
  title = {{Chameleon: Taming Dynamic Operator Sequences for Memory-Intensive LLM Training}},
  year = {2025},
  eprint = {2509.11076},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.11076},
}
Figure