Large and Small Model Collaboration for Air Interface

Yiming Cui, Junran Guo, Xiao Li, Chao-Kai Wen, Jin ShiarXiv 2025 · DOI · arXiv

TL;DR

This paper introduces a collaborative framework that pairs "Large AI Models" (LAMs) with "Small AI Models" (SAMs) to optimize wireless air interfaces. By treating the LAM as a static, general knowledge base and the SAM as a lightweight, environment-specific plugin, the authors achieve superior performance in Channel State Information (CSI) feedback with significantly lower training costs and faster adaptation than traditional fine-tuning.

What Problem Does This Paper Address?

In the push toward 6G, Large AI Models (LAMs) like Transformers show immense promise for wireless tasks due to their ability to learn complex channel patterns. However, deploying them in the real world hits a "wall" of practical constraints:

  1. Prohibitive Costs: Fine-tuning a massive model for every specific base station or environment is computationally expensive and slow.
  2. Catastrophic Forgetting: Adapting a model to a new environment often causes it to "forget" the generalized physical laws it learned during pre-training.
  3. Black-Box Barriers: Many high-performing LAMs are proprietary; researchers often cannot access internal parameters to perform fine-tuning.
  4. Inference Efficiency: Running massive models for dozens of users simultaneously creates massive latency and memory overhead.

The authors seek a way to get the "environment-specific" gains of fine-tuning without actually touching the LAM's parameters.

Methods

The authors propose LASCO (Large and Small COllaboration) and its evolution, E-LASCO.

The Core Architecture: The "Plugin" Concept

Instead of modifying the LAM, they use it as a fixed Foundational Channel Knowledge Base. To handle local environment nuances (like specific building layouts or interference patterns), they attach a SAM as an Environment-Specific Plugin.

The Dual-SAM Strategy (LASCO)

To mimic the effect of fine-tuning a large model without actually doing it, LASCO uses two small models:

  • Reference SAM: A small model trained to mimic the output of the pre-trained LAM.
  • Proxy SAM: An identical small model that is fine-tuned on data from the target environment.
  • The Shift: The difference between the Proxy and Reference SAMs represents the "environment shift." This shift is then added to the original LAM’s output to produce the final result.

Elastic Collaboration (E-LASCO)

Recognizing that some environments need more "general knowledge" and others need more "local adaptation," the authors introduced E-LASCO. This version adds a learnable coefficient ($\alpha$) that automatically balances the contributions of the LAM and the SAM based on the specific environment's data.

Datasets

The study utilizes synthetic CSI data generated via the QuaDRiGa simulator, following 3GPP 38.901 UMi (Urban Micro) scenarios.

  • Scale: 110 distinct circular regions (5m radius) within a 200m cell.
  • Diversity: A mix of Line-of-Sight (LOS) and Non-Line-of-Sight (NLOS) environments.
  • Partitioning: 100 regions were used for general pre-training, while 10 regions were reserved to test "environment adaptation" performance.

Key Insights and Ideas

  • Physics vs. Nuance: LAMs are excellent at capturing the broad "physics" of wireless propagation (preventing "hallucinations" or physically impossible channel reconstructions), while SAMs are better at capturing the low-dimensional "local shifts" of a specific street corner.
  • The Black-Box Advantage: Because the framework only requires the output of the LAM, it works with proprietary, closed-source models. You don't need to see the weights to benefit from the collaboration.
  • Stability-Plasticity Balance: By keeping the LAM static, the system retains its "stability" (general knowledge) while the SAM provides the "plasticity" (rapid adaptation), solving the catastrophic forgetting problem.
  • Efficiency in Multi-User Scenarios: Since the LAM is static and shared, it can process batches of users once, while only the tiny SAMs need to be run individually for specific environment tweaks, drastically reducing GPU memory usage.

Key Results

  • Superior Accuracy: LASCO and E-LASCO consistently outperformed both "Pre-trained LAMs" and "Fine-tuned SAMs" across various compression ratios (codeword lengths).
  • Sample Efficiency: LASCO achieved high performance with as few as 1,000 training samples in a new environment, whereas baseline models often failed to converge or required much more data.
  • Faster Convergence: In 50% of test cases, E-LASCO required significantly fewer training epochs to adapt to a new environment compared to traditional fine-tuning methods.
  • The $\alpha$ Factor: Testing showed that the optimal balance ($\alpha$) between the large and small models varies by environment (e.g., $\alpha \approx 0.9$ for one area vs. $0.3$ for another), proving that the "elastic" approach in E-LASCO is necessary for peak performance.

Limitations and Future Work

  • Model Alignment: The success of the method relies on the Reference SAM being a "good enough" mimic of the LAM. If the architectures are too different, the "shift" might be noisy.
  • Inference Overhead: While much lighter than fine-tuning a LAM, the system still requires running three models (one LAM, two SAMs) during the adaptation phase, though the SAMs are designed to be very small (e.g., 2 Transformer blocks vs. 20 in the LAM).
  • Future Directions: The authors suggest extending this framework beyond CSI feedback to other "Air Interface" tasks like beam management, channel estimation, and signal detection.

Summary updated

Abstract

Large artificial intelligence models (LAMs) have shown strong capability in wireless communications, yet existing works mainly rely on their generalized knowledge across environments while overlooking the potential gains of environment-specific adaptation. Directly fine-tuning LAMs for adaptation is often impractical due to prohibitive training costs, low inference efficiency in multi-user scenarios, and the risk of catastrophic forgetting, in addition to the limited accessibility of model parameters. To address these limitations, we establish a collaborative framework for air interface. In this framework, unlike prior approaches that either depend solely on LAMs or require direct fine-tuning, LAMs are exploited as a universal channel knowledge base while small artificial intelligence models (SAMs) are employed as lightweight plugins to capture environment-specific knowledge, facilitating efficient environment-specific adaptation of LAMs. Subsequently, we instantiate this framework for CSI feedback tasks, and develop a large and small collaboration framework for CSI feedback, referred to as LASCO. LASCO operates by letting the base LAM produce an initial CSI reconstruction, learning the environment-induced reconstruction shift through a reference SAM and a proxy SAM, and transferring this shift back to the LAM. To further enhance adaptability, we introduce elastic-LASCO (E-LASCO), which augments LASCO with learnable collaboration coefficients that control the contribution of LAMs and SAMs across different environments. Numerical results demonstrate that LASCO and E-LASCO enables LAMs to achieve environment-specific performance gains with significantly reduced training costs, lower data collection requirements, and faster adaptation speed.
BibTeX
@misc{cui2025large,
  author = {Yiming Cui and Junran Guo and Xiao Li and Chao-Kai Wen and Jin Shi},
  title = {{Large and Small Model Collaboration for Air Interface}},
  year = {2025},
  eprint = {2512.12170},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.12170},
  howpublished = {arXiv},
}
Figure