Deep learning models are expanding at a rate that far outpaces the hardware capabilities of edge devices like smartphones, IoT sensors, and vehicular networks. Running a multi-billion parameter model entirely on-device is often impossible due to strict memory and power constraints. Conversely, offloading the entire task to a centralized cloud server introduces high latency, heavy bandwidth consumption, and severe privacy risks regarding raw user data.
Model partitioning solves this dilemma by treating a neural network not as a monolithic block, but as a graph of sequential layers or functional modules that can be physically distributed across different devices. By cutting the network at a specific "cut layer," systems can execute the initial, lightweight layers locally on an edge device, transmit the resulting intermediate features over the network, and offload the remaining computationally heavy layers to a powerful server or a collaborative cluster of nearby devices.
The Anatomy of a Split
To understand why this works, consider the sequential nature of a standard deep neural network. The early layers typically perform basic feature extraction—such as token embeddings in language models or low-level convolutions in vision models. These layers require relatively little computational power and memory compared to the deep, parameter-dense layers that follow.
In a split inference configuration, as described by Chang Liu et al. in Resource Allocation in Large Language Model Integrated 6G Vehicular Networks, the edge device processes only these initial layers. Instead of transmitting raw input data (like high-resolution video or sensitive voice recordings), the device transmits the intermediate activations at the cut layer. A remote server receives these activations and resumes the forward pass through the rest of the network.
[ Input Data ]
│
▼
┌───────────────────────────┐
│ Client-Side Layers │ <- Low memory, low compute
└───────────────────────────┘
│
▼ [ Intermediate Activations / Tensors ] ───► Transmitted over Network
┌───────────────────────────┐
│ Server-Side Layers │ <- High memory, high compute
└───────────────────────────┘
│
▼
[ Final Prediction / Output ]
This modularity can also be functional rather than strictly sequential. For instance, Yongwei Wang et al. in Advances in edge-cloud collaboration and evolution for large-small models highlight how systems can separate functional components like encoders and decoders across infrastructure tiers. This design choice keeps low-latency data ingestion local while leveraging cloud resources for heavy generative processing.
Training Across the Cut: Split Learning
The same partitioning logic applies to model training, a paradigm known as Split Learning. During a forward pass, the client computes activations up to the cut layer and sends them to the server, which completes the forward pass and calculates the loss. During the backward pass, the server computes gradients up to the cut layer and transmits them back to the client, allowing the client to update its local weights without ever seeing the full model.
To scale this process to multiple clients simultaneously, researchers have integrated split learning with federated learning architectures. Chandra Thapa et al. proposed a hybrid framework in SplitFed: When Federated Learning Meets Split Learning, where a dual-server setup manages the heavy computational load of the upper layers on one server while aggregating client-side weights on another. This prevents clients from needing to store or execute the full global model, providing an inherent layer of security for the model architecture itself since neither party possesses the complete network.
Furthermore, to handle the synchronization bottleneck of massive client networks, Shraman Pal et al. in Server-Side Local Gradient Averaging and Learning Rate Acceleration for Scalable Split Learning demonstrated that the server can average the gradients received at the boundary cut layer and broadcast them back. This allows individual edge devices to benefit from collective updates without the massive overhead of full model weight synchronization.
Asymmetric and Parameter-Efficient Partitioning
As models grow into massive foundation models, cutting a network sequentially becomes less efficient because even the early layers or embedding tables can overwhelm edge memory. This has driven a shift toward asymmetric model partitioning.
Instead of splitting a monolithic network down the middle, asymmetric architectures deploy a lightweight "shadow" or "target" model on the client that interacts with a massive, frozen backbone in the cloud. For example, in large language model fine-tuning, the framework can host the massive frozen backbone on a cloud server while keeping only the embedding layers and trainable adapter modules on the local edge device.
Similarly, Chenhe Dong et al. in Tunable Soft Prompts are Messengers in Federated Learning utilize tiny, tunable soft prompts on the client side as messengers. The clients optimize these small prompts locally and send them to the server, reducing local compute and memory requirements by nearly an order of magnitude while protecting the proprietary model weights on the server.
Another axis of asymmetry is precision. Ngoc Duy Pham et al. in Binarizing Split Learning for Data Privacy Enhancement and Computation Reduction introduced asymmetric precision split learning. Here, the client-side layers are heavily binarized to minimize the computational and memory footprint on resource-constrained hardware, while the server-side layers retain full-precision weights to preserve the overall predictive capacity of the network.
Managing Communication and Privacy Trade-offs
While model partitioning dramatically reduces the computational burden on edge devices, it introduces a critical trade-off: network communication overhead. Transmitting intermediate activation tensors can sometimes require more bandwidth than transmitting raw data, especially in early convolutional layers where feature maps are large and multi-channeled.
To mitigate this, authors have focused on optimizing the compressibility of the cut-layer features. Ivan V. Bajic et al. in Multi-Task Learning with Compressible Features for Collaborative Intelligence introduced a differentiable compressibility loss during training. This forces the edge-based encoder to produce intermediate feature tensors that are inherently easier to compress without sacrificing downstream task performance.
In generative contexts, Ruikang Zhong in Enabling Distributed Generative Artificial Intelligence in 6G: Mobile-Edge Generation demonstrated that splitting the generation process by transmitting compressed "latent seeds" from an edge server allows the final image reconstruction to happen locally on the user device, minimizing network traffic by offloading the rendering process to the receiver.
Privacy is another nuanced trade-off. Although split learning avoids sharing raw data, Alsharif Abuadbba et al. in Can We Use Split Learning on 1D CNN Models for Privacy Preserving Training? warned that intermediate activations can still leak significant information. In sequential data processing, raw signals can often be visually reconstructed from the intermediate data sent to the server. This highlights that model partitioning is not a standalone guarantee for privacy, but a structural optimization that must be paired with compression, binarization, or cryptographic techniques to achieve true security across the edge-cloud continuum.