In the standard narrative of Federated Learning (FL), we imagine thousands of smartphones training a global model on similar types of data—like photos or text messages. This is Horizontal Federated Learning, where the feature space is the same across all participants, but the individuals (samples) are different.
However, a different and often more complex problem exists in enterprise sectors like finance and healthcare: Vertical Federated Learning (VFL). Here, different organizations hold different attributes (features) for the same set of individuals. For example, a bank may have a customer's credit history, while an e-commerce platform has their shopping habits. To build a truly predictive credit scoring model, these "data islands" need to be bridged without actually sharing the raw, sensitive records.
The Architecture of a Vertical Split
The core intuition behind VFL is to decompose a neural network into local and global components. As Praneeth Vepakomma and colleagues detailed in their work on Split Learning, the model is "split" at a specific layer, known as the cut layer.
- Local Processing: Each participant (or "passive party") runs the raw data through their own local sub-network. This sub-network acts as a feature extractor, transforming sensitive raw data into a latent representation or "embedding."
- The Cut Layer: These embeddings are sent to a central server (or an "active party" that typically holds the labels).
- Global Fusion: The server aggregates these embeddings—often by concatenation or summation—and passes them through the remaining layers of the network to produce a prediction.
- Privacy-Preserving Backpropagation: To update the model, the server calculates the loss and sends the gradients back across the cut layer. Each participant then updates their local sub-network using these gradients, never having seen the other parties' features or the global labels.
This mechanism allows for multi-modal fusion across institutions. As noted in Recent Methodological Advances in Federated Learning for Healthcare, this is essential for precision medicine, where one hospital might hold imaging data while another holds longitudinal clinical records for the same patient.
Aligning the "Data Islands"
Before training can begin, VFL requires a crucial preprocessing step: Entity Alignment. Since the parties have different features for the same people, they must first identify which records belong to whom without revealing their entire customer list.
Researchers like Qiang Yang and Dashan Gao emphasize the use of Private Set Intersection (PSI). PSI uses cryptographic protocols to find the overlap of user IDs (like hashed emails or SSNs) between parties. Only the overlapping samples are used for the collaborative training, ensuring that neither party learns about customers they don't already have in their own database.
Balancing Security and Speed
The primary challenge in VFL is the "performance tax" of privacy. Traditional distributed learning is fast because it shares raw data or simple gradients. VFL often employs heavy-duty cryptography to protect the embeddings and gradients.
Qinghe Jing and Kai Chen quantified this in Quantifying the Performance of Federated Transfer Learning, finding that VFL can be 17x to 24x slower than centralized learning due to the overhead of Homomorphic Encryption (HE).
To solve this, newer frameworks like CAESAR and SPNN propose an "algorithmic-cryptographic co-design." Instead of encrypting the entire process, they use Secure Multi-Party Computation (SMPC) or lightweight masking only for the most sensitive interactions—like the initial feature-combining layer—while allowing the rest of the network to train at near-plaintext speeds.
Why VFL Matters for Industry
While Horizontal FL is popular for consumer apps, VFL is the "unsung hero" of B2B collaboration. Qinbin Li and Bingsheng He argue in their system-level surveys that VFL bridges the gap between data utility and regulatory compliance (like GDPR).
Practical applications are already showing significant gains:
- Credit Scoring: Fanglan Zheng demonstrated that VFL can automate interpretable scorecards, reducing training time from hundreds of hours to just 30 while improving accuracy (FL-LRBC).
- Advertising: In the FedAds benchmark, integrating publisher-side browsing data with advertiser-side conversion labels via VFL significantly outperformed models trained on isolated data.
- Fraud Detection: By combining VFL with differential privacy, researchers achieved a 16% boost in detecting financial anomalies (Privacy-Preserving Financial Anomaly Detection).
As we enter the era of Foundation Models, VFL remains a critical research frontier. As Nathalie Baracaldo points out, even the most powerful Large Language Models cannot inherently "see" across data silos; VFL provides the structural plumbing necessary to adapt these models to vertically partitioned, multi-institutional data.